Phone line, modem, and internet provider are mainly responsible for connection speed, e.g. if you have a 56K modem, your phone line must be in perfect clear condition to connect at that speed.

Also cable modem speeds vary considerably between 64 Kbps & 2 Mbps depending on service.