Download speeds are mostly dependant on the ability of the remote
server to pump out the data. If a bunch of people are connected to a busy server, each one gets a slice of attention from the server.

And if the server from which you are downloading is geographically distant, the data has to travel over more hops to reach you. If there's a weak link in the chain along the way, you will see slower download speeds.

Many popular download services such as Tucows or Download.com offer you a choice of servers, and will list the location of the remote server.

Always pick the server that's closest to you. You'll get faster download speeds and you'll have the satisfaction of being a good netizen.

Oh, and if it's a really big file, try downloading in the early hours when the rest of the world is sleeping!