I agree with zeta101. Sometimes, even if something is really popular and lots of people are sharing, you can still pick a bad torrent. Look for the number of seeds and the number of leechers (most torrent search engines give those numbers as a value and as a percentage). The more seeds the better, and lower the leechers to seeds ratio the better.
In general, if there are a couple of torrents available for something I'm after that appear to have a good set of seeds, then I'll start two and cancel the slowest of the two. Don't forget that they sometimes also take a few minutes to get up to full speed. I downloaded the Ubuntu Dapper Drake ISO at 490KB/s (I'm on a 4Mb line) so it really does work!
As a final note, many ISPs (certainly in the UK, and I would imagine the US, not sure about anywhere else) throttle the traffic on bittorrent ports, or block it completely. One of the up-and-coming UK ISPs (TalkTalk) throttle the torrent traffic right down during the day, but allows unlimited use between midnight and 8am. This could explain why some people just never see good speeds.