I just got my iPhone yesterday and faced the same issue. My solution was to create a new Smart Playlist and put all the songs I wanted to load onto the iPhone into it. Then I told iTunes to sync just that playlist. Because it is a Smart Playlist, it is easy to build up all the music you want quickly. This is what I did, and it worked quite well.
For example, are you a big Jimmy Buffett fan (isn't everyone?
). I have a huge collection, all of which was covered in one Smart Playlist entry - basically I told it to include anything that had Jimmy Buffett in it. I repeated this for all my major artists of interest, and then I had it sync a few of my existing "compilation" playlists to pull in the rest. Worked like a charm.
BTW, I must credit my wife for this great solution. I was going down the path of seeing if I could just establish a separate iTunes Library (sort of like a secondary profile) just for the iPhone. She suggested the Smart Playlist, which she uses all the time for her iPods and it made perfect sense. If you have a LOT of artists, it may take half an hour or so to build the list, but once done, it is like a virtual profile for that specific device. Really kind of an elegant solution.