I have the same issues at times. I have over 85,000 music tracks (600+ Gb) and over 3000 Video tracks (700=+ Gb) in iTunes. everything is spread across 7 internal and external drives, and iTunes is on my main boot drive (8th drive). It can get sluggish at times, even on the Quad G5. iTunes is not all that good with large library database files it seems.
There are several options:
1) Delete your library and rebuild it.
2) Create separate libraries for Music and Video or even Music, TV shows and Movies. Hold down Ctrl + Shift (I think) when starting iTunes and it will give the option select a library or create a new one. You can create as many as you want, but to switch between them you have to restart iTunes.
I've done option 1 and it helps a bit. Option 2 works pretty well but can get cumbersome when you want to switch between libraries.
I'm guessing that you are using a network drive so that may be part of your issue. Other than that, I don't know of any real tricks to speed up performance other than get a faster computer or reduce the overall size of the library.