Getting iPod Touch 4G tomorrow.

I want all the music, videos and apps I have on my current iPod Touch, to be transferred to my new one.

How do I do this via iTunes? Do I literally have to have everything backed up on my iTunes, then synced to the new iPod?

Well that is pretty much how you should do it. It's easy as well but painful since it can take a very long time. If you don't have any contacts, calendar events, notes, etc, then you can simply make playlists and sync your music, videos, movies, etc, without using the backup.