DVI does not carry audio - at all - it is not capable of carrying audio - it will require a separate audio cable.
With an older HD TV with no HDMI, would guess you are still using component (the three connection red, green, blue) &/or S-video for your other external devices and a separate audio cable.
10 years ago would have been right about the end of the component era and the beginning of the HDMI era. HDMI wins because it carries HD video and multichannel audio in a single cable.
On a TV set of that age, would guess it's one of the top main stream brands - Sony, Panasonic, a couple others - there were not that many that even had a DVI input back then - most were still VGA only.
The issue - you will need a separate audio cable and two choices - getting the audio to the TV or to an audio receiver.
First - the ATV only has audio via HDMI or optical - both are "digital" only.
The TV is going to have only "analog" audio inputs.
Connecting the audio from the ATV to the TV.
"If" the TV has an audio port that works with it's DVI port - it's going to be the standard stereo RCA (red & white) analog port. You are going to need to look at the back of the TV - does the TV have an audio port labeled for the DVI port?
If it does, you will need a digital to analog audio converter - something like
this one.
Note: this one will not convert 5.1 digital - it will only convert digital 2 channel to analog 2 channel.
Which means that, for listening to /managing your music (which would likely all be stereo) on the TV, it will work for that. It is not going to work for pretty much any iTunes purchased/rented movies/TV shows as just about everything today is 5.1 audio.
edit:
If you try this one, you might also try changing the settings of the Apple TV - Dolby Digital to Off first - then try Audio Output to 16 bit
Upgrade a little and possibly try one like
this - which claims it will decode all the common 5.1/7.1 formats - it will probably do an ok job if you're listening to your TV speakers.
Next step up is going to be finding one that actually bears the Dolby and DTS logos and probably pay good money for those logos on the package.
Along with any of the above, you'll also need:
toslink cable
RCA stereo cable
Connecting the ATV to a audio receiver
This is a better option if you're using a stereo or surround receiver rather than your TV speakers.
Any receiver made since the mid-late 90s should have either an optical or coaxial (digital) input.
If it has optical - just need an optical cable and you're in business.
If it has a coax port - it should be orange and looks just like the red/white RCA ports - you will again need a converter - "optical to coaxial" - something like
this.
edit: have no problem recommending monoprice.com - I don't know anything about Parts Express.