Explanation of options: (I'll probably get called out on some mistakes or oversimplifications)
MP3: MPEG-1 layer 3 audio. Currently the most popular format for compressed digital music. A standardized format using techniques patented by Thomson and Fraunhofer. Widely used and supported, but often yields lower quality than newer formats.
WMA: Windows Media Audio. Audio format developed and controlled by Microsoft; their preferred replacement for MP3.
AAC: Advanced Audio Codec; part of the MPEG-4 standard. Used by Apple in iTunes and, in modified form, on the iTunes music store. Uses techniques patented by many companies including Dolby.
OGG: Ogg Vorbis audio format, a Free (no patents, no licensing) audio codec. Rarely seen thus far.
These formats are all "lossy:" they use various tricks to shrink the file sizes. Compared to AIFF or WAV, the quality is slightly lower but the file size is many times smaller.
As a compromise, formats like FLAC, Apple Lossless, and the Windows Media lossless format cut file sizes in half (compared to AIFF/WAV) but retain 100% of the quality.