Jailbreaking allows you to install apps that Apple doesn't approve off because it
A. Allows you to install apps/features to iOS that apple doesn't want you to have from Cydia. Most devs that can't get their apps into the Apple App Store end up here where they can also sell their apps but with the downside that only Jailbreakers can get at them.
B. Could allow one to install stolen/pirated apps one will find on sites like the PirateBay.
Now, I Jailbroke my iPod Touch for pretty much just one reason.
The Apple Bluetooth that is in iOS will not allow you to connect to non-apple hardware thus making it pretty useless. What I installed once I had jailbroken it was roqyBluetooth 4. This little app once installed allowed me to use my external Bluetooth GPS device with every map application that is out there making my iPod Touch a super handy navigator.
Now to the Cydia store, it gets it's app lists from "Repositories" which can come form several sites. List of legal repositories:
Cydia™ | iPhone Apps | Search powered by ModMyi.com
Once you have added a Repository to cydia, all apps in it will be shown to you in list form and you can then search/buy/download apps.
Another thing why some people Jailbreak is because of theming. There are some pretty awesome iOS themes out there that look absolutely stunning.
The ModMyI Site is a pretty good place to get more info, as well as their forum.
Apple, iPhone and iPad Forums | ModMyi.com
Apps list of the Tweaks category.
IMG_0125.PNG IMG_0127.PNG