This is one of the most obvious immediate differences between Windows and OS X. The icons on your desktop are virtual disk images. They are similar to the the icon that appears in My Computer when you insert a CD to install a software package.
To install the software, you open the image, and physically drag the software icon to your Applications folder. Once it's copied to your Applications folder, you can Ctrl-Click the desktop image and select "Eject". The image disappears but the software stays installed.
Unlike Windows software, which is often made up of many thousands of files, most (but not all) OS X applications are a single file on your drive. Copy that file to your Applications folder and it's installed. Delete that file from your Applications folder and the software is uninstalled. It seems odd, but once you get used to it, it's ridiculously simple.