Hi,
First, tap the file with the mouse so it's selected. Then, hold the command key down and hit the delete key with the command key depressed. The Command keys are on either side of the space bar, where the windows button would be on a PC. They have the apple logo on them and a square with four loops. The file should then go to the trash bin. You can also tap with two fingers to open file properties and send it to the trash that way, or just drag it there.
The Command key can be used like the control key in Windows, you can select non sequential files with it as well, for deletion by holding it down and using the mouse to click on the next file you want. That way, multiple files can be selected.
I had that same exact question right after I converted to the church of Mac!
Cheers