If you want to delete the previous character then hit the delete key, which you already know. To delete the next character, hold down the function key (fn key next the ctrl) and press delete.

The fn, crtl, alt (known as the option key) and the Apple key (known as the Command key) are filled with hidden functions depending on the combination of keys held down. For instance, if you want to type a # character, hold the alt (Option) key and press 3 (not sure if this is printed on the US keyboards anyway but on the UK keyboard 3 is the £ key. It took me ages to find the shortcut. Hopefully others will find this helpful even if you don't).