As jc said, you need to format the drive in a file system that both OSes can read and write to. While OS X does support FAT, it's a pretty lousy file system.

If your Windows PCs are running a Windows NT variant (Windows NT/2000/XP/Vista), then I'd suggest, as a free alternative, installing some software in OS X that will enable it to read/write to NTFS-formatted disks. To do so, you need to install 2 different packages: MacFUSE, then NTFS-3G. A tutorial is here:
http://macntfs-3g.blogspot.com/

AFAIK, you would have to use Windows to format a drive in NTFS still, but once done, OS X will read/write it just fine with those packages installed. And NTFS is a much more robust, safer file system than FAT is. Now... there are other commercial programs that will let Windows read Mac-formatted drives and vice versa, but this solution works very well.