1. Click on the Apple icon in the upper left corner of the desktop, then choose System Preferences.
2. In the third row of icons is one labeled "Network." Click on that. In the resulting window are two columns.
3. On the left is a list of all of your network connections. One will be green, showing "connected." It's going to be either "Ethernet" if you use wired connections, or "Wi-Fi" if you use wireless. Click on the one showing "connected."
4. Now the right had part of the window shows the information for THAT connection. Click on the button labelled "Advanced" and the window will open with a series of buttons across the top labelled "Wi-Fi," "TCP/IP," "DNS," etc.
5. Click on the DNS box. Two windows open showing on the left the DNS Servers and on the right Search Domains. Typically the DNS server is four numbers separated by dots. For most ISPs the default is 192.168.1.1, but it may be something different for you. I don't use Windows, so I cannot tell you how to check what is being used by your Windows machine, but if you can find the equivalent list on Windows and then make sure that the list in OSX is the same, then things should work.
Another alternative is to simply add the Google DNS server to the left list. To do that, click on the "+" box at the bottom right of the left list and enter "8.8.8.8" and click "OK." It should show up in the box. You can also call your ISP to get their DNS server address and add that to the list. Then use the "-" box to delete all other entries in that list. It's ok to have more than one DNS server listed, they are referenced in the order of the list. Now you can leave System Preferences and it should work better. You may need to reboot the Mac, but I think the settings take effect right away.