'Spaces' Trackpad Annoyance

OK, so I have had an issue recently that has annoyed the crap out of me and I hope someone here can help me out!

If I have four spaces setup with applications, say I'm in the bottom right space and have Firefox open. For example, I'm searching google and I want to bring my pointer to the search box in the top left of the window so I use my trackpad to bring the pointer there. Nearly everytime I do this, it will automatically switch to Space #1, or the top left space. This seems to happen with every space except that first one.

I look at the system preferences and I don't think I have an option that enables switching between spaces with the trackpad. I'm also a beginner with Mac, so I could be wrong!

Can someone please enlighten me and help me to get rid of this problem? Any input is much appreciated!!

its not hot corners doing somethign you didnt know it was set to is it?

have you checked the settings under-
System Prefs-->Expose&Spaces-->Expose

I do have hot corners enabled, but that isn't the problem because it would happen even when I was no where near a corner.

Here's the weird thing, it hasn't happened today yet, and I'm pretty sure I have yet to change any settings! Hopefully it won't continue.

So it started up again - particularly when I started running a program remotely using X11 - does that make any sense at all??

It can be really sensitive, for instance when I first choose a space, a small trackpad gesture will switch to another space (which I don't want). It also switches occasionally if I draw a circle with the trackpad.

This is so odd, I can't put my finger on what gesture is causing this to happen, has anyone out there had a problem remotely close to this? There doesn't seem to be any consistency with that, except for the fact that it doesn't happen when I'm in the top left space. From there I can't directly switch to other spaced.

AHHH, help!

I'm pretty positive this is due to X11... as soon as I exited the program it wouldn't randomly switch between spaces like before.

If anyone has insight as to why this might be the case, let me know.. I'm just glad I know what is causing it.