An optical mouse uses sight, not movement. It takes a snapshot of the surface below hundreds of times a second. It compares each image with the previous image, and from the difference it knows how far the mouse moved.
I found that wood grain tends to confuse the optical mouses a lot. I think there is too much going on in the image for it to pick a good place marker. So it'll confuse some grains of the wood with others.
One would think that an image with lots of character gives the optics lots of placemarkers to pick and therefore get better results. But I found that a blank piece of paper, or a flat black mouse pad gives the best, actually perfect, results.