They are pretty much equal cameras. The Canon cameras have larger spot meters, so Nikon wins that edge. Canon at that level has better noise control on images shot ISO 800-3200. I think Canon has ISO 100 on the XT and 20D, both of those points would lead to Canon. Nikon cameras are more solidly built than Canon, I know what you mean about size I have seen the 20D and remember thinking how small the camera appears. I have not used the 20D, this was a camera being used by someone near me.
Lenses will be your most expensive purchases. Nikon and Canon both make outstanding lenses, you could go that route shopping your lenses you would want of both manufacturers and seeing if there is money savings that way.
I own nikon equipment, and actually am selling a D70, 18-70 kit lens, 300mm f/4, Adobe elements 3.0, 3 batteries total (two nikon one promaster) assorted extras for 1500.00 (U.S.) If interested let me know, the D70 is one year old and has 2 years left on an extended waranty.
After moving digital with the D70 fell in love with digital and now own a D2X, and no longer use the D70 as much.