You never really only use a tablet when working. I always have one hand on the keyboard to quickly switch tools.
Some people always use them and never use a mouse or trackpad. I'd find that really annoying to be honest... I'm left handed so I often use a mouse and a tablet at the same time! Kinda handy.