He's not trying to be annoying. Toddlers just don't understand the concept of privacy. They're used to being with you wherever you go, and natural inquisitiveness makes them wonder what you could possibly be doing all alone in the bathroom. (Plus, they have no sense of shame -- if they did, they'd never get through diaper changes or potty training.)
To teach your toddler about giving you space, tell your child that bathroom time is alone time and when the door is closed he shouldn't open it, peek underneath, or rattle the doorknob. He should knock if he needs your attention. There's no need to explain that Mommy and Daddy have different parts that are private (yet -- he'll start asking about that soon enough).
When he goes to the bathroom, stand on the other side of the door and turn away. If you respect his privacy, maybe he'll respect yours.
If your child slips up and bursts in on you -- and he will -- calmly remind him about bathroom privacy and tell him to close the door and wait for you to come out. He'll get the idea down in time.