Yes I think you’re wrong, because if he’s good enough to be allowed into your house then he’s good enough to show hospitality to! A bathroom break should have been expected in advance. since you hired him for work. Anytime someone is in my house I think they may need to use the restroom