just do it. don't preoccupy yourself with letting him down. as a man, one must learn to accept all forms of rejection and not take it personally. you can have some of the ppl some of the time but never all of the people all of the time.