You should let him go because even though you guys can continue the relationship, you still need to worry whether if your bf is committed to you or not in the future. It is the best choice to let go of him. You certainly will feel very overwhelmed at the beginning but I trust you can overcome it.