Where in scripture does it say that? I agree with you that there are legit reasons not to take it, like if you are an unbeliever, a child, or if you fail to recognise the body and blood of Christ which the bread and wine symbolises and you take it in an unworthy manner. However, it is not just for the baptised. It's for all who are saved.