Okay first of all, it's "large language model". As far as "only as good as it's training data," the paper "noise2noise" illustrates how models performance often exceeds the quality of their training data. Fact-checked data is... Entirely not the issue. Some models like RAG pull directly from data sources as you're describing, most LLM, esp the new ones, are not RAG like. RAG is more common in the corpo chatbots that are 99% requoting company policy to users.