Hitler and his cronies were literally terrified of the German people turning on them all the time. He tried to push his anti-Semitic agenda on them soon after taking power but the people rejected it. He has to take another 3 years to slowly build up the resentment toward them in order to actually get them on his side, and even when the camps started, they made the mistake of putting them too close to German cities and the public were NOT having it.