That is a recurring statist argument, and I'm glad you're tackling it. But I think it isn't so weak as you suggest. There's an argument used by many authors (I recall it from Fichte) that says something like: a man can't be free in a society that isn't; and a society can't be free, if its men aren't.
So, how can we be free? Maybe its a long process that cross generations, and if we, living in a statist society, were to allow a power vaccuum, the process might have to begin from scratch. We have tamed our states to some degree, and by getting rid of democracy, we could get a tyranny in its stead.
It is because I agree with you, that free men don't need government, that I think the power vacuum is indeed a reasonable argument for those who aren't free.