Well, if you were free, white and over 21 (and male)... So, you see, it wasn't everyone by a long shot.
+1. The language in the Constitution and the BOR divides the American populace into three broad overlapping categories. The broadest category is "person(s)" or "the people", which refers to more or less everyone, or at least those who have not had rights taken away by due process of law. The next broadest category is "citizen(s)", which is most generally defined as the subset of "the people" who are allowed to vote. The Constitution further grants certain specific powers and/or duties to smaller subsets of the population or to certain individuals. "The militia" falls into this final category.
Rhetoric of certain founders notwithstanding, the language in the document strongly implies that the words "the militia" does NOT refer to everyone, but rather to a specific group of people. The composition of "the militia" is presumably to be determined by Congress or by the states.
The 2A maintains this linguistic distinction; the "militia" is to be well-regulated, but "the right of the people
to keep and bear Arms shall not be infringed." Not the militia- the PEOPLE!
If one argues solely along insurrectionist lines, they're not only scaring folks, they're missing the central argument of the RKBA.
At best, insurrectionist arguments implicitly support the collective rights or militia interpretation of the 2A, and supporters are prone to falling into a semantic and/or historical trap regarding the composition of the militia and what exactly it means to be "well-regulated."
At worst, insurrectionist arguments implicitly support anarchy. What good is representative government if unsanctioned private armed groups can use violence to oppose it, with unclear legal consequences?