There are more than one kind of permit. In some states, a permit is required before you can purchase a firearm (or just a handgun). In the majority of states, a permit (or license, depending on what the state calls it) is required before you can carry a handgun -- in some states to carry at all, in others to carry concealed. These are not the same permit. A purchase permit does not authorize carry ... but in many states a carry permit also serves as proof of a background check when purchasing.
In my state, we can purchase a handgun without any permission slips if we're willing to wait. I think the period is ten days, but it may be two weeks. With a carry permit, we can walk out with the new toy as soon as the ink is dry on the 4473. Without the carry permit, once you get home with your new handgun you cannot take it anywhere off your property, except to a repair facility or to a gun shop to be sold. PERIOD. Not to the range. Not to your brother-in-law's house to show it off. NOWHERE ... not even unloaded, locked in a titanium case and transported in the trunk of your car.
Each state is different. Ignore that at your peril.