Technically, yes, although it is true for anything, not just guns. Usually, however, you will have paid sales tax anyway if you actually bought something somewhere else. Otherwise, what you owe is called a use tax, or at least it is here. Of course, you only have to pay the tax once.
People also love for governments to spend money, too, you know. No one really seems to be willing for the government, particularly the federal government, to spend less.
Shoot low, sheriff. They're riding Shetlands!
Underneath the starry flag, civilize 'em with a Krag,
and return us to our own beloved homes!
Buy War Bonds.