I'm not required to go to stores to buy food or clothing? I could live in the woods. We long ago decided that businesses did not have the absolute right to refuse patronage based on what was called protected classes.
The store owner benefits from the protections that all taxpayers provide. Fire, police, etc. Thus, he or she has no right to deny patronage unless he or she gives up the services provided by the state.
I regard the right of self-defense as another instance of a right that makes one protected against the vagaries of a business that is open to the public and expects tax payer based services.
Once you decide you are open to the public, you give up rights. The mall owner must provide toilets. Isn't it the right of the store owner not to have a toilet?
Private property is not inviolate, you cannpt poop on your lawn as it is detrimental to society. Not allowing people to defend themselves and taking services from the tax payers voids that belief that a business is an inviolate castle.