Yes there is minimal protection for the ear away from the sound. I have heavy deafness in my right ear. I'm right handed one hand pistol shooter. My left ear has some hearing loss, but much less than my right ear.
It only stands to reason that the ear not directly receiving the brunt of the sound wave would get less damage.
(Bear in mind this report is merely anecdotal)