I'm not a parent, but have friends who are. Many people fear that a toddler is going to break into a lockbox, learn to how to load a gun, take off the safety, rack the slide (which some adults have trouble with) and pull the 5 pound trigger and shoot someone.
While home invasions are rare, so are kids shooting themselves or others.
And if you survived a home invasion, you'd likely feel quite bad that you were unable to prevent any of the consequences of same.
I'd feel quite comfortable with Frank's approach. Lockboxes with guns, ammo, flashlight around the home would be fine. There are also things called life jackets which mount to studs and lock a long gun stored vertically on the wall.
I'd also figure out if the 4 year old can reach high enough on a chair to access a long gun stored on hooks above the closet door, for instance.
Also, when they can understand, they must be taught about gun safety as others have chimed in.