Personally, I would work on my awareness and skip the 380 shopping. Yes, this sounds aloof and yes I carry a Keltec 32 ACP occasionally.
What you have with a pocket 32 or 380 is a caliber which really isn't getting it done. In 32, the choice is simple, carry the fmj, prepare to shoot them all and run away fast. This is because even decent hits can be ineffective. In 380, wow, maybe you carry the fmj and live with what 9" - 10" of penetration?? Or over 14" with the FMJ's. Maybe you alternate in the mag, who knows. Whatever you do, ammowise is kind of hoping you have enough gun.
Above, sounded like good 380 gun advice, so I will not add.
I get over the hump and have a nice pocket carry gun, you might look at a Kahr PM 9 or PM 40. Small, pocketable and adequate.
I also like. . .and most often carry. . .a S&W J frame. It sounds like my 640 would be too heavy in your mind, but how about the S&W 340PDin 357 mag, or something like a S&W 642.
As for non-CCW looking CCW gear, I generally wear shorts with large enough rear pockets to slip a revolver in and an untucked button up shirt. I do this because untucked shirts are in style and make me look thinner. . .somewhat.
I also, occasionally wear guns in horizontal shoulder holsters under a tucked or untucked shirt.
In the dead heat of summer, I don't mind keeping my P40 in my Maxpedition either, just have to keep that on me, then.
Last, getting to a good CCW is a process. Maybe the first step needs to be developing confident trigger control and handgun accuracy with a 22. Lot's or good 22's. You can't go too wrong with most.