Like everything else in life, it depends. How you plan to use the gun makes a big difference. Generally, long barrels for hunting and some types of target shooting, short barrels for concealed carry, and something in between for everything else. Except as a necessity for concealed carry, I wouldn't go with less 4" and in my opinion 5"-6" is ideal for general purpose/home defense. The longer barrel will give you a longer sight radius and better velocity without being cumbersome or muzzle heavy. Shorter barrels are just trading velocity and accuracy for reduced performance and increased muzzle blast unless you have a specific need for them. Longer barrels improve performance but at some point make the handgun harder to conceal, slower to draw, and muzzle heavy.
"Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety."
- Benjamin Franklin