It's not that a 5'11" cannot fight. It's that typically the 5'6" 110 lb guy isn't the one picking a fight with the 5'11" 185 lb guy. It's the 6'3" 220 lb guy and if he can handle himself he's going to wear down the smaller fighter. You can't teach size, which is why there are weight classes in pretty much all levels of organized fighting.
I'm 5ft 8in and to go toe-to-toe with a 6ft 2 inch 200 muscle man, well that is why I learned the art of Glockdo.
Yes if all have is hands and feet well, that's all I got but I will sure look around for anything as a weapon (lamp, table, chair, bottle, belt, rock, book, towel, etc.. and I'll attack first if need be.
But, as Ernest Hemingway said, "the comforting feel of well worn checkered grips" is not something to sneeze at.