Looks like you're hot on the powder charge. I looked here and got 4.1 gr as a starting load and 5.0 as the max:
http://data.hodgdon.com/cartridge_load.asp
I can't get this to format, but you can see the numbers...
Hodgdon Powder Company
Cartridge Load Recipe Report - 2/10/2010
data.hodgdon.com
40 S&W
Load Type: Pistol Powder: 231
BW: 180
Cartridge Information
Case: Hornady Barrel Length: 4"
Twist: 1:16" Trim Length: .845"
Primer: WINCHESTER SP
40 S&W
Cartridge Load Data Starting Loads Maximum Loads
Bullet Weight (Gr.) Powder Bullet Diam. C.O.L. Grs. Vel. (ft/s) Pressure Grs. Vel. (ft/s) Pressure
180 GR. HDY XTP 231 .400" 1.125" 4.1 797 23,800 PSI 5.0 947 32,900 PSI
NEVER EXCEED MAXIMUM LOADS