I'm not a LEO or have ever worn a vest. However in your specific situation, if you're already used to wearing it most of the day, and it's on hand already. I see no reason to not at least give it a test run. See if in your off duty time your activities make the vest more cumbersome than practical.
This general topic has come up a few times in the past on the board. I've come to think that in a self defense situation it hard to argue that a purely defensive item is a bad idea. To me, a vest would help mitigate some of the factors of which you have no control. Furthermore as a LEO, you may feel a higher moral and ethical duty to intervene in situations when off duty than the average CCW'er would. Possibly even having a legal obligation to do so.