"Notwithstanding the provisions of subsection (a) of this section, a person is not justified in using deadly physical force upon another person if he or she knows that he or she can avoid the necessity of using such force with complete safety (1) by retreating, except that the actor shall not be required to retreat if he or she is in his or her dwelling, as defined in section 53a-100, or place of work and was not the initial aggressor"
So the law says we don't have to retreat. But it doesn't say anywhere that we can actively pursue anybody in our house or not.
It looks like he was correct in saying we can only hold our ground in where we are at at the time of the invasion. We can't sweep or otherwise protect our home except hope the invader comes at us and is the aggressor.