Possession is 9/10th's of the law. Also, there is no such thing as "HOLDING" a firearm for someone as far as the courts concerned. When he left them in your possession, the courts view that as being "transferred" to YOU. As in you own them and your responsible for them until such time as you transfer them back. The only thing he'd have a leg to stand on is if he had a letter claiming a sale of the guns and you didn't pay.
Quite frankly, a call to his parole agent will solve this in about 2 seconds. Also, when he was convicted he would have been required to give them up immediately, and since it was you who had them that will be good enough for the courts.
If it really bugs you, you could offer them the used value of the rifle your ex bought and leave it at that.