Practically speaking, it appears that the ATF is not particularly concerned about "old" firearms with long ago defaced serial numbers. I had the same situation with a Colt New Army/Navy that's been in the family for decades. Sometime in the dusty past, somebody filed the numbers off the butt.
I called the local ATF office about it when I inherited the gun. The agent explained that in the hierarchy of people they're concerned about, the guy with grandpa's eighty or hundred year old pistol rates slightly below "who's gonna take the trash out this afternoon".
Now, you'll have to discover your own definition of "old" and "long ago". In my case, I'm not worried at all. Your comfort level may vary.
Well we don't rent pigs and I figure it's better to say it right out front because a man that does like to rent pigs is... he's hard to stop - Gus McCrae