The only thing I know of in which you give up that right is if you have an FFL.
This is correct. The ATF can conduct an annual "compliance" inspection of an FFL. Also, the standard for a warrant is marginally eased for FFLs.
Non-FFL firearm owners (including Title II firearms) are still protected by the 4th amendment. Probable cause is required for a search warrant. Even then, the ATF/AUSA will sometimes screw that up by not being specific in their warrants (these warrants can usually get tossed).