The president has DIRECT control over the management, policies and practices at the BATFE. Or he can choose to de-fund them entirely. He also controls the DOJ. Congress allocates money, the Prez spends it more or less, except for major entitlement programs.
Would Dr. Paul spend less money than Congress gives him? Yeah. He sure as hell would. He gave back his entire salary as a congresscritter every single year, he pared his office budget to the bone (including fewer than normal staffers) and gave back a big part of his office budget to the general treasury every year.
We have never had a President wired quite that way in our entire history.