To expand a bit on what Willie Sutton wrote and provide citations:
18 USC 922(a)(5) prohibits a resident of one State from transferring, selling, trading, giving, transporting, or delivering any firearm to any person who the transferor knows or has reasonable cause to believe lives in another State. But the statute makes an exception for a temporary loan or rental of a gun for a lawful sporting purpose. So if your buddy is visiting you from another State, you may loan him a gun so he can go target shooting or hunting with. Or an outfitter may rent a gun to a client from another State to participate in a guided hunt.
But 18 USC 922(a)(3) prohibits any person from transporting into or receiving in the State where he lives a gun obtained in another State. There's no exception here for a loan.
So for example, you can loan your buddy visiting here from another State a gun, but he can't take it home with him (nor could you ship a gun to him in his home State "on loan"). Your letting him take the loaned gun home to another State, or you making an interstate loan, would violate 18 USC 922(a)(3).
"It is long been a principle of ours that one is no more armed because he has possession of a firearm than he is a musician because he owns a piano. There is no point in having a gun if you are not capable of using it skillfully." -- Jeff Cooper