I've seen some pretty convincing evidence that shows microscopic (but still toxic) pieces of lead getting scattered throughout the meat of deer that was shot with a traditional lead bullet. I can see how those would cause problems both in carrion eaters and in humans.
Seeing as we now have excellent non-toxic ammo available, I really wouldn't be terribly opposed to a lead ban for hunting ammo. In theory, the street price of non-toxic stuff should come down if every manufacturer were producing it. Actually, I'd be more conserned with a ban in shotgun ammo than in rifle ammo. The only think right now keeping me from using Barnes TSX or Hornady GMX ammo is the price.