Personally, I wouldn't feel comfortable shooting any deer with any 20 ga round. Yesterday I shot a medium size white tail at about 30 yards with 3-1/2" magnum 00 buck out of my Benelli supernova 12 ga. it shot about an 8 inch spread, and i hit him right in the vitals. perfect shot. It only penetrated 1 lung. He dropped immediately, but it took him a while to die.
Remember thats 18 .33" pellets. The #4 you are gonna be able to find is a .24" pellet. I shot at a tree with my cousins 20 ga with #4 buck and it barely penetrated the bark. While it might be legal, it isn't ethical imo to shoot a deer with a 20. Ask your neighbor if him or any of his buddies have a spare 12 ga u can use. If you have to use the 20 ga, keep it within 25 yards at the absolute max.
look at the 20 ga
http://www.youtube.com/watch?v=PDjNaTeUt2Y
vs the 12 ga
http://www.youtube.com/watch?v=PDjNaTeUt2Y
and remeber i took that deer with 3-1/2" 00 buck, not 3" in the video and he still kicked for a while.