You should do what you are comfortable with, not what we think you should do. You are paying the guide for a hunt, you should have the experience you want. The guide may accommodate or tell you to get lost.
Personally, I just do not hunt or shoot with anyone needing a drink or toke beforehand, whatever the reason. There may be legal risks, and safety risks. I don't need that. Some guys may do OK partaking beforehand but it is simpler for me to just walk away from that. There are other times, places and buddies with whom to hunt.
Him offering you guys (non-medical cases) a toke indicates he is more than just a medical case himself and demonstrates very poor judgment. I can't imagine that the guide doesn't already know.
"The ultimate authority ... resides in the people alone. ... The advantage of being armed, which the Americans possess over the people of almost every other nation ... forms a barrier against the enterprises of ambition."
- James Madison