I'll second that suggestion of gas checks for hot loads. If that gun shot straight with jacketed bullets it should shoot straight with cast. If it don't shoot straight with either type you may have a problem with some aspect of the gun. Finally, when shooting cast you need to know your barrel size and and cylinder throats size. The cylinder throats need to be about .001 bigger than the barrel. Bullets need to be sized to bore size. If you don't have a caliper then beg, borrow, steal, or buy one. There are lots of folks here who can give you better info about this than I.
The difference between a stumbling block and a stepping stone is the height to which one raises one's foot.