Just to expand a bit on what the difference between pistol and rifle primers are and why removing them is the right thing to do...
Mainly two different benefits from using rifle primers in rifle loads.
1) You get a hotter, longer and more intense burn with a rifle primer which is advantageous for the larger rifle case and the (sometimes) harder to ignite rifle powders
2) The primer cup thickness and durability is much more so in rifle primers, containing the extreme pressure of rifle ammo. Standard small pistol primers are typically used in cartridges running up to 35,000 PSI max pressure. The .204 Ruger runs north of 57,000 PSI max pressure. Some odd handgun rounds that run pressure beyond 35,000 PSI usually are loaded with small rifle primers simply to combat that pressure. .327 Federal Magnum is an obvious example -- it's a handgun round all the way, but run up to 45,000 PSI max and it uses small rifle primers exclusively.
Running a small pistol primer in .204 Ruger is a BAD idea. Good on you for asking before doing, and for catching your own mistake.
Attention Brass rats and other reloaders: I really need .327 Federal Magnum brass, no lot size too small. Tell me what caliber you need and I'll see what I have to swap. PM me and we'll discuss.