You will have a hard time getting a very shiny blue with rust bluing, so I assume you intend to use hot tank or caustic blue. For that purpose, the polishing you describe will work fine, but note that jewelers' rouge is very fine so you will want to use a more abrasive compound first to polish out the tool marks common on military rifles. Then work down (or up, if you prefer) to the jewelers' rouge. The result will be a very high shine (mirror finish).
(Just to cover the bases, you also won't get a mirror finish with cold blue of any kind, no matter how hard you polish and the blue won't last.)