Respectfully, that may be your experience but it is not mine; I have no alternative but to presume that you're scratching something on top of the wood.
I find no scratches in American walnut after using scotchbrite, with or without a lubricant such as boiled linseed oil or mineral spirits, or when dry. I've spent the morning doing this, dry. I have no special technique, although maybe I have an advantage because I was trained to do bodywork in a pro shop when I was a kid, and I'm good at it; I know when to stop and when to not stop, if you take my meaning. I rub by hand in circular patterns, with fairly light pressure. I don't 'scrub' the wood. I use very fine scotchbrite. It takes some time but I like the results.
On the previous page is a Walnut stock that I finished by sanding 150 grit down to 600 grit, then with Very Fine Scotchbrite, then with a few dozen coats of boiled linseed oil. You will note no scratches. Here is a photo of the stock I've worked over the last two days with scotchbrite; this morning I used it dry. No scratches. Here's a close-up photo of the results. I'm oiling in about 15 minutes.
