I need several questions answered

[patrolman705]

I recently purchased a 300 WSM in Remington 700 VSSF, I then sent it away and had a suarez Barrel put on with a 1 in 10 twist. I installed a Barrett Bors system with a Leupold Mark 4 8.5-25X50 wide duplex. I started testing loads immediately and found a great three shot group with the following: Case- Winchester, Primer- CCI Large Rifle Magnum, Powder IMR-4320, Charge 56 gr., and Bullet- Combined Technology 168 Ballistic Silvertip. I then took my rifle home cleaned it and reloaded the rest of the ammunition(about 50 rds) before I decided to go out again. I went out again today the temp was 67 compared to 84 the first day and there was some steady rain but no wind. I started out at 100 yds and my CBS was right on the red. My second shot was about 2 inches right and 5 inches high. It had to be me I thought so I continued on with the scope tracking test. Again first shot right on red six clicks up put me about 1.8 inches high six clicks right put me 2.2 inches right, six clicks down out beside of red, six clicks left put me low and to the right on the dot. I took a couple more shots and things continued to bounce around unlike before with the inititial zero I had been getting damn near hole for hole. I am thinking it has to do with my sight picture. I am not for sure any help advice or just plain tellin me what to do to fix this would be greatly appreciated.