In my work with cast lead bullets in rifles, I always used at least a 9x scope, usually higher magnification when working on finding an accurate load. Then with guns (Win. 94, etc.),that I had no intention of having a scope on when hunting, I would remove the scopes. Just using iron sights during load development is a handicap inasmuch as it seems to make it more difficult to differentiate between an inaccurate load and human error.
Aside from that, if you are shooting a SKS, then what you got may be all that you are going to get out of cast bullets and your gun. However, if like me, you have a Ruger M77 in 7.62 x 39, you may have some significant improvement to look forward to.