A thought: In doing your magnetic test, are you detecting the SAME level of pull on the frame as on the barrel?
Where on the frame are you touching the magnet to?
Quite often in doing the magnet test, it reacts to steel parts INSIDE alloy outer assemblies or underneath the part you're touching it to.
I have a magnet kept here on my office desk, I frequently use it to test for steel on various guns. It can appear to react to alloy or plastic front sights, for instance, when it's actually reacting to the steel under those sights.
On my alloy-framed Smith J just now I tried both sides of the frame behind the cylinder & the magnet showed attraction to the steel guts inside. I KNOW it wasn't attracted to the frame itself.
Touched the magnet to the triggerguard away from any steel parts, no attraction whatever.
Do your test again, touch the magnet to the triggerguard, or the base of the gripframe, away from any steel internals.
Denis