You'd have to work real hard to hurt the temper of a rifle barrel with a propane torch, absent deliberate vandalism. Remember, many sights are held on via silver-solder.
I was more concerned about discoloration of the finish, but careful heating for a small time shouldn't matter.
(For that matter, you can hold the end of the barrel over a low flame of a gas kitchen stove for just a short time.)
Short time = ten to twenty seconds. Just enough to get some expansion of the sight, without heating through it to the barrel. The torch provides a more focussed flame.
Art