I don't think I have any stainless bbl'd rifles at the moment- but no more than what I do with rifles these days, I don't think it makes a whit of difference to me.
Most of what I'd heard between the two is just stuff I've read on the net and cain't be backed up or substantiated anyhow. But, I did read that stainless bbls can be finished smoother on the inside during the birthing mode. I've never hammer forged, button rifled, or gave birth to one so I just don't know. I've also heard they last just a wee bit longer on average, but again- I don't know.
To me, I look at the overall asthetics of the rifle- or even price and availability. But I do try to go with what catches my eye and looks the rightest.
edited for clarificationses.