agree, great question... I'd probably leave mine in also, since the buck shot is smaller diameter than the bore, probably a little tighter group with the choke in ???
as an aside... I have a custom 444 barrel, made on a 44 Hot Shot barrel ( TC's offering using a 44 mag case, & a huge shot tube to equal a normal shot shell ) I load my 444 cases like a 3" .410, & load a shot load, a .375" round ball load, & light 444 bullet loads, as this barrel also has the straight rifled choke tube... I do get tighter groups with my 444, with ball loads, with the choke tube in ( at 15 yards, pretty much shoots a 1" hole, with the choke tube, & a 3" group with it out... hotshot choke tubes screw on the outside... buck / ball loads might be a little hard on the threads of a screw in barrel, with internal threads, like the .410 barrel