Depends on what the gun is for. Defensive factory .357 ammo is tuned to work out of 4" barrels because that's what people usually carry. For defense purposes, 4" is perfect.
If you're using non-expanding hardcast hunting ammo, or hunting JHPs that are "deep punch, delayed expansion", then driving them faster doesn't hurt and often helps. It gives you better range, and more "bone shattering" ability.
For target, faster means a flatter trajectory.
As the barrel gets longer, the sight picture gets more accurate but takes a bit longer to line up. Shorter tubes line up faster, which is why a lot of people like combat wheelguns to have a 3" or 4" tube. Which is why the ammo makers optimize the JHP expansion for that barrel length.