For .38 and .357 both, I use 3.5 grains of Titegroup and either a Lyman 356637 (147 grain BB RFN, a 9mm bullet really, but I had hundreds of 'em on hand.) or an Ideal 358416 (158 grain RFN). In .45 Colt, I use 5.5 grains of Titegroup and a Lyman 454190 (250 grain RFN). I haven't come up a CAS load for .44-40 yet, but I'm working on it.
In that relatively big .357 case, 3.5 grains of powder isn't much. I don't use any filler, and I haven't had any problems so far. (fingers crossed)