Interesting. I am dealing with exactly the same issue right now. My previous (38 spl) loads, using HP-38 and 125 grain plated bullets....were in the range of 4.2 to 4.4 grains. I also found excessive residue/ unburned powder with these loads.
I have since bumped my load, with the same 125 gr. bullets, to 4.8 grains. I am no longer seeing unburned powder myself.
Incidently, 4.9 grains is, per the Hodgdon website, a maximum charge. However, even though it is listed as maximum, the published chamber pressure is in the 15,000 psi range. Normal maximum standard load pressure for 38 spl is around 17,000 psi. So, it is my belief that there is wiggle room with the maximum. I have not tested this myself....as I have not needed to do so. But, if unburned powder shows up with a load of 4.8 grains....it is indicative of low pressure.
So, the OP MIGHT want to test, CAREFULLY, charges a bit above 4.8 grains. If so, watch carefully for any signs of excess pressure. But, he might find that the unburned powder issue might go away with, say, 5.0 grains....with no ill effects. Just a suggestion.....
I can say, without fear of contradiction, that HP-38, when used in relatively large cases, like 38 spl, does NOT like light charges. HP-38 does not seem to be nearly as flexible as regards charge weights, compared to powders such as Bullseye or Titegroup. In smaller cases, such as 9mm Luger, HP-38 does not seem to exhibit the issue.