Like most have mentioned the alloy will determine the upper end of your velocity to a point, then fit and the pressure of your load will take over.
Shooting cast is simply a balancing act of getting things into the right order to balance out and work like you need them to. Some powders with X alloy will shoot just fine but others wil lead up terribly. Same with the alloy, some will shoot fine with everything under the sun, but some won't shoot with anything but a specific low velocity load.
I hunt with a variety of handguns from .357 up through the 454. As such when I started out casting I initially wanted to duplicate the jacketed loads I knew worked from each revolver. This said that involved some alloy testing which included both successful and miserable failures. So far I have managed to get within a couple hundred feet per second of my top end velocities with all but the 44 which I haven't messed with much, mainly due to having so much fun with the others.
In my .357 I found if I wanted the top end velocities, I had to use the slower burning powders for starts, and use an alloy with at least around a 10'ish BHN. Through some small trial blends I managed to even get an alloy which expands very well up to just over 1300fps, which can be seen here,
M p 358-640 hp The media in which these were shot into was probably akin to being driven through the thinner part of a shoulder blade on a deer. They were shot through the lid of a 6 gallon plastic bucket which was filled with very fine dampened sand similar almost to talc powder. When I say damp, it had about 8-10 oz of water soaked into the whole full bucket of sand so just wet enough to clump if you grabbed a hand full and squeezed it but not wet. I have found this will give me a very similar results to bullets I have recovered from deer and hogs.
Anyway, I suggest you try the powders like Unique, 2400, AA-9 is what I use but #5 is also a vbery good one if you have either. Test your loads with a clean barrel between each round so you will detect if your going to have any leading. If you find that you see a build up in the4 first inch or so of your barrel after 5 rounds you might as well try another powder or drop that particular load, or call that the top end for which ever load you have worked up to. If you see lead on the muzzle end your loosing your lube and might simply need to add some tumble lube via Lee Alox. It is simple to use and cheap aliet a bit messy. It does however work, I have used it to drive some 454 bullets up to 1700'ish FPS with it and no ill effects.
When you start out just load 5 rounds up from the start data, increase in .2gr increments. Open your cylinder after each shot and look down the muzzle to verify no leading is started in the breech end of the barrel. It will be pretty easy to see and look like skid marks in the rifling. In a worst case it will look like this, (this was after only 4 rounds)
I knew going into this the alloy might not handle the load but I was testing and when testing you have some failures. It does scrub out pretty easily if you catch it at this stage though, however the more you lay down the harder it gets to remove. Worst case get some Chore Boy Copper pads and wrap a string of it around a bore brush and go to town. Just make sure you check the pads with a magnet as there is a LOT of it out there which is only copper coated or plated steel. You don't want to use that in your bore.
That said if you see this, everything should be fine, this one had more than a hundred rounds through it when the pic was taken,
Good luck, hope this helps and get back with results and or questions after you get a few down range.