The least damaging is to make a scraper from brass.
You can use a rifle case by smashing the case mouth together, or do what I did, and make a tool from brass rod that looks like a brass screwdriver blade.
This allows scraping the lead off without damaging the finish or scratching the soft aluminum.
Another method is to use solvent and a brass "toothbrush".