Pure lead outide the body isn't much of an issue. Lead is extremely stable. You need a reactive chemical, such as gastric acids, in order to make it dangerous. Lead became an issue because industry started putting it in products with other chemicals. Gasoline, paint, etc.
Lead pipes are mostly 75+ years old. If they were losing any significant amount of material they would have been long gone.
There was an informal survey here several years ago where several people participating in different shooting activities were tested. Reloading, casting, shooting, etc. Although very limited in scope and not all that scientific, the only pattern that developed was those shooting indoor had "dangerous" levels. It was suggested that lead is used in most primers in combination with the exact type of volatile chemicals that would make it dangerous.
I too am past my brains "developmental stage." I have been working with pure lead rotor weights at work recently. I am not at all worried about it. I just make sure I wash up well before breaks, and try to keep from disturbing dust. It isn't like pure lead floats for long. Use PPE of course.
IMO, don't shoot indoors and avoid ingesting it with regularity. I imagine you would have to survive off of small game harvested with shot for an extended period of time in order for ingestion to be a problem.