Update
Well I loaded some of my .45 Colt rounds with felt wads and some without. I noticed no real difference in performance.
I enjoy cleaning the pistols and did not notice a lot of gunk or lead build up.
The biggest concern I really had was about leaving too much space between the black powder and the bullet. I understand that this is a no-no! I will use the felt wads to take up space on the rounds I make in which the bullet does not reach and compress the powder when I seat it.
Thank you to everyone who helped me with this.