You didn't specify whether you have a percussion sidelock or an inline with a removable breechplug, or what type of powder you were shooting with.
People clean their guns in different ways, some use hot soapy water and some use black powder solvents.
You need to have some of the right gadgets to help get the job done.
A caliber appropriate cleaning jag and a patch removal "worm" to help you remove a patch if it comes loose down inside the barrel, is almost a necessity if it's a sidelock. Sometimes the jag/patch combo. will get wedged into the barrel along with your cleaning rod if the patch is too oversized, so I use an undersized jag and larger homemade patches. A .41 caliber jag virtually fits almost all of my larger caliber rifles and I use a lot of cleaning strokes to remove as much fouling as possible. But you have to use a tight fitting patch, but not too tight to get stuck, so I cut my own to size.
Some folks fill up a small tub or coffee can with hot soapy water and stick the breech end of the barrel into it with the nipple removed. Then the suction of the tight patch being stroked up and down will flush out much of the powder residue. But then you have to dry the barrel as quickly and thoroughly as possible, and then lubricate. Many use a water displacing product like WD40, ballistol, RemOil...but there may also much better products for long term protection depending on personal preference.
I use TC Bore Butter or Wonderlube, but then I don't put water in my barrels to begin with, I use blackpowder solvents. You really have to clean all of the residue off of any threads and where ever it may have accumulated inside and out on your gun each and every time you shoot.
Hopefully your gun only has some surface rust and no permanent damage or pitting will result that will have a noticiable affect. Use a bronze brush on your cleaning rod as suggested and stroke the bore until it's clean with the additional help of any kind of solvent, and then lots of patches until it's as clean as possible. Then repeat if necessary. Brass or nylon brushes help to remove stubborn deposits on the outside (but try not to overdue it with a brass one, although it's usually totally harmless). And try to soak the nipple in something to help keep it clean and free from forming rust that will block the channel, and try to do the same for the drum where the nipple fits. I use pipe cleaners or little pieces of wooden dowel with a piece of patch and use a twisting motion to remove stubborn deposits in there. That's where solvents can help.
Make sure everything gets coated with a good lubricant, and after a few days I sometimes will go back and stroke the barrel a few more times to insure that I removed all of the residue and that no oxidation is occurring.
I'll even try to remove any residue from the exposed areas of the lock and mainspring located in the stock and under the barrel, and apply oil with a cotton swab.
Good luck and remember that half of muzzle loading is about keeping the gun clean. The culprit is corrosive salts that are the by products of shooting powder containing sulpher. Even the "non-corrosive" powders that don't contain sulpher need to be thoroughly cleaned so that the residue won't absorb moisture and cause rust that way. They may give a person a little more time to do the cleaning, but they don't always perform as well since, every product has it's advantages and disadvantages.
There are other methods people do use to help smooth out a rough barrel, including using toothpaste (silica), bore paste, valve lapping compound (water soluble), # 0000 steel wool, 3M teflon cleaning scrungy pads etc....but some of these methods should only be used on barrels that have problems that are either from more long term damage or are of a more distinctive nature where an alternative remedy and their use may be required to solve a problem. You also don't want to end up having to spin a cleaning rod with an abrasive material attached inside your barrel with an electric drill unless it's absolutely necessary.