Just don't do it.
That's the best advice. There is no reason you should overlook the cheapest component of the whole operation, a small square or round piece of cloth. The patch does lots of things.
1) It keeps the ball tightly squeezed in the rifling, holding it against the powder so you don't get a dangerous situation with airspace between the powder and the ball.
2) In squeezing it against the rifling, it imparts the spin of the rifling to the ball and stabilizes the ball in flight. A ball just "rattling down the pipe" doesn't have spin on it and, well, you've seen knuckle balls pitched in baseball, right? Same thing.
3) The patch transfers the lube that hopefully you have used with it, and makes loading easier and also cleaning the bore easier after the shot.
4) It aids the patch in sealing the rifling behind the ball and transferring the "push" of the burning powder to the entire rear surface of the ball, rather than whistling around it through the rifling and airspace between ball and barrel (blow-by).
5) There is a good chance that if the powder isn't compressed with a nice tightly patched ball, it won't ignite (misfire), or only partially burn and not completely eject the ball, another problem in itself.
6) If the ball isn't patched, as mentioned, you won't have a nice tight seal, and if not fired very soon after loading, the powder might absorb moisture from the air and not ignite properly or at all.
7) The ball might just roll out of the barrel and embarrass you in front of the other guys at the range, whom will then blackball you from the club for not using a patch in the first place.
8)Just don't do it.