You must be shooting a revolver, right? What is the make and model? Bought new or used?
The two common sizes of cap are #10 and #11. #10 caps are slightly smaller than #11 caps. If you're using #11, get some #10 and see if they fit better. They should fit snugly, that is, you should have to press a bit to get it all the way down on the cone of the nipple. WARNING: Do NOT push the cap onto the nipple with your finger or thumb. If the cap sparks from the pressure, the powder in the charge hole will make an ugly mess of your finger or thumb, not to mention send the ball out the side of the frame. Get a good quality capper or use something like a Popsicle stick.
If the caps split when you press them down, then the nipples are too big. Take them out, chuck them into a drill and slowly and carefully sand or file them down, checking frequently, until the caps fit snugly.
Besides being a pain in the butt, a cap falling off means the nipple is open and when the cap under the hammer fires, it could send a spark into the other nipple and set off the charge; this is called a "chain fire". It's usually not harmful to the gun, but it is scary and can be dangerous.