Small pistol, not a lot to hang on to...short sight radius, rather rudimentary sights, and I'd venture a guess at a less than match grade trigger pull....
Some people can overcome these drawbacks to accuracy better than others. Ask that guy who gets "quarter size groups" if you can shoot his pistol, or at least what ammo he's using.
Heck, ask HIM to shoot yours. If HE gets quarter size groups with your gun, the problem isn't the gun...
Generally speaking, the usual reason pocket guns are not very accurate isn't mechanical, its because they are difficult to shoot well for most people.
Have a few other people shoot your gun and see what, on average, they can do with it. That will tell you if the gun needs work, or if you do. And, its not unvommon for even good shots with "regular" guns to not be as proficient with the small pocket models. Some folks are, some just aren't.
Personally, I shoot bigger guns, with better sights and triggers better than I do the small ones. It is an individual thing.