Under what scenario would you be best off using the shotgun?
I have practiced enough with a full-stocked shotgun that I can walk with it ready and sufficiently ready to retain in a struggle. Sure a long gun makes it a PITA to negotiate doors (closed to save heating), clear rooms in the narrow hallway, etc.
However. If I have to light an intruder up, much less in MY OWN HOME, I want the best tool for the job. Yeah, M4's have replaced the combat shotgun on all fronts, but still ,when each shot has to matter, my shot is low recoil 00 Buck. There is just no disputing the stopping power, within HD ranges. I just feel better stacking the deck in my favor.
I also plan for the local m.o. That is a violent entry of multiple armed assailants. You could likely have to engage two, maybe more, criminals before they flee. Almost always this is just punks looking to head up on their supplier or dealer, probably the occasional rich folks in a nice home. My home is the 16th most dangerous city in the US (and all violent crime, property crime is below average), good deal of murders go down 2 miles down the road (between home and work) and while that makes me a more likely random target, lifestyle choices that avoid trouble and BG interaction allow me to feel safe in my own home. But you can never be too careful. i have to think though, those crack dealers and whatnot, while lacking any gunfighting skills (they do know how to pick out the iron though, .223s and SKSs are pretty popular with BGs here,apparently, they turn up in raids regularly), they have an advantage in staying sharp-the realistic expectation that people could bust in and start shooting. If not, even having already experienced shooting someone, that subconcious question or hesitation gone. And these are the people we'll be dealing with! This is why I trained in scenario terms. Visualise something, most likely situations, and work out the best way to address them, then repeat several times. Hoping it will help reaction and efficiency, and buy time if I go on autopilot. Revert to what I practiced, so I keep it simple and practical. I have always been interested in the mindset aspect too, studied great gunfighters (and other combatants too) throughout history trying to find something that might help me survive in that situation. A loved one could be counting on that one day.
I live alone and am perfectly fine answering a pounding/screaming at the door in the middle of the night with a 12 gauge, or a window being messed with... Something that out of the ordinary is cause to roll out and have ready the artillery.
A shotgun should at least be kept to fortify a safe-room. Even a 20 gauge is not a huge compromise.
The Carbine is a good choice too, and IMO better than a handgun anytime you have the time or opportunity, and elbowspace to use one. Handguns have traditionally been the close quarter gun. History, well common sense in fact, shows that the rifle (and SG) is the superior fight stopper. More power. Better hit probability, hell, accuracy potential. More control. I am not blowing off handguns nor consider them no good for anything but carry and backup. A fine tuned 1911 (or other fine shooting SA, or a striker, or TDA) is IMO a fine choice of a HD weapon on it's own, if you are satisfied with the limited power a handgun can offer, and you trust your skills with it. I don't feel comfortable with either.
I used to use a Mini 14 with 18x 40 gr. JHPs/unchambered as my first resort HD weapon, though the Mossberg and .45 were loaded too. I was concerned with overpenetration, as I then lived at home with my parents and the threat would likely come from the direction of their room.
I wouldn't use that load today. They may virtually explode but they're still just 40 grains. .223 is the heaviest I'd go indoors whether in a cabin or thin walled tentament. Many cannot understand a handgun cal. carbine which is understandable. But the mildness of this setup is a big advantage in weapon control, no giant muzzle flash or ear splitting report to further disorient you in a fight or flight altered reality, no getting rocked momentarily by a 12 gauge recoilling. Sort of a .22 with real punch on the muzzle end. For the portability you give up, vs. the same round in a pistol or revolver, you get something that increases the power of the shot, and the chance it will go where you want.
I would use the handgun, as a backup (tucked in when I go for the shotgun), to replace a dry or jammed primary, or pull if a disarm was going on. In the event it would be faster access (meet the threat ASAP with whatever you have adequate). Or, having one handy when I walk into the house after work, or later at night. Carrying at home sounds more and more like a good idea.
I just know I for one will have a lot better of a chance if using a rifle or shotgun. Could be from training more with them, but... say a 25,30 yard target. That's cake for anyone with a M4 or 870. Same target, no stress, the results will be a lot different using your best pistol. Does this apply to HD, those ranges? I think so, just pointing out the more inherent 'hitting potential', magnified under stress. Headshots with a carbine are not the likely miss that they are to many pistoleers fighting for their life (Just a point, COM is the way to go). People don't miss as much with long guns, they're just a lot easier to use effectively. And when they strike the target, more likely gets the results you need. To me those are definate advantages that matter, and take priority.
What works for me, and why, no advisory intended
If you think that you can't drop a bad guy with a round heavy enough to stop him because you are worrying about possibly hitting a family member or neighbor, use a knife or invest in a state of the art home security system and don't waste your money on guns and ammo. When you commit to using firearms there is always a chance of someone getting hurt or killed that you didn't intend. IF YOU ARE SO WORRIED ABOUT THIS TYPE OF ACCIDENT, DON'T USE A GUN. 