Okay, bad guy is in your house. He is armed and paranoid. You decide to scare him off by racking your shotgun. Your intent is that he will be scared off by the awesome sound that obviously indicates superior firepower. Instead, the bad guy just unloads his pistol in the direction of the racking shotgun. That is an example of the violent type of response that can result. By racking the shotgun, you have given up the opportunity of surprise and have given the bad guy insight into where you are located.
You're making a whole lot of big, general assumptions.
assume that he is armed, paranoid, ready to unload his firearm into the dark just because he heard something, assuming that it is dark, assuming this and that?
You seem to also be clearly saying that a person in a defensive situation should sit quietly as a mouse, retaining the element of surprise, rather than trying the alternative of maybe even shouting "get out, or I'll rack my shotgun at you!"
I don't suggest it as a tactic, but it should be obvious to everyone that in at least SOME cases, just the sound of a weapon cocking will cause some unarmed and even armed criminals to flee instead of fight. a little old lady with a .25 auto scared off a couple of teenagers here recently.
It's equally obvious that at least a few of the crazier tweakers out there will decide that hearing a shotgun at the end of the hall would be a good reason to mow down as much of the house as possible, rather than risk taking a round from that shotgun.
In some cases it WILL work, just read the police reports. Sticking a laser dot on an opponent sometimes works. Sometimes a toy poodle will drive off an armed intruder. Lots of things work, it all depends on your opponent.
In some cases it won't work. it will, in some cases, cause escalation in conflict; so will inviting the guy in for a cup of tea. maybe he hates tea.
without statistics, arguing that one scenario is more likely than the other, or that one is more likely to get you killed, is ridiculous.
If the guy wants to rack his shotgun and hope for the best, that is probably safer than asking him in for tea.
I see little to gain, or lose, in a general, unknown situation.
My personal actions will probably run along the lines of telling them leave or die and that 911 has been alerted, and then, doing whatever I have to to make them less of a threat. Unless the invader is simply an assassin, or very determined to take whatever I have at any cost, they will MOST LIKELY leave, rather than take the chances of being shot. I will give them the option to survive, and the option of not spending prison time if they kill me.
any home invasion scenario is flying without a manual.
Of course, in many states and circumstances, the answer is obvious. He has invaded your home. You fear for your life. Kill him and get it over with. That's why we have castle doctrine. No need for a warning.