If one is careful about what the gun is loaded with and can handle the recoil, a .44 Magnum can make a fine self-defense cartridge. The problem comes when someone chooses a gun/ammunition combination with too much recoil and/or uses overly penetrative ammunition.
A .44 Magnum, while certainly powerful, isn't a howitzer and there is suitable ammunition that can be used. What you want is an aggressively expanding bullet that will reliably open in an erect biped. Loadings that have good reputations for those qualities include Federal 180gr SJHP, Winchester 210gr Silvertip, and Remington 240gr SJHP. Hornady's 180gr Critical Defense and Speer's 200gr Short Barrel Gold Dot also look promising though I don't have any experience with them myself.
What you need to avoid is JSP bullets, heavy-for-caliber bullets (anything much over 240-250gr), or "tough" JHP bullets specifically designed to penetrate deeply on large, heavy game. These types of bullets are designed expand moderately at the higher velocities achieved with long barrels and will give little, if any, expansion in an average person particularly when fired from the shorter barrels most likely to be found on a defensive revolver. Obviously, non-expanding bullets like FMJ or hardcast are to be avoided for much the same reason.
Also, don't fall into the trap of loading too powerful ammunition in too light a gun. While the nuclear-level bear loads from the likes of Buffalo Bore, Cor-Bon, Double Tap, and Grizzly make produce impressive numbers, the recoil of such is likely to impede your speed and accuracy to an unacceptable level in anything but the largest and heaviest revolvers. If you carry one of the lightweight guns, this is even more true and you should probably be looking at the lower-end .44 Magnums like the aforementioned Speer Gold Dot or perhaps even .44 Special ammo. Personally, the 240gr Remington SJHP at 1180fps is about the most recoil that I'm comfortable with in a defensive load from my 4" S&W 629. If I had a lighter revolver like a Mountain Gun or 329PD, I'd likely be looking for something with considerably less recoil still.
Furthermore, one needs to understand that, for most people, being able to shoot a .44 Magnum effectively isn't something that's going to be mastered overnight. There is substantially more recoil than most popular defensive cartridges and it takes a dedicated shooter willing to expend the time and money (ammo isn't cheap) to master it. Don't think that you're going to transition to it as smoothly as you would from a 9mm to a .45 ACP and don't try to shoot 100+ rounds in a sitting unless you want to develop a severe flinch. A full 50 round box, and maybe less for some people, is a full day's shooting and a couple cylinders full at each range session isn't a bad way to build up your skill if you don't mind taking a while to do it. Be sure that your gun has a comfortable set of grips that fills you hand and distributes recoil too.
All that being said, a .44 Magnum is a very formidable weapon in the hands of someone who's invested the time, money, and thought to build skill and load appropriate ammunition. Report and flash, while considerable, are no worse IMHO than a .357 Magnum, .357 Sig, or even a .40 S&W loaded with light bullets. Likewise, recoil, while substantial, is noticeably less than many of the small-frame, lightweight .357 Magnum revolvers like the Ruger LCR or S&W 340PD which seem to be so popular. Overpenetration, while a concern, isn't the boogeyman it has been made out to be and is best addressed by observing Rule #4 (be sure of your target and what is beyond it). This becomes particularly evident when one considers that any bullet which meets the FBI's penetration criteria of 12-16" (as most premium JHP's in popular defensive cartridges like 9mm, .40 S&W, and .45 ACP do) will very likely pass completely through the upper torso of the average person in a straight-on shot since most people are less than 12" thick at the torso, much less in a shot to an extremity which is even thinner. So, I suppose my point in all this is that, while a .44 Magnum certainly isn't the defensive gun for everyone, that doesn't mean that it's not a good choice for anyone either.