T-shot,
Since we're all guessing, mine is that your improvised shooting position from the window goobered up your cheek weld/gun mount, and you either shot high or low, fringing the critter. Who knows. An odd shooting position can mess up hit probability, even with a shotgun.
While a humane kill is a desired objective in a hunt, I do not see this circumstance in the same light as a game animal and fair chase. That 'yote was not there for a cup of coffee and share your morning paper. He was there for your wife's poodle, your daughters cat, your chicken house, etc. I nearly lost a full grown lab, 90-100 lb male, to a 'yote pack. My vet bill to stitch him up was nearly $500 bucks. Happened across the lane from my house in the dark, not 100 yds away. A single 'yote howled, he left the yard after it, and the pack was waiting. No sympathy for a maurading coyote, and any law that does not let me deal with it on my place, from my bedroom window or other, is a bad one.
Regards T-shot. There was a comment that coyote needed bigger pellets. T-shot is plenty for coyotes at 30 yds, even with steel. The OP mentions a 3" load of steel Tshot. T-shot steel comes 52 +/- pellets to the oz. Not 36. A standard 3" load of Tshot steel will be 1-1/4 oz. That's 60+ pellets of the stuff. And to put .20" dia in perspective, that is bigger (and heavier) than a steel BB your Red Ryder used to shoot. Traveling what........1200-1300+ fps at the muzzle? No way Wiley T. is gonna survive a load of that, centered at 30 yds from a suitably choked gun. We don't know the choke of the gun, (OP may have used his stubby HD gun) and the distance of the shot in question is up for debate, but you do not need larger buck shot to kill coyotes at shotgun distances.
In fact, my own experience with large buck, (OO buck is what I have shot the most) is that much beyond 40 yds, the patterns thin to the degree that getting sufficient number of hits on a deer, or even an IDPA torso type target, much less a target as small and spindly as a 'yote, is a role of the dice. Even #4 buck, (.24 dia, and 27 pellets to a 1-1/4 oz load) is pretty thin past 35 yds, especially if the choke is on the open side and with a simple bead sight.