Lots of reason, lots of history...
There are a lot of reasons that go into the general preferences of the buying public. And tied in with that are the products made and sold.
The capabilities, and drawbacks of the early designs form a huge opinion base, for both good and ill.
The biggest reason pump shotguns are virtually universal and pump rifles are scarce is simply that the pump action gives a greater benefit in shotgun use than it does in most rifle use.
And you might also notice that there are a huge number of successful lever action shotguns out there, too. (Sarcasm)
Size does play a part. The handing does too. Also, who's design got there first, and how well it works.
In the "deer rifle" market where I grew up, Remington pumps and semi autos competed with Winchester and Marlin levers. And, when it came to who sold the most, of what, Remington lost.
Lots of reasons, here's a few...
pre WWII you could get lots of very good Winchester, Marlin, and even Savage lever guns, and by and large, they could all use the common ammo (which was mostly Winchester designs). Remington had a line of their own. Remington guns only took Remington ammo.
That's a factor in popularity right there.
Out of the entire line of Remington cartridges, .25, .30, .32, & .35 Remington, the only one that has survived commercially has been the .35Rem. And that's because 1) there was no directly competing Winchester cartridge, and 2) It was chambered in a popular Marlin lever action rifle!
After WWII, when sporting rifle design and manufacturing began again, Remington got smarter. Their next generation of pumps and semis could take advantage of the full range of standard rifle cartridges, .30-06 length.
This opened up wider market, but one that was already well filled by the bolt action, and where the speed advantage of the pump over the bolt was essentially nullified.
The semi auto's competition is the other thing that ensured pump rifles would never dominate the market, absent some artificial rule or restriction. Simply because a lot of people who would buy a pump rifle would buy a semi instead.
I believe Pennsylvania has a game law forbidding semi auto rifles for deer hunting. The pump rifle is very popular there. Still lots of levers and bolts too.