I'm not sure that one negatively impacts the other.
I think this may have come about because of our schedules. We usually only have X amount of time for shooting related activities, and rarely is that X large enough for you to both do shotgun and rifle/pistol. So shooting one type means you aren't practicing the other, and skills DO atrophy.
Let's say you have 3 hours per week that you can spend shooting. By the time you go to the range and shoot, you have to pick one or the other. If you're like most people, you get on streaks- a couple months will be shotgun, a couple will be rifle/pistol. So when you get back to the shotgun, you've lost a few birds because you weren't practicing.
I've been doing a lot of shotgun this year, and I've been getting decent. However, that also means I haven't shot rifle/pistol much at all, so the next time I manage to get to the range I expect I'm going to have quite the time getting myself dialed back in.