I prefer handguns. Why? I do not know. I got into guns with rifles; but like another poster stated, I am right handed but left-eye dominant, so handgun shooting is more natural for me, whereas I have to close an eye to shoot rifles or shotguns. I can shoot a handgun in either hand passably though my performance is dramatically better two handed. I can manage a pretty quick snap shot with a rifle, but my follow through sucks because I have to choose between seeing my surroundings and keeping an eye in the scope, plus, that eye being weaker it takes me longer to focus in. I am a decent shot with a rifle, I can put up a 2-3 group from a semi supported position at 100 yards shooting five rounds quickly. Ok but not great.
Handguns are just more fun to shoot; it is faster paced but also less involved. I stay proficient with my rifle because I hunt with it and while I do enjoy shooting it the set up is always more of a chore, for example the 100 walk between groups. With a pistol I often set up several targets that I will double-tap one after the other, ending in one of those spinning metal targets. There are all kinds of "drills" and games I play with handguns that I wouldn't with a rifle. I would never ever give up my rifle, but I have more fun shooting handguns. In contrast to some others here I actually maintain rifle skills in a utilitarian way more than anything because I think a grown man ought to be able to shoot a rifle passably. With handguns it is more fun, it is a quick set up and the time between mags and new targets is much smaller.
I also love shooting clays.
Now, all that said, part of my problem with the rifle is that I only have access to a 100 yard range, so I don't feel at all challenged by it really. I sometimes shoot my target model Ruger MKII at 50 yards, and it is an interesting challenge. 100 yards with a Mauser though? It's too easy.
Overall, handguns, but I wouldn't ever let the long guns go.