Don't get me wrong, I like my 20 gauge, but if I only had one shotgun it would be the 12 gauge. Even if you don't like the 12 gauge recoil with some loads there are always lighter recoil loads available for it. With the 20 gauge, the choices are very limited, usually you have to take what you can find.(see my shotgun slug thread question down below)
You should definitely drop Hogdogs and Zippy13 a PM before you decide, I think they've shot every type of shotgun and shell in the world.