Theres no Gander Mountain in VT, but there is one like 20 miles from me in Plattsburgh NY, I can and do legally own both handguns and rifles in VT, can I purchase a gun in NY?
Not a handgun - at least, not in person. You can purchase a handgun from an out-of-state vendor, but the sale has to go through an FFL in your home state.
I believe you can purchase a long gun directly in NY STATE with no permit or anything. As long as you are legally allowed to posses one in your home state.
A pistol however will have to be shipped to an FFL in your home state and transferred there. But like Musketeer said, they most likely won't even let you touch it without you showing them a NYS pistol permit.
None of this applies in the city of course. There you pretty much can't buy a gun. I was in a shop in NYC wanting to look at a Mini-14 and they wouldn't even take it off the rack for me because I didn't have a permit.