Stressfire,
As a native New Yorker (Upstate), I have to say right off the bat - DO NOT bring any firearms into New York State. Only state residents may possess firearms within the state's boundaries (there is a small exception for visiting hunters, but that only allows them to borrow a gun from a friend who is a NY resident for the duration of the hunt).
OC/pepper spray is also heavily regulated, although I don't know the exact specifics so I'll just have to say be careful with regards to that.
I'd be careful of any folder over 3" without knowing what the local regs are. Some areas will regulate more than others (especially the more populated upstate cities like Buffalo, Rochester, Syracuse, and Albany).
I have to third Don H's and Aarond's suggestion about a cane/walking stick. With a little knowledge of how to use a baton, they can be a very effective defensive tool without raising any warning flags or getting you in legal trouble just for having one (and canes aren't illegal in NY - yet - Bloomberg hasn't decided wheelchairs are better for your health so far this week).