Definitely trickier for women, as they TEND to not wear carry-friendly clothing
My wife does not carry, but she has commented that it would make her nervous because she may set her purse down or drop it in the grocery cart, or its not necessarily under her control at all times. We have kids, they shouldn't be going through her purse, but most kids do at some point. Same reason I don't carry in my laptop bag nor backpack.