Since I live in the middle of the woods(all posted,private property) and have run trespassing slob hunters off, literally within feet of our yard boundary, hunter orange is worn during firearm season hunting or not.
If I'm out and about in the yard or woods, carrying a gun or not, I've got a hunter orange hooded jacket that just becomes normal wear.
Too, having had several close calls , again caused by trespassing slob hunters hunting squirrels with a .22 , while I have been sitting in tree stands during our deer bow season in which I was totally camo'd out, I'm considering wearing orange during our bow season as well.