Luke, just take it easy, get some ideas and see what works.
Nothing in life is 100% safe, from the water coming out your tap to the car you drive or anything else. EDC is just the same, it is a matter of carefully weighing the options.
Two options that I humbly suggest, as they work for me and I think they are good at balancing the convenience/risk factors:
1) If you want to have access to your firearm at home but you have young kids:
- carry your firearm at home.
- when it is too uncomfortable (my case) stow your handgun in a quick to access push button portable safe like the one I linked above.
- when leaving the house take your EDC with you. If kids are in the car with you, wear it in a holster or put it in a portable safe like the one I linked above.
- All other firearms must remain locked and un-loaded in your firearm safe.
2) If you don't care about having access to your firearm at home, where you have young kids.
- Store your firearm in your firearm safe in a locked room
- when leaving the house take your EDC with you. If kids are in the car with you, wear it in a holster or put it in a portable safe like the one I linked above.
3) if you don't have young kids and you are the only one at home.
- liberally leave your firearms all over the place
(just kidding)
One last thing: you will not forget about your firearm in you carry it in an IWB holster, and if you do, it is not a problem because it is in your possession. The firearm is most secure there. As you take it off, wherever you are (car, home wherever) immediately put it in the safe. Again, portable safes are convenient for that.