Unless you know specifically when you are going to need to defend your home, you would have to wear it all the time. I doubt a burglar or home invader will wait for you to put it on.
That is just plain silly. Plenty of situation do not develop that quickly or where there isn't time to take precautions.
For example, this guy was on the phone with 911 for 4 minutes before shooting an intruder...
https://komonews.com/news/local/i-h...1-call-released-of-homeowner-shooting-burglar
This woman was on the phone with 911 for 21 minutes as two men tried to gain entry. She had time to get armed, give her baby a bottle, and talk extensively with 911...
https://abcnews.go.com/US/okla-woman-shoots-kills-intruder911-operators-shoot/story?id=15285605
This woman was on the phone for over 10 minute with 911 after arming herself with a shotgun...
https://oklahoman.com/article/3422809/homeowner-shoots-kills-intruder-in-lincoln-county
These folks were on the phone for several minutes with 911 before shooting through the door and killing the intruder...
https://www.myplainview.com/news/article/Man-who-shot-musician-told-911-he-tried-firing-8717798.php
The hero of the Sutherland Springs church shooting had time to get an AR15 from his safe, a magazine, ammo, and load the magazine while responding to the shooting. He had time to put on a vest...
https://en.wikipedia.org/wiki/Sutherland_Springs_church_shooting
This lady spent time on the phone describing what was going on, when entry was made, the person she saw, etc. etc. She could have put on a vest...
https://www.youtube.com/watch?v=Zbdqyw9sASU
While controversial, John Horn had plenty of time to put on a vest before confronting burglars of a house he was watching...
https://en.wikipedia.org/wiki/Joe_Horn_shooting_controversy
Another...
https://www.youtube.com/watch?v=fB1CPeWvplI
These people had over 15 minutes...
https://www.youtube.com/watch?v=X_CNgS_1rjw
This woman had quite a bit of time...
https://www.youtube.com/watch?v=q5OQ2PMPYu0
No doubt I could find you many more exampless where people had time to do a lot of things before either they shot intruders or ended up dead or injured themselves, but that would be overkill and these will suffice. My point here is that if you don't have a ballistic vest, then you definitely can't access it in a time of need. That does not mean that you will always have time to access it, but a lot of people certainly would have had time. The notion that you need to be wearing it all the time and sleeping in it for it to be useful is just plain naive.
You have time to access one if you hear a bump in the night and decide to go an investigate it as well.