Letting your guns get rusted is unrelated to cleaning (the bore) after shooting. One is neglect, the other hurts nothing.
I used to clean my guns every time I used them. I *hate* cleaning guns. It is not part of the fun or the experience to me. It is dreaded. Despised. Loathed.
Modern powder and primer residue is not hydroscopic to any significant degree. In fact, a coating of powder residue is more likely to PROTECT your bore than harm it.
More guns will be damaged by improper cleaning than by not being cleaned. It can be no other way, since not cleaning them can not hurt them while cleaning them can. It doesn't matter how unlikely it is, it can't be less than the zero chance you have by not cleaning.
Keep the outsides and functional parts clean, dry and appropriately oiled. Clean the bore when (if) accuracy degrades.
My only exception is a break in that I do, which involves cleaning the bore after each of the first 10 shots and then after #15 and 20.
My guns stay accurate and do not copper foul for many rounds, if ever. The bores still have a mirror sheen when cleaned and with many powders still shine even after shooting.