I am not a .223 user, but I can tell you, from a technical point of view....yes, 20" is just fine. The .223 round, in any loadings of which I have heard, does not use a slow enough powder to necessitate a barrel longer than 20". That is the real factor here. A longer barrel will NOT enhance accuracy. That concept is an "old wives tale", so to speak. As long as the twist rate of the barrel in question is proper for the cartridge and bullet weight used and the barrel is long enough to allow for complete combustion of the powder used in the load....one is fine with the minimum length necessary to fit those criteria (except as specified in the next paragraph). In fact, a shorter barrel (within the needed limits) will often produce better accuracy, because it is stiffer by nature and less subject to barrel harmonics.
For LONG range shooting however, the barrel length needs to be chosen also to try to ensure sufficient remaining velocity of the bullet, as it reaches the distant target. So, for distances greater than 300 - 400 yards, it might well be better to choose a barrel at the long end of the acceptable range. Perhaps 22" would be better - but most likely 26" would be too long. Whatever the optimum number might be, for the .223 (and the distances at which you will shoot it), I do not know (as I don't shoot that cartridge). No doubt someone here will be able to assist you with that specific info.
The OP mentioned a "varmit" rifle.....so presumably he intends to use a scope (probably exclusively). So, additional barrel length just for a longer sight radius (for iron sight use) is probably moot.