.38 Special has a bullet diameter of .357", but earlier ".38's", including those cartridges made for use in converted percussion revolvers, used heel type bullets of .375"-.380", the same diameter as the bullets and balls used in ".36 caliber" percussion revolvers.
Once the designation ".38" caught on for the rimfire calibers, the same name for subsequent cartridges followed naturally, even when it was as inaccurate as ".36 caliber."
As for "odd numbers not used to [sic] calibers", how about .25, .31, .35, .41, and .45?
Jim