It is not just military arms. Did you ever try buying S&W sideplate screws from anyone but S&W?
Arms makers began the whole industrial process in this country and made up their own sizes as required to meet the need to be proof against vibration, etc. Before standards were ever set, most gun makers made their own screws, so there were often as many sizes as there were companies.
In fact, 1/4 x 28 is National Fine and one of the more common American thread sizes. I didn't know it was "elusive" except maybe at the most ill-stocked hardware stores.
The trend to metric is ongoing, but mostly in international companies (e.g., GM). Small companies may be a while getting on board. The metric thread system, incidentally, is not clean and neat either. Thread pitch is only part of the story - thread angle and shape is the other, and metric stuff is all over the map.
Jim