It is an interesting idea and has been done in custom shops. If the law limited you to one receiver, a 1911 or a Witness frame would be the way to go. But, the law does not put such restrictions on gun owners (yet), so I prefer one slide per frame.
Also, to get the most out of a pistol, the slide and frame need to be matched to each other. If you are satisfied with average accuracy and a little slop in the slide to frame fit, then multiple slides are fine. At least until the frame is worn out, then you have to have a new frame matched to multiple slides.
Now consider costs. Bare slides cost only slightly less than bare frames. Once you factor in all the parts that go into completing top ends and frames, the cost difference narrows considerably. A pistol pack that has 9mm, 40 S&W and 10mm top ends is nice, but the 40 and the 10 overlap. Why not just buy a 9mm complete gun and a 10mm complete gun. If you want to shoot 40, a barrel and recoil spring swap put you back in the game. Or cheaper yet, those 10mm mid range loads brought to you by the FBI. Now you have two complete guns for just a little more than one with a bunch of top ends. And, if one frame fails, you still have the other gun to shoot while repairs are made.