Most simply, if you only look at measured caliber, the .40 is going to be 'bigger' as the 9mm is commonly sized at .355 or .356 for reloading purposes.
I would imagine that the .40 derringer has to be larger to contain the pressure (~28,000 cup, IRC) behind the larger bullet weight, and although the 9mm generates higher pressures, it would likely have less recoil (smaller bullet and powder charge).
HTH.
