This is very true. There being four main dungeons has to do with each cardinal direction, therefore it wouldn't exactly fit if they had more. But they were able to fit The Ancient Castle of Ikana. I would not be able to figure out a way to implement another dungeon into the game. The Ancient...
There are five dungeons in Majora's Mask if you include The Ancient Castle of Ikana and exclude The Moon. This is not very many compared to the other Zelda games. OoT and ALttP are generally considered the two best Zelda games made, and they are two that have many dungeons.
Also - Compare it to...