Personally, I think the 'gap between dimensions' definitely doesn't refer to somewhere literal, as in between Hyrule and Termina, because I just thought that Termina existed in the same 'realm' as Hyrule, or at least running on the same timeline. Termina wasn't mentioned in OoT because it wasn't relevant at the time as the game was only about saving Hyrule and not anywhere else.
The reason the twilight wasn't mentioned in OoT would probably be for the same reason, furthermore, the people living in the world of light in that game don't know about the world of shadows which is why they're so confused when weird bugs and stuff appear. This is why it is Link who is the hero, because he is the soul person who does not just appear in spirit form in twilight, but as a wolf.
Anyway, I don't think Phantom Ganon was banished there ever. I think possibly SS needs to be incooperated at this point as it gives the backstory to Hyrule's original creation and also the return of Hylia's people to the surface (I hope this isn't giving away too many spoilers for that game). Anyway, all three (MM, TP, SS) considered, I would say his phantom was possibly banished to some time before the events of OoT, perhaps just as the essence of demise or even in a third timeline. This is just speculation though.
^^ Maybe that's where ALttP fits