Let me start by saying that I'm not a Zelda history/timeline/storyline buff ... not at all. However, I think I can introduce one common-sense aspect to this discussion that may help everyone out...
First of all, consider the base/common name of this game ... the LEGEND of Zelda, with emphasis on LEGEND.
Legend is built on Lore, and "history" got it's literal name (the word history) from "His Story" ...
We all know that over time, before writing/print that HISTORY was told via verbal stories. There weren't necessarily drawings, plans, blueprints, dimensions, etc. People would have just talked about places, things, temples, forests, etc, as well as what happened with people, heroes, enemies, so on and so forth.
Keep all that in mind when viewing this game, and any other Zelda game. Not every detail needs to be identical to what happened before. Landscapes change over time, at least to a degree. Buildings do start to fall apart and need to be refurbished, renovated, or even rebuilt.
In an effort to keep this post from being too long, I will sum up by saying that there's no real sense in trying to argue about such trivial details about the location of a temple, or its relation to another place from a past version of the game ... or anything else along those lines. In THIS game, the Temple of Time is the Temple of Time, period. There is no interpretation required or even worth pondering. It is what it is.
As for the Forgotten Temple ... I don't know what it is, what it was, or if it even matters. No, wait ... it DOESN'T matter ... there's no significance to it in this game other than it's one more place you need to go to find a shrine, and then eventually go back to in order to get the reward for finding all the other shrines... and, much less importantly, to find a couple of treasure chests and a couple of Koroks/seeds.
That's it ... no need to try to make it something it isn't ...