I hold that after OoT a bunch of people (maybe refugees from Kakariko, or who knows) settled in Lon Lon Ranch and built the new Hyrule Castle just above it. I also claim that the Temple Of Time in TP is the same ToT as in OoT, but in a different place. How does that make any sense? Well, warps...