Correct. That’s why, unlike the account of the Four Swords backstory to the events of Four Swords, it doesn’t then go “ages flow by.” It just says that for a period, people thought their troubles were over.
The Zonai’s presence in Hyrule explicitly came to be after, at the minimum, relatively late in the timeline. Considering even the masters of the Lomei Labyrinths - likely some of the first Zonai who were present on the surface considering they predate Rauru’s kingdom - consider Ocarina of Time Ganondorf to be ancient per their description of the Evil Spirit Armor. This is further reinforced by the Seven Heroines of the Gerudo and the Eighth Heroine predating the Zonai arriving on the surface, hence why the Temple of Lightning and other Zonai-Gerudo aesthetics utilize the Heroines (and the Eighth Heroine statue is even literally just a giant Zonai puzzle.) I’ve already explained the first part, so I assume the second part will be received the same way, but I can only hope other people are reading and interpret what I’m saying in good faith.
And no, this isn’t the only evidence, as established.