Yeah you're right, it makes absolutely no sense lol. But hey it's Zelda. Nintendo has screwed with the mechanics of time travel so much, I say just go with it
It's The Legend of Zelda, where the space time continuum doesn't appear to exist, Hyrule's economy is stable enough to let some pottery get smashed and fish people evolve into bird people despite the fact that they're now in a world more suitable for fish people.
