I think that the Goron's were down to 3 because the islands that they were on eventually ran out of resources (rocks) and they died out as they couldn't move to other islands.
Revelation: Godessess turned Zora into Ruto.
Why, you ask? Because they wanted to keep Hyrule hidden forever. They sent all the species to the mountains (excluding Gerudo) but the Zora were probably partying like "Woohoo! Hyrules gonna be an ocean! F*** Yeah!" and so the Godesses sent lord...