I think he was "chosen" by the gods.
Okay, he stole that triforce of power.
The golds could have stop him, but they didn't.
So they knew that he was going to steal the triforce.
Maybe they considered it was some kind of challenge for Ganondorf.
If he was able to go into the Sacred Realm, then...