The Gerudo had basically shut themselves off from the outside world, only venturing out to steal or find a husband... even when Ganondorf took over Hyrule the vast majority of them kept themselves shut away in the Gerudo's Fortress... their behaviour is defensive, not offensive... so it seems...
I doubt he thought they would go to war... there's no mention of the Gerudo having been involved in the civil war that took place prior to OoT, and there's no evidence of ill will between the two races that implies they had ever been at war before... the Gerudo were just reclusive thieves...
Prophecies are typically depicted as being rather vague, requiring some degree of interpretation to determine their true meaning.
The same appears to be true of Zelda's prophetic dreams... in them she only saw dark clouds that represented the evil threatening Hyrule... she interpreted that as...