Anyway, in most of the games the main religion is the one that follows the three goddesses and the triforce and/or Hylia. So, it's kind of indicated that the only temple used for the religious purpose is probably OoT's Spirit Temple, which depicts the Goddess of the Sand, suggesting an entirely different religion for the Gerudo race, possibly explaining some of the cultural differences.
This isn't entirely true. According to SS, the region of the Gerudo Dessert was the location of the original Temple of Time and thus under direct jurisdiction of the Goddess Hylia herself. The Colossus is most likely the last structural evidence of her presence by the time of OoT and TP era. We also know that a part of this region was under the Sages control with the construction of Arbiter Grounds, indicating their local affiliation concerning the Hylian religion. It is possible the Gerudo warped Hylia's identity to their benefit (Koume and Kotake for instance) by turning it into their base of operations, and thus following Ganondorf on his quest for the Triforce.
We can deduct that the Sages presence in regards to these temples might of served as outpost to keep the Sacred Realm safe from invaders. It just so happens Ganondorf learned more about accessing the SR from the Spirit Temple itself. I don't have the images, but upon playing through OoT 3DS, there is a room in the Gerudo Fortress that has maps outlining points of interests in Hyrule that concern accessing the Triforce.
I need to add something. Has no one thought that these temples are in fact locations of spiritual importance because in OOT, Saria entered the Forest Temple to aid the Forest Spirits and awaken as a Sage of the Forest. Each Temple to me seems to be in the most empiric of spiritual locations for each race of people. It's only natural that a Sage of the Sheikah could only awaken in a place of secrecy and shadows Hence the Shadow Temple. The Gerudo Sages awaken in the Spirit Temple because in the desert what else is there too do but meditate upon your spirit. Each Temple is there for it's respective sage to overcome the trials within and Awaken/ascend to Sage.
This is a very good point, and I'll add to the importance of each Temple reflected by Arbiter Grounds. I believe they were once places of worship but also sectors made to keep an eye on those whom might want to claim the Triforce. It should be noted that a Gossip Stone is located almost near every temple in Hyrule. I'm sure a great deal of criminals spoken of in TP dealt with more or less reasons Ganondorf was executed.
Like Repentence said, I think they were built as places for Link to prove himself, or atleast they were built in the mind of “The hero” having to go through it to grow mentally and physically. I'm not sure if the Gods actually built them, maybe they sent divine visions urging people to build them, but that's all.
For OoT this is true, and is especially evident within the Spirit Temple. I would also say that the presence of Time-Blocks throughout the temples symbolized this notion as well.
