I'll rate the temples from the point of view of playing the game for the first time, be prepared for a quite unusual list:
1. Forest Temple:
This is where the difficulty jumps up considerably compared to the previous dungeons.
Physical difficulty: You start off in the place with relatively low HP (~7 hearts) because it's still quite early in the game and yet you have to battle multiple Stalfosses which can drain your HP very quickly (up to 2 hearts when they jump-attack you and 1 heart for the regular attack). What's annoying is that they can resurrect themselves if you don't kill the second one quickly enough, a fact which you probably don't know if it's your first time playing the game.
Not knowing the "Triforce corner camping trick" in the Phantom Ganon battle can also prove to be quite deadly because he can inflict 3 dmg when charging you. So the physical difficulty is quite high in the dungeon.
Puzzles: On top of that there's also a considerable amount of puzzle difficulty in the dungeon: For example there are those twisted hallways which can easily confuse you because they have to be twisted/untwisted several times.
And the nastiest thing is probably that key in the very first room which is IMO much easier to miss than any of the Water Temple keys, for two reasons: First, you aren't used to getting keys in that part of the game yet and second you do not have the compass by then so you don't see anything on your map. I'm wondering how many % of all gamers missed that key on their first playthrough!
2. Spirit Temple:
The physical difficulty here is above-average too, especially the battles with the ironknuckles can be tough if you don't know the back-and-forth jump slashing trick and they take lots of hits before going down when battled with the Kokiri sword only.
The final battle isn't that hard though if you know what to do, which isn't much of an issue to figure out.
For the puzzle difficulty, the nonlinear design of the place will make you have to use your brain a bit more than in some other dungeons. Especially that switch puzzle with the statues which have to stop exactly on the switch to open the door always takes me forever! Needless to say you can lose quite a few hearts during that too.
3. Shadow Temple
This is an average dungeon for me in terms of difficulty. There are not that many puzzles and most of them center around the Lens of Truth and are therefore very easy to figure out. On top of that, the place is quite linear with little to no backtracking involved.
The physical difficulty is kept within limits too, the traps aren't hard to dodge and the enemies which are scarce anyway aren't too bad.
Only the boss battle is among the more difficult ones, you can lose quite a few hearts and it's the only boss which gives me serious trouble on the 3DS boss gauntlet because I often fail to shoot quickly enough against the 2nd hand (I have trouble locating it quickly enough).
4. Water Temple:
Contrary to popular opinion, I always found this place to be one of the easier dungeons in the game.
The physical difficulty is very low, there aren't really that many regular enemies in the dungeon and none of them are anywhere near threatening, just a few Tektites, seashell thingies (I'm too lazy to look up their name right now) and spike-rock thingies, no problem here.
Only Dark Link can be slightly challenging, but if you use the Biggoron's Sword or alternatively the Hammer or Din's Fire, he'll be down in no time. Same goes for Morpha, even if you don't use the corner trick you can dodge its tentacles relatively easily.
The puzzles aren't that hard either, finding the keys isn't much of a problem if you simply use the compass and map, I sometimes suspect that many players don't make much use of them but this dungeon really implies that they are there for a reason! I can understand the frustration though if you're standing keyless in front of that door which holds the chest with the Master Key which means a lot of backtracking! Luckily I did it in the right order to start with when I first played the game so thankfully I was saved from that nuisance.
The only other thing which makes this temple a bit annoying is the issue with the iron boots which was fixed in the 3DS version. It can be countered a bit though if you get the golden scale before going through the temple.
5. Fire Temple:
This is the easiest one for me, mainly because of its linearity, basically you just scale it up and finally drop down to land right in front of the boss door. Sure, you can theoretically miss a key or too but even if you do then there's not too much backtracking involved because keys have to be used relatively closely to where you find them.
The enemies too are rather weak here, nothing a simple sword strike or too can't handle.
Volvagia could theoretically pose a problem if you're not fast enough with the hammer and he slaps you but his other two regular attacks (fire breath and rock drop) can be easily dodged by climbing down the edge of the platform a bit and camping there.