There are actually plenty of moments I don't look forward to. I haven't played all of the Zelda games--far form it--but here's my take on it:
Ocarina of Time: Dodongo's Cavern. Call me crazy, but I just... I can't. I don't. What. The boss is easy and done pretty soon, but I have a tendency to take FOREVER in Dodongo's Cavern. I just... I can't. I don't even with this. I like to get very, very lost. And, y'know. WATER TEMPLE. Also, most of the intro I hate.
Majora's Mask: The Gerudo Fortress. It's just, what? I don't ever bother to snag the Stone Mask because I'm lazy, so really, it's kidna tricky. Sneaking around in 3D is NOT my strong suit. I can do it on the handhelds just fine, but... yeah.
And the Great Bay Temple. I get lost. I get freaking lost and do everything wrong. And yeah. Also, Gyorg I mean wtf, I do not have the grace an accuracy of a gazelle, go away you giant fish. I just get lost a lot in the Great Bay. A lot. It's all I ever really do.
Twilight Princess: The first fight with King Bublin, where he captures Colin. I can't do that battle. I die a lot. I always dread having to go to that fight--it was the first thing that came to my mind when I saw this thread. I hate it. I HATE IT. Really, both of the initial fights with King Bublin I despise, but I love his character shift at the end, and the Arbiter's Grounds and Hyrule Castle fights aren't too bad.
Oh, speaking of Hyrule Castle, the Darknut fights in that place. I just.... WHAT. NO. NOT FAIR, GAME. I can't say I'm too fond of Hyrule Castle in general.
Phantom Hourglass: Everything, really. It's all just too easy--the only parts up to my standards for Zelda are the final boss battles, the last temple, and the last two runs of the Temple of the Ocean King. I found those to be legitly fun. The rest of it... bleh. The dialogue with the spirits is crappy, the cannon is annoying, the temples are ridiculously easy for me.
Spirit Tracks: Whoa, I actually don't like some parts of this game? Yes. Shush. There are segments I despise. The Tower of Spirits level just after the Ocean Temple, for one. Any fight with those giant squid things underwater. ANYTHING involving Linebeck III, that miniquest was unbearable for me! And the buildup to the Sand Temple. It was all just, no. I died. A lot. Still do. I have a lot of fun in ST--it surpasses PH by far for me--but I can't stand these segments.