I never really had a dislike for any of the dungeons in The Legend of Zelda franchise. There are some in Zelda II: The Adventure of Link I find frustrating. There is the infamous Water Temple in Ocarina of Time. Ganon's Tower in A Link to the Past can be a pain in the butt if you do not know...
Being a retro player, I always liked the dungeons in The Legend of Zelda (NES) and the ones in Zelda II: The Adventure of Link.
Here is a small list of the dungeons I do not like in no particular order:
Forest Temple (OoT)
Water Temple (OoT)
Shadow Temple (OoT)
Turtle Rock (The Legend of...