I'd have to agree with those who say any dungeon from Zelda II: The Adventure of Link. As I am a person who does not use walkthroughs/cheat codes on my first play through of any game, I began pulling my hair out at Death Mountain! Not only was the difficulty level of this game high, but if you were redirected to the Game Over screen, you were forced to start from the exact place from which your adventure began. This I found, got to be quite annoying! While I do agree that AoL does not recieve as much love compared to all other Legend if Zelda games, its not hard to see why many fans dislike it.
I also found some of the MQ temples to be quite difficult.