Skull Woods (Alttp): I would turn back for the Village of Outcasts and not think twice about it. That place is a nightmare.
Ice Caverns (OoT): I don't want to be froze to death. Burn me if you must, but death by ice is not a good way to go. I could handle some of the Ice Dungeons in Zelda, but...