Also, OoT first came out over a decade ago, when many people here were young kids. At that age, it would certainly seem scary. Of course, this is all just an educated guess of mine.
This. I know I can't speak for everyone, but I know the main reason I found it scary- I was 6 at the time. And that impression of it being scary stayed with me until I played through it again fairly recently, and was like "Actually, that wasn't as bad as I remember".
I think what scared me more than the blood on the walls and the music and stuff was the fact I didn't feel like I was in control of the situation (same reason I was scared of redeads, I didn't feel like I could escape from them, wasn't confident I could defeat them)... being fairly young, I wasn't certain I could get out/succeed in a situation if one arose (e.g. accidentally falling down through one of the fake floors into a room full of Redeads or something, I wasn't confident enough in my fine motor skills or fighting strategy that I could get Link out of there alive. And yeah, I know, Game Overs aren't that big of a deal... but at the time, they were) If something like that happened now I'd be like "Ahh crud" *breaks out Din's Fire* *calmly finds way back to room I was in*. But back when I was 6, I wouldn't have been able to keep a cool enough head and would have just panicked and got scared, and then run into another situation, panicked and got more scared until I threw the controller at my dad [noparse]>.>[/noparse]
Although the music was pretty creepy. And I'm still mildly paranoid about the fake floors