There aren't that many things that made the Forest Temple that scary or eerie. It seems scary because it's the first actual temple that you come across in the game, and that the theme for enemies found within the temple is along the lines or haunted mansion with the Staflos', Poe Sisters...