Yes, I really enjoyed the Silent Realms. ^^ They certainly got my heart pounding, and they added a bit of strategy and stealth into the game (and, for whatever reason, I've always been rather fond of stealth missions, like the Eldin Volcano one). The only frustrating one was Farore's Silent Realm, since, you know, it was the first one and I had no idea what I was doing. I always thought you had to actually sprint to get away from the Guardians, without realizing that just plain running would suffice, so I kept running out of stamina.
Most difficult... probably Skyloft. I actually didn't fail any of them except for the first one, so I'm only basing off of my re-runs in the Thunder Dragon's Lightning Round (which I did out of complete boredom and the small hope of obtaining a Heart Piece). I'm not very good at beating the Skyloft one on a limit; I died four times before finally finding the last tear. Then I again, I didn't retry Eldin or Lanayru, so who knows...