The only thing that irks me about OoT is riding Epona in Hyrule Field. It's a bit too buggy. Sometimes you'd be riding at full speed, and then you'd come to a small hill, and Epona gets, like, scared of it for some reason. For instance: if you try to cross the bridge to Kakariko going too fast, then Epona stops, bucks, and walks backwards.
It totally kills the momentum of the game.
Exactly this, and a few other reasons.
The other thing I hated about Ocarina of Time was not being able to skip cutscenes, which was very, very annoying. Another than that annoyed me was falling off the bridge in Gerudo Valley as a child, it sent you all the way back to Lake Hylia, along with that, Young Link is slow, because he can't use Epona, or anything like a skateboard.
Another thing was the Owl, ... for a few reasons, one: Because I couldn't pronounce his name, ( -.- ) and two: He'd just go on and on with boring information, and if you pressed the wrong answer by accident, he'd say it all again, back in the day I use to accidently keep telling him to say what he said again, which drove me insane.
And lastly: All the secret keys in the Water Temple, though I was smart on my first try, by not going through any doors with a keyhole until I had explored the rest of the temple I had access to, this was very helpful. And ended up saving me from going all the way back at one point.
The other thing was the easy mini-boss. Sure, it may have been hard if you were fighting normally, but you could easily cheat your way out with some weapons/spells such as Din's Fire, Biggoron Sword, Broken Biggoron Sword and the Megaton Hammer.