I dislike him because he's a persistent companion character with no development whose only purpose is to feed you quest information, tell you where to go, and physically restrict you from exploring the map.
As anyone who sees me post has no doubt sussed out, I wasn't a fan of the direction the series took after ALTTP. When I was younger, I enjoyed games like Ocarina of Time and Twilight Princess. Heck, I played through TP multiple times. I would regularly tout Zelda as my favorite video game...