Well, I'm reevaluating all the games I've played so far and a lot of them have been moved around, so I don't know quite where PH is on my list of Zelda games yet. Anyways, here are some reasons why I believe that people dislike PH so much.
#1. Difficulty
Simply because it's incredibly easy. It's the easiest game in the series, and it never gave you a challenge. Not to mention, the difficulty was skewed between the TotOK and the other dungeons, the TotOK basically being a difficulty leech and leeching away all the difficulty of the other dungeons.
#2. Story
The story is actually, in my opinion, very good in PH but really it doesn't seem that many other fans like it so much because "it's a LA clone" in terms of plot. The only similarity I see is the dream scenario that it was all just a dream. But it's debatable in PH but undeniable in LA and the only way the two games are similar is that they're both portable games as well as PH possibly being little more than a dream. But again, it's debatable whether or not PH was a dream.
#3. Casual Gamers
Most Zelda fans these days bag on PH simply because it was geared towards casual gamers. It's somewhat unfair to bag on a game because it appealed to casual gamers instead of core gamers if you ask me. Simply because it's geared towards casual gamers doesn't make it less of a game and, let's face it, PH is actually geared towards both. It's easy enough for casual gamers to just be happy with beating the game and doing it, but it's hard enough that core gamers will be searching for a long time for everything to get a 100% completion. After all, I had to spend many upon many playthroughs to find every single thing there is to get there.
#4. Temple of the Ocean King
The Temple of the Ocean King, while a cool little concept, was just overall not good for PH. It sucked away the difficulty from the other dungeons while still being incredibly easy in and of itself, not to mention you had to go through a ton every single time you had to visit it, even if you used the yellow warp.
#5. Gameplay
Another reason why people seem to dislike the game so much is because of the gameplay. Apparently, people disliked the new control scheme of the game and the lack of ability to use the classic controls. Apparently, people also disliked the sailing in PH too (as do I) in addition to other things that I'm probably forgetting.