I have two least favourite Miniblin and floormaster.
They always annoy me no matter how many times a play.
I always get taken by the floormaster and then the miniblin. There little poking sticks anger me so much Grrrrrr.
I think there's an annoying enemy from every game I've played:
Ocarina of Time: ReDeads/ Gibdo (I am not for one on face-noming), Deku Scrubs (in child from if I had no Deku Shield), Mini Peahats, Shellblades
Phantom Hourglass: Crows, Green and Blue Chu Chus,
Twilight Princess: Chillfos and Freezards. ReDeads were pretty annoying, but they lost their frightening aspect with the amazingness that is the ball and chain.
Skyward Sword: kulltullas especially ones that are on the ground. They attack like ReDeads too, which is not very welcome for me.
Ok non boss ememy? Like-likes and redeads are a close pair, but Wall masters trump them both... Like-likes you can defeat so easily, as creepy as they may be. and Redeads and Gibdos (at least in OOT) you can stun and kill super easy too.... Wall masters you just gotta know where they are caus if you dont: "Whats that sound? whats up with that shadow over... AHHHH!!!" :/ not fun.
I would have to say the wallmasters from Zelda 1, A Link to the Past, Ocarina of Time, and Minish Cap because they usually come the hell out of nowhere and drag you all the way back to the start of the dungeon. I can't count how many times I've had to restart the Skull Woods from A Link to the Past because of those annoying, oversized hands.
Ooh.. Probably Like Likes, they always stole my stuff! And i hate the seahats from Wind Waker, they terrified me. And i don't know if this counts, but the giant octopuses in Wind Waker, there were 8 all over the great sea, i had a mini heart attack everytime i came across one.