Skyward Sword's rendition of the Bokoblin easily tops my list. It dashes at you to assume a defensive position and it just makes things go entirely too slow! How could you get anymore annoying than this?! By the end of the game they're a LOT more manageable than beginning-to-mid game, but they're still annoying!
Another honorable mention and actually the thing I thought was the most annoying is Takkuri Bird. This thing is hard to kill (initially), it steals any of your useable items (including the Ocarina of Time, if memory serves), it's hideous and it just keeps coming back! Finally if you want to get your items back, you have to pay an overpriced amount for any one of them, or else play the Song of Time in a last ditch attempt! Talk about disatrous!