By far Odolwa form MM. I kept dealing large amounts of damage but he didn't die, then later he killed me and due to the great save system in MM, I had to start the Woodfall Temple all over again.
There are a lot of boss battles that are very annoying for me. Just about anything that is tricky or lasts a long time is what I classify as annoying. Something that you have to find out a complex strategy or something like that.