Most annoying is probably the sailing in Phantom Hourglass. I rather enjoyed sailing in the Wind Waker, if felt free and open - even if it could be boring at times.
However, sailing in Phantom Hourglass felt like the antithesis of the Wind Waker, it was constricted, limited, the process itself...