There were a lot of things that I didn't like about Spirit Tracks, but I suppose I'll stick to what we have on the options.
I never really had a problem with the Pan Flute, it always worked very well for me. The only thing I ever had trouble using in the game that should have been easy was drawing the symbols on doors. It took me exactly 48 tries (I counted) to draw a symbol on a door because it apparently wasn't the right symbol. It was ridiculous. It expected you to get every little pixel right, apparently.
The Spirit Train was probably my biggest dislike in the game. The travel was slow and boring, and the enemies did nothing to help that. The only thing the enemies did was make sure that I couldn't set my DS down somewhere and go grab something to drink or whatever and wait for me to arrive at my destination. Instead they made me slave over the DS and wait for the train to arrive. The portals were helpful, I suppose, but I didn't find the locations incredibly convenient half the time and the travel was still incredibly boring.
The Tower of Spirits is another thing that I disliked, but definitely not as much as the Spirit Train. I just find central dungeons in general a horrible idea, but at least it wasn't quite as annoying or repetitive as The Temple of the Ocean King. I completely dislike the idea of having to go to one place over and over again after each dungeon. It starts to seem more like a boring task rather than trying to enjoy a game.