I hate to make more comparisons to Phantom Hourglass' Temple of the Ocean King, but frankly they're unavoidable; the Tower of Spirits was supposed to be like the Temple of the Ocean King.
It really just wasn't as good as the Temple of the Ocean King for me. I mean, I'm even tempted to say it wasn't as good as a fact, but I know there are those who are disagree. I just really think that from a design standpoint, they did not put that much work into it. To me the Tower of Spirits feels a lot like a rapid barrage of puzzles, not like an artfully designed, smooth area. When I play the Temple of the Ocean King, yeah, sure it can get a little annoying redoing early parts, but that's diminished a lot by you learning the shortcuts. It was designed better and had more depth. It also had the stealth aspect, and heavily so, making it unique from all the rest of the game (with the exception of the Ghost Ship). The Tower of Spirits had a little of the stealth, but it was cut down to an absolute minimum and it's really mostly just puzzles.
On its own I liked the Tower of Spirits fine enough... The co-op gameplay was awesome, and the story moments there were great. It was a good part of the game overall. But the way it was designed got annoying at times, and a lot of the floors of the Tower just... I dunno, dragged on. It got kinda boring. I wish they had made it more like the Temple of the Ocean King, but perhaps with more portals to skip sections.