The time-traveling system is only annoying/inconvenient for the first three days of Majora's Mask.
After you retrieve the Ocarina of Time from the Skull Kid, however, it becomes an invaluable asset to the game. Going back in time resets everything that Link did, but allows him to keep some of his gear. Maybe that felt like an irksome waste to many people; it certainly did for me, so much that I was almost dissuaded enough to not continue. That is, until I noticed it had far more uses than just following the story:
It's the Dawn of the First Day, and 72 hours remain before the Moon will make contact with Clock Town's tower...again. You must've gone through this routine at least on a dozen counts already; you didn't have enough time to finish a Dungeon because you needed to get some transformation mask or restore some Zora's voice, but didn't know how. But that's all right. You deserve a little break. The Crazy Mask Salesman can wait a bit longer before the Mask is placed back in his graspy palms. So you set out with a different objective in mind.
First off you gather supplies from the field outside Clock Town, a tremendous spot to find all sorts of goodies. After refilling your stock with arrows, bombs, and Rupees, you wonder where you can go next. There's that Ranch with the strange little girl rambling on about ghost aliens appearing at night, but her place is blocked by a boulder right now. Maybe you could Warp to Woodfall and make your way to the Deku Palace? Nah, they'd be holding yet another ceremony to sacrifice the monkey. Plus if you showed up as a Hylian then they'd kick you out for not receiving an invitation. Then there's the Goron Village; you still haven't uncovered the Rock Salmon, which you need to get that Frog Mask. But the entire bawling Goron-baby debacle had started over again.
With a sigh of frustration you realize half the day has gone by, as well as a good chunk of you break. So you decide a refreshing swim in Great Bay will clear your head and help you pick out something fun to do before returning to the quest. Time, after all, is on your side.
That's just a little illustration to help show the worth of the time-travel attribute. There's a whole lot more to do than trying to get to the next Dungeon to defeat the next Boss and free the next Giant. You have an entire set of masks to collect that aid your journey, and a plethora of sidequests that you could work on instead, such as fulfilling the wishes of every person in the Bombers' Notebook.
Even so, all this could be achieved without time-travelling back and forth. But the 3-Day Cycle drives you to make a plan and set a goal of what can be done within that timespan. It might put too much pressure on some people, yet it's great for pushing one to succeed in their purpose before the clock run out. Besides, in a game like Majora's Mask, where everything is stationed around Termina's world coming to an end, this is necessary. The cycle contributed greatly to the atmosphere of the upcoming apocalypse, and going back in time helped keep focus on that central threat.
The Mask of Majora is a puppetmaster that used its victim to spread havoc across the land, and keep an eye on his mischief. Meanwhile the Moon continues to drop, hoping to bring an end to the hectic ants below its odious ever-leering state.
Time travel was the only advantage against this menace, for their was not enough time to finish the task before its ultimate descent.