It is impossible to travel between timelines unless you time traveling, and altering the past, created the alternate timeline. So Link ended up in a different timeline once (The Hero of Time leaving the Adult Timeline and creating the Child Timeline). He just can't return to the Adult Timeline...