Well, I kind of see what you mean, but since there is one individual, he grows up to be adult Link, then as soon as that quest is complete, he goes further back in time and changes the future, so he doesn't grow up in that time line. So he doesn't vanish into thin air, but he doesn't exist in that timeline anymore. That's what I think is supposed to be the case in the game.
The thing with time travel is that it's been done in many ways in many different stories with many different rules, so you have to figure out what rules apply to each story. It only makes sense if you let the writers tell you what the rules are, then you squint at the product from a distance and turn your head to the left.