However, that would mean that he is erasing what he has done in the future by traveling back. Only if he went straight back to his original time would the events continue as he left them in the future.
But I can agree with you on the split timeline being a bit "iffy"...
Its easier to...