Well if you think about this logically and look at the state that hyrule is in even before zant invades its in a pretty bad shape. There are ruins every where towns have been abandoned and quality of life is on the decline as well as morale. There are a couple of reasons why this could be
1) natural disasters
It is actualy canon that there was an earthquake a couple years back. this took out water supplies and trade routes causing starvation and collapse of economy. The only race that would benefit slightly from this are the gorons who are powerful enough to move debris and could be hired to do so. Also the original zora's river has seemed to dried up becoming the new kakariko village and the zora's moved to the gerudo valley which seems to have come under climate change of its own.
2) Disease
is something that comes with a very poor population this could explain why medical rates are so expensive.
4) War
there are quite afew hints of war in the last few decades, the ruins and barracks around hyrule, the snow peak ruins and arbitor's grounds are just afew. In between mm and tp hyrule has launched campaigns on at least; the gerudo desert, snow peak , and possibly oradon. Wars have a drain on the population, morale, and economy of a country and at least one of those wars seems to have been dragged out for years, the snow peak ruins tells a dark tale of the genocide of the yeti race with only a few left. It is also canon that there is a renegade group who before the capture of hyrule were already a group who were planning to usurp the crown.
These are some of the things I think could have happened between games.