In may of 2004, Eiji Aonuma stated that at the time, the Four Swords title for the Gameboy Advanced was the "oldest tale in the Zelda timeline". He stated that Four Swords Adventures was a sequel taking placing "sometime after that".
If you believe The Minish Cap to be a prequel to Four Swords...