He did not fight Ganondorf in Majora's Mask because it is a sequel to OoT and he already killed Ganondorf and he was riding his horse when he left kokiri forest ( i think xp)