That Gerudo lady in the cage told me they like bananas so I realised that's what they were good for.
However I went in all Chuck Norris on their asses and killed them all. Ice arrow to their face, followed by a whirlwind charge attack. Enough hits on them to kill them before the sword one...