I deliberately went for him after getting 20 hearts and the master sword/ Hylian shield combo. I just swam up waterfalls, doing so can skip all the way to the sanctum undetected! I beat him on my second try, having all 4 divine beasts made it go faster.