Yes it's possible. You can even (barely) do it with 3 hearts. The only time you should come close to drowning is getting that key underneath the central tower. You get 8 seconds for every heart you have, and you can just barely make it through that room within the 24 second countdown if you take the iron boots on and off at the right times to mess with the clock (to delay it from counting down). I can pull it off with 3 hearts about half the time. With 4 hearts or more, it's close to 100% of the time. Technically you can just skip that room and its small key anyway because you can bypass a locked door in the dungeon due to its design.
You can also beat the Fire Temple without the Goron Tunic, but you need much more health or at least two fairies if you try with 3 hearts. Only certain rooms of the dungeon start the countdown clock for heat so there are plenty of safe zones.
I've tried a lot of glitch-free minimalist runs through the game so I've learned a lot of weird things that the game lets you do.