At first I thought it was a glitch but then I looked it up and I'm pretty sure it happens to everyone. Was it their intention to make your rupees constantly deplete? Because when you first obtain the armour, it says it takes rupees instead of health when you get hit. But this was not the case.
Personally, I didn't bother haha. I got enough Skulltulas to earn the wallet but stopped there. And I only got a few Big Poe Souls. I wasn't very good at chasing them and then shooting them tbh.