Breath of the Wild Yiga Clan Hideout Not Working?

[DIO'sNotDead]

Apologies if I'm three years late to the BOTW party lol.

After getting the "Divine Beast Nah Boris" quest and talking to lots of guards around Gerudo Town, I headed out for the Yiga Clan Hideout, but I entered without a sand seal and I went through a hole near the entrance of the octagonal room. Problem is, I've heard this should've been very difficult without being stealthy, but I only encountered three Yiga Clan Members near the entrance and another one below. There's no one else around the area and I am yet to encounter a boss fight.
I restarted the game and entered again, but with a sand seal and through the entrance where youre supposed to go. Pretty much the same enemies spawned and I feel stuck.

Have I done something wrong? Is there anything that I missed before heading to the hideout?

 

[Shrimpdragon]

Did you talk to Gerudo chief before getting Naboris.
She will tell of Lightning helmet that has been stolen by Yiga, Captain of guards will tell you where hideout is.
I got into area from Tower near there and dropped thru hole, Shrine in there.
Kill several Yiga, get into entrance, use torches to burn separate entrances for treats and enemies, one will take you to find Barta-behind bars-listen to her advice about Bananas, sneak up stairs all way to top, collect masses of Bananas and stone, sneak thru main room, via walkways above Yiga heads, don’t get seen!
Finally exit room opposite diagonal corner, collect more treats, use magnesi, open wall block and face Boss.
Eventually Gerudo Chief will use helmet to shield you whilst shooting Naboris feet!
Not sure if you’ve done this backwards!