I'd say I know a lot of casual fans. A few of my irl friends played the hell out of Breath of the Wild, but I can't recall if they're fans of the entire series. My girlfriend has enjoyed a lot of the games on account of me, and I remember her saying she had played WW before meeting me.
I'm sure more do, but I'm always too shy to ask. Zelda isn't a social multiplayer franchise, despite having obscure multiplayer variants; you can't just simply ask someone to join you for a round of Zelda.