Navi only came after GDT ordered her to when he was 7
Wow. Seven years is the lowest guess for Young Link's age I've ever seen.
That was just the first time Link had been given a guardian fairy. Obviously, living in an enchanted forest where all his neighbors had their own fairy fluttering around their heads, Link would have been in the presence of plenty of fairies. How do we know Link hadn't been interacting with fairies his entire life? Throughout his infancy they could have been feeding him, diapering him, cleaning him and all the usual baby maintenance stuff. If they did so in solitude, perhaps inside the Deku Tree or in some remote corner of the forest where the Kokiri children wouldn't find him, then one day perhaps as a toddler they could have had Link show up in the Kokiri village. If the Kokiri didn't know better, the Deku Tree could have explained him as a new Kokiri. We don't really know how much the Kokiri knew Link wasn't one of them. They could have been under orders from the Deku Tree to keep it a secret. It didn't stop Mido from discriminating against him in any case.
But Navi was Link's personal guardian fairy - his sole dedicated fairy companion. Countless fairies other than Navi assist Link in his adventure. Do fairies not assist or even show themselves to non Kokiri or are they just more elusive? Would the fairies actively shun Link unless the Deku Tree told them otherwise?
If the answer to those questions is "no" then it is entirely possible that it was the fairies who nurtured Link as an infant.