A visit to gamerankings places it at 86%--by no means a bad rating, but certainly a good deal lower than its contemporaries. Comparatively, the N64's Banjo-Tooie was placed at 91%.
I can tell you as someone who was with the N64 from the beginning that there was a sense of collective disappointment at the game. This is anecdotal, but many of the people I knew who loved the other N64 platformers couldn't stand DK64. It was simply too big, too disorganized, and too packed with busywork. Rare's other games--even their other deeply flawed N64 game, Jet Force Gemini (which admittedly got a 79% on gamerankings)--showed a much more solid understanding of basic game design philosophy. Don't encumber the player with pointless tasks; let them get to the meat of real gameplay. Let the platforming or minigames themselves be the challenge, not questions about which character to use. Let the only thing standing between you and those collectibles be environmental obstacles or (occasionally) unlearned moves.
Rare was a lot like Nintendo. While games were trending toward the cinematic, Rare, at least early in the N64 era, either incorporated the cinematic elements (see Banjo-Kazooie, Goldeneye) or eschewed them almost entirely in favor of raw gameplay (see Blast Corps, Diddy Kong Racing). And the gameplay, at its best, was engaging. You had to think of new ways to use Banjo-Kazooie's moves. You had to pick the right vehicle in Blast Corps. You had to traverse levels carefully and explore in Goldeneye. You could generally do all this without backtracking or slowly plodding toward goals; it came down to your skill as a gamer and your understanding of the game's core mechanics. And that's just scratching the surface of those games.
The cheesy cutscenes in DK64 were just painful to watch. Most of us who'd grown up with the system were too old for things like that when the N64 was released (I was 10 in 1996), and were far less willing to tolerate it so late in the system's life cycle! If DK64 had been backed by solid, consistently engaging gameplay, I would have been content to ignore it, but it seemed like this ill-advised atmosphere was really the whole point. There was little seamless about its design, little intelligence to its level layouts, little fun about its sprawling tasks.
I can guarantee you that as someone who watched the N64's whole life cycle very closely, DK64 was considered a low point, and a sad end to Rare's consistent upward trajectory.