We'd still have the basic foundation of videogames (which Pong is more likely to be). But honestly, Zelda is like a Citizen Kane for videogames: It didn't invent the medium or anything like that, but it showed that games could be much more than just moving a random figure through the screen. Also, there's the key feature to any deep single-player expirience: saving.
Seriously, try to imagine playing ANY RPG/action-adventure/platformer/any genre knowing that the moment you turn off the machine, everything you did will be erased. It's such a simple feature, but it was groundbreaking at the time. That along with using weapons or tools you found in unexpected ways made it unique and far more ambitious than any of the abundant side scrollers or arcade style games at the time.
Other than that, there wouldn't be much of a difference.