Well the difficulty of a game isn't the most important thing to me, but I have to admit that in some cases I've been frustrated either with the difficulty being too high or not high enough. Personally I felt the difficulty set in the original Fable game was fine, it wasn't too hard, nor was it too easy. And if anyone did feel it was too easy they could challenge themselves by "boasting" before a quest, like doing the quest naked, vow to not take a single potion or eat a single food item during the quest, there was even the option to vow doing a quest without using weapons or magic at all - so the game offered you the chance to set your own difficulty, making it hard enough to make it fun, but at the same time you could avoid making it so hard it became unplayable.
However, the boasting option vanished in the following Fable games, they also removed the ressurection potion from the inventory and with it the possibility of being killed in the game, you simply fainted on the spot and then you could get right back up and continue fighting. The game suddenly became painstakingly easy, there was honestly no challenge left. The only consequence of being "beaten" was that your character would get a ton of scars that just wouldn't go away.
With the Buffy games, the creators took a different route in terms of difficulty. The original game had a decent difficulty and if you felt it got too easy you could choose to play the game at a higher difficulty level - again we got the choice to up or lower the challenges so the game remained playable. Some levels were easy, some were much more challenging, but all in all the game was enjoyable and beatable. With the sequel they made some changes that got to be frustrating. Firstly there were no checkpoints for an area, you had to play through a rather long level before you could even fight the boss, and should you fail to beat the boss in that same sitting, the next time you returned to the game you had to replay the whole area all over again. I was stuck at Sunnydale high playing as Xander for ages because of this (he was a weak fighter, had no powers, equipment or skills that helped me at all and the boss was frickin' Anyanka...) and it was just maddening.
So difficulty isn't the most important thing, but if it is way too high or way too low it does affect the gameplay. I don't think anyone really wants a game that offers no challenge whatsoever, and at the same time I don't think anyone wants a game they have no chance in hell to beat...