When I think of the Master Sword, the first thing that comes to my head is The Sword in the Stone, a novel by British Author T. H. White. In the book, there is, as the title implies, a powerful sword stuck inside a stone. There is an engravement in the stone that reads "Whoso Pulleth Out This Sword of this Stone and Anvil, is Rightwise King Born of all England." Naturally, everybody in town tries to pull the sword out, but failure is all that results. A young boy, constantly looked down upon by his family, and especially his brother that overshadows him in many ways, is finally the one that manages to pull the sword out and, as a result, becomes King of England.
The Master Sword is a very similar concept in that it is a sword stuck in a stone pedestal that none but only the true hero can pull out and use. Link is the true hero, much like Arthur, who was the true king, and is therefore the only one that can wield the sword. However, just because he is the only one that can wield it does not mean he is the only one that can touch it; it means he is the only one that can use it to its full potential. King Arthur was the only one that could pull the sword out of the stone, but I'm sure anybody could have touched/held it after that, Arthur just would have been the only one that can use it. The Master Sword is the same.
Another obvious comparison that can be made is Thor's Hammer. Only the one worthy of holding it can hold the legendary hammer Mjolnir, and that person is Thor. Likewise, the only one worthy of using the Master Sword is Link.
So yes, it does seem a little strange that other people in the Zelda world can hold the Master Sword, but it's not a problem since we never see those people actually using the sword. Link, as the true hero, is able to use it as his own weapon since he is the only one worthy of using it. And this is the way I like it. I don't think it needs to change.