Sampling should be completely legal. I think most IP laws are too restrictive, so that's a given.
Furthermore, I've heard sounds and movie clips sampled to great effect.
However, I've yet to hear a song I like that samples heavily from another song. I think one of the most egregious examples of a bad song that samples from a good song is "Power" by Kanye West. He sampled from the important and historic King Crimson song, "21st Century Schizoid Man," without so much as asking permission from Robert Fripp until it had been heard by over a million people.
That's just disgusting conduct. It's part of what drove Fripp, the brains behind Crimson, out of the music business. I also think it's rather sad that a good number of people will be introduced to the Crimson song with West's tasteless, profanity-laced lyrics that have nothing to do with the subject matter of the original song and sap every bit of atmosphere from it. I was honestly appalled the first time I heard West's version. I've heard he does the same thing with other songs as well, so I have no desire to hear anything else by him. As I understand it, he was fully within his legal rights here, so it's not a legal issue.
This is all, of course, personal opinion. Art speaks for itself, and artists should be free to use all tools at their disposal. What I mentioned was but one example that left a sour taste in my mouth, and I'm not very informed about the practice, so I make no moral claims. I just hope other artists show more tact and bother to ask permission from the original composer, if possible, before sampling, or at least preserve something of the original message, atmosphere, and content.