Oh it was more complex, really it was started over... taxes. The North was expanding, the South wasn't. This made Southerners angry because they felt they were over taxed but not getting the expansion they wanted. So the broke off and formed the Confederacy. Key fact here, the North had slaves also at the time. Not as many as in the south, but they had them.