Just southwest of North Castle is a cave. Enter the cave to find the first Magic Jar.
After obtaining the hammer, break the boulder next to where you get out and walk over the remains to fall. Continue in this cave to find the second magic jar.
On your way to Maze Palace, take the outer road to eventually find a hidden pit where the third magic jar is at. Be careful since there are hidden enemies on the way as well.
The last magic jar may be kinda tricky. You must have obtained the first three to even get the last jar. If you have the three jars, make your way to the Hidden Town of Kasuto (refer to the image). This town resides in a small forest after venturing through a cave. You must chop down the certain brush to get to it, (again refer to the image). Once you found the town, talk to an old lady in a blue dress. You then follow her into the house and in the basement is where the last magic jar is.