Ugh...this has gone on too long. why even argue if the PotFS is canon. in no way does it connect Alttp to the FS saga. sure, there are four swords in it. big deal. it doesn't mean they have to be fragments of the four sword.
Although they literally come together at the end to reform the Four Sword. That pretty much signifies that it was in four separate sections.
I myself do not agree that the Palace is a huge deal in connecting the Four Sword Saga to ALttP. It definitely seems more like an added quest that gives players some extra replay value, and a reward of sorts after beating both games. Had they given a little more story to the Palace, then it would have been obivous as a connection. But there are no characters in the game that even talk about the Palace aside from the creature at the beginning of it, who says something along the lines of "Only true heroes are allowed in". This doesn't give the Palace any more importance towards the timeline, nor does it connect ALttP to FSA in any way.
Now what can be made from the Palace is some really good guessing. You could technically say that the Four Sword was broken (not necessarily physically, but separated into four separate swords), which would explain how Ganon escaped between FSA and ALttP. This is the point that Zemen is trying to make, and its a point I have brought up as well in the past, but its not the best point to use.
The fact is, the Palace of the Four Sword doesn't even have to be there for a person to connect FSA to ALttP. The GBA Version of ALttP technically doesn't even have to be there to connect the two. FSA makes plenty of references to Ganondorf (man) going to some Pyramid (such as the one in ALttP), to search for the Trident (which he has in ALttP). At the end of FSA, we see Ganondorf has apparently gotten the Trident of Power and he has transformed into Ganon (resembling his form from ALttP). At the end of FSA he is sealed within the sword; We find out during ALttP that he is no longer in the sword.