I am willing to accept the validity of the story of FSA to be canon. How ever I think the gameplay/graphics are still somewhat of a spinoff of LttP mixed with WW.
But I still can not agree with you that FS was/is canon.
FS sets up FSA. If FSA is canon, then FS has to be. FS is also a continuation of MC, which is also canon. All the games belong in the Four Swords Saga.
Vaati comes to power, Vaati is sealed away - MC
*We also see how the Four Sword came to be in MC, and at the end, learn that it was placed in a shrine.
Vaati escapes, he is sealed away again - FS
*Here, we actually see the Four Sword in the shrine it was placed in after MC
Vaati escapes a second time, is seemingly destroyed - FSA
All these titles work together as a continuation. Without FS, there would be no FSA, which is definitely canon. They would also never mention a non-canon game's story or plot in the backstory of a canon title. If you believe FSA is canon, then you can't deny that FS is canon as well, as explained in FSA's BS.