Interesting theory. I'm not sure it was an academy for Knights but if not I am guessing there would have been a military presence there once for some reason or other. Ashei seems to have connections to snowpeak so perhaps it even belonged to her family.
Though...what if the humans once...