How can you judge by their appearance? they literally do not change in the 7 years between links visit as child and adult, plus it takes a regular house spider 8 hours to spin a web judging by their sizes i'd say it'd take significantly less, they could have been in that house for less than a month or have been there for a 100 years. Seeing as their greed led them to it, i'd be more inclined to call them Gerudo's who stole a sheikah artifact but Gerudo are for the most part female, could've been the menfolk they take to get pregnant but again i doubt that, it seems more probable they were a family who stole a artifact and were cursed for the theft.
Plus Sheikah are a dying species, we only ever see two in OOT, and judging from Impa's age i'd say it was likely 50 years or more, she was born and raised in that town and it was due to her efforts that the town was opened to non Sheikah. So it's 50/50 either they were Sheikah from before the town was opened or they came to the town stole something and were cursed for it.
Also Impa herself was the one to seal up Bongo Bongo in the Well Bottom, which bar from one or two people no one knows anything about, for all we know Bongo Bongo could've been the patron deity of the Sheikah and took exception to the fact a group of sheikah would steal from their own people. Next to Ganondorf and Twinrova, Bongo Bongo would be the next strongest, he managed to take out the legendary hero pretty easily in shadow form whereas Twinrova failed, and magically speaking it's a giant undead ghost monster, fairly sure it knows some curses.
The very fact there are 100 Skultulas that are involved in their curse might indicated something, Maybe the last skultula to be freed was the worst transgressor which is why it takes all 100 to free them where as the first requires only 10 and so on up until 50 Skultulas are destroyed.