There was to be a Goron island where everyone smoked tobacco called stovepipe isle as we can see in Hyrule historia yet WW was rushed and content was cut.
I don't think there is any real reason there weren't many Gorons in WW apart from the fact their territory just wasn't included in time.