Few blacksmiths, yes, but lots of farmers, weavers, tanners, masons, and carpenters. In this game, we aren't Mongol or Native American horse tribes (which probably had the maximum percentage of fighters per population of any group, ever, but I'm too lazy to look it up and find out for sure). We're not especially nomadic and we do build defensive structures, so a few blacksmiths wouldn't cut it.