It is mentioned that only a Dunyain Woman can give birth to a true Dunyain son, which kind of says that while the Dunyain have no breeding issues in Ishual, any Dunyain would have trouble breeding with any other woman. I think we are being led to believe that the Dunyain were isolated for so long that they are one the brink of speciation.
Any evolutionary biologists in the house? How many generations does it take to speciate two separated groups of the same animal? I know that some long running experiment managed to speciate some species of insect (fruit flies or something?)... But I think it took like 20+ years, and the reproductive cycle of flies is rather fast.
I wonder , given the limited data we have, if anyone has tried to come up with some kind of extremely rough correlation that gives you speciation time in years (or decades/centuries/etc.) given a given reproductive cycle/rate... Probably varies way too drastically between animals to do something like that.
Probably a stretch IRL, but throw in some magic and a little bit of Alien blood and I think its a plausible timeframe

Food for thought: one common trait of the offspring of inter-species couples is infertility, if, that is, the offspring can be brought to term and born alive.
Maitha has no kids (right), nor do any of Kellhus' kids.
I'm not 100% sure what that might mean for Esmi or Mitha's mother, but these facts seem to point to the Dunyain being a different species, which then (if you take a large leap) leads me to believe that the Dunyain had this in mind all the time. Nerdanel: Dunyain always planned on eventually copulating with the Nonmen to create viable hybrid children (to what end, I don't know.).