They could be inbred but exhibit little to no effects of typical inbred populations if they took extraordinary precautions as only the Dunyain could have.
But how does that come into play here? Are you suggesting that because the Dunyain are so inbred, there would likely not be may differences, genetically, between Kellhus and Moenghus?
What was the stated goal for Moenghus to go out into the world? If the Dunyain were getting so inbred that they were having significant trouble procreating, its likely that more than just 1 or 2 would have been sent to the world so that they could conquer it (or at least part of it) and send fresh genetic stock regularly to the Dunyain. We've seen the unfortunate effects of the half-dunyain, and breeding with the regular humans would set them back hundreds of years, but if they kept isolated for much longer they wouldn't be able to have children at all.
This is why I don't believe that to be the case. It doesn't seem like that was what they are/were trying to do with Moenghus/Kellhus. I think they have a profound understanding of genetics and have been extremely careful about it since their inception, and have fine tuned their plan with their super-dunyain brains every generation or two.
It could be that the Anasurimbor line needed to be culled. For whatever reason too pure (in this case meaning lots of inbreding i guess), perhaps a failed line a centuries long experiment with that bloodline. As Alia mentioned in a different thread, when breeding cats and such, if you suspect a negetive genetic trait to run in the family you often breed a mother/son or father/daughter pair, and if the children show the defect, you simply remove them from the breeding....
The Dunyain recognized that the Anasurimbor line has some issue, so they bred Moenghus with his mother a few times. Maybe Kellhus was find but his brothers/sisters had the defect, so they killed them off. Instead of killing Moe outright, they sent him out to scout the world. He came back, realized he was going to be killed, and left. No idea why they didn't just kill Kel though
Edit:
For inbred dunyain populations, see:
http://www.second-apocalypse.com/index.php?topic=993.msg8263#msg8263