Well if you consider any ability at all to preserve sperm/egg that could be kept after the owners death, even for just an extra 1 or 2 generations, there would be little chance of any real inbreeding. Stretched to the very limits, I came up with 5 generations separating potential partners.. If that could be extended to 6 or 7 you would be fine with an extremely small number of original families.
I was more focused on the number of bloodlines/families, rather than total number of refugees, though it is almost the same question. There were likely not many children or elderly in the group, but there could still have been 1-5 members of a family. 5*32=160.
I just don't know how popular the Dunyain cult was, and how sranc infested the north was when they chose to flee. Those original Dunyain were only marginally more gifted than normal people, if any at all, so a fair number of them should have died on their trek through the wilds (Anyone ever played the old Oregon Trail game?). They may or may not have had a destination, and even if they did think they knew where they were going (ishterebinth? some other abandoned city?), they probably took a roundabout way to get there.
I want to say that I feel like 200 is too much, but then again a tiny group is more susceptible to failure due to accident/disease, while a large group can survive some misfortune. If most/all the weapon races were tethered to the No-God farther south, a large group could go through the North unchallenged. Its also likely that the Dunyain cult was comprised of whole families, like your standard religious family, so a medium-large size group isn't unreasonable.
I'm torn

I'm just glad I thought through the thing and was able to figure out how a small group could keep themselves producing viable offspring for at least a 100 years with only a little bit of creativity. Once you get past 100 years, it gets more complicated, but the Dunyain certainly had the foresight to breed themselves in such a way to preserve themselves for 200-400 years. By that time they should have started becoming smarter/faster/stronger.
I'd say 10-20 generations is enough to see some results, so they may have been able to plan much farther into the future. They are in the mountains after all, and freezing some sperm isnt so high-tech. If they ever discovered how to do that, it would be smooth sailing from then on, they could just always breed with older/uncrossed family lines, and never have to breed together children that had the more complex bloodline mixtures.