Premise of research. Vascular plants depend on sexual recombination for generating new genetic variability to meet environmental needs. Nevertheless, members of the early-divergent fern genus Botrychium (Ophioglossaceae) typically maintain gametophytic selfing and show strong inbreeding within populations. To explain this evolutionary anomaly, the existence of previous or current but undiscovered outcrossing, genetically rich precursors of the existing genetically depauperate taxa, has been hypothesized.

Methodology. Using allele expression at 13 independently assorting enzyme loci, we compared allelic diversity and levels of heterozygosity in 471 B. lunaria individuals across 16 populations in the Alps and Jura Mountains of Switzerland. We examined habitat characteristics influencing mating systems and investigated population genetic structure on the basis of a discriminant analysis of principal components (DPAC) and a graph-theoretic framework. We tested the pattern of isolation by distance and explored the biological processes favoring or limiting spore dispersal at a regional scale.

Pivotal results. We found high genetic diversity within and among populations and similar observed heterozygosity (H_O) and expected heterozygosity (H_S) across loci (H_O = 0.020–0.450 and H_S = 0.040–0.590, respectively). Estimates of H_O (0.050–0.272) were lower than those of H_S (0.144–0.305) across individuals, indicating a weak deficit in heterozygotes. Mean values of departure (D) from expected heterozygosity were close to what would be expected with random mating (DL=0.061 and DP=0.059 across loci and populations, respectively). The mean inbreeding coefficient (F_IS) was low (FIS=0.247), and the overall genetic differentiation (F_ST) was moderate (global FST=0.083).

Conclusions. Together, the DAPC and the population graph revealed a complex population genetic structure and weak genetic differentiation, with substantial gene flow among most populations. One population (Arosa), sampled from a site recently released from permafrost, demonstrated a probable founder effect not correlated with geographical isolation. The topology of the population graph identified three subgroups with several key populations (Bosco Gurin, Chandolin Low, Grand St. Bernard, San Bernardino, and Val Roseg) that maintained genetic connectivity of populations from distant locations.