The density of wild ungulates has increased in the last century, and browsing has become a major driver of forest succession in the northern hemisphere. In addition, tree species are expected to respond differently to future climate conditions, especially an increased frequency of late frost events. The aim of this study was to analyze the influence of intraspecific genetic variation on the recovery of two tree species to frost and browsing. An experiment with saplings from 90 Abies alba and 72 Picea abies seed sources was conducted. Five‐year‐old saplings were clipped at three intensities before budburst in spring. Growth (height, diameter, leader shoot length, and biomass) and quality (e.g. stem form, multistemming, reaction type) were assessed before and 1–2 years after clipping or 3–4 years after natural frost events, and provenance differences were related to environmental differences at the seed source. For Abies, frost and clipping resulted in reduced height growth in the first year after the stress and reduced height for two (clipping) to four (frost) vegetation periods. Sapling biomass, diameter increment, and quality decreased after heavy clipping. For Picea, which grew twice as fast as Abies, such effects were only found after frost damage. Population differences were significant for both species for all investigated growth traits and for Picea also for some quality variables. The “reaction type” after browsing (e.g. new shoot, existing twig bending upward) seems to be species specific and independent of seed source. In contrast, the time lag between clipping and formation of a clear new leader shoot increased for Abies with lower temperatures at the seed source. Lowland populations with warmer climates grew faster, and for Picea also qualitatively better, and recovered faster from leader shoot loss (Abies) or reacted at the uppermost meristem (Picea). Thus, the investigated stressors increased the existing differences among populations.