Forests are undergoing increasing risks of drought-induced tree mortality. Species replacement patterns following mortality may have a significant impact on the global carbon cycle. Among major hardwoods, deciduous oaks (Quercus spp.) are increasingly reported as replacing dying conifers across the Northern Hemisphere. Yet, our knowledge on the growth responses of these oaks to drought is incomplete, especially regarding post-drought legacy effects. The objectives of this study were to determine the occurrence, duration, and magnitude of legacy effects of extreme droughts and how that vary across species, sites, and drought characteristics. The legacy effects were quantified by the deviation of observed from expected radial growth indices in the period 1940–2016. We used stand-level chronologies from 458 sites and 21 oak species primarily from Europe, north-eastern America, and eastern Asia. We found that legacy effects of droughts could last from 1 to 5 years after the drought and were more prolonged in dry sites. Negative legacy effects (i.e., lower growth than expected) were more prevalent after repetitive droughts in dry sites. The effect of repetitive drought was stronger in Mediterranean oaks especially in Quercus faginea. Species-specific analyses revealed that Q. petraea and Q. macrocarpa from dry sites were more negatively affected by the droughts while growth of several oak species from mesic sites increased during post-drought years. Sites showing positive correlations to winter temperature showed little to no growth depression after drought, whereas sites with a positive correlation to previous summer water balance showed decreased growth. This may indicate that although winter warming favors tree growth during droughts, previous-year summer precipitation may predispose oak trees to current-year extreme droughts. Our results revealed a massive role of repetitive droughts in determining legacy effects and highlighted how growth sensitivity to climate, drought seasonality and species-specific traits drive the legacy effects in deciduous oak species.