Tree rings provide an invaluable long-term record for understanding how climate and other drivers shape tree growth and forest productivity. However, conventional tree-ring analysis methods were not designed to simultaneously test effects of climate, tree size, and other drivers on individual growth. This has limited the potential to test ecologically relevant hypotheses on tree growth sensitivity to environmental drivers and their interactions with tree size. Here, we develop and apply a new method to simultaneously model nonlinear effects of primary climate drivers, reconstructed tree diameter at breast height (DBH), and calendar year in generalized least squares models that account for the temporal autocorrelation inherent to each individual tree's growth. We analyze data from 3811 trees representing 40 species at 10 globally distributed sites, showing that precipitation, temperature, DBH, and calendar year have additively, and often interactively, influenced annual growth over the past 120 years. Growth responses were predominantly positive to precipitation (usually over ≥3-month seasonal windows) and negative to temperature (usually maximum temperature, over ≤3-month seasonal windows), with concave-down responses in 63% of relationships. Climate sensitivity commonly varied with DBH (45% of cases tested), with larger trees usually more sensitive. Trends in ring width at small DBH were linked to the light environment under which trees established, but basal area or biomass increments consistently reached maxima at intermediate DBH. Accounting for climate and DBH, growth rate declined over time for 92% of species in secondary or disturbed stands, whereas growth trends were mixed in older forests. These trends were largely attributable to stand dynamics as cohorts and stands age, which remain challenging to disentangle from global change drivers. By providing a parsimonious approach for characterizing multiple interacting drivers of tree growth, our method reveals a more complete picture of the factors influencing growth than has previously been possible.