Nucleation and subsequent growth of new aerosol particles in the atmosphere is a major source of cloud condensation nuclei and persistent large uncertainty in climate models. Newly formed particles need to grow rapidly to avoid scavenging by pre-existing aerosols and become relevant for the climate and air quality. In the continental atmosphere, condensation of oxygenated organic molecules is often the dominant mechanism for rapid growth. However, the huge variety of different organics present in the continental boundary layer makes it challenging to predict nanoparticle growth rates from gas-phase measurements. Moreover, recent studies have shown that growth rates of nanoparticles derived from particle size distribution measurements show surprisingly little dependency on potentially condensable vapors observed in the gas phase. Here, we show that the observed nanoparticle growth rates in the sub-10 nm size range can be predicted in the boreal forest only for springtime conditions, even with state-of-the-art mass spectrometers and particle sizing instruments. We find that, especially under warmer conditions, observed growth is slower than predicted from gas-phase condensation. We show that only a combination of simple particle-phase reaction schemes, phase separation due to non-ideal solution behavior, or particle-phase diffusion limitations can explain the observed lower growth rates. Our analysis provides first insights as to why atmospheric nanoparticle growth rates above 10 nm h⁻¹ are rarely observed. Ultimately, a reduction of experimental uncertainties and improved sub-10 nm particle hygroscopicity and chemical composition measurements are needed to further investigate the occurrence of such a growth rate-limiting process.