How climatic factors influence the balance of sulfur (S) with carbon (C) and nitrogen (N) in soils is not well understood. Here we report the results of S, C and N concentrations in 550 soil samples along a 3500-km aridity gradient across arid and semi-arid grasslands in northern China. Our results showed that soil available S (AS) concentrations decreased with increasing aridity index (AI, calculated as '1—the ratio of mean annual precipitation to potential evapotranspiration') when AI < 0.91 (90–450 mm rainfall year⁻¹), but increased when AI > 0.91 (30–90 mm rainfall year⁻¹), having a threshold at AI = 0.91 (90 mm rainfall year⁻¹). Soil AS concentrations were positively related to soil organic C (OC), soil total N (TN), microbial biomass C and N concentrations, and arylsulphatase activities when AI < 0.91, but were not positively related to these variables when AI > 0.91. Topsoil AS:OC and AS:TN ratios mainly depended on OC and TN concentrations when AI < 0.91, but mainly depended on AS concentrations when AI > 0.91. The decreased (AI < 0.91) and increased total S concentrations (AI > 0.91) with increasing AI along the gradient provided further evidence for the existence of this fundamental threshold. High concentrations of sulfate in drier soils were associated with long-term atmosphere inputs and groundwater-derived salts with minimal leaching rates. Overall, our findings imply that aridity asymmetrically controls the relationships of soil AS with OC and TN on the two sides of the threshold, i.e. biological processes exerted stronger controls in wetter sites, while geochemical processes played more significant roles in drier sites. These results should be incorporated into predictive models of global biogeochemical cycling under various global climate change scenarios.