Soil microbial diversity has major influences on ecosystem functions and services. However, due to its complexity and uneven distribution of abundant and rare taxa, quantification of soil microbial diversity remains challenging and thereby impeding its integration into long-term monitoring programs. Using metabarcoding, we analyzed soil bacterial and fungal communities at 30 long-term soil monitoring sites from the three land-use types arable land, permanent grassland, and forest with a yearly sampling between snowmelt and first fertilization over five years. Unlike soil microbial biomass and alpha-diversity, microbial community compositions and structures were site- and land-use-specific with CAP reclassification success rates of 100%. The temporally stable site core communities included 38.5% of bacterial and 33.1% of fungal OTUs covering 95.9% and 93.2% of relative abundances. We characterized bacterial and fungal core communities and their land-use associations at the family-level. In general, fungal families revealed stronger land-use associations as compared to bacteria. This is likely due to a stronger vegetation effect on fungal core taxa, while bacterial core taxa were stronger related to soil properties. The assessment of core communities can be used to form cultivation-independent reference lists of microbial taxa, which may facilitate the development of microbial indicators for soil quality and the use of soil microbiota for long-term soil biomonitoring.