Understanding how plant-associated microbial communities assemble and the role they play in plant performance are major goals in microbial ecology. For nitrogen-fixing rhizobia, community assembly is generally driven by host plant selection and soil conditions. Here, we aimed to determine the relative importance of neutral and deterministic processes in the assembly of bacterial communities of root nodules of a legume shrub adapted to extreme nutrient limitation, rooibos (Aspalathus linearis Burm. Dahlgren). We grew rooibos seedlings in soil from cultivated land and wild habitats, and mixtures of these soils, sampled from a wide geographic area, and with a fertilization treatment. Bacterial communities were characterized using next generation sequencing of part of the nodA gene (i.e. common to the core rhizobial symbionts of rooibos), and part of the gyrB gene (i.e. common to all bacterial taxa). Ecological drift alone was a major driver of taxonomic turnover in the bacterial communities of root nodules (62.6% of gyrB communities). In contrast, the assembly of core rhizobial communities (genus Mesorhizobium) was driven by dispersal limitation in concert with drift (81.1% of nodA communities). This agrees with a scenario of rooibos-Mesorhizobium specificity in spatially separated subpopulations, and low host filtering of other bacteria colonizing root nodules in a stochastic manner.