We demonstrate that collective vibrational strong coupling of molecules in thermal equilibrium can give rise to significant local electronic polarizations in the thermodynamic limit. We do so by first showing that the full nonrelativistic Pauli-Fierz problem of an ensemble of strongly coupled molecules in the dilute-gas limit reduces in the cavity Born-Oppenheimer approximation to a cavity-Hartree equation for the electronic structure. Consequently, each individual molecule experiences a self-consistent coupling to the dipoles of all other molecules, which amount to non-negligible values in the thermodynamic limit (large ensembles). Thus, collective vibrational strong coupling can alter individual molecules strongly for localized ”hotspots” within the ensemble. Moreover, the discovered cavity-induced polarization pattern possesses a zero net polarization, which resembles a continuous form of a spin glass (or better polarization glass). Our findings suggest that the thorough understanding of polaritonic chemistry, requires a self-consistent treatment of dressed electronic structure, which can give rise to numerous, so far overlooked, physical mechanisms.