The appearance of a plateau in the magnetization of a quantum spin system subject to continuously varying magnetic field invites the identification of a topological quantization. Indeed, the magnetization plateaus at 1/8 and 1/2 of saturation in TmB4 have been suggested to be intrinsic, resulting from such a topological quantization, or, alternatively, to be metastable phases. By means of neutron- and x-ray-scattering experiments and magnetization measurements, we show that the 1/8 plateau is metastable, arising because the spin dynamics are frozen below T ≈ 4.5 K. Our experiments show that in this part of the phase diagram of TmB₄, many long-ranged orders with different propagation vectors may appear and coexist, particularly as the applied field drives the system from one plateau to another. The magnetic structures accommodating a magnetization of ≈1/8 seem to be particularly favorable, but still only appear if the system has sufficient dynamics to reorganize into a superstructure as it is driven toward the expected plateau. This work demonstrates that TmB₄ represents a model material for the study of slow dynamics, in and out of equilibrium.