The recording of planetary magnetic fields is often attributed to uniformly-magnetised nanoscopic iron oxides, called single-domain. Yet, the main magnetic constituents of rocks are more complex, non-uniformly magnetised grains in single or multi-vortex states. We know little about their behaviour due to limitations in defining their precise shape and internal magnetic structure. Here we combine non-destructive Ptychographic X-ray Computed Nano-tomography with micromagnetic modelling to explore the magnetic stability of remanence-bearing minerals. Applied to a microscopic rock sample, we identified hundreds of nanoscopic grains of magnetite/maghemite with diverse morphologies. Energy barrier calculations were performed for these irregularly shaped grains. For some grains, these morphological irregularities near the transition from single-domain to the single-vortex state allow for multiple domain states, some unstable and unable to record the field for significant periods. Additionally, some other grains exhibit temperature-dependent occupancy probabilities, potentially hampering experiments to recover the intensity of past magnetic fields.