Atomistic deformation simulations in the nominally elastic regime are performed for a model binary glass with strain rates as low as 104/s (corresponding to 0.01 shear strain per 1 µs). A strain rate dependent elastic softening due to a micro-plasticity is observed, which is mediated by thermally-activated localized structural transformations (LSEs). A closer inspection of the atomic-scale structure indicates the material response is distinctly different for two types of local atomic environments. A system spanning iscosahedrally coordinated substructure responds purely elastically, whereas the remaining substructure admits both elastic and microplastic evolution. This leads to a heterogeneous internal stress distribution which, upon unloading, results in negative creep and complete residual-strain recovery. A detailed structural analysis in terms of local stress, atomic displacement, and SU(2) local bonding topology shows such microscopic processes can result in large changes in local stress and are more likely to occur in geometrically frustrated regions characterized by higher free volume and softer elastic stiffness. The thermally-activated LSE activity also mediates structural relaxation, and in this way should be distinguished from stress-driven shear transformation activity which only rejuvenates glass structure. The frequency of LSE activity, and therefore the amount of micro-plasticity, is found to be related to the degree to which the glassy state is relaxed. These insights shed atomistic light onto the structural origins that may govern recent experimental observations of significant structural evolution in response to elastic loading protocols.