Diamagnetic oxides can, under certain conditions, become ferromagnetic at room temperature and therefore are promising candidates for future material in spintronic devices. Contrary to early predictions, doping ZnO with uniformly distributed magnetic ions is not essential to obtain ferromagnetic samples. Instead, the nanostructure seems to play the key role, as room temperature ferromagnetism was also found in nanograined, undoped ZnO. However, the origin of room temperature ferromagnetism in primarily non-magnetic oxides like ZnO is still unexplained and a controversial subject within the scientific community. Using low energy muon spin relaxation in combination with SQUID and TEM techniques, we demonstrate that the magnetic volume fraction is strongly related to the sample volume fraction occupied by grain boundaries. With molecular dynamics and density functional theory we find ferromagnetic coupled electron states in ZnO grain boundaries. Our results provide evidence and a microscopic model for room temperature ferromagnetism in oxides.