Judiciously designed two-dimensional THz metamaterials consisting of resonant metallic structures embedded in a dielectric environment locally enhance the electromagnetic field of an incident THz pulse to values sufficiently high to cause nonlinear responses of the environment. In semiconductors, the response is attributed to nonlinear transport phenomena via intervalley scattering, impact ionization, or interband tunneling and can affect the resonant behavior of the metallic structure, which results, for instance, in mode switching. However, details of mode switching, especially time scales, are still debated. By using metallic split-ring resonators with nm-size gaps on intrinsic semiconductors with different bandgaps, we identify the most relevant carrier generation processes. In addition, by combining nonlinear THz time-domain spectroscopy with simulations, we establish the fastest time constant for mode switching to around hundred femtoseconds. Our results not only elucidate dominant carrier generation mechanisms and dynamics but also pave the route toward optically driven modulators with THz bandwidth.