The advancement of the techniques of producing electromagnetic radiation in the terahertz range (1–20 THz) attracts a wide range of potential applications and the quest for powerful THz sources is ever-increasing. The means to generate THz is an active research field and the energy per pulse is limited due to breakdown of the medium used. In this paper, we propose a compact yet powerful narrowband THz source based on two pillars of active research in the field of novel accelerators: laser plasma wakefield acceleration and high gradient dielectric lined structures. The laser plasma wakefield (LPW) injector is used to provide the relativistic electron bunches to an interaction tube, a dielectric lined waveguide (DLW), in order to generate THz. Due to the high gradient nature of both devices, the combination ends up with a very compact solution for mJ scale THz pulse generation. Potentially these pulses can be used as drivers for nonlinear phenomena in condensed matter, which can be probed using ultrashort x-ray pulses at the same free electron laser facility.