Wave-front propagation simulations have been a tool to design and optimize X-ray interferometry devices. The often used plane wave approaches, however, lack the angular resolution to describe effects like system imperfections or inhomogeneous samples in conjunction with the X-ray source size. We developed a framework that allows to simulate optical components as well as samples with any source size in arbitrary configurations by inducing the mentioned effects within the wave propagation instead of adding intermediate models. The simulation results were able to predict and explain the impact of local grating defects for different focal spot sizes and provided a spectral sampling optimization for image acquisition. The simulation framework can run on GPU, do out-of-memory calculations, and is publicly available on Github.