Although ptychography does not require a precise knowledge of the illumination wavefront, common implementations rely upon assumptions such as accurate knowledge of the scan positions and constant illumination. Limited validity of these assumptions results in deterioration of the reconstruction quality. We present a generalized ptychography method that optimizes the reconstruction along multiple directions. In our manuscript, we demonstrate that the additional flexibility not only helps to amend imprecisions of the ptychography model in a self-consistent way but additionally leads to faster convergence without a significant increase of the computational cost per iteration.