Cross-borehole seismic tomography is a powerful tool to investigate the subsurface with a very high spatial resolution. In a set of boreholes, comprehensive three-dimensional investigations at different depths can be conducted to analyse velocity anisotropy effects due to local changes within the medium. Especially in glaciological applications, the drilling of boreholes with hot water is cost-efficient and provides rapid access to the internal structure of the ice. In turn, movements of the subsurface such as the continuous flow of ice masses cause deformations of the boreholes and complicate a precise determination of the source and receiver positions along the borehole trajectories. Here, we present a three-dimensional inversion scheme that considers the deviations of the boreholes as additional model parameters next to the common velocity inversion parameters. Instead of introducing individual parameters for each source and receiver position, we describe the borehole trajectory with two orthogonal polynomials and only invert for the polynomial coefficients. This significantly reduces the number of additional model parameters and leads to much more stable inversion results. In addition, we also discuss whether the inversion of the borehole parameters can be separated from the velocity inversion, which would enhance the flexibility of our inversion scheme. In that case, updates of the borehole trajectories are only performed if this further reduces the overall error in the data sets. We apply this sequential inversion scheme to a synthetic data set and a field data set from a temperate Alpine glacier. With the sequential inversion, the number of artefacts in the velocity model decreases compared to a velocity inversion without borehole adjustments. In combination with a rough approximation of the borehole trajectories, for example, from additional a priori information, heterogeneities in the velocity model can be imaged similarly to an inversion with fully correct borehole coordinates. Furthermore, we discuss the advantages and limitations of our approach in the context of an inherent seismic anisotropy of the medium and extend our algorithm to consider an elliptic velocity anisotropy. With this extended version of the algorithm, we analyse the interference between a seismic anisotropy in the medium and the borehole coordinate adjustment. Our analysis indicates that the borehole inversion interferes with seismic velocity anisotropy. The inversion can compensate for such a velocity anisotropy. Based on the modelling results, we propose considering polynomials up to degree 3. For such a borehole trajectory inversion, third-order polynomials are a good compromise between a good representation of the true borehole trajectories and minimising compensation for velocity anisotropy.