Fission track detectors in the center of moderating spheres are routinely used to measure the ambient dose equivalent due to neutrons in the environmental dosimetry at Paul Scherrer Institut (PSI). Originally, the system was designed to cope with neutrons from skyshine effects. Later, the system was also adapted behind the shielding of PSI's accelerators. Nowadays, as a consequence of continuously upgrading accelerator energies and intensities, the neutron energy behind thick shielding can range from fractions of eV to about 1 GeV (e.g. at CERN). For this reason a measurement campaign in a high-energy stray radiation field at CERN's High-Energy Reference Field Facility (CERF) was initiated to study and compare the response of the already existing detector-moderator configuration and a new design, the "GSI ball". Employing an additional lead layer in a moderator sphere of 32.5 cm diameter, the GSI ball was primarily designed for the use with thermoluminescent based dosimeters in its center in order to optimize the response for the measurement of H* (10) to higher neutron energies. In this work, the measurement results for fission track detectors using two different radiator materials in the PSI and the GSI moderator are presented. Based on these studies, on the one hand, field calibration factors for the use in presumably similar high-energy fields around accelerators could be deduced. On the other hand, it could be shown that there is no need to replace the established PSI moderator by the GSI moderator since the combination of fission track detector and GSI moderator does not result in a significant sensitivity improvement.