To assess the incidence and severity of changes in hearing threshold in patients undergoing high-dose pencil-beam-scanning proton therapy (PBS-PT). This retrospective cohort study included fifty-one patients (median 50 years (range, 13–68)) treated with PBS-PT for skull base tumors. No chemotherapy was delivered. Pure tone averages (PTAs)were determined before (baseline) and after PBS-PT as the average hearing thresholds at frequencies of 0.5, 1, 2, and 4 kHz. Hearing changes were calculated as PTA differences between pre-and post-PBS-PT. A linear mixed-effects model was used to assess the relationship between the PTA at the follow-up and the baseline, the cochlea radiation dose intensity, the increased age, and the years after PBS-PT. Included patients were treated for chordoma (n = 24), chondrosarcoma (n = 9), head and neck tumors (n = 9), or meningioma (n = 3), with a mean tumor dose of 71.1 Gy (RBE) (range, 52.0–77.8), and a mean dose of 37 Gy (RBE) (range, 0.0–72.7) was delivered to the cochleas. The median time to the first follow-up was 11 months (IQR, 5.5–33.7). The PTA increased from a median of 15 dB (IQR 10.0–25) at the baseline to 23.8 (IQR 11.3–46.3) at the first follow-up. In the linear mixed-effect model, the baseline PTA (estimate 0.80, 95%CI 0.64 to 0.96, p ≤ 0.001), patient’s age (0.30, 0.03 to 0.57, p = 0.029), follow-up time (2.07, 0.92 to 3.23, p ≤ 0.001), and mean cochlear dose in Gy (RBE) (0.34, 0.21 to 0.46, p ≤ 0.001) were all significantly associated with an increase in PTA at follow-up. The applied cochlear dose and baseline PTA, age, and time after treatment were significantly associated with hearing loss after proton therapy.