Rashba materials have appeared as an ideal playground for spin-to-charge conversion in prototype spintronics devices. Among them, α-GeTe(111) is a non-centrosymmetric ferroelectric semiconductor for which a strong spin-orbit interaction gives rise to giant Rashba coupling. Its room temperature ferroelectricity was recently demonstrated as a route towards a new type of highly energy-efficient non-volatile memory device based on switchable polarization. Currently based on the application of an electric field, the writing and reading processes could be outperformed by the use of femtosecond light pulses requiring exploration of the possible control of ferroelectricity on this timescale. Here, we probe the room temperature transient dynamics of the electronic band structure of α-GeTe(111) using time and angle-resolved photoemission spectroscopy. Our experiments reveal an ultrafast modulation of the Rashba coupling mediated on the fs timescale by a surface photovoltage, namely an increase corresponding to a 13% enhancement of the lattice distortion. This opens the route for the control of the ferroelectric polarization in α-GeTe(111) and ferroelectric semiconducting materials in quantum heterostructures.