The air temperature and relative humidity inside refrigerated containers should be kept at optimal conditions to prevent fruit losses. However, each shipment evolves differently in terms of shipping duration and temperature. Moreover, due to the dense stacking of the pallets, non-uniform hygrothermal conditions develop throughout the cargo. Currently, little is known about the fruit quality evolution inside containers, inhibiting proper product management and causing fruit losses to occur. The goal of this study is to identify where and when fruit quality is lost during overseas transport. More specifically, it is aimed to pinpoint where quality issues are expected to occur: consistently on a specific location within the cargo, or only within certain shipments. To achieve this goal, a digital twin was developed of a refrigerated container filled with ‘Valencia’ oranges. This digital twin consisted of a multiphysics model in which airflow, heat, and moisture transport were calculated. Moreover, different fruit quality attributes were predicted. Supply air temperature sensor data of shipments from South Africa to Europe and Asia, as well as the fruit packaging type, were fed to the digital twin. Except for mass loss, the results showed limited quality gradients within the container. Also, when shifting the packaging type, the average quality of the cargo did not change significantly. In contrast to the limited intra-container quality heterogeneity, significant inter-container differences were observed. This indicates the added value of monitoring each shipment. Overall, the digital twin serves as a first step towards real-time quality monitoring within fresh fruit supply chains.