Human life expectancy is constantly increasing and aging has become a major risk factor for many diseases, although the underlying gene regulatory mechanisms are still unclear. Using transcriptomic and chromosomal conformation capture (Hi-C) data from human skin fibroblasts from individuals across different age groups, we identified a tight coupling between the changes in co-regulation and co-localization of genes. We obtained transcription factors, cofactors, and chromatin regulators that could drive the cellular aging process by developing a time-course prize-collecting Steiner tree algorithm. In particular, by combining RNA-Seq data from different age groups and protein–protein interaction data we determined the key transcription regulators and gene regulatory changes at different life stage transitions. We then mapped these transcription regulators to the 3D reorganization of chromatin in young and old skin fibroblasts. Collectively, we identified key transcription regulators whose target genes are spatially rearranged and correlate with changes in their expression, thereby providing potential targets for reverting cellular aging.