Vapor sorption in hygroscopic porous materials is accompanied by latent heat release/storage, which can influence indoor thermal comfort and building heating and cooling energy consumption. There is a need to better understand the coupled vapor and heat transport during adsorption and desorption. In this study, longitudinal spruce samples are exposed to adsorption and desorption experiments. Neutron radiography provides accurate measurement of moisture content variations spatially and temporally. Wireless thermocouples provide accurate measurements of temperature at different locations. Large changes in moisture content and temperature are observed during both adsorption and desorption experiments. Both moisture content and temperature variations seen in experiments are well simulated with hygrothermal modeling. The latent heat associated with vapor sorption is found to be the source of the large variations in temperature. It is found that vapor permeability influences both vapor and thermal transport while thermal conductivity influences only thermal transport. The vapor transfer coefficient has a small influence on vapor transport while the convective heat transfer coefficient has an influence on heat transport. The validated hygrothermal model is further used to simulate the coupled vapor and heat transport occurring in moisture buffering tests. It is found that moisture buffering values are different by up to 14% depending on the presence or absence of thermal insulation around the samples. For more hygroscopic materials, the difference can be even much larger. It is recommended not only to seal and but also to insulate samples for moisture buffering tests.