Heatwaves threaten ecosystem carbon balances, yet the mechanisms driving short-term carbon flux responses remain poorly understood. Here, integrating high-frequency eddy covariance (EC) data from 140 global flux tower sites (872 site-years) with detailed process-based modeling, we examine ecosystem responses during and immediately after heatwaves. We show that heatwaves caused a −40% (range [−29%, −128%]) reduction in net ecosystem productivity (NEP) compared to pre-heatwave values, with this reduction persisting over the following two weeks (−38% range [+3%, −154%]). We attributed NEP decreases to photosynthesis decreases more than to ecosystem respiration (RE) increases. Forest sites had greater NEP decreases during heatwaves than non-forest sites, but remained carbon sinks afterwards, indicating resilience. Our modeling analysis of extreme heatwaves at selected EC sites shows that decreased photosynthesis, increased maintenance respiration, and decreased plant non-structural carbon reserves during heatwaves drive carbon cycle changes that persist for weeks. Consistent with phenocam observations, we modeled a reduction in leaf area index caused by reduced non-structural carbon reserves, leading to early leaf senescence and longer-term impacts. Ongoing increases in heatwaves are therefore likely to reduce NEP across a range of ecosystems, exacerbating carbon cycle feedback.