The El Niño–Southern Oscillation (ENSO) is a key driver of global climate variability, and its impact on phytoplankton concentrations in the eastern equatorial Pacific via nutrient supply changes is well established. However, the extent to which phytoplankton feedback influences ENSO remains unclear. Chlorophyll in phytoplankton warms the upper ocean by absorbing solar radiation, and this effect weakens during El Niño and strengthens during La Niña, yet its overall impact is not well quantified. Using a simple nitrogen-based Nitrate–Phytoplankton–Zooplankton–Detritus pelagic model, here we show that phytoplankton concentration anomalies significantly dampen ENSO by cooling sea surface temperature by 0.69 °C during El Niño development and warming it by 1.09 °C during La Niña, with mean amplitudes of 1.71 °C and 1.42 °C, respectively. This may partially contributes to the amplitude asymmetry of ENSO, and accounts for 16.8% of total shortwave radiation-related damping during El Niño and 17.4% during La Niña. Our offline modeling approach successfully isolates this direct heating effect by excluding indirect dynamical effects of phytoplankton to physical variables.