Nitrogen (N) legacy effects are widely recognized as a primary obstacle to the improvements of water quality following mitigation efforts. The exploration of long-term nutrient trajectories for nitrogen model (ELEMeNT-N) has been applied to estimate legacy effects in several watersheds globally. However, ELEMeNT-N does not account for the accumulation-release processes of nitrate within soil profile, introducing potential uncertainty in long-term simulations. This study enhances ELEMeNT-N by incorporating a soil nitrate accumulation coefficient and a deep soil nitrate cycling module to better assess the long-term impact of legacy N on water quality. Long-term simulation for riverine N flux in the Yongan watershed, China (1980–2022) indicate that the modified model, ELEMeNT-N-SP, demonstrated significantly improved performance relative to the original model (Nash–Sutcliff coefficient: 0.89 vs 0.72, R2: 0.89 vs. 0.71). ELEMeNT-N-SP results indicate that ∼39.5% of N input accumulated in soil and groundwater, mainly in agricultural soil-groundwater. Legacy N contributed to 81.7% (Temporal: 65.6%–92.0%; i.e. cumulative contributions of historical N inputs) of riverine N flux (466 kg N ha−1). Among 12 modeling units, legacy N accumulation (SON: 175–1083 kg N ha−1; soil nitrate: −11–259 kg N ha−1; groundwater: 147–656 kg N ha−1) and its contribution to riverine N flux (Spatial: 64.0%–85.5%) showed considerable spatial heterogeneity. Agricultural soil-groundwater served as the primary accumulation sink for N input and the important source of riverine N pollution. Model validation, which included comparisons with monitored soil-groundwater N content and dual stable isotope data, confirmed the reliability of ELEMeNT-N-SP. ELEMeNT-N-SP offers a valuable framework for improving watershed nutrient management globally, emphasizing the importance of incorporating soil N cycling dynamics into predictive tools.