Abstract Initialized climate prediction systems provide forecasts on seasonal‐to‐decadal timescales, but their computational cost has motivated the development of cheaper alternatives such as analog‐based approaches. Recent studies suggest that constraining large climate model ensembles using analogs of past sea surface temperature (SST) can yield skillful forecasts on multi‐annual to multi‐decadal timescales. Here, we adapt an analog‐based method to a perfect‐model framework, using large ensembles to assess maximum potential skill arising from internal variability. Results show that constraining internal variability produces skillful forecasts for global and regional SSTs up to 2–3 years, with declining predictability on longer timescales. Beyond 5 years, skill largely vanishes, indicating that multi‐decadal skill in previous studies might be driven by forced climate response rather than internal variability. These findings benchmark the limits of a collection of analog‐based prediction approaches, based on SST pattern correlations, and provide a framework for disentangling sources of skill in near‐term climate predictions.