Ongoing climate change threatens to cause mismatches between the phenology of many organisms and their resources. Populations of migratory birds may need to undergo ‘evolutionary rescue’ if resource availability moves to earlier dates in the year, as shifted arrival dates at the breeding grounds may be required for persistence under new environmental schedules. Here we show a counterintuitive process that can reduce the strength of selection for early arrival when the resource peaks earlier. This happens when two processes combine to determine selection for early arrival: breeding success is higher if a bird does not miss the resource peak, but this occurs together with a ‘zero-sum game’ where birds acquire good territories ahead of their competitors if they arrive early. The latter process can relax if the population has experienced a recent decline. Therefore, climate change can have two opposing effects: its direct effect on breeding success strengthens selection for early arrival, but this combines with an indirect effect of relaxed selection due to population declines, if territoriality is a significant determinant of population dynamics and fitness. We show that the latter process can predominate, and this can cause a failure for a population to adapt to a new schedule under changing environmental conditions.