The ability of plants to sequester carbon is highly variable over the course of the year and reflects seasonal variation in photosynthetic efficiency. This seasonal variation is most prominent during autumn, when leaves of deciduous tree species such as sugar maple (Acer saccharum Marsh.) undergo senescence, which is associated with downregulation of photosynthesis and a change of leaf color. The remote sensing of leaf color by spectral reflectance measurements and digital repeat images is increasingly used to improve models of growing season length and seasonal variation in carbon sequestration. Vegetation indices derived from spectral reflectance measurements and digital repeat images might not adequately reflect photosynthetic efficiency of red-senescing tree species during autumn due to the changes in foliar pigment content associated with autumn phenology. In this study, we aimed to assess how effectively several widely used vegetation indices capture autumn phenology and reflect the changes in physiology and photosynthetic pigments during autumn. Chlorophyll fluorescence and pigment content of green, yellow, orange and red leaves were measured to represent leaf senescence during autumn and used as a reference to validate and compare vegetation indices derived from leaf-level spectral reflectance measurements and color analysis of digital images. Vegetation indices varied in their suitability to track the decrease of photosynthetic efficiency and chlorophyll content despite increasing anthocyanin content. Commonly used spectral reflectance indices such as the normalized difference vegetation index and photochemical reflectance index showed major constraints arising from a limited representation of gradual decreases in chlorophyll content and an influence of high foliar anthocyanin levels. The excess green index and green–red vegetation index were more suitable to assess the process of senescence. Similarly, digital image analysis revealed that vegetation indices such as Hue and normalized difference index are superior compared with the often-used green chromatic coordinate. We conclude that indices based on red and green color information generally represent autumn phenology most efficiently.