A growing body of research has examined whether people’s judgments of their own memory functioning accurately reflect their memory performance at cross-section and over time. Relatively less is known about whether these judgments are specifically based on memory performance, or reflect general cognitive change. The aim of the present study was to examine longitudinal associations of subjective memory with performance in tests of episodic memory and a wide range of other cognitive tests, including the Wechsler Adult Intelligence Scale—Revised (WAIS–R) Block Design, Comprehension, Digit Span, Digit Symbol, and Vocabulary subtests. We applied latent growth curve models to five occasions over up to 16 years of neuropsychological assessments from 956 participants of the Seattle Longitudinal Study (SLS; 57% women; age at baseline: M = 65.1, SD = 11.4, 38 – 96 years). Results revealed that lower self-reported Frequency of Forgetting was significantly associated with better performance in all cognitive domains at baseline. The baseline correlation of Frequency of Forgetting with memory performance was stronger than its correlations with performance in other cognitive tests. Furthermore, additional analyses with baseline data showed that a latent memory performance factor reliably predicted Frequency of Forgetting after controlling for a general cognitive factor. Over time, steeper increases in Frequency of Forgetting were associated with steeper declines in tests of memory performance and in the Block Design and Digit Symbol subtests. Taken together, these findings suggest that although self-reported Frequency of Forgetting reflects performance in a broad range of other cognitive domains, it also shows some specificity for memory performance.