Objective: To assess whether neurodegenerative pathologies are differentially related to trajectories of change in different cognitive abilities. Methods: At annual intervals for up to 21 years, 915 older participants in a longitudinal clinical-pathologic cohort study completed a battery of 15 tests from which previously established composite measures of episodic memory, semantic memory, working memory, and perceptual speed were derived. At death, they underwent a neuropathologic examination to quantify Alzheimer’s disease (AD) pathology, Lewy bodies, transactive response DNA-binding protein 43 (TDP-43) pathology, and hippocampal sclerosis plus multiple markers of cerebrovascular disease. Time-varying effect models were used to assess change over time in the relation of neuropathologic markers to cognitive trajectories. Results: Controlling for pathology, decline in perceptual speed was evident about 15 years before death; modest decline in semantic and working memory occurred later; and there was little change in episodic memory. Each neurodegenerative marker was associated with lower episodic memory function beginning about 10 to16 years before death. As time before death decreased, AD pathology, Lewy bodies, and hippocampal sclerosis were associated with impairment in other cognitive domains but the association of TDP-43 pathology with cognition continued to be mainly confined to episodic memory. Conclusions: The results suggest that episodic memory impairment is an early sign of multiple neurodegenerative conditions which primarily differ in their associations with other cognitive systems.