A long-standing concern has been whether epilepsy contributes to cognitive decline or so-called ‘epileptic dementia’. Although global cognitive decline is generally reported in the context of chronic refractory epilepsy, it is largely unknown what percentage of patients is at risk for decline. This review is focused on the identification of risk factors and characterization of aberrant cognitive trajectories in epilepsy. Evidence is found that the cognitive trajectory of patients with epilepsy over time differs from processes of cognitive ageing in healthy people, especially in adulthood-onset epilepsy. Cognitive deterioration in these patients seems to develop in a ‘second hit model’ and occurs when epilepsy hits on a brain that is already vulnerable or vice versa when comorbid problems develop in a person with epilepsy. Processes of ageing may be accelerated due to loss of brain plasticity and cognitive reserve capacity for which we coin the term ‘accelerated cognitive ageing’. We believe that the concept of accelerated cognitive ageing can be helpful in providing a framework understanding global cognitive deterioration in epilepsy.