Several cross-sectional studies have found cognitive impairment in subjects taking benzodiazepines for long periods. However, it is not known whether long-term use of benzodiazepines accelerates cognitive decline in the elderly. The authors addressed this issue in a follow-up study of 1,389 people aged 60 to 70 years recruited from the electoral rolls of the city of Nantes (Epidemiology of Vascular Aging study). Data on cognitive functioning (five cognitive tests), depressive symptoms (Center for Epidemiologic Studies-Depression Scale), anxiety symptoms (Spielberger Inventory Scale), use of psychotropics and other drugs, and tobacco and alcohol consumption were collected at baseline, 2-year, and 4-year examinations. People reporting to take benzodiazepines at one, two and three examinations were classified as episodic, recurrent, and chronic users, respectively. Among the 1,176 subjects (85%) who participated in the three examinations, the proportions of episodic, recurrent, and chronic users were 10%, 6%, and 7%, respectively. Chronic users of benzodiazepines had a significantly higher risk of cognitive decline in the global cognitive test (Mini Mental State Examination) and the two attention tests than nonusers (Mini Mental State Examination: odds ratio [OR] [95% confidence interval (CI)] = 1.9 [1.0–3.5]; Digit Symbol Substitution test: OR [95% CI] = 2.7 [1.6–4.7]; Trail Making test, part B: OR [95% CI] = 2.1 [1.2–3.7]). Overall, episodic and recurrent users had lower cognitive scores than nonusers, but differences were not statistically significant. These results were independent of age, sex, education, alcohol and tobacco use, anxiety and depression scores, and use of psychotropic drugs other than benzodiazepines. The findings suggest that long-term use of benzodiazepines is risk factor of increased cognitive decline in the elderly.