Hearing and cognition are commonly involved in both normal and pathological aging. Current clinical interest lies in whether peripheral hearing loss promotes cognitive decline. In our previous publication, the authors have shown a causal relationship between hearing and cognitive impairments in C57BL/6 mice. Here we extended the follow-up to 12 months to determine the long-term effects of hearing loss on cognition and to observe hippocampal p-tau and lipofuscin. One month old male mice were randomly allocated into two groups, the control (n = 12) and noise-induced hearing loss (NIHL) (n = 12). After baseline hearing and cognitive measurements, the mice in the NIHL group were exposed to 110 dB SPL white noise for 1 h every day for 20 consecutive days. Cognitive function was assessed by radial arm maze and novel object recognition tests. p-Tau was observed by the western blot, immunofluorescence, and immunogold staining. The mice in the NIHL group showed elevated auditory brainstem response thresholds and poorer performances in spatial working and recognition memories than the controls. They exhibited more p-tau and lipofuscin in the hippocampus. The cognitive impact of hearing loss varied with the types of memory. Working memory impairment was reversible, whereas recognition memory impairment was permanent. Our results provide behavioral and histopathological evidence for hearing-related cognitive decline. Early hearing loss is suggested to be one of the important determinants between normal and pathological cognitive aging.