Alzheimer's disease (AD) is the most common type of dementia and is characterized by a progression from decline of episodic memory to a global impairment of cognitive function. Astrogliosis is a hallmark feature of AD, and reactive gliosis has been considered as an important target for intervention in various neurological disorders. We previously found in astrocyte cultures that the expression of the intermediate conductance calcium-activated potassium channel KCa3.1 was increased in reactive astrocytes induced by TGF-β, while pharmacological blockade or genetic deletion of KCa3.1 attenuated astrogliosis. In this study, we sought to suppress reactive gliosis in the context of AD by inhibiting KCa3.1 and evaluate its effects on the cognitive impairment using murine animal models such as the senescence-accelerated mouse prone 8 (SAMP8) model that exhibits some AD-like symptoms. We found KCa3.1 expression was increased in reactive astrocytes as well as neurons in the brains of both SAMP8 mice and Alzheimer's disease patients. Blockade of KCa3.1 with the selective inhibitor TRAM-34 in SAMP8 mice resulted in a decrease in astrogliosis as well as microglia activation, and moreover an attenuation of memory deficits. Using KCa3.1 knockout mice, we further confirmed that deletion of KCa3.1 reduced the activation of astrocytes and microglia, and rescued the memory loss induced by intrahippocampal Aβ1–42 peptide injection. We also found in astrocyte cultures that blockade of KCa3.1 or deletion of KCa3.1 suppressed Aβ oligomer-induced astrogliosis. Our data suggest that KCa3.1 inhibition might represent a promising therapeutic strategy for AD treatment.