Evidence from both human and animal studies suggests that gonadal steroids, such as testosterone, exert activational effects on adult spatial behavior. Endogenous testosterone levels decline gradually but variably as men age; it remains to be shown whether these decreases are associated with age-related declines in visuo-spatial performance or constituent abilities indicative of generalized age-related cognitive decline. Ninety-six healthy, community dwelling men aged between 38 and 69 years completed the Vandenberg and Kuse Mental Rotation Test (MRT) together with a battery of tests including processing speed, executive function, perceptual discrimination, working memory, and reaction time measures. Significant main effects of tertiles of calculated free testosterone levels (cEFT) were found on composite measures of processing speed, executive function, and perceptual discrimination ability in a subset of men aged over 50 years in age and crystallized intelligence controlled analyses; higher cEFT levels were associated with poorer performance. Hierarchical multiple regression and path analyses on the whole data set showed that cEFT levels negatively moderated processing speed performance, which in turn predicted both working memory and MRT performance with aging. Together these data suggest that age-related declines in endogenous testosterone levels in healthy middle-to-older aged men are not associated with generalized age-related cognitive decline.