To examine the relationship of total and free testosterone and sex hormone-binding globulin (SHBG) with central obesity in men, we studied 1548 men aged 25–84 years that took part in the 1994–1995 survey of the Tromsø study. Total testosterone and SHBG were measured by immuno-assay and the free testosterone fraction was calculated. These measurements were analyzed in relation to anthropometric data and lifestyle factors. The age-adjusted correlation between waist circumference (WC) and total testosterone was −0.34 (p < 0.001), between WC and free testosterone −0.09 (p < 0.001) and, between WC and SHBG −0.44 (p < 0.001). Adjusting for BMI and lifestyle factors weakened, but did not eliminate these associations. All hormone and SHBG associations were stronger for WC than for waist-hip ratio or BMI. In age- and BMI-adjusted analyses men with a WC ≥ 102 cm had significantly lower levels of total testosterone and SHBG compared to men with an optimal WC, defined as < 94 cm (12.3 vs. 13.9 nmol/l; p < 0.01 and 48.5 vs. 55.1 nmol/l; p < 0.001, respectively). The lowest levels of total and free testosterone were observed in men with relatively high WC despite relatively low overall obesity (BMI), suggesting that WC should be the preferred anthropometric measurement in predicting endogenous testosterone levels.