Objective: Criminologists consistently find a correlation between criminal victimization and criminal offending across a variety of populations and types of crime; however, research on the relationship between adolescent and adult sexual assault (ASA) perpetration and ASA victimization is limited. Comparing the sexual attitudes of men who are both victims and perpetrators of ASA, men who are victims only, men who are perpetrators only, and men who are neither victims nor perpetrators may highlight unique sexual attitudes that create a dual vulnerability to ASA perpetration and victimization. Method: A convenience sample of community men (N = 268) completed an online survey and reported on their ASA perpetration and victimization history and their experiences of child sexual abuse. They also completed measures of traumatic sexual beliefs, dysfunctional sexual behaviors, sexual avoidance, sexual preoccupation, and distorted sexual self-concept. Results: Results revealed a statistically significant overlap between men’s perpetration and victimization of ASA. Compared with victims-only, perpetrators-only, and men with no ASA history, men with a history of both perpetration and victimization reported the highest rates of childhood sexual abuse and the highest levels of distorted sexual self-concept. Conclusions: Targeting men’s sexual self-concept may be an effective way to reduce their risk of both ASA victimization and ASA perpetration.