The purpose of this study was to determine the influence of various types of childhood adversity on later sexual deviance and sexually violent behavior. Data were collected from more than 700 convicted sexual offenders in outpatient and confinement-based treatment programs throughout the U.S. Using the 10-item Adverse Childhood Experiences (ACE) Scale, participants were surveyed about childhood maltreatment and family dysfunction. For male sex offenders, factors that significantly predicted sexual deviance included childhood sexual abuse, emotional neglect, and having unmarried parents. Factors that significantly predicted violent sexual offending included child physical abuse, substance abuse in the childhood home, mental illness in the home, and having an incarcerated family member. ACE scores were significantly higher for generalist offenders than for those specializing in sexual crime. The results underscore the need for clinicians to assess the existence of early adversity, to understand the role of traumatic events in the development of criminality and abusive behaviors, and to utilize trauma-informed counseling practices. In terms of policy, investing in prevention services for maltreated children and at-risk families is an important step in disrupting the cycle of interpersonal violence and crime in our communities.