Fragile X syndrome (FXS), the most common inherited cause of intellectual disability and autism spectrum disorder, is associated with significant behavioral, social, and neurocognitive deficits. Understanding structural brain network topology in FXS provides an important link between neurobiological and behavioral/cognitive symptoms of this disorder. We investigated the connectome via whole-brain structural networks created from group-level morphological correlations. Participants included 100 individuals: 50 with FXS and 50 with typical development, age 11-23 years. Results indicated alterations in topological properties of structural brain networks in individuals with FXS. Significantly reduced small-world index indicates a shift in the balance between network segregation and integration and significantly reduced clustering coefficient suggests that reduced local segregation shifted this balance. Caudate and amygdala were less interactive in the FXS network further highlighting the importance of subcortical region alterations in the neurobiological signature of FXS. Modularity analysis indicates that FXS and typically developing groups’ networks decompose into different sets of interconnected sub networks, potentially indicative of aberrant local interconnectivity in individuals with FXS. These findings advance our understanding of the effects of fragile X mental retardation protein on large-scale brain networks and could be used to develop a connectome-level biological signature for FXS.