Brain-behavior associations in fMRI studies are typically restricted to a single level of analysis: either a circumscribed brain region-of-interest (ROI) or a larger network of brain regions. However, this common practice may not always account for the interdependencies among ROIs of the same network or potentially unique information at the ROI-level, respectively. To account for both sources of information, we combined measurement and structural components of structural equation modeling (SEM) approaches to empirically derive networks from ROI activity, and to assess the association of both individual ROIs and their respective whole-brain activation networks with task performance using three large task-fMRI datasets and two separate brain parcellation schemes. The results for working memory and relational tasks revealed that well-known ROI-performance associations are either non-significant or reversed when accounting for the ROI's common association with its corresponding network, and that the network as a whole is instead robustly associated with task performance. The results for the arithmetic task revealed that in certain cases, an ROI can be robustly associated with task performance, even when accounting for its associated network. The SEM framework described in this study provides researchers additional flexibility in testing brain-behavior relationships, as well as a principled way to combine ROI- and network-levels of analysis.