Observed touch interactions provide useful information on how others communicate with the external world. Previous studies revealed shared neural circuits between the direct experience and the passive observation of simple touch, such as being stroked/slapped. Here, we investigate the complexity of the neural representations underlying the understanding of others' socio-affective touch interactions. Importantly, we use a recently developed touch database that contains a larger range of more complex social and non-social touch interactions. Participants judged affective aspects of each touch event and were scanned while watching the same videos. Using correlational multivariate pattern analysis methods, we obtained neural similarity matrices in 18 regions of interest from five different networks: somatosensory, pain, the theory of mind, visual and motor regions. Among them, four networks except motor cortex represent the social nature of the touch, whereas fine-detailed affective information is reflected in more targeted areas such as social brain regions and somatosensory cortex. Lastly, individual social touch preference at the behavioral level was correlated with the involvement of somatosensory areas on representing affective information, suggesting that individuals with higher social touch preference exhibit stronger vicarious emotional responses to others' social touch experiences. Together, these results highlight the overall complexity and the individual modulation of the distributed neural representations underlying the processing of observed socio-affective touch.