aFunctional Brain Center, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, 64239 Tel Aviv, IsraelbFilm and Television Department, Tel Aviv University, 69978 Tel Aviv, IsraelcSagol School of Neuroscience, Tel Aviv University, 69978 Tel Aviv, IsraeldDepartment of Information Engineering, University of Brescia, 38, 25123 Brescia, ItalyeSchool of Psychological Sciences, Tel Aviv University, 69978 Tel Aviv, IsraelfDepartment of Psychology, University of Haifa, 3498838 Haifa, IsraelgDepartment of Neurosurgery, Tel Aviv Sourasky Medical Center, 64239 Tel Aviv, IsraelhMovement Disorders Unit, Neurological Institute, Tel-Aviv Sourasky Medical Center, 64239 Tel Aviv, IsraeliSackler Faculty of Medicine, Tel Aviv University, 69978 Tel Aviv, IsraeljMusicology Department, Hebrew University of Jerusalem, 9190501 Jerusalem, IsraelkDepartment of Cognitive Neuroscience, Maastricht University, 6211 LK Maastricht, The Netherlands
Checking for direct PDF access through Ovid
Major methodological advancements have been recently made in the field of neural decoding, which is concerned with the reconstruction of mental content from neuroimaging measures. However, in the absence of a large-scale examination of the validity of the decoding models across subjects and content, the extent to which these models can be generalized is not clear. This study addresses the challenge of producing generalizable decoding models, which allow the reconstruction of perceived audiovisual features from human magnetic resonance imaging (fMRI) data without prior training of the algorithm on the decoded content. We applied an adapted version of kernel ridge regression combined with temporal optimization on data acquired during film viewing (234 runs) to generate standardized brain models for sound loudness, speech presence, perceived motion, face-to-frame ratio, lightness, and color brightness. The prediction accuracies were tested on data collected from different subjects watching other movies mainly in another scanner.Substantial and significant (QFDR<0.05) correlations between the reconstructed and the original descriptors were found for the first three features (loudness, speech, and motion) in all of the 9 test movies (Symbol=0.62, Symbol = 0.60, Symbol = 0.60, respectively) with high reproducibility of the predictors across subjects. The face ratio model produced significant correlations in 7 out of 8 movies (Symbol=0.56). The lightness and brightness models did not show robustness (Symbol=0.23, Symbol = 0). Further analysis of additional data (95 runs) indicated that loudness reconstruction veridicality can consistently reveal relevant group differences in musical experience.The findings point to the validity and generalizability of our loudness, speech, motion, and face ratio models for complex cinematic stimuli (as well as for music in the case of loudness). While future research should further validate these models using controlled stimuli and explore the feasibility of extracting more complex models via this method, the reliability of our results indicates the potential usefulness of the approach and the resulting models in basic scientific and diagnostic contexts.