Yuanfang Zhao¹ (), Emalie McMahon, Leyla Isik; ¹Johns Hopkins University

Dynamic social vision has been proposed to rely on the lateral visual cortex and superior temporal sulcus (STS), collectively referred to as the “third visual pathway.” However, disentangling distinct neural responses within this pathway has been challenging with hypothesis-driven approaches. To address this, we employed a data-driven voxel decomposition technique (i.e., non-negative matrix factorization) to decompose distinct neural responses in the third pathway in dynamic social vision. Using a large-scale fMRI dataset of 200 three-second video clips depicting two individuals in everyday activities, our analysis identified two components with distinct functional profiles shared across participants. One component, predominantly weighted in the extrastriate body area (EBA) of the lateral visual cortex, responds strongly to videos depicting physical interactions between people, while the other, heavily weighted in the STS, responds strongly to videos depicting communicative interactions. To ensure the robustness of these findings, we replicated the analysis with the BOLD Moments Dataset, analyzing neural responses to 601three-second video clips with one or more people engaging in a broader range of everyday activities. This replication not only confirmed the two components with similar functional profiles, but also revealed a third component, which responds strongly to videos depicting isolated single body movements. This third component is also weighted heavily in the EBA but has a distinct spatial distribution from the component representing interpersonal physical interactions, suggesting a distinction between single and multi-person physical actions. Importantly, none of the components can be explained by motion energy, suggesting that these responses are not driven simply by motion in the videos. Together, our findings suggest that EBA and STS within the third pathway differentially respond to body movement, physical interactions, and communicative interactions, offering new insights into the neural mechanisms underlying social interaction perception in the third visual pathway.

Acknowledgements: This work is supported by R01 grant: NIH R01MH132826