Diana C Dima^1,2, Jody C Culham¹, Yalda Mohsenzadeh^1,2; ¹Western University, ²Vector Institute for Artificial Intelligence

In everyday life, we rapidly recognize and react to many actions performed by others despite variations in the setting, agents, and objects involved. How are everyday actions processed in the ventral “vision-for-perception” and dorsal “vision-for-action” streams? Human neuroimaging research has found representations of perceptual features and action goals in the lateral occipito-temporal cortex (LOTC), while primate research suggests that parietal areas encode action classes with shared motor goals. Here, we collected fMRI data from 20 participants while they viewed two-second videos of everyday actions and read sentences describing the same actions. Representational similarity analysis on neural patterns revealed LOTC representations of effectors (e.g., leg or arm), action class (e.g., locomotion or manipulation), and action target (e.g., an object, the self, or another person). In contrast, univariate analyses on activation levels revealed differences among action classes in parietal cortex, especially the superior parietal lobule (SPL). Cross-decoding of multivariate representations across videos and sentences revealed that only LOTC, but not parietal cortex, encoded modality-invariant responses to actions. Together, our results reveal different representations of natural actions in the ventral and dorsal streams.

Acknowledgements: This work was supported by BrainsCAN at Western University through the Canada First Research Excellence Fund, a Western Interdisciplinary Development Initiatives Grant, a Vector Institute for Artificial Intelligence Research Grant, and a Western Postdoctoral Fellowship.