Sholei Croom¹, Chaz Firestone¹; ¹Johns Hopkins University

Ordinary observers can infer the goals of others’ actions, such as where someone is walking or which object they are reaching for. However, actions are more than their goals; underlying visually-guided behavior are complex dynamics between an agent’s body and the environment. What do observers know about such dynamics and the behaviors that emerge from them? Here, we explore this question through “pantomimed actions”, in which people perform actions with imaginary objects. Pantomimed actions differ kinematically from genuine object-directed actions because the absence of visual information disrupts the typical perception-action feedback cycle. If observers can distinguish real actions from pantomimed actions, this would reveal finer-grained intuitions about the dynamics underlying visually-guided action. We created a set of videos in which agents performed object-directed actions involving no physical contact with the objects (e.g., stepping over a box, ducking under an overhang, or weaving between poles). In half of the videos, actors interacted with real boxes, real overhangs, etc.; in the other half, they were instructed to move *as if* interacting with these objects (in both cases, a censor box occluded where the object was or would have been). Then, independent subjects watched these videos and had to determine which was which; which videos showed real actions and which showed pantomimes? Collapsing across all actions, observers discriminated real actions from pantomimes at rates above chance. However, certain actions were more easily discriminable than others; for some actions, actors successfully ‘fooled’ observers into thinking an object was present. Our work supports two conclusions: (1) Observers are sensitive to kinematic differences distinguishing genuine visually-guided actions from their pantomimed counterparts (revealing surprisingly fine-grained intuitions about visuomotor processing); (2) The ability to “fake” actions may be more robust than previously suggested (e.g., findings that actors cannot make a light box seem heavy).

Acknowledgements: NSF BCS #2021053