Ning Tang¹ (), Enjie Xu², Jifan Zhou², Mowei Shen², Tao Gao^3,4,5; ¹Department of Psychology, Soochow University, ²Department of Psychology and Behavioral Sciences, Zhejiang University, ³Department of Communication, UCLA, ⁴Department of Psychology, UCLA, ⁵Department of Statistics, UCLA

The Heider-Simmel display illustrates how we transition from low-level motion of geometric shapes to high-level social storytelling, engaging both perception and cognition. While it is accepted that perception serves as the visual root for social inference, it remains unclear which aspects of Heider-Simmel experience are attributed solely to perception. In this study, we propose a new paradigm to investigate the online perceptual component of this experience, inspired by research on time perception and working memory. Studies show that our experience of "the present" is not infinitesimally small, but rather a temporal window of about 3 seconds, which corresponds to the duration of working memory. This window also aligns with the time for generating and understanding intentional actions, such as waving to a friend. We hypothesize that perception and cognition can be dissociated by slowing down the display to extend critical motion beyond the 3-second window. Disrupting this window would likely impair online perception, while offline cognition would be more immune to it. In two experiments, we adapted two types of visual displays: one resembling the Heider-Simmel display, described by action verbs like 'stalk' and 'argue,' heavily relying on perceptual processing; the other using a food truck task, where participants engaged in a sophisticated inference about the agent’s desire preference among three trucks, requiring more cognitive reasoning. In both experiments, participants focused on a visual search task while viewing display clips at normal speed or 0.5x speed. Afterward, they were asked a surprise question to interpret the social interaction or preference. Results showed a drop in accuracy in the action verb task at slower speeds, but no difference in the desire preference task. Our study demonstrates that agency understanding is more closely tied to perception when captured through verb semantics, while inferring desire preference may rely more on domain-general cognitive inference.

Acknowledgements: This work was performed with the support of grants from Natural Science Foundation of Jiangsu Province (BK20240791 to Ning Tang)