Zitong Lu¹ (), Julie D. Golomb¹; ¹Department of Psychology, The Ohio State University

Although visual input is initially recorded in 2D on our retinas, we live in a 3D world, and our visual systems must integrate 2D representations with various depth cues to achieve 3D perception. However, it remains unclear how exactly the brain integrates 2D and depth information into 3D representations during visual processing in time and space. In this study, we collected fMRI and EEG data from participants viewing 3D stimuli with red-green anaglyph glasses. Participants first completed a behavioral session involving depth judgment tasks and a 3D cube adjustment to quantify and equate across individual differences in depth perception. They then participated in two EEG sessions and two fMRI sessions in which they viewed peripheral stimuli presented across 64 possible 3D locations (4×4×4 grid centered on fixation). With a large number of trials (>7,000 trials per participant) and by integrating EEG, fMRI, and computational methods via representational similarity analysis, we address several previously unexplored questions. We first successfully tracked the spatiotemporal dynamics of neural representations of object spatial information (horizontal, vertical, depth, radius, polar angle, etc.) in various coordinate systems (Cartesian, Cylindrical Polar, and Spherical Polar coordinates), finding that neural representations reflect different types of spatial information across different brain areas and points in time. Second, we revealed an overall preference for representations consistent with a 2D polar coordiante system and a 3D cylindrical coordinate system, though there was evidence for Cartesian preference in some brain areas. Finally, we asked whether there is evidence of 2D and/or 3D spatial integration (integrated representations of space). Partial correlations revealed unique representations of 2D and 3D integrated representations (Euclidean distance), beyond the individual spatial components. These findings provide valuable insights into the neural mechanisms underlying our ability to perceive the world in 3D.

Acknowledgements: NIH R01-EY025648 (JG), NSF 1848939 (JG)