Yu Hu^1,2, Yalda Mohsenzadeh^1,2; ¹Western University, ²Vector Institute for Artificial Intelligence

When we hear the sound of an object, it may trigger associated visual memory and imagery. This process involves perceptual, memory retrieval, and imagery activities. Based on previous research, this process may recruit sensory- and memory-related brain areas. However, the underlying spatiotemporal dynamics remain poorly investigated. Here, we used naturalistic videos of audiovisual events and recorded fMRI data during the tasks in which human participants (n=16) recalled visual contents when hearing associated sounds and recalled sounds when watching silent videos, after they well-memorized the video contents. With time-resolved fMRI multivariate pattern analyses, we observed reverse spatiotemporal hierarchy during the sound-induced visual memory retrieval: the neural activity started in auditory areas, then high-level visual areas, and finally reached the primary visual cortex. A similar trend was found during visual-induced auditory memory retrieval, where the activity of the high-level superior temporal gyrus was earlier than mid-level planum temporale. However, the primary auditory area was not involved, suggesting modality differences in the role of primary sensory areas in corresponding memory recall. We also observed the activity of the hippocampus, the parahippocampal cortex, the retrosplenial cortex, and the precuneus, which were involved in similar timing of imagery-related sensory areas. Overall, our study provided both spatial and temporal accounts of neural activity during the cross-modal memory retrieval and imagery.