Ian Charest¹, Peter Brotherwood¹, Mathias Salvas-Hebert¹, Kendrick Kay², Frédéric Gosselin¹; ¹Département de psychologie, Université de Montréal, Montréal, Québec, Canada, ²Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, MN, USA

Understanding spatio-temporal dynamics of neural signals at the scale of the whole brain is essential for understanding brain computation. We present a novel method in which large-scale electroencephalography (EEG) and ultra-high field functional magnetic resonance imaging (fMRI) datasets are combined, yielding a full spatio-temporal trajectory of visual processing in the human brain. We leverage data from the Natural Scenes Dataset (NSD; Allen et al., 2022), which includes single-trial 7T fMRI responses, and the NSD-EEG companion dataset (Brotherwood et al.; 2024), recorded with high-density (164-channel Biosemi) electrodes during viewing of the same 73000 natural scene images (albeit by different participants). EEG channel amplitudes were used as predictors in a cross-validated fractional ridge regression model (Rokem and Kay; 2020) to predict voxelwise fMRI activity. This process is carried out time point by time point, revealing temporal information contained within each fMRI voxel. Our results reveal a distinct spatio-temporal pattern: significant prediction first emerges in the early visual cortex, consistent with initial feedforward visual processing. Over time, predicted activity patterns spread ventrally, dorsally, and peaking in prediction accuracy in parietal regions (Pearson correlation = 0.46), stabilizing roughly 300 ms after stimulus onset and persisting until 750 ms. This data-driven, voxel-level “EEG-to-fMRI” mapping effectively performs an empirical source reconstruction, linking channel-level EEG measurements to the underlying cortical generators. The observed pattern of activation, and its timing, fits well with the known biophysics of EEG—where the measured signals are spatiotemporal mixtures of cortical sources diffused through tissue. This method, when applied to sufficient amounts of high-quality data, provides a promising new avenue for understanding the dynamic neural representations with unprecedented spatio-temporal precision.