Rico Stecher¹, Daniel Kaiser¹; ¹Justus-Liebig-University Gießen

Recent research suggests that when we generate a mental image, our brain employs feedback-related alpha rhythms to reactivate visual content representations. However, what visual features are encoded in these rhythms has only scarcely been examined. To answer this question, we acquired a large EEG data set through extensive sampling of individual participants. Each participant imagined 16 natural scenes according to short text prompts for a total of 10 sessions (i.e. 4,320 trials per participant, 44,320 trials in total). In order to characterize the visual feature information alpha rhythms carry during imagery, we approximated participants’ mental images using generative AI. We fed the imagined scene prompts into a latent text-to-image diffusion model and generated a large set of candidate images for each prompt (100 images per prompt, 1,600 in total). We then evaluated the AI-generated candidate images with a convolutional neural network (CNN) trained on scene categorization. Finally, we generated representational dissimilarity matrices (RDMs) from both the CNN activations and the EEG responses in the alpha band and compared the CNN RDMs to the EEG RDMs to determine how CNN features extracted across the CNN hierarchy predict the neural representation of mental images. Across individual participants, we found remarkable variations in how features of different complexity (coded at different depths of the CNN) contribute to the representations of mental images. On the group level, we found that the correspondence between participants’ alpha-band representations was most pronounced in intermediate CNN layers, suggesting that mental images of scenes are prominently shaped by mid-level visual features. Taken together, our results show that, when we conjure complex natural environments before our minds’ eye, alpha rhythms reactivate a complex range of visual feature information.

Acknowledgements: DK is supported by the DFG (SFB/TRR135,222641018; KA4683/5-1, 518483074, KA4683/6-1, 536053998), “The Adaptive Mind”, and an ERC Starting Grant (PEP, ERC-2022-STG 101076057).