Diana Kollenda^1,2 (), Sophie-Luise Schwenger¹, Sophie Halstenberg¹, Benjamin de Haas^1,2; ¹Experimental Psychology, Justus Liebig University, Giessen, Germany, ²Center for Mind, Brain and Behavior (CMBB), Universities of Marburg, Giessen and Darmstadt, Germany

Some images are inherently more memorable than others, a quality that can be predicted with ResMem, a residual neural network sensitive to semantic image features. Large-scale human data further indicate that semantic features strongly contribute to the memorability of object images. This raises a compelling question: Is image memorability tied to visual features, or at least in part modality-independent? Here, we tested whether the memorability of an image generalizes to its verbal description. We prepared descriptions of 160 scenes from the OSIE dataset using two approaches: a) a standardized protocol used by one of the experimenters, and b) a multimodal transformer model (ChatGPT-4). We then assessed the memorability of these descriptions in independent groups of participants (n = 81 each). During the encoding phase, participants were tasked to categorize 80 descriptions as depicting either outdoor or indoor scenes. In a subsequent surprise memory task, they were presented with the same 80 descriptions, intermingled with 80 lures, and asked to indicate for each description whether they had seen it during the preceding encoding phase or not. Half of the targets and lures were descriptions of images with low ResMem scores (0.64 ± 0.04) and half descriptions of images with high scores (0.91 ± 0.02). Recognition sensitivity (d’) was much higher for descriptions of high-memorability images, regardless of whether the descriptions were prepared by an experimenter (t = 10.8, p <.001, d = 1.2), or Chat-GPT (t = 8.67, p <.001, d = 0.96). Replicating the experiment with images instead of descriptions showed a comparable memorability effect (t = 11.3, p <.001, d = 1.25). The predictiveness of ResMem scores for description memorability held up when controlling for a range of image features. These results provide clear evidence that image-computable memorability is at least in part modality-independent.

Acknowledgements: This work was supported by ERC Starting Grant 852885 INDIVISUAL; BdH was further supported by DFG Project Nos. 222641018-SFB/TRR 135 TP C9 as well as "The Adaptive Mind", funded by the Excellence Program of the Hessian Ministry of Higher Education, Science, Research and Art.