Yikai Tang^1,2, William Cunningham^1,2, Dirk Bernhardt-Walther; ¹University of Toronto, ²Vector Institute

Ambiguity, the state where competing perceptual interpretations or behavioral affordances coexist (Hirsh & Mar, 2011), has been shown to be aversive in various contexts. While extensively studied in decision-making, researchers suggest that aversion to ambiguity is a broader principle spanning perception and cognition. If this principle extends to visual preferences, individuals may favor images with clear, dominant interpretations and find ambiguous ones less appealing. Here, we focus on categorization as a fundamental form of interpretation. To test this hypothesis, we analyzed model-derived ambiguity estimates for natural object and scene images from the BOLD5000 dataset. For object images (N = 1,915), ambiguity was operationalized using classification models (e.g., VGG16, VGG19, AlexNet) by calculating the maximal confidence of their SoftMax outputs. Correlating these values with pre-collected pleasure ratings, we found that higher ambiguity was consistently associated with lower pleasure ratings across all models. We extended this analysis to scene images (N = 3,000), which often feature multiple objects with varying interpretability. Using the You Only Look Once (YOLO) model, we computed the average classification confidence across detected objects within each scene. Consistent with the findings for object images, higher average ambiguity in scene images correlated with lower pleasure ratings, suggesting that aversion to ambiguity influences preferences for more complex visual stimuli as well. Finally, we investigated whether the negative effect of ambiguity stems from its inherent aversiveness or the cognitive effort required to resolve it. Using BOLD signal changes in ventral visual areas (also from the BOLD5000 dataset) as a measure of processing costs, we found that these costs partially mediated ambiguity’s negative effect on pleasure. This may suggest that aversion to ambiguity reflects both the aversiveness of ambiguity itself and the effort involved in resolving it. Overall, our study provides valuable insights into affective responses to categorization ambiguity in real-world images.

Acknowledgements: This work was supported by an NSERC Discovery Grant (RGPIN-2018-05946) to WC and an NSERC Discovery Grant (RGPIN-2020-04097) as well as a SSHRC Insight Grant (435-2023-0015) to DBW.