Herrick Fung¹ (), N Apurva Ratan Murty¹, Dobromir Rahnev¹; ¹School of Psychology, Georgia Institute of Technology

Humans vary greatly in many aspects of behavior. Deep neural network models are often seen as promising models of human perceptual decisions but are typically assumed to lack such individual differences. Recent work has revealed substantial differences in the representational geometries among model instances differing only in their random weight initializations (Mehrer et al., 2020). However, whether these representational differences translate into individual differences in behavior remains unknown. Here we examined three datasets of human perceptual decisions that involve tasks of varying complexity: 16-choice object categorization (N=36), 8-choice digit recognition (N=60), and 2-choice Gabor orientation discrimination (N=20). For each task, we trained deep neural networks of several different architectures. For each architecture, we randomly initialized many network instances that matched the number of human subjects and trained them in identical fashions to reach similar overall accuracy on the validation dataset. We then tested the performance of each model instance on new stimuli on which the model was never trained. We found substantial individual differences in accuracy, confidence, response bias, and reaction time across model instances. Similar to humans, some instances performed reliably better or had consistent response biases compared to other instances. To quantify these individual differences, we computed pairwise image-by-image correlations among model instances across all behavioral metrics. We found that the average pairwise correlations between model instances of different behavioral metrics ranged from r = 0.203 to r = 0.836, demonstrating large individual differences in the pattern of image-by-image responses. Our findings establish the existence of robust individual behavioral differences among different model instances, highlighting the importance of analyzing multiple network instances to draw reliable inferences about network behavior. These results also open a promising avenue for using individual differences in neural networks as a framework to model human individual differences.

Acknowledgements: This work was supported by the National Institute of Health (award: R01MH119189) and the Office of Naval Research (award: N00014-20-1-2622).