Touching Sounds: Examining the Impact of Non-Visual Exposure on the Development of Audio-Tactile Sound-Shape Correspondences

Poster Presentation: Saturday, May 17, 2025, 8:30 am – 12:30 pm, Pavilion
Session: Multisensory Processing: Visual-haptic and visual-vestibular integration

Shibo Cao1 (), Rong Tan1, Julia Kelly1, Cuong Nguyen1, Vivian M Ciaramitaro1; 1University of Massachusetts Boston

In one type of crossmodal correspondence, nonsense words, such as "bouba", are associated with rounded abstract shapes, and "kiki" with angular shapes. Such associations are found between auditory and visual objects (AV), and between auditory and tactile objects, touched but not seen (AT). Visual experience can influence AT associations: AT associations are weak in early-blind adults (Fryer et.al, 2014) and fully-sighted 6-8-year-olds, with naïve visual experience (Chow et.al, 2021), and can be enhanced if fully-sighted children see the visual shapes first, prior AV exposure. Here, we examine how the type and amount of prior non-visual exposure influences AT associations. Children (N=122, 6-8-year-olds) completed 4 or 8 trials of a tactile attention (TT), or visual imagery (TI) task. For TT exposure, children felt a round and spiky shape inside a box, hidden from view, and then felt a smaller shape and judged which larger shape best matched the smaller shape. For TI exposure, children felt either a round or spiky shape inside a box and judged if they imagined the shape best resembled a flower or star. After TT or TI exposure, children completed 16 AT test trials: they felt a round and spiky shape inside a box and judged which shape best matched a nonsense sound. No feedback was provided during exposure or test trials. We found that 8 trials of prior TT, but not TI, exposure enhanced AT associations. However, neither 4 trials of prior TT or TI exposure enhanced AT associations, complementing our related findings that 8, but not 4, trials of prior AV exposure enhanced AT associations. Our results suggest that the type and amount of prior exposure is important in helping children understand how to represent objects experienced only via touch when forming sound-shape correspondences and that direct visual exposure may be sufficient but not necessary.