Time/Room: Friday, May 10, 3:30 – 5:30 pm, Royal 4-5
Organizer: Jeremy Freeman, New York University; Elisha P. Merriam, Departments of Psychology and Neural Science, New York University; and Talia Konkle, Department of Psychology, Harvard University
Presenters: Elisha P. Merriam, Seong-Gi Kim, Adam Kohn, Talia Konkle, Kalanit Grill-Spector, J. Swaroop Guntupalli
Symposium Description
With the advent of functional neuroimaging (fMRI), hemodynamic responses can be measured over the whole brain with relatively high spatial precision. A formidable challenge, however, is that fMRI responses appear to reflect a multitude of signals, both neural and non-neural, at multiple spatial scales. At times, evidence for multiple scales appears to border on contradiction, e.g. suggesting that the same stimulus dimension is encoded both through both coarse-scale topographic organization and fine-scale columnar structure. A diversity of analyses and methods have highlighted a set of key questions: First, over what expanse of cortex is information meaningfully represented – to what degree is cortical function clustered, if there is clustering at all? Second, if information can be decoded from patterns of fMRI response, what should we infer about the corresponding brain region – does it imply fine-scale structure, or reflect coarse-scale topography? Finally, and crucially, how does the spatial scale of fMRI selectivity reflect the tuning of the underlying neurophysiological signals – does it reflect spikes, field potentials, hemodynamic events, or a complex combination of all three? In this symposium, we have brought together six investigators who are pushing the cutting edge both analytically and methodologically to characterize the spatial structure of visual representation in cortex. Four investigators apply novel multivariate analytical approaches to fMRI data from humans, studying the representation of both low-level features (orientation) and complex objects; our fifth investigator studies the biophysics of the fMRI response using high-resolution fMRI in conjunction with other imaging modalities; and our final investigator records electrophysiological signals from macaques using multi-electrode arrays, a technique that may shed important light on what the fMRI signal reflects. Each talk will introduce the analysis method and experimental approach, and what it has revealed about neural organization, with a critical examination of (i) what is assumed and tested about the spatial scale of the neural response; (ii) what the analysis method can and cannot reveal about the nature of the underlying representations; (iii) what the analysis implies about other, complementary measurements of brain activity. Lively moderated discussions will emphasize points of agreement – or disagreement – across the different approaches. Combined, these talks demonstrate the richness of representational structure at multiple spatial scales across the cortex, and highlight the inferential strengths and weaknesses of current analyses, and the benefits of integrating information across multiple experimental techniques. This symposium should attract all investigators and students studying vision using fMRI and decoding (alongside associated behavioral measures), which is a rapidly growing contingent of the VSS community. The particular controversies about decoding and spatial scale that we plan to address have attracted large audiences at recent VSS meetings. Furthermore, due to its inter-disciplinary nature, the symposium is likely to attract investigators and students using a range of experimental techniques, including fMRI and electrophysiology, and motivate them to find new ways to combine these techniques through collaboration.
Presentations
Orientation decoding in humans – evidence for a columnar contribution?
Speaker: Elisha P. Merriam, Department of Psychology and Neural Science, New York University
Authors: Jeremy Freeman, Departments of Psychology and Neural Science, New York University; David J. Heeger, Departments of Psychology and Neural Science, New York University
The representation of orientation in primary visual cortex (V1) has been examined extensively at a fine spatial scale corresponding to the columnar architecture. In humans, orientation can be decoded from functional magnetic resonance imaging (fMRI) signals using multivariate classification methods, but it is unknown whether orientation decoding depends on the fine-scale, columnar architecture in cortex. We have shown that orientation is also represented in human cortex at a coarse spatial scale, and we have argued that this organization provides the basis for orientation decoding (Freeman et al., 2011). This topic remains highly controversial, and several labs have provided new evidence suggesting that a columnar-scale signal is present in fMRI measurements at conventional resolution. In this talk, I will review the evidence for and against a columnar contribution to orientation decoding. I will present recent evidence from our lab in which we measure fMRI responses in V1 to a variety of spatially structured stimuli – including oriented gratings with different spatial configurations and logarithmic spirals – and apply decoding and spatial filtering analyses to the data. Together, the results of our analyses strongly suggest that orientation decoding does not reflect the irregular spatial arrangements of orientation columns. Rather, it is likely that the coarse-scale topographic map of orientation in V1 is the major, if not the only, source of information that is exploited by multivariate decoding methods.
Underlying sources for decoding of oriented gratings in fMRI
Speaker: Seong-Gi Kim, Department of Neurobiology, University of Pittsburgh
Authors: Amir Shmuel, Department of Neurobiology, McGill University
Multivariate machine learning algorithms were applied to BOLD fMRI data obtained from human subjects for decoding the orientation of gratings, with voxels larger than the width of orientation columns. Contributions to this successful decoding using low-resolution BOLD fMRI can potentially be made by 1) functionally selective large blood vessels, 2) orientation bias in large-scale organization, and 3) local orientation irregularities. In order to examine this issue, we re-analyzed cerebral blood volume-weighted fMRI data from cat visual cortex (Fukuda et al., J. of Neurosci, 2006). To remove large vessel contributions, ferrous iron oxide contrast agent was injected into blood. The functional data were obtained with 0.156 x 0.156 x 1 mm3. Then, high-resolution data were down-sampled to low-resolution up to 3 mm (the average orientation cycle is 1.0-1.4 mm). Linear support vector machine analysis showed that the presented orientation can be predicted above the chance level, even at 3-mm voxel resolution. To separate contributions from local orientation irregularities and from large-scale organizations, data were band-pass filtered with center frequency of 0.4 cycles/mm (frequency range of local irregularities) and 0.1 cycles/mm (low frequency). In both conditions, the presented orientation can be predicted above the chance level, with slightly better accuracy for the spatial filter of higher frequencies. Our analysis indicates that 1) large vessel contribution is not essential, and 2) local orientation irregularities can contribute for decoding of orientations in low-resolution fMRI data.
The relationship between the local field potential and spiking activity in primary visual cortex
Speaker: Adam Kohn, Albert Einstein College of Medicine
Authors: Xiaoxuan Jia, Albert Einstein College of Medicine
The local field potential (LFP) represents the summed electrical activity in a local region of cortex. It provides a mesoscopic view of network activity and function, between local measures such as single unit spiking activity and more global measures such as BOLD-fMRI and EEG. However, the relationship between the LFP and these signals remains unclear, making it difficult to relate findings across scales of study. We therefore investigated how the LFP is related to spiking activity in primary visual cortex of macaque monkeys, and found a flexible relationship for the gamma frequency components of the LFP. Small sinusoidal gratings, and those masked with noise, induce gamma power that is tuned similarly to spiking activity. Large gratings induce a ‘global’ gamma rhythm characterized by a distinctive spectral bump. This signal is well tuned for orientation and spatial and temporal frequency, but with a preference that is similar across millimeters of cortex. The preference of this gamma is sensitive to adaptation and the location of a stimulus in visual space. We argue that these properties indicate the global gamma rhythm reflects and magnifies an underlying bias in the neuronal representation of visual stimuli in V1. Our results show that there is not a single, fixed neuronal ensemble contributing to gamma and that the global gamma rhythm may be a useful signal for detecting and characterizing biased representations in visual cortex.
Macro-organization of object responses in occipito-temporal cortex
Speaker: Talia Konkle, Department of Psychology, Harvard University
Authors: Alfonso Caramazza, Department of Psychology, Harvard University
What are the dimensions that organize object representation? A common assumption is that the mosaic of category-selective areas are the only large-scale clusters, while the remaining object responses have more heterogenous response profiles with structure primarily at a finer spatial scale. In contrast, I will present results showing a large-scale of object responses spanning the entire ventral and lateral occipito-temporal cortex, based on the dimensions of animacy and size. Zones with systematic animacy-size preferences are arranged in a spoked organization emanating from the occipital pole along a single ventral-medial-to-lateral-to-dorsal-medial axis, bearing marked similarity to the organization of early visual areas. Regions selective for faces, bodies, and scenes fit within these zones, demonstrating consistent meso-scale structure. These results suggest that object cortex, just like early visual cortex, has structure that can be explained at multiple spatial scales. I will argue that understanding this multi-scale representation is valuable for inferring the nature of the underlying cognitive architecture.
High-resolution fMRI reveals cortical tiling of face and limb selectivity in human high-level visual cortex
Speaker: Kalanit Grill-Spector, Department of Psychology and Neuroscience Institute, Stanford University
Authors: Kevin Weiner, Department of Psychology, Stanford University
Functional magnetic resonance imaging (fMRI) studies identify areas responding selectively to images of faces and body parts compared to a variety of control objects throughout ventral temporal and lateral occipitotemporal cortices (VTC and LOTC, respectively). Previous research indicates that the location of each region is variable relative to both gross anatomical landmarks, as well as to other high-level visual areas. Using higher-resolution fMRI scanning methods, we conducted a series of experiments revealing that the fine-scale spatial organization of face and limb selectivity is much more consistent than once thought. These experiments reveal a topographic organization of face- and limb-selective regions extending from LOTC to VTC where each high-level region is defined by a combination of anatomical and functional boundaries separating them from neighboring regions just millimeters away. We propose a multi-factor organization framework resulting from these empirical measurements where any region in human high-level visual cortex can be defined using the following criteria: 1) precise anatomical location, 2) preserved spatial relationship among functional regions, 3) preserved relationship relative to known visual field maps, and 4) reliable functional differences among regions. Methodologically, we demonstrate how these organizational features allow consistent parcellation of cortical regions across subjects. Theoretically, we refer to this inter-related structure of multiple maps as cortical tiling and hypothesize that tiling is a universal organizational strategy of the brain. Finally, we discuss computational benefits of this organization serving to accommodate multidimensional information in a concentrated neural territory to increase the repertoire, flexibility, and efficiency of visual processing.
Exploring the scale of common dimensions of information coding in ventral temporal cortex
Speaker: J. Swaroop Guntupalli, Department of Psychological and Brain Sciences, Dartmouth College
Authors: Andrew C. Connolly, Department of Psychological and Brain Sciences, Dartmouth College; James V. Haxby, Department of Psychological and Brain Sciences, Dartmouth College, Center for Mind/Brain Sciences, Universita degli studi di Trento
Scale of representation can refer to either categorical (super-ordinate vs sub-ordinate) or spatial (coarse-scale topographies, fine-scale topographies). Typically, we assume that they map one-to-one – coarse-scale categorical distinctions (animate vs inanimate) are reflected in coarse-scale topographies (medial vs lateral ventral temporal cortex (VT)). This is reflected in the fact that we can use a common decoding model to classify super-ordinate categories (houses vs faces) across-subjects, but fine-scale categorical distinctions (human faces vs animal faces) require individually tailored decoding models. We proposed a method that aligns representations, even at fine-scale, across subjects into common dimensions of encoding in VT. We showed that about 35 orthogonal dimensions are required to decode movie scenes, faces and objects, and 6 animal species from VT. This suggests that category decoding models reveal at least more than two dozen categorical dimensions in VT. Now the question remains: are these common categorical dimensions represented in large-scale cortical topographies? Decoding super-ordinate & sub-ordinate categorical information from a localized cortical patch after removing the low-frequency information can elucidate the spatial scale of representation of both coarse & fine- scale categorical information. We use PCA or MDS to identify common coarse-scale categorical dimensions and decode fine-scale categorical information both on those coarse dimensions and from the space orthogonal to those dimensions across different spatial frequency bands. This can elucidate the scale of representation of fine-scale information both categorically & spatially. We present results of both these analyses in VT on our studies using movies, faces and objects, and animal species.