Member-Initiated Symposia
2008 Symposia
Perceptual expectations and the neural processing of complex images
Cortical organization
and dynamics for visual perception and beyond
Crowding
Visual Memory and the Brain
Bayesian models applied to perceptual
behavior
Action for perception: functional
significance of eye movements for vision
The past,
present, and future of the written word
Surface material perception
Symposia from Past Meetings |
Surface material perception
Friday, May 9, 2008, 3:30 - 5:30 pm
Royal Palm 6-8
Organizer:
Roland W Fleming (Max Planck Institute for
Biological Cybernetics, Tübingen, Germany)
Presenters:
Roland W. Fleming (Max Planck Institute
for Biological Cybernetics, Tübingen, Germany),
Melvyn A. Goodale (The
University of Western Ontario), Isamu
Motoyoshi (NTT Communication Science
Laboratories), Daniel Kersten (University of Minnesota),
Laurence T Maloney (New York University),
Edward H Adelson (MIT)
Symposium Description
When we look at an everyday
object we gain information about its location and shape and also about the
material it is made of. The apparent color of an orange signals whether it is
ripe; its apparent gloss and mesoscale texture inform us whether it is fresh.
All of these judgments are visual judgments about the physical chemistry of
surfaces, their material properties. In the past few years, researchers have
begun to study the visual assessment of surface material properties, notably
gloss and mesoscale texture (“roughness”). Their research has been facilitated
by advances in computer graphics, statistical methodology, and experimental
methods and also by a growing realization that the visual system is best studied
using stimuli that approximate the environment we live in. This symposium
concerns recent research in material perception presented by six researchers in
computer science, neuroscience and visual perception.
The successive mappings from
surface property to retinal image to neural state to material judgments are
evidently complex. Coming to understand how each step leads to the next is a
fascinating series of challenges that crosses disciplines. An initial challenge
is to work out how changes in surface material properties are mirrored in
changes in retinal information, to identify the cues that could potentially
signal a surface material property such as gloss or roughness.
A second challenge is to
determine which cues are actually used by the visual system in assessing
material properties. Of particular interest are recent claims that very simple
image statistics contain considerable information relevant to assessing surface
material properties. A third challenge concerns the neural encoding of surface
properties and what we can learn from neuroimaging, a fourth, how variations in
one surface material property affect perception of a second.
A final – and fundamental --
challenge is to work out how the organism learns to use visual estimates of
material properties to guide everyday actions -- to decide which oranges to eat
and which to avoid.
The symposium is likely to
be of interest to a very wide range or researchers in computer vision, visual
neuroscience and visual perception, especially perception of color. lightness
and texture.
Abstracts
Perception of materials that transmit light
Roland W. Fleming, Max
Planck Institute for Biological Cybernetics, Tübingen, Germany
Many materials that we
commonly encounter, such as ice, marmalade and wax, transmit some proportion of
incident light. Broadly, these can
be separated into transparent and translucent materials.
Transparent materials (e.g. gemstones, water) are dominated by specular
reflection and refraction, leading to a characteristic glistening, pellucid
appearance. Translucent materials
(e.g. marble, cheese) exhibit sub-surface light scattering, in which light
bleeds diffusely through the object creating a distinctive soft or glowing
appearance. Importantly, both types
of material are poorly approximated by Metelli’s episcotister or other models of
thin neutral density filters that have shaped our understanding of transparency
to date. I will present various
psychophysical and theoretical studies that we have performed using physically
based computer simulations of light transport through solid transmissive
objects. One important observations
is that these materials do not exhibit many image features traditionally thought
to be central to transparency perception (e.g. X-junctions).
However, they compensate with a host of novel cues, which I will
describe. I will discuss the
perceptual scales of refractive index and translucency and report systematic
failures of constancy across changes in illumination, 3D shape and context.
I will discuss conditions under which various low-level image statistics
succeed and fail to predict material appearance.
I will also discuss the difficulties posed by transmissive materials for
the estimation of 3D shape. Under
many conditions, human vision appears to use simple image heuristics rather than
correctly inverting the physics. I
will show how this can be exploited to create illusions of material appearance.
How we see stuff: fMRI and behavioural
studies of visual routes to the material properties of objects
Melvyn A. Goodale
Almost all studies of visual
object recognition have focused on the geometric structure of objects rather
than their material properties (as revealed by surface-based visual cues such as
colour and texture). But recognizing the material from which an object is made
can assist in its identification – and can also help specify the forces required
to pick up that object. In two
recent fMRI studies (Cant & Goodale, 2007; Cant et al., submitted), we
demonstrated that the processing of object form engages more lateral regions of
the ventral stream such as area LO whereas the processing of an object’s surface
properties engages more medial regions in the ventral stream, particularly areas
in the lingual, fusiform, and parahippocampal cortex. These neuroimaging data
are consistent with observations in neurological patients with visual form
agnosia (who can still perceive colour and visual texture) and patients with
cerebral achromatopsia (who can still perceive form).
The former often have lesions in area LO and the latter in more medial
ventral-stream areas. In a
behavioural study with healthy observers (Cant et al., in press), we showed that
participants were able to ignore form while making surface-property
classifications, and to ignore surface properties while making form
classifications – even though we could demonstrate mutual interference between
different form cues. Taken together, these findings suggest that the perception
of the material properties depends on medial occipito-temporal areas that are
anatomically and functionally distinct from more lateral occipital areas
involved in the perception of object shape.
Histogram skewness and glossiness perception
Isamu Motoyoshi
Human can effortlessly judge
the glossiness of natural surfaces with complex mesostructure. The visual system
may utilize simple statistics of the image to achieve this ability (Motoyoshi,
Sharan, Nishida & Adelson, 2007a;
Motoyoshi, Nishizawa & Uchikawa, 2007b). We have shown that the perceived
glossiness of various surfaces is highly correlated with the skewness (3rd-order
moment) of the luminance histogram, and that this image property can be easily
computed by the known early visual mechanisms. Our 'skewness aftereffect'
demonstrated the existence of such skewness detectors and their link to the
perceived glossiness. However,
simple skewness detectors are not very sensitive to image spatial structures.
They might not be able to distinguish a glossy surface from, say, a matte
surface covered with white dusts while humans can do. These unsolved issues and
questions will be discussed together with our latest psychophysical data. Our
glossiness study suggests that the perception of material properties may be
generally based on simple 'pictorial
cues' in the 2D image, rather than on complex inverse optics computations. This
hypothesis is supported by the finding that simple image manipulation techniques
can dramatically alter the apparent
surface qualities including translucency and metallicity (Motoyoshi, Nishida &
Adelson, 2005).
Object lightness and shininess
Daniel Kersten
Under everyday viewing
conditions, observers can determine material properties at a glance--such as
whether an object has light or dark pigmentation, or whether it is shiny or
matte. How do we do this? The first problem--lightness perception--has a long
history in perception research, yet many puzzles remain, such as the nature of
the neural mechanisms for representing and combining contextual information. The
second--"shininess"--has a shorter history, and seems to pose even stiffer
challenges to our understanding of how vision arrives at determinations of
material properties. I will describe results from two approaches to these two
problems. For the first problem, I will describe neuroimaging results showing
that cortical MR activity in retinotopic areas, including V1, is correlated with
context-dependent lightness variations, even when local luminance remains
constant. Further, responses to
these lightness variations, measured with a dynamic version of the Craik-O'Brien
illusion, are resistant to a distracting attentional task. For the second
problem, I will describe an analysis of natural constraints that determine human
perception of shininess given surface curvature, and given object motion. One
set of demonstrations show that apparent shininess is a function of how
statistical patterns of natural illumination interact with surface curvature. A
second set of demonstrations illustrates how the visual system is sensitive to
the way that specularities slide across a surface.
Multiple surface material properties,
multiple visual cues
Laurence T. Maloney
Previous research on visual
perception of surface material has typically focused on single material
properties and single visual cues, with no consideration of possible
interactions. I’ll first describe recent work in which we examined how multiple
visual cues contribute to visual perception of a single material property, the
roughness of 3D rendered surfaces, viewed binocularly. We found that the
visual system made substantial use of visual cues that were in fact useless in
estimating roughness under the conditions of our experiments. I’ll discuss what
the existence of pseudo-cues implies about surface material perception.
In a separate experiment, we used a conjoint measurement design to
determine how observers represent perceived 3D texture (“bumpiness”) and
specularity (“glossiness”) and modeled how each of these two surface material
properties affects perception of the other. Observers made judgments of
“bumpiness” and “glossiness” of surfaces that varied in both surface texture and
specularity. We found that a simple additive model captures visual perception of
texture and specularity and their interactions. We quantify how changes in each
surface material property affect judgments of the other. Conjoint measurement is
potentially a powerful tool for analyzing surface material perception in
realistic environments.
What is material perception good for?
Edward H. Adelson
What are the essential ways
in which vision helps us interface with the physical world? What is the special
role of material perception? One way to approach this question is: 1. Marry a
vision scientist. 2. Have children with her. 3. Take videos of your children
interacting with the world. 4. Study these videos, taking note of the essential
tasks children must master. 5. Make your colleagues watch these videos. For some
tasks (e.g., learning the alphabet or recognizing giraffes) material perception
is relatively unimportant, but for others
(e.g., eating, walking, getting dressed, playing outside, taking a bath) it is
critical. The mastery of materials -- the way they look, feel, and respond to
manipulation -- is one of the main tasks of childhood. Why, then, is so little
known about material perception, as compared to, say, object recognition? One of
the issues seems to be that material perception is embedded in procedural
knowledge (knowing how to do), whereas object recognition is embedded in
declarative knowledge (knowing how to describe). This suggests
that material perception should be approached from multiple modalities including
vision, touch, and motor control. It suggests that the brain might contain
mechanisms devoted to the joint visual/haptic analysis of stiffness,
slipperiness, roughness, and the like. In pursuit of this program, we have
recently been showing our home videos to colleagues in other fields.
|
