Inferring Shape from Shading Flow: light source(s) as an emergent property
23.536, Saturday, May 11, 8:30 am - 12:30 pm, Vista Ballroom
Benjamin Kunsberg1, Steven Zucker2; 1Applied Mathematics Program, Yale University, 2Department of Computer Science, Yale University
The inference of shape from shading information depends on the light field, which is initially unknown, sometimes impossible to estimate from the image, and inconveniently inconsistent with priors on its position. We address this problem directly by analytically solving for all surface and light source pairs for every image patch, and then use boundary information to select the correct pair at each patch. Although the work is mathematical, it is motivated by neurobiological constructs. Just as columns of orientationally-selective neurons indicate all possible "edge" directions at a point, we derive an analogous column for surface patches. Curvature-based conditions indicate how edges fit together; now surface properties indicate how the patches fit together. Finally, surface curvature information from the boundary "selects" a consistent global surface across all patches, and the light source position(s) emerge naturally. Technically, covariant derivatives of the shading flow "cancel out" the local light source directions in our irradiance equations, and calculations equate the shape operator, dN, uniquely to image properties. Our only assumptions are Lambertian shading, locally constant albedo, and smooth surfaces. dN can be interpreted in terms of curvature selective cells in V4. Solving these equations leads to an infinite surface solution set (a local fiber analogous to an orientation column) parametrized by surface gradient. Each member of the fiber cor- responds to a different local surface/light source configuration. The global problem involves picking a single surface from each fiber, which starts with normal curvature information from the boundary (as viewed from the image) and propagates inwards. Tests confirm proof-of- concept. Summary: shape from shading flows is an intermediate-level visual inference analogous to curve inference. Light sources are an emergent property of finding global solutions, and hence need not be represented explicitly in the early visual system.