Imaging Beyond Pixels: Future Forecasts

This blog post references my feature article 'Imaging Beyond Pixels: Enhancing The Elementary Picture' in EDN's March 15, 2007 edition.

Ironically, one of my primary motivations for writing this article didn't make the final print page count cut! It came in the form of a series of articles in the August 2006 issue of IEEE Computer Magazine, which provide a compelling glimpse into how future cameras' optics, sensors, flash illumination subystems and processors might combine to radically transform image capture processes. Instead of trying to describe them myself, I'll quote from the overview (PDF), written by Oliver Bimber of Bauhaus-University Wiemar:

In “Computational Cameras: Redefining the Image,” Shree K. Nayar describes different examples of computational cameras. Optical extensions, such as curved mirrors or spectral filters in combination with ingenious computer vision techniques enable wide-angle, high-dynamic-range, multispectral, and depth imaging. While programmable imaging allows emulating several specialized functionalities with a single imaging system, programmable illumination realizes smart flashes. Nayar provides examples of both.

Most digital cameras allow capturing small movie sequences. Instead of a simple playback, however, future cameras will register the corresponding video frames into a spacetime slab. This data structure, together with appropriate processing techniques, offers higher image quality—less noise, larger depth of field, higher dynamic range—and opens completely new possibilities, such as consistent group shooting, motion-invariant image stitching, or playback of motion loops. Michael F. Cohen and Richard Szeliski describe this technique in “The Moment Camera.”

Light that a point in space reflects or emits usually travels in all unoccluded directions. Nevertheless, conventional cameras can capture only a small subset of those light rays that travel through its optical system. In “Light Fields and Computational Imaging,” Marc Levoy discusses new imaging devices that capture a much larger number of light rays that travel in many parameterized directions—called the 4D light field. This novel concept truly revolutionizes digital imaging in many areas and enables new applications, such as multiperspective panoramas and synthetic aperture photography.

One other limitation of conventional cameras is that they illuminate scene points with light rays coming only from a single direction—from the integrated flash. In addition to capturing light rays propagating from the scene in multiple directions, advanced lighting systems can illuminate the scene with light rays coming from multiple directions. Paul Debevec describes such a system, called the Light Stage, in “Virtual Cinematography: Relighting through Computation.” This system enables virtual relighting of previously captured images or video sequences.

Continue reading with 'Future Forecasts: Other Innovations'….