Thoughts on Coded Aperture Imaging (Coded Imaging)

See our SIGGRAPH 2007 paper on Coded Aperture Photography and Light Field Capture Using Optical Heterodyning

Coded Exposure: 
Our coded exposure (flutter shutter) paper showed that high spatial frequencies are lost in motion blurred images and by coding the exposure, these spatial frequencies can be retained. Thus, the ill-posed problem of motion deblurring can be converted into a well-posed problem.

This immediately leads us to the question: How can we extend this notion of coding to space (2D) from time (1D).
In a traditional camera, the out of focus blur acts as a low pass filter and high spatial frequencies are lost in out of focus blurred images. It will be useful to apply the notion of coding to make this defocus point spread function broadband. This may enable us to recover sharp images from out of focus blurred images.

Coded Imaging:
An alternate class of imaging techniques that allows imaging gamma rays/X rays. These rays cannot bend as light and traditional lens based focusing imaging is thus not useful. These techniques are also referred to as multiplexing techniques. Coded aperture imaging is used in high energy astronomy to capture image of distant stars. A pinhole camera is ideal for this kind of situation as it has infinite depth of field and no chromatic aberrations. However, a pinhole is after all a pinhole and cannot capture large amount of light. So multiple pinholes are generally used. But the observed image is then the sum of images due to all the pinholes. To successfully decode the image, a "coded pattern" of pinholes is used and hence the name "coded aperture imaging". Similarly, in other fields such as medical imaging, the observed image is coded according to physics of the imaging apparatus and needs to be decoded to be useful to human observers.

However, we are interested in coded imaging applied to consumer photography. Traditional coded aperture as used in astronomy is suitable only for point light source distributions and not for area light sources. See [3,4]. The SNR adavtange is lost for natural scenes. In addition, the absence of lens in traditional coded aperture will lead to severe degradation of image quality.

Coded Aperture with Lens:

For consumer photography, it will be desirable to use some kind of coded imaging with lens. The system by Hiura and
Matsuyama  [1] shows how to achieve this. In their system, they put a coded aperture with four pinholes in the optical path of the camera and capture multiple images. They use these images to solve for the depth of the scene using "depth from defocus" techniques. They also show how to compute a blur free image. Thus, coded aperture imaging systems with lens are not new.

The paper by Hiura also shows that by structuring the aperture shape, one can design blurring characterstics suitable for depth measurement. They argued that the aperture pattern can be modified as a high pass or band pass system, which can presrve useful information for depth measurement. Thus, one can have a random shape aperture in the lens to suit for a particular task.

A range finding system using coded aperture:

A very interesting analysis of coded aperture systems is given in [2,5]. In this paper, the authors analyze the capability of a coded aperture system for range finding based on Fisher information. They argued that a coded aperture designed for range finding will be ill-suited for capturing intensities. Similarly, a coded aperture designed to make the defocus blur broadband will be a bad choice for finding range (or depth). Thus, there is an inherent tradeoff between capturing image intensities and measuring depths using a single lens, single image coded aperture camera. Using Fisher information and Cramer-Rao bounds, authors prove a necessary condition for a coded aperture system to be able to measure depths. This condition says the OTF should have zeros as a function of misfocus. Note that this directly contradicts with the notion of OTF being broadband for measuring intensities.

[1] Shinsaku Hiura and Takashi Matsuyama,  "Depth Measurement by the Multi-Focus Camera", IEEE Conference on Computer Vision and Pattern Recognition  (CVPR) 1998

[2] E. R. Dowski, Jr. and W. T. Cathey, ``Single-Lens, Single-Image, Incoherent Passive Ranging Systems,'' Applied Optics, 33, 6762-6773 (1994). 

[3] Roberto Accorsi, Francesca Gasparini and Richard C. Lanza, "Optimal coded aperture patterns for improved SNR in nuclear medicine imaging", Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Volume 474, Issue 3, December 2001, Pages 273-284

[4] Roberto Accorsi,  "Analytic derivation of the Contrast to Noise Ratio in Coded Aperture Imaging",  Personal Communications