SIGGRAPH 2007 Computational photography
Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing
The idea
Encode the rays. Refocus after capture.
A patterned mask inside a camera modulates incoming light so that a 2D photograph can encode a 4D light field. A related aperture-mask design enables full-resolution refocusing of layered Lambertian scenes and editing image contents by depth, without recovering the complete light field.
Two camera
designs
A patterned planar mask attenuates rays instead of bending them. Placed near the sensor, it reversibly modulates the incoming light field, encoding angular information in the captured image without additional lenses.
A broad-band mask at the lens supports coded aperture refocusing. This partial encoding of ray-space information allows unwanted occluders to be removed or suppressed in layered Lambertian scenes.
02 / Fourier-domain illustration
The same scene, with and without a mask
A scene of cones at different depths illustrates how the mask changes the captured image and its spectrum. The mask casts a soft shadow on the sensor, dappling the incoming light and creating spectral replicas.
In this example, the cosine mask has four harmonics, giving 2 × 4 + 1 = 9 impulses in its frequency response. The two spatial directions produce a 9 × 9 array of 81 spectral replicas.
From 2D photo
to 4D light field
- Compute the 2D Fourier transform of the captured photograph.
- Rearrange the 81 spectral tiles into four dimensions, using the known mask structure.
- Compute the inverse 4D Fourier transform to recover the light field.
The recovered sub-aperture views correspond to images seen through a narrow aperture placed at different positions on the lens.
Abstract
We describe reversible modulation of 4D light field by inserting a patterned planar mask in the optical path of a lens based camera. We can reconstruct the 4D light field from a 2D camera image without any additional lenses as required by previous light field cameras. The patterned mask attenuates light rays inside the camera instead of bending them, and the attenuation recoverably encodes the ray on the 2D sensor. Our mask-equipped camera focuses just as a traditional camera might to capture conventional 2D photos at full sensor resolution, but the raw pixel values also hold a modulated 4D light field. The light field can be recovered by rearranging the tiles of the 2D Fourier transform of sensor values into 4D planes, and computing the inverse Fourier transform.
We also show how a broad-band mask placed at the lens enables us to compute refocusing at full sensor resolution for images of layered Lambertian scenes. This partial encoding of 4D ray-space data enables editing of image contents by depth to remove or suppress unwanted occluders, yet does not require computational recovery of the complete 4D light field.
Notes and
resources
Errata: Equations 11 and 12 in the published version contain errors. These have been corrected in the online PDFs linked above.