Dappled Photography: Mask
Enhanced Cameras for
Figure 1. Prototype camera designs derived from our
Fourier domain theory of mask-enhanced cameras.
Graphical Illustration of Computing 4D Light Field from 2D Photo
Consider a scene consisting of several cones at different depths. If we take an image with a traditional camera, we get the photo shown above. The magnitude
of the 2D FFT of the image shows high energy at low frequencies. This is a well known fact that most energy in natural images is concentrated
in low frequencies.
Now consider taking a 2D photo of the same scene with our Heterodyne Light Field Camera. In this new camera, we place a cosine mask near the
sensor. We get an image shown as below.
Although this image looks similar to the previous photo captured using a traditional camera, notice the effect of mask on the input photo. The mask
casts a soft shadow on the sensor. It dapples the light reaching the sensor. In theory, mask modulates the incoming 4D light field to make spectral
Now consider the 2D FFT of this new image. Notice that the spectral replicas are clearly seen!!!. Compare this FFT image with the previous FFT image to
visualize the differences. These replicas are due to the cosine mask placed near the sensor. The cosine mask modulates the incoming 4D light field.
In this example, the mask has 4 harmonics, or 4*2+1 = 9 impulses in its frequency response. Thus, we get 9*9=81 replicas in both x and y direction.
Now lets see how we can obtain the 4D light field from the modulated 2D photo captured from our Heterodyne Light Field Camera. The entire algorithm
is shown graphically below.
1. Compute the 2D FFT of the captured 2D photo.
2. We know that we will get 9*9 replicas due to the physical mask placed near the sensor. Rearrange these 81 tiles into 4D.
3. Compute the inverse 4D FFT to get the light field.
Click on the above image to see a video (ViewCones1.avi) of different 'views' obtained from the light field. These views are essentially images that we
would obtain if we look through a narrow aperture on the lens at different positions.
Coded Exposure Photography: Motion Deblurring using Fluttered Shutter, SIGGRAPH 2006
Resolving Objects at Higher Resolution from a Single Motion-Blurred Image, CVPR 2007