Coded Exposure Photography: Motion Deblurring using Fluttered Shutter

Ramesh Raskar, Amit Agrawal, and Jack Tumblin

Low res pdf, High res pdf

Talk Slides (PPT)
As presented during SIGGRAPH 2006 with no text on any slide
With text

View Slides as html

Flutter Shutter Video Camera (On-Chip Fluttered Integration)

Thoughts on Coded Aperture Imaging

Super-resolution from Motion Blur using Flutter Shutter Camera

Related Papers:

Resolving Objects at Higher Resolution from a Single Motion-Blurred Image, CVPR 2007
Dappled Photography: Mask Enahnced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing, SIGGRAPH 2007

In a conventional single-exposure photograph, moving objects or moving cameras cause motion blur. The exposure time defines a temporal box
filter that smears the moving object across the image by convolution.  This box filter destroys important high-frequency spatial details
so that deblurring via deconvolution becomes an ill-posed problem.

Rather than leaving the shutter open for the entire exposure duration, we ``flutter'' the camera's shutter open and closed during the
chosen exposure time with a binary pseudo-random sequence. The flutter changes the box filter to a broad-band filter that preserves
high-frequency spatial details in the blurred image and the corresponding deconvolution becomes a well-posed problem. We demonstrate that
manually-specified point spread functions are sufficient for several challenging cases of motion-blur removal including extremely large
motions, textured backgrounds and partial occluders.

Single Photo (Exposure Time: 200ms)


Rectified image

Deblurred Output
Top left shows image of a fast moving taxi captured using our new camera (exposure time 200ms). Bottom left shows the rectified image to make motion lines 
parallel to scan lines. An approximate cutout of the blurred object is shown in top right. The deblurred image obtained by solving a
linear system is shown in bottom right. Note that all high frequencies are recovered and there are no deconvolution artifacts.

Key idea and differences with previous methods:

Motion deblurring is an ill-posed problem. The relative motion between camera and the scene results in a blurred image in
which high frequencies are lost, due to which deblurring results in increased noise, ringing and other artifacts. Note that this is
independent of the deblurring algorithm used. As soon as the image is captured, high frequencies are lost and the damage is done.

In this paper, we propose a novel coded exposure camera and show that by fluttering the shutter in a single exposure, one can
retain high frequencies. The deblurring then become a well-posed problem. Our approach is thus fundamentally different from other
deblurring algorithms. In fact, we use the simplest possible deconvolution approach: solving a linear system AX=B to recover
the sharp image X from the blurred image B and show high quality results with large motion blurs (~300 pixels) with no ringing and
deconvolution artifacts. The change in hardware amounts to putting an external ferro-electric shutter in front of the lens,
synchronized with the start of the exposure time and can be done on any off the shelf digital camera.

Comparison of Short Exposure, Traditional Camera, and Coded Exposure Camera

Our portable prototype camera

Photo (A) taken with a traditional camera
Exposure time: 1 sec

Photo (B) taken with our camera
Exposure time: 1 sec

Photo (C) taken with a traditional camera
Exposure time: 100 ms

  Photo (D) taken with our camera
Exposure time: 100 ms
Comparison of images taken with a traditional camera and flutter shutter camera. In images A and B, a bright LED (light emitting diode) is moved rapidly in front of the camera by
the person in the photo. For traditional camera, the image of LED is a continuous smear (on left). For coded exposure, due to fluttering of the shutter, the smear becomes discontinuous
preserving high frequencies (on right).  Notice the headlights of the car in photos C and D. In a conventional photo (left), the headlights will leave a continuous trail. In our camera,
due to fluttering of the shutter, the headlights leave a discontinuous smear (on right).

Additional Results
Outdoor Taxi
Comparisons with Short Exposure and Traditional Camera
License Plate Retrieval from Blurred Image

Flutter Shutter Video Camera