Invertible Motion Blur in Video

 

Amit Agrawal, Yi Xu and Ramesh Raskar
ACM SIGGRAPH 2009



Summary

Linear motion-blur in a single photo is non-invertible but we make the blur in video invertible by changing the exposure time of successive frames. We show an automatic deblurring approach without special camera hardware by creating jointly invertible PSF, estimating PSF and segmenting moving objects on non-uniform background.


Abstract

We show that motion blur in successive video frames is invertible even if the point-spread function (PSF) due to motion smear in a single photo is non-invertible. Blurred photos exhibit nulls (zeros) in the frequency transform of the PSF, leading to an ill-posed deconvolution. Hardware solutions to avoid this require specialized devices such as the coded exposure camera or accelerating sensor motion. We employ ordinary video cameras and introduce the notion of null-filling along with joint-invertibility of multiple blur-functions. The key idea is to record the same object with varying PSFs, so that the nulls in the frequency component of one frame can be filled by other frames. The combined frequency transform becomes null-free, making deblurring well-posed. We achieve jointly-invertible blur simply by changing the exposure time of successive frames. We address the problem of automatic deblurring of objects moving with constant velocity by solving the four critical components: preservation of all spatial frequencies, segmentation of moving parts, motion estimation of moving parts, and non-degradation of the static parts of the scene. We demonstrate several challenging cases of object motion blur including textured backgrounds and partial occluders.


Paper (Preprint)

pdf (13 MB)

Video (Quicktime, 41 MB)

SIGGRAPH 2009 Talk (ppt, 67 MB)



Related Papers in Motion/Focus Deblurring

SIGGRAPH 2006      Coded exposure for motion deblurring
SIGGRAPH 2007      Coded aperture and a new theory of light field capture

CVPR 2007              Simultaneous motion deblurring and super-resolution
CVPR 2009              Optimal Single Image Capture for Motion Deblurring
CVPR 2009              New codes for coded exposure deblurring to help in PSF Estimation
      
Back to my homepage





Key Idea: A traditional camera results in a box-shaped motion blur (low frequency blur) which is difficult to remove. Coded exposure makes the motion blur invertible by fluttering the shutter. However, it is single photo based and PSF estimation is manual. Our technique makes motion blur invertible via PSF null-filling by capturing a varying exposure video. It does not require any hardware modifications and multiple photos/video make PSF estimation and segmentation easier for deblurring.


Additional Results



Faces:
Our approach can be easily used with standard digital SLR's. Three photos of a person standing up were captured using the AEB (Auto Exposure Bracketing) mode on Canon Digital Rebel XT. The exposures used were 1/10, 1/13 and 1/8 secs. The deblurring result shows the features of the face clearly.




Deblurring in presence of occluders
. In this example, a toy car is moved in front of a textured background. A flower in front occludes the car in all four captured photos. We obtain a sharp deblurring result. The user simply marks the location of the occluder in the first photo. While deblurring, the occluded pixels are not considered. The composite shows a novel synthesis, where the moving object is sharp and the occluder is blurred.  




Rotating CD:
PSF null-filling can also be applied to non-linear motions that lead to linear blur after image rectification. Using polar coodinates, input photos of a rotating CD can be rectified to have linear blur as shown. Deblurring followed by inverse rectification results in a sharp image of the CD.



Back to my homepage