Optimal Coded Sampling for Temporal
SuperResolution


Amit
Agrawal, Mohit Gupta, Ashok Veeraraghavan and Srinivasa Narasimhan CVPR 2010 
Summary
Coded sampling to utilize multiple low cost low speed cameras
to obtain an effective high speed camera. Abstract


Paper (Preprint)
Related Papers in Motion/Focus Deblurring SIGGRAPH 2006 Coded exposure for motion deblurring CVPR 2007
Simultaneous motion
deblurring and superresolution
Sampling Strategy: Suppose we have N cameras each running at frame rate f. How can we use them to get a video with frame rate N*f? There are several sampling schemes possible. Point Sampling: Reduce the exposure time of each camera to 1/Nf and stagger the start of integration. Pros: The interleaved video automatically gives a higher frame rate video. Blur is avoided becasue each camera captures a sharp image. N cameras give a frame rate increase of N. No postprocessing required. Cons: No light benefit compared to an equivalent high speed camera running at frame rate Nf Box Sampling: Use larger exposure time for each camera (1/f) and stagger the start of integration. The interleaved video removes aliasing. But every frame has motion blur. Pros: More light (N times) compared to an equivalent high speed camera running at frame rate Nf Cons: Box blur supppress high frequency content leading to illposed system. Noise is increased in reconstruction. Requires solving a big linear system. Dififcult to get N times increase in frame rate with N cameras. Coded Sampling (Ours): Use larger exposure time for each camera (1/f), but temporally modulate every frame. Each camera has a different code, but the code is same for all frames of a given camera. Pros: More light (N/2 times) compared to an equivalent high speed camera running at frame rate Nf Easy to get N times increase in frame rate with N cameras. Codes can be chosen to get the maximum possible SNR in reconstruction. Coded blur leads to wellposed system which can be solved independently for each set of N output frames. This allows streaming reconstruction with low computational complexity (solving only a N by N linear system). Cons: N/2 times more light instead of N times more light in each camera compared to box sampling Results
Note that
reconstrcuted frames are sharp with low noise. Thus, coded sampling
allows streaming output with low computation. On the other hand, for
box sampling, one needs to solve a much bigger linear system (of
several frames). Since box sampling is illposed it leads to increased
noise. By using regularization, noise can be reduced, but blur is
increased in the output.



