Home

SIGGRAPH Asia 2010 Computational imaging

Axial-Cones: Modeling Spherical Catadioptric Cameras for Wide-Angle Light Field Rendering

Yuichi Taguchi, Amit Agrawal, Ashok Veeraraghavan, Srikumar Ramalingam, and Ramesh Raskar

Mitsubishi Electric Research Labs (MERL) & MIT Media Lab

The idea

One shot. A wide-angle light field.

Capture and render wide-angle light fields using an array of spherical mirrors or refractive spheres. Axial-cone modeling enables GPU-based digital refocusing and dense volumetric 3D reconstruction from a single photograph.

Outdoor scene reflected in an array of twelve spherical mirrors
A single photograph captured through 12 spherical mirrors. The outdoor scene shows a person trying to catch a ball.
Scene captured through an array of twelve refractive acrylic ball lenses
A single photograph captured through 12 refractive acrylic ball lenses.
01

Axial-cone
modeling

Represent a multi-perspective image as a collection of virtual perspective cameras lying on an axis, each with its own viewpoint and cone of rays. The model avoids geometric approximations and lets algorithms developed for perspective cameras work with spherical catadioptric images.

Spherical mirror geometry represented by virtual axial-cone cameras
Spherical mirrors modeled using virtual axial-cone cameras.
Refractive sphere geometry represented by virtual axial-cone cameras
Refractive spheres modeled using virtual axial-cone cameras.

02 / Results

Refocus the scene. Recover its depth.

Wide-angle digital refocusing produces depth-of-field effects after capture. The same geometric model supports dense depth estimation and spherical distortion correction. The GPU implementation achieves up to three orders of magnitude speedup over ray tracing.

Wide-angle refocusing results and depth maps from the spherical mirror array
Spherical mirror array: wide-angle refocusing results and depth maps.
Wide-angle refocusing results and depth maps from the refractive ball array
Refractive sphere array: wide-angle refocusing results and depth maps.

03 / Capture systems

The experimental setups

Camera positions for the reflective steel ball and refractive acrylic ball arrays
Camera configurations for the reflective steel ball array and the refractive acrylic ball array.
Handheld reflective steel ball and refractive acrylic ball arrays, with an outdoor capture example
Handheld reflective and refractive arrays, with an outdoor capture example.
04

Abstract

Catadioptric imaging systems are commonly used for wide-angle imaging, but lead to multi-perspective images which do not allow algorithms designed for perspective cameras to be used. Efficient use of such systems requires accurate geometric ray modeling as well as fast algorithms. We present accurate geometric modeling of the multi-perspective photo captured with a spherical catadioptric imaging system using axial-cone cameras: multiple perspective cameras lying on an axis each with a different viewpoint and a different cone of rays. This modeling avoids geometric approximations and allows several algorithms developed for perspective cameras to be applied to multi-perspective catadioptric cameras.

We demonstrate axial-cone modeling in the context of rendering wide-angle light fields, captured using a spherical mirror array. We present several applications such as spherical distortion correction, digital refocusing for artistic depth of field effects in wide-angle scenes, and wide-angle dense depth estimation. Our GPU implementation using axial-cone modeling achieves up to three orders of magnitude speed up over ray tracing for these applications.