Home

ICRA 2012 3D reconstruction

Convex Bricks: A New Primitive for Visual Hull Modeling and Reconstruction

Visesh Chari, Amit Agrawal, Yuichi Taguchi, and Srikumar Ramalingam

Mitsubishi Electric Research Labs (MERL)

Duck visual hull reconstruction comparing voxel carving with adaptive convex bricks
Visual hull reconstruction on the Duck dataset using 10 silhouettes. Voxel carving from 643 and 1283 grids produces 21,784 and 160,372 voxels, with 6,503 and 26,080 surface voxels, respectively. At the same precision as the 1283 grid, our reconstruction uses only 109 convex bricks (shades of red-brown) with 1,551 vertices. A surface mesh is obtained by removing internal planes.
01

Summary

We propose convex bricks, a new 3D primitive for modeling and reconstructing visual hulls from silhouettes. Unlike voxel based visual hull reconstruction where the shape of voxel remains fixed, the shape of convex bricks can adapt to the given silhouettes, thereby reducing the number of primitives required for a volumetric representation. Using convex bricks, polyhedral visual hull can be generated from polygonal silhouettes as a combination of 3D convex polytopes.

Convex bricks are unique in that they allow both volumetric and surface based representation. Being a volumetric approach, our technique avoids explicit computation of triple points/frontier points and does not require local orientation or connectivity rules. Interestingly, convex bricks also provides a watertight surface mesh, bridging the gap between volumetric and surface based approaches. In addition to high-quality reconstruction, our representation is useful for applications that require convex decomposition of 3D shapes.

02

Advantages
over voxels

  1. Independent of a fixed 3D grid. Unlike voxels, convex bricks avoid blockiness and quantization artifacts. Their size and shape depend on the inherent non-convexity of the object rather than its surface area.
  2. Fewer primitives. High-quality reconstruction requires fewer primitives than voxel carving.
  3. Volume and surface together. The representation provides both volumetric and surface information, rather than either one alone.
  4. Automatic shape decomposition. A decomposition of the 3D shape into convex parts is obtained automatically.
03

Implementation

Our implementation uses well-studied computational geometry operations such as (a) 2D convex decomposition, (b) 2D intersections and (c) 3D-3D convex intersections. The implementation is similar to voxel carving. Starting from a bounding box, each convex brick is projected on to a new silhouette. If it lies outside, the CB is discarded. If it lies completely inside, it is retained.

If it intersects with the silhouette, then the intersection is divided into k convex partitions. Each convex partition is then back-projected to subdivide the convex brick into k smaller convex polytopes. Thus, while voxel carving subdivides each voxel into 8 smaller voxels, our approach subdivides each convex brick into k convex polytopes depending on local silhouette shape.

04

Results

Convex-brick visual hull reconstructions of complex non-convex objects
High-quality visual hull reconstruction on real data using convex bricks. The shapes are complex and highly non-convex. From left to right: 23,107, 21,120, and 35,932 convex bricks, with 106,872, 101,325, and 156,243 vertices. Data courtesy of Yasutaka Furukawa.
Dinosaur reconstruction comparing different numbers of convex bricks with voxel carving
Comparison of reconstruction using different numbers of convex bricks and voxel carving (2,102 voxels, 1,659 on the surface).
Convex-brick reconstruction precision controlled by alpha, compared with blocky voxel carving results
Precision is controlled by the parameter alpha: larger alpha means lower precision. Even at larger alpha, the visual hull retains high-frequency details. Voxel carving requires significantly more voxels for similar precision and results in blocky reconstruction.