Home

CVPR 2006 Computational photography

Edge Suppression by Gradient Field Transformation Using Cross-Projection Tensors

Amit Agrawal, Ramesh Raskar, and Rama Chellappa

The idea

Suppress shared edges. Reveal what changed.

Use one image to suppress corresponding edges in another, even when illumination changes. Cross-projection tensors transform the gradient field, separating foreground structure from shared background texture through gradient integration.

Image A showing a foreground box against a textured background
Image A · foreground and background
Image B showing the background from the same viewpoint under different illumination
Image B · background under different illumination
Absolute pixel difference of images A and B, affected by illumination variations
Absolute difference · |A − B|
Foreground recovered through cross-projection tensor edge suppression
Cross-projection result · background texture suppressed
01

Foreground
recovery

Images A and B are captured from the same viewpoint with significant illumination differences. To recover the foreground layer, remove edges from A that also appear in B. A 2 × 2 projection tensor is estimated at every pixel and used to apply an affine transformation to A’s gradient field. Integrating that transformed field gives the foreground result.

The example removes background texture, including texture within the box’s shadow. Simple frame differencing and cross-correlation struggle with the illumination changes. Only one background image is available, making statistical background modeling difficult, and the box has a similar color to the red book behind it. This approach uses gradient direction rather than pixel intensity.

Limitations: illumination and reflectance edges are assumed not to coincide. A foreground reflectance edge coinciding with an illumination edge in B cannot be recovered. The box’s shadow remains because it introduces a new edge in A; the flash/no-flash examples below demonstrate shadow removal.

02

The algorithm

Algorithm pipeline estimating projection tensors, transforming the gradient field, and integrating the transformed and residual fields
Transforming and integrating the gradients separates edges unique to A from edges shared with B.

Estimate cross-projection tensors from A and B, then transform the gradient field of A. Integration produces A′, with the corresponding shared edges suppressed. Integrating the residual field produces A″, containing the edges in A that also occur in B.

The reference image B can be captured under different illumination. Applications include removing shadows from color images, suppressing glass reflections, and recovering intrinsic images for non-Lambertian scenes.

03 / Applications

Removing shadows with flash/no-flash pairs

A flash image provides the reference structure for removing shadows from an ambient photograph. The method avoids transferring flash shadows and can handle different color tones in the two inputs.

Ambient and flash photographs of a book, shadow removal, and recovered illumination map
Book example: remove shadows from the ambient image without transferring shadows from the flash image. The estimated illumination map is free of the book’s complex texture.
No-flash and flash mannequin photographs with different color tones, a shadow-free result, and an ambient illumination map
Mannequin example: the shadow-free result does not inherit flash shadows. Different ambient and flash color tones are handled without color calibration or white balancing.
Glass reflection removal using a low-contrast ambient photograph and a flash photograph with checkerboard reflections
Checkerboard reflections are removed from the flash image using a low-contrast no-flash image, despite differences in color tone.

04 / Glass reflections

Remove reflections
without color artifacts.

A low-contrast no-flash photograph guides removal of checkerboard reflections from the flash image. Although the input color tones differ, the reconstructed result avoids color artifacts.

05

Our SIGGRAPH 2005 gradient-projection method removes glass reflections by projecting a flash-image gradient onto the ambient-image gradient at each pixel.

This paper shows that projection is a special case of an affine transformation of the gradient field. Building on research in restoration using diffusion tensors, it derives projection tensors from one image and applies them to another for edge suppression under variable illumination.