Consider a pair
of flash/no-flash images as shown on the right. The ambient image has
reflections of the photographer who is taking this picture. You can see
the hands and face of the photographer and even the tripod getting
reflected in this image. The flash image is a short exposure image
which removes these reflections but
has a hotspot. This hotspot is caused by the direct reflection of flash
from the glossy surface of the painting. We wish to combine these two
images to get an image free of ambient reflections and flash hotspot.
highlighted patch in these images. This patch does not have
any reflections. So the direction of the ambient and flash image
gradient vector will be the same for any pixel in this patch.
another image patch at the bottom which
overlaps with reflections. The direction of the gradient vector in
flash and ambient images will not be the same. Due to the reflections
in the ambient image, the direction of the ambient image gradient
vector is perturbed.
the direction of the image gradient
in flash and ambient images should remain the same, we can constrain
the direction of the ambient image gradient vector by taking a vector
projection of the ambient image gradient on to the flash image gradient
The "result" image
is obtained by a 2D integration of the projected
residual gradients removed by taking the projection
can be integrated to obtain the "reflection layer".
Thus by using
gradient projection, we can decompose a image with
reflections into a reflection free image and the reflection layer.
One can ask why do
we need to do a projection. We know the direction of
the gradient vector so we can choose some other magnitude for it. The
important thing is to maintain the same direction.
The reason for
doing projection is that for a given gradient field, the
gradient field which lies orthogonal to it does not have much visual
information. Suppose (gx,gy) denotes the original gradient field and
(gx_90,gy_90) denotes the gradient field obtained by rotating each
gradient vector by 90 degrees. One can see that the divergence of the
rotated gradient field is zero everywhere. So the reconstruction from
the rotated gradient field will be zero everywhere.
We took the Lena
image and find gradients in x and y directions. Then
we start rotating these gradient vectors by certain angles and
reconstructing from the rotated gradient field. One can see that at a
rotation angle of 90 degress, the reconstruction does not have any
visual information. At rotation angle of 180 degrees, the obtained
is the negative of the original image.
My CVPR 2006 paper for a
more general algorithm for edge suppression by gradient field