Course Projects at UMD

The MATLAB implementation detects independently moving objects observed by a moving stereo camera. Depth from stereo differs from depth inferred through optical flow for independently moving objects, providing a segmentation cue.

The camera is calibrated and stereo frames are rectified to reduce correspondence search to one dimension. Disparities and calibration give a stereo depth map. Stereo depth and optical flow yield a linear ego-motion problem, solved robustly using Least Median of Squares. The estimated ego-motion gives depth from optical flow; regions where the two depth estimates disagree are marked as independently moving.

The project replaces a banner in an ice-skating video while preserving skaters that occlude it. The planar banner is related across frames by homographies.

After manually selecting its first-frame corners, a corner detector finds features inside the banner. Block matching tracks them, with prior-frame homographies helping transfer positions. More than the minimum four correspondences are used, with Least Median of Squares for robustness to occlusion.

All frames are warped to the first. A temporal median recovers the unobstructed banner under the assumption that each pixel is occluded in fewer than half the frames. Comparing against that median identifies visible pixels, where the replacement banner is inserted.

For a camera rotating around its vertical axis without translation, stationary scene points follow a homography while a moving person does not.

Feature matches and robust homography estimates align frames to one view. Their median forms a background mosaic, assuming that a moving person occupies any pixel in fewer than half the frames. Warping the median mosaic back to each frame produces a video without the person.

Feature correspondences between two images determine the fundamental matrix. Camera calibration gives the essential matrix, whose decomposition yields candidate rotations and translations. Of the four possible solutions, positive scene depths identify the physically consistent configuration.

The feature matrix from corresponding points across multiple images is factorized to estimate shape and camera motion under orthographic projection. Both orthographic and para-perspective factorization were implemented in MATLAB.

Factorization is a batch method. An Extended Kalman Filter provides a sequential alternative that updates shape and inter-frame camera motion as new features arrive. A batch reconstruction initializes the filter and its covariance matrices. See the structure-from-motion report.

Supervised and unsupervised texture-segmentation methods were implemented and compared in MATLAB, modeling the underlying texture as a Markov random field. See the texture-segmentation report.