Video Stitching and Processing

[CS445]Computational Photography (Instructor: Prof. Derek Hoiem)

Part 1. Background Subtraction

The first goal of this project is to generate a movie that only includes moving objects. First, I compute Homography between two frames by using the extracted keypoints in images and map another frame onto the reference frame. Then, I create a background panorama by taking the mean value over overlapped pixels to remove moving objects and map it to the movie coordinates to create a background movie. Finally, a foreground movie is created by subtracting background color to make it contain only the pixels of moving things.

Panorama using five key frames

Map the video to the reference plane

Create background panorama

Create background movie & foreground movie

Part 2. Insert an unexpected object in the video

For the Bells and Whistles, I inserted a blue ball into the original video. The object should be below foreground pixels and above background pixels. And it should be fixed on the ground, so its position does not change during video playback. I used the inverse of homography matrices that are already computed in the previous part.