Graffiti Art into the Picture

[CS445] Final Project with Mohit Vyas

1. Overview

Painting on the wall in a picture without considering 3D information is likely to look unrealistic and unnatural. In this project, we made a pipeline that creates realistic graffiti on the walls in a picture. We used the COLMAP to obtain a sparse point cloud from images. The first step is to compute the projection matrix between the 3d coordinates and the 2d image plane. Then warp the image to make it parallel to the image plane axes. Now we're ready to paint on the wall!

The next steps are including image segmentation and a blending technique. To avoid drawing on the window of the building, we segment the wall pixels from the background in an image by using the Graph-Cut Algorithm. Then blend the new painting on the foreground pixels. We've compared two blending methods: (i) Laplacian blending, and (ii) Gradient-based domain fusion using mixed gradients. Laplacian blending keeps taking the texture from the source image, however, the gradient-based domain fusion enables the texture of bricks to be preserved.

Figure 1. Goal: Create a graffiti on the wall by taking into account its 3D structure.


Image transformation Pipeline

Wall Painting Pipeline

2. Results

Figure 2.Left to Right: Source Image, Laplacian Blending, and Gradient Domain Fusion using Mixed Gradients

Figure 3. Left to Right: Source Image, Laplacian Blending, and Gradient Domain Fusion using Mixed Gradients