Image Quilting & Texture Transfer

[CS445]Computational Photography (Instructor: Prof. Derek Hoiem)

Reference article is SIGGRAPH 2001 paper by Efros and Freeman.

Part 1. Image Quilting by Cutting after Seam Finding

Image Quilting is used as a fast and simple texture synthesis algorithm. The seam finding method makes image quilting look much more seamless than using random sampling or overlapping patches. To describe how the cut function works, figure 1 illustrates how the seam finding works. First, overlap two sample patches. Then compute the cost function over the overlapped area. More white means that the overlapped pixels have a big difference in RGB values. The boundary between black and white in the third image is the Min-Cost path found by the cut function. After applying the masks and summing up, we can create a seamless image.

Figure 1. Input Texture
Figure 2. Illustration of the Seam Finding
Figure 3. Left to Right: Random placement of blocks, Neighboring blocks constrained by overlap, and Seam Finding technique

Figure 4. Marble Pattern

Part 2. Texture Transfer

The source image is a part of the painting by Georges Seurat, which is one of the most famous of the Pointillism painting. I turned this image to grayscale as costs caused by color difference over the overlapped area can make errors in the texture transferred image.

Figure 5, 6. (Left) Part of "A Sunday Afternoon on the Island of La Grande Jatte", Painting by Georges Seurat, (Right)Image converted to Grayscale

The weight and alpha determine the tradeoff between the texture synthesis and the fidelity to the target image correspondence map. I used image intensities as a correspondence map and set alpha as 0.1 which is same with alpha for 1st iteration of texture transfer in the paper.

Figure 7. Texture transfer result: My Face

Figure 8. Texture transfer result: Baby Elephant and My Friend