Assignment 2: Part 2:Mask generation for poisson blending using Graphcuts

1.1 Description:

The objective was to automatically create mask for poisson blending two images using graphcuts. The images were taken from flickr and were used under creative commons license. Aeroplanes were blended into the sky for this assignment. The image of the sky was selected from a database of 50 images taken from flickr using the GIST image descriptor. Specifically, sky & city ,were searched together to get these images.

The graphcut algorithm used in this project was the Max-flow min-cut algorithm taken from here. This implements Boykov-Kolmogorov algorithm. The files can be downloaded from here. As these files are written in C/C++, a MATLAB wrapper was used. The wrapper can be downloaded from here. This site has plenty of description of how to setup the graphcut algorithm.
The MATLAB wrapper is quite old, hence the README file is quite outdated. Proceed according to the README file but keep the folder name as maxflow-v3.01. Also, edit the make.m file by replacing everything in it by the following:
mex -largeArrayDims 'maxflowmex.cpp' 'maxflow-v3.01/graph.cpp' 'maxflow-v3.01/maxflow.cpp'
Now run the make.m file to finally set up the graphcut algorithm.This completes the graphcut algorithm part.
To be able to use GIST descriptor, download and unzip the files and place the folder hence created in working directory.

The image was modelled as a graph and graphcut algorithm was used to extract the object. Then the output of graphcut was converted into mask to further use in poisson blending.

1.3.1 Image selection using GIST :

I have created a reference imageset of 10 images of sky-city under the folder 'city'. Using these 'ideal' images I computed the centroid of their GIST descriptors.This centroid is used as a reference to select a proper image from dataset of 50 images. Euclidean distance is used to measure the similarity of the descriptors with the reference. The user can input a number n to get the best n matches and these are processed to get the max-flow. The image having maximum of the max-flow from n images taken is treated as the best image.
Functions: gistMean.m gistCompare.m

1.3.2 Image Positioning :

The image of city-sky and aeroplane hence obtained are resized and aligned. The target image (city-sky image) is assumed to have upper half as sky. The target image is divided into 6 equal parts. The aeroplane image ( also called as source) is aligned and resized to fit the centre upper block of target image.
Function: resizeAlign.m

1.3.3 Mask creation using graphcuts

The source image was converted into a 4 connected graph using the function in the wrapper. A rough mask is assumed to be provided initially which separates the background and the foreground. Using joint histograms RGB channels ( multiply individual probability,ie, Joint P =Pb*Pr*Pg ), probability is found out for a pixel to belong to foreground and background. The histograms are smoothed using a gaussian filter to get a smoother pdf.
It is important that the source to pixel flow > pixel to sink flow to force edge values to influence the cut.
The metric used for edges between pixel was absolute difference between gray values.
This mask was used for poisson blending which was implemented in assignment 2,part 1.
Function: labels.m

Target Image