Assignment 1: Prokudin-Gorskii image alignment using MATLAB

1.1 Description:

Prokudin-Gorskii captured three photographs for R, G and B color channels by a set of three cameras displaced by some vertical offset. The aim of this assignment is to align these three image channels so that we get a good color image. The images were taken from here. Multiple metrics were experimented with to get a perfect alignment. The alignment was done to sub-pixel levels and small rotaions in the image were also considered.

The algorithm exhaustively seraches for small shifts to find the correct alignment. It uses edges in the image to find the alignment by using normalized cross correlation (NCC) metric. For very large images, the algorithm creates an image pyramid with three levels. By such a procedure, the serach range in 2-D can be adaptively changed according to the level in the image pyramid. This accelerates the algorithm, hence it is able to find alignment for a 3760*3225 image in 12 seconds!
For the case of large images, its assumed that image is larger than 1024*1024.

1.2.1 Algorithm:

1. Break the image into three channels in the order B, G, R from top
2. Set search range in 2-D ~ 21*21 matrix
3. If image is very large, downsample it to size less than 512*512, set dsampleFlag=1
4. Find shifts in channels with B channel as reference, function - shiftAlign
5. If dsampleFlag==1, find the shifts at finer levels using the same function used above with adjusted search range
6. Find small rotations, function - rotate
7. Find small subpixel shifts using convolution, function - sPix

1.2.2 Function Description:

1. shiftAlign
1. Adjust size of image according to downsampling level
2. Find edges in all three channels
3. Using NCC, find the best shifts in X and Y directions on edge images

2. rotate
1. Adjust size of image according to downsampling level
2. Find edges in all three channels
3. Using NCC, find the best rotation in range -1 to 1 degree for the edge images

3. sPix
1. Extract small portion of the image for faster results
2. Create matrices in X and Y directions for sub-pixel shifts
3. Using convolution, apply the matrices and find optimal shifts for the color channel image.
Edge image was not considered as edge estimating functions are not sub-pixel accurate.

1. When image is very large, NCC is calculate only for a region of 512*512 at the centre.
But this works on the assumption that large image is atleast 1024*1024

2. Search range is adaptively changed
For the smallest downsampled image (downsampling factor = k0, k lies between 0 and 1), search range is kept high (21*21) so that we get a coarse estimate.
For the image levels in between, if error in k1 level is 1 pix, then search range in finer level (k2) should be
For error in k1 = 1 pix,
error in k2 = ceiling(k2/k1) Hence, range = 2ceiling(k2/k1)+1 * 2ceiling(k2/k1)+1
As a precautionary measure I have kept range = 2ceiling(k2/k1)+2 * 2ceiling(k2/k1)+2

Original Image

Bad Alignment

Final result

Following shows the aligned images at various levels in the image pyramid with their corresponding normalised cross correlation (NCC) maps. Note how the NCC map size changes according to level.

Smallest downsampled aligned image

Corresponding NCC map for smallest downsampled image

NCC for scale = 0.5 stage

NCC for scale =1 stage

Scale = 1, aligned but not rotated

Scale =1 , aligned and rotated

Please change the path for the input image. Download all the following files and keep them in the same folder.