COMP 776 Spring 2009

Assignment 1: Demosaicing

Due date: January 29, 5PM

The goal of the assignment is to get started with image processing in MATLAB by implementing a very simple demosaicing algorithm. You will be working with the following data:

  • original image, "mosaic" image

    The "mosaic" image was created by taking the original color image and keeping only one color component for each pixel, according to the standard Bayer pattern: 

    
R G . . .
G B
.
.
.
    So, once you read the "mosaic" image into a matrix, entry (1,1) will be the value of the "red" component for pixel (1,1), entry (1,2) will be "green", etc.

    Part 1: Linear Interpolation

    Implement a very simple linear interpolation approach to demosaicing: for each pixel, fill in the two missing channels by averaging either the four or the two neighboring known channel values:

    Avoid using loops! Instead, use the imfilter function (filter2 or conv2 work just as well).

    The above method, being very simple, does not work perfectly. You can see where it makes mistakes by computing a map of squared differences (summed over the three color components) between the original and reconstructed color value for each pixel. Compute such a map and display it using the imshow or imagesc commands (see the documentation for those commands to figure out how to scale the values for good visibility). In addition, compute the average and maximum per-pixel errors for the image. Finally, show a close-up of some patch of the reconstructed image where the artifacts are particularly apparent and explain the cause of these artifacts.

    Part 2: The Freeman Method

    Bill Freeman proposed an improvement of the simple bilinear interpolation approach. Since the G channel is sampled at a higher rate than the R and B channels, one might expect interpolation to work better for G values. Then it would make sense to use the interpolated G channel to modify the interpolated R and B channels. The improved algorithm begins with linear interpolation applied separately to each channel, just as you have already done above. The estimated G channel is not changed, but R and B channels are modified as follows. First, compute the difference images R-G and B-G between the respective interpolated channels. Mosaicing artifacts tend to show up as small "splotches" in these images. To eliminate the "splotches", apply median filtering (medfilt2 command in MATLAB) to the R-G and B-G images. Finally, create the modified R and B channels by adding the G channel to the respective difference images.

    Implement the above algorithm and visualize the quality of the results in the same way as for Part 1 by displaying the error image and computing average and maximum error. Compare the output to that of Part 1. Are there visible improvements (especially in the close-up patch selected in Part 1)?

    Hint: Implementing this method should take you about two lines of code.

    For Bonus Points

    Here are some optional ideas for digging a little deeper:
    • Take some additional color images, create a fake "mosaic" by removing the two color channels according to the Bayer pattern (or, better yet, find some unprocessed camera images), and apply the demosaicing algorithm to them. Try to find images for which it "breaks" particularly badly.
    • Propose (or look up online) additional improvements to make demosaicing work better. Implement some improvements and compare the results.

    Turning in the Assignment

    Please email me your MATLAB code and a brief report (in a single PDF file) showin all your results and answering the above questions by 5PM, Thursday, January 29th.

    Useful MATLAB Functions

    • imread, imwrite
    • imfilter, filter2, conv2
    • imshow, imagesc
    • medfilt2
    • max, mean