COMP 776 - Final Project: Ribbon Carving, progress

Ryan Schubert
April 3, 2008


Things done:

• Loading videos into Matlab
• Manipulating video data in Matlab; currently only worrying about color videos with RGB at each pixel (non-colormapped)
• Can save out a new video with edited data (for exporting the results of the ribbon carving)
• Can look at arbitrary x-time slices of the video volume at any y value
• Can calculate an energy function for a given x-time slice of the image--there's currently a magic number to scale the importance of temporal gradient as compared to spatial gradient
• Calculates an average of all energy functions across all y slices of a video volume

Things still to be done:

• Generate the summed area table using the averaged energy function
• Remove the lowest energy ribbon as identified in the summed area table
• Save out resulting video with -1 frame, and repeat process to remove new frames
• Optimize ribbon removal rather than recomputing full energy function after each ribbon is removed

Some results of progress so far:

Matlab code:
ribbon.m - main file
calcEnergy.m - calculates the energy of the given x-time slice
findSeam.m - does nothing yet

I'm currently using the following video I took as a simple test case: test.avi
The video consists of one person flailing around on one side of the video for a short amount of time, someone else jumping into the video halfway through, and the first person jumping out of the video for the remainder. Ideally, this video would be shortened such that there is more temporal overlap with both people on the screen at the same time (as much as is possible without them spatially intersecting). There is also several frames at the beginning before the first person starts moving that have low energy and would likely be removed before others in the video.

This is an x-time slice of the test video at y = 370	This is the energy function for the slice at y = 370 (with temporal gradient 10x as important as spatial)

This is the average energy function in x-time for ALL y slices of the test video.