COMP 776 Project Ideas from Gary Bishop

1. Motion detection for a move-to-music game: We are developing a generalized version of Dance-Dance Revolution for kids with movement disorders, blindness, or other disabilities. In the original game, players step on a pad according to visual instructions that scroll by on the screen. Scoring is based on accuracy and timing. In the generalized game I want kids to move in time to the music. We've got a proof of concept system working with the keyboard, mouse, and DDR pad as possible inputs. We'd like to extend it to include image-based input. So a kid in a wheel chair might move one arm, or her head, or even a leg. I'd like to detect the motion in images and "score" it based on how well it correlates with the beat of the music. I've got all the musical part well in hand. I can automatically detect the beat in music using open source software. I'd like help with the computer vision aspects of the project. I'm eager to work with a student. I think a crude measure of velocity might work well without tracking specific body parts.
   
   
2. Image classification as a switch replacement: People with movement disorders often use one or two switches as the primary means of communication with computers (and the world). The switches typically are a "mover" and a "chooser". Hit the mover to go from A to B to C, etc. and the chooser when you get to the letter you want. Higher level systems do the same with words. Steven Hawking uses a single switch and lists of words to compose his books and speech. I'd like to replace the expensive and difficult to use switches with a cheap webcam. In my ideal system we'd point the camera at a kid and say "Show me some movement that is easy today". They might move an arm or their head. "OK, that is your mover; now show me something else." And they might move a leg or the other arm. "OK, that is your chooser". Then later in the day when those movements are no longer easy (or even possible) they could choose another two.