Comp 770: Computer Graphics

Project Proposal: Shape Deformation for Articulated Meshes

Goal of the Project:

The aim of this project is to adapt a template human body model (triangular mesh + skeleton [1]) so that it fits given silhouettes observed by multiple cameras. The video data may correspond to a person whose body shape differs from that of the original model. The deformation process should maintain the physical constraints on the body and the skeleton.

Motivation:

Motion capture or 3D pose estimation techniques require an initial model which is then updated at every frame to represent the current pose. The model representing the initial pose is typically acquired offline with the help of devices like body scanners, optical or magnetic sensors, motion suits etc. This may, however, not be feasible for applications such as surveillance or when a large number people need to be tracked. For such cases, an automatic model initialization can be achieved by deforming the shape of a mean human body model to fit the target person based on video data. While a lot of work has been done for modeling the deformations of articualted mesh models for representing different poses, relatively fewer works address the problem of modeling deformations to capture shape variability. A shape deformation method for automatic model initialization will allow motion capture to be used for a wider range of applications. Further, pose estimation finds direct application to character animation via motion tranfer from real videos. Shape deformation can also be used for crowd simulation by simulating people with different appearances from a given template.

Subtasks:

1. Modeling mesh deformations to capture variations in body shape: using the SCAPE model described in [2].

2. Fitting a template mesh to multiple-view silhouettes: registering the mesh to visual hull computed from silhouettes.

3. Computing the skeleton associated with the deformed mesh: learning the pose deformation transfer between original mesh and the deformed mesh[3].

References:

[1] Lewis J P, Cordner M, and Fong N. Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. SIGGRAPH, 165-172 (2000)

[2] Anguelov D, Srinivasan P, Koller D, Thrun S, Rodgers J, and Davis J. SCAPE: Shape completion and animation of people. SIGGRAPH 24, 408-416 (2005)

[3] Sumner R W, and Popovic J. Deformation transfer for triangle meshes. SIGGRAPH 23(3), 399-405 (2004)

Update 1:

Implemented a model for skeleton-driven mesh deformation to represent various poses of a human body. The model consists of an underlying skeleton or an articulated chain whose configuration changes depending on the movement of various bones which are linked in a hieracrchy. The amount of motion a certain joint can undergo is constrained by the number of degrees of freedom it has and by the upper and lower bounds for each degree of freedom. These constraints are used to ensure that the skeleton pose doesn't violate any physical constraints of human body motion. The mesh deformation occurs depending on the current configuration of the animated skeleton. I have implemented the 'Linear Blend Skinning' algorithm for mesh deformations.