Dealing with dominant planes

The projective structure and motion approach described in the previous section assumes that both motion and structure are general. When this is not the case, the approach can fail. In the case of motion this will happen when the camera is purely rotating. A solution to this problem was proposed in [155]. Here we will assume that care is taken during acquisition to not take multiple images from the same position so that this problem doesn't occur^E1.

Scene related problems occur when (part of) the scene is purely planar. In this case it is not possible anymore to determine the epipolar geometry uniquely. If the scene is planar, the image motion can be fully described by a homography. Since ${\bf F}=[{\tt e}']_\times {\bf H}$ (with $[{\tt e}']_\times$ the vector product with the epipole ${\tt e}'$), there is a 2 parameter family of solutions for the epipolar geometry. In practice robust techniques would pick a random solution based on the inclusion of some outliers.

Assuming we would be able to detect this degeneracy, the problem is not completely solved yet. Obviously, the different subsequences containing sufficient general 3D structure could be reconstructed separately. The structure of subsequences containing only a single plane could also be reconstructed as such. These planar reconstructions could then be inserted into the neighboring 3D projective reconstructions. However, there remains an ambiguity on the transformation relating two 3D projective reconstruction only sharing a common plane. The plane shared by the two reconstructions can be uniquely parameterized by three 3D points ($3 \times 3$ parameters) and a fourth point in the plane (2 free parameters) to determine the projective basis within the plane. The ambiguity therefore has 15-11=4 degrees of freedom. An illustration is given on the left side of Figure 5.5.

Figure 5.5: Left: Illustration of the four-parameter ambiguity between two projective reconstructions sharing a common plane. If the base of the cube is shared, a projective transformation can still affect the height of the cube and the position of the third vanishing point. Right: A fundamental problem for many (man-made) scenes is that it is not possible to see A,B and C at the same time and therefore when moving from position 1 to position 3 the planar ambiguity problem will be encountered.

Note also that it can be very hard to avoid this type of degeneracy as can be seen from the right side of Figure 5.5. Many scenes have a configuration similar to this one.

In [100] a practical solution was proposed for recovering the 3D structure and camera motion of these types of sequences without ambiguities. This approach is described in the following sections.