Partial projective structure and motion recovery

The sequence is first traversed and separated in subsequences. For subsequences with sufficient 3D structure (case A) the approach described in Section 5 is followed so that the projective structure and motion is recovered. When a triplet corresponds to case B, only planar features are tracked and reconstructed (in 2D). A possible partitioning of an image sequence is given in Table 5.2. Note that the triplet 3-4-5 would cause an approach based on [155] to fail.

Table: Example on how a sequence would be partitioned based on the different cases obtained in the model selection step. Underlined F correspond to cases that would not be dealt with appropriately using a pairwise analysis.

case	AABAABBBBBAAA
3D	PPPP PPPPP
2D	HH HHHHHH
3D	PPPP
	FFFFFFHHHHFFFF

Suppose the plane ${\tt\Pi }$ is labeled as a dominant plane from view $i$ based on features tracked in views $(i-1,i,i+1)$. In general, some feature points ${\tt M}_{\tt\Pi}$ located on ${\tt\Pi }$ will have been reconstructed in 3D from previous views (e.g. $i$ and $(i-1)$). Therefore, the coefficients of ${\tt\Pi }$ can be computed from ${\tt M}_{\tt\Pi}^\top {\tt\Pi}=0$. Define ${\bf M}_{\tt\Pi}$ as the right null space of ${\tt\Pi}^\top$ ($4 \times 3$ matrix). ${\bf M}_{\tt\Pi}$ represents 3 supporting points for the plane ${\tt\Pi }$ and let ${\bf m}_{\tt\Pi i}={\bf P}_i {\bf M}_{\tt\Pi}$ be the corresponding image projections. Define the homography ${\bf H}_{i{\tt\Pi}}={\bf m}_{\tt\Pi i}^{-1}$, then the 3D reconstruction of image points located in the plane ${\tt\Pi }$ are obtained as follows:

$${\tt M}_i={\bf M}_{\tt\Pi} {\bf H}_{i{\tt\Pi}} {\tt m}_i$$ (E11)

Similarly, a feature ${\tt m}_j$ seen in view $j (>i)$ can be reconstructed as:

$${\tt M}_j={\bf M}_{\tt\Pi} {\bf H}_{i{\tt\Pi}} ({\bf H}^{\tt\Pi}_{ij})^{-1} {\tt m}_j$$ (E12)

where ${\bf H}^{\tt\Pi}_{ij}={\bf H}^{\tt\Pi}_{i(i+1)} \ldots {\bf H}^{\tt\Pi}_{(j-1)j}$.