Refining and extending structure

The structure is refined using an iterated linear reconstruction algorithm on each point. Equation 3.8 can be rewritten to become linear in ${\tt M}$:

$$\begin{array}{ccc} {\tt P}_3 {\tt M}x - {\tt P}_1 {\tt M} & ... ...0\\ {\tt P}_3 {\tt M}y - {\tt P}_2 {\tt M} & = & 0 \end{array}$$ (E6)

with ${\tt P}_i$ the $i$-th row of ${\bf P}$ and $(x,y)$ being the image coordinates of the point. An estimate of ${\tt M}$ is computed by solving the system of linear equations obtained from all views where a corresponding image point is available. To obtain a better solution the criterion $\sum D({\bf P}{\tt M},{\tt m})$ should be minimized. This can be approximately obtained by iteratively solving the following weighted linear equations (in matrix form):

$$\frac{1}{{\tt P}_3 \tilde{\tt M}} \left[\begin{array}{c} {\t... ...}_1 \\ {\tt P}_3 y - {\tt P}_2 \end{array} \right] {\tt M} =0$$ (E7)

where ${\tilde{\tt M}}$ is the previous solution for ${\tt M}$. This procedure can be repeated a few times. By solving this system of equations through SVD a normalized homogeneous point is automatically obtained. If a 3D point is not observed the position is not updated. In this case one can check if the point was seen in a sufficient number of views to be kept in the final reconstruction. This minimum number of views can for example be put to three. This avoids to have an important number of outliers due to spurious matches.

Of course in an image sequence some new features will appear in every new image. If point matches are available that were not related to an existing point in the structure, then a new point can be initialized as in section 5.1.2.

After this procedure has been repeated for all the images, one disposes of camera poses for all the views and the reconstruction of the interest points. In the further modules mainly the camera calibration is used. The reconstruction itself is used to obtain an estimate of the disparity range for the dense stereo matching.