Metric bundle adjustment

For high accuracy the recovered metric structure should be refined using a maximum likelihood approach such as the bundle adjustment (see Appendix A). In this case, however, the metric structure and not the projective structure is retrieved. This means that the camera projection matrices should be parametrized using intrinsic and extrinsic parameters (and not in homogeneous form as in the projective case). If one assumes that the error is only due to mislocalization of the image features and that this error is uniform and normally distributed^F5, the bundle adjustment corresponds to a maximum likelihood estimator. For this to be satisfied the camera model should be general enoguh so that no systematic errors remain in the data (e.g. due to lens distortion). In these circumstances the maximum likelihood estimation corresponds to the solution of a least-squares problem. In this case a criterion of the type of equation (6.2) should be minimized:

$${\cal C}_{ML}({\tt M}_l, {\bf K}_i, {\bf R}_i, {\tt t}_i) = ... ...\frac{{\tt P}_{i2}{\tt M}_l}{{\tt P}_{i3}{\tt M}_l})^2 \right)$$ (F21)

where $I_i$ is the set of indices corresponding to the points seen in view $i$ and ${\bf P}_i \equiv \left[ {\tt P}_{i1}^\top {\tt P}_{i2}^\top {\tt P}_{i3}^\top \right]^\top = {\bf K}_i [ {\bf R}_i^\top \vert \mbox{-}{\bf R}_i^\top{\tt t}_i ]$. This criterion should be extended with terms that reflect the (un)certainty on the intrinsic camera parameters. This would yield a criterion of the following form:

$$\begin{array}{rcl} {\cal C}'_{ML}({\tt M}_l, {\bf K}_i, {\bf... ...n_{i=1} \sum_{k=1}^m \lambda_k C_{ki}({\bf K}_i)^2 \end{array}$$ (F22)

with $\lambda_k$ a regularization factor and $C_{ki}({\bf K}_i)$ representing the constraints on the intrinsic camera parameters, e.g. $C_{1i}={f_x}_i-{f_y}_i$ (known aspect ratio), $C_{2i}={u_x}_i$ (known principal point) or ${f_x}_i-f_x$ (constant focal length). The values of the factors $\lambda_k$ depend on how strongly the constraints should be enforced.