non-linear self-calibration refinement

Before going for a bundle-adjustment it can still be interesting to refine the linear self-calibration results through a minimization that only involves the camera projection matrices (although in practice this is often not necessary since the results of the linear computation are good enough to allow the convergence of the bundle adjustment to the desired solution). Let us define the functions $f(.),r(.),u(.),v(.)$ and $s(.)$ that respectively extract the focal length, aspect ratio, coordinates of the principal point and skew from a projection matrix (in practice this is done based on QR-decomposition). Then our expectations for the distributions of the parameters could be translated to the following criterion (for a projection matrix normalized as in Equation (6.12)):

$${\cal C}({\bf T})=\sum_i \left( \frac{\log(f({\bf P}_i{\bf T... ...2}{0.1^2} + \frac{s({\bf P}_i{\bf T}^{-1})^2}{0.01^2} \right)$$ (F18)

Note that since $f$ and $r$ indicate relative and not absolute values, it is more meaningful to use logarithmic values in the minimization. This also naturally avoids that the focal length would collapse to zero for some degenerate cases. In this criterion ${\bf T}$ should be parametrized with 8 parameters and initialized with the solution of the linear algorithm. The refined solution for the transformation can then be obtained as:

$${\bf T}_{opt} = \arg\min \, {\cal C}({\bf T})$$ (F19)

Some terms can also be added to enforce constant parameters, e.g. $\frac{(\log(f({\bf P}_i{\bf T}^{-1}))-\overline{\log {f}})^2}{\log(0.1)^2}$ with $\overline{\log f}$ the average logarithm of the observed focal length. The metric structure and motion is then obtained as

$${\bf P}_M={\bf P}{\bf T}^{-1} \mbox{ and } {\tt M}_M={\bf T}{\tt M}$$ (F20)

This result can then further be refined through bundle adjustment. In this case the constraints on the intrinsics should also be enforced during the minimization process. For more details the reader is referred to [158].