Gary Bishop

F06 Calibration Math

A short overview of how the camera calibration math works.

We want a 3×4 matrix M such that:

$ [[ U t ],[V t],[t]] = M [ [ x ],[ y ],[ z ],[ 1 ]] $

given a large set of corresponding $U,V$ and $x,y,z$ values. Let’s make up names for the elements of M such as:

$ M = [ [ a, b, c, d ], [e, f, g, h], [i, j, k, l] ] $

Evaluating the matrix product we have:

$ U t = a x + b y + c z + d $, $ V t = e x + f y + g z + h $, and $ t = i x + j y + k z + l $. So dividing by $t$ we get

$ U = (a x + b y + c z + d) / (i x + j y + k z + l) $ and $ V = (e x + f y + g z + h) / (i x + j y + k z + l) $

Multiply both sides by the denominator ($t$).

$ i x U + j y U + k z U + l U = a x + b y + c z + d $ and $ i x V + j y V + k z V + l V = e x + f y + g z + h $.

Rearranging terms we have:

$ -a x – b y – c z – d + i x U + j y U + k z U + l U = 0 $ and $ – e x – f y – g z – h + i x V + j y V + k z V + l V = 0 $

Remember our knowns are $U,V,x,z,y$ and the unknowns are $a,b,c,d,e,f,g,h,i,j,k,l$. Writing this as a matrix equation we get:

$ [ [ -x,-y,-z,-1,0,0,0,0,x U,y U,z U,U],[0,0,0,0,-x,-y,-z,-1,x V,y V,z V,V] ] [[ a],[ b],[ c],[ d],[ e],[ f],[ g],[ h],[ i],[ j],[ k],[ l]] = 0 $

I have only shown 2 rows above. You’d get 2 rows for each set of $U,V$ and corresponding $x,y,z$ you are given for input. So for $N$ measured points you’d have a $2 N$ by 9 matrix, multiplying a 12 by 1 vector of unknowns to produce 0.

Unfortunately, solving such a homogeneous system of equations is hard because the trivial zero solution hides the least-squares solution we’d like to find. We can fix this by assuming a value for one of the values in the matrix. We can do this because any scaling of the matrix gets divided out. So I’ll say that $k=1$. This is safe as long as $k$ doesn’t turn out to be zero. In this came I’m sure because of the experimental setup that it isn’t zero. But if there was some chance it could be, then I’d have to do some case analysis.

Setting $k=1$ I now have a constant term that I can move over to the right side of the equation. The new equation will look like this:

$ [ [ -x,-y,-z,-1,0,0,0,0,x U,y U,U],[0,0,0,0,-x,-y,-z,-1,x V,y V,V] ] [[ a],[ b],[ c],[ d],[ e],[ f],[ g],[ h],[ i],[ j],[ l]] = [ [ -z U ],[ -z V ] ]$

I can use a variety of methods to solve this equation (given more rows from many more measurements) for the values that give me the best fit in the least-squares sense.

Given the values for $a$ through $l$ and knowing that $k=1$ I can write my matrix $M$.