Here we present a simple algorithm for rectification which can deal with all possible camera geometries. Only the oriented fundamental matrix is required. All transformations are done in the images. The image size is as small as can be achieved without compressing parts of the images. This is achieved by preserving the length of the epipolar lines and by determining the width independently for every half epipolar line.
For traditional stereo applications the limitations of standard rectification algorithms are not so important. The main component of camera displacement is parallel to the images for classical stereo setups. The limited vergence keeps the epipoles far from the images. New approaches in uncalibrated structure and motion as presented in this text however make it possible to retrieve 3D models of scenes acquired with hand-held cameras. In this case forward motion can no longer be excluded. Especially when a street or a similar kind of scene is considered.