Epipolar geometry and stereo vision


By Kendall Merriman



Project:

My original goal was much more ambitious, but in the end, numerous setbacks effectively made the project scope much smaller then the initial goal. The following tasks were accomplished:
The fundamental matrix is generated by using the basic 8-point algorithm, which requires only 8 pairs of matched points. However, I found that this method, though correct, produced matrices that were not entirely useful. As such, I decided to try and get the matrix that minimized the error over as many matches as the user provides, allowing for more accurate epipolar lines.

For more information on the fundamental matrix and epipolar lines, see the references.

This project uses DirectShow to connect to the cameras and grab frames, which are processed to produce the results shown. Once the images are pulled, they are displayed to the user, who, after selecting enough matches between the images, can generate the matrix and start exploring the scene, mainly by cricking on things. The following options are available:

The application will store the matrix it calculates automatically, so if the setup has not changed, it will not need to be recalculated. The user will be prompted about this at startup.

System Requirements:

Results:


An example of an epipolar line. The line shown in the right window is for the point in the left window. The point in the right is the match chosen for that point in the left.

An example of selecting objects. The red and green squares indicate the red and green balls, which were selected to add to the depth display. Notice that the red is in front of the green, as they are in the image.

Download here

References

Hartley, R.~I. and Zisserman, A., Multiple View Geometry in Computer Vision, Cambridge University Press, 2004
Wikipedia on Epipolar Geometry
Wikipedia on The Fundamental Matrix