Lens Distortion Non-linear Distortion and Perspective Distortion

Question

Hello everyone, I'm a member of the vision team in the Intelligent Ground Vehicle team in Cal State Northridge, an autonomous robot that uses real time feed and laser data as sensors.
So we need a fully undistorted video feed for us to do image analysis with the lowest processing time.
We have a distortion situation with our current vision system where we have to fix Non-linear Distortion due to the use of a wide lens ( About 144 Deg ) and also Perspective Distortion of the projected image. The camera is mounted on top of our robot with an angle with the ground (about 45 Deg).
I have been using labview for about 6 months, and have been trying to correct this distortion using Vision Assistant program to generate VI's.
First I'll list my attempts here and then I will direct some questions.
First attempt, a picture of a dot grid was taken with our camera and that picture was correct by Vision Assistant's nonlinear image calibration. Then a VI was generated by Vision Assistant, and we changed in the input from images to video.
This method worked but not accurately.
Second attempt, we tried to remap the image pixels using a correction function generated through Excel, this method did not work accurately either. I'm not even sure how to do this again since it was done in the past.
So my questions are here:
1) what's the best approach to correct the distortion in our camera? and how please.
2) what are we doing wrong/right in general?
3) Generally, how would I improve any vision VI to reach faster processing times?
Everyone's help is very much appreciated,
Omar
Cal State Northridge
The Intelligent Ground Vehicle team
[email protected] 

Answer 1

I'd like to correct the information provided by Bruce. It was correct up til Vision Development Module 2010. You need to update, Bruce ;-)
In Vision Development Module 2011, we improved the accuracy and precision of our grid calibration, by computing a lens distortion model (Division or Polynomial Model), that take into account radial and tangential non-linear distortion of the lens. The new algorithm also allows to compute the internal parameters of the lens (focal length and optical center), that you can use to determine the relationship of the camera to the object under inspection.
The microplane algorithm that Bruce described is still availale, as it can be used to solve applications in which the object you're trying to calibrate is non planar, and can be calibrated by wrapping the grid around the object.
A calibration will be valid for a specific perspective plane, and will need to be recomputed if the camera moves or object is in a different plane.
Typically, the calibration process is an offline process. For performance reasons, you don't necessarily want to correct the entire image (although VDM provides you with that function). As Bruce mentioned, most algorithms, like edge detection, pattern matching etc, return their data both in pixels and real-word data, talking into account the calibration information that you learned. For the few algorithms that only provide pixel results, you can use VIs to convert Pixels to Real-World. This operation is fast.
What type of features are you trying to measure in the image? Can it be solved in 2D, or do you need 3D information? How many cameras do you have on the robot?
-Christophe