NVIDIA VPI GStreamer Plug-in - Examples - Undistort

GstVPI
GstVPI Basics
Getting Started
Evaluating GstVPI Getting the Code Building the Plugin
Examples
Gaussian Filter Undistort Harris Keypoints Detector KLT Tracker Perspective Warp Overlay Videoconvert Videoscale Convolution Separable Convolution
Performance
Nano TX1 TX2 Xavier
Contact Us

Introduction

The vpiundistort element applies the Lens Distortion Correction algorithm from the NVIDIA VPI library to a video stream. In order to use this algorithm, you must provide the camera distortion parameters. See the next section to learn how to obtain them.

How to Obtain the Camera Calibration Parameters

The vpiundistort element requires the calibration parameters and the distortion coefficients of the input camera. In order to extract them, we provide a tool that will simplify this process. Check this section to learn how to use it.

1. Go into the tests/examples/vpiundistort directory for GstVPI professional version:

cd tests/examples/vpiundistort

Or into <path-to-eval>/tool for GstVPI evaluation version:

cd <path-to-eval>/tool

2. Open the settings.json file in this directory, and edit the following lines:

"cameraWidth": 1280,
"cameraHeight": 800,
"chessboardInnerCornersWidth": 6,
"chessboardInnerCornersHeight": 9,
"camIndex": 0,

with the following values:

• cameraWidth: your camera resolution width
• cameraHeight: your camera resolution height
• chessboardInnerCornersWidth: Number of inner corners in the calibration pattern counted horizontally
• chessboardInnerCornersHeight: Number of inner corners in the calibration pattern counted vertically
• camIndex: The identifier number of your camera following the video word when you run ls /dev/video*

3. Run the calibrator. The following options are available for the script:

• -c: Execute the calibration process.
• -p $IMAGE_PATH: Path to store the calibration images. Default: imgs/calibration.
• -r: Remove the images from the $IMAGE_PATH.
• -s $SETTINGS_PATH: Path to the settings file. Default: settings.json.
• -t: Run a test routine to see the calibration results.
• -v: Use the vl42 plugin to capture images. By default, the NVIDIA plugins will be used.
• --cli: Perform calibration with images located at $IMAGE_PATH. No graphical user interface mode to capture images.

The easiest way to calibrate the camera is to run:

python3 calibration_tool.py -c
# Or for evaluation version
python3 calibration_tool.pyc -c

But feel free to explore the other options in case you need them.

• GUI mode:

- If you did not use the --cli option, a window with the camera image will appear. You must show the calibration pattern to the camera until the frame turns green, which means that the pattern has been detected (see animation below).

- Move the pattern to fully cover the area and tilt it in both horizontal and vertical directions, as shown in the animation, to obtain a good calibration.

Error creating thumbnail: Unable to save thumbnail to destination

- Once the saved images counter has reached 40 ~ 50 images, you may press the q key to proceed to parameters calculation.

• CLI mode:

- If you chose to use the --cli mode with pre-captured images, make sure the calibration pattern is in different positions and visible in all of them. You may take as a reference the images displayed in the animation above. These images must be in .png format.

- The parameters calculation will begin immediately.

4. You will obtain a string with the parameters ready to be used in a GStreamer pipeline as properties of the vpiundistort element. This will be shown in your console in a format like this:

  ============================================= 
 |  The following variables can be used as the |
 | properties of the vpiundistort element.     |
 | Use the parameters of your preferred model.  |
  ============================================= 
 
 
# ======= 
# FISHEYE 
# ======= 
model=fisheye k1=0.053 k2=0.013 k3=0.001 k4=-0.002 intrinsic="<<305.950, 0.000, 654.029>, <0.000, 305.226, 415.647>>"
 
# ============= 
# BROWN_CONRADY 
# ============= 
model=polynomial k1=0.522 k2=-0.016 p1=-0.000 p2=0.000 k3=-0.002 k4=0.801 k5=0.052 k6=-0.008 intrinsic="<<305.950, 0.000, 654.029>, <0.000, 305.226, 415.647>>"

Element properties

• backend
  

Backend to use to execute VPI algorithms. Available options:

cpu: CPU backend

cuda: CUDA backend

pva: PVA backend (Xavier only)

vic: VIC backend

Flags: readable, writable
Default: "cuda"

• extrinsinc
  

3x4 matrix resulting of concatenation of 3x3 rotation matrix with 3x1 vector containing the position of the origin world coordinate system expressed in coordinates of camera-centered system.
Type: GstValueArray of GValues of type GstValueArray
Flags: readable, writable
Default: <<1.0,0.0,0.0,0.0>,<0.0,1.0,0.0,0.0>,<0.0,0.0,1.0,0.0>>

• intrinsic
  

2x3 matrix with the parameters: [[fx,s,cx],[0,fy,cy]] where:
- fx, fy: focal length in pixels in x and y direction
- s: skew
- cx, cy: the principal point in x and y direction
If not provided, a default calibration matrix will be created.
Type: GstValueArray of GValues of type GstValueArray
Flags: readable, writable
Default: <<f, 0.0, width/2>,<0.0, f, height/2>>. Where [math]\displaystyle{ f = 7.5 * width / 22.2 }[/math]

• interpolator
  

Interpolation method to be used. Available options:

nearest: Nearest neighbor interpolation

linear: Fast linear interpolation

catmull: Fast Catmull-Rom cubic interpolation

Flags: readable, writable
Default: "catmull"

• model
  

Type of distortion model to use. Available options:

fisheye: Fisheye distortion model

polynomial: Polynomial (Brown-Conrady) distortion model

Flags: readable, writable
Default: "fisheye"

• mapping
  

Type of fisheye lens mapping types (only for fisheye model). Available options:

equidistant: Specifies the equidistant fisheye mapping

equisolid: Specifies the equisolid fisheye mapping

ortographic: Specifies the ortographic fisheye mapping

stereographic: Specifies the stereographic fisheye mapping

Flags: readable, writable
Default: "equidistant"

• k1
  

Distortion coefficient 1 of the chosen distortion model (fisheye or polynomial).
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

• k2
  

Distortion coefficient 2 of the chosen distortion model (fisheye or polynomial).
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

• k3
  

Distortion coefficient 3 of the chosen distortion model (fisheye or polynomial).
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

• k4
  

Distortion coefficient 4 of the chosen distortion model (fisheye or polynomial).
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

• k5
  

Distortion coefficient 5. Only for polynomial model.
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

• k6
  

Distortion coefficient 6. Only for polynomial model.
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

• p1
  

Tangential distortion coefficient 1. Only for polynomial model.
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

• p2
  

Tangential distortion coefficient 2. Only for polynomial model.
Type: Double
Range: -1,797693e+308 - 1,797693e+308
Flags: readable, writable
Default: 0

Example

The following example pipes will help you correct the distortion from your camera lens. You may modify the properties values according to the information above. In these examples we provide some distortion coefficients and calibration matrices for the camera we used, however, you should modify them according to your camera's parameters (see How to Obtain the Camera Calibration Parameters).

Polynomial Distortion

gst-launch-1.0 v4l2src ! nvvidconv bl-output=false ! 'video/x-raw(memory:NVMM)' ! vpiupload ! vpiundistort k1=0.52 k2=-0.02 p1=0 p2=0 k3=-0.001 k4=0.8 k5=0.05 k6=-0.008 model=polynomial intrinsic='<<305.95,0.0,654.03>,<0.0,305.23,415.65>>' ! vpidownload ! nvvidconv ! nvoverlaysink

Fisheye Distortion

gst-launch-1.0 v4l2src ! nvvidconv bl-output=false ! 'video/x-raw(memory:NVMM)' ! vpiupload ! vpiundistort k1=0.05 k2=0.01 k3=0 k4=0 model=fisheye intrinsic='<<305.95,0.0,654.03>,<0.0,305.23,415.65>>' mapping=equisolid ! vpidownload ! nvvidconv ! nvoverlaysink