CUDA ISP for NVIDIA Jetson/Examples/GStreamer usage

CUDA ISP for NVIDIA Jetson
CUDA ISP for NVIDIA Jetson Basics
Getting Started
Evaluating the CUDA ISP Getting the code Building CUDA ISP for NVIDIA Jetson Building custom v4l2src element
User Manual
GStreamer
cudadebayer cudashift cudaawb
Examples
API usage GStreamer usage
Performance
Library GStreamer Elements
Contact Us

General

CUDA ISP provides three different GStreamer elements to apply the different algorithms to raw image data coming from a camera sensor. The 3 elements are:

• cudashift
• cudadebayer
• awb

In the following sections you can see characteristics about the elements and examples on how to use them. You can see more information about the algorithms this elements apply in CUDA ISP Basics

cudashift

This element applies the shifting algorithm. It allows the following image formats:

• Input:
  • Bayer 10/12/14/16
• Output
  • Bayer 8

The cudashift elements has a property:

• shift: number of bits to shift right per pixel.

The following pipeline receives a bayer 10 image with 4k resolution from a camera sensor, applies a shifting with a value of 5 and outputs a bayer 8 image.

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, format=rggb' ! cudashift shift=5 ! fakesink

Error creating thumbnail: Unable to save thumbnail to destination

cudadebayer

This element applies the debayering algorithm. It is also capable of applying the shifting algorithm. This element allows the following image formats:

• Input
  • Bayer 10/12/14/16
• Output
  • RGB
  • I420

This element has a shifting property.

• shift: number of bits to shift right per pixel.

This element has two output options: RGB or I420. You can choose which output option you want by modifying the format name in the source caps. The following pipeline receives a bayer 10 image with 4K resolution and outputs an RGB image.

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! fakesink

This next pipeline receives a bayer 10 image with 4K resolution and outputs an I420 image. You can see that source caps were added to choose which output format you desire,in this case it would be I420.

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! 'video/x-raw, format=I420' ! fakesink

This next pipeline receives a bayer 10 image with 4K resolution and outputs an RGB image with shifting added. In this case the output format is RGB.

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer shift=5 ! 'video/x-raw, format=RGB' ! fakesink

Error creating thumbnail: Unable to save thumbnail to destination

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! queue ! videoconvert ! 'video/x-raw' ! videoscale ! 'video/x-raw, width=640, height=480' ! ximagesink display=localhost:10.0 sync=false

GST_DEBUG=WARNING,*debayer*:LOG gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, format=rggb' ! cudashift shift=5 ! 'video/x-bayer, bpp=8, format=rggb' ! cudadebayer ! 'video/x-raw, format=RGBA' ! fakesink

cudaawb

This element applies de auto-white balance algorithm. It allows the following image formats:

• Input
  • RGB
• Output
  • RGB
  • I420

The following pipeline receives an bayer10 image with 4K resolution and output an RGB image after applying the auto-white balance algorithm. This element requires an RGB image input. So before using the cudaawb element we should add the cudadebayer element to obtain the RGB image that the AWB element will use as input.

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb ! fakesink

Error creating thumbnail: Unable to save thumbnail to destination

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudashift shift=0 ! cudadebayer ! cudaawb ! queue ! videoconvert ! 'video/x-raw' ! videoscale ! 'video/x-raw, width=640, height=480' ! ximagesink display=localhost:10.0 sync=false