CUDA ISP for NVIDIA Jetson: GStreamer usage
| CUDA ISP for NVIDIA Jetson | |
|---|---|
 | |
| CUDA ISP for NVIDIA Jetson Basics | |
|
|
|
| Getting Started | |
|
|
|
| User Manual | |
|
|
|
| GStreamer | |
|
|
|
| Examples | |
|
|
|
| Performance | |
|
|
|
| Contact Us | |
|
|
General
 | lleon95: Please, provide a brief introduction to these plugins. For instance, "CUDA ISP provides GStreamer plugin to integrate image signal processing to your pipeline... facilitating..." (please remove this box when addressed) |
CUDA ISP provides GStreamer plugins that facilitate the usage of the library with GStreamer pipelines. CUDA ISP provides three different GStreamer elements for image signal processing. to The three elements are:
- cudashift
- cudadebayer
- cudaawb
In the following sections, you can see the characteristics of the elements and examples of how to use them. You can see more information about the algorithms these elements apply in CUDA ISP Basics
GStreamer Elements
cudashift
| lleon95: I think we have data replication. We have the information about the properties and capabilities here (please remove this box when addressed) |
This element applies the shifting algorithm. For more information about the element properties and capabilities, check here: cudashift element
| lleon95: I think what we really want in this section is to illustrate how to use the element. In this case, we want to explain: how to choose a proper value of shift (do you remember the wiki about offsets? we need some of this info here). Then, how to apply this knowledge to this. (please remove this box when addressed) |
| anonymous: lleon95 (please remove this box when addressed) |
The following pipeline receives a bayer 10 image with 4k resolution from a camera sensor, applies a shifting with a value of 5 and outputs a bayer 8 image.
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, format=rggb' ! cudashift shift=5 ! fakesink
| lleon95: A diagram would be really useful. A box with some formats coming and Bayer 8 as output (please remove this box when addressed) |
cudadebayer
This element applies the debayering algorithm. For more information about the element properties and capabilities, check here: cudadebayer element
This element has two output options: RGB or I420. Next, you will see examples on how to use the cudadebayer element.
The following pipeline receives a bayer 10 image with 4K resolution from a camera sensor and outputs an RGB image.
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! fakesink
This next pipeline receives a bayer 10 image with 4K resolution and outputs an I420 image. You can see that source caps were added to choose which output format you desire,in this case it would be I420.
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! 'video/x-raw, format=I420' ! fakesink
This next pipeline receives a bayer 10 image with 4K resolution and outputs an RGB image with shifting added. In this case the output format is RGB.
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer shift=5 ! 'video/x-raw, format=RGB' ! fakesink
cudaawb
| lleon95: Same comments (please remove this box when addressed) |
This element applies de auto-white balance algorithm. It allows the following image formats:
- Input
- RGB
- I420
- Output
- RGB
- I420
The following pipeline receives a bayer10 image with 4K resolution from a camera sensor and outputs an RGB image after applying the auto-white balance algorithm. This element requires an RGB image input. So, when using a camera sensor, before using the cudaawb element, we should add the cudadebayer element to obtain the RGB image that the AWB element will use as input.
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb ! fakesink
| lleon95: You need to rephrase it. The sentences is intricate (please remove this box when addressed) |
The following pipeline receives a bayer10 image with 4K resolution from a camera sensor and outputs an I420 image after applying the auto-white balance algorithm.
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb ! 'video/x-raw, format=I420' ! fakesink
| lleon95: We are missing a case where we don't need debayering (please remove this box when addressed) |
Use all!
| lleon95: Some questions that we need to answer: 1) Can we use cudaawb alone? 2) When do we need to use cudashift and cudadebayer and when do we need to use cudadebayer only? Some answers rely on latency and framerate. Splitting the elements will lead to more latency but to higher framerate. I strongly recommend you to write a section of "Use Cases" (please remove this box when addressed) |
This pipeline receives a bayer 10 image with 4K resolution and outputs a RGB image, using the three elements.
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudashift shift=5 ! cudadebayer ! cudaawb ! fakesink
