Difference between revisions of "CUDA ISP for NVIDIA Jetson/Examples/GStreamer usage"

From RidgeRun Developer Connection
Jump to: navigation, search
(→‎cudaawb)
Line 4: Line 4:
  
 
{{DISPLAYTITLE:CUDA ISP for NVIDIA Jetson: GStreamer usage|noerror}}
 
{{DISPLAYTITLE:CUDA ISP for NVIDIA Jetson: GStreamer usage|noerror}}
 +
 +
{{Review|Please, make it second level heading|lleon95}}
  
 
=== General ===
 
=== General ===
−
CUDA ISP provides three different GStreamer elements to apply the different algorithms to raw image data coming from a camera sensor. The 3 elements are:
+
 
 +
{{Review|Please, provide a brief introduction to these plugins. For instance, "CUDA ISP provides GStreamer plugin to integrate image signal processing to your pipeline... facilitating..."|lleon95}}
 +
 
 +
CUDA ISP provides three different GStreamer elements to apply the different algorithms to raw image data coming from a camera sensor. The three elements are:
 +
 
 
* cudashift
 
* cudashift
 
* cudadebayer
 
* cudadebayer
−
* awb
+
* cudaawb
−
In the following sections you can see characteristics about the elements and examples on how to use them. You can see more information about the algorithms this elements apply in [[CUDA_ISP_for_NVIDIA_Jetson/CUDA_ISP_for_NVIDIA_Jetson_Basics|CUDA ISP Basics]]
+
 
 +
In the following sections, you can see the characteristics of the elements and examples of how to use them. You can see more information about the algorithms these elements apply in [[CUDA_ISP_for_NVIDIA_Jetson/CUDA_ISP_for_NVIDIA_Jetson_Basics|CUDA ISP Basics]]
 +
 
 +
{{Review|Add a second level heading: GStreamer Elements|lleon95}}
  
 
=== cudashift ===
 
=== cudashift ===
 +
 +
{{Review|I think we have data replication. We have the information about the properties and capabilities [https://ridgerun.github.io/libcudaisp-docs/md__builds_ridgerun_rnd_libcudaisp_docs_cudadebayer.html here]|lleon95}}
 +
 
This element applies the shifting algorithm. It allows the following image formats:
 
This element applies the shifting algorithm. It allows the following image formats:
 
* Input:
 
* Input:
Line 18: Line 30:
 
* Output
 
* Output
 
** Bayer 8
 
** Bayer 8
−
The cudashift elements has a property:
+
The cudashift details has a property:
 
* shift: number of bits to shift right per pixel.  
 
* shift: number of bits to shift right per pixel.  
 +
 +
{{Review|I think what we really want in this section is to illustrate how to use the element. In this case, we want to explain: how to choose a proper value of shift (do you remember the wiki about offsets? we need some of this info here). Then, how to apply this knowledge to this.|lleon95}}
 +
 +
{{Review|Another thing we are missing is the fact that we need a patch to make v4l2src work with bayer != 8|lleon95}}
  
 
The following pipeline receives a bayer 10 image with 4k resolution from a camera sensor, applies a shifting with a value of 5 and outputs a bayer 8 image.  
 
The following pipeline receives a bayer 10 image with 4k resolution from a camera sensor, applies a shifting with a value of 5 and outputs a bayer 8 image.  
Line 30: Line 46:
 
<br>
 
<br>
 
<br><br>
 
<br><br>
 +
 +
{{Review|A diagram would be really useful. A box with some formats coming and Bayer 8 as output|lleon95}}
  
 
=== cudadebayer ===
 
=== cudadebayer ===
 +
 +
{{Review|Same comments|lleon95}}
 +
 
This element applies the debayering algorithm. It is also capable of applying the shifting algorithm. This element allows the following image formats:
 
This element applies the debayering algorithm. It is also capable of applying the shifting algorithm. This element allows the following image formats:
 
* Input
 
* Input
Line 41: Line 62:
 
This element has a shifting property.
 
This element has a shifting property.
 
* shift: number of bits to shift right per pixel.
 
* shift: number of bits to shift right per pixel.
 +
 +
{{Review|These pipelines are well placed! Thanks|lleon95}}
  
 
This element has two output options: RGB or I420. You can choose which output option you want by modifying the format name in the source caps. The following pipeline receives a bayer 10 image with 4K resolution from a camera sensor and outputs an RGB image.
 
This element has two output options: RGB or I420. You can choose which output option you want by modifying the format name in the source caps. The following pipeline receives a bayer 10 image with 4K resolution from a camera sensor and outputs an RGB image.
Line 62: Line 85:
  
 
=== cudaawb ===
 
=== cudaawb ===
 +
 +
{{Review|Same comments|lleon95}}
 +
 
This element applies de auto-white balance algorithm. It allows the following image formats:
 
This element applies de auto-white balance algorithm. It allows the following image formats:
 
* Input
 
* Input
Line 69: Line 95:
 
** RGB
 
** RGB
 
** I420
 
** I420
−
The following pipeline receives an bayer10 image with 4K resolution from a camera sensor and outputs an RGB image after applying the auto-white balance algorithm. This element requires an RGB image input. So, when using a camera sensor, before using the cudaawb element we should add the cudadebayer element to obtain the RGB image that the AWB element will use as input.  
+
The following pipeline receives a bayer10 image with 4K resolution from a camera sensor and outputs an RGB image after applying the auto-white balance algorithm. This element requires an RGB image input. So, when using a camera sensor, before using the cudaawb element, we should add the cudadebayer element to obtain the RGB image that the AWB element will use as input.  
  
 
<source lang=bash>
 
<source lang=bash>
Line 80: Line 106:
 
<br><br>
 
<br><br>
  
−
The following pipeline receives an bayer10 image with 4K resolution from a camera sensor and outputs an I420 image after applying the auto-white balance algorithm.
+
{{Review|You need to rephrase it. The sentences is intricate|lleon95}}
 +
 
 +
The following pipeline receives a bayer10 image with 4K resolution from a camera sensor and outputs an I420 image after applying the auto-white balance algorithm.
 
<source lang=bash>
 
<source lang=bash>
 
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb ! 'video/x-raw, format=I420' ! fakesink
 
gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb ! 'video/x-raw, format=I420' ! fakesink
 
</source>
 
</source>
 +
 +
{{Review|We are missing a case where we don't need debayering|lleon95}}
  
 
<br>
 
<br>
Line 92: Line 122:
  
 
=== Use all! ===
 
=== Use all! ===
 +
 +
 +
{{Review|Some questions that we need to answer: 1) Can we use cudaawb alone? 2) When do we need to use cudashift and cudadebayer and when do we need to use cudadebayer only? Some answers rely on latency and framerate. Splitting the elements will lead to more latency but to higher framerate. I strongly recommend you to write a section of "Use Cases"|lleon95}}
  
 
This pipeline receives a bayer 10 image with 4K resolution and outputs a RGB image, using the three elements.  
 
This pipeline receives a bayer 10 image with 4K resolution and outputs a RGB image, using the three elements.  
Line 102: Line 135:
 
<br>
 
<br>
 
<br><br>
 
<br><br>
−
 
−
 
−
 
−
 
−
 
−
 
−
 
−
 
−
 
−
 
  
  

Revision as of 11:53, 16 March 2023


  Index  


CUDA ISP for NVIDIA Jetson
Cudaisp-transparent.png
CUDA ISP for NVIDIA Jetson Basics
Getting Started
User Manual
GStreamer
Examples
Performance
Contact Us





‎

‎

‎

General

‎

‎

‎

CUDA ISP provides three different GStreamer elements to apply the different algorithms to raw image data coming from a camera sensor. The three elements are:

  • cudashift
  • cudadebayer
  • cudaawb

In the following sections, you can see the characteristics of the elements and examples of how to use them. You can see more information about the algorithms these elements apply in CUDA ISP Basics

‎

‎

‎

cudashift

‎

‎

‎

This element applies the shifting algorithm. It allows the following image formats:

  • Input:
    • Bayer 10/12/14/16
  • Output
    • Bayer 8

The cudashift details has a property:

  • shift: number of bits to shift right per pixel.

‎

‎

‎

‎

‎

‎

The following pipeline receives a bayer 10 image with 4k resolution from a camera sensor, applies a shifting with a value of 5 and outputs a bayer 8 image. 

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, format=rggb' ! cudashift shift=5 ! fakesink




Error creating thumbnail: Unable to save thumbnail to destination




‎

‎

‎

cudadebayer

‎

‎

‎

This element applies the debayering algorithm. It is also capable of applying the shifting algorithm. This element allows the following image formats:

  • Input
    • Bayer 10/12/14/16
  • Output
    • RGB
    • I420

This element has a shifting property.

  • shift: number of bits to shift right per pixel.

‎

‎

‎

This element has two output options: RGB or I420. You can choose which output option you want by modifying the format name in the source caps. The following pipeline receives a bayer 10 image with 4K resolution from a camera sensor and outputs an RGB image. 

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! fakesink

This next pipeline receives a bayer 10 image with 4K resolution and outputs an I420 image. You can see that source caps were added to choose which output format you desire,in this case it would be I420. 

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! 'video/x-raw, format=I420' ! fakesink

This next pipeline receives a bayer 10 image with 4K resolution and outputs an RGB image with shifting added. In this case the output format is RGB. 

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer shift=5 ! 'video/x-raw, format=RGB' ! fakesink




Error creating thumbnail: Unable to save thumbnail to destination




cudaawb

‎

‎

‎

This element applies de auto-white balance algorithm. It allows the following image formats:

  • Input
    • RGB
    • I420
  • Output
    • RGB
    • I420

The following pipeline receives a bayer10 image with 4K resolution from a camera sensor and outputs an RGB image after applying the auto-white balance algorithm. This element requires an RGB image input. So, when using a camera sensor, before using the cudaawb element, we should add the cudadebayer element to obtain the RGB image that the AWB element will use as input. 

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb ! fakesink




Error creating thumbnail: Unable to save thumbnail to destination




‎

‎

‎

The following pipeline receives a bayer10 image with 4K resolution from a camera sensor and outputs an I420 image after applying the auto-white balance algorithm. 

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudadebayer ! cudaawb ! 'video/x-raw, format=I420' ! fakesink

‎

‎

‎




Error creating thumbnail: Unable to save thumbnail to destination




Use all!

‎

‎

‎

This pipeline receives a bayer 10 image with 4K resolution and outputs a RGB image, using the three elements. 

gst-launch-1.0 -ve v4l2src io-mode=userptr ! 'video/x-bayer, bpp=10, width=3840, height=2160' ! cudashift shift=5 ! cudadebayer ! cudaawb ! fakesink




Error creating thumbnail: Unable to save thumbnail to destination






  Index  



Retrieved from "http://developer.ridgerun.com/wiki/index.php?title=CUDA_ISP_for_NVIDIA_Jetson/Examples/GStreamer_usage&oldid=47988"