NVIDIA Jetson Orin/GStreamer Pipelines/Capture and Display

NVIDIA®Jetson AGX Orin™
Introduction
SoM Overview Carrier Boards NVIDIA developer kit Auvidea X230D Connect Tech Forge Software Development Support
JetPack 5.0.2
Getting Started Installing JetPack Flashing the board with GUI installer JetPack Components Compiling Source Code Flashing the Board from Cmdline Performance Tuning Evaluating Performance Set Values Manually Tuning Power Maximizing Performance Software Encoders For Jetson Orin Nano
Development in the Board
Getting into the Board Using Serial Console Using SSH Using Ubuntu Login Installing Packages
Emulation
Flashing commands for emulation
Supported Cameras
IMX477+J20 Driver
GStreamer Pipelines
Capture and Display H264 H265
RidgeRun Products
GstPTZR GstCUDA GstColorTransfer GstWebRTC GstRtspSink GstInterpipe GstInference GstVPI GStreamer Daemon GstShark
RidgeRun Reference Designs
Panoramic Stitching and WebRTC Streaming
Reference Documentation
Contact Us

This section presents GStreamer pipelines to capture from MIPI CSI-2 and USB cameras and render the video to the display connected to the Orin. Key performance indicators are measured using three power profiles (operation modes). You can find information on how the performance metrics were extracted in the Performance Measurement subsection.

MIPI CSI-2 Camera

For the pipelines in this section, we used the IMX477 camera.

1920x1080@30 using nv3dsink

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12' ! nv3dsink

Table 1: Performance of pipeline at 1920x1080@30 using nv3dsink

Operation Mode	CPU (%)	GPU 1 (%)	GPU 2 (%)	FPS	latency (ms)
0 (max performance)
1 (min power)
2 (30W)

1920x1080@30 using xvimagesink

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12' ! nvvidconv ! xvimagesink

Table 2: Performance of pipeline at 1920x1080@30 using xvimagesink

Operation Mode	CPU (%)	GPU 1 (%)	GPU 2 (%)	FPS	latency (ms)
0 (max performance)
1 (min power)
2 (30W)

USB Camera

For the pipelines in this section, we used the LI-10635 USB camera.

1280x720@30 using xvimagesink

 gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1,width=1280,height=720 ! nvvidconv ! xvimagesink

Table 3: Performance of pipeline at 1280x720@30 using xvimagesink

Operation Mode	CPU (%)	GPU 1 (%)	GPU 2 (%)	FPS	latency (ms)
0 (max performance)	32.27	22.2	23.87	30.01	0.13
1 (min power)	23.97	6.27	63.4	30.01	0.13
2 (30W)	27.6	5.87	64.87	30.01	0.16

1280x720@30 using nv3dsink

 gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1,width=1280,height=720 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nv3dsink

Performance:

Table 4: Performance of pipeline at 1280x720@30 using nv3dsink

Operation Mode	CPU (%)	GPU 1 (%)	GPU 2 (%)	FPS	latency (ms)
0 (max performance)	24.5	31.33	28.27	30.01	1.92
1 (min power)	32.87	17.93	52.17	30.01	2.47
2 (30W)	28.5	14.9	52.63	30.01	2.34

Performance Measurement

The performance measurements are extracted using the specified power profile (operation mode) and with the GStreamer pipeline running in the system.

Operation Mode:

• To change the Orin AGX power profile (operation mode), please check our Performance Tuning - Tuning Power section.

CPU usage:

• For the CPU percentage utilization, the tegrastats utility was used as follows: sudo tegrastats -readall. This command shows the percentage utilization of the 12 cores that the Jetson AGX Orin devkit has. For the measurement, we compute the average over 30 samples of the utilization values for the CPU cores.

GPU Usage:

• The same procedure described for the CPU usage is done in the case of GPU percentage. As the board has 2 GPUs, 30 samples of each one are taken, and then, the average is obtained.

FPS:

• For the FPS, RidgeRun's gst-perf plugin is used. This plugin computes the FPS mean. For the measurement, we compute the average over 30 samples of the FPS mean values. The following example shows a pipeline with the perf element incorporated:

 gst-launch-1.0 -v v4l2src device=/dev/video0 ! perf ! video/x-raw,framerate=30/1,width=1280,height=720 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nv3dsink

Latency:

• In the case of the Latency metric, the RidgeRun GstShark plugin is used. For this metric, it is important to take into account that it is not a glass-to-glass measurement. GstShark provides the time needed by a buffer to travel from the source pad of the source element to the source pad of the remaining elements of the pipeline. The following pipeline shows an example of the GstShark interlatency tracer execution:

 GST_DEBUG="GST_TRACER:7" GST_TRACERS="interlatency" gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1,width=1280,height=720 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nv3dsink

Remote Development

Often we want to run tests in a Jetson that is in another physical location. In these cases, you will need to have a display connected to the Jetson and also find out the proper value of the DISPLAY environment variable. For this, before you move to developing remotely or with someone's help at the Jetson location, connect a keyboard to the Jetson, open a terminal and check for the DISPLAY variable executing the following command:

echo $DISPLAY

This command will print the needed value, usually something like :0. Then you can run the GStreamer Pipeline from an SSH terminal as follows:

 DISPLAY=:0 GST_DEBUG=2 gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1,width=1280,height=720 ! nvvidconv ! xvimagesink sync=true

Make sure to replace the :0 value in the pipeline command according to your $DISPLAY environment variable. More information about accessing the Jetson Orin AGX through SSH can be found in our SSH section.