NVIDIA Jetson Xavier - Overview of the video capture and display subsystem

NVIDIA®Jetson Xavier™
Introduction
SoM Overview Carrier Boards NVIDIA Xavier Developer Kit Connect Tech Rogue Connect Tech Mimic Software Support NVIDIA JetPack
JetPack 5.0.2
Getting Started Installing JetPack Flashing the board with GUI installer JetPack Components Compiling Source Code Flashing the Board from Cmdline Performance Tuning Evaluating Performance Set Values Manually Tuning Power Maximizing Performance
Development in the Board
Getting into the Board Using Serial Console Using SSH Using Ubuntu Login Installing Packages
Video Capture and Display
Description Supported Cameras Imx219 Driver Software Support Using GStreamer Using Libargus
Processors
HD Audio and Video Subystem Video Encoder Video Decoder JPEG Encoder and Decoder Video Image Compositor Audio Processing Engine Deep Learning Accelerator Volta GPU Description Using CUDA Using OpenGL
Interfaces
I2C SD/eMMC Controller USB Controller
GStreamer Pipelines
Capture and Display H264 H265 VP9 MPEG4 Simultaneous Streams Compositor
Deep Learning
TensorRT Using TensorRT integrated with Tensorflow Parsing a Tensorflow model for TensorRT Parsing Caffe model for TensorRT Building TensorRT API examples Deep Learning Accelerator Software Support Deep Learning Tutorials Jetson Inference Jetson Reinforcement
RidgeRun Products
GstPTZR GstCUDA GstColorTransfer GstWebRTC GstRtspSink GstInterpipe GstInference GStreamer Daemon GstShark
Reference Documentation
Contact Us

The NVIDIA Xavier has a video capture subsystem that allows image capture from camera sensors and does image processing. The Xavier SOM also integrates a display controller and three independent display outputs. In this section, you will get a general knowledge of the principal characteristics of the video capture subsystem: interfaces and formats supported, system modes, and image processing capabilities. Also, you will learn about the available software for this subsystem and how to use it.

Likewise, a list of cameras/decoders supported for this platform by NVIDIA or by RidgeRun will be provided.

Camera Architecture Stack

The Linux for Tegra architecture framework with the application and kernel-mode V4L2 drivers is as follows:

Camera Architecture Stack

Available Camera Interfaces

The following table summarizes the different standard interfaces that could be used to interface a camera module with the SoM.

Interface	Description
3xUSB 3.1	Has support for 1080p@60FPS video and is backward compatible with USB 3.0 and USB 2.0
Micro-USB 2.0 OTG	Used through a micro-B to female USB-A adapter has enough bandwidth for video 720p@30FPS.
Gigabit Ethernet	Allows support for IP Cameras and has enough bandwidth for uncompressed 1080p video streams
MIPI CSI-2	Provides direct access to the ISP and can support up to 4 quad lane cameras or four dual-lane cameras plus two quad lane cameras or 6 dual-lane cameras (total of 6 in any configuration). It can also support up to 16 cameras via the virtual channel.
PCIe	This connector allows for the connection of SLVS cameras
GPIOs	40-pin GPIO header allows for communication cameras that follow the UART, SPI, or CAN protocol

Video Input Subsystem (VI)

The Video Input (VI) subsystem is in charge of taking data from the MIPI-CSI receiver and preparing it for either the Image Signal Processor or the system memory. It provides formatting for RGB, YCbCr, and raw Bayer data for up to six active streams.

Image Signal Processor (ISP)

The Image Signal Processor (ISP) takes data from the Video Input Subsystem or directly from the CSI block in raw Bayer format and processes it to YUV, and removes artifacts introduced by high-megapixel CMOS sensors, camera lens, and color-space conversion.

The ISP enables various features such as:

• CSI Virtual Channel allows for up to 16 total virtual channels.
• Local Tone Map.
• Bayer Histogram statistics for auto-exposure.
• Bayer average map for auto white balance and auto-exposure.
• Sharpness map for autofocus.
• Parity Protection
• Bad pixel count.
• Deadlock detection.