CUDA ISP for NVIDIA Jetson: API usage

CUDA ISP for NVIDIA Jetson
CUDA ISP for NVIDIA Jetson Basics
Getting Started
Evaluating the CUDA ISP Getting the code Building CUDA ISP for NVIDIA Jetson Building custom v4l2src element
User Manual
GStreamer
cudadebayer cudashift cudaawb
Examples
API usage GStreamer usage
Performance
Library GStreamer Elements
Contact Us

General

CUDA ISP provides several algorithms to process raw image data from a camera sensor in a variety of ways.

Here we illustrate a simple example that involves all the basic algorithms. This example processes a raw BAYER10 image into a I420 image with white balancing using the following steps:

• The shift algorithm converts BAYER10 into BAYER8
• The debayer algorithm converts BAYER8 to RGB
• The white balancer algorithm improves the color balance in the RGB image without changing the format
• The color space converter converts RGB to I420

The following diagram illustrates the complete processing pipeline:

A simple process of this type might be used to encode camera information into a video file or a network stream.

Backend

We first create a backend object. The backend object provides factories to create algorithms and buffers. We specify CUDA as the desired backend type.

auto backend = rr::IBackend::Build(rr::kCUDA);

Creating the algorithms

The creation of any given algorithm requires a single function call:

auto shift = backend->CreateAlgorithm(rr::kShift);
auto debayer = backend->CreateAlgorithm(rr::kDebayer);
auto white_balancer = backend->CreateAlgorithm(rr::kWhiteBalancer);
auto color_space_converter = backend->CreateAlgorithm(rr::kColorSpaceConverter);

The shift object converts BAYERXX formats (BAYER10, BAYER12, BAYER14, or BAYER16) into a BAYER8 format. Each pixel in a BAYERXX format takes up 16 bits of information, but with the appropiate rightwards bit shift, the 8 most significant bits can be placed in the 8 least significant bit positions of the pixel memory. The shift object then completes the BAYERXX to BAYER8 conversion by only returning the 8 least significant bits of each pixel.

The debayer object converts BAYER8 to either RGB or RGBA.