Difference between revisions of "CUDA ISP for NVIDIA Jetson/Performance/Library"
Line 52: | Line 52: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaShift | | style="text-align:left; font-weight:bold;" | CudaShift | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.82 |
| style="background-color:#ffd6a5;" | 1.52 | | style="background-color:#ffd6a5;" | 1.52 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 63: | Line 63: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaDebayer | | style="text-align:left; font-weight:bold;" | CudaDebayer | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.68 |
| style="background-color:#ffd6a5;" | 1.30 | | style="background-color:#ffd6a5;" | 1.30 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 74: | Line 74: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.84 |
| style="background-color:#ffd6a5;" | 1.66 | | style="background-color:#ffd6a5;" | 1.66 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 85: | Line 85: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 1.24 |
| style="background-color:#ffd6a5;" | 1.91 | | style="background-color:#ffd6a5;" | 1.91 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 96: | Line 96: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | | style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.91 |
| style="background-color:#ffd6a5;" | 1.60 | | style="background-color:#ffd6a5;" | 1.60 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 110: | Line 110: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaShift | | style="text-align:left; font-weight:bold;" | CudaShift | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 1216 |
| style="background-color:#ffd6a5;" | 660 | | style="background-color:#ffd6a5;" | 660 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 121: | Line 121: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaDebayer | | style="text-align:left; font-weight:bold;" | CudaDebayer | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 1479 |
| style="background-color:#ffd6a5;" | 771 | | style="background-color:#ffd6a5;" | 771 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 132: | Line 132: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 1197 |
| style="background-color:#ffd6a5;" | 603 | | style="background-color:#ffd6a5;" | 603 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 143: | Line 143: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 807 |
| style="background-color:#ffd6a5;" | 522 | | style="background-color:#ffd6a5;" | 522 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 154: | Line 154: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | | style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 1104 |
| style="background-color:#ffd6a5;" | 623 | | style="background-color:#ffd6a5;" | 623 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 168: | Line 168: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaShift | | style="text-align:left; font-weight:bold;" | CudaShift | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.070 |
| style="background-color:#ffd6a5;" | 0.077 | | style="background-color:#ffd6a5;" | 0.077 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 179: | Line 179: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaDebayer | | style="text-align:left; font-weight:bold;" | CudaDebayer | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.043 |
| style="background-color:#ffd6a5;" | 0.057 | | style="background-color:#ffd6a5;" | 0.057 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 190: | Line 190: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.071 |
| style="background-color:#ffd6a5;" | 0.098 | | style="background-color:#ffd6a5;" | 0.098 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 201: | Line 201: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.092 |
| style="background-color:#ffd6a5;" | 0.099 | | style="background-color:#ffd6a5;" | 0.099 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 212: | Line 212: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | | style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 0.061 |
| style="background-color:#ffd6a5;" | 0.071 | | style="background-color:#ffd6a5;" | 0.071 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 226: | Line 226: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaShift | | style="text-align:left; font-weight:bold;" | CudaShift | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 89.7 |
| style="background-color:#ffd6a5;" | 90.8 | | style="background-color:#ffd6a5;" | 90.8 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 237: | Line 237: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaDebayer | | style="text-align:left; font-weight:bold;" | CudaDebayer | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 44.4 |
| style="background-color:#ffd6a5;" | 44.5 | | style="background-color:#ffd6a5;" | 44.5 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 248: | Line 248: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 99.8 |
| style="background-color:#ffd6a5;" | 99.1 | | style="background-color:#ffd6a5;" | 99.1 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 259: | Line 259: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 99.2 |
| style="background-color:#ffd6a5;" | 99.3 | | style="background-color:#ffd6a5;" | 99.3 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 270: | Line 270: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | | style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 44.1 |
| style="background-color:#ffd6a5;" | 44.3 | | style="background-color:#ffd6a5;" | 44.3 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 284: | Line 284: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaShift | | style="text-align:left; font-weight:bold;" | CudaShift | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 12.29 |
| style="background-color:#ffd6a5;" | 7.81 | | style="background-color:#ffd6a5;" | 7.81 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 295: | Line 295: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaDebayer | | style="text-align:left; font-weight:bold;" | CudaDebayer | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 11.6 |
| style="background-color:#ffd6a5;" | 13.04 | | style="background-color:#ffd6a5;" | 13.04 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 306: | Line 306: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 9.75 |
| style="background-color:#ffd6a5;" | 19.27 | | style="background-color:#ffd6a5;" | 19.27 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 317: | Line 317: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 8.89 |
| style="background-color:#ffd6a5;" | 24.42 | | style="background-color:#ffd6a5;" | 24.42 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 328: | Line 328: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | | style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 4.85 |
| style="background-color:#ffd6a5;" | 9.14 | | style="background-color:#ffd6a5;" | 9.14 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 342: | Line 342: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaShift | | style="text-align:left; font-weight:bold;" | CudaShift | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 26.8 |
| style="background-color:#ffd6a5;" | 28.4 | | style="background-color:#ffd6a5;" | 28.4 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 353: | Line 353: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaDebayer | | style="text-align:left; font-weight:bold;" | CudaDebayer | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 6.0 |
| style="background-color:#ffd6a5;" | 6.1 | | style="background-color:#ffd6a5;" | 6.1 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 364: | Line 364: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Gray World Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 33.7 |
| style="background-color:#ffd6a5;" | 32.7 | | style="background-color:#ffd6a5;" | 32.7 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 375: | Line 375: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | | style="text-align:left; font-weight:bold;" | CudaWhiteBalancer (Histogram Stretch Algorithm) | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 33.2 |
| style="background-color:#ffd6a5;" | 33.4 | | style="background-color:#ffd6a5;" | 33.4 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | | ||
Line 386: | Line 386: | ||
|- style="text-align:right;" | |- style="text-align:right;" | ||
| style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | | style="text-align:left; font-weight:bold;" | CudaColorSpaceConverter | ||
− | | style="background-color:#ffd6a5;" | | + | | style="background-color:#ffd6a5;" | 5.4 |
| style="background-color:#ffd6a5;" | 6.0 | | style="background-color:#ffd6a5;" | 6.0 | ||
| style="background-color:#ffadad;" | | | style="background-color:#ffadad;" | |
Revision as of 10:24, 13 April 2023
CUDA ISP for NVIDIA Jetson | |
---|---|
![]() | |
CUDA ISP for NVIDIA Jetson Basics | |
|
|
Getting Started | |
|
|
User Manual | |
|
|
GStreamer | |
|
|
Examples | |
|
|
Performance | |
|
|
Contact Us | |
|
This page is still under development. Please, do not take the performance data as definitive.
Library API performance
To measure the CUDA ISP API performance, we built a simple example (provided upon request) that iterates over the Apply
methods for each algorithm and records performance metrics for each iteration. We measured the duration of each algorithm's Apply
method. We also measured CPU, CPU RAM, GPU, and GPU RAM usage for the complete processing pipeline iterating at 30fps. We ran the experiments on both 1080p and 4K buffers. We also ran the experiments on the Jetson Nano, Jetson Xavier NX, Jetson Xavier AGX, and Jetson AGX Orin.
- We measured the duration of each
Apply
method separately using thechrono
library. - We used the
sys/times.h
library to obtain the CPU usage. - We read the
/proc/self/status
file to obtain the CPU RAM usage. - We used jtop to measure GPU usage on the Jetson Nano and Jetson Xavier NX. We use jetson-stats to measure GPU usage on the Jetson Xavier AGX and the Jetson AGX Orin.
- We used
cudaMemGetInfo
from CUDA to measure GPU RAM usage.
This is the hardware setup we used:
- On the Jetson Nano, we used Jetpack 4.5.3 and MAXN Power Mode (NVP model 0)
- On the Jetson Xavier NX, we used Jetpack 4.5.3 and 20W 6 Core Power Mode (NVP model 8)
- On the Jetson Xavier AGX, we used Jetpack 4.5.1 and 30W 8 Core Power Mode (NVP model 3)
- On the Jetson AGX Orin, we used Jetpack 5.0.2 and MAXN Power Mode (NVP model 0)
The following table summarizes CUDA ISP's performance results. The values in parenthesis next to each processing time is the corresponding theoretical framerate calculated as the inverse of the time.
Platform | Jetson AGX Orin | Jetson Xavier AGX | Jetson Xavier NX | Jetson Nano | ||||
---|---|---|---|---|---|---|---|---|
Buffer size | 1080p | 4K | 1080p | 4K | 1080p | 4K | 1080p | 4K |
Duration (ms) | ||||||||
CudaShift | 0.82 | 1.52 | ||||||
CudaDebayer | 0.68 | 1.30 | ||||||
CudaWhiteBalancer (Gray World Algorithm) | 0.84 | 1.66 | ||||||
CudaWhiteBalancer (Histogram Stretch Algorithm) | 1.24 | 1.91 | ||||||
CudaColorSpaceConverter | 0.91 | 1.60 | ||||||
Framerate (fps) | ||||||||
CudaShift | 1216 | 660 | ||||||
CudaDebayer | 1479 | 771 | ||||||
CudaWhiteBalancer (Gray World Algorithm) | 1197 | 603 | ||||||
CudaWhiteBalancer (Histogram Stretch Algorithm) | 807 | 522 | ||||||
CudaColorSpaceConverter | 1104 | 623 | ||||||
CPU usage (%) | ||||||||
CudaShift | 0.070 | 0.077 | ||||||
CudaDebayer | 0.043 | 0.057 | ||||||
CudaWhiteBalancer (Gray World Algorithm) | 0.071 | 0.098 | ||||||
CudaWhiteBalancer (Histogram Stretch Algorithm) | 0.092 | 0.099 | ||||||
CudaColorSpaceConverter | 0.061 | 0.071 | ||||||
CPU RAM (MB) | ||||||||
CudaShift | 89.7 | 90.8 | ||||||
CudaDebayer | 44.4 | 44.5 | ||||||
CudaWhiteBalancer (Gray World Algorithm) | 99.8 | 99.1 | ||||||
CudaWhiteBalancer (Histogram Stretch Algorithm) | 99.2 | 99.3 | ||||||
CudaColorSpaceConverter | 44.1 | 44.3 | ||||||
GPU usage (%) | ||||||||
CudaShift | 12.29 | 7.81 | ||||||
CudaDebayer | 11.6 | 13.04 | ||||||
CudaWhiteBalancer (Gray World Algorithm) | 9.75 | 19.27 | ||||||
CudaWhiteBalancer (Histogram Stretch Algorithm) | 8.89 | 24.42 | ||||||
CudaColorSpaceConverter | 4.85 | 9.14 | ||||||
GPU RAM (MB) | ||||||||
CudaShift | 26.8 | 28.4 | ||||||
CudaDebayer | 6.0 | 6.1 | ||||||
CudaWhiteBalancer (Gray World Algorithm) | 33.7 | 32.7 | ||||||
CudaWhiteBalancer (Histogram Stretch Algorithm) | 33.2 | 33.4 | ||||||
CudaColorSpaceConverter | 5.4 | 6.0 |