OmniVision OV5647 Linux driver

From RidgeRun Developer Connection
Jump to: navigation, search

Nvidia-preferred-partner-badge-rgb-for-screen.png

Error something wrong.jpg Problems running the pipelines shown on this page?
Please see our GStreamer Debugging guide for help.

RR Contact Us.png


Keywords: OV5647 Jetson TX1, GStreamer, Raspberry PI, NVIDIA, RidgeRun, V4L2 Driver, OmniVision

OmniVision OV5647 Linux driver overview video


OV5647 features

The OmniVision OV5647 is a CMOS image sensor with the following features:

  • Automatic image control functions:
    • Automatic exposure control (AEC)
    • Automatic white balance (AWB)
    • Automatic band filter (ABF)
    • Automatic 50/60 Hz luminance detection
    • Automatic black level calibration (ABLC)
  • Image quality controls: lens correction, defective pixel canceling
  • CSI2 serial data output (MIPI interface 2 lanes)
  • Support for output formats: 8-/10-bit raw RGB data
  • Maximum image transfer rate:
    • QSXGA "Mpixel" (2592 x 1944): 15 fps
    • 1080p: 30 fps
    • 960p: 45 fps
    • 720p: 60 fps
    • VGA (640 x 480): 90 fps
    • QVGA (320 x 240): 120 fps

RidgeRun has developed a driver for the Jetson TX1 platform with the following support:

  • V4l2 Media controller driver
  • Tested resolution 1080p @ 30 fps. (More resolutions support are in the developing process).
  • Output format: RAW10 Bayer BGGR pattern.
  • Tested with J100 and J20 Auvidea boards
  • Capture with v4l2src and nvcamerasrc
  • Support for sextuple video capture. (Tested using the 6 available ports on Auvidea J20 expansion board)

Currently available for:

  • OV5647 - TX1 - Jetpack 3.1 - J100/J20
  • OV5647 - TX2 - Jetpack 3.3 - J100/J20/J90
  • OV5647 - TX1 - Jetpack 3.3 - J100 (work in progress)

Enabling the driver

In order to use this driver, you have to patch and compile the kernel source.

Using Jetpack

  • If you didn't have download the toolchain yet, follow the instructions in [[Compiling_Jetson_TX1/TX2_source_code#Toolchain Compiling Jetson TX1/TX2 source code (Toolchain)]
  • Once you have the source code, apply the patches for fix kernel error during compilation and add the support for the OV5647:
cd $DEVDIR/64_TX1/Linux_for_Tegra_64_tx1/sources/
PATCHES=<path to the tarball>
tar -xzf $PATCHES/ov5647-driver-for-tegra-x1-patches.tar.gz
mv ov5647-driver-for-tegra-x1-patches patches
quilt push -a

Please change the path in PATCHES for the one where you downloaded the tarball with the OV5647 patches.

Make sure to enable OV5647 and J20 driver support in step 5:

-> Device Drivers                                                                                                                        
 -> Multimedia support                                                                                           
   -> Encoders, decoders, sensors and other helper chips
      -> <*> Auvidea J20 Expansion board
      -> <*> Omnivision OV5647 camera sensor support

Using the driver (GStreamer examples)

The GStreamer version distributed with Jetpack doesn't support Bayer RAW10 only RAW8 so GStreamer needs to be patched in order to capture using v4l2src. Follow the steps in the following wiki page to add the support for RAW10:

Compile_gstreamer_on_Jetson_TX1_and_TX2

DISPLAY=:0 gst-launch-1.0 ...

Getting_started_guide_for_Auvidea_J20_board

Demo Video Pipeline Example

By following this link Overview Demo Video you can see the results of executing the below pipeline and get some CPU/GPU load percentage and framerate measurements.

The below pipeline executes the following actions:

  • Sextuple RGB video capture at 1080p@30fps
  • Downscale to 320x240 and color-space format conversion to YUV I420
  • Mix of the 6 streams into one window (Mosaic)
  • Display
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, \
height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, \
height=(int)240, format=(string)I420, framerate=(fraction)30/1' ! perf ! mixer.sink_1  nvcamerasrc sensor-id=1 \
fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \
framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, height=(int)240, format=(string)I420, \
framerate=(fraction)30/1' ! mixer.sink_2 nvcamerasrc sensor-id=2 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \
width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, \
width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)30/1' ! mixer.sink_3 nvcamerasrc \
sensor-id=3 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \
framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, height=(int)240, format=(string)I420, \
framerate=(fraction)30/1' ! mixer.sink_4 nvcamerasrc sensor-id=4 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \
width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, \
width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)30/1' ! mixer.sink_5 nvcamerasrc \
sensor-id=5 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \
framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)320, height=(int)240, format=(string)I420, \
framerate=(fraction)30/1' ! mixer.sink_6 videotestsrc pattern="black" ! video/x-raw,width=1,height=1 ! videomixer \
name=mixer sink_0::xpos=0 sink_0::ypos=0 sink_0::alpha=0 sink_1::xpos=320 sink_1::ypos=0 sink_2::xpos=320 \
sink_2::ypos=240 sink_3::xpos=0 sink_3::ypos=0 sink_4::xpos=0 sink_4::ypos=480 sink_5::xpos=0 sink_5::ypos=240 \
sink_6::xpos=320 sink_6::ypos=480 ! queue ! ximagesink sync=false -v

This is a snapshot of the Jetson TX1 display while running the above pipeline:

Alt
Figure 1.Sextuple capture demo video snapshot


Performance statistics

  • Tegrastats in normal operation:
RAM 827/3995MB (lfb 660x4MB) cpu [4%,3%,2%,6%]@102 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [6%,2%,3%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [1%,1%,2%,10%]@102 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [4%,4%,5%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [3%,1%,4%,5%]@102 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [6%,3%,2%,6%]@102 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [3%,1%,2%,10%]@102 GR3D 0%@76 EDP limit 0
RAM 828/3995MB (lfb 660x4MB) cpu [9%,2%,2%,2%]@102 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [9%,3%,2%,3%]@403 GR3D 0%@76 EDP limit 0
RAM 827/3995MB (lfb 660x4MB) cpu [0%,0%,3%,0%]@102 GR3D 0%@76 EDP limit 0
  • Tegrastats with the above pipeline running
RAM 1821/3995MB (lfb 404x4MB) cpu [57%,56%,60%,48%]@1734 GR3D 44%@76 EDP limit 0
RAM 1819/3995MB (lfb 404x4MB) cpu [59%,58%,54%,55%]@1734 GR3D 20%@76 EDP limit 0
RAM 1821/3995MB (lfb 404x4MB) cpu [53%,66%,52%,56%]@1734 GR3D 28%@76 EDP limit 0
RAM 1822/3995MB (lfb 404x4MB) cpu [57%,66%,53%,54%]@1734 GR3D 42%@76 EDP limit 0
RAM 1820/3995MB (lfb 404x4MB) cpu [50%,61%,52%,53%]@1734 GR3D 46%@76 EDP limit 0
RAM 1822/3995MB (lfb 404x4MB) cpu [55%,63%,52%,59%]@1734 GR3D 29%@76 EDP limit 0
RAM 1822/3995MB (lfb 404x4MB) cpu [56%,62%,56%,51%]@1734 GR3D 42%@76 EDP limit 0
RAM 1820/3995MB (lfb 404x4MB) cpu [55%,64%,52%,56%]@1734 GR3D 20%@76 EDP limit 0
RAM 1822/3995MB (lfb 404x4MB) cpu [57%,62%,50%,55%]@1734 GR3D 34%@76 EDP limit 0
RAM 1822/3995MB (lfb 404x4MB) cpu [59%,66%,49%,57%]@1734 GR3D 34%@76 EDP limit 0

Snapshots

In order to check the snapshot, you can use the following tool:

https://github.com/jdthomas/bayer2rgb

So, run the following commands to download the tool and compile it:

git clone git@github.com:jdthomas/bayer2rgb.git
cd bayer2rgb
make
cp bayer2rgb /usr/bin/

Bayer2rgb will convert naked (no header) Bayer grid data into RGB data. There are several choices of interpolation (though they all look essentially the same to my eye). It can output tiff files and can integrate with ImageMagick to output other formats.

  • 1920x1080
gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! "video/x-bayer, format=bggr, width=1920, height=1080" \
! multifilesink location=test%d_1920x1080.bayer

Check the snapshot with:

./bayer2rgb --input=test#_1920x1080.bayer --output=data.tiff --width=1920 --height=1080 --bpp=16 --first=BGGR \
--method=BILINEAR --tiff

Use image_magik to convert the tiff to png:

convert data.tiff data.png


Snapshots with nvcamerasrc

The following pipeline will create a file for each captured frame. You can visualize the file on the following web page: http://rawpixels.net/

gst-launch-1.0 -v nvcamerasrc sensor-id=1 fpsRange="30 30" num-buffers=100 ! 'video/x-raw(memory:NVMM), width=(int)1920, \
height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)1920, \
height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! multifilesink location=test_%d.yuv

Single Capture

V4l2src

You can use the raw2rgbpnm tool to check all the buffers:

https://github.com/martinezjavier/raw2rgbpnm

So, run the following commands to download the tool and compile it:

git clone git clone git@github.com:martinezjavier/raw2rgbpnm.git
cd raw2rgbpnm

Open the file raw2rgbpnm.c and change line 489 with:

int c = getopt(argc, argv, "a:b:f:ghs:wn");

This is to enable the option to extract multiple frames from a file. Now, you can build the application:

make

In order to capture 10 buffers and save them in a file, you can run the following pipelines:

  • 1920x1080
gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! "video/x-bayer, format=bggr, width=1920, height=1080" \
! filesink location=test_1920x1080.bayer

Check the buffers with:

./raw2rgbpnm -f SGRBG10 -s 1920x1080 -b 5.0 -n test_1920x1080.bayer output_1920x1080

Nvcamerasrc

  • 1920x1080
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, \
height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink -e

This is an image captured with the above pipeline:

Alt
Figure 2.OV5647 capture (1920x1080) with nvcamerasrc


Sextuple Capture

The following image consists of a Jetson TX1 with six ov5647 cameras plugged in the Auvidea J20 expansion board and doing video capture at the same time:

Alt
Figure 3.OV5647 camera sextuple video capture with Jetson TX1


Sextuple Capture Connection Diagram:

Alt
Figure 4.OV5647 Sextuple Capture Connection Diagram


Using the following pipelines we can test the performance of the Jetson TX1 when doing sextuple capture:

V4l2src

  • Pipeline for sextuple video capture using v4l2src, at 1920x1080 @30fps:
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \
v4l2src device=/dev/video1 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \
v4l2src device=/dev/video2 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \
v4l2src device=/dev/video3 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \
v4l2src device=/dev/video4 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink \
v4l2src device=/dev/video5 ! 'video/x-bayer,format=bggr,width=1920,height=1080' ! fakesink
Performance statistics
  • Tegrastats in normal operation:
RAM 1579/3994MB (lfb 378x4MB) cpu [3%,5%,2%,7%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [3%,4%,5%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [2%,1%,8%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [4%,2%,8%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [2%,3%,7%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [3%,3%,6%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [5%,1%,5%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [1%,2%,7%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1579/3994MB (lfb 378x4MB) cpu [8%,1%,1%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1578/3994MB (lfb 378x4MB) cpu [8%,2%,2%,6%]@102 GR3D 0%@76 EDP limit 0
  • Tegrastats with the above pipeline running:
RAM 1677/3994MB (lfb 370x4MB) cpu [20%,3%,5%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [18%,5%,6%,2%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [18%,3%,4%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [18%,7%,3%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [11%,9%,7%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [17%,9%,4%,2%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [17%,4%,8%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [15%,6%,2%,5%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [21%,6%,4%,7%]@102 GR3D 0%@76 EDP limit 0
RAM 1677/3994MB (lfb 370x4MB) cpu [19%,3%,5%,4%]@102 GR3D 0%@76 EDP limit 0

Nvcamerasrc

  • Pipeline for sextuple video capture using nvcamerasrc, at 1920x1080 @30fps:
gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \
format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=1 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \
width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=2 \
fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' \
! fakesink nvcamerasrc sensor-id=3 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \
format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=4 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \
width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! fakesink nvcamerasrc sensor-id=5 \
fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' \
! fakesink
Performance statistics
  • Tegrastats in normal operation:
RAM 1490/3994MB (lfb 412x4MB) cpu [5%,3%,5%,2%]@102 GR3D 0%@76 EDP limit 0
RAM 1490/3994MB (lfb 412x4MB) cpu [5%,1%,5%,3%]@307 GR3D 0%@76 EDP limit 0
RAM 1490/3994MB (lfb 412x4MB) cpu [5%,2%,4%,0%]@102 GR3D 0%@76 EDP limit 0
RAM 1490/3994MB (lfb 412x4MB) cpu [8%,1%,6%,2%]@102 GR3D 0%@76 EDP limit 0
RAM 1490/3994MB (lfb 412x4MB) cpu [7%,5%,6%,9%]@102 GR3D 0%@76 EDP limit 0
RAM 1489/3994MB (lfb 412x4MB) cpu [6%,3%,6%,5%]@102 GR3D 0%@76 EDP limit 0
RAM 1489/3994MB (lfb 412x4MB) cpu [6%,2%,3%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1489/3994MB (lfb 412x4MB) cpu [7%,4%,3%,8%]@102 GR3D 0%@76 EDP limit 0
RAM 1489/3994MB (lfb 412x4MB) cpu [6%,2%,1%,5%]@102 GR3D 0%@76 EDP limit 0
RAM 1489/3994MB (lfb 412x4MB) cpu [8%,4%,1%,4%]@102 GR3D 0%@76 EDP limit 0
  • Tegrastats with the above pipeline running
RAM 1785/3994MB (lfb 408x4MB) cpu [45%,44%,51%,40%]@1428 GR3D 0%@76 EDP limit 0
RAM 1785/3994MB (lfb 408x4MB) cpu [47%,53%,50%,45%]@1036 GR3D 0%@76 EDP limit 0
RAM 1785/3994MB (lfb 408x4MB) cpu [46%,54%,42%,47%]@1326 GR3D 0%@76 EDP limit 0
RAM 1785/3994MB (lfb 408x4MB) cpu [44%,45%,47%,49%]@1036 GR3D 0%@76 EDP limit 0
RAM 1785/3994MB (lfb 408x4MB) cpu [46%,50%,48%,47%]@1036 GR3D 0%@76 EDP limit 0
RAM 1785/3994MB (lfb 408x4MB) cpu [48%,48%,49%,45%]@1224 GR3D 0%@76 EDP limit 0
RAM 1785/3994MB (lfb 408x4MB) cpu [50%,50%,43%,46%]@1326 GR3D 0%@76 EDP limit 0
RAM 1786/3994MB (lfb 408x4MB) cpu [50%,47%,48%,43%]@1326 GR3D 0%@76 EDP limit 0
RAM 1785/3994MB (lfb 408x4MB) cpu [40%,46%,47%,46%]@1132 GR3D 0%@76 EDP limit 0
RAM 1786/3994MB (lfb 408x4MB) cpu [46%,53%,40%,49%]@1428 GR3D 0%@76 EDP limit 0

Sextuple Capture, and Display

Using the following pipelines we can test the performance of the Jetson TX1 when doing sextuple video capture and display:

V4l2src

V4l2 can't be used with the ISP accelerated hardware unit of the Jetson TX1 to do the de-Bayer process (convert to YUV color-space) yet. Because of that, we can't link v4l2src with any hardware-accelerated encoder/decoder available on Jetson TX1 or any actual video sinks, if the camera sensor output is in Bayer color-space format.

Nvcamerasrc

  • Pipeline for sextuple video capture, at 1920x1080 resolution @30fps and display:
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \
format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink nvcamerasrc sensor-id=1 fpsRange="30 30" ! \
'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! \
nveglglessink nvcamerasrc sensor-id=2 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \ 
format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink nvcamerasrc sensor-id=3 fpsRange="30 30" ! \
'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! \
nveglglessink nvcamerasrc sensor-id=4 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \
format=(string)I420, framerate=(fraction)30/1' ! nvegltransform ! nveglglessink nvcamerasrc sensor-id=5 fpsRange="30 30" ! \
'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! perf ! \
nvegltransform ! nveglglessink -e 
Performance statistics
  • Tegrastats in normal operation:
RAM 1661/3994MB (lfb 334x4MB) cpu [3%,7%,5%,1%]@102 GR3D 0%@76 EDP limit 0
RAM 1661/3994MB (lfb 334x4MB) cpu [2%,1%,2%,1%]@102 GR3D 0%@76 EDP limit 0
RAM 1660/3994MB (lfb 334x4MB) cpu [5%,2%,1%,5%]@102 GR3D 0%@76 EDP limit 0
RAM 1660/3994MB (lfb 334x4MB) cpu [3%,3%,4%,6%]@102 GR3D 0%@76 EDP limit 0
RAM 1660/3994MB (lfb 334x4MB) cpu [3%,4%,1%,7%]@102 GR3D 0%@76 EDP limit 0
RAM 1660/3994MB (lfb 334x4MB) cpu [5%,3%,1%,6%]@102 GR3D 0%@76 EDP limit 0
RAM 1661/3994MB (lfb 334x4MB) cpu [3%,3%,1%,9%]@102 GR3D 0%@76 EDP limit 0
RAM 1661/3994MB (lfb 334x4MB) cpu [3%,4%,3%,3%]@102 GR3D 0%@76 EDP limit 0
RAM 1661/3994MB (lfb 334x4MB) cpu [3%,2%,3%,4%]@102 GR3D 0%@76 EDP limit 0
RAM 1661/3994MB (lfb 334x4MB) cpu [2%,4%,4%,6%]@102 GR3D 0%@76 EDP limit 0
  • Tegrastats with the above pipeline running
RAM 2253/3994MB (lfb 284x4MB) cpu [63%,63%,64%,61%]@1734 GR3D 52%@307 EDP limit 0
RAM 2253/3994MB (lfb 284x4MB) cpu [63%,64%,60%,61%]@1734 GR3D 59%@230 EDP limit 0
RAM 2252/3994MB (lfb 284x4MB) cpu [64%,64%,64%,61%]@1734 GR3D 61%@230 EDP limit 0
RAM 2252/3994MB (lfb 284x4MB) cpu [65%,58%,64%,60%]@1734 GR3D 65%@307 EDP limit 0
RAM 2252/3994MB (lfb 284x4MB) cpu [61%,67%,62%,64%]@1734 GR3D 54%@307 EDP limit 0
RAM 2252/3994MB (lfb 284x4MB) cpu [66%,58%,63%,64%]@1734 GR3D 78%@230 EDP limit 0
RAM 2254/3994MB (lfb 283x4MB) cpu [65%,64%,64%,63%]@1734 GR3D 57%@307 EDP limit 0
RAM 2254/3994MB (lfb 283x4MB) cpu [65%,64%,62%,64%]@1734 GR3D 51%@230 EDP limit 0
RAM 2254/3994MB (lfb 283x4MB) cpu [67%,59%,63%,58%]@1734 GR3D 73%@230 EDP limit 0
RAM 2254/3994MB (lfb 283x4MB) cpu [62%,69%,67%,61%]@1734 GR3D 45%@307 EDP limit 0

Sextuple Capture, Downscale and Display

Using the following pipelines we can test the performance of the Jetson TX1 when doing sextuple video capture, downscale, and display:

V4l2src

V4l2 can't be used with the ISP accelerated hardware unit of the Jetson TX1 to do the de-Bayer process (convert to YUV color-space) yet. Because of that, we can't link v4l2src with any hardware-accelerated encoder/decoder available on Jetson TX1 or any actual video sinks, if the camera sensor output is in Bayer color-space format.

Nvcamerasrc

This was the same pipeline used in the ov5647 sextuple capture on the Jetson TX1 demo video in the Overview Video section at the beginning of this wiki.

  • Pipeline for sextuple video capture, downscale from 1920x1080 to 640x480 resolution @30fps and display:
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \
format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, \
framerate=(fraction)30/1' ! xvimagesink  nvcamerasrc sensor-id=1 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, \
height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, \
format=(string)I420, framerate=(fraction)30/1' ! xvimagesink nvcamerasrc sensor-id=2 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), \
width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, \
height=(int)480, format=(string)I420, framerate=(fraction)30/1' ! xvimagesink nvcamerasrc sensor-id=3 fpsRange="30 30" ! \
'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! \
'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1' ! xvimagesink nvcamerasrc sensor-id=4 \
fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! \
nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1' ! xvimagesink \
nvcamerasrc sensor-id=5 fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, \
framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw, width=(int)640, height=(int)480, format=(string)I420, \
framerate=(fraction)30/1' ! xvimagesink -v

The following picture is a screenshot of the Jetson TX1 when it was running the above pipeline:

Alt
Figure 5.OV5647 camera sextuple video capture on Jetson TX1 screenshot


Performance statistics
  • Tegrastats in normal operation:
RAM 1694/3994MB (lfb 351x4MB) cpu [4%,3%,1%,8%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [4%,3%,3%,7%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [4%,4%,3%,6%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [4%,5%,0%,7%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [1%,4%,0%,5%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [7%,5%,1%,5%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [8%,3%,1%,2%]@518 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [3%,0%,0%,0%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [3%,1%,0%,0%]@102 GR3D 0%@76 EDP limit 0
RAM 1694/3994MB (lfb 351x4MB) cpu [4%,3%,1%,0%]@102 GR3D 0%@76 EDP limit 0
  • Tegrastats with the above pipeline running
RAM 2065/3994MB (lfb 340x4MB) cpu [68%,57%,60%,59%]@1734 GR3D 54%@76 EDP limit 0
RAM 2065/3994MB (lfb 340x4MB) cpu [66%,59%,56%,57%]@1734 GR3D 64%@76 EDP limit 0
RAM 2065/3994MB (lfb 340x4MB) cpu [60%,65%,58%,62%]@1734 GR3D 53%@76 EDP limit 0
RAM 2065/3994MB (lfb 340x4MB) cpu [60%,62%,60%,63%]@1734 GR3D 46%@76 EDP limit 0
RAM 2065/3994MB (lfb 340x4MB) cpu [60%,58%,60%,65%]@1734 GR3D 54%@76 EDP limit 0
RAM 2065/3994MB (lfb 340x4MB) cpu [61%,64%,62%,60%]@1734 GR3D 42%@76 EDP limit 0
RAM 2065/3994MB (lfb 340x4MB) cpu [63%,61%,59%,65%]@1734 GR3D 54%@76 EDP limit 0
RAM 2069/3994MB (lfb 339x4MB) cpu [62%,63%,60%,58%]@1734 GR3D 51%@76 EDP limit 0
RAM 2069/3994MB (lfb 339x4MB) cpu [63%,57%,62%,59%]@1734 GR3D 57%@76 EDP limit 0
RAM 2069/3994MB (lfb 339x4MB) cpu [61%,66%,64%,62%]@1734 GR3D 56%@76 EDP limit 0

Video Encoding Transport Stream 1920x1080@30fps

The following pipeline will generate a video, you can visualize it with any video-player like VLC for example.

gst-launch-1.0 -v nvcamerasrc sensor-id=1 fpsRange="30 30" num-buffers=500 ! 'video/x-raw(memory:NVMM), width=(int)1920, \
height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh264enc ! qtmux ! filesink location=test.ts

OV5647 4x camera 720p recording test

On OV5647_4x_camera_720p_recording_test you will find a very descriptive step by step 4x camera 720p video stream recording to disk test. The test consists in save to disk the 4 video streams in RAW, H264, and H265 encoded formats in each case. A SSD and a SD Card (class 10) were used as disk units. Also, you will find the performance statistics of each test case, so you can make comparisons between them.

See also


RidgeRun Resources

Quick Start Client Engagement Process RidgeRun Blog Homepage
Technical and Sales Support RidgeRun Online Store RidgeRun Videos Contact Us

OOjs UI icon message-progressive.svg Contact Us

Visit our Main Website for the RidgeRun Products and Online Store. RidgeRun Engineering informations are available in RidgeRun Professional Services, RidgeRun Subscription Model and Client Engagement Process wiki pages. Please email to support@ridgerun.com for technical questions and contactus@ridgerun.com for other queries. Contact details for sponsoring the RidgeRun GStreamer projects are available in Sponsor Projects page. Ridgerun-logo.svg
RR Contact Us.png