Revision as of 15:43, 8 November 2016

IMX219 Features

The Sony IMX219 is a CMOS image sensor with the following features:

CSI2 serial data output (selection of 4 or 2 lanes)
Max. 30 frame/s in all-pixel scan mode
180 frame/s 720p with 2x2 analog (special) binning, 60 frame/s @ 1080p with V-crop
Data rate: Max. 722 Mbps/lane(@4lane), 912 Mbps/Lane(@2lane)
Max resolution of 3280 (H) x 2464 (V) approx. 8.08 M pixels

RidgeRun has developed a driver for the Tegra X1 platform with the following support:

L4T 24.2 and Jetpack 2.3
V4l2 Media controller driver
Tested resolution 3280 x 2464 @ 15 fps
Tested resolution 1080p @ 47 fps
Tested resolution 720p @ 78 fps
Tested resolution 1640x1232 @ 30 fps
Tested resolution 820x616 @ 30 fps
Tested with J100 and J20 Auvidea boards.

Getting the source code

In order to get the source code please send an email to support@ridgerun.com or contact us

Enabling the driver

In order to use this driver, you have to patch and compile the kernel source, and there are two ways to do it:

Using RidgeRun SDK

Through the SDK you can easily patch the kernel and generate an image with the required changes to get the imx219 sensor to work. In this wiki Getting_Started_Guide_for_Tegra_X1_Jetson you can find all the information required to build a Tegra X1 SDK from scratch.

In order to add the IMX219 driver follow this steps:

Go to to your SDK directory
Go to the kernel directory
Copy the patches in the patches directory

0001-add-imx219-subdevice-driver.patch
0002-add-imx219-dtb.patch

In the case that you will use the driver with the J20 board, you will need to copy the following patch:

0003-add-j20-board-driver.patch

Modify the series file in the kernel directory. You have to add the 3 above patches.
Run make config and select the IMX219 in the Kernel Configuration like this:

-> Kernel Configuration
 -> Device Drivers                                                                                                                        
  -> Multimedia support                                                                                           
    -> Encoders, decoders, sensors and other helper chips
       -> <*> IMX219 camera sensor support

Then make the SDK and install following the Started Guide mentioned before

Using Jetpack

Follow the instructions in Compiling_Tegra_X1_source_code (Downloading the code) to get the kernel source code.

Once you have the source code, apply the following two patches if you haven't yet, for fixing kernel errors during compilation.

kernel_r7_asm.patch
logical_comparison.patch

Apply the driver patches:

0001-add-imx219-subdevice-driver.patch
0002-add-imx219-dtb.patch

In the case that you will use the driver with the J20 board, you will need to apply the following patch:

0003-add-j20-board-driver.patch

Follow the instructions in Compiling_Tegra_X1_source_code (Build Kernel) for building the kernel, and then flash the image.

Make sure to enable imx219 driver support:

make menuconfig

-> Device Drivers                                                                                                                        
  -> Multimedia support                                                                                           
    -> Encoders, decoders, sensors and other helper chips
       -> <*> IMX219 camera sensor support

Using the driver

Gstreamer examples

The Gstreamer version distributed with Jetpack doesn't support bayer RAW10 only RAW8 so gstreamer needs to be patched in order to capture using v4l2src. Follow the steps in the following wiki page to add the support for RAW10:

http://developer.ridgerun.com/wiki/index.php?title=Compile_gstreamer_on_tegra_X1

Snapshots

3280x2464

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! "video/x-bayer, format=rggb, width=3280, height=2464" ! filesink location=test_3280x2464.bayer

1920x1080

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! "video/x-bayer, format=rggb, width=1920, height=1080" ! filesink location=test_1920x1080.bayer

1280x720

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! "video/x-bayer, format=rggb, width=1280, height=720" ! filesink location=test_1280x720.bayer

1640x1232 (Binning x2)

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! "video/x-bayer, format=rggb, width=1640, height=1232" ! filesink location=test_1640x1232.bayer

820x616 (Binning x4)

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! "video/x-bayer, format=rggb, width=820, height=616" ! filesink location=test_820x616.bayer

In order to check the snapshot, you can use the following tool:

https://github.com/jdthomas/bayer2rgb

So, run the following commands to download the tool and compile it:

git clone git@github.com:jdthomas/bayer2rgb.git
cd bayer2rgb
make
cp bayer2rgb /usr/bin/

Bayer2rgb will convert naked (no header) bayer grid data into rgb data. There are several choices of interpolation (though they all look essentially the same to my eye). It can output tiff files, and can integrate with ImageMagick to output other formats.

./bayer2rgb --input=<input file> --output=data.tiff --width=<width> --height=<height> --bpp=16 --first=RGGB --method=BILINEAR --tiff

Use image_magik to convert the tiff to png:

convert data.tiff data.png

Capture

In order to capture 10 buffers and save them in a file, you can run the following pipelines:

3280x2464

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! "video/x-bayer, format=rggb, width=3280, height=2464" ! filesink location=test_3280x2464.bayer

1920x1080

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! "video/x-bayer, format=rggb, width=1920, height=1080" ! filesink location=test_1920x1080.bayer

1280x720

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! "video/x-bayer, format=rggb, width=1280, height=720" ! filesink location=test_1280x720.bayer

1640x1232 (Binning x2)

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! "video/x-bayer, format=rggb, width=1640, height=1232" ! filesink location=test_1640x1232.bayer

820x616 (Binning x4)

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! "video/x-bayer, format=rggb, width=820, height=616" ! filesink location=test_820x616.bayer

You can use the raw2rgbpnm tool to check the 10 buffers:

https://github.com/martinezjavier/raw2rgbpnm

So, run the following commands to download the tool and compile it:

git clone git clone git@github.com:martinezjavier/raw2rgbpnm.git
cd raw2rgbpnm

Open the file raw2rgbpnm.c and change the line 489 with:

int c = getopt(argc, argv, "a:b:f:ghs:wn");

This is to enable the option to extract multiple frames from a file. Now, you can build the application:

make

This tool converts from GRBG10 to pnm. We capture RGGB in the IMX219, so you will see that the colors at the output of the image are wrong.

./raw2rgbpnm -f SGRBG10 -s <resolution> -b 5.0 -n <input file> <output file>

Dual Capture

Using the following pipeline we can test the performance of the Tegra X1 when doing dual capture:

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-bayer,format=rggb,width=3280,height=2464' ! fakesink 
v4l2src device=/dev/video3 ! 'video/x-bayer,format=rggb,width=3280,height=2464' ! fakesink

We noticed that using two cameras with the max resolution 3280x2464, the cpu load consumption measured with tegrastats doesn't change considerably, and it remains almost the same:

Tegrastats in normal operation:

RAM 563/3994MB (lfb 748x4MB) cpu [51%,0%,0%,0%]@518 GR3D 0%@76 EDP limit 0
RAM 563/3994MB (lfb 748x4MB) cpu [50%,0%,0%,0%]@518 GR3D 0%@76 EDP limit 0
RAM 563/3994MB (lfb 748x4MB) cpu [50%,0%,0%,0%]@518 GR3D 0%@76 EDP limit 0
RAM 563/3994MB (lfb 748x4MB) cpu [51%,0%,0%,0%]@518 GR3D 0%@76 EDP limit 0
RAM 563/3994MB (lfb 748x4MB) cpu [49%,0%,0%,0%]@518 GR3D 0%@76 EDP limit 0
RAM 563/3994MB (lfb 748x4MB) cpu [53%,0%,0%,0%]@518 GR3D 0%@76 EDP limit 0
RAM 563/3994MB (lfb 748x4MB) cpu [52%,0%,0%,0%]@403 GR3D 0%@76 EDP limit 0

Tegrastats with the above pipeline running

RAM 608/3994MB (lfb 651x4MB) cpu [57%,3%,0%,0%]@825 GR3D 0%@76 EDP limit 0
RAM 608/3994MB (lfb 651x4MB) cpu [52%,29%,0%,0%]@403 GR3D 0%@76 EDP limit 0
RAM 608/3994MB (lfb 651x4MB) cpu [56%,0%,1%,0%]@403 GR3D 0%@76 EDP limit 0
RAM 608/3994MB (lfb 651x4MB) cpu [60%,1%,1%,0%]@403 GR3D 0%@76 EDP limit 0
RAM 608/3994MB (lfb 651x4MB) cpu [57%,1%,0%,0%]@403 GR3D 0%@76 EDP limit 0
RAM 608/3994MB (lfb 651x4MB) cpu [58%,0%,1%,0%]@403 GR3D 0%@76 EDP limit 0
RAM 608/3994MB (lfb 651x4MB) cpu [58%,1%,0%,0%]@403 GR3D 0%@76 EDP limit 0

These tests were done using the J20 board from Auvidea Getting_started_guide_for_Auvidea_J20_board

@@ Line 141: / Line 141: @@
 </pre>
-*820x616 (Binning x2)
+*820x616 (Binning x4)
 <pre style="background:#d6e4f1">

Difference between revisions of "Sony IMX219 Linux driver"

Revision as of 15:43, 8 November 2016

Contents

IMX219 Features

Getting the source code

Enabling the driver

Using RidgeRun SDK

Using Jetpack

Using the driver

Gstreamer examples

Snapshots

Capture

Dual Capture

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Useful Links

Legal

Services

Tools