ON Semiconductor AR1335 Linux Driver

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.

ON Semiconductor AR1335 Features

The AR1335 is a 1/3.2-inch CMOS active-pixel digital image sensor with a pixel array of 4208H x 3120V. The AR1335 digital image sensor, features breakthrough 1.1µm pixel technology that delivers superior low-light image quality through leading sensitivity, quantum efficiency and linear full well. This allows image quality that rivals digital still cameras. With a sensor architecture focused on low power and a high Chief Ray Angle (CRA) for low Z-heights, the AR1335 is ideal for smartphone and other mobile device applications. It incorporates sophisticated on-chip camera functions such as windowing, mirroring, column and row skip modes, and snapshot mode. It is programmable through a simple two-wire serial interface. The AR1335 sensor can generate full resolution image at up to 30 frames per second (fps) and supports advanced video modes including 4K 30fps, 1080P 60fps and 720P 120fps. (Taken from OnSemi webpage).

Learn more about the sensor it the OnSemi webpage!

Supported Platforms

• NVIDIA Jetson TX2
• NVIDIA Jetson Nano

Features Included in the Driver

‎

‎

‎

Feature	Details	SDK Support	Platforms
1920x1080@60fps	4 Lanes	L4T 32.2.1 / Jetpack 4.2.2	TX2
3840x2160@30fps	4 Lanes	L4T 32.2.1 / Jetpack 4.2.2	TX2
4208x3120@30fps	4 Lanes	L4T 32.2.1 / Jetpack 4.2.2	TX2
4208x3120@15fps	2 Lanes	L4T 32.4.3 / Jetpack 4.4	Nano
4208x3120@15fps	2 Lanes	L4T 32.5 / Jetpack 4.5	Nano
2104x1560@29fps	2 Lanes	L4T 32.4.3 / Jetpack 4.4	Nano
2104x1560@29fps	2 Lanes	L4T 32.5 / Jetpack 4.5	Nano
512x512@86fps	2 Lanes	L4T 32.4.3 / Jetpack 4.4	Nano
512x512@86fps	2 Lanes	L4T 32.5 / Jetpack 4.5	Nano
Gain control		L4T 32.2.1 / Jetpack 4.2.2	TX2
Exposure control		L4T 32.2.1 / Jetpack 4.2.2	TX2
Framerate control		L4T 32.2.1 / Jetpack 4.2.2	TX2
White balance control	Digital gain	L4T 32.2.1 / Jetpack 4.2.2	TX2
Window control		L4T 32.2.1 / Jetpack 4.2.2	TX2
OTPM	Write/Read/Invalidate. (explain what this means)	L4T 32.2.1 / Jetpack 4.2.2	TX2
2 simultaneous cameras		L4T 32.2.1 / Jetpack 4.2.2	TX2
GRGB10 capture		L4T 32.2.1 / Jetpack 4.2.2	TX2
Temperature reading		L4T 32.2.1 / Jetpack 4.2.2	TX2
V4l2 Media Controller driver		L4T 32.2.1 / Jetpack 4.2.2	TX2
ISP usage through NvArgusCameraSrc		L4T 32.2.1 / Jetpack 4.2.2	TX2
GRBG10 capture	4208x3120@15fps only	L4T 32.5 / Jetpack 4.5	Nano
GRBG10 capture	4208x3120@15fps only	L4T 32.4.3 / Jetpack 4.4	Nano
GRBG8 capture		L4T 32.5 / Jetpack 4.5	Nano
GRBG8 capture		L4T 32.4.3 / Jetpack 4.4	Nano
GREY10 capture		L4T 32.5 / Jetpack 4.5	Nano
GREY10 capture		L4T 32.4.3 / Jetpack 4.4	Nano
GREY8 capture		L4T 32.5 / Jetpack 4.5	Nano
GREY8 capture		L4T 32.4.3 / Jetpack 4.4	Nano
V4l2 Media Controller driver		L4T 32.4.3 / Jetpack 4.4	Nano
V4l2 Media Controller driver		L4T 32.5 / Jetpack 4.5	Nano
ISP usage through NvArgusCameraSrc	Only GRBG10 support	L4T 32.4.3 / Jetpack 4.4	Nano
ISP usage through NvArgusCameraSrc	Only GRBG10 support	L4T 32.5 / Jetpack 4.5	Nano
Gain control		L4T 32.4.3 / Jetpack 4.4	Nano
Exposure control		L4T 32.4.3 / Jetpack 4.4	Nano
Window control		L4T 32.4.3 / Jetpack 4.4	Nano
Gain control		L4T 32.5 / Jetpack 4.5	Nano
Exposure control		L4T 32.5 / Jetpack 4.5	Nano
Window control		L4T 32.5 / Jetpack 4.5	Nano
2 simultaneous cameras		L4T 32.5 / Jetpack 4.5	Nano
2 simultaneous cameras		L4T 32.4.3 / Jetpack 4.4	Nano

Enabling the Driver

In order to use this driver, you have to patch and compile the kernel source using JetPack:

• Follow the instructions in (Downloading sources) to get the kernel source code.

• Once you have the source code, apply the following three patches in order to add the changes required for the AR1335 camera at kernel and dtb level.

4.2.2_ar1335.patch

• Follow the instructions in (Build Kernel) for building the kernel, and then flash the image.

Make sure to enable AR1335 driver support:

make menuconfig

-> Device Drivers                                                                                                                        
  -> Multimedia support                                                                                           
    -> NVIDIA overlay Encoders, decoders, sensors and other helper chips 
       -> <*> AR1335 camera sensor support

Using the Driver

GStreamer Examples

Capture and Display

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1' ! autovideosink

Video Encoding

CAPS="video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1"

gst-launch-1.0 nvarguscamerasrc sensor-id=0 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test.ts

The sensor will capture in the 3840x2160@30 mode and the pipeline will encode the video and save it into test.ts file.

Dual Capture + Encoding

gst-launch-1.0 nvarguscamerasrc sensor-id=0 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test_0.ts nvarguscamerasrc sensor-id=1 num-buffers=500 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! mpegtsmux ! filesink location=test_1.ts

The sensor will capture from 3 cameras in the 3840x2160@30 mode and the pipeline will encode the videos and save them into test_0.ts and test_1.ts files.

Window Control Examples

ROI Selection

The AR1335 driver exposes x_addr and y_addr controls in order to select the region of interest. These controls can be used for all available resolutions.

v4l2-ctl -d /dev/video0 -c x_addr=100 -c y_addr=300

Bin/Scale Enable

By default 1080p mode has an output frame size of about 1920x1080 but covers a full ROI of 3840x2160. However, the binning/scaling process can be disabled using this control to selects a 1920x1080 ROI without changing the frame size. The frame that comes from the camera needs to have a constant size once the streaming begins, so the idea is to start the capture using 1080p mode with the binning/scaling enabled covering the 3840x2160 region and then, select a 1920x1080 ROI in the desired area. This ensures a constant frame size avoiding inconsistent streaming.

v4l2-ctl -d /dev/video0 -c bin_scale_enable=0

Analog gain

Analog gain can range from 25 to 2325.

v4l2-ctl -c gain=<value>

Exposure

Exposure time is in us, with maximum value being 127595us.

v4l2-ctl -c exposure=<value>

Digital Gain Examples

The driver allows supporting manual white balance handle through red, green, blue digital gain controls. This gain controls only affect the digital contributions to modify the image colors and keep the current analog gain.

v4l2-ctl -d /dev/video0 -c red_gain=100 greenr_gain=100 greenb_gain=100 blue_gain=64

Temperature Reading Example

The AR1335 temperature sensor works only when the camera sensor is in streaming mode, so the temperature will be set to zero when the sensor is in low-power-mode.

The thermal_zone devices registered will be available on /sys/class/thermal/, so you can read temperature as the following:

cat /sys/devices/virtual/thermal/thermal_zone8/temp

Test Patterns Example

Support was added to allow you to enable test pattern mode and select your desired pattern between the following options:

0: Normal operation: no test pattern
1: Solid color
2: 100% color bars
3: Fade−to−gray color bars
4: Walking 1s (10-bit)
5: Walking 1s (8-bit)

Try different test patterns selecting a number between 1 and 5:

v4l2-ctl -d /dev/video0 -c test_pattern=1
v4l2-ctl -d /dev/video0 -c test_pattern=2
v4l2-ctl -d /dev/video0 -c test_pattern=3
v4l2-ctl -d /dev/video0 -c test_pattern=4
v4l2-ctl -d /dev/video0 -c test_pattern=5

Performance

ARM Load

‎

‎

‎

Tegrastats display the following output when capturing with the sensor:

RAM 1263/7855MB (lfb 1501x4MB) CPU [0%@2035,off,off,0%@2035,0%@2035,0%@2035] 
RAM 1263/7855MB (lfb 1501x4MB) CPU [23%@960,off,off,17%@960,16%@960,23%@960] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [17%@345,off,off,17%@345,18%@345,20%@345] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,16%@345,18%@345,15%@345] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,13%@345,15%@345,14%@345] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,15%@345,12%@345,15%@345] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,15%@345,15%@345,16%@345] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [20%@345,off,off,18%@345,18%@345,17%@345] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [16%@345,off,off,15%@345,27%@345,17%@345] 
RAM 1263/7855MB (lfb 1500x4MB) CPU [19%@345,off,off,18%@345,17%@345,19%@345] 

Framerate

Using the next pipeline we were able to measure the framerate for single capture with perf element:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)30/1' ! perf  ! fakesink

GST-PERF INFO -->  Timestamp: 0:07:19.108602798; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:20.141189052; Bps: 782; fps: 30.3 
GST-PERF INFO -->  Timestamp: 0:07:21.174265435; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:22.207318757; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:23.240543516; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:24.273697886; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:25.306822764; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:26.340117514; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:27.373087284; Bps: 782; fps: 30.3 
GST-PERF INFO -->  Timestamp: 0:07:28.406069581; Bps: 782; fps: 30.3 
GST-PERF INFO -->  Timestamp: 0:07:29.439238457; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:30.472398102; Bps: 782; fps: 30.0 
GST-PERF INFO -->  Timestamp: 0:07:31.472948042; Bps: 808; fps: 30.0 

The results show the framerate constant at 30FPS that use nvarguscamerasrc and passing frames through the ISP to convert from Bayer to YUV.

Latency measurement

‎

‎

‎

The glass to glass latency measured is about ~150 ms with nvarguscamerasrc.

RidgeRun Resources

Quick Start Client Engagement Process RidgeRun Blog Homepage Technical and Sales Support RidgeRun Online Store RidgeRun Videos Contact Us

Contact Us Visit our Main Website for the RidgeRun Products and Online Store. RidgeRun Engineering informations are available in RidgeRun Professional Services, RidgeRun Subscription Model and Client Engagement Process wiki pages. Please email to support@ridgerun.com for technical questions and contactus@ridgerun.com for other queries. Contact details for sponsoring the RidgeRun GStreamer projects are available in Sponsor Projects page.