ON Semiconductor AP1302 Linux driver

Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help.

AP1302 Features

The AP1302 is an image sensor with the following features:

• Up to 13 Mp (4224x3156) sensor support.
• 450 Mp/second processing (13Mp at 30/1080p at 120/720p at 240 - subject to sensor limitations).
• Dual on-chip 32-bit RISC processor cores.
• Optimized for operation with Aptina's Clarity+™ Pixel technology sensors as well as Bayer pattern CMOS Image Sensors (CIS).
• Designed to support Aptina sensors that provide video interlaced HDR (iHDR).
• Color gaining and gamma correction.
• Frame rate reduction, image resizing and clipping.
• Auto exposure, auto white balance, auto focus, auto flicker detection and control.
• Adaptive Tone Mapping, Local Tone Mapping.
• Scene mode support: scene selection and automatic detection.
• Preferred Color reproduction.
• Motion-assisted anti-shake with ISO control, face detection and smile detection.
• Video (Movie) stabilization assist for video resolutions up to 1080p.
• Batch-mode capture, Best-picture assist.
• Smooth digital zoom up to 16X.
• Exposure bracketing for improved dynamic range.
• OSD, special effects.
• Test pattern generator.
• Programmable JPEG encoder with EXIF header support.
• Scalado SpeedTagsTM Encode support.
• Two-wire serial interface (I2C) for sensor and peripheral control and register access supporting Standard (100kbps), Fast mode (400kbps), FM+ (1Mbps) and HS (3.4 Mbps).
• Dual sensor support (second camera or 3D bridge application).
• 6 MIPI data lanes shared between two sensor interfaces (4+2/3+3/4+1/3+2).
• SPI External EEPROM/FLASH (4 M-bit) support for code and/or configuration data storage. Additional four-wire serial interface (SPI) for register access supporting up to 25 Mbps.
• 12 GPIOs (shared functionality with SPI master and slave, second I2C master).
• Xenon, LED Flash, pulsed LED Flash Support.
• Fail-safe IO, programmable slew-rate control.

RidgeRun has developed a driver for the Jetson TX2 with the following support:

• L4T 32.2.0 and Jetpack 4.2
• V4L2 Media controller driver
• Supported Formats:
  • Raw10
  • UYVY
• Supported Resolutions:
  • 640x360@30
  • 640x400@30
  • 1280x720@30
  • 1920x1080@30
  • 3840x2160@25
  • 3968x2160@25
  • 4096x2160@20
  • 3968x3120@15
  • 4208x3120@15

Enabling the driver

In order to use this driver, you have to patch and compile the kernel source using JetPack:

• Follow the instructions in (Downloading sources) to get the kernel source code.

• Once you have the source code, apply the following patche in order to add the changes required for the AP1302 camera at kernel and dtb level.

4.2_ap1302.patch

• Follow the instructions in (Build Kernel) for building the kernel, and then flash the image.

Make sure to enable AP1302 driver support:

make menuconfig

-> Device Drivers                                                                                                                        
  -> Multimedia support                                                                                           
    -> NVIDIA overlay Encoders, decoders, sensors and other helper chips
       -> <*> AP1302 camera sensor support

Using the driver

GStreamer examples

Capture

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=(int)4208, height=(int)3120, format=(string)UYVY' ! autovideosink

The sensor will capture in the 4208x3120@15 mode

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Video Encoding

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=(int)4208, height=(int)3120, format=(string)UYVY' ! nvvidconv ! omxh264enc ! mpegtsmux ! filesink location=test.ts

The sensor will capture in the 4208x3120@15 mode and the pipeline will encode the video and save it into test.ts file.

Performance

ARM Load

Tegrastats display the following output when capturing with the sensor:

RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [27%@2035,off,off,17%@2035,12%@2035,9%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [31%@2035,off,off,17%@2035,13%@2035,4%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [27%@2035,off,off,18%@2035,9%@2035,13%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [51%@2035,off,off,4%@2035,9%@2035,3%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [51%@2035,off,off,4%@2035,8%@2035,0%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [44%@2035,off,off,0%@2035,22%@2035,0%@2035]
RAM 667/7860MB (lfb 1687x4MB) SWAP 0/3930MB (cached 0MB) CPU [46%@2035,off,off,8%@2035,13%@2035,0%@2035]

Framerate

Using the next pipeline we were able to measure the framerate for single capture with the perf element:

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, width=(int)4208, height=(int)3120, format=(string)UYVY' ! perf ! fakesink

INFO: perf: perf0; timestamp: 0:20:41.612503008; bps: 0,000; mean_bps: 0,000; fps: 0,000; mean_fps: 0,000
INFO: perf: perf0; timestamp: 0:20:42.641163584; bps: 3279792536,737; mean_bps: 3279792536,737; fps: 15,554; mean_fps: 15,554
INFO: perf: perf0; timestamp: 0:20:43.684277280; bps: 3234348559,450; mean_bps: 3257070548,093; fps: 15,339; mean_fps: 15,446
INFO: perf: perf0; timestamp: 0:20:44.728300096; bps: 3231532135,405; mean_bps: 3248557743,864; fps: 15,325; mean_fps: 15,406
INFO: perf: perf0; timestamp: 0:20:45.773840800; bps: 3226840683,574; mean_bps: 3243128478,791; fps: 15,303; mean_fps: 15,380
INFO: perf: perf0; timestamp: 0:20:46.816163744; bps: 3236802278,431; mean_bps: 3241863238,719; fps: 15,350; mean_fps: 15,374
INFO: perf: perf0; timestamp: 0:20:47.862452032; bps: 3224535071,925; mean_bps: 3238975210,920; fps: 15,292; mean_fps: 15,361
INFO: perf: perf0; timestamp: 0:20:48.903999360; bps: 3239212649,586; mean_bps: 3239009130,730; fps: 15,362; mean_fps: 15,361
INFO: perf: perf0; timestamp: 0:20:49.950199680; bps: 3224806201,550; mean_bps: 3237233764,582; fps: 15,293; mean_fps: 15,352
INFO: perf: perf0; timestamp: 0:20:50.991857952; bps: 3238867650,446; mean_bps: 3237415307,456; fps: 15,360; mean_fps: 15,353
INFO: perf: perf0; timestamp: 0:20:52.035787040; bps: 3231822274,886; mean_bps: 3236856004,199; fps: 15,327; mean_fps: 15,351
INFO: perf: perf0; timestamp: 0:20:53.082210528; bps: 3224118455,567; mean_bps: 3235698045,232; fps: 15,290; mean_fps: 15,345
INFO: perf: perf0; timestamp: 0:20:54.126004192; bps: 3232241578,351; mean_bps: 3235410006,326; fps: 15,329; mean_fps: 15,344
INFO: perf: perf0; timestamp: 0:20:55.167571168; bps: 3239151545,450; mean_bps: 3235697817,028; fps: 15,361; mean_fps: 15,345
INFO: perf: perf0; timestamp: 0:20:56.211503776; bps: 3231811377,617; mean_bps: 3235420214,212; fps: 15,327; mean_fps: 15,344
INFO: perf: perf0; timestamp: 0:20:57.255406240; bps: 3231904700,246; mean_bps: 3235185846,615; fps: 15,327; mean_fps: 15,343
INFO: perf: perf0; timestamp: 0:20:58.301784736; bps: 3224257085,650; mean_bps: 3234502799,054; fps: 15,291; mean_fps: 15,339
INFO: perf: perf0; timestamp: 0:20:59.343773824; bps: 3237839358,256; mean_bps: 3234699067,243; fps: 15,355; mean_fps: 15,340

The results show the framerate constant at 15FPS that uses v4l2src.

RidgeRun Resources

Quick Start Client Engagement Process RidgeRun Blog Homepage Technical and Sales Support RidgeRun Online Store RidgeRun Videos Contact Us

Contact Us Visit our Main Website for the RidgeRun Products and Online Store. RidgeRun Engineering informations are available in RidgeRun Professional Services, RidgeRun Subscription Model and Client Engagement Process wiki pages. Please email to support@ridgerun.com for technical questions and contactus@ridgerun.com for other queries. Contact details for sponsoring the RidgeRun GStreamer projects are available in Sponsor Projects page.