i.MX8 - Multimedia - GStreamer Support - Hardware-accelerated plugins
< IMX8 | Multimedia
|
Contents
Introduction
This wiki is intended to be used as a reference for the hardware-accelerated capabilities of the i.MX8 Series Applications Processors. This family of SOCs comes in different versions, each with specialized uses according to the hardware configuration. While this assortment of boards provides a solution for each use case, it makes it difficult to find real performance metrics and information for each of the versions available. Therefore, in this wiki you will find compatibility information and performance metrics of the different hardware-accelerated GStreamer plugins available in the i.MX8 platform.
iMX8 vendors
- SolidRun (iMX8-SOM)
- Variscite (VAR-SOM-MX8)
- CompuLab (CL-SOM-IMX8)
- Toradex (Apalis iMX8)
- Boundary Devices (Nitrogen8M)
- PhyTec (PhyCore-iMX8)
- Seco (Q7-C25)
- iWave (iMX8-Quadmax-Smarc SOM)
Hardware-accelerated plugins
Test platforms
The different plugins available for iMX8 platforms depend on the specific configuration of the devices. The following devices were used for testing:
DART-MX8M-PLUS (i.MX8M Plus)
- 2D GPU: GC520L
- 3D GPU: GC7000 UltraLite
- 2D APIs: OpenVG 1.1 (uses 3D GPU), G2D
- 3D APIs: OpenGL ES 3.1, Vulkan
- Video decode (VPU):
- 1080p60 HEVC/H.265 Main, Main 10 (up to level 5.1)
- 1080p60 VP9 Profile 0, 2
- 1080p60 VP8
- 1080p60 AVC/H.264 Baseline, Main, High decoder
- Video encode (VPU):
- 1080p60 AVC/H.264 encoder
- 1080p60 HEVC/H.265 encoder
- ISP: 2x ISP supporting 375 Mpixel/s aggregate performance and up to 3-exposure HDR processing.
- When one camera is used, support up to 12MP@30fps or 4kp45
- When two cameras are used, each supports up to 1080p80
- NPU: 2.3 TOP/s Neural Network performance.
- Camera interfaces: 2 MIPI CSI
- HDMI: v2.0 4k@60
- MIPI DSI: x1 4-Lane (1080p60)
i.MX8M EVK and Nitrogen8M (i.MX8M Quad)
- 3D GPU: GC7000 Lite
- 2D APIs: OpenVG 1.1 (uses 3D GPU)
- 3D APIs: OpenGL ES 3.1, Vulkan
- Video decode (VPU):
- 4Kp60 HEVC/H.265 main, and main 10 decoder
- 4Kp60 VP9 decoder
- 4Kp30 AVC/H.264 decoder
- 1080p60 MPEG-2, MPEG-4p2, VC-1, VP8, RV9, AVS, MJPEG, H.263 decoder
- Video encode (SW only):
- 1080p30 (h.264)
- Camera interfaces: 2 MIPI CSI
- HDMI: v2.0 4k@60
- MIPI DSI: x1 4-Lane (1080p60)
Nitrogen8M Mini (i.MX8M Mini)
- 3D GPU: GC7000 NanoUltra
- 2D GPU: GC320
- 2D APIs: OpenVG 1.1 (uses 3D GPU), G2D
- 3D APIs: OpenGL ES 3.1, Vulkan
- Video decode (VPU):
- 1080p60 VP9 Profile 0, 2 (10-bit)
- 1080p60 HEVC/H.265 decoder
- 1080p60 AVC/H.264 Baseline, Main, High decoder
- 1080p60 VP8
- Video encode (VPU):
- 1080p60 AVC/H.264 encoder
- 1080p60 VP8
- Camera interfaces: 1 MIPI CSI
- MIPI DSI: x1 4-Lane (1080p60)
Latency measurement
Latency measurements were performed using gst-shark interlatency tracer by exporting the following variables before executing the pipelines.
export GST_DEBUG="GST_TRACER:7" GST_TRACERS="interlatency"It was also noted that printing traces to the serial console caused distorted latency results. The problem is solved by connecting to the board using SSH or by dumping the traces to a file. The latency tests were performed adding the following to the commands:
&> /dev/nullVideo decoding
All i.MX8 boards with a Video Processing Unit (VPU) have the vpudec element capable of different formats in 1080p and some even 4K. The following is the gst-inspect of the vpudec element in the i.MX8M Plus.
gst-inspect-1.0 vpudec (i.MX8M Plus)
|
|---|
root@imx8mp-var-dart:~# gst-inspect-1.0 vpudec
Factory Details:
Rank primary + 1 (257)
Long-name IMX VPU-based video decoder
Klass Codec/Decoder/Video
Description Decode compressed video to raw data
Author Multimedia Team <shmmmw@freescale.com>
Plugin Details:
Name vpu
Description VPU video codec
Filename /usr/lib/gstreamer-1.0/libgstvpu.so
Version 4.6.0
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstVpuDec
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-h265
video/x-vp9
video/x-vp8
video/x-h264
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
disable-reorder : disable vpu reorder when end to end streaming
flags: readable, writable
Boolean. Default: false
frame-drop : enable adaptive frame drop for smoothly playback
flags: readable, writable
Boolean. Default: true
frame-plus : set number of addtional frames for smoothly playback
flags: readable, writable
Unsigned Integer. Range: 0 - 16 Default: 3
or
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: 10
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "vpudec0"
y for MJPEG)
flags: readable, writable
Enum "GstVpuDecOutputFormat" Default: 0, "auto"
fault)
(1): NV12 - NV12 format
(2): I420 - I420 format
(3): YV12 - YV12 format
(4): Y42B - Y42B format
(5): NV16 - NV16 format
(6): Y444 - Y444 format
(7): NV24 - NV24 format
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: true
use-vpu-memory : use vpu allocate video frame buffer
flags: readable, writable
Boolean. Default: true |
H.264
Performance
1080p
File: 1080p-h264-24fps.mp4
gst-launch-1.0 filesrc location="1080p-h264-24fps.mp4" ! qtdemux ! queue ! vpudec ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 2 % | 250.812 | 17 |
| iMX8M EVK (i.MX8M Quad) | 6 % | 179.214 | 19 |
| Nitrogen8M (i.MX8M Quad) | 6 % | 171.293 | 29 |
| Nitrogen8M Mini (i.MX8M Mini) | 6 % | 172.197 | 21 |
File: 1080p-h264-60fps.mp4
gst-launch-1.0 filesrc location="1080p-h264-60fps.mp4" ! qtdemux ! queue ! vpudec ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 2 % | 184.335 | 19 |
| iMX8M EVK (i.MX8M Quad) | 2 % | 136.260 | 23 |
| Nitrogen8M (i.MX8M Quad) | 2 % | 128.650 | 25 |
| Nitrogen8M Mini (i.MX8M Mini) | 2 % | 130.824 | 24 |
HEVC
Performance
1080p
gst-launch-1.0 filesrc location="1080p-hevc-30fps.mkv" ! matroskademux ! queue ! vpudec ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 7 % | 181.232 | 29 |
| iMX8M EVK (i.MX8M Quad) | 6 % | 143.147 | 35 |
| Nitrogen8M (i.MX8M Quad) | 6 % | 133.876 | 36 |
| Nitrogen8M Mini (i.MX8M Mini) | 7 % | 144.752 | 35 |
4K
gst-launch-1.0 filesrc location="4k-hevc-60fps.mkv" ! matroskademux ! queue ! vpudec ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | Not supported | ||
| iMX8M EVK (i.MX8M Quad) | 3 % | 94.255 | 29 |
| Nitrogen8M (i.MX8M Quad) | 3 % | 91.139 | 31 |
| Nitrogen8M Mini (i.MX8M Mini) | Not supported |
VP8
Performance
1080p
gst-launch-1.0 filesrc location="1080p-vp8-60fps.mkv" ! matroskademux ! queue ! vpudec ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 2 % | 228.686 | 18 |
| iMX8M EVK (i.MX8M Quad) | 2 % | 150.754 | 20 |
| Nitrogen8M (i.MX8M Quad) | 2 % | 148.773 | 26 |
| Nitrogen8M Mini (i.MX8M Mini) | 2 % | 154.011 | 27 |
VP9
Performance
4K HDR
gst-launch-1.0 filesrc location="4k-vp9-hdr-60fps.mkv" ! matroskademux ! queue ! vpudec ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
DART-MX8M-PLUS (i.MX8M Plus) | Not supported |
|---|---|---|---|
| iMX8M EVK (i.MX8M Quad) | 3 % | 95.341 | 57 |
| Nitrogen8M (i.MX8M Quad) | 3 % | 92.792 | 62 |
| Nitrogen8M Mini (i.MX8M Mini) | Not supported |
JPEG
Performance
1080p
gst-launch-1.0 filesrc location="1080p-mjpeg-30fps.mov" ! qtdemux ! queue ! vpudec ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
DART-MX8M-PLUS (i.MX8M Plus) | Not supported |
|---|---|---|---|
| iMX8M EVK (i.MX8M Quad) | 4 % | 158.914 | 11 |
| Nitrogen8M (i.MX8M Quad) | 4 % | 157.880 | 11 |
| Nitrogen8M Mini (i.MX8M Mini) | Not supported |
Video encoding
Some i.MX8 boards VPUs are capable of encoding 1080p video to some formats.
H.264
gst-inspect-1.0 vpuenc_h264 (i.MX8M Plus)
|
|---|
root@imx8mp-var-dart:~# gst-inspect-1.0 vpuenc_h264
Factory Details:
Rank primary + 1 (257)
Long-name IMX VPU-based AVC/H264 video encoder
Klass Codec/Encoder/Video
Description Encode raw data to compressed video
Author Multimedia Team <shmmmw@freescale.com>
Plugin Details:
Name vpu
Description VPU video codec
Filename /usr/lib/gstreamer-1.0/libgstvpu.so
Version 4.6.0
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----vpuenc_h264
Implemented Interfaces:
GstPreset
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
ng)BGRx, (string)BGR16 }
width: [ 64, 1920 ]
height: [ 64, 1088 ]
framerate: [ 0/1, 2147483647/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h264
stream-format: { (string)avc, (string)byte-stream }
alignment: { (string)au, (string)nal }
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
bitrate : set bit rate in kbps (0 for automatic)
flags: readable, writable
Unsigned Integer. Range: 0 - 2147483647 Default: 0
gop-size : How many frames a group-of-picture shall contain
flags: readable, writable
Unsigned Integer. Range: 0 - 32767 Default: 30
n nanoseconds
flags: readable, writable
ult: 0
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "vpuenc_h264-0"
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
quant : set quant value: H.264(0-51) (-1 for automatic)
flags: readable, writable
Integer. Range: -1 - 51 Default: -1
stream-multislice : the number of slices a picture contains
flags: readable, writable
Integer. Range: 1 - 2147483647 Default: 1 |
Performance
Using OV5640 camera at 1080p 30 fps:
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,width=1920,height=1080 ! queue ! vpuenc_h264 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 1 % | 30.004 | 4.7 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
Using videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080 ! queue ! vpuenc_h264 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 26 % | 65.335 | 5.1 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | 26 % | 62.486 | 13.7 |
These results are probably impacted by the CPU load of videotestsrc.
HEVC
gst-inspect-1.0 vpueenc_hevc (i.MX8M Plus)
|
|---|
root@imx8mp-var-dart:~# gst-inspect-1.0 vpuenc_hevc
Factory Details:
Rank primary + 1 (257)
Long-name IMX VPU-based HEVC video encoder
Klass Codec/Encoder/Video
Description Encode raw data to compressed video
Author Multimedia Team <shmmmw@freescale.com>
Plugin Details:
Name vpu
Description VPU video codec
Filename /usr/lib/gstreamer-1.0/libgstvpu.so
Version 4.6.0
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----vpuenc_hevc
Implemented Interfaces:
GstPreset
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
ng)BGRx, (string)BGR16 }
width: [ 64, 1920 ]
height: [ 64, 1088 ]
framerate: [ 0/1, 2147483647/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h265
variant: itu
width: [ 64, 1920 ]
height: [ 64, 1088 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
bitrate : set bit rate in kbps (0 for automatic)
flags: readable, writable
Unsigned Integer. Range: 0 - 2147483647 Default: 0
gop-size : How many frames a group-of-picture shall contain
flags: readable, writable
Unsigned Integer. Range: 0 - 32767 Default: 30
n nanoseconds
flags: readable, writable
ult: 0
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "vpuenc_hevc0"
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
quant : set quant value: HEVC(0-51) (-1 for automatic)
flags: readable, writable
Integer. Range: -1 - 51 Default: -1
stream-multislice : the number of slices a picture contains
flags: readable, writable
Integer. Range: 1 - 2147483647 Default: 1 |
Performance
Using OV5640 camera at 1080p 30 fps:
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,width=1920,height=1080 ! queue ! vpuenc_hevc ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 1 % | 30.001 | 4.3 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | Not supported |
Using videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080 ! queue ! vpuenc_hevc ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 26 % | 65.461 | 4.7 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | Not supported |
These results are probably impacted by the CPU load of videotestsrc.
VP8
gst-inspect-1.0 vpueenc_vp8 (i.MX8M Mini)
|
|---|
root@nitrogen8mm:~# gst-inspect-1.0 vpuenc_vp8
Factory Details:
Rank primary + 1 (257)
Long-name IMX VPU-based VP8 video encoder
Klass Codec/Encoder/Video
Description Encode raw data to compressed video
Author Multimedia Team <shmmmw@freescale.com>
Plugin Details:
Name vpu
Description VPU video codec
Filename /usr/lib/gstreamer-1.0/libgstvpu.so
Version 4.6.1
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----vpuenc_vp8
Implemented Interfaces:
GstPreset
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-vp8
variant: itu
width: [ 64, 1920 ]
height: [ 64, 1088 ]
framerate: [ 0/1, 2147483647/1 ]
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YUY2, (string)UYVY, (string)RGBA, (string)RGBx, (string)RGB16, (string)RGB15, (st}
width: [ 64, 1920 ]
height: [ 64, 1088 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
bitrate : set bit rate in kbps (0 for automatic)
flags: readable, writable
Unsigned Integer. Range: 0 - 2147483647 Default: 0
gop-size : How many frames a group-of-picture shall contain
flags: readable, writable
Unsigned Integer. Range: 0 - 32767 Default: 30
min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "vpuenc_vp8-0"
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
quant : set quant value: VP8(1-31) (-1 for automatic)
flags: readable, writable
Integer. Range: -1 - 31 Default: -1 |
Performance
Using OV5640 camera at 1080p 30 fps:
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,width=1920,height=1080 ! queue ! vpuenc_vp8 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | Not supported | ||
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
Using videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080 ! queue ! vpuenc_vp8 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | Not supported | ||
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | 27 % | 63.150 | 14.3 |
These results are probably impacted by the CPU load of videotestsrc.
Video conversion
G2D API
gst-inspect-1.0 imxvideoconvert_g2d (i.MX8M Plus)
|
|---|
root@imx8mp-var-dart:~# gst-inspect-1.0 imxvideoconvert_g2d
Factory Details:
Rank primary + 1 (257)
Long-name IMX g2d Video Converter
Klass Filter/Converter/Video
Description Video CSC/Resize/Rotate.
Author Multimedia Team <shmmmw@freescale.com>
Plugin Details:
Name imxvideoconvert
Description IMX Video Convert Plugins
Filename /usr/lib/gstreamer-1.0/libgstimxvideoconvert.so
Version 4.6.0
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseTransform
+----GstVideoFilter
+----imxvideoconvert_g2d
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
string)YV12, (string)NV16, (string)NV21 }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
string)YV12, (string)NV16, (string)NV21 }
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
ng)xBGR }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
ng)xBGR }
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
composition-meta-enable: Enable overlay composition meta processing
flags: readable, writable
Boolean. Default: false
ideo overlay composition will blended onto input buffer
flags: readable, writable
Boolean. Default: false
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "imxvideoconvert_g2d0"
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events
flags: readable, writable
Boolean. Default: true
rotation : Rotation that shall be applied to output frames
flags: readable, writable
Enum "ImxVideoConvertRotationMode" Default: 0, "none"
(0): none - No rotation
(1): rotate-90 - Rotate 90 degrees
(2): rotate-180 - Rotate 180 degrees
(3): rotate-270 - Rotate 270 degrees
(4): horizontal-flip - Flip horizontally
(5): vertical-flip - Flip vertically
videocrop-meta-enable: Enable videocrop meta processing
flags: readable, writable
Boolean. Default: false |
Performance
Color Space Conversion (CSC)
Using OV5640 camera at 1080p 30 fps:
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! imxvideoconvert_g2d ! video/x-raw,format=RGB16 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 0 % | 30.001 | 4.6 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
Using videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! imxvideoconvert_g2d ! video/x-raw,format=RGB16 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 26 % | 34.164 | 4.5 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | 26 % | 34.172 | 5.7 |
These results are probably impacted by the CPU load of videotestsrc.
Resize
Using OV5640 camera at 1080p 30 fps:
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=NV12,width=1920,height=1080 ! imxvideoconvert_g2d ! video/x-raw, width=1280, height=720 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 0 % | 30.002 | 4.2 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
Using videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! imxvideoconvert_g2d ! video/x-raw, width=1280, height=720 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 22 % | 28.152 | 4.2 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | 22 % | 27.213 | 5.5 |
OpenGL
gst-inspect-1.0 glcolorconvert (nitrogen8M)
|
|---|
root@nitrogen8m:~# gst-inspect-1.0 glcolorconvert
Factory Details:
Rank none (0)
Long-name OpenGL color converter
Klass Filter/Converter/Video
Description Converts between color spaces using OpenGL shaders
Author Matthew Waters <matthew@centricular.com>
Plugin Details:
Name opengl
Description OpenGL plugin
Filename /usr/lib/gstreamer-1.0/libgstopengl.so
Version 1.18.0
License LGPL
Source module gst-plugins-base
Source release date 2020-09-08
Binary package GStreamer Base Plug-ins source release
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseTransform
+----GstGLBaseFilter
+----GstGLColorConvertElement
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw(memory:GLMemory)
format: { (string)RGBA, (string)RGB, (string)RGBx, (string)BGR, (string)BGRx, (string)BGRA, (string)xRGB, (string)xBGR, (string)ARGB, (string)ABGR, (string)Y444, (string)I420, (string)YV12, (string)Y42B, (string)Y41B, (string)NV12, (string)NV21, (string)NV16, (string)NV61, (s}
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
texture-target: { (string)2D, (string)rectangle, (string)external-oes }
video/x-raw(memory:GLMemory, meta:GstVideoOverlayComposition)
format: { (string)RGBA, (string)RGB, (string)RGBx, (string)BGR, (string)BGRx, (string)BGRA, (string)xRGB, (string)xBGR, (string)ARGB, (string)ABGR, (string)Y444, (string)I420, (string)YV12, (string)Y42B, (string)Y41B, (string)NV12, (string)NV21, (string)NV16, (string)NV61, (s}
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
texture-target: { (string)2D, (string)rectangle, (string)external-oes }
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw(memory:GLMemory)
format: { (string)RGBA, (string)RGB, (string)RGBx, (string)BGR, (string)BGRx, (string)BGRA, (string)xRGB, (string)xBGR, (string)ARGB, (string)ABGR, (string)Y444, (string)I420, (string)YV12, (string)Y42B, (string)Y41B, (string)NV12, (string)NV21, (string)NV16, (string)NV61, (s}
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
texture-target: { (string)2D, (string)rectangle, (string)external-oes }
video/x-raw(memory:GLMemory, meta:GstVideoOverlayComposition)
format: { (string)RGBA, (string)RGB, (string)RGBx, (string)BGR, (string)BGRx, (string)BGRA, (string)xRGB, (string)xBGR, (string)ARGB, (string)ABGR, (string)Y444, (string)I420, (string)YV12, (string)Y42B, (string)Y41B, (string)NV12, (string)NV21, (string)NV16, (string)NV61, (s}
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
texture-target: { (string)2D, (string)rectangle, (string)external-oes }
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
context : Get OpenGL context
flags: readable
Object of type "GstGLContext"
disable-passthrough : Disable passthrough mode
flags: readable, writable
Boolean. Default: false
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "glcolorconvertelement0"
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events
flags: readable, writable
Boolean. Default: true |
Performance
Color Space Conversion (CSC)
Using OV5640 camera at 1080p 30 fps:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! queue ! glupload ! glcolorconvert ! gldownload ! video/x-raw,format=RGB16 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | OpenGL upload latency (ms) |
Element latency (ms) |
OpenGL download latency (ms) |
|---|---|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 2% | 30.007 | 0.51 | 0.64 | 5.98 |
| iMX8M EVK (i.MX8M Quad) | 2% | 30.004 | 1.26 | 2.45 | 6.52 |
| Nitrogen8M (i.MX8M Quad) | Not tested | ||||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
Using videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! glupload ! glcolorconvert ! gldownload ! video/x-raw,format=RGB ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | OpenGL upload latency (ms) |
Element latency (ms) |
OpenGL download latency (ms) |
|---|---|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 26% | 34.253 | 0.38 | 0.65 | 0.13 |
| iMX8M EVK (i.MX8M Quad) | 27% | 28.421 | 0.50 | 2.18 | 0.10 |
| Nitrogen8M (i.MX8M Quad) | 26% | 29.485 | 0.44 | 2.10 | 0.13 |
| Nitrogen8M Mini (i.MX8M Mini) | 26% | 35.690 | 0.29 | 0.55 | 0.07 |
These results are probably impacted by the CPU load of videotestsrc.
Resize
Using OV5640 camera at 1080p 30 fps:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! queue ! glupload ! glcolorscale ! gldownload ! video/x-raw, width=1280, height=720 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | OpenGL upload latency (ms) |
Element latency (ms) |
OpenGL download latency (ms) |
|---|---|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 2% | 30.005 | 0.45 | 0.70 | 3.28 |
| iMX8M EVK (i.MX8M Quad) | 2% | 30.003 | 1.46 | 3.60 | 2.14 |
| Nitrogen8M (i.MX8M Quad) | Not tested | ||||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
Using videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! glupload ! glcolorscale ! gldownload ! video/x-raw, width=1280, height=720 ! perf print-cpu-load=true ! queue ! fakesink| Board | CPU % | FPS | OpenGL upload latency (ms) |
Element latency (ms) |
OpenGL download latency (ms) |
|---|---|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 26% | 34.123 | 0.38 | 0.69 | 5.23 |
| iMX8M EVK (i.MX8M Quad) | 27% | 28.352 | 1.10 | 1.62 | 6.61 |
| Nitrogen8M (i.MX8M Quad) | 26% | 29.476 | 1.14 | 1.60 | 6.72 |
| Nitrogen8M Mini (i.MX8M Mini) | 27% | 35.207 | 0.27 | 0.81 | 8.56 |
These results are probably impacted by the CPU load of videotestsrc.
Video composition
G2D API
Note: the imxcompositor_g2d plugin caused an exception when not adding output caps. Also, internal errors are thrown when any of the images is out of the screen.
gst-inspect-1.0 imxcompositor_g2d (i.MX8M Plus)
|
|---|
root@imx8mp-var-dart:~# gst-inspect-1.0 imxcompositor_g2d
Factory Details:
Rank primary + 1 (257)
Long-name IMX g2d Video Compositor
Klass Filter/Editor/Video/ImxCompositor
Description Composite multiple video streams
Author Multimedia Team <shmmmw@freescale.com>
Plugin Details:
Name imxcompositor
Description IMX Video Composition Plugins
Filename /usr/lib/gstreamer-1.0/libgstimxcompositor.so
Version 4.6.0
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstAggregator
+----GstVideoAggregator
+----imxcompositor_g2d
Implemented Interfaces:
GstChildProxy
Pad Templates:
SINK template: 'sink_%u'
Availability: On request
Capabilities:
video/x-raw
string)YV12, (string)NV16, (string)NV21 }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
string)YV12, (string)NV16, (string)NV21 }
Type: GstImxCompositorPad
Pad Properties:
alpha : Alpha of the picture
flags: readable, writable, controllable
ault: 1
uld be used when scaling and converting this pad's video frames
flags: readable, writable
Boxed pointer of type "GstStructure"
emit-signals : Send signals to signal data consumption
flags: readable, writable
Boolean. Default: false
height : Target height of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
keep-ratio : Keep the video aspect ratio after resize
flags: readable, writable, controllable
Boolean. Default: false
behaviour on EOS is not affected
, PAUSED or PLAYING state
Default: 18446744073709551615
OS
flags: readable, writable, controllable
Boolean. Default: false
rotate : Rotation that shall be applied to input frames
flags: readable, writable, controllable
Enum "ImxCompositorRotationMode" Default: 0, "none"
(0): none - No rotation
(1): rotate-90 - Rotate 90 degrees
(2): rotate-180 - Rotate 180 degrees
(3): rotate-270 - Rotate 270 degrees
(4): horizontal-flip - Flip horizontally
(5): vertical-flip - Flip vertically
width : Target width of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
xpos : X Position of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
ypos : Y Position of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
zorder : Z Order of the picture
flags: readable, writable, controllable
Unsigned Integer. Range: 0 - 4294967295 Default: 0
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
ng)xBGR }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
ng)xBGR }
Type: GstAggregatorPad
Pad Properties:
emit-signals : Send signals to signal data consumption
flags: readable, writable
Boolean. Default: false
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SRC: 'src'
Pad Template: 'src'
Element Properties:
back-enable : Enable Fill Background color
flags: readable, writable
Boolean. Default: true
background : Background color (ARGB format)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 0
composition-meta-enable: Enable overlay composition meta processing
flags: readable, writable
Boolean. Default: false
emit-signals : Send signals
flags: readable, writable
Boolean. Default: false
e longer to produce buffers for the current position (in nanoseconds)
flags: readable, writable
ult: 0
s)
flags: readable, writable
ult: 0
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "imxcompositor_g2d0"
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
start-time : Start time to use if start-time-selection=set
flags: readable, writable
ult: 18446744073709551615
start-time-selection: Decides which start time is output
flags: readable, writable
"
ECTION_ZERO
ECTION_FIRST
ECTION_SET
Element Signals:
"samples-selected" : void user_function (GstElement* object,
GstSegment* arg0,
guint64 arg1,
guint64 arg2,
guint64 arg3,
GstStructure* arg4,
gpointer user_data); |
Performance
Composite with color space conversion, resize, and rotate with red background color
Using OV5640 camera at 1080p 30 fps and videotestsrc:
gst-launch-1.0 imxcompositor_g2d background=0x000000FF name=comp sink_0::width=960 sink_0::height=540 sink_0::rotate=4 sink_1::width=960 sink_1::height=540 sink_1::xpos=960 sink_1::ypos=540 sink_1::rotate=1 ! video/x-raw,format=RGBA,width=1920,height=1080 ! perf print-cpu-load=true ! queue ! fakesink v4l2src device=/dev/video1 ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! comp.sink_0 videotestsrc ! video/x-raw,format=I420,width=1920,height=1080 ! queue ! comp.sink_1| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 26 % | 30.015 | 110 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
The CPU usage is elevated due to the videotestsrc element.
Using 2 videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 imxcompositor_g2d background=0x000000FF name=comp sink_0::width=960 sink_0::height=540 sink_0::rotate=4 sink_1::width=960 sink_1::height=540 sink_1::xpos=960 sink_1::ypos=540 sink_1::rotate=1 ! video/x-raw,format=RGBA,width=1920,height=1080 ! perf print-cpu-load=true ! queue ! fakesink videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! comp.sink_0 videotestsrc ! video/x-raw,format=I420,width=1920,height=1080 ! queue ! comp.sink_1| Board | CPU % | FPS | Element latency (ms) |
|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 50 % | 31.889 | 10 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||
| Nitrogen8M Mini (i.MX8M Mini) | 50 % | 31.538 | 6 |
These results are probably impacted by the CPU load of videotestsrc.
OpenGL
gst-inspect-1.0 glvideomixerelement (Nitrogen8M Mini)
|
|---|
root@nitrogen8mm:~# gst-inspect-1.0 glvideomixerelement
Factory Details:
Rank none (0)
Long-name OpenGL video_mixer
Klass Filter/Effect/Video/Compositor
Description OpenGL video_mixer
Author Matthew Waters <matthew@centricular.com>
Plugin Details:
Name opengl
Description OpenGL plugin
Filename /usr/lib/gstreamer-1.0/libgstopengl.so
Version 1.18.0
License LGPL
Source module gst-plugins-base
Source release date 2020-09-08
Binary package GStreamer Base Plug-ins source release
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstAggregator
+----GstVideoAggregator
+----GstGLBaseMixer
+----GstGLMixer
+----GstGLVideoMixer
Implemented Interfaces:
GstChildProxy
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw(memory:GLMemory)
format: RGBA
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Type: GstAggregatorPad
Pad Properties:
emit-signals : Send signals to signal data consumption
flags: readable, writable
Boolean. Default: false
SINK template: 'sink_%u'
Availability: On request
Capabilities:
video/x-raw(memory:GLMemory)
format: RGBA
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Type: GstGLVideoMixerPad
Pad Properties:
alpha : Alpha of the picture
flags: readable, writable, controllable
Double. Range: 0 - 1 Default: 1
blend-constant-color-alpha: Blend Constant Color Alpha
flags: readable, writable, controllable
Double. Range: 0 - 1 Default: 0
blend-constant-color-blue: Blend Constant Color Green
flags: readable, writable, controllable
Double. Range: 0 - 1 Default: 0
blend-constant-color-green: Blend Constant Color Green
flags: readable, writable, controllable
Double. Range: 0 - 1 Default: 0
blend-constant-color-red: Blend Constant Color Red
flags: readable, writable, controllable
Double. Range: 0 - 1 Default: 0
blend-equation-alpha: Blend Equation for Alpha
flags: readable, writable, controllable
Enum "GstGLVideoMixerBlendEquation" Default: 0, "add"
(0): add - Add
(1): subtract - Subtract
(2): reverse-subtract - Reverse Subtract
blend-equation-rgb : Blend Equation for RGB
flags: readable, writable, controllable
Enum "GstGLVideoMixerBlendEquation" Default: 0, "add"
(0): add - Add
(1): subtract - Subtract
(2): reverse-subtract - Reverse Subtract
blend-function-dst-alpha: Blend Function for Destination Alpha
flags: readable, writable, controllable
Enum "GstGLVideoMixerBlendFunction" Default: 7, "one-minus-src-alpha"
(0): zero - Zero
(1): one - One
(2): src-color - Source Color
(3): one-minus-src-color - One Minus Source Color
(4): dst-color - Destination Color
(5): one-minus-dst-color - One Minus Destination Color
(6): src-alpha - Source Alpha
(7): one-minus-src-alpha - One Minus Source Alpha
(8): dst-alpha - Destination Alpha
(9): one-minus-dst-alpha - One Minus Destination Alpha
(10): constant-color - Constant Color
(11): one-minus-contant-color - One Minus Constant Color
(12): constant-alpha - Constant Alpha
(13): one-minus-contant-alpha - One Minus Constant Alpha
(14): src-alpha-saturate - Source Alpha Saturate
blend-function-dst-rgb: Blend Function for Destination RGB
flags: readable, writable, controllable
Enum "GstGLVideoMixerBlendFunction" Default: 7, "one-minus-src-alpha"
(0): zero - Zero
(1): one - One
(2): src-color - Source Color
(3): one-minus-src-color - One Minus Source Color
(4): dst-color - Destination Color
(5): one-minus-dst-color - One Minus Destination Color
(6): src-alpha - Source Alpha
(7): one-minus-src-alpha - One Minus Source Alpha
(8): dst-alpha - Destination Alpha
(9): one-minus-dst-alpha - One Minus Destination Alpha
(10): constant-color - Constant Color
(11): one-minus-contant-color - One Minus Constant Color
(12): constant-alpha - Constant Alpha
(13): one-minus-contant-alpha - One Minus Constant Alpha
(14): src-alpha-saturate - Source Alpha Saturate
blend-function-src-alpha: Blend Function for Source Alpha
flags: readable, writable, controllable
Enum "GstGLVideoMixerBlendFunction" Default: 1, "one"
(0): zero - Zero
(1): one - One
(2): src-color - Source Color
(3): one-minus-src-color - One Minus Source Color
(4): dst-color - Destination Color
(5): one-minus-dst-color - One Minus Destination Color
(6): src-alpha - Source Alpha
(7): one-minus-src-alpha - One Minus Source Alpha
(8): dst-alpha - Destination Alpha
(9): one-minus-dst-alpha - One Minus Destination Alpha
(10): constant-color - Constant Color
(11): one-minus-contant-color - One Minus Constant Color
(12): constant-alpha - Constant Alpha
(13): one-minus-contant-alpha - One Minus Constant Alpha
(14): src-alpha-saturate - Source Alpha Saturate
blend-function-src-rgb: Blend Function for Source RGB
flags: readable, writable, controllable
Enum "GstGLVideoMixerBlendFunction" Default: 6, "src-alpha"
(0): zero - Zero
(1): one - One
(2): src-color - Source Color
(3): one-minus-src-color - One Minus Source Color
(4): dst-color - Destination Color
(5): one-minus-dst-color - One Minus Destination Color
(6): src-alpha - Source Alpha
(7): one-minus-src-alpha - One Minus Source Alpha
(8): dst-alpha - Destination Alpha
(9): one-minus-dst-alpha - One Minus Destination Alpha
(10): constant-color - Constant Color
(11): one-minus-contant-color - One Minus Constant Color
(12): constant-alpha - Constant Alpha
(13): one-minus-contant-alpha - One Minus Constant Alpha
(14): src-alpha-saturate - Source Alpha Saturate
emit-signals : Send signals to signal data consumption
flags: readable, writable
Boolean. Default: false
height : Height of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
max-last-buffer-repeat: Repeat last buffer for time (in ns, -1=until EOS), behaviour on EOS is not affected
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 18446744073709551615
repeat-after-eos : Repeat the last frame after EOS until all pads are EOS
flags: readable, writable, controllable
Boolean. Default: false
width : Width of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
xpos : X Position of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
ypos : Y Position of the picture
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
zorder : Z Order of the picture
flags: readable, writable, controllable
Unsigned Integer. Range: 0 - 4294967295 Default: 0
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SRC: 'src'
Pad Template: 'src'
Element Properties:
background : Background type
flags: readable, writable
Enum "GstGLVideoMixerBackground" Default: 0, "checker"
(0): checker - Checker pattern
(1): black - Black
(2): white - White
(3): transparent - Transparent Background to enable further compositing
context : Get OpenGL context
flags: readable
Object of type "GstGLContext"
emit-signals : Send signals
flags: readable, writable
Boolean. Default: false
latency : Additional latency in live mode to allow upstream to take longer to produce buffers for the current position (in nanoseconds)
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
min-upstream-latency: When sources with a higher latency are expected to be plugged in dynamically after the aggregator has started playing, this allows overriding the minimum latency reported by the initial source(s). This is only taken into account when larger than the actually reported minimum latency. (nanoseconds)
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "glvideomixer0"
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
start-time : Start time to use if start-time-selection=set
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 18446744073709551615
start-time-selection: Decides which start time is output
flags: readable, writable
Enum "GstAggregatorStartTimeSelection" Default: 0, "zero"
(0): zero - GST_AGGREGATOR_START_TIME_SELECTION_ZERO
(1): first - GST_AGGREGATOR_START_TIME_SELECTION_FIRST
(2): set - GST_AGGREGATOR_START_TIME_SELECTION_SET
Element Signals:
"samples-selected" : void user_function (GstElement* object,
GstSegment* arg0,
guint64 arg1,
guint64 arg2,
guint64 arg3,
GstStructure* arg4,
gpointer user_data); |
Performance
Composite with color space conversion, and resize with checkers background
Using OV5640 camera at 1080p 30 fps and videotestsrc:
gst-launch-1.0 glvideomixerelement name=comp sink_0::width=960 sink_0::height=540 sink_1::width=960 sink_1::height=540 sink_1::xpos=960 sink_1::ypos=540 ! gldownload ! perf print-cpu-load=true ! queue ! fakesink \
v4l2src device=/dev/video2 ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! glupload ! comp.sink_0 \
videotestsrc ! video/x-raw,format=I420,width=1920,height=1080 ! queue ! glupload ! comp.sink_1| Board | CPU % | FPS | OpenGL upload latency (ms) |
Element latency (ms) |
OpenGL download latency (ms) |
|---|---|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 26% | 20.009 | 0.4 | 1.18 | 0.1 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||||
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
The CPU usage is elevated due to the videotestsrc element.
Using 2 videotestsrc at 1080p with the framerate unlocked:
gst-launch-1.0 glvideomixerelement name=comp sink_0::width=960 sink_0::height=540 sink_1::width=960 sink_1::height=540 sink_1::xpos=960 sink_1::ypos=540 ! gldownload ! perf print-cpu-load=true ! queue ! fakesink \
videotestsrc ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! glupload ! comp.sink_0 \
videotestsrc ! video/x-raw,format=I420,width=1920,height=1080 ! queue ! glupload ! comp.sink_1| Board | CPU % | FPS | OpenGL upload latency (ms) |
Element latency (ms) |
OpenGL download latency (ms) |
|---|---|---|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 51 % | 31.975 | 0.7 | 1.3 | 0.1 |
| iMX8M EVK (i.MX8M Quad) | Not supported | ||||
| Nitrogen8M (i.MX8M Quad) | Not supported | ||||
| Nitrogen8M Mini (i.MX8M Mini) | 54 % | 31.565 | 7 | 80 | 0.1 |
These results are probably impacted by the CPU load of videotestsrc.
Video decoding, conversion, composition and encoding
Using OV5640 camera at 1080p 30 fps and 1080p-hevc-30fps.mkv file:
gst-launch-1.0 imxcompositor_g2d background=0x000000FF name=comp sink_0::width=960 sink_0::height=540 sink_0::rotate=4 sink_1::width=960 sink_1::height=540 sink_1::xpos=960 sink_1::ypos=540 sink_1::rotate=1 ! video/x-raw,format=RGBA,width=1920,height=1080 ! vpuenc_hevc ! perf print-cpu-load=true ! queue ! fakesink \
\
v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=1920,height=1080 ! queue ! comp.sink_0 \
\
filesrc location="1080p-hevc-30fps.mkv" ! matroskademux ! queue ! vpudec ! video/x-raw,format=NV12,width=1920,height=1080 ! queue ! comp.sink_1| Board | CPU % | FPS |
|---|---|---|
| DART-MX8M-PLUS (i.MX8M Plus) | 3 % | 30.027 |
| iMX8M EVK (i.MX8M Quad) | Not supported | |
| Nitrogen8M (i.MX8M Quad) | Not supported | |
| Nitrogen8M Mini (i.MX8M Mini) | Not tested |
