View source for Xilinx ZYNQ UltraScale+ MPSoC/GStreamer/Accelerating Image Processing/Accelerated Elements

<noinclude>
{{Xilinx ZYNQ UltraScale+ MPSoC/Head|previous=|next=}}
</noinclude>

Xilinx MPSoC's VCU IP is capable of simultaneously encoding and decoding up to 32 streams of H.264 and H.265 codecs, with a maximum bandwidth equivalent to 4K60. 8K encoding/decoding is also supported at a 15 FPS reduced rate mode.

It is important to notice that to make use of the VCU, firmware with the VCU enabled must be loaded into the FPGA. For this wiki, the smartcam example firmware was used during testing. Xilinx seems to have stopped including these apps in the newer versions of Petalinux, for this reason, you may need to add them manually. The firmware can be found in this [https://github.com/Xilinx/kria-apps-firmware repository] and the instructions to add a firmware recipe to the Petalinux build are located [[Xilinx ZYNQ UltraScale+ MPSoC/Development/Petalinux|here]].

== Video Encoding ==

=== H.264 ===

The H.264 encoder is available through the omxh264enc GStreamer element.

{|border=1" cellspacing=0" cellpadding="2" class="mw-collapsible mw-{{{state|collapsed}}}" style="width:100%;margin:0;padding:0;border:1px solid #999999;border-collapse:collapse;background:#FFFFFF"
|-
! style="text-align:{{{ta1|center}}};" | <div style="font-weight:{{{fw1|bold}}};background-color:{{{bg1|#CCCCFF}}};{{{headercss|}}}">{{{header| gst-inspect-1.0 omxh264enc }}}&nbsp;</div>
|-
|style="font-weight:{{{fw2|normal}}};background-color:{{{bg2|transparent}}};text-align:{{{ta2|left}}};{{{contentcss|}}}"|
{{{content|
<syntaxhighlight>
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh264enc
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                OpenMAX H.264 Video Encoder
  Klass                    Codec/Encoder/Video/Hardware
  Description              Encode H.264 video streams
  Author                   Sebastian Dr?ge <sebastian.droege@collabora.co.uk>

Plugin Details:
  Name                     omx
  Description              GStreamer OpenMAX Plug-ins
  Filename                 /usr/lib/gstreamer-1.0/libgstomx.so
  Version                  1.18.5
  License                  LGPL
  Source module            gst-omx
  Source release date      2021-09-08
  Binary package           GStreamer OpenMAX Plug-ins
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoEncoder
                         +----GstOMXVideoEnc
                               +----GstOMXH264Enc
                                     +----GstOMXH264Enc-omxh264enc

Implemented Interfaces:
  GstPreset

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw(memory:XLNXLL)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
  
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-h264
                  width: [ 16, 4096 ]
                 height: [ 16, 4096 ]
              framerate: [ 0/1, 2147483647/1 ]
          stream-format: byte-stream
              alignment: { (string)au, (string)nal }

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  aspect-ratio        : Display aspect ratio of the video sequence to be written in SPS/VUI
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncAspectRatio" Default: 0, "auto"
                           (0): auto             - 4:3 for SD video,16:9 for HD video,unspecified for unknown format
                           (1): 1-1              - 1:1 aspect ratio
                           (2): 4-3              - 4:3 aspect ratio
                           (3): 16-9             - 16:9 aspect ratio
                           (4): none             - Aspect ratio information is not present in the stream
  b-frames            : Number of B-frames between two consecutive P-frames
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  constrained-intra-prediction: If enabled, prediction only uses residual data and decoded samples from neighbouring coding blocks coded using intra prediction modes
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  control-rate        : Bitrate control method
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncControlRate" Default: -1, "default"
                           (0): disable          - Disable
                           (1): variable         - Variable
                           (2): constant         - Constant
                           (3): variable-skip-frames - Variable Skip Frames
                           (4): constant-skip-frames - Constant Skip Frames
                           (2130706434): capped-variable  - Capped Variable
                           (2130706433): low-latency      - Low Latency
                           (-1): default          - Component Default
  cpb-size            : Coded Picture Buffer as specified in the HRD model in msec. Not used when control-rate=disable
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 3000 
  default-roi-quality : The default quality level to apply to each Region of Interest
                        flags: readable, writable
                        Enum "GstOMXVideoEncRoiQuality" Default: 0, "high"
                           (0): high             - Delta QP of -5
                           (1): medium           - Delta QP of 0
                           (2): low              - Delta QP of +5
                           (3): dont-care        - Maximum delta QP value
                           (4): intra            - Region all LCU encoded with intra prediction mode
  dependent-slice     : If encoding with multiple slices, specify whether the additional slices are dependent slice segments or regular slices
                        flags: readable, writable
                        Boolean. Default: false
  entropy-mode        : Entropy mode for encoding process
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXH264EncEntropyMode" Default: -1, "default"
                           (0): CAVLC            - CAVLC entropy mode
                           (1): CABAC            - CABAC entropy mode
                           (-1): default          - Component Default
  filler-data         : Enable/Disable Filler Data NAL units for CBR rate control
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: true
  gdr-mode            : Gradual Decoder Refresh scheme mode. Only used if gop-mode=low-delay-p
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncGdrMode" Default: 0, "disabled"
                           (0): disabled         - No GDR
                           (1): vertical         - Gradual refresh using a vertical bar moving from left to right
                           (2): horizontal       - Gradual refresh using a horizontal bar moving from top to bottom
  gop-length          : Distance between two consecutive I frames
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 1000 Default: 30 
  gop-mode            : Group Of Pictures mode
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncGopMode" Default: 0, "basic"
                           (0): basic            - Basic GOP settings
                           (1): basic-b          - Basic GOP settings, includes only B-frames
                           (2): pyramidal        - Advanced GOP pattern with hierarchical B-frames
                           (3): pyramidal-b      - Advanced GOP pattern with hierarchical B-frames, includes only B-frames
                           (5): low-delay-p      - Single I-frame followed by P-frames only
                           (6): low-delay-b      - Single I-frame followed by B-frames only
                           (4): adaptive         - Advanced GOP pattern with adaptive B-frames
  hlg-sdr-compatible  : If enabled and input caps contain HLG colorimetry, insert BT2020 EOTF and ATC SEI instead
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  initial-delay       : The initial removal delay as specified in the HRD model in msec. Not used when control-rate=disable
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 1500 
  input-crop          : Cropping parameters for input video buffer ('<left, top, width, height>')
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  interval-intraframes: Interval of coding Intra frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
-h264,alignment=nal ! ...).
                        flags: readable, writable
                        Enum "GstOMXVideoEncLatencyMode" Default: 0, "normal"
                           (0): normal           - Normal mode
                           (1): low-latency      - Low latency mode
                           (-1): default          - Component Default
  long-term-freq      : Periodicity of LongTerm reference picture marking in encoding process Units in frames, distance between two consequtive long-term reference pictures
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  long-term-ref       : If enabled, encoder accepts dynamically inserting and using long-term reference picture events from upstream elements
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  look-ahead          : The number of frames processed ahead of second pass encoding. If smaller than 2, dual pass encoding is disabled
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  loop-filter-alpha-c0-offset: Alpha C0 offset for the deblocking filter, used only when loop-filter-mode is enabled
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Integer. Range: -6 - 6 Default: -1 
  loop-filter-beta-offset: Beta offset for the deblocking filter, used only when loop-filter-mode is enabled
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Integer. Range: -6 - 6 Default: -1 
  loop-filter-mode    : Enable or disable the deblocking filter (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXH264EncLoopFilter" Default: -1, "default"
                           (0): enable           - Enable deblocking filter
                           (1): disable          - Disable deblocking filter
                           (2): disable-slice-boundary - Disables deblocking filter on slice boundary
                           (-1): default          - Component Default
  low-bandwidth       : If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  max-bitrate         : Max bitrate in Kbps, only used if control-rate=variable (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  max-consecutive-skip: Max number of consecutive frames to be skipped. Only applicable if skip-frame is enabled
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
ze will not have any effect
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
icture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  max-qp              : Maximum QP value allowed for the rate control
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 51 Default: 51 
le
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  min-qp              : Minimum QP value allowed for the rate control
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 51 Default: 10 
  name                : The name of the object
                        flags: readable, writable, 0x2000
                        String. Default: "omxh264enc-omxh264enc0"
egularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 1 - 4294967295 Default: 4294967295 
  output-crop         : Cropping parameters for output bitstream ('<left, top, width, height>')
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  parent              : The parent of the object
                        flags: readable, writable, 0x2000
                        Object of type "GstObject"
  periodicity-idr     : Periodicity of IDR frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  periodicty-idr      : Periodicity of IDR frames (0xffffffff=component default) DEPRECATED - only for backwards compat
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  prefetch-buffer     : Enable/Disable L2Cache buffer in encoding process
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  qos                 : Handle Quality-of-Service events from downstream
                        flags: readable, writable
                        Boolean. Default: false
  qp-mode             : QP control mode used by the VCU encoder
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncQpMode" Default: -1, "default"
                           (0): uniform          - Use the same QP for all coding units of the frame
                           (1): roi              - Adjust QP according to the regions of interest defined on each frame. Must be set to handle ROI metadata.
                           (2): auto             - Let the VCU encoder change the QP for each coding unit according to its content
                           (3): load-qp-absolute - Uses absolute QP values set by user. Must be set to use External QP buffer
                           (4): load-qp-relative - Uses Relative/Delta QP values set by user. Must be set to use External QP buffer
                           (-1): default          - Component Default
  quant-b-frames      : Quantization parameter for B-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-i-frames      : Quantization parameter for I-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-p-frames      : Quantization parameter for P-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  ref-frames          : Number of reference frames used for inter-motion search (0=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unknown type 16 "guchar"
  scaling-list        : Scaling list mode
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncScalingList" Default: 1, "default"
                           (1): default          - Default scaling list mode
                           (0): flat             - Flat scaling list mode
Skip. Only used if control-rate=constant or variable and b-frames are less than 2
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  slice-size          : Target slice size (in bytes) that the encoder uses to automatically split the bitstream into approximately equally-sized slices
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 65535 Default: 0 
  target-bitrate      : Target bitrate in Kbps (0xffffffff=component default)
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  uniform-slice-type  : Enable/Disable uniform slice type in slice header
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  use-out-port-pool   : Use a buffer pool on the output port
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
cture size will be limited to max-picture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  y444-to-gray        : Enable/Disable YUV format manipulation to Y8
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
</syntaxhighlight>
}}}
|}

==== Performance ====

Using AR1335 camera through AP1302 ISP at 1080p:
<syntaxhighlight>
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=1920, height=1080, format=NV12, framerate=30/1 ! omxh264enc ! perf ! fakesink
</syntaxhighlight>

<center>
<table class="wikitable">
  <tr>
    <th style="background:MidnightBlue; color:White">Board</th>
    <th style="background:MidnightBlue; color:White">CPU %</th> 
    <th style="background:MidnightBlue; color:White">FPS</th>
    <th style="background:MidnightBlue; color:White">Element<br/> latency (ms)</th>
  </tr>
  <tr>
    <td>KV260</td>
    <td>2 %</td> 
    <td>30.000</td> 
    <td>Not tested</td>
  </tr>
</table>
</center>

Using AR1335 camera through AP1302 ISP at 4K:
<syntaxhighlight>
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=30/1 ! omxh264enc ! perf ! fakesink
</syntaxhighlight>

<center>
<table class="wikitable">
  <tr>
    <th style="background:MidnightBlue; color:White">Board</th>
    <th style="background:MidnightBlue; color:White">CPU %</th> 
    <th style="background:MidnightBlue; color:White">FPS</th>
    <th style="background:MidnightBlue; color:White">Element<br/> latency (ms)</th>
  </tr>
  <tr>
    <td>KV260</td>
    <td>3 %</td> 
    <td>30.000</td> 
    <td>Not tested</td>
  </tr>
</table>
</center>

=== H.265 ===

The H.265 encoder is available through the omxh265enc GStreamer element.

{|border=1" cellspacing=0" cellpadding="2" class="mw-collapsible mw-{{{state|collapsed}}}" style="width:100%;margin:0;padding:0;border:1px solid #999999;border-collapse:collapse;background:#FFFFFF"
|-
! style="text-align:{{{ta1|center}}};" | <div style="font-weight:{{{fw1|bold}}};background-color:{{{bg1|#CCCCFF}}};{{{headercss|}}}">{{{header| gst-inspect-1.0 omxh265enc }}}&nbsp;</div>
|-
|style="font-weight:{{{fw2|normal}}};background-color:{{{bg2|transparent}}};text-align:{{{ta2|left}}};{{{contentcss|}}}"|
{{{content|
<syntaxhighlight>
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh265enc
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                OpenMAX H.265 Video Encoder
  Klass                    Codec/Encoder/Video/Hardware
  Description              Encode H.265 video streams
  Author                   Sebastian Dr?ge <sebastian.droege@collabora.co.uk>

Plugin Details:
  Name                     omx
  Description              GStreamer OpenMAX Plug-ins
  Filename                 /usr/lib/gstreamer-1.0/libgstomx.so
  Version                  1.18.5
  License                  LGPL
  Source module            gst-omx
  Source release date      2021-09-08
  Binary package           GStreamer OpenMAX Plug-ins
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoEncoder
                         +----GstOMXVideoEnc
                               +----GstOMXH265Enc
                                     +----GstOMXH265Enc-omxh265enc

Implemented Interfaces:
  GstPreset

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw(format:Interlaced)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
         interlace-mode: alternate
      video/x-raw(memory:XLNXLL)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
  
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-h265
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
          stream-format: byte-stream
              aligmment: { (string)au, (string)nal }

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  aspect-ratio        : Display aspect ratio of the video sequence to be written in SPS/VUI
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncAspectRatio" Default: 0, "auto"
                           (0): auto             - 4:3 for SD video,16:9 for HD video,unspecified for unknown format
                           (1): 1-1              - 1:1 aspect ratio
                           (2): 4-3              - 4:3 aspect ratio
                           (3): 16-9             - 16:9 aspect ratio
                           (4): none             - Aspect ratio information is not present in the stream
  b-frames            : Number of B-frames between two consecutive P-frames
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  constrained-intra-prediction: If enabled, prediction only uses residual data and decoded samples from neighbouring coding blocks coded using intra prediction modes
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  control-rate        : Bitrate control method
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncControlRate" Default: -1, "default"
                           (0): disable          - Disable
                           (1): variable         - Variable
                           (2): constant         - Constant
                           (3): variable-skip-frames - Variable Skip Frames
                           (4): constant-skip-frames - Constant Skip Frames
                           (2130706434): capped-variable  - Capped Variable
                           (2130706433): low-latency      - Low Latency
                           (-1): default          - Component Default
  cpb-size            : Coded Picture Buffer as specified in the HRD model in msec. Not used when control-rate=disable
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 3000 
  default-roi-quality : The default quality level to apply to each Region of Interest
                        flags: readable, writable
                        Enum "GstOMXVideoEncRoiQuality" Default: 0, "high"
                           (0): high             - Delta QP of -5
                           (1): medium           - Delta QP of 0
                           (2): low              - Delta QP of +5
                           (3): dont-care        - Maximum delta QP value
                           (4): intra            - Region all LCU encoded with intra prediction mode
  dependent-slice     : If encoding with multiple slices, specify whether the additional slices are dependent slice segments or regular slices
                        flags: readable, writable
                        Boolean. Default: false
  filler-data         : Enable/Disable Filler Data NAL units for CBR rate control
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: true
  gdr-mode            : Gradual Decoder Refresh scheme mode. Only used if gop-mode=low-delay-p
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncGdrMode" Default: 0, "disabled"
                           (0): disabled         - No GDR
                           (1): vertical         - Gradual refresh using a vertical bar moving from left to right
                           (2): horizontal       - Gradual refresh using a horizontal bar moving from top to bottom
  gop-length          : Distance between two consecutive I frames
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 1000 Default: 30 
  gop-mode            : Group Of Pictures mode
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncGopMode" Default: 0, "basic"
                           (0): basic            - Basic GOP settings
                           (1): basic-b          - Basic GOP settings, includes only B-frames
                           (2): pyramidal        - Advanced GOP pattern with hierarchical B-frames
                           (3): pyramidal-b      - Advanced GOP pattern with hierarchical B-frames, includes only B-frames
                           (5): low-delay-p      - Single I-frame followed by P-frames only
                           (6): low-delay-b      - Single I-frame followed by B-frames only
                           (4): adaptive         - Advanced GOP pattern with adaptive B-frames
  hlg-sdr-compatible  : If enabled and input caps contain HLG colorimetry, insert BT2020 EOTF and ATC SEI instead
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  initial-delay       : The initial removal delay as specified in the HRD model in msec. Not used when control-rate=disable
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 1500 
  input-crop          : Cropping parameters for input video buffer ('<left, top, width, height>')
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  interval-intraframes: Interval of coding Intra frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
-h264,alignment=nal ! ...).
                        flags: readable, writable
                        Enum "GstOMXVideoEncLatencyMode" Default: 0, "normal"
                           (0): normal           - Normal mode
                           (1): low-latency      - Low latency mode
                           (-1): default          - Component Default
  long-term-freq      : Periodicity of LongTerm reference picture marking in encoding process Units in frames, distance between two consequtive long-term reference pictures
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  long-term-ref       : If enabled, encoder accepts dynamically inserting and using long-term reference picture events from upstream elements
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  look-ahead          : The number of frames processed ahead of second pass encoding. If smaller than 2, dual pass encoding is disabled
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  loop-filter-beta-offset: Beta offset for the deblocking filter, used only when loop-filter-mode is enabled
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Integer. Range: -6 - 6 Default: -1 
  loop-filter-mode    : Enable or disable the deblocking filter (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXH265EncLoopFilter" Default: -1, "default"
                           (0): enable           - Enable deblocking filter
                           (1): disable          - Disable deblocking filter
                           (2): disable-cross-slice - Disable deblocking filter on slice boundary
                           (3): disable-cross-tile - Disable deblocking filter on tile boundary
                           (4): disable-slice-and-tile - Disable deblocking filter on slice and tile boundary
                           (-1): default          - Component Default
  loop-filter-tc-offset: TC offset for the deblocking filter, used only when loop-filter-mode is enabled
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Integer. Range: -6 - 6 Default: -1 
  low-bandwidth       : If enabled, decrease the vertical search range used for P-frame motion estimation to reduce the bandwidth
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  max-bitrate         : Max bitrate in Kbps, only used if control-rate=variable (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  max-consecutive-skip: Max number of consecutive frames to be skipped. Only applicable if skip-frame is enabled
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
ze will not have any effect
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
icture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  max-qp              : Maximum QP value allowed for the rate control
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 51 Default: 51 
le
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  min-qp              : Minimum QP value allowed for the rate control
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 51 Default: 10 
  name                : The name of the object
                        flags: readable, writable, 0x2000
                        String. Default: "omxh265enc-omxh265enc0"
egularly as possible. If slice-size is defined as well more slices may be produced to fit the slice-size requirement (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 1 - 4294967295 Default: 4294967295 
  output-crop         : Cropping parameters for output bitstream ('<left, top, width, height>')
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  parent              : The parent of the object
                        flags: readable, writable, 0x2000
                        Object of type "GstObject"
  periodicity-idr     : Periodicity of IDR frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  prefetch-buffer     : Enable/Disable L2Cache buffer in encoding process
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  qos                 : Handle Quality-of-Service events from downstream
                        flags: readable, writable
                        Boolean. Default: false
  qp-mode             : QP control mode used by the VCU encoder
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncQpMode" Default: -1, "default"
                           (0): uniform          - Use the same QP for all coding units of the frame
                           (1): roi              - Adjust QP according to the regions of interest defined on each frame. Must be set to handle ROI metadata.
                           (2): auto             - Let the VCU encoder change the QP for each coding unit according to its content
                           (3): load-qp-absolute - Uses absolute QP values set by user. Must be set to use External QP buffer
                           (4): load-qp-relative - Uses Relative/Delta QP values set by user. Must be set to use External QP buffer
                           (-1): default          - Component Default
  quant-b-frames      : Quantization parameter for B-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-i-frames      : Quantization parameter for I-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  quant-p-frames      : Quantization parameter for P-frames (0xffffffff=component default)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  scaling-list        : Scaling list mode
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstOMXVideoEncScalingList" Default: 1, "default"
                           (1): default          - Default scaling list mode
                           (0): flat             - Flat scaling list mode
Skip. Only used if control-rate=constant or variable and b-frames are less than 2
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  slice-size          : Target slice size (in bytes) that the encoder uses to automatically split the bitstream into approximately equally-sized slices
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 65535 Default: 0 
  target-bitrate      : Target bitrate in Kbps (0xffffffff=component default)
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295 
  uniform-slice-type  : Enable/Disable uniform slice type in slice header
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
  use-out-port-pool   : Use a buffer pool on the output port
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false
cture size will be limited to max-picture-size-x value. If set it to 0 then max-picture-size-x will not have any effect
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  y444-to-gray        : Enable/Disable YUV format manipulation to Y8
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false

</syntaxhighlight>
}}}
|}

==== Performance ====

Using AR1335 camera through AP1302 ISP at 1080p:
<syntaxhighlight>
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=1920, height=1080, format=NV12, framerate=30/1 ! omxh265enc ! perf ! fakesink
</syntaxhighlight>

<center>
<table class="wikitable">
  <tr>
    <th style="background:MidnightBlue; color:White">Board</th>
    <th style="background:MidnightBlue; color:White">CPU %</th> 
    <th style="background:MidnightBlue; color:White">FPS</th>
    <th style="background:MidnightBlue; color:White">Element<br/> latency (ms)</th>
  </tr>
  <tr>
    <td>KV260</td>
    <td>2 %</td> 
    <td>30.000</td> 
    <td>Not tested</td>
  </tr>
</table>
</center>

Using AR1335 camera through AP1302 ISP at 4K:
<syntaxhighlight>
gst-launch-1.0 mediasrcbin media-device=/dev/media0 v4l2src0::io-mode=4 ! video/x-raw, width=3840, height=2160, format=NV12, framerate=30/1 ! omxh265enc ! perf ! fakesink
</syntaxhighlight>

<center>
<table class="wikitable">
  <tr>
    <th style="background:MidnightBlue; color:White">Board</th>
    <th style="background:MidnightBlue; color:White">CPU %</th> 
    <th style="background:MidnightBlue; color:White">FPS</th>
    <th style="background:MidnightBlue; color:White">Element<br/> latency (ms)</th>
  </tr>
  <tr>
    <td>KV260</td>
    <td>3 %</td> 
    <td>30.000</td> 
    <td>Not tested</td>
  </tr>
</table>
</center>

== Video Decoding ==

=== H.264 ===


The H.264 decoder is available through the omxh264dec GStreamer element.

{|border=1" cellspacing=0" cellpadding="2" class="mw-collapsible mw-{{{state|collapsed}}}" style="width:100%;margin:0;padding:0;border:1px solid #999999;border-collapse:collapse;background:#FFFFFF"
|-
! style="text-align:{{{ta1|center}}};" | <div style="font-weight:{{{fw1|bold}}};background-color:{{{bg1|#CCCCFF}}};{{{headercss|}}}">{{{header| gst-inspect-1.0 omxh264dec }}}&nbsp;</div>
|-
|style="font-weight:{{{fw2|normal}}};background-color:{{{bg2|transparent}}};text-align:{{{ta2|left}}};{{{contentcss|}}}"|
{{{content|
<syntaxhighlight>
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh264dec
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                OpenMAX H.264 Video Decoder
  Klass                    Codec/Decoder/Video/Hardware
  Description              Decode H.264 video streams
  Author                   Sebastian Dr?ge <sebastian.droege@collabora.co.uk>

Plugin Details:
  Name                     omx
  Description              GStreamer OpenMAX Plug-ins
  Filename                 /usr/lib/gstreamer-1.0/libgstomx.so
  Version                  1.18.5
  License                  LGPL
  Source module            gst-omx
  Source release date      2021-09-08
  Binary package           GStreamer OpenMAX Plug-ins
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstOMXVideoDec
                               +----GstOMXH264Dec
                                     +----GstOMXH264Dec-omxh264dec

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h264
              alignment: au
          stream-format: byte-stream
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/x-h264
              alignment: nal
          stream-format: byte-stream
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
  
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:GLMemory)
                 format: RGBA
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:XLNXLL, format:Interlaced)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
         interlace-mode: alternate
      video/x-raw(format:Interlaced)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
         interlace-mode: alternate
      video/x-raw(memory:XLNXLL)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
g high bitrate streams. Decreasing it reduces the memory footprint
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 2 - 16 Default: 5 
t to 'au' for normal latency and 'nal' for subframe latency (e.g.  ... ! video/x-h264,alignment=nal ! omxh264dec low-latency=1 ! ...).
                        flags: readable, writable
                        Enum "GstOMXVideoDecLatencyMode" Default: 0, "normal"
                           (0): normal           - Normal mode
                           (1): reduced-latency  - Low ref dpb mode(reduced-latency)
                           (2): low-latency      - Low latency mode
                           (-1): default          - Component Default
  low-latency         : When enabled, buffers will be pushed before they are evicted from the DBP, reducing decoding latency.
                        flags: readable, writable
                        Boolean. Default: false
  max-errors          : Max consecutive decoder errors before returning flow error
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: 10 
  name                : The name of the object
                        flags: readable, writable, 0x2000
                        String. Default: "omxh264dec-omxh264dec0"
  output-position     : Position coordinates of start of active pixel data in video buffer ('<position_x, position_y>')
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  parent              : The parent of the object
                        flags: readable, writable, 0x2000
                        Object of type "GstObject"
  qos                 : Handle Quality-of-Service events from downstream
                        flags: readable, writable
                        Boolean. Default: true
 buffer and process them.
                        flags: readable, writable
                        Boolean. Default: false

</syntaxhighlight>
}}}
|}

=== H.265 ===


The H.265 decoder is available through the omxh265dec GStreamer element.

{|border=1" cellspacing=0" cellpadding="2" class="mw-collapsible mw-{{{state|collapsed}}}" style="width:100%;margin:0;padding:0;border:1px solid #999999;border-collapse:collapse;background:#FFFFFF"
|-
! style="text-align:{{{ta1|center}}};" | <div style="font-weight:{{{fw1|bold}}};background-color:{{{bg1|#CCCCFF}}};{{{headercss|}}}">{{{header| gst-inspect-1.0 omxh265dec }}}&nbsp;</div>
|-
|style="font-weight:{{{fw2|normal}}};background-color:{{{bg2|transparent}}};text-align:{{{ta2|left}}};{{{contentcss|}}}"|
{{{content|
<syntaxhighlight>
root@xilinx-kv260-starterkit-20222:~# gst-inspect-1.0 omxh265dec                                                                                                            

  Rank                     primary + 1 (257)
  Long-name                OpenMAX H.265 Video Decoder
  Klass                    Codec/Decoder/Video/Hardware
  Description              Decode H.265 video streams
  Author                   Sebastian Dr?ge <sebastian.droege@collabora.co.uk>

Plugin Details:
  Name                     omx
  Description              GStreamer OpenMAX Plug-ins
  Filename                 /usr/lib/gstreamer-1.0/libgstomx.so
  Version                  1.18.5
  License                  LGPL
  Source module            gst-omx
  Source release date      2021-09-08
  Binary package           GStreamer OpenMAX Plug-ins
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstOMXVideoDec
                               +----GstOMXH265Dec
                                     +----GstOMXH265Dec-omxh265dec

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
              alignment: au
          stream-format: byte-stream
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/x-h265
              alignment: nal
          stream-format: byte-stream
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
  
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:GLMemory)
                 format: RGBA
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:XLNXLL, format:Interlaced)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
         interlace-mode: alternate
      video/x-raw(format:Interlaced)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
         interlace-mode: alternate
      video/x-raw(memory:XLNXLL)
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)NV16_10LE32, (string)NV12_10LE32, (string)NV16, (string)NV12, (string)GRAY10_LE32, (string)GRAY8, (string)Y444, (string)Y444_10LE32 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
g high bitrate streams. Decreasing it reduces the memory footprint
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 2 - 16 Default: 5 
t to 'au' for normal latency and 'nal' for subframe latency (e.g.  ... ! video/x-h264,alignment=nal ! omxh264dec low-latency=1 ! ...).
                        flags: readable, writable
                        Enum "GstOMXVideoDecLatencyMode" Default: 0, "normal"
                           (0): normal           - Normal mode
                           (1): reduced-latency  - Low ref dpb mode(reduced-latency)
                           (2): low-latency      - Low latency mode
                           (-1): default          - Component Default
  low-latency         : When enabled, buffers will be pushed before they are evicted from the DBP, reducing decoding latency.
                        flags: readable, writable
                        Boolean. Default: false
  max-errors          : Max consecutive decoder errors before returning flow error
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: 10 
  name                : The name of the object
                        flags: readable, writable, 0x2000
                        String. Default: "omxh265dec-omxh265dec0"
  output-position     : Position coordinates of start of active pixel data in video buffer ('<position_x, position_y>')
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only
  parent              : The parent of the object
                        flags: readable, writable, 0x2000
                        Object of type "GstObject"
  qos                 : Handle Quality-of-Service events from downstream
                        flags: readable, writable
                        Boolean. Default: true
 buffer and process them.
                        flags: readable, writable
                        Boolean. Default: false
</syntaxhighlight>
}}}
|}

<noinclude>{{Xilinx ZYNQ UltraScale+ MPSoC/Foot||}}</noinclude>