Difference between revisions of "IMX8/Multimedia/VPU"
m |
|||
| (3 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | <noinclude>{{IMX8/Head}}</noinclude> | + | <noinclude>{{IMX8/Head|previous=Multimedia/GPU/Vulkan|next=Multimedia/VPU/Encoding System|metakeywords=multimedia,vpu,Video Processing Unit,Video Decoding System,Hantro G2 + G1 hardware,OpenMAX, OpenMAX API,OpenMAX IL API,OMX API,VPU G1,VPU G2}}</noinclude> |
| − | =General Description= | + | ==General Description== |
The Video Processing Unit (VPU) hardware performs all of the codec computation and most of the bitstream parsing/packeting. The software takes advantage of less control and effort to implement a complex and efficient multimedia codec system. | The Video Processing Unit (VPU) hardware performs all of the codec computation and most of the bitstream parsing/packeting. The software takes advantage of less control and effort to implement a complex and efficient multimedia codec system. | ||
| Line 24: | Line 24: | ||
|} | |} | ||
| − | ==Video Decoding System== | + | ===Video Decoding System=== |
The VPU supports video decoding for various video format using the Hantro G2 + G1 hardware based decoders. | The VPU supports video decoding for various video format using the Hantro G2 + G1 hardware based decoders. | ||
| − | ===OpenMAX Integration Layer API=== | + | ====OpenMAX Integration Layer API==== |
| − | OpenMAX IL API it's a standardized multimedia component interface that allows easier and faster integration of multimedia components into a multimedia system. The interface itself is independent of the execution environment and operating system. However, the overall performance and capabilities of the decoders | + | OpenMAX IL API it's a standardized multimedia component interface that allows easier and faster integration of multimedia components into a multimedia system. The interface itself is independent of the execution environment and operating system. However, the overall performance and capabilities of the decoders are system dependent. |
| − | The decoder is implemented in Linux environment. The Bellagio OMX IL Core for Linux is used for development and testing purposes. The OMX API is implemented on top of the Hantro | + | The decoder is implemented in a Linux environment. The Bellagio OMX IL Core for Linux is used for development and testing purposes. The OMX API is implemented on top of the Hantro |
| − | 8170/8190/9170/9190/G1/G2 decoder AVS, H.264, H.263, HEVC, MPEG-2, MPEG-4, VC-1, RV, DivX, VP6, VP8, VP9, JPEG, WebP and post-processing APIs. | + | 8170/8190/9170/9190/G1/G2 decoder AVS, H.264, H.263, HEVC, MPEG-2, MPEG-4, VC-1, RV, DivX, VP6, VP8, VP9, JPEG, WebP, and post-processing APIs. |
| − | The API is developed on Linux environment and the implementation uses an Operating System Abstraction Layer (OSAL) to isolate all OS specific functionality. The following diagram shows the blocks of the OpenMAX video decoder system. | + | The API is developed on a Linux environment and the implementation uses an Operating System Abstraction Layer (OSAL) to isolate all OS specific functionality. The following diagram shows the blocks of the OpenMAX video decoder system. |
[[Image:OMX.png|thumb|center|450px|Image source: https://www.nxp.com]] | [[Image:OMX.png|thumb|center|450px|Image source: https://www.nxp.com]] | ||
| − | ===Hantro Capabilities=== | + | ====Hantro Capabilities==== |
The Hantro 8170/8190/9170/9190/G1/G2 hardware based decoders are able to decode: H.264, H.263, MPEG-2, MPEG-4, VC-1, RealVideo (supported in 9170/9190/G1), DivX (supported in 9170/9190/G1), VP6 (supported in 9190/G1), AVS (supported in G1), VP8 (supported in G1), HEVC (supported in G2) and VP9 (supported in G2) standards video streams, JPEG standard still images and WebP (supported in G1) still images. | The Hantro 8170/8190/9170/9190/G1/G2 hardware based decoders are able to decode: H.264, H.263, MPEG-2, MPEG-4, VC-1, RealVideo (supported in 9170/9190/G1), DivX (supported in 9170/9190/G1), VP6 (supported in 9190/G1), AVS (supported in G1), VP8 (supported in G1), HEVC (supported in G2) and VP9 (supported in G2) standards video streams, JPEG standard still images and WebP (supported in G1) still images. | ||
| − | =VPU G1= | + | ==VPU G1== |
The 8170/8190/9170/9190/G1/G2 decoder has two input ports and one output port: | The 8170/8190/9170/9190/G1/G2 decoder has two input ports and one output port: | ||
* The '''first input port''' format is for compressed AVS, H.264, H.263, HEVC, MJPEG, MPEG-2, MPEG-4, VC-1, RV, DivX, VP6, VP8, VP9, JPEG or WebP stream. | * The '''first input port''' format is for compressed AVS, H.264, H.263, HEVC, MJPEG, MPEG-2, MPEG-4, VC-1, RV, DivX, VP6, VP8, VP9, JPEG or WebP stream. | ||
| − | * The '''second input port''' is used for post-processing alpha blending feature and its format is uncompressed YUV or RGB video data. | + | * The '''second input port''' is used for the post-processing alpha blending feature and its format is uncompressed YUV or RGB video data. |
| − | * The '''output port''' format is uncompressed YUV/RGB video data. For post-processor input the port format is uncompressed YUV video data. | + | * The '''output port''' format is uncompressed YUV/RGB video data. For post-processor input, the port format is uncompressed YUV video data. |
| − | =VPU G2= | + | ==VPU G2== |
The decoder is able to decode HEVC standard Main/Main 10 profile compatible video streams, and Google's VP9 Profile 0 compatible video streams. | The decoder is able to decode HEVC standard Main/Main 10 profile compatible video streams, and Google's VP9 Profile 0 compatible video streams. | ||
| Line 55: | Line 55: | ||
===Compressed format=== | ===Compressed format=== | ||
| − | G2 decoder supports compressed tiled output | + | G2 decoder supports compressed tiled output when reference buffer compression (RFC) is enabled when synthesis and user allow compressed reference frame output directly. |
Reference frame compression is a feature added in G2 decoder to compress frame buffer so that the bandwidth of storing/loading reference frame can be reduced, especially when | Reference frame compression is a feature added in G2 decoder to compress frame buffer so that the bandwidth of storing/loading reference frame can be reduced, especially when | ||
| − | the resolution of decoded stream is of high definition. | + | the resolution of the decoded stream is of high definition. |
<noinclude>{{IMX8/Foot|Multimedia/GPU/Vulkan|Multimedia/VPU/Encoding System}}</noinclude> | <noinclude>{{IMX8/Foot|Multimedia/GPU/Vulkan|Multimedia/VPU/Encoding System}}</noinclude> | ||
Latest revision as of 12:21, 9 March 2023
|
Contents
General Description
The Video Processing Unit (VPU) hardware performs all of the codec computation and most of the bitstream parsing/packeting. The software takes advantage of less control and effort to implement a complex and efficient multimedia codec system.
| VPU Encoding & Decoding Support | ||
|---|---|---|
| Product Family | Video Encode Resolution (Top Codecs) | Video Decode Resolution (Top Codecs) |
| i.MX8 |
|
|
| i.MX 8M |
|
|
| i.MX 8M Mini |
|
|
| i.MX 8X |
|
|
Video Decoding System
The VPU supports video decoding for various video format using the Hantro G2 + G1 hardware based decoders.
OpenMAX Integration Layer API
OpenMAX IL API it's a standardized multimedia component interface that allows easier and faster integration of multimedia components into a multimedia system. The interface itself is independent of the execution environment and operating system. However, the overall performance and capabilities of the decoders are system dependent.
The decoder is implemented in a Linux environment. The Bellagio OMX IL Core for Linux is used for development and testing purposes. The OMX API is implemented on top of the Hantro 8170/8190/9170/9190/G1/G2 decoder AVS, H.264, H.263, HEVC, MPEG-2, MPEG-4, VC-1, RV, DivX, VP6, VP8, VP9, JPEG, WebP, and post-processing APIs.
The API is developed on a Linux environment and the implementation uses an Operating System Abstraction Layer (OSAL) to isolate all OS specific functionality. The following diagram shows the blocks of the OpenMAX video decoder system.
Hantro Capabilities
The Hantro 8170/8190/9170/9190/G1/G2 hardware based decoders are able to decode: H.264, H.263, MPEG-2, MPEG-4, VC-1, RealVideo (supported in 9170/9190/G1), DivX (supported in 9170/9190/G1), VP6 (supported in 9190/G1), AVS (supported in G1), VP8 (supported in G1), HEVC (supported in G2) and VP9 (supported in G2) standards video streams, JPEG standard still images and WebP (supported in G1) still images.
VPU G1
The 8170/8190/9170/9190/G1/G2 decoder has two input ports and one output port:
- The first input port format is for compressed AVS, H.264, H.263, HEVC, MJPEG, MPEG-2, MPEG-4, VC-1, RV, DivX, VP6, VP8, VP9, JPEG or WebP stream.
- The second input port is used for the post-processing alpha blending feature and its format is uncompressed YUV or RGB video data.
- The output port format is uncompressed YUV/RGB video data. For post-processor input, the port format is uncompressed YUV video data.
VPU G2
The decoder is able to decode HEVC standard Main/Main 10 profile compatible video streams, and Google's VP9 Profile 0 compatible video streams. The decoder conforms to the HEVC Main, Main 10 profiles and can decode streams up to level 5.1. The maximum supported resolution is 4096 x 2304 based on RTL synthesis.
Compressed format
G2 decoder supports compressed tiled output when reference buffer compression (RFC) is enabled when synthesis and user allow compressed reference frame output directly. Reference frame compression is a feature added in G2 decoder to compress frame buffer so that the bandwidth of storing/loading reference frame can be reduced, especially when the resolution of the decoded stream is of high definition.
