Difference between revisions of "Video crop and scaling with DM816x and DM814x"
(→Video crop + upscaling) |
m |
||
| (23 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
| + | <seo title="Video Scaling & Cropping with DM816x and DM814x" titlemode="replace" keywords="GStreamer, Linux SDK, Linux BSP, Embedded Linux, Device Drivers, Nvidia, Xilinx, TI, NXP, Freescale, Embedded Linux driver development, Linux Software development, Embedded Linux SDK, Embedded Linux Application development, GStreamer Multimedia Framework." description="This document presents a brief overview of the crop and scaling features available for the DM816x and DM814x platforms. Read this useful wiki page now."></seo> | ||
| + | |||
= Introduction = | = Introduction = | ||
| Line 9: | Line 11: | ||
The capture's crop and scaling features are part of the '''v4l2src'' GStreamer plugin and are exposed trough the ''crop-area'' property. This property is set following the next format: | The capture's crop and scaling features are part of the '''v4l2src'' GStreamer plugin and are exposed trough the ''crop-area'' property. This property is set following the next format: | ||
| − | crop-area=<left>,<top>@<width>x<height> | + | <pre style="background:#FFFFC0">crop-area=<left>,<top>@<width>x<height> </pre> |
where ''left'' and ''top'' are the coordinates for the upper left corner of the crop region and the ''width'' and ''height'' values set its size. The figure 1 shows this concept. | where ''left'' and ''top'' are the coordinates for the upper left corner of the crop region and the ''width'' and ''height'' values set its size. The figure 1 shows this concept. | ||
| Line 25: | Line 27: | ||
* If the capabilities after the v4l2src element are of the same size of the input buffer (the case shown) or just bigger than the cropped area, the output buffer will look such as "Output buffer (a)", the remaining space will be basically old data in memory. The following pipeline is an example of this setup. | * If the capabilities after the v4l2src element are of the same size of the input buffer (the case shown) or just bigger than the cropped area, the output buffer will look such as "Output buffer (a)", the remaining space will be basically old data in memory. The following pipeline is an example of this setup. | ||
| − | <pre> | + | <pre style="background:#FFFFC0">VIDEO_STD=720P_60 |
| − | VIDEO_STD=720P_60 | ||
CLEFT=20 | CLEFT=20 | ||
CTOP=30 | CTOP=30 | ||
| Line 43: | Line 44: | ||
* If the width and height of the capabilities is the same that the ones specified in the crop-area parameter the output should look like "Output buffer (b)" where we have cropped the input video and the scaling feature has resized the output buffer to fit the cropped region. The following pipeline is an example of this setup. | * If the width and height of the capabilities is the same that the ones specified in the crop-area parameter the output should look like "Output buffer (b)" where we have cropped the input video and the scaling feature has resized the output buffer to fit the cropped region. The following pipeline is an example of this setup. | ||
| − | <pre> | + | |
| − | VIDEO_STD=720P_60 | + | <pre style="background:#FFFFC0">VIDEO_STD=720P_60 |
CLEFT=100 | CLEFT=100 | ||
CTOP=100 | CTOP=100 | ||
| Line 59: | Line 60: | ||
* If the ''crop-area'' parameter is omitted, v4l2src will use the width and height values from the pipeline capabilities and the left and top values will be set to 0. The following pipeline is an example of this setup. | * If the ''crop-area'' parameter is omitted, v4l2src will use the width and height values from the pipeline capabilities and the left and top values will be set to 0. The following pipeline is an example of this setup. | ||
| − | <pre> | + | <pre style="background:#FFFFC0"> |
VIDEO_STD=720P_60 | VIDEO_STD=720P_60 | ||
| Line 69: | Line 70: | ||
</pre> | </pre> | ||
| − | [[File:Crop_feature_02.png|400px|thumb|center|Figure | + | [[File:Crop_feature_02.png|400px|thumb|center|Figure 2. Crop-area working model]] |
= Video display crop = | = Video display crop = | ||
| Line 75: | Line 76: | ||
If it is necessary to work with the input buffer on its original size but it is desired to crop it for display it is possible to play with the ''v4l2sink'' element's properties to do the trick as shown in the following pipeline: | If it is necessary to work with the input buffer on its original size but it is desired to crop it for display it is possible to play with the ''v4l2sink'' element's properties to do the trick as shown in the following pipeline: | ||
| − | <pre> | + | <pre style="background:#FFFFC0"> |
CAPS="video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1" | CAPS="video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1" | ||
CLEFT=100 | CLEFT=100 | ||
| Line 97: | Line 98: | ||
Subframe 1 | Subframe 1 | ||
| − | gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,0@960x540 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=960,height=540,framerate=(fraction)60/1' ! | + | <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,0@960x540 ! \ |
| − | + | 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' \ | |
| + | ! omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ | ||
| + | height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v | ||
| + | </pre> | ||
Subframe 2 | Subframe 2 | ||
| − | gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,0@960x540 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=960,height=540,framerate=(fraction)60/1' ! | + | <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,0@960x540 ! \ |
| − | + | 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' \ | |
| + | ! omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ | ||
| + | height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v | ||
| + | </pre> | ||
Subframe 3 | Subframe 3 | ||
| − | gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,540@960x540 ! 'video/x-raw-yuv,format=(fourcc)NV12,width=960,height=540,framerate=(fraction)60/1' ! | + | <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,540@960x540 ! \ |
| − | + | 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' ! \ | |
| + | omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ | ||
| + | height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v | ||
| + | </pre> | ||
| + | Subframe 4 | ||
| − | + | <pre style="background:#FFFFC0">gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,540@960x540 ! \ | |
| + | 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' ! \ | ||
| + | omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ | ||
| + | height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -v | ||
| + | </pre> | ||
| + | |||
| + | = Video crop in the scaling stage = | ||
| + | |||
| + | It's also possible to do cropping and scaling at the same time in the middle of the pipeline, using the De-Interlacer or the Scaler OpenMax components. Both upscaling and downscaling are supported, so it depends only of the user requirements about the display resolution or other specifications. For example, in the next pipeline the system captures 1080p video, then it crops the input to get the subframe 1, according with Figure 3; and finally it scales the video to be displayed at 720p. | ||
| + | |||
| + | <pre style="background:#FFFFC0"> | ||
| + | gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! 'video/x-raw-yuv, \ | ||
| + | format=(fourcc)NV12, width=1920, height=1080, framerate=(fraction)60/1' ! omxbufferalloc \ | ||
| + | numBuffers=8 ! omx_mdeiscaler crop-area=0,0@960x540 name=d d.src_00 ! 'video/x-raw-yuv, \ | ||
| + | width=(int)1280, height=(int)720' ! gstperf print-fps=true print-arm-load=true ! v4l2sink \ | ||
| + | device=/dev/video1 d.src_01 ! fakesink silent=true -v | ||
| + | </pre> | ||
| − | + | In the next pipeline, omx_scaler plugin is used instead of the omx_deiscaler plugin. In this case, the subframe 3 is gotten and it's scaled to 1080p again to be displayed. | |
| − | |||
| + | <pre style="background:#FFFFC0"> | ||
| + | gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! 'video/x-raw-yuv, \ | ||
| + | format=(fourcc)NV12, width=1920, height=1080, framerate=(fraction)60/1' ! omxbufferalloc \ | ||
| + | numBuffers=8 ! omx_scaler crop-area=960,540@960x540 ! 'video/x-raw-yuv, width=(int)1920, \ | ||
| + | height=(int)1080' ! gstperf print-fps=true print-arm-load=true ! v4l2sink device=/dev/video1 | ||
| + | </pre> | ||
| − | [[Category: | + | [[Category:DM8168]] [[Category:HowTo]] [[Category:Whitepaper]][[Category:DM8148]][[Category:RidgeRunTechnology]] |
Latest revision as of 14:00, 15 December 2017
Contents
Introduction
This document presents a brief overview of the crop and scaling features available for the DM816x and DM814x platforms.
Crop is oriented to the capability of cut a region of an image and just process this section in the next processing units whereas scaling is just referred to resize an image's size to fit a target width and height values. Those features can be done in the at either the capture stage or the display stage.
Video capture crop and scaling
The capture's crop and scaling features are part of the 'v4l2src GStreamer plugin and are exposed trough the crop-area property. This property is set following the next format:
crop-area=<left>,<top>@<width>x<height>
where left and top are the coordinates for the upper left corner of the crop region and the width and height values set its size. The figure 1 shows this concept.
NOTE: The left, top, width and height values MUST be set avoiding to request a region out of the input image's boundaries. <\blockquote>
The crop-area feature was added together with a resizing feature where together execute the following actions:
- The crop feature cuts the input buffer and just considers the data from an specific region as shown in the figure 2.
- If the capabilities after the v4l2src element are of the same size of the input buffer (the case shown) or just bigger than the cropped area, the output buffer will look such as "Output buffer (a)", the remaining space will be basically old data in memory. The following pipeline is an example of this setup.
VIDEO_STD=720P_60 CLEFT=20 CTOP=30 CWIDTH=640 CHEIGHT=480 gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 crop-area=$CLEFT,$CTOP@$CWIDTHx$CHEIHT ! \ 'video/x-raw-yuv, format=(fourcc)NV12, width=1280, height=720, framerate=60/1' ! \ omxbufferalloc numBuffers=12 ! \ omx_scaler ! \ v4l2sink sync=false
- The resize feature is the one that actually determines the size of the output buffer, not the crop feature. It is worth to mention that the resizer is only able to do a downscaling process, upscaling is not supported by the architecture.
- If the width and height of the capabilities is the same that the ones specified in the crop-area parameter the output should look like "Output buffer (b)" where we have cropped the input video and the scaling feature has resized the output buffer to fit the cropped region. The following pipeline is an example of this setup.
VIDEO_STD=720P_60 CLEFT=100 CTOP=100 CWIDTH=640 CHEIGHT=480 gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 crop-area=$CLEFT,$CTOP@$CWIDTHx$CHEIHT ! \ 'video/x-raw-yuv, format=(fourcc)NV12, width=640, height=480, framerate=60/1' ! \ omxbufferalloc numBuffers=12 ! \ omx_scaler ! \ v4l2sink sync=false
- If the crop-area parameter is omitted, v4l2src will use the width and height values from the pipeline capabilities and the left and top values will be set to 0. The following pipeline is an example of this setup.
VIDEO_STD=720P_60 gst-launch -e v4l2src videoStd=$VIDEO_STD device=/dev/video0 always-copy=false queue-size=8 ! \ 'video/x-raw-yuv, format=(fourcc)NV12, width=640, height=480, framerate=60/1' ! \ omxbufferalloc numBuffers=12 ! \ omx_scaler ! \ v4l2sink sync=falseVideo display crop
If it is necessary to work with the input buffer on its original size but it is desired to crop it for display it is possible to play with the v4l2sink element's properties to do the trick as shown in the following pipeline:
CAPS="video/x-raw-yuv,format=(fourcc)NV12,width=1920,height=1080,framerate=(fraction)60/1" CLEFT=100 CTOP=100 CWIDTH=640 CHEIGHT=480 gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! capsfilter caps="$CAPS" ! \ omxbufferalloc numBuffers=12 ! gstperf ! omx_scaler ! \ v4l2sink device=/dev/video1 crop-left=$CLEFT crop-top=$CTOP crop-width-$CWIDTH crop-height=$CHEIGHTwhere the left, top, width and heightproperties work in the same way that in the v4l2src element.
Video crop + upscaling
Other feature available is crop plus upscaling, an use case for example would be to separate a 1080p frame in 4 subframes (960x540), and upscale each one of them into a larger frame (1080p).
Subframe 1
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,0@960x540 ! \ 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' \ ! omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -vSubframe 2
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,0@960x540 ! \ 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' \ ! omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -vSubframe 3
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=0,540@960x540 ! \ 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' ! \ omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -vSubframe 4
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 crop-area=960,540@960x540 ! \ 'video/x-raw-yuv, format=(fourcc)NV12, width=960, height=540, framerate=(fraction)60/1' ! \ omxbufferalloc numBuffers=8 ! omx_mdeiscaler name=d d.src_00 ! 'video/x-raw-yuv, width=1920, \ height=1080' ! omx_ctrl display-mode=OMX_DC_MODE_1080P_60 ! omx_videosink sync=false d.src_00 -vVideo crop in the scaling stage
It's also possible to do cropping and scaling at the same time in the middle of the pipeline, using the De-Interlacer or the Scaler OpenMax components. Both upscaling and downscaling are supported, so it depends only of the user requirements about the display resolution or other specifications. For example, in the next pipeline the system captures 1080p video, then it crops the input to get the subframe 1, according with Figure 3; and finally it scales the video to be displayed at 720p.
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! 'video/x-raw-yuv, \ format=(fourcc)NV12, width=1920, height=1080, framerate=(fraction)60/1' ! omxbufferalloc \ numBuffers=8 ! omx_mdeiscaler crop-area=0,0@960x540 name=d d.src_00 ! 'video/x-raw-yuv, \ width=(int)1280, height=(int)720' ! gstperf print-fps=true print-arm-load=true ! v4l2sink \ device=/dev/video1 d.src_01 ! fakesink silent=true -vIn the next pipeline, omx_scaler plugin is used instead of the omx_deiscaler plugin. In this case, the subframe 3 is gotten and it's scaled to 1080p again to be displayed.
gst-launch v4l2src device=/dev/video0 always-copy=false queue-size=8 ! 'video/x-raw-yuv, \ format=(fourcc)NV12, width=1920, height=1080, framerate=(fraction)60/1' ! omxbufferalloc \ numBuffers=8 ! omx_scaler crop-area=960,540@960x540 ! 'video/x-raw-yuv, width=(int)1920, \ height=(int)1080' ! gstperf print-fps=true print-arm-load=true ! v4l2sink device=/dev/video1
