V4L2 FPGA - GStreamer Pipelines - Filter

Overview

This wiki section describes how to use the V4L2-FPGA for communicating with a hardware accelerator, which receives an image, does some processing, and sends it back to the system. You can try using one of the following examples for performing these tests:

To use them, you need to configure your FPGA before running any of the pipelines shown below.

Basically, you can implement the pipeline illustrated in Figure 1 using GStreamer, which is performed in the following section.

Error creating thumbnail: Unable to save thumbnail to destination

V4L2 common pipeline

Camera Pipeline

The following pipelines generate a video pattern, sends it to the hardware accelerator implemented on the FPGA and retrieves the result.

4k Resolution

WIDTH=", width=(int)3840"
HEIGHT=", height=(int)2160"

gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw $WIDTH $HEIGHT" ! perf ! fakesink sync=false -v &
gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw $WIDTH $HEIGHT" ! v4l2sink sync=false device=/dev/video2 -v

For seeing the result frame, you can change fakesink by xvimagesink or similar.

1080p Resolution

WIDTH=", width=(int)1920"
HEIGHT=", height=(int)1080"

gst-launch-1.0 v4l2src device=/dev/video1 ! "video/x-raw $WIDTH $HEIGHT" ! perf ! fakesink sync=false -v &
gst-launch-1.0 videotestsrc is-live=true ! "video/x-raw $WIDTH $HEIGHT" ! v4l2sink sync=false device=/dev/video2 -v

Performance Summary

Table 1. Performance for 3x3 convolution on several standard resolutions running at 30fps on a PicoEVB
Resolution	CPU usage (%)	Memory (MB)	Memory Bandwidth Usage (%)
4k@7	2.13	60	1
1080p@30	3.75	25	1
720p@30	2.53	15	1

Table 2. Maximum framerate using 3x3 convolution for several standard resolutions on a PicoEVB
Resolution	Maximum framerate (fps)
4k	7.478
1080p	29.485
720p	64.972

Table 3. Performance for a passthrough on several standard resolutions running at 30fps on a PicoEVB
Resolution	CPU usage (%)	Memory (MB)	Memory Bandwidth Usage (%)
4k@30	2.625	60	4
1080p@30	2.93	25	1
720p@30	2.34	15	1

Table 4. Maximum framerate using a passthrough for several standard resolutions on a PicoEVB
Resolution	Maximum framerate (fps)
4k	38.90
1080p	144.5
720p	294.729

CPU Usage

These measurements were taken with the tegrastats app to see the behavior of the pipeline while running at 30fps.

Cpu usage for convolution for different resolutions at 30fps.

Cpu usage for passthrough for different resolutions at 30fps.

Memory Usage

These measurements were taken with tegrastats app to see the behavior of the pipeline while running at 30fps.

Memory usage for convolution for different resolutions at 30fps.

Memory usage for passthrough for different resolutions at 30fps.

Maximum Framerate

These measurements were taken with gstperf tool to see the behavior of the pipeline while running, with videotestsrc we are able to push buffers as much as possible to the element. The image below shows the maximum framerate that the debayer element supports. It depends on the execution time of the debayer kernel algorithm.