GStreamer DispTEC Motion Detection Plugin

Overview

Gst-Dtmotion is a GStreamer element which integrates DispTEC library's algorithms for motion detection, making it possible to incorporate their functionality into GStreamer pipelines, such as motion detection in a video sequence, even with non-stationary cameras!

With GstDispTEC, it is possible to add several DispTEC-based elements in the same pipeline to strengthen image analysis, connect your own applications to the pipeline for receiving the analysis data and much more amazing things. Also, the algorithms can be run either on CPU or GPU, making the most of the computational resources and reducing the processing time.

Description

The motion detection element is based on DispTEC's MCD and SCBU algorithms, designed to run on both CPU and GPU. Its primary feature is the motion detection inside a non-stationary capture, although it can also be used on stationary captures. When deciding which of the algorithms to use, consider the following guidelines :

• MCD algorithm is faster, especially for smaller resolutions, and the result contains less noise than its counterpart.
• SCBU adjusts better to brightness changes on the captured video but it is more sensible to fast changes in movement. The output is more uniform but contains more noise than MCD.

Dtmotion is the designed GStreamer element that integrates the previously described capabilities into a videofilter derived element that can be easily incorporated into a pipeline. It receives any input and generates a new frame built upon a binary mask that exposes the detected motion. In order to adjust the element to the desired application, the following properties can be modified:

Name	Description
algorithm	Both MCD and SCBU algorithms are available to run on CPU or GPU, depending on the target application. If not specified, the default MCD algorithm on CPU is selected. The available options are the following: (0): cpu-mcd - MCD on CPU (1): cpu-scbu - SCBU on CPU (2): gpu-mcd - MCD on GPU (3): gpu-scbu - SCBU On GPU
block-size	(Only for SCBU) Unsigned Integer. Range: 0 - 4294967295. Default: 4. It is an important measure of the algorithm's granularity. The captured image is divided into a number of blocks depending on the specified size. If it is set to be too small, the slightest change between frames may result in motion detection. It will also require greater memory usage, but the processing will be faster and more efficient. Greater value will reduce the amount of noise on the output, and require less memory, but the processing is less efficient.
grid-size	(Only for SCBU) Unsigned Integer. Range: 0 - 4294967295 Default: 60 Grid size determines the distance between the resulting amount of points that will be used to determine how much the background moved. The more points are used, the more precise the measurement. However, the algorithm used is fairly resistant to noise, so it won't make much difference to use a smaller value. A large grid size (fewer points) usually yields good results.
max-learning-rate	(Only for SCBU) Unsigned Integer. Range: 0 - 4294967295 Default: 5 The varying learning rate property is a key feature if trying to detect motion inside of non-static captures, given that it allows adaptation to object movement vs background movement. Increasing the learning rate decreases the sensibility to object motion detection, but improves the rate at which it adapts to background changes. A low maximum value helps in noisy captures due to the fact that the detected moving objects won't be absorbed as background.

Architecture

GstDispTEC has been created based on DispTEC library. However, GstDispTEC is independent of the DispTEC's dependencies and it can be easily integrated whichever library that DispTEC is based on (OpenCV, LTI-lib, etc). This is achieved by an Abstraction layer just on the base of GstDispTEC plug-in, which can adapt it to several DispTEC variations.

By using DispTEC as the GstDispTEC base, it is possible to perform and run Computer Vision algorithms easily on CPU/GPU, accelerating the performance and delivery time of video analyses. Nevertheless, joining DispTEC with GStreamer, GstDispTEC offers the best of both worlds: the powerful algorithms available in DispTEC and the flexibility of GStreamer to process videos, allowing live-streaming, pipelining and more.

Error creating thumbnail: Unable to save thumbnail to destination

GstDispTEC architecture diagram.

GstDispTEC Arquitecture Description illustrates the general architecture of the plug-in and what makes it one of the best solutions for image analysis for GStreamer.

Supported platforms

GstDispTEC currently supports platforms capable to run Open Computer Vision Library (OpenCV). For a better experience, it is recommended to run on a platform with CUDA support for accelerating the motion detection element. Some of the platforms already tested are:

Platform	Operating System
Nvidia Jetson TX1	Ubuntu 16.04 64 bits
Personal Computer (PC)	Ubuntu 16.04 64 bits

Both platforms were running OpenCV 3.3.

In case of not having a platform not capable of running CUDA, GstDispTEC will run on CPU.

Building GstDispTEC

Dependencies

Before trying to install, make sure of having the following dependencies already installed on the target platform:

• OpenCV Core >= 3.3
• OpenCV Contribution Modules >= 3.3
• DispTEC >= 0.2
• CUDA >= 9.0
• GStreamer >= 1.8.1
• Autotools

Installing dependencies

DispTEC

sudo apt install cmake
git clone https://github.com/RidgeRun/dispTEC_2017
cmake .
make
sudo make install

OpenCV

For installing OpenCV, please, follow the steps in the link: OpenCV Installation

GStreamer

sudo apt-get install \
gstreamer1.0-x \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev

Autotools

sudo apt install \
autotools-dev \
autoconf

Installing GstDispTEC

When you purchase GstDispTEC, you will get a Gitlab repository with the source code inside. You need it to build it in your system and add it to your GStreamer plug-ins catalogue. So, in a temporary path, clone the repository using:

git clone git://git@gitlab.com/RidgeRun/orders/${CUSTOMER_DIRECTORY}/gst-disptec.git

Where ${CUSTOMER_DIRECTORY} contains the name of your customer directory given after the purchase.

Afterwards, find the path where GStreamer looks for plug-ins and libraries (LIBDIR). Take as a reference to the following table:

Platform	LIBDIR path
PC 32-bits/x86	/usr/lib/i386-linux-gnu/
PC 64-bits/x86	/usr/lib/x86_64-linux-gnu/
Nvidia Jetson	/usr/lib/aarch64-linux-gnu/

According to the table, define an environment variable which stores the path accordingly to your system. For example, for an x64 PC, it will be:

LIBDIR=/usr/lib/x86_64-linux-gnu/

Then, do the installation using:

./autogen.sh
./configure --libdir $LIBDIR
make
sudo make install

Test installation

For testing if everything is properly installed, run:

gst-inspect-1.0 disptec

It should show:

Plugin Details:
  Name                     disptec
  Description              DispTEC Image processing library plugin
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstdisptec.so
  Version                  0.3.0
  License                  Proprietary
  Source module            gst-disptec
  Binary package           RidgeRun
  Origin URL               https://www.ridgerun.com

  dttracker: DispTEC tracker element
  dtmotion: DispTEC motion detector

  2 features:
  +-- 2 elements

Examples

Example 1: Default settings

gst-launch-1.0 filesrc location=$PATH_TO_FILE ! decodebin name=dec ! videoconvert ! dtmotion algorithm=cpu-mcd ! videoconvert ! xvimagesink

Example 2: Setting all properties

gst-launch-1.0 filesrc location=$PATH_TO_FILE ! decodebin name=dec ! videoconvert ! dtmotion algorithm=cpu-scbu max-learning-rate=5 block-size=4 grid-size=60 ! videoconvert ! xvimagesink

Example 3: Capturing from camera

CAPTURE_DEVICE=/dev/video0
CAPS='video/x-raw, format=(string)UYVY, width=(int)640, height=(int)360, framerate=30/1'
gst-launch v4l2src device=$CAPTURE_DEVICE ! $CAPS !  dtmotion algorithm=cpu-mcd ! videoconvert ! xvimagesink