GStreamer DispTEC Tracker Plugin

From RidgeRun Developer Connection
Jump to: navigation, search

Error something wrong.jpg Problems running the pipelines shown on this page?
Please see our GStreamer Debugging guide for help.

RR Contact Us.png

GstDispTEC Object Tracking Overview

GstDispTEC object tracker is a GStreamer element which integrates DispTEC [1] library's algorithms for object tracking, making it possible to incorporate their functionalities into GStreamer pipelines, such as showing where the object is located on a frame. This allows the user to strengthen image analysis by connecting its own applications to the pipeline for receiving the analysis data and much more amazing things, as shown in the following demo.


Object Tracking Description

Object tracking is implemented based on Kernelized Correlation Filters (KCF). Each tracker element is initialized with the coordinates of the object in the current frame, snipping the object from the frame and taking it as a template. Afterward, the tracker element updates the filter every time that there is an object hit, adapting itself to the possible changes which an object might have during the video, making it robust enough to cope with potential perspective changes.

This object tracking technique gives the following features:

  • Non-stationary camera resilient.
  • Tolerates object scaling, rotation, and transformation.
  • Resilient to changes in color and light.

GstDispTEC object tracker goes even further, making it possible to reset the object position at run-time, even if the pipeline is in the Play state. This includes:

  • Possibility of tracker cascading.
  • Tracker reinitialization at run-time.
  • Application integrability.
  • Visual feedback of object position overlaid to frames.
  • Multi-tracker deployment in the same element (coming soon).


The object tracker element gets some parameters to surround the object to get a template and start tracking. Also, it is possible to enable the visual feedback overlaid on the frame highlighting the current position. These parameters are described below:

Parameter Description
x (Unsigned int) Object x coordinate at the top-left corner in pixels. Range: 0 - 4294967295. Default: 0
y (Unsigned int) Object y coordinate at the top-left corner in pixels. Range: 0 - 4294967295. Default: 0
width (Unsigned int) Object width in pixels. Range: 0 - 4294967295. Default: 100
height (Unsigned int) Object height in pixels. Range: 0 - 4294967295. Default: 100
render-bbox (Boolean) Render a bounding box that gives a visual representation of where the object is currently located. Default: TRUE

The tracker element also generates a signal for third-party applications for retrieving the current object position (coordinates), whose coordinates are expressed as integers in the following sequence: x, y, width, and height.

Object Tracker Architecture

GstDispTEC object tracker has been created based on the DispTEC library. However, it is independent of the DispTEC's dependencies and it can be easily integrated whichever library that DispTEC is based on (OpenCV, LTI-lib, etc). This is achieved by an Abstraction layer just on the base of the plug-in, which can adapt it to several DispTEC variations.

By using DispTEC as the base, and joining it with GStreamer, we offer the best of both worlds: the powerful algorithms available in DispTEC and the flexibility of GStreamer to process videos, allowing live-streaming, pipelining, and more.


Error creating thumbnail: Unable to save thumbnail to destination
GstDispTEC architecture diagram.

GstDispTEC Architecture Description illustrates the general architecture of the plug-in and what makes it one of the best solutions for image analysis for GStreamer.

Supported platforms

GstDispTEC object tracker currently supports platforms that are able to run the Open Computer Vision Library (OpenCV). Some of the platforms already tested are:

Platform Operating System
Nvidia Jetson TX1 Ubuntu 16.04 64 bits
Personal Computer (PC) Ubuntu 16.04 64 bits

Both platforms were running OpenCV 3.3.

Getting the code

Please visit RidgeRun Contact if you are interested in purchasing the GstDispTEC object tracker plugin, with full source code.

Building GstDispTEC

Dependencies

Before trying to install, make sure of having the following dependencies already installed on the target platform:

  • OpenCV Core >= 3.3
  • OpenCV Contribution Modules >= 3.3
  • DispTEC >= 0.2
  • GStreamer >= 1.8.1
  • Autotools

Installing dependencies

DispTEC

sudo apt install cmake
git clone https://github.com/RidgeRun/dispTEC_2017
cmake .
make
sudo make install

OpenCV

For installing OpenCV, please, follow the steps in the link: OpenCV Installation

GStreamer

sudo apt-get install \
gstreamer1.0-x \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev

Autotools

sudo apt install \
autotools-dev \
autoconf

Installing GstDispTEC Object Tracker Plugin

When you purchase the gst-dttracker plugin, you will get a Gitlab repository with the source code inside. You need it to build it in your system and add it to your GStreamer plug-ins catalogue. So, in a temporary path, clone the repository using: 

git clone git://git@gitlab.com/RidgeRun/orders/${CUSTOMER_DIRECTORY}/gst-disptec.git

Where ${CUSTOMER_DIRECTORY} contains the name of your customer directory given after the purchase.

Afterwards, find the path where GStreamer looks for plug-ins and libraries (LIBDIR). Take as a reference to the following table:

Platform LIBDIR path
PC 32-bits/x86 /usr/lib/i386-linux-gnu/
PC 64-bits/x86 /usr/lib/x86_64-linux-gnu/
Nvidia Jetson /usr/lib/aarch64-linux-gnu/

According to the table, define an environment variable which stores the path accordingly to your system. For example, for an x64 PC, it will be: 

LIBDIR=/usr/lib/x86_64-linux-gnu/

Then, do the installation using: 

./autogen.sh
./configure --libdir $LIBDIR
make
sudo make install

Test installation

For testing if everything is properly installed, run: 

gst-inspect-1.0 dttracker

It should show: 

Factory Details:
  Rank                     none (0)
  Long-name                DispTEC tracker element
  Klass                    Generic
  Description              DispTEC object tracking element over a video stream
  Author                   <luis.leon@ridgerun.com, carolina.trejos@ridgerun.com>

Plugin Details:
  Name                     disptec
  Description              DispTEC Image processing library plugin
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstdisptec.so
  Version                  0.3.0
  License                  Proprietary
  Source module            gst-disptec
  Binary package           RidgeRun
  Origin URL               https://www.ridgerun.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseTransform
                         +----GstVideoFilter
                               +----GstDttracker

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: BGR
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: BGR
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_element_change_state_func

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "dttracker0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  qos                 : Handle Quality-of-Service events
                        flags: readable, writable
                        Boolean. Default: true
  x                   : Initial left corner X position
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  y                   : Initial left corner Y position
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  width               : Initial object width
                        flags: readable, writable
                        Unsigned Integer. Range: 1 - 4294967295 Default: 100 
  height              : Initial object height
                        flags: readable, writable
                        Unsigned Integer. Range: 1 - 4294967295 Default: 100 
  render-bbox         : Enables or disables the bounding box around the tracked object
                        flags: readable, writable
                        Boolean. Default: true

Element Signals:
  "coordinates" :  void user_function (GstElement* object,
                                       guint arg0,
                                       guint arg1,
                                       guint arg2,
                                       guint arg3,
                                       gpointer user_data);

Object Tracking Examples

Example 1: Ball pattern with visual feedback 

gst-launch-1.0 videotestsrc pattern=ball ! videoconvert ! dttracker name=tracker x=130 y=90 width=50 height=50 enable=true ! videoconvert ! xvimagesink

For this case, the ball always starts at x=130, y=90 and has a width around 50x50 px.

Example 2: Video loaded from a file 

INITIAL_OBJ_POSITION="x=100 y=100 width=100 height=100"
gst-launch-1.0 filesrc location=$PATH_TO_FILE ! videoconvert ! dttracker name=tracker $INITIAL_OBJ_POSITION enable=true ! videoconvert ! xvimagesink

Set the object initial position into INITIAL_OBJ_POSITION

Example 3: Capturing from camera 

INITIAL_OBJ_POSITION="x=100 y=100 width=100 height=100"
gst-launch v4l2src device=$CAPTURE_DEVICE ! $CAPS ! dttracker name=tracker $INITIAL_OBJ_POSITION enable=true ! videoconvert ! xvimagesink

References

  1. ↑ http://www.ie.tec.ac.cr/palvarado/pmwiki/index.php/Proyectos/DispTEC2?userlang=en


RidgeRun Resources

Quick Start Client Engagement Process RidgeRun Blog Homepage
Technical and Sales Support RidgeRun Online Store RidgeRun Videos Contact Us

OOjs UI icon message-progressive.svg Contact Us

Visit our Main Website for the RidgeRun Products and Online Store. RidgeRun Engineering informations are available in RidgeRun Professional Services, RidgeRun Subscription Model and Client Engagement Process wiki pages. Please email to support@ridgerun.com for technical questions and contactus@ridgerun.com for other queries. Contact details for sponsoring the RidgeRun GStreamer projects are available in Sponsor Projects page. Ridgerun-logo.svg
RR Contact Us.png
Retrieved from "http://developer.ridgerun.com/wiki/index.php?title=GStreamer_DispTEC_Tracker_Plugin&oldid=45590"