GstInference - Introduction

GstInference
Introduction
Getting started
Getting the code Building the plugin Quick Starting Guide
Supported architectures
InceptionV1 InceptionV2 InceptionV3 InceptionV4 TinyYoloV2 TinyYoloV3 YoloV2 ResNetV1 MobileNetV2 MobileNetV2 + SSD AlexNet
Supported backends
TensorFlow Tensorflow-Lite Coral from Google Caffe TensorRT ONNXRT ONNXRT ACL ONNXRT OpenVINO
Metadata and Signals
GstClassificationMeta GstInferenceMeta Signals
Overlay Elements
Inference Overlay
Utils Elements
InferenceFilter InferenceCrop InferenceDebug InferenceBin
Legacy pipelines
PC NANO TX2 Xavier IMX8
Example pipelines
Example applications
Classification Detection Hand Pose Recognition
Benchmarks
Model Zoo
Project Status
Releases Roadmap
Contact Us

Deep Learning has revolutionized classic computer vision techniques to enable even more intelligent and autonomous systems. Multimedia frameworks, such as GStreamer, are a basic complement of automatic recognition and classification systems. GstInference is an ongoing open-source project from Ridgerun Engineering that allows easy integration of deep learning networks into your existing pipeline.

Software Stack

The following diagram shows how GstInference plays along other software and hardware modules.

Error creating thumbnail: Unable to save thumbnail to destination

GstInference Software Stack

HW Modules

A deep learning model can be inferred in different hardware units. In the most general case, even though performance won't be great, the general purpose CPU may be used. This is ideal for quick prototyping without requiring specialized hardware.

One of the most popular processors to infer models is the GPGPU. Platforms such as CUDA or OpenCL have enabled the concurrent processing power of these units to execute very efficiently deep learning models.

More recently embedded systems are now equipped with TPUs (Tensor Processing Units) or DLAs (Deep Learning Accelerators) which are HW units whose architecture is designed to host deep learning models.

Third party backends

There are a variety of machine learning frameworks that may be used for deep learning inference. Among the most popular we can mention TensorFlow, Caffe, TensorRT, NCSDK, etc...

They serve as abstractions for a (or several) HW unit(s). For example, TensorFlow may be compiled to make use the CPU or the GPU. Similarly, TensorRT may be configured to use the GPU or the DLA.

This provides the benefit of exposing a single interface for several hardware modules.

R2Inference

R2Inference is an open source project by RidgeRun that serves as an abstraction layer in C/C++ for a variety of machine learning frameworks.

As such, a single C/C++ application may work with a Caffe or TensorFlow model, for example.

This is specially useful for hybrid solutions, where multiple models need to be inferred. R2Inference may be able to execute one model on the DLA and another on the CPU, for instance.

GstInference

GstInference is the Gstreamer wrapper over R2Inference. It provides base classes, utilities and a set of elements to bring inference to a Gstreamer pipeline.

Each model architecture requires custom image pre-processing and output interpretation. Similarly, each model has a predefined input format.

Each model architecture (i.e: GoogLeNet) is represented as an independent Gstreamer element (GstGoogleNet). A single element may infer models from different backends, transparently.

GstInference Hierarchy

The following diagram shows how different GstInference modules are related. Details have been omitted for simplicity.

GstInference class diagram (simplified)