Difference between revisions of "GstInference/Example Applications/Classification"

From RidgeRun Developer Connection
Jump to: navigation, search
(→‎Building the Example)
m
 
(26 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
<noinclude>
 
<noinclude>
−
{{GstInference/Head}}
+
{{GstInference/Head|previous=Example Applications|next=Example Applications/Detection|metakeywords=GstInference,classification application,classification architecture,InceptionV4 |title=GstInference Classification application}}
 
</noinclude>
 
</noinclude>
 +
 
<!-- If you want a custom title for the page, un-comment and edit this line:
 
<!-- If you want a custom title for the page, un-comment and edit this line:
 
{{DISPLAYTITLE:GstInference - <descriptive page name>|noerror}}
 
{{DISPLAYTITLE:GstInference - <descriptive page name>|noerror}}
 
-->
 
-->
  
−
This example receives an input video file and classifies each frame into one of 1000 possible classes. For each classified frame the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic. Simultaneously, the pipeline displays the captured frames with the associated label in a window. Not that the image currently being displayed '''not necessarily''' matches de one being handled by the signal. The display is meant for visualization and debugging purposes.  
+
This example receives an input video file and classifies each frame into one of the 1000 possible classes. For each classified frame, the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic. Simultaneously, the pipeline displays the captured frames with the associated label in a window. Note that the image currently being displayed '''not necessarily''' matches the one being handled by the signal. The display is meant for visualization and debugging purposes.  
  
−
The classification architecture being used by the example is InceptionV4 trained using the [http://image-net.org/challenges/LSVRC/2012/browse-synsets Large Scale Visual Recognition Challenge 2012 (ILSVRC2012) Dataset]. A pre-trained model can be downloaded from the [https://www.ridgerun.com/store/Deep-Learning-Models-and-Binaries-c33344794 GstInference Model Zoo]
+
The classification architecture being used by the example is InceptionV4 trained using the [http://image-net.org/challenges/LSVRC/2012/browse-synsets Large Scale Visual Recognition Challenge 2012 (ILSVRC2012) Dataset]. A pre-trained model can be downloaded from the [https://developer.ridgerun.com/wiki/index.php?title=GstInference/Model_Zoo GstInference Model Zoo]
  
 
This examples serves both as an example and as a starting point for a classification application.
 
This examples serves both as an example and as a starting point for a classification application.
Line 16: Line 17:
 
== Building the Example ==
 
== Building the Example ==
  
−
The example builds along the GstInference project. Make sure you follow the instructions in [GstInference/Building the Plugin] to make sure all the dependencies are correctly fulfilled.
+
The example builds along with the GstInference project. Make sure you follow the instructions in [[GstInference/Getting_started/Building_the_plugin|Building the Plug-In]] to make sure all the dependencies are correctly fulfilled.
  
 
Once the project is built the example may be built independently by running '''make''' within the example directory.
 
Once the project is built the example may be built independently by running '''make''' within the example directory.
Line 29: Line 30:
 
== Running the Example ==
 
== Running the Example ==
  
−
The classification application provides a series of cmdline options to control de behavior of the example. The basic usage is:
+
The classification application provides a series of cmdline options to control the behavior of the example. The basic usage is:
  
 
<syntaxhighlight lang=bash lines=1>
 
<syntaxhighlight lang=bash lines=1>
Line 38: Line 39:
 
:Mandatory. Path to the InceptionV4 trained model
 
:Mandatory. Path to the InceptionV4 trained model
 
;-f|--file
 
;-f|--file
−
:Mandatory. Path to the video file to be used
+
:Optional. Path to the video file to be used.
−
;-b|--backed
+
:If it is not set, a camera stream will be used.
−
:Mandatory. Name of the backed to be used. See [GstInference/Supported backends] for a list of possible options.
+
;-b|--backend
 +
:Mandatory. Name of the backend to be used. See [[GstInference/Supported backends|Supported Backends]] for a list of possible options.
 
;-v
 
;-v
 
:Optional. Run verbosely.
 
:Optional. Run verbosely.
Line 46: Line 48:
 
You may always run '''--help''' for further details.
 
You may always run '''--help''' for further details.
  
−
<pre>
+
<syntaxhighlight lang=bash lines=1>
 
./classification --help
 
./classification --help
 
Usage:
 
Usage:
Line 61: Line 63:
 
   -f, --file                        File path
 
   -f, --file                        File path
 
   -b, --backend                    Backend used for inference, example: tensorflow
 
   -b, --backend                    Backend used for inference, example: tensorflow
−
</pre>
+
</syntaxhighlight>
  
 
== Extending the Application ==
 
== Extending the Application ==
 +
 +
The example is composed of the following main files:
 +
;gstclassification.c
 +
:Source file with GStreamer logic
 +
;customlogic.c
 +
:Placeholder for custom logic
 +
;customlogic.h
 +
:Header file for custom logic source
 +
;Makefile
 +
:Build script
 +
;classification
 +
:Executable generated on build
 +
 +
The application can be extended by filling in the placeholders in '''customlogic.c'''. In the most simple cases, the information provided in '''handle_prediction'''should suffice to react to the prediction.
 +
 +
<syntaxhighlight lang=c line=1>
 +
void
 +
handle_prediction (unsigned char *image,
 +
                  int width,
 +
                  int height,
 +
                  int size,
 +
                  double *probabilities,
 +
                  int num_probabilities)
 +
{
 +
    /* FILLME: Handle image and prediction here */
 +
}
 +
</syntaxhighlight>
 +
 +
;image
 +
:Image data in '''RGB''' raster format.
 +
;width
 +
:The width of the image, in pixels
 +
;height
 +
:The height of the image, in pixels
 +
;size
 +
:The size of the total image, in bytes
 +
;probabilities
 +
:An array of probabilities for each class
 +
;num_probabilities
 +
:The length of the probabilities array
 +
<!-------
 +
<pre style=background-color:yellow>
 +
NOTE: The signal will block buffer streaming. Long processing operations should perform asynchronously to avoid blocking the pipeline
 +
</pre>
 +
---->
 +
{{Ambox
 +
|type=notice
 +
|small=left
 +
|issue='''The signal will block buffer streaming. Long processing operations should perform asynchronously to avoid blocking the pipeline'''
 +
|style=width:unset;
 +
}}
  
 
== Troubleshooting ==
 
== Troubleshooting ==
 +
 +
The first debug level is GStreamer debug. You may execute the application with debug enabled by running:
 +
 +
<syntaxhighlight lang=bash line=1>
 +
./classification --gst-debug=2
 +
</syntaxhighlight>
 +
 +
For advanced support, please contact [mailto:support@ridgerun.com] with the output of the following command:
 +
<syntaxhighlight lang=bash line=1>
 +
./classification --gst-debug=2,videoinference:6,inceptionv4:6
 +
</syntaxhighlight>
 +
 +
== Reporting a Bug ==
 +
 +
Please feel free to report bugs using [https://github.com/RidgeRun/gst-inference/issues GitHub's issue tracker].
  
 
<noinclude>
 
<noinclude>
−
{{GstInference/Foot|<Replace with "previous" page>|<Replace with "next" page>}}
+
{{GstInference/Foot|Example Applications|Example Applications/Detection}}
 
</noinclude>
 
</noinclude>

Latest revision as of 14:38, 27 February 2023



Previous: Example Applications Index Next: Example Applications/Detection


GstInference
GstInference
Introduction
Getting started
Supported architectures
Supported backends
Metadata and Signals
Overlay Elements
Utils Elements
Legacy pipelines
Example pipelines
Example applications
Benchmarks
Model Zoo
Project Status
Contact Us

Coral from Google




This example receives an input video file and classifies each frame into one of the 1000 possible classes. For each classified frame, the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic. Simultaneously, the pipeline displays the captured frames with the associated label in a window. Note that the image currently being displayed not necessarily matches the one being handled by the signal. The display is meant for visualization and debugging purposes.

The classification architecture being used by the example is InceptionV4 trained using the Large Scale Visual Recognition Challenge 2012 (ILSVRC2012) Dataset. A pre-trained model can be downloaded from the GstInference Model Zoo

This examples serves both as an example and as a starting point for a classification application.

Building the Example

The example builds along with the GstInference project. Make sure you follow the instructions in Building the Plug-In to make sure all the dependencies are correctly fulfilled.

Once the project is built the example may be built independently by running make within the example directory. 

1 cd tests/examples/classification
2 make

The example is not meant to be installed.

Running the Example

The classification application provides a series of cmdline options to control the behavior of the example. The basic usage is: 

./classification -m MODEL -f FILE -b BACKEND [-v]
-m|--model
Mandatory. Path to the InceptionV4 trained model
-f|--file
Optional. Path to the video file to be used.
If it is not set, a camera stream will be used.
-b|--backend
Mandatory. Name of the backend to be used. See Supported Backends for a list of possible options.
-v
Optional. Run verbosely.

You may always run --help for further details. 

./classification --help
Usage:
  classification [OPTIONS] - GstInference Classification Example

Help Options:
  -h, --help                        Show help options
  --help-all                        Show all help options
  --help-gst                        Show GStreamer Options

Application Options:
  -v, --verbose                     Be verbose
  -m, --model                       Model path
  -f, --file                        File path
  -b, --backend                     Backend used for inference, example: tensorflow

Extending the Application

The example is composed of the following main files:

gstclassification.c
Source file with GStreamer logic
customlogic.c
Placeholder for custom logic
customlogic.h
Header file for custom logic source
Makefile
Build script
classification
Executable generated on build

The application can be extended by filling in the placeholders in customlogic.c. In the most simple cases, the information provided in handle_predictionshould suffice to react to the prediction. 

 1 void
 2 handle_prediction (unsigned char *image, 
 3                    int width, 
 4                    int height,
 5                    int size,
 6                    double *probabilities,
 7                    int num_probabilities)
 8 {
 9     /* FILLME: Handle image and prediction here */
10 }
image
Image data in RGB raster format.
width
The width of the image, in pixels
height
The height of the image, in pixels
size
The size of the total image, in bytes
probabilities
An array of probabilities for each class
num_probabilities
The length of the probabilities array

Troubleshooting

The first debug level is GStreamer debug. You may execute the application with debug enabled by running: 

1 ./classification --gst-debug=2

For advanced support, please contact [1] with the output of the following command: 

1 ./classification --gst-debug=2,videoinference:6,inceptionv4:6

Reporting a Bug

Please feel free to report bugs using GitHub's issue tracker.


Previous: Example Applications Index Next: Example Applications/Detection



Retrieved from "http://developer.ridgerun.com/wiki/index.php?title=GstInference/Example_Applications/Classification&oldid=46830"