Difference between revisions of "GstInference/Example Applications/Detection"
(→Running the Example) |
m |
||
(9 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
<noinclude> | <noinclude> | ||
− | {{GstInference/Head|previous=Example Applications/Classification|next=Example Applications/ | + | {{GstInference/Head|previous=Example Applications/Classification|next=Example Applications/DispTec|metakeywords=GstInference|title=GstInference Detection application}} |
</noinclude> | </noinclude> | ||
+ | |||
<!-- If you want a custom title for the page, un-comment and edit this line: | <!-- If you want a custom title for the page, un-comment and edit this line: | ||
{{DISPLAYTITLE:GstInference - <descriptive page name>|noerror}} | {{DISPLAYTITLE:GstInference - <descriptive page name>|noerror}} | ||
--> | --> | ||
− | This example receives an input video file and detects objects in each buffer, the possible objects to detect are available | + | This example receives an input video file and detects objects in each buffer, the possible objects to detect are available in the classes section. |
For each frame the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic. Simultaneously, the pipeline displays the captured frames with every detected object marked with a label and a square. Note that the image currently being displayed '''not necessarily''' matches the one being handled by the signal. The display is meant for visualization and debugging purposes. | For each frame the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic. Simultaneously, the pipeline displays the captured frames with every detected object marked with a label and a square. Note that the image currently being displayed '''not necessarily''' matches the one being handled by the signal. The display is meant for visualization and debugging purposes. | ||
− | The detection example is based on TinyYoloV2, trained using the Pascal VOC dataset, more information | + | The detection example is based on TinyYoloV2, trained using the Pascal VOC dataset, more information can be found in [https://pjreddie.com/darknet/yolo/ YOLO: Real-Time Object Detection]. |
+ | A pre-trained model can be downloaded from the [https://www.ridgerun.com/store/Deep-Learning-Models-and-Binaries-c33344794 GstInference Model Zoo] | ||
This examples serves both as an example and as a starting point for detection application. | This examples serves both as an example and as a starting point for detection application. | ||
Line 41: | Line 43: | ||
== Building the Example == | == Building the Example == | ||
− | The example builds along the GstInference project. Make sure you follow the instructions in [[GstInference/Getting_started/Building_the_plugin|Building the Plug-In]] to make sure all the dependencies are correctly fulfilled. | + | The example builds along with the GstInference project. Make sure you follow the instructions in [[GstInference/Getting_started/Building_the_plugin|Building the Plug-In]] to make sure all the dependencies are correctly fulfilled. |
Once the project is built the example may be built independently by running '''make''' within the example directory. | Once the project is built the example may be built independently by running '''make''' within the example directory. | ||
Line 91: | Line 93: | ||
== Extending the Application == | == Extending the Application == | ||
− | The example is composed | + | The example is composed of the following main files: |
;gstdetection.c | ;gstdetection.c | ||
:Source file with GStreamer logic | :Source file with GStreamer logic | ||
Line 131: | Line 133: | ||
:The length of the boxes array | :The length of the boxes array | ||
− | + | {{Ambox | |
− | + | |type=notice | |
− | + | |small=left | |
+ | |issue='''The signal will block buffer streaming. Long processing operations should perform asynchronously to avoid blocking the pipeline.''' | ||
+ | |style=width:unset; | ||
+ | }} | ||
== Troubleshooting == | == Troubleshooting == | ||
Line 143: | Line 148: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | For advanced support, please contact [mailto:support@ridgerun.com] with the output of the following command: | + | For advanced support, please [https://www.ridgerun.com/contact <U>Contact</U>] RidgeRun OR email to [mailto:support@ridgerun.com <u>support@ridgerun.com</u>] with the output of the following command: |
+ | |||
<syntaxhighlight lang=bash line=1> | <syntaxhighlight lang=bash line=1> | ||
./detection --gst-debug=2,videoinference:6,tinyyolo:6 | ./detection --gst-debug=2,videoinference:6,tinyyolo:6 | ||
Line 154: | Line 160: | ||
<noinclude> | <noinclude> | ||
− | {{GstInference/Foot|Example Applications/Classification|Example Applications/ | + | {{GstInference/Foot|Example Applications/Classification|Example Applications/DispTec}} |
</noinclude> | </noinclude> |
Latest revision as of 14:39, 27 February 2023
Make sure you also check GstInference's companion project: R2Inference |
This example receives an input video file and detects objects in each buffer, the possible objects to detect are available in the classes section.
For each frame the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic. Simultaneously, the pipeline displays the captured frames with every detected object marked with a label and a square. Note that the image currently being displayed not necessarily matches the one being handled by the signal. The display is meant for visualization and debugging purposes.
The detection example is based on TinyYoloV2, trained using the Pascal VOC dataset, more information can be found in YOLO: Real-Time Object Detection. A pre-trained model can be downloaded from the GstInference Model Zoo
This examples serves both as an example and as a starting point for detection application.
Contents
Pascal VOC Classes
The VOC dataset used for training TinyYoloV2 supports 20 classes for inference. Classes according to label ID:
- aeroplane
- bicycle
- bird
- boat
- bottle
- bus
- car
- cat
- chair
- cow
- diningtable
- dog
- horse
- motorbike
- person
- pottedplant
- sheep
- sofa
- train
- tvmonitor
Building the Example
The example builds along with the GstInference project. Make sure you follow the instructions in Building the Plug-In to make sure all the dependencies are correctly fulfilled.
Once the project is built the example may be built independently by running make within the example directory.
1 cd tests/examples/detection
2 make
The example is not meant to be installed.
Running the Example
The detection application provides a series of cmdline options to control the behavior of the example. The basic usage is:
./detection -m MODEL -f FILE -b BACKEND [-v]
- -m|--model
- Mandatory. Path to the TinyYoloV2 trained model
- -f|--file
- Mandatory. Path to the video file to be used.
- If it is not set, a camera stream will be used.
- -b|--backend
- Mandatory. Name of the backed to be used. See Supported Backends for a list of possible options.
- -v
- Optional. Run verbosely.
You may always run --help for further details.
./detection --help Usage: detection [OPTIONS] - GstInference Detection Example Help Options: -h, --help Show help options --help-all Show all help options --help-gst Show GStreamer Options Application Options: -v, --verbose Be verbose -m, --model Model path -f, --file File path -b, --backend Backend used for inference, example: tensorflow
Extending the Application
The example is composed of the following main files:
- gstdetection.c
- Source file with GStreamer logic
- customlogic.c
- Placeholder for custom logic
- customlogic.h
- Header file for custom logic source
- Makefile
- Build script
- detection
- Executable generated on build
The application can be extended by filling in the placeholders in customlogic.c. In the most simple cases, the information provided in handle_predictionshould suffice to react to the prediction.
1 void
2 handle_prediction (unsigned char *image,
3 int width,
4 int height,
5 unsigned int size,
6 BoundingBox * boxes,
7 int num_boxes)
8 {
9 /* FILLME: Handle image and prediction here */
10 }
- image
- Image data in RGB raster format.
- width
- The width of the image, in pixels
- height
- The height of the image, in pixels
- size
- The size of the total image, in bytes
- boxes
- An array of boxes with all detected objects in the frame
- num_boxes
- The length of the boxes array
The signal will block buffer streaming. Long processing operations should perform asynchronously to avoid blocking the pipeline. |
Troubleshooting
The first debug level is GStreamer debug. You may execute the application with debug enabled by running:
1 ./detection --gst-debug=2
For advanced support, please Contact RidgeRun OR email to support@ridgerun.com with the output of the following command:
1 ./detection --gst-debug=2,videoinference:6,tinyyolo:6
Reporting a Bug
Please feel free to report bugs using GitHub's issue tracker.