GstInference Embedding application
 | Make sure you also check GstInference's companion project: R2Inference |
This example receives video file, images, or camera stream as input and process each frame to detect if a face is valid. For each processed frame the application captures the signal emitted by GstInference, forwarding the prediction to a placeholder for external logic which computes if the metadata belongs to a valid face. Simultaneously, the pipeline displays the captured frames with the associated label in a window in case of a valid face, otherwise, a fail label will be displayed. Note that the image currently being displayed not necessarily matches the one being handled by the signal. The display is meant for visualization and debugging purposes.
The recognition architecture being used by the example is FaceNetV1 trained using the Large Scale CelebFaces Attributes Dataset. A pre-trained model can be downloaded from the GstInference Model Zoo
These examples serve both as an example and as a starting point for a Face Recognition application.
Contents
Building the Example
The example builds along with the GstInference project. Make sure you follow the instructions in Building the Plug-In to make sure all the dependencies are correctly fulfilled.
Once the project is built the example may be built independently by running make within the example directory.
1 cd tests/examples/embedding
2 make
The example is not meant to be installed.
Running the Example
The embedding application provides a series of cmdline options to control the behavior of the example. The basic usage is:
./embedding -m MODEL -f FILE -b BACKEND -e EMBEDDINGS -l LABELS [-v]
- -m|--model
- Mandatory. Path to the FaceNetV1 trained model
- -f|--file
- Optional. Path to the video file or image to be used
- If it is not set, a camera stream will be used.
- -b|--backend
- Mandatory. Name of the backend to be used. See Supported Backends for a list of possible options.
- -e|--embeddings
- Mandatory. Path to the text file with embeddings data containing valid faces
- -l|--labels
- Mandatory. Path to the text file with labels associated with the embeddings file
- -v
- Optional. Run verbosely.
You may always run --help for further details.
./embedding --help Usage: embedding [OPTION...] - GstInference Embedding Example Help Options: -h, --help Show help options --help-all Show all help options --help-gst Show GStreamer Options Application Options: -v, --verbose Be verbose -m, --model Model path -e, --embeddings Path to file with encoded valid faces -l, --labels Path to labels from valid faces -f, --file File path (or camera, if omitted) -b, --backend Backend used for inference, example: tensorflow
Extending the Application
The example is composed by the following main files:
- gstembedding.c
- Source file with GStreamer logic
- customlogic.c
- Placeholder for custom logic
- customlogic.h
- Header file for custom logic source
- Makefile
- Build script
- embedding
- Executable generated on build
- images
- Images directory with the valid faces used to generate demo labels and embeddings
- embeddings
- Directory with demo files embeddings.txt and labels.txt using the faces from images directory
The application can be extended by filling in the placeholders in customlogic.c. In the most simple cases, the information provided in handle_prediction should suffice to react to the prediction.
1 void
2 handle_prediction (unsigned char *image,
3 int width,
4 int height,
5 unsigned int size,
6 double *embeddings,
7 int num_dimensions,
8 int verbose,
9 char **embeddings_list,
10 char **labels,
11 int num_embeddings)
12 {
13 /* FILLME: Handle image and prediction here */
14 }
- image
- Image data in RGB raster format.
- width
- The width of the image, in pixels
- height
- The height of the image, in pixels
- size
- The size of the total image, in bytes
- embeddings
- An array of encoded data
- num_dimensions
- The length of the embeddings array
- verbose
- A flag to display the encoded metadata from the current frame
- embeddings_list
- An array of strings containing the metadata from valid faces
- labels
- The list of labels used to generate the embeddings file
- num_embeddings
- The amount of valid faces
 | The signal will block buffer streaming. Long processing operations should perform asynchronously to avoid blocking the pipeline. |
Troubleshooting
The first debug level is GStreamer debug. You may execute the application with debug enabled by running:
1 ./embeddings --gst-debug=2
For advanced support, please contact [1] with the output of the following command:
1 ./embeddings --gst-debug=2,videoinference:6,facenetv1:6
Reporting a Bug
Please feel free to report bugs using GitHub's issue tracker.
