View source for Qualcomm Robotics RB5/AI hardware acceleration/Neural Processing SDK/Example project

<noinclude>
{{Qualcomm Robotics RB5/Head|previous=AI_hardware_acceleration/Neural_Processing_SDK/Install_qtimlesnpe|next=AI_hardware_acceleration/TensorFlow|keywords=neuralsdk}}
</noinclude>

{{DISPLAYTITLE:Qualcomm Robotics RB5/RB6 - Neural Processing SDK: Example project|noerror}}

In this section, we are going to see how to work with the Neural Processing SDK. We are going to transform a trained model to a Deep Learning Container (DLC) file, and then use this model with the GStreamer plugin named <code>qtimlesnpe</code> to use the tools from the SDK and GStreamer to execute the model in a live pipeline. Finally, we are going to measure its performance. To continue with these steps, the previous sections should be completed. If not, please check our sections [[Qualcomm_Robotics_RB5/AI_hardware_acceleration/Neural_Processing_SDK/Getting_started|Downloading Requirements]], [[Qualcomm_Robotics_RB5/AI_hardware_acceleration/Neural_Processing_SDK/Setup_environment|Setup SDK Environment]] and [[Qualcomm_Robotics_RB5/AI_hardware_acceleration/Neural_Processing_SDK/Install_qtimlesnpe|Install qtimlesnpe]].

==Download ML Models and Convert them to DLC Format==

We are going to use a script in the SDK to download a model and convert it to DLC. This script will download the Inception V3 TensorFlow model and convert it to DLC.

===Inception v3===
The Neural Processing SDK does not come with any model files, that's why it has scripts to download some popular and public models and converts them into the Deep Learning container needed to use the <code>qtimlesnpe</code> element. In this example, we are using the Inception v3 TensorFlow model for classification. The SDK has a python script that downloads the frozen model and converts it to a DLC file. Please follow the next steps:

'''1'''. Open a terminal in your host computer where you downloaded your Neural Processing SDK.
<syntaxhighlight lang=bash>
cd $SNPE_ROOT
</syntaxhighlight>
<br>

'''2'''. Since this model comes from TensorFlow, we need to setup the environment for this framework. First, lets check where is TensorFlow installed with the following command:
<syntaxhighlight lang=bash>
python -m pip show tensorflow | grep Location
</syntaxhighlight>

The output of the above command should be similar to the following:
<syntaxhighlight lang=bash>
Location: /home/user/.local/lib/python3.6/site-packages
</syntaxhighlight>
<br>

'''3'''. Now, lets define an environment variable for this path. TensorFlow is installed in the above path, but we need to add the specific directory of the package at the end, like the following:
<syntaxhighlight lang=bash>
export TENSORFLOW_DIR=/home/edson/.local/lib/python3.6/site-packages/tensorflow
</syntaxhighlight>
<br>

'''4'''. Lets setup the environment for the SDK with the chosen framework.
<syntaxhighlight lang=bash>
source bin/envsetup.sh -t $TENSORFLOW_DIR
</syntaxhighlight>

You should see the following output:
<syntaxhighlight lang=bash>
[INFO] Setting TENSORFLOW_HOME=/home/user/.local/lib/python3.6/site-packages/tensorflow
</syntaxhighlight>
<br>

'''5'''. Create a new temporary folder for the download.
<syntaxhighlight lang=bash>
mkdir -p /home/user/tmpdir
</syntaxhighlight>
<br>

'''6'''. Finally, lets download the Inception v3 model and convert it to a DLC file.
<syntaxhighlight lang=bash>
python models/inception_v3/scripts/setup_inceptionv3.py -a ~/tmpdir -d
</syntaxhighlight>

You should see an output similar to the one below:
<syntaxhighlight lang=bash>
2023-03-15 15:44:04,338 - 214 - INFO - Processed 0 quantization encodings
2023-03-15 15:44:04,742 - 214 - INFO - INFO_INITIALIZATION_SUCCESS: 
2023-03-15 15:44:04,991 - 214 - INFO - INFO_CONVERSION_SUCCESS: Conversion completed successfully
2023-03-15 15:44:05,350 - 214 - INFO - INFO_WRITE_SUCCESS: 
INFO: Setup inception_v3 completed.
</syntaxhighlight>
<br>

'''7'''. Now, we are going to transfer the DLC and labels files to the board.
<syntaxhighlight lang=bash>
scp models/inception_v3/dlc/inception_v3.dlc qrb5:/data/misc/camera/.
scp models/inception_v3/data/imagenet_slim_labels.txt qrb5:/data/misc/camera/.
</syntaxhighlight>
<br>

You can now continue with the GStreamer pipeline section to check how to use your model with <code>qtimlesnpe</code>.

==GStreamer pipeline==

Now that we have our model in the DLC format, we can use it with the <code>qtimlesnpe</code> element in GStreamer! In the previous steps, we moved the DLC and labesl files to the board, specifically to the <code>/data/misc/camera/</code> directory. We are noe going to build our pipeline and use these files. We are using the following pipeline:

<syntaxhighlight lang=bash>
gst-launch-1.0 qtiqmmfsrc ! "video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,camera=0" ! queue ! qtimlesnpe model=/data/misc/camera/inception_v3.dlc labels=/data/misc/camera/imagenet_slim_labels.txt postprocessing="classification" runtime=1 ! queue ! qtioverlay ! waylandsink qos=false enable-last-sample=false sync=false fullscreen=true
</syntaxhighlight>
<br>

In the pipeline above, we use <code>qtiqmmfsrc</code> to capture frames from the camera in the Qualcomm Robotics RB5/RB6. We then set the caps of the capture, where we specify the format, resolution and framerate. Then we have our main element, the <code>qtimlesnpe</code>, where we specify the path of the model and the labels, also the type of postporcessing we are using; that in our case is a classification model, and finally where is the pipeline running. The <code>qtimlesnpe</code> plugin has 4 possible runtime options<ref name="SNPEruntime">Snapdragon NPE Runtime. Retrieved March 14, 2023, from [https://developer.qualcomm.com/sites/default/files/docs/snpe/snapdragon_npe_runtime.html]</ref>:
*0 - CPU: Runs the model on the CPU. Supports 32-bit floating point or 8-bit quantized execution.
*1 - DSP: Runs the model on Hexagon DSP using Q6 and Hexagon NN, executing on HVX; supports 8-bit quantized execution.
*2 - GPU: Runs the model on the GPU; supports hybrid or full 16-bit floating point modes.
*3 - AIP:  Runs the model on Hexagon DSP using Q6, Hexagon NN, and HTA; supports 8-bit quantized execution.

Later in the pipeline we use <code>qtioverlay</code> to overlay the metadata from the model in our video frames to finally display it in the monitor with the <code>waylandsink</code> element.

==Measurements==

For the pipeline in GStreamer pipeline section, we are measuring its CPU usage percentage and framerate, changing the possible runtimes. To measure the former, we are using the <code>top</code> command and then pressing <code>Shift</code>+<code>i</code> to get the average CPU usage percentage per core. For the latter, we use the [https://github.com/RidgeRun/gst-perf gst-perf] plugin from RidgeRun that computes the mean FPS, we get the average of 30 samples.

{| class="wikitable" style="margin: auto;"
|+ Table 1: Performance of pipeline with nnapi as delegate.
! style="width: 70px; background: #4479BA;" | {{fontcolor|#FFF|Runtime}}
! style="width: 70px; background: #4479BA;" | {{fontcolor|#FFF|CPU (%)}}
! style="width: 70px; background: #4479BA;" | {{fontcolor|#FFF|Framerate (fps)}}
|-
!CPU
|88.5
|5.284
|-
!DSP
|86.4
|5.301
|}

==References==
<references/>

<noinclude>
{{Qualcomm Robotics RB5/Foot|AI_hardware_acceleration/Neural_Processing_SDK/Install_qtimlesnpe|AI_hardware_acceleration/TensorFlow}}
</noinclude>