Getting started with AI on NXP i.MX8M Plus - Neural Processing Unit - Use Case experiments: Smart Parking - Introduction to the use case

Getting started with AI on NXP i.MX8M Plus
Introduction
Overview i.MX8M Plus hardware specifications Development tools Development kit DART-MX8M-PLUS Development kit Quick getting started Yocto minimal image Flashing SD card First steps on the board
Development
Gatesgarth and Hardknott Yocto versions Downloading Yocto images for NXP based BSP Building an imx-core-image Flashing SD card Integrating multimedia software stack Installing GStreamer Installing OpenCV Integrating Artificial Intelligence software stack Installing TensorFlow Lite for NXP based images Installing R2Inference Installing GstInference Developing software for the board Building the SDK Crosscompiling apps for GStreamer, TensorFlow Lite, and OpenCV Exploring TensorFlow Lite delegates for prototyping
Neural Processing Unit
Introduction Profiling the NPU Installing gputop Setting the board on debug mode Use Case experiments: Smart Parking Introduction to the use case License Plate Detection TinyYOLO version 3 Training a model compatible with GstInference License Plate Recognition Rosetta Getting the model prototyping Bash scripts for CPU usage and time estimation Serial experiments Parallel Experiments Results
GStreamer
Exploring Rosetta element for character recognition Example pipelines Smart Parking Video inferences Image inferences Video streaming
Contact Us

Smart Parking as the use case

In the fig.1 we can see the execution order for the Smart Parking application. Here we have two main stages: i) License Plate Detection, ii) License Plate Recognition. In the first stage, we detect the plate from a frame, and then it is cropped, this cropped license goes through the second stage, which is intended to extract the characters and have a final phrase in char/string format.

Figure 1 Smart Parking data flow.

Main idea

In fig.2 we can see the general architecture diagram for the Smart Parking application. This solution includes the GstInference and R2Inference RidgeRun plugins that support NPU delegate, TinyYOLO version 3 model pre-process and post-process, and Rosetta Facebook model with its pre-process and post-process. The module in light blue is related to the first stage of the application which is license plate detection, the result will be caught by the Rosetta element, which is related to character extraction. Those stages are going to be handled in a GStreamer pipeline.

Figure 2 GStreamer wrapper.

Base of the experiments

In the next sections, we are going to talk about experiments, but before going further, the GStreamer behavior understanding is required to understand the experiments.

In fig.3 we have a GStreamer pipeline, so when the TinyYOLO v3 inference is executed, GStreamer will look for resources in order to execute the Rosetta inference for the past frame.

Figure 3 GStreamer behavior.

Prototype

Here is the prototype Google Colaboratory, where the Smart Parking dataflow is developed using Python as the main prototyping programming language.