Difference between revisions of "GstInference/Supported backends/NCSDK"
| Line 8: | Line 8: | ||
The NCSDK Intel® Movidius™ Neural Compute SDK (Intel® Movidius™ NCSDK) enables deployment of deep neural networks on compatible devices such as the Intel® Movidius™ Neural Compute Stick. The NCSDK includes a set of software tools to compile, profile, and validate DNNs (Deep Neural Networks) as well as APIs on C/C++ and Python for application development. | The NCSDK Intel® Movidius™ Neural Compute SDK (Intel® Movidius™ NCSDK) enables deployment of deep neural networks on compatible devices such as the Intel® Movidius™ Neural Compute Stick. The NCSDK includes a set of software tools to compile, profile, and validate DNNs (Deep Neural Networks) as well as APIs on C/C++ and Python for application development. | ||
| − | + | To use the ncsdk on Gst-Inference be sure to run the R2Inference configure with the flag <code> --enable-ncsdk </code> and use the property <code> backend=ncsdk </code> on the Gst-Inference plugins. | |
| − | |||
| − | |||
| − | |||
=Installation= | =Installation= | ||
| Line 19: | Line 16: | ||
We also provide an installation guide with troubleshooting on the [[Intel_Movidius_NCSDK_Installation | Intel Movidius Installation wiki page]] | We also provide an installation guide with troubleshooting on the [[Intel_Movidius_NCSDK_Installation | Intel Movidius Installation wiki page]] | ||
| − | = | + | =Generating a graph= |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | When you use the ncsdk backeng you will need a compiled ncs graph file. You can obtain this file from tensorflow's protobuff and weights filer; or caffe's prototxt and caffemodel files. mvNCCompile is a tool included with the ncsdk installation that | |
| − | + | compiles a network and produces a graph file that is compatible with the NCAPI and the Gst-Inference plugins using the ncsdk backend. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
For example, giving a caffe model (bvlc_googlenet.caffemodel) and a network description (deploy.prototxt): | For example, giving a caffe model (bvlc_googlenet.caffemodel) and a network description (deploy.prototxt): | ||
| Line 76: | Line 25: | ||
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt | mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| − | This command will output the '''graph''' and '''output_expected.npy''' files, that | + | This command will output the '''graph''' and '''output_expected.npy''' files, that can be used later with the googlenet plugin. |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
=Options= | =Options= | ||
| Line 210: | Line 48: | ||
The <code>backend::log-level=1</code> section of the pipeline sets the <code>NC_RW_LOG_LEVEL</code> option of the NCSDK C API to <code>1</code>. | The <code>backend::log-level=1</code> section of the pipeline sets the <code>NC_RW_LOG_LEVEL</code> option of the NCSDK C API to <code>1</code>. | ||
| + | |||
| + | To learn more about the NCSDK C API option, please check the [[R2Inference/Supported_backends/NCSDK| NCSDK wiki page]] on the R2Inference subwiki. | ||
==Device Options== | ==Device Options== | ||
| Line 355: | Line 195: | ||
|} | |} | ||
| + | = NCSDK Tools= | ||
| + | |||
| + | The NCSDK installation include some useful tools to analyze, optimize and compile models. We will mention these tools here, but if you want some examples and a more complete description please check the [[R2Inference/Supported_backends/NCSDK| NCSDK wiki page]] on the R2Inference subwiki. | ||
| + | * '''mvNCCheck''': Checks the validity of a Caffe or TensorFlow model on a neural compute device. The check is done by running an inference on both the device and in software and then comparing the results to determine a if the network passes or fails. | ||
| + | * '''mvNCCompile''': Compiles a network and weights files from Caffe or TensorFlow models into a graph file that is compatible with the NCAPI. | ||
| + | * '''mvNCProfile''': Compiles a network, runs it on a connected neural compute device, and outputs profiling info on the terminal and on an HTML file. The profiling data contains layer performance and execution time of the model. The html version of the report also contains a graphical representation of the model. | ||
<noinclude> | <noinclude> | ||
{{GstInference/Foot|Supported backends|Example pipelines}} | {{GstInference/Foot|Supported backends|Example pipelines}} | ||
</noinclude> | </noinclude> | ||
Revision as of 12:52, 26 December 2018
 | Make sure you also check GstInference's companion project: R2Inference |
The NCSDK Intel® Movidius™ Neural Compute SDK (Intel® Movidius™ NCSDK) enables deployment of deep neural networks on compatible devices such as the Intel® Movidius™ Neural Compute Stick. The NCSDK includes a set of software tools to compile, profile, and validate DNNs (Deep Neural Networks) as well as APIs on C/C++ and Python for application development.
To use the ncsdk on Gst-Inference be sure to run the R2Inference configure with the flag --enable-ncsdk and use the property backend=ncsdk on the Gst-Inference plugins.
Contents
Installation
You can install the NCSDK on a system running Linux directly, downloading a Docker container, on a virtual machine or using a Python virtual environment. All the possible installation paths are documented on the official installation guide.
We also provide an installation guide with troubleshooting on the Intel Movidius Installation wiki page
Generating a graph
When you use the ncsdk backeng you will need a compiled ncs graph file. You can obtain this file from tensorflow's protobuff and weights filer; or caffe's prototxt and caffemodel files. mvNCCompile is a tool included with the ncsdk installation that compiles a network and produces a graph file that is compatible with the NCAPI and the Gst-Inference plugins using the ncsdk backend.
For example, giving a caffe model (bvlc_googlenet.caffemodel) and a network description (deploy.prototxt):
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
This command will output the graph and output_expected.npy files, that can be used later with the googlenet plugin.
Options
You can find the full documentation of the C API here and the Python API here. Gst-Inference uses only the C API and R2Inference takes care of devices, graphs, models and fifos. Because of this, we will only take a look at the options that you can change when using the C API through R2Inference.
The following syntax is used to change backend options on Gst-Inference plugins:
backend::<property>
For example to change the NCSDK API log level of the googlenet plugin you need to run the pipeline like this:
gst-launch-1.0 \
googlenet name=net model-location=/root/r2inference/examples/r2i/ncsdk/graph_googlenet backend=ncsdk backend::log-level=1 \
videotestsrc ! tee name=t \
t. ! queue ! videoconvert ! videoscale ! net.sink_model \
t. ! queue ! net.sink_bypass \
net.src_bypass ! fakesink
The backend::log-level=1 section of the pipeline sets the NC_RW_LOG_LEVEL option of the NCSDK C API to 1.
To learn more about the NCSDK C API option, please check the NCSDK wiki page on the R2Inference subwiki.
Device Options
All the device options are read only.
| Property | C API Counterpart | Value | Description |
|---|---|---|---|
| thermal-throttling-level | NC_RO_THERMAL_THROTTLING_LEVEL | Integer (0,1,2) |
|
| device-state | NC_RO_DEVICE_STATE | Integer (0,1,2,3) | The current state of the device:
|
| current-memory-used | NC_RO_DEVICE_CURRENT_MEMORY_USED | Integer | Current memory used on the device. |
| memory-size | NC_RO_DEVICE_MEMORY_SIZE | Integer | Total memory available on the device. |
| max-fifo-num | NC_RO_DEVICE_MAX_FIFO_NUM | Integer | Max number of fifos. |
| allocated-fifo-num | NC_RO_DEVICE_ALLOCATED_FIFO_NUM | Integer | Number of fifos currently allocated. |
| max-graph-num | NC_RO_DEVICE_MAX_GRAPH_NUM | Integer | Max number of graphs. |
| allocated-graph-num | NC_RO_ALLOCATED_GRAPH_NUM | Integer | Number of graphs currently allocated. |
| option-class-limit | NC_RO_DEVICE_OPTION_CLASS_LIMIT | Integer | Highest option class supported. |
| device-name | NC_RO_DEVICE_NAME | String | Device name. |
Fifo Options
Most of the R/W options on the FIFO can only be modified between creation and allocation, and R2Inference does both in a single method (Engine->Start()), so it is impossible to write on these options. R2Inference also fixates those options to our specific implementation, so they are not exposed on the plugin.
Global Options
Pay special attention to the log level enumeration, because it is ordered counter intuitively. 1 is actually the highest log level, 4 is the lowest and 0 the default.
| Property | C API Counterpart | Value | Description |
|---|---|---|---|
| log-level | NC_RW_LOG_LEVEL | Integer | NCSDK debug log level from ncLogLevel_t enum
|
Graph Options
| Property | C API Counterpart | Value | Description |
|---|---|---|---|
| graph-state | NC_RO_GRAPH_STATE | Integer | The current state of the graph from ncGraphState_t enum
|
| graph-input-count | NC_RO_GRAPH_INPUT_TENSOR_DESCRIPTORS | Integer | Array of graph inputs. Returns the size of the array instead of the array itself. |
| graph-output-count | NC_RO_GRAPH_OUTPUT_TENSOR_DESCRIPTORS | Integer | Array of graph outputs. Returns the size of the array instead of the array itself. |
| graph-debug-info | NC_RO_GRAPH_DEBUG_INFO | String | Debug information. |
| graph-name | NC_RO_GRAPH_NAME | String | Graph name. |
| graph-option-class-limit | NC_RO_GRAPH_OPTION_CLASS_LIMIT | Integer | The highest option class supported. |
| graph-version | NC_RO_GRAPH_VERSION | String | The version ([major, minor]) of the compiled graph. |
NCSDK Tools
The NCSDK installation include some useful tools to analyze, optimize and compile models. We will mention these tools here, but if you want some examples and a more complete description please check the NCSDK wiki page on the R2Inference subwiki.
- mvNCCheck: Checks the validity of a Caffe or TensorFlow model on a neural compute device. The check is done by running an inference on both the device and in software and then comparing the results to determine a if the network passes or fails.
- mvNCCompile: Compiles a network and weights files from Caffe or TensorFlow models into a graph file that is compatible with the NCAPI.
- mvNCProfile: Compiles a network, runs it on a connected neural compute device, and outputs profiling info on the terminal and on an HTML file. The profiling data contains layer performance and execution time of the model. The html version of the report also contains a graphical representation of the model.
