DeepStream Reference Designs - Customizing the Project - Implementing Custom Media
|
| DeepStream Reference Designs | |
|---|---|
 | |
| Getting Started | |
|
|
|
| Project Architecture | |
|
|
|
| Reference Designs | |
|
|
|
| Customizing the Project | |
|
|
|
| Contact Us | |
|
|
Contents
The Media Module
This element is responsible for encapsulating and managing the information data received through the system's cameras. The main functionality of this component is that it allows the Camera Capture subsystem to carry out a transparent and simple manipulation of the data received, through the operations provided by the Media interface. In addition, the flexibility of the system is increased since the Camera Capture module is not aware of the underlying technology used for processing camera data, so one media can be replaced by another without affecting overall functionality. For example, in the case of the APLVR reference design, the Gstd plugin is used to manage media instances. And within the media descriptor, the camera protocol is indicated, which corresponds to RTSP in said system. Other examples of components that could be used for this purpose were mentioned in the High-Level Design section.
As indicated in other modules of this system, the design of this component is not restricted to a specific implementation, so different technologies, plugins, or your own components can be added within the project structure, however, certain conditions must be respected prior to the incorporation of a custom component. After reading this wiki page, you will know what are the requirements to add your custom Media module to this reference design.
Class Diagram
The following figure shows a class diagram representation of the Media structure. To have a better understanding of the component, the diagram also shows which modules within the system are related to the Media instance.
Communication Between Modules
- Camera Capture: This module is in charge of transmitting the inference data that is obtained in real-time through the DeepStream pipeline. Once a new inference is obtained, a callback function is activated, to continue with the expected processing flow. For this, it is necessary to use the inference parser, since the information received depends on the context of the application used, where it can come with specific formats, parameters specific to each inference model, amount of data, etc. Regardless of the previously mentioned conditions, the custom Inference Parser will know how to interpret and transform said information into a format that is understandable by the rest of the system components. For the Inference Listener to be able to use the custom Parser, it must be previously registered using the method provided by the Inference Listener interface called register_inference_parser. Later in this wiki page, there will be a brief demonstration of how to make this connection in code.
- Media Factory: This class simply represents a Data Transfer Object (DTO), which will contain the information parsed by the custom Inference Parser. The idea is that the rest of the system modules can share this DTO to transmit the information obtained from the inference process so that everyone can interpret it regardless of the context of the application with which they are working. It is the responsibility of the Inference Parser to produce a new Inference Info after parsing the information received.
Media Interface Operations
As shown in the diagram, any custom Media module must implement the operations that are defined by the interface named "Media". As a user, you can add more elements to the design of the said component, for example, add new methods, modify the constructor of the class, and even carry out your implementations of each of the operations exposed by the interface. The important thing is that these methods are maintained. Next, there will be a brief explanation of the purpose of each of the operations defined by the interface. Remember that the specific implementation is up to your criteria or needs.
- start: As its name indicates, the purpose of this method is to initialize the data transmission process coming from the cameras. The idea is that Camera Capture can manage the right times to start receiving information from each Media.
- stop: Method that represents the counterpart of the start operation. As its name indicates, the idea is that the Camera Capture module can stop the flow of information received by the cameras, through this function. The fact that a Media stops its operation does not mean that it has been eliminated, it is simply in a paused state, which can be reactivated later.
- get_name: This is a simple method that allows you to get the name with which a certain instance of Media is created. Other modules or components of the system may require some type of management with the Media, so when obtaining the identifier they can carry out the respective processing tasks.
- get_triggers: According to the system design, each Media created will have one or more associated triggers, which abstract the business rules of the application and determine the moment necessary to activate a certain action in that specific Media. With this method provided by the interface, it is possible to obtain a list of the triggers associated with each Media.
- register_error_callback: Through this function, the Camera Capture module that manages the Media can register a callback function for error handling. The idea is that this callback is in charge of monitoring the flow of information coming from each Media, and being able to carry out actions when an error occurs in the camera system or in the process of obtaining the information. For example, in the APLVR application, the error callback is responsible for restarting the Media instance that reported an error and stopped. What is sought with this type of strategy is that the system can recover automatically in any eventuality, without the need for manual intervention by the user. Of course, the above is just an example of what an implementation of this type of callback could be and the user can implement other types of actions.
Supporting a New Camera
