Full Body Pose Estimation for Sports Analysis - Getting Started

Full Body Pose Estimation for Sports Analysis on NVIDIA ®Jetson™
About
Camera Abstraction
Camera Calibration AprilTag Camera Calibration Chessboard Camera Calibration 3D Pose Estimation Pose Representation Skeleton Calibration Temporal Filter Kinematic Fitting Pose Optimization Pose Alignment Pose Correction Performance
Introduction Required Hardware How to Get the Code How to Install the Library How to Use the Library
Getting Started
Contact Us

Introduction

This wiki is intended to guide the user in the first steps of getting the code and installation of the full-body pose estimation library. Additionally, a demo of an application is attached to explain the main concepts of how to use the library.

Required Hardware

The project requires the hardware shown in the next image. The use of the Nvidia Jetson TX2 and the Connect Tech Astro Carrier Board allows the synchronization by the hardware of the Leopard Imaging LI-OV10635-SER camera modules, which is required by the system to perform multi-view pose estimation.

Error creating thumbnail: Unable to save thumbnail to destination

Required hardware.

You can position the cameras however you want. The more different the views captured, the more information the system will have to estimate the human pose. In the next image, you can appreciate the camera setup used by us for testing the system with limited space, shown using a 3D scene render. Please note, that even with the cameras facing the person in a similar way, they are far apart from one another and point from different angles.

Error creating thumbnail: Unable to save thumbnail to destination

Render showing a possible hardware setup.

How to Get the Code

To get the code you only need to clone the repository of the full-body pose estimation library as follows:

 $git clone https://gitlab.com/RidgeRun/tec/disptec-2020

How to Install the Library

In order to install the library, you need to first install the dependencies listed below and then follow the instructions in the installation section.

Dependencies

• OpenCV >= 4
• Python >= 3.7
• Real-time 3D Multi-person Pose Estimation Library

Installation

1. Go to the cloned repository directory:

 cd disptec-2020/

2. Install the Python requirements:

 pip3 install -r requirements.txt

3. Generate the library package:

  python3 setup.py sdist bdist_wheel

4. Install the rrpose library:

  pip3 install dist/rrpose-0.0.1.tar.gz

How to Use the Library

The library comprises several independent modules, which together confirm the full body pose estimation system. In our repository, we include a demo application that encompasses functionalities of every module, to perform pose estimation on input video data. However, if you would like to get deeper into each module functionality, we recommend you visit the specific module's page in this wiki.

Along with the demo application, in our repository you will find a data directory, which contains two sample videos for you to test the system with. Both videos correspond to the same sequence, and they were captured with cameras synchronized by hardware. We also include the corresponding camera calibration file, so you can easily test our demo.

To run the demo application, you just need to use the following command:

 python3 demo.py

Now, we included the sample data to make testing the demo as simple and fast as possible, however, you are free to customize the demo by changing its available parameters.

 python3 demo.py --param1 <value> --param2 <value> ...

--pose_estimation_model: Model to be used for initial pose estimation.
--kinematic_fitting_model: Model to be used for kinematic fitting.
--device: Optional. Specify the target device to infer on: CPU or GPU. The demo will look for a suitable plug-in for the device specified (it defaults to GPU).
--height-size: Optional. Network input layer height size.
--delay: Delay used to display result.
--joints_thresholds: Filter threshold values to define when it is considered that a joint has moved a lot (cm). Defaults to [5, 10, 0, 7, 3, 5, 20, 3, 5, 7, 3, 5, 20, 3, 5]).
--symetric_joints: List of identifiers that define the pairs of symmetric limbs'. Defaults to [[3, 9], [4, 10], [5, 11], [6, 12], [7, 13], [8, 14]]).
--calibration_file: Camera calibration file.
--skeleton_file: Skeleton file to use.
--color_file: Color file to use.
--camera_type: Camera type from our camera abstraction module. Defaults to opencvcam.
--cameras: List of the cameras device to use. Defaults to sample videos: ./data/cam0.mkv,./data/cam1.mkv.
--ip: IP address for Gst RTSP Dual Camera.
--port: Port number for Gst RTSP Dual Camera.
--mapping0: Camera 0 stream name for Gst RTSP Dual Camera.
--mapping1: Camera 1 stream name for Gst RTSP Dual Camera.