GStreamer Daemon - ROS

← GStreamer Daemon - Digital PTZ

⌂ Home

→ API Examples

---

ROS (Robot operating system) has a big impact on the development of multimedia products, obtaining images, audio samples and data from hardware sensors all around the system. GStreamer and ROS can work together to deliver modular and optimized systems using images and audio to make decisions on actions to be performed by the robot.

GScam is a GStreamer-based ROS project with the interface to enable, configure and launch GStreamer pipelines, publishing data from any GStreamer source to a specific ROS topic.

Installing GScam

In order to use GScam, you first need to set up your ROS environment and ROS workspace. We will use official ROS documentation:

• Install ROS Melodic. You can use another distro if you'd like.

ROS Melodic Ubuntu Installation

• Set up your workspace:

ROS Environment and workspace setup

• Install GSCam:

cd ~/catkin_ws/
git clone https://github.com/RidgeRun/gscam.git src/gscam
catkin_make

Using GstD with GScam

RidgeRun has modified GScam adding an example to use the Shared Memory elements shmsrc/shmsink to pass buffers from GstD to the ROS node. These elements are needed because of the GstD limitation where the GStreamer buffers (and data in general) are available within the GstD process only, and can't be accessed by the GstD Client process or any other process. Shared memory elements allow us to test the integration between GstD and ROS by using GstD as the source pipeline, and ROS handling the sink pipeline for data publishing.

Let's run the ROS + GstD example:

• Make sure you have added the following line to your ~/.bashrc file, otherwise, run the command for each terminal opened.

source ~/catkin_ws/devel/setup.bash

• Start roscore. Open a new terminal and run:

roscore

• Open a new terminal and start GStreamer Daemon:

gstd

• Create the test sender pipeline:

gstd-client pipeline_create shm videotestsrc name=vtsrc is-live=true ! "video/x-raw, format=YUY2, color-matrix=sdtv, chroma-site=mpeg2, width=(int)320, height=(int)240, framerate=(fraction)30/1"  ! perf ! queue leaky=2 max-size-buffers=5 ! shmsink socket-path=/tmp/blah shm-size=2000000

• Start the test sender pipeline:

gstd-client pipeline_play shm

• Run GScam node, launching the GstD demo.

roslaunch gscam gstd-demo.launch

• Now the GScam node is publising the RAW data to the topic /gstd/camera/image_raw, you can see the available topics running:

rostopic list

• You can check the data being published through the topic in 2 ways, using console:

rostopic echo /gstd/camera/image_raw # this will print the pixel values in the console

• Or using the ROS node image_view:

rosrun image_view image_view image:=/gstd/camera/image_raw # this should open a window that displays the videotestsrc video

• We can control the pipeline using GstD. For example, changing the image pattern with the following command. You will see in the image_view window the change from the color bars to the ball pattern

gstd-client element_set shm vtsrc pattern 18

• Make sure to stop the Gstd pipeline when you are done.

gstd-client pipeline_stop shm

Benchmark

GStreamer Daemon working with ROS is a relatively new feature, it's important to determine the performance for this case in which one Linux process (GStreamer Daemon) handles the multimedia part and sends the data to a shared memory path and another Linux process (ROS) receives the data that GScam is publishing from that path. Also, it's important to define what's the impact of that shared memory handling.

For this purpose, benchmarking is the best option, several tests were applied on the RidgeRun GStreamer Daemon - ROS demo with modified GstD pipelines and GScam configurations, and the results are presented in this wiki.

Considerations

For this benchmark, the following metrics were considered:

• CPU usage
• Memory usage
• Framerate
• Glass to glass latency

CPU and memory usage were measured for all the processes involved with a modified version of a GNU GPL V3 licensed bash script that monitors a specific Linux process with the top command, it writes the data on a .csv file and generates a .png with the respective plot. It's important to mention that the top command is by default in Irix mode, in this mode, CPU usage is scaled to 100% being full use of a single core, therefore, it can go over 100% depending on the number of cores.

Framerate is divided into two parts: the framerate at the GStreamer pipeline, measured with GstShark, and the framerate at the ROS topic, measured with ROS command-line tools.

Finally, the glass to glass latency is measured by running a high-resolution bash timer, pointing the camera towards the PC screen where the timer is displayed and proceeding to display the captured video on a different monitor, then subtracting the difference between the values on both monitors.

Configuration

In order to test different resolutions and framerates, the GstD sender pipeline and the GScam configuration (located in the ROS launch file) must be modified. In this section, all the pipelines and GScam configurations used are presented.

Note: GRAY8 is used because GScam internally only accepts RGB and GRAY8 and the element nvvidconv is not compatible with RGB, videconvert could be used but it consumes too much cpu. For this to work, we need to add the following parameter in the ROS launch file:

<param name="image_encoding" value="mono8"/>

1280x720@30fps

GstD pipeline:

nvarguscamerasrc sensor_mode=0 ! "video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1" ! nvvidconv ! "video/x-raw, width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1" ! queue leaky=2 max-size-buffers=5 ! shmsink socket-path=/tmp/blah shm-size=20000000

GScam configuration:

shmsrc socket-path=/tmp/blah ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM),format=GRAY8 ! nvvidconv ! video/x-raw,format=GRAY8 ! identity

1920x1080@20fps

GstD pipeline:

nvarguscamerasrc sensor_mode=0 ! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)20/1" ! nvvidconv ! "video/x-raw, width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)20/1" ! queue leaky=2 max-size-buffers=5 ! shmsink socket-path=/tmp/blah shm-size=20000000

GScam configuration:

shmsrc socket-path=/tmp/blah ! video/x-raw,format=NV12,width=1920,height=1080,framerate=20/1 ! nvvidconv ! video/x-raw(memory:NVMM),format=GRAY8 ! nvvidconv ! video/x-raw,format=GRAY8 ! identity

3840x2160@15fps

GstD pipeline:

nvarguscamerasrc sensor_mode=0 ! "video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)15/1" ! nvvidconv ! "video/x-raw, width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)15/1" ! queue leaky=2 max-size-buffers=5 ! shmsink socket-path=/tmp/blah shm-size=20000000

GScam configuration:

shmsrc socket-path=/tmp/blah ! video/x-raw,format=NV12,width=3840,height=2160,framerate=15/1 ! nvvidconv ! video/x-raw(memory:NVMM),format=GRAY8 ! nvvidconv ! video/x-raw,format=GRAY8 ! identity

Results

CPU and Memory Usage

Process	CPU Usage (%)			Memory Usage (%)
	1280x720@30fps	1920x1080@20fps	3840x2160@15fps	1280x720@30fps	1920x1080@20fps	3840x2160@15fps
GstD	9.325	11.7	14.35	1.0	1.3	2.6
GScam	7.1375	13.75	18.75	1.3	1.5	1.8
Nvargus	24.9125	15.55	12.5875	6.8	6.2	5.1
ROS (roscore,rosmaster,rosout,roslaunch)	2.0	2.0	2.0	4.4	4.3	4.3
Note: This values are average for a test time of 60 seconds, ignoring idle states.

Framerate

Process	Framerate
	1280x720@30fps	1920x1080@20fps	3840x2160@15fps
GstD pipeline	30.0	20.0	15.0
ROS topic	29.912	20.0	14.991
Note: This values are average for a test time of 15 seconds.

Glass to glass latency

Process	Glass to glass latency (ms)
1280x720@30fps	121.83
1920x1080@20fps	215.66
3840x2160@15fps	-

Known Issues

At high resolutions/framerates the GScam node can easily publish the data through the ROS topic, but trying to visualize the data (with ROS image_view for example) could be a problem. When executing image_view, the CPU usage gets high too and then the system becomes unstable and most likely kills the process. This is the reason why latency is not measured for the 3840x2160@15fps test and why the framerate was set to 20fps instead of 30fps for the 1920x1080 resolution test.

---

← GStreamer Daemon - Digital PTZ

⌂ Home

→ API Examples