RidgeRun Toolbox Demo

From RidgeRun Developer Connection
(Redirected from RidgeRun GstPlugins Demo)
Jump to: navigation, search
Nvidia-preferred-partner-badge-rgb-for-screen.png

Error something wrong.jpg Problems running the pipelines shown on this page?
Please see our GStreamer Debugging guide for help.

RR Contact Us.png

Introduction

RidgeRun provides support for embedded Linux development, specializing in the use of hardware accelerators in multimedia applications. Also develops software products and development using GStreamer Technology. Several GStreamer plugins have been developed, this wiki shows some examples with four test scripts that can be used as a quick demo of their capabilities.

The demo scripts use Gstd which is a process that runs independently and exposes a public interface for other processes to communicate with and control the daemon. Using this daemon along with GstInterpipe which allows communication between two or more independent pipelines, allows us to demonstrate the use of some of our other gstreamer plugins such as GstCuda, which enables easy CUDA algorithm integration into GStreamer pipelines. Gst NvStabilize that performs video stabilization on a sequence of images accelerated with the GPU. GstMetaData which inserts text data in a Transport Stream video. GstRtspsink which permits high performance streaming to multiple computers using the RTSP / RTP protocols and GstInference that allows easy integration of deep learning networks into your existing pipeline.

Dependencies

This wiki assumes all the following plugins and dependencies are successfully installed:

Summary

The following image shows the data path for some of the most common use cases that will be presented in this demo:

Toolbox-pipelines.png

Visualization

To receive incoming buffers from each of the pipelines you can use programs like vlc, totem or GStreamer. For example to view the output buffers from H264/H265 pipelines you can use: 

vlc rtsp://${HOST_IP}:6005/stream0

To view the output buffers from VP9/VP8 pipelines you can use: 

totem rtsp://${HOST_IP}:6005/stream0

To receive and display Transport Stream data using GStreamer, you can use below pipelines: Receive and display: 

gst-launch-1.0 rtspsrc latency=0 location=rtsp://${HOST_IP}:5005/stream0 ! rtpmp2tdepay ! queue ! tsdemux ! h264parse ! avdec_h264 ! videoconvert !  queue ! ximagesink sync=false async=false

If Transport Stream data contains metadata, you can also show the metadata with following pipeline: 

gst-launch-1.0 rtspsrc latency=0 location=rtsp://${HOST_IP}:5005/stream0 ! rtpmp2tdepay ! queue ! tsdemux name=demux demux.! h264parse ! avdec_h264 ! videoconvert !  queue ! ximagesink sync=false async=false demux. ! queue ! 'meta/x-klv' ! metasink

Considerations

Each script includes some definitions needed to run, you can change them to match your needs. For example camera settings: 

CAM0=0
CAM0_WIDTH=1280
CAM0_HEIGHT=720
CAM0_FRAMERATE=120

Please pay special attention to definitions at:

  • test3.sh:

Update the PATH_TO_CUDA_FILTER definition to match the path where you downloaded the Gst-cuda eval. 

PATH_TO_CUDA_FILTER=/home/user/gst-cuda/tests/examples/cudafilter_algorithms/gray-scale-filter/gray-scale-filter.so
  • test4.sh:

Update the MODEL_LOCATION and LABELS definitions needed for the Gst-Inference plugin.

labels.txt: attached to this wiki

graph_tinyyolov2_tensorflow.pb: you can get the tensorflow model for free at the following link. 

MODEL_LOCATION='/home/user/graph_tinyyolov2_tensorflow.pb'
LABELS='/home/user/labels.txt'

Tests

Before running each test please ensure gstreamer daemon (gstd) is up and running: 

gstd -k
gstd

test1.sh

CAM0=0
CAM0_SRC=cam${CAM0}_src
CAM0_PIPE=${CAM0_SRC}_pipe
CAM0_WIDTH=1280
CAM0_HEIGHT=720
CAM0_FRAMERATE=120


########## GREEN PIPELINE ##########################
#1. Create camera input
echo -e "===> Create the h264 ${CAM0_PIPE}\n"
gstd-client pipeline_create ${CAM0_PIPE} nvarguscamerasrc sensor-id=${CAM0} ! "video/x-raw(memory:NVMM), width=(int)${CAM0_WIDTH}, \\
height=(int)${CAM0_HEIGHT}, format=(string)NV12, framerate=(fraction)${CAM0_FRAMERATE}/1" ! nvvidconv \
! interpipesink name=${CAM0_SRC} sync=false async=false

#2.1 Create h264 encoding
gstd-client pipeline_create x264_pipe interpipesrc name=x264_src listen-to=${CAM0_SRC} \
! omxh264enc ! "video/x-h264,stream-format=(string)byte-stream" \
! h264parse ! "video/x-h264,mapping=/stream0" ! \
interpipesink name=x264_sink sync=false async=false

#2.2 Create h265 encoding
gstd-client pipeline_create x265_pipe interpipesrc name=x265_src listen-to=${CAM0_SRC} \
! omxh265enc ! "video/x-h265,stream-format=(string)byte-stream" \
! h265parse ! "video/x-h265,mapping=/stream0" ! \
interpipesink name=x265_sink sync=false async=false

#3.1 Create rtspsink 1
echo -e "===> Create the output rtsp sinks \n"
gstd-client pipeline_create stream0_pipe interpipesrc name=rtsp0 listen-to=x264_sink \
! rtspsink service=6005

#3.2 Create rtspsink 2
gstd-client pipeline_create stream1_pipe interpipesrc name=rtsp1 listen-to=x264_sink \
! rtspsink service=6006

#3.3 Create rtspsink 2
echo -e "===> Create the output rtsp sinks \n"
gstd-client pipeline_create stream2_pipe interpipesrc name=rtsp0 listen-to=x265_sink \
! rtspsink service=6007

#3.4 Create rtspsink 4
gstd-client pipeline_create stream3_pipe interpipesrc name=rtsp1 listen-to=x265_sink \
! rtspsink service=6008


gstd-client pipeline_play ${CAM0_PIPE}
gstd-client pipeline_play x264_pipe
gstd-client pipeline_play x265_pipe
gstd-client pipeline_play stream0_pipe
gstd-client pipeline_play stream1_pipe
gstd-client pipeline_play stream2_pipe
gstd-client pipeline_play stream3_pipe


####################################################

test2.sh

CAM1=0
CAM1_SRC=cam${CAM1}_src
CAM1_PIPE=${CAM1_SRC}_pipe
CAM1_WIDTH=1280
CAM1_HEIGHT=720
CAM1_FRAMERATE=120


########## PURPLE PIPELINE ##########################
#1. Create camera input
echo -e "===> Create the ${CAM1_PIPE}\n"
gstd-client pipeline_create ${CAM1_PIPE} nvarguscamerasrc sensor-id=${CAM1} ! "video/x-raw(memory:NVMM), width=(int)${CAM1_WIDTH}, \\
height=(int)${CAM1_HEIGHT}, format=(string)NV12, framerate=(fraction)${CAM1_FRAMERATE}/1" \
! interpipesink name=${CAM1_SRC} sync=false async=false

#2.1 Create h264 encoding
gstd-client pipeline_create vp9_pipe interpipesrc name=vp9_src listen-to=${CAM1_SRC} \
! omxvp9enc ! "video/x-vp9, mapping=/stream0" ! queue \
! interpipesink name=vp9_sink sync=false async=false

#3.1 Create rtspsink 1
echo -e "===> Create the output rtsp sinks \n"
gstd-client pipeline_create stream0_pipe interpipesrc name=rtsp0 listen-to=vp9_sink \
! rtspsink service=6005

gstd-client pipeline_create stream1_pipe interpipesrc name=rtsp1 listen-to=vp9_sink \
! rtspsink service=6006

gstd-client pipeline_create stream2_pipe interpipesrc name=rtsp2 listen-to=vp9_sink \
! rtspsink service=6007

gstd-client pipeline_create stream3_pipe interpipesrc name=rtsp3 listen-to=vp9_sink \
! rtspsink service=6008


gstd-client pipeline_play ${CAM1_PIPE}
gstd-client pipeline_play vp9_pipe
gstd-client pipeline_play stream0_pipe
gstd-client pipeline_play stream1_pipe
gstd-client pipeline_play stream2_pipe
gstd-client pipeline_play stream3_pipe

####################################################

test3.sh

CAM0=0
CAM0_SRC=cam${CAM0}_src
CAM0_PIPE=${CAM0_SRC}_pipe
CAM0_WIDTH=1280
CAM0_HEIGHT=720
CAM0_FRAMERATE=120

PATH_TO_CUDA_FILTER=/home/user/gst-cuda/tests/examples/cudafilter_algorithms/gray-scale-filter/gray-scale-filter.so

########## RED PIPELINE ##########################
#1. Create camera input
echo -e "===> Create the ${CAM0_PIPE} pipeline\n"
gstd-client pipeline_create ${CAM0_PIPE} nvarguscamerasrc sensor-id=${CAM0} ! "video/x-raw(memory:NVMM), width=(int)${CAM0_WIDTH}, \\
height=(int)${CAM0_HEIGHT}, format=(string)NV12, framerate=(fraction)${CAM0_FRAMERATE}/1" \
! interpipesink name=${CAM0_SRC} sync=false async=false

#2.1 Use GstNvStabilize element
gstd-client pipeline_create stabilize_pipe interpipesrc name=stabilize_src listen-to=${CAM0_SRC} \
! nvvidconv ! nvstabilize crop-margin=0.1 queue-size=5 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvvidconv \
! interpipesink name=stabilize_sink sync=false async=false

#2.2 Use GstCuda element
gstd-client pipeline_create cuda_pipe interpipesrc name=cuda_src listen-to=${CAM0_SRC} \
! nvvidconv ! "video/x-raw(memory:NVMM),width=1280,height=720,format=I420,framerate=120/1" ! nvvidconv \
! cudafilter in-place=true location=${PATH_TO_CUDA_FILTER} \
! interpipesink name=cuda_sink sync=false async=false

#3.1 Create h264 encoding
gstd-client pipeline_create x264_pipe interpipesrc name=x264_src listen-to=cuda_sink \
! omxh264enc ! "video/x-h264,stream-format=(string)byte-stream" \
! h264parse ! "video/x-h264,mapping=/stream0" ! \
interpipesink name=x264_sink sync=false async=false

#3.2 Create h264 encoding
gstd-client pipeline_create x264_pipe2 interpipesrc name=x264_src2 listen-to=stabilize_sink \
! omxh264enc ! "video/x-h264,stream-format=(string)byte-stream" \
! h264parse ! "video/x-h264,mapping=/stream0" ! \
interpipesink name=x264_sink2 sync=false async=false


#4.1 Create rtspsink 1
echo -e "===> Create the output rtsp sinks \n"
gstd-client pipeline_create stream0_pipe interpipesrc name=rtsp0 listen-to=x264_sink \
! rtspsink service=6005

#4.2 Create rtspsink 2
gstd-client pipeline_create stream1_pipe interpipesrc name=rtsp1 listen-to=x264_sink2 \
! rtspsink service=6006

gstd-client pipeline_play ${CAM0_PIPE}
gstd-client pipeline_play stabilize_pipe
gstd-client pipeline_play cuda_pipe
gstd-client pipeline_play x264_pipe
gstd-client pipeline_play x264_pipe2
gstd-client pipeline_play stream0_pipe
gstd-client pipeline_play stream1_pipe


###################################################

test4.sh

CAM1='/dev/video1'
CAM1_WIDTH=1280
CAM1_HEIGHT=720
MODEL_LOCATION='/home/user/graph_tinyyolov2_tensorflow.pb'
INPUT_LAYER='input/Placeholder'
OUTPUT_LAYER='add_8'
LABELS='/home/user/labels.txt'

############# BLUE PIPELINE ####################
#1. Create camera input
echo -e "===> Create the ${CAM1} pipeline\n"
gstd-client pipeline_create cam1_pipe v4l2src device=$CAM1 \
! "video/x-raw, width=${CAM1_WIDTH}, height=${CAM1_HEIGHT}" \
! interpipesink name=cam1_src sync=false async=false

#2.1 Use Gst-inference plugin
gstd-client pipeline_create inference_pipe interpipesrc name=inference_src listen-to=cam1_src \
! videoconvert ! tee name=t t. ! videoscale ! queue ! net.sink_model t. ! queue \
! net.sink_bypass tinyyolov2 name=net model-location=$MODEL_LOCATION backend=tensorflow \
backend::input-layer=$INPUT_LAYER backend::output-layer=$OUTPUT_LAYER net.src_bypass \
! detectionoverlay labels="$(cat $LABELS)" font-scale=1 thickness=2 ! videoconvert \
! omxh264enc ! "video/x-h264,stream-format=(string)byte-stream" \
! h264parse ! "video/x-h264,mapping=/stream0" \
! rtspsink service=6007

#2.2 Encode h264 video
gstd-client pipeline_create cam1_empty interpipesrc name=cam_src_empty listen-to=cam1_src \
! videoconvert ! omxh264enc ! "video/x-h264,stream-format=(string)byte-stream" \
! h264parse ! "video/x-h264,mapping=/stream0" \
! rtspsink service=6008

gstd-client pipeline_play cam1_pipe
gstd-client pipeline_play cam1_empty
gstd-client pipeline_play inference_pipe

###############################################

test5.sh

CAM0=0
CAM0_SRC=cam${CAM0}_src
CAM0_PIPE=${CAM0_SRC}_pipe
CAM0_WIDTH=640
CAM0_HEIGHT=480
CAM0_FRAMERATE=30

########## YELLOW PIPELINE ##########################
#Data to send
METADATA=$(date +"%T")

#1. Create trap to handle CTRL-C for clean exit 
trap "gstd-client pipeline_stop meta_pipe; gstd-client pipeline_stop no_meta_pipe; gstd-client pipeline_stop ${CAM0_PIPE}; gstd-client pipeline_delete meta_pipe; gstd-client pipeline_delete no_meta_pipe; gstd-client pipeline_delete ${CAM0_PIPE}; exit" SIGINT

#2. Create video source pipeline
echo -e "===> Create the video source \n"
gstd-client pipeline_create ${CAM0_PIPE} v4l2src device=/dev/video${CAM0} ! videoconvert ! "video/x-raw,format=(string)I420,width=${CAM0_WIDTH},height=${CAM0_HEIGHT},framerate=(fraction)${CAM0_FRAMERATE}/1" ! x264enc ! "video/x-h264,stream-format=(string)byte-stream" ! h264parse ! queue ! interpipesink name=${CAM0_SRC} sync=false async=false

#3. Create pipeline that inserts metadata
echo -e "===> Create the medatada interpipe \n"
gstd-client pipeline_create meta_pipe interpipesrc listen-to=${CAM0_SRC} ! queue ! mpegtsmux name=muxmeta ! tsparse ! "video/mpegts,mapping=/stream0" ! queue ! rtspsink service=6005 metasrc metadata=$METADATA name=meta_src ! 'meta/x-klv' ! muxmeta.

#4. Create pipeline that doesn't insert metadata
echo -e "===> Create the interpipe without metadata \n"
gstd-client pipeline_create no_meta_pipe interpipesrc listen-to=${CAM0_SRC} ! queue ! mpegtsmux name=mux ! tsparse ! "video/mpegts,mapping=/stream0" ! queue ! rtspsink service=6006

#5. Run pipelines
# Set pipelines to PLAYING STATE
echo -e "\n ====> Start cam_src_pipe and live_preview_pipe \n"
gstd-client pipeline_play ${CAM0_PIPE}
gstd-client pipeline_play meta_pipe
gstd-client pipeline_play no_meta_pipe

echo -e "====> Type Ctrl+C to stop the example execution, otherwise it will iterate infinitely!\n"
#6. Enable loop to keep updating metadata once per second
while :
do

  sleep 1
  METADATA=$(date +"%T")
  echo -e "Current value on METADATA: ${METADATA} \n"
  gstd-client element_set meta_pipe meta_src metadata ${METADATA}

done

####################################################

labels.txt

The file is needed by the gst-inference demo:

labels.txt 

aeroplane;bicycle;bird;boat;bottle;bus;car;cat;chair;cow;diningtable;dog;horse;motorbike;person;pottedplant;sheep;sofa;train;tvmonitor


RidgeRun Resources

Quick Start Client Engagement Process RidgeRun Blog Homepage
Technical and Sales Support RidgeRun Online Store RidgeRun Videos Contact Us

OOjs UI icon message-progressive.svg Contact Us

Visit our Main Website for the RidgeRun Products and Online Store. RidgeRun Engineering informations are available in RidgeRun Professional Services, RidgeRun Subscription Model and Client Engagement Process wiki pages. Please email to support@ridgerun.com for technical questions and contactus@ridgerun.com for other queries. Contact details for sponsoring the RidgeRun GStreamer projects are available in Sponsor Projects page. Ridgerun-logo.svg
RR Contact Us.png
Retrieved from "http://developer.ridgerun.com/wiki/index.php?title=RidgeRun_Toolbox_Demo&oldid=42795"