H.264 Motion Vector Extractor - User Guide - Preparing the buffer

H.264 Motion Vector Extractor
H.264 Motion Vector Extractor Basics
Getting Started
Evaluating H.264 Motion Vector Extractor Getting the Code Building H.264 Motion Vector Extractor
User Guide
Preparing the buffer Retrieving the motion information Error codes
Examples
File motion vector extraction
Performance
x86 i.MX6 i.MX8
Contact Us

This page provides the basic steps for creating the input buffer used by the motion vector extractor. The input buffer can be either an H.264 byte stream or an AVC bitstream.

Parsing requirements

In order to parse the H.264 stream, this library requires all types of NALU to be processed. This means that the first NALUs to be processed must be the PPS and SPS. After that, the library supports an entire frame per buffer or partial frames, in which case a RuntimeError::StillProcessing will be returned.

The main methods of the class require the following parameters:

Buffer

Name: buffer

Type: std::shared_ptr<unsigned int>

A shared pointer to the start of the binary buffer that contains the H.264 stream.

Size

Name: size

Type: integer

Specifies the total size in bytes of the memory buffer pointed by Buffer.

Prefix size (AVC only)

Name: prefixSize

Type: integer

Specifies the size in bytes of the AVC NAL unit length prefix.

Frame

Name: frame

Type: Frame struct

Output structure containing the information extracted from the H.264 stream. You only need to pass an empty structure.

Creating the buffer

The shared pointer allows memory to be automatically released after it goes out of scope. If creating a buffer from scratch, the following portion of code avoids problems with the deleter of the shared pointer.

std::shared_ptr<unsigned char> buffer(new unsigned char[size](), [](unsigned char *p) { delete[] p; });

In case you are using the buffer from a GStreamer's appsink the following code will help you create it to take advantage of the shared pointer features and extract the size at the same time.

/** Function to easily wrap GstBuffer's data in shared_ptr of unsigned char and
    output its size.
    @param buffer GstBuffer's pointer to wrap.
    @param size output of the buffer's size.
    @return std::shared_ptr with the correct deleter set up.
    */
using GstBufferPtr = std::shared_ptr<unsigned char>;
GstBufferPtr gst_buffer_make(GstBuffer *buffer, int &size) {
  /* Map buffer information to retrieve data and size */
  GstMapInfo info;
  if (!gst_buffer_map(buffer, &info, GST_MAP_READ)) {
    return nullptr;
  }
  gst_buffer_ref(buffer);

  /* Output buffer's size */
  size = info.size;

  /* Create deleter for both the buffer and the mapped info */
  auto deleter = [buffer, info](unsigned char *) mutable {
    gst_buffer_unmap(buffer, &info);
    gst_buffer_unref(buffer);
  };
  return GstBufferPtr(info.data, deleter);
}

Extracting the motion vectors from the buffer

Before extracting the motion vectors you need to make sure to call the Init method of the VectorExtractor with the width and height of the stream.

RuntimeError Init(const unsigned int width, const unsigned int height);

After that just pass the previously explained parameters to the ExtractMotionVectors method according to the stream's format.

Bytestream

  RuntimeError ExtractMotionVectorsBytestream(
      const std::shared_ptr<unsigned char> buffer,
      const int size,
      Frame& frame);

AVC

  RuntimeError ExtractMotionVectorsAVC(
      const std::shared_ptr<unsigned char> buffer,
      const int size,
      const int prefixSize,
      Frame& frame);

Then follow the instructions here to retrieve the motion vectors.