Difference between revisions of "Image Stitching for NVIDIA Jetson/User Guide/Controlling the Stitcher"
(→Case: 3 Images) |
|||
| Line 50: | Line 50: | ||
One panoramic use case is to compute the homographies for both left '''(0)''' and right '''(2)''' images, using the center image '''(1)''' as the reference. | One panoramic use case is to compute the homographies for both left '''(0)''' and right '''(2)''' images, using the center image '''(1)''' as the reference. | ||
| − | The JSON file would look like: | + | The JSON file would look like this: |
<pre> | <pre> | ||
Revision as of 08:48, 10 March 2021
| Image Stitching for NVIDIA®Jetson™ | |
|---|---|
 | |
| Image Stitching for NVIDIA Jetson Basics | |
|
|
|
| Getting Started | |
|
|
|
| User Guide | |
|
|
|
| Compiling OpenCV for Image Stitching | |
|
|
|
| Examples | |
|
|
|
| 360 Video | |
|
|
|
| Performance | |
|
|
|
| Contact Us |
|
This page serves as a guide to configure the stitcher in order to meet different application requirements.
Contents
Homography List
The homography list is just a JSON file that defines the transformations and the relationships between the images. Here we will explore (by examples) how to create this file in order to stitch the corresponding images.
Case: 2 Images
Let's assume we have only 2 images (with indices 0 an 1). These 2 images are related by a homography which can be computed using the Homography Estimation Guide. The computed homography transforms the Target image from the Reference image perspective.
This way, to fully describe a homography, we need to declare 3 parameters:
- Matrix: the 3x3 transformation matrix.
- Target: the index of the target image (i.e. the image to be transformed).
- Reference: the index of the reference image (i.e. the image used as a reference to transform the target image).
Having this information, we build the Homography JSON file:
{
"homographies":[
{
"images":{
"target":1,
"reference":0
},
"matrix":{
"h00": 1, "h01": 0, "h02": 510,
"h10": 0, "h11": 1, "h12": 0,
"h20": 0, "h21": 0, "h22": 1
}
}
]
}
With this file we are describing a set of 2 images (0 and 1), where the given matrix will transform the image 1 based on 0.
Case: 3 Images
Similar to the 2 images case, we use homographies to relate the set of images. The rule is to use N-1 number of homographies, where N is the number of images.
One panoramic use case is to compute the homographies for both left (0) and right (2) images, using the center image (1) as the reference. The JSON file would look like this:
{
"homographies":[
{
"images":{
"target":0,
"reference":1
},
"matrix":{
"h00": 1, "h01": 0, "h02": -510,
"h10": 0, "h11": 1, "h12": 0,
"h20": 0, "h21": 0, "h22": 1
}
},
{
"images":{
"target":2,
"reference":1
},
"matrix":{
"h00": 1, "h01": 0, "h02": 510,
"h10": 0, "h11": 1, "h12": 0,
"h20": 0, "h21": 0, "h22": 1
}
}
]
}
V0.6.0 and earlier
The stitcher uses the following parameters to determine its runtime behavior:
- Border Width: Border width for the blender.
- Left-Center Homography: Left-center homography from the stitching of three images.
- Right-Center Homography: Right-center homography from the stitching of two or three images.
- async-homography: Enable the async estimation of the homographies.
- async-homography-time: Time in milliseconds for the estimation of the homography.
- fov: Field of view of the cameras.
- overlap: Overlap between the images.
- stitching-quality-treshold: Stitching quality treshold, between 0 and 1, where 1 is the best score. This threshold does not allow new homographies bellow this value.
All of the properties should be configured at start-up. Currently, the stitcher is able to operate in either two or three image stitching modes. For the two image stitching mode, the left image is kept as a reference and the right one is transformed, for the three image cases, the center image is the reference.
In the following image the effect of these parameters can be seen:
The first step on the stitching is the image warping, this uses the Homography for the corresponding image to transform the input image in such a way as it matches the reference image. Depending on the camera setup this can introduce distortion on the input image, but note that the effect on the previous image is exaggerated.
If you want to learn how to calculate this visit the following page.
The second step is copying over the images over to the output buffer, this is handled by the homography and no parameter is involved in this process.
Finally, the stitch is blended, this is marked with the washed-out purple and orange in the previous image. This section is not copied over from the input images but is a mix of both of these and is controlled by the border-width parameter. This is especially useful when adjacent cameras don't have the same exposure.
Warning: If the homography between the two images is not optimal or the cameras have too much distortion a too large border width can introduce ghosting.
Example Pipeline
Check the following simple pipeline to see how each parameter is used:
LC_HOMOGRAPHY="{\
\"h00\": 7.3851e-01, \"h01\": 1.0431e-01, \"h02\": 1.4347e+03, \
\"h10\":-1.0795e-01, \"h11\": 9.8914e-01, \"h12\":-9.3916e+00, \
\"h20\":-2.3449e-04, \"h21\": 3.3206e-05, \"h22\": 1.0000e+00}"
RC_HOMOGRAPHY="{\
\"h00\": 7.3851e-01, \"h01\": 1.0431e-01, \"h02\": 1.4347e+03, \
\"h10\":-1.0795e-01, \"h11\": 9.8914e-01, \"h12\":-9.3916e+00, \
\"h20\":-2.3449e-04, \"h21\": 3.3206e-05, \"h22\": 1.0000e+00}"\
BORDER_WIDTH=10
gst-launch-1.0 -e cudastitcher name=stitcher \
left-center-homography="$LC_HOMOGRAPHY" \
right-center-homography="$RC_HOMOGRAPHY" \
border-width=$BORDER_WIDTH \
nvarguscamerasrc maxperf=true sensor-id=0 ! nvvidconv ! stitcher.sink_0 \
nvarguscamerasrc maxperf=true sensor-id=1 ! nvvidconv ! stitcher.sink_1 \
nvarguscamerasrc maxperf=true sensor-id=2 ! nvvidconv ! stitcher.sink_2 \
stitcher. ! perf print-arm-load=true ! queue ! nvvidconv ! nvoverlaysink
You can find more complex examples here.
