Difference between revisions of "How to Use A/B Filesystem Redundancy and OTA with NVIDIA Jetpack"

From RidgeRun Developer Connection
Jump to: navigation, search
(How to Perform an OTA Update)
Line 1: Line 1:
 
<seo title="A/B Filesystem Redundancy and OTA with NVIDIA Jetpack | RidgeRun Developer" titlemode="replace" keywords="GStreamer, Linux SDK, Linux BSP,  Embedded Linux, Device Drivers, NVIDIA, NVIDIA Jetson, Xilinx, TI, NXP, Freescale, Embedded Linux driver development, Linux Software development, Embedded Linux SDK, Embedded Linux Application development, OTA, Over The Air updates, A/B Filesystem Redundancy, Filesystem Redundancy, NVIDIA Jetpack, Jetpack 4.6, Jetson TX2, Xavier AGX, Xavier NX, Jetson Nano, OTA Update, Jetpack, A/B Root File System Support, Root File System Support, NVIDIA Jetpack 4.6."  description="This wiki shows how to use the new root file system redundancy, as well as the image-based OTA update features, available in Jetpack 4.6."></seo>
 
<seo title="A/B Filesystem Redundancy and OTA with NVIDIA Jetpack | RidgeRun Developer" titlemode="replace" keywords="GStreamer, Linux SDK, Linux BSP,  Embedded Linux, Device Drivers, NVIDIA, NVIDIA Jetson, Xilinx, TI, NXP, Freescale, Embedded Linux driver development, Linux Software development, Embedded Linux SDK, Embedded Linux Application development, OTA, Over The Air updates, A/B Filesystem Redundancy, Filesystem Redundancy, NVIDIA Jetpack, Jetpack 4.6, Jetson TX2, Xavier AGX, Xavier NX, Jetson Nano, OTA Update, Jetpack, A/B Root File System Support, Root File System Support, NVIDIA Jetpack 4.6."  description="This wiki shows how to use the new root file system redundancy, as well as the image-based OTA update features, available in Jetpack 4.6."></seo>
  
==Introduction==
+
=Introduction=
  
 
This wiki shows how to use the new root file system redundancy, as well as the image-based OTA update features, available since Jetpack 4.6.
 
This wiki shows how to use the new root file system redundancy, as well as the image-based OTA update features, available since Jetpack 4.6.
Line 37: Line 37:
 
|}
 
|}
  
==Image based OTA==
+
=Image based OTA Update=
  
 
It is the process to upgrade the whole system by updating partition by partition.
 
It is the process to upgrade the whole system by updating partition by partition.
Line 98: Line 98:
 
'''Note''': There needs to be free space available in EMMC (~100MB for TX2 and ~300MB for Jetson AGX Xavier) to build an intermediate boot path, in case there is a reboot during OTA so that the device can fallback to recovery kernel from this path.
 
'''Note''': There needs to be free space available in EMMC (~100MB for TX2 and ~300MB for Jetson AGX Xavier) to build an intermediate boot path, in case there is a reboot during OTA so that the device can fallback to recovery kernel from this path.
  
===How to Perform an OTA Update===
+
==How to Perform an OTA Update==
  
 
These instructions were extracted from [https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html#wwpID0E0UI0HA NVIDIA official documentation].
 
These instructions were extracted from [https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html#wwpID0E0UI0HA NVIDIA official documentation].
Line 104: Line 104:
 
'''Note''': You cannot use OTA update to update a Jetson device on which Jetpack is installed.
 
'''Note''': You cannot use OTA update to update a Jetson device on which Jetpack is installed.
  
====Preparing for an Image-Based OTA Update====
+
===Steps on the host machine===
 
 
=====Steps on the host machine=====
 
  
 
1. Flash the target device with any of the BSP listed above in order to perform the update to the R32.6.1 BSP.
 
1. Flash the target device with any of the BSP listed above in order to perform the update to the R32.6.1 BSP.
Line 128: Line 126:
 
4. Unpack and apply the new release sample root filesystem.
 
4. Unpack and apply the new release sample root filesystem.
  
* For simplicity, lets define a variable for the NVIDIA downloads directory. This will change if you changed the default downloads directory.
+
For simplicity, lets define a variable for the NVIDIA downloads directory. This will change if you changed the default downloads directory.
 
<source lang=bash>
 
<source lang=bash>
 
NVIDIA_DOWNLOADS_DIR=/home/${USER}/Downloads/nvidia/sdkm_downloads/
 
NVIDIA_DOWNLOADS_DIR=/home/${USER}/Downloads/nvidia/sdkm_downloads/
Line 149: Line 147:
 
</source>
 
</source>
  
5. Assign the ToT_BSP variable to the ''Linux_for_Tegra'' folder of the new release (32.6.1).
+
5. Assign the ToT_BSP (Top of Tree, latest release) and TARGET_BSP variables to the ''Linux_for_Tegra'' folder of the new release (32.6.1).
  
 
<source lang=bash>
 
<source lang=bash>
 
#After last commands you should be on this ''Linux_for_Tegra'' directory
 
#After last commands you should be on this ''Linux_for_Tegra'' directory
 
ToT_BSP=$PWD
 
ToT_BSP=$PWD
 +
TARGET_BSP=$PWD
 
</source>
 
</source>
  
Line 160: Line 159:
 
<source lang=bash>
 
<source lang=bash>
 
cd ..
 
cd ..
tar xpf ota_tools_r32.6.1_aarch64.tbz2  
+
tar <OTA_TOOLS_DOWNLOAD_PATH>/xpf ota_tools_r32.6.1_aarch64.tbz2  
 +
</source>
 +
 
 +
7. '''Optional''': If you want to update to other release different than ToT, such as 32.4 or 32.5 follow the next instructions, otherwise go to step 8.
 +
 
 +
'''Note''': Updating to 32.5 is not supported on the Jetson Xavier NX P3668-0000.
 +
 
 +
Create a directory that contains the BSP package that you want to update to, for example for 32.5:
 +
 
 +
<source lang=bash>
 +
mkdir L4T_32.5/
 +
cd L4T_32.5/
 +
tar xpf $NVIDIA_DOWNLOADS_DIR/Jetson_Linux_R32.5.0_aarch64.tbz2
 +
cd Linux_for_Tegra/rootfs/
 +
sudo tar xpf $NVIDIA_DOWNLOADS_DIR/Tegra_Linux_Sample-Root-Filesystem_R32.5.0_aarch64.tbz2
 +
cd ..
 +
sudo ./apply_binaries.sh
 +
# Redefine TARGET_BSP variable
 +
TARGET_BSP=$PWD
 
</source>
 
</source>
  
7. Generate the base recovery image and recovery DTB:
+
Install in the TARGET_BSP the required OTA tools files:
 +
 
 +
<source lang=bash>
 +
cd ..
 +
tar xpf <OTA_TOOLS_DOWNLOAD_PATH>/ota_tools_r32.6.1_aarch64.tbz2
 +
cd $TARGET_BSP
 +
cp ${ToT_BSP}/tools/ota_tools/version_upgrade/* ./tools/ota_tools/version_upgrade
 +
</source>
 +
 
 +
Generate slot metadata files for TARGET_BSP:
 +
 
 +
<source lang=bash>
 +
cd ${TARGET_BSP}
 +
cp ./bootloader/smd_info.cfg ./tools/ota_tools/version_upgrade/smd_info.boot_slot_A.cfg
 +
cp ./bootloader/smd_info.cfg ./tools/ota_tools/version_upgrade/smd_info.boot_slot_B.cfg
 +
sed -i -e 's/^15/# 15/' -e 's/^##15/15/' -e 's/^##14/14/' ./tools/ota_tools/version_upgrade/smd_info.boot_slot_A.cfg
 +
sed -i -e 's/^15/# 15/' -e 's/^##15/14/' -e 's/^##14/15/' ./tools/ota_tools/version_upgrade/smd_info.boot_slot_B.cfg
 +
./bootloader/nv_smd_generator ./tools/ota_tools/version_upgrade/smd_info.boot_slot_A.cfg ./tools/ota_tools/version_upgrade/slot_metadata.bin.A
 +
./bootloader/nv_smd_generator ./tools/ota_tools/version_upgrade/smd_info.boot_slot_B.cfg ./tools/ota_tools/version_upgrade/slot_metadata.bin.B
 +
</source>
 +
 
 +
Copy the files from ${ToT_BSP} and modify p2771-0000.conf.common, p2972-0000.conf.common, and p3668.conf.common:
 +
 
 +
<source lang=bash>
 +
cp ${ToT_BSP}/flash.sh ./
 +
cp ${ToT_BSP}/l4t_sign_image.sh ./
 +
cp ${ToT_BSP}/l4t_generate_soc_bup.sh ./
 +
cp ${ToT_BSP}/bootloader/tegraparser_v2 ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegraflash.py ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegrahost_v2 ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegrabct_v2 ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegraflash_internal.py ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegrasign_v3_util.py ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegrasign_v3_internal.py ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegrasign_v3.py ./bootloader/
 +
cp ${ToT_BSP}/bootloader/tegraopenssl ./bootloader/
 +
sudo cp ${ToT_BSP}/rootfs/opt/nvidia/l4t-bootloader-config/nv-l4t-bootloader-config.sh ./rootfs/opt/nvidia/l4t-bootloader-config/nv-l4t-bootloader-config.sh
 +
sudo cp ${ToT_BSP}/rootfs/usr/sbin/nvbootctrl ./rootfs/usr/sbin/nvbootctrl
 +
sudo cp ${ToT_BSP}/rootfs/usr/sbin/nv_bootloader_payload_updater ./rootfs/usr/sbin/nv_bootloader_payload_updater
 +
sudo cp ${ToT_BSP}/rootfs/usr/sbin/nv_update_engine ./rootfs/usr/sbin/nv_update_engine
 +
echo "SMDFILE=\"slot_metadata.bin\";" >>p2771-0000.conf.common
 +
echo "SMDFILE=\"slot_metadata.bin\";" >>p2972-0000.conf.common
 +
echo "SMDFILE=\"slot_metadata.bin\";" >>p3668.conf.common
 +
</source>
 +
 
 +
8. Generate the base recovery image and recovery DTB:
  
 
First, make sure you have the device tree compiler by running:
 
First, make sure you have the device tree compiler by running:
Line 198: Line 260:
 
</source>
 
</source>
  
After running this command you should have a recovery image and a recovery DTB under ''${ToT_BSP}/bootloader''. These are typically named ''recovery.dtb.<bsp_version>'' and ''recovery.img.<bsp_version>''.
+
After running this command you should have a recovery image and a recovery DTB under ''${TARGET_BSP}/bootloader''. These are typically named ''recovery.dtb.<bsp_version>'' and ''recovery.img.<bsp_version>''.
  
8. Generate the OTA update payload package:
+
9. Generate the OTA update payload package:
  
 
<source lang=bash>
 
<source lang=bash>
Line 229: Line 291:
 
This script generates the file '''ota_payload_package.tar.gz''' under '''${TARGET_BSP}/bootloader/<target_board>'''. This package would normally be uploaded to an OTA server in order to implement a complete OTA solution. The ota_payload_package.tar.gz archive that the script generates from the default rootfs occupies about 2 GB. A self-generated rootfs may yield a larger or smaller archive. Decompressing the archive on the Jetson device to apply the image-based OTA requires at least 6 GB of free space.
 
This script generates the file '''ota_payload_package.tar.gz''' under '''${TARGET_BSP}/bootloader/<target_board>'''. This package would normally be uploaded to an OTA server in order to implement a complete OTA solution. The ota_payload_package.tar.gz archive that the script generates from the default rootfs occupies about 2 GB. A self-generated rootfs may yield a larger or smaller archive. Decompressing the archive on the Jetson device to apply the image-based OTA requires at least 6 GB of free space.
  
'''Note''': Steps 7 and 8 use the L4T sample rootfs from '''${BASE_BSP}/rootfs/''' to build the recovery kernel, recovery DTB and OTA package. This option lets you overwrite the default rootfs image with a self-generated rootfs image, for example, an image cloned from a Jetson device that has been fully customized with a top-of-tree release of Jetson Linux. If you have customized the sample rootfs, for example, you must add required libraries and binaries into rootfs, and apply them before running these scripts.  
+
'''Note''': Steps 8 and 9 use the L4T sample rootfs from '''${BASE_BSP}/rootfs/''' to build the recovery kernel, recovery DTB and OTA package. This option lets you overwrite the default rootfs image with a self-generated rootfs image, for example, an image cloned from a Jetson device that has been fully customized with a top-of-tree release of Jetson Linux. If you have customized the sample rootfs, for example, you must add required libraries and binaries into rootfs, and apply them before running these scripts.  
  
=====Steps on the target board=====
+
===Steps on the target board===
  
9. Check ''/boot/extlinux/extlinux.conf'' to see if INITRD variable is set, otherwise add:
+
10. Check ''/boot/extlinux/extlinux.conf'' to see if INITRD variable is set, otherwise add:
  
 
<source lang=bash>
 
<source lang=bash>
Line 245: Line 307:
 
</source>
 
</source>
  
10. Create a working directory for the OTA update process and assign it to the ''WORKDIR'' variable:
+
11. Create a working directory for the OTA update process and assign it to the ''WORKDIR'' variable:
  
 
<source lang=bash>
 
<source lang=bash>
Line 253: Line 315:
 
</source>
 
</source>
  
11. Download the OTA update tool package ''ota_tools_<release>_aarch64.tbz2'' from step 6 and unpack it inside ''$WORKDIR''.
+
12. Download the OTA update tool package ''ota_tools_<release>_aarch64.tbz2'' from step 6 and unpack it inside ''$WORKDIR''.
  
 
An example with SSH would be the following:
 
An example with SSH would be the following:
Line 261: Line 323:
 
</source>
 
</source>
  
12. Download the '''ota_payload_package.tar.gz''' from step 8 and put in the directory (''/ota'').
+
13. Download the '''ota_payload_package.tar.gz''' from step 8 and put in the directory (''/ota'').
  
 
An example with SSH would be the following:
 
An example with SSH would be the following:
Line 269: Line 331:
 
</source>
 
</source>
  
13. Prepare to start OTA:
+
14. Prepare to start OTA:
  
 
<source lang=bash>
 
<source lang=bash>
Line 278: Line 340:
 
where ''/dev/mmcblk0'' is the device on which the partitions for the backup boot path are placed and ''/ota/ota_payload_package.tar.gz'' is the OTA payload package. The ''nv_ota_start'' tool will handle automatically all use cases whether rootfs A/B is enabled or not and whether there are partition layout changes or not.
 
where ''/dev/mmcblk0'' is the device on which the partitions for the backup boot path are placed and ''/ota/ota_payload_package.tar.gz'' is the OTA payload package. The ''nv_ota_start'' tool will handle automatically all use cases whether rootfs A/B is enabled or not and whether there are partition layout changes or not.
  
14. If no error occurs in the last step, reboot the target board.
+
15. If no error occurs in the last step, reboot the target board.  
 
 
It boots to the recovery kernel, which starts the OTA update process. The OTA update process performs a fail-safe update. If the update fails due to power loss, the target device can reboot into the recovery kernel and continue the update. The update process stores logs in ''/ota_logs/'' on the target device. They record the details of the OTA update process.
 
  
'''Note''': To create a complete, secure OTA update solution, you must implement an OTA client that executes steps 10−14 on the target device, and you must apply security mechanisms according to your security policy requirements.
+
16. Wait for the device to reboot. It may reboot a couple of times and get stuck at a black screen for a few minutes but then it will boot back up with the new L4T release.
 
 
15. Wait for the device to reboot. It may reboot a couple of times and get stuck at a black screen for a few minutes but then it will boot back up with the new L4T release.
 
  
 
* If rootfs A/B was enabled on the target, the Bootloader partitions and rootfs partitions are updated only on the inactive slot. If the boot fails, the system will boot to the good slot, which was the one from where the update command was triggered, otherwise the target board reboots to the just updated slot.
 
* If rootfs A/B was enabled on the target, the Bootloader partitions and rootfs partitions are updated only on the inactive slot. If the boot fails, the system will boot to the good slot, which was the one from where the update command was triggered, otherwise the target board reboots to the just updated slot.
Line 290: Line 348:
 
* If rootfs A/B is disabled the target first boots into the recovery kernel, which runs the OTA update process, and then reboots into the updated L4T.
 
* If rootfs A/B is disabled the target first boots into the recovery kernel, which runs the OTA update process, and then reboots into the updated L4T.
  
==A/B Root File System Support==
+
'''Note''': To create a complete, secure OTA update solution, you must implement an OTA client that executes steps 11−15 on the target device, and you must apply security mechanisms according to your security policy requirements.
 +
 
 +
=A/B Root File System Support=
  
 
Starting from JetPack 4.5 NVIDIA provides full support for root filesystems redundancy on Jetson Xavier NX, Jetson Xavier AGX, and Jetson TX2 series.
 
Starting from JetPack 4.5 NVIDIA provides full support for root filesystems redundancy on Jetson Xavier NX, Jetson Xavier AGX, and Jetson TX2 series.
Line 308: Line 368:
 
'''Note''': If you enable root filesystem redundancy you can't apply the Image-based OTA Update explained above, otherwise you will lose the redundancy support.
 
'''Note''': If you enable root filesystem redundancy you can't apply the Image-based OTA Update explained above, otherwise you will lose the redundancy support.
  
===How to Create Redundant Root Filesystems===
+
==How to Create Redundant Root Filesystems==
  
 
The JetPack 4.5 release provides a simple flash command to create and flash the target board with root filesystem redundancy. Go to the ''Linux_for_Tegra'' directory of the JetPack 4.5 release in your host machine:
 
The JetPack 4.5 release provides a simple flash command to create and flash the target board with root filesystem redundancy. Go to the ''Linux_for_Tegra'' directory of the JetPack 4.5 release in your host machine:
Line 321: Line 381:
 
Before flashing you can do some customizations.
 
Before flashing you can do some customizations.
  
====Root Filesystem Size Customization====
+
===Root Filesystem Size Customization===
  
 
When root filesystems redundancy is in use, the size of each rootfs partition (APP and APP_b) is half the size that the rootfs would be if rootfs redundancy were not in use. That size is specified by
 
When root filesystems redundancy is in use, the size of each rootfs partition (APP and APP_b) is half the size that the rootfs would be if rootfs redundancy were not in use. That size is specified by
Line 351: Line 411:
 
</source>
 
</source>
  
====SMD Partition Image Customization====
+
===SMD Partition Image Customization===
  
 
The SMD partition holds information about the status of a slot. A device with redundant filesystems has two partitions, SMD for slot A and SMD_b for slot B. The root filesystems redundancy requires the SMD partitions to function. The SMD partition image is stored in the file ''slot_metadata.bin.rootfsAB'', which the script ''nv_smd_generator'' generates from ''smd_info.rootfs_AB.cfg''.
 
The SMD partition holds information about the status of a slot. A device with redundant filesystems has two partitions, SMD for slot A and SMD_b for slot B. The root filesystems redundancy requires the SMD partitions to function. The SMD partition image is stored in the file ''slot_metadata.bin.rootfsAB'', which the script ''nv_smd_generator'' generates from ''smd_info.rootfs_AB.cfg''.
Line 407: Line 467:
 
</source>
 
</source>
  
====Flashing the Board with Filesystems Redundancy====
+
===Flashing the Board with Filesystems Redundancy===
  
 
Now, to flash the board install the dependencies:
 
Now, to flash the board install the dependencies:
Line 460: Line 520:
 
</source>
 
</source>
  
===Managing the Root Filesystem Slots with nvbootctrl===
+
==Managing the Root Filesystem Slots with nvbootctrl==
  
 
The nvbootctrl tool can help you test redundant filesystems from your NVIDIA Jetson device. You can use it to display the status of both root filesystems or to change the active rootfs slot between slots A and B.
 
The nvbootctrl tool can help you test redundant filesystems from your NVIDIA Jetson device. You can use it to display the status of both root filesystems or to change the active rootfs slot between slots A and B.
Line 519: Line 579:
 
'''Note''': <slot> is 0 for A and 1 for B.
 
'''Note''': <slot> is 0 for A and 1 for B.
  
===Debian OTA Update with Root Filesystem Redundancy Enabled===
+
==Debian OTA Update with Root Filesystem Redundancy Enabled==
  
 
Jetson Linux uses Debian packages to implement Over-the-Air (OTA) update. The OTA update process is the same for a system with root filesystem redundancy as for one with a
 
Jetson Linux uses Debian packages to implement Over-the-Air (OTA) update. The OTA update process is the same for a system with root filesystem redundancy as for one with a
 
single root filesystem, which means that it is not a feature exclusive to the 32.6 release. Also, keep in mind that the Debian packages only update files in the current rootfs slot. The system does not switch the active root filesystem slot after rebooting. This is a guide based on [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html#wwpID0E06B0HA NVIDIA Jetson Linux Developer Guide].
 
single root filesystem, which means that it is not a feature exclusive to the 32.6 release. Also, keep in mind that the Debian packages only update files in the current rootfs slot. The system does not switch the active root filesystem slot after rebooting. This is a guide based on [https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html#wwpID0E06B0HA NVIDIA Jetson Linux Developer Guide].
  
==== Upgrade packages to a new point release ====
+
=== Upgrade packages to a new point release ===
  
 
This in case you want to update to a new point release of the same minor release; for example, from r32.4.1 to r32.4.2.
 
This in case you want to update to a new point release of the same minor release; for example, from r32.4.1 to r32.4.2.
Line 540: Line 600:
 
2. Reboot your Jetson device when the upgrade is finished (remember that it will boot from the same slot in which the packages were updated).
 
2. Reboot your Jetson device when the upgrade is finished (remember that it will boot from the same slot in which the packages were updated).
  
==== Upgrade packages to a new minor release ====
+
=== Upgrade packages to a new minor release ===
  
 
This is in case you want to update to a new minor release; for example, from r32.3.x to r32.4.2.
 
This is in case you want to update to a new minor release; for example, from r32.3.x to r32.4.2.
Line 582: Line 642:
 
5.Reboot your Jetson device when the upgrade is finished.
 
5.Reboot your Jetson device when the upgrade is finished.
  
===Update Partition Images and Rootfs Image===
+
==Update Partition Images and Rootfs Image==
  
==== Bootloader, Kernel and Kernel-dtb ====
+
=== Bootloader, Kernel and Kernel-dtb ===
  
 
The NVIDIA update engine tool updates the images to the unused bootloader partitions and switches the active bootloader slot. The system then boots from the new (updated) active partition. This is the same process as previous bootloader updates. However, now the kernel and kernel-dtb partitions do not belong to the bootloader when rootfs redundancy is enabled, so the update process must identify them by the rootfs slot ID, not the bootloader slot ID. For example, if the current rootfs slot ID is 0, then the kernel and kernel-dtb images must be updated to slot 1.
 
The NVIDIA update engine tool updates the images to the unused bootloader partitions and switches the active bootloader slot. The system then boots from the new (updated) active partition. This is the same process as previous bootloader updates. However, now the kernel and kernel-dtb partitions do not belong to the bootloader when rootfs redundancy is enabled, so the update process must identify them by the rootfs slot ID, not the bootloader slot ID. For example, if the current rootfs slot ID is 0, then the kernel and kernel-dtb images must be updated to slot 1.
Line 640: Line 700:
 
'''Note''': The bootloader updates will cause the device to reboot in the other bootloader slot.
 
'''Note''': The bootloader updates will cause the device to reboot in the other bootloader slot.
  
==== Root Filesystem ====
+
=== Root Filesystem ===
  
===== Customization =====
+
==== Customization ====
  
 
Usually, when you need to update the filesystem with some files, these files are generated natively inside the Jetson device. This means that you need a way to move the complete filesystem from the target into the host in order to be able to distribute it to customers so they can apply the update too.  
 
Usually, when you need to update the filesystem with some files, these files are generated natively inside the Jetson device. This means that you need a way to move the complete filesystem from the target into the host in order to be able to distribute it to customers so they can apply the update too.  
Line 682: Line 742:
 
6. Download the ''rootfs_update.img'' to your Jetson device. Be sure you have enough free space for the rootfs tarball.
 
6. Download the ''rootfs_update.img'' to your Jetson device. Be sure you have enough free space for the rootfs tarball.
  
===== Update =====
+
==== Update ====
  
 
1. Create an update script to unpack the rootfs in the unused rootfs slot. The script will likely do the following steps:
 
1. Create an update script to unpack the rootfs in the unused rootfs slot. The script will likely do the following steps:

Revision as of 13:48, 17 August 2021

Introduction

This wiki shows how to use the new root file system redundancy, as well as the image-based OTA update features, available since Jetpack 4.6.

The OTA update feature enables the user of NVIDIA Jetson platforms to update their devices. Instead of going through the process of re-flashing a board with a new image of the latest Jetpack version, the user can now simply perform the update, as long as the device supports it. This provides benefits such as the option to maintain files after the update, customize the file system before updating, and a fail-safe update procedure.

Similar to OTA update, file system redundancy is a feature integrated into the newer versions of Jetpack. It provides the user with the ability to have two root file systems in the Jetson Device. This feature proves to be highly beneficial for applications where system reliability is of the highest priority. With options like independent creation and update of the two file systems, file system switching and fail-over, this feature is of great value to NVIDIA Jetson platform users.

Feature support table
Board OTA Update File System Redundancy
Jetson TX2 Yes Yes
Jetson AGX Xavier Yes Yes
Jetson Xavier NX Yes Yes
Jetson TX2 NX Yes No
Jetson Nano No No

Image based OTA Update

It is the process to upgrade the whole system by updating partition by partition. Partition-based OTA can be used when the partition layout has changed between the currently running version and the target version (such as when upgrading from L4T 28.2 to L4T 32.6.1) and also can be used when there is no partition layout changes between the currently running version and the target version. The update process is performed by rebooting the device into recovery mode and running recovery kernel and recovery initrd, and then updating the partitions one by one.

It is currently supported only for the Jetson AGX Xavier, Jetson Xavier NX, Jetson TX2 and Jetson TX2 NX. The next table shows the updates supported out-of-the-box, but it is possible to update to other releases such as 32.4 or 32.5 by slightly modifying the procedure.

Board Supported upgrades
Jetson TX21
  • R28.2 → R32.6.1
  • R28.3 → R32.6.1
  • R28.4 → R32.6.1
  • R32.1 → R32.6.1
  • R32.2 → R32.6.1
  • R32.3 → R32.6.1
  • R32.4 → R32.6.1
  • R32.5 → R32.6.1
Jetson AGX Xavier
  • R32.1 → R32.6.1
  • R32.2 → R32.6.1
  • R32.3 → R32.6.1
  • R32.4 → R32.6.1
  • R32.5 → R32.6.1
Jetson Xavier NX
  • R32.4 → R32.6.1
  • R32.5 → R32.6.1
Jetson TX2 NX
  • R32.5 → R32.6.1

1 Only the original TX2 module.

The following updates have been tested:

Board Supported upgrades
Jetson TX2
  • R32.5 → R32.6.1

Note: There needs to be free space available in EMMC (~100MB for TX2 and ~300MB for Jetson AGX Xavier) to build an intermediate boot path, in case there is a reboot during OTA so that the device can fallback to recovery kernel from this path.

How to Perform an OTA Update

These instructions were extracted from NVIDIA official documentation.

Note: You cannot use OTA update to update a Jetson device on which Jetpack is installed.

Steps on the host machine

1. Flash the target device with any of the BSP listed above in order to perform the update to the R32.6.1 BSP.

2. Assign the BASE_BSP environment variable to the Linux_for_Tegra directory of the JetPack version you just flashed or the old one you had.

#Usually is in a path similar to this
BASE_BSP=/home/${USER}/nvidia/nvidia_sdk/JetPack_{DETAILS_OF_YOUR_VERSION_AND_DEVICE}/Linux_for_Tegra/

3. Download the SDK packages from the version you want to update to (32.6.1) and unpack them in the install directory.

# This is the default install directory for SDK manager, change according to your case.
cd /home/${USER}/nvidia/
mkdir L4T_32.6.1/
cd L4T_32.6.1/

4. Unpack and apply the new release sample root filesystem.

For simplicity, lets define a variable for the NVIDIA downloads directory. This will change if you changed the default downloads directory.

NVIDIA_DOWNLOADS_DIR=/home/${USER}/Downloads/nvidia/sdkm_downloads/

Now, let's continue with the next step.

# Modify this if you changed the default downloads directory
tar xpf $NVIDIA_DOWNLOADS_DIR/Jetson_Linux_R32.6.1_aarch64.tbz2
cd Linux_for_Tegra/rootfs/
sudo tar xpf $NVIDIA_DOWNLOADS_DIR/Tegra_Linux_Sample-Root-Filesystem_R32.6.1_aarch64.tbz2

Optional: Apply your customizations to the rootfs at this moment (e.g. Add files, libraries, binaries).

cd ..
sudo ./apply_binaries.sh

5. Assign the ToT_BSP (Top of Tree, latest release) and TARGET_BSP variables to the Linux_for_Tegra folder of the new release (32.6.1).

#After last commands you should be on this ''Linux_for_Tegra'' directory
ToT_BSP=$PWD
TARGET_BSP=$PWD

6. Unpack the OTA tools package (this will add some extra files to the Linux_for_Tegra directory). This package can be found in the official NVIDIA release page, in the Tools section, under the name Jetson Platform Over-The-Air Update Tools.

cd ..
tar <OTA_TOOLS_DOWNLOAD_PATH>/xpf ota_tools_r32.6.1_aarch64.tbz2

7. Optional: If you want to update to other release different than ToT, such as 32.4 or 32.5 follow the next instructions, otherwise go to step 8.

Note: Updating to 32.5 is not supported on the Jetson Xavier NX P3668-0000.

Create a directory that contains the BSP package that you want to update to, for example for 32.5:

mkdir L4T_32.5/
cd L4T_32.5/
tar xpf $NVIDIA_DOWNLOADS_DIR/Jetson_Linux_R32.5.0_aarch64.tbz2
cd Linux_for_Tegra/rootfs/
sudo tar xpf $NVIDIA_DOWNLOADS_DIR/Tegra_Linux_Sample-Root-Filesystem_R32.5.0_aarch64.tbz2
cd ..
sudo ./apply_binaries.sh
# Redefine TARGET_BSP variable
TARGET_BSP=$PWD

Install in the TARGET_BSP the required OTA tools files:

cd ..
tar xpf <OTA_TOOLS_DOWNLOAD_PATH>/ota_tools_r32.6.1_aarch64.tbz2
cd $TARGET_BSP
cp ${ToT_BSP}/tools/ota_tools/version_upgrade/* ./tools/ota_tools/version_upgrade

Generate slot metadata files for TARGET_BSP:

cd ${TARGET_BSP}
cp ./bootloader/smd_info.cfg ./tools/ota_tools/version_upgrade/smd_info.boot_slot_A.cfg
cp ./bootloader/smd_info.cfg ./tools/ota_tools/version_upgrade/smd_info.boot_slot_B.cfg
sed -i -e 's/^15/# 15/' -e 's/^##15/15/' -e 's/^##14/14/' ./tools/ota_tools/version_upgrade/smd_info.boot_slot_A.cfg
sed -i -e 's/^15/# 15/' -e 's/^##15/14/' -e 's/^##14/15/' ./tools/ota_tools/version_upgrade/smd_info.boot_slot_B.cfg
./bootloader/nv_smd_generator ./tools/ota_tools/version_upgrade/smd_info.boot_slot_A.cfg ./tools/ota_tools/version_upgrade/slot_metadata.bin.A
./bootloader/nv_smd_generator ./tools/ota_tools/version_upgrade/smd_info.boot_slot_B.cfg ./tools/ota_tools/version_upgrade/slot_metadata.bin.B

Copy the files from ${ToT_BSP} and modify p2771-0000.conf.common, p2972-0000.conf.common, and p3668.conf.common:

cp ${ToT_BSP}/flash.sh ./
cp ${ToT_BSP}/l4t_sign_image.sh ./
cp ${ToT_BSP}/l4t_generate_soc_bup.sh ./
cp ${ToT_BSP}/bootloader/tegraparser_v2 ./bootloader/
cp ${ToT_BSP}/bootloader/tegraflash.py ./bootloader/
cp ${ToT_BSP}/bootloader/tegrahost_v2 ./bootloader/
cp ${ToT_BSP}/bootloader/tegrabct_v2 ./bootloader/
cp ${ToT_BSP}/bootloader/tegraflash_internal.py ./bootloader/
cp ${ToT_BSP}/bootloader/tegrasign_v3_util.py ./bootloader/
cp ${ToT_BSP}/bootloader/tegrasign_v3_internal.py ./bootloader/
cp ${ToT_BSP}/bootloader/tegrasign_v3.py ./bootloader/
cp ${ToT_BSP}/bootloader/tegraopenssl ./bootloader/
sudo cp ${ToT_BSP}/rootfs/opt/nvidia/l4t-bootloader-config/nv-l4t-bootloader-config.sh ./rootfs/opt/nvidia/l4t-bootloader-config/nv-l4t-bootloader-config.sh
sudo cp ${ToT_BSP}/rootfs/usr/sbin/nvbootctrl ./rootfs/usr/sbin/nvbootctrl
sudo cp ${ToT_BSP}/rootfs/usr/sbin/nv_bootloader_payload_updater ./rootfs/usr/sbin/nv_bootloader_payload_updater
sudo cp ${ToT_BSP}/rootfs/usr/sbin/nv_update_engine ./rootfs/usr/sbin/nv_update_engine
echo "SMDFILE=\"slot_metadata.bin\";" >>p2771-0000.conf.common
echo "SMDFILE=\"slot_metadata.bin\";" >>p2972-0000.conf.common
echo "SMDFILE=\"slot_metadata.bin\";" >>p3668.conf.common

8. Generate the base recovery image and recovery DTB:

First, make sure you have the device tree compiler by running:

sudo apt install device-tree-compiler

Optional: Backup files that you want to keep after the update. NVIDIA provides as part of the OTA tools from R32.6.1 release a file named ota_backup_files_list.txt inside the Linux_for_Tegra/tools/ota_tools/version_upgrade/ directory. You can add there the files or directories that you want to backup and then restore after the update. They should be listed using the absolute path without the first / that indicates the root.

And then run:

cd $ToT_BSP
sudo ./tools/ota_tools/version_upgrade/build_base_recovery_image.sh [-u <PKC key file>] [-v <SBK key file>] <target_board> <bsp_version> ${BASE_BSP} ${BASE_BSP}/rootfs ${TARGET_BSP}

where:

  • <target_board> can be:
    • For Jetson Xavier NX P3668-0000: jetson-xavier-nx-devkit
    • For Jetson Xavier NX P3668-0001: jetson-xavier-nx-devkit-emmc
    • For Jetson TX2 NX: jetson-xavier-nx-devkit-tx2-nx
    • For Jetson AGX Xavier P2888-0001 (16 GB) or P2888-0004 (32 GB): jetson-agx-xavier-devkit
    • For Jetson TX2 P3310-1000: jetson-tx2-devkit
  • <bsp_version> can be R28-2, R28-3, R28-4, R32-1, R32-2, R32-3, R32-4 or R32-5
  • <PKC key file> is an optional PKC key file. It must be the same as the one used in flashing images to the target board.
  • <SBK key file> is an optional SBK key file. It must be the same as the one used in flashing images to the target board.

So, an example for TX2 upgrading from 32.5 to 32.6.1 without PCK and SBK files would be:

sudo ./tools/ota_tools/version_upgrade/build_base_recovery_image.sh jetson-tx2-devkit R32-5 ${BASE_BSP} ${BASE_BSP}/rootfs ${TARGET_BSP}

After running this command you should have a recovery image and a recovery DTB under ${TARGET_BSP}/bootloader. These are typically named recovery.dtb.<bsp_version> and recovery.img.<bsp_version>.

9. Generate the OTA update payload package:

cd $TARGET_BSP
sudo ./tools/ota_tools/version_upgrade/l4t_generate_ota_package.sh [-s] [‑b] [‑r] [-u <PKC_key_file>] [-v <SBK_key_file>]  [-o <rootfs_updater>] [-f <rootfs_image>] <target_board> <bsp_version>

where the parameters are:

  • ‑s: Skip generating the rootfs image. Use this option if a rootfs image has been generated or specified with the -f option.
  • -b: Generate an OTA payload package for updating Bootloader only. This option is meaningful only for an update without partition layout change.
  • -r: Generate an OTA payload package for updating rootfs only. This option is valid only for an update without partition layout change.
  • <PKC_key_file> is an optional PKC key file. It must be the same as the one used in flashing images to the target board.
  • <SBK_key_file> is an optional SBK key file. It must be the same as the one used in flashing images to the target board.
  • <rootfs_updater> specifies the script to be used to update the rootfs partition. (For more information, see NVIDIA official guidelines).
  • <rootfs_image> is the path of the rootfs image to be written to the rootfs partition.
  • <target_board> is the configuration name specified for build_base_recovery_image.sh in last step.
  • <bsp_version> is the base BSP version specified for build_base_recovery_image.sh in last step.

Note: ‑s, ‑b, and ‑r may be used separately, as shown, or may be combined in a single word, e.g. ‑sb or ‑sr.

So, an example for perfoming a TX2 full upgrade from 32.5 to 32.6.1 without PCK and SBK files would be:

sudo ./tools/ota_tools/version_upgrade/l4t_generate_ota_package.sh jetson-tx2-devkit R32-5

This script generates the file ota_payload_package.tar.gz under ${TARGET_BSP}/bootloader/<target_board>. This package would normally be uploaded to an OTA server in order to implement a complete OTA solution. The ota_payload_package.tar.gz archive that the script generates from the default rootfs occupies about 2 GB. A self-generated rootfs may yield a larger or smaller archive. Decompressing the archive on the Jetson device to apply the image-based OTA requires at least 6 GB of free space.

Note: Steps 8 and 9 use the L4T sample rootfs from ${BASE_BSP}/rootfs/ to build the recovery kernel, recovery DTB and OTA package. This option lets you overwrite the default rootfs image with a self-generated rootfs image, for example, an image cloned from a Jetson device that has been fully customized with a top-of-tree release of Jetson Linux. If you have customized the sample rootfs, for example, you must add required libraries and binaries into rootfs, and apply them before running these scripts.

Steps on the target board

10. Check /boot/extlinux/extlinux.conf to see if INITRD variable is set, otherwise add:

LINUX /boot/Image line:
INITRD /boot/initrd # Add this line

Check if the root device is set, otherwise add it into the APPEND line:

root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4

11. Create a working directory for the OTA update process and assign it to the WORKDIR variable:

mkdir ota_workdir
cd ota_workdir
WORKDIR=${PWD}

12. Download the OTA update tool package ota_tools_<release>_aarch64.tbz2 from step 6 and unpack it inside $WORKDIR.

An example with SSH would be the following:

scp <host_user>@<host_ip_address>:<path-to-ota-tools-tarball> .
tar xpf ota_tools_r32.6.1_aarch64.tbz2

13. Download the ota_payload_package.tar.gz from step 8 and put in the directory (/ota).

An example with SSH would be the following:

sudo mkdir /ota
sudo scp <host_user>@<host_ip_address>:<path-to-ota-payload-tarball> /ota/

14. Prepare to start OTA:

cd ${WORKDIR}/Linux_for_Tegra/tools/ota_tools/version_upgrade
sudo ./nv_ota_start.sh /dev/mmcblk0 /ota/ota_payload_package.tar.gz

where /dev/mmcblk0 is the device on which the partitions for the backup boot path are placed and /ota/ota_payload_package.tar.gz is the OTA payload package. The nv_ota_start tool will handle automatically all use cases whether rootfs A/B is enabled or not and whether there are partition layout changes or not.

15. If no error occurs in the last step, reboot the target board.

16. Wait for the device to reboot. It may reboot a couple of times and get stuck at a black screen for a few minutes but then it will boot back up with the new L4T release.

  • If rootfs A/B was enabled on the target, the Bootloader partitions and rootfs partitions are updated only on the inactive slot. If the boot fails, the system will boot to the good slot, which was the one from where the update command was triggered, otherwise the target board reboots to the just updated slot.
  • If rootfs A/B is disabled the target first boots into the recovery kernel, which runs the OTA update process, and then reboots into the updated L4T.

Note: To create a complete, secure OTA update solution, you must implement an OTA client that executes steps 11−15 on the target device, and you must apply security mechanisms according to your security policy requirements.

A/B Root File System Support

Starting from JetPack 4.5 NVIDIA provides full support for root filesystems redundancy on Jetson Xavier NX, Jetson Xavier AGX, and Jetson TX2 series. It uses two root filesystems, one designated rootfs A and stored in the usual filesystem partition, APP, and the other designated rootfs B and stored in a new partition, APP_b, located after APP on the disk.

As a failover mechanism, if the current active rootfs were damaged or corrupted, and then can’t boot up the system, after automatically rebooting 3 times (not customizable), the device will failover to the other available unused rootfs to boot up. If both rootfs A and B are unbootable, the device will boot into the recovery kernel image.

This feature is currently supported just on NVIDIA Jetson Xavier AGX, Xavier NX, and TX2.

Here are some useful terms in order to understand the feature:

  • active rootfs slot: the slot that the next boot operation will attempt to boot from.
  • current rootfs slot: the slot that is currently being used. The opposite of this is the unused rootfs slot.

Note: If you enable root filesystem redundancy you can't apply the Image-based OTA Update explained above, otherwise you will lose the redundancy support.

How to Create Redundant Root Filesystems

The JetPack 4.5 release provides a simple flash command to create and flash the target board with root filesystem redundancy. Go to the Linux_for_Tegra directory of the JetPack 4.5 release in your host machine:

# Usually on this directory
cd /home/${USER}/nvidia/nvidia_sdk/JetPack_4.5_Linux_{DEVICE_NAME}/Linux_for_Tegra/

Make sure you apply the sample root filesystem.

Before flashing you can do some customizations.

Root Filesystem Size Customization

When root filesystems redundancy is in use, the size of each rootfs partition (APP and APP_b) is half the size that the rootfs would be if rootfs redundancy were not in use. That size is specified by ROOTFSSIZE in the configuration file for the target board. You can change the amount of space allocated to the root filesystems by changing the value of ROOTFSSIZE before you flash the target.

In order to do this you need to modify the next file according to your board:

Board Configuration File
Xavier NX p3668.conf.common
Xavier AGX p2972-0000.conf.common
TX2 p2771-0000.conf.common

For example, If you require to modify the size of the root filesystem in an Xavier NX, edit the ROOTFSSIZE variable in the following snippet located in the p3668.conf.common file:

if [ "${ROOTFS_AB}" == 1 ]; then
	rootfs_ab=1
	ROOTFSSIZE=7GiB; #Edit
fi;

SMD Partition Image Customization

The SMD partition holds information about the status of a slot. A device with redundant filesystems has two partitions, SMD for slot A and SMD_b for slot B. The root filesystems redundancy requires the SMD partitions to function. The SMD partition image is stored in the file slot_metadata.bin.rootfsAB, which the script nv_smd_generator generates from smd_info.rootfs_AB.cfg.

In order to customize this, edit the smd_info.rootfs_AB.cfg file, located in the bootloader directory:

cd bootloader
${YOUR_EDITOR} smd_info.rootfs_AB.cfg

There, you may disable the root filesystems redundancy and change the active slot, if that is what you require. You can achieve such task, by modifying the <ROOTFS_AB 1>, <ROOTFS_ACTIVE_A 1> and <ROOTFS_ACTIVE_B 1> variables. See the next snippet taken from the default configuration:

# Config 2: Enable rootfs A/B support (default)
#
< REDUNDANCY_ENABLE 1 >
< ROOTFS_AB 1 > #Disable root filesystems redundancy by replacing 1 with 0

# Select rootfs A as the active rootfs
< ROOTFS_ACTIVE_A 1 > #To set slot B, comment this line and uncomment the next one
##< ROOTFS_ACTIVE_B 1 >

You can also change the number of times CBoot tries to use the primary file system, before changing the roles and using the secondary. By setting:

# Set the maximum rootfs slot retry count
# Please make sure this field is set before slot info config
# The valid setting is 1 to 3
< MAX_ROOTFS_AB_RETRY_COUNT 3 >

Finally, if you require to perform bootloader related configurtions. You can set the following variables:

# Enable/disable unified bootloader AB and rootfs AB
# Set 1 to enable, set 0 to disable. Default is enabled
# This option must be defined after "< ROOTFS_AB 1 >"
# When < ROOTFS_BL_UNIFIED_AB 1 > is set,
# auto sync for both BL and RF are disabled.
< ROOTFS_BL_UNIFIED_AB 1 >
 
# To disable bootloader autosync, use < BL_AUTOSYNC_DISABLE 1 >, default is disabled.
# REDUNDANCY_ENABLE or REDUNDANCY_USER must be defined before
# BL_AUTOSYNC_DISABLE !
< BL_AUTOSYNC_DISABLE 1 >

Once the file is modified, generate the new slot_metadata.bin.rootfs_AB.cfg by running:

#./nv_smd_generator <config_file> <output_file>
./nv_smd_generator smd_info.rootfs_AB.cfg slot_metadata.bin.rootfsAB

Flashing the Board with Filesystems Redundancy

Now, to flash the board install the dependencies:

sudo apt install libxml2-utils

and then run on the Linux_for_Tegra directory:

sudo ROOTFS_AB=1 ./flash.sh [options] <target_board> <rootdev>

where <target_board> may be:

Board <target_board>
Xavier NX jetson-xavier-nx-devkit-emmc or jetson-xavier-nx-devkit
Xavier AGX jetson-agx-xavier-devkit
TX2 jetson-tx2-devkit

and <rootdev> specifies the location of the root filesystems (i.e mmcblk0p1).

For example:

sudo ROOTFS_AB=1 ./flash.sh jetson-xavier-nx-devkit mmcblk0p1

Keep in mind that the flashing might take a while. If A/B redundancy was enabled you should see at some point some log like this:

...
Making system.img... #FILESYSTEM A
	populating rootfs from ${LOCATION_OF_THE_LINUX_FOR_TEGRA_DIRECTORY} ... done.
	populating /boot/extlinux/extlinux.conf ... done.
	Sync'ing system.img ... done.
	Converting RAW image to Sparse image... done.
system.img built successfully. 
Making system.img_b... #FILESYSTEM B
	populating rootfs from ${LOCATION_OF_THE_LINUX_FOR_TEGRA_DIRECTORY} ... done.
...

Managing the Root Filesystem Slots with nvbootctrl

The nvbootctrl tool can help you test redundant filesystems from your NVIDIA Jetson device. You can use it to display the status of both root filesystems or to change the active rootfs slot between slots A and B.

In order to use it you may run:

sudo nvbootctrl -t rootfs <command>

where <command> can be:

<command> Description
get-number-slots Prints the number of slots.
get-current-slot Prints the index of the currently running slot.
mark-boot-successful Marks the current slot as good.
set-active-boot-slot <slot> Causes next boot to load and execute the specified slot.
set-slot-as-unbootable <slot> Marks <slot> as invalid.
is-slot-bootable <slot> Returns 0 if <slot> is bootable, or a non-zero value otherwise.
is-slot-marked-successful <slot> Returns 0 only if <slot> is marked as good.
get-suffix <slot> Prints the suffix of <slot>.
dump-slots-info Prints information about slots.
is-autosync-enabled Prints the status of autosync (enabled or disabled). For Bootloader only.
toggle-autosync Toggles the enablement status of autosync. For Bootloader only.
is-unified-enabled Returns:
  • 0 if both unified A/B and unified A/B are enabled
  • 69 if both unified A/B and rootfs A/B are disabled
  • 70 if rootfs A/B is enabled and unified A/B is disabled

Note: <slot> is 0 for A and 1 for B.

Debian OTA Update with Root Filesystem Redundancy Enabled

Jetson Linux uses Debian packages to implement Over-the-Air (OTA) update. The OTA update process is the same for a system with root filesystem redundancy as for one with a single root filesystem, which means that it is not a feature exclusive to the 32.6 release. Also, keep in mind that the Debian packages only update files in the current rootfs slot. The system does not switch the active root filesystem slot after rebooting. This is a guide based on NVIDIA Jetson Linux Developer Guide.

Upgrade packages to a new point release

This in case you want to update to a new point release of the same minor release; for example, from r32.4.1 to r32.4.2.

1. Run the following commands:

#Read a list of packages from the remote APT repository and identifies new and upgradable packages
sudo apt update
#Display a list of new and upgradable packages
sudo apt list --upgradable
sudo apt upgrade

2. Reboot your Jetson device when the upgrade is finished (remember that it will boot from the same slot in which the packages were updated).

Upgrade packages to a new minor release

This is in case you want to update to a new minor release; for example, from r32.3.x to r32.4.2.

1. Open the apt source configuration file in a text editor, for example:

sudo vim /etc/apt/sources.list.d/nvidia-l4t-apt-source.list

2. Change the repository name from r32 to r32.4 (or any other newer minor version you want), and <platform> to the appropriate processor name for your platform, where the original commands are:

#Note that r32 may be r32.4 or other name repo according to your current release
deb https://repo.download.nvidia.com/jetson/common r32 main
deb https://repo.download.nvidia.com/jetson/<platform> r32 main

Here <platform> identifies the platform’s processor:

  • t186 for Jetson TX2 series
  • t194 for Jetson AGX Xavier series or Jetson Xavier NX
  • t210 for Jetson Nano or Jetson TX1

For example, if your platform is Jetson Xavier NX it should look like this:

deb https://repo.download.nvidia.com/jetson/common r32.4 main
deb https://repo.download.nvidia.com/jetson/t194 r32.4 main

3. Save and close the source configuration file.

4. Run the commands:

sudo apt update
sudo apt dist-upgrade

5.Reboot your Jetson device when the upgrade is finished.

Update Partition Images and Rootfs Image

Bootloader, Kernel and Kernel-dtb

The NVIDIA update engine tool updates the images to the unused bootloader partitions and switches the active bootloader slot. The system then boots from the new (updated) active partition. This is the same process as previous bootloader updates. However, now the kernel and kernel-dtb partitions do not belong to the bootloader when rootfs redundancy is enabled, so the update process must identify them by the rootfs slot ID, not the bootloader slot ID. For example, if the current rootfs slot ID is 0, then the kernel and kernel-dtb images must be updated to slot 1.

1. Plug in the Jetson device in recovery mode to the host and generate the bootloader and kernel payloads named bl_only_payload, kernel_only_payload and bl_update_payload by running:

cd <path-to-Linux_for_Tegra-directory>
sudo ./build_l4t_bup.sh <target-device> mmcblk0p1

where:

  • <target-device> is the device name (e.g. jetson-xavier-nx-devkit, jetson-tx2, etc).

The resulting files should be stored at bootloader/payloads_<processor-number>.

2. Download the payloads to the Jetson device.

3. Apply the updates in the Jetson device.

  • For kernel and bootloader

Copy bl_update_payload to /opt/ota_package/bl_update_payload and then run:

sudo nv_update_engine --install
  • For bootloader only

Copy bl_only_payload to /opt/ota_package/bl_update_payload and then run:

sudo nv_update_engine --install
  • For kernel only

Copy kernel_only_payload to /opt/ota_package/bl_update_payload and then run:

sudo nvbootctrl -t rootfs get-current-slot

This will print the current slot, so if it is 0 you must update the kernel_b and kernel-dtb_b partitions and if is 1 then the kernel and kernel-dtb partitions.

sudo nv_bootloader_payload_updater --part <kernel | kernel_b>
sudo nv_bootloader_payload_updater --part <kernel-dtb | kernel-dtb_b>

Where the kernel partitions depend on the current slot as stated above.

Note: The bootloader updates will cause the device to reboot in the other bootloader slot.

Root Filesystem

Customization

Usually, when you need to update the filesystem with some files, these files are generated natively inside the Jetson device. This means that you need a way to move the complete filesystem from the target into the host in order to be able to distribute it to customers so they can apply the update too.

The NVIDIA tool for updating the rootfs requires a tarball with the new filesystem to be updated, so in the next step, you will learn how to obtain this tarball from the Jetson device.

1. Apply the customizations needed in your Jetson device.

2. Put your Jetson device in recovery mode and plug it into the host computer.

3. Run the following command to perform a clone of the rootfs partition. This will generate two image files named <backup-name>.img and <backup-name>.img.raw in case of the APP partition and one file named <backup-name>.img for the APP_b partition. When using the APP partition the .img file generated is a space-optimized version to reduce storage consumption but it cannot be mounted and modified. The img.raw file is the complete image without optimizations, which makes it mountable and editable, so you will be using this one. However, when using APP_b partition it only generates one .img file, in this case, this is the one you can mount and edit.

cd <path-to-Linux_for_Tegra-directory>
sudo ROOTFS_AB=1 ./flash.sh -r -k <partition-name> -G <backup-name>.img <target-device> mmcblk0p1

where:

  • <target-device> is the device name (e.g. jetson-xavier-nx-devkit, jetson-tx2, etc).
  • <partition-name> is either APP or APP_b for rootfs A or B, respectively.
  • <backup-name> is the name without extension of the rootfs images that will be generated.

4. Mount the image with no optimizations.

# For APP partition clone
sudo mount -o loop <backup-name>.img.raw /mnt
# For APP_b partition clone
sudo mount -o loop <backup-name>.img /mnt

5. Create a tarball named rootfs_update.img with the contents of the mounted image

sudo tar -cpf rootfs_update.img /mnt/*

6. Download the rootfs_update.img to your Jetson device. Be sure you have enough free space for the rootfs tarball.

Update

1. Create an update script to unpack the rootfs in the unused rootfs slot. The script will likely do the following steps:

  • Receive the tarball name as a parameter.
  • Get the unused rootfs slot ID with the nvbootctrl tool.
  • Back up all configurations files that should be restored after the update.
  • Mount the device that represents the unused rootfs slot. This is where the update will be applied:
    • For APP partition is /dev/mmcblk0p1.
    • For the APP_b partition is /dev/mmcblk0p1.
  • Format the destination rootfs before updating, or update directly by unpacking the rootfs_update.img on top of the current rootfs from the unused slot.
  • Restore the backed-up configuration files.

We provide a basic example of this script. This example does not include the backup part of the steps above.

#!/bin/bash
ROOTFS_TAR=$1

# Get unused rootfs
nvbootctrl -t rootfs get-current-slot > tmp.log
USED_ROOTFS="$(tail -1 tmp.log)"

if [ "$USED_ROOTFS" = "0" ]; then
  UNUSED_ROOTFS="APP_b"
  MOUNT_DEVICE=/dev/mmcblk0p2
elif [ "$USED_ROOTFS" = "1" ]; then
  UNUSED_ROOTFS="APP"
  MOUNT_DEVICE=/dev/mmcblk0p1
else
  echo "Unable to determine unused rootfs"
  exit 1
fi

echo "Mounting unused rootfs $UNUSED_ROOTFS $MOUNT_DEVICE"

# Mount unused rootfs
mount $MOUNT_DEVICE /mnt
# Extract files in /mnt (new rootfs)
tar -C /mnt -xpf $ROOTFS_TAR

2. Download the script to your Jetson device.

3. Run the NVIDIA Update Engine with the following parameters:

nv_update_engine -i rf --payload <rootfs_image> --rf-updater <rootfs_updater> [--no-reboot]

where:

  • <rootfs_image>: is the path to the rootfs_update.img generated in the customization section.
  • <rootfs_updater>: is the absolute path to the update script created in the previous steps.
  • [--no-reboot]: is an optional parameter to tell the updater not to reboot after the update (by default it does it).

4. Once the process is done the active slot will be the other slot, so after reboot, it will boot from the updated slot.

Note: This operation cannot update an earlier release of L4T, such as r32.4, to an r32.5.* release. ‏‎‏‏