OpenWRT: Manage correctly the process to upgrade to the new version!

Article published the ; modified the
8 minutes to read

This article has 1578 words.
RAW source of the article:

Description

OpenWRT has a tool to upgrade the system, named sysupgrade. You can invoque on the Luci’s web admin.

If you are more interesting by sysupgrade on LuCI, go to this note: Flash on LuCI


The following procedure explains step by step the sysupgrade in CLI mode, while preserving the user configuration…

Process

The first action is install the tool curl, because the native wget does not support TLS.

# opkg install curl

opkgscript.sh

The opkgscript.sh script is used to save the list of packages installed in addition to the base.

⇒ Now, fetch opkgscript.sh:
$ curl -O https://raw.githubusercontent.com/richb-hanover/OpenWrtScripts/master/opkgscript.sh

⇒ Put the +x needed rights:
# chmod 0700 opkgscript.sh

⇒ backup the list of installed packages; to re-install easy them after the sysupgrade process:

# ./opkgscript.sh -v write

This script write the list into file /etc/config/opkg.installed.

Download firmware

Let’s retrieve the new firmware version:

  • e.g. the current version:

# v="22.03.2"

⇒ for the Ubiquiti EdgeRouter X :

curl -O https://downloads.openwrt.org/releases/"${v}"/targets/ramips/mt7621/{openwrt-"${v}"-ramips-mt7621-ubnt-erx-squashfs-sysupgrade.bin,sha256sums}

⇒ for the Xiaomi Redmi Router AC2100 :

curl -O https://downloads.openwrt.org/releases/"${v}"/targets/ramips/mt7621/{openwrt-"${v}"-ramips-mt7621-xiaomi_redmi-router-ac2100-squashfs-sysupgrade.bin,sha256sums}

And, we check the checksum:
# sha256sum -c sha256sums 2> /dev/null | grep OK

⇒ Correct result for Xiaomi Redmi Router AC2100:
openwrt-22.03.2-ramips-mt7621-xiaomi_redmi-router-ac2100-squashfs-sysupgrade.bin: OK

Danger

Backup configuration

The next step check the backup configuration:

# sysupgrade -l

If necessary, you can edit the file /etc/sysupgrade.conf to add some folders/files; for instance, in the case where a user has been added to the **sudo** group , you need to add those:

  • /etc/sudoers
  • /etc/sudoers.d/

Check again; and, backup the configuration:

# sysupgrade -b /tmp/backup-${HOSTNAME}-$(date +%F).tar.gz

And, after, it is necessary to retrieve this backup:
$ scp root@openwrt:/tmp/backup*.tar.gz $(pwd)
(where ‘openwrt’ is the adresse IP about your router)

Info

Upgrade system

Now, it’s time to upgrade the system:

# sysupgrade -v openwrt-"${v}"-*-sysupgrade.bin

Example, for the Ubiquiti EdgeRouter X:

# sysupgrade -v openwrt-"${v}"-ramips-mt7621-ubnt-erx-squashfs-sysupgrade.bin`
Saving config files...
etc/config/dhcp
etc/config/dhcp-opkg
etc/config/dropbear
etc/config/firewall
etc/config/firewall-opkg
etc/config/https-dns-proxy
etc/config/https-dns-proxy-opkg
etc/config/luci
etc/config/luci-opkg
etc/config/network
etc/config/ntpclient
etc/config/opkg.installed
etc/config/rpcd
etc/config/system
etc/config/ucitrack
etc/config/ucitrack-opkg
etc/config/uhttpd
etc/dropbear/authorized_keys
etc/dropbear/dropbear_rsa_host_key
etc/group
etc/hosts
etc/inittab
etc/luci-uploads/.placeholder
etc/opkg/keys/0b26f36ae0f4106d
etc/opkg/keys/1035ac73cc4e59e3
etc/opkg/keys/5151f69420c3f508
etc/opkg/keys/72a57f2191b211e0
etc/opkg/keys/792d9d9b39f180dc
etc/opkg/keys/9ef4694208102c43
etc/opkg/keys/b2d571e0880ff617
etc/opkg/keys/b5043e70f9a75cde
etc/opkg/keys/c10b9afab19ee428
etc/opkg/keys/dace9d4df16896bf
etc/opkg/keys/dd6de0d06bbd3d85
etc/opkg/keys/f94b9dd6febac963
etc/passwd
etc/profile
etc/profile.d/opkg.sh
etc/rc.local
etc/shadow
etc/shells
etc/sysctl.conf
etc/uhttpd.crt
etc/uhttpd.key
etc/uhttpd.key
etc/uhttpd.crt
Commencing upgrade. Closing all shell sessions.
Tip

At this moment, your SSH session close, and the router reboot!


Info

Verify the new OS version

After the reboot, you will find the information about the new version:

  • In LuCI, go to Status > Overview, see the section “System” and “Firmware version”.
  • In SSH, the login banner has the release information.

Upgrade packages

Warning

Let’s upgrade the third-party packages:

⇒ On the SSH console, do:

# opkg update && opkg list-upgradable

If the result show some binaries to update, launch this command:

# for name in `opkg list-upgradable | awk '{print $1}'`; do opkg upgrade "${name}"; done

Restore “user profil”

To restore the user profile:

After repeating the step about curl and the script opkgscript.sh, we run the script to reinstall all packages previously intalled by you:

# ./opkgscript.sh -v install

And, after… the last but not the least: reboot!

Ultimates Checks

Check your configuration:

  • your various network interfaces are always present and operational?
  • your firewall configuration is correct?
  • your different services run correctly? Have you a IPv6 tunnel, OpenVPN, or others services, usually, accessed by the menu “Services”?

Voila!


Notes

You will find various useful informations, only for certain contexts:

Flash on LuCI

Since the menu “System” > “Backup / Flash firmware”:

1/ It can be usefull in the ‘Configuration’ tab to modify the list of custom files to save…
If you installed some packages, relatives files and folders are not backuped if there are not wroted here.

2/ Make a backup of your OpenWRT configuration before, from the ‘Actions’ tab.

3/ During the process of flashing a new sysupgrade image:

⇒ use those options:

  • KEEP SETTINGS AND RETAIN THE CURRENT CONFIGURATION, at least
  • INCLUDE IN BACKUP A LIST OF CURRENT INSTALLED PACKAGES AT /ETC/BACKUP/INSTALLED_PACKAGES.TXT.

If those are not checked, you lost your entire configuration of OpenWRT.

In all cases, all previous installed packages need to be reinstall, perhaps to configure again.

Keep on mind that changes included by upgrades to major version can cause problems during migration. There may be critical changes.

SSH v9.0 and higher

If your SSH client version is ≥ 9.0:

Since SSH v9.0, the behavior of scp has changed.
The above command will fail with the error message:

scp router:/tmp/backup-***-2022-04-21.tar.gz .
ash: /usr/libexec/sftp-server: not found
scp: Connection closed

In the fact, Dropbrear cant discuss with…
To resolve this, add the -O option to the command, instance:
$ scp -O root@openwrt:/tmp/backup*.tar.gz $(pwd)
(this restore the old SFTP behavior of scp)

Another tips is to install the OpenSSH server instead of Dropbear .

Freeing memory

This section is most useful if /tmp is not enough large to store the sysupgrade OpenWRT image. The following actions will temporary free up space in RAM.

Let’s make sure of the memory and disk space with the commands free and df, or cat /proc/meminfo; the goal is to verify that the size of free RAM is larger than the size of the downloaded image. If it’s the case, then go to continue the process… else, “houston, we have a problem” and go to discuss on the OpenWRT forum!


Take an example to better understand:

⇒ the size of the sysupgrade binary to the current version:

$ ll -h
(…)
-rw-rw-r-- 1 root root 6.7M Sep  10 13:53 openwrt-22.03.0-ramips-mt7621-xiaomi_redmi-router-ac2100-squashfs-sysupgrade.bin
(…)

it is 6.7 Mb.

⇒ the available space in /tmp:

# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 3.8M      3.8M         0 100% /rom
tmpfs                    59.7M    540.0K     59.1M   1% /tmp
/dev/ubi0_1              97.2M      7.8M     84.7M   8% /overlay
overlayfs:/overlay       97.2M      7.8M     84.7M   8% /
tmpfs                   512.0K         0    512.0K   0% /dev

In the context of the Xiaomi Redmi AC2100, the available space is 59.1Mb, which is more enough to retrieve the sysupgrade image and manage it.

⇒ Also, the space freed memory:

# free -m
              total        used        free      shared  buff/cache   available
Mem:         122220       40532       61400         540       20288       46700
Swap:             0           0           0

In this context, the free memory space is about 60Mb. Again, there is enough space to manage the sysupgrade image.

If, in your context, the total space available, i.e. memory + space /tmp is not enough, it’s possible to try the following actions — delete what is not useless:

  • package list files:
    # rm -r /tmp/opkg-lists/
  • caches:
    # sync && echo 3 > /proc/sys/vm/drop_caches
  • the following wifi drivers:
    # rm /etc/modules.d/*{80211,ath9k,b43}*
  • and, finally, check that there are no symbolic links into /etc/modules.d; if, it’s the cas, delete-them. This which will free up RAM at the next startup. And reboot before the sysupgrade process.

SSH Remote Host Identification

When connecting again via SSH, it is normally possible that you will get the following error message:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:0atP7BnQQ98EVJciOCBDYAUD245lKm2tbau8BgWMpQ0.
Please contact your system administrator.
Add correct host key in /home/you/.ssh/known_hosts to get rid of this message.
Offending RSA key in /home/you/.ssh/known_hosts:93
  remove with:
  ssh-keygen -f "/home/you/.ssh/known_hosts" -R "192.168.xyz.1"
ED25519 host key for 192.168.xyz.1 has changed and you have requested strict checking.
Host key verification failed.

DO NOT PANIC! It’s a normal process.
We changed the version, and the identification host, too.
Apply the ssh-keygen command, like wroted, to delete the older. And, after retry your connection.

Warning

Migration to ≥ 21.02.*

Warning

You chose to migrate at the 21.02.* version from v19.7.*, it’s fine!

But, you need to connect on the WebUI, and click on the “Interfaces” menu. The interface will propose you to migrate the configuration, do-it now!

And after, reboot!


Documentations

As you can read in the documentation of the OpenWRT wiki, there are other methods; the one I propose seems the most relevant…