177 lines
8.4 KiB
Markdown
177 lines
8.4 KiB
Markdown
# Admin-pushed firmware update (experimental `-ota` images)
|
|
|
|
> [!WARNING]
|
|
> Part of an experimental fork, not meant for regular users: its releases
|
|
> are a rolling release on MeshCore `dev` (see the [README](../README.md)).
|
|
|
|
The `*-ota` and `*-observer-ota` images of this fork's releases can take a new
|
|
firmware from their admin -- over the USB or BLE console, or **over the mesh,
|
|
as remote admin CLI**. Nothing changes in the bootloader and there is no second
|
|
bank: the node stores the package in free flash, checks all of it, and only on
|
|
an explicit `fw apply` rewrites itself from a routine running in RAM.
|
|
|
|
The point is the **delta**: a package against the image the node runs is
|
|
usually a few kB, so it goes through LoRa in minutes. A full image (~300 kB)
|
|
goes only over USB or BLE.
|
|
|
|
**Rolling release on `dev`, for experienced users.** These `-ota` images track
|
|
MeshCore `dev`, merged continuously, so a new release lands roughly twice a day
|
|
and each delta stays small. There is no stable channel, no second bank, and no
|
|
automatic rollback: a bad image that still boots keeps running until you push the
|
|
next one, and one that will not boot needs USB/BLE DFU in hand. Run it only on
|
|
nodes you can reach, and try every release on one within reach first.
|
|
|
|
## What a release carries
|
|
|
|
For every `<board>_<role>-ota` / `-observer-ota` image:
|
|
|
|
| file | what |
|
|
|---|---|
|
|
| `<image>-<release>.zip` / `.uf2` | the image, flashed the usual way (first time) |
|
|
| `<image>-<release>.mcfw` | full package: USB/BLE console only |
|
|
| `<image>-<release>.from-<earlier>.mcfw` | delta from an earlier release's same image |
|
|
| `manifest.json` | image sha256, target, build epoch, packages, deltas by base |
|
|
| `SHA256SUMS`, `SHA256SUMS.sig` | checksums, signed by `meshcore-ota@tth-pub` |
|
|
|
|
Check a download:
|
|
|
|
ssh-keygen -Y verify -f tools/ota_allowed_signers -I meshcore-ota@tth-pub \
|
|
-n meshcore-ota -s SHA256SUMS.sig < SHA256SUMS
|
|
sha256sum -c --ignore-missing SHA256SUMS
|
|
|
|
`tools/ota_pack.py fetch <image> --base <sha8>` does both and picks the delta
|
|
for the node's image (`--full` for the full package, `--release <tag>` for
|
|
another than the latest).
|
|
|
|
## On the node
|
|
|
|
```
|
|
fw status state, battery, the running image's size and sha256 prefix
|
|
fw ver target and build epoch a package for this image needs
|
|
fw begin <bytes> [sid] start a transfer
|
|
fw w <offset> <base64> one 96 B chunk (text)
|
|
fw missing what is still missing
|
|
fw verify queue the full check, poll fw status
|
|
fw apply battery gate, then rewrite and reboot
|
|
fw apply downgrade local console only: an older (or the same) image
|
|
fw abort
|
|
```
|
|
|
|
What the node checks before it touches its own image:
|
|
|
|
1. The **manifest**, as soon as it is in: an Ed25519 signature by the key the
|
|
image was built with (`[ota] pubkey`), the target (board + variant: an
|
|
`-observer-ota` node takes no `-ota` package), the size, for a delta the
|
|
base (the sha256 of the image it runs), and the build epoch.
|
|
2. Over the mesh: **only a delta, and only a newer build.** An older or equal
|
|
one is refused right after the manifest; a full image is refused when the
|
|
transfer begins. Locally a full image is fine, and an older one takes
|
|
`fw apply downgrade`.
|
|
3. `fw verify`: the payload hash, and a delta applied **dry** -- the same code
|
|
the real run uses, with the flash untouched -- must produce the signed
|
|
image hash.
|
|
4. `fw apply`: battery at 3.8 V or more unless the node runs on USB. The
|
|
bootloader settings get the new image's CRC **first**, so a power cut during
|
|
the rewrite leaves the node in the bootloader (back with USB/BLE DFU), not
|
|
in half an image.
|
|
|
|
## When an update goes wrong
|
|
|
|
No second bank, no automatic rollback. The checks above stop a *corrupt* package,
|
|
but firmware that passes them and still boots into a broken state keeps running.
|
|
Two watchdogs catch that. The deaf-receiver one is **off by default** -- turn it
|
|
on for nodes you push to; the TX stall one is on in repeaters:
|
|
|
|
set radio.silence <min> re-arm the receiver, then reboot, if it hears nothing
|
|
for <min> (persisted, off by default); catches an OTA
|
|
that boots with a dead radio
|
|
set radio.txstall <min> the same when the TX queue stalls (repeater only, on
|
|
at 15 min by default, a change holds until reboot)
|
|
get radio.reboots watchdog reboots so far (survives power loss)
|
|
get boot why the board last reset, plus the hardfault record
|
|
|
|
Details: `docs/cli_commands.md`, "Radio silence watchdog" and "TX stall watchdog".
|
|
|
|
A node that will not boot at all has to be reflashed over USB/BLE DFU. After an
|
|
`fw apply` over **BLE** the bond can drop (a supervision timeout while the node
|
|
reboots) -- re-pair from the same host; it is not a brick.
|
|
|
|
## Tools
|
|
|
|
```bash
|
|
# a delta for the node's image, from the latest release
|
|
tools/ota_pack.py fetch Xiao_nrf52_repeater-ota --base <sha8 from fw status>
|
|
|
|
# over the mesh, through a companion (USB port or tcp://host:port), as admin
|
|
tools/ota_mesh_push.py /dev/serial/by-id/<companion> <node pubkey prefix> pkg.mcfw --apply
|
|
|
|
# over the node's own USB console
|
|
tools/ota_push.py /dev/serial/by-id/<node> pkg.mcfw --apply
|
|
```
|
|
|
|
### From a phone
|
|
|
|
The [tth fork of meshcore-open](https://git.rfa.cz/tth-pub/meshcore-open)
|
|
(Android, Obtainium) has it in the repeater management of an admin: Firmware
|
|
update (OTA). With no file chosen it reads the node's target and image and
|
|
takes the delta from the latest release (checked against `SHA256SUMS`); the
|
|
side channel below is a switch there.
|
|
|
|
### Off the shared channel: `--side`
|
|
|
|
tools/ota_mesh_push.py <companion> <node> pkg.mcfw --apply \
|
|
--side 869.525,62.5,7,5 --duty 10 --gap 1
|
|
|
|
Login, `fw status` and `get radio` still go over the mesh; then the node gets
|
|
`tempradio <side>,3`, the companion retunes to the same preset, logs in again
|
|
zero-hop, and the chunks go there. The window grows to `--window` (15 min) once
|
|
the link is proven and is renewed while the transfer runs. The node comes back
|
|
by itself when the window runs out, on `fw apply` (the reboot), or when the
|
|
tool sends it back with `tempradio <its preset>,1`. Nothing new in the firmware:
|
|
any node with `tempradio` takes it.
|
|
|
|
- The companion must be in **direct** range of the node, and it must be a
|
|
companion **firmware** that retunes its own radio. An openHop virtual
|
|
companion answers `set_radio` with OK and stays put; the tool checks that
|
|
and stops, and the node returns within 3 minutes.
|
|
- The companion's preset is put back at the end -- unless the tool is killed
|
|
hard; then `set_radio` it back yourself.
|
|
- The example preset sits inside 869.4-869.65 MHz (10 % duty) between the CZ
|
|
mesh (869.432/62.5) and EU Narrow (869.618/62.5). The node keeps its TX
|
|
power on the side channel: pick a band where that power is allowed.
|
|
|
|
The transfer in both tools is the OTA engine of the
|
|
[tth.meshcore Ansible collection](https://git.rfa.cz/tth-pub/meshcore-ansible)
|
|
(`plugins/module_utils/meshcore_ota_engine.py`) -- its `meshcore_fw_ota` module
|
|
runs the same code, so a fix lands everywhere at once. The tools need only its
|
|
files, not Ansible: an installed collection
|
|
(`ansible-galaxy collection install git+https://git.rfa.cz/tth-pub/meshcore-ansible.git`),
|
|
a checkout named by `MESHCORE_ANSIBLE`, or one next to this repository
|
|
(`tools/ota_collection.py` prints which one it uses).
|
|
|
|
`ota_mesh_push.py` needs the `meshcore` Python library. It logs in as admin
|
|
(password from `[secrets] admin_pw` of `platformio.local.ini`, or asked for),
|
|
sends binary chunks with a pause between them, asks which are missing and
|
|
resends only those, and keeps an airtime budget (`--duty`, default 1 % per
|
|
hour: the 865-868 MHz band). A refused package costs the two manifest chunks,
|
|
not the whole transfer. Check `get radio` on both ends first: this transmits.
|
|
|
|
## Your own builds
|
|
|
|
A build with `-D FW_OTA=1` trusts `[ota] pubkey` of `platformio.ini` -- the
|
|
release key. To sign your own packages, set your key in the gitignored
|
|
`platformio.local.ini`:
|
|
|
|
```ini
|
|
[ota]
|
|
pubkey = <tools/ota_pack.py pubkey --key your.key>
|
|
```
|
|
|
|
and pack with `tools/ota_pack.py pack|diff <env> --key your.key`. Target and
|
|
build epoch come from the ELF, never from the command line. Keep every image
|
|
you flash: a delta needs the exact bytes the node runs as its base.
|
|
|
|
Cost: about 10 kB of RAM on a repeater, the applier code in RAM, a 32 kB
|
|
buffer while a full package is verified or applied. Classic images (without
|
|
`-ota`) have none of it.
|