meshcore_fw_ota: OTA firmware update of a meshcore-tth repeater, over RF or its BLE console
ci / test (push) Successful in 2m21s
ci / test (push) Successful in 2m21s
Picks the package of a release that fits the image `fw status` reports (changed=false when the node runs it already, no_package when none fits or the node has no `fw`), transfers it -- over RF as binary chunks with a bitmap of the missing ones and text chunks as the fallback, inside an airtime budget; over the BLE console as text, a full image too -- then verifies and applies. Port of tools/ota_mesh_push.py / ota_push.py of meshcore-tth; radio-free parts in module_utils/meshcore_ota.py with unit tests (the airtime formula they caught is fixed in the tool as well). Link.cli takes retries, timeout, quiet and a before_send hook. Tested on two live repeaters forced onto their current image (BLE console; RF through a companion: 'current', 0.6 s airtime) and with a package of another base (no_package). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d6f2d5cf28
commit
891be3afbe
7 files changed
+966
-5
No files matched your search
@@ -46,10 +46,20 @@ z AUR, jinde venv a `ansible_python_interpreter` na něj.
|
||||
| `meshcore_filter_hash` | zákaz přeposílání kanálů podle hashe | meshcore-tth |
|
||||
| `meshcore_filter` | stáří zpráv a dry-run filtru | meshcore-tth |
|
||||
| `meshcore_fairness` | okno na adverty podle původce a dry-run | meshcore-tth |
|
||||
| `meshcore_fw_ota` | OTA aktualizace firmwaru z balíčků releasu (`.mcfw`): vybere deltu podle `fw status`, přes RF v limitu airtime (`duty`), přes BLE konzoli uzlu bez vysílání; `no_package` = žádná delta nesedí | meshcore-tth, obraz `-ota` |
|
||||
|
||||
Moduly pro meshcore-tth na běžném firmwaru skončí chybou nebo `skipped`
|
||||
(`Unknown command`), nic jiného nepošlou.
|
||||
|
||||
`meshcore_fw_ota` čte balíčky na hostu, kde běží (`packages`: soubor nebo
|
||||
adresář s balíčky jednoho releasu). Stažení a ověření releasu (podepsaný
|
||||
`SHA256SUMS`) je na volajícím, uzel ale každý balíček kontroluje sám (podpis
|
||||
Ed25519, cíl, základní obraz, nesmí být starší) a cizí odmítne po prvních
|
||||
kouscích. Přes RF jdou jen delty; dlouhý přenos pouštějte s `async`. Výpadek
|
||||
napájení během ~30 s přepisu po `apply` nechá uzel v bootloaderu (polovičatý
|
||||
obraz se nespustí, oprava je DFU na místě), pod hranicí baterie uzel `apply`
|
||||
odmítne.
|
||||
|
||||
## Jak to funguje
|
||||
|
||||
- Každý repeater je host v inventáři, úlohy běží na hostu s companionem
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
"""The text console of a meshcore-tth repeater over BLE (Nordic UART Service).
|
||||
|
||||
A local console: no login, no RF, nothing on air -- the OTA module prefers it
|
||||
wherever a bonded host hears the node. The node echoes each command and answers
|
||||
with a line ' -> <reply>'. Bond the node once on the host the module runs on
|
||||
(the console is SECMODE_ENC_WITH_MITM); the module never takes the PIN, so it
|
||||
cannot end up in a log.
|
||||
|
||||
Learned on the masts:
|
||||
* BLE connects are flaky: a TimeoutError on a node that answers the next try is
|
||||
normal, hence the retries;
|
||||
* a connected central stops the node advertising -- stop whatever holds it
|
||||
(a bridge) and `bluetoothctl disconnect` it first;
|
||||
* a NUS write carries at most ATT_MTU - 3 bytes and BlueZ does not split it;
|
||||
* do not send a bare newline to wake it: over BLE that reply comes late and is
|
||||
taken for the answer to the next command.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
import time
|
||||
|
||||
try:
|
||||
from bleak import BleakClient, BleakScanner
|
||||
|
||||
HAS_BLEAK = True
|
||||
except ImportError: # reported by the module via missing_required_lib
|
||||
HAS_BLEAK = False
|
||||
|
||||
NUS_RX = "6e400002-b5a3-f393-e0a9-e50e24dcca9e" # write (host -> node)
|
||||
NUS_TX = "6e400003-b5a3-f393-e0a9-e50e24dcca9e" # notify (node -> host)
|
||||
REPLY = re.compile(rb"-> ?(.*?)\r?\n")
|
||||
|
||||
|
||||
class ConsoleError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class BleConsole:
|
||||
def __init__(self, address, connect_tries=5):
|
||||
self.address = address
|
||||
self.connect_tries = connect_tries
|
||||
self.client = None
|
||||
self.buf = bytearray()
|
||||
self.chunk = 20
|
||||
self.log = []
|
||||
|
||||
async def open(self):
|
||||
last = None
|
||||
for attempt in range(self.connect_tries):
|
||||
try:
|
||||
dev = await BleakScanner.find_device_by_address(self.address, timeout=20)
|
||||
if dev is None:
|
||||
raise ConsoleError("not advertising (a central still connected?)")
|
||||
client = BleakClient(dev, timeout=30)
|
||||
await client.connect()
|
||||
await client.start_notify(NUS_TX, lambda _h, data: self.buf.extend(data))
|
||||
self.client = client
|
||||
self.chunk = max(20, int(getattr(client, "mtu_size", 23)) - 3)
|
||||
self.log.append(f"BLE console {self.address} open (attempt {attempt + 1}, mtu {self.chunk + 3})")
|
||||
return
|
||||
except Exception as e: # noqa: BLE001 -- bleak/BlueZ raise many kinds, all mean "try again"
|
||||
last = e
|
||||
self.log.append(f"BLE connect {attempt + 1}/{self.connect_tries}: {type(e).__name__}: {e}")
|
||||
await asyncio.sleep(4)
|
||||
raise ConsoleError(f"cannot open the BLE console of {self.address}: {last}")
|
||||
|
||||
async def close(self):
|
||||
if self.client is not None:
|
||||
try:
|
||||
await self.client.disconnect()
|
||||
except Exception as e: # noqa: BLE001 -- after `fw apply` the node is gone already
|
||||
self.log.append(f"BLE disconnect: {type(e).__name__}")
|
||||
|
||||
async def cmd(self, line, timeout=5.0, quiet=False):
|
||||
"""One console command -> the reply after '-> '."""
|
||||
self.buf.clear()
|
||||
data = line.encode() + b"\r\n"
|
||||
for i in range(0, len(data), self.chunk):
|
||||
await self.client.write_gatt_char(NUS_RX, data[i : i + self.chunk], response=False)
|
||||
await asyncio.sleep(0.05)
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
m = REPLY.search(bytes(self.buf))
|
||||
if m:
|
||||
text = m.group(1).decode(errors="replace").strip()
|
||||
if not quiet:
|
||||
self.log.append(f"$ {line} -> {text}")
|
||||
return text
|
||||
await asyncio.sleep(0.02)
|
||||
shown = line if len(line) <= 24 else line[:24] + "..."
|
||||
raise ConsoleError(f"no reply to `{shown}` on the BLE console")
|
||||
@@ -187,29 +187,38 @@ class Link:
|
||||
pfx = (msg.get("pubkey_prefix") or "").lower()
|
||||
return bool(pfx) and self.target_pk.startswith(pfx)
|
||||
|
||||
async def cli(self, cmd, expect):
|
||||
async def cli(self, cmd, expect, retries=None, timeout=None, quiet=False, before_send=None):
|
||||
"""Send one CLI command and return the reply text that matches `expect` (regex).
|
||||
|
||||
A reply that does not match (a late answer to an earlier command, or
|
||||
another node) is drained and the command is resent, up to `retries`.
|
||||
quiet: no trace line on success (hundreds of firmware chunks).
|
||||
before_send: awaited before every attempt (the OTA airtime budget).
|
||||
"""
|
||||
retries = self.retries if retries is None else retries
|
||||
timeout = self.reply_timeout if timeout is None else timeout
|
||||
# never log a secret the command carries (the trace is returned to Ansible)
|
||||
shown = cmd
|
||||
for secret_cmd in ("password ", "ble pin "):
|
||||
if cmd.startswith(secret_cmd):
|
||||
shown = secret_cmd + "***"
|
||||
for attempt in range(1 + self.retries):
|
||||
if quiet and len(shown) > 24:
|
||||
shown = shown[:24] + "..."
|
||||
for attempt in range(1 + retries):
|
||||
self.inbox.clear()
|
||||
if before_send is not None:
|
||||
await before_send()
|
||||
if self.companion:
|
||||
await self.send_cli_command(cmd)
|
||||
else:
|
||||
await self.mc.commands.send_cmd(self.contact, cmd)
|
||||
deadline = time.monotonic() + self.reply_timeout
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
for msg in list(self.inbox):
|
||||
text = (msg.get("text") or "").strip()
|
||||
if self._from_target(msg) and re.search(expect, text):
|
||||
self.log.append(f"$ {shown} -> {self._redact(text)}")
|
||||
if not quiet:
|
||||
self.log.append(f"$ {shown} -> {self._redact(text)}")
|
||||
# The reply carries the NODE's clock: what fw_facts reports as skew.
|
||||
self.last_reply_ts = msg.get("sender_timestamp")
|
||||
self.last_reply_at = time.time()
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
"""Radio-free parts of the OTA firmware update of meshcore-tth (src/helpers/FwUpdate).
|
||||
|
||||
Kept apart from meshcore_fw_ota so they can be unit-tested: which package fits
|
||||
the node, the chunk framing, and what a transfer costs on air. The protocol
|
||||
itself is the reference client's, tools/ota_mesh_push.py of meshcore-tth.
|
||||
|
||||
A package (`.mcfw`, tools/ota_pack.py) is a 256 B manifest and a payload:
|
||||
0 magic "MCFW" | 4 format 1 (full image) or 2 (delta) | 5 layout
|
||||
16 raw_sha256 = the image it installs | 160 base_sha256 (delta: the image it starts from)
|
||||
192 Ed25519 signature over 0..191 -- checked by the node, not here
|
||||
"""
|
||||
|
||||
import math
|
||||
import re
|
||||
import time
|
||||
|
||||
CHUNK = 96 # FW_OTA_CHUNK: text chunk (`fw w <off> <base64>`), answered "ok"
|
||||
BCHUNK = 128 # FW_OTA_BCHUNK: binary chunk (REQ_TYPE_FW_OTA 'w'), unanswered
|
||||
UNIT = 32 # FW_OTA_UNIT: what the missing-units bitmap counts
|
||||
MANIFEST = 256
|
||||
REQ_FW = 0xF0 # REQ_TYPE_FW_OTA
|
||||
# The companion takes host frames of at most 176 B, and a binary request frame
|
||||
# is 1 (cmd) + 32 (pubkey) + payload: 160 B chunks were dropped, every one.
|
||||
MAX_FRAME = 176
|
||||
REPLY_TEXT = 40 # typical length of a reply, for the airtime estimate
|
||||
|
||||
|
||||
class Package:
|
||||
"""One `.mcfw` file: name, bytes, and what the manifest says about it."""
|
||||
|
||||
def __init__(self, name, data):
|
||||
if data[:4] != b"MCFW" or len(data) < MANIFEST:
|
||||
raise ValueError(f"{name}: not an OTA package (no MCFW manifest)")
|
||||
self.name = name
|
||||
self.data = data
|
||||
self.format = data[4]
|
||||
self.image_sha = data[16:48].hex()
|
||||
self.base_sha = data[160:192].hex() if self.format == 2 else None
|
||||
self.version = data[128:156].rstrip(b"\0").decode(errors="replace")
|
||||
|
||||
@property
|
||||
def delta(self):
|
||||
return self.format == 2
|
||||
|
||||
|
||||
def parse_status(text):
|
||||
"""`fw status` -> (base image size, sha256 prefix) or None (no `fw`, not an -ota build).
|
||||
|
||||
Reply: '> idle, batt 4137mV (min 3800), free 352256 B, base 456492 fe748410, delta'.
|
||||
"""
|
||||
m = re.search(r"\bbase (\d+) ([0-9a-f]{8})", text or "")
|
||||
return (int(m.group(1)), m.group(2)) if m else None
|
||||
|
||||
|
||||
def pick(packages, base, full_ok):
|
||||
"""('current', pkg) | ('delta', pkg) | ('full', pkg) | ('none', None).
|
||||
|
||||
current: the node already runs the image these packages install. delta: the
|
||||
one that starts from the node's image. full: no delta fits and a full image
|
||||
may go (a local console; the node refuses one over the mesh).
|
||||
"""
|
||||
full = None
|
||||
for p in packages:
|
||||
if p.image_sha.startswith(base):
|
||||
return "current", p
|
||||
for p in packages:
|
||||
if p.delta and p.base_sha.startswith(base):
|
||||
return "delta", p
|
||||
if not p.delta:
|
||||
full = p
|
||||
if full is not None and full_ok:
|
||||
return "full", full
|
||||
return "none", None
|
||||
|
||||
|
||||
def crc16(data, crc=0xFFFF):
|
||||
"""Nordic crc16_compute (CCITT, init 0xFFFF) -- FwUpdate.cpp crc16_update."""
|
||||
for b in data:
|
||||
crc = ((crc >> 8) | (crc << 8)) & 0xFFFF
|
||||
crc ^= b
|
||||
crc ^= (crc & 0xFF) >> 4
|
||||
crc ^= (crc << 12) & 0xFFFF
|
||||
crc ^= ((crc & 0xFF) << 5) & 0xFFFF
|
||||
return crc
|
||||
|
||||
|
||||
def w_payload(sid, off, chunk):
|
||||
"""REQ_TYPE_FW_OTA 'w': [F0]['w'][sid u16][off u32][len u8][crc16][data].
|
||||
|
||||
The crc binds the data to this session and offset (FwUpdate::handleBinary),
|
||||
so late chunks of an earlier transfer are dropped.
|
||||
"""
|
||||
head = sid.to_bytes(2, "little") + off.to_bytes(4, "little") + bytes([len(chunk)])
|
||||
return bytes([REQ_FW, ord("w")]) + head + crc16(chunk, crc16(head)).to_bytes(2, "little") + chunk
|
||||
|
||||
|
||||
def parse_missing(data):
|
||||
"""Reply to REQ_TYPE_FW_OTA 'm' -> (state, [missing unit indices]) or None.
|
||||
|
||||
[state u8][2 bytes][count u16][count x unit u16]; state 3 = refused, 4 = busy.
|
||||
"""
|
||||
if len(data) < 5:
|
||||
return None
|
||||
cnt = int.from_bytes(data[3:5], "little")
|
||||
return data[0], [int.from_bytes(data[5 + 2 * i : 7 + 2 * i], "little") for i in range(cnt)]
|
||||
|
||||
|
||||
def text_missing(reply):
|
||||
"""`fw missing` -> chunk offsets ([] for 'none'), or None for a reply that is no list."""
|
||||
if not reply.startswith(">"):
|
||||
return None
|
||||
body = reply[1:].strip()
|
||||
return [] if body == "none" else [int(x) for x in body.split()]
|
||||
|
||||
|
||||
# ── airtime ─────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def lora_airtime(n, sf, bw_khz, cr, preamble=16):
|
||||
"""Seconds on air for an n-byte LoRa frame (explicit header, CRC on)."""
|
||||
tsym = (2**sf) / (bw_khz * 1000)
|
||||
de = 1 if tsym > 0.016 else 0
|
||||
# Semtech AN1200.13: ceil(...) * (CR + 4), CR = 1..4 for 4/5..4/8
|
||||
payload = 8 + max(math.ceil((8 * n - 4 * sf + 28 + 16) / (4 * (sf - 2 * de))) * (cr + 4), 0)
|
||||
return (preamble + 4.25) * tsym + payload * tsym
|
||||
|
||||
|
||||
def text_frame_len(text):
|
||||
"""Direct text frame: header+path_len, dest/src hash, MAC, timestamp+flags+text in AES blocks."""
|
||||
return 6 + 16 * math.ceil((5 + len(text.encode())) / 16)
|
||||
|
||||
|
||||
def req_frame_len(payload_len):
|
||||
"""Direct REQ: header+path_len, dest/src hash, MAC, timestamp+payload in AES blocks."""
|
||||
return 6 + 16 * math.ceil((4 + payload_len) / 16)
|
||||
|
||||
|
||||
def hops_of(contact):
|
||||
"""out_path_len: low 6 bits = hops, top 2 = hash size - 1; -1 = flood (counted as 0)."""
|
||||
v = int(contact.get("out_path_len", -1))
|
||||
return 0 if v < 0 else v & 0x3F
|
||||
|
||||
|
||||
class Bucket:
|
||||
"""Airtime budget: a token bucket of `duty` % of an hour.
|
||||
|
||||
It counts what the transfer puts on the CHANNEL -- request and reply, each
|
||||
times (hops + 1), since every repeater on the path sends both again.
|
||||
`clock` is injectable for the tests.
|
||||
"""
|
||||
|
||||
def __init__(self, duty, clock=time.monotonic):
|
||||
if not 0 < duty <= 100:
|
||||
raise ValueError(f"duty {duty} % is not in (0, 100]")
|
||||
self.rate = duty / 100.0
|
||||
self.cap = self.rate * 3600
|
||||
self.level = self.cap
|
||||
self.clock = clock
|
||||
self.t = clock()
|
||||
self.used = 0.0
|
||||
|
||||
def wait_for(self, secs):
|
||||
"""Seconds to wait before `secs` of airtime may be spent (0 = now); spends it when 0."""
|
||||
now = self.clock()
|
||||
self.level = min(self.cap, self.level + (now - self.t) * self.rate)
|
||||
self.t = now
|
||||
if self.level >= secs:
|
||||
self.level -= secs
|
||||
self.used += secs
|
||||
return 0.0
|
||||
return (secs - self.level) / self.rate
|
||||
@@ -0,0 +1,533 @@
|
||||
# Copyright: (c) 2026, TeTeHacko <tth@rfa.cz>
|
||||
# GNU General Public License v3.0+ (see LICENSE or https://www.gnu.org/licenses/gpl-3.0.txt)
|
||||
DOCUMENTATION = r"""
|
||||
module: meshcore_fw_ota
|
||||
short_description: OTA firmware update of a meshcore-tth repeater, over RF or its BLE console
|
||||
author: TeTeHacko (@TeTeHacko)
|
||||
description:
|
||||
- Updates a repeater that runs an C(-ota) image of meshcore-tth (the C(fw)
|
||||
commands of src/helpers/FwUpdate) from the packages of a release (C(.mcfw),
|
||||
tools/ota_pack.py), and applies it. The node checks each package itself
|
||||
(Ed25519 signature, target, base image, not older) and refuses a wrong one
|
||||
after its first chunks.
|
||||
- Reads C(fw status) and picks the package that fits the image the node runs.
|
||||
C(changed=false) when the node already runs the image, C(no_package=true)
|
||||
(not a failure) when no package starts from its image or the node has no
|
||||
C(fw) at all -- then the caller falls back to a full DFU.
|
||||
- Over RF (a companion transport, as the other modules) only deltas go, the
|
||||
node refuses a full image there. Chunks are binary requests (128 B, no
|
||||
reply, missing ones found by a bitmap query), with text chunks as the
|
||||
fallback, paced by an airtime budget (C(duty)). A 28 kB delta over two hops
|
||||
took 40 min at 3 %.
|
||||
- Over the node's BLE console (C(transport.type=ble_console)) nothing goes on
|
||||
air and a full image may go too. Prefer it wherever a host with a bond
|
||||
hears the node.
|
||||
- A power cut during the ~30 s rewrite after C(apply) leaves the node in its
|
||||
bootloader (no half image runs; recovery = DFU on site). The node refuses
|
||||
to apply below its battery gate.
|
||||
- In check mode it reads C(fw status) and says which package would go.
|
||||
options:
|
||||
transport:
|
||||
description:
|
||||
- "Companion to talk through (see meshcore_acl), or the node's own BLE
|
||||
console C({type: ble_console, address}), bonded on the host the module
|
||||
runs on."
|
||||
type: dict
|
||||
required: true
|
||||
target: {description: Full 64-hex pubkey of the node (not needed for ble_console)., type: str}
|
||||
name: {description: Name for the contact if the companion has to create it., type: str}
|
||||
password: {description: Admin password (empty = blank login; not used on ble_console)., type: str, default: ""}
|
||||
packages:
|
||||
description:
|
||||
- A C(.mcfw) package, or a directory with the packages of one release
|
||||
(the full image and its deltas), on the host the module runs on.
|
||||
type: path
|
||||
required: true
|
||||
apply: {description: Apply after a good C(fw verify) (else the package stays staged)., type: bool, default: true}
|
||||
duty:
|
||||
description:
|
||||
- RF only. Airtime budget in percent per hour (token bucket). Counts what
|
||||
the transfer puts on the channel, request and reply times (hops + 1).
|
||||
869.4-869.65 MHz allows 10 %, other sub-bands 1 %.
|
||||
type: float
|
||||
default: 3
|
||||
gap: {description: RF only. Pause in seconds after every reply., type: float, default: 5}
|
||||
resume:
|
||||
description:
|
||||
- Continue a transfer that stopped (the node keeps what it got until the
|
||||
next C(fw begin) or a reboot) -- no C(fw begin), only what C(fw missing)
|
||||
lists.
|
||||
type: bool
|
||||
default: false
|
||||
apply_wait: {description: Seconds to wait after C(fw apply) before reading the new version., type: int, default: 60}
|
||||
requirements: [meshcore (RF), bleak (ble_console)]
|
||||
"""
|
||||
|
||||
EXAMPLES = r"""
|
||||
- name: Update over RF through the companion next to the mast
|
||||
tth.meshcore.meshcore_fw_ota:
|
||||
transport: {type: tcp, host: 127.0.0.1, port: 5000, expect_freq: 869.432}
|
||||
target: "{{ mc_target }}"
|
||||
password: "{{ vault_meshcore_admin_password }}"
|
||||
packages: /tmp/ota/ # the .mcfw files of one release
|
||||
duty: 3
|
||||
async: 7200
|
||||
poll: 30
|
||||
register: ota
|
||||
|
||||
- name: The same over the node's BLE console (bonded on this host)
|
||||
tth.meshcore.meshcore_fw_ota:
|
||||
transport: {type: ble_console, address: "AA:BB:CC:DD:EE:FF"}
|
||||
packages: /tmp/ota/
|
||||
|
||||
- name: No package fits the node's image -- full DFU instead
|
||||
ansible.builtin.debug:
|
||||
msg: "{{ ota.base }}: no delta"
|
||||
when: ota.no_package
|
||||
"""
|
||||
|
||||
RETURN = r"""
|
||||
state:
|
||||
description: current | no_package | would_update | staged | applied
|
||||
returned: always
|
||||
type: str
|
||||
package:
|
||||
description: Name of the package that went (or would go).
|
||||
returned: when one fits
|
||||
type: str
|
||||
kind:
|
||||
description: delta | full
|
||||
returned: when one fits
|
||||
type: str
|
||||
base:
|
||||
description: Size and sha256 prefix of the image the node ran.
|
||||
returned: when the node has fw
|
||||
type: str
|
||||
no_package:
|
||||
description: No package starts from the node's image, or the node has no fw.
|
||||
returned: always
|
||||
type: bool
|
||||
version:
|
||||
description: Version the package installs.
|
||||
returned: when one fits
|
||||
type: str
|
||||
version_after:
|
||||
description: The `ver` reply after the apply, when the node answered.
|
||||
returned: when applied
|
||||
type: str
|
||||
airtime:
|
||||
description: Channel airtime of the transfer in seconds (RF).
|
||||
returned: over RF
|
||||
type: float
|
||||
log:
|
||||
description: Trace of the exchange (no secrets).
|
||||
returned: always
|
||||
type: list
|
||||
elements: str
|
||||
"""
|
||||
import asyncio
|
||||
import base64
|
||||
import os
|
||||
import random
|
||||
|
||||
from ansible.module_utils.basic import AnsibleModule, missing_required_lib
|
||||
from ansible_collections.tth.meshcore.plugins.module_utils.meshcore_console import (
|
||||
HAS_BLEAK,
|
||||
BleConsole,
|
||||
ConsoleError,
|
||||
)
|
||||
from ansible_collections.tth.meshcore.plugins.module_utils.meshcore_link import HAS_MESHCORE, Link, LinkError, run
|
||||
from ansible_collections.tth.meshcore.plugins.module_utils.meshcore_ota import (
|
||||
BCHUNK,
|
||||
CHUNK,
|
||||
MANIFEST,
|
||||
MAX_FRAME,
|
||||
REPLY_TEXT,
|
||||
REQ_FW,
|
||||
UNIT,
|
||||
Bucket,
|
||||
Package,
|
||||
hops_of,
|
||||
lora_airtime,
|
||||
parse_missing,
|
||||
parse_status,
|
||||
pick,
|
||||
req_frame_len,
|
||||
text_frame_len,
|
||||
text_missing,
|
||||
w_payload,
|
||||
)
|
||||
|
||||
try:
|
||||
from meshcore import EventType
|
||||
except ImportError: # HAS_MESHCORE says so
|
||||
EventType = None
|
||||
|
||||
# what each reply looks like -- a late answer to an earlier command must not pass
|
||||
EXPECT = {
|
||||
"status": r"^(>|ERR|Err|Unknown|\?)",
|
||||
"begin": r"^(OK|ERR|Err)",
|
||||
"w": r"^(ok|ERR|Err)",
|
||||
"list": r"^(>|ERR|Err)",
|
||||
"apply": r"^(OK|ERR|Err)",
|
||||
"ver": r"\(Build: ",
|
||||
}
|
||||
|
||||
|
||||
class OtaError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def load_packages(path):
|
||||
names = sorted(f for f in os.listdir(path) if f.endswith(".mcfw")) if os.path.isdir(path) else [path]
|
||||
out = []
|
||||
for n in names:
|
||||
full = os.path.join(path, n) if os.path.isdir(path) else n
|
||||
with open(full, "rb") as f:
|
||||
out.append(Package(os.path.basename(n), f.read()))
|
||||
if not out:
|
||||
raise OtaError(f"no .mcfw package in {path}")
|
||||
return out
|
||||
|
||||
|
||||
class Console:
|
||||
"""`fw` commands over the BLE console: local, nothing on air."""
|
||||
|
||||
rf = False
|
||||
|
||||
def __init__(self, transport):
|
||||
self.con = BleConsole(transport["address"])
|
||||
self.log = self.con.log
|
||||
self.airtime = None
|
||||
|
||||
async def open(self):
|
||||
await self.con.open()
|
||||
|
||||
async def close(self):
|
||||
await self.con.close()
|
||||
|
||||
async def ask(self, line, kind, timeout=5.0, tries=3, quiet=False):
|
||||
del kind # a console answers in order
|
||||
for attempt in range(tries):
|
||||
try:
|
||||
return await self.con.cmd(line, timeout=timeout, quiet=quiet)
|
||||
except ConsoleError:
|
||||
if attempt + 1 == tries:
|
||||
raise
|
||||
return ""
|
||||
|
||||
async def after_apply(self, wait):
|
||||
await self.con.close()
|
||||
await asyncio.sleep(wait)
|
||||
await self.con.open()
|
||||
|
||||
|
||||
class Radio:
|
||||
"""`fw` commands as remote admin CLI through a companion, inside an airtime budget."""
|
||||
|
||||
rf = True
|
||||
|
||||
def __init__(self, module):
|
||||
p = module.params
|
||||
self.link = Link(p["transport"], p["target"], p["password"], name=p["name"])
|
||||
self.log = self.link.log
|
||||
self.bucket = Bucket(p["duty"])
|
||||
self.gap = p["gap"]
|
||||
self.radio = None
|
||||
|
||||
@property
|
||||
def airtime(self):
|
||||
return round(self.bucket.used, 1)
|
||||
|
||||
async def open(self):
|
||||
await self.link.open()
|
||||
si = self.link.mc.self_info or {}
|
||||
self.radio = (int(si["radio_sf"]), float(si["radio_bw"]), int(si["radio_cr"]) - 4)
|
||||
await self.spend(lora_airtime(48, *self.radio))
|
||||
await self.login()
|
||||
|
||||
async def login(self):
|
||||
await self.link.login()
|
||||
await self.link.mc.ensure_contacts(follow=True) # the login may have taught it a path
|
||||
self.link.contact = self.link._find_contact() or self.link.contact
|
||||
self.log.append(f"path: {hops_of(self.link.contact)} hops")
|
||||
|
||||
async def close(self):
|
||||
await self.link.close()
|
||||
|
||||
async def spend(self, secs):
|
||||
while True:
|
||||
wait = self.bucket.wait_for(secs)
|
||||
if wait == 0:
|
||||
return
|
||||
await asyncio.sleep(wait)
|
||||
|
||||
def hops(self):
|
||||
return hops_of(self.link.contact or {})
|
||||
|
||||
async def ask(self, line, kind, timeout=20.0, tries=4, quiet=False):
|
||||
one = lora_airtime(text_frame_len(line), *self.radio) + lora_airtime(
|
||||
text_frame_len("x" * REPLY_TEXT), *self.radio
|
||||
)
|
||||
reply = await self.link.cli(
|
||||
line,
|
||||
EXPECT[kind],
|
||||
retries=tries - 1,
|
||||
timeout=timeout,
|
||||
quiet=quiet,
|
||||
before_send=lambda: self.spend(one * (self.hops() + 1)),
|
||||
)
|
||||
if self.gap:
|
||||
await asyncio.sleep(self.gap)
|
||||
return reply
|
||||
|
||||
# ── binary chunks (REQ_TYPE_FW_OTA) ──────────────────────────────────────
|
||||
|
||||
def req_cost(self, payload_len, reply_len=0):
|
||||
t = lora_airtime(req_frame_len(payload_len), *self.radio)
|
||||
if reply_len:
|
||||
t += lora_airtime(req_frame_len(reply_len), *self.radio)
|
||||
return t * (self.hops() + 1)
|
||||
|
||||
async def send_req(self, payload):
|
||||
if 1 + 32 + len(payload) > MAX_FRAME:
|
||||
raise OtaError(f"binary request of {len(payload)} B exceeds the companion frame")
|
||||
ev = await self.link.mc.commands.send(
|
||||
b"\x32" + bytes.fromhex(self.link.target_pk) + payload, [EventType.MSG_SENT, EventType.ERROR]
|
||||
)
|
||||
if ev is None or ev.type != EventType.MSG_SENT:
|
||||
return None
|
||||
tag = ev.payload.get("expected_ack")
|
||||
return tag.hex() if isinstance(tag, (bytes, bytearray)) else str(tag)
|
||||
|
||||
async def write(self, sid, off, chunk):
|
||||
payload = w_payload(sid, off, chunk)
|
||||
c = self.req_cost(len(payload))
|
||||
await self.spend(c)
|
||||
await self.send_req(payload)
|
||||
# no reply comes: give the packet time to go out along the whole path
|
||||
await asyncio.sleep(max(self.gap, c + 1.0))
|
||||
|
||||
async def missing(self, timeout=20.0):
|
||||
"""(state, missing unit indices) or None when the node does not answer."""
|
||||
for _attempt in range(2):
|
||||
await self.spend(self.req_cost(2, 160))
|
||||
fut = asyncio.get_running_loop().create_future()
|
||||
|
||||
def got(ev, fut=fut):
|
||||
if not fut.done():
|
||||
fut.set_result(ev)
|
||||
|
||||
sub = self.link.mc.subscribe(EventType.BINARY_RESPONSE, got)
|
||||
try:
|
||||
tag = await self.send_req(bytes([REQ_FW, ord("m")]))
|
||||
ev = await asyncio.wait_for(fut, timeout)
|
||||
except TimeoutError:
|
||||
ev = None
|
||||
finally:
|
||||
self.link.mc.unsubscribe(sub)
|
||||
if ev is None or (tag and ev.payload.get("tag") != tag):
|
||||
continue
|
||||
st = parse_missing(bytes.fromhex(ev.payload["data"]))
|
||||
if st is None:
|
||||
continue
|
||||
await asyncio.sleep(self.gap)
|
||||
return st
|
||||
return None
|
||||
|
||||
async def settled(self):
|
||||
"""The bitmap once the node is done checking (state 4 = busy). A lost
|
||||
query after a long round must not throw the round away."""
|
||||
lost = 0
|
||||
st = None
|
||||
for _query in range(10):
|
||||
st = await self.missing()
|
||||
if st is None and lost < 4:
|
||||
lost += 1
|
||||
await asyncio.sleep(10)
|
||||
continue
|
||||
if st is None or st[0] != 4:
|
||||
return st
|
||||
await asyncio.sleep(3)
|
||||
return st
|
||||
|
||||
async def after_apply(self, wait):
|
||||
await asyncio.sleep(wait)
|
||||
await self.login()
|
||||
|
||||
|
||||
async def binary_rounds(io, data, sid):
|
||||
"""Binary chunks in rounds; returns the text chunks still to send ([] = all in), or None = no binary."""
|
||||
if await io.missing() is None:
|
||||
io.log.append("node does not answer binary requests: text chunks")
|
||||
return None
|
||||
todo = list(range((len(data) + BCHUNK - 1) // BCHUNK))
|
||||
last_miss = 1 << 30
|
||||
manifest_checked = False
|
||||
for rnd in range(6):
|
||||
for k, i in enumerate(todo):
|
||||
off = i * BCHUNK
|
||||
await io.write(sid, off, data[off : off + BCHUNK])
|
||||
if not manifest_checked and off + BCHUNK >= MANIFEST and k + 1 < len(todo):
|
||||
# the node checks the manifest as soon as it is in: ask before
|
||||
# the rest goes on air, a refused package would cost the round
|
||||
manifest_checked = True
|
||||
st = await io.settled()
|
||||
if st is not None and st[0] == 3:
|
||||
raise OtaError(f"node refused the package: {await io.ask('fw status', 'status')}")
|
||||
st = await io.settled()
|
||||
if st is None:
|
||||
raise OtaError("no answer to the missing-units query (the node keeps what it has: resume)")
|
||||
state, miss = st
|
||||
io.log.append(f"binary round {rnd + 1}: {len(todo)} chunks sent, {len(miss)} units missing")
|
||||
if state == 3:
|
||||
raise OtaError(f"node refused the package: {await io.ask('fw status', 'status')}")
|
||||
if not miss:
|
||||
return []
|
||||
if rnd > 0 and len(miss) >= last_miss:
|
||||
io.log.append("binary chunks do not arrive: text chunks for the rest")
|
||||
return sorted({u * UNIT // CHUNK for u in miss})
|
||||
last_miss = len(miss)
|
||||
todo = sorted({u * UNIT // BCHUNK for u in miss})
|
||||
raise OtaError("binary transfer did not converge in 6 rounds")
|
||||
|
||||
|
||||
async def text_rounds(io, data, todo):
|
||||
"""Text chunks (each answered) until `fw missing` says none."""
|
||||
stuck = 0
|
||||
for rnd in range(40):
|
||||
for i in todo:
|
||||
off = i * CHUNK
|
||||
r = await io.ask(f"fw w {off} {base64.b64encode(data[off : off + CHUNK]).decode()}", "w", quiet=True)
|
||||
if r != "ok":
|
||||
raise OtaError(f"chunk {off}: {r}")
|
||||
reply = await io.ask("fw missing", "list")
|
||||
new = text_missing(reply)
|
||||
io.log.append(f"text round {rnd + 1}: {len(todo)} chunks sent, missing: {reply[:60]}")
|
||||
if new == []:
|
||||
return
|
||||
if new is None:
|
||||
stuck += 1 # a reply that is no list: ask again next round
|
||||
if stuck > 3:
|
||||
raise OtaError(f"`fw missing` does not answer: {reply}")
|
||||
todo = []
|
||||
continue
|
||||
new = [o // CHUNK for o in new]
|
||||
stuck = stuck + 1 if new == todo else 0
|
||||
if stuck > 3:
|
||||
raise OtaError("the same chunks keep missing")
|
||||
todo = new
|
||||
raise OtaError("text transfer did not converge in 40 rounds")
|
||||
|
||||
|
||||
async def work(module, io):
|
||||
p = module.params
|
||||
pkgs = load_packages(p["packages"])
|
||||
out = {"changed": False, "no_package": False}
|
||||
await io.open()
|
||||
try:
|
||||
status = await io.ask("fw status", "status")
|
||||
base = parse_status(status)
|
||||
if base is None:
|
||||
out.update(state="no_package", no_package=True, status=status)
|
||||
io.log.append("no base image in `fw status`: not an -ota build")
|
||||
return out
|
||||
out["base"] = f"{base[0]} {base[1]}"
|
||||
kind, pkg = pick(pkgs, base[1], full_ok=not io.rf)
|
||||
if kind == "current":
|
||||
out.update(state="current", version=pkg.version)
|
||||
return out
|
||||
if kind == "none":
|
||||
out.update(state="no_package", no_package=True)
|
||||
io.log.append(
|
||||
f"no package starts from {base[1]}" + (" (a full image does not go over RF)" if io.rf else "")
|
||||
)
|
||||
return out
|
||||
out.update(package=pkg.name, kind=kind, version=pkg.version, changed=True)
|
||||
if module.check_mode:
|
||||
out["state"] = "would_update"
|
||||
return out
|
||||
data = pkg.data
|
||||
if p["resume"]:
|
||||
reply = await io.ask("fw missing", "list")
|
||||
todo = text_missing(reply)
|
||||
if todo is None:
|
||||
raise OtaError(f"resume: {reply}")
|
||||
if todo:
|
||||
await text_rounds(io, data, [o // CHUNK for o in todo])
|
||||
else:
|
||||
sid = random.randint(1, 0xFFFF) # chunks of an earlier transfer are dropped
|
||||
r = await io.ask(f"fw begin {len(data)} {sid}", "begin")
|
||||
if not r.startswith("OK"):
|
||||
raise OtaError(f"fw begin: {r}")
|
||||
todo = None
|
||||
if io.rf:
|
||||
todo = await binary_rounds(io, data, sid)
|
||||
if todo is None:
|
||||
todo = list(range((len(data) + CHUNK - 1) // CHUNK))
|
||||
if todo:
|
||||
await text_rounds(io, data, todo)
|
||||
r = await io.ask("fw verify", "list", timeout=40)
|
||||
for _poll in range(20): # the node runs it in loop(), not in the handler
|
||||
if "verifying" not in r:
|
||||
break
|
||||
await asyncio.sleep(3)
|
||||
r = await io.ask("fw status", "status")
|
||||
if not r.startswith(">") or "ready" not in r:
|
||||
raise OtaError(f"fw verify: {r}")
|
||||
await io.ask("fw bl", "list")
|
||||
if not p["apply"]:
|
||||
out["state"] = "staged"
|
||||
return out
|
||||
r = await io.ask("fw apply", "apply", timeout=30, tries=1)
|
||||
if not r.startswith("OK"):
|
||||
raise OtaError(f"fw apply: {r}")
|
||||
out["state"] = "applied"
|
||||
try:
|
||||
await io.after_apply(p["apply_wait"])
|
||||
out["version_after"] = await io.ask("ver", "ver")
|
||||
await io.ask("fw status", "status")
|
||||
except (LinkError, ConsoleError) as e:
|
||||
# applied all the same; the caller reads the version its own way
|
||||
io.log.append(f"after the apply: {e}")
|
||||
return out
|
||||
finally:
|
||||
if io.airtime is not None:
|
||||
out["airtime"] = io.airtime
|
||||
await io.close()
|
||||
|
||||
|
||||
def main():
|
||||
module = AnsibleModule(
|
||||
argument_spec={
|
||||
"transport": {"type": "dict", "required": True},
|
||||
"target": {"type": "str"},
|
||||
"name": {"type": "str"},
|
||||
"password": {"type": "str", "default": "", "no_log": True},
|
||||
"packages": {"type": "path", "required": True},
|
||||
"apply": {"type": "bool", "default": True},
|
||||
"duty": {"type": "float", "default": 3},
|
||||
"gap": {"type": "float", "default": 5},
|
||||
"resume": {"type": "bool", "default": False},
|
||||
"apply_wait": {"type": "int", "default": 60},
|
||||
},
|
||||
supports_check_mode=True,
|
||||
)
|
||||
console = module.params["transport"].get("type") == "ble_console"
|
||||
if console and not HAS_BLEAK:
|
||||
module.fail_json(msg=missing_required_lib("bleak"))
|
||||
if not console and not HAS_MESHCORE:
|
||||
module.fail_json(msg=missing_required_lib("meshcore"))
|
||||
if not console and not module.params["target"]:
|
||||
module.fail_json(msg="target (the node's pubkey) is required over RF")
|
||||
io = None
|
||||
try:
|
||||
io = Console(module.params["transport"]) if console else Radio(module)
|
||||
result = run(work(module, io))
|
||||
except (LinkError, ConsoleError, OtaError, ValueError, OSError) as e:
|
||||
module.fail_json(msg=str(e), log=io.log if io else [])
|
||||
module.exit_json(log=io.log, **result)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Python on the host the modules run on (the one with the companion).
|
||||
meshcore>=2.3
|
||||
# meshcore_companion_acl only (BLE link to a companion).
|
||||
# meshcore_companion_acl (BLE link to a companion), meshcore_fw_ota over a BLE console.
|
||||
bleak>=0.22
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Unit tests for the radio-free OTA parts. Replies are real ones from the masts."""
|
||||
|
||||
import base64
|
||||
|
||||
import pytest
|
||||
from ansible_collections.tth.meshcore.plugins.module_utils.meshcore_ota import (
|
||||
Bucket,
|
||||
Package,
|
||||
crc16,
|
||||
hops_of,
|
||||
lora_airtime,
|
||||
parse_missing,
|
||||
parse_status,
|
||||
pick,
|
||||
text_missing,
|
||||
w_payload,
|
||||
)
|
||||
|
||||
IMG_OLD = "fe7484101f1cad46e055d09488b5aa0a29d00e3172e4394df31fa8d9cc7744a3"
|
||||
IMG_MID = "43a84e0150b8649544af685a5ee19c8254428cccc2b525ed6591a77efacd158a"
|
||||
IMG_NEW = "490418bc6fa12b22edbf6c3925cd8536145a44ebfc3006f7a4ba3a2e65faa9bb"
|
||||
|
||||
|
||||
def mcfw(fmt, image, base=None, version="v1.18.0-tthcc0598b"):
|
||||
head = bytearray(256)
|
||||
head[0:4] = b"MCFW"
|
||||
head[4] = fmt
|
||||
head[5] = 2
|
||||
head[16:48] = bytes.fromhex(image)
|
||||
head[128 : 128 + len(version)] = version.encode()
|
||||
if base:
|
||||
head[160:192] = bytes.fromhex(base)
|
||||
return bytes(head) + b"\x00" * 64
|
||||
|
||||
|
||||
RELEASE = [
|
||||
Package("x-v3.mcfw", mcfw(1, IMG_NEW)),
|
||||
Package("x-v3.from-v1.mcfw", mcfw(2, IMG_NEW, IMG_OLD)),
|
||||
Package("x-v3.from-v2.mcfw", mcfw(2, IMG_NEW, IMG_MID)),
|
||||
]
|
||||
|
||||
|
||||
# ── fw status ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_status_of_an_ota_node():
|
||||
s = "> idle, batt 4137mV (min 3800), free 352256 B, base 456492 fe748410, delta"
|
||||
assert parse_status(s) == (456492, "fe748410")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("reply", ["Unknown command", "", "> idle"])
|
||||
def test_status_without_fw(reply):
|
||||
assert parse_status(reply) is None
|
||||
|
||||
|
||||
# ── which package ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_delta_from_the_node_image():
|
||||
kind, p = pick(RELEASE, "43a84e01", full_ok=False)
|
||||
assert (kind, p.name) == ("delta", "x-v3.from-v2.mcfw")
|
||||
|
||||
|
||||
def test_already_runs_the_image():
|
||||
assert pick(RELEASE, "490418bc", full_ok=True)[0] == "current"
|
||||
|
||||
|
||||
def test_no_delta_full_image_only_where_allowed():
|
||||
assert pick(RELEASE, "deadbeef", full_ok=False) == ("none", None)
|
||||
kind, p = pick(RELEASE, "deadbeef", full_ok=True)
|
||||
assert (kind, p.name) == ("full", "x-v3.mcfw")
|
||||
|
||||
|
||||
def test_package_manifest():
|
||||
p = RELEASE[1]
|
||||
assert p.delta and p.base_sha == IMG_OLD and p.image_sha == IMG_NEW
|
||||
assert p.version == "v1.18.0-tthcc0598b"
|
||||
assert not RELEASE[0].delta and RELEASE[0].base_sha is None
|
||||
|
||||
|
||||
def test_not_a_package():
|
||||
with pytest.raises(ValueError):
|
||||
Package("zip.zip", b"PK\x03\x04" + b"\x00" * 300)
|
||||
|
||||
|
||||
# ── framing ─────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_crc16_ccitt():
|
||||
assert crc16(b"123456789") == 0x29B1 # CRC-16/CCITT-FALSE check value
|
||||
|
||||
|
||||
def test_binary_chunk_layout():
|
||||
chunk = bytes(range(128))
|
||||
p = w_payload(0x1234, 0x100, chunk)
|
||||
assert p[:2] == b"\xf0w"
|
||||
assert p[2:4] == b"\x34\x12" and p[4:8] == b"\x00\x01\x00\x00" and p[8] == 128
|
||||
assert int.from_bytes(p[9:11], "little") == crc16(chunk, crc16(p[2:9]))
|
||||
assert p[11:] == chunk
|
||||
assert 1 + 32 + len(p) <= 176 # the companion frame
|
||||
|
||||
|
||||
def test_missing_bitmap_reply():
|
||||
assert parse_missing(bytes([0, 0, 0, 2, 0, 5, 0, 9, 0])) == (0, [5, 9])
|
||||
assert parse_missing(bytes([3, 0, 0, 0, 0])) == (3, [])
|
||||
assert parse_missing(b"\x00") is None
|
||||
|
||||
|
||||
def test_text_missing():
|
||||
assert text_missing("> none") == []
|
||||
assert text_missing("> 96 192") == [96, 192]
|
||||
assert text_missing("ERR no transfer") is None
|
||||
|
||||
|
||||
def test_text_chunk_fits_the_console_line():
|
||||
line = f"fw w 999936 {base64.b64encode(bytes(96)).decode()}"
|
||||
assert len(line) < 160 # command[160] on the node
|
||||
|
||||
|
||||
# ── airtime ─────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_airtime_matches_the_semtech_calculator():
|
||||
# SX1262 calculator: 10 B, SF7, BW125, CR4/5, preamble 8, explicit header, CRC = 41.22 ms
|
||||
assert lora_airtime(10, 7, 125, 1, preamble=8) == pytest.approx(0.04122, abs=1e-4)
|
||||
|
||||
|
||||
def test_hops():
|
||||
assert hops_of({"out_path_len": -1}) == 0
|
||||
assert hops_of({"out_path_len": 0x42}) == 2 # 2 hops, 2-byte hashes
|
||||
|
||||
|
||||
def test_bucket_paces_to_the_duty():
|
||||
now = [0.0]
|
||||
b = Bucket(1, clock=lambda: now[0]) # 1 % = 36 s per hour, all of it at once
|
||||
assert b.wait_for(30) == 0
|
||||
w = b.wait_for(30)
|
||||
assert w == pytest.approx((30 - 6) / 0.01)
|
||||
now[0] += w
|
||||
assert b.wait_for(30) == 0
|
||||
assert b.used == pytest.approx(60)
|
||||
|
||||
|
||||
def test_bucket_refuses_a_silly_duty():
|
||||
with pytest.raises(ValueError):
|
||||
Bucket(0)
|
||||
Reference in new issue
Block a user