meshcore-repeater-observer 0.1.0
ci / test (push) Successful in 1m15s

Reads a MeshCore repeater's console over BLE or USB and publishes what it
hears: the `pktlog` raw-frame feed in the meshcoretomqtt format (community
analyzer with a JWT signed by the node's key, audience = broker hostname, or
an own broker without auth) and optionally stats, `nodes`, `rxlog` to a local
MQTT broker. Needs the observer build of meshcore-tth.

Tested on a XIAO with the observer build over USB and over BLE: feed in the
meshcoretomqtt format, packet hashes matching those of other observers.
Unit tests for the parsers, the packet transform, the Ed25519 signature
(checked against `cryptography`), the JWT and paho-mqtt 1.x/2.x; README
(Czech), AGENTS.md, CI, Renovate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
tthandClaude Opus 5.5 committed 2026-11-03 00:20:03 +01:00
commit 0da25dd15b
10 files changed
+1868

No files matched your search

+44
View File
@@ -0,0 +1,44 @@
# Lint + tests on every push and PR; the image is built by image.yml.
name: ci
"on":
push:
pull_request:
workflow_dispatch:
env:
RUFF_VERSION: "0.15.7"
MYPY_VERSION: "1.19.1"
YAMLLINT_VERSION: "1.38.0"
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: gitleaks
uses: https://git.rfa.cz/tth-pub/ci-actions/gitleaks@v11
with:
mode: block
- uses: actions/setup-python@v7
with:
python-version: "3.15"
- name: Install
run: |
set -euo pipefail
python -m pip install --quiet ".[dev]" \
"ruff==$RUFF_VERSION" "mypy==$MYPY_VERSION" "yamllint==$YAMLLINT_VERSION"
- name: Lint
run: |
set -euo pipefail
ruff check .
ruff format --check .
mypy meshcore_repeater_observer.py
yamllint .gitea .yamllint
- name: Tests
run: python -m pytest -q
+10
View File
@@ -0,0 +1,10 @@
__pycache__/
*.egg-info/
.venv/
.pytest_cache/
.mypy_cache/
.ruff_cache/
build/
dist/
cursors.json
*.env
+5
View File
@@ -0,0 +1,5 @@
extends: default
rules:
line-length: {max: 120}
document-start: disable
truthy: {check-keys: false}
+25
View File
@@ -0,0 +1,25 @@
# Instructions for an AI agent setting this up
meshcore-repeater-observer reads a MeshCore repeater's console and publishes
what the repeater hears. README.md is the reference; these are the rules.
1. **Read only, with two exceptions the user must approve:** the observer
itself sets the node's clock (`time <epoch>`) and, for the community feed,
reads its private key (`get prv.key`) into memory. Never send other CLI
commands to the repeater (`set …`, `reboot`, `erase`, `advert`) — it is
part of a community network.
2. The repeater needs the observer build of meshcore-tth. Ask which board and
firmware it runs; do not flash anything yourself.
3. Prefer USB (`/dev/serial/by-id/…`, never `/dev/ttyACM0`). BLE: the console
is off after flashing; the user turns it on (`ble on`) after setting their
own PIN. Pairing is interactive (`bluetoothctl pair`) — the user does it.
Never `bluetoothctl remove` a paired repeater.
4. Community feed only for a node on the Czech preset (869.432 / 62.5 / SF7 /
CR5), with `MC_OBS_IATA` set; leave `MC_OBS_AUDIENCE` at its default.
5. Passwords and PINs go to the env file (mode 600), never into commands you
print, commits or logs. The private key never leaves the process.
6. Done = the log shows `observer: připojen k …` and packets are counted;
on the community analyzer the node appears under observers within minutes.
Developing: `python3 -m pytest`, `ruff check`, `ruff format --check`,
`mypy meshcore_repeater_observer.py`. `--help` must work without bleak/paho.
+1
View File
@@ -0,0 +1 @@
@AGENTS.md
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 TeTeHacko
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+165
View File
@@ -0,0 +1,165 @@
# meshcore-repeater-observer
Udělá z MeshCore **repeateru observer**: přes jeho konzoli (Bluetooth nebo USB)
průběžně čte, co repeater slyší, a posílá to do MQTT — do komunitního
analyzeru ([analyzer.meshcore.cz](https://analyzer.meshcore.cz)), do vlastního
CoreScope, nebo obojí. Žádné druhé rádio vedle repeateru není potřeba.
```
repeater ── BLE / USB ── meshcore-repeater-observer ──┬── MQTT ── komunitní analyzer
(observer build) └── MQTT ── vlastní broker / CoreScope
```
*English: turns a MeshCore repeater into an observer by reading its console
(`pktlog`, `rxlog`, `nodes`) over BLE or USB and publishing to MQTT in the
meshcoretomqtt format. Docs below are Czech; `--help` is English.*
## Kdy to dává smysl
- Repeater už stojí (stožár, střecha) a vedle něj nejde nebo nechcete dát
další rádio. Počítač s Bluetooth v dosahu, nebo USB kabel, stačí.
- Jinak je jednodušší samostatný observer na [openHopu](https://github.com/openhop-dev/openhop_repeater),
který MQTT do analyzerů umí sám a repeater nechá na pokoji.
## Co je potřeba
1. **Repeater s observer buildem** firmwaru [meshcore-tth](https://git.rfa.cz/tth-pub/meshcore-tth):
soubory `*_repeater-observer-*` v [Releases](https://git.rfa.cz/tth-pub/meshcore-tth/releases)
(XIAO, SenseCAP Solar, RAK4631, T1000-E, Heltec T114; `.uf2` přetažením na
disk bootloaderu, `.zip` pro DFU přes BLE nebo USB). Běžný MeshCore
firmware příkazy `pktlog`/`rxlog` nemá; observer pak feed tiše vynechá.
Observer build má Bluetooth konzoli **po flashi vypnutou** a zástupný PIN
`123456`: nejdřív si nastavte vlastní PIN a admin heslo, pak `ble on`.
2. **Linux** s Pythonem 3.10+ (Raspberry Pi stačí), u BLE s BlueZ.
3. Pro komunitní analyzer: repeater na **českém nastavení** (869,432 MHz /
62,5 kHz / SF7 / CR5) — komunita chce jen uzly na CZ presetu.
## Instalace
```bash
pipx install git+https://git.rfa.cz/tth-pub/meshcore-repeater-observer
meshcore-repeater-observer --help
```
### USB (doporučeno)
Repeater na USB kabelu; port najdete v `ls -l /dev/serial/by-id/` (trvalé
jméno, `/dev/ttyACM0` se po přepojení mění).
```bash
MC_SERIAL_PORT=/dev/serial/by-id/usb-…-if00 \
MC_OBS_ENABLE=1 MC_OBS_IATA=PRG \
meshcore-repeater-observer
```
USB se s rádiem repeateru nepere a čte se nepřetržitě.
### Bluetooth
1. Na repeateru (USB konzole, nebo z aplikace přes admin CLI) nastavte vlastní
PIN a heslo a zapněte konzoli — po flashi je **vypnutá**:
`ble on` (uloží se). Pozor: dokud má nějaký počítač otevřený jeho USB port,
**Bluetooth konzole se neukáže** (neběží BLE advertising, nejde se
připojit). Na repeater jako takový to vliv nemá — přijímá, přeposílá
a posílá LoRa adverty dál normálně. Jen BLE a USB konzole se střídají.
2. Jednou spárujte:
```
bluetoothctl
agent KeyboardOnly
default-agent
scan on # počkejte na MAC repeateru
pair AA:BB:CC:DD:EE:FF
trust AA:BB:CC:DD:EE:FF
quit
```
3. Spusťte s `MC_BLE_ADDR=AA:BB:CC:DD:EE:FF` místo `MC_SERIAL_PORT`.
Observer se po každém vyčtení odpojí a `MC_BLE_OFF_INTERVAL` sekund (výchozí
120) počká: starší firmware s připojeným BLE klientem zahazoval přímé
(DIRECT) pakety, které měl přeposlat. V meshcore-tth je to opravené, interval
jde zkrátit; pokud log hlásí `pktlog DROP`, ring se během pauzy přepsal.
### Jako služba (systemd)
```ini
# /etc/systemd/system/meshcore-repeater-observer.service
[Unit]
Description=MeshCore repeater observer
Wants=network-online.target bluetooth.target
After=network-online.target bluetooth.target
[Service]
EnvironmentFile=/etc/meshcore-repeater-observer.env
ExecStart=/opt/meshcore-repeater-observer/bin/meshcore-repeater-observer
StateDirectory=meshcore-repeater-observer
Restart=on-failure
RestartSec=30
[Install]
WantedBy=multi-user.target
```
V `/etc/meshcore-repeater-observer.env` řádky `MC_…=…` z tabulky níž.
`StateDirectory` drží pozici v ringu repeateru, takže restart nic neztratí.
## Nastavení (proměnné prostředí)
| proměnná | výchozí | |
|---|---|---|
| `MC_SERIAL_PORT` | — | USB konzole repeateru (`/dev/serial/by-id/…`) |
| `MC_BLE_ADDR` / `MC_BLE_NAME` | — | BLE: MAC, nebo jméno, které repeater advertuje |
| `MC_NODE` | `repeater` | jméno v lokálních MQTT tématech |
| `MC_OBS_ENABLE` | vypnuto | `1` = observer feed z `pktlog` |
| `MC_OBS_IATA` | — | **povinné s feedem**: kód letiště regionu (`PRG`, `BRQ`, `OSR`, …) |
| `MC_OBS_MQTT_HOST` | `mqtt1.meshcore.cz` | broker feedu |
| `MC_OBS_MQTT_PORT` / `MC_OBS_TRANSPORT` / `MC_OBS_TLS` | `443` / `websockets` / `1` | |
| `MC_OBS_AUTH` | `token` | `token` = JWT podepsaný klíčem uzlu; `none` = vlastní broker bez hesla |
| `MC_OBS_AUDIENCE` | = `MC_OBS_MQTT_HOST` | audience v JWT; komunitní brokery chtějí svůj hostname |
| `MC_OBS_OWNER` / `MC_OBS_EMAIL` / `MC_OBS_MODEL` | — / — / `MeshCore repeater` | volitelné údaje do tokenu a statusu |
| `MC_MQTT_HOST` | — | lokální MQTT (volitelné): stats, `nodes`, `rxlog`, kopie paketů |
| `MC_MQTT_PORT` / `MC_MQTT_TLS` / `MC_MQTT_USER` / `MC_MQTT_PASS` | `1883` / — | |
| `MC_MQTT_TOPIC` | `repeater-observer/<MC_NODE>` | kořen lokálních témat |
| `MC_INTERVAL` | `300` | s mezi plným čtením (stats, senzory, sousedé) |
| `MC_NODES_INTERVAL` / `MC_RXLOG_INTERVAL` | `60` / `5` | registr `nodes` / vyčítání `rxlog`+`pktlog` |
| `MC_BLE_OFF_INTERVAL` | `120` | BLE: pauza bez spojení mezi vyčteními |
| `MC_CURSOR_FILE` | `$STATE_DIRECTORY/cursors.json` | pozice v ringu repeateru |
Komunitní brokery: `mqtt1.meshcore.cz`, `mqtt2.meshcore.website`,
`mqtt.meshcore-analyzer.eu` (všechny WSS :443, JWT). Audience musí být
hostname brokeru — s jinou broker odpoví `not authorised`.
Vlastní CoreScope: `MC_OBS_MQTT_HOST=<broker> MC_OBS_MQTT_PORT=1883
MC_OBS_TRANSPORT=tcp MC_OBS_TLS=0 MC_OBS_AUTH=none`.
## Klíč uzlu
Pro komunitní feed se observer přihlašuje tokenem podepsaným **klíčem
repeateru**, takže si při spojení přečte z konzole `get prv.key`. Klíč drží jen
v paměti procesu, nikam ho nezapisuje ani neloguje, a před použitím ověří, že
k němu patří veřejný klíč uzlu. S `MC_OBS_AUTH=none` se klíč nečte vůbec.
## Co se publikuje
- Feed (`MC_OBS_ENABLE=1`): `meshcore/<IATA>/<PUBKEY>/packets` a `…/status`
ve formátu [meshcoretomqtt](https://github.com/Cisien/meshcoretomqtt).
- Lokálně (`MC_MQTT_HOST`), pod `MC_MQTT_TOPIC`: `neighbours`, `stats`,
`sensors`, `nodes` (retained), `rxlog`, `packets`, `ble`/`serial` stav.
Observer navíc srovnává hodiny repeateru bez GPS (`time <epoch>`), když jsou
pozadu: starý čas v advertech by ostatní uzly zahazovaly jako replay.
## Když to nejde
| příznak | příčina |
|---|---|
| `config: …` a konec | chybí transport (`MC_SERIAL_PORT`/`MC_BLE_ADDR`), IATA, nebo není kam publikovat |
| `BLE … nenalezen ve scanu` | repeater má BLE vypnuté (`ble on`), má otevřený USB port (BLE se pak neukazuje), nebo BlueZ visí na starém spojení: `bluetoothctl disconnect <MAC>` — **nikdy `remove`**, to zruší spárování |
| `FW nezná pktlog` | repeater nemá observer build |
| `broker odmítl spojení rc=5` | špatná audience, nebo uzel není na CZ presetu |
| `pktlog DROP` | ring repeateru se přepsal dřív, než se vyčetl: kratší `MC_BLE_OFF_INTERVAL`, nebo USB |
## Licence
MIT.
File diff suppressed because it is too large. Load diff
+37
View File
@@ -0,0 +1,37 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "meshcore-repeater-observer"
dynamic = ["version"]
description = "Observer feed and stats from a MeshCore repeater's console over BLE or USB"
readme = "README.md"
license = "MIT"
requires-python = ">=3.10"
dependencies = [
"bleak>=0.22",
"paho-mqtt>=1.6",
"pyserial>=3.5",
]
[project.optional-dependencies]
dev = ["pytest", "ruff", "mypy", "cryptography"]
[project.scripts]
meshcore-repeater-observer = "meshcore_repeater_observer:cli"
[tool.setuptools]
py-modules = ["meshcore_repeater_observer"]
[tool.setuptools.dynamic]
version = { attr = "meshcore_repeater_observer.__version__" }
[tool.ruff]
line-length = 120
[tool.ruff.lint]
select = ["E", "F", "W", "I", "B", "UP"]
[tool.mypy]
ignore_missing_imports = true
+167
View File
@@ -0,0 +1,167 @@
"""Parsers, the meshcoretomqtt transform and the JWT signature -- no radio, no broker."""
import hashlib
import importlib
import json
import os
import subprocess
import sys
from base64 import urlsafe_b64decode
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
def load(env=None):
"""Import the module with a clean MC_* environment (it reads env at import)."""
for k in [k for k in os.environ if k.startswith("MC_")]:
del os.environ[k]
os.environ.update(env or {})
import meshcore_repeater_observer as mod
return importlib.reload(mod)
@pytest.fixture
def ob():
return load()
def test_parse_pktlog(ob):
text = " -> P,7,1793600000,40,-92,1104AABB\r\nP,8,1793600001,-12,-110,15\r\nE,8,3,8\r\n"
recs, last, oldest, newest = ob.parse_pktlog(text)
assert [r["seq"] for r in recs] == [7, 8] and (last, oldest, newest) == (8, 3, 8)
assert recs[0]["snr"] == 40 and recs[0]["rssi"] == -92 and recs[0]["raw"] == "1104AABB"
def test_parse_rxlog(ob):
recs, last, oldest, newest = ob.parse_rxlog("R,5,1793600000,abcd,11,24,-95,5361\nE,5,1,5\n")
assert recs == [
{"seq": 5, "when": 1793600000, "pkthash": "abcd", "hdr": 0x11, "snr": 6.0, "rssi": -95, "path": "5361"}
]
assert (last, oldest, newest) == (5, 1, 5)
def test_parse_nodes_name_with_commas(ob):
rows, nxt, total = ob.parse_nodes("N,ab12,2,50100000,14200000,20,-90,30,deadbeef,5361,name, with, commas\nE,1,1\n")
assert rows[0]["name"] == "name, with, commas" and rows[0]["lat"] == 50.1 and rows[0]["snr"] == 5.0
assert (nxt, total) == (1, 1)
def test_paged_done(ob):
assert ob._paged_done(bytearray(b"P,1,2,3,4,ab\r\nE,1,1,1\r\n"))
assert ob._paged_done(bytearray(b" -> E,0,0,0\r\n"))
assert not ob._paged_done(bytearray(b"P,1,2,3,4,ab\r\n"))
def test_clock_both_formats(ob):
assert ob.parse_clock("12:34 - 7/8/2026 UTC (epoch 1786095587)") == (1786095587, True)
from datetime import datetime, timezone
node, exact = ob.parse_clock("12:34 - 7/8/2026 UTC")
assert not exact and node == int(datetime(2026, 8, 7, 12, 34, tzinfo=timezone.utc).timestamp())
assert ob.parse_clock("garbage") is None
def test_frame_and_hash(ob):
payload = bytes(range(40))
flood_advert = bytes([(4 << 2) | 1, 0]) + payload # route F, type 4, no path
route, ptype, plb, pl = ob.parse_raw_frame(flood_advert)
assert (route, ptype, plb, pl) == (1, 4, 0, payload)
assert ob.meshcore_packet_hash(4, 0, payload) == hashlib.sha256(bytes([4]) + payload).digest()[:8].hex().upper()
# 2-byte path hashes, two hops: (2-1)<<6 | 2
with_path = bytes([(4 << 2) | 1, 0x42]) + b"\x53\x61\x77\x88" + payload
assert ob.parse_raw_frame(with_path)[3] == payload
assert ob.parse_raw_frame(b"\x11") is None
def test_packet_json(ob):
ob.OBS.name, ob.OBS.pub = "my-rpt", "AB" * 32
raw = (bytes([(4 << 2) | 1, 0]) + bytes(40)).hex()
msg = ob.obs_packet_json({"seq": 1, "when": 1793600000, "snr": 41, "rssi": -100, "raw": raw})
assert msg["origin"] == "my-rpt" and msg["route"] == "F" and msg["packet_type"] == "4"
assert msg["SNR"] == "10" and msg["RSSI"] == "-100" and msg["raw"] == raw.upper() and len(msg["hash"]) == 16
assert ob.obs_packet_json({"seq": 1, "when": 0, "snr": 0, "rssi": 0, "raw": "zz"}) is None
def orlp_keypair(seed: bytes, ob):
"""orlp/ed25519 keypair layout: prv64 = clamped scalar || prefix (lib/ed25519/keypair.c)."""
h = bytearray(hashlib.sha512(seed).digest())
h[0] &= 248
h[31] &= 63
h[31] |= 64
a = int.from_bytes(h[:32], "little")
pub = ob._ed_compress(ob._ed_mul(ob._ED_B, a))
return pub, bytes(h)
def test_signature_verifies_with_cryptography(ob):
ed = pytest.importorskip("cryptography.hazmat.primitives.asymmetric.ed25519")
seed = bytes(range(32))
pub, prv = orlp_keypair(seed, ob)
# the same key derived the standard way (RFC 8032 seed) must give the same public key
std = ed.Ed25519PrivateKey.from_private_bytes(seed).public_key()
from cryptography.hazmat.primitives import serialization
assert std.public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw) == pub
sig = ob.ed25519_sign_orlp(b"hello", pub, prv)
std.verify(sig, b"hello") # raises on a bad signature
def test_jwt_audience_is_broker_host():
ob = load({"MC_OBS_ENABLE": "1", "MC_OBS_IATA": "PRG"})
pub, prv = orlp_keypair(b"\x01" * 32, ob)
tok = ob.make_auth_token(pub.hex(), prv.hex())
head, body, sig = tok.split(".")
payload = json.loads(urlsafe_b64decode(body + "=" * (-len(body) % 4)))
assert json.loads(urlsafe_b64decode(head + "==")) == {"alg": "Ed25519", "typ": "JWT"}
assert payload["aud"] == "mqtt1.meshcore.cz" == ob.OBS_HOST
assert payload["publicKey"] == pub.hex().upper() and payload["exp"] - payload["iat"] == ob.OBS_TOKEN_TTL
assert len(sig) == 128 and sig == sig.upper() # uppercase hex, not base64url
ob2 = load({"MC_OBS_MQTT_HOST": "mqtt2.meshcore.website"})
assert ob2.OBS_AUDIENCE == "mqtt2.meshcore.website"
@pytest.mark.parametrize(
"env,expect",
[
({}, "MC_SERIAL_PORT"),
({"MC_SERIAL_PORT": "/dev/x", "MC_OBS_ENABLE": "1"}, "MC_OBS_IATA"),
({"MC_SERIAL_PORT": "/dev/x"}, "nothing to publish"),
({"MC_SERIAL_PORT": "/dev/x", "MC_OBS_ENABLE": "1", "MC_OBS_IATA": "PRG"}, None),
({"MC_BLE_ADDR": "AA:BB:CC:DD:EE:FF", "MC_MQTT_HOST": "localhost"}, None),
],
)
def test_config_error(env, expect):
err = load(env).config_error()
assert (err is None) if expect is None else (expect in err)
def test_local_mqtt_optional():
ob = load({"MC_SERIAL_PORT": "/dev/x", "MC_OBS_ENABLE": "1", "MC_OBS_IATA": "PRG"})
assert isinstance(ob.make_mqtt(), ob.NullMQTT)
assert ob.MQTT_TOPIC == "repeater-observer/repeater"
def test_help_without_dependencies():
code = (
"import sys, builtins; real = builtins.__import__\n"
"def imp(n, *a, **k):\n"
" if n.split('.')[0] in ('bleak', 'paho', 'serial'): raise ImportError(n)\n"
" return real(n, *a, **k)\n"
"builtins.__import__ = imp; sys.argv = ['x', '--help']\n"
"import runpy; runpy.run_path('meshcore_repeater_observer.py', run_name='__main__')"
)
r = subprocess.run([sys.executable, "-c", code], cwd=ROOT, capture_output=True, text=True)
assert r.returncode == 0 and "MC_SERIAL_PORT" in r.stdout
def test_mqtt_client_with_installed_paho(ob):
# paho 2.x refuses Client() without callback_api_version; 1.x has no such argument
c = ob.mqtt_client(client_id="t", clean_session=True, transport="websockets")
c.on_connect = lambda cl, ud, fl, rc: None
c.on_disconnect = lambda cl, ud, rc: None
c.username_pw_set("v1_AB", "token")