Files
meshcore-open/tools/aeic/README.md
T
zjs81 aa00a863c3 Refresh documentation and update Dutch translation
Documentation:
- Correct web transport support: USB via Web Serial in Chrome only; no
  BLE or TCP in the browser.
- Fix region scope semantics, routing modes (Auto/Flood/Manual), the
  156-byte DM limit, repeater CLI framing, the guessed-location algorithm,
  radio stats contents and settings labels.
- Add regions, image messages, radio statistics and troubleshooting guides.
- Restore coding conventions, localization workflow, storage scoping and
  transport entry points to CLAUDE.md/AGENTS.md; align build/CI commands.
- Replace the obsolete docs/BLE_PROTOCOL.md with a pointer to the
  maintained protocol reference.

Dutch (nl):
- Translate the 79 missing strings and correct existing mistranslations,
  typos and grammar (e.g. "Uitzoomen", "Handmatig", "Slowaaks", frequency
  range 150-2500 MHz).
- Use consistent terminology and informal "je" throughout; keep mesh terms
  such as repeater, node, flood, hop and advert in English.

Fixes #538
2026-09-27 13:12:23 -07:00

78 lines
4.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Regenerating the AEIC test fixtures
The image codec's tests are pinned against data produced by the reference
implementation, not by the Dart code. That is the whole point: a Dart encoder
and a Dart decoder that agree with each other prove nothing, because they would
agree just as happily on a wrong wire format. These scripts produce the data the
Dart side is checked against.
They are committed here so `test/services/golden/` is reproducible. Nothing in
the app runs them.
| script | produces | used by |
|---|---|---|
| `export_golden.py` | `golden/aeic_cdf_ft32.bin`, `golden/vectors/*.gv` | `rans_coder_test.dart`, `entropy_tables_test.dart` |
| `record_entropy_io.py` | `golden/e2e/*.aeicrec`, `golden/e2e/manifest.json` | `image_codec_e2e_test.dart` |
## What they need
Neither script is self-contained. Both drive the real AEIC model, so they need
the research checkout that is **not** part of this repository:
- the AEIC source (`github.com/LuizScarlet/AEIC`) with the two local patches:
pybind11 bumped to v2.13.6 (2.10.4 silently returns stride-0 arrays under
NumPy 2), and `.contiguous()` on `z_hat` in both `compress()` and
`decompress()` (the encoder and decoder otherwise land in different memory
layouts, and the ~2.8e-7 drift that causes desynchronises the rANS decoder on
roughly one image in 26, with no error raised)
- the `AEIC_SE_ft32.pkl` checkpoint
- the compiled C++ rANS extension (`src/cpp`, CMake) — the golden bitstreams are
produced by it, which is what makes them worth comparing against
- PyTorch, onnxruntime, NumPy 2
## Preparing the research layout
The scripts import helpers that are not included in this app repository: `aeic_runner.py` and, for recording, `bitexact_encoder.py`. Obtain the matching research/export environment before running them. The app does not pin that external checkout’s revision; record its commit, checkpoint checksum, dependency versions, and patch diff alongside regenerated fixtures so others can reproduce your results.
Use this layout in the research checkout:
```text
exp/
export_golden.py
record_entropy_io.py
aeic_runner.py
bitexact_encoder.py
onnx/
aeic_entropy_side_fp32_op17.onnx
aeic_entropy_decode_fp32_op17.onnx
results/golden/
data/kodak_raw/
data/custom/
```
Copy the two scripts from this directory into `exp/`. Their output paths are resolved relative to the scripts’ parent directory, not just the shell’s working directory. Supply the ft32 checkpoint and expected image corpus in the research environment.
## Running them
From the prepared research checkout, with its dependencies installed:
```bash
AEIC_DEVICE=cpu python exp/export_golden.py --ckpt /absolute/path/AEIC_SE_ft32.pkl
AEIC_DEVICE=cpu python exp/record_entropy_io.py --ckpt /absolute/path/AEIC_SE_ft32.pkl --scratch /absolute/path/aeic-scratch
python exp/record_entropy_io.py --verify-only
```
The recorder’s default scratch path is machine-specific; always supply `--scratch` or `AEIC_SCRATCH`. `--decode-graph` can select an existing recording graph; `--images` selects an explicit corpus. The exporter defaults to ten images; `--all` uses the full configured corpus.
Copy the generated contents of `results/golden/` into the app’s `test/services/golden/`. Keep paired `.gv`, `.bin`, JSON metadata, recording files, and manifest together. Review generated differences and run the relevant tests:
```bash
flutter test test/services/rans_coder_test.dart test/services/entropy_tables_test.dart test/services/image_codec_e2e_test.dart
```
## When fixtures need regeneration
Regenerate when changing the reference model, entropy tables, rANS format, or recorded tensor interface. These fixtures validate the codec/entropy pipeline, **not the separate mesh chunk headers**.
Chunk framing and metadata are defined in [image_chunk_transport.dart](../../lib/services/image_chunk_transport.dart). Changes there need transport-focused test updates; a chunk-layout-only change does not automatically require new reference codec fixtures. See the [image-message guide](../../documentation/image-messages.md).