Commit Graph
23 Commits
Author SHA1 Message Date
ericz aae2bf6d60 Backup&Restore 2026-11-08 00:40:23 +01:00
zjs81 a9d369f42f Address message filter review findings
False positives
- Same-sender repeats need at least 3 words and 10 characters, so short
  replies ("ok", "test") repeated within 10 minutes stay visible.
- Repeated characters needs a run of 20+ of one non-digit character;
  "Noooooooo!", separators, a row of emoji and long numbers are chat.

Correctness
- Rules apply to stored messages: adding a rule hides matching messages
  already in the chat, and removing or disabling one shows again what only
  it was hiding. Other filter reasons are left alone. Also applied when
  history is loaded.
- The filter runs after mesh dedup, inside _addChannelMessage, so each
  message is evaluated and counted by the duplicate tracker once.
- Channels excluded from filtering no longer feed implicit clean examples
  to the classifier.
- Unread counting and the unread anchor treat filtered messages as normal
  when filtering is off for the channel.

Performance
- Classifier training runs in a background isolate (injectable trainer for
  widget tests); app start-up no longer waits for it.
- Reply previews look up the quoted message through an id index instead
  of scanning the channel per bubble.
- The latest-messages channel sort computes each channel's latest visible
  message once.

Cleanup
- Use MessageFilterText.isStructuredPayload instead of a screen copy.

Docs: message-filter.md explains how each check works, including the
learned classifier, and drops the outdated "new messages only" limit.
2026-11-06 00:54:09 -07:00
zjs81 1332c35884 Add channel message filter for spam and repeated messages
Client-side filter for channel messages, with no model download:
- User rules on message text and/or sender name (contains, whole word,
  exact), normalised for case, accents, zero-width and look-alike letters.
- Automatic checks: repeated characters, same-sender repeats within 10
  minutes, and the same sentence (>= 6 words) from 3+ senders within 30
  minutes. Windows use send times, so synced history is not a flood.
- On-device n-gram classifier trained by Mark as spam / Not spam taps.

Filtered messages are kept, never notify or count as unread, are not
auto-translated or used for channel previews, and are concealed in reply
quotes. They show as a calm, collapsible placeholder that animates open
and closed, or can be hidden completely. Includes a Filtered messages
list, per-channel toggle, settings screen, translations for all locales
and documentation.
2026-11-05 23:35:27 -07:00
zjs81 6213da118e Arm the telemetry send ack only after preparing the path
A background retry's RESP_CODE_SENT arriving while the path was being
prepared was taken as the telemetry request's ack, so its tag and timeout
replaced ours and a sensor's tagged reply was rejected.
2026-10-25 11:26:15 -07:00
zjs81 dfc4cd54d8 Keep telemetry fixes off the advert timestamp and across full syncs
Writing the telemetry time into lastSeen sent the phone's clock back to
the radio as the node's advert timestamp on the next path or flag write,
which could later make the radio's record look newer and drop the fix.
The fix now records its own telemetryFixAt and leaves lastSeen as the
advert time; the map and lists show the later of the two as last heard.

A full contact sync (pull to refresh, reconnect) cleared the list before
the radio resent it, so fixes were lost even for an unchanged advert.
They are now held for the sync and restored onto matching records.
2026-10-25 11:17:44 -07:00
zjs81 f0144ca959 Show telemetry GPS fixes on the map and add a map location request
Telemetry replies carried a GPS fix that was only shown on the telemetry
screen, so the map kept the node at its last advert position. The fix now
updates the contact and survives radio contact refreshes until the node
adverts again (compared on the node's own advert clock). The map node card
gains an "Update location" button for companions and sensors; repeaters
and rooms only share GPS with logged-in admins, so they don't get it.
2026-10-25 11:10:27 -07:00
zjs81 603172b3ec Batch per-frame persistence so busy meshes don't stall the UI isolate
Every received packet copy re-saved its channel's whole history, every
unread change re-saved the channel list, and adverts/path records re-saved
full contact and path-history blobs. Under load the SharedPreferences
backlog starved the Dart event loop (frames and timers stopped), the radio's
offline queue overflowed, and CPU stayed pinned after traffic ended.

- Add CoalescedWrites: keeps the latest write per key and runs them once
  per window (750 ms; 5 s for path history). Writes start synchronously so
  stores still read the current scoped key.
- Flush before the app is backgrounded, on disconnect, before switching
  radios, before the channel slot map changes, before channel history
  reloads, before deleteChannel drops a list, and before path history
  reloads an evicted contact.
- Cap in-memory channel history at ChannelMessageStore.maxStoredMessages;
  storage already kept only that many, but the list grew unbounded during a
  session and every save grew with it.
- Cache Contact.publicKeyHex and build pubKeyToHex from a hex-digit lookup; it was
  rebuilt on every equality/hash and per-advert contact scan.

Load test (fake TCP radio, 350 contacts, 17 channels, 10-150 packets/s,
Linux profile build): CPU at 10 packets/s 104% -> 45%, recovers to idle
after traffic instead of staying at 160%, and queued messages drained went
from 555 to 3237 over the same run.
2026-10-24 23:38:28 -07:00
zjs81 a4670a9275 Add a setting to show link images in all chats by default
'Show link images in all chats by default' (default off) applies to
every channel and contact without an explicit per-chat choice: the
per-chat setting is now unset/on/off and resolves to perChat ?? default.
The 'Load images from links' master switch still overrides it.
2026-10-24 22:01:30 -07:00
zjs81 0620e8f27a Always clear a deleted channel's storage when no other slot holds it
Keep refreshing channels before clearing so in-flight queue messages
can't write the history back, but decide whether the channel is still
held while ignoring the deleted slot, and clear even if the refresh
fails or is incomplete.
2026-10-24 21:54:16 -07:00
zjs81 7e7e7a48db Merge remote-tracking branch 'origin/dev' into pr-593
# Conflicts:
#	lib/models/app_settings.dart
#	lib/screens/channel_chat_screen.dart
#	lib/screens/chat_screen.dart
#	untranslated.json
2026-10-24 21:45:13 -07:00
zjs81 7c2f481520 Merge remote-tracking branch 'origin/dev' into fix/ui-issues
# Conflicts:
#	untranslated.json
2026-10-24 21:13:02 -07:00
zjs81 21e727cf21 Don't re-create a removed contact when syncing its path or flags
The firmware's CMD_ADD_UPDATE_CONTACT adds the key when it isn't in the
contact table. Screens that stay open after a contact is removed (e.g. a
repeater admin screen after the repeater's keys were changed and the old
contact deleted) sync the path before every send, so the next command,
status or telemetry request silently added the old contact back. Skip
path and flag writes for contacts no longer in the contact list.
2026-10-24 21:11:04 -07:00
zjs81 f2a27f0f71 Wait for the radio before removing a contact, and explain re-adds
removeContact now waits for the radio's reply and only drops the contact
locally when the radio confirms (or reports it doesn't have it), so a
lost command no longer leaves a contact that the next sync restores.
Failures are shown to the user. The delete dialog warns when the radio
auto-adds that contact type, since the contact returns on its next
advert.
2026-10-24 21:03:39 -07:00
zjs81 f667320608 Fix review findings in channel notification modes
- Detect mentions on the original text, as @[name] or a whole-word @name
- Save and restore the full mode map in review mode
- Open the mode picker from the chat bell and show the mode in its tooltip
- Share one mode picker sheet; fix the stale sheet context
- Persist modes by name and skip unknown entries
- Keep writing muted_channels so older builds keep the user's mutes
- Remove the muteChannel/unmuteChannel/isChannelMuted shims
- Update notifications docs to match the behaviour
2026-10-24 20:46:49 -07:00
ericz 050016f599 ai fix tcp reconnect test fail 2026-10-24 16:46:32 +02:00
ZIERandCopilot 4000c49213 Merge zjs81/dev into ez_share_urlimage_PR
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-13 10:16:13 +02:00
zjs81 6a8df764bb Stamp legacy images inside the migration; hold early channel frames
- Image entries are stamped with their channel ID before the migration is
  marked done, so an interrupted run repeats it instead of leaving them
  unstamped (and invisible) forever.
- Channel message/data frames that arrive before this session's slot map
  is confirmed are held and replayed once it is. Current firmware only
  sends them in reply to SYNC_NEXT_MESSAGE, which already waits; this
  guards against a stale saved map filing them under the wrong channel.
2026-10-11 18:04:53 -07:00
zjs81 679116ed03 Key per-channel storage by PSK-derived channel ID
Channel messages, settings, region, order and images were persisted by
radio slot index, so a slot reused for a different channel inherited the
old channel's history and settings (#359).

- Channel.storageId: first 8 bytes of SHA-256("meshcore-channel:" + psk).
- All per-channel stores key by channel ID; in-memory state stays by slot.
- The connector keeps a slot -> channel ID map, diffs it against the saved
  map on every channel sync and reloads slots whose channel changed.
- Queued channel frames wait until this session's map is confirmed.
- One-shot per-radio migration from slot keys, using the cached channel
  list the legacy data was written under; rerun-safe if interrupted.
- Deleting a channel clears its data; a channel removed elsewhere keeps its
  history and gets it back when re-added. A PSK edit keeps its place in
  the custom order.
- Slots sharing a PSK share history and settings.
- Image entries record their channel ID; a slot change resets that slot's
  in-progress reassembly.
2026-10-11 17:52:59 -07:00
zjs81 25f197c979 Fix review findings, add default flood scope, and address recent issues
Review fixes:
- Keep stored message history beyond the 200-message window; stop
  messages leaking between radios; reset link/session state on disconnect
- Accept firmware's 176-byte RX frames over USB/TCP
- Hold APP_START during contact sync; contact-sync watchdog; replay DMs
  from not-yet-known senders; tombstone late command replies; finish
  channel sync on ERR_NOT_FOUND; fail region-scoped sends cleanly
- DM retry watchdog when RESP_CODE_SENT never arrives; late repeater CLI
  replies; ACK hash collisions; UTF-8-safe name truncation
- Image pipeline: delete-during-decode, lifecycle cancels, decoder memory
  release, entry cap, dead worker recovery, atomic model writes
- Storage: deterministic reaction hash across web/native, Smaz
  double-decode, per-entry parsing, per-radio path history and regions,
  review-mode data isolation
- UI: safe channel slot selection, byte-length name limits, double-send
  guard, mounted checks, localized hardcoded strings
- Skip persisted-store reload on repeat SELF_INFO

Features:
- Radio default flood scope (firmware ver 11+) and explicit unscoped
  sends (ver 12+)
- Scan for nearby nodes (#570)
- Optional distinct first-hop repeater counting for channel resend (#589)
- Optional region next to hop count (#580)

Issues:
- Message text follows system font size (#569)
- Repeater/room password field accepts 20 chars (#576)
- TCP auto-reconnect and keepalive (#571)
- Order feeds by local receive time; flag future timestamps (#584)
- Coding rate help text (#568), French "Favoris" (#575)
- iOS/macOS app badge now clears

Translations for all new strings in every locale.
2026-10-11 01:25:31 -07:00
Zach d74587c167 Refactor LoRa airtime tests and add new functionality
- Updated LoRa airtime calculations in `lora_airtime_test.dart` to reflect new reference values.
- Introduced new tests for LDRO behavior and preamble symbol calculations.
- Added `contact_path_width_test.dart` to validate path hash width handling in contacts.
- Enhanced `model_changes_test.dart` to ensure legacy data migration works correctly.
- Created `remote_node_stats_test.dart` to decode and validate remote node statistics.
- Implemented `map_marker_parse_test.dart` for parsing marker text with coordinate validation.
- Developed `map_tile_cache_key_test.dart` to ensure API keys are excluded from cache keys.
- Added `message_retry_failure_test.dart` to test message retry service behavior on failures.
- Extended `path_history_service_test.dart` to preserve learned state across path attempts.
- Enhanced `received_image_store_test.dart` to handle concurrent writes and review fixes.
- Implemented `repeater_command_service_test.dart` to validate command response handling.
- Created `store_tolerant_parsing_test.dart` to ensure stores skip malformed records gracefully.
- Cleared untranslated JSON file for localization updates.
2026-09-25 10:01:18 -07:00
kingult dfcf13a97b fix: lat/lon double-write in buildUpdateContactPathFrame
The function emitted two consecutive 8-byte position blocks instead of
one, producing a frame 8 bytes longer than the documented layout. When
a caller passed lastModified, the firmware parsed the duplicated second
lat as the timestamp, giving wildly wrong "last seen" values on
imported contacts.

Delete the unconditional first block; keep the conditional block that
correctly skips the optional tail when neither location nor
lastModified is set, zero-fills position slots when only lastModified
is present, and appends the optional timestamp.

Adds test/connector/build_update_contact_path_frame_test.dart with
five cases covering all four optional-tail combinations plus the
fixed-point lat/lon encoding.

Fixes #427
2026-05-06 16:03:03 -07:00
just-stuff-tm 2f770bbd53 fix(tcp): reset state on aborted pre-handshake connect 2026-03-10 21:38:35 -04:00
just-stuff-tm 9db79e9d40 test(tcp): harden cancel-race handling and add coverage
- tighten late TCP connect error suppression to manual-cancel disconnecting/disconnected windows
- keep TCP handshake failures surfaced outside explicit cancel flow
- allow TcpScreen connect action when connector is scanning
- add connector-level tests for late-error suppression classifier
- add TcpScreen test covering connect from scanning state
2026-03-10 20:06:05 -04:00