af5b631a53f1
add SYNC_STATUS.md scratchpad before sync rewrite
- time
- 2026-04-30 20:12 UTC · 6 months ago
- machine
- koh2
- parent
- a85d969e6a00
↓ peach-browser-af5b631a.tar.gzatlas at this save
files changed1 file
SYNC_STATUS.md+445 −0SYNC_STATUS.md+445 −0
@@ -0,0 +1,445 @@1+# Peach Sync — Honest Status2+3+Snapshot as of 2026-04-30, HEAD `a85d969e6a00`, package v2.13.33, native v0.1.12.4+5+This document distinguishes what is **implemented and verified**, what is6+**implemented but dormant** (compiles, tests in isolation, not reachable from7+any live runtime path), what is **stubbed**, and what is **missing entirely**.8+9+It exists because earlier session summaries called Phases 6–9 "done." That is10+true at the spec-checklist level. It is **not** true at the "two real devices11+can pair and sync" level. The system has multiple dead-ends between the12+parts.13+14+---15+16+## TL;DR — can two devices sync today?17+18+**No.**19+20+The minimum walking skeleton requires four things, none of which currently21+work end-to-end:22+23+1. Some user gesture must call `runSyncSession`. Nothing does.24+2. First-time pairing must succeed without prior trust. It cannot — the25+ responder hard-rejects un-trusted initiators.26+3. Android must run a Noise NK initiator over WebSocket. There is no JS27+ Noise client.28+4. The native host must actually be advertising. Nothing in the live29+ runtime path calls `advertise()`.30+31+Every individual piece below is real and tested. They just are not connected32+to one another yet.33+34+---35+36+## What is solid (implemented + verified)37+38+### Native Rust host (`native/peach-sync-host/`, v0.1.12)39+40+- **Per-install X25519 identity** (`identity.rs`) — generates on first run,41+ persisted at `<config_dir>/peach-sync-identity.key` mode 0o600 on Unix,42+ bails on corrupt keys. SYN-001's original `[0u8; 32]` private-key bug is43+ gone. Tested.44+- **Noise NK transport** (`noise.rs`) — snow-based, X25519+ChaChaPoly+BLAKE2s.45+ HKDF-SHA256 over the captured handshake hash gives both peers an identical46+ 32-byte session key per labeled purpose. Tested (deterministic, label47+ separation, session separation).48+- **mDNS** (`mdns.rs`) — advertises and browses `_peach-sync._tcp.local.`,49+ TXT records `pk` / `did` / `name` / `pair` / `wsport`. Stop/restart50+ lifecycle clears state cleanly. Tested.51+- **TCP transport** (`tcp.rs`) — listener at OS-assigned port, length-prefix52+ framing, full Noise handshake, challenge/proof auth round, transport-mode53+ encrypt/decrypt, per-IP rate limiting (1 conn/sec/IP, 60s pruning),54+ authenticated read loop emitting via broadcast. Loopback initiator↔55+ responder integration test passes.56+- **WebSocket transport** (`websocket.rs`) — same Noise/auth/transport stack57+ over `tokio_tungstenite` with one Binary message per logical frame, shares58+ the connection map / channels with TCP via `SharedConnectionState`.59+ Loopback test against a raw `tokio_tungstenite` client passes.60+- **Trusted devices store** (`trusted_devices.rs`) — JSON file in config dir.61+ Tested.62+- **Native messaging framing** (`lib.rs`) — u32-length-prefixed JSON over63+ stdin/stdout, 1 MiB cap, parse errors do not kill the host.64+- **`cargo test`** — 29 passing, including the WS↔TCP shared-state path.65+66+### Extension TypeScript67+68+- **`src/background/sync.ts`** — full client mirror of the host protocol:69+ GET_IDENTITY, GET_TRUSTED_DEVICES, TRUST_DEVICE, REMOVE_TRUSTED_DEVICE,70+ ROTATE_IDENTITY, ADVERTISE / STOP_ADVERTISING, DISCOVER / STOP_DISCOVERY,71+ CONNECT, DISCONNECT, SYNC_SEND, GET_SESSION_KEY, STATUS. Length-checked72+ base64url session-key decode. Tested.73+- **`src/shared/sync-protocol.ts`** — canonical-JSON SHA-256 manifests,74+ diff into request/offer/conflict buckets, deterministic conflict resolver75+ (`chooseConflictWinner` lex-compares hashes; both peers reach the same76+ physical-hash winner; loser archives via `buildConflictedCopy`). Tested.77+- **`src/shared/sync-entry-payload.ts`** — AES-256-GCM envelope keyed by the78+ derived session key, strips `passkey.privateKeyJwk` and legacy79+ `privateKeyJwk` before encryption, validates id / modifiedAt / deletedAt80+ binding on decrypt. Tested.81+- **`src/shared/sync-wire.ts`** — base64url JSON-over-Noise wire codec for82+ `SyncMessage`. Tested.83+- **`src/background/sync-coordinator.ts`** —84+ - `dispatchSyncFrame(frame, key, deps)` — pure stateless dispatcher,85+ handles SYNC_HELLO / SYNC_MANIFEST / SYNC_REQUEST / SYNC_ENTRIES /86+ SYNC_DONE / SYNC_ERROR, wires the conflict resolver into the87+ MANIFEST path (resolver chooses winner, loser archives). Tested.88+ - `runSyncSession({ role, deviceId, vaultVersion, transport, deps })` —89+ preflight (Pro + unlocked), per-connection session key, half-close90+ protocol (only peer's DONE/ERROR/close ends our side), merge totals91+ aggregated. Tested.92+ - `createNativeSyncTransport(connectionId, bindings?)` — adapter from93+ `sync.ts` to `SyncSessionTransport`, scoped to a connection id.94+ Tested.95+- **`src/shared/sync-bridge.ts`** — `getSyncBridge()` returns a96+ `CapacitorSyncBridge` (NsdManager via Sync plugin) on Capacitor,97+ `NativeMessagingSyncBridge` (sync.ts) on extension. Discovery only.98+ Helper `peerWebSocketUrl(peer)` builds the URL Capacitor must connect to.99+ Tested.100+- **Settings UX** — Sync section with status, last-sync, paired devices101+ + Remove, Pair-a-new-device, Pair-with-another-device, trusted networks,102+ Reset sync identity in Advanced (clears local devices, best-effort native103+ rotate). Discover-and-tap pairing flow listens for SYNC_PEER_DISCOVERED /104+ SYNC_PEER_LOST broadcasts.105+- **Incoming pairing banner** — App.tsx renders 60-second countdown banner106+ over any view; Accept/Reject calls RESPOND_INCOMING_PAIRING.107+- **Manifest installer** (`scripts/install-native-sync-host.mjs`,108+ `npm run native:install`) — writes `com.peachpasswords.sync.json` to the109+ per-OS NativeMessagingHosts directory; Windows prints registry hint.110+ Tested for all platform/browser combos via `--show-dir` JSON output.111+- **`nativeMessaging`** permission live in both `manifest.json` and112+ `manifest.firefox.json` since v2.13.27.113+- **`vitest`** — 301 passing.114+115+### Android116+117+- **`SyncPlugin.kt`** — NsdManager-based mDNS browser of `_peach-sync._tcp.`,118+ WiFi MulticastLock acquired during discovery, parses TXT records, emits119+ Capacitor `peerDiscovered` / `peerLost` listeners. Methods120+ `startDiscovery` / `stopDiscovery` / `getDiscoveredPeers`.121+- Plugin registered in `MainActivity.kt`.122+- `INTERNET`, `ACCESS_WIFI_STATE`, `CHANGE_WIFI_MULTICAST_STATE`,123+ `CHANGE_NETWORK_STATE` permissions present in `AndroidManifest.xml`.124+- `./gradlew assembleDebug` succeeds.125+- `npx cap sync android` clean.126+127+### Verification matrix all green128+129+- `cargo test` — 29/29130+- `npm run typecheck` — clean131+- `npm test` — 301/301132+- `npm run build:chrome` / `:firefox` / `:mobile` — clean (only the133+ pre-existing dynamic-import warning on `Import.tsx`)134+- `npx cap sync android` — clean135+- `./gradlew assembleDebug` — BUILD SUCCESSFUL136+137+---138+139+## What is wired but wrong (live but broken)140+141+### B1. `INITIATE_PAIR_WITH_PEER` cannot work for first-time pairing142+143+**Where:** `src/background/messages.ts` `INITIATE_PAIR_WITH_PEER` →144+`sync.connectToPeer(host, port, x25519PublicKey)` →145+`tcp.rs::authenticate_responder_side` line ~503.146+147+**What happens:** the responder rejects any initiator whose public key is148+not already in `trusted_devices`:149+150+```rust151+if !trusted_devices.read().await.is_trusted(&normalized_public_key_hex) {152+ anyhow::bail!("initiator is not trusted");153+}154+```155+156+The discover-and-tap flow then locally adds the peer to `peach_paired_devices`157+on success — but the connect attempt has already failed at the responder.158+The user sees "pair failed" and the local store ends up out of sync with the159+peer's reality. **First-time pairing cannot succeed via this path.**160+161+**Why it shipped this way:** I built the pairing UI shell against the162+authenticated transport without first wiring an unauthenticated pairing163+channel. The spec defines `PAIR_REQUEST` / `PAIR_ACCEPT` / `PAIR_REJECT`164+messages exactly for this; they are present in the type union but stubbed165+in the dispatcher.166+167+### B2. The pairing protocol is a no-op stub168+169+**Where:** `src/background/sync-coordinator.ts:343-347`:170+171+```ts172+case "PAIR_REQUEST":173+case "PAIR_ACCEPT":174+case "PAIR_REJECT":175+ // Pairing flow is handled outside the data-sync state machine.176+ return { outbound: [] };177+```178+179+There is no out-of-band pairing channel either. The comment is a lie —180+nothing handles these "outside the data-sync state machine" because no such181+machine exists.182+183+### B3. The incoming pairing banner never fires from real traffic184+185+**Where:** `src/background/messages.ts` — the only writer to186+`setIncomingPairingRequest` is the `DEBUG_INJECT_INCOMING_PAIRING` test187+hook. `sync.ts::onIncomingConnection` exists but no one subscribes to it.188+The native host emits `PEER_CONNECTED` over native messaging when an189+incoming TCP/WS connection completes the handshake; this event is190+unhandled in the runtime path.191+192+**What's missing:** a subscriber on `onIncomingConnection` that, when the193+connection's pairing flag is set, calls `setIncomingPairingRequest` and194+broadcasts `INCOMING_PAIRING_REQUEST` to the popup.195+196+### B4. The extension never advertises197+198+**Where:** `src/background/sync.ts:303` exports `advertise()`. It is called199+from **nowhere** in the runtime path. Search:200+201+```202+grep -rn "advertise\|Advertise" src/popup/ → no matches203+grep -rn "sync.advertise\|advertise(" src/background/ → only the export204+```205+206+Consequence: even if Phase 10 ran with two extensions on the same network,207+neither would be discoverable by the other. mDNS works only because **both208+sides advertise**. The native host has the code; the extension never asks209+it to start.210+211+### B5. `TRIGGER_SYNC` is a Pro-gate stub212+213+**Where:** `src/background/messages.ts:634-637`:214+215+```ts216+case "TRIGGER_SYNC": {217+ const gate = await requirePro("sync");218+ if (!gate.allowed) return { ok: false, error: "pro_required" };219+ return { ok: true };220+}221+```222+223+The "Sync now" button in Settings calls this. It does nothing. The224+coordinator I built in Phase 5 (`runSyncSession`) is fully formed and tested225+but has no live caller. **No user action currently triggers a sync.**226+227+---228+229+## What is implemented but dormant (compiles + tests, not reachable)230+231+These are real and correct; they just have no live caller upstream of them.232+Once the gaps above are fixed they should light up without further changes.233+234+- `runSyncSession` (`sync-coordinator.ts:368`) — preflight, lifecycle,235+ half-close. No caller.236+- `createNativeSyncTransport` (`sync-coordinator.ts:95`) — adapter. No237+ caller.238+- `dispatchSyncFrame` (`sync-coordinator.ts:193`) — full SYNC_*239+ state machine. Only called from inside `runSyncSession` and tests.240+- `getSyncBridge` / `CapacitorSyncBridge` / `NativeMessagingSyncBridge`241+ (`sync-bridge.ts`) — discovery. No caller in popup or background.242+- `advertise` / `stopAdvertising` (`sync.ts`) — native host control. No243+ caller.244+- `onIncomingConnection` (`sync.ts:415`) — incoming-pair native event. No245+ subscriber.246+- `RESET_SYNC_IDENTITY`'s native-side rotate — best-effort `try/catch`247+ silently swallowed because the host is not connected at the moment of248+ call.249+- `START_PEER_DISCOVERY` / `STOP_PEER_DISCOVERY` / `INITIATE_PAIR_WITH_PEER`250+ — handlers exist; they connect the popup to native discovery. They run,251+ but produce nothing because no one advertises (B4) and the trust path is252+ broken (B1, B2).253+254+---255+256+## What is missing entirely (not even started)257+258+### M1. JavaScript Noise NK initiator for Capacitor259+260+**This is the biggest gap.** Android has discovery (`SyncPlugin.kt`) and261+the bridge stub (`sync-bridge.ts`), but no code that:262+263+- opens the WebSocket264+- runs the X25519 + ChaChaPoly + BLAKE2s Noise NK handshake265+- runs the challenge/proof auth round266+- runs the transport-mode encrypt/decrypt loop267+- derives the HKDF session key from the handshake hash268+269+Until this exists, Android can list peers but cannot connect to one. The270+desktop-to-desktop case can sidestep this because the native host (Rust)271+runs Noise on both ends; mobile cannot, because Capacitor's WebView cannot272+load the native messaging host and cannot open raw TCP.273+274+**Rough scope:** a `src/shared/noise-js.ts` using `@stablelib/x25519`,275+`@stablelib/chacha20poly1305`, `blakejs` (or a maintained noise-js276+package), plus a `CapacitorSyncTransport` that satisfies the same277+`SyncSessionTransport` interface as `createNativeSyncTransport`. Probably278+500–800 lines plus tests.279+280+### M2. Pairing protocol implementation281+282+A pairing-mode connection that:283+284+- Both sides put their host into "pairing" advertising state (TXT `pair=1`).285+- Initiator opens an **un-authenticated** Noise NK connection (no286+ trust check on responder side; only allowed when responder is in pairing287+ mode).288+- Initiator sends `PAIR_REQUEST { ed_public_key, name, code }`.289+- Responder pops the inline approval banner; on Accept, calls290+ `TRUST_DEVICE` locally and replies with `PAIR_ACCEPT { ed_public_key,291+ name }`. On Reject, replies with `PAIR_REJECT { reason }` and closes.292+- Initiator on PAIR_ACCEPT calls `TRUST_DEVICE` locally and persists the293+ PairedDevice record.294+- Both sides exit pairing mode.295+296+The native responder needs a `pairing_mode: bool` flag that, when true,297+allows one un-trusted handshake to succeed for the purpose of the PAIR298+exchange (and rejects all data-sync messages on that connection until299+trust is mutually recorded). This requires coordination between300+`tcp.rs`/`websocket.rs` (skip the trust check), `auth.rs` (still require301+proof — pairing should still bind the keypair, just not require prior302+trust), and the dispatcher in TS.303+304+### M3. Live trigger for sync305+306+After pairing, something needs to actually call `runSyncSession`. Sensible307+triggers:308+309+- "Sync now" button → connect to each paired device → `runSyncSession`310+ per peer.311+- Periodic alarm → for each paired device whose last-seen broadcast was312+ recent → run a session.313+- mDNS PEER_DISCOVERED whose `did` is in `peach_paired_devices` → run a314+ session if vault is unlocked.315+316+None of these exist.317+318+### M4. PEER_CONNECTED → incoming pairing banner subscriber319+320+When the native host accepts an incoming pairing-mode connection, it emits321+`PEER_CONNECTED { connection_id, peer_addr }`. The extension needs to:322+323+1. Subscribe via `sync.onIncomingConnection`.324+2. Read the first frame on that connection (PAIR_REQUEST).325+3. Call `setIncomingPairingRequest({ deviceName, publicKeyHex, ... })`.326+4. Broadcast `INCOMING_PAIRING_REQUEST` to the popup.327+5. On user Accept, send `PAIR_ACCEPT` over the connection and call328+ `TRUST_DEVICE`. On Reject, send `PAIR_REJECT` and `disconnect`.329+330+### M5. `INITIATE_PAIR_WITH_PEER` re-implementation331+332+Currently it locally records the peer as "paired" before the handshake333+finishes. After M2 it should:334+335+1. Open a **pairing-mode** connection.336+2. Send `PAIR_REQUEST { our_public_key_hex, our_device_name, code }`.337+3. Wait for `PAIR_ACCEPT` (or `PAIR_REJECT` / timeout).338+4. On accept, call native `TRUST_DEVICE` for the peer's pubkey.339+5. **Then** add to `peach_paired_devices`.340+6. Disconnect the pairing-mode connection. Subsequent sync uses a fresh341+ trusted connection.342+343+### M6. Auto-advertise on Pro+unlocked344+345+Some lifecycle should call `advertise()` when:346+347+- vault is unlocked and348+- user is Pro and349+- (current network is trusted or Tailscale)350+351+And `stopAdvertising` on lock or Pro loss. This is what makes the device352+discoverable to its peers in the first place.353+354+### M7. WebSocket-side rate limiting355+356+`tcp.rs::AcceptRateLimiter` is shared with `websocket.rs` (good), but each357+manager has its own instance (bad). A single attacker scanning both ports358+gets 2 attempts per second per IP instead of 1. Low priority but worth a359+note.360+361+### M8. Connection close from `sync.ts`362+363+`disconnectFromPeer(connectionId)` sends `DISCONNECT` to the native host.364+The native side `handle_disconnect` removes the connection from the map365+but **does not actively close the TCP/WS stream** — it just drops the366+write half. The read loop will exit on next failed read, which only367+happens when the peer also closes. For clean shutdown the native side368+should explicitly shutdown the TcpStream / send a Close frame on WS.369+370+### M9. Phase 10 (manual two-device E2E)371+372+Has not been run. This is the only honest verification that the above is373+all wrong-or-right. Until M1–M8 land, this can't be run for mobile, and374+can only be run for desktop↔desktop after M1 (kidding — without M2/M3/M4375+desktop↔desktop also doesn't work past pairing).376+377+---378+379+## What changed under SYN-001 that genuinely improved the codebase380+381+Even though the system isn't end-to-end working, these are real wins382+already on `dev`:383+384+- **The original SYN-001 critical bug is fixed.** The Noise responder385+ no longer uses an all-zero static private key; per-install identity is386+ generated and persisted. The full audit checklist in the spec was walked387+ through in v2.13.26, including the rate limiter (gap discovered during388+ that walk).389+- **Native host compiles and runs cleanly**, which it didn't at the start.390+- **The protocol layer is fully realized in pure functions** (sync-protocol,391+ sync-entry-payload, sync-wire) with thorough tests. When the live wiring392+ is fixed, these don't have to change.393+- **The coordinator's dispatcher and lifecycle are correct** (modulo the394+ PAIR_* stub) and tested with synthetic transports — the half-close395+ protocol is real.396+- **Conflict resolution is deterministic and symmetric** — both peers agree397+ on a physical-hash winner without a round-trip; the loser archives. This398+ is one of the harder bits and it's done.399+- **Manifests carry `nativeMessaging`** and the per-OS installer ships, so400+ the desktop runtime stack is at least *plumbed* even if not yet401+ *triggered*.402+- **Android shell is in place** with discovery and the right permissions —403+ M1 lands cleanly when written.404+405+---406+407+## Suggested next-bite order408+409+If picking this up, the cheapest path to a working E2E is roughly:410+411+1. **M6 + M3 + B5 first** — make `TRIGGER_SYNC` (or popup unlock) call412+ `advertise()` and a paired-peer sync session. This requires no new413+ protocol; it just connects existing parts. Verifies the desktop↔desktop414+ data-sync path against an *already-trusted* pair. Lets you see real415+ `runSyncSession` runs against real native infrastructure.416+2. **M2 + B1 + B2 + B3 + M4 + M5** — pairing protocol, in one go since417+ they share the same surface. After this, two unrelated desktops can418+ pair and sync.419+3. **M1** — JS Noise NK. After this, an Android emulator can join.420+4. **M7, M8** — small cleanups.421+5. **M9 / Phase 10** — actually pair and sync two devices end-to-end.422+423+Roughly: 1 is half a day if disciplined; 2 is one to two days; 3 is the424+biggest single chunk, probably two to four days depending on whether a425+maintained noise-js package is acceptable or a hand-rolled implementation426+is required.427+428+---429+430+## How this report differs from earlier session notes431+432+CONTINUITY.md has a sequence of `[CODE]` outcome entries that mark433+Phases 3, 5, 6, 7, 8, 9 as DONE. Those are accurate **at the spec434+checklist level** — the items the spec asked for in those phases were435+shipped. They are not accurate at the "system works" level. Specifically:436+437+- Phase 5 ("extension background sync") shipped `runSyncSession` etc.438+ but not a caller for it — see B5, M3.439+- Phase 6 ("Android sync") shipped discovery and the sync-bridge shell,440+ but not the JS Noise initiator — see M1.441+- Phase 7 ("Settings sync UX") shipped Reset / banner / discover-tap UI,442+ but the banner is unreachable from real traffic and the tap path443+ cannot succeed for first-time pairing — see B1, B2, B3.444+445+That's an overstatement on my part. This document is the correction.