retoor/tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

whatsapp: fix link/login 401, add image send, contact sync and search; harden failover

Browse Source 
Root causes, each reproduced against the live server or endpoint:
- Server-only JIDs decoded as "@s.whatsapp.net" and pairing replies copied them
  into `to`, so the device was never registered and every login returned 401.
  Decoder/encoder fixed, replies use s.whatsapp.net, ADV hosted prefixes follow
  whatsmeow. The link wizard now waits for the restart, logs in, uploads prekeys
  and retries once with a fresh device on 401/403.
- Closed sockets made every receive loop spin until its deadline (the 7 minute
  CPU burn). Loops stop on a closed stream.
- whatsapp_send had no image argument and send_image was a stub. Added WaMedia
  (byte-exact against 8 wa-go golden vectors), media_conn upload, image send and
  the DeviceSentMessage copy for the user's other devices.
- Phone contacts never reached the client: added app-state sync (WaAppState,
  WaReceiver, LTHash and MAC checks verified against a whatsmeow-built oracle)
  and the historySyncConfig declaration at registration.
- Contact lookup: FTS5 trigram plus fuzzy ranking, ambiguity is reported and
  never guessed, global search lists contacts. Send caps (10/min, 200/day) are
  enforced and survive unlink.

Failover and agent:
- Every failed backend candidate is printed the moment it fails.
- muse runs with low reasoning effort, an empty preset and one stable workspace
  (36.5s down to 2.8s on the real prompt); Ctrl-C no longer counts as a backend
  failure; CLI backends use the circuit breaker; opencode runs are capped at 30s.
- Orphan tool results are dropped before sending (devplace HTTP 400, reproduced);
  compaction no longer starts its window on a tool result.
- Terminal art pasted as a prompt is refused before any model call.

Tests: whatsapp_test, muse_flags_test, llm_history_test; full suite 40 steps
(agent.e2e reads live tmux panes and is flaky, green on re-run).
Research archived under research/whatsapp-automation.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
retoor
2026-10-27 21:32:38 +01:00
co-authored by Claude Sonnet 5.5
parent bfc117b0c5
commit 1611f3baeb
50 changed files with 3350 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files
taicpp/docs/whatsapp.md
+90 -1
View File
@@ -1,5 +1,17 @@
# whatsapp (taicpp): research result and build specification
`whatsmeow` is an open source Go library for the WhatsApp Web multidevice API
(author Tulir Asokan, module `go.mau.fi/whatsmeow`, license MPL-2.0). It speaks
the WhatsApp protocol directly over one WebSocket, with no browser involved:
QR and pair-code linking, Noise handshake, Signal encryption, send and receive,
media upload, and app-state (contact) sync. detz does not link or ship it: the
native C++ module `detz::wa` is transpiled from `wa-go` (MIT) with `whatsmeow`
as the behavior reference (section 3), and the test oracle
`tests/whatsapp_data/appstate_oracle.go` imports its Go packages to generate
byte-exact vectors. Any comment saying "ported from whatsmeow" means the logic
or byte layout was copied from upstream and must stay compatible with it. Full
comparison: section 2.1.
Status: **built and integrated; the offline core is verified, the live link
is verified through QR display, message send is built but unverified** (it
needs a phone-linked account, which development does not have). What is in the
@@ -169,7 +181,7 @@ pair, as the rest of taicpp.
| `WaTools` | agent tools `whatsapp_send`, `whatsapp_contacts`, `whatsapp_alias` | none, detz design |
Out of scope for v1, listed so nobody assumes otherwise: groups (sender keys),
app-state sync (LTHash contact list), history sync, calls, status, newsletters,
history sync, calls, status, newsletters,
receiving media, editing, reactions.
## 5. Protocol walkthrough and crypto mapping
@@ -557,3 +569,80 @@ parser-focused tests and the first sanitizer runs because those never exercised
the capture hook (which turned an unreproducible failure into a deterministic
one), and the rule that a round-trip test must cover the populated case, not
just the empty one. The honest open item from 13.6 is now closed.
## 14. App-state sync: the contact list (`WaAppState`, `WaReceiver`)
Status: implemented and verified offline. The live path needs a re-linked
phone: nothing here has been run against a phone that holds contacts.
Why: the phone's address book reaches a linked companion only through app-state
sync (collection `critical_unblock_low`, index `["contact", "<jid>"]`). Without
it `wa_contact` stays empty and names cannot be resolved.
| Piece | Files | What it does |
|---|---|---|
| Key receipt | `wa_receiver.*`, `wa_proto.*` (`parse_protocol_message`), `WaClient::handle_message` | Decrypts `<enc>` from our own account (pkmsg via X3DH responder, msg via session), unpads, stores each APP_STATE_SYNC_KEY_SHARE key in `wa_appstate_key`. Only the phone's own messages are decrypted; other senders are acked and ignored as before. |
| Key request | `WaClient::request_app_state_keys`, `send_peer_message`, `encode_app_state_key_request` | Peer message to the primary phone (category `peer`), at most `kWaAppStateKeyIdsMax` ids, one request per key id per `kWaAppStateKeyRequestInterval` (24 h), stored in `wa_appstate_keyreq`. |
| Key expansion, MACs, values | `wa_appstate.*` (`WaAppStateKeys`, `WaAppStateCrypto`) | HKDF-SHA256 `WhatsApp Mutation Keys` to 160 bytes; value MAC HMAC-SHA512 truncated to 32; index, snapshot and patch MACs HMAC-SHA256; AES-256-CBC values. `WaCrypto::hmac_sha512` added. |
| LTHash | `wa_appstate.*` (`WaLtHash`) | 128-byte state, HKDF-SHA256 info `WhatsApp Patch Integrity` (whatsmeow, not BLAKE2b; see research conclusion `appstate-lthash-hkdf-not-blake2b`). |
| Patch fetch and apply | `WaAppStateSync::sync` | iq `w:sync:app:state`, `return_snapshot` at version 0, `version=N` otherwise; external blobs via `WaMedia::download` (`md-app-state`); snapshot MAC, patch MAC, index and value MACs checked before anything is stored. |
| Contacts | `WaAppStateSync::apply_contact` | SET stores jid = index[1], name = fullName else firstName; empty name is counted and skipped; REMOVE deletes the row. |
| Login hook | `WaClient::login` calls `sync_app_state` after prekeys | Never fails the login (errors are logged). If a key is missing, requests it, waits up to `kWaAppStateKeyWait` s for a key share, then syncs once more. |
Caps (`wa_consts.h`): `kWaAppStatePatchMax` 50 patches per sync,
`kWaAppStateMutationMax` 10000 mutations, `kWaAppStateTimeout` 30 s per iq,
`kWaAppStateBlobMax` 16 MiB per external blob, `kWaAppStateKeyIdsMax` 16 key
ids per request, `kWaAppStateKeyWait` 20 s. A cap stops the sync loudly
(`capped` in the report, a log line) and the rest waits for the next login.
Failure rule: a patch or snapshot that fails a MAC or hash check is not stored,
the sync stops, and the collection version stays at the last verified state.
This skips the bad patch and everything after it, because each later patch
depends on the hash chain.
Tables: `wa_appstate_key`, `wa_appstate_version`, `wa_appstate_mac` (value MAC
per index MAC, needed to undo a SET or REMOVE), `wa_appstate_keyreq`. `clear_device`
empties them, so a new link syncs from version 0.
Verification (offline, `build/whatsapp_test`, section `appstate`):
- expanded keys equal whatsmeow `expandAppStateKeys` bytes;
- LTHash add/remove cancel and are order-free;
- a snapshot plus one patch (SET, REMOVE, in-patch replacement, unnamed SET, a
non-contact SET) decodes to the oracle's final hash and version, and the
contact table matches the oracle's expected names;
- a re-sync at version 2 changes nothing;
- a tampered patch is rejected and not stored;
- a missing key stores nothing and reports the key id;
- the key share and key request bytes match whatsmeow's Message encoding;
- a pkmsg built with `WaSession` decrypts through `WaReceiver` to the key share
Message, and a corrupted pkmsg is refused.
The oracle is `tests/whatsapp_data/appstate_vectors.json`, generated by
`tests/whatsapp_data/appstate_oracle.go` (a Go program using whatsmeow's own
`lthash`, `hkdfutil` and protobuf packages). It is not run by `make test`.
Not verified: the live server's answers to the iq, key share timing, the media
host for external blobs, and whether a REMOVE of an unknown index (whatsmeow
only warns) causes trouble. Known gap: a SET with an empty name keeps any older
name the row had; the spec says skip, not delete.
## 15. Incident WA-2026-10-27b: link ended in 401, no picture, no contacts
Everything here was measured on 2026-10-27/28 against the live server and the user's phone.
| Symptom | Root cause | Fix | Evidence |
|---|---|---|---|
| `Linked, but the login confirmation did not complete`; later login `failure 401` | The decoder printed a server-only JID (empty user) as `@s.whatsapp.net`; the pairing replies copied that into `to`, so the server refused `pair-device-sign` and never registered the device | decoder returns the bare server name; encoder writes bare server names as JID pairs; replies use `WaConsts::kServerJid` as whatsmeow does | captured `pair-success` decoded (`from=@s.whatsapp.net`), live login probe 401 before, `logged in` after re-link; test `server-only JID round-trips` |
| wizard never reported the real login result | `run_until` returned at once because `paired_event_` stayed set | `run_until` is stateless; `WaClient::await_pair_restart` + `WaClient::login`; one fresh pairing after a 401/403 (`kWaLinkRetries`) | wizard source |
| `user 7m` CPU while linking | a closed socket made every receive return instantly and the loops spun until their deadline | `WaSocket` clears `connected_` on close, every loop checks `WaClient::socket_open()` | probe after the 401: 1.7 s, 0.01 s CPU (was 20 s of spinning) |
| PNG could not be sent | the `whatsapp_send` tool had no image argument and `send_image` was a stub | `WaMedia` (HKDF `WhatsApp Image Keys`, AES-CBC, 10-byte HMAC, `media_conn`, HTTPS upload), `send_image`, DeviceSentMessage copy to our other devices, tool argument `image` | 8/8 wa-go golden vectors byte-exact; live: image to own number acknowledged by the server |
| Nigel not found | the phone address book reaches a companion only through app-state sync, which did not exist; and the phone only pushes history and keys to a companion that declares `DeviceProps.historySyncConfig`, which registration omitted | section 14 (`WaAppState`, `WaReceiver`); `WaDeviceProps` now sends `historySyncConfig` (field 5, whatsmeow values); peer key request encrypted to the phone's LID identity | unit tests; the live address book needs a link made with this build |
| name lookups were exact or one fuzzy guess | contact table only, one `FuzzyMatcher::Best` | `WaStore::search_contacts`: FTS5 trigram candidates + fuzzy rank; ambiguity (two scores within 0.08) is reported, never guessed; the global `search` tool lists matching contacts | tests `contact search: ...` |
| no send limits | caps were specified, not enforced | `wa_send_log`, `kWaSendPerMinute` 10 and `kWaSendPerDay` 200 enforced in `whatsapp_send` | test `send log counts sends in the window` |
Gaps: the contact sync and the address-book key share are not verified against a phone yet; a newly linked device
must be created with this build for the phone to send history. Set `DETZ_WA_TRACE=1` to log every node's tag and
attributes when debugging a protocol exchange.