tai/taicpp/docs/whatsapp.md

# whatsapp (taicpp): research result and build specification

`whatsmeow` is an open source Go library for the WhatsApp Web multidevice API
(author Tulir Asokan, module `go.mau.fi/whatsmeow`, license MPL-2.0). It speaks
the WhatsApp protocol directly over one WebSocket, with no browser involved:
QR and pair-code linking, Noise handshake, Signal encryption, send and receive,
media upload, and app-state (contact) sync. detz does not link or ship it: the
native C++ module `detz::wa` is transpiled from `wa-go` (MIT) with `whatsmeow`
as the behavior reference (section 3), and the test oracle
`tests/whatsapp_data/appstate_oracle.go` imports its Go packages to generate
byte-exact vectors. Any comment saying "ported from whatsmeow" means the logic
or byte layout was copied from upstream and must stay compatible with it. Full
comparison: section 2.1.

Status: **built and integrated; the offline core is verified, the live link
is verified through QR display, message send is built but unverified** (it
needs a phone-linked account, which development does not have). What is in the
tree now (`src/whatsapp/`, namespace `detz::wa`):

| Piece | Files | State |
|---|---|---|
| `WaCrypto` (X25519, AES-GCM/CTR/CBC, HKDF, PBKDF2, SHA, XEdDSA) | `wa_crypto.*` | verified: RFC vectors + XEdDSA roundtrips |
| `WaBinary` (binary-XML codec, token dict v3) | `wa_binary.*`, `wa_tokens_data.h`, `wa_node.h` | verified: wa-go codec battery 19/19 |
| `WaProto` (protobuf + the needed messages) | `wa_proto.*` | verified: ClientPayload byte-exact |
| `WaNoise` (Noise XX + transport) | `wa_noise.*` | verified: golden transcript **and live server** |
| `WaSignal` (X3DH + Double Ratchet) | `wa_signal.*` | verified: golden session 4/4 byte-exact |
| `WaDevice` (identity + ClientPayload) | `wa_device.*` | verified via the handshake |
| `WaSocket` (libcurl WebSocket) | `wa_socket.*` | verified live (connect + frames) |
| `WaClient` (connect, pair, login, send) | `wa_client.*` | connect/QR live-verified; pair-complete + send built, unverified |
| `WaStore` (vault SQLite) | `wa_store.*` | verified: round-trips device + contacts |
| `WaQr` (terminal QR) | `wa_qr.*` | verified: renders a scannable code |
| `WaWizard` (`--whatsapp-link`) | `wa_wizard.*` | live-verified through QR display + device persist |
| agent tools `whatsapp_send`, `whatsapp_contacts` | `src/agent/detz_tools.cpp`, `tools_ops.cpp` | wired; contacts verified, send unverified |

Verified against the live server (no phone needed): `detz --whatsapp-link`
connects, completes the Noise handshake, receives the `pair-device` node, renders
the QR, and saves the device. A phone scanning that QR would link it; the
pair-success handler (ADV HMAC + account-signature verify, device signing,
`pair-device-sign`) is ported faithfully from whatsmeow but cannot be exercised
in-tree. `whatsapp_send` performs a real usync + prekey-fetch + Signal-encrypt +
send-node sequence; it is faithful to whatsmeow but unverified without a live
linked account. Unit test: `build/whatsapp_test` (16/16), vectors in
`tests/whatsapp_data/`. Dependencies added: none at runtime (OpenSSL was already
linked for the static build; added to the default link), plus two vendored
libraries, `third_party/ed25519` (zlib) and `third_party/qrcodegen` (MIT).

Remaining to make send fully work: own-device fan-out (DeviceSentMessage),
media upload for `send_image` (`WaMedia`), prekey upload after login, retry
receipts, and a live test with a sacrificial number.

Original research and plan below (kept as written; some "not built yet" lines
are now superseded by the table above).

---

This file states what was found, what was
decided, and the exact plan to build a native C++ WhatsApp client into detz.
Research archive: [`../../research/whatsapp-automation/`](../../research/whatsapp-automation/)
(sources and three conclusions). Every number below was measured on 2026-10-27
on this machine unless a line says otherwise.

## 1. Goal

One sentence must work in detz:

> "Create screenshot and explain what is genius about it and send to Niggel on WhatsApp"

The flow breaks into four steps. Three of them already exist:

| Step | Owner | Exists |
|---|---|---|
| take a screenshot | `shell` tool (`grim`, `scrot`, `import`) | yes |
| explain it | `describe_image` tool (src/agent/tools_core.cpp) | yes |
| resolve "Niggel" to a WhatsApp JID | `WaContacts` (this spec) | no |
| send text plus image | `whatsapp_send` tool over `WaClient` (this spec) | no |

The missing part is one module: a WhatsApp companion device written in C++.

## 2. Answers to the questions asked

### 2.1 Best Python libraries for a normal phone number

There is no Python library that speaks the current WhatsApp protocol itself.
Measured on the cloned repositories:

| Library | Language of the protocol code | How it works | Verdict |
|---|---|---|---|
| neonize (Apache-2.0, 759 stars, 62 commits in 90 days) | **Go** (whatsmeow) | 4362 lines of Go export 93 C functions (`//export`), 14918 lines of Python call them through `ctypes` | best "Python" option, but it is whatsmeow with a Python skin |
| whatsfly | Go (whatsmeow) | same idea, PyPI 0.0.1, linux 32 bit only | early, not usable |
| whatsapp-mcp (lharries, MIT) | Go (whatsmeow) | Go bridge with HTTP `/api/send`, Python MCP server with `send_message`, `send_file` | usable shortcut, see section 10 |
| yowsup (GPL-3.0) | Python | old phone-registration protocol (`e1.whatsapp.net:443`), last commit 2021-12-14, README warns that registration gives an instant ban | dead, do not use |
| pywhatkit, Astra, wa-automation, whalesong | Python | drive WhatsApp Web in a real browser | browser, rejected |
| Supbot2 | Python | drives the Android app with Appium | needs a phone or emulator, rejected |
| pywa and other Cloud API clients | Python | official Meta Business Cloud API | needs a business number; the number cannot stay on the normal app, rejected for a personal number |

Conclusion: transpiling Python gets us nothing. The real implementations are:

| Project | Language | License | Activity (commits in last 90 days) | Size (non-blank, non-test, non-generated) |
|---|---|---|---|---|
| whatsmeow (tulir) | Go | MPL-2.0 | 78 | 24,301 lines, plus 107,816 generated protobuf lines |
| libsignal-protocol-go (tulir), used by whatsmeow | Go | **GPL-3.0** | not measured | 5,416 |
| wa-go (jfelipesjc) | Go | **MIT** | 17, created 2026-06-26, 1 star | 23,278 (wire 2778, signal 1625, keys 311, media 517, client 14379) |
| Baileys (WhiskeySockets) | TypeScript | MIT | 7 | 48,898 |
| whatsapp-rust (oxidezap) | Rust | MIT | at least 100 (API page limit) | about 650,000 including VoIP and plugins |
| libsignal-protocol-c (signalapp) | C | **GPL-3.0** | last commit 2020-03-27 | 29,456 |

No C or C++ library implements the multi-device protocol. Search found only
WebKit wrappers (WasIstLos, YAWF) and a REST wrapper (Green-API).

### 2.2 Is a browser required

**No.** whatsmeow, Baileys, wa-go and whatsapp-rust all speak the protocol
directly over one WebSocket. The phone is needed exactly once, to link the
device: scan a QR code shown in the terminal, or type an 8-character pairing
code into WhatsApp > Linked devices. After that the companion runs alone. The
phone must come online now and then; one directory page reports
re-authentication after about 20 days (not verified).

### 2.3 Can it be done in C++ with what detz already links

Yes. detz already links libcurl and OpenSSL:

- libcurl 8.18.0 on this machine lists `ws wss` as protocols.
  `curl -H "Origin: https://web.whatsapp.com" wss://web.whatsapp.com/ws/chat`
  answered **`101 Switching Protocols`**, so the transport is proven reachable
  from here with the library we already ship.
- OpenSSL 3.5.5 covers every primitive except XEdDSA signing (section 5).
- Protobuf needs no library: we need about 15 messages, and a hand-written
  varint and length-delimited writer and reader is about 200 lines.

New dependencies: **none.**

## 3. Decision: native C++, transpiled

Build `detz::wa` as a native C++ module. Transpile from two sources:

1. **wa-go (MIT) is the code source** for wire, Noise, binary codec, keys,
   Signal (X3DH, Double Ratchet), media crypto and pairing. Reasons: MIT, small
   (the whole Signal layer is 1625 lines against 3585 for the libsignal-go
   subset), layered exactly like detz wants (`internal/wire`, `signal`, `keys`,
   `media`), and it ships **788 KB of golden vectors captured from real Baileys**
   (`testdata/traces/connect_pair`, `testdata/signal/session_ab.json`,
   `testdata/media/media_vectors.json`, `testdata/paircode/vectors.json`). Those
   become our C++ test fixtures as they are.
2. **whatsmeow (MPL-2.0) is the behaviour reference**: it is the most used and
   most maintained (78 commits in 90 days). When wa-go and whatsmeow disagree,
   whatsmeow wins. The current client version, the server certificate key and
   the node shapes are read from whatsmeow at build time of each release.

Do **not** transpile libsignal-go or libsignal-protocol-c: both are GPL-3.0,
and detz is commercial. MPL-2.0 is file-level: a C++ file transpiled from a
whatsmeow file keeps the MPL notice and its source stays available. Copying
code from wa-go, or reading whatsmeow to learn a value, does not trigger this.
This is a fact about the licenses, not legal advice.

## 4. Module layout

Namespace `detz::wa`, sources in `src/whatsapp/`, one class per `.h`/`.cpp`
pair, as the rest of taicpp.

| Class | Responsibility | Transpiled from (wa-go / whatsmeow) |
|---|---|---|
| `WaConstants` | URL, origin, Noise pattern, header, dict version, cert key, client version, all caps (section 8) | `socket/constants.go`, `store/clientpayload.go` |
| `WaProtoWriter`, `WaProtoReader` | protobuf wire format: varint, zigzag not needed, length-delimited, fixed32 | none, hand-written |
| `WaMessages` | the ~15 protobuf messages as plain structs with `encode()`/`decode()` | field numbers from whatsmeow `proto/*.proto` |
| `WaCrypto` | static: X25519, AES-256-GCM, AES-256-CTR, AES-256-CBC, HKDF-SHA256, HMAC-SHA256, SHA-256, SHA-512, PBKDF2-SHA256, random, over OpenSSL EVP | `util/*util`, wa-go `keys` |
| `WaXeddsa` | XEdDSA sign and verify over Curve25519 keys | libsignal-go `ecc/SignCurve25519.go` logic, wa-go `keys` (section 5) |
| `WaSocket` | libcurl WebSocket, 3-byte big-endian frame length, `WA` header on first frame, 16 MiB frame cap | wa-go `wire/framing.go`, `ws` |
| `WaNoise` | Noise XX handshake and the post-handshake read/write ciphers | wa-go `wire/noise.go`, whatsmeow `socket/noisehandshake.go`, `handshake.go` |
| `WaNode`, `WaBinaryCodec`, `WaTokens` | binary XMPP-like nodes, token dictionary v3 (236 single-byte, about 1030 double-byte tokens), nibble/hex packing, JID encoding | wa-go `wire/codec.go`, `token.go`, `node.go` |
| `WaStore` | SQLite tables: device identity, signed prekey, prekeys, sessions, identities, contacts, aliases | wa-go `store`, whatsmeow `store/sqlstore` |
| `WaPairing` | QR pairing (`pair-device`, ref rotation, `pair-success` with ADV signature check) and phone pairing code | wa-go `client` pairing, whatsmeow `pair.go`, `pair-code.go` |
| `WaSignal` | X3DH, Double Ratchet, `pkmsg`/`msg` encrypt and decrypt, session records | wa-go `internal/signal` (x3dh, ratchet, session, message) |
| `WaPrekeys` | upload 50 prekeys plus signed prekey after pairing, refill below threshold | whatsmeow `prekeys.go` |
| `WaUsync` | device list of a recipient (`usync` iq), "is this number on WhatsApp" | whatsmeow `user.go` (subset) |
| `WaMedia` | media key expansion (`WhatsApp Image Keys`), AES-CBC plus HMAC, `media_conn` iq, HTTPS upload to `/mms/image/<token>` | wa-go `media`, whatsmeow `upload.go`, `mediaconn.go` |
| `WaSender` | 1:1 send: device fan-out, encrypt per device, own other devices get a `DeviceSentMessage`, `<message><participants>` node, `phash` check | whatsmeow `send.go` (DM path only) |
| `WaReceiver` | decrypt incoming 1:1 text, send receipts and acks | whatsmeow `message.go`, `receipt.go` (subset) |
| `WaClient` | owns the above; connect, login, keepalive, reconnect, event loop on one thread | whatsmeow `client.go`, `keepalive.go`, `connectionevents.go` |
| `WaContacts` | name to JID: user aliases first, then contacts learned from push names, then fuzzy match with `detz::FuzzyMatcher` (floor 0.3) | none, detz design |
| `WaTools` | agent tools `whatsapp_send`, `whatsapp_contacts`, `whatsapp_alias` | none, detz design |

Out of scope for v1, listed so nobody assumes otherwise: groups (sender keys),
history sync, calls, status, newsletters,
receiving media, editing, reactions.

## 5. Protocol walkthrough and crypto mapping

Order of one session, with the OpenSSL call for each primitive.

1. **Connect.** WebSocket to `wss://web.whatsapp.com/ws/chat`, header
   `Origin: https://web.whatsapp.com`. Every frame is a 3-byte big-endian
   length plus payload; the first frame is prefixed with
   `'W','A',6,3` (magic 6, dictionary version 3). Frame cap `1 << 24`.
2. **Noise XX handshake.** Pattern string
   `Noise_XX_25519_AESGCM_SHA256\0\0\0\0` (32 bytes, used as the initial hash
   as is). Send `HandshakeMessage.ClientHello{ephemeral}`; receive
   `ServerHello{ephemeral, static (encrypted), payload (encrypted cert)}`;
   mix X25519 shared secrets with HKDF-SHA256 into AES-256-GCM keys, counter IV;
   verify the certificate chain against the pinned key `WACertPubKey` in
   whatsmeow `handshake.go`; send `ClientFinish{static (encrypted noise key),
   payload (encrypted ClientPayload)}`. Then split into a read and a write key.
   The proto also defines post-quantum modes (`XXKEM`, `IKKEM`, `WA_PQ`);
   whatsmeow does not use them today, so neither do we.
3. **ClientPayload.** Register (unpaired: `devicePairingData` with registration
   id, identity public key, signed prekey and its signature, build hash = MD5 of
   the version string) or login (paired: `username` = phone number, `device` =
   device id). The client version is a constant
   (`2.3000.1050728964` in whatsmeow on 2026-10-27). A stale version gives
   connect failure **405 client outdated**; whatsmeow can fetch the live value
   from `web.whatsapp.com` (`"client_revision":N`).
4. **Pairing.** QR: the server sends `pair-device` with refs; the QR string is
   `https://wa.me/settings/linked_devices#ref,noisePub,identityPub,advSecret,clientType`
   (keys base64), shown in the terminal, refs
   rotate. Phone code: `PairPhone` sends a `link_code_companion_reg` iq; the
   8-character code derives an AES-256-CTR key with **PBKDF2-SHA256, 131072
   (2<<16) iterations**. On `pair-success` verify the account signature
   (XEdDSA), sign the device identity (XEdDSA), send `pair-device-sign`, store
   the JID, reconnect and log in.
5. **Prekeys.** Upload signed prekey plus a batch of 50 one-time prekeys (iq
   `xmlns="encrypt"`, whatsmeow `WantedPreKeyCount`). Refill when the server
   reports fewer than 5 (`MinPreKeyCount`).
6. **Send to a contact.** `usync` iq for the recipient's device list (and our
   own); fetch prekey bundles for devices without a session; X3DH, Double
   Ratchet encrypt `Message{conversation}` or `Message{imageMessage}` padded;
   send `<message to type="text|media"><participants><to jid><enc type="pkmsg|msg"
   v="2">` and compare the returned `phash`. Addresses can be phone JIDs
   (`@s.whatsapp.net`) or LID JIDs (`@lid`); whatsmeow `send.go` mentions LID 45
   times, so LID handling is part of the send path, not an extra.
7. **Image.** Random 32-byte media key; HKDF-SHA256 with info
   `WhatsApp Image Keys` to 112 bytes (IV, cipher key, MAC key, ref key);
   AES-256-CBC, HMAC-SHA256 truncated to 10 bytes appended; `media_conn` iq gives
   hosts and auth; HTTPS POST to `https://<host>/mms/image/<token>`; put
   `url`, `directPath`, `mediaKey`, `fileSha256`, `fileEncSha256`,
   `fileLength`, `mimetype`, `caption` into `ImageMessage`.

| Primitive | Where | OpenSSL 3.5 |
|---|---|---|
| X25519 | Noise, X3DH, ratchet | `EVP_PKEY_X25519`, `EVP_PKEY_derive` |
| AES-256-GCM | Noise transport | `EVP_aes_256_gcm` |
| AES-256-CTR | pairing code | `EVP_aes_256_ctr` |
| AES-256-CBC | Signal message keys, media | `EVP_aes_256_cbc` |
| HKDF-SHA256 | Noise, ratchet chains, media keys | `EVP_KDF "HKDF"` |
| HMAC-SHA256 | ratchet, media MAC | `EVP_MAC "HMAC"` |
| PBKDF2-SHA256 | pairing code | `PKCS5_PBKDF2_HMAC` |
| SHA-256, SHA-512, MD5 | everywhere, XEdDSA, build hash | `EVP_Digest` |
| **XEdDSA sign** | signed prekey, pairing device signature | **not in OpenSSL** |
| XEdDSA verify | pairing account signature | Montgomery u to Edwards y, then `EVP_PKEY_ED25519` verify (to be proven) |

XEdDSA needs Edwards25519 scalar and point arithmetic, which OpenSSL does not
expose. The algorithm is 60 lines in libsignal-go `ecc/SignCurve25519.go`
(clamped scalar a, A = aB with sign bit forced to 0, r = SHA-512 of a
prefix, a, message and 64 random bytes, R = rB, s = r + SHA-512(R,A,M) a mod L).
The arithmetic underneath (filippo.io/edwards25519, BSD-3-Clause) is what has to
be ported or replaced. That is the one hard crypto item, see section 6.

## 6. Build versus deep research, decided up front

**Build** means mechanical transpile with a known oracle. **Deep research** means
the answer is not written down anywhere we have read, and has to be
established first, with an archive entry, before any code.

| Item | Kind | Oracle or open question |
|---|---|---|
| protobuf writer/reader, message structs | build | field numbers in whatsmeow `.proto`; wa-go `client_payload.json` trace |
| `WaCrypto` wrappers | build | RFC test vectors; wa-go `media_vectors.json` |
| framing, Noise, binary codec, tokens | build | wa-go `traces/connect_pair` (`noise.json`, `frames_raw.jsonl`, `nodes.jsonl`), `codec_battery/nodes.jsonl` |
| Signal X3DH, ratchet, session | build | wa-go `testdata/signal/session_ab.json` |
| pairing code crypto | build | wa-go `testdata/paircode/vectors.json` |
| media encrypt, upload | build | wa-go `media_vectors.json`, then one live send |
| usync, prekey upload, send node shape | build | whatsmeow source, then one live send |
| **XEdDSA sign** | **deep research** | port the Edwards25519 field/scalar/point subset (BSD-3, from filippo.io/edwards25519) or find a permissive C implementation; verify against wa-go `keys/testdata/verify_sig.mjs` style vectors |
| **current client version and 405 policy** | **deep research** | how often whatsmeow bumps `waVersion`; whether to fetch `client_revision` at start (adds one HTTPS call, kills the treadmill) |
| **LID addressing** | **deep research** | when the server hands out `@lid`, how sessions map phone JID to LID; read whatsmeow `send.go` and `store` LID tables |
| **ban behaviour** | **deep research, measured only with a sacrificial number** | no maintainer publishes heuristics; one vendor article names account age, message rate and reports |
| contact names without app-state | build | push names in incoming messages plus user aliases; app-state sync is v2 |

## 7. User flow (the customer view)

Two commands, nothing else to configure:

```
detz --whatsapp-link            # shows a QR in the terminal, scan in WhatsApp > Linked devices
detz --whatsapp-link 31612345678  # prints an 8-character code to type on the phone instead
```

Then, in any detz session:

```
> remember that Niggel is 31687654321 on WhatsApp
> create a screenshot, explain what is genius about it and send it to Niggel on WhatsApp
```

The first line calls `whatsapp_alias`. The second runs `shell` (screenshot),
`describe_image`, then `whatsapp_send{to:"Niggel", text, image}`. `whatsapp_send`
resolves the name through `WaContacts`; when the best match is not exact, the
tool result names what it resolved to, the same note pattern as `@agent`
mentions. `detz --whatsapp-unlink` logs the device out and deletes its rows.

State lives in the profile's vault (SQLite, the existing `store`), not in a new
file. The link step is the only step that touches the phone.

Agent tools:

| Tool | Arguments | Result |
|---|---|---|
| `whatsapp_send` | `to` (name, alias or number), `text`, optional `image` (path) | message id, resolved JID and name, `phash` match |
| `whatsapp_contacts` | optional `query` | known names and JIDs |
| `whatsapp_alias` | `name`, `number` | stored alias, after a usync check that the number is on WhatsApp |

## 8. Caps (all static, per CLAUDE.md "everything gets a hard cap")

| Constant | Value | Why |
|---|---|---|
| `kWaFrameMax` | 16 MiB (`1 << 24`) | protocol limit |
| `kWaImageMax` | 16 MiB | upload cap; refuse larger, loud |
| `kWaSendPerMinute` | 10 | ban exposure; the agent must not burst |
| `kWaSendPerDay` | 200 | same |
| `kWaRecipientsPerSend` | 1 | v1 is 1:1 only |
| `kWaPrekeyBatch` | 50 | whatsmeow `WantedPreKeyCount` |
| `kWaConnectTimeout` | 20 s | handshake plus login |
| `kWaPairTimeout` | 120 s | QR or code wait |
| `kWaReconnectFloor` | 5 s, doubling to 300 s | never hammer the server |
| `kWaContactsMax` | 5000 | contact and alias rows per profile |

These are judgments, not measurements. They become measured once the live
tests below run.

## 9. Tests

Unit (`tests/whatsapp_test.cpp`, `make test`):

- every golden vector from wa-go `testdata/` (copied with its MIT notice into
  `tests/whatsapp_data/`): Noise transcript, raw frames, decoded nodes, codec
  battery, ClientPayload bytes, Signal session A to B, pairing code, media.
- XEdDSA: sign then verify round trip, plus verify of a signature made by
  libsignal (vector to be produced once, archived).
- protobuf: encode equals the bytes in the traces, decode of every trace.

Integration (`tests/whatsapp.py`, real binary, real WhatsApp, per AGENTS.md):

- needs a sacrificial number linked once (`WA_TEST_PEER` names a second number);
- cases: link state survives restart, send text, send image, receive the
  reply text, `405` handling with a forced old version;
- verification reads vault rows (message id, session rows), never CLI text.

## 10. The shortcut for the day-after-tomorrow promise

The native module is roughly the size measured in section 2.1, so it does
not ship in two days. What can work now, with no C++:

1. clone `lharries/whatsapp-mcp`, **bump its whatsmeow** (`go.mod` pins
   `v0.0.0-20250318233852`, 19 months old; a stale whatsmeow risks the 405
   client-outdated failure), run `whatsapp-bridge` once and scan the QR;
2. add the MCP server to detz (`mcp_add`, stdio transport,
   `uv --directory .../whatsapp-mcp-server run main.py`);
3. the sentence then runs as `shell` + `describe_image` + `send_file`.

This path was **not run end to end**: it needs a linked phone, and a local
probe (an unpaired whatsmeow connect that prints QR events, which proves the
full handshake without a phone) was refused by this session's permission
classifier because it executes downloaded code. To run it yourself:

```
mkdir probe && cd probe && go mod init probe
go get go.mau.fi/whatsmeow@latest modernc.org/sqlite
# main.go: sqlstore.New("sqlite", "file:probe.db"), NewDevice, NewClient,
# GetQRChannel, Connect, print each event; expect "code" events within seconds
go run .
```

## 11. Phases

| Phase | Content | Exit test |
|---|---|---|
| 0 | MCP shortcut (section 10) | the sentence works once, live |
| 1 | `WaProto*`, `WaCrypto`, `WaXeddsa`, `WaSocket`, `WaNoise`, codec, tokens | all wire and crypto golden vectors green; live: QR events from the real server |
| 2 | `WaStore`, `WaPairing`, `WaPrekeys`, login | live: link by QR and by code, restart stays logged in |
| 3 | `WaSignal`, `WaUsync`, `WaSender`, `WaReceiver` | Signal vectors green; live: text to a peer and its reply decrypted |
| 4 | `WaMedia`, `WaContacts`, `WaTools`, CLI flags | live: the full sentence, then the MCP shortcut is removed |

Size guide (measured Go source to transpile, non-blank, non-comment): whatsmeow
files on the 1:1 path (handshake, pair, pair-code, prekeys, send, message,
receipt, upload, mediaconn, user, request, keepalive, client,
connectionevents, qrchan, retry, socket, binary, keys, crypto utils) total
8,616 lines, and only part of send, message and user is needed; wa-go wire,
keys, signal and media total 5,231. The C++ size is not measured until it exists.

## 12. Risks and gaps

- **Terms of Service.** Every unofficial client breaks WhatsApp's terms; every
  project README and the one vendor article found say a number can be banned.
  Use a sacrificial number for tests. The sanctioned path (Business Cloud API)
  cannot use a personal number.
- **Protocol drift.** whatsmeow changes about 6 times a week (78 commits in
  90 days). We follow whatsmeow; each release re-checks the version constant,
  the cert key and the token dictionary.
- **wa-go is young** (created 2026-06-26, one author, 1 star). Its golden
  vectors come from Baileys, which keeps them honest, but its code is not
  battle tested. whatsmeow stays the referee.
- **Unverified here:** the full handshake (probe refused), XEdDSA verify via
  OpenSSL Ed25519, the 20-day re-auth claim, the 405 behaviour.

## 13. Robustness against hostile input (incident WA-2026-10-27, and the standing rule)

Every byte this module parses comes off a network socket we do not control.
That makes the wire parsers (`WaBinary`, `WaProto`, and the Noise/Signal frame
readers) a security and stability boundary, not ordinary code. This section
records a crash that got through, exactly why, how it was fixed, and the rule
and test that keep it from ever coming back. **We do not accept regressions
here.**

### 13.1 What happened

Running `detz --whatsapp-link` on a real terminal, the wizard connected, showed
the QR, and after the code auto-refreshed it aborted:

```
terminate called after throwing an instance of 'std::length_error'
  what():  cannot create std::vector larger than max_size()
Aborted (core dumped)
```

A core dump, not a handled error. The process died mid-session instead of
logging a bad frame and carrying on.

### 13.2 Root cause

Two independent faults lined up:

1. **An overflowable bounds check.** Both the protobuf reader and the binary-node
   decoder guarded a length-delimited field with `pos + len > size`. `pos` is a
   `size_t` and `len` was read straight from the wire (up to 2^32-1, or a
   4-byte length sign-extended into a negative `int`). `pos + len` then wrapped
   around past `SIZE_MAX`, computed a small sum, and the check passed. The code
   went on to `vector::assign(begin, begin + len)` with a garbage `len`, and the
   standard library threw `std::length_error` from inside `vector`.
2. **An unguarded top-level loop.** `WaClient::dispatch` routed nodes straight
   into `handle_pair_success` and the other handlers with no `try/catch`. The
   decode path in `recv_node` was wrapped, but the handlers that re-parse
   sub-messages (the ADV identity in pairing, for instance) were not, so a
   `std::length_error` raised there propagated out of `main` and called
   `std::terminate`.

Either fault alone is survivable. Together they turn one malformed frame into a
process abort. This is the classic C++ failure: an arithmetic overflow in a
size computation silently defeats a bounds check, and an uncaught exception is
not a logged error but a dead process.

### 13.3 The fix (commit f16c5c6)

- **Overflow-safe length checks everywhere bytes are sliced.** The comparison is
  now `len > size - pos` (never `pos + len > size`). Because `pos <= size` is an
  invariant at every call site, the subtraction cannot wrap, so no attacker
  value can slip through. `src/whatsapp/wa_proto.cpp` (`ProtoReader::next`) and
  `src/whatsapp/wa_binary.cpp` (`Decoder::need`).
- **Lengths are read unsigned and range-checked.** The binary decoder reads
  4-byte lengths into a `uint32_t` (never a sign-extendable `int`), rejects any
  value larger than the whole frame, and `read_bytes` rejects negatives. A bad
  length now raises `WireError`, a `std::exception` we catch, long before any
  allocation.
- **Dispatch can no longer abort the process.** `WaClient::dispatch` wraps all
  node routing in `try/catch(const std::exception&)`; a bad node is logged
  (`dropped node <tag>: <reason>`) and the client keeps running.

### 13.4 The standing rule (project principle)

The goal is to make C++ safe to use here, which means a hostile or buggy peer
can waste our time but can never crash us or corrupt us. Three invariants, in
order of importance:

1. **No size arithmetic that can overflow a bounds check.** Never write
   `pos + len <= size`. Write `len <= size - pos` against an established
   `pos <= size`. Read lengths into an unsigned type wide enough to hold them,
   and range-check before you allocate or slice. Treat every length, count and
   index off the wire as adversarial.
2. **No unchecked allocation from an attacker-controlled count.** A length field
   is a claim, not a fact. Validate it against the bytes actually present before
   `reserve`, `resize`, `assign`, or a range constructor. A caller must never be
   able to make us ask for a 4 GB vector.
3. **No exception may escape a top-level loop.** Any loop that reads and
   dispatches untrusted input (`recv_node`, `dispatch`, the send and pairing
   waits) wraps the per-item work in `try/catch(const std::exception&)` and
   turns a failure into a logged, skipped item. An uncaught `std::exception` in
   this module is a bug even if the parse that raised it was "impossible".

These apply to any code that reads untrusted bytes, not just WhatsApp. When a
new wire format is added (CAP, MCP, a new media type), it inherits this rule.

### 13.5 How we stop the regression

Prose does not prevent regressions; a failing test does. `tests/whatsapp_test.cpp`
has a `test_hostile` section (run by `build/whatsapp_test`, step `whatsapp.unit`
in `tests/run_all.py`, and `make test`) that feeds the parsers exactly the shapes
that used to crash and asserts they throw a *handled* exception and specifically
**not** `std::length_error`:

- a binary node whose content length claims 0xFFFFFFFF in an 11-byte buffer;
- a protobuf field whose length varint is ~4 billion with no payload;
- truncated and short frames;
- 2000 random byte strings through both `WaBinary` and `ProtoReader` (surviving
  the loop is the proof: a single unchecked allocation would abort the run).

Each case `catch`es `std::length_error` and fails the test if it is thrown, so
if anyone reintroduces a `pos + len` check or an unguarded slice, this test goes
red. If you add a new parser, add its hostile cases here in the same pass; a
parser with no adversarial test is treated as unfinished.

### 13.6 Follow-up: the first fix was not enough, and how we proved it

Section 13.3's fix stopped the abort but was declared "verified" on a
hostile-input test that never exercised the live decode path. That was
overconfident: a later run still logged `dropped node <iq>: cannot create
std::vector larger than max_size()`. The crash was gone (the `try/catch` held),
but a real frame was still being mis-handled and dropped.

The cure was to stop reasoning and use a tool. Building the parsers under
AddressSanitizer + UndefinedBehaviorSanitizer + `_GLIBCXX_ASSERTIONS`
([`../tools/san-whatsapp.sh`](../tools/san-whatsapp.sh)) immediately found a
second, independent undefined-behavior bug that two manual reviews had missed:
a protobuf varint loop that shifted by up to 70 bits (shift >= 64 is UB) on
malformed input. It is now fixed and guarded by a deterministic test case and
the sanitizer net. Two more defenses were added:

- **Frame capture.** `WaClient` records the exact bytes of any frame that fails
  to decode or dispatch to `<home>/whatsapp/last-error.log` (bounded). The next
  time the still-unreproduced live `<iq>` failure happens, the precise input is
  on disk and becomes a one-paste regression test and a deterministic fix.
- **The sanitizer net is now the standing requirement**, not a one-off. See the
  project-wide playbook, [cpp-safety.md](cpp-safety.md), which this incident
  produced: overflow-safe checked-reader parsing, cheap always-on release
  hardening, the ASan+UBSan test net, and capture.

The honest open item: the exact frame the user hit is still unreproduced
in-session (the server did not resend it within a 160 s sanitizer run). The
capture hook exists precisely so that gap closes on its next occurrence rather
than by more guessing.

### 13.7 Root cause, found and fixed (the capture hook paid off)

The capture hook from 13.6 did its job: the next `--whatsapp-link` run recorded
the exact 726-byte frame to `<home>/whatsapp/last-error.log`, and replaying it
made the bug deterministic in minutes. The verdict overturns every earlier
guess: **it was never a parser bug.** The frame decoded perfectly (it was a
normal `pair-success` with a `device-identity`, `device`, `biz`, and
`encryption-metadata`). The `std::vector larger than max_size()` was thrown by
`WaDevice::serialize()`, in `WaClient::handle_pair_success` saving the newly
paired device:

```cpp
put_blob(b, Bytes((jid ? *jid : "").begin(), (jid ? *jid : "").end()));
```

`(jid ? *jid : "")` builds a **fresh temporary `std::string` for each
evaluation**, so `.begin()` and `.end()` point into two different, already
destroyed objects. The range constructor computed a garbage length and the
vector threw. It only misfired once `jid` was set, which is why it appeared
only after the phone scanned, and why the initial unpaired save and every
parser test passed. The fix binds the value to a named `const std::string`
before taking iterators. Verified by replaying the captured frame end to end
(serialize, save, encode the response: no throw) and by a paired-device
serialize round-trip test (`tests/whatsapp_test.cpp` `test_device`) that runs
under the sanitizer net.

The lesson is in [cpp-safety.md](cpp-safety.md): this was a C++ object-lifetime
bug, a different class from the earlier size-overflow, and it slipped past the
parser-focused tests and the first sanitizer runs because those never exercised
`serialize()` with a set optional. Two defenses were strengthened as a result:
the capture hook (which turned an unreproducible failure into a deterministic
one), and the rule that a round-trip test must cover the populated case, not
just the empty one. The honest open item from 13.6 is now closed.

## 14. App-state sync: the contact list (`WaAppState`, `WaReceiver`)

Status: implemented and verified offline. The live path needs a re-linked
phone: nothing here has been run against a phone that holds contacts.

Why: the phone's address book reaches a linked companion only through app-state
sync (collection `critical_unblock_low`, index `["contact", "<jid>"]`). Without
it `wa_contact` stays empty and names cannot be resolved.

| Piece | Files | What it does |
|---|---|---|
| Key receipt | `wa_receiver.*`, `wa_proto.*` (`parse_protocol_message`), `WaClient::handle_message` | Decrypts `<enc>` from our own account (pkmsg via X3DH responder, msg via session), unpads, stores each APP_STATE_SYNC_KEY_SHARE key in `wa_appstate_key`. Own means our phone number or our LID (whatsmeow `IsFromMe`); other senders are acked and ignored as before. Every arrival is logged before the filter, so a trace shows dropped mail too. An empty share (keys without key data, openclaw/wacli PR 443) is reported and ends the wait without storing; a consumed peer message gets a `peer_msg` protocol receipt. |
| Key request | `WaClient::request_app_state_keys`, `send_peer_message`, `encode_app_state_key_request` | Peer message to the primary phone (category `peer`, meta + enc, device-identity only on a prekey message), at most `kWaAppStateKeyIdsMax` ids, one request per key id per `kWaAppStateKeyRequestInterval` (24 h), stored in `wa_appstate_keyreq`. The send is verified (Signal session must exist, the server ack must carry no error); when it fails the 24 h marks are rolled back so the next sync asks again (whatsmeow PR 1268). |
| Notifications, receipts | `WaClient::handle_notification`, `handle_receipt` | `server_sync` re-syncs the contact collection (other collections are named and skipped); `encrypt` with fewer than `kWaMinPrekeys` left refills prekeys; a retry receipt (the phone could not decrypt our message) is logged loudly. Notification syncs never nest inside a sync (guard + one more pass). |
| Key expansion, MACs, values | `wa_appstate.*` (`WaAppStateKeys`, `WaAppStateCrypto`) | HKDF-SHA256 `WhatsApp Mutation Keys` to 160 bytes; value MAC HMAC-SHA512 truncated to 32; index, snapshot and patch MACs HMAC-SHA256; AES-256-CBC values. `WaCrypto::hmac_sha512` added. |
| LTHash | `wa_appstate.*` (`WaLtHash`) | 128-byte state, HKDF-SHA256 info `WhatsApp Patch Integrity` (whatsmeow, not BLAKE2b; see research conclusion `appstate-lthash-hkdf-not-blake2b`). |
| Patch fetch and apply | `WaAppStateSync::sync` | iq `w:sync:app:state`, `return_snapshot` at version 0, `version=N` otherwise; external blobs via `WaMedia::download` (`md-app-state`); snapshot MAC, patch MAC, index and value MACs checked before anything is stored. |
| Contacts | `WaAppStateSync::apply_contact` | SET stores jid = index[1], name = fullName else firstName; empty name is counted and skipped; REMOVE deletes the row. |
| Push-name learning | `WaClient::handle_message`, `push_name_candidate`, `WaStore::put_contact_if_absent` | Every incoming message carries the sender's self-reported name in `notify` (whatsmeow `message.go`); group senders come from `participant`, the PN alternate is preferred when present. Stored only when the jid has no name yet, so a saved, imported or aliased name always wins; truncated to `kWaPushNameMax` (128). No server query returns names for regular users (`GetUserInfo` usync has status, picture, devices, LID and business verified names only), so this plus VCF/alias/app-state is the complete name-resolution stack. |
| Login hook | `WaClient::login` calls `sync_app_state` after prekeys | Never fails the login (errors are logged). If a key is missing, requests it, waits up to `kWaAppStateKeyWait` s for a key share, then syncs once more. |
| Long sync | `detz --whatsapp-sync` (`WaWizard::sync`) | Logs in, re-syncs with a `kWaSyncOnlineWait` (300 s) key wait for a sleeping phone, reports the contact count. Exit 0 with contacts, 1 without, 2 when not linked or login failed. |
| VCF fallback | `detz --whatsapp-import FILE` (`WaVcf`) | Imports a .vcf exported on the phone (FN else N, TEL lines, international digits) into the same contact search. Loud skips, capped at `kWaContactsMax`, file capped at `kWaVcfFileMax`. |

Caps (`wa_consts.h`): `kWaAppStatePatchMax` 50 patches per sync,
`kWaAppStateMutationMax` 10000 mutations, `kWaAppStateTimeout` 30 s per iq,
`kWaAppStateBlobMax` 16 MiB per external blob, `kWaAppStateKeyIdsMax` 16 key
ids per request, `kWaAppStateKeyWait` 20 s, `kWaSyncOnlineWait` 300 s,
`kWaVcfFileMax` 8 MiB. A cap stops the sync loudly
(`capped` in the report, a log line) and the rest waits for the next login.

Failure rule: a patch or snapshot that fails a MAC or hash check is not stored,
the sync stops, and the collection version stays at the last verified state.
This skips the bad patch and everything after it, because each later patch
depends on the hash chain.

Tables: `wa_appstate_key`, `wa_appstate_version`, `wa_appstate_mac` (value MAC
per index MAC, needed to undo a SET or REMOVE), `wa_appstate_keyreq`. `clear_device`
empties them, so a new link syncs from version 0.

Verification (offline, `build/whatsapp_test`, section `appstate`):

- expanded keys equal whatsmeow `expandAppStateKeys` bytes;
- LTHash add/remove cancel and are order-free;
- a snapshot plus one patch (SET, REMOVE, in-patch replacement, unnamed SET, a
  non-contact SET) decodes to the oracle's final hash and version, and the
  contact table matches the oracle's expected names;
- a re-sync at version 2 changes nothing;
- a tampered patch is rejected and not stored;
- a missing key stores nothing and reports the key id;
- the key share and key request bytes match whatsmeow's Message encoding;
- a pkmsg built with `WaSession` decrypts through `WaReceiver` to the key share
  Message, and a corrupted pkmsg is refused;
- the 24 h request mark can be cleared for an immediate retry;
- own senders accept PN and LID, the peer node carries device-identity only on
  a prekey message, server_sync collections and retry receipts parse;
- an empty key share parses with empty key data;
- a 5-card .vcf imports 4 numbers (FN, N fallback, unfolding, 00 form) and the
  typo-tolerant search finds them;
- push-name candidates parse for 1:1 (PN, LID, PN alternate), groups
  (participant, PN alternate), and reject own senders, `-`, missing notify,
  non-user senders and over-long names; fill-if-absent keeps the first name and
  never overwrites a saved one.

The oracle is `tests/whatsapp_data/appstate_vectors.json`, generated by
`tests/whatsapp_data/appstate_oracle.go` (a Go program using whatsmeow's own
`lthash`, `hkdfutil` and protobuf packages). It is not run by `make test`.

Not verified: the live server's answers to the iq, key share timing, the media
host for external blobs, and whether a REMOVE of an unknown index (whatsmeow
only warns) causes trouble. Known gaps: a SET with an empty name keeps any
older name the row had (the spec says skip, not delete); a retry receipt is
logged, not answered with a resend (whatsmeow `retry.go`); history sync
notifications are not downloaded.



## 15. Incident WA-2026-10-27b: link ended in 401, no picture, no contacts

Everything here was measured on 2026-10-27/28 against the live server and the user's phone.

| Symptom | Root cause | Fix | Evidence |
|---|---|---|---|
| `Linked, but the login confirmation did not complete`; later login `failure 401` | The decoder printed a server-only JID (empty user) as `@s.whatsapp.net`; the pairing replies copied that into `to`, so the server refused `pair-device-sign` and never registered the device | decoder returns the bare server name; encoder writes bare server names as JID pairs; replies use `WaConsts::kServerJid` as whatsmeow does | captured `pair-success` decoded (`from=@s.whatsapp.net`), live login probe 401 before, `logged in` after re-link; test `server-only JID round-trips` |
| wizard never reported the real login result | `run_until` returned at once because `paired_event_` stayed set | `run_until` is stateless; `WaClient::await_pair_restart` + `WaClient::login`; one fresh pairing after a 401/403 (`kWaLinkRetries`) | wizard source |
| `user 7m` CPU while linking | a closed socket made every receive return instantly and the loops spun until their deadline | `WaSocket` clears `connected_` on close, every loop checks `WaClient::socket_open()` | probe after the 401: 1.7 s, 0.01 s CPU (was 20 s of spinning) |
| PNG could not be sent | the `whatsapp_send` tool had no image argument and `send_image` was a stub | `WaMedia` (HKDF `WhatsApp Image Keys`, AES-CBC, 10-byte HMAC, `media_conn`, HTTPS upload), `send_image`, DeviceSentMessage copy to our other devices, tool argument `image` | 8/8 wa-go golden vectors byte-exact; live: image to own number acknowledged by the server |
| Nigel not found | the phone address book reaches a companion only through app-state sync, which did not exist; and the phone only pushes history and keys to a companion that declares `DeviceProps.historySyncConfig`, which registration omitted | section 14 (`WaAppState`, `WaReceiver`); `WaDeviceProps` now sends `historySyncConfig` (field 5, whatsmeow values); peer key request encrypted to the phone's LID identity | unit tests; the live address book needs a link made with this build |
| name lookups were exact or one fuzzy guess | contact table only, one `FuzzyMatcher::Best` | `WaStore::search_contacts`: FTS5 trigram candidates + fuzzy rank; ambiguity (two scores within 0.08) is reported, never guessed; the global `search` tool lists matching contacts | tests `contact search: ...` |
| no send limits | caps were specified, not enforced | `wa_send_log`, `kWaSendPerMinute` 10 and `kWaSendPerDay` 200 enforced in `whatsapp_send` | test `send log counts sends in the window` |

Gaps: the contact sync and the address-book key share are not verified against a phone yet; a newly linked device
must be created with this build for the phone to send history. Set `DETZ_WA_TRACE=1` to log every node's tag and
attributes when debugging a protocol exchange.

## 16. Incident WA-2026-10-28: the phone never sends the app-state sync key

Full record: `problem.md`. A newly linked companion (`31630045827:54`) logged
in, uploaded prekeys and sent an image to its own number (visible in "Message
yourself", participant hash matching the server's), but the phone never sent
the app-state sync key: 100 to 120 s of `DETZ_WA_TRACE=1` showed only
`<success>`, `<ib edge_routing>` and `<ib dirty>`, no `<message>` at all.

The request path was checked line by line against whatsmeow `send.go`,
`message.go`, `appstate.go`, `notification.go`, `client.go`, `pair.go` and
`store/clientpayload.go` (main, fetched 2026-10-27). Registration, login,
DeviceProps (including `historySyncConfig`), the peer node shape, the key
request bytes and the padding all match. whatsmeow answers the open questions
this way:

- A peer message needs a Signal session with the phone first:
  `encryptMessageForDevice` with no bundle and no session returns
  `ErrNoSession` and the request never leaves the process (hence whatsmeow PR
  1268, which rolls back the 24 h mark on send failure, and Volt-Chat PR 5,
  which adds an explicit session setup).
- Own senders are the PN user or the LID user (`parseMessageSource`); no
  `recipient` attribute is needed. The `<ib dirty type=account_sync>` node
  needs no answer (whatsmeow `handleIB` leaves it alone).
- The `jurisdiction` and `encryption-metadata` nodes in `pair-success` are
  server-side account metadata; whatsmeow `handlePairSuccess` ignores every
  child it does not know, as we do. Not related to key sharing.

Fixes in this build (all local; the phone side is still unseen):

| Gap | Fix | Evidence |
|---|---|---|
| a failed key request still stamped the 24 h mark, blocking the retry | `clear_app_key_request_time`, rollback on send failure (whatsmeow PR 1268) | test `key request rollback` |
| a peer send was trusted without checking the session or the ack | session verified after the prekey fetch, ack error attribute checked and logged (Volt-Chat PR 5) | code, log lines |
| device-identity rode on every peer message, whatsmeow only on pkmsg | `build_peer_message_node` matches whatsmeow | test `peer message helpers` |
| key shares from our LID were silently dropped (PN-only filter) | `is_own_sender` accepts PN or LID, every arrival is logged before the filter | test `peer message helpers` |
| an empty key share looked like no answer | stored/empty counted separately, the wait ends, `peer_msg` receipt sent (openclaw/wacli PR 443) | test `empty key share` |
| notifications and receipts were ignored | `server_sync` re-syncs, `encrypt` refills prekeys, retry receipts logged loudly | test `peer message helpers` |
| 20 s online is short for a sleeping phone | `detz --whatsapp-sync` stays online 300 s (`kWaSyncOnlineWait`) | wizard source |
| no fallback when the phone never answers | `detz --whatsapp-import FILE` imports a .vcf into the contact search | test `vcf import` |

What to run now: link fresh if needed, then `detz --whatsapp-sync` with
WhatsApp open on the phone (no battery restriction). If no `<message>` arrives
within the 300 s, the phone is gating on its side; import the .vcf and retry
the sync another day. Research: `research/whatsapp-automation/`
(conclusion `key-share-never-arrives.md`).

## 17. Transpilation pin, harness and reuse table (M0, 2026-10-28)

The full whatsmeow transpilation (`full_transpilation_and_integration.md`,
milestones M0..M10) starts here. This section holds the pinned source, the
harness that gates every milestone, the baseline coverage, and the moloapp
reuse table. The pin never moves.

### 17.1 Pinned source

| Field | Value |
|---|---|
| Module | `go.mau.fi/whatsmeow` |
| Version | `v0.0.0-20261027104441-1eec1136a50b` |
| Commit | `1eec1136a50b2ef148b05f1819e77a9d989e0457` |
| Commit date | 2026-10-27T10:44:41Z |
| Location | `$(go env GOMODCACHE)/go.mau.fi/whatsmeow@v0.0.0-20261027104441-1eec1136a50b` |
| Provenance | `cache/download/go.mau.fi/whatsmeow/@v/<version>.info`, `Origin.Hash` |

`tools/wa-api-inventory.sh` refuses to run when the directory is missing or
the `.info` hash differs from the pinned commit, so a moved cache fails
loudly instead of silently rescoping the work.

### 17.2 Harness

| Tool | Job |
|---|---|
| `tools/wa-api-inventory.sh` | Extracts the mechanical scope into `build/wa-inventory.tsv` (`package, kind, key, file`) and the `proto.Marshal`/`proto.Unmarshal` call sites into `build/wa-marshal-sites.txt`. Re-verifies the pin on every run and aborts on a duplicate key (parser self-check). |
| `tools/wa-mapping.tsv` | Committed source of truth: every inventory key maps to its C++ counterpart (`detz::wa::...`, file) or to `UNMAPPED`. Mapping rule: a named C++ entity (function, method, type, const, field) performs the counterpart role; an inline fragment with no named entity stays `UNMAPPED` for a later milestone to name. |
| `tools/wa-coverage.sh [--slice FILE]` | Re-runs the inventory, then gates: the mapping key set must match the inventory exactly (a missing or stale key fails as "out of sync"), and every gated entry must be mapped. Exit 0 only at 100%. A slice file (`file:`, `pkg:`, `sym:`, `re:` lines) scopes the gate to one milestone's API. |
| `tools/wa-stub-scan.sh` | Fails on absolute stub patterns (`TODO`, `FIXME`, `XXX`, `HACK`, `unimplemented`, `not implemented`, `NOT_IMPLEMENTED`, `panic(`, `assert(false`, `std::abort`) anywhere in `src/whatsapp/`, and on any new `return std::nullopt` / `return false` / `return {}` (diff vs `HEAD`, plus untracked files) without a log line (`log(`, `log_(`, `TRACE`, `cerr`, `stderr`, `fprintf`, `perror`) within the 6 lines above it or on its own line. No allowlist. |

Scope rules (mechanical, encoded in the inventory script): in scope is every
exported symbol (func, method, type, const, var) in non-test, non-generated Go
source. Out of scope: `*_test.go`; files whose header marks them generated
(`DO NOT EDIT`, `DO NOT MODIFY`, `generated by`: the 57 `*.pb.go` files,
`internals.go`, `binary/proto/legacy.go`); files with `//go:build ignore`
(`internals_generate.go`). A generated protobuf message counts as in scope
when `<pkg>.<Msg>` is referenced by qualified name in an in-scope file (69
marshal call sites motivate the 129 used messages). Methods on unexported
receivers with exported names (`patchOutput.AddMAC`, `waLog.noopLogger.*`)
stay in scope: the name is exported, so it is listed and must map.

Validation (2026-10-28): the extractor was diffed against `go doc` as an
independent method. `Client` methods match exactly (the two `go doc` extras
are the `NewClient` constructor, inventoried as `func`, and
`DangerousInternals`, defined only in the excluded generated/ignored files).
The coverage gate was shown to fail below 100% and pass on an all-mapped
slice; the stub scanner was shown to fail on an injected `TODO` and on a new
silent `return false` (tracked and untracked files) and to pass on a logged
one and on the clean tree.

### 17.3 Baseline coverage (detz 1.46.1)

`tools/wa-coverage.sh`: **137/1500 mapped (9.1%)**, exit 1 (expected: the
100% gate is the M10 exit, per-milestone slices gate M1..M9).

| Kind | Mapped | Total |
|---|---|---|
| const | 13 | 330 |
| func | 14 | 60 |
| method | 49 | 523 |
| proto | 28 | 129 |
| type | 24 | 320 |
| var | 9 | 138 |

Mapped packages (mapped/total): appstate 8/136, binary 18/46, cbcutil 2/5,
gcmutil 2/3, hkdfutil 1/1, keys 4/7, lthash 3/4, socket 17/34, store 26/143,
types 6/232, waAdv 3/3, waCompanionReg 2/10, waE2E 7/35, waServerSync 6/9,
waSyncAction 2/19, waWa6 8/8, whatsmeow 22/369. Everything else (groups,
newsletters, calls, presence, pair-code/passkey, history sync, inbox, read
receipts, retry-resend, event bus, sqlstore schema, error taxonomy, the
remaining crypto/store/util surface) is `UNMAPPED`, claimed by M1..M10 in the
milestone table. The 137 mapped entries were each verified against the header
or source named in `tools/wa-mapping.tsv`; subset notes on the value name the
gap the owning milestone closes.

### 17.4 moloapp reuse table

Prompt section 5: before writing any HTTP client, SSE/websocket, LLM
provider, config, CLI, tenancy or app-pattern code, grep moloapp
(`../../moloapp/src/`, 1039 headers) and record the decision here. Reuse is
by include plus link only; moloapp is never modified. A duplicated facility
fails review.

| Date | Need | moloapp facility checked | Decision |
|---|---|---|---|
| 2026-10-28 | M0 harness scripts | none applicable (bash + python3 stdlib; no HTTP/SSE/LLM/config/CLI surface) | wrote local `tools/wa-*.sh` |
| 2026-10-28 | WebSocket client transport | `realtime/websocket/client/websocket_client.h` (exists, but needs a caller-provided transport: the caller does TLS and the HTTP handshake itself) | keep local `WaSocket` on libcurl: live-verified against the real server (connect + frames), libcurl already linked. Revisit if libcurl ws proves insufficient (e.g. M1 keepalive pings). |
| 2026-10-28 | media HTTPS upload/download | `clients/` HTTP client, pool, proxy (heavier connection-pool stack) | keep local `WaMedia` on `curl_easy` (`wa_media.cpp`): two calls (one POST, one GET), libcurl already linked |
| 2026-10-28 | vault storage for device/sessions/contacts | detz-owned `store::SqlDb` (already the vault; the schema is detz's) | keep local `WaStore` on `store::SqlDb` (`wa_store.h`); no moloapp store facility needed |
| 2026-10-28 | M2 store tables (identities/LID/contacts/secrets/tokens/settings) | `tenancy/pool_factory.h` (sqlite named only in a comment), `cli/cli/cli_app.cpp` (scaffolder `--backend sqlite` flag): no store facility | keep `WaStore`/`WaLidMap` on detz-owned `store::SqlDb`; schema is detz's |

## 18. M2 store (2026-10-28)

Milestone M2 transpiles the whatsmeow store area named in the plan-table row:
identities with trust tracking, the LID map, sessions, contacts
(saved/push/business), privacy tokens, message secrets and chat settings.
Coverage slice `tools/wa-slice-m2.txt` holds the 60 inventory keys;
`tools/wa-coverage.sh --slice tools/wa-slice-m2.txt` reports 60/60 (100%).

### 18.1 What landed

New classes (one per `.h`/`.cpp`, namespace `detz::wa`):

- `WaJid` (`wa_jid.*`): the JID subset the store needs (parse, `str`,
  non-AD form, PN/LID classification, signal user), mirroring whatsmeow
  `types/jid.go`, proven byte-for-byte against the oracle (`jid_non_ad`,
  `signal_user:*`, `jid_parse_fails` vectors).
- `WaLidMap` (`wa_lidmap.*`): the cached LID map (`CachedLIDMap` plus
  `Device.GetAltJID` as `alt_jid`).
- `WaStore` gains the `SessionStore` remainder (`has_session`,
  `get_many_sessions`, `put_many_sessions`, `delete_session`,
  `delete_all_sessions`, `migrate_pn_to_lid`), the full `IdentityStore`,
  `ContactStore`, `ChatSettingsStore`, `MsgSecretStore` and
  `PrivacyTokenStore`, with the `WaContactInfo`, `WaChatSettings`,
  `WaPrivacyToken`, `WaMessageSecret`, `WaContactEntry`, `WaRedactedPhone`
  and `WaMessageSecretInsert` structs.

New tables in the profile database (`wa_identity`, `wa_sender_key`,
`wa_chat_settings`, `wa_msg_secret`, `wa_privacy_token`, `wa_lid_map`) plus
the five whatsmeow contact columns on `wa_contact` (added with
`ALTER TABLE` when missing, so older databases migrate in place). The
detz `name` column and the FTS index keep their meaning (aliases, VCF
imports, learned push names); the whatsmeow name columns live beside them.
`clear_device` now also drops identities, sender keys, secrets, tokens and
chat settings (a relinked device must not inherit trust); contacts and the
LID map survive, like upstream, where both outlive a device delete.

New static caps in `wa_consts.h`: `kWaIdentityKeyLen` (32),
`kWaStoreJidMax` (1024), `kWaStoreBlobMax` (1 MiB, enforced before
allocation on every store blob read), `kWaStoreBatchMax` (300, the
upstream contact batch size), `kWaMutedForever` (253402300799, pinned by
the oracle against `store.MutedForever.Unix()`).

### 18.2 Tested divergences from upstream (all oracle-pinned)

Four deliberate deviations, each recorded with its failing input:

1. **Signal address format.** detz stores `user.device` (see
   `WaClient::addr_of`); upstream stores `user:device`. Prefix operations
   (`delete_all_sessions`, `delete_all_identities`, the migrate arms) use
   `LIKE 'phone.%'` where upstream uses `LIKE 'phone:%'`. Observable
   behavior is identical; the vectors record outcomes, never addresses.
2. **Migrate renames plain users.** Upstream `MigratePNToLID` renames
   signal users, so LID-side rows embed the agent (`100000000000001_1:0`,
   pinned in `mig_lid_addr`); detz addresses have no agent suffix, so
   `migrate_pn_to_lid` renames plain users (`100000000000001.0`), the only
   spelling detz lookups use.
3. **`DeleteIdentity` is exact.** Upstream accidentally deletes with the
   `LIKE` query, so deleting `9%9:0` also wiped `999:0` (the oracle
   records this in `trust_pct_sibling_after_pct_delete`). detz deletes
   with `=`: the sibling row survives, proven by the test.
4. **Not-muted representation.** Upstream reads a missing/cleared mute as
   zero `time.Time` (Unix -62135596800); detz reads it as 0. The stored
   row (0) is identical; both mean not muted (`chat_*_zero` vectors).

Two loud-error mirrors: `put_privacy_tokens({})` throws (upstream fails
its SQL on empty input; both loud), and a corrupt identity length throws
`SqlError` (upstream `ErrInvalidLength`).

### 18.3 Evidence

- Oracle: `tests/whatsapp_data/store_oracle.go` drives pinned whatsmeow
  itself (`sqlstore` over scratch SQLite) through one ordered op script
  and emits `tests/whatsapp_data/store_vectors.json` (105 keys).
- Differential test: `test_store_m2` in `tests/whatsapp_test.cpp`
  replays the script against `WaStore` and compares every outcome;
  `test_store_hostile` adds corrupt rows, overlong keys, malformed JIDs,
  wide inputs (2000-address get-many, 650-row chunked batch) and a
  3000-string fuzz loop with a JID round-trip check.
- `build/whatsapp_test` green; `tools/san-whatsapp.sh` green (the script
  was broken — it never compiled the store TUs — and now builds every TU
  the test exercises; the first sanitizer run caught a real bug, the
  agent-suffixed migrate address in section 18.2.2, plus unsafe optional
  derefs in the test, both fixed); `make`, `make lint` (with `-Werror`)
  and `tools/wa-stub-scan.sh` green (one harness fix: a `grep` with no
  match in a clean untracked file aborted the scan under `pipefail`).

### 18.4 Left for later milestones (not M2 scope)

Sender-key put/get (M4, the table and the migrate arm exist),
app-state sync-key/version/MAC stores (M7, beside the existing M0
app-state port), prekey upload tracking (`GetOrGenPreKeys`,
`MarkPreKeysAsUploaded`, `UploadedPreKeyCount`, `RemovePreKey`; M3),
the event/retry buffer (M4), the device container and `NoopStore`
(M10/integration), rewiring push-name learning to `put_push_name` (M4/M8,
which own receive and user events), and searching the synced
first/full names via FTS (M7).