tai/taicpp/docs/whatsapp.md

whatsapp (taicpp): research result and build specification

whatsmeow is an open source Go library for the WhatsApp Web multidevice API (author Tulir Asokan, module go.mau.fi/whatsmeow, license MPL-2.0). It speaks the WhatsApp protocol directly over one WebSocket, with no browser involved: QR and pair-code linking, Noise handshake, Signal encryption, send and receive, media upload, and app-state (contact) sync. detz does not link or ship it: the native C++ module detz::wa is transpiled from wa-go (MIT) with whatsmeow as the behavior reference (section 3), and the test oracle tests/whatsapp_data/appstate_oracle.go imports its Go packages to generate byte-exact vectors. Any comment saying "ported from whatsmeow" means the logic or byte layout was copied from upstream and must stay compatible with it. Full comparison: section 2.1.

Status: built and integrated; the offline core is verified, the live link is verified through QR display, message send is built but unverified (it needs a phone-linked account, which development does not have). What is in the tree now (src/whatsapp/, namespace detz::wa):

Piece	Files	State
WaCrypto (X25519, AES-GCM/CTR/CBC, HKDF, PBKDF2, SHA, XEdDSA)	wa_crypto.*	verified: RFC vectors + XEdDSA roundtrips
WaBinary (binary-XML codec, token dict v3)	wa_binary.*, wa_tokens_data.h, wa_node.h	verified: wa-go codec battery 19/19
WaProto (protobuf + the needed messages)	wa_proto.*	verified: ClientPayload byte-exact
WaNoise (Noise XX + transport)	wa_noise.*	verified: golden transcript and live server
WaSignal (X3DH + Double Ratchet)	wa_signal.*	verified: golden session 4/4 byte-exact
WaDevice (identity + ClientPayload)	wa_device.*	verified via the handshake
WaSocket (libcurl WebSocket)	wa_socket.*	verified live (connect + frames)
WaClient (connect, pair, login, send)	wa_client.*	connect/QR live-verified; pair-complete + send built, unverified; M3: pair-code, passkey, logout, hardened pair-success, all built, unverified
WaPairing (pair-code crypto, QR builder, pair/logout nodes, pair errors)	wa_pairing.*	verified: Go oracle byte-exact (section 18)
WaPasskey (WebAuthn, prologue, sealed request)	wa_passkey.*	verified: Go oracle byte-exact; no in-tree authenticator (section 18.2)
WaQrChannel (timed code pump) + pairing events	wa_qr_channel.*, wa_pair_events.h	verified: fake-clock unit tests
WaStore (vault SQLite)	wa_store.*	verified: round-trips device + contacts
WaQr (terminal QR)	wa_qr.*	verified: renders a scannable code
WaWizard (--whatsapp-link)	wa_wizard.*	live-verified through QR display + device persist; M3: --whatsapp-link PHONE code flow + server logout on unlink, built, unverified
agent tools whatsapp_send, whatsapp_contacts	src/agent/detz_tools.cpp, tools_ops.cpp	wired; contacts verified, send unverified

Verified against the live server (no phone needed): detz --whatsapp-link connects, completes the Noise handshake, receives the pair-device node, renders the QR, and saves the device. A phone scanning that QR would link it; the pair-success handler (ADV HMAC + account-signature verify, device signing, pair-device-sign) is ported faithfully from whatsmeow but cannot be exercised in-tree. whatsapp_send performs a real usync + prekey-fetch + Signal-encrypt + send-node sequence; it is faithful to whatsmeow but unverified without a live linked account. Unit test: build/whatsapp_test (all sections green, M3 adds the four pairing sections), vectors in tests/whatsapp_data/. Dependencies added: none at runtime (OpenSSL was already linked for the static build; added to the default link), plus two vendored libraries, third_party/ed25519 (zlib) and third_party/qrcodegen (MIT).

Remaining to make send fully work: own-device fan-out (DeviceSentMessage), media upload for send_image (WaMedia), prekey upload after login, retry receipts, and a live test with a sacrificial number.

Original research and plan below (kept as written; some "not built yet" lines are now superseded by the table above).

---

This file states what was found, what was decided, and the exact plan to build a native C++ WhatsApp client into detz. Research archive: ../../research/whatsapp-automation/ (sources and three conclusions). Every number below was measured on 2026-10-27 on this machine unless a line says otherwise.

1. Goal

One sentence must work in detz:

"Create screenshot and explain what is genius about it and send to Niggel on WhatsApp"

The flow breaks into four steps. Three of them already exist:

Step	Owner	Exists
take a screenshot	shell tool (grim, scrot, import)	yes
explain it	describe_image tool (src/agent/tools_core.cpp)	yes
resolve "Niggel" to a WhatsApp JID	WaContacts (this spec)	no
send text plus image	whatsapp_send tool over WaClient (this spec)	no

The missing part is one module: a WhatsApp companion device written in C++.

2. Answers to the questions asked

2.1 Best Python libraries for a normal phone number

There is no Python library that speaks the current WhatsApp protocol itself. Measured on the cloned repositories:

Library	Language of the protocol code	How it works	Verdict
neonize (Apache-2.0, 759 stars, 62 commits in 90 days)	Go (whatsmeow)	4362 lines of Go export 93 C functions (//export), 14918 lines of Python call them through ctypes	best "Python" option, but it is whatsmeow with a Python skin
whatsfly	Go (whatsmeow)	same idea, PyPI 0.0.1, linux 32 bit only	early, not usable
whatsapp-mcp (lharries, MIT)	Go (whatsmeow)	Go bridge with HTTP /api/send, Python MCP server with send_message, send_file	usable shortcut, see section 10
yowsup (GPL-3.0)	Python	old phone-registration protocol (e1.whatsapp.net:443), last commit 2021-12-14, README warns that registration gives an instant ban	dead, do not use
pywhatkit, Astra, wa-automation, whalesong	Python	drive WhatsApp Web in a real browser	browser, rejected
Supbot2	Python	drives the Android app with Appium	needs a phone or emulator, rejected
pywa and other Cloud API clients	Python	official Meta Business Cloud API	needs a business number; the number cannot stay on the normal app, rejected for a personal number

Conclusion: transpiling Python gets us nothing. The real implementations are:

Project	Language	License	Activity (commits in last 90 days)	Size (non-blank, non-test, non-generated)
whatsmeow (tulir)	Go	MPL-2.0	78	24,301 lines, plus 107,816 generated protobuf lines
libsignal-protocol-go (tulir), used by whatsmeow	Go	GPL-3.0	not measured	5,416
wa-go (jfelipesjc)	Go	MIT	17, created 2026-06-26, 1 star	23,278 (wire 2778, signal 1625, keys 311, media 517, client 14379)
Baileys (WhiskeySockets)	TypeScript	MIT	7	48,898
whatsapp-rust (oxidezap)	Rust	MIT	at least 100 (API page limit)	about 650,000 including VoIP and plugins
libsignal-protocol-c (signalapp)	C	GPL-3.0	last commit 2020-03-27	29,456

No C or C++ library implements the multi-device protocol. Search found only WebKit wrappers (WasIstLos, YAWF) and a REST wrapper (Green-API).

2.2 Is a browser required

No. whatsmeow, Baileys, wa-go and whatsapp-rust all speak the protocol directly over one WebSocket. The phone is needed exactly once, to link the device: scan a QR code shown in the terminal, or type an 8-character pairing code into WhatsApp > Linked devices. After that the companion runs alone. The phone must come online now and then; one directory page reports re-authentication after about 20 days (not verified).

2.3 Can it be done in C++ with what detz already links

Yes. detz already links libcurl and OpenSSL:

• libcurl 8.18.0 on this machine lists ws wss as protocols. curl -H "Origin: https://web.whatsapp.com" wss://web.whatsapp.com/ws/chat answered 101 Switching Protocols, so the transport is proven reachable from here with the library we already ship.
• OpenSSL 3.5.5 covers every primitive except XEdDSA signing (section 5).
• Protobuf needs no library: we need about 15 messages, and a hand-written varint and length-delimited writer and reader is about 200 lines.

New dependencies: none.

3. Decision: native C++, transpiled

Build detz::wa as a native C++ module. Transpile from two sources:

1. wa-go (MIT) is the code source for wire, Noise, binary codec, keys, Signal (X3DH, Double Ratchet), media crypto and pairing. Reasons: MIT, small (the whole Signal layer is 1625 lines against 3585 for the libsignal-go subset), layered exactly like detz wants (internal/wire, signal, keys, media), and it ships 788 KB of golden vectors captured from real Baileys (testdata/traces/connect_pair, testdata/signal/session_ab.json, testdata/media/media_vectors.json, testdata/paircode/vectors.json). Those become our C++ test fixtures as they are.
2. whatsmeow (MPL-2.0) is the behaviour reference: it is the most used and most maintained (78 commits in 90 days). When wa-go and whatsmeow disagree, whatsmeow wins. The current client version, the server certificate key and the node shapes are read from whatsmeow at build time of each release.

Do not transpile libsignal-go or libsignal-protocol-c: both are GPL-3.0, and detz is commercial. MPL-2.0 is file-level: a C++ file transpiled from a whatsmeow file keeps the MPL notice and its source stays available. Copying code from wa-go, or reading whatsmeow to learn a value, does not trigger this. This is a fact about the licenses, not legal advice.

4. Module layout

Namespace detz::wa, sources in src/whatsapp/, one class per .h/.cpp pair, as the rest of taicpp.

Class	Responsibility	Transpiled from (wa-go / whatsmeow)
WaConstants	URL, origin, Noise pattern, header, dict version, cert key, client version, all caps (section 8)	socket/constants.go, store/clientpayload.go
WaProtoWriter, WaProtoReader	protobuf wire format: varint, zigzag not needed, length-delimited, fixed32	none, hand-written
WaMessages	the ~15 protobuf messages as plain structs with encode()/decode()	field numbers from whatsmeow proto/*.proto
WaCrypto	static: X25519, AES-256-GCM, AES-256-CTR, AES-256-CBC, HKDF-SHA256, HMAC-SHA256, SHA-256, SHA-512, PBKDF2-SHA256, random, over OpenSSL EVP	util/*util, wa-go keys
WaXeddsa	XEdDSA sign and verify over Curve25519 keys	libsignal-go ecc/SignCurve25519.go logic, wa-go keys (section 5)
WaSocket	libcurl WebSocket, 3-byte big-endian frame length, WA header on first frame, 16 MiB frame cap	wa-go wire/framing.go, ws
WaNoise	Noise XX handshake and the post-handshake read/write ciphers	wa-go wire/noise.go, whatsmeow socket/noisehandshake.go, handshake.go
WaNode, WaBinaryCodec, WaTokens	binary XMPP-like nodes, token dictionary v3 (236 single-byte, about 1030 double-byte tokens), nibble/hex packing, JID encoding	wa-go wire/codec.go, token.go, node.go
WaStore	SQLite tables: device identity, signed prekey, prekeys, sessions, identities, contacts, aliases	wa-go store, whatsmeow store/sqlstore
WaPairing	QR pairing (pair-device, ref rotation, pair-success with ADV signature check) and phone pairing code	wa-go client pairing, whatsmeow pair.go, pair-code.go
WaSignal	X3DH, Double Ratchet, pkmsg/msg encrypt and decrypt, session records	wa-go internal/signal (x3dh, ratchet, session, message)
WaPrekeys	upload 50 prekeys plus signed prekey after pairing, refill below threshold	whatsmeow prekeys.go
WaUsync	device list of a recipient (usync iq), "is this number on WhatsApp"	whatsmeow user.go (subset)
WaMedia	media key expansion (WhatsApp Image Keys), AES-CBC plus HMAC, media_conn iq, HTTPS upload to /mms/image/<token>	wa-go media, whatsmeow upload.go, mediaconn.go
WaSender	1:1 send: device fan-out, encrypt per device, own other devices get a DeviceSentMessage, <message><participants> node, phash check	whatsmeow send.go (DM path only)
WaReceiver	decrypt incoming 1:1 text, send receipts and acks	whatsmeow message.go, receipt.go (subset)
WaClient	owns the above; connect, login, keepalive, reconnect, event loop on one thread	whatsmeow client.go, keepalive.go, connectionevents.go
WaContacts	name to JID: user aliases first, then contacts learned from push names, then fuzzy match with detz::FuzzyMatcher (floor 0.3)	none, detz design
WaTools	agent tools whatsapp_send, whatsapp_contacts, whatsapp_alias	none, detz design

Out of scope for v1, listed so nobody assumes otherwise: groups (sender keys), history sync, calls, status, newsletters, receiving media, editing, reactions.

5. Protocol walkthrough and crypto mapping

Order of one session, with the OpenSSL call for each primitive.

1. Connect. WebSocket to wss://web.whatsapp.com/ws/chat, header Origin: https://web.whatsapp.com. Every frame is a 3-byte big-endian length plus payload; the first frame is prefixed with 'W','A',6,3 (magic 6, dictionary version 3). Frame cap 1 << 24.
2. Noise XX handshake. Pattern string Noise_XX_25519_AESGCM_SHA256\0\0\0\0 (32 bytes, used as the initial hash as is). Send HandshakeMessage.ClientHello{ephemeral}; receive ServerHello{ephemeral, static (encrypted), payload (encrypted cert)}; mix X25519 shared secrets with HKDF-SHA256 into AES-256-GCM keys, counter IV; verify the certificate chain against the pinned key WACertPubKey in whatsmeow handshake.go; send ClientFinish{static (encrypted noise key), payload (encrypted ClientPayload)}. Then split into a read and a write key. The proto also defines post-quantum modes (XXKEM, IKKEM, WA_PQ); whatsmeow does not use them today, so neither do we.
3. ClientPayload. Register (unpaired: devicePairingData with registration id, identity public key, signed prekey and its signature, build hash = MD5 of the version string) or login (paired: username = phone number, device = device id). The client version is a constant (2.3000.1050728964 in whatsmeow on 2026-10-27). A stale version gives connect failure 405 client outdated; whatsmeow can fetch the live value from web.whatsapp.com ("client_revision":N).
4. Pairing. QR: the server sends pair-device with refs; the QR string is https://wa.me/settings/linked_devices#ref,noisePub,identityPub,advSecret,clientType (keys base64), shown in the terminal, refs rotate. Phone code: PairPhone sends a link_code_companion_reg iq; the 8-character code derives an AES-256-CTR key with PBKDF2-SHA256, 131072 (2<<16) iterations. On pair-success verify the account signature (XEdDSA), sign the device identity (XEdDSA), send pair-device-sign, store the JID, reconnect and log in.
5. Prekeys. Upload signed prekey plus a batch of 50 one-time prekeys (iq xmlns="encrypt", whatsmeow WantedPreKeyCount). Refill when the server reports fewer than 5 (MinPreKeyCount).
6. Send to a contact. usync iq for the recipient's device list (and our own); fetch prekey bundles for devices without a session; X3DH, Double Ratchet encrypt Message{conversation} or Message{imageMessage} padded; send <message to type="text|media"><participants><to jid><enc type="pkmsg|msg" v="2"> and compare the returned phash. Addresses can be phone JIDs (@s.whatsapp.net) or LID JIDs (@lid); whatsmeow send.go mentions LID 45 times, so LID handling is part of the send path, not an extra.
7. Image. Random 32-byte media key; HKDF-SHA256 with info WhatsApp Image Keys to 112 bytes (IV, cipher key, MAC key, ref key); AES-256-CBC, HMAC-SHA256 truncated to 10 bytes appended; media_conn iq gives hosts and auth; HTTPS POST to https://<host>/mms/image/<token>; put url, directPath, mediaKey, fileSha256, fileEncSha256, fileLength, mimetype, caption into ImageMessage.

Primitive	Where	OpenSSL 3.5
X25519	Noise, X3DH, ratchet	EVP_PKEY_X25519, EVP_PKEY_derive
AES-256-GCM	Noise transport	EVP_aes_256_gcm
AES-256-CTR	pairing code	EVP_aes_256_ctr
AES-256-CBC	Signal message keys, media	EVP_aes_256_cbc
HKDF-SHA256	Noise, ratchet chains, media keys	EVP_KDF "HKDF"
HMAC-SHA256	ratchet, media MAC	EVP_MAC "HMAC"
PBKDF2-SHA256	pairing code	PKCS5_PBKDF2_HMAC
SHA-256, SHA-512, MD5	everywhere, XEdDSA, build hash	EVP_Digest
XEdDSA sign	signed prekey, pairing device signature	not in OpenSSL
XEdDSA verify	pairing account signature	Montgomery u to Edwards y, then EVP_PKEY_ED25519 verify (to be proven)

XEdDSA needs Edwards25519 scalar and point arithmetic, which OpenSSL does not expose. The algorithm is 60 lines in libsignal-go ecc/SignCurve25519.go (clamped scalar a, A = aB with sign bit forced to 0, r = SHA-512 of a prefix, a, message and 64 random bytes, R = rB, s = r + SHA-512(R,A,M) a mod L). The arithmetic underneath (filippo.io/edwards25519, BSD-3-Clause) is what has to be ported or replaced. That is the one hard crypto item, see section 6.

6. Build versus deep research, decided up front

Build means mechanical transpile with a known oracle. Deep research means the answer is not written down anywhere we have read, and has to be established first, with an archive entry, before any code.

Item	Kind	Oracle or open question
protobuf writer/reader, message structs	build	field numbers in whatsmeow .proto; wa-go client_payload.json trace
WaCrypto wrappers	build	RFC test vectors; wa-go media_vectors.json
framing, Noise, binary codec, tokens	build	wa-go traces/connect_pair (noise.json, frames_raw.jsonl, nodes.jsonl), codec_battery/nodes.jsonl
Signal X3DH, ratchet, session	build	wa-go testdata/signal/session_ab.json
pairing code crypto	build	wa-go testdata/paircode/vectors.json
media encrypt, upload	build	wa-go media_vectors.json, then one live send
usync, prekey upload, send node shape	build	whatsmeow source, then one live send
XEdDSA sign	deep research	port the Edwards25519 field/scalar/point subset (BSD-3, from filippo.io/edwards25519) or find a permissive C implementation; verify against wa-go keys/testdata/verify_sig.mjs style vectors
current client version and 405 policy	deep research	how often whatsmeow bumps waVersion; whether to fetch client_revision at start (adds one HTTPS call, kills the treadmill)
LID addressing	deep research	when the server hands out @lid, how sessions map phone JID to LID; read whatsmeow send.go and store LID tables
ban behaviour	deep research, measured only with a sacrificial number	no maintainer publishes heuristics; one vendor article names account age, message rate and reports
contact names without app-state	build	push names in incoming messages plus user aliases; app-state sync is v2

7. User flow (the customer view)

Two commands, nothing else to configure:

detz --whatsapp-link            # shows a QR in the terminal, scan in WhatsApp > Linked devices
detz --whatsapp-link 31612345678  # prints an 8-character code to type on the phone instead

Then, in any detz session:

> remember that Niggel is 31687654321 on WhatsApp
> create a screenshot, explain what is genius about it and send it to Niggel on WhatsApp

The first line calls whatsapp_alias. The second runs shell (screenshot), describe_image, then whatsapp_send{to:"Niggel", text, image}. whatsapp_send resolves the name through WaContacts; when the best match is not exact, the tool result names what it resolved to, the same note pattern as @agent mentions. detz --whatsapp-unlink logs the device out and deletes its rows.

State lives in the profile's vault (SQLite, the existing store), not in a new file. The link step is the only step that touches the phone.

Agent tools:

Tool	Arguments	Result
whatsapp_send	to (name, alias or number), text, optional image (path)	message id, resolved JID and name, phash match
whatsapp_contacts	optional query	known names and JIDs
whatsapp_alias	name, number	stored alias, after a usync check that the number is on WhatsApp

8. Caps (all static, per CLAUDE.md "everything gets a hard cap")

Constant	Value	Why
kWaFrameMax	16 MiB (1 << 24)	protocol limit
kWaImageMax	16 MiB	upload cap; refuse larger, loud
kWaSendPerMinute	10	ban exposure; the agent must not burst
kWaSendPerDay	200	same
kWaRecipientsPerSend	1	v1 is 1:1 only
kWaPrekeyBatch	50	whatsmeow WantedPreKeyCount
kWaConnectTimeout	20 s	handshake plus login
kWaPairTimeout	120 s	QR or code wait
kWaReconnectFloor	5 s, doubling to 300 s	never hammer the server
kWaContactsMax	5000	contact and alias rows per profile

These are judgments, not measurements. They become measured once the live tests below run.

9. Tests

Unit (tests/whatsapp_test.cpp, make test):

• every golden vector from wa-go testdata/ (copied with its MIT notice into tests/whatsapp_data/): Noise transcript, raw frames, decoded nodes, codec battery, ClientPayload bytes, Signal session A to B, pairing code, media.
• XEdDSA: sign then verify round trip, plus verify of a signature made by libsignal (vector to be produced once, archived).
• protobuf: encode equals the bytes in the traces, decode of every trace.

Integration (tests/whatsapp.py, real binary, real WhatsApp, per AGENTS.md):

• needs a sacrificial number linked once (WA_TEST_PEER names a second number);
• cases: link state survives restart, send text, send image, receive the reply text, 405 handling with a forced old version;
• verification reads vault rows (message id, session rows), never CLI text.

10. The shortcut for the day-after-tomorrow promise

The native module is roughly the size measured in section 2.1, so it does not ship in two days. What can work now, with no C++:

1. clone lharries/whatsapp-mcp, bump its whatsmeow (go.mod pins v0.0.0-20250318233852, 19 months old; a stale whatsmeow risks the 405 client-outdated failure), run whatsapp-bridge once and scan the QR;
2. add the MCP server to detz (mcp_add, stdio transport, uv --directory .../whatsapp-mcp-server run main.py);
3. the sentence then runs as shell + describe_image + send_file.

This path was not run end to end: it needs a linked phone, and a local probe (an unpaired whatsmeow connect that prints QR events, which proves the full handshake without a phone) was refused by this session's permission classifier because it executes downloaded code. To run it yourself:

mkdir probe && cd probe && go mod init probe
go get go.mau.fi/whatsmeow@latest modernc.org/sqlite
# main.go: sqlstore.New("sqlite", "file:probe.db"), NewDevice, NewClient,
# GetQRChannel, Connect, print each event; expect "code" events within seconds
go run .

11. Phases

Phase	Content	Exit test
0	MCP shortcut (section 10)	the sentence works once, live
1	WaProto*, WaCrypto, WaXeddsa, WaSocket, WaNoise, codec, tokens	all wire and crypto golden vectors green; live: QR events from the real server
2	WaStore, WaPairing, WaPrekeys, login	live: link by QR and by code, restart stays logged in
3	WaSignal, WaUsync, WaSender, WaReceiver	Signal vectors green; live: text to a peer and its reply decrypted
4	WaMedia, WaContacts, WaTools, CLI flags	live: the full sentence, then the MCP shortcut is removed

Size guide (measured Go source to transpile, non-blank, non-comment): whatsmeow files on the 1:1 path (handshake, pair, pair-code, prekeys, send, message, receipt, upload, mediaconn, user, request, keepalive, client, connectionevents, qrchan, retry, socket, binary, keys, crypto utils) total 8,616 lines, and only part of send, message and user is needed; wa-go wire, keys, signal and media total 5,231. The C++ size is not measured until it exists.

12. Risks and gaps

• Terms of Service. Every unofficial client breaks WhatsApp's terms; every project README and the one vendor article found say a number can be banned. Use a sacrificial number for tests. The sanctioned path (Business Cloud API) cannot use a personal number.
• Protocol drift. whatsmeow changes about 6 times a week (78 commits in 90 days). We follow whatsmeow; each release re-checks the version constant, the cert key and the token dictionary.
• wa-go is young (created 2026-06-26, one author, 1 star). Its golden vectors come from Baileys, which keeps them honest, but its code is not battle tested. whatsmeow stays the referee.
• Unverified here: the full handshake (probe refused), XEdDSA verify via OpenSSL Ed25519, the 20-day re-auth claim, the 405 behaviour.

13. Robustness against hostile input (incident WA-2026-10-27, and the standing rule)

Every byte this module parses comes off a network socket we do not control. That makes the wire parsers (WaBinary, WaProto, and the Noise/Signal frame readers) a security and stability boundary, not ordinary code. This section records a crash that got through, exactly why, how it was fixed, and the rule and test that keep it from ever coming back. We do not accept regressions here.

13.1 What happened

Running detz --whatsapp-link on a real terminal, the wizard connected, showed the QR, and after the code auto-refreshed it aborted:

terminate called after throwing an instance of 'std::length_error'
  what():  cannot create std::vector larger than max_size()
Aborted (core dumped)

A core dump, not a handled error. The process died mid-session instead of logging a bad frame and carrying on.

13.2 Root cause

Two independent faults lined up:

1. An overflowable bounds check. Both the protobuf reader and the binary-node decoder guarded a length-delimited field with pos + len > size. pos is a size_t and len was read straight from the wire (up to 2^32-1, or a 4-byte length sign-extended into a negative int). pos + len then wrapped around past SIZE_MAX, computed a small sum, and the check passed. The code went on to vector::assign(begin, begin + len) with a garbage len, and the standard library threw std::length_error from inside vector.
2. An unguarded top-level loop. WaClient::dispatch routed nodes straight into handle_pair_success and the other handlers with no try/catch. The decode path in recv_node was wrapped, but the handlers that re-parse sub-messages (the ADV identity in pairing, for instance) were not, so a std::length_error raised there propagated out of main and called std::terminate.

Either fault alone is survivable. Together they turn one malformed frame into a process abort. This is the classic C++ failure: an arithmetic overflow in a size computation silently defeats a bounds check, and an uncaught exception is not a logged error but a dead process.

13.3 The fix (commit f16c5c6)

• Overflow-safe length checks everywhere bytes are sliced. The comparison is now len > size - pos (never pos + len > size). Because pos <= size is an invariant at every call site, the subtraction cannot wrap, so no attacker value can slip through. src/whatsapp/wa_proto.cpp (ProtoReader::next) and src/whatsapp/wa_binary.cpp (Decoder::need).
• Lengths are read unsigned and range-checked. The binary decoder reads 4-byte lengths into a uint32_t (never a sign-extendable int), rejects any value larger than the whole frame, and read_bytes rejects negatives. A bad length now raises WireError, a std::exception we catch, long before any allocation.
• Dispatch can no longer abort the process. WaClient::dispatch wraps all node routing in try/catch(const std::exception&); a bad node is logged (dropped node <tag>: <reason>) and the client keeps running.

13.4 The standing rule (project principle)

The goal is to make C++ safe to use here, which means a hostile or buggy peer can waste our time but can never crash us or corrupt us. Three invariants, in order of importance:

1. No size arithmetic that can overflow a bounds check. Never write pos + len <= size. Write len <= size - pos against an established pos <= size. Read lengths into an unsigned type wide enough to hold them, and range-check before you allocate or slice. Treat every length, count and index off the wire as adversarial.
2. No unchecked allocation from an attacker-controlled count. A length field is a claim, not a fact. Validate it against the bytes actually present before reserve, resize, assign, or a range constructor. A caller must never be able to make us ask for a 4 GB vector.
3. No exception may escape a top-level loop. Any loop that reads and dispatches untrusted input (recv_node, dispatch, the send and pairing waits) wraps the per-item work in try/catch(const std::exception&) and turns a failure into a logged, skipped item. An uncaught std::exception in this module is a bug even if the parse that raised it was "impossible".

These apply to any code that reads untrusted bytes, not just WhatsApp. When a new wire format is added (CAP, MCP, a new media type), it inherits this rule.

13.5 How we stop the regression

Prose does not prevent regressions; a failing test does. tests/whatsapp_test.cpp has a test_hostile section (run by build/whatsapp_test, step whatsapp.unit in tests/run_all.py, and make test) that feeds the parsers exactly the shapes that used to crash and asserts they throw a handled exception and specifically not std::length_error:

• a binary node whose content length claims 0xFFFFFFFF in an 11-byte buffer;
• a protobuf field whose length varint is ~4 billion with no payload;
• truncated and short frames;
• 2000 random byte strings through both WaBinary and ProtoReader (surviving the loop is the proof: a single unchecked allocation would abort the run).

Each case catches std::length_error and fails the test if it is thrown, so if anyone reintroduces a pos + len check or an unguarded slice, this test goes red. If you add a new parser, add its hostile cases here in the same pass; a parser with no adversarial test is treated as unfinished.

13.6 Follow-up: the first fix was not enough, and how we proved it

Section 13.3's fix stopped the abort but was declared "verified" on a hostile-input test that never exercised the live decode path. That was overconfident: a later run still logged dropped node <iq>: cannot create std::vector larger than max_size(). The crash was gone (the try/catch held), but a real frame was still being mis-handled and dropped.

The cure was to stop reasoning and use a tool. Building the parsers under AddressSanitizer + UndefinedBehaviorSanitizer + _GLIBCXX_ASSERTIONS (../tools/san-whatsapp.sh) immediately found a second, independent undefined-behavior bug that two manual reviews had missed: a protobuf varint loop that shifted by up to 70 bits (shift >= 64 is UB) on malformed input. It is now fixed and guarded by a deterministic test case and the sanitizer net. Two more defenses were added:

• Frame capture. WaClient records the exact bytes of any frame that fails to decode or dispatch to <home>/whatsapp/last-error.log (bounded). The next time the still-unreproduced live <iq> failure happens, the precise input is on disk and becomes a one-paste regression test and a deterministic fix.
• The sanitizer net is now the standing requirement, not a one-off. See the project-wide playbook, cpp-safety.md, which this incident produced: overflow-safe checked-reader parsing, cheap always-on release hardening, the ASan+UBSan test net, and capture.

The honest open item: the exact frame the user hit is still unreproduced in-session (the server did not resend it within a 160 s sanitizer run). The capture hook exists precisely so that gap closes on its next occurrence rather than by more guessing.

13.7 Root cause, found and fixed (the capture hook paid off)

The capture hook from 13.6 did its job: the next --whatsapp-link run recorded the exact 726-byte frame to <home>/whatsapp/last-error.log, and replaying it made the bug deterministic in minutes. The verdict overturns every earlier guess: it was never a parser bug. The frame decoded perfectly (it was a normal pair-success with a device-identity, device, biz, and encryption-metadata). The std::vector larger than max_size() was thrown by WaDevice::serialize(), in WaClient::handle_pair_success saving the newly paired device:

put_blob(b, Bytes((jid ? *jid : "").begin(), (jid ? *jid : "").end()));

(jid ? *jid : "") builds a fresh temporary std::string for each evaluation, so .begin() and .end() point into two different, already destroyed objects. The range constructor computed a garbage length and the vector threw. It only misfired once jid was set, which is why it appeared only after the phone scanned, and why the initial unpaired save and every parser test passed. The fix binds the value to a named const std::string before taking iterators. Verified by replaying the captured frame end to end (serialize, save, encode the response: no throw) and by a paired-device serialize round-trip test (tests/whatsapp_test.cpp test_device) that runs under the sanitizer net.

The lesson is in cpp-safety.md: this was a C++ object-lifetime bug, a different class from the earlier size-overflow, and it slipped past the parser-focused tests and the first sanitizer runs because those never exercised serialize() with a set optional. Two defenses were strengthened as a result: the capture hook (which turned an unreproducible failure into a deterministic one), and the rule that a round-trip test must cover the populated case, not just the empty one. The honest open item from 13.6 is now closed.

14. App-state sync: the contact list (WaAppState, WaReceiver)

Status: implemented and verified offline. The live path needs a re-linked phone: nothing here has been run against a phone that holds contacts.

Why: the phone's address book reaches a linked companion only through app-state sync (collection critical_unblock_low, index ["contact", "<jid>"]). Without it wa_contact stays empty and names cannot be resolved.

Piece	Files	What it does
Key receipt	wa_receiver.*, wa_proto.* (parse_protocol_message), WaClient::handle_message	Decrypts <enc> from our own account (pkmsg via X3DH responder, msg via session), unpads, stores each APP_STATE_SYNC_KEY_SHARE key in wa_appstate_key. Own means our phone number or our LID (whatsmeow IsFromMe); other senders are acked and ignored as before. Every arrival is logged before the filter, so a trace shows dropped mail too. An empty share (keys without key data, openclaw/wacli PR 443) is reported and ends the wait without storing; a consumed peer message gets a peer_msg protocol receipt.
Key request	WaClient::request_app_state_keys, send_peer_message, encode_app_state_key_request	Peer message to the primary phone (category peer, meta + enc, device-identity only on a prekey message), at most kWaAppStateKeyIdsMax ids, one request per key id per kWaAppStateKeyRequestInterval (24 h), stored in wa_appstate_keyreq. The send is verified (Signal session must exist, the server ack must carry no error); when it fails the 24 h marks are rolled back so the next sync asks again (whatsmeow PR 1268).
Notifications, receipts	WaClient::handle_notification, handle_receipt	server_sync re-syncs the contact collection (other collections are named and skipped); encrypt with fewer than kWaMinPrekeys left refills prekeys; a retry receipt (the phone could not decrypt our message) is logged loudly. Notification syncs never nest inside a sync (guard + one more pass).
Key expansion, MACs, values	wa_appstate.* (WaAppStateKeys, WaAppStateCrypto)	HKDF-SHA256 WhatsApp Mutation Keys to 160 bytes; value MAC HMAC-SHA512 truncated to 32; index, snapshot and patch MACs HMAC-SHA256; AES-256-CBC values. WaCrypto::hmac_sha512 added.
LTHash	wa_appstate.* (WaLtHash)	128-byte state, HKDF-SHA256 info WhatsApp Patch Integrity (whatsmeow, not BLAKE2b; see research conclusion appstate-lthash-hkdf-not-blake2b).
Patch fetch and apply	WaAppStateSync::sync	iq w:sync:app:state, return_snapshot at version 0, version=N otherwise; external blobs via WaMedia::download (md-app-state); snapshot MAC, patch MAC, index and value MACs checked before anything is stored.
Contacts	WaAppStateSync::apply_contact	SET stores jid = index[1], name = fullName else firstName; empty name is counted and skipped; REMOVE deletes the row.
Push-name learning	WaClient::handle_message, push_name_candidate, WaStore::put_contact_if_absent	Every incoming message carries the sender's self-reported name in notify (whatsmeow message.go); group senders come from participant, the PN alternate is preferred when present. Stored only when the jid has no name yet, so a saved, imported or aliased name always wins; truncated to kWaPushNameMax (128). No server query returns names for regular users (GetUserInfo usync has status, picture, devices, LID and business verified names only), so this plus VCF/alias/app-state is the complete name-resolution stack.
Login hook	WaClient::login calls sync_app_state after prekeys	Never fails the login (errors are logged). If a key is missing, requests it, waits up to kWaAppStateKeyWait s for a key share, then syncs once more.
Long sync	detz --whatsapp-sync (WaWizard::sync)	Logs in, re-syncs with a kWaSyncOnlineWait (300 s) key wait for a sleeping phone, reports the contact count. Exit 0 with contacts, 1 without, 2 when not linked or login failed.
VCF fallback	detz --whatsapp-import FILE (WaVcf)	Imports a .vcf exported on the phone (FN else N, TEL lines, international digits) into the same contact search. Loud skips, capped at kWaContactsMax, file capped at kWaVcfFileMax.

Caps (wa_consts.h): kWaAppStatePatchMax 50 patches per sync, kWaAppStateMutationMax 10000 mutations, kWaAppStateTimeout 30 s per iq, kWaAppStateBlobMax 16 MiB per external blob, kWaAppStateKeyIdsMax 16 key ids per request, kWaAppStateKeyWait 20 s, kWaSyncOnlineWait 300 s, kWaVcfFileMax 8 MiB. A cap stops the sync loudly (capped in the report, a log line) and the rest waits for the next login.

Failure rule: a patch or snapshot that fails a MAC or hash check is not stored, the sync stops, and the collection version stays at the last verified state. This skips the bad patch and everything after it, because each later patch depends on the hash chain.

Tables: wa_appstate_key, wa_appstate_version, wa_appstate_mac (value MAC per index MAC, needed to undo a SET or REMOVE), wa_appstate_keyreq. clear_device empties them, so a new link syncs from version 0.

Verification (offline, build/whatsapp_test, section appstate):

• expanded keys equal whatsmeow expandAppStateKeys bytes;
• LTHash add/remove cancel and are order-free;
• a snapshot plus one patch (SET, REMOVE, in-patch replacement, unnamed SET, a non-contact SET) decodes to the oracle's final hash and version, and the contact table matches the oracle's expected names;
• a re-sync at version 2 changes nothing;
• a tampered patch is rejected and not stored;
• a missing key stores nothing and reports the key id;
• the key share and key request bytes match whatsmeow's Message encoding;
• a pkmsg built with WaSession decrypts through WaReceiver to the key share Message, and a corrupted pkmsg is refused;
• the 24 h request mark can be cleared for an immediate retry;
• own senders accept PN and LID, the peer node carries device-identity only on a prekey message, server_sync collections and retry receipts parse;
• an empty key share parses with empty key data;
• a 5-card .vcf imports 4 numbers (FN, N fallback, unfolding, 00 form) and the typo-tolerant search finds them;
• push-name candidates parse for 1:1 (PN, LID, PN alternate), groups (participant, PN alternate), and reject own senders, -, missing notify, non-user senders and over-long names; fill-if-absent keeps the first name and never overwrites a saved one.

The oracle is tests/whatsapp_data/appstate_vectors.json, generated by tests/whatsapp_data/appstate_oracle.go (a Go program using whatsmeow's own lthash, hkdfutil and protobuf packages). It is not run by make test.

Not verified: the live server's answers to the iq, key share timing, the media host for external blobs, and whether a REMOVE of an unknown index (whatsmeow only warns) causes trouble. Known gaps: a SET with an empty name keeps any older name the row had (the spec says skip, not delete); a retry receipt is logged, not answered with a resend (whatsmeow retry.go); history sync notifications are not downloaded.

15. Incident WA-2026-10-27b: link ended in 401, no picture, no contacts

Everything here was measured on 2026-10-27/28 against the live server and the user's phone.

Symptom	Root cause	Fix	Evidence
Linked, but the login confirmation did not complete; later login failure 401	The decoder printed a server-only JID (empty user) as @s.whatsapp.net; the pairing replies copied that into to, so the server refused pair-device-sign and never registered the device	decoder returns the bare server name; encoder writes bare server names as JID pairs; replies use WaConsts::kServerJid as whatsmeow does	captured pair-success decoded (from=@s.whatsapp.net), live login probe 401 before, logged in after re-link; test server-only JID round-trips
wizard never reported the real login result	run_until returned at once because paired_event_ stayed set	run_until is stateless; WaClient::await_pair_restart + WaClient::login; one fresh pairing after a 401/403 (kWaLinkRetries)	wizard source
user 7m CPU while linking	a closed socket made every receive return instantly and the loops spun until their deadline	WaSocket clears connected_ on close, every loop checks WaClient::socket_open()	probe after the 401: 1.7 s, 0.01 s CPU (was 20 s of spinning)
PNG could not be sent	the whatsapp_send tool had no image argument and send_image was a stub	WaMedia (HKDF WhatsApp Image Keys, AES-CBC, 10-byte HMAC, media_conn, HTTPS upload), send_image, DeviceSentMessage copy to our other devices, tool argument image	8/8 wa-go golden vectors byte-exact; live: image to own number acknowledged by the server
Nigel not found	the phone address book reaches a companion only through app-state sync, which did not exist; and the phone only pushes history and keys to a companion that declares DeviceProps.historySyncConfig, which registration omitted	section 14 (WaAppState, WaReceiver); WaDeviceProps now sends historySyncConfig (field 5, whatsmeow values); peer key request encrypted to the phone's LID identity	unit tests; the live address book needs a link made with this build
name lookups were exact or one fuzzy guess	contact table only, one FuzzyMatcher::Best	WaStore::search_contacts: FTS5 trigram candidates + fuzzy rank; ambiguity (two scores within 0.08) is reported, never guessed; the global search tool lists matching contacts	tests contact search: ...
no send limits	caps were specified, not enforced	wa_send_log, kWaSendPerMinute 10 and kWaSendPerDay 200 enforced in whatsapp_send	test send log counts sends in the window

Gaps: the contact sync and the address-book key share are not verified against a phone yet; a newly linked device must be created with this build for the phone to send history. Set DETZ_WA_TRACE=1 to log every node's tag and attributes when debugging a protocol exchange.

16. Incident WA-2026-10-28: the phone never sends the app-state sync key

Full record: problem.md. A newly linked companion (31630045827:54) logged in, uploaded prekeys and sent an image to its own number (visible in "Message yourself", participant hash matching the server's), but the phone never sent the app-state sync key: 100 to 120 s of DETZ_WA_TRACE=1 showed only <success>, <ib edge_routing> and <ib dirty>, no <message> at all.

The request path was checked line by line against whatsmeow send.go, message.go, appstate.go, notification.go, client.go, pair.go and store/clientpayload.go (main, fetched 2026-10-27). Registration, login, DeviceProps (including historySyncConfig), the peer node shape, the key request bytes and the padding all match. whatsmeow answers the open questions this way:

• A peer message needs a Signal session with the phone first: encryptMessageForDevice with no bundle and no session returns ErrNoSession and the request never leaves the process (hence whatsmeow PR 1268, which rolls back the 24 h mark on send failure, and Volt-Chat PR 5, which adds an explicit session setup).
• Own senders are the PN user or the LID user (parseMessageSource); no recipient attribute is needed. The <ib dirty type=account_sync> node needs no answer (whatsmeow handleIB leaves it alone).
• The jurisdiction and encryption-metadata nodes in pair-success are server-side account metadata; whatsmeow handlePairSuccess ignores every child it does not know, as we do. Not related to key sharing.

Fixes in this build (all local; the phone side is still unseen):

Gap	Fix	Evidence
a failed key request still stamped the 24 h mark, blocking the retry	clear_app_key_request_time, rollback on send failure (whatsmeow PR 1268)	test key request rollback
a peer send was trusted without checking the session or the ack	session verified after the prekey fetch, ack error attribute checked and logged (Volt-Chat PR 5)	code, log lines
device-identity rode on every peer message, whatsmeow only on pkmsg	build_peer_message_node matches whatsmeow	test peer message helpers
key shares from our LID were silently dropped (PN-only filter)	is_own_sender accepts PN or LID, every arrival is logged before the filter	test peer message helpers
an empty key share looked like no answer	stored/empty counted separately, the wait ends, peer_msg receipt sent (openclaw/wacli PR 443)	test empty key share
notifications and receipts were ignored	server_sync re-syncs, encrypt refills prekeys, retry receipts logged loudly	test peer message helpers
20 s online is short for a sleeping phone	detz --whatsapp-sync stays online 300 s (kWaSyncOnlineWait)	wizard source
no fallback when the phone never answers	detz --whatsapp-import FILE imports a .vcf into the contact search	test vcf import

What to run now: link fresh if needed, then detz --whatsapp-sync with WhatsApp open on the phone (no battery restriction). If no <message> arrives within the 300 s, the phone is gating on its side; import the .vcf and retry the sync another day. Research: research/whatsapp-automation/ (conclusion key-share-never-arrives.md).

17. Transpilation pin, harness and reuse table (M0, 2026-10-28)

The full whatsmeow transpilation (full_transpilation_and_integration.md, milestones M0..M10) starts here. This section holds the pinned source, the harness that gates every milestone, the baseline coverage, and the moloapp reuse table. The pin never moves.

17.1 Pinned source

Field	Value
Module	go.mau.fi/whatsmeow
Version	v0.0.0-20261027104441-1eec1136a50b
Commit	1eec1136a50b2ef148b05f1819e77a9d989e0457
Commit date	2026-10-27T10:44:41Z
Location	$(go env GOMODCACHE)/go.mau.fi/whatsmeow@v0.0.0-20261027104441-1eec1136a50b
Provenance	cache/download/go.mau.fi/whatsmeow/@v/<version>.info, Origin.Hash

tools/wa-api-inventory.sh refuses to run when the directory is missing or the .info hash differs from the pinned commit, so a moved cache fails loudly instead of silently rescoping the work.

17.2 Harness

Tool	Job
tools/wa-api-inventory.sh	Extracts the mechanical scope into build/wa-inventory.tsv (package, kind, key, file) and the proto.Marshal/proto.Unmarshal call sites into build/wa-marshal-sites.txt. Re-verifies the pin on every run and aborts on a duplicate key (parser self-check).
tools/wa-mapping.tsv	Committed source of truth: every inventory key maps to its C++ counterpart (detz::wa::..., file) or to UNMAPPED. Mapping rule: a named C++ entity (function, method, type, const, field) performs the counterpart role; an inline fragment with no named entity stays UNMAPPED for a later milestone to name.
tools/wa-coverage.sh [--slice FILE]	Re-runs the inventory, then gates: the mapping key set must match the inventory exactly (a missing or stale key fails as "out of sync"), and every gated entry must be mapped. Exit 0 only at 100%. A slice file (file:, pkg:, sym:, re: lines) scopes the gate to one milestone's API.
tools/wa-stub-scan.sh	Fails on absolute stub patterns (TODO, FIXME, XXX, HACK, unimplemented, not implemented, NOT_IMPLEMENTED, panic(, assert(false, std::abort) anywhere in src/whatsapp/, and on any new return std::nullopt / return false / return {} (diff vs HEAD, plus untracked files) without a log line (log(, log_(, TRACE, cerr, stderr, fprintf, perror) within the 6 lines above it or on its own line. No allowlist.

Scope rules (mechanical, encoded in the inventory script): in scope is every exported symbol (func, method, type, const, var) in non-test, non-generated Go source. Out of scope: *_test.go; files whose header marks them generated (DO NOT EDIT, DO NOT MODIFY, generated by: the 57 *.pb.go files, internals.go, binary/proto/legacy.go); files with //go:build ignore (internals_generate.go). A generated protobuf message counts as in scope when <pkg>.<Msg> is referenced by qualified name in an in-scope file (69 marshal call sites motivate the 129 used messages). Methods on unexported receivers with exported names (patchOutput.AddMAC, waLog.noopLogger.*) stay in scope: the name is exported, so it is listed and must map.

Validation (2026-10-28): the extractor was diffed against go doc as an independent method. Client methods match exactly (the two go doc extras are the NewClient constructor, inventoried as func, and DangerousInternals, defined only in the excluded generated/ignored files). The coverage gate was shown to fail below 100% and pass on an all-mapped slice; the stub scanner was shown to fail on an injected TODO and on a new silent return false (tracked and untracked files) and to pass on a logged one and on the clean tree.

17.3 Baseline coverage (detz 1.46.1)

tools/wa-coverage.sh: 137/1500 mapped (9.1%), exit 1 (expected: the 100% gate is the M10 exit, per-milestone slices gate M1..M9).

Kind	Mapped	Total
const	13	330
func	14	60
method	49	523
proto	28	129
type	24	320
var	9	138

Mapped packages (mapped/total): appstate 8/136, binary 18/46, cbcutil 2/5, gcmutil 2/3, hkdfutil 1/1, keys 4/7, lthash 3/4, socket 17/34, store 26/143, types 6/232, waAdv 3/3, waCompanionReg 2/10, waE2E 7/35, waServerSync 6/9, waSyncAction 2/19, waWa6 8/8, whatsmeow 22/369. Everything else (groups, newsletters, calls, presence, pair-code/passkey, history sync, inbox, read receipts, retry-resend, event bus, sqlstore schema, error taxonomy, the remaining crypto/store/util surface) is UNMAPPED, claimed by M1..M10 in the milestone table. The 137 mapped entries were each verified against the header or source named in tools/wa-mapping.tsv; subset notes on the value name the gap the owning milestone closes.

17.4 moloapp reuse table

Prompt section 5: before writing any HTTP client, SSE/websocket, LLM provider, config, CLI, tenancy or app-pattern code, grep moloapp (../../moloapp/src/, 1039 headers) and record the decision here. Reuse is by include plus link only; moloapp is never modified. A duplicated facility fails review.

Date	Need	moloapp facility checked	Decision
2026-10-28	M0 harness scripts	none applicable (bash + python3 stdlib; no HTTP/SSE/LLM/config/CLI surface)	wrote local tools/wa-*.sh
2026-10-28	WebSocket client transport	realtime/websocket/client/websocket_client.h (exists, but needs a caller-provided transport: the caller does TLS and the HTTP handshake itself)	keep local WaSocket on libcurl: live-verified against the real server (connect + frames), libcurl already linked. Revisit if libcurl ws proves insufficient (e.g. M1 keepalive pings).
2026-10-28	media HTTPS upload/download	clients/ HTTP client, pool, proxy (heavier connection-pool stack)	keep local WaMedia on curl_easy (wa_media.cpp): two calls (one POST, one GET), libcurl already linked
2026-10-28	vault storage for device/sessions/contacts	detz-owned store::SqlDb (already the vault; the schema is detz's)	keep local WaStore on store::SqlDb (wa_store.h); no moloapp store facility needed
2026-10-28	M1 keepalive pings (revisit of the M0 transport row)	realtime/websocket/client/websocket_client.h send_ping, realtime/websocket/server/heartbeat_manager.h, realtime/engineio/heartbeat_scheduler.h	wrote local WaKeepAlive (wa_keepalive.h): whatsmeow keepalive is an application iq (w:p get), not a websocket ping frame, with whatsmeow-specific timing (20-30 s jitter, 10 s deadline, 3 min horizon); moloapp's facilities are ws-protocol/engine.io level and none fits. WaSocket stays on libcurl.
2026-10-28	M2 store tables (identities/LID/contacts/secrets/tokens/settings)	tenancy/pool_factory.h (sqlite named only in a comment), cli/cli/cli_app.cpp (scaffolder --backend sqlite flag): no store facility	keep WaStore/WaLidMap on detz-owned store::SqlDb; schema is detz's
2026-10-28	M5 media HTTPS (all types up/down, stickers, newsletter, deletes)	clients/SPEC.md + client.h: a full coroutine HTTP stack, but its TLS is a marked wolfSSL stub, so it cannot do real HTTPS; media needs real TLS	keep local WaMedia on curl_easy (libcurl already linked), per the M0 row; the injectable WaMedia::HttpClient (the SetMediaHTTPClient counterpart) keeps every path fake-testable

18. M2 store (2026-10-28)

Milestone M2 transpiles the whatsmeow store area named in the plan-table row: identities with trust tracking, the LID map, sessions, contacts (saved/push/business), privacy tokens, message secrets and chat settings. Coverage slice tools/wa-slice-m2.txt holds the 60 inventory keys; tools/wa-coverage.sh --slice tools/wa-slice-m2.txt reports 60/60 (100%).

18.1 What landed

New classes (one per .h/.cpp, namespace detz::wa):

• WaJid (wa_jid.*): the JID subset the store needs (parse, str, non-AD form, PN/LID classification, signal user), mirroring whatsmeow types/jid.go, proven byte-for-byte against the oracle (jid_non_ad, signal_user:*, jid_parse_fails vectors).
• WaLidMap (wa_lidmap.*): the cached LID map (CachedLIDMap plus Device.GetAltJID as alt_jid).
• WaStore gains the SessionStore remainder (has_session, get_many_sessions, put_many_sessions, delete_session, delete_all_sessions, migrate_pn_to_lid), the full IdentityStore, ContactStore, ChatSettingsStore, MsgSecretStore and PrivacyTokenStore, with the WaContactInfo, WaChatSettings, WaPrivacyToken, WaMessageSecret, WaContactEntry, WaRedactedPhone and WaMessageSecretInsert structs.

New tables in the profile database (wa_identity, wa_sender_key, wa_chat_settings, wa_msg_secret, wa_privacy_token, wa_lid_map) plus the five whatsmeow contact columns on wa_contact (added with ALTER TABLE when missing, so older databases migrate in place). The detz name column and the FTS index keep their meaning (aliases, VCF imports, learned push names); the whatsmeow name columns live beside them. clear_device now also drops identities, sender keys, secrets, tokens and chat settings (a relinked device must not inherit trust); contacts and the LID map survive, like upstream, where both outlive a device delete.

New static caps in wa_consts.h: kWaIdentityKeyLen (32), kWaStoreJidMax (1024), kWaStoreBlobMax (1 MiB, enforced before allocation on every store blob read), kWaStoreBatchMax (300, the upstream contact batch size), kWaMutedForever (253402300799, pinned by the oracle against store.MutedForever.Unix()).

18.2 Tested divergences from upstream (all oracle-pinned)

Four deliberate deviations, each recorded with its failing input:

1. Signal address format. detz stores user.device (see WaClient::addr_of); upstream stores user:device. Prefix operations (delete_all_sessions, delete_all_identities, the migrate arms) use LIKE 'phone.%' where upstream uses LIKE 'phone:%'. Observable behavior is identical; the vectors record outcomes, never addresses.
2. Migrate renames plain users. Upstream MigratePNToLID renames signal users, so LID-side rows embed the agent (100000000000001_1:0, pinned in mig_lid_addr); detz addresses have no agent suffix, so migrate_pn_to_lid renames plain users (100000000000001.0), the only spelling detz lookups use.
3. DeleteIdentity is exact. Upstream accidentally deletes with the LIKE query, so deleting 9%9:0 also wiped 999:0 (the oracle records this in trust_pct_sibling_after_pct_delete). detz deletes with =: the sibling row survives, proven by the test.
4. Not-muted representation. Upstream reads a missing/cleared mute as zero time.Time (Unix -62135596800); detz reads it as 0. The stored row (0) is identical; both mean not muted (chat_*_zero vectors).

Two loud-error mirrors: put_privacy_tokens({}) throws (upstream fails its SQL on empty input; both loud), and a corrupt identity length throws SqlError (upstream ErrInvalidLength).

18.3 Evidence

• Oracle: tests/whatsapp_data/store_oracle.go drives pinned whatsmeow itself (sqlstore over scratch SQLite) through one ordered op script and emits tests/whatsapp_data/store_vectors.json (105 keys).
• Differential test: test_store_m2 in tests/whatsapp_test.cpp replays the script against WaStore and compares every outcome; test_store_hostile adds corrupt rows, overlong keys, malformed JIDs, wide inputs (2000-address get-many, 650-row chunked batch) and a 3000-string fuzz loop with a JID round-trip check.
• build/whatsapp_test green; tools/san-whatsapp.sh green (the script was broken — it never compiled the store TUs — and now builds every TU the test exercises; the first sanitizer run caught a real bug, the agent-suffixed migrate address in section 18.2.2, plus unsafe optional derefs in the test, both fixed); make, make lint (with -Werror) and tools/wa-stub-scan.sh green (one harness fix: a grep with no match in a clean untracked file aborted the scan under pipefail).

18.4 Left for later milestones (not M2 scope)

Sender-key put/get (M4, the table and the migrate arm exist), app-state sync-key/version/MAC stores (M7, beside the existing M0 app-state port), prekey upload tracking (GetOrGenPreKeys, MarkPreKeysAsUploaded, UploadedPreKeyCount, RemovePreKey; M3), the event/retry buffer (M4), the device container and NoopStore (M10/integration), rewiring push-name learning to put_push_name (M4/M8, which own receive and user events), and searching the synced first/full names via FTS (M7).

19. M1 foundations: tokens, socket, iq, Noise (2026-10-28)

Milestone M1 of full_transpilation_and_integration.md (slice tools/wa-slices/m1-foundations.txt, 91 entries, 100% mapped; full inventory 210/1522, 13.8%). Four pieces, each proven by a Go oracle that calls the pinned whatsmeow itself, plus hostile-input tests and a sanitizer run.

19.1 Harness repairs found by M1

Three harness repairs surfaced and were fixed in this milestone (all minimal, all required by the M1 gates):

• tools/wa-api-inventory.sh cut // comments before stripping string literals, so the // inside the "https://..." token broke brace-depth counting and the var ( block in binary/token/token.go swallowed the rest of the file. token.DictVersion, 3 lookup funcs and 18 wire consts (22 entries) were missing from the scope. Fixed by stripping strings first; the inventory is now 1522 entries (was 1500) with zero removals.
• tools/wa-stub-scan.sh died silently (exit 1, no verdict) on untracked files without matches: grep exits 1, and set -e/pipefail killed the script. Fixed with || true plus a comment; M0's validation (fail on an injected TODO / silent return) still holds.
• tools/san-whatsapp.sh compiled only 6 parser TUs, but the test now links the whole module. It compiles all of src/whatsapp/ plus the support TUs (json, seal_base64, sqldb, fuzzy, qrcodegen) and links curl and sqlite3.

19.2 Token dictionary freshness

tests/whatsapp_data/token_oracle.go dumps whatsmeow's own binary/token tables (version, 236 + 4x256 strings, 18 consts) and whatsmeow-marshaled wire bytes for 14 token-probe nodes (single-byte tokens, one token per double-byte dictionary, an untokenized literal). build/whatsapp_test (test_tokens) compares every string verbatim, every const, and byte-exact re-encodes all 14 probes. Re-running the oracle after a pin move fails loudly on drift. The tables were already correct; the check is the deliverable.

Two encoding forms differ cosmetically and both are valid: whatsmeow writes a typed server-JID to as JID_PAIR while detz writes the same string as its 1-byte token, and whatsmeow writes "" as NIBBLE_8 while detz writes BINARY_8. Both pairs decode identically (proven in test_iq/test_tokens); the server accepts ours (proven live, section 19.5).

19.3 Socket keepalive pings

WaKeepAlive (wa_keepalive.h) is the whatsmeow keepAliveLoop schedule as pure timing state over an injected millisecond clock: jittered 20-30 s intervals, error counting, forced reconnect past the 3 min horizon. Caps in wa_consts.h (kWaKeepAliveResponseDeadline 10 s, kWaKeepAliveIntervalMin 20 s, kWaKeepAliveIntervalMax 30 s, kWaKeepAliveMaxFailTime 180 s). WaClient::run_until starts it and sends a w:p get on schedule via query_iq; any reply proves liveness, exactly as sendKeepAlive. On a forced reconnect the client logs and disconnects; the redial lands with M9 auto-reconnect. The KeepAliveTimeout/KeepAliveRestored events belong to the M9 event bus; until then failures are loud log lines.

19.4 Full iq request/response

WaIq (wa_iq.h) mirrors request.go for the synchronous client: WaIqQuery plus build (id/xmlns/type, smax_id, to, target, one child), parse_error (whatsmeow parseIQError, overflow-safe code parse, never fails), is_disconnect_node / is_auth_error_disconnect (401, replaced, device_removed), and whatsmeow-shaped request ids (new_prefix draws <byte0>.<byte1>-, generate_id appends the counter). WaIqError carries the 13 ErrIQ* sentinels plus timed_out() with message()/is(); WaDisconnectedError carries for_iq() with message()/is(). WaClient::send_iq now classifies every reply (error replies log code/text and are returned; disconnects and timeouts log loudly), defaults to the 75 s kWaIqDefaultTimeout, and query_iq fills ids. ErrClientIsNil maps to the class itself: methods take references, a nil receiver is unrepresentable. Retry-on-reconnect needs M9 auto-reconnect (NoRetry arrives with it); today a disconnect during an iq is a loud WaDisconnectedError, never a silent drop. Vectors: iq_oracle.go (whatsmeow-marshaled ping, count, 404, 500, bare, result, three disconnects; code/text extracted by whatsmeow's own getters; byte-exact encode checks where deterministic).

19.5 Noise parity

• WaNoise::mix_shared_secret names the X25519-into-key step (MixSharedSecretIntoKey); the handshake calls it three times. Behavior is unchanged (existing transcript test) and now differentially proven: the oracle runs whatsmeow's own NoiseHandshake over the wa-go trace and our decrypted server static + payload match byte-for-byte.
• WaFraming::kLengthSize names the 3-byte prefix, and unframe uses overflow-safe subtraction checks (it had the forbidden pos + 3 form).
• WaServerCert (wa_cert.h) verifies the Noise certificate chain exactly as verifyServerCert: pinned-key intermediate check, issuer serials, 64-byte signatures, key lengths, validity windows, leaf-key-vs-static match. Any failure throws WaCertError with the reason, and the handshake fails closed. This caught two real proto bugs: parse_cert_chain had leaf and intermediate swapped (fields are leaf = 1, intermediate = 2), and parse_cert_details read the wrong fields (serial = 1, issuer = 2, key = 3, not_before = 4, not_after = 5). Vectors: cert_oracle.go (decrypted bytes, per-field values, per-check verdicts from whatsmeow + libsignal; the captured leaf is expired today, so the test time-travels inside its window for the accept case and asserts expiry now, plus 6 tamper and 4 hostile rejections).
• WaSocket gained the already-open and not-connected guards plus WaSocketStatusError (HTTP status on connect failure); WaClient refuses a second connect loudly and reports closes through the new on_disconnect hook.

Live verification (2026-10-28, scratch TAI_HOME, no phone): detz --whatsapp-link completed the Noise handshake against web.whatsapp.com with the new certificate verification passing ("handshake ok (registration)") and rendered the QR. A 60 s session fired keepalive pings on schedule with server replies (zero unanswered/failed lines, connection stayed open until the timeout kill). The cert accept path and the ping shape are therefore live-proven, not just oracle-proven.

| 2026-10-28 | M3 --whatsapp-link [PHONE] optional-value flag | cli/cli/flag.hpp molo::cli::Flag (only takes_value bool, no argparse nargs="?" const semantics) | keep local Nargs::Optional (src/cli/flag_spec.cpp): detz's parser is argparse-compatible with parity tests, moloapp's flag cannot express bare-vs-valued |

20. Pairing: pair-code, passkey, QR channel, logout (M3, 2026-10-28)

M3 transpiles whatsmeow pair.go, pair-code.go, pair-passkey.go, qrchan.go, types/passkey.go, plus Client.Logout, the pairing error taxonomy, the pairing events and the seven pairing protobuf messages: 67 API keys, all mapped (tools/wa-slice-m3.txt, tools/wa-coverage.sh --slice 67/67, full inventory now 200/1500). New code:

Piece	Files	whatsmeow source
WaPairing (pair-code crypto, QR builder, hello/finish/error/logout nodes, error taxonomy, client types)	wa_pairing.*	pair-code.go, pair.go (shapes), errors.go (pair/QR/phone subset), client.go Logout iq
WaPasskey (WebAuthn JSON, handoff key, prologue, confirmation, sealed request)	wa_passkey.*	pair-passkey.go, types/passkey.go
WaQrChannel (timed code pump + terminal items)	wa_qr_channel.*	qrchan.go
pairing events	wa_pair_events.h	types/events (8 pairing types)
7 pairing messages	wa_proto.* (extended)	proto/waCompanionReg
client flows + notification routing + hardened pair-success	wa_client.* (extended)	pair.go, notification.go (4 pairing branches)
code linking, server logout, passkey UX	wa_wizard.* (extended), --whatsapp-link [PHONE]	user flow

20.1 Pair-code linking

WaClient::pair_phone mirrors PairPhone: strip the number to digits (WaPairing::normalize_phone, same errors), wrap a fresh ephemeral key with PBKDF2-SHA256/131072 + AES-CTR (wrap_companion_ephemeral), send link_code_companion_reg stage companion_hello, keep the {jid, linking_code, pairing_ref, eph} cache. The link_code_companion_reg notification runs finish_code_pairing (unwrap, double ECDH, sealed key bundle, adv_secret HKDF) and answers with the companion_finish iq; the ADV secret lands in memory and persists at pair-success, exactly like Go. detz --whatsapp-link 31612345678 prints the ABCD-EFGH code (Chrome client type, Chrome (Linux) display name, push notification on). Decisions: detz pins the Chrome client type (WaClient default, live-verified in the QR flow) instead of whatsmeow's UNKNOWN/WEB derivation (OtherWebClient); the derivation itself exists as the pure wa_default_qr_client_type for callers. The 160 s code expiry is kWaPairCodeExpiry.

20.2 Passkey

WaPasskey covers the whole handshake as pure functions: WebAuthn options parse (parse_public_key, base64url strict, options capped at kWaPasskeyOptionsMax), Go-byte-exact response marshal (marshal_response, same field order and encoding/json escaping), derive_handoff_key (5-minute WaPasskeyHandoffKey::valid), build_prologue, handoff_proof, derive_encryption_key, derive_confirmation_code (linking-base32 of the XOR, ABCD-EFGH), and seal_pairing_request. WaClient::send_passkey_response / send_passkey_confirmation run the iq exchanges, including the fetch-the-options fallback and the handoff proof. Gap, stated plainly: detz has no built-in authenticator, so nothing in-tree can produce a WebAuthnResponse; the wizard surfaces the request (rpId + challenge note) and the confirmation code (user compares, enter confirms; handoff codes auto-confirm through the channel), and the client methods are complete and unit-tested for the day an authenticator exists.

20.3 QR hardening

WaQrChannel is qrchan.go without goroutines: the client drives poll() from its run loop (run loop and iq waits, so PairPhone never stalls the pump). Same semantics: first code of a six-code batch lives 60 s, the rest 20 s (kWaQrCodeTimeout, kWaQrFirstCodeTimeout); exhaustion emits the timeout item and the client disconnects; success/error/outdated/unexpected map like Go; rotate_adv rewrites the secret and re-emits the current code at once (verified against qrchan.go's re-queue, not assumed). New routing: ib/downgrade_webclient feeds the multidevice hint, failure 405 feeds outdated, any other failure/success while armed feeds unexpected, a dropped socket feeds disconnected, and companion_reg_refresh notifications rotate the ADV secret (WaClient::rotate_adv_secret, memory-only like Go). Pair-success now sends sendPairError nodes with Go's codes on every failure path, parses biz/platform/client-props, consults pre_pair_callback before storing, tracks the server clock offset, and fires on_pair_success/on_pair_error; get_qr_channel refuses late arming with Go's error texts. The oracle test caught a real bug here first: iterators taken from two rebuilt temporaries in finish_code_pairing (cpp-safety rule 7), fixed before the gate ran.

20.4 Integration

--whatsapp-link takes an optional PHONE (Nargs::Optional, bare means QR; help text updated). --whatsapp-unlink now logs in briefly and sends remove-companion-device before dropping local rows, and says so when the server was unreachable (offline unlink still unlinks locally). The wizard renders QR codes from the channel (deduped first paint via on_qr), prints multidevice/outdated/timeout guidance, and pauses the wait for a passkey code compare. New caps in wa_consts.h: QR timeouts, PBKDF2 iterations, handoff validity, phone digits, options/ref/display-name caps.

20.5 Evidence

tests/whatsapp_data/pairing_oracle.go (pinned whatsmeow's hkdfutil, gcmutil, CompanionReg protos and passkey JSON types; 64 vectors in pairing_vectors.json) pins every byte: linking-base32 (7 lengths), QR string, hello wrap, key bundle + ADV secret, all 7 protos, handoff key, prologue, encryption key, confirmation code, sealed request, and 4 WebAuthn JSON shapes. build/whatsapp_test adds [pairing: Go oracle differentials] (byte-exact), [pairing: node shapes and errors] (structural, exact error texts), [pairing: QR channel] (fake clock) and [pairing: hostile input] (new parsers + 3000 mutated inputs, never length_error). tools/san-whatsapp.sh (TU list repaired: it no longer linked) and make lint are green. Two harness repairs shipped in M3: the sanitizer script compiles the full module closure, and wa-stub-scan.sh no longer aborts on a clean untracked file (|| true on the grep).

20.6 Open seams (owned by later milestones)

M2: the main-device identity row and LID mapping at pair-success (sessions trust on first use until then). M4: sendUnifiedSession after pairing. M9: unifying the pairing callbacks into the 47-type event bus, and the generic ErrClientIsNil/ErrNotLoggedIn guards. Live verification of the four flows needs a linked phone and has not run; the refactored QR display was exercised against the live server on 2026-10-28 (Noise handshake, 6 pair-device refs, terminal render, then disconnected without scanning — no phone was linked).

21. M5 media (2026-10-28)

Milestone M5 transpiles the whatsmeow media area named in the plan-table row: upload and download of every type, thumbnails, stickers and newsletter media. Coverage slice tools/wa-slice-m5.txt holds the 61 inventory keys (download.go, upload.go, download-to-file.go, mediaconn.go, types/sticker.go, the media arms of errors.go and cbcutil, the FB download envelope, the media HTTP hook); tools/wa-coverage.sh --slice tools/wa-slice-m5.txt reports 61/61 (100%).

21.1 What landed

New classes (one per .h/.cpp, namespace detz::wa):

• WaMediaType + WaMediaTypes (wa_mediatype.*): the eight media kinds, their HKDF app-info strings, the mms-type table, for_message (GetMediaType) and for_thumbnail. Unknown inputs throw UnknownMediaType; upstream cannot even name them (a non-downloadable message is a compile-time exclusion from GetMediaType).
• WaMediaError + WaDownloadHttpError (wa_mediaerror.*): the media error taxonomy with byte-exact sentinel messages, the status-code Is match, and the terminal_for_host_failover / retryable predicates that drive the download loop exactly like the errors.Is checks upstream.
• WaMediaConn + WaMediaConnHost (wa_mediaconn.*): the full connection set (auth, TTLs, buckets, hosts) with expiry and a strict <media_conn> parser. It replaces WaClient's private {auth, host} pair; query_media_conn now parses through it.
• WaStickerPack + WaStickerPackItem (wa_sticker.*): the sticker structs with their JSON tags and all five getters, plus the pack-array parser.
• WaMedia grows the throwing M5 surface beside the untouched v1 upload()/download() pair (single host, still serving the app-state blob sync): per-type encrypt plus the video sidecar, CBC streaming (cbc_encrypt_stream, cbc_decrypt_file), split/validate/decrypt primitives, downloads (download, download_any, download_thumbnail, download_fb, download_media_with_path, download_media_with_only_path and all four to-file variants) with host failover and per-host retry with Retry-After backoff, uploads (upload_encrypted, upload_reader, upload_newsletter, upload_newsletter_reader), delete_media, and fetch_sticker_pack. The injectable WaMedia::HttpClient is the SetMediaHTTPClient counterpart, and the injectable sleeper pins the backoff without waiting.

New static caps in wa_consts.h: kWaMediaMax (16 MiB, every blob up or down; kWaImageMax is now an alias), kWaMediaMacLen (10), kWaMediaKeyLen (32), kWaMediaDownloadRetries (5), kWaMediaHostsMax (16), kWaMediaSidecarChunk (64 KiB), kWaMediaStreamChunk (32 KiB), kWaStickerJsonMax (1 MiB), kWaMediaUrlMax (8192), kWaMediaTimeout (60 s).

21.2 Tested divergences from upstream (all pinned)

1. Strict padding check on memory decrypts. Upstream cbcutil.Decrypt truncates by the last byte with only a length check; decrypt_blob validates every pad byte (the v1 download() already did). Hostile ciphertext must not yield garbage plaintext silently. The file decryptor mirrors upstream exactly (truncate by last byte), so the two arms differ on purpose and both are tested.
2. Empty hosts fail loudly. Upstream ranges over zero hosts and returns success with nil data; we throw BadResponse. A success that downloaded nothing is silent corruption.
3. Bounded where upstream is not. Downloads take max_bytes, to-file bodies are capped (upstream streams unbounded), URLs refuse beyond 8192 bytes, sticker bodies beyond 1 MiB. Every refusal is a loud WaMediaError.
4. Escaped sticker id. Upstream interpolates the pack id raw into the query; we URL-escape it (a strict superset for sane ids, a closed break-out for hostile ones).
5. No Messenger upload arm. The wa-msgr/mms prefix and ptt audio remap need MessengerConfig, which lives with the Messenger/FB transports (M9).

Source-copied, not oracle-pinned (no exported accessor exists): the mms-type table and the download/upload/delete URL formulas, read from the whatsmeow call sites and pinned as exact-string tests so drift fails loudly. The sidecar oracle is a second implementation of the same 20 unexported lines, documented in media_oracle.go.

21.3 Evidence

• Oracle: tests/whatsapp_data/media_oracle.go drives pinned whatsmeow itself (hkdfutil, cbcutil, GetMediaType, the sentinels, StickerPack marshal, retryafter) and emits tests/whatsapp_data/media_vectors_m5.json (16 per-type encrypt cases, 5 stream cases, 5 decrypt-file cases, 4 sidecars, the type map, the error strings, one sticker pack, the retry table).
• Differential test: test_media_m5 in tests/whatsapp_test.cpp replays every vector byte-for-byte and exercises all 61 APIs, including failover, retry/backoff and the last-host trail over a scripted fake transport; test_media_hostile mutates blobs, JSON, nodes, sidecar inputs and retry values (3500 inputs) plus four deterministic edges, asserting handled errors and never std::length_error.
• build/whatsapp_test green (83 new checks); tools/san-whatsapp.sh green; make, make lint (391 files) and tools/wa-stub-scan.sh green. The gates caught two real bugs: a temporary-iterator UB in the test's fake upload reply (the exact Layer 1 rule 7 shape) and a silent return false in the new code, both fixed.

21.4 Left for later milestones (not M5 scope)

Media-retry receipts and notifications (mediaretry.go, M4), message secrets and poll/reaction/comment E2E (msgsecret.go, M4), history-sync download (DownloadHistorySync, M7), the waE2E message protos (M4), app-state sticker indexes (M7), presence/privacy sticker settings (M8), the waConsumerApplication FB codecs and the Messenger upload arm (M9), and sending the new media kinds (send_video etc., M4/M10, which own the send paths; send_image keeps working through the new WaMediaConn).