whatsapp: M0 transpilation harness
This commit is contained in:
@@ -1,5 +1,9 @@
|
||||
# Changelog
|
||||
|
||||
## [Unreleased] - 2026-10-28
|
||||
- WhatsApp transpilation harness (M0): whatsmeow `v0.0.0-20261027104441-1eec1136a50b` (commit `1eec1136a50b...0457`) pinned in [docs/whatsapp.md](docs/whatsapp.md) section 17; `tools/wa-api-inventory.sh` extracts the mechanical scope (1500 entries: every exported symbol in non-test, non-generated Go source plus the 129 used protobuf messages), `tools/wa-mapping.tsv` maps each entry to its `detz::wa` counterpart or `UNMAPPED`, `tools/wa-coverage.sh` fails below 100% (baseline 137/1500, 9.1%), `tools/wa-stub-scan.sh` fails on stub patterns and new silent early-outs. Section 17 also opens the moloapp reuse table.
|
||||
|
||||
|
||||
## [1.46.1] - 2026-10-27
|
||||
- WhatsApp learns contact names from incoming messages (push names)
|
||||
|
||||
|
||||
@@ -709,3 +709,94 @@ WhatsApp open on the phone (no battery restriction). If no `<message>` arrives
|
||||
within the 300 s, the phone is gating on its side; import the .vcf and retry
|
||||
the sync another day. Research: `research/whatsapp-automation/`
|
||||
(conclusion `key-share-never-arrives.md`).
|
||||
|
||||
## 17. Transpilation pin, harness and reuse table (M0, 2026-10-28)
|
||||
|
||||
The full whatsmeow transpilation (`full_transpilation_and_integration.md`,
|
||||
milestones M0..M10) starts here. This section holds the pinned source, the
|
||||
harness that gates every milestone, the baseline coverage, and the moloapp
|
||||
reuse table. The pin never moves.
|
||||
|
||||
### 17.1 Pinned source
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Module | `go.mau.fi/whatsmeow` |
|
||||
| Version | `v0.0.0-20261027104441-1eec1136a50b` |
|
||||
| Commit | `1eec1136a50b2ef148b05f1819e77a9d989e0457` |
|
||||
| Commit date | 2026-10-27T10:44:41Z |
|
||||
| Location | `$(go env GOMODCACHE)/go.mau.fi/whatsmeow@v0.0.0-20261027104441-1eec1136a50b` |
|
||||
| Provenance | `cache/download/go.mau.fi/whatsmeow/@v/<version>.info`, `Origin.Hash` |
|
||||
|
||||
`tools/wa-api-inventory.sh` refuses to run when the directory is missing or
|
||||
the `.info` hash differs from the pinned commit, so a moved cache fails
|
||||
loudly instead of silently rescoping the work.
|
||||
|
||||
### 17.2 Harness
|
||||
|
||||
| Tool | Job |
|
||||
|---|---|
|
||||
| `tools/wa-api-inventory.sh` | Extracts the mechanical scope into `build/wa-inventory.tsv` (`package, kind, key, file`) and the `proto.Marshal`/`proto.Unmarshal` call sites into `build/wa-marshal-sites.txt`. Re-verifies the pin on every run and aborts on a duplicate key (parser self-check). |
|
||||
| `tools/wa-mapping.tsv` | Committed source of truth: every inventory key maps to its C++ counterpart (`detz::wa::...`, file) or to `UNMAPPED`. Mapping rule: a named C++ entity (function, method, type, const, field) performs the counterpart role; an inline fragment with no named entity stays `UNMAPPED` for a later milestone to name. |
|
||||
| `tools/wa-coverage.sh [--slice FILE]` | Re-runs the inventory, then gates: the mapping key set must match the inventory exactly (a missing or stale key fails as "out of sync"), and every gated entry must be mapped. Exit 0 only at 100%. A slice file (`file:`, `pkg:`, `sym:`, `re:` lines) scopes the gate to one milestone's API. |
|
||||
| `tools/wa-stub-scan.sh` | Fails on absolute stub patterns (`TODO`, `FIXME`, `XXX`, `HACK`, `unimplemented`, `not implemented`, `NOT_IMPLEMENTED`, `panic(`, `assert(false`, `std::abort`) anywhere in `src/whatsapp/`, and on any new `return std::nullopt` / `return false` / `return {}` (diff vs `HEAD`, plus untracked files) without a log line (`log(`, `log_(`, `TRACE`, `cerr`, `stderr`, `fprintf`, `perror`) within the 6 lines above it or on its own line. No allowlist. |
|
||||
|
||||
Scope rules (mechanical, encoded in the inventory script): in scope is every
|
||||
exported symbol (func, method, type, const, var) in non-test, non-generated Go
|
||||
source. Out of scope: `*_test.go`; files whose header marks them generated
|
||||
(`DO NOT EDIT`, `DO NOT MODIFY`, `generated by`: the 57 `*.pb.go` files,
|
||||
`internals.go`, `binary/proto/legacy.go`); files with `//go:build ignore`
|
||||
(`internals_generate.go`). A generated protobuf message counts as in scope
|
||||
when `<pkg>.<Msg>` is referenced by qualified name in an in-scope file (69
|
||||
marshal call sites motivate the 129 used messages). Methods on unexported
|
||||
receivers with exported names (`patchOutput.AddMAC`, `waLog.noopLogger.*`)
|
||||
stay in scope: the name is exported, so it is listed and must map.
|
||||
|
||||
Validation (2026-10-28): the extractor was diffed against `go doc` as an
|
||||
independent method. `Client` methods match exactly (the two `go doc` extras
|
||||
are the `NewClient` constructor, inventoried as `func`, and
|
||||
`DangerousInternals`, defined only in the excluded generated/ignored files).
|
||||
The coverage gate was shown to fail below 100% and pass on an all-mapped
|
||||
slice; the stub scanner was shown to fail on an injected `TODO` and on a new
|
||||
silent `return false` (tracked and untracked files) and to pass on a logged
|
||||
one and on the clean tree.
|
||||
|
||||
### 17.3 Baseline coverage (detz 1.46.1)
|
||||
|
||||
`tools/wa-coverage.sh`: **137/1500 mapped (9.1%)**, exit 1 (expected: the
|
||||
100% gate is the M10 exit, per-milestone slices gate M1..M9).
|
||||
|
||||
| Kind | Mapped | Total |
|
||||
|---|---|---|
|
||||
| const | 13 | 330 |
|
||||
| func | 14 | 60 |
|
||||
| method | 49 | 523 |
|
||||
| proto | 28 | 129 |
|
||||
| type | 24 | 320 |
|
||||
| var | 9 | 138 |
|
||||
|
||||
Mapped packages (mapped/total): appstate 8/136, binary 18/46, cbcutil 2/5,
|
||||
gcmutil 2/3, hkdfutil 1/1, keys 4/7, lthash 3/4, socket 17/34, store 26/143,
|
||||
types 6/232, waAdv 3/3, waCompanionReg 2/10, waE2E 7/35, waServerSync 6/9,
|
||||
waSyncAction 2/19, waWa6 8/8, whatsmeow 22/369. Everything else (groups,
|
||||
newsletters, calls, presence, pair-code/passkey, history sync, inbox, read
|
||||
receipts, retry-resend, event bus, sqlstore schema, error taxonomy, the
|
||||
remaining crypto/store/util surface) is `UNMAPPED`, claimed by M1..M10 in the
|
||||
milestone table. The 137 mapped entries were each verified against the header
|
||||
or source named in `tools/wa-mapping.tsv`; subset notes on the value name the
|
||||
gap the owning milestone closes.
|
||||
|
||||
### 17.4 moloapp reuse table
|
||||
|
||||
Prompt section 5: before writing any HTTP client, SSE/websocket, LLM
|
||||
provider, config, CLI, tenancy or app-pattern code, grep moloapp
|
||||
(`../../moloapp/src/`, 1039 headers) and record the decision here. Reuse is
|
||||
by include plus link only; moloapp is never modified. A duplicated facility
|
||||
fails review.
|
||||
|
||||
| Date | Need | moloapp facility checked | Decision |
|
||||
|---|---|---|---|
|
||||
| 2026-10-28 | M0 harness scripts | none applicable (bash + python3 stdlib; no HTTP/SSE/LLM/config/CLI surface) | wrote local `tools/wa-*.sh` |
|
||||
| 2026-10-28 | WebSocket client transport | `realtime/websocket/client/websocket_client.h` (exists, but needs a caller-provided transport: the caller does TLS and the HTTP handshake itself) | keep local `WaSocket` on libcurl: live-verified against the real server (connect + frames), libcurl already linked. Revisit if libcurl ws proves insufficient (e.g. M1 keepalive pings). |
|
||||
| 2026-10-28 | media HTTPS upload/download | `clients/` HTTP client, pool, proxy (heavier connection-pool stack) | keep local `WaMedia` on `curl_easy` (`wa_media.cpp`): two calls (one POST, one GET), libcurl already linked |
|
||||
| 2026-10-28 | vault storage for device/sessions/contacts | detz-owned `store::SqlDb` (already the vault; the schema is detz's) | keep local `WaStore` on `store::SqlDb` (`wa_store.h`); no moloapp store facility needed |
|
||||
|
||||
Executable
+219
@@ -0,0 +1,219 @@
|
||||
#!/usr/bin/env bash
|
||||
# wa-api-inventory.sh: extract the mechanical transpilation scope from the pinned
|
||||
# whatsmeow source (M0 harness; see docs/whatsapp.md section 17 and
|
||||
# full_transpilation_and_integration.md prompt section 1).
|
||||
#
|
||||
# Scope, mechanically:
|
||||
# 1. Every exported symbol (functions, methods, types, constants, vars) in
|
||||
# whatsmeow's non-test, non-generated Go source. Skipped: *_test.go files,
|
||||
# files whose header marks them generated (DO NOT EDIT / DO NOT MODIFY /
|
||||
# "generated by": the *.pb.go files, internals.go, binary/proto/legacy.go),
|
||||
# and files with "//go:build ignore" (the generator tool).
|
||||
# 2. Every generated protobuf message the library uses in non-test code: a
|
||||
# message struct from a *.pb.go file counts when "<pkg>.<Msg>" is
|
||||
# referenced in an in-scope file. The proto.Marshal/proto.Unmarshal call
|
||||
# sites that motivate this are listed separately as evidence.
|
||||
#
|
||||
# Output (all under build/, deterministic, sorted):
|
||||
# build/wa-inventory.tsv package<tab>kind<tab>key<tab>file
|
||||
# build/wa-marshal-sites.txt file:line of every proto.Marshal/Unmarshal call
|
||||
# Kinds: func, method, type, const, var, proto. Key is package-qualified
|
||||
# ("whatsmeow.Client.Connect", "types.JID", "waE2E.Message").
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
WA_VERSION="v0.0.0-20261027104441-1eec1136a50b"
|
||||
WA_HASH="1eec1136a50b2ef148b05f1819e77a9d989e0457"
|
||||
OUT="build/wa-inventory.tsv"
|
||||
SITES="build/wa-marshal-sites.txt"
|
||||
|
||||
GOMODCACHE="$(go env GOMODCACHE)"
|
||||
WA_SRC="$GOMODCACHE/go.mau.fi/whatsmeow@$WA_VERSION"
|
||||
INFO="$GOMODCACHE/cache/download/go.mau.fi/whatsmeow/@v/$WA_VERSION.info"
|
||||
test -d "$WA_SRC" || { echo "wa-api-inventory: missing $WA_SRC" >&2; exit 1; }
|
||||
test -f "$INFO" || { echo "wa-api-inventory: missing $INFO" >&2; exit 1; }
|
||||
GOT_HASH="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["Origin"]["Hash"])' "$INFO")"
|
||||
test "$GOT_HASH" = "$WA_HASH" || {
|
||||
echo "wa-api-inventory: pin drift: info says $GOT_HASH, want $WA_HASH" >&2; exit 1; }
|
||||
|
||||
mkdir -p build
|
||||
WA_SRC="$WA_SRC" OUT="$OUT" SITES="$SITES" python3 - <<'EOF'
|
||||
import os, re, sys
|
||||
|
||||
wa = os.environ["WA_SRC"]
|
||||
out, sites = os.environ["OUT"], os.environ["SITES"]
|
||||
|
||||
def header_lines(path):
|
||||
with open(path, encoding="utf-8") as f:
|
||||
return [f.readline() for _ in range(8)]
|
||||
|
||||
def is_generated(path):
|
||||
return any("DO NOT EDIT" in ln or "DO NOT MODIFY" in ln or
|
||||
"generated by" in ln.lower() for ln in header_lines(path))
|
||||
|
||||
def is_ignored(path):
|
||||
with open(path, encoding="utf-8") as f:
|
||||
for ln in f:
|
||||
if ln.startswith("//go:build ignore"):
|
||||
return True
|
||||
if ln.startswith("package "):
|
||||
return False
|
||||
return False
|
||||
|
||||
def pkg_of(text):
|
||||
m = re.search(r"^package (\w+)", text, re.M)
|
||||
return m.group(1) if m else "?"
|
||||
|
||||
def strip_line(ln):
|
||||
ln = ln.split("//", 1)[0]
|
||||
ln = re.sub(r"`[^`]*`", "", ln)
|
||||
ln = re.sub(r'"(?:[^"\\]|\\.)*"', "", ln)
|
||||
return ln
|
||||
|
||||
def exported(name):
|
||||
return bool(name) and name[0].isupper() and name != "_"
|
||||
|
||||
# Collect files.
|
||||
gofiles = []
|
||||
for root, _, files in os.walk(wa):
|
||||
for fn in files:
|
||||
if fn.endswith(".go"):
|
||||
gofiles.append(os.path.relpath(os.path.join(root, fn), wa))
|
||||
gofiles.sort()
|
||||
|
||||
scoped, pbfiles = [], []
|
||||
for rel in gofiles:
|
||||
if rel.endswith("_test.go"):
|
||||
continue
|
||||
full = os.path.join(wa, rel)
|
||||
if is_generated(full) or is_ignored(full):
|
||||
if rel.endswith(".pb.go"):
|
||||
pbfiles.append(rel)
|
||||
elif not rel.endswith(".pb.go"):
|
||||
sys.stderr.write("note: generated/ignored, out of scope: %s\n" % rel)
|
||||
continue
|
||||
scoped.append(rel)
|
||||
|
||||
rows = [] # (pkg, kind, key, file)
|
||||
allsrc = []
|
||||
for rel in scoped:
|
||||
text = open(os.path.join(wa, rel), encoding="utf-8").read()
|
||||
allsrc.append(text)
|
||||
pkg = pkg_of(text)
|
||||
# Blank multi-line raw-string bodies: SQL inside const blocks must not
|
||||
# parse as constant names, and braces inside strings must not move depth.
|
||||
rawlines = []
|
||||
in_raw = False
|
||||
for rln in text.split("\n"):
|
||||
if in_raw:
|
||||
if "`" in rln:
|
||||
in_raw = False
|
||||
rawlines.append("")
|
||||
elif rln.count("`") % 2 == 1:
|
||||
in_raw = True
|
||||
rawlines.append(re.sub(r"`[^`]*$", "", rln))
|
||||
else:
|
||||
rawlines.append(rln)
|
||||
lines = rawlines
|
||||
i, n = 0, len(lines)
|
||||
while i < n:
|
||||
ln = lines[i]
|
||||
m = re.match(r"^func \(([^)]*)\) ([A-Za-z_]\w*)", ln)
|
||||
if m:
|
||||
recv = m.group(1).strip().split()
|
||||
rtype = recv[-1].lstrip("*").split("[")[0] if recv else "?"
|
||||
meth = m.group(2)
|
||||
if exported(meth):
|
||||
rows.append((pkg, "method", "%s.%s.%s" % (pkg, rtype, meth), rel))
|
||||
i += 1
|
||||
continue
|
||||
m = re.match(r"^func ([A-Za-z_]\w*)", ln)
|
||||
if m and exported(m.group(1)):
|
||||
rows.append((pkg, "func", "%s.%s" % (pkg, m.group(1)), rel))
|
||||
i += 1
|
||||
continue
|
||||
m = re.match(r"^type ([A-Za-z_]\w*)", ln)
|
||||
if m and exported(m.group(1)):
|
||||
rows.append((pkg, "type", "%s.%s" % (pkg, m.group(1)), rel))
|
||||
i += 1
|
||||
continue
|
||||
m = re.match(r"^type \($", ln.strip())
|
||||
blk = None
|
||||
if m:
|
||||
blk = "type"
|
||||
elif ln.strip() in ("const (", "var ("):
|
||||
blk = "const" if ln.startswith("const") else "var"
|
||||
elif re.match(r"^const ([A-Za-z_]\w*)", ln):
|
||||
if exported(re.match(r"^const ([A-Za-z_]\w*)", ln).group(1)):
|
||||
rows.append((pkg, "const", "%s.%s" % (pkg, re.match(r"^const ([A-Za-z_]\w*)", ln).group(1)), rel))
|
||||
i += 1
|
||||
continue
|
||||
elif re.match(r"^var ([A-Za-z_]\w*)", ln):
|
||||
if exported(re.match(r"^var ([A-Za-z_]\w*)", ln).group(1)):
|
||||
rows.append((pkg, "var", "%s.%s" % (pkg, re.match(r"^var ([A-Za-z_]\w*)", ln).group(1)), rel))
|
||||
i += 1
|
||||
continue
|
||||
if blk is None:
|
||||
i += 1
|
||||
continue
|
||||
depth = 0
|
||||
i += 1
|
||||
while i < n:
|
||||
bln = lines[i]
|
||||
if bln.strip() == ")" and depth == 0:
|
||||
break
|
||||
code = strip_line(bln)
|
||||
if depth == 0:
|
||||
tok = re.match(r"\s*([A-Za-z_]\w*)", code)
|
||||
if tok and exported(tok.group(1)):
|
||||
rows.append((pkg, blk, "%s.%s" % (pkg, tok.group(1)), rel))
|
||||
depth += code.count("{") - code.count("}")
|
||||
i += 1
|
||||
i += 1
|
||||
|
||||
# Proto messages used in non-test code.
|
||||
blob = "\n".join(allsrc)
|
||||
proto_count = 0
|
||||
for rel in sorted(pbfiles):
|
||||
text = open(os.path.join(wa, rel), encoding="utf-8").read()
|
||||
pkg = pkg_of(text)
|
||||
plines = text.split("\n")
|
||||
for j, pln in enumerate(plines):
|
||||
m = re.match(r"^type (\w+) struct \{$", pln)
|
||||
if not m:
|
||||
continue
|
||||
window = "\n".join(plines[j:j + 3])
|
||||
if "protoimpl.MessageState" not in window:
|
||||
continue
|
||||
msg = m.group(1)
|
||||
if re.search(r"\b%s\.%s\b" % (re.escape(pkg), re.escape(msg)), blob):
|
||||
rows.append((pkg, "proto", "%s.%s" % (pkg, msg), rel))
|
||||
proto_count += 1
|
||||
|
||||
rows.sort(key=lambda r: (r[0], r[1], r[2], r[3]))
|
||||
seen = {}
|
||||
for pkg, kind, key, rel in rows:
|
||||
if key in seen:
|
||||
sys.stderr.write("DUPKEY %s first %s then %s:%s\n" % (key, seen[key], kind, rel))
|
||||
sys.exit("wa-api-inventory: duplicate key, parser bug")
|
||||
seen[key] = "%s:%s" % (kind, rel)
|
||||
with open(out, "w", encoding="utf-8") as f:
|
||||
for pkg, kind, key, rel in rows:
|
||||
f.write("%s\t%s\t%s\t%s\n" % (pkg, kind, key, rel))
|
||||
|
||||
# Marshal/Unmarshal call sites (evidence, not coverable entries).
|
||||
with open(sites, "w", encoding="utf-8") as f:
|
||||
for rel in scoped:
|
||||
full = os.path.join(wa, rel)
|
||||
for j, ln in enumerate(open(full, encoding="utf-8"), 1):
|
||||
if re.search(r"proto\.(Marshal|Unmarshal)\b", ln):
|
||||
f.write("%s:%d: %s" % (rel, j, ln.strip()[:160] + "\n"))
|
||||
|
||||
kinds = {}
|
||||
for _, k, _, _ in rows:
|
||||
kinds[k] = kinds.get(k, 0) + 1
|
||||
sys.stderr.write("scoped go files: %d, generated pb files: %d\n" % (len(scoped), len(pbfiles)))
|
||||
sys.stderr.write("inventory entries: %d %s\n" % (len(rows), kinds))
|
||||
sys.stderr.write("wrote %s\n" % out)
|
||||
EOF
|
||||
echo "inventory: $(wc -l < "$OUT") entries, $(wc -l < "$SITES") marshal sites"
|
||||
Executable
+98
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env bash
|
||||
# wa-coverage.sh: gate whatsmeow API coverage against the C++ port (M0 harness;
|
||||
# see docs/whatsapp.md section 17 and full_transpilation_and_integration.md
|
||||
# prompt sections 1 and 3).
|
||||
#
|
||||
# Usage: tools/wa-coverage.sh [--slice FILE]
|
||||
#
|
||||
# tools/wa-api-inventory.sh is re-run first so the inventory is never stale.
|
||||
# tools/wa-mapping.tsv maps every inventory key to its C++ counterpart or to
|
||||
# UNMAPPED. The key sets must match exactly: an inventory key missing from the
|
||||
# mapping, or a mapping key no longer in the inventory, fails loudly (the
|
||||
# mapping is stale, extend it, do not waive it). There is no allowlist.
|
||||
#
|
||||
# A slice file scopes the gate to one milestone's API: lines of
|
||||
# file:<whatsmeow-relative-path> every entry from that Go file
|
||||
# pkg:<go-package> every entry from that package
|
||||
# sym:<exact-key> one entry
|
||||
# re:<python-regex> every key the regex matches
|
||||
# ("#" comments and blank lines skipped; an entry matching ANY line counts.)
|
||||
# Without --slice the whole inventory is gated.
|
||||
#
|
||||
# Exit 0 only at 100% mapped; otherwise the unmapped keys are listed.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
SLICE=""
|
||||
if [ "${1:-}" = "--slice" ]; then SLICE="${2:?need a slice file}"; shift 2; fi
|
||||
test $# -eq 0 || { echo "usage: tools/wa-coverage.sh [--slice FILE]" >&2; exit 1; }
|
||||
test -z "$SLICE" || test -f "$SLICE" || { echo "wa-coverage: no slice $SLICE" >&2; exit 1; }
|
||||
|
||||
./tools/wa-api-inventory.sh >/dev/null
|
||||
MAP="tools/wa-mapping.tsv"
|
||||
test -f "$MAP" || { echo "wa-coverage: missing $MAP" >&2; exit 1; }
|
||||
|
||||
SLICE="$SLICE" MAP="$MAP" python3 - <<'EOF'
|
||||
import os, re, sys
|
||||
|
||||
inv = {}
|
||||
for ln in open("build/wa-inventory.tsv", encoding="utf-8"):
|
||||
pkg, kind, key, rel = ln.rstrip("\n").split("\t")
|
||||
inv[key] = (pkg, kind, rel)
|
||||
|
||||
mapping = {}
|
||||
for ln in open(os.environ["MAP"], encoding="utf-8"):
|
||||
if not ln.strip() or ln.startswith("#"):
|
||||
continue
|
||||
key, cpp = ln.rstrip("\n").split("\t", 1)
|
||||
mapping[key] = cpp.strip()
|
||||
|
||||
fail = []
|
||||
for key in sorted(set(inv) - set(mapping)):
|
||||
fail.append("missing from mapping: %s (%s %s)" % (key, inv[key][1], inv[key][2]))
|
||||
for key in sorted(set(mapping) - set(inv)):
|
||||
fail.append("stale mapping key (not in inventory): %s" % key)
|
||||
if fail:
|
||||
print("wa-coverage: mapping out of sync with inventory:")
|
||||
for f in fail[:40]:
|
||||
print(" " + f)
|
||||
if len(fail) > 40:
|
||||
print(" ... and %d more" % (len(fail) - 40))
|
||||
sys.exit(1)
|
||||
|
||||
slicepath = os.environ["SLICE"]
|
||||
wanted = None
|
||||
if slicepath:
|
||||
rules = []
|
||||
for ln in open(slicepath, encoding="utf-8"):
|
||||
ln = ln.strip()
|
||||
if not ln or ln.startswith("#"):
|
||||
continue
|
||||
kind, _, val = ln.partition(":")
|
||||
rules.append((kind.strip(), val.strip()))
|
||||
wanted = set()
|
||||
for key, (pkg, _, rel) in inv.items():
|
||||
for kind, val in rules:
|
||||
if (kind == "file" and rel == val) or \
|
||||
(kind == "pkg" and pkg == val) or \
|
||||
(kind == "sym" and key == val) or \
|
||||
(kind == "re" and re.search(val, key)):
|
||||
wanted.add(key)
|
||||
break
|
||||
if not wanted:
|
||||
print("wa-coverage: slice %s matched nothing" % slicepath)
|
||||
sys.exit(1)
|
||||
|
||||
keys = sorted(wanted if wanted is not None else inv)
|
||||
unmapped = [k for k in keys if mapping[k] == "UNMAPPED"]
|
||||
pct = 100.0 * (len(keys) - len(unmapped)) / len(keys)
|
||||
scope = ("slice %s" % slicepath) if slicepath else "full inventory"
|
||||
print("wa-coverage [%s]: %d/%d mapped (%.1f%%)" % (scope, len(keys) - len(unmapped), len(keys), pct))
|
||||
if unmapped:
|
||||
for k in unmapped[:60]:
|
||||
print(" UNMAPPED %s (%s %s)" % (k, inv[k][1], inv[k][2]))
|
||||
if len(unmapped) > 60:
|
||||
print(" ... and %d more" % (len(unmapped) - 60))
|
||||
sys.exit(1)
|
||||
print("wa-coverage: 100%%")
|
||||
EOF
|
||||
File diff suppressed because it is too large
Load Diff
Executable
+91
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env bash
|
||||
# wa-stub-scan.sh: fail on stubs in the WhatsApp C++ module (M0 harness; see
|
||||
# docs/whatsapp.md section 17 and full_transpilation_and_integration.md
|
||||
# prompt section 1). There is no allowlist: a hit means fix the code, never
|
||||
# waive the line.
|
||||
#
|
||||
# Two checks over src/whatsapp/:
|
||||
# 1. Absolute: TODO, FIXME, XXX, HACK, unimplemented, not implemented,
|
||||
# NOT_IMPLEMENTED, panic(, assert(false, std::abort. Any hit fails.
|
||||
# 2. New silent early-outs: an added line (vs HEAD, plus untracked files)
|
||||
# matching "return std::nullopt", "return false" or "return {}" must have
|
||||
# a loud log line at the call site: a match of log(|log_(|TRACE|cerr|
|
||||
# stderr|fprintf|perror within the 6 lines above it or on its own line.
|
||||
# "New" is the diff because the rule governs code being added; committed
|
||||
# lines were already gated.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
fails=0
|
||||
ABS='TODO|FIXME|XXX|HACK|unimplemented|not implemented|NOT_IMPLEMENTED|panic\(|assert\(false|std::abort'
|
||||
if grep -rnE "$ABS" src/whatsapp/; then
|
||||
echo "wa-stub-scan: absolute stub patterns hit ^^^" >&2
|
||||
fails=1
|
||||
else
|
||||
echo "wa-stub-scan: absolute patterns clean"
|
||||
fi
|
||||
|
||||
LOG='log_?\(|TRACE|cerr|stderr|fprintf|perror'
|
||||
RET='return (std::nullopt|false|\{\})'
|
||||
new_returns="$(mktemp)"
|
||||
trap 'rm -f "$new_returns"' EXIT
|
||||
if git rev-parse --verify HEAD >/dev/null 2>&1; then
|
||||
# Diff paths are repo-relative (taicpp/...); strip the cwd prefix back to local.
|
||||
PREFIX="$(git rev-parse --show-prefix)"
|
||||
export PREFIX
|
||||
git diff -U0 HEAD -- src/whatsapp/ | python3 -c '
|
||||
import os, re, sys
|
||||
prefix = os.environ.get("PREFIX", "")
|
||||
cur, new = None, None
|
||||
for ln in sys.stdin:
|
||||
m = re.match(r"\+\+\+ b/(.*)", ln)
|
||||
if m:
|
||||
cur = m.group(1).strip()
|
||||
if prefix and cur.startswith(prefix):
|
||||
cur = cur[len(prefix):]
|
||||
continue
|
||||
m = re.match(r"@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@", ln)
|
||||
if m:
|
||||
new = int(m.group(1))
|
||||
continue
|
||||
if cur is None or new is None or ln.startswith("-"):
|
||||
continue
|
||||
if ln.startswith("+") and not ln.startswith("+++"):
|
||||
if re.search(r"return (std::nullopt|false|\{\})", ln[1:]):
|
||||
print("%s:%d" % (cur, new))
|
||||
new += 1
|
||||
elif not ln.startswith("\\"):
|
||||
new += 1
|
||||
' > "$new_returns"
|
||||
while IFS= read -r f; do
|
||||
test -n "$f" || continue
|
||||
grep -nE "$RET" "$f" | sed "s|^|$f:|" | cut -d: -f1,2 >> "$new_returns"
|
||||
done < <(git status --porcelain -- src/whatsapp/ | grep "^??" | sed -e "s|^?? ||" -e "s|^$PREFIX||" || true)
|
||||
else
|
||||
echo "wa-stub-scan: no HEAD, scanning the whole tree for silent returns" >&2
|
||||
grep -rnE "$RET" src/whatsapp/ | cut -d: -f1,2 > "$new_returns"
|
||||
fi
|
||||
|
||||
silent=0
|
||||
while IFS=: read -r f n; do
|
||||
test -n "$f" || continue
|
||||
if ! test -f "$f"; then
|
||||
echo "wa-stub-scan: internal error: reported file missing: $f:$n" >&2
|
||||
fails=1
|
||||
continue
|
||||
fi
|
||||
lo=$((n > 6 ? n - 6 : 1))
|
||||
if ! sed -n "${lo},${n}p" "$f" | grep -qE "$LOG"; then
|
||||
echo "$f:$n: new silent early-out (no log within 6 lines above):"
|
||||
sed -n "${lo},${n}p" "$f" | sed 's/^/ /'
|
||||
silent=1
|
||||
fi
|
||||
done < "$new_returns"
|
||||
if [ "$silent" -eq 1 ]; then
|
||||
echo "wa-stub-scan: new silent early-outs ^^^ (log loudly or restructure)" >&2
|
||||
fails=1
|
||||
else
|
||||
echo "wa-stub-scan: new early-outs all loud ($(wc -l < "$new_returns" | tr -d ' ') checked)"
|
||||
fi
|
||||
|
||||
exit "$fails"
|
||||
Reference in New Issue
Block a user