# Hugging Face - Specification Owned directory: `src/vendors/huggingface/`. Namespace `molo::huggingface`. Depends on `molo::llm` (`ApiClient`, `Service`, `SseDecoder`, `ChatModel`, `Config`) and `molo::clients`. Built by `make ai` (the vendor glob in `src/ai.mk` picks the directory up). Demo: `example/app/ai/HuggingfaceDemo.cpp`, routes under `/ai/huggingface`. Plans: `plans/17-llm-vendor-platforms.md`, `plans/18-breadth.md`. ## 1. Scope Hugging Face is five HTTP surfaces on four hosts. All are covered: | Surface | Host | Class | What | |---|---|---|---| | Hub | `https://huggingface.co` | `Hub`, `Discussions`, `Collections`, `Webhooks` | models, datasets, spaces (list, info, revision), repo tree / refs / commits / compare / paths-info / likers / safetensors metadata, file download (`/resolve`), search, trending, tags, tasks, papers, Spaces catalogs, Inference Provider mappings (`/api/partners/{provider}/models`), whoami, discussions, collections, webhooks | | Router (Inference Providers) | `https://router.huggingface.co` | `RouterChat`, `InferenceTasks` | OpenAI-compatible chat (`/v1/chat/completions`, routing suffix `:fastest` `:cheapest` `:preferred` `:`), `/v1/models`, hf-inference task APIs (25 tasks, the classic Inference API), hf-inference chat | | Inference Endpoints | `https://api.endpoints.huggingface.cloud` | `Endpoints` | the whole management API v2 / v3: endpoints (list, get, create, update, pause, resume, scale to zero, delete), logs, 15 metric series, replicas, vendors and quotas, search, CIDR lists, custom domains, private services, audit logs | | TGI server | any URL | `TgiServer` | native (`/generate`, `/generate_stream`, `/tokenize`, `/info`, `/health`, `/metrics`) and OpenAI-compatible (`/v1/chat/completions`, `/v1/completions`, `/v1/models`, `/invocations`) | | TEI server | any URL | `TeiServer` | `/embed`, `/embed_all`, `/embed_sparse`, `/predict`, `/rerank`, `/similarity`, `/tokenize`, `/decode`, `/v1/embeddings`, `/info`, `/health`, `/metrics` | Plus `HuggingfaceChatModel : llm::ChatModel` over the router (default), a TGI server, or any `ChatEndpoint`. Not covered (explicit, see `tools/check_spec.py` EXCLUDED and section 8): repository writes (commit, upload, LFS, branches, tags, settings), organizations and users, billing, Jobs, buckets / containers / kernels, gated-access requests, SCIM, OAuth, `datasets-server.huggingface.co`, the `huggingface.co/api/blog` and posts comment APIs, the provider-specific router paths. ## 2. Module map (one class per concern) | File | Concern | |---|---| | `caps.h` | `HfCaps`: static ceilings | | `calls.h/.cpp` | `Call` enum and `kCalls`: the ONE table of 132 HTTP calls (surface, method, path, spec kind, reference, auth, flags, body fields, query parameters) | | `request_plan.h/.cpp` | `RequestPlan`, `PathArgs`, `Query`, `RepoRef`, `RepoType`: renders and validates one call from the table | | `platform.h/.cpp` | `Platform`, `Access`, `RequestBudget`: hosts, `HF_TOKEN`, anonymous clients, shared call budget, server URL check | | `hf_service.h/.cpp` | `HfService : llm::Service`: surface check, token check, budget, JSON / raw / bytes / paged / SSE calls | | `hub.h/.cpp` | `Hub` | | `community.h/.cpp` | `Discussions`, `Collections`, `Webhooks` (+ `CollectionItem`, `WebhookSpec`) | | `providers.h/.cpp` | `RouterChat`, `RouterModel`, `TaskCatalog`, `InferenceTasks` | | `endpoints.h/.cpp` | `Endpoints`, `MetricKind` | | `servers.h/.cpp` | `TgiServer`, `TeiServer` | | `chat_wire.h/.cpp` | `ChatWire`, `ChatEndpoint`: neutral chat <-> OpenAI-compatible wire | | `chat_model.h/.cpp` | `HuggingfaceChatModel` | | `tools/check_spec.py` | table vs published documents | ## 3. Static caps (caps.h) | Knob | Value | |---|---| | Path segment / repo-relative file path / segments | 256 / 1024 chars / 32 | | Query parameters / one query value | 32 / 2048 chars | | Model id / server URL | 256 / 2048 chars | | Local call budget | 1000 calls per 300 s per process (observed server limit: 500 per 300 s, `RateLimit-Policy: "fixed window";"api";q=500;w=300`) | | Page walk | 20 pages, 5000 items, page `limit` 1000 | | `paths-info` paths | 256 | | Download | 64 MiB (the `llm` response cap; larger fails loudly) | | Inference input (binary) | 16 MiB | | Batch inputs (TEI, embeddings) | 256 entries, 4 MiB text | | Stream events | 200000 | | Messages / tools / stop sequences | 4096 / 128 / 4 | | Webhook domains | 8 | ## 4. Contracts - **Token.** `HF_TOKEN` (or the legacy `HUGGING_FACE_HUB_TOKEN`) via `llm::Config`. It travels as `Authorization: Bearer` only, never in a URL or an error. A row with `auth: token` and no token throws `NotConfiguredError` (503 `NOT_CONFIGURED`) before any traffic. Rows with `auth: none` or `optional` (public Hub reads, `GET /v1/models`, `GET /v2/provider`, TGI / TEI) run without a token. - **Anonymous requests.** `llm::ApiClient` cannot send an unauthenticated request, so without a token `Platform` builds the client with a marker header `X-Molo-Anonymous: 1` instead of `Authorization`. Foundation gap, worked around inside the package. - **One table.** Every path, query parameter and body field comes from `kCalls`. `RequestPlan::make` refuses a query parameter or body field the row does not list, an unused or missing path argument, a body on a bodiless row, and rows flagged `unsupported`. - **Ids.** `namespace`, `repo`, `name`: `[A-Za-z0-9._-]`, 1..256, not `.` / `..`. Repo ids must be `owner/name`; legacy ids without an owner (`gpt2`) are refused because the Hub answers them with a redirect to the canonical id. A revision or tag is one percent-encoded segment (`refs/pr/1` -> `refs%2Fpr%2F1`). File paths are validated per segment and percent-encoded. Query values are percent-encoded; a value containing `..` has its dots encoded because `ApiClient` refuses `..` anywhere in the request path. - **Lists.** Hub list calls follow `Link: <...>; rel="next"` cursors, accept only the `cursor` of a Link to the same host, and return `Page` with `truncated`, `truncated_by` (`items` or `pages`) and `next_cursor`. Never silent. - **Writes and deletes.** Rows flagged `write` need a token. Deletes are `remove_*` methods with no defaults, flagged `destructive` in the table (tested: 7 DELETE rows, all flagged). The demo runs every paid or changing action as a dry run unless the body has `"confirm":true`. - **Chat wire.** `ChatWire` documents its mapping in `chat_wire.h`: tool arguments that are not JSON arrive as `{"_invalid_json": raw}`; reasoning is read from `reasoning_content` then `reasoning`; `response_schema` maps to a strict `json_schema`; usage is requested in streams. - **Errors.** Everything is an `ApiError`; the Hub error body `{"error":"..."}` and the Endpoints body `{"error":"401 Unauthorized","code":"UNAUTHORIZED"}` keep their message and status. A local budget stop is `429 LOCAL_BUDGET`. - **Tests.** `MOLO_ENV_FILES=off`; `MOLO_LIVE_NET=1` enables the live tests; `MOLO_LIVE_PAID=1` additionally allows the one chat completion. ## 5. Credentials | Env var | Page | |---|---| | `HF_TOKEN` | https://huggingface.co/settings/tokens (docs: https://huggingface.co/docs/hub/security-tokens). A fine-grained token needs "Make calls to Inference Providers" for the router, "Make calls to Inference Endpoints" and "Manage Inference Endpoints" for the endpoints API, and write scopes for discussions / collections / webhooks. Inference Providers bills the token owner (or the organization named by `RouterChat::set_bill_to`) | No key exists in the development environment: see sections 7 and 8. ## 6. Sources Machine-readable documents, fetched 2026-10-25: - Hub OpenAPI 3.1 `https://huggingface.co/.well-known/openapi.json` (374 operations, 307 paths; also `/.well-known/openapi.md`). It omits the list / info routes of models, datasets and spaces (`GET /api/models`, `/api/models/{ns}/{repo}`, ...), `/api/tasks` and `/api/spaces/{ns}/{repo}/runtime`; those 11 Hub rows cite `huggingface_hub` `hf_api.py` (spec kind `src`); the three hf-inference router rows cite `hf_inference.py` (also `src`). - Inference Endpoints OpenAPI 2.0.0 `https://api.endpoints.huggingface.cloud/openapi.json` (48 operations). The old `/api-doc/openapi.json` answers 404; the swagger page at `/` points at `/openapi.json`. - TGI `docs/openapi.json` (3.3.6-dev0, 12 operations) and TEI `docs/openapi.json` (1.9.4, 12 operations) from the `huggingface/text-generation-inference` and `huggingface/text-embeddings-inference` repositories, branch `main`. - `huggingface_hub` `src/huggingface_hub/hf_api.py`, `constants.py`, `inference/_providers/__init__.py` and `hf_inference.py`: endpoints and parameters of the routes the OpenAPI omits, the router URL template `https://router.huggingface.co/{provider}`, the 25-task hf-inference map, the feature-extraction / sentence-similarity `/pipeline/{task}` path, binary-input tasks, base64 form with parameters. - `huggingface.js` `packages/tasks/src/tasks//spec/input.json` for the chat-completion body fields (26 tasks fetched; `audio-to-audio` and `tabular-classification` have no schema file). - Unauthenticated probes, section 8. ## 7. Verified against spec `python3 -I src/vendors/huggingface/tools/check_spec.py --probe`, 2026-10-25: ``` 132 rows checked; spec operations hub=374 ie=48 tgi=12 tei=12; 0 problems ``` For `ie`, `tgi`, `tei`: every document operation, query parameter and body field is in the table (equality). For `hub`: every row's method, path (expanded over models / datasets / spaces where the document has one path per type), query parameters and body fields exist in the document; the other 314 operations are excluded by explicit rules in `EXCLUDED`, counted per reason in the output (jobs 23, buckets / containers / kernels 33, organizations / users / SCIM 77, ...). The chat row is checked against the huggingface.js chat-completion schema (equality of fields). `--probe` GETs the `src` and `none` rows without a credential: all answered 200 (public) or 401 (the Hub's answer for an unknown or private repo id), none 404. Local tests: `make ai-test` runs 66 `huggingface_*` tests (table integrity, id and path rules, query and body checks, caps, Link walks, budget window, token header, anonymous header, error mapping, chat wire, SSE transcript, every surface's request shapes). Replies in tests are `RECORDED` (copied from a probe, trimmed) or `SHAPE` (written to the documented schema). Nothing is a scripted claim about vendor behavior. Example route tests: `tests/example/ai/huggingface/`. ## 8. Verified live Run on 2026-10-25 with no credential unless stated. All through the real HTTP transport (`MOLO_LIVE_NET=1 make ai-test`: 4 live tests, 3 run without a key). | Request | Result | |---|---| | `GET huggingface.co/api/models?limit=2`, `/api/datasets?limit=2`, `/api/spaces?limit=2` | 200 JSON array; `Link: <...&cursor=...>; rel="next"`; `RateLimit-Policy: "fixed window";"api";q=500;w=300` | | `GET /api/models/gpt2`, `/api/datasets/squad`, `/api/models/gpt2/refs` | 307 to the canonical `openai-community/gpt2` / `rajpurkar/squad` (the reason legacy ids are refused) | | `GET /api/models/openai-community/gpt2`, `/refs`, `/tree/main`, `/discussions`; `POST /paths-info/main`; `GET /openai-community/gpt2/resolve/main/config.json` (665 bytes, `X-Repo-Commit` header) | 200 | | `GET /api/spaces/gradio/hello_world`, `/api/spaces/hardware`, `/api/trending`, `/api/models-tags-by-type`, `/api/tasks`, `/api/daily_papers`, `/api/quicksearch`, `/api/collections?limit=2`, `/api/collections/nvidia/flora-6ad6a8986320a55467825cf5`, `/api/partners/groq/models` | 200 | | `GET /api/whoami-v2`, `/api/settings/webhooks` without a token | 401 `{"error":"Invalid username or password."}` | | `GET router.huggingface.co/v1/models` | 200, 134 models, per model a `providers` array (status, context_length, pricing, supports_tools, latency) | | `POST router.huggingface.co/v1/chat/completions`, `/v1/embeddings`, `/hf-inference/models/gpt2`, any other path | 401 `text/html` without a token, including a made-up path. The router answers 401 before routing, so these probes do NOT prove a path exists | | `GET router.huggingface.co/hf-inference/models/gpt2` | 401 JSON `{"error":"Invalid username or password."}` | | `GET api.endpoints.huggingface.cloud/v2/provider` | 200 (public vendor / region / compute / price list) | | `GET .../v2/endpoint/huggingface`, `/v2/provider/huggingface`, `/v2/search/huggingface`, `/v2/endpoint/huggingface/x/logs` | 401 `{"error":"401 Unauthorized","code":"UNAUTHORIZED"}` | | Invalid token `hf_invalid_molo_probe` on `whoami` and `endpoints list` | 401 with the two envelopes above (live test) | | `api-inference.huggingface.co` | host does not resolve (the classic host is gone; hf-inference lives behind the router) | | `datasets-server.huggingface.co/is-valid?dataset=squad` | 404 `{"error":"The dataset has been renamed..."}` (out of scope, noted) | ## 9. Unverified No token was available, so no authenticated success body was seen: - every `auth: token` call: whoami, space runtime, discussions create / comment / status / title / delete, collections create / update / add item / delete, webhooks (all), Endpoints (all except `GET /v2/provider`); - the router chat completion, its routing suffixes, the `X-HF-Bill-To` header (taken from huggingface_hub / docs, unprobed), streaming usage in the last chunk, `reasoning` vs `reasoning_content` per provider; - every hf-inference task call and reply shape: the `/hf-inference/models/{model}` and `/pipeline/{task}` paths and the binary / base64 input forms come from `huggingface_hub` source, not from a response; - the response shapes our code reads (`choices`, `usage`, tool call fields) are the OpenAI-compatible shapes of the chat-completion spec, not seen from the router; - TGI and TEI: no server was reachable; request shapes follow their OpenAPI documents; `ready()` treats 503 as "starting" per TGI behavior and the documents, unobserved; - `Endpoints::validate_spec` checks only the required top-level keys, the name rule from the schema description and the `type` enum; - ZeroGPU / pro quotas and per-token rate limits other than the observed anonymous `q=500;w=300`; - text ranges in `/resolve` (no range support), xet / LFS downloads of files over 64 MiB. ## 10. Gaps - `llm::ApiClient` cannot send an unauthenticated request (workaround: marker header), has no GET stream (the three Endpoints SSE routes `GET .../sse`, `.../logs/sse` and `/v3/.../logs/sse` are in the table with flag `unsupported` and refused), no per-call timeout, and drops response headers on `request()` (paging uses `request_raw`). - `llm::ApiClient::build` refuses `..` in any request path including the query; query values with `..` are dot-encoded as a workaround. - Provider-specific router paths (`router.huggingface.co/{provider}/...`, for example text-to-image on fal-ai or Replicate) have no per-provider schema published; only `hf-inference` tasks and the generic chat route are wrapped. - Collection item update / delete (`.../items/{itemId}`) are not wrapped: the document names the path with the `{slug}-{id}` split. - Repo writes (commit, upload, branches, tags, LFS) are out of scope; `Hub` reads and downloads only. Legacy ids without owner are refused. - `kernels` repo type exists on the Hub and in the discussions document; not wrapped. - Services `Discussions`, `Collections`, `Webhooks` share `Surface::kHub`. - Google services used: none.