15 KiB
Hugging Face - Specification
Owned directory: src/vendors/huggingface/. Namespace molo::huggingface.
Depends on molo::llm (ApiClient, Service, SseDecoder, ChatModel,
Config) and molo::clients. Built by make ai (the vendor glob in
src/ai.mk picks the directory up). Demo: example/app/ai/HuggingfaceDemo.cpp,
routes under /ai/huggingface. Plans: plans/17-llm-vendor-platforms.md,
plans/18-breadth.md.
1. Scope
Hugging Face is five HTTP surfaces on four hosts. All are covered:
| Surface | Host | Class | What |
|---|---|---|---|
| Hub | https://huggingface.co |
Hub, Discussions, Collections, Webhooks |
models, datasets, spaces (list, info, revision), repo tree / refs / commits / compare / paths-info / likers / safetensors metadata, file download (/resolve), search, trending, tags, tasks, papers, Spaces catalogs, Inference Provider mappings (/api/partners/{provider}/models), whoami, discussions, collections, webhooks |
| Router (Inference Providers) | https://router.huggingface.co |
RouterChat, InferenceTasks |
OpenAI-compatible chat (/v1/chat/completions, routing suffix :fastest :cheapest :preferred :<provider>), /v1/models, hf-inference task APIs (25 tasks, the classic Inference API), hf-inference chat |
| Inference Endpoints | https://api.endpoints.huggingface.cloud |
Endpoints |
the whole management API v2 / v3: endpoints (list, get, create, update, pause, resume, scale to zero, delete), logs, 15 metric series, replicas, vendors and quotas, search, CIDR lists, custom domains, private services, audit logs |
| TGI server | any URL | TgiServer |
native (/generate, /generate_stream, /tokenize, /info, /health, /metrics) and OpenAI-compatible (/v1/chat/completions, /v1/completions, /v1/models, /invocations) |
| TEI server | any URL | TeiServer |
/embed, /embed_all, /embed_sparse, /predict, /rerank, /similarity, /tokenize, /decode, /v1/embeddings, /info, /health, /metrics |
Plus HuggingfaceChatModel : llm::ChatModel over the router (default), a TGI
server, or any ChatEndpoint.
Not covered (explicit, see tools/check_spec.py EXCLUDED and section 8):
repository writes (commit, upload, LFS, branches, tags, settings), organizations
and users, billing, Jobs, buckets / containers / kernels, gated-access requests,
SCIM, OAuth, datasets-server.huggingface.co, the huggingface.co/api/blog
and posts comment APIs, the provider-specific router paths.
2. Module map (one class per concern)
| File | Concern |
|---|---|
caps.h |
HfCaps: static ceilings |
calls.h/.cpp |
Call enum and kCalls: the ONE table of 132 HTTP calls (surface, method, path, spec kind, reference, auth, flags, body fields, query parameters) |
request_plan.h/.cpp |
RequestPlan, PathArgs, Query, RepoRef, RepoType: renders and validates one call from the table |
platform.h/.cpp |
Platform, Access, RequestBudget: hosts, HF_TOKEN, anonymous clients, shared call budget, server URL check |
hf_service.h/.cpp |
HfService : llm::Service: surface check, token check, budget, JSON / raw / bytes / paged / SSE calls |
hub.h/.cpp |
Hub |
community.h/.cpp |
Discussions, Collections, Webhooks (+ CollectionItem, WebhookSpec) |
providers.h/.cpp |
RouterChat, RouterModel, TaskCatalog, InferenceTasks |
endpoints.h/.cpp |
Endpoints, MetricKind |
servers.h/.cpp |
TgiServer, TeiServer |
chat_wire.h/.cpp |
ChatWire, ChatEndpoint: neutral chat <-> OpenAI-compatible wire |
chat_model.h/.cpp |
HuggingfaceChatModel |
tools/check_spec.py |
table vs published documents |
3. Static caps (caps.h)
| Knob | Value |
|---|---|
| Path segment / repo-relative file path / segments | 256 / 1024 chars / 32 |
| Query parameters / one query value | 32 / 2048 chars |
| Model id / server URL | 256 / 2048 chars |
| Local call budget | 1000 calls per 300 s per process (observed server limit: 500 per 300 s, RateLimit-Policy: "fixed window";"api";q=500;w=300) |
| Page walk | 20 pages, 5000 items, page limit 1000 |
paths-info paths |
256 |
| Download | 64 MiB (the llm response cap; larger fails loudly) |
| Inference input (binary) | 16 MiB |
| Batch inputs (TEI, embeddings) | 256 entries, 4 MiB text |
| Stream events | 200000 |
| Messages / tools / stop sequences | 4096 / 128 / 4 |
| Webhook domains | 8 |
4. Contracts
- Token.
HF_TOKEN(or the legacyHUGGING_FACE_HUB_TOKEN) viallm::Config. It travels asAuthorization: Beareronly, never in a URL or an error. A row withauth: tokenand no token throwsNotConfiguredError(503NOT_CONFIGURED) before any traffic. Rows withauth: noneoroptional(public Hub reads,GET /v1/models,GET /v2/provider, TGI / TEI) run without a token. - Anonymous requests.
llm::ApiClientcannot send an unauthenticated request, so without a tokenPlatformbuilds the client with a marker headerX-Molo-Anonymous: 1instead ofAuthorization. Foundation gap, worked around inside the package. - One table. Every path, query parameter and body field comes from
kCalls.RequestPlan::makerefuses a query parameter or body field the row does not list, an unused or missing path argument, a body on a bodiless row, and rows flaggedunsupported. - Ids.
namespace,repo,name:[A-Za-z0-9._-], 1..256, not./... Repo ids must beowner/name; legacy ids without an owner (gpt2) are refused because the Hub answers them with a redirect to the canonical id. A revision or tag is one percent-encoded segment (refs/pr/1->refs%2Fpr%2F1). File paths are validated per segment and percent-encoded. Query values are percent-encoded; a value containing..has its dots encoded becauseApiClientrefuses..anywhere in the request path. - Lists. Hub list calls follow
Link: <...>; rel="next"cursors, accept only thecursorof a Link to the same host, and returnPagewithtruncated,truncated_by(itemsorpages) andnext_cursor. Never silent. - Writes and deletes. Rows flagged
writeneed a token. Deletes areremove_*methods with no defaults, flaggeddestructivein the table (tested: 7 DELETE rows, all flagged). The demo runs every paid or changing action as a dry run unless the body has"confirm":true. - Chat wire.
ChatWiredocuments its mapping inchat_wire.h: tool arguments that are not JSON arrive as{"_invalid_json": raw}; reasoning is read fromreasoning_contentthenreasoning;response_schemamaps to a strictjson_schema; usage is requested in streams. - Errors. Everything is an
ApiError; the Hub error body{"error":"..."}and the Endpoints body{"error":"401 Unauthorized","code":"UNAUTHORIZED"}keep their message and status. A local budget stop is429 LOCAL_BUDGET. - Tests.
MOLO_ENV_FILES=off;MOLO_LIVE_NET=1enables the live tests;MOLO_LIVE_PAID=1additionally allows the one chat completion.
5. Credentials
| Env var | Page |
|---|---|
HF_TOKEN |
https://huggingface.co/settings/tokens (docs: https://huggingface.co/docs/hub/security-tokens). A fine-grained token needs "Make calls to Inference Providers" for the router, "Make calls to Inference Endpoints" and "Manage Inference Endpoints" for the endpoints API, and write scopes for discussions / collections / webhooks. Inference Providers bills the token owner (or the organization named by RouterChat::set_bill_to) |
No key exists in the development environment: see sections 7 and 8.
6. Sources
Machine-readable documents, fetched 2026-10-25:
- Hub OpenAPI 3.1
https://huggingface.co/.well-known/openapi.json(374 operations, 307 paths; also/.well-known/openapi.md). It omits the list / info routes of models, datasets and spaces (GET /api/models,/api/models/{ns}/{repo}, ...),/api/tasksand/api/spaces/{ns}/{repo}/runtime; those 11 Hub rows citehuggingface_hubhf_api.py(spec kindsrc); the three hf-inference router rows citehf_inference.py(alsosrc). - Inference Endpoints OpenAPI 2.0.0
https://api.endpoints.huggingface.cloud/openapi.json(48 operations). The old/api-doc/openapi.jsonanswers 404; the swagger page at/points at/openapi.json. - TGI
docs/openapi.json(3.3.6-dev0, 12 operations) and TEIdocs/openapi.json(1.9.4, 12 operations) from thehuggingface/text-generation-inferenceandhuggingface/text-embeddings-inferencerepositories, branchmain. huggingface_hubsrc/huggingface_hub/hf_api.py,constants.py,inference/_providers/__init__.pyandhf_inference.py: endpoints and parameters of the routes the OpenAPI omits, the router URL templatehttps://router.huggingface.co/{provider}, the 25-task hf-inference map, the feature-extraction / sentence-similarity/pipeline/{task}path, binary-input tasks, base64 form with parameters.huggingface.jspackages/tasks/src/tasks/<task>/spec/input.jsonfor the chat-completion body fields (26 tasks fetched;audio-to-audioandtabular-classificationhave no schema file).- Unauthenticated probes, section 8.
7. Verified against spec
python3 -I src/vendors/huggingface/tools/check_spec.py --probe, 2026-10-25:
132 rows checked; spec operations hub=374 ie=48 tgi=12 tei=12; 0 problems
For ie, tgi, tei: every document operation, query parameter and body
field is in the table (equality). For hub: every row's method, path (expanded
over models / datasets / spaces where the document has one path per type),
query parameters and body fields exist in the document; the other 314
operations are excluded by explicit rules in EXCLUDED, counted per reason in
the output (jobs 23, buckets / containers / kernels 33, organizations / users /
SCIM 77, ...). The chat row is checked against the huggingface.js
chat-completion schema (equality of fields). --probe GETs the src and
none rows without a credential: all answered 200 (public) or 401 (the Hub's
answer for an unknown or private repo id), none 404.
Local tests: make ai-test runs 66 huggingface_* tests (table integrity, id
and path rules, query and body checks, caps, Link walks, budget window, token
header, anonymous header, error mapping, chat wire, SSE transcript, every
surface's request shapes). Replies in tests are RECORDED (copied from a probe,
trimmed) or SHAPE (written to the documented schema). Nothing is a scripted
claim about vendor behavior. Example route tests: tests/example/ai/huggingface/.
8. Verified live
Run on 2026-10-25 with no credential unless stated. All through the real HTTP
transport (MOLO_LIVE_NET=1 make ai-test: 4 live tests, 3 run without a key).
| Request | Result |
|---|---|
GET huggingface.co/api/models?limit=2, /api/datasets?limit=2, /api/spaces?limit=2 |
200 JSON array; Link: <...&cursor=...>; rel="next"; RateLimit-Policy: "fixed window";"api";q=500;w=300 |
GET /api/models/gpt2, /api/datasets/squad, /api/models/gpt2/refs |
307 to the canonical openai-community/gpt2 / rajpurkar/squad (the reason legacy ids are refused) |
GET /api/models/openai-community/gpt2, /refs, /tree/main, /discussions; POST /paths-info/main; GET /openai-community/gpt2/resolve/main/config.json (665 bytes, X-Repo-Commit header) |
200 |
GET /api/spaces/gradio/hello_world, /api/spaces/hardware, /api/trending, /api/models-tags-by-type, /api/tasks, /api/daily_papers, /api/quicksearch, /api/collections?limit=2, /api/collections/nvidia/flora-6ad6a8986320a55467825cf5, /api/partners/groq/models |
200 |
GET /api/whoami-v2, /api/settings/webhooks without a token |
401 {"error":"Invalid username or password."} |
GET router.huggingface.co/v1/models |
200, 134 models, per model a providers array (status, context_length, pricing, supports_tools, latency) |
POST router.huggingface.co/v1/chat/completions, /v1/embeddings, /hf-inference/models/gpt2, any other path |
401 text/html without a token, including a made-up path. The router answers 401 before routing, so these probes do NOT prove a path exists |
GET router.huggingface.co/hf-inference/models/gpt2 |
401 JSON {"error":"Invalid username or password."} |
GET api.endpoints.huggingface.cloud/v2/provider |
200 (public vendor / region / compute / price list) |
GET .../v2/endpoint/huggingface, /v2/provider/huggingface, /v2/search/huggingface, /v2/endpoint/huggingface/x/logs |
401 {"error":"401 Unauthorized","code":"UNAUTHORIZED"} |
Invalid token hf_invalid_molo_probe on whoami and endpoints list |
401 with the two envelopes above (live test) |
api-inference.huggingface.co |
host does not resolve (the classic host is gone; hf-inference lives behind the router) |
datasets-server.huggingface.co/is-valid?dataset=squad |
404 {"error":"The dataset has been renamed..."} (out of scope, noted) |
9. Unverified
No token was available, so no authenticated success body was seen:
- every
auth: tokencall: whoami, space runtime, discussions create / comment / status / title / delete, collections create / update / add item / delete, webhooks (all), Endpoints (all exceptGET /v2/provider); - the router chat completion, its routing suffixes, the
X-HF-Bill-Toheader (taken from huggingface_hub / docs, unprobed), streaming usage in the last chunk,reasoningvsreasoning_contentper provider; - every hf-inference task call and reply shape: the
/hf-inference/models/{model}and/pipeline/{task}paths and the binary / base64 input forms come fromhuggingface_hubsource, not from a response; - the response shapes our code reads (
choices,usage, tool call fields) are the OpenAI-compatible shapes of the chat-completion spec, not seen from the router; - TGI and TEI: no server was reachable; request shapes follow their OpenAPI
documents;
ready()treats 503 as "starting" per TGI behavior and the documents, unobserved; Endpoints::validate_specchecks only the required top-level keys, the name rule from the schema description and thetypeenum;- ZeroGPU / pro quotas and per-token rate limits other than the observed
anonymous
q=500;w=300; - text ranges in
/resolve(no range support), xet / LFS downloads of files over 64 MiB.
10. Gaps
llm::ApiClientcannot send an unauthenticated request (workaround: marker header), has no GET stream (the three Endpoints SSE routesGET .../sse,.../logs/sseand/v3/.../logs/sseare in the table with flagunsupportedand refused), no per-call timeout, and drops response headers onrequest()(paging usesrequest_raw).llm::ApiClient::buildrefuses..in any request path including the query; query values with..are dot-encoded as a workaround.- Provider-specific router paths (
router.huggingface.co/{provider}/..., for example text-to-image on fal-ai or Replicate) have no per-provider schema published; onlyhf-inferencetasks and the generic chat route are wrapped. - Collection item update / delete (
.../items/{itemId}) are not wrapped: the document names the path with the{slug}-{id}split. - Repo writes (commit, upload, branches, tags, LFS) are out of scope;
Hubreads and downloads only. Legacy ids without owner are refused. kernelsrepo type exists on the Hub and in the discussions document; not wrapped.- Services
Discussions,Collections,WebhooksshareSurface::kHub. - Google services used: none.