feat(clients): cohere: whole API, CohereChatModel, spec checker, demo and tests

Package src/vendors/cohere (molo::cohere): Chat v2 with streaming, tools,
thinking and citations; embed, rerank, classify, tokenize, detokenize,
parse, transcription; datasets, embed jobs, batches, connectors,
fine-tuning, models, check-api-key; legacy v1 chat/generate/summarize;
CohereChatModel : ChatModel. All 42 operations of cohere-openapi.yaml are
in one call table, verified by tools/check_spec.py. Demo routes under
/ai/cohere, 51 unit tests and 2 route tests; live tests are opt-in.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-25 10:49:17 +01:00
co-authored by Claude Sonnet 5.5
parent 3db67fcf1f
commit f32947427b
36 changed files with 6045 additions and 3 deletions
+196
View File
@@ -0,0 +1,196 @@
# Cohere API platform - Specification
Owned directory: `src/vendors/cohere/`. Namespace `molo::cohere`. Depends on the
`src/llm` foundation (`ApiClient`, `Service`, `SseDecoder`, `ApiError`,
`Config`, `ChatModel`). Plans: `plans/17-llm-vendor-platforms.md`,
`plans/18-breadth.md`. Credential: `COHERE_API_KEY` via `llm::Config`
(`CO_API_KEY`, the official SDKs' name, is accepted as a fallback).
Tests: `tests/src/vendors/cohere/`. Demo: `example/app/ai/CohereDemo.cpp`,
routes `/ai/cohere`, script `example/assets/ai_cohere.js`, route tests
`tests/example/ai/cohere/`.
## 1. Scope
Every operation of Cohere's OpenAPI document (42 of 42, host
`https://api.cohere.com`, Bearer auth):
- Chat v2 (`POST /v2/chat`): messages (system, user with text and image parts,
assistant, tool), tools and `strict_tools`, `tool_choice`, documents and
citation options, JSON response format with schema, safety mode, thinking
(`token_budget`), sampling and penalties, logprobs, priority, streaming
(all eleven event types), `Conversation` with a capped tool loop.
- Legacy v1 that remains in the spec: chat, generate, summarize (Json bodies
checked against the call table), embed, rerank, classify.
- Embed v2 (texts, images, mixed `inputs`, `embedding_types`, `output_dimension`),
Rerank v2 (`relevant_spans`, `max_tokens_per_doc`), Tokenize, Detokenize,
Parse (`/v2/parse`), audio transcription (`/v2/audio/transcriptions`, multipart).
- Datasets (multipart upload, list, get, usage, delete), Embed jobs, Batches
(`/v2/batches`, cancel), Connectors (spec marks them deprecated), Fine-tuning
(spec marks it deprecated: list, create, get, update, delete, events,
training-step metrics), Models (list, get), `check-api-key`.
- `CohereChatModel : llm::ChatModel` over Chat v2.
Not here, said plainly: the spec defines cluster-job schemas
(`CreateClusterJobRequest` and friends) and `FeedbackResponse` as components,
but no path uses them, so there is nothing to call.
## 2. Module map (one class per concern)
| File | Concern |
|---|---|
| `caps.h` | Static ceilings, never autotuned |
| `calls.h/.cpp` | THE call table (42 rows) and the schema-field table; `render` (id and query validation), `check_body` |
| `rules.h/.cpp` | Input rules (model, ids, ranges, enums, URLs); all throw `ApiError::invalid` before any traffic |
| `types.h/.cpp` | Tolerant readers `J`, `TokenUsage`, `Page`, `ModelInfo`, `Batch`, `EmbedJob`, `Dataset`, `Connector`, `FinetunedModel` |
| `platform.h/.cpp` | Host, Bearer auth, `X-Client-Name: molo` header, key lookup |
| `chat_params.h/.cpp` | `Messages` builders, `ToolDef`, `ChatParams` (validate + body) |
| `chat_result.h/.cpp` | `ChatResult`, `StreamDelta`, `ChatStreamAccumulator` |
| `chat.h/.cpp` | `Chat` (create, stream, legacy v1), `Conversation` (history, tool loop) |
| `chat_model.h/.cpp` | `CohereChatModel : llm::ChatModel` and its documented mapping |
| `inference.h/.cpp` | `Embeddings`, `Rerank`, `Classify`, `Tokenizer`, `Legacy` |
| `resources.h/.cpp` | `Account`, `Datasets`, `Batches`, `EmbedJobs`, `Connectors`, `FineTuning`, `Documents` |
| `catalog.h/.cpp` | Model names from the docs page, defaults |
| `tools/check_spec.py` | Spec checker (see Verified against spec) |
## 3. Static caps
| Knob | Value | Origin |
|---|---|---|
| Embed texts / images / inputs per call | 96 | spec `maxItems` (texts); ours for the rest |
| Classify inputs | 96 | spec `maxItems` |
| Tokenize text / detokenize tokens | 65536 | spec `maxLength` / `maxItems` |
| Chat stop sequences | 5 | spec description |
| `k` | 0..500, `p` 0.01..0.99, `temperature` 0..1, `priority` 0..999, penalties 0..1 | spec |
| Page size | 1..1000 | spec description |
| Messages / tools / documents | 4096 / 256 / 1000 | ours |
| Stream events | 200000 | ours |
| List walk | 20 pages, 5000 items | ours; truncation is reported (`Page::truncated`) |
| Upload (datasets, audio) | 256 MiB | ours (llm cap is 512 MiB) |
| Tool loop rounds | 16 | ours |
| Dataset validation polls | 60, wait supplied by the caller | ours |
| Body / response / stream bytes | `llm::Caps` | foundation |
## 4. Contracts
- No key: `configured()` is false and every call throws `NotConfiguredError`
(503, `NOT_CONFIGURED`) before any network traffic. The key travels in the
`Authorization` header only.
- Every request body is built from typed params or checked with
`Calls::check_body`: a field that is not in the call table is rejected
loudly. Path ids go through `Service::valid_id`; query names must be in the row.
- Destructive or cancelling calls (`Datasets::remove`, `Batches::cancel`,
`EmbedJobs::cancel`, `Connectors::remove`, `FineTuning::remove`) are explicit
methods with no defaults. Calls that cost money or train a model
(`FineTuning::create`, `Batches::create`, `EmbedJobs::create`) validate their
settings first.
- A stream without `message-end` throws `502 STREAM_TRUNCATED`; a `message-end`
with `delta.error` throws `502 STREAM_ERROR`; a JSON error body in place of a
stream throws (SSE decoder).
- Tool arguments arrive as the model's JSON text; `ToolCallRecord::arguments_valid()`
tells, and `CohereChatModel` keeps invalid text as `{"_invalid_json": "..."}`.
- `CohereChatModel` mapping (also in `chat_model.h`): `tool_choice` required/none
-> `REQUIRED`/`NONE`, a tool name is rejected (Cohere cannot force one tool);
`json_mode`/`response_schema` -> `response_format` json_object (+ `json_schema`);
`reasoning_effort` none/off/disabled -> thinking disabled, anything else ->
enabled; finish reasons COMPLETE and STOP_SEQUENCE -> stop, MAX_TOKENS ->
length, TOOL_CALL -> tool_calls; usage from `usage.tokens` with `billed_units`
as the fallback.
- Datasets: `keep_fields` and `optional_fields` are repeated query parameters
(what the official Python SDK does by passing a sequence to httpx).
- `Datasets::await_validation` never sleeps; the caller passes the wait.
## 5. Example routes (`/ai/cohere`)
GET: page, `/status`, `/calls` (no credential). POST: `chat`, `stream`,
`tools`, `legacy`, `embed`, `rerank`, `classify`, `tokenize`, `detokenize`,
`parse`, `models`, `key`, `datasets`, `batches`, `embedjobs`, `connectors`,
`finetune` (resource routes take an `action`). 422 on bad input, 503
`cohere_not_configured` without a key, 502 `cohere_error` for vendor errors.
Chat, stream, tools, embed, rerank, classify, parse, legacy and every create,
update, cancel and delete are dry runs unless the body has `"confirm":true`.
Reads and tokenize run directly once a key exists.
## Sources
1. `cohere-openapi.yaml` in `cohere-ai/cohere-developer-experience` (branch
`main`, fetched 2026-10-25, 1,140,155 bytes, ETag
`148fca0f27bb02cdf86b2eb95b7d9a02efa1bca0263cae8d6c9dcc571cb17d60`),
https://raw.githubusercontent.com/cohere-ai/cohere-developer-experience/main/cohere-openapi.yaml .
OpenAPI 3.1, 42 operations, 191 component schemas. Its `x-fern-examples` and
`x-examples` are the fixtures in `tests/src/vendors/cohere/fixtures.h`.
2. Official Python SDK `cohere-ai/cohere-python` (`src/cohere/*/raw_client.py`,
fetched 2026-10-25): 31 path templates match the spec; the repeated query
parameter behaviour for datasets comes from here.
3. Docs model list https://docs.cohere.com/docs/models (read 2026-10-25): model
names, status and limits in `catalog.cpp`.
4. Live unauthenticated probes (below).
## Verified against spec
`python3 -I src/vendors/cohere/tools/check_spec.py` (2026-10-25, against the
fetched document): `cohere spec ok: 42 calls of 42 spec operations (exclusions: 0),
75 schema rows, 263 fields`. The checker parses the YAML with a small stdlib
reader (compared with PyYAML on the whole file: identical except for long flow
arrays inside examples, which it does not use). It fails on a method+path or
operationId missing from the spec, a body field or query parameter we send that
the operation does not have, a spec-required field missing from the row, a
schema field we read that is not a property (dotted paths descend into nested
objects and arrays), a response name that is neither a component nor an
operationId, and any spec operation that is neither in the table nor in the
(empty) exclusion list. Negative runs with a wrong field, a missing required
field, a wrong response name and a wrong schema field each fail as expected.
## Verified live
Unauthenticated probes, 2026-10-25, no credential used:
- TLS and DNS for `api.cohere.com` work (HTTP/2).
- `POST /v2/chat` with no key: 401 `{"id":"...","message":"no api key supplied"}`.
- `GET /v1/models` and `POST /v2/chat` with an invalid bearer: 401
`{"id":"...","message":"Incorrect API key provided: ***...robe. You can find your API key at https://dashboard.cohere.com/api-keys."}`.
- `POST /v1/check-api-key` with an invalid bearer: HTTP 200 `{"valid":false}`.
- All 42 documented method+path pairs, called without a key: 42 x 401. A made-up
path (`GET /v1/nothing-here`) also answers 401, so the gateway authenticates
before routing and a 401 does NOT prove a route exists. Existence is verified
against the spec and the SDK only.
- Opt-in tests: `MOLO_LIVE_NET=1` runs the invalid-key probes
(`cohere_live.invalid_key_probes` passed on 2026-10-25); with `COHERE_API_KEY`
also key check, chat model list and ONE tiny chat; `MOLO_LIVE_PAID=1` adds one
embed and one rerank. Without them each prints a loud SKIP. Never run so far
with a key (none exists here).
## Unverified
No success response from Cohere has been seen. Everything below is inferred:
- Real reply bodies for models list/get, datasets, batches, embed jobs,
connectors, fine-tuning, parse, transcription and generate. Tests use minimal
documents labelled SHAPE, built from the OpenAPI schemas.
- Streaming of tool calls and citations beyond the two documented transcripts
(text, tools). Citation, thinking and logprob stream events are handled from
the schema.
- That `keep_fields`/`optional_fields` are repeated query parameters on the wire.
- `X-Client-Name` is optional in the spec; we send `molo`.
- Whether `command-a-03-2025`, `embed-v4.0`, `rerank-v3.5` stay valid defaults;
the docs page lists them (2026-10-25) and the live list is the truth.
- Tokenize and detokenize pricing and whether they need a model that supports
them; the demo defaults to `command-a-03-2025`.
- Spec says `GET /v1/models` pages with `page_size` up to 1000 (our walk uses 1000).
- Classify without a fine-tuned model: the 2-example minimum is our guard, not a
documented rule. The endpoint is marked deprecated in the spec.
- Behaviour of deprecated surfaces (connectors, fine-tuning, classify, generate,
summarize): present in the spec, not exercised.
## Gaps
- `src/llm`: `ApiClient::stream` is POST-only; `post_multipart` parses JSON replies
only; no per-call timeout; `ApiError` keeps no `Retry-After`. Cohere's 429
semantics (trial key limits) are therefore not surfaced beyond the status.
- No `CO-Api-Key` style header or v1 streaming (`/v1/chat` stream, `/v1/generate`
stream use a different event format); both are rejected by `stream:true` on
purpose.
- Embed results are kept as `Json`; typed accessors exist for floats only.
- `FineTuning` settings and connector OAuth are passed as `Json` (validated for
the required keys only).
- Credential console: https://dashboard.cohere.com/api-keys (named in Cohere's own
401 body, 2026-10-25). Owner action: add the row to `src/llm/SPEC.md` section 5.
+214
View File
@@ -0,0 +1,214 @@
#include "vendors/cohere/calls.h"
#include <cstring>
#include <set>
#include "clients/url.h"
#include "llm/api_client.h"
#include "llm/error.h"
namespace molo::cohere {
// CALLS-BEGIN (parsed by tools/check_spec.py; one row per line, strings in order:
// http method, path, operationId, body fields, query params, response schema)
static const CallSpec kCalls[] = {
{Call::kChatV1, "POST", "/v1/chat", "chat", "message,model,stream,preamble,chat_history,conversation_id,prompt_truncation,connectors,search_queries_only,documents,citation_quality,temperature,max_tokens,max_input_tokens,k,p,seed,stop_sequences,frequency_penalty,presence_penalty,raw_prompting,tool_results,tools,force_single_step,response_format,safety_mode", "", "chat"},
{Call::kChat, "POST", "/v2/chat", "chatv2", "model,messages,stream,tools,strict_tools,documents,citation_options,response_format,safety_mode,max_tokens,stop_sequences,temperature,seed,frequency_penalty,presence_penalty,k,p,logprobs,tool_choice,thinking,priority", "", "ChatResponseV2"},
{Call::kParse, "POST", "/v2/parse", "parsev2", "model,document,output_format", "", "ParseResponse"},
{Call::kGenerate, "POST", "/v1/generate", "generate", "model,prompt,stream,num_generations,max_tokens,truncate,temperature,seed,preset,end_sequences,stop_sequences,k,p,frequency_penalty,presence_penalty,return_likelihoods,raw_prompting", "", "Generation"},
{Call::kEmbedV1, "POST", "/v1/embed", "embed", "texts,images,model,input_type,embedding_types,truncate", "", "embed"},
{Call::kEmbed, "POST", "/v2/embed", "embedv2", "model,input_type,texts,images,inputs,max_tokens,output_dimension,embedding_types,truncate,priority", "", "EmbedByTypeResponse"},
{Call::kTranscribe, "POST", "/v2/audio/transcriptions", "create-transcription", "model,language,temperature,file", "", "create-transcription"},
{Call::kBatchList, "GET", "/v2/batches", "ListBatches", "", "page_size,page_token,order_by", "ListBatchesResponse"},
{Call::kBatchCreate, "POST", "/v2/batches", "CreateBatch", "name,input_dataset_id,model", "", "CreateBatchResponse"},
{Call::kBatchGet, "GET", "/v2/batches/{id}", "GetBatch", "", "", "GetBatchResponse"},
{Call::kBatchCancel, "POST", "/v2/batches/{id}:cancel", "CancelBatch", "", "", "CancelBatchResponse"},
{Call::kEmbedJobCreate, "POST", "/v1/embed-jobs", "create-embed-job", "model,dataset_id,input_type,name,embedding_types,truncate", "", "CreateEmbedJobResponse"},
{Call::kEmbedJobList, "GET", "/v1/embed-jobs", "list-embed-jobs", "", "", "ListEmbedJobResponse"},
{Call::kEmbedJobGet, "GET", "/v1/embed-jobs/{id}", "get-embed-job", "", "", "EmbedJob"},
{Call::kEmbedJobCancel, "POST", "/v1/embed-jobs/{id}/cancel", "cancel-embed-job", "", "", ""},
{Call::kRerankV1, "POST", "/v1/rerank", "rerank", "model,query,documents,top_n,rank_fields,return_documents,max_chunks_per_doc", "", "rerank"},
{Call::kRerank, "POST", "/v2/rerank", "rerankv2", "model,query,documents,top_n,max_tokens_per_doc,relevant_spans,priority", "", "rerankv2"},
{Call::kClassify, "POST", "/v1/classify", "classify", "inputs,examples,model,preset,truncate", "", "classify"},
{Call::kDatasetCreate, "POST", "/v1/datasets", "create-dataset", "data,eval_data", "name,type,keep_original_file,skip_malformed_input,keep_fields,optional_fields,text_separator,csv_delimiter", "create-dataset"},
{Call::kDatasetList, "GET", "/v1/datasets", "list-datasets", "", "datasetType,before,after,limit,offset,validationStatus", "list-datasets"},
{Call::kDatasetUsage, "GET", "/v1/datasets/usage", "get-dataset-usage", "", "", "get-dataset-usage"},
{Call::kDatasetGet, "GET", "/v1/datasets/{id}", "get-dataset", "", "", "get-dataset"},
{Call::kDatasetDelete, "DELETE", "/v1/datasets/{id}", "delete-dataset", "", "", ""},
{Call::kSummarize, "POST", "/v1/summarize", "summarize", "text,length,format,model,extractiveness,temperature,additional_command", "", "summarize"},
{Call::kTokenize, "POST", "/v1/tokenize", "tokenize", "text,model", "", "tokenize"},
{Call::kDetokenize, "POST", "/v1/detokenize", "detokenize", "tokens,model", "", "detokenize"},
{Call::kConnectorList, "GET", "/v1/connectors", "list-connectors", "", "limit,offset", "ListConnectorsResponse"},
{Call::kConnectorCreate, "POST", "/v1/connectors", "create-connector", "name,url,description,excludes,oauth,active,continue_on_failure,service_auth", "", "CreateConnectorResponse"},
{Call::kConnectorGet, "GET", "/v1/connectors/{id}", "get-connector", "", "", "GetConnectorResponse"},
{Call::kConnectorUpdate, "PATCH", "/v1/connectors/{id}", "update-connector", "name,url,excludes,oauth,active,continue_on_failure,service_auth", "", "UpdateConnectorResponse"},
{Call::kConnectorDelete, "DELETE", "/v1/connectors/{id}", "delete-connector", "", "", ""},
{Call::kConnectorAuthorize, "POST", "/v1/connectors/{id}/oauth/authorize", "oAuthAuthorize-connector", "", "after_token_redirect", "OAuthAuthorizeResponse"},
{Call::kModelGet, "GET", "/v1/models/{model}", "get-model", "", "", "GetModelResponse"},
{Call::kModelList, "GET", "/v1/models", "list-models", "", "page_size,page_token,endpoint,default_only", "ListModelsResponse"},
{Call::kCheckApiKey, "POST", "/v1/check-api-key", "checkAPIKey", "", "", "checkAPIKey"},
{Call::kFinetuneList, "GET", "/v1/finetuning/finetuned-models", "ListFinetunedModels", "", "page_size,page_token,order_by", "ListFinetunedModelsResponse"},
{Call::kFinetuneCreate, "POST", "/v1/finetuning/finetuned-models", "CreateFinetunedModel", "name,settings", "", "CreateFinetunedModelResponse"},
{Call::kFinetuneUpdate, "PATCH", "/v1/finetuning/finetuned-models/{id}", "UpdateFinetunedModel", "name,settings", "", "UpdateFinetunedModelResponse"},
{Call::kFinetuneGet, "GET", "/v1/finetuning/finetuned-models/{id}", "GetFinetunedModel", "", "", "GetFinetunedModelResponse"},
{Call::kFinetuneDelete, "DELETE", "/v1/finetuning/finetuned-models/{id}", "DeleteFinetunedModel", "", "", ""},
{Call::kFinetuneEvents, "GET", "/v1/finetuning/finetuned-models/{finetuned_model_id}/events", "ListEvents", "", "page_size,page_token,order_by", "ListEventsResponse"},
{Call::kFinetuneMetrics, "GET", "/v1/finetuning/finetuned-models/{finetuned_model_id}/training-step-metrics", "ListTrainingStepMetrics", "", "page_size,page_token", "ListTrainingStepMetricsResponse"},
};
// CALLS-END
// SCHEMAS-BEGIN (parsed by tools/check_spec.py; one row per line: schema, fields)
static const SchemaFields kSchemas[] = {
{"ChatResponseV2", "id,finish_reason,message,usage,logprobs"},
{"AssistantMessageResponse", "role,tool_calls,tool_plan,content,citations"},
{"ChatTextContent", "type,text"},
{"ChatThinkingContent", "type,thinking"},
{"ToolCallV2", "id,type,function"},
{"ToolCallV2.function", "name,arguments"},
{"Usage", "billed_units,tokens,cached_tokens"},
{"Usage.billed_units", "input_tokens,output_tokens,search_units,classifications"},
{"Usage.tokens", "input_tokens,output_tokens"},
{"Citation", "start,end,text,sources,content_index,type"},
{"ChatMessageStartEvent", "type,id,delta"},
{"ChatContentStartEvent", "type,index,delta"},
{"ChatContentStartEvent.delta.message.content", "type,text,thinking"},
{"ChatContentDeltaEvent", "type,index,delta,logprobs"},
{"ChatContentDeltaEvent.delta.message.content", "text,thinking"},
{"ChatContentEndEvent", "type,index"},
{"ChatToolPlanDeltaEvent", "type,delta"},
{"ChatToolPlanDeltaEvent.delta.message", "tool_plan"},
{"ChatToolCallStartEvent", "type,index,delta"},
{"ChatToolCallStartEvent.delta.message", "tool_calls"},
{"ChatToolCallDeltaEvent", "type,index,delta"},
{"ChatToolCallDeltaEvent.delta.message.tool_calls.function", "arguments"},
{"ChatToolCallEndEvent", "type,index"},
{"CitationStartEvent", "type,index,delta"},
{"CitationStartEvent.delta.message", "citations"},
{"ChatMessageEndEvent", "type,id,delta"},
{"ChatMessageEndEvent.delta", "error,finish_reason,usage"},
{"chat", "text,generation_id,response_id,citations,documents,is_search_required,search_queries,search_results,finish_reason,tool_calls,chat_history,meta"},
{"ParseResponse", "id,pages,meta,finish_reason"},
{"Generation", "id,prompt,generations,meta"},
{"embed", "id,embeddings,texts,images,meta,response_type"},
{"EmbedByTypeResponse", "id,embeddings,texts,images,meta,response_type"},
{"EmbedByTypeResponse.embeddings", "float,int8,uint8,binary,ubinary,base64"},
{"ApiMeta", "api_version,billed_units,tokens,cached_tokens,warnings"},
{"create-transcription", "text"},
{"ListBatchesResponse", "batches,next_page_token"},
{"Batch", "id,name,status,created_at,updated_at,input_dataset_id,output_dataset_id,model,input_tokens,output_tokens,num_records,num_successful_records,num_failed_records,status_reason,creator_id,org_id"},
{"CreateBatchResponse", "batch"},
{"GetBatchResponse", "batch"},
{"CreateEmbedJobResponse", "job_id,meta"},
{"ListEmbedJobResponse", "embed_jobs"},
{"EmbedJob", "job_id,name,status,created_at,input_dataset_id,output_dataset_id,model,truncate,meta"},
{"rerank", "id,results,meta"},
{"rerank.results", "index,relevance_score,document"},
{"rerankv2", "id,results,meta"},
{"rerankv2.results", "index,relevance_score,relevant_spans"},
{"classify", "id,classifications,meta"},
{"classify.classifications", "id,input,prediction,predictions,confidence,confidences,labels,classification_type"},
{"create-dataset", "id"},
{"list-datasets", "datasets"},
{"get-dataset", "dataset"},
{"get-dataset-usage", "organization_usage"},
{"Dataset", "id,name,created_at,updated_at,dataset_type,validation_status,validation_error,validation_warnings,dataset_parts,required_fields,preserve_fields"},
{"DatasetPart", "id,name,url,index,size_bytes,num_rows,original_url,samples"},
{"summarize", "id,summary,meta"},
{"tokenize", "tokens,token_strings,meta"},
{"detokenize", "text,meta"},
{"ListConnectorsResponse", "connectors,total_count"},
{"Connector", "id,organization_id,name,url,created_at,updated_at,excludes,auth_type,oauth,auth_status,active,continue_on_failure"},
{"CreateConnectorResponse", "connector"},
{"GetConnectorResponse", "connector"},
{"UpdateConnectorResponse", "connector"},
{"OAuthAuthorizeResponse", "redirect_url"},
{"GetModelResponse", "name,is_deprecated,endpoints,finetuned,context_length,tokenizer_url,default_endpoints,features,sampling_defaults"},
{"ListModelsResponse", "models,next_page_token"},
{"checkAPIKey", "valid,organization_id,owner_id"},
{"ListFinetunedModelsResponse", "finetuned_models,next_page_token,total_size"},
{"FinetunedModel", "id,name,creator_id,organization_id,settings,status,created_at,updated_at,completed_at,last_used"},
{"CreateFinetunedModelResponse", "finetuned_model"},
{"UpdateFinetunedModelResponse", "finetuned_model"},
{"GetFinetunedModelResponse", "finetuned_model"},
{"ListEventsResponse", "events,next_page_token,total_size"},
{"Event", "user_id,status,created_at"},
{"ListTrainingStepMetricsResponse", "step_metrics,next_page_token"},
{"TrainingStepMetrics", "created_at,step_number,metrics"},
};
// SCHEMAS-END
const CallSpec& Calls::spec_of(Call call) {
for (const CallSpec& s : kCalls)
if (s.call == call) return s;
throw llm::ApiError::invalid("cohere", "unknown call");
}
clients::Method Calls::method_of(Call call) {
const char* m = spec_of(call).http_method;
if (!std::strcmp(m, "GET")) return clients::Method::kGet;
if (!std::strcmp(m, "POST")) return clients::Method::kPost;
if (!std::strcmp(m, "PATCH")) return clients::Method::kPatch;
return clients::Method::kDelete;
}
std::size_t Calls::count() { return sizeof(kCalls) / sizeof(kCalls[0]); }
const CallSpec& Calls::at(std::size_t index) {
if (index >= count()) throw llm::ApiError::invalid("cohere", "call index out of range");
return kCalls[index];
}
std::size_t Calls::schema_count() { return sizeof(kSchemas) / sizeof(kSchemas[0]); }
const SchemaFields& Calls::schema_at(std::size_t index) {
if (index >= schema_count()) throw llm::ApiError::invalid("cohere", "schema index out of range");
return kSchemas[index];
}
namespace {
std::set<std::string> split(const char* list) {
std::set<std::string> out;
std::string cur;
for (const char* p = list;; ++p) {
if (*p == ',' || *p == '\0') {
if (!cur.empty()) out.insert(cur);
cur.clear();
if (*p == '\0') break;
} else {
cur += *p;
}
}
return out;
}
} // namespace
std::string Calls::render(Call call, const std::string& id,
const Query& query) {
const CallSpec& s = spec_of(call);
std::string path = s.path;
auto open = path.find('{');
if (open != std::string::npos) {
auto close = path.find('}', open);
if (!llm::Service::valid_id(id)) throw llm::ApiError::invalid("cohere", "bad id");
path.replace(open, close - open + 1, id);
} else if (!id.empty()) {
throw llm::ApiError::invalid("cohere", "this call takes no id");
}
std::set<std::string> allowed = split(s.query_params);
std::string qs;
for (const auto& [k, v] : query) {
if (!allowed.count(k)) throw llm::ApiError::invalid("cohere", "query parameter not in the call table: " + k);
qs += qs.empty() ? '?' : '&';
qs += clients::Url::encode_component(k) + "=" + clients::Url::encode_component(v);
}
return path + qs;
}
void Calls::check_body(Call call, const clients::body::Json& body) {
if (!body.is_object()) throw llm::ApiError::invalid("cohere", "body must be a JSON object");
std::set<std::string> allowed = split(spec_of(call).body_fields);
for (const auto& [k, v] : body.as_object())
if (!allowed.count(k)) throw llm::ApiError::invalid("cohere", "field not in the call table: " + k);
}
} // namespace molo::cohere
+66
View File
@@ -0,0 +1,66 @@
#pragma once
// The ONE table of REST calls this module makes. tools/check_spec.py checks
// every row against Cohere's OpenAPI document (cohere-openapi.yaml) and fails
// on any spec operation that has no row. Code builds every path, query string
// and body-field check from this table, never from a literal.
#include <cstddef>
#include <string>
#include <utility>
#include <vector>
#include <string>
#include "clients/body/json.h"
#include "clients/request.h"
namespace molo::cohere {
enum class Call {
kChatV1, kChat, kParse, kGenerate, kEmbedV1, kEmbed, kTranscribe,
kBatchList, kBatchCreate, kBatchGet, kBatchCancel,
kEmbedJobCreate, kEmbedJobList, kEmbedJobGet, kEmbedJobCancel,
kRerankV1, kRerank, kClassify,
kDatasetCreate, kDatasetList, kDatasetUsage, kDatasetGet, kDatasetDelete,
kSummarize, kTokenize, kDetokenize,
kConnectorList, kConnectorCreate, kConnectorGet, kConnectorUpdate, kConnectorDelete,
kConnectorAuthorize,
kModelGet, kModelList, kCheckApiKey,
kFinetuneList, kFinetuneCreate, kFinetuneUpdate, kFinetuneGet, kFinetuneDelete,
kFinetuneEvents, kFinetuneMetrics,
};
struct CallSpec {
Call call;
const char* http_method; // GET POST PATCH DELETE
const char* path; // from the host; one {placeholder} at most
const char* operation_id; // operationId in the OpenAPI document
const char* body_fields; // comma list: request body fields we send (JSON or multipart)
const char* query_params; // comma list: query parameters we send
const char* response; // component schema name, or the operationId of an inline 200 body
};
struct SchemaFields {
const char* schema; // component name, operationId (inline 200 body), optionally ".a.b" property path
const char* fields; // comma list: fields we read
};
using Query = std::vector<std::pair<std::string, std::string>>; // repeat a name for list values
struct Calls {
static const CallSpec& spec_of(Call call);
static clients::Method method_of(Call call);
static std::size_t count();
static const CallSpec& at(std::size_t index);
static std::size_t schema_count();
static const SchemaFields& schema_at(std::size_t index);
// Path with the placeholder replaced by a validated id and the query
// parameters appended (percent-encoded). A query name that is not in the
// row, or an id that is not a valid path segment, throws ApiError::invalid.
static std::string render(Call call, const std::string& id = "",
const Query& query = {});
// Throws ApiError::invalid when `body` has a field the row does not declare.
static void check_body(Call call, const clients::body::Json& body);
};
} // namespace molo::cohere
+37
View File
@@ -0,0 +1,37 @@
#pragma once
// Static ceilings for the Cohere module. Never autotuned. "Documented" caps
// come from cohere-openapi.yaml (see SPEC.md ## Sources); the rest are ours.
#include <cstddef>
namespace molo::cohere {
struct CohereCaps {
// Documented (OpenAPI constraints).
static constexpr std::size_t kMaxEmbedTexts = 96; // /v2/embed texts maxItems
static constexpr std::size_t kMaxClassifyInputs = 96; // /v1/classify inputs maxItems
static constexpr std::size_t kMaxTokenizeChars = 65536; // /v1/tokenize text maxLength
static constexpr std::size_t kMaxDetokenizeTokens = 65536;
static constexpr std::size_t kMaxStopSequences = 5; // chat stop_sequences ("up to 5")
static constexpr int kMaxTopK = 500;
static constexpr int kMaxPriority = 999;
static constexpr int kMaxPageSize = 1000; // batches, models, finetuning pages
// Our own caps.
static constexpr std::size_t kMaxMessages = 4096;
static constexpr std::size_t kMaxTools = 256;
static constexpr std::size_t kMaxDocuments = 1000; // chat documents, rerank documents
static constexpr std::size_t kMaxEmbedInputs = 96;
static constexpr std::size_t kMaxStreamEvents = 200000;
static constexpr std::size_t kMaxListPages = 20; // list_all page walks
static constexpr std::size_t kMaxItemsReturned = 5000;
static constexpr std::size_t kMaxUploadBytes = 256u * 1024u * 1024u; // dataset parts
static constexpr std::size_t kMaxFilenameBytes = 512;
static constexpr std::size_t kMaxIdChars = 256;
static constexpr std::size_t kMaxNameChars = 256;
static constexpr std::size_t kMaxModelChars = 128;
static constexpr int kDatasetListMaxLimit = 1000;
static constexpr int kConnectorListMaxLimit = 1000;
};
} // namespace molo::cohere
+43
View File
@@ -0,0 +1,43 @@
#include "vendors/cohere/catalog.h"
#include "llm/error.h"
namespace molo::cohere {
namespace {
constexpr ModelName kModels[] = {
{"command-a-plus-05-2026", ModelKind::kChat, ModelStatus::kLive, "MoE, text and images, 128k context, 64k output"},
{"command-a-03-2025", ModelKind::kChat, ModelStatus::kLive, "text, 256k context, 8k output, tool use and RAG"},
{"command-a-reasoning-08-2025", ModelKind::kChat, ModelStatus::kLive, "reasoning model"},
{"command-a-vision-07-2025", ModelKind::kChat, ModelStatus::kLive, "image input"},
{"command-a-translate-08-2025", ModelKind::kChat, ModelStatus::kLive, "translation"},
{"command-r7b-12-2024", ModelKind::kChat, ModelStatus::kLive, "small, 128k context, 4k output"},
{"command-r-plus-08-2024", ModelKind::kChat, ModelStatus::kLive, "128k context, 4k output"},
{"command-r-08-2024", ModelKind::kChat, ModelStatus::kLive, "128k context, 4k output"},
{"command-r-03-2024", ModelKind::kChat, ModelStatus::kDeprecated, "deprecated Sept 15, 2025"},
{"command-r-plus-04-2024", ModelKind::kChat, ModelStatus::kDeprecated, "deprecated Sept 15, 2025"},
{"embed-v5.0-pro", ModelKind::kEmbed, ModelStatus::kLive, "text, images, mixed; dimensions 256..2048 (default 2048)"},
{"embed-v5.0-fast", ModelKind::kEmbed, ModelStatus::kLive, "faster, lighter embed-v5.0-pro"},
{"embed-v4.0", ModelKind::kEmbed, ModelStatus::kLive, "text and images"},
{"rerank-v5.0-pro", ModelKind::kRerank, ModelStatus::kLive, "highest quality, 32k context, calibrated scores"},
{"rerank-v5.0-fast", ModelKind::kRerank, ModelStatus::kLive, "low latency"},
{"rerank-v4.0-pro", ModelKind::kRerank, ModelStatus::kLive, "listed on the models page"},
{"rerank-v4.0-fast", ModelKind::kRerank, ModelStatus::kLive, "listed on the models page"},
{"rerank-v3.5", ModelKind::kRerank, ModelStatus::kLive, "multilingual, listed on the models page"},
};
} // namespace
std::size_t ModelCatalog::count() { return sizeof(kModels) / sizeof(kModels[0]); }
const ModelName& ModelCatalog::at(std::size_t index) {
if (index >= count()) throw llm::ApiError::invalid("cohere", "model index out of range");
return kModels[index];
}
ModelStatus ModelCatalog::status_of(const std::string& id) {
for (const ModelName& m : kModels)
if (id == m.id) return m.status;
return ModelStatus::kUnknown;
}
} // namespace molo::cohere
+31
View File
@@ -0,0 +1,31 @@
#pragma once
// Model names the docs page https://docs.cohere.com/docs/models lists (read
// 2026-10-25). The live GET /v1/models list is the truth; this table only
// explains names and gives the defaults the module uses.
#include <cstddef>
#include <string>
namespace molo::cohere {
enum class ModelKind { kChat, kEmbed, kRerank };
enum class ModelStatus { kLive, kDeprecated, kUnknown };
struct ModelName {
const char* id;
ModelKind kind;
ModelStatus status;
const char* note;
};
struct ModelCatalog {
static std::size_t count();
static const ModelName& at(std::size_t index);
static ModelStatus status_of(const std::string& id);
static const char* default_chat() { return "command-a-03-2025"; }
static const char* default_embed() { return "embed-v4.0"; }
static const char* default_rerank() { return "rerank-v3.5"; }
};
} // namespace molo::cohere
+83
View File
@@ -0,0 +1,83 @@
#include "vendors/cohere/chat.h"
#include "vendors/cohere/caps.h"
#include "vendors/cohere/rules.h"
namespace molo::cohere {
ChatResult Chat::create(const ChatParams& params) {
Json doc = client_->post(Calls::render(Call::kChat), params.to_json(false));
return ChatResult::parse(doc);
}
ChatResult Chat::stream(const ChatParams& params, const DeltaSink& on_delta) {
Json body = params.to_json(true);
ChatStreamAccumulator acc;
bool stopped = false;
std::size_t events = 0;
client_->stream(Calls::render(Call::kChat), body, [&](const llm::SseEvent& ev) {
if (++events > CohereCaps::kMaxStreamEvents)
throw llm::ApiError("cohere", 502, "STREAM_TOO_LONG", "stream over the event cap");
StreamDelta d = acc.feed(ev.data);
if (on_delta && !on_delta(d)) {
stopped = true;
return false;
}
return true;
});
if (!stopped && !acc.done())
throw llm::ApiError("cohere", 502, "STREAM_TRUNCATED", "stream ended without message-end");
return acc.result();
}
Json Chat::legacy(const Json& body) {
Calls::check_body(Call::kChatV1, body);
Rules::require(body.find("message") && body.find("message")->is_string(), "message is required");
const Json* s = body.find("stream");
Rules::require(!s || (s->is_bool() && !s->as_bool()), "legacy chat streaming is not supported here");
return client_->post(Calls::render(Call::kChatV1), body);
}
Conversation& Conversation::system(const std::string& text) {
params_.add(Messages::system(text));
return *this;
}
Conversation& Conversation::user(const std::string& text) {
params_.add(Messages::user(text));
return *this;
}
Conversation& Conversation::tool_result(const std::string& call_id, const std::string& content) {
params_.add(Messages::tool(call_id, content));
return *this;
}
ChatResult Conversation::send(Chat& chat) {
ChatResult r = chat.create(params_);
params_.add(r.history_message());
return r;
}
ChatResult Conversation::send_stream(Chat& chat, const Chat::DeltaSink& on_delta) {
ChatResult r = chat.stream(params_, on_delta);
params_.add(r.history_message());
return r;
}
ChatResult Conversation::run_tools(Chat& chat, const ToolHandler& handler, int max_rounds) {
Rules::require(max_rounds >= 1 && max_rounds <= kMaxToolRounds, "max_rounds must be 1..16");
Rules::require(!params_.tools.empty(), "run_tools needs tools");
ChatResult r;
for (int round = 0; round < max_rounds; ++round) {
r = send(chat);
if (r.tool_calls.empty()) return r;
for (const ToolCallRecord& call : r.tool_calls) {
if (!call.arguments_valid())
throw llm::ApiError("cohere", 502, "BAD_TOOL_ARGUMENTS",
"tool call '" + call.name + "' has invalid JSON arguments");
tool_result(call.id, handler(call.name, call.parsed(), call.id));
}
}
throw llm::ApiError::invalid("cohere", "tool loop hit the round cap with calls still pending");
}
} // namespace molo::cohere
+56
View File
@@ -0,0 +1,56 @@
#pragma once
// Chat v2 (POST /v2/chat) with streaming, Conversation (history keeper and
// tool loop) and the legacy v1 chat (POST /v1/chat, still in the spec).
#include <functional>
#include "vendors/cohere/calls.h"
#include "vendors/cohere/chat_params.h"
#include "vendors/cohere/chat_result.h"
#include "llm/api_client.h"
namespace molo::cohere {
class Chat : public llm::Service {
public:
explicit Chat(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
ChatResult create(const ChatParams& params);
using DeltaSink = std::function<bool(const StreamDelta&)>; // false: stop early
// Streams events; returns the folded result. A stream that ends without
// message-end (and was not stopped by the sink) throws 502 STREAM_TRUNCATED.
ChatResult stream(const ChatParams& params, const DeltaSink& on_delta);
// Legacy v1 chat. `body` fields must be in the call table; `message` and a
// non-streaming request are required (stream:true is rejected, v1 streams use a different event format).
Json legacy(const Json& body);
};
class Conversation {
public:
static constexpr int kMaxToolRounds = 16;
explicit Conversation(ChatParams base) : params_(std::move(base)) {}
Conversation& system(const std::string& text);
Conversation& user(const std::string& text);
Conversation& tool_result(const std::string& call_id, const std::string& content);
ChatResult send(Chat& chat);
ChatResult send_stream(Chat& chat, const Chat::DeltaSink& on_delta);
// handler(tool name, parsed arguments, call id) -> tool output text. Loops
// while the model asks for tools, at most `max_rounds` requests (<= 16).
using ToolHandler = std::function<std::string(const std::string&, const Json&, const std::string&)>;
ChatResult run_tools(Chat& chat, const ToolHandler& handler, int max_rounds = 8);
const ChatParams& params() const { return params_; }
ChatParams& params() { return params_; }
private:
ChatParams params_;
};
} // namespace molo::cohere
+206
View File
@@ -0,0 +1,206 @@
#include "vendors/cohere/chat_model.h"
#include <map>
#include "vendors/cohere/platform.h"
#include "vendors/cohere/rules.h"
#include "security/crypto/Base64.h"
namespace molo::cohere {
CohereChatModel::CohereChatModel(std::shared_ptr<Chat> chat, std::string default_model)
: chat_(std::move(chat)), default_model_(std::move(default_model)) {}
std::shared_ptr<CohereChatModel> CohereChatModel::from_env(const std::string& default_model) {
return std::make_shared<CohereChatModel>(std::make_shared<Chat>(Platform::from_env().client), default_model);
}
namespace {
Json image_part_of(const llm::ImageData& img) {
if (!img.url.empty()) return Messages::image_part(img.url);
Rules::require(!img.bytes.empty(), "image has neither url nor bytes");
Rules::require(!img.mime.empty(), "image bytes need a mime type");
return Messages::image_part("data:" + img.mime + ";base64," + security::crypto::Base64::encode(img.bytes));
}
void append(ChatParams& params, const llm::Message& m) {
switch (m.role) {
case llm::Role::kSystem:
params.add(Messages::system(m.text()));
return;
case llm::Role::kUser: {
bool has_image = false;
for (const llm::ContentPart& p : m.parts) has_image = has_image || p.image.has_value();
if (!has_image) {
params.add(Messages::user(m.text()));
return;
}
std::vector<Json> parts;
for (const llm::ContentPart& p : m.parts) {
if (!p.text.empty()) parts.push_back(Messages::text_part(p.text));
if (p.image) parts.push_back(image_part_of(*p.image));
}
params.add(Messages::user_parts(parts));
return;
}
case llm::Role::kAssistant: {
std::string thinking;
Json calls = Json::array();
for (const llm::ContentPart& p : m.parts) {
thinking += p.reasoning;
if (p.tool_call)
calls.push(Messages::tool_call(p.tool_call->id, p.tool_call->name,
p.tool_call->arguments.is_null() ? "{}" : p.tool_call->arguments.dump()));
}
params.add(Messages::assistant(m.text(), "", calls, thinking));
return;
}
case llm::Role::kTool: {
bool any = false;
for (const llm::ContentPart& p : m.parts)
if (p.tool_result) {
params.add(Messages::tool(p.tool_result->call_id, p.tool_result->content));
any = true;
}
Rules::require(any, "a tool message needs a tool result part");
return;
}
}
}
llm::Usage usage_of(const TokenUsage& u) {
llm::Usage out;
out.input_tokens = static_cast<int>(u.input_tokens ? u.input_tokens : u.billed_input_tokens);
out.output_tokens = static_cast<int>(u.output_tokens ? u.output_tokens : u.billed_output_tokens);
out.total_tokens = out.input_tokens + out.output_tokens;
out.cached_tokens = static_cast<int>(u.cached_tokens);
return out;
}
Json arguments_of(const ToolCallRecord& call) {
if (call.arguments_valid()) return call.parsed();
Json bad = Json::object();
bad.set("_invalid_json", Json(call.arguments));
return bad;
}
} // namespace
std::string CohereChatModel::neutral_finish(const std::string& r) {
if (r == "COMPLETE" || r == "STOP_SEQUENCE") return "stop";
if (r == "MAX_TOKENS") return "length";
if (r == "TOOL_CALL") return "tool_calls";
if (r == "ERROR") return "error";
if (r == "TIMEOUT") return "timeout";
return r;
}
ChatParams CohereChatModel::to_params(const llm::ChatRequest& r) const {
ChatParams p;
p.model = r.model.empty() ? default_model_ : r.model;
if (!r.system.empty()) p.add(Messages::system(r.system));
for (const llm::Message& m : r.messages) append(p, m);
for (const llm::ToolSpec& t : r.tools) {
ToolDef d;
d.name = t.name;
d.description = t.description;
d.parameters = t.parameters;
p.tools.push_back(d);
}
if (r.tool_choice == "required") p.tool_choice = "REQUIRED";
else if (r.tool_choice == "none") p.tool_choice = "NONE";
else if (!r.tool_choice.empty() && r.tool_choice != "auto")
Rules::fail("Cohere cannot force a named tool; use required, none or auto");
p.max_tokens = r.max_tokens;
p.temperature = r.temperature;
p.p = r.top_p;
p.stop = r.stop;
if (r.json_mode || r.response_schema.is_object()) {
p.json_mode = true;
if (r.response_schema.is_object()) p.json_schema = r.response_schema;
}
if (!r.reasoning_effort.empty()) {
bool off = r.reasoning_effort == "none" || r.reasoning_effort == "off" || r.reasoning_effort == "disabled";
p.thinking = off ? Thinking::kDisabled : Thinking::kEnabled;
}
if (r.extra.is_object()) {
Json rest = Json::object();
for (const auto& [k, v] : r.extra.as_object()) {
if (k == "thinking" && v.is_object()) {
std::string type = J::str(v, "type");
if (type == "enabled") p.thinking = Thinking::kEnabled;
else if (type == "disabled") p.thinking = Thinking::kDisabled;
else if (!type.empty()) Rules::fail("thinking.type must be enabled or disabled");
if (v.find("token_budget") && v.find("token_budget")->is_number()) {
if (type.empty() && p.thinking == Thinking::kDefault) p.thinking = Thinking::kEnabled;
p.thinking_budget = v.find("token_budget")->as_int();
}
} else {
rest.set(k, v);
}
}
p.extra = rest;
}
return p;
}
llm::ChatResponse CohereChatModel::to_response(const ChatResult& result) {
llm::ChatResponse out;
out.id = result.id;
out.message.role = llm::Role::kAssistant;
if (!result.thinking.empty()) {
llm::ContentPart r;
r.reasoning = result.thinking;
out.message.parts.push_back(r);
}
if (!result.text.empty() || result.tool_calls.empty())
out.message.parts.push_back(llm::ContentPart::of_text(result.text));
for (const ToolCallRecord& call : result.tool_calls) {
llm::ToolCall tc;
tc.id = call.id;
tc.name = call.name;
tc.arguments = arguments_of(call);
out.message.parts.push_back(llm::ContentPart::of_call(tc));
}
out.finish_reason = neutral_finish(result.finish_reason);
out.usage = usage_of(result.usage);
out.raw = result.raw;
return out;
}
llm::ChatResponse CohereChatModel::generate(const llm::ChatRequest& request) {
ChatParams params = to_params(request);
llm::ChatResponse out = to_response(chat_->create(params));
out.model = params.model;
return out;
}
void CohereChatModel::stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
ChatParams params = to_params(request);
chat_->stream(params, [&](const StreamDelta& d) {
if (!d.text.empty() || !d.thinking.empty()) {
llm::ChatChunk c;
c.text_delta = d.text;
c.reasoning_delta = d.thinking;
if (!on_chunk(c)) return false;
}
if (d.type == "tool-call-start" || d.type == "tool-call-delta") {
llm::ChatChunk c;
c.tool_call_id = d.tool_id;
c.tool_call_name = d.tool_name;
c.tool_args_delta = d.args_delta;
if (!on_chunk(c)) return false;
}
if (d.type == "message-end") {
llm::ChatChunk c;
c.finish_reason = neutral_finish(d.finish_reason);
if (d.has_usage) c.usage = usage_of(d.usage);
if (!on_chunk(c)) return false;
}
return true;
});
}
} // namespace molo::cohere
+47
View File
@@ -0,0 +1,47 @@
#pragma once
// CohereChatModel: the llm::ChatModel seam over Chat v2.
//
// Mapping notes (each one is a decision):
// - ChatRequest.system becomes a leading system message.
// - tool_choice: "required" -> REQUIRED, "none" -> NONE, ""/"auto" -> omitted;
// a tool NAME is rejected loudly (Cohere cannot force one named tool).
// - json_mode -> response_format json_object; response_schema -> the same plus
// json_schema (Cohere's json_object accepts a schema).
// - reasoning_effort "none"/"off"/"disabled" -> thinking disabled; any other
// non-empty value -> thinking enabled (Cohere has no effort levels, only
// token_budget, which ChatRequest.extra {"thinking":{"token_budget":N}} sets).
// - Assistant reasoning parts are sent back as thinking content.
// - Tool arguments that are not valid JSON arrive as {"_invalid_json": "<raw>"}.
// - finish_reason: COMPLETE and STOP_SEQUENCE -> stop, MAX_TOKENS -> length,
// TOOL_CALL -> tool_calls, ERROR -> error, TIMEOUT -> timeout.
// - Usage: counts come from usage.tokens (actual), billed_units is the fallback;
// cached_tokens from usage.cached_tokens.
#include <memory>
#include "vendors/cohere/chat.h"
#include "llm/chat_types.h"
namespace molo::cohere {
class CohereChatModel : public llm::ChatModel {
public:
explicit CohereChatModel(std::shared_ptr<Chat> chat, std::string default_model = "command-a-03-2025");
static std::shared_ptr<CohereChatModel> from_env(const std::string& default_model = "command-a-03-2025");
std::string vendor() const override { return "cohere"; }
llm::ChatResponse generate(const llm::ChatRequest& request) override;
void stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
ChatParams to_params(const llm::ChatRequest& request) const;
static llm::ChatResponse to_response(const ChatResult& result);
static std::string neutral_finish(const std::string& finish_reason);
private:
std::shared_ptr<Chat> chat_;
std::string default_model_;
};
} // namespace molo::cohere
+205
View File
@@ -0,0 +1,205 @@
#include "vendors/cohere/chat_params.h"
#include "vendors/cohere/calls.h"
#include "vendors/cohere/caps.h"
#include "vendors/cohere/rules.h"
namespace molo::cohere {
Json Messages::text_part(const std::string& text) {
Json p = Json::object();
p.set("type", Json("text"));
p.set("text", Json(text));
return p;
}
Json Messages::image_part(const std::string& url, const std::string& detail) {
Rules::image_url(url);
if (!detail.empty()) Rules::one_of(detail, {"auto", "low", "high"}, "image detail");
Json u = Json::object();
u.set("url", Json(url));
if (!detail.empty()) u.set("detail", Json(detail));
Json p = Json::object();
p.set("type", Json("image_url"));
p.set("image_url", u);
return p;
}
static Json message(const char* role) {
Json m = Json::object();
m.set("role", Json(role));
return m;
}
Json Messages::system(const std::string& text) {
Json m = message("system");
m.set("content", Json(text));
return m;
}
Json Messages::user(const std::string& text) {
Json m = message("user");
m.set("content", Json(text));
return m;
}
Json Messages::user_parts(const std::vector<Json>& parts) {
Rules::require(!parts.empty(), "a user message needs at least one part");
Json arr = Json::array();
for (const Json& p : parts) arr.push(p);
Json m = message("user");
m.set("content", arr);
return m;
}
Json Messages::assistant(const std::string& text, const std::string& tool_plan, const Json& tool_calls,
const std::string& thinking) {
Json m = message("assistant");
Json parts = Json::array();
if (!thinking.empty()) {
Json t = Json::object();
t.set("type", Json("thinking"));
t.set("thinking", Json(thinking));
parts.push(t);
}
if (!text.empty()) parts.push(text_part(text));
if (!parts.as_array().empty()) m.set("content", parts);
if (!tool_plan.empty()) m.set("tool_plan", Json(tool_plan));
if (tool_calls.is_array() && !tool_calls.as_array().empty()) m.set("tool_calls", tool_calls);
return m;
}
Json Messages::tool(const std::string& call_id, const std::string& content) {
Rules::require(!call_id.empty(), "tool_call_id must not be empty");
Json m = message("tool");
m.set("tool_call_id", Json(call_id));
m.set("content", Json(content));
return m;
}
Json Messages::tool_call(const std::string& id, const std::string& name, const std::string& arguments_json) {
Json f = Json::object();
f.set("name", Json(name));
f.set("arguments", Json(arguments_json));
Json c = Json::object();
c.set("id", Json(id));
c.set("type", Json("function"));
c.set("function", f);
return c;
}
Json ToolDef::to_json() const {
Rules::require(!name.empty() && name.size() <= 128, "tool name must be 1..128 chars");
for (char c : name)
Rules::require((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_' || c == '-',
"tool name has invalid characters");
Json f = Json::object();
f.set("name", Json(name));
if (!description.empty()) f.set("description", Json(description));
f.set("parameters", parameters.is_object() ? parameters : Json::object());
Json t = Json::object();
t.set("type", Json("function"));
t.set("function", f);
return t;
}
ChatParams ChatParams::prompt(const std::string& text) {
ChatParams p;
p.add(Messages::user(text));
return p;
}
ChatParams& ChatParams::add(Json message) {
messages.push_back(std::move(message));
return *this;
}
void ChatParams::validate() const {
Rules::model(model);
Rules::require(!messages.empty(), "messages must not be empty");
Rules::require(messages.size() <= CohereCaps::kMaxMessages, "too many messages");
Rules::require(tools.size() <= CohereCaps::kMaxTools, "too many tools");
Rules::require(documents.size() <= CohereCaps::kMaxDocuments, "too many documents");
if (!citation_mode.empty())
Rules::one_of(citation_mode, {"ENABLED", "DISABLED", "FAST", "ACCURATE", "OFF"}, "citation mode");
if (!safety_mode.empty()) Rules::one_of(safety_mode, {"CONTEXTUAL", "STRICT", "OFF"}, "safety mode");
if (!tool_choice.empty()) {
Rules::one_of(tool_choice, {"REQUIRED", "NONE"}, "tool_choice");
Rules::require(tool_choice != "REQUIRED" || !tools.empty(), "tool_choice REQUIRED needs tools");
}
Rules::require(!strict_tools || !tools.empty(), "strict_tools needs tools");
Rules::require(!json_schema.is_object() || json_mode, "json_schema needs json_mode");
if (max_tokens) Rules::require(*max_tokens >= 1, "max_tokens must be >= 1");
Rules::stop(stop);
if (temperature) Rules::temperature(*temperature);
if (seed) Rules::require(*seed >= 0, "seed must be >= 0");
if (frequency_penalty) Rules::range(*frequency_penalty, 0.0, 1.0, "frequency_penalty");
if (presence_penalty) Rules::range(*presence_penalty, 0.0, 1.0, "presence_penalty");
if (k) Rules::top_k(*k);
if (p) Rules::top_p(*p);
if (priority) Rules::priority(*priority);
if (thinking_budget) {
Rules::require(*thinking_budget >= 1, "thinking token budget must be >= 1");
Rules::require(thinking == Thinking::kEnabled, "a thinking budget needs thinking enabled");
}
}
Json ChatParams::to_json(bool stream) const {
validate();
Json b = Json::object();
b.set("model", Json(model));
Json msgs = Json::array();
for (const Json& m : messages) msgs.push(m);
b.set("messages", msgs);
if (stream) b.set("stream", Json(true));
if (!tools.empty()) {
Json t = Json::array();
for (const ToolDef& d : tools) t.push(d.to_json());
b.set("tools", t);
}
if (strict_tools) b.set("strict_tools", Json(true));
if (!documents.empty()) {
Json d = Json::array();
for (const Json& x : documents) d.push(x);
b.set("documents", d);
}
if (!citation_mode.empty()) {
Json c = Json::object();
c.set("mode", Json(citation_mode));
b.set("citation_options", c);
}
if (json_mode) {
Json f = Json::object();
f.set("type", Json("json_object"));
if (json_schema.is_object()) f.set("json_schema", json_schema);
b.set("response_format", f);
}
if (!safety_mode.empty()) b.set("safety_mode", Json(safety_mode));
if (max_tokens) b.set("max_tokens", Json(*max_tokens));
if (!stop.empty()) {
Json s = Json::array();
for (const std::string& x : stop) s.push(Json(x));
b.set("stop_sequences", s);
}
if (temperature) b.set("temperature", Json(*temperature));
if (seed) b.set("seed", Json(*seed));
if (frequency_penalty) b.set("frequency_penalty", Json(*frequency_penalty));
if (presence_penalty) b.set("presence_penalty", Json(*presence_penalty));
if (k) b.set("k", Json(*k));
if (p) b.set("p", Json(*p));
if (logprobs) b.set("logprobs", Json(true));
if (!tool_choice.empty()) b.set("tool_choice", Json(tool_choice));
if (thinking != Thinking::kDefault) {
Json t = Json::object();
t.set("type", Json(thinking == Thinking::kEnabled ? "enabled" : "disabled"));
if (thinking_budget) t.set("token_budget", Json(*thinking_budget));
b.set("thinking", t);
}
if (priority) b.set("priority", Json(*priority));
if (extra.is_object())
for (const auto& [key, v] : extra.as_object()) b.set(key, v);
Calls::check_body(Call::kChat, b);
return b;
}
} // namespace molo::cohere
+70
View File
@@ -0,0 +1,70 @@
#pragma once
// Chat v2 request: message builders (Messages), tool definitions and
// ChatParams, which validates and serializes to the /v2/chat body.
#include <optional>
#include <string>
#include <vector>
#include "vendors/cohere/types.h"
namespace molo::cohere {
// Builders for the four v2 message roles. Content is a plain string or an
// array of typed parts (text, image_url; assistant: text, thinking).
struct Messages {
static Json system(const std::string& text);
static Json user(const std::string& text);
static Json user_parts(const std::vector<Json>& parts);
// tool_plan and tool_calls are omitted when empty.
static Json assistant(const std::string& text, const std::string& tool_plan = "",
const Json& tool_calls = Json::array(), const std::string& thinking = "");
// Result of one tool call; `content` is text (the docs also allow document parts).
static Json tool(const std::string& call_id, const std::string& content);
static Json text_part(const std::string& text);
static Json image_part(const std::string& url, const std::string& detail = "");
static Json tool_call(const std::string& id, const std::string& name, const std::string& arguments_json);
};
struct ToolDef {
std::string name;
std::string description;
Json parameters = Json::object(); // JSON Schema
Json to_json() const;
};
enum class Thinking { kDefault, kEnabled, kDisabled };
struct ChatParams {
std::string model = "command-a-03-2025";
std::vector<Json> messages;
std::vector<ToolDef> tools;
bool strict_tools = false;
std::vector<Json> documents; // each a string or {"data":{...},"id":"..."}
std::string citation_mode; // ENABLED DISABLED FAST ACCURATE OFF
bool json_mode = false;
Json json_schema; // null: none; else response_format.json_schema
std::string safety_mode; // CONTEXTUAL STRICT OFF
std::optional<int> max_tokens;
std::vector<std::string> stop;
std::optional<double> temperature;
std::optional<long> seed;
std::optional<double> frequency_penalty;
std::optional<double> presence_penalty;
std::optional<int> k;
std::optional<double> p;
bool logprobs = false;
std::string tool_choice; // REQUIRED, NONE or empty
Thinking thinking = Thinking::kDefault;
std::optional<long> thinking_budget;
std::optional<int> priority;
Json extra; // merged into the body last; keys must exist in the call table
static ChatParams prompt(const std::string& text);
ChatParams& add(Json message);
void validate() const;
Json to_json(bool stream = false) const;
};
} // namespace molo::cohere
+132
View File
@@ -0,0 +1,132 @@
#include "vendors/cohere/chat_result.h"
#include "vendors/cohere/chat_params.h"
#include "llm/error.h"
namespace molo::cohere {
bool ToolCallRecord::arguments_valid() const {
try {
return Json::parse(arguments.empty() ? "{}" : arguments).is_object();
} catch (const std::exception&) {
return false;
}
}
Json ToolCallRecord::parsed() const {
try {
Json j = Json::parse(arguments.empty() ? "{}" : arguments);
if (j.is_object()) return j;
} catch (const std::exception&) {
}
return Json::object();
}
static ToolCallRecord tool_of(const Json& c) {
ToolCallRecord t;
t.id = J::str(c, "id");
if (const Json* f = J::obj(c, "function")) {
t.name = J::str(*f, "name");
t.arguments = J::str(*f, "arguments");
}
return t;
}
ChatResult ChatResult::parse(const Json& doc) {
ChatResult r;
r.raw = doc;
r.id = J::str(doc, "id");
r.finish_reason = J::str(doc, "finish_reason");
if (const Json* m = J::obj(doc, "message")) {
if (const Json* content = J::arr(*m, "content"))
for (const Json& part : content->as_array()) {
if (J::str(part, "type") == "thinking") r.thinking += J::str(part, "thinking");
else r.text += J::str(part, "text");
}
r.tool_plan = J::str(*m, "tool_plan");
if (const Json* calls = J::arr(*m, "tool_calls"))
for (const Json& c : calls->as_array()) r.tool_calls.push_back(tool_of(c));
if (const Json* cites = J::arr(*m, "citations"))
for (const Json& c : cites->as_array()) r.citations.push_back(c);
}
r.usage = TokenUsage::parse(J::obj(doc, "usage"));
if (const Json* l = J::arr(doc, "logprobs")) r.logprobs = *l;
return r;
}
Json ChatResult::history_message() const {
Json calls = Json::array();
for (const ToolCallRecord& c : tool_calls) calls.push(Messages::tool_call(c.id, c.name, c.arguments));
return Messages::assistant(text, tool_plan, calls, thinking);
}
StreamDelta ChatStreamAccumulator::feed(const Json& ev) {
StreamDelta d;
d.type = J::str(ev, "type");
d.index = static_cast<int>(J::num(ev, "index"));
const Json* delta = J::obj(ev, "delta");
const Json* msg = delta ? J::obj(*delta, "message") : nullptr;
if (d.type == "message-start") {
result_.id = J::str(ev, "id");
} else if (d.type == "content-start" || d.type == "content-delta") {
const Json* content = msg ? J::obj(*msg, "content") : nullptr;
if (content) {
d.text = J::str(*content, "text");
d.thinking = J::str(*content, "thinking");
result_.text += d.text;
result_.thinking += d.thinking;
}
} else if (d.type == "tool-plan-delta") {
if (msg) d.tool_plan = J::str(*msg, "tool_plan");
result_.tool_plan += d.tool_plan;
} else if (d.type == "tool-call-start") {
if (const Json* c = msg ? J::obj(*msg, "tool_calls") : nullptr) {
ToolCallRecord t = tool_of(*c);
d.tool_start = true;
d.tool_id = t.id;
d.tool_name = t.name;
d.args_delta = t.arguments;
tools_[d.index] = t;
}
} else if (d.type == "tool-call-delta") {
if (const Json* c = msg ? J::obj(*msg, "tool_calls") : nullptr)
if (const Json* f = J::obj(*c, "function")) {
d.args_delta = J::str(*f, "arguments");
ToolCallRecord& t = tools_[d.index];
t.arguments += d.args_delta;
d.tool_id = t.id;
d.tool_name = t.name;
}
} else if (d.type == "citation-start") {
if (const Json* c = msg ? J::obj(*msg, "citations") : nullptr) {
d.citation = *c;
result_.citations.push_back(*c);
}
} else if (d.type == "message-end") {
if (delta) {
std::string error = J::str(*delta, "error");
if (!error.empty()) throw llm::ApiError("cohere", 502, "STREAM_ERROR", error);
d.finish_reason = J::str(*delta, "finish_reason");
result_.finish_reason = d.finish_reason;
if (const Json* u = J::obj(*delta, "usage")) {
d.usage = TokenUsage::parse(u);
d.has_usage = true;
result_.usage = d.usage;
}
}
done_ = true;
} else if (d.type.empty()) {
throw llm::ApiError("cohere", 502, "STREAM_ERROR", "stream event without a type");
}
// content-end, tool-call-end, citation-end and unknown future events carry nothing we keep.
return d;
}
ChatResult ChatStreamAccumulator::result() const {
ChatResult r = result_;
r.tool_calls.clear();
for (const auto& [i, t] : tools_) r.tool_calls.push_back(t);
return r;
}
} // namespace molo::cohere
+68
View File
@@ -0,0 +1,68 @@
#pragma once
// Chat v2 response and the streaming accumulator. Stream events are the
// `type` values of StreamedChatResponseV2 (message-start ... message-end).
#include <map>
#include <string>
#include <vector>
#include "vendors/cohere/types.h"
namespace molo::cohere {
struct ToolCallRecord {
std::string id;
std::string name;
std::string arguments; // JSON text, as sent by the model
bool arguments_valid() const;
Json parsed() const; // {} when invalid
};
struct ChatResult {
std::string id;
std::string finish_reason; // COMPLETE STOP_SEQUENCE MAX_TOKENS TOOL_CALL ERROR TIMEOUT
std::string text; // text parts joined
std::string thinking; // thinking parts joined
std::string tool_plan;
std::vector<ToolCallRecord> tool_calls;
std::vector<Json> citations;
TokenUsage usage;
Json logprobs; // array or null
Json raw;
static ChatResult parse(const Json& doc);
// The assistant message to put back into the history for the next turn.
Json history_message() const;
};
struct StreamDelta {
std::string type; // event type, e.g. "content-delta"
std::string text;
std::string thinking;
std::string tool_plan;
int index = 0;
bool tool_start = false; // tool-call-start: id and name are set
std::string tool_id;
std::string tool_name;
std::string args_delta;
Json citation;
std::string finish_reason; // message-end
bool has_usage = false;
TokenUsage usage;
};
class ChatStreamAccumulator {
public:
// Throws llm::ApiError (502 STREAM_ERROR) on an error event or a message-end error.
StreamDelta feed(const Json& event);
bool done() const { return done_; }
ChatResult result() const;
private:
ChatResult result_;
std::map<int, ToolCallRecord> tools_;
bool done_ = false;
};
} // namespace molo::cohere
+273
View File
@@ -0,0 +1,273 @@
#include "vendors/cohere/inference.h"
#include "vendors/cohere/calls.h"
#include "vendors/cohere/caps.h"
#include "vendors/cohere/rules.h"
namespace molo::cohere {
namespace {
Json string_array(const std::vector<std::string>& v) {
Json a = Json::array();
for (const std::string& s : v) a.push(Json(s));
return a;
}
} // namespace
void EmbedParams::validate() const {
Rules::model(model);
Rules::one_of(input_type, {"search_document", "search_query", "classification", "clustering", "image"}, "input_type");
Rules::require(!texts.empty() || !images.empty() || !inputs.empty(), "give texts, images or inputs");
Rules::require(texts.size() <= CohereCaps::kMaxEmbedTexts, "at most 96 texts per call");
Rules::require(images.size() <= CohereCaps::kMaxEmbedTexts, "at most 96 images per call");
Rules::require(inputs.size() <= CohereCaps::kMaxEmbedInputs, "at most 96 inputs per call");
for (const std::string& t : texts) Rules::require(!t.empty(), "texts must not be empty");
for (const std::string& i : images) Rules::image_url(i);
for (const std::string& t : embedding_types)
Rules::one_of(t, {"float", "int8", "uint8", "binary", "ubinary", "base64"}, "embedding type");
if (!truncate.empty()) Rules::one_of(truncate, {"NONE", "START", "END"}, "truncate");
if (max_tokens) Rules::require(*max_tokens >= 1, "max_tokens must be >= 1");
if (output_dimension) Rules::require(*output_dimension >= 1, "output_dimension must be >= 1");
if (priority) Rules::priority(*priority);
}
Json EmbedParams::to_json() const {
validate();
Json b = Json::object();
b.set("model", Json(model));
b.set("input_type", Json(input_type));
if (!texts.empty()) b.set("texts", string_array(texts));
if (!images.empty()) b.set("images", string_array(images));
if (!inputs.empty()) {
Json a = Json::array();
for (const Json& i : inputs) a.push(i);
b.set("inputs", a);
}
if (!embedding_types.empty()) b.set("embedding_types", string_array(embedding_types));
if (!truncate.empty()) b.set("truncate", Json(truncate));
if (max_tokens) b.set("max_tokens", Json(*max_tokens));
if (output_dimension) b.set("output_dimension", Json(*output_dimension));
if (priority) b.set("priority", Json(*priority));
return b;
}
std::vector<std::vector<double>> EmbedResult::floats() const {
std::vector<std::vector<double>> out;
const Json* rows = nullptr;
if (embeddings.is_array()) rows = &embeddings;
else if (embeddings.is_object() && embeddings.find("float") && embeddings.find("float")->is_array())
rows = embeddings.find("float");
if (!rows) return out;
for (const Json& row : rows->as_array()) {
std::vector<double> v;
if (row.is_array())
for (const Json& x : row.as_array())
if (x.is_number()) v.push_back(x.as_number());
out.push_back(std::move(v));
}
return out;
}
EmbedResult EmbedResult::parse(const Json& doc) {
EmbedResult r;
r.raw = doc;
r.id = J::str(doc, "id");
r.response_type = J::str(doc, "response_type");
r.texts = J::strings(doc, "texts");
if (const Json* e = doc.is_object() ? doc.find("embeddings") : nullptr) r.embeddings = *e;
r.usage = TokenUsage::parse(J::obj(doc, "meta"));
if (const Json* meta = J::obj(doc, "meta"))
if (const Json* b = J::obj(*meta, "billed_units")) {
r.usage.billed_input_tokens = J::num(*b, "input_tokens");
r.usage.present = true;
}
return r;
}
void RerankParams::validate() const {
Rules::model(model);
Rules::require(!query.empty(), "query must not be empty");
Rules::require(!documents.empty() && documents.size() <= CohereCaps::kMaxDocuments, "documents must be 1..1000");
for (const std::string& d : documents) Rules::require(!d.empty(), "documents must not be empty");
if (top_n) Rules::require(*top_n >= 1, "top_n must be >= 1");
if (max_tokens_per_doc) Rules::require(*max_tokens_per_doc >= 1, "max_tokens_per_doc must be >= 1");
if (!relevant_spans.empty()) Rules::one_of(relevant_spans, {"enabled", "disabled"}, "relevant_spans");
Rules::require(relevant_spans.empty() || relevant_spans == "enabled" || (!spans_threshold && !max_spans),
"span options need relevant_spans enabled");
if (spans_threshold) Rules::range(*spans_threshold, 0.0, 1.0, "span threshold");
if (max_spans) Rules::require(*max_spans >= 1, "max_spans must be >= 1");
if (priority) Rules::priority(*priority);
}
Json RerankParams::to_json() const {
validate();
Json b = Json::object();
b.set("model", Json(model));
b.set("query", Json(query));
b.set("documents", string_array(documents));
if (top_n) b.set("top_n", Json(*top_n));
if (max_tokens_per_doc) b.set("max_tokens_per_doc", Json(*max_tokens_per_doc));
if (!relevant_spans.empty()) {
Json s = Json::object();
s.set("mode", Json(relevant_spans));
if (spans_threshold) s.set("threshold", Json(*spans_threshold));
if (max_spans) s.set("max_spans", Json(*max_spans));
b.set("relevant_spans", s);
}
if (priority) b.set("priority", Json(*priority));
return b;
}
RerankResult RerankResult::parse(const Json& doc) {
RerankResult r;
r.raw = doc;
r.id = J::str(doc, "id");
if (const Json* a = J::arr(doc, "results"))
for (const Json& x : a->as_array()) {
RerankHit h;
h.index = static_cast<int>(J::num(x, "index"));
h.score = J::real(x, "relevance_score");
if (const Json* s = J::arr(x, "relevant_spans")) h.spans = *s;
r.hits.push_back(h);
}
r.usage = TokenUsage::parse(J::obj(doc, "meta"));
return r;
}
void ClassifyParams::validate() const {
Rules::require(!inputs.empty() && inputs.size() <= CohereCaps::kMaxClassifyInputs, "inputs must be 1..96");
for (const std::string& i : inputs) Rules::require(!i.empty(), "inputs must not be empty");
if (!model.empty()) Rules::model(model);
Rules::require(!model.empty() || examples.size() >= 2, "without a fine-tuned model, give at least 2 examples");
for (const ClassifyExample& e : examples)
Rules::require(!e.text.empty() && !e.label.empty(), "examples need text and label");
if (!truncate.empty()) Rules::one_of(truncate, {"NONE", "START", "END"}, "truncate");
}
Json ClassifyParams::to_json() const {
validate();
Json b = Json::object();
b.set("inputs", string_array(inputs));
if (!examples.empty()) {
Json a = Json::array();
for (const ClassifyExample& e : examples) {
Json x = Json::object();
x.set("text", Json(e.text));
x.set("label", Json(e.label));
a.push(x);
}
b.set("examples", a);
}
if (!model.empty()) b.set("model", Json(model));
if (!truncate.empty()) b.set("truncate", Json(truncate));
return b;
}
ClassifyResult ClassifyResult::parse(const Json& doc) {
ClassifyResult r;
r.raw = doc;
r.id = J::str(doc, "id");
if (const Json* a = J::arr(doc, "classifications"))
for (const Json& x : a->as_array()) {
Classification c;
c.id = J::str(x, "id");
c.input = J::str(x, "input");
c.prediction = J::str(x, "prediction");
if (c.prediction.empty()) {
std::vector<std::string> p = J::strings(x, "predictions");
if (!p.empty()) c.prediction = p.front();
}
c.confidence = J::real(x, "confidence");
if (c.confidence == 0)
if (const Json* cs = J::arr(x, "confidences"))
if (!cs->as_array().empty() && cs->as_array().front().is_number()) c.confidence = cs->as_array().front().as_number();
c.classification_type = J::str(x, "classification_type");
if (const Json* l = J::obj(x, "labels")) c.labels = *l;
r.items.push_back(c);
}
return r;
}
TokenizeResult TokenizeResult::parse(const Json& doc) {
TokenizeResult r;
if (const Json* a = J::arr(doc, "tokens"))
for (const Json& t : a->as_array())
if (t.is_number()) r.tokens.push_back(t.as_int());
r.token_strings = J::strings(doc, "token_strings");
return r;
}
EmbedResult Embeddings::embed(const EmbedParams& params) {
Json body = params.to_json();
Calls::check_body(Call::kEmbed, body);
return EmbedResult::parse(client_->post(Calls::render(Call::kEmbed), body));
}
EmbedResult Embeddings::embed_v1(const EmbedParams& params) {
Rules::require(params.inputs.empty() && !params.output_dimension && !params.max_tokens && !params.priority,
"v1 embed takes texts and images only");
Json body = params.to_json();
Calls::check_body(Call::kEmbedV1, body);
return EmbedResult::parse(client_->post(Calls::render(Call::kEmbedV1), body));
}
RerankResult Rerank::rerank(const RerankParams& params) {
Json body = params.to_json();
Calls::check_body(Call::kRerank, body);
return RerankResult::parse(client_->post(Calls::render(Call::kRerank), body));
}
Json Rerank::rerank_v1(const Json& body) {
Calls::check_body(Call::kRerankV1, body);
Rules::require(body.find("query") && body.find("query")->is_string(), "query is required");
const Json* docs = body.find("documents");
Rules::require(docs && docs->is_array() && !docs->as_array().empty() &&
docs->as_array().size() <= CohereCaps::kMaxDocuments,
"documents must be 1..1000");
return client_->post(Calls::render(Call::kRerankV1), body);
}
ClassifyResult Classify::classify(const ClassifyParams& params) {
Json body = params.to_json();
Calls::check_body(Call::kClassify, body);
return ClassifyResult::parse(client_->post(Calls::render(Call::kClassify), body));
}
TokenizeResult Tokenizer::tokenize(const std::string& model, const std::string& text) {
Rules::model(model);
Rules::require(!text.empty() && text.size() <= CohereCaps::kMaxTokenizeChars, "text must be 1..65536 chars");
Json b = Json::object();
b.set("model", Json(model));
b.set("text", Json(text));
return TokenizeResult::parse(client_->post(Calls::render(Call::kTokenize), b));
}
std::string Tokenizer::detokenize(const std::string& model, const std::vector<long>& tokens) {
Rules::model(model);
Rules::require(!tokens.empty() && tokens.size() <= CohereCaps::kMaxDetokenizeTokens, "tokens must be 1..65536");
Json arr = Json::array();
for (long t : tokens) {
Rules::require(t >= 0, "token ids must be >= 0");
arr.push(Json(t));
}
Json b = Json::object();
b.set("model", Json(model));
b.set("tokens", arr);
return J::str(client_->post(Calls::render(Call::kDetokenize), b), "text");
}
Json Legacy::generate(const Json& body) {
Calls::check_body(Call::kGenerate, body);
Rules::require(body.find("prompt") && body.find("prompt")->is_string(), "prompt is required");
const Json* s = body.find("stream");
Rules::require(!s || (s->is_bool() && !s->as_bool()), "generate streaming is not supported here");
return client_->post(Calls::render(Call::kGenerate), body);
}
Json Legacy::summarize(const Json& body) {
Calls::check_body(Call::kSummarize, body);
Rules::require(body.find("text") && body.find("text")->is_string(), "text is required");
return client_->post(Calls::render(Call::kSummarize), body);
}
} // namespace molo::cohere
+139
View File
@@ -0,0 +1,139 @@
#pragma once
// Stateless model calls: Embed (v2, v1), Rerank (v2, v1), Classify, Tokenize /
// Detokenize, and the legacy Generate and Summarize. Parameters validate
// before any network traffic; bodies are checked against the call table.
#include <optional>
#include <string>
#include <vector>
#include "vendors/cohere/types.h"
#include "llm/api_client.h"
namespace molo::cohere {
struct EmbedParams {
std::string model = "embed-v4.0";
std::string input_type = "search_document"; // search_document search_query classification clustering image
std::vector<std::string> texts; // <= 96
std::vector<std::string> images; // data URLs
std::vector<Json> inputs; // v2 mixed inputs: {"content":[{type,text|image_url}]}
std::vector<std::string> embedding_types; // float int8 uint8 binary ubinary base64
std::string truncate; // NONE START END
std::optional<int> max_tokens;
std::optional<int> output_dimension;
std::optional<int> priority;
void validate() const;
Json to_json() const;
};
struct EmbedResult {
std::string id;
std::string response_type;
std::vector<std::string> texts;
Json embeddings; // {"float":[[...]],...} (v2) or [[...]] (v1 floats)
TokenUsage usage;
Json raw;
// Float vectors (v2 "float", or the v1 array). Empty when not requested.
std::vector<std::vector<double>> floats() const;
static EmbedResult parse(const Json& doc);
};
struct RerankParams {
std::string model = "rerank-v3.5";
std::string query;
std::vector<std::string> documents; // 1..1000
std::optional<int> top_n;
std::optional<int> max_tokens_per_doc;
std::string relevant_spans; // "", enabled, disabled
std::optional<double> spans_threshold;
std::optional<int> max_spans;
std::optional<int> priority;
void validate() const;
Json to_json() const;
};
struct RerankHit {
int index = 0;
double score = 0;
Json spans; // array or null
};
struct RerankResult {
std::string id;
std::vector<RerankHit> hits; // best first, as sent
TokenUsage usage;
Json raw;
static RerankResult parse(const Json& doc);
};
struct ClassifyExample {
std::string text;
std::string label;
};
struct ClassifyParams {
std::vector<std::string> inputs; // 1..96
std::vector<ClassifyExample> examples;
std::string model; // empty: the default classifier
std::string truncate;
void validate() const;
Json to_json() const;
};
struct Classification {
std::string id, input, prediction, classification_type;
double confidence = 0;
Json labels;
};
struct ClassifyResult {
std::string id;
std::vector<Classification> items;
Json raw;
static ClassifyResult parse(const Json& doc);
};
struct TokenizeResult {
std::vector<long> tokens;
std::vector<std::string> token_strings;
static TokenizeResult parse(const Json& doc);
};
class Embeddings : public llm::Service {
public:
explicit Embeddings(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
EmbedResult embed(const EmbedParams& params); // POST /v2/embed
EmbedResult embed_v1(const EmbedParams& params); // POST /v1/embed (texts/images only)
};
class Rerank : public llm::Service {
public:
explicit Rerank(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
RerankResult rerank(const RerankParams& params); // POST /v2/rerank
Json rerank_v1(const Json& body); // POST /v1/rerank
};
class Classify : public llm::Service {
public:
explicit Classify(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
ClassifyResult classify(const ClassifyParams& params); // POST /v1/classify (deprecated in the spec)
};
class Tokenizer : public llm::Service {
public:
explicit Tokenizer(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
TokenizeResult tokenize(const std::string& model, const std::string& text);
std::string detokenize(const std::string& model, const std::vector<long>& tokens);
};
// Deprecated endpoints that remain in the spec; bodies are Json checked against the table.
class Legacy : public llm::Service {
public:
explicit Legacy(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
Json generate(const Json& body); // POST /v1/generate (prompt required, stream:true rejected)
Json summarize(const Json& body); // POST /v1/summarize (text required)
};
} // namespace molo::cohere
+33
View File
@@ -0,0 +1,33 @@
#include "vendors/cohere/platform.h"
#include "clients/transports/http_transport.h"
#include "llm/config.h"
namespace molo::cohere {
llm::ApiClientConfig Platform::config() {
llm::ApiClientConfig c;
c.vendor = "cohere";
c.base_url = kHost;
c.key_env = kKeyEnv;
c.auth = llm::AuthStyle::kBearer;
c.default_headers["X-Client-Name"] = kClientName;
return c;
}
Platform Platform::from_env() {
std::string key = llm::Config::get(kKeyEnv);
if (key.empty()) key = llm::Config::get(kKeyEnvAlt);
Platform p;
p.client = std::make_shared<llm::ApiClient>(std::make_unique<clients::transports::HttpTransport>(),
config(), key);
return p;
}
Platform Platform::with(std::unique_ptr<clients::Transport> transport, const std::string& key) {
Platform p;
p.client = std::make_shared<llm::ApiClient>(std::move(transport), config(), key);
return p;
}
} // namespace molo::cohere
+26
View File
@@ -0,0 +1,26 @@
#pragma once
// api.cohere.com, Bearer auth. Credential COHERE_API_KEY (CO_API_KEY, the
// official SDKs' name, is accepted as a fallback), read through llm::Config.
#include <memory>
#include "llm/api_client.h"
namespace molo::cohere {
struct Platform {
static constexpr const char* kHost = "https://api.cohere.com";
static constexpr const char* kKeyEnv = "COHERE_API_KEY";
static constexpr const char* kKeyEnvAlt = "CO_API_KEY";
static constexpr const char* kClientName = "molo";
std::shared_ptr<llm::ApiClient> client;
static llm::ApiClientConfig config();
static Platform from_env();
static Platform with(std::unique_ptr<clients::Transport> transport, const std::string& key);
bool configured() const { return client && client->configured(); }
};
} // namespace molo::cohere
+445
View File
@@ -0,0 +1,445 @@
#include "vendors/cohere/resources.h"
#include <functional>
#include "clients/body/form.h"
#include "vendors/cohere/calls.h"
#include "vendors/cohere/caps.h"
#include "vendors/cohere/rules.h"
namespace molo::cohere {
namespace {
using clients::Method;
void add_page(Query& q, std::optional<int> page_size, const std::string& token) {
if (page_size) {
Rules::page_size(*page_size);
q.emplace_back("page_size", std::to_string(*page_size));
}
if (!token.empty()) {
Rules::require(token.size() <= 2048, "page_token too long");
q.emplace_back("page_token", token);
}
}
Json strings(const std::vector<std::string>& v) {
Json a = Json::array();
for (const std::string& s : v) a.push(Json(s));
return a;
}
const Json* field(const Json& doc, const char* key) { return doc.is_object() ? doc.find(key) : nullptr; }
Json require_object(const Json& j, const char* what) {
Rules::require(j.is_object(), std::string(what) + " must be a JSON object");
return j;
}
} // namespace
// ---- Account ---------------------------------------------------------------
ApiKeyCheck Account::check_api_key() {
return ApiKeyCheck::parse(client_->request(Method::kPost, Calls::render(Call::kCheckApiKey)));
}
ModelList Account::models(const std::string& endpoint, bool default_only, std::optional<int> page_size,
const std::string& page_token) {
Query q;
if (!endpoint.empty()) {
Rules::one_of(endpoint, {"chat", "embed", "classify", "summarize", "rerank", "rate", "generate"}, "endpoint");
q.emplace_back("endpoint", endpoint);
}
if (default_only) {
Rules::require(!endpoint.empty(), "default_only needs an endpoint");
q.emplace_back("default_only", "true");
}
add_page(q, page_size, page_token);
return ModelList::parse(client_->get(Calls::render(Call::kModelList, "", q)));
}
ModelList Account::all_models(const std::string& endpoint) {
ModelList all;
std::string token;
for (std::size_t page = 0; page < CohereCaps::kMaxListPages; ++page) {
ModelList l = models(endpoint, false, CohereCaps::kMaxPageSize, token);
for (ModelInfo& m : l.models) all.models.push_back(std::move(m));
token = l.page.next_page_token;
if (token.empty() || all.models.size() >= CohereCaps::kMaxItemsReturned) break;
}
all.page.next_page_token = token;
all.page.truncated = !token.empty();
return all;
}
ModelInfo Account::model(const std::string& name) {
Rules::model(name);
return ModelInfo::parse(client_->get(Calls::render(Call::kModelGet, name)));
}
// ---- Datasets --------------------------------------------------------------
const std::vector<std::string>& DatasetUpload::types() {
static const std::vector<std::string> t = {
"embed-input", "embed-result", "cluster-result", "cluster-outliers", "reranker-finetune-input",
"single-label-classification-finetune-input", "chat-finetune-input",
"multi-label-classification-finetune-input", "batch-chat-input", "batch-openai-chat-input",
"batch-embed-v2-input", "batch-chat-v2-input"};
return t;
}
static void file_name(const std::string& name) {
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxFilenameBytes, "filename must be 1..512 bytes");
for (char c : name)
Rules::require(c != '"' && c != '\r' && c != '\n' && c != '\\' && c != '/', "filename has invalid characters");
}
void DatasetUpload::validate() const {
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxNameChars, "dataset name must be 1..256 chars");
Rules::one_of(type, types(), "dataset type");
file_name(filename);
Rules::require(!data.empty(), "data is empty");
Rules::require(data.size() + eval_data.size() <= CohereCaps::kMaxUploadBytes, "upload over 256 MiB");
Rules::require(eval_data.empty() == eval_filename.empty(), "eval_data and eval_filename go together");
if (!eval_filename.empty()) file_name(eval_filename);
Rules::require(text_separator.size() <= 16 && csv_delimiter.size() <= 4, "separator too long");
}
void DatasetFilter::validate() const {
if (!dataset_type.empty()) Rules::one_of(dataset_type, DatasetUpload::types(), "dataset type");
if (!validation_status.empty())
Rules::one_of(validation_status, {"unknown", "queued", "processing", "failed", "validated", "skipped"},
"validation status");
if (limit) Rules::require(*limit >= 1 && *limit <= CohereCaps::kDatasetListMaxLimit, "limit must be 1..1000");
if (offset) Rules::require(*offset >= 0, "offset must be >= 0");
}
std::string Datasets::create(const DatasetUpload& u) {
u.validate();
Query q;
q.emplace_back("name", u.name);
q.emplace_back("type", u.type);
if (u.keep_original_file) q.emplace_back("keep_original_file", "true");
if (u.skip_malformed_input) q.emplace_back("skip_malformed_input", "true");
for (const std::string& f : u.keep_fields) q.emplace_back("keep_fields", f);
for (const std::string& f : u.optional_fields) q.emplace_back("optional_fields", f);
if (!u.text_separator.empty()) q.emplace_back("text_separator", u.text_separator);
if (!u.csv_delimiter.empty()) q.emplace_back("csv_delimiter", u.csv_delimiter);
clients::body::MultipartBody form;
form.add_file("data", u.filename, u.content_type, u.data);
if (!u.eval_data.empty()) form.add_file("eval_data", u.eval_filename, u.content_type, u.eval_data);
return J::str(client_->post_multipart(Calls::render(Call::kDatasetCreate, "", q), form), "id");
}
std::vector<Dataset> Datasets::list(const DatasetFilter& f) {
f.validate();
Query q;
if (!f.dataset_type.empty()) q.emplace_back("datasetType", f.dataset_type);
if (!f.validation_status.empty()) q.emplace_back("validationStatus", f.validation_status);
if (f.limit) q.emplace_back("limit", std::to_string(*f.limit));
if (f.offset) q.emplace_back("offset", std::to_string(*f.offset));
if (!f.before.empty()) q.emplace_back("before", f.before);
if (!f.after.empty()) q.emplace_back("after", f.after);
Json doc = client_->get(Calls::render(Call::kDatasetList, "", q));
std::vector<Dataset> out;
if (const Json* a = J::arr(doc, "datasets"))
for (const Json& d : a->as_array()) {
if (out.size() >= CohereCaps::kMaxItemsReturned) break;
out.push_back(Dataset::parse(d));
}
return out;
}
Dataset Datasets::get(const std::string& id) {
Json doc = client_->get(Calls::render(Call::kDatasetGet, id));
const Json* d = field(doc, "dataset");
return Dataset::parse(d ? *d : Json::object());
}
Json Datasets::usage() {
Json doc = client_->get(Calls::render(Call::kDatasetUsage));
const Json* u = field(doc, "organization_usage");
return u ? *u : Json();
}
void Datasets::remove(const std::string& id) { client_->del(Calls::render(Call::kDatasetDelete, id)); }
Dataset Datasets::await_validation(const std::string& id, int max_polls, const std::function<void()>& wait) {
Rules::require(max_polls >= 1 && max_polls <= 60, "max_polls must be 1..60");
Dataset d;
for (int i = 0; i < max_polls; ++i) {
d = get(id);
if (d.validation_status == "validated" || d.validation_status == "failed" || d.validation_status == "skipped")
return d;
if (wait) wait();
}
return d;
}
// ---- Batches ---------------------------------------------------------------
BatchList Batches::list(std::optional<int> page_size, const std::string& page_token, const std::string& order_by) {
Query q;
add_page(q, page_size, page_token);
if (!order_by.empty()) {
Rules::one_of(order_by, {"created_at", "updated_at"}, "order_by");
q.emplace_back("order_by", order_by);
}
return BatchList::parse(client_->get(Calls::render(Call::kBatchList, "", q)));
}
Batch Batches::create(const std::string& name, const std::string& input_dataset_id, const std::string& model) {
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxNameChars, "batch name must be 1..256 chars");
Rules::id("dataset", input_dataset_id);
Rules::model(model);
Json b = Json::object();
b.set("name", Json(name));
b.set("input_dataset_id", Json(input_dataset_id));
b.set("model", Json(model));
Calls::check_body(Call::kBatchCreate, b);
Json doc = client_->post(Calls::render(Call::kBatchCreate), b);
const Json* x = field(doc, "batch");
return Batch::parse(x ? *x : Json::object());
}
Batch Batches::get(const std::string& id) {
Json doc = client_->get(Calls::render(Call::kBatchGet, id));
const Json* x = field(doc, "batch");
return Batch::parse(x ? *x : Json::object());
}
void Batches::cancel(const std::string& id) {
client_->request(Method::kPost, Calls::render(Call::kBatchCancel, id));
}
// ---- Embed jobs ------------------------------------------------------------
Json EmbedJobParams::to_json() const {
Rules::model(model);
Rules::id("dataset", dataset_id);
Rules::one_of(input_type, {"search_document", "search_query", "classification", "clustering", "image"}, "input_type");
for (const std::string& t : embedding_types)
Rules::one_of(t, {"float", "int8", "uint8", "binary", "ubinary", "base64"}, "embedding type");
if (!truncate.empty()) Rules::one_of(truncate, {"START", "END"}, "truncate");
Rules::require(name.size() <= CohereCaps::kMaxNameChars, "name too long");
Json b = Json::object();
b.set("model", Json(model));
b.set("dataset_id", Json(dataset_id));
b.set("input_type", Json(input_type));
if (!name.empty()) b.set("name", Json(name));
if (!embedding_types.empty()) b.set("embedding_types", strings(embedding_types));
if (!truncate.empty()) b.set("truncate", Json(truncate));
Calls::check_body(Call::kEmbedJobCreate, b);
return b;
}
std::string EmbedJobs::create(const EmbedJobParams& p) {
return J::str(client_->post(Calls::render(Call::kEmbedJobCreate), p.to_json()), "job_id");
}
std::vector<EmbedJob> EmbedJobs::list() {
Json doc = client_->get(Calls::render(Call::kEmbedJobList));
std::vector<EmbedJob> out;
if (const Json* a = J::arr(doc, "embed_jobs"))
for (const Json& j : a->as_array()) {
if (out.size() >= CohereCaps::kMaxItemsReturned) break;
out.push_back(EmbedJob::parse(j));
}
return out;
}
EmbedJob EmbedJobs::get(const std::string& id) { return EmbedJob::parse(client_->get(Calls::render(Call::kEmbedJobGet, id))); }
void EmbedJobs::cancel(const std::string& id) {
client_->request(Method::kPost, Calls::render(Call::kEmbedJobCancel, id));
}
// ---- Connectors ------------------------------------------------------------
Json ConnectorParams::to_json(bool create) const {
Json b = Json::object();
if (create) Rules::require(!name.empty() && !url.empty(), "a connector needs name and url");
if (!name.empty()) {
Rules::require(name.size() <= CohereCaps::kMaxNameChars, "name too long");
b.set("name", Json(name));
}
if (!url.empty()) {
Rules::require(Rules::is_http_url(url), "connector url must be http(s)");
b.set("url", Json(url));
}
if (create && !description.empty()) b.set("description", Json(description));
if (!excludes.empty()) b.set("excludes", strings(excludes));
if (active) b.set("active", Json(*active));
if (continue_on_failure) b.set("continue_on_failure", Json(*continue_on_failure));
if (!oauth.is_null()) {
Json o = require_object(oauth, "oauth");
Rules::require(o.find("authorize_url") && o.find("token_url"), "oauth needs authorize_url and token_url");
b.set("oauth", o);
}
if (!service_auth.is_null()) {
Json s = require_object(service_auth, "service_auth");
Rules::require(s.find("type") && s.find("token"), "service_auth needs type and token");
b.set("service_auth", s);
}
Rules::require(!b.as_object().empty(), "nothing to send");
return b;
}
std::vector<Connector> Connectors::list(std::optional<int> limit, std::optional<int> offset) {
Query q;
if (limit) {
Rules::require(*limit >= 1 && *limit <= CohereCaps::kConnectorListMaxLimit, "limit must be 1..1000");
q.emplace_back("limit", std::to_string(*limit));
}
if (offset) {
Rules::require(*offset >= 0, "offset must be >= 0");
q.emplace_back("offset", std::to_string(*offset));
}
Json doc = client_->get(Calls::render(Call::kConnectorList, "", q));
std::vector<Connector> out;
if (const Json* a = J::arr(doc, "connectors"))
for (const Json& c : a->as_array()) {
if (out.size() >= CohereCaps::kMaxItemsReturned) break;
out.push_back(Connector::parse(c));
}
return out;
}
static Connector connector_of(const Json& doc) {
const Json* c = field(doc, "connector");
return Connector::parse(c ? *c : Json::object());
}
Connector Connectors::create(const ConnectorParams& p) {
Json b = p.to_json(true);
Calls::check_body(Call::kConnectorCreate, b);
return connector_of(client_->post(Calls::render(Call::kConnectorCreate), b));
}
Connector Connectors::get(const std::string& id) { return connector_of(client_->get(Calls::render(Call::kConnectorGet, id))); }
Connector Connectors::update(const std::string& id, const ConnectorParams& p) {
Json b = p.to_json(false);
Calls::check_body(Call::kConnectorUpdate, b);
return connector_of(client_->patch(Calls::render(Call::kConnectorUpdate, id), b));
}
void Connectors::remove(const std::string& id) { client_->del(Calls::render(Call::kConnectorDelete, id)); }
std::string Connectors::oauth_authorize(const std::string& id, const std::string& after_token_redirect) {
Query q;
if (!after_token_redirect.empty()) q.emplace_back("after_token_redirect", after_token_redirect);
return J::str(client_->request(Method::kPost, Calls::render(Call::kConnectorAuthorize, id, q)), "redirect_url");
}
// ---- Fine-tuning -----------------------------------------------------------
FineTuning::ModelPage FineTuning::list(std::optional<int> page_size, const std::string& page_token,
const std::string& order_by) {
Query q;
add_page(q, page_size, page_token);
if (!order_by.empty()) {
Rules::one_of(order_by, {"created_at", "updated_at"}, "order_by");
q.emplace_back("order_by", order_by);
}
Json doc = client_->get(Calls::render(Call::kFinetuneList, "", q));
ModelPage out;
if (const Json* a = J::arr(doc, "finetuned_models"))
for (const Json& m : a->as_array()) out.models.push_back(FinetunedModel::parse(m));
out.page.next_page_token = J::str(doc, "next_page_token");
out.page.total = J::num(doc, "total_size", -1);
return out;
}
static FinetunedModel finetuned_of(const Json& doc) {
const Json* m = field(doc, "finetuned_model");
return FinetunedModel::parse(m ? *m : Json::object());
}
static void check_settings(const Json& settings) {
Rules::require(settings.is_object(), "settings must be a JSON object");
const Json* base = settings.find("base_model");
Rules::require(base && base->is_object() && base->find("base_type"), "settings.base_model.base_type is required");
const Json* ds = settings.find("dataset_id");
Rules::require(ds && ds->is_string() && llm::Service::valid_id(ds->as_string()), "settings.dataset_id is required");
}
FinetunedModel FineTuning::create(const std::string& name, const Json& settings) {
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxNameChars, "name must be 1..256 chars");
check_settings(settings);
Json b = Json::object();
b.set("name", Json(name));
b.set("settings", settings);
Calls::check_body(Call::kFinetuneCreate, b);
return finetuned_of(client_->post(Calls::render(Call::kFinetuneCreate), b));
}
FinetunedModel FineTuning::get(const std::string& id) {
return finetuned_of(client_->get(Calls::render(Call::kFinetuneGet, id)));
}
FinetunedModel FineTuning::update(const std::string& id, const std::string& name, const Json& settings) {
Rules::require(!name.empty() || !settings.is_null(), "give a name or settings");
Json b = Json::object();
if (!name.empty()) b.set("name", Json(name));
if (!settings.is_null()) {
check_settings(settings);
b.set("settings", settings);
}
Calls::check_body(Call::kFinetuneUpdate, b);
return finetuned_of(client_->patch(Calls::render(Call::kFinetuneUpdate, id), b));
}
void FineTuning::remove(const std::string& id) { client_->del(Calls::render(Call::kFinetuneDelete, id)); }
Json FineTuning::events(const std::string& id, std::optional<int> page_size, const std::string& page_token) {
Query q;
add_page(q, page_size, page_token);
return client_->get(Calls::render(Call::kFinetuneEvents, id, q));
}
Json FineTuning::step_metrics(const std::string& id, std::optional<int> page_size, const std::string& page_token) {
Query q;
add_page(q, page_size, page_token);
return client_->get(Calls::render(Call::kFinetuneMetrics, id, q));
}
// ---- Parse and transcription ------------------------------------------------
Json Documents::parse(const std::string& model, const std::string& image_url, const std::string& output_format) {
Rules::model(model);
Rules::image_url(image_url);
Json doc = Json::object();
doc.set("type", Json("image_url"));
doc.set("image_url", Json(image_url));
Json b = Json::object();
b.set("model", Json(model));
b.set("document", doc);
if (!output_format.empty()) {
Rules::one_of(output_format, {"markdown", "blocks"}, "output_format");
b.set("output_format", Json(output_format));
}
Calls::check_body(Call::kParse, b);
return client_->post(Calls::render(Call::kParse), b);
}
std::string Documents::transcribe(const std::string& model, const std::string& language, const std::string& filename,
const std::string& content_type, const std::string& audio,
std::optional<double> temperature) {
Rules::model(model);
Rules::require(!language.empty() && language.size() <= 16, "language is required (for example en)");
file_name(filename);
Rules::require(!audio.empty(), "audio is empty");
Rules::require(audio.size() <= CohereCaps::kMaxUploadBytes, "audio over 256 MiB");
clients::body::MultipartBody form;
form.add_text("model", model);
form.add_text("language", language);
if (temperature) {
Rules::range(*temperature, 0.0, 2.0, "temperature");
form.add_text("temperature", std::to_string(*temperature));
}
form.add_file("file", filename, content_type.empty() ? "application/octet-stream" : content_type, audio);
return J::str(client_->post_multipart(Calls::render(Call::kTranscribe), form), "text");
}
} // namespace molo::cohere
+150
View File
@@ -0,0 +1,150 @@
#pragma once
// Account-level resources: models and key check, datasets, batches, embed
// jobs, connectors (deprecated in the spec), fine-tuned models (deprecated in
// the spec), plus Parse and audio Transcribe. Destructive and cancelling
// calls are explicit methods with no defaults. List walks are capped and
// report truncation.
#include <optional>
#include <string>
#include <vector>
#include "vendors/cohere/types.h"
#include "llm/api_client.h"
namespace molo::cohere {
class Account : public llm::Service {
public:
explicit Account(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
ApiKeyCheck check_api_key(); // POST /v1/check-api-key
ModelList models(const std::string& endpoint = "", bool default_only = false,
std::optional<int> page_size = std::nullopt,
const std::string& page_token = ""); // GET /v1/models, one page
ModelList all_models(const std::string& endpoint = ""); // walks pages (cap 20)
ModelInfo model(const std::string& name); // GET /v1/models/{model}
};
struct DatasetUpload {
std::string name;
std::string type; // DatasetType, e.g. embed-input, chat-finetune-input, batch-chat-v2-input
std::string filename;
std::string content_type = "application/octet-stream";
std::string data;
std::string eval_filename;
std::string eval_data;
bool keep_original_file = false;
bool skip_malformed_input = false;
std::vector<std::string> keep_fields;
std::vector<std::string> optional_fields;
std::string text_separator;
std::string csv_delimiter;
static const std::vector<std::string>& types();
void validate() const;
};
struct DatasetFilter {
std::string dataset_type;
std::string validation_status; // unknown queued processing failed validated skipped
std::optional<int> limit; // 1..1000
std::optional<int> offset;
std::string before, after; // ISO timestamps
void validate() const;
};
class Datasets : public llm::Service {
public:
explicit Datasets(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
std::string create(const DatasetUpload& upload); // POST /v1/datasets (multipart); returns the id
std::vector<Dataset> list(const DatasetFilter& filter = {});
Dataset get(const std::string& id);
Json usage(); // organization_usage (bytes as sent)
void remove(const std::string& id); // DELETE, explicit
// Polls get() until validated/failed/skipped, at most `max_polls` (<= 60) times; the caller
// supplies the wait through `wait` (no sleeping inside the library). Returns the last state.
Dataset await_validation(const std::string& id, int max_polls, const std::function<void()>& wait);
};
class Batches : public llm::Service {
public:
explicit Batches(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
BatchList list(std::optional<int> page_size = std::nullopt, const std::string& page_token = "",
const std::string& order_by = "");
Batch create(const std::string& name, const std::string& input_dataset_id, const std::string& model);
Batch get(const std::string& id);
void cancel(const std::string& id); // POST /v2/batches/{id}:cancel, explicit
};
struct EmbedJobParams {
std::string model = "embed-v4.0";
std::string dataset_id;
std::string input_type = "search_document";
std::string name;
std::vector<std::string> embedding_types;
std::string truncate; // START or END
Json to_json() const;
};
class EmbedJobs : public llm::Service {
public:
explicit EmbedJobs(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
std::string create(const EmbedJobParams& params); // returns job_id
std::vector<EmbedJob> list();
EmbedJob get(const std::string& id);
void cancel(const std::string& id); // explicit
};
struct ConnectorParams {
std::string name;
std::string url;
std::string description;
std::vector<std::string> excludes;
std::optional<bool> active;
std::optional<bool> continue_on_failure;
Json oauth; // {client_id, client_secret, authorize_url, token_url, scope}
Json service_auth; // {type, token}
Json to_json(bool create) const;
};
class Connectors : public llm::Service {
public:
explicit Connectors(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
std::vector<Connector> list(std::optional<int> limit = std::nullopt, std::optional<int> offset = std::nullopt);
Connector create(const ConnectorParams& params);
Connector get(const std::string& id);
Connector update(const std::string& id, const ConnectorParams& params);
void remove(const std::string& id); // explicit
std::string oauth_authorize(const std::string& id, const std::string& after_token_redirect = "");
};
class FineTuning : public llm::Service {
public:
explicit FineTuning(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
struct ModelPage {
std::vector<FinetunedModel> models;
Page page;
};
ModelPage list(std::optional<int> page_size = std::nullopt, const std::string& page_token = "",
const std::string& order_by = "");
// `settings` = {base_model:{base_type,...}, dataset_id, hyperparameters?, wandb?}. Costs money.
FinetunedModel create(const std::string& name, const Json& settings);
FinetunedModel get(const std::string& id);
FinetunedModel update(const std::string& id, const std::string& name, const Json& settings);
void remove(const std::string& id); // explicit
Json events(const std::string& id, std::optional<int> page_size = std::nullopt, const std::string& page_token = "");
Json step_metrics(const std::string& id, std::optional<int> page_size = std::nullopt, const std::string& page_token = "");
};
class Documents : public llm::Service {
public:
explicit Documents(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
// POST /v2/parse: document is an image URL (http(s) or data URL); output_format markdown|blocks.
Json parse(const std::string& model, const std::string& image_url, const std::string& output_format = "");
// POST /v2/audio/transcriptions (multipart); returns the text.
std::string transcribe(const std::string& model, const std::string& language, const std::string& filename,
const std::string& content_type, const std::string& audio,
std::optional<double> temperature = std::nullopt);
};
} // namespace molo::cohere
+64
View File
@@ -0,0 +1,64 @@
#include "vendors/cohere/rules.h"
#include "vendors/cohere/caps.h"
#include "llm/api_client.h"
namespace molo::cohere {
void Rules::fail(const std::string& why) { throw llm::ApiError::invalid("cohere", why); }
void Rules::model(const std::string& name) {
require(!name.empty() && name.size() <= CohereCaps::kMaxModelChars, "model must be 1..128 chars");
for (char c : name)
require((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
c == '_' || c == '.' || c == ':',
"model has invalid characters");
}
void Rules::id(const std::string& what, const std::string& id) {
require(llm::Service::valid_id(id), "bad " + what + " id");
}
void Rules::range(double v, double lo, double hi, const char* what) {
require(v >= lo && v <= hi, std::string(what) + " must be " + std::to_string(lo) + ".." + std::to_string(hi));
}
void Rules::temperature(double v) { range(v, 0.0, 1.0, "temperature"); }
void Rules::top_p(double v) { range(v, 0.01, 0.99, "p"); }
void Rules::top_k(long v) { require(v >= 0 && v <= CohereCaps::kMaxTopK, "k must be 0..500"); }
void Rules::priority(long v) { require(v >= 0 && v <= CohereCaps::kMaxPriority, "priority must be 0..999"); }
void Rules::stop(const std::vector<std::string>& stop) {
require(stop.size() <= CohereCaps::kMaxStopSequences, "at most 5 stop sequences");
for (const std::string& s : stop) require(!s.empty(), "stop sequences must not be empty");
}
void Rules::page_size(long v) { require(v >= 1 && v <= CohereCaps::kMaxPageSize, "page_size must be 1..1000"); }
void Rules::one_of(const std::string& value, const std::vector<std::string>& allowed, const char* what) {
for (const std::string& a : allowed)
if (a == value) return;
std::string list;
for (const std::string& a : allowed) list += (list.empty() ? "" : ", ") + a;
fail(std::string(what) + " must be one of: " + list);
}
bool Rules::is_http_url(const std::string& url) {
if (url.size() > 8192) return false;
bool ok = url.rfind("http://", 0) == 0 || url.rfind("https://", 0) == 0;
if (!ok) return false;
for (char c : url)
if (c == ' ' || c == '\r' || c == '\n') return false;
return url.find('/', 8) != std::string::npos || url.size() > 8;
}
void Rules::image_url(const std::string& url) {
if (url.rfind("data:image/", 0) == 0) {
require(url.find(";base64,") != std::string::npos, "data URL must be base64");
require(url.size() <= 8u * 1024u * 1024u, "image data URL over 8 MiB");
return;
}
require(is_http_url(url), "image url must be http(s) or a data:image/...;base64, URL");
}
} // namespace molo::cohere
+36
View File
@@ -0,0 +1,36 @@
#pragma once
// Input rules shared by every request type. All throw llm::ApiError::invalid
// (HTTP 400 INVALID_ARGUMENT, vendor "cohere") before any network traffic.
#include <string>
#include <vector>
#include "llm/error.h"
namespace molo::cohere {
struct Rules {
[[noreturn]] static void fail(const std::string& why);
static void require(bool ok, const std::string& why) {
if (!ok) fail(why);
}
// 1..128 of [A-Za-z0-9._:-]; the server decides which names exist.
static void model(const std::string& name);
// Single path segment id (llm::Service::valid_id).
static void id(const std::string& what, const std::string& id);
static void range(double v, double lo, double hi, const char* what);
static void temperature(double v); // 0..1
static void top_p(double v); // 0.01..0.99
static void top_k(long v); // 0..500
static void priority(long v); // 0..999
static void stop(const std::vector<std::string>& stop); // <= 5, non-empty
static void page_size(long v); // 1..1000
// One of the listed values.
static void one_of(const std::string& value, const std::vector<std::string>& allowed, const char* what);
// http(s) URL, or data:...;base64, URL for images, <= 8 MiB
static bool is_http_url(const std::string& url);
static void image_url(const std::string& url);
};
} // namespace molo::cohere
+424
View File
@@ -0,0 +1,424 @@
#!/usr/bin/env python3
"""Check src/vendors/cohere/calls.cpp against Cohere's published OpenAPI document.
Cohere publishes cohere-openapi.yaml in the cohere-ai/cohere-developer-experience
repository (the file their Fern SDK generation reads). This checker parses that
file with a small block-YAML reader (Python 3 standard library only, no PyYAML)
and fails (exit 1) when:
row a declared call's HTTP method + path or operationId is not in the spec;
field a request body field or query parameter we send is not in the spec's
request body / parameters of that operation;
required a field or parameter the spec marks required is not declared in the row;
schema a field we read (SCHEMAS table) is not a property of that schema;
response a declared response schema is neither a component nor an operationId;
coverage a spec operation is neither in the table nor in EXCLUDED_OPERATIONS.
python3 -I src/vendors/cohere/tools/check_spec.py [--spec path-to-openapi.yaml] [alternative calls.cpp]
Network required unless --spec is given. No credentials.
"""
import os
import re
import sys
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
CALLS = os.path.join(HERE, "..", "calls.cpp")
SPEC_URL = ("https://raw.githubusercontent.com/cohere-ai/cohere-developer-experience/main/"
"cohere-openapi.yaml")
# Operations that exist in the spec but are deliberately not wrapped (reason each).
EXCLUDED_OPERATIONS = {}
STR = re.compile(r'"([^"]*)"')
METHODS = ("get", "post", "put", "patch", "delete")
# ---------------------------------------------------------------- YAML reader
class Yaml:
"""Block-style YAML subset: mappings, sequences, plain / quoted scalars (single
line or folded continuation), block scalars (| >), empty flow [] and {}."""
def __init__(self, text):
self.lines = []
for raw in text.replace("\t", " ").split("\n"):
self.lines.append(raw.rstrip("\r"))
self.i = 0
@staticmethod
def indent_of(line):
return len(line) - len(line.lstrip(" "))
def skip_blank(self):
while self.i < len(self.lines):
s = self.lines[self.i].strip()
if s == "" or s.startswith("#") or s == "---":
self.i += 1
else:
return
def parse(self):
self.skip_blank()
return self.node(self.indent_of(self.lines[self.i]))
def node(self, indent):
self.skip_blank()
if self.i >= len(self.lines):
return None
line = self.lines[self.i]
if self.indent_of(line) < indent:
return None
body = line.strip()
if body.startswith("- ") or body == "-":
return self.seq(self.indent_of(line))
if self.key_split(body) is not None:
return self.mapping(self.indent_of(line))
self.i += 1
return self.scalar(body, self.indent_of(line))
@staticmethod
def key_split(body):
"""(key, rest) for `key: rest`, else None. Handles quoted keys."""
if body[:1] in ("'", '"'):
q = body[0]
j = 1
while j < len(body):
if body[j] == q and (q == "'" or body[j - 1] != "\\"):
break
j += 1
if j + 1 < len(body) and body[j + 1] == ":" and (j + 2 == len(body) or body[j + 2] == " "):
return body[1:j], body[j + 2:].strip()
return None
if body[:1] in ("[", "{", "|", ">", "#", "*", "&", "!"):
return None
m = re.match(r"^([^:#]+?|[^:#]*?[^:\s#][^#]*?):(?:\s+(.*))?$", body)
if not m:
return None
key = m.group(1)
if " #" in key:
return None
return key.strip(), (m.group(2) or "").strip()
def mapping(self, indent):
out = {}
while True:
self.skip_blank()
if self.i >= len(self.lines):
break
line = self.lines[self.i]
if self.indent_of(line) != indent:
break
body = line.strip()
kv = self.key_split(body)
if kv is None or body.startswith("- "):
break
key, rest = kv
self.i += 1
out[key] = self.value(rest, indent)
return out
def seq(self, indent):
out = []
while True:
self.skip_blank()
if self.i >= len(self.lines):
break
line = self.lines[self.i]
if self.indent_of(line) != indent:
break
body = line.strip()
if not (body.startswith("- ") or body == "-"):
break
rest = body[1:].strip()
if rest == "":
self.i += 1
out.append(self.node(indent + 1))
elif self.key_split(rest) is not None and not rest.startswith(("'", '"')) or \
(rest[:1] in ("'", '"') and self.key_split(rest) is not None):
# `- key: value`: treat the text after "- " as a mapping line at indent+2
self.lines[self.i] = " " * (indent + 2) + rest
out.append(self.mapping(indent + 2))
else:
self.i += 1
out.append(self.scalar(rest, indent))
return out
def value(self, rest, indent):
if rest == "" or rest.startswith("#"):
self.skip_blank()
if self.i < len(self.lines):
nxt = self.lines[self.i]
ni = self.indent_of(nxt)
nb = nxt.strip()
if ni > indent or (ni == indent and (nb.startswith("- ") or nb == "-")):
return self.node(ni)
return None
if rest[0] in "|>":
return self.block_scalar(rest, indent)
return self.scalar(rest, indent)
def block_scalar(self, header, indent):
fold = header[0] == ">"
buf = []
block_indent = None
while self.i < len(self.lines):
line = self.lines[self.i]
if line.strip() == "":
buf.append("")
self.i += 1
continue
li = self.indent_of(line)
if li <= indent:
break
if block_indent is None:
block_indent = li
buf.append(line[block_indent:] if li >= block_indent else line.strip())
self.i += 1
text = ("\n" if not fold else " ").join(buf).strip()
return text
def scalar(self, text, indent):
if text[0] in ("'", '"'):
q = text[0]
acc = text
# continuation lines until the closing quote
while not self.closed(acc, q) and self.i < len(self.lines):
acc += " " + self.lines[self.i].strip()
self.i += 1
body = acc[1:Yaml.end_of(acc, q)]
if q == '"':
body = body.replace('\\"', '"').replace("\\n", "\n").replace("\\\\", "\\")
else:
body = body.replace("''", "'")
return body
if text in ("[]", "{}"):
return [] if text == "[]" else {}
if text[0] in "[{":
acc = text
while acc.count("[") + acc.count("{") > acc.count("]") + acc.count("}") and self.i < len(self.lines):
acc += " " + self.lines[self.i].strip()
self.i += 1
return acc # flow collections beyond [] and {} are kept as text (not used by the checker)
# plain scalar: strip trailing comment, fold continuation lines that are more indented
text = re.sub(r"\s+#.*$", "", text)
while self.i < len(self.lines):
nxt = self.lines[self.i]
if nxt.strip() == "" or self.indent_of(nxt) <= indent:
break
if self.key_split(nxt.strip()) is not None and nxt.strip()[:1] not in ("'", '"'):
break
text += " " + nxt.strip()
self.i += 1
return text
@staticmethod
def end_of(acc, q):
j = 1
while j < len(acc):
c = acc[j]
if q == '"' and c == "\\":
j += 2
continue
if c == q:
if q == "'" and j + 1 < len(acc) and acc[j + 1] == "'":
j += 2
continue
return j
j += 1
return len(acc)
@staticmethod
def closed(acc, q):
"""True when the quoted scalar that starts acc has its closing quote."""
j = 1
while j < len(acc):
c = acc[j]
if q == '"' and c == "\\":
j += 2
continue
if c == q:
if q == "'" and j + 1 < len(acc) and acc[j + 1] == "'":
j += 2
continue
return True
j += 1
return False
# ------------------------------------------------------------------ spec model
class Spec:
def __init__(self, doc):
self.doc = doc
self.schemas = doc["components"]["schemas"]
self.params = doc["components"].get("parameters", {})
def ref(self, node):
seen = 0
while isinstance(node, dict) and "$ref" in node and seen < 20:
name = node["$ref"].split("/")[-1]
kind = node["$ref"].split("/")[-2]
node = (self.params if kind == "parameters" else self.schemas).get(name)
seen += 1
return node
def props(self, node, depth=0):
"""Union of property names (and required names) through $ref / allOf / oneOf / anyOf / items."""
props, required = {}, set()
node = self.ref(node)
if not isinstance(node, dict) or depth > 12:
return props, required
for key in ("allOf", "oneOf", "anyOf"):
for sub in node.get(key) or []:
p, r = self.props(sub, depth + 1)
props.update(p)
if key == "allOf":
required |= r
if isinstance(node.get("properties"), dict):
props.update(node["properties"])
if isinstance(node.get("required"), list):
required |= set(node["required"])
return props, required
def descend(self, node, path):
"""Follow a dotted property path; arrays are entered through items."""
for step in [p for p in path.split(".") if p]:
node = self.ref(node)
if isinstance(node, dict) and node.get("type") == "array" and "items" in node:
node = self.ref(node["items"])
props, _ = self.props(node)
if step not in props:
return None
node = props[step]
node = self.ref(node)
if isinstance(node, dict) and node.get("type") == "array" and "items" in node:
node = self.ref(node["items"])
return node
def operations(self):
for path, item in self.doc["paths"].items():
for method, op in item.items():
if method in METHODS and isinstance(op, dict):
yield method.upper(), path, op
def op_params(self, op):
out = {}
for p in op.get("parameters") or []:
p = self.ref(p)
if isinstance(p, dict) and p.get("in") in ("query", "path"):
out[p["name"]] = (p["in"], bool(p.get("required")))
return out
def op_body(self, op):
rb = op.get("requestBody")
if not rb:
return None, set()
content = rb.get("content") or {}
for ct in ("application/json", "multipart/form-data"):
if ct in content:
return self.props(content[ct].get("schema"))
return {}, set()
def op_response(self, op):
r = (op.get("responses") or {}).get("200") or {}
content = r.get("content") or {}
if "application/json" in content:
return content["application/json"].get("schema")
return None
# ------------------------------------------------------------------------ main
def rows(text, begin, end):
a, b = text.index(begin), text.index(end)
out = []
for line in text[a:b].splitlines()[1:]:
s = line.strip()
if s.startswith("{") and '"' in s:
out.append(STR.findall(s))
return out
def load_spec(argv):
if "--spec" in argv:
i = argv.index("--spec")
path = argv[i + 1]
del argv[i:i + 2]
with open(path, encoding="utf-8") as f:
return f.read()
req = urllib.request.Request(SPEC_URL, headers={"User-Agent": "molo-cohere-spec-check/1"})
with urllib.request.urlopen(req, timeout=120) as r:
return r.read().decode("utf-8", "replace")
def main():
argv = sys.argv[1:]
spec_text = load_spec(argv)
calls_path = argv[0] if argv else CALLS
text = open(calls_path).read()
calls = rows(text, "// CALLS-BEGIN", "// CALLS-END")
schemas = rows(text, "// SCHEMAS-BEGIN", "// SCHEMAS-END")
spec = Spec(Yaml(spec_text).parse())
ops = {(m, p): op for m, p, op in spec.operations()}
by_id = {op.get("operationId"): op for op in ops.values()}
bad = []
covered = set()
for method, path, op_id, body, query, response in calls:
op = ops.get((method, path))
if op is None:
bad.append("%s %s: not a path of the spec" % (method, path))
continue
covered.add((method, path))
if op.get("operationId") != op_id:
bad.append("%s %s: operationId is %s in the spec, %s in the table"
% (method, path, op.get("operationId"), op_id))
params = spec.op_params(op)
body_props, body_required = spec.op_body(op)
declared_body = {f for f in body.split(",") if f}
declared_query = {q for q in query.split(",") if q}
for f in sorted(declared_body):
if f not in body_props:
bad.append("%s %s: body field %s is not in the request body" % (method, path, f))
for q in sorted(declared_query):
if q not in params or params[q][0] != "query":
bad.append("%s %s: query parameter %s is not in the spec" % (method, path, q))
for name, (where, required) in params.items():
if where == "query" and required and name not in declared_query:
bad.append("%s %s: required query parameter %s missing in the row" % (method, path, name))
for f in sorted(body_required):
if f in body_props and f not in declared_body:
bad.append("%s %s: required body field %s missing in the row" % (method, path, f))
if response and response not in spec.schemas and response not in by_id:
bad.append("%s %s: response schema %s is neither a component nor an operationId" % (method, path, response))
for (method, path), op in sorted(ops.items()):
if (method, path) not in covered and op.get("operationId") not in EXCLUDED_OPERATIONS:
bad.append("unwrapped operation %s %s (%s): add a row or an exclusion"
% (method, path, op.get("operationId")))
checked_fields = 0
for name, fields in schemas:
head, _, tail = name.partition(".")
if head in spec.schemas:
node = spec.schemas[head]
elif head in by_id:
node = spec.op_response(by_id[head])
else:
bad.append("schema %s: neither a component nor an operationId" % head)
continue
target = spec.descend(node, tail) if tail else spec.ref(node)
if target is None:
bad.append("schema %s: property path not found" % name)
continue
props, _ = spec.props(target)
for f in filter(None, fields.split(",")):
checked_fields += 1
if f not in props:
bad.append("schema %s: field %s is not a property" % (name, f))
if bad:
print("SPEC CHECK FAILED")
for b in bad:
print(" " + b)
return 1
print("cohere spec ok: %d calls of %d spec operations (exclusions: %d), %d schema rows, %d fields"
% (len(calls), len(ops), len(EXCLUDED_OPERATIONS), len(schemas), checked_fields))
return 0
if __name__ == "__main__":
sys.exit(main())
+213
View File
@@ -0,0 +1,213 @@
#include "vendors/cohere/types.h"
#include <algorithm>
namespace molo::cohere {
std::string J::str(const Json& doc, const char* key, const std::string& fallback) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_string() ? v->as_string() : fallback;
}
long J::num(const Json& doc, const char* key, long fallback) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_number() ? v->as_int() : fallback;
}
double J::real(const Json& doc, const char* key, double fallback) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_number() ? v->as_number() : fallback;
}
bool J::flag(const Json& doc, const char* key, bool fallback) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_bool() ? v->as_bool() : fallback;
}
const Json* J::obj(const Json& doc, const char* key) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_object() ? v : nullptr;
}
const Json* J::arr(const Json& doc, const char* key) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_array() ? v : nullptr;
}
std::vector<std::string> J::strings(const Json& doc, const char* key) {
std::vector<std::string> out;
if (const Json* a = arr(doc, key))
for (const Json& v : a->as_array())
if (v.is_string()) out.push_back(v.as_string());
return out;
}
TokenUsage TokenUsage::parse(const Json* usage) {
TokenUsage u;
if (!usage || !usage->is_object()) return u;
u.present = true;
if (const Json* t = J::obj(*usage, "tokens")) {
u.input_tokens = J::num(*t, "input_tokens");
u.output_tokens = J::num(*t, "output_tokens");
}
if (const Json* b = J::obj(*usage, "billed_units")) {
u.billed_input_tokens = J::num(*b, "input_tokens");
u.billed_output_tokens = J::num(*b, "output_tokens");
u.search_units = J::num(*b, "search_units");
u.classifications = J::num(*b, "classifications");
}
u.cached_tokens = J::num(*usage, "cached_tokens");
return u;
}
static Page page_of(const Json& doc, const char* total_key = nullptr) {
Page p;
p.next_page_token = J::str(doc, "next_page_token");
if (total_key) p.total = J::num(doc, total_key, -1);
return p;
}
bool ModelInfo::supports(const std::string& endpoint) const {
return std::find(endpoints.begin(), endpoints.end(), endpoint) != endpoints.end();
}
ModelInfo ModelInfo::parse(const Json& doc) {
ModelInfo m;
m.name = J::str(doc, "name");
m.deprecated = J::flag(doc, "is_deprecated");
m.finetuned = J::flag(doc, "finetuned");
m.context_length = J::real(doc, "context_length");
m.tokenizer_url = J::str(doc, "tokenizer_url");
m.endpoints = J::strings(doc, "endpoints");
m.default_endpoints = J::strings(doc, "default_endpoints");
m.features = J::strings(doc, "features");
m.raw = doc;
return m;
}
ModelList ModelList::parse(const Json& doc) {
ModelList l;
if (const Json* a = J::arr(doc, "models"))
for (const Json& m : a->as_array()) l.models.push_back(ModelInfo::parse(m));
l.page = page_of(doc);
return l;
}
const ModelInfo* ModelList::find(const std::string& name) const {
for (const ModelInfo& m : models)
if (m.name == name) return &m;
return nullptr;
}
ApiKeyCheck ApiKeyCheck::parse(const Json& doc) {
ApiKeyCheck c;
c.valid = J::flag(doc, "valid");
c.organization_id = J::str(doc, "organization_id");
c.owner_id = J::str(doc, "owner_id");
return c;
}
bool Batch::terminal() const {
return status == "BATCH_STATUS_COMPLETED" || status == "BATCH_STATUS_FAILED" ||
status == "BATCH_STATUS_CANCELED";
}
Batch Batch::parse(const Json& doc) {
Batch b;
b.id = J::str(doc, "id");
b.name = J::str(doc, "name");
b.status = J::str(doc, "status");
b.created_at = J::str(doc, "created_at");
b.updated_at = J::str(doc, "updated_at");
b.input_dataset_id = J::str(doc, "input_dataset_id");
b.output_dataset_id = J::str(doc, "output_dataset_id");
b.model = J::str(doc, "model");
b.status_reason = J::str(doc, "status_reason");
b.input_tokens = J::str(doc, "input_tokens");
b.output_tokens = J::str(doc, "output_tokens");
b.num_records = J::num(doc, "num_records");
b.num_successful_records = J::num(doc, "num_successful_records");
b.num_failed_records = J::num(doc, "num_failed_records");
b.raw = doc;
return b;
}
BatchList BatchList::parse(const Json& doc) {
BatchList l;
if (const Json* a = J::arr(doc, "batches"))
for (const Json& b : a->as_array()) l.batches.push_back(Batch::parse(b));
l.page = page_of(doc);
return l;
}
bool EmbedJob::terminal() const { return status == "complete" || status == "cancelled" || status == "failed"; }
EmbedJob EmbedJob::parse(const Json& doc) {
EmbedJob j;
j.job_id = J::str(doc, "job_id");
j.name = J::str(doc, "name");
j.status = J::str(doc, "status");
j.created_at = J::str(doc, "created_at");
j.input_dataset_id = J::str(doc, "input_dataset_id");
j.output_dataset_id = J::str(doc, "output_dataset_id");
j.model = J::str(doc, "model");
j.truncate = J::str(doc, "truncate");
j.raw = doc;
return j;
}
DatasetPart DatasetPart::parse(const Json& doc) {
DatasetPart p;
p.id = J::str(doc, "id");
p.name = J::str(doc, "name");
p.url = J::str(doc, "url");
p.original_url = J::str(doc, "original_url");
p.index = J::num(doc, "index");
p.size_bytes = J::num(doc, "size_bytes");
p.num_rows = J::num(doc, "num_rows");
return p;
}
Dataset Dataset::parse(const Json& doc) {
Dataset d;
d.id = J::str(doc, "id");
d.name = J::str(doc, "name");
d.created_at = J::str(doc, "created_at");
d.updated_at = J::str(doc, "updated_at");
d.dataset_type = J::str(doc, "dataset_type");
d.validation_status = J::str(doc, "validation_status");
d.validation_error = J::str(doc, "validation_error");
d.validation_warnings = J::strings(doc, "validation_warnings");
if (const Json* a = J::arr(doc, "dataset_parts"))
for (const Json& p : a->as_array()) d.parts.push_back(DatasetPart::parse(p));
d.raw = doc;
return d;
}
Connector Connector::parse(const Json& doc) {
Connector c;
c.id = J::str(doc, "id");
c.organization_id = J::str(doc, "organization_id");
c.name = J::str(doc, "name");
c.url = J::str(doc, "url");
c.created_at = J::str(doc, "created_at");
c.updated_at = J::str(doc, "updated_at");
c.auth_type = J::str(doc, "auth_type");
c.auth_status = J::str(doc, "auth_status");
c.excludes = J::strings(doc, "excludes");
c.active = J::flag(doc, "active");
c.continue_on_failure = J::flag(doc, "continue_on_failure");
c.raw = doc;
return c;
}
FinetunedModel FinetunedModel::parse(const Json& doc) {
FinetunedModel m;
m.id = J::str(doc, "id");
m.name = J::str(doc, "name");
m.creator_id = J::str(doc, "creator_id");
m.organization_id = J::str(doc, "organization_id");
m.status = J::str(doc, "status");
m.created_at = J::str(doc, "created_at");
m.updated_at = J::str(doc, "updated_at");
m.completed_at = J::str(doc, "completed_at");
m.last_used = J::str(doc, "last_used");
if (const Json* s = J::obj(doc, "settings")) m.settings = *s;
return m;
}
} // namespace molo::cohere
+125
View File
@@ -0,0 +1,125 @@
#pragma once
// Typed views of Cohere documents. Every parser reads only fields the call
// table's schema rows declare; the full document stays available as `raw`.
#include <string>
#include <vector>
#include "clients/body/json.h"
namespace molo::cohere {
using Json = clients::body::Json;
// Tolerant readers: a missing or mistyped field gives the fallback.
struct J {
static std::string str(const Json& doc, const char* key, const std::string& fallback = "");
static long num(const Json& doc, const char* key, long fallback = 0);
static double real(const Json& doc, const char* key, double fallback = 0);
static bool flag(const Json& doc, const char* key, bool fallback = false);
static const Json* obj(const Json& doc, const char* key);
static const Json* arr(const Json& doc, const char* key);
static std::vector<std::string> strings(const Json& doc, const char* key);
};
// Usage (chat) and ApiMeta (everything else) share these fields.
struct TokenUsage {
long input_tokens = 0; // tokens.input_tokens
long output_tokens = 0; // tokens.output_tokens
long billed_input_tokens = 0; // billed_units.input_tokens
long billed_output_tokens = 0;
long search_units = 0;
long classifications = 0;
long cached_tokens = 0;
bool present = false;
static TokenUsage parse(const Json* usage);
};
struct Page {
std::string next_page_token;
long total = -1; // total_size / total_count when sent
bool truncated = false; // more pages existed than the walk cap allowed
};
struct ModelInfo {
std::string name;
bool deprecated = false;
bool finetuned = false;
double context_length = 0;
std::string tokenizer_url;
std::vector<std::string> endpoints;
std::vector<std::string> default_endpoints;
std::vector<std::string> features;
Json raw;
bool supports(const std::string& endpoint) const;
static ModelInfo parse(const Json& doc);
};
struct ModelList {
std::vector<ModelInfo> models;
Page page;
const ModelInfo* find(const std::string& name) const;
static ModelList parse(const Json& doc);
};
struct ApiKeyCheck {
bool valid = false;
std::string organization_id;
std::string owner_id;
static ApiKeyCheck parse(const Json& doc);
};
struct Batch {
std::string id, name, status, created_at, updated_at, input_dataset_id, output_dataset_id, model;
std::string status_reason, input_tokens, output_tokens; // token counts are strings in the spec
long num_records = 0, num_successful_records = 0, num_failed_records = 0;
Json raw;
bool terminal() const; // COMPLETED, FAILED or CANCELED
static Batch parse(const Json& doc);
};
struct BatchList {
std::vector<Batch> batches;
Page page;
static BatchList parse(const Json& doc);
};
struct EmbedJob {
std::string job_id, name, status, created_at, input_dataset_id, output_dataset_id, model, truncate;
Json raw;
bool terminal() const; // complete, cancelled or failed
static EmbedJob parse(const Json& doc);
};
struct DatasetPart {
std::string id, name, url, original_url;
long index = 0, size_bytes = 0, num_rows = 0;
static DatasetPart parse(const Json& doc);
};
struct Dataset {
std::string id, name, created_at, updated_at, dataset_type, validation_status, validation_error;
std::vector<std::string> validation_warnings;
std::vector<DatasetPart> parts;
Json raw;
bool validated() const { return validation_status == "validated"; }
static Dataset parse(const Json& doc);
};
struct Connector {
std::string id, organization_id, name, url, created_at, updated_at, auth_type, auth_status;
std::vector<std::string> excludes;
bool active = false, continue_on_failure = false;
Json raw;
static Connector parse(const Json& doc);
};
struct FinetunedModel {
std::string id, name, creator_id, organization_id, status, created_at, updated_at, completed_at, last_used;
Json settings;
bool ready() const { return status == "STATUS_READY"; }
static FinetunedModel parse(const Json& doc);
};
} // namespace molo::cohere