feat(clients): cohere: whole API, CohereChatModel, spec checker, demo and tests
Package src/vendors/cohere (molo::cohere): Chat v2 with streaming, tools, thinking and citations; embed, rerank, classify, tokenize, detokenize, parse, transcription; datasets, embed jobs, batches, connectors, fine-tuning, models, check-api-key; legacy v1 chat/generate/summarize; CohereChatModel : ChatModel. All 42 operations of cohere-openapi.yaml are in one call table, verified by tools/check_spec.py. Demo routes under /ai/cohere, 51 unit tests and 2 route tests; live tests are opt-in. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Vendored
+196
@@ -0,0 +1,196 @@
|
||||
# Cohere API platform - Specification
|
||||
|
||||
Owned directory: `src/vendors/cohere/`. Namespace `molo::cohere`. Depends on the
|
||||
`src/llm` foundation (`ApiClient`, `Service`, `SseDecoder`, `ApiError`,
|
||||
`Config`, `ChatModel`). Plans: `plans/17-llm-vendor-platforms.md`,
|
||||
`plans/18-breadth.md`. Credential: `COHERE_API_KEY` via `llm::Config`
|
||||
(`CO_API_KEY`, the official SDKs' name, is accepted as a fallback).
|
||||
Tests: `tests/src/vendors/cohere/`. Demo: `example/app/ai/CohereDemo.cpp`,
|
||||
routes `/ai/cohere`, script `example/assets/ai_cohere.js`, route tests
|
||||
`tests/example/ai/cohere/`.
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Every operation of Cohere's OpenAPI document (42 of 42, host
|
||||
`https://api.cohere.com`, Bearer auth):
|
||||
|
||||
- Chat v2 (`POST /v2/chat`): messages (system, user with text and image parts,
|
||||
assistant, tool), tools and `strict_tools`, `tool_choice`, documents and
|
||||
citation options, JSON response format with schema, safety mode, thinking
|
||||
(`token_budget`), sampling and penalties, logprobs, priority, streaming
|
||||
(all eleven event types), `Conversation` with a capped tool loop.
|
||||
- Legacy v1 that remains in the spec: chat, generate, summarize (Json bodies
|
||||
checked against the call table), embed, rerank, classify.
|
||||
- Embed v2 (texts, images, mixed `inputs`, `embedding_types`, `output_dimension`),
|
||||
Rerank v2 (`relevant_spans`, `max_tokens_per_doc`), Tokenize, Detokenize,
|
||||
Parse (`/v2/parse`), audio transcription (`/v2/audio/transcriptions`, multipart).
|
||||
- Datasets (multipart upload, list, get, usage, delete), Embed jobs, Batches
|
||||
(`/v2/batches`, cancel), Connectors (spec marks them deprecated), Fine-tuning
|
||||
(spec marks it deprecated: list, create, get, update, delete, events,
|
||||
training-step metrics), Models (list, get), `check-api-key`.
|
||||
- `CohereChatModel : llm::ChatModel` over Chat v2.
|
||||
|
||||
Not here, said plainly: the spec defines cluster-job schemas
|
||||
(`CreateClusterJobRequest` and friends) and `FeedbackResponse` as components,
|
||||
but no path uses them, so there is nothing to call.
|
||||
|
||||
## 2. Module map (one class per concern)
|
||||
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings, never autotuned |
|
||||
| `calls.h/.cpp` | THE call table (42 rows) and the schema-field table; `render` (id and query validation), `check_body` |
|
||||
| `rules.h/.cpp` | Input rules (model, ids, ranges, enums, URLs); all throw `ApiError::invalid` before any traffic |
|
||||
| `types.h/.cpp` | Tolerant readers `J`, `TokenUsage`, `Page`, `ModelInfo`, `Batch`, `EmbedJob`, `Dataset`, `Connector`, `FinetunedModel` |
|
||||
| `platform.h/.cpp` | Host, Bearer auth, `X-Client-Name: molo` header, key lookup |
|
||||
| `chat_params.h/.cpp` | `Messages` builders, `ToolDef`, `ChatParams` (validate + body) |
|
||||
| `chat_result.h/.cpp` | `ChatResult`, `StreamDelta`, `ChatStreamAccumulator` |
|
||||
| `chat.h/.cpp` | `Chat` (create, stream, legacy v1), `Conversation` (history, tool loop) |
|
||||
| `chat_model.h/.cpp` | `CohereChatModel : llm::ChatModel` and its documented mapping |
|
||||
| `inference.h/.cpp` | `Embeddings`, `Rerank`, `Classify`, `Tokenizer`, `Legacy` |
|
||||
| `resources.h/.cpp` | `Account`, `Datasets`, `Batches`, `EmbedJobs`, `Connectors`, `FineTuning`, `Documents` |
|
||||
| `catalog.h/.cpp` | Model names from the docs page, defaults |
|
||||
| `tools/check_spec.py` | Spec checker (see Verified against spec) |
|
||||
|
||||
## 3. Static caps
|
||||
|
||||
| Knob | Value | Origin |
|
||||
|---|---|---|
|
||||
| Embed texts / images / inputs per call | 96 | spec `maxItems` (texts); ours for the rest |
|
||||
| Classify inputs | 96 | spec `maxItems` |
|
||||
| Tokenize text / detokenize tokens | 65536 | spec `maxLength` / `maxItems` |
|
||||
| Chat stop sequences | 5 | spec description |
|
||||
| `k` | 0..500, `p` 0.01..0.99, `temperature` 0..1, `priority` 0..999, penalties 0..1 | spec |
|
||||
| Page size | 1..1000 | spec description |
|
||||
| Messages / tools / documents | 4096 / 256 / 1000 | ours |
|
||||
| Stream events | 200000 | ours |
|
||||
| List walk | 20 pages, 5000 items | ours; truncation is reported (`Page::truncated`) |
|
||||
| Upload (datasets, audio) | 256 MiB | ours (llm cap is 512 MiB) |
|
||||
| Tool loop rounds | 16 | ours |
|
||||
| Dataset validation polls | 60, wait supplied by the caller | ours |
|
||||
| Body / response / stream bytes | `llm::Caps` | foundation |
|
||||
|
||||
## 4. Contracts
|
||||
|
||||
- No key: `configured()` is false and every call throws `NotConfiguredError`
|
||||
(503, `NOT_CONFIGURED`) before any network traffic. The key travels in the
|
||||
`Authorization` header only.
|
||||
- Every request body is built from typed params or checked with
|
||||
`Calls::check_body`: a field that is not in the call table is rejected
|
||||
loudly. Path ids go through `Service::valid_id`; query names must be in the row.
|
||||
- Destructive or cancelling calls (`Datasets::remove`, `Batches::cancel`,
|
||||
`EmbedJobs::cancel`, `Connectors::remove`, `FineTuning::remove`) are explicit
|
||||
methods with no defaults. Calls that cost money or train a model
|
||||
(`FineTuning::create`, `Batches::create`, `EmbedJobs::create`) validate their
|
||||
settings first.
|
||||
- A stream without `message-end` throws `502 STREAM_TRUNCATED`; a `message-end`
|
||||
with `delta.error` throws `502 STREAM_ERROR`; a JSON error body in place of a
|
||||
stream throws (SSE decoder).
|
||||
- Tool arguments arrive as the model's JSON text; `ToolCallRecord::arguments_valid()`
|
||||
tells, and `CohereChatModel` keeps invalid text as `{"_invalid_json": "..."}`.
|
||||
- `CohereChatModel` mapping (also in `chat_model.h`): `tool_choice` required/none
|
||||
-> `REQUIRED`/`NONE`, a tool name is rejected (Cohere cannot force one tool);
|
||||
`json_mode`/`response_schema` -> `response_format` json_object (+ `json_schema`);
|
||||
`reasoning_effort` none/off/disabled -> thinking disabled, anything else ->
|
||||
enabled; finish reasons COMPLETE and STOP_SEQUENCE -> stop, MAX_TOKENS ->
|
||||
length, TOOL_CALL -> tool_calls; usage from `usage.tokens` with `billed_units`
|
||||
as the fallback.
|
||||
- Datasets: `keep_fields` and `optional_fields` are repeated query parameters
|
||||
(what the official Python SDK does by passing a sequence to httpx).
|
||||
- `Datasets::await_validation` never sleeps; the caller passes the wait.
|
||||
|
||||
## 5. Example routes (`/ai/cohere`)
|
||||
|
||||
GET: page, `/status`, `/calls` (no credential). POST: `chat`, `stream`,
|
||||
`tools`, `legacy`, `embed`, `rerank`, `classify`, `tokenize`, `detokenize`,
|
||||
`parse`, `models`, `key`, `datasets`, `batches`, `embedjobs`, `connectors`,
|
||||
`finetune` (resource routes take an `action`). 422 on bad input, 503
|
||||
`cohere_not_configured` without a key, 502 `cohere_error` for vendor errors.
|
||||
Chat, stream, tools, embed, rerank, classify, parse, legacy and every create,
|
||||
update, cancel and delete are dry runs unless the body has `"confirm":true`.
|
||||
Reads and tokenize run directly once a key exists.
|
||||
|
||||
## Sources
|
||||
|
||||
1. `cohere-openapi.yaml` in `cohere-ai/cohere-developer-experience` (branch
|
||||
`main`, fetched 2026-10-25, 1,140,155 bytes, ETag
|
||||
`148fca0f27bb02cdf86b2eb95b7d9a02efa1bca0263cae8d6c9dcc571cb17d60`),
|
||||
https://raw.githubusercontent.com/cohere-ai/cohere-developer-experience/main/cohere-openapi.yaml .
|
||||
OpenAPI 3.1, 42 operations, 191 component schemas. Its `x-fern-examples` and
|
||||
`x-examples` are the fixtures in `tests/src/vendors/cohere/fixtures.h`.
|
||||
2. Official Python SDK `cohere-ai/cohere-python` (`src/cohere/*/raw_client.py`,
|
||||
fetched 2026-10-25): 31 path templates match the spec; the repeated query
|
||||
parameter behaviour for datasets comes from here.
|
||||
3. Docs model list https://docs.cohere.com/docs/models (read 2026-10-25): model
|
||||
names, status and limits in `catalog.cpp`.
|
||||
4. Live unauthenticated probes (below).
|
||||
|
||||
## Verified against spec
|
||||
|
||||
`python3 -I src/vendors/cohere/tools/check_spec.py` (2026-10-25, against the
|
||||
fetched document): `cohere spec ok: 42 calls of 42 spec operations (exclusions: 0),
|
||||
75 schema rows, 263 fields`. The checker parses the YAML with a small stdlib
|
||||
reader (compared with PyYAML on the whole file: identical except for long flow
|
||||
arrays inside examples, which it does not use). It fails on a method+path or
|
||||
operationId missing from the spec, a body field or query parameter we send that
|
||||
the operation does not have, a spec-required field missing from the row, a
|
||||
schema field we read that is not a property (dotted paths descend into nested
|
||||
objects and arrays), a response name that is neither a component nor an
|
||||
operationId, and any spec operation that is neither in the table nor in the
|
||||
(empty) exclusion list. Negative runs with a wrong field, a missing required
|
||||
field, a wrong response name and a wrong schema field each fail as expected.
|
||||
|
||||
## Verified live
|
||||
|
||||
Unauthenticated probes, 2026-10-25, no credential used:
|
||||
|
||||
- TLS and DNS for `api.cohere.com` work (HTTP/2).
|
||||
- `POST /v2/chat` with no key: 401 `{"id":"...","message":"no api key supplied"}`.
|
||||
- `GET /v1/models` and `POST /v2/chat` with an invalid bearer: 401
|
||||
`{"id":"...","message":"Incorrect API key provided: ***...robe. You can find your API key at https://dashboard.cohere.com/api-keys."}`.
|
||||
- `POST /v1/check-api-key` with an invalid bearer: HTTP 200 `{"valid":false}`.
|
||||
- All 42 documented method+path pairs, called without a key: 42 x 401. A made-up
|
||||
path (`GET /v1/nothing-here`) also answers 401, so the gateway authenticates
|
||||
before routing and a 401 does NOT prove a route exists. Existence is verified
|
||||
against the spec and the SDK only.
|
||||
- Opt-in tests: `MOLO_LIVE_NET=1` runs the invalid-key probes
|
||||
(`cohere_live.invalid_key_probes` passed on 2026-10-25); with `COHERE_API_KEY`
|
||||
also key check, chat model list and ONE tiny chat; `MOLO_LIVE_PAID=1` adds one
|
||||
embed and one rerank. Without them each prints a loud SKIP. Never run so far
|
||||
with a key (none exists here).
|
||||
|
||||
## Unverified
|
||||
|
||||
No success response from Cohere has been seen. Everything below is inferred:
|
||||
|
||||
- Real reply bodies for models list/get, datasets, batches, embed jobs,
|
||||
connectors, fine-tuning, parse, transcription and generate. Tests use minimal
|
||||
documents labelled SHAPE, built from the OpenAPI schemas.
|
||||
- Streaming of tool calls and citations beyond the two documented transcripts
|
||||
(text, tools). Citation, thinking and logprob stream events are handled from
|
||||
the schema.
|
||||
- That `keep_fields`/`optional_fields` are repeated query parameters on the wire.
|
||||
- `X-Client-Name` is optional in the spec; we send `molo`.
|
||||
- Whether `command-a-03-2025`, `embed-v4.0`, `rerank-v3.5` stay valid defaults;
|
||||
the docs page lists them (2026-10-25) and the live list is the truth.
|
||||
- Tokenize and detokenize pricing and whether they need a model that supports
|
||||
them; the demo defaults to `command-a-03-2025`.
|
||||
- Spec says `GET /v1/models` pages with `page_size` up to 1000 (our walk uses 1000).
|
||||
- Classify without a fine-tuned model: the 2-example minimum is our guard, not a
|
||||
documented rule. The endpoint is marked deprecated in the spec.
|
||||
- Behaviour of deprecated surfaces (connectors, fine-tuning, classify, generate,
|
||||
summarize): present in the spec, not exercised.
|
||||
|
||||
## Gaps
|
||||
|
||||
- `src/llm`: `ApiClient::stream` is POST-only; `post_multipart` parses JSON replies
|
||||
only; no per-call timeout; `ApiError` keeps no `Retry-After`. Cohere's 429
|
||||
semantics (trial key limits) are therefore not surfaced beyond the status.
|
||||
- No `CO-Api-Key` style header or v1 streaming (`/v1/chat` stream, `/v1/generate`
|
||||
stream use a different event format); both are rejected by `stream:true` on
|
||||
purpose.
|
||||
- Embed results are kept as `Json`; typed accessors exist for floats only.
|
||||
- `FineTuning` settings and connector OAuth are passed as `Json` (validated for
|
||||
the required keys only).
|
||||
- Credential console: https://dashboard.cohere.com/api-keys (named in Cohere's own
|
||||
401 body, 2026-10-25). Owner action: add the row to `src/llm/SPEC.md` section 5.
|
||||
Vendored
+214
@@ -0,0 +1,214 @@
|
||||
#include "vendors/cohere/calls.h"
|
||||
|
||||
#include <cstring>
|
||||
#include <set>
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "llm/api_client.h"
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
// CALLS-BEGIN (parsed by tools/check_spec.py; one row per line, strings in order:
|
||||
// http method, path, operationId, body fields, query params, response schema)
|
||||
static const CallSpec kCalls[] = {
|
||||
{Call::kChatV1, "POST", "/v1/chat", "chat", "message,model,stream,preamble,chat_history,conversation_id,prompt_truncation,connectors,search_queries_only,documents,citation_quality,temperature,max_tokens,max_input_tokens,k,p,seed,stop_sequences,frequency_penalty,presence_penalty,raw_prompting,tool_results,tools,force_single_step,response_format,safety_mode", "", "chat"},
|
||||
{Call::kChat, "POST", "/v2/chat", "chatv2", "model,messages,stream,tools,strict_tools,documents,citation_options,response_format,safety_mode,max_tokens,stop_sequences,temperature,seed,frequency_penalty,presence_penalty,k,p,logprobs,tool_choice,thinking,priority", "", "ChatResponseV2"},
|
||||
{Call::kParse, "POST", "/v2/parse", "parsev2", "model,document,output_format", "", "ParseResponse"},
|
||||
{Call::kGenerate, "POST", "/v1/generate", "generate", "model,prompt,stream,num_generations,max_tokens,truncate,temperature,seed,preset,end_sequences,stop_sequences,k,p,frequency_penalty,presence_penalty,return_likelihoods,raw_prompting", "", "Generation"},
|
||||
{Call::kEmbedV1, "POST", "/v1/embed", "embed", "texts,images,model,input_type,embedding_types,truncate", "", "embed"},
|
||||
{Call::kEmbed, "POST", "/v2/embed", "embedv2", "model,input_type,texts,images,inputs,max_tokens,output_dimension,embedding_types,truncate,priority", "", "EmbedByTypeResponse"},
|
||||
{Call::kTranscribe, "POST", "/v2/audio/transcriptions", "create-transcription", "model,language,temperature,file", "", "create-transcription"},
|
||||
{Call::kBatchList, "GET", "/v2/batches", "ListBatches", "", "page_size,page_token,order_by", "ListBatchesResponse"},
|
||||
{Call::kBatchCreate, "POST", "/v2/batches", "CreateBatch", "name,input_dataset_id,model", "", "CreateBatchResponse"},
|
||||
{Call::kBatchGet, "GET", "/v2/batches/{id}", "GetBatch", "", "", "GetBatchResponse"},
|
||||
{Call::kBatchCancel, "POST", "/v2/batches/{id}:cancel", "CancelBatch", "", "", "CancelBatchResponse"},
|
||||
{Call::kEmbedJobCreate, "POST", "/v1/embed-jobs", "create-embed-job", "model,dataset_id,input_type,name,embedding_types,truncate", "", "CreateEmbedJobResponse"},
|
||||
{Call::kEmbedJobList, "GET", "/v1/embed-jobs", "list-embed-jobs", "", "", "ListEmbedJobResponse"},
|
||||
{Call::kEmbedJobGet, "GET", "/v1/embed-jobs/{id}", "get-embed-job", "", "", "EmbedJob"},
|
||||
{Call::kEmbedJobCancel, "POST", "/v1/embed-jobs/{id}/cancel", "cancel-embed-job", "", "", ""},
|
||||
{Call::kRerankV1, "POST", "/v1/rerank", "rerank", "model,query,documents,top_n,rank_fields,return_documents,max_chunks_per_doc", "", "rerank"},
|
||||
{Call::kRerank, "POST", "/v2/rerank", "rerankv2", "model,query,documents,top_n,max_tokens_per_doc,relevant_spans,priority", "", "rerankv2"},
|
||||
{Call::kClassify, "POST", "/v1/classify", "classify", "inputs,examples,model,preset,truncate", "", "classify"},
|
||||
{Call::kDatasetCreate, "POST", "/v1/datasets", "create-dataset", "data,eval_data", "name,type,keep_original_file,skip_malformed_input,keep_fields,optional_fields,text_separator,csv_delimiter", "create-dataset"},
|
||||
{Call::kDatasetList, "GET", "/v1/datasets", "list-datasets", "", "datasetType,before,after,limit,offset,validationStatus", "list-datasets"},
|
||||
{Call::kDatasetUsage, "GET", "/v1/datasets/usage", "get-dataset-usage", "", "", "get-dataset-usage"},
|
||||
{Call::kDatasetGet, "GET", "/v1/datasets/{id}", "get-dataset", "", "", "get-dataset"},
|
||||
{Call::kDatasetDelete, "DELETE", "/v1/datasets/{id}", "delete-dataset", "", "", ""},
|
||||
{Call::kSummarize, "POST", "/v1/summarize", "summarize", "text,length,format,model,extractiveness,temperature,additional_command", "", "summarize"},
|
||||
{Call::kTokenize, "POST", "/v1/tokenize", "tokenize", "text,model", "", "tokenize"},
|
||||
{Call::kDetokenize, "POST", "/v1/detokenize", "detokenize", "tokens,model", "", "detokenize"},
|
||||
{Call::kConnectorList, "GET", "/v1/connectors", "list-connectors", "", "limit,offset", "ListConnectorsResponse"},
|
||||
{Call::kConnectorCreate, "POST", "/v1/connectors", "create-connector", "name,url,description,excludes,oauth,active,continue_on_failure,service_auth", "", "CreateConnectorResponse"},
|
||||
{Call::kConnectorGet, "GET", "/v1/connectors/{id}", "get-connector", "", "", "GetConnectorResponse"},
|
||||
{Call::kConnectorUpdate, "PATCH", "/v1/connectors/{id}", "update-connector", "name,url,excludes,oauth,active,continue_on_failure,service_auth", "", "UpdateConnectorResponse"},
|
||||
{Call::kConnectorDelete, "DELETE", "/v1/connectors/{id}", "delete-connector", "", "", ""},
|
||||
{Call::kConnectorAuthorize, "POST", "/v1/connectors/{id}/oauth/authorize", "oAuthAuthorize-connector", "", "after_token_redirect", "OAuthAuthorizeResponse"},
|
||||
{Call::kModelGet, "GET", "/v1/models/{model}", "get-model", "", "", "GetModelResponse"},
|
||||
{Call::kModelList, "GET", "/v1/models", "list-models", "", "page_size,page_token,endpoint,default_only", "ListModelsResponse"},
|
||||
{Call::kCheckApiKey, "POST", "/v1/check-api-key", "checkAPIKey", "", "", "checkAPIKey"},
|
||||
{Call::kFinetuneList, "GET", "/v1/finetuning/finetuned-models", "ListFinetunedModels", "", "page_size,page_token,order_by", "ListFinetunedModelsResponse"},
|
||||
{Call::kFinetuneCreate, "POST", "/v1/finetuning/finetuned-models", "CreateFinetunedModel", "name,settings", "", "CreateFinetunedModelResponse"},
|
||||
{Call::kFinetuneUpdate, "PATCH", "/v1/finetuning/finetuned-models/{id}", "UpdateFinetunedModel", "name,settings", "", "UpdateFinetunedModelResponse"},
|
||||
{Call::kFinetuneGet, "GET", "/v1/finetuning/finetuned-models/{id}", "GetFinetunedModel", "", "", "GetFinetunedModelResponse"},
|
||||
{Call::kFinetuneDelete, "DELETE", "/v1/finetuning/finetuned-models/{id}", "DeleteFinetunedModel", "", "", ""},
|
||||
{Call::kFinetuneEvents, "GET", "/v1/finetuning/finetuned-models/{finetuned_model_id}/events", "ListEvents", "", "page_size,page_token,order_by", "ListEventsResponse"},
|
||||
{Call::kFinetuneMetrics, "GET", "/v1/finetuning/finetuned-models/{finetuned_model_id}/training-step-metrics", "ListTrainingStepMetrics", "", "page_size,page_token", "ListTrainingStepMetricsResponse"},
|
||||
};
|
||||
// CALLS-END
|
||||
|
||||
// SCHEMAS-BEGIN (parsed by tools/check_spec.py; one row per line: schema, fields)
|
||||
static const SchemaFields kSchemas[] = {
|
||||
{"ChatResponseV2", "id,finish_reason,message,usage,logprobs"},
|
||||
{"AssistantMessageResponse", "role,tool_calls,tool_plan,content,citations"},
|
||||
{"ChatTextContent", "type,text"},
|
||||
{"ChatThinkingContent", "type,thinking"},
|
||||
{"ToolCallV2", "id,type,function"},
|
||||
{"ToolCallV2.function", "name,arguments"},
|
||||
{"Usage", "billed_units,tokens,cached_tokens"},
|
||||
{"Usage.billed_units", "input_tokens,output_tokens,search_units,classifications"},
|
||||
{"Usage.tokens", "input_tokens,output_tokens"},
|
||||
{"Citation", "start,end,text,sources,content_index,type"},
|
||||
{"ChatMessageStartEvent", "type,id,delta"},
|
||||
{"ChatContentStartEvent", "type,index,delta"},
|
||||
{"ChatContentStartEvent.delta.message.content", "type,text,thinking"},
|
||||
{"ChatContentDeltaEvent", "type,index,delta,logprobs"},
|
||||
{"ChatContentDeltaEvent.delta.message.content", "text,thinking"},
|
||||
{"ChatContentEndEvent", "type,index"},
|
||||
{"ChatToolPlanDeltaEvent", "type,delta"},
|
||||
{"ChatToolPlanDeltaEvent.delta.message", "tool_plan"},
|
||||
{"ChatToolCallStartEvent", "type,index,delta"},
|
||||
{"ChatToolCallStartEvent.delta.message", "tool_calls"},
|
||||
{"ChatToolCallDeltaEvent", "type,index,delta"},
|
||||
{"ChatToolCallDeltaEvent.delta.message.tool_calls.function", "arguments"},
|
||||
{"ChatToolCallEndEvent", "type,index"},
|
||||
{"CitationStartEvent", "type,index,delta"},
|
||||
{"CitationStartEvent.delta.message", "citations"},
|
||||
{"ChatMessageEndEvent", "type,id,delta"},
|
||||
{"ChatMessageEndEvent.delta", "error,finish_reason,usage"},
|
||||
{"chat", "text,generation_id,response_id,citations,documents,is_search_required,search_queries,search_results,finish_reason,tool_calls,chat_history,meta"},
|
||||
{"ParseResponse", "id,pages,meta,finish_reason"},
|
||||
{"Generation", "id,prompt,generations,meta"},
|
||||
{"embed", "id,embeddings,texts,images,meta,response_type"},
|
||||
{"EmbedByTypeResponse", "id,embeddings,texts,images,meta,response_type"},
|
||||
{"EmbedByTypeResponse.embeddings", "float,int8,uint8,binary,ubinary,base64"},
|
||||
{"ApiMeta", "api_version,billed_units,tokens,cached_tokens,warnings"},
|
||||
{"create-transcription", "text"},
|
||||
{"ListBatchesResponse", "batches,next_page_token"},
|
||||
{"Batch", "id,name,status,created_at,updated_at,input_dataset_id,output_dataset_id,model,input_tokens,output_tokens,num_records,num_successful_records,num_failed_records,status_reason,creator_id,org_id"},
|
||||
{"CreateBatchResponse", "batch"},
|
||||
{"GetBatchResponse", "batch"},
|
||||
{"CreateEmbedJobResponse", "job_id,meta"},
|
||||
{"ListEmbedJobResponse", "embed_jobs"},
|
||||
{"EmbedJob", "job_id,name,status,created_at,input_dataset_id,output_dataset_id,model,truncate,meta"},
|
||||
{"rerank", "id,results,meta"},
|
||||
{"rerank.results", "index,relevance_score,document"},
|
||||
{"rerankv2", "id,results,meta"},
|
||||
{"rerankv2.results", "index,relevance_score,relevant_spans"},
|
||||
{"classify", "id,classifications,meta"},
|
||||
{"classify.classifications", "id,input,prediction,predictions,confidence,confidences,labels,classification_type"},
|
||||
{"create-dataset", "id"},
|
||||
{"list-datasets", "datasets"},
|
||||
{"get-dataset", "dataset"},
|
||||
{"get-dataset-usage", "organization_usage"},
|
||||
{"Dataset", "id,name,created_at,updated_at,dataset_type,validation_status,validation_error,validation_warnings,dataset_parts,required_fields,preserve_fields"},
|
||||
{"DatasetPart", "id,name,url,index,size_bytes,num_rows,original_url,samples"},
|
||||
{"summarize", "id,summary,meta"},
|
||||
{"tokenize", "tokens,token_strings,meta"},
|
||||
{"detokenize", "text,meta"},
|
||||
{"ListConnectorsResponse", "connectors,total_count"},
|
||||
{"Connector", "id,organization_id,name,url,created_at,updated_at,excludes,auth_type,oauth,auth_status,active,continue_on_failure"},
|
||||
{"CreateConnectorResponse", "connector"},
|
||||
{"GetConnectorResponse", "connector"},
|
||||
{"UpdateConnectorResponse", "connector"},
|
||||
{"OAuthAuthorizeResponse", "redirect_url"},
|
||||
{"GetModelResponse", "name,is_deprecated,endpoints,finetuned,context_length,tokenizer_url,default_endpoints,features,sampling_defaults"},
|
||||
{"ListModelsResponse", "models,next_page_token"},
|
||||
{"checkAPIKey", "valid,organization_id,owner_id"},
|
||||
{"ListFinetunedModelsResponse", "finetuned_models,next_page_token,total_size"},
|
||||
{"FinetunedModel", "id,name,creator_id,organization_id,settings,status,created_at,updated_at,completed_at,last_used"},
|
||||
{"CreateFinetunedModelResponse", "finetuned_model"},
|
||||
{"UpdateFinetunedModelResponse", "finetuned_model"},
|
||||
{"GetFinetunedModelResponse", "finetuned_model"},
|
||||
{"ListEventsResponse", "events,next_page_token,total_size"},
|
||||
{"Event", "user_id,status,created_at"},
|
||||
{"ListTrainingStepMetricsResponse", "step_metrics,next_page_token"},
|
||||
{"TrainingStepMetrics", "created_at,step_number,metrics"},
|
||||
};
|
||||
// SCHEMAS-END
|
||||
|
||||
const CallSpec& Calls::spec_of(Call call) {
|
||||
for (const CallSpec& s : kCalls)
|
||||
if (s.call == call) return s;
|
||||
throw llm::ApiError::invalid("cohere", "unknown call");
|
||||
}
|
||||
|
||||
clients::Method Calls::method_of(Call call) {
|
||||
const char* m = spec_of(call).http_method;
|
||||
if (!std::strcmp(m, "GET")) return clients::Method::kGet;
|
||||
if (!std::strcmp(m, "POST")) return clients::Method::kPost;
|
||||
if (!std::strcmp(m, "PATCH")) return clients::Method::kPatch;
|
||||
return clients::Method::kDelete;
|
||||
}
|
||||
|
||||
std::size_t Calls::count() { return sizeof(kCalls) / sizeof(kCalls[0]); }
|
||||
|
||||
const CallSpec& Calls::at(std::size_t index) {
|
||||
if (index >= count()) throw llm::ApiError::invalid("cohere", "call index out of range");
|
||||
return kCalls[index];
|
||||
}
|
||||
|
||||
std::size_t Calls::schema_count() { return sizeof(kSchemas) / sizeof(kSchemas[0]); }
|
||||
|
||||
const SchemaFields& Calls::schema_at(std::size_t index) {
|
||||
if (index >= schema_count()) throw llm::ApiError::invalid("cohere", "schema index out of range");
|
||||
return kSchemas[index];
|
||||
}
|
||||
|
||||
namespace {
|
||||
std::set<std::string> split(const char* list) {
|
||||
std::set<std::string> out;
|
||||
std::string cur;
|
||||
for (const char* p = list;; ++p) {
|
||||
if (*p == ',' || *p == '\0') {
|
||||
if (!cur.empty()) out.insert(cur);
|
||||
cur.clear();
|
||||
if (*p == '\0') break;
|
||||
} else {
|
||||
cur += *p;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::string Calls::render(Call call, const std::string& id,
|
||||
const Query& query) {
|
||||
const CallSpec& s = spec_of(call);
|
||||
std::string path = s.path;
|
||||
auto open = path.find('{');
|
||||
if (open != std::string::npos) {
|
||||
auto close = path.find('}', open);
|
||||
if (!llm::Service::valid_id(id)) throw llm::ApiError::invalid("cohere", "bad id");
|
||||
path.replace(open, close - open + 1, id);
|
||||
} else if (!id.empty()) {
|
||||
throw llm::ApiError::invalid("cohere", "this call takes no id");
|
||||
}
|
||||
std::set<std::string> allowed = split(s.query_params);
|
||||
std::string qs;
|
||||
for (const auto& [k, v] : query) {
|
||||
if (!allowed.count(k)) throw llm::ApiError::invalid("cohere", "query parameter not in the call table: " + k);
|
||||
qs += qs.empty() ? '?' : '&';
|
||||
qs += clients::Url::encode_component(k) + "=" + clients::Url::encode_component(v);
|
||||
}
|
||||
return path + qs;
|
||||
}
|
||||
|
||||
void Calls::check_body(Call call, const clients::body::Json& body) {
|
||||
if (!body.is_object()) throw llm::ApiError::invalid("cohere", "body must be a JSON object");
|
||||
std::set<std::string> allowed = split(spec_of(call).body_fields);
|
||||
for (const auto& [k, v] : body.as_object())
|
||||
if (!allowed.count(k)) throw llm::ApiError::invalid("cohere", "field not in the call table: " + k);
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+66
@@ -0,0 +1,66 @@
|
||||
#pragma once
|
||||
|
||||
// The ONE table of REST calls this module makes. tools/check_spec.py checks
|
||||
// every row against Cohere's OpenAPI document (cohere-openapi.yaml) and fails
|
||||
// on any spec operation that has no row. Code builds every path, query string
|
||||
// and body-field check from this table, never from a literal.
|
||||
|
||||
#include <cstddef>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
#include <string>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "clients/request.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
enum class Call {
|
||||
kChatV1, kChat, kParse, kGenerate, kEmbedV1, kEmbed, kTranscribe,
|
||||
kBatchList, kBatchCreate, kBatchGet, kBatchCancel,
|
||||
kEmbedJobCreate, kEmbedJobList, kEmbedJobGet, kEmbedJobCancel,
|
||||
kRerankV1, kRerank, kClassify,
|
||||
kDatasetCreate, kDatasetList, kDatasetUsage, kDatasetGet, kDatasetDelete,
|
||||
kSummarize, kTokenize, kDetokenize,
|
||||
kConnectorList, kConnectorCreate, kConnectorGet, kConnectorUpdate, kConnectorDelete,
|
||||
kConnectorAuthorize,
|
||||
kModelGet, kModelList, kCheckApiKey,
|
||||
kFinetuneList, kFinetuneCreate, kFinetuneUpdate, kFinetuneGet, kFinetuneDelete,
|
||||
kFinetuneEvents, kFinetuneMetrics,
|
||||
};
|
||||
|
||||
struct CallSpec {
|
||||
Call call;
|
||||
const char* http_method; // GET POST PATCH DELETE
|
||||
const char* path; // from the host; one {placeholder} at most
|
||||
const char* operation_id; // operationId in the OpenAPI document
|
||||
const char* body_fields; // comma list: request body fields we send (JSON or multipart)
|
||||
const char* query_params; // comma list: query parameters we send
|
||||
const char* response; // component schema name, or the operationId of an inline 200 body
|
||||
};
|
||||
|
||||
struct SchemaFields {
|
||||
const char* schema; // component name, operationId (inline 200 body), optionally ".a.b" property path
|
||||
const char* fields; // comma list: fields we read
|
||||
};
|
||||
|
||||
using Query = std::vector<std::pair<std::string, std::string>>; // repeat a name for list values
|
||||
|
||||
struct Calls {
|
||||
static const CallSpec& spec_of(Call call);
|
||||
static clients::Method method_of(Call call);
|
||||
static std::size_t count();
|
||||
static const CallSpec& at(std::size_t index);
|
||||
static std::size_t schema_count();
|
||||
static const SchemaFields& schema_at(std::size_t index);
|
||||
// Path with the placeholder replaced by a validated id and the query
|
||||
// parameters appended (percent-encoded). A query name that is not in the
|
||||
// row, or an id that is not a valid path segment, throws ApiError::invalid.
|
||||
static std::string render(Call call, const std::string& id = "",
|
||||
const Query& query = {});
|
||||
// Throws ApiError::invalid when `body` has a field the row does not declare.
|
||||
static void check_body(Call call, const clients::body::Json& body);
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+37
@@ -0,0 +1,37 @@
|
||||
#pragma once
|
||||
|
||||
// Static ceilings for the Cohere module. Never autotuned. "Documented" caps
|
||||
// come from cohere-openapi.yaml (see SPEC.md ## Sources); the rest are ours.
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
struct CohereCaps {
|
||||
// Documented (OpenAPI constraints).
|
||||
static constexpr std::size_t kMaxEmbedTexts = 96; // /v2/embed texts maxItems
|
||||
static constexpr std::size_t kMaxClassifyInputs = 96; // /v1/classify inputs maxItems
|
||||
static constexpr std::size_t kMaxTokenizeChars = 65536; // /v1/tokenize text maxLength
|
||||
static constexpr std::size_t kMaxDetokenizeTokens = 65536;
|
||||
static constexpr std::size_t kMaxStopSequences = 5; // chat stop_sequences ("up to 5")
|
||||
static constexpr int kMaxTopK = 500;
|
||||
static constexpr int kMaxPriority = 999;
|
||||
static constexpr int kMaxPageSize = 1000; // batches, models, finetuning pages
|
||||
// Our own caps.
|
||||
static constexpr std::size_t kMaxMessages = 4096;
|
||||
static constexpr std::size_t kMaxTools = 256;
|
||||
static constexpr std::size_t kMaxDocuments = 1000; // chat documents, rerank documents
|
||||
static constexpr std::size_t kMaxEmbedInputs = 96;
|
||||
static constexpr std::size_t kMaxStreamEvents = 200000;
|
||||
static constexpr std::size_t kMaxListPages = 20; // list_all page walks
|
||||
static constexpr std::size_t kMaxItemsReturned = 5000;
|
||||
static constexpr std::size_t kMaxUploadBytes = 256u * 1024u * 1024u; // dataset parts
|
||||
static constexpr std::size_t kMaxFilenameBytes = 512;
|
||||
static constexpr std::size_t kMaxIdChars = 256;
|
||||
static constexpr std::size_t kMaxNameChars = 256;
|
||||
static constexpr std::size_t kMaxModelChars = 128;
|
||||
static constexpr int kDatasetListMaxLimit = 1000;
|
||||
static constexpr int kConnectorListMaxLimit = 1000;
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+43
@@ -0,0 +1,43 @@
|
||||
#include "vendors/cohere/catalog.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
namespace {
|
||||
constexpr ModelName kModels[] = {
|
||||
{"command-a-plus-05-2026", ModelKind::kChat, ModelStatus::kLive, "MoE, text and images, 128k context, 64k output"},
|
||||
{"command-a-03-2025", ModelKind::kChat, ModelStatus::kLive, "text, 256k context, 8k output, tool use and RAG"},
|
||||
{"command-a-reasoning-08-2025", ModelKind::kChat, ModelStatus::kLive, "reasoning model"},
|
||||
{"command-a-vision-07-2025", ModelKind::kChat, ModelStatus::kLive, "image input"},
|
||||
{"command-a-translate-08-2025", ModelKind::kChat, ModelStatus::kLive, "translation"},
|
||||
{"command-r7b-12-2024", ModelKind::kChat, ModelStatus::kLive, "small, 128k context, 4k output"},
|
||||
{"command-r-plus-08-2024", ModelKind::kChat, ModelStatus::kLive, "128k context, 4k output"},
|
||||
{"command-r-08-2024", ModelKind::kChat, ModelStatus::kLive, "128k context, 4k output"},
|
||||
{"command-r-03-2024", ModelKind::kChat, ModelStatus::kDeprecated, "deprecated Sept 15, 2025"},
|
||||
{"command-r-plus-04-2024", ModelKind::kChat, ModelStatus::kDeprecated, "deprecated Sept 15, 2025"},
|
||||
{"embed-v5.0-pro", ModelKind::kEmbed, ModelStatus::kLive, "text, images, mixed; dimensions 256..2048 (default 2048)"},
|
||||
{"embed-v5.0-fast", ModelKind::kEmbed, ModelStatus::kLive, "faster, lighter embed-v5.0-pro"},
|
||||
{"embed-v4.0", ModelKind::kEmbed, ModelStatus::kLive, "text and images"},
|
||||
{"rerank-v5.0-pro", ModelKind::kRerank, ModelStatus::kLive, "highest quality, 32k context, calibrated scores"},
|
||||
{"rerank-v5.0-fast", ModelKind::kRerank, ModelStatus::kLive, "low latency"},
|
||||
{"rerank-v4.0-pro", ModelKind::kRerank, ModelStatus::kLive, "listed on the models page"},
|
||||
{"rerank-v4.0-fast", ModelKind::kRerank, ModelStatus::kLive, "listed on the models page"},
|
||||
{"rerank-v3.5", ModelKind::kRerank, ModelStatus::kLive, "multilingual, listed on the models page"},
|
||||
};
|
||||
} // namespace
|
||||
|
||||
std::size_t ModelCatalog::count() { return sizeof(kModels) / sizeof(kModels[0]); }
|
||||
|
||||
const ModelName& ModelCatalog::at(std::size_t index) {
|
||||
if (index >= count()) throw llm::ApiError::invalid("cohere", "model index out of range");
|
||||
return kModels[index];
|
||||
}
|
||||
|
||||
ModelStatus ModelCatalog::status_of(const std::string& id) {
|
||||
for (const ModelName& m : kModels)
|
||||
if (id == m.id) return m.status;
|
||||
return ModelStatus::kUnknown;
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+31
@@ -0,0 +1,31 @@
|
||||
#pragma once
|
||||
|
||||
// Model names the docs page https://docs.cohere.com/docs/models lists (read
|
||||
// 2026-10-25). The live GET /v1/models list is the truth; this table only
|
||||
// explains names and gives the defaults the module uses.
|
||||
|
||||
#include <cstddef>
|
||||
#include <string>
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
enum class ModelKind { kChat, kEmbed, kRerank };
|
||||
enum class ModelStatus { kLive, kDeprecated, kUnknown };
|
||||
|
||||
struct ModelName {
|
||||
const char* id;
|
||||
ModelKind kind;
|
||||
ModelStatus status;
|
||||
const char* note;
|
||||
};
|
||||
|
||||
struct ModelCatalog {
|
||||
static std::size_t count();
|
||||
static const ModelName& at(std::size_t index);
|
||||
static ModelStatus status_of(const std::string& id);
|
||||
static const char* default_chat() { return "command-a-03-2025"; }
|
||||
static const char* default_embed() { return "embed-v4.0"; }
|
||||
static const char* default_rerank() { return "rerank-v3.5"; }
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+83
@@ -0,0 +1,83 @@
|
||||
#include "vendors/cohere/chat.h"
|
||||
|
||||
#include "vendors/cohere/caps.h"
|
||||
#include "vendors/cohere/rules.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
ChatResult Chat::create(const ChatParams& params) {
|
||||
Json doc = client_->post(Calls::render(Call::kChat), params.to_json(false));
|
||||
return ChatResult::parse(doc);
|
||||
}
|
||||
|
||||
ChatResult Chat::stream(const ChatParams& params, const DeltaSink& on_delta) {
|
||||
Json body = params.to_json(true);
|
||||
ChatStreamAccumulator acc;
|
||||
bool stopped = false;
|
||||
std::size_t events = 0;
|
||||
client_->stream(Calls::render(Call::kChat), body, [&](const llm::SseEvent& ev) {
|
||||
if (++events > CohereCaps::kMaxStreamEvents)
|
||||
throw llm::ApiError("cohere", 502, "STREAM_TOO_LONG", "stream over the event cap");
|
||||
StreamDelta d = acc.feed(ev.data);
|
||||
if (on_delta && !on_delta(d)) {
|
||||
stopped = true;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
if (!stopped && !acc.done())
|
||||
throw llm::ApiError("cohere", 502, "STREAM_TRUNCATED", "stream ended without message-end");
|
||||
return acc.result();
|
||||
}
|
||||
|
||||
Json Chat::legacy(const Json& body) {
|
||||
Calls::check_body(Call::kChatV1, body);
|
||||
Rules::require(body.find("message") && body.find("message")->is_string(), "message is required");
|
||||
const Json* s = body.find("stream");
|
||||
Rules::require(!s || (s->is_bool() && !s->as_bool()), "legacy chat streaming is not supported here");
|
||||
return client_->post(Calls::render(Call::kChatV1), body);
|
||||
}
|
||||
|
||||
Conversation& Conversation::system(const std::string& text) {
|
||||
params_.add(Messages::system(text));
|
||||
return *this;
|
||||
}
|
||||
Conversation& Conversation::user(const std::string& text) {
|
||||
params_.add(Messages::user(text));
|
||||
return *this;
|
||||
}
|
||||
Conversation& Conversation::tool_result(const std::string& call_id, const std::string& content) {
|
||||
params_.add(Messages::tool(call_id, content));
|
||||
return *this;
|
||||
}
|
||||
|
||||
ChatResult Conversation::send(Chat& chat) {
|
||||
ChatResult r = chat.create(params_);
|
||||
params_.add(r.history_message());
|
||||
return r;
|
||||
}
|
||||
|
||||
ChatResult Conversation::send_stream(Chat& chat, const Chat::DeltaSink& on_delta) {
|
||||
ChatResult r = chat.stream(params_, on_delta);
|
||||
params_.add(r.history_message());
|
||||
return r;
|
||||
}
|
||||
|
||||
ChatResult Conversation::run_tools(Chat& chat, const ToolHandler& handler, int max_rounds) {
|
||||
Rules::require(max_rounds >= 1 && max_rounds <= kMaxToolRounds, "max_rounds must be 1..16");
|
||||
Rules::require(!params_.tools.empty(), "run_tools needs tools");
|
||||
ChatResult r;
|
||||
for (int round = 0; round < max_rounds; ++round) {
|
||||
r = send(chat);
|
||||
if (r.tool_calls.empty()) return r;
|
||||
for (const ToolCallRecord& call : r.tool_calls) {
|
||||
if (!call.arguments_valid())
|
||||
throw llm::ApiError("cohere", 502, "BAD_TOOL_ARGUMENTS",
|
||||
"tool call '" + call.name + "' has invalid JSON arguments");
|
||||
tool_result(call.id, handler(call.name, call.parsed(), call.id));
|
||||
}
|
||||
}
|
||||
throw llm::ApiError::invalid("cohere", "tool loop hit the round cap with calls still pending");
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+56
@@ -0,0 +1,56 @@
|
||||
#pragma once
|
||||
|
||||
// Chat v2 (POST /v2/chat) with streaming, Conversation (history keeper and
|
||||
// tool loop) and the legacy v1 chat (POST /v1/chat, still in the spec).
|
||||
|
||||
#include <functional>
|
||||
|
||||
#include "vendors/cohere/calls.h"
|
||||
#include "vendors/cohere/chat_params.h"
|
||||
#include "vendors/cohere/chat_result.h"
|
||||
#include "llm/api_client.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
class Chat : public llm::Service {
|
||||
public:
|
||||
explicit Chat(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
|
||||
ChatResult create(const ChatParams& params);
|
||||
|
||||
using DeltaSink = std::function<bool(const StreamDelta&)>; // false: stop early
|
||||
// Streams events; returns the folded result. A stream that ends without
|
||||
// message-end (and was not stopped by the sink) throws 502 STREAM_TRUNCATED.
|
||||
ChatResult stream(const ChatParams& params, const DeltaSink& on_delta);
|
||||
|
||||
// Legacy v1 chat. `body` fields must be in the call table; `message` and a
|
||||
// non-streaming request are required (stream:true is rejected, v1 streams use a different event format).
|
||||
Json legacy(const Json& body);
|
||||
};
|
||||
|
||||
class Conversation {
|
||||
public:
|
||||
static constexpr int kMaxToolRounds = 16;
|
||||
|
||||
explicit Conversation(ChatParams base) : params_(std::move(base)) {}
|
||||
|
||||
Conversation& system(const std::string& text);
|
||||
Conversation& user(const std::string& text);
|
||||
Conversation& tool_result(const std::string& call_id, const std::string& content);
|
||||
|
||||
ChatResult send(Chat& chat);
|
||||
ChatResult send_stream(Chat& chat, const Chat::DeltaSink& on_delta);
|
||||
|
||||
// handler(tool name, parsed arguments, call id) -> tool output text. Loops
|
||||
// while the model asks for tools, at most `max_rounds` requests (<= 16).
|
||||
using ToolHandler = std::function<std::string(const std::string&, const Json&, const std::string&)>;
|
||||
ChatResult run_tools(Chat& chat, const ToolHandler& handler, int max_rounds = 8);
|
||||
|
||||
const ChatParams& params() const { return params_; }
|
||||
ChatParams& params() { return params_; }
|
||||
|
||||
private:
|
||||
ChatParams params_;
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+206
@@ -0,0 +1,206 @@
|
||||
#include "vendors/cohere/chat_model.h"
|
||||
|
||||
#include <map>
|
||||
|
||||
#include "vendors/cohere/platform.h"
|
||||
#include "vendors/cohere/rules.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
CohereChatModel::CohereChatModel(std::shared_ptr<Chat> chat, std::string default_model)
|
||||
: chat_(std::move(chat)), default_model_(std::move(default_model)) {}
|
||||
|
||||
std::shared_ptr<CohereChatModel> CohereChatModel::from_env(const std::string& default_model) {
|
||||
return std::make_shared<CohereChatModel>(std::make_shared<Chat>(Platform::from_env().client), default_model);
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
Json image_part_of(const llm::ImageData& img) {
|
||||
if (!img.url.empty()) return Messages::image_part(img.url);
|
||||
Rules::require(!img.bytes.empty(), "image has neither url nor bytes");
|
||||
Rules::require(!img.mime.empty(), "image bytes need a mime type");
|
||||
return Messages::image_part("data:" + img.mime + ";base64," + security::crypto::Base64::encode(img.bytes));
|
||||
}
|
||||
|
||||
void append(ChatParams& params, const llm::Message& m) {
|
||||
switch (m.role) {
|
||||
case llm::Role::kSystem:
|
||||
params.add(Messages::system(m.text()));
|
||||
return;
|
||||
case llm::Role::kUser: {
|
||||
bool has_image = false;
|
||||
for (const llm::ContentPart& p : m.parts) has_image = has_image || p.image.has_value();
|
||||
if (!has_image) {
|
||||
params.add(Messages::user(m.text()));
|
||||
return;
|
||||
}
|
||||
std::vector<Json> parts;
|
||||
for (const llm::ContentPart& p : m.parts) {
|
||||
if (!p.text.empty()) parts.push_back(Messages::text_part(p.text));
|
||||
if (p.image) parts.push_back(image_part_of(*p.image));
|
||||
}
|
||||
params.add(Messages::user_parts(parts));
|
||||
return;
|
||||
}
|
||||
case llm::Role::kAssistant: {
|
||||
std::string thinking;
|
||||
Json calls = Json::array();
|
||||
for (const llm::ContentPart& p : m.parts) {
|
||||
thinking += p.reasoning;
|
||||
if (p.tool_call)
|
||||
calls.push(Messages::tool_call(p.tool_call->id, p.tool_call->name,
|
||||
p.tool_call->arguments.is_null() ? "{}" : p.tool_call->arguments.dump()));
|
||||
}
|
||||
params.add(Messages::assistant(m.text(), "", calls, thinking));
|
||||
return;
|
||||
}
|
||||
case llm::Role::kTool: {
|
||||
bool any = false;
|
||||
for (const llm::ContentPart& p : m.parts)
|
||||
if (p.tool_result) {
|
||||
params.add(Messages::tool(p.tool_result->call_id, p.tool_result->content));
|
||||
any = true;
|
||||
}
|
||||
Rules::require(any, "a tool message needs a tool result part");
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
llm::Usage usage_of(const TokenUsage& u) {
|
||||
llm::Usage out;
|
||||
out.input_tokens = static_cast<int>(u.input_tokens ? u.input_tokens : u.billed_input_tokens);
|
||||
out.output_tokens = static_cast<int>(u.output_tokens ? u.output_tokens : u.billed_output_tokens);
|
||||
out.total_tokens = out.input_tokens + out.output_tokens;
|
||||
out.cached_tokens = static_cast<int>(u.cached_tokens);
|
||||
return out;
|
||||
}
|
||||
|
||||
Json arguments_of(const ToolCallRecord& call) {
|
||||
if (call.arguments_valid()) return call.parsed();
|
||||
Json bad = Json::object();
|
||||
bad.set("_invalid_json", Json(call.arguments));
|
||||
return bad;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string CohereChatModel::neutral_finish(const std::string& r) {
|
||||
if (r == "COMPLETE" || r == "STOP_SEQUENCE") return "stop";
|
||||
if (r == "MAX_TOKENS") return "length";
|
||||
if (r == "TOOL_CALL") return "tool_calls";
|
||||
if (r == "ERROR") return "error";
|
||||
if (r == "TIMEOUT") return "timeout";
|
||||
return r;
|
||||
}
|
||||
|
||||
ChatParams CohereChatModel::to_params(const llm::ChatRequest& r) const {
|
||||
ChatParams p;
|
||||
p.model = r.model.empty() ? default_model_ : r.model;
|
||||
if (!r.system.empty()) p.add(Messages::system(r.system));
|
||||
for (const llm::Message& m : r.messages) append(p, m);
|
||||
for (const llm::ToolSpec& t : r.tools) {
|
||||
ToolDef d;
|
||||
d.name = t.name;
|
||||
d.description = t.description;
|
||||
d.parameters = t.parameters;
|
||||
p.tools.push_back(d);
|
||||
}
|
||||
if (r.tool_choice == "required") p.tool_choice = "REQUIRED";
|
||||
else if (r.tool_choice == "none") p.tool_choice = "NONE";
|
||||
else if (!r.tool_choice.empty() && r.tool_choice != "auto")
|
||||
Rules::fail("Cohere cannot force a named tool; use required, none or auto");
|
||||
p.max_tokens = r.max_tokens;
|
||||
p.temperature = r.temperature;
|
||||
p.p = r.top_p;
|
||||
p.stop = r.stop;
|
||||
if (r.json_mode || r.response_schema.is_object()) {
|
||||
p.json_mode = true;
|
||||
if (r.response_schema.is_object()) p.json_schema = r.response_schema;
|
||||
}
|
||||
if (!r.reasoning_effort.empty()) {
|
||||
bool off = r.reasoning_effort == "none" || r.reasoning_effort == "off" || r.reasoning_effort == "disabled";
|
||||
p.thinking = off ? Thinking::kDisabled : Thinking::kEnabled;
|
||||
}
|
||||
if (r.extra.is_object()) {
|
||||
Json rest = Json::object();
|
||||
for (const auto& [k, v] : r.extra.as_object()) {
|
||||
if (k == "thinking" && v.is_object()) {
|
||||
std::string type = J::str(v, "type");
|
||||
if (type == "enabled") p.thinking = Thinking::kEnabled;
|
||||
else if (type == "disabled") p.thinking = Thinking::kDisabled;
|
||||
else if (!type.empty()) Rules::fail("thinking.type must be enabled or disabled");
|
||||
if (v.find("token_budget") && v.find("token_budget")->is_number()) {
|
||||
if (type.empty() && p.thinking == Thinking::kDefault) p.thinking = Thinking::kEnabled;
|
||||
p.thinking_budget = v.find("token_budget")->as_int();
|
||||
}
|
||||
} else {
|
||||
rest.set(k, v);
|
||||
}
|
||||
}
|
||||
p.extra = rest;
|
||||
}
|
||||
return p;
|
||||
}
|
||||
|
||||
llm::ChatResponse CohereChatModel::to_response(const ChatResult& result) {
|
||||
llm::ChatResponse out;
|
||||
out.id = result.id;
|
||||
out.message.role = llm::Role::kAssistant;
|
||||
if (!result.thinking.empty()) {
|
||||
llm::ContentPart r;
|
||||
r.reasoning = result.thinking;
|
||||
out.message.parts.push_back(r);
|
||||
}
|
||||
if (!result.text.empty() || result.tool_calls.empty())
|
||||
out.message.parts.push_back(llm::ContentPart::of_text(result.text));
|
||||
for (const ToolCallRecord& call : result.tool_calls) {
|
||||
llm::ToolCall tc;
|
||||
tc.id = call.id;
|
||||
tc.name = call.name;
|
||||
tc.arguments = arguments_of(call);
|
||||
out.message.parts.push_back(llm::ContentPart::of_call(tc));
|
||||
}
|
||||
out.finish_reason = neutral_finish(result.finish_reason);
|
||||
out.usage = usage_of(result.usage);
|
||||
out.raw = result.raw;
|
||||
return out;
|
||||
}
|
||||
|
||||
llm::ChatResponse CohereChatModel::generate(const llm::ChatRequest& request) {
|
||||
ChatParams params = to_params(request);
|
||||
llm::ChatResponse out = to_response(chat_->create(params));
|
||||
out.model = params.model;
|
||||
return out;
|
||||
}
|
||||
|
||||
void CohereChatModel::stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
|
||||
ChatParams params = to_params(request);
|
||||
chat_->stream(params, [&](const StreamDelta& d) {
|
||||
if (!d.text.empty() || !d.thinking.empty()) {
|
||||
llm::ChatChunk c;
|
||||
c.text_delta = d.text;
|
||||
c.reasoning_delta = d.thinking;
|
||||
if (!on_chunk(c)) return false;
|
||||
}
|
||||
if (d.type == "tool-call-start" || d.type == "tool-call-delta") {
|
||||
llm::ChatChunk c;
|
||||
c.tool_call_id = d.tool_id;
|
||||
c.tool_call_name = d.tool_name;
|
||||
c.tool_args_delta = d.args_delta;
|
||||
if (!on_chunk(c)) return false;
|
||||
}
|
||||
if (d.type == "message-end") {
|
||||
llm::ChatChunk c;
|
||||
c.finish_reason = neutral_finish(d.finish_reason);
|
||||
if (d.has_usage) c.usage = usage_of(d.usage);
|
||||
if (!on_chunk(c)) return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+47
@@ -0,0 +1,47 @@
|
||||
#pragma once
|
||||
|
||||
// CohereChatModel: the llm::ChatModel seam over Chat v2.
|
||||
//
|
||||
// Mapping notes (each one is a decision):
|
||||
// - ChatRequest.system becomes a leading system message.
|
||||
// - tool_choice: "required" -> REQUIRED, "none" -> NONE, ""/"auto" -> omitted;
|
||||
// a tool NAME is rejected loudly (Cohere cannot force one named tool).
|
||||
// - json_mode -> response_format json_object; response_schema -> the same plus
|
||||
// json_schema (Cohere's json_object accepts a schema).
|
||||
// - reasoning_effort "none"/"off"/"disabled" -> thinking disabled; any other
|
||||
// non-empty value -> thinking enabled (Cohere has no effort levels, only
|
||||
// token_budget, which ChatRequest.extra {"thinking":{"token_budget":N}} sets).
|
||||
// - Assistant reasoning parts are sent back as thinking content.
|
||||
// - Tool arguments that are not valid JSON arrive as {"_invalid_json": "<raw>"}.
|
||||
// - finish_reason: COMPLETE and STOP_SEQUENCE -> stop, MAX_TOKENS -> length,
|
||||
// TOOL_CALL -> tool_calls, ERROR -> error, TIMEOUT -> timeout.
|
||||
// - Usage: counts come from usage.tokens (actual), billed_units is the fallback;
|
||||
// cached_tokens from usage.cached_tokens.
|
||||
|
||||
#include <memory>
|
||||
|
||||
#include "vendors/cohere/chat.h"
|
||||
#include "llm/chat_types.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
class CohereChatModel : public llm::ChatModel {
|
||||
public:
|
||||
explicit CohereChatModel(std::shared_ptr<Chat> chat, std::string default_model = "command-a-03-2025");
|
||||
static std::shared_ptr<CohereChatModel> from_env(const std::string& default_model = "command-a-03-2025");
|
||||
|
||||
std::string vendor() const override { return "cohere"; }
|
||||
llm::ChatResponse generate(const llm::ChatRequest& request) override;
|
||||
void stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
|
||||
|
||||
ChatParams to_params(const llm::ChatRequest& request) const;
|
||||
static llm::ChatResponse to_response(const ChatResult& result);
|
||||
static std::string neutral_finish(const std::string& finish_reason);
|
||||
|
||||
private:
|
||||
std::shared_ptr<Chat> chat_;
|
||||
std::string default_model_;
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+205
@@ -0,0 +1,205 @@
|
||||
#include "vendors/cohere/chat_params.h"
|
||||
|
||||
#include "vendors/cohere/calls.h"
|
||||
#include "vendors/cohere/caps.h"
|
||||
#include "vendors/cohere/rules.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
Json Messages::text_part(const std::string& text) {
|
||||
Json p = Json::object();
|
||||
p.set("type", Json("text"));
|
||||
p.set("text", Json(text));
|
||||
return p;
|
||||
}
|
||||
|
||||
Json Messages::image_part(const std::string& url, const std::string& detail) {
|
||||
Rules::image_url(url);
|
||||
if (!detail.empty()) Rules::one_of(detail, {"auto", "low", "high"}, "image detail");
|
||||
Json u = Json::object();
|
||||
u.set("url", Json(url));
|
||||
if (!detail.empty()) u.set("detail", Json(detail));
|
||||
Json p = Json::object();
|
||||
p.set("type", Json("image_url"));
|
||||
p.set("image_url", u);
|
||||
return p;
|
||||
}
|
||||
|
||||
static Json message(const char* role) {
|
||||
Json m = Json::object();
|
||||
m.set("role", Json(role));
|
||||
return m;
|
||||
}
|
||||
|
||||
Json Messages::system(const std::string& text) {
|
||||
Json m = message("system");
|
||||
m.set("content", Json(text));
|
||||
return m;
|
||||
}
|
||||
|
||||
Json Messages::user(const std::string& text) {
|
||||
Json m = message("user");
|
||||
m.set("content", Json(text));
|
||||
return m;
|
||||
}
|
||||
|
||||
Json Messages::user_parts(const std::vector<Json>& parts) {
|
||||
Rules::require(!parts.empty(), "a user message needs at least one part");
|
||||
Json arr = Json::array();
|
||||
for (const Json& p : parts) arr.push(p);
|
||||
Json m = message("user");
|
||||
m.set("content", arr);
|
||||
return m;
|
||||
}
|
||||
|
||||
Json Messages::assistant(const std::string& text, const std::string& tool_plan, const Json& tool_calls,
|
||||
const std::string& thinking) {
|
||||
Json m = message("assistant");
|
||||
Json parts = Json::array();
|
||||
if (!thinking.empty()) {
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("thinking"));
|
||||
t.set("thinking", Json(thinking));
|
||||
parts.push(t);
|
||||
}
|
||||
if (!text.empty()) parts.push(text_part(text));
|
||||
if (!parts.as_array().empty()) m.set("content", parts);
|
||||
if (!tool_plan.empty()) m.set("tool_plan", Json(tool_plan));
|
||||
if (tool_calls.is_array() && !tool_calls.as_array().empty()) m.set("tool_calls", tool_calls);
|
||||
return m;
|
||||
}
|
||||
|
||||
Json Messages::tool(const std::string& call_id, const std::string& content) {
|
||||
Rules::require(!call_id.empty(), "tool_call_id must not be empty");
|
||||
Json m = message("tool");
|
||||
m.set("tool_call_id", Json(call_id));
|
||||
m.set("content", Json(content));
|
||||
return m;
|
||||
}
|
||||
|
||||
Json Messages::tool_call(const std::string& id, const std::string& name, const std::string& arguments_json) {
|
||||
Json f = Json::object();
|
||||
f.set("name", Json(name));
|
||||
f.set("arguments", Json(arguments_json));
|
||||
Json c = Json::object();
|
||||
c.set("id", Json(id));
|
||||
c.set("type", Json("function"));
|
||||
c.set("function", f);
|
||||
return c;
|
||||
}
|
||||
|
||||
Json ToolDef::to_json() const {
|
||||
Rules::require(!name.empty() && name.size() <= 128, "tool name must be 1..128 chars");
|
||||
for (char c : name)
|
||||
Rules::require((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_' || c == '-',
|
||||
"tool name has invalid characters");
|
||||
Json f = Json::object();
|
||||
f.set("name", Json(name));
|
||||
if (!description.empty()) f.set("description", Json(description));
|
||||
f.set("parameters", parameters.is_object() ? parameters : Json::object());
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("function"));
|
||||
t.set("function", f);
|
||||
return t;
|
||||
}
|
||||
|
||||
ChatParams ChatParams::prompt(const std::string& text) {
|
||||
ChatParams p;
|
||||
p.add(Messages::user(text));
|
||||
return p;
|
||||
}
|
||||
|
||||
ChatParams& ChatParams::add(Json message) {
|
||||
messages.push_back(std::move(message));
|
||||
return *this;
|
||||
}
|
||||
|
||||
void ChatParams::validate() const {
|
||||
Rules::model(model);
|
||||
Rules::require(!messages.empty(), "messages must not be empty");
|
||||
Rules::require(messages.size() <= CohereCaps::kMaxMessages, "too many messages");
|
||||
Rules::require(tools.size() <= CohereCaps::kMaxTools, "too many tools");
|
||||
Rules::require(documents.size() <= CohereCaps::kMaxDocuments, "too many documents");
|
||||
if (!citation_mode.empty())
|
||||
Rules::one_of(citation_mode, {"ENABLED", "DISABLED", "FAST", "ACCURATE", "OFF"}, "citation mode");
|
||||
if (!safety_mode.empty()) Rules::one_of(safety_mode, {"CONTEXTUAL", "STRICT", "OFF"}, "safety mode");
|
||||
if (!tool_choice.empty()) {
|
||||
Rules::one_of(tool_choice, {"REQUIRED", "NONE"}, "tool_choice");
|
||||
Rules::require(tool_choice != "REQUIRED" || !tools.empty(), "tool_choice REQUIRED needs tools");
|
||||
}
|
||||
Rules::require(!strict_tools || !tools.empty(), "strict_tools needs tools");
|
||||
Rules::require(!json_schema.is_object() || json_mode, "json_schema needs json_mode");
|
||||
if (max_tokens) Rules::require(*max_tokens >= 1, "max_tokens must be >= 1");
|
||||
Rules::stop(stop);
|
||||
if (temperature) Rules::temperature(*temperature);
|
||||
if (seed) Rules::require(*seed >= 0, "seed must be >= 0");
|
||||
if (frequency_penalty) Rules::range(*frequency_penalty, 0.0, 1.0, "frequency_penalty");
|
||||
if (presence_penalty) Rules::range(*presence_penalty, 0.0, 1.0, "presence_penalty");
|
||||
if (k) Rules::top_k(*k);
|
||||
if (p) Rules::top_p(*p);
|
||||
if (priority) Rules::priority(*priority);
|
||||
if (thinking_budget) {
|
||||
Rules::require(*thinking_budget >= 1, "thinking token budget must be >= 1");
|
||||
Rules::require(thinking == Thinking::kEnabled, "a thinking budget needs thinking enabled");
|
||||
}
|
||||
}
|
||||
|
||||
Json ChatParams::to_json(bool stream) const {
|
||||
validate();
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
Json msgs = Json::array();
|
||||
for (const Json& m : messages) msgs.push(m);
|
||||
b.set("messages", msgs);
|
||||
if (stream) b.set("stream", Json(true));
|
||||
if (!tools.empty()) {
|
||||
Json t = Json::array();
|
||||
for (const ToolDef& d : tools) t.push(d.to_json());
|
||||
b.set("tools", t);
|
||||
}
|
||||
if (strict_tools) b.set("strict_tools", Json(true));
|
||||
if (!documents.empty()) {
|
||||
Json d = Json::array();
|
||||
for (const Json& x : documents) d.push(x);
|
||||
b.set("documents", d);
|
||||
}
|
||||
if (!citation_mode.empty()) {
|
||||
Json c = Json::object();
|
||||
c.set("mode", Json(citation_mode));
|
||||
b.set("citation_options", c);
|
||||
}
|
||||
if (json_mode) {
|
||||
Json f = Json::object();
|
||||
f.set("type", Json("json_object"));
|
||||
if (json_schema.is_object()) f.set("json_schema", json_schema);
|
||||
b.set("response_format", f);
|
||||
}
|
||||
if (!safety_mode.empty()) b.set("safety_mode", Json(safety_mode));
|
||||
if (max_tokens) b.set("max_tokens", Json(*max_tokens));
|
||||
if (!stop.empty()) {
|
||||
Json s = Json::array();
|
||||
for (const std::string& x : stop) s.push(Json(x));
|
||||
b.set("stop_sequences", s);
|
||||
}
|
||||
if (temperature) b.set("temperature", Json(*temperature));
|
||||
if (seed) b.set("seed", Json(*seed));
|
||||
if (frequency_penalty) b.set("frequency_penalty", Json(*frequency_penalty));
|
||||
if (presence_penalty) b.set("presence_penalty", Json(*presence_penalty));
|
||||
if (k) b.set("k", Json(*k));
|
||||
if (p) b.set("p", Json(*p));
|
||||
if (logprobs) b.set("logprobs", Json(true));
|
||||
if (!tool_choice.empty()) b.set("tool_choice", Json(tool_choice));
|
||||
if (thinking != Thinking::kDefault) {
|
||||
Json t = Json::object();
|
||||
t.set("type", Json(thinking == Thinking::kEnabled ? "enabled" : "disabled"));
|
||||
if (thinking_budget) t.set("token_budget", Json(*thinking_budget));
|
||||
b.set("thinking", t);
|
||||
}
|
||||
if (priority) b.set("priority", Json(*priority));
|
||||
if (extra.is_object())
|
||||
for (const auto& [key, v] : extra.as_object()) b.set(key, v);
|
||||
Calls::check_body(Call::kChat, b);
|
||||
return b;
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+70
@@ -0,0 +1,70 @@
|
||||
#pragma once
|
||||
|
||||
// Chat v2 request: message builders (Messages), tool definitions and
|
||||
// ChatParams, which validates and serializes to the /v2/chat body.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/cohere/types.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
// Builders for the four v2 message roles. Content is a plain string or an
|
||||
// array of typed parts (text, image_url; assistant: text, thinking).
|
||||
struct Messages {
|
||||
static Json system(const std::string& text);
|
||||
static Json user(const std::string& text);
|
||||
static Json user_parts(const std::vector<Json>& parts);
|
||||
// tool_plan and tool_calls are omitted when empty.
|
||||
static Json assistant(const std::string& text, const std::string& tool_plan = "",
|
||||
const Json& tool_calls = Json::array(), const std::string& thinking = "");
|
||||
// Result of one tool call; `content` is text (the docs also allow document parts).
|
||||
static Json tool(const std::string& call_id, const std::string& content);
|
||||
static Json text_part(const std::string& text);
|
||||
static Json image_part(const std::string& url, const std::string& detail = "");
|
||||
static Json tool_call(const std::string& id, const std::string& name, const std::string& arguments_json);
|
||||
};
|
||||
|
||||
struct ToolDef {
|
||||
std::string name;
|
||||
std::string description;
|
||||
Json parameters = Json::object(); // JSON Schema
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
enum class Thinking { kDefault, kEnabled, kDisabled };
|
||||
|
||||
struct ChatParams {
|
||||
std::string model = "command-a-03-2025";
|
||||
std::vector<Json> messages;
|
||||
std::vector<ToolDef> tools;
|
||||
bool strict_tools = false;
|
||||
std::vector<Json> documents; // each a string or {"data":{...},"id":"..."}
|
||||
std::string citation_mode; // ENABLED DISABLED FAST ACCURATE OFF
|
||||
bool json_mode = false;
|
||||
Json json_schema; // null: none; else response_format.json_schema
|
||||
std::string safety_mode; // CONTEXTUAL STRICT OFF
|
||||
std::optional<int> max_tokens;
|
||||
std::vector<std::string> stop;
|
||||
std::optional<double> temperature;
|
||||
std::optional<long> seed;
|
||||
std::optional<double> frequency_penalty;
|
||||
std::optional<double> presence_penalty;
|
||||
std::optional<int> k;
|
||||
std::optional<double> p;
|
||||
bool logprobs = false;
|
||||
std::string tool_choice; // REQUIRED, NONE or empty
|
||||
Thinking thinking = Thinking::kDefault;
|
||||
std::optional<long> thinking_budget;
|
||||
std::optional<int> priority;
|
||||
Json extra; // merged into the body last; keys must exist in the call table
|
||||
|
||||
static ChatParams prompt(const std::string& text);
|
||||
ChatParams& add(Json message);
|
||||
void validate() const;
|
||||
Json to_json(bool stream = false) const;
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+132
@@ -0,0 +1,132 @@
|
||||
#include "vendors/cohere/chat_result.h"
|
||||
|
||||
#include "vendors/cohere/chat_params.h"
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
bool ToolCallRecord::arguments_valid() const {
|
||||
try {
|
||||
return Json::parse(arguments.empty() ? "{}" : arguments).is_object();
|
||||
} catch (const std::exception&) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
Json ToolCallRecord::parsed() const {
|
||||
try {
|
||||
Json j = Json::parse(arguments.empty() ? "{}" : arguments);
|
||||
if (j.is_object()) return j;
|
||||
} catch (const std::exception&) {
|
||||
}
|
||||
return Json::object();
|
||||
}
|
||||
|
||||
static ToolCallRecord tool_of(const Json& c) {
|
||||
ToolCallRecord t;
|
||||
t.id = J::str(c, "id");
|
||||
if (const Json* f = J::obj(c, "function")) {
|
||||
t.name = J::str(*f, "name");
|
||||
t.arguments = J::str(*f, "arguments");
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
ChatResult ChatResult::parse(const Json& doc) {
|
||||
ChatResult r;
|
||||
r.raw = doc;
|
||||
r.id = J::str(doc, "id");
|
||||
r.finish_reason = J::str(doc, "finish_reason");
|
||||
if (const Json* m = J::obj(doc, "message")) {
|
||||
if (const Json* content = J::arr(*m, "content"))
|
||||
for (const Json& part : content->as_array()) {
|
||||
if (J::str(part, "type") == "thinking") r.thinking += J::str(part, "thinking");
|
||||
else r.text += J::str(part, "text");
|
||||
}
|
||||
r.tool_plan = J::str(*m, "tool_plan");
|
||||
if (const Json* calls = J::arr(*m, "tool_calls"))
|
||||
for (const Json& c : calls->as_array()) r.tool_calls.push_back(tool_of(c));
|
||||
if (const Json* cites = J::arr(*m, "citations"))
|
||||
for (const Json& c : cites->as_array()) r.citations.push_back(c);
|
||||
}
|
||||
r.usage = TokenUsage::parse(J::obj(doc, "usage"));
|
||||
if (const Json* l = J::arr(doc, "logprobs")) r.logprobs = *l;
|
||||
return r;
|
||||
}
|
||||
|
||||
Json ChatResult::history_message() const {
|
||||
Json calls = Json::array();
|
||||
for (const ToolCallRecord& c : tool_calls) calls.push(Messages::tool_call(c.id, c.name, c.arguments));
|
||||
return Messages::assistant(text, tool_plan, calls, thinking);
|
||||
}
|
||||
|
||||
StreamDelta ChatStreamAccumulator::feed(const Json& ev) {
|
||||
StreamDelta d;
|
||||
d.type = J::str(ev, "type");
|
||||
d.index = static_cast<int>(J::num(ev, "index"));
|
||||
const Json* delta = J::obj(ev, "delta");
|
||||
const Json* msg = delta ? J::obj(*delta, "message") : nullptr;
|
||||
if (d.type == "message-start") {
|
||||
result_.id = J::str(ev, "id");
|
||||
} else if (d.type == "content-start" || d.type == "content-delta") {
|
||||
const Json* content = msg ? J::obj(*msg, "content") : nullptr;
|
||||
if (content) {
|
||||
d.text = J::str(*content, "text");
|
||||
d.thinking = J::str(*content, "thinking");
|
||||
result_.text += d.text;
|
||||
result_.thinking += d.thinking;
|
||||
}
|
||||
} else if (d.type == "tool-plan-delta") {
|
||||
if (msg) d.tool_plan = J::str(*msg, "tool_plan");
|
||||
result_.tool_plan += d.tool_plan;
|
||||
} else if (d.type == "tool-call-start") {
|
||||
if (const Json* c = msg ? J::obj(*msg, "tool_calls") : nullptr) {
|
||||
ToolCallRecord t = tool_of(*c);
|
||||
d.tool_start = true;
|
||||
d.tool_id = t.id;
|
||||
d.tool_name = t.name;
|
||||
d.args_delta = t.arguments;
|
||||
tools_[d.index] = t;
|
||||
}
|
||||
} else if (d.type == "tool-call-delta") {
|
||||
if (const Json* c = msg ? J::obj(*msg, "tool_calls") : nullptr)
|
||||
if (const Json* f = J::obj(*c, "function")) {
|
||||
d.args_delta = J::str(*f, "arguments");
|
||||
ToolCallRecord& t = tools_[d.index];
|
||||
t.arguments += d.args_delta;
|
||||
d.tool_id = t.id;
|
||||
d.tool_name = t.name;
|
||||
}
|
||||
} else if (d.type == "citation-start") {
|
||||
if (const Json* c = msg ? J::obj(*msg, "citations") : nullptr) {
|
||||
d.citation = *c;
|
||||
result_.citations.push_back(*c);
|
||||
}
|
||||
} else if (d.type == "message-end") {
|
||||
if (delta) {
|
||||
std::string error = J::str(*delta, "error");
|
||||
if (!error.empty()) throw llm::ApiError("cohere", 502, "STREAM_ERROR", error);
|
||||
d.finish_reason = J::str(*delta, "finish_reason");
|
||||
result_.finish_reason = d.finish_reason;
|
||||
if (const Json* u = J::obj(*delta, "usage")) {
|
||||
d.usage = TokenUsage::parse(u);
|
||||
d.has_usage = true;
|
||||
result_.usage = d.usage;
|
||||
}
|
||||
}
|
||||
done_ = true;
|
||||
} else if (d.type.empty()) {
|
||||
throw llm::ApiError("cohere", 502, "STREAM_ERROR", "stream event without a type");
|
||||
}
|
||||
// content-end, tool-call-end, citation-end and unknown future events carry nothing we keep.
|
||||
return d;
|
||||
}
|
||||
|
||||
ChatResult ChatStreamAccumulator::result() const {
|
||||
ChatResult r = result_;
|
||||
r.tool_calls.clear();
|
||||
for (const auto& [i, t] : tools_) r.tool_calls.push_back(t);
|
||||
return r;
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+68
@@ -0,0 +1,68 @@
|
||||
#pragma once
|
||||
|
||||
// Chat v2 response and the streaming accumulator. Stream events are the
|
||||
// `type` values of StreamedChatResponseV2 (message-start ... message-end).
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/cohere/types.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
struct ToolCallRecord {
|
||||
std::string id;
|
||||
std::string name;
|
||||
std::string arguments; // JSON text, as sent by the model
|
||||
bool arguments_valid() const;
|
||||
Json parsed() const; // {} when invalid
|
||||
};
|
||||
|
||||
struct ChatResult {
|
||||
std::string id;
|
||||
std::string finish_reason; // COMPLETE STOP_SEQUENCE MAX_TOKENS TOOL_CALL ERROR TIMEOUT
|
||||
std::string text; // text parts joined
|
||||
std::string thinking; // thinking parts joined
|
||||
std::string tool_plan;
|
||||
std::vector<ToolCallRecord> tool_calls;
|
||||
std::vector<Json> citations;
|
||||
TokenUsage usage;
|
||||
Json logprobs; // array or null
|
||||
Json raw;
|
||||
|
||||
static ChatResult parse(const Json& doc);
|
||||
// The assistant message to put back into the history for the next turn.
|
||||
Json history_message() const;
|
||||
};
|
||||
|
||||
struct StreamDelta {
|
||||
std::string type; // event type, e.g. "content-delta"
|
||||
std::string text;
|
||||
std::string thinking;
|
||||
std::string tool_plan;
|
||||
int index = 0;
|
||||
bool tool_start = false; // tool-call-start: id and name are set
|
||||
std::string tool_id;
|
||||
std::string tool_name;
|
||||
std::string args_delta;
|
||||
Json citation;
|
||||
std::string finish_reason; // message-end
|
||||
bool has_usage = false;
|
||||
TokenUsage usage;
|
||||
};
|
||||
|
||||
class ChatStreamAccumulator {
|
||||
public:
|
||||
// Throws llm::ApiError (502 STREAM_ERROR) on an error event or a message-end error.
|
||||
StreamDelta feed(const Json& event);
|
||||
bool done() const { return done_; }
|
||||
ChatResult result() const;
|
||||
|
||||
private:
|
||||
ChatResult result_;
|
||||
std::map<int, ToolCallRecord> tools_;
|
||||
bool done_ = false;
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+273
@@ -0,0 +1,273 @@
|
||||
#include "vendors/cohere/inference.h"
|
||||
|
||||
#include "vendors/cohere/calls.h"
|
||||
#include "vendors/cohere/caps.h"
|
||||
#include "vendors/cohere/rules.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
namespace {
|
||||
Json string_array(const std::vector<std::string>& v) {
|
||||
Json a = Json::array();
|
||||
for (const std::string& s : v) a.push(Json(s));
|
||||
return a;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
void EmbedParams::validate() const {
|
||||
Rules::model(model);
|
||||
Rules::one_of(input_type, {"search_document", "search_query", "classification", "clustering", "image"}, "input_type");
|
||||
Rules::require(!texts.empty() || !images.empty() || !inputs.empty(), "give texts, images or inputs");
|
||||
Rules::require(texts.size() <= CohereCaps::kMaxEmbedTexts, "at most 96 texts per call");
|
||||
Rules::require(images.size() <= CohereCaps::kMaxEmbedTexts, "at most 96 images per call");
|
||||
Rules::require(inputs.size() <= CohereCaps::kMaxEmbedInputs, "at most 96 inputs per call");
|
||||
for (const std::string& t : texts) Rules::require(!t.empty(), "texts must not be empty");
|
||||
for (const std::string& i : images) Rules::image_url(i);
|
||||
for (const std::string& t : embedding_types)
|
||||
Rules::one_of(t, {"float", "int8", "uint8", "binary", "ubinary", "base64"}, "embedding type");
|
||||
if (!truncate.empty()) Rules::one_of(truncate, {"NONE", "START", "END"}, "truncate");
|
||||
if (max_tokens) Rules::require(*max_tokens >= 1, "max_tokens must be >= 1");
|
||||
if (output_dimension) Rules::require(*output_dimension >= 1, "output_dimension must be >= 1");
|
||||
if (priority) Rules::priority(*priority);
|
||||
}
|
||||
|
||||
Json EmbedParams::to_json() const {
|
||||
validate();
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
b.set("input_type", Json(input_type));
|
||||
if (!texts.empty()) b.set("texts", string_array(texts));
|
||||
if (!images.empty()) b.set("images", string_array(images));
|
||||
if (!inputs.empty()) {
|
||||
Json a = Json::array();
|
||||
for (const Json& i : inputs) a.push(i);
|
||||
b.set("inputs", a);
|
||||
}
|
||||
if (!embedding_types.empty()) b.set("embedding_types", string_array(embedding_types));
|
||||
if (!truncate.empty()) b.set("truncate", Json(truncate));
|
||||
if (max_tokens) b.set("max_tokens", Json(*max_tokens));
|
||||
if (output_dimension) b.set("output_dimension", Json(*output_dimension));
|
||||
if (priority) b.set("priority", Json(*priority));
|
||||
return b;
|
||||
}
|
||||
|
||||
std::vector<std::vector<double>> EmbedResult::floats() const {
|
||||
std::vector<std::vector<double>> out;
|
||||
const Json* rows = nullptr;
|
||||
if (embeddings.is_array()) rows = &embeddings;
|
||||
else if (embeddings.is_object() && embeddings.find("float") && embeddings.find("float")->is_array())
|
||||
rows = embeddings.find("float");
|
||||
if (!rows) return out;
|
||||
for (const Json& row : rows->as_array()) {
|
||||
std::vector<double> v;
|
||||
if (row.is_array())
|
||||
for (const Json& x : row.as_array())
|
||||
if (x.is_number()) v.push_back(x.as_number());
|
||||
out.push_back(std::move(v));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
EmbedResult EmbedResult::parse(const Json& doc) {
|
||||
EmbedResult r;
|
||||
r.raw = doc;
|
||||
r.id = J::str(doc, "id");
|
||||
r.response_type = J::str(doc, "response_type");
|
||||
r.texts = J::strings(doc, "texts");
|
||||
if (const Json* e = doc.is_object() ? doc.find("embeddings") : nullptr) r.embeddings = *e;
|
||||
r.usage = TokenUsage::parse(J::obj(doc, "meta"));
|
||||
if (const Json* meta = J::obj(doc, "meta"))
|
||||
if (const Json* b = J::obj(*meta, "billed_units")) {
|
||||
r.usage.billed_input_tokens = J::num(*b, "input_tokens");
|
||||
r.usage.present = true;
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
void RerankParams::validate() const {
|
||||
Rules::model(model);
|
||||
Rules::require(!query.empty(), "query must not be empty");
|
||||
Rules::require(!documents.empty() && documents.size() <= CohereCaps::kMaxDocuments, "documents must be 1..1000");
|
||||
for (const std::string& d : documents) Rules::require(!d.empty(), "documents must not be empty");
|
||||
if (top_n) Rules::require(*top_n >= 1, "top_n must be >= 1");
|
||||
if (max_tokens_per_doc) Rules::require(*max_tokens_per_doc >= 1, "max_tokens_per_doc must be >= 1");
|
||||
if (!relevant_spans.empty()) Rules::one_of(relevant_spans, {"enabled", "disabled"}, "relevant_spans");
|
||||
Rules::require(relevant_spans.empty() || relevant_spans == "enabled" || (!spans_threshold && !max_spans),
|
||||
"span options need relevant_spans enabled");
|
||||
if (spans_threshold) Rules::range(*spans_threshold, 0.0, 1.0, "span threshold");
|
||||
if (max_spans) Rules::require(*max_spans >= 1, "max_spans must be >= 1");
|
||||
if (priority) Rules::priority(*priority);
|
||||
}
|
||||
|
||||
Json RerankParams::to_json() const {
|
||||
validate();
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
b.set("query", Json(query));
|
||||
b.set("documents", string_array(documents));
|
||||
if (top_n) b.set("top_n", Json(*top_n));
|
||||
if (max_tokens_per_doc) b.set("max_tokens_per_doc", Json(*max_tokens_per_doc));
|
||||
if (!relevant_spans.empty()) {
|
||||
Json s = Json::object();
|
||||
s.set("mode", Json(relevant_spans));
|
||||
if (spans_threshold) s.set("threshold", Json(*spans_threshold));
|
||||
if (max_spans) s.set("max_spans", Json(*max_spans));
|
||||
b.set("relevant_spans", s);
|
||||
}
|
||||
if (priority) b.set("priority", Json(*priority));
|
||||
return b;
|
||||
}
|
||||
|
||||
RerankResult RerankResult::parse(const Json& doc) {
|
||||
RerankResult r;
|
||||
r.raw = doc;
|
||||
r.id = J::str(doc, "id");
|
||||
if (const Json* a = J::arr(doc, "results"))
|
||||
for (const Json& x : a->as_array()) {
|
||||
RerankHit h;
|
||||
h.index = static_cast<int>(J::num(x, "index"));
|
||||
h.score = J::real(x, "relevance_score");
|
||||
if (const Json* s = J::arr(x, "relevant_spans")) h.spans = *s;
|
||||
r.hits.push_back(h);
|
||||
}
|
||||
r.usage = TokenUsage::parse(J::obj(doc, "meta"));
|
||||
return r;
|
||||
}
|
||||
|
||||
void ClassifyParams::validate() const {
|
||||
Rules::require(!inputs.empty() && inputs.size() <= CohereCaps::kMaxClassifyInputs, "inputs must be 1..96");
|
||||
for (const std::string& i : inputs) Rules::require(!i.empty(), "inputs must not be empty");
|
||||
if (!model.empty()) Rules::model(model);
|
||||
Rules::require(!model.empty() || examples.size() >= 2, "without a fine-tuned model, give at least 2 examples");
|
||||
for (const ClassifyExample& e : examples)
|
||||
Rules::require(!e.text.empty() && !e.label.empty(), "examples need text and label");
|
||||
if (!truncate.empty()) Rules::one_of(truncate, {"NONE", "START", "END"}, "truncate");
|
||||
}
|
||||
|
||||
Json ClassifyParams::to_json() const {
|
||||
validate();
|
||||
Json b = Json::object();
|
||||
b.set("inputs", string_array(inputs));
|
||||
if (!examples.empty()) {
|
||||
Json a = Json::array();
|
||||
for (const ClassifyExample& e : examples) {
|
||||
Json x = Json::object();
|
||||
x.set("text", Json(e.text));
|
||||
x.set("label", Json(e.label));
|
||||
a.push(x);
|
||||
}
|
||||
b.set("examples", a);
|
||||
}
|
||||
if (!model.empty()) b.set("model", Json(model));
|
||||
if (!truncate.empty()) b.set("truncate", Json(truncate));
|
||||
return b;
|
||||
}
|
||||
|
||||
ClassifyResult ClassifyResult::parse(const Json& doc) {
|
||||
ClassifyResult r;
|
||||
r.raw = doc;
|
||||
r.id = J::str(doc, "id");
|
||||
if (const Json* a = J::arr(doc, "classifications"))
|
||||
for (const Json& x : a->as_array()) {
|
||||
Classification c;
|
||||
c.id = J::str(x, "id");
|
||||
c.input = J::str(x, "input");
|
||||
c.prediction = J::str(x, "prediction");
|
||||
if (c.prediction.empty()) {
|
||||
std::vector<std::string> p = J::strings(x, "predictions");
|
||||
if (!p.empty()) c.prediction = p.front();
|
||||
}
|
||||
c.confidence = J::real(x, "confidence");
|
||||
if (c.confidence == 0)
|
||||
if (const Json* cs = J::arr(x, "confidences"))
|
||||
if (!cs->as_array().empty() && cs->as_array().front().is_number()) c.confidence = cs->as_array().front().as_number();
|
||||
c.classification_type = J::str(x, "classification_type");
|
||||
if (const Json* l = J::obj(x, "labels")) c.labels = *l;
|
||||
r.items.push_back(c);
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
TokenizeResult TokenizeResult::parse(const Json& doc) {
|
||||
TokenizeResult r;
|
||||
if (const Json* a = J::arr(doc, "tokens"))
|
||||
for (const Json& t : a->as_array())
|
||||
if (t.is_number()) r.tokens.push_back(t.as_int());
|
||||
r.token_strings = J::strings(doc, "token_strings");
|
||||
return r;
|
||||
}
|
||||
|
||||
EmbedResult Embeddings::embed(const EmbedParams& params) {
|
||||
Json body = params.to_json();
|
||||
Calls::check_body(Call::kEmbed, body);
|
||||
return EmbedResult::parse(client_->post(Calls::render(Call::kEmbed), body));
|
||||
}
|
||||
|
||||
EmbedResult Embeddings::embed_v1(const EmbedParams& params) {
|
||||
Rules::require(params.inputs.empty() && !params.output_dimension && !params.max_tokens && !params.priority,
|
||||
"v1 embed takes texts and images only");
|
||||
Json body = params.to_json();
|
||||
Calls::check_body(Call::kEmbedV1, body);
|
||||
return EmbedResult::parse(client_->post(Calls::render(Call::kEmbedV1), body));
|
||||
}
|
||||
|
||||
RerankResult Rerank::rerank(const RerankParams& params) {
|
||||
Json body = params.to_json();
|
||||
Calls::check_body(Call::kRerank, body);
|
||||
return RerankResult::parse(client_->post(Calls::render(Call::kRerank), body));
|
||||
}
|
||||
|
||||
Json Rerank::rerank_v1(const Json& body) {
|
||||
Calls::check_body(Call::kRerankV1, body);
|
||||
Rules::require(body.find("query") && body.find("query")->is_string(), "query is required");
|
||||
const Json* docs = body.find("documents");
|
||||
Rules::require(docs && docs->is_array() && !docs->as_array().empty() &&
|
||||
docs->as_array().size() <= CohereCaps::kMaxDocuments,
|
||||
"documents must be 1..1000");
|
||||
return client_->post(Calls::render(Call::kRerankV1), body);
|
||||
}
|
||||
|
||||
ClassifyResult Classify::classify(const ClassifyParams& params) {
|
||||
Json body = params.to_json();
|
||||
Calls::check_body(Call::kClassify, body);
|
||||
return ClassifyResult::parse(client_->post(Calls::render(Call::kClassify), body));
|
||||
}
|
||||
|
||||
TokenizeResult Tokenizer::tokenize(const std::string& model, const std::string& text) {
|
||||
Rules::model(model);
|
||||
Rules::require(!text.empty() && text.size() <= CohereCaps::kMaxTokenizeChars, "text must be 1..65536 chars");
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
b.set("text", Json(text));
|
||||
return TokenizeResult::parse(client_->post(Calls::render(Call::kTokenize), b));
|
||||
}
|
||||
|
||||
std::string Tokenizer::detokenize(const std::string& model, const std::vector<long>& tokens) {
|
||||
Rules::model(model);
|
||||
Rules::require(!tokens.empty() && tokens.size() <= CohereCaps::kMaxDetokenizeTokens, "tokens must be 1..65536");
|
||||
Json arr = Json::array();
|
||||
for (long t : tokens) {
|
||||
Rules::require(t >= 0, "token ids must be >= 0");
|
||||
arr.push(Json(t));
|
||||
}
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
b.set("tokens", arr);
|
||||
return J::str(client_->post(Calls::render(Call::kDetokenize), b), "text");
|
||||
}
|
||||
|
||||
Json Legacy::generate(const Json& body) {
|
||||
Calls::check_body(Call::kGenerate, body);
|
||||
Rules::require(body.find("prompt") && body.find("prompt")->is_string(), "prompt is required");
|
||||
const Json* s = body.find("stream");
|
||||
Rules::require(!s || (s->is_bool() && !s->as_bool()), "generate streaming is not supported here");
|
||||
return client_->post(Calls::render(Call::kGenerate), body);
|
||||
}
|
||||
|
||||
Json Legacy::summarize(const Json& body) {
|
||||
Calls::check_body(Call::kSummarize, body);
|
||||
Rules::require(body.find("text") && body.find("text")->is_string(), "text is required");
|
||||
return client_->post(Calls::render(Call::kSummarize), body);
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+139
@@ -0,0 +1,139 @@
|
||||
#pragma once
|
||||
|
||||
// Stateless model calls: Embed (v2, v1), Rerank (v2, v1), Classify, Tokenize /
|
||||
// Detokenize, and the legacy Generate and Summarize. Parameters validate
|
||||
// before any network traffic; bodies are checked against the call table.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/cohere/types.h"
|
||||
#include "llm/api_client.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
struct EmbedParams {
|
||||
std::string model = "embed-v4.0";
|
||||
std::string input_type = "search_document"; // search_document search_query classification clustering image
|
||||
std::vector<std::string> texts; // <= 96
|
||||
std::vector<std::string> images; // data URLs
|
||||
std::vector<Json> inputs; // v2 mixed inputs: {"content":[{type,text|image_url}]}
|
||||
std::vector<std::string> embedding_types; // float int8 uint8 binary ubinary base64
|
||||
std::string truncate; // NONE START END
|
||||
std::optional<int> max_tokens;
|
||||
std::optional<int> output_dimension;
|
||||
std::optional<int> priority;
|
||||
void validate() const;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct EmbedResult {
|
||||
std::string id;
|
||||
std::string response_type;
|
||||
std::vector<std::string> texts;
|
||||
Json embeddings; // {"float":[[...]],...} (v2) or [[...]] (v1 floats)
|
||||
TokenUsage usage;
|
||||
Json raw;
|
||||
// Float vectors (v2 "float", or the v1 array). Empty when not requested.
|
||||
std::vector<std::vector<double>> floats() const;
|
||||
static EmbedResult parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct RerankParams {
|
||||
std::string model = "rerank-v3.5";
|
||||
std::string query;
|
||||
std::vector<std::string> documents; // 1..1000
|
||||
std::optional<int> top_n;
|
||||
std::optional<int> max_tokens_per_doc;
|
||||
std::string relevant_spans; // "", enabled, disabled
|
||||
std::optional<double> spans_threshold;
|
||||
std::optional<int> max_spans;
|
||||
std::optional<int> priority;
|
||||
void validate() const;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct RerankHit {
|
||||
int index = 0;
|
||||
double score = 0;
|
||||
Json spans; // array or null
|
||||
};
|
||||
|
||||
struct RerankResult {
|
||||
std::string id;
|
||||
std::vector<RerankHit> hits; // best first, as sent
|
||||
TokenUsage usage;
|
||||
Json raw;
|
||||
static RerankResult parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct ClassifyExample {
|
||||
std::string text;
|
||||
std::string label;
|
||||
};
|
||||
|
||||
struct ClassifyParams {
|
||||
std::vector<std::string> inputs; // 1..96
|
||||
std::vector<ClassifyExample> examples;
|
||||
std::string model; // empty: the default classifier
|
||||
std::string truncate;
|
||||
void validate() const;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct Classification {
|
||||
std::string id, input, prediction, classification_type;
|
||||
double confidence = 0;
|
||||
Json labels;
|
||||
};
|
||||
|
||||
struct ClassifyResult {
|
||||
std::string id;
|
||||
std::vector<Classification> items;
|
||||
Json raw;
|
||||
static ClassifyResult parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct TokenizeResult {
|
||||
std::vector<long> tokens;
|
||||
std::vector<std::string> token_strings;
|
||||
static TokenizeResult parse(const Json& doc);
|
||||
};
|
||||
|
||||
class Embeddings : public llm::Service {
|
||||
public:
|
||||
explicit Embeddings(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
EmbedResult embed(const EmbedParams& params); // POST /v2/embed
|
||||
EmbedResult embed_v1(const EmbedParams& params); // POST /v1/embed (texts/images only)
|
||||
};
|
||||
|
||||
class Rerank : public llm::Service {
|
||||
public:
|
||||
explicit Rerank(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
RerankResult rerank(const RerankParams& params); // POST /v2/rerank
|
||||
Json rerank_v1(const Json& body); // POST /v1/rerank
|
||||
};
|
||||
|
||||
class Classify : public llm::Service {
|
||||
public:
|
||||
explicit Classify(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
ClassifyResult classify(const ClassifyParams& params); // POST /v1/classify (deprecated in the spec)
|
||||
};
|
||||
|
||||
class Tokenizer : public llm::Service {
|
||||
public:
|
||||
explicit Tokenizer(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
TokenizeResult tokenize(const std::string& model, const std::string& text);
|
||||
std::string detokenize(const std::string& model, const std::vector<long>& tokens);
|
||||
};
|
||||
|
||||
// Deprecated endpoints that remain in the spec; bodies are Json checked against the table.
|
||||
class Legacy : public llm::Service {
|
||||
public:
|
||||
explicit Legacy(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
Json generate(const Json& body); // POST /v1/generate (prompt required, stream:true rejected)
|
||||
Json summarize(const Json& body); // POST /v1/summarize (text required)
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+33
@@ -0,0 +1,33 @@
|
||||
#include "vendors/cohere/platform.h"
|
||||
|
||||
#include "clients/transports/http_transport.h"
|
||||
#include "llm/config.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
llm::ApiClientConfig Platform::config() {
|
||||
llm::ApiClientConfig c;
|
||||
c.vendor = "cohere";
|
||||
c.base_url = kHost;
|
||||
c.key_env = kKeyEnv;
|
||||
c.auth = llm::AuthStyle::kBearer;
|
||||
c.default_headers["X-Client-Name"] = kClientName;
|
||||
return c;
|
||||
}
|
||||
|
||||
Platform Platform::from_env() {
|
||||
std::string key = llm::Config::get(kKeyEnv);
|
||||
if (key.empty()) key = llm::Config::get(kKeyEnvAlt);
|
||||
Platform p;
|
||||
p.client = std::make_shared<llm::ApiClient>(std::make_unique<clients::transports::HttpTransport>(),
|
||||
config(), key);
|
||||
return p;
|
||||
}
|
||||
|
||||
Platform Platform::with(std::unique_ptr<clients::Transport> transport, const std::string& key) {
|
||||
Platform p;
|
||||
p.client = std::make_shared<llm::ApiClient>(std::move(transport), config(), key);
|
||||
return p;
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+26
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
// api.cohere.com, Bearer auth. Credential COHERE_API_KEY (CO_API_KEY, the
|
||||
// official SDKs' name, is accepted as a fallback), read through llm::Config.
|
||||
|
||||
#include <memory>
|
||||
|
||||
#include "llm/api_client.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
struct Platform {
|
||||
static constexpr const char* kHost = "https://api.cohere.com";
|
||||
static constexpr const char* kKeyEnv = "COHERE_API_KEY";
|
||||
static constexpr const char* kKeyEnvAlt = "CO_API_KEY";
|
||||
static constexpr const char* kClientName = "molo";
|
||||
|
||||
std::shared_ptr<llm::ApiClient> client;
|
||||
|
||||
static llm::ApiClientConfig config();
|
||||
static Platform from_env();
|
||||
static Platform with(std::unique_ptr<clients::Transport> transport, const std::string& key);
|
||||
bool configured() const { return client && client->configured(); }
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+445
@@ -0,0 +1,445 @@
|
||||
#include "vendors/cohere/resources.h"
|
||||
|
||||
#include <functional>
|
||||
|
||||
#include "clients/body/form.h"
|
||||
#include "vendors/cohere/calls.h"
|
||||
#include "vendors/cohere/caps.h"
|
||||
#include "vendors/cohere/rules.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
namespace {
|
||||
using clients::Method;
|
||||
|
||||
void add_page(Query& q, std::optional<int> page_size, const std::string& token) {
|
||||
if (page_size) {
|
||||
Rules::page_size(*page_size);
|
||||
q.emplace_back("page_size", std::to_string(*page_size));
|
||||
}
|
||||
if (!token.empty()) {
|
||||
Rules::require(token.size() <= 2048, "page_token too long");
|
||||
q.emplace_back("page_token", token);
|
||||
}
|
||||
}
|
||||
|
||||
Json strings(const std::vector<std::string>& v) {
|
||||
Json a = Json::array();
|
||||
for (const std::string& s : v) a.push(Json(s));
|
||||
return a;
|
||||
}
|
||||
|
||||
const Json* field(const Json& doc, const char* key) { return doc.is_object() ? doc.find(key) : nullptr; }
|
||||
|
||||
Json require_object(const Json& j, const char* what) {
|
||||
Rules::require(j.is_object(), std::string(what) + " must be a JSON object");
|
||||
return j;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
// ---- Account ---------------------------------------------------------------
|
||||
|
||||
ApiKeyCheck Account::check_api_key() {
|
||||
return ApiKeyCheck::parse(client_->request(Method::kPost, Calls::render(Call::kCheckApiKey)));
|
||||
}
|
||||
|
||||
ModelList Account::models(const std::string& endpoint, bool default_only, std::optional<int> page_size,
|
||||
const std::string& page_token) {
|
||||
Query q;
|
||||
if (!endpoint.empty()) {
|
||||
Rules::one_of(endpoint, {"chat", "embed", "classify", "summarize", "rerank", "rate", "generate"}, "endpoint");
|
||||
q.emplace_back("endpoint", endpoint);
|
||||
}
|
||||
if (default_only) {
|
||||
Rules::require(!endpoint.empty(), "default_only needs an endpoint");
|
||||
q.emplace_back("default_only", "true");
|
||||
}
|
||||
add_page(q, page_size, page_token);
|
||||
return ModelList::parse(client_->get(Calls::render(Call::kModelList, "", q)));
|
||||
}
|
||||
|
||||
ModelList Account::all_models(const std::string& endpoint) {
|
||||
ModelList all;
|
||||
std::string token;
|
||||
for (std::size_t page = 0; page < CohereCaps::kMaxListPages; ++page) {
|
||||
ModelList l = models(endpoint, false, CohereCaps::kMaxPageSize, token);
|
||||
for (ModelInfo& m : l.models) all.models.push_back(std::move(m));
|
||||
token = l.page.next_page_token;
|
||||
if (token.empty() || all.models.size() >= CohereCaps::kMaxItemsReturned) break;
|
||||
}
|
||||
all.page.next_page_token = token;
|
||||
all.page.truncated = !token.empty();
|
||||
return all;
|
||||
}
|
||||
|
||||
ModelInfo Account::model(const std::string& name) {
|
||||
Rules::model(name);
|
||||
return ModelInfo::parse(client_->get(Calls::render(Call::kModelGet, name)));
|
||||
}
|
||||
|
||||
// ---- Datasets --------------------------------------------------------------
|
||||
|
||||
const std::vector<std::string>& DatasetUpload::types() {
|
||||
static const std::vector<std::string> t = {
|
||||
"embed-input", "embed-result", "cluster-result", "cluster-outliers", "reranker-finetune-input",
|
||||
"single-label-classification-finetune-input", "chat-finetune-input",
|
||||
"multi-label-classification-finetune-input", "batch-chat-input", "batch-openai-chat-input",
|
||||
"batch-embed-v2-input", "batch-chat-v2-input"};
|
||||
return t;
|
||||
}
|
||||
|
||||
static void file_name(const std::string& name) {
|
||||
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxFilenameBytes, "filename must be 1..512 bytes");
|
||||
for (char c : name)
|
||||
Rules::require(c != '"' && c != '\r' && c != '\n' && c != '\\' && c != '/', "filename has invalid characters");
|
||||
}
|
||||
|
||||
void DatasetUpload::validate() const {
|
||||
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxNameChars, "dataset name must be 1..256 chars");
|
||||
Rules::one_of(type, types(), "dataset type");
|
||||
file_name(filename);
|
||||
Rules::require(!data.empty(), "data is empty");
|
||||
Rules::require(data.size() + eval_data.size() <= CohereCaps::kMaxUploadBytes, "upload over 256 MiB");
|
||||
Rules::require(eval_data.empty() == eval_filename.empty(), "eval_data and eval_filename go together");
|
||||
if (!eval_filename.empty()) file_name(eval_filename);
|
||||
Rules::require(text_separator.size() <= 16 && csv_delimiter.size() <= 4, "separator too long");
|
||||
}
|
||||
|
||||
void DatasetFilter::validate() const {
|
||||
if (!dataset_type.empty()) Rules::one_of(dataset_type, DatasetUpload::types(), "dataset type");
|
||||
if (!validation_status.empty())
|
||||
Rules::one_of(validation_status, {"unknown", "queued", "processing", "failed", "validated", "skipped"},
|
||||
"validation status");
|
||||
if (limit) Rules::require(*limit >= 1 && *limit <= CohereCaps::kDatasetListMaxLimit, "limit must be 1..1000");
|
||||
if (offset) Rules::require(*offset >= 0, "offset must be >= 0");
|
||||
}
|
||||
|
||||
std::string Datasets::create(const DatasetUpload& u) {
|
||||
u.validate();
|
||||
Query q;
|
||||
q.emplace_back("name", u.name);
|
||||
q.emplace_back("type", u.type);
|
||||
if (u.keep_original_file) q.emplace_back("keep_original_file", "true");
|
||||
if (u.skip_malformed_input) q.emplace_back("skip_malformed_input", "true");
|
||||
for (const std::string& f : u.keep_fields) q.emplace_back("keep_fields", f);
|
||||
for (const std::string& f : u.optional_fields) q.emplace_back("optional_fields", f);
|
||||
if (!u.text_separator.empty()) q.emplace_back("text_separator", u.text_separator);
|
||||
if (!u.csv_delimiter.empty()) q.emplace_back("csv_delimiter", u.csv_delimiter);
|
||||
clients::body::MultipartBody form;
|
||||
form.add_file("data", u.filename, u.content_type, u.data);
|
||||
if (!u.eval_data.empty()) form.add_file("eval_data", u.eval_filename, u.content_type, u.eval_data);
|
||||
return J::str(client_->post_multipart(Calls::render(Call::kDatasetCreate, "", q), form), "id");
|
||||
}
|
||||
|
||||
std::vector<Dataset> Datasets::list(const DatasetFilter& f) {
|
||||
f.validate();
|
||||
Query q;
|
||||
if (!f.dataset_type.empty()) q.emplace_back("datasetType", f.dataset_type);
|
||||
if (!f.validation_status.empty()) q.emplace_back("validationStatus", f.validation_status);
|
||||
if (f.limit) q.emplace_back("limit", std::to_string(*f.limit));
|
||||
if (f.offset) q.emplace_back("offset", std::to_string(*f.offset));
|
||||
if (!f.before.empty()) q.emplace_back("before", f.before);
|
||||
if (!f.after.empty()) q.emplace_back("after", f.after);
|
||||
Json doc = client_->get(Calls::render(Call::kDatasetList, "", q));
|
||||
std::vector<Dataset> out;
|
||||
if (const Json* a = J::arr(doc, "datasets"))
|
||||
for (const Json& d : a->as_array()) {
|
||||
if (out.size() >= CohereCaps::kMaxItemsReturned) break;
|
||||
out.push_back(Dataset::parse(d));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
Dataset Datasets::get(const std::string& id) {
|
||||
Json doc = client_->get(Calls::render(Call::kDatasetGet, id));
|
||||
const Json* d = field(doc, "dataset");
|
||||
return Dataset::parse(d ? *d : Json::object());
|
||||
}
|
||||
|
||||
Json Datasets::usage() {
|
||||
Json doc = client_->get(Calls::render(Call::kDatasetUsage));
|
||||
const Json* u = field(doc, "organization_usage");
|
||||
return u ? *u : Json();
|
||||
}
|
||||
|
||||
void Datasets::remove(const std::string& id) { client_->del(Calls::render(Call::kDatasetDelete, id)); }
|
||||
|
||||
Dataset Datasets::await_validation(const std::string& id, int max_polls, const std::function<void()>& wait) {
|
||||
Rules::require(max_polls >= 1 && max_polls <= 60, "max_polls must be 1..60");
|
||||
Dataset d;
|
||||
for (int i = 0; i < max_polls; ++i) {
|
||||
d = get(id);
|
||||
if (d.validation_status == "validated" || d.validation_status == "failed" || d.validation_status == "skipped")
|
||||
return d;
|
||||
if (wait) wait();
|
||||
}
|
||||
return d;
|
||||
}
|
||||
|
||||
// ---- Batches ---------------------------------------------------------------
|
||||
|
||||
BatchList Batches::list(std::optional<int> page_size, const std::string& page_token, const std::string& order_by) {
|
||||
Query q;
|
||||
add_page(q, page_size, page_token);
|
||||
if (!order_by.empty()) {
|
||||
Rules::one_of(order_by, {"created_at", "updated_at"}, "order_by");
|
||||
q.emplace_back("order_by", order_by);
|
||||
}
|
||||
return BatchList::parse(client_->get(Calls::render(Call::kBatchList, "", q)));
|
||||
}
|
||||
|
||||
Batch Batches::create(const std::string& name, const std::string& input_dataset_id, const std::string& model) {
|
||||
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxNameChars, "batch name must be 1..256 chars");
|
||||
Rules::id("dataset", input_dataset_id);
|
||||
Rules::model(model);
|
||||
Json b = Json::object();
|
||||
b.set("name", Json(name));
|
||||
b.set("input_dataset_id", Json(input_dataset_id));
|
||||
b.set("model", Json(model));
|
||||
Calls::check_body(Call::kBatchCreate, b);
|
||||
Json doc = client_->post(Calls::render(Call::kBatchCreate), b);
|
||||
const Json* x = field(doc, "batch");
|
||||
return Batch::parse(x ? *x : Json::object());
|
||||
}
|
||||
|
||||
Batch Batches::get(const std::string& id) {
|
||||
Json doc = client_->get(Calls::render(Call::kBatchGet, id));
|
||||
const Json* x = field(doc, "batch");
|
||||
return Batch::parse(x ? *x : Json::object());
|
||||
}
|
||||
|
||||
void Batches::cancel(const std::string& id) {
|
||||
client_->request(Method::kPost, Calls::render(Call::kBatchCancel, id));
|
||||
}
|
||||
|
||||
// ---- Embed jobs ------------------------------------------------------------
|
||||
|
||||
Json EmbedJobParams::to_json() const {
|
||||
Rules::model(model);
|
||||
Rules::id("dataset", dataset_id);
|
||||
Rules::one_of(input_type, {"search_document", "search_query", "classification", "clustering", "image"}, "input_type");
|
||||
for (const std::string& t : embedding_types)
|
||||
Rules::one_of(t, {"float", "int8", "uint8", "binary", "ubinary", "base64"}, "embedding type");
|
||||
if (!truncate.empty()) Rules::one_of(truncate, {"START", "END"}, "truncate");
|
||||
Rules::require(name.size() <= CohereCaps::kMaxNameChars, "name too long");
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
b.set("dataset_id", Json(dataset_id));
|
||||
b.set("input_type", Json(input_type));
|
||||
if (!name.empty()) b.set("name", Json(name));
|
||||
if (!embedding_types.empty()) b.set("embedding_types", strings(embedding_types));
|
||||
if (!truncate.empty()) b.set("truncate", Json(truncate));
|
||||
Calls::check_body(Call::kEmbedJobCreate, b);
|
||||
return b;
|
||||
}
|
||||
|
||||
std::string EmbedJobs::create(const EmbedJobParams& p) {
|
||||
return J::str(client_->post(Calls::render(Call::kEmbedJobCreate), p.to_json()), "job_id");
|
||||
}
|
||||
|
||||
std::vector<EmbedJob> EmbedJobs::list() {
|
||||
Json doc = client_->get(Calls::render(Call::kEmbedJobList));
|
||||
std::vector<EmbedJob> out;
|
||||
if (const Json* a = J::arr(doc, "embed_jobs"))
|
||||
for (const Json& j : a->as_array()) {
|
||||
if (out.size() >= CohereCaps::kMaxItemsReturned) break;
|
||||
out.push_back(EmbedJob::parse(j));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
EmbedJob EmbedJobs::get(const std::string& id) { return EmbedJob::parse(client_->get(Calls::render(Call::kEmbedJobGet, id))); }
|
||||
|
||||
void EmbedJobs::cancel(const std::string& id) {
|
||||
client_->request(Method::kPost, Calls::render(Call::kEmbedJobCancel, id));
|
||||
}
|
||||
|
||||
// ---- Connectors ------------------------------------------------------------
|
||||
|
||||
Json ConnectorParams::to_json(bool create) const {
|
||||
Json b = Json::object();
|
||||
if (create) Rules::require(!name.empty() && !url.empty(), "a connector needs name and url");
|
||||
if (!name.empty()) {
|
||||
Rules::require(name.size() <= CohereCaps::kMaxNameChars, "name too long");
|
||||
b.set("name", Json(name));
|
||||
}
|
||||
if (!url.empty()) {
|
||||
Rules::require(Rules::is_http_url(url), "connector url must be http(s)");
|
||||
b.set("url", Json(url));
|
||||
}
|
||||
if (create && !description.empty()) b.set("description", Json(description));
|
||||
if (!excludes.empty()) b.set("excludes", strings(excludes));
|
||||
if (active) b.set("active", Json(*active));
|
||||
if (continue_on_failure) b.set("continue_on_failure", Json(*continue_on_failure));
|
||||
if (!oauth.is_null()) {
|
||||
Json o = require_object(oauth, "oauth");
|
||||
Rules::require(o.find("authorize_url") && o.find("token_url"), "oauth needs authorize_url and token_url");
|
||||
b.set("oauth", o);
|
||||
}
|
||||
if (!service_auth.is_null()) {
|
||||
Json s = require_object(service_auth, "service_auth");
|
||||
Rules::require(s.find("type") && s.find("token"), "service_auth needs type and token");
|
||||
b.set("service_auth", s);
|
||||
}
|
||||
Rules::require(!b.as_object().empty(), "nothing to send");
|
||||
return b;
|
||||
}
|
||||
|
||||
std::vector<Connector> Connectors::list(std::optional<int> limit, std::optional<int> offset) {
|
||||
Query q;
|
||||
if (limit) {
|
||||
Rules::require(*limit >= 1 && *limit <= CohereCaps::kConnectorListMaxLimit, "limit must be 1..1000");
|
||||
q.emplace_back("limit", std::to_string(*limit));
|
||||
}
|
||||
if (offset) {
|
||||
Rules::require(*offset >= 0, "offset must be >= 0");
|
||||
q.emplace_back("offset", std::to_string(*offset));
|
||||
}
|
||||
Json doc = client_->get(Calls::render(Call::kConnectorList, "", q));
|
||||
std::vector<Connector> out;
|
||||
if (const Json* a = J::arr(doc, "connectors"))
|
||||
for (const Json& c : a->as_array()) {
|
||||
if (out.size() >= CohereCaps::kMaxItemsReturned) break;
|
||||
out.push_back(Connector::parse(c));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
static Connector connector_of(const Json& doc) {
|
||||
const Json* c = field(doc, "connector");
|
||||
return Connector::parse(c ? *c : Json::object());
|
||||
}
|
||||
|
||||
Connector Connectors::create(const ConnectorParams& p) {
|
||||
Json b = p.to_json(true);
|
||||
Calls::check_body(Call::kConnectorCreate, b);
|
||||
return connector_of(client_->post(Calls::render(Call::kConnectorCreate), b));
|
||||
}
|
||||
|
||||
Connector Connectors::get(const std::string& id) { return connector_of(client_->get(Calls::render(Call::kConnectorGet, id))); }
|
||||
|
||||
Connector Connectors::update(const std::string& id, const ConnectorParams& p) {
|
||||
Json b = p.to_json(false);
|
||||
Calls::check_body(Call::kConnectorUpdate, b);
|
||||
return connector_of(client_->patch(Calls::render(Call::kConnectorUpdate, id), b));
|
||||
}
|
||||
|
||||
void Connectors::remove(const std::string& id) { client_->del(Calls::render(Call::kConnectorDelete, id)); }
|
||||
|
||||
std::string Connectors::oauth_authorize(const std::string& id, const std::string& after_token_redirect) {
|
||||
Query q;
|
||||
if (!after_token_redirect.empty()) q.emplace_back("after_token_redirect", after_token_redirect);
|
||||
return J::str(client_->request(Method::kPost, Calls::render(Call::kConnectorAuthorize, id, q)), "redirect_url");
|
||||
}
|
||||
|
||||
// ---- Fine-tuning -----------------------------------------------------------
|
||||
|
||||
FineTuning::ModelPage FineTuning::list(std::optional<int> page_size, const std::string& page_token,
|
||||
const std::string& order_by) {
|
||||
Query q;
|
||||
add_page(q, page_size, page_token);
|
||||
if (!order_by.empty()) {
|
||||
Rules::one_of(order_by, {"created_at", "updated_at"}, "order_by");
|
||||
q.emplace_back("order_by", order_by);
|
||||
}
|
||||
Json doc = client_->get(Calls::render(Call::kFinetuneList, "", q));
|
||||
ModelPage out;
|
||||
if (const Json* a = J::arr(doc, "finetuned_models"))
|
||||
for (const Json& m : a->as_array()) out.models.push_back(FinetunedModel::parse(m));
|
||||
out.page.next_page_token = J::str(doc, "next_page_token");
|
||||
out.page.total = J::num(doc, "total_size", -1);
|
||||
return out;
|
||||
}
|
||||
|
||||
static FinetunedModel finetuned_of(const Json& doc) {
|
||||
const Json* m = field(doc, "finetuned_model");
|
||||
return FinetunedModel::parse(m ? *m : Json::object());
|
||||
}
|
||||
|
||||
static void check_settings(const Json& settings) {
|
||||
Rules::require(settings.is_object(), "settings must be a JSON object");
|
||||
const Json* base = settings.find("base_model");
|
||||
Rules::require(base && base->is_object() && base->find("base_type"), "settings.base_model.base_type is required");
|
||||
const Json* ds = settings.find("dataset_id");
|
||||
Rules::require(ds && ds->is_string() && llm::Service::valid_id(ds->as_string()), "settings.dataset_id is required");
|
||||
}
|
||||
|
||||
FinetunedModel FineTuning::create(const std::string& name, const Json& settings) {
|
||||
Rules::require(!name.empty() && name.size() <= CohereCaps::kMaxNameChars, "name must be 1..256 chars");
|
||||
check_settings(settings);
|
||||
Json b = Json::object();
|
||||
b.set("name", Json(name));
|
||||
b.set("settings", settings);
|
||||
Calls::check_body(Call::kFinetuneCreate, b);
|
||||
return finetuned_of(client_->post(Calls::render(Call::kFinetuneCreate), b));
|
||||
}
|
||||
|
||||
FinetunedModel FineTuning::get(const std::string& id) {
|
||||
return finetuned_of(client_->get(Calls::render(Call::kFinetuneGet, id)));
|
||||
}
|
||||
|
||||
FinetunedModel FineTuning::update(const std::string& id, const std::string& name, const Json& settings) {
|
||||
Rules::require(!name.empty() || !settings.is_null(), "give a name or settings");
|
||||
Json b = Json::object();
|
||||
if (!name.empty()) b.set("name", Json(name));
|
||||
if (!settings.is_null()) {
|
||||
check_settings(settings);
|
||||
b.set("settings", settings);
|
||||
}
|
||||
Calls::check_body(Call::kFinetuneUpdate, b);
|
||||
return finetuned_of(client_->patch(Calls::render(Call::kFinetuneUpdate, id), b));
|
||||
}
|
||||
|
||||
void FineTuning::remove(const std::string& id) { client_->del(Calls::render(Call::kFinetuneDelete, id)); }
|
||||
|
||||
Json FineTuning::events(const std::string& id, std::optional<int> page_size, const std::string& page_token) {
|
||||
Query q;
|
||||
add_page(q, page_size, page_token);
|
||||
return client_->get(Calls::render(Call::kFinetuneEvents, id, q));
|
||||
}
|
||||
|
||||
Json FineTuning::step_metrics(const std::string& id, std::optional<int> page_size, const std::string& page_token) {
|
||||
Query q;
|
||||
add_page(q, page_size, page_token);
|
||||
return client_->get(Calls::render(Call::kFinetuneMetrics, id, q));
|
||||
}
|
||||
|
||||
// ---- Parse and transcription ------------------------------------------------
|
||||
|
||||
Json Documents::parse(const std::string& model, const std::string& image_url, const std::string& output_format) {
|
||||
Rules::model(model);
|
||||
Rules::image_url(image_url);
|
||||
Json doc = Json::object();
|
||||
doc.set("type", Json("image_url"));
|
||||
doc.set("image_url", Json(image_url));
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
b.set("document", doc);
|
||||
if (!output_format.empty()) {
|
||||
Rules::one_of(output_format, {"markdown", "blocks"}, "output_format");
|
||||
b.set("output_format", Json(output_format));
|
||||
}
|
||||
Calls::check_body(Call::kParse, b);
|
||||
return client_->post(Calls::render(Call::kParse), b);
|
||||
}
|
||||
|
||||
std::string Documents::transcribe(const std::string& model, const std::string& language, const std::string& filename,
|
||||
const std::string& content_type, const std::string& audio,
|
||||
std::optional<double> temperature) {
|
||||
Rules::model(model);
|
||||
Rules::require(!language.empty() && language.size() <= 16, "language is required (for example en)");
|
||||
file_name(filename);
|
||||
Rules::require(!audio.empty(), "audio is empty");
|
||||
Rules::require(audio.size() <= CohereCaps::kMaxUploadBytes, "audio over 256 MiB");
|
||||
clients::body::MultipartBody form;
|
||||
form.add_text("model", model);
|
||||
form.add_text("language", language);
|
||||
if (temperature) {
|
||||
Rules::range(*temperature, 0.0, 2.0, "temperature");
|
||||
form.add_text("temperature", std::to_string(*temperature));
|
||||
}
|
||||
form.add_file("file", filename, content_type.empty() ? "application/octet-stream" : content_type, audio);
|
||||
return J::str(client_->post_multipart(Calls::render(Call::kTranscribe), form), "text");
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+150
@@ -0,0 +1,150 @@
|
||||
#pragma once
|
||||
|
||||
// Account-level resources: models and key check, datasets, batches, embed
|
||||
// jobs, connectors (deprecated in the spec), fine-tuned models (deprecated in
|
||||
// the spec), plus Parse and audio Transcribe. Destructive and cancelling
|
||||
// calls are explicit methods with no defaults. List walks are capped and
|
||||
// report truncation.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/cohere/types.h"
|
||||
#include "llm/api_client.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
class Account : public llm::Service {
|
||||
public:
|
||||
explicit Account(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
ApiKeyCheck check_api_key(); // POST /v1/check-api-key
|
||||
ModelList models(const std::string& endpoint = "", bool default_only = false,
|
||||
std::optional<int> page_size = std::nullopt,
|
||||
const std::string& page_token = ""); // GET /v1/models, one page
|
||||
ModelList all_models(const std::string& endpoint = ""); // walks pages (cap 20)
|
||||
ModelInfo model(const std::string& name); // GET /v1/models/{model}
|
||||
};
|
||||
|
||||
struct DatasetUpload {
|
||||
std::string name;
|
||||
std::string type; // DatasetType, e.g. embed-input, chat-finetune-input, batch-chat-v2-input
|
||||
std::string filename;
|
||||
std::string content_type = "application/octet-stream";
|
||||
std::string data;
|
||||
std::string eval_filename;
|
||||
std::string eval_data;
|
||||
bool keep_original_file = false;
|
||||
bool skip_malformed_input = false;
|
||||
std::vector<std::string> keep_fields;
|
||||
std::vector<std::string> optional_fields;
|
||||
std::string text_separator;
|
||||
std::string csv_delimiter;
|
||||
static const std::vector<std::string>& types();
|
||||
void validate() const;
|
||||
};
|
||||
|
||||
struct DatasetFilter {
|
||||
std::string dataset_type;
|
||||
std::string validation_status; // unknown queued processing failed validated skipped
|
||||
std::optional<int> limit; // 1..1000
|
||||
std::optional<int> offset;
|
||||
std::string before, after; // ISO timestamps
|
||||
void validate() const;
|
||||
};
|
||||
|
||||
class Datasets : public llm::Service {
|
||||
public:
|
||||
explicit Datasets(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
std::string create(const DatasetUpload& upload); // POST /v1/datasets (multipart); returns the id
|
||||
std::vector<Dataset> list(const DatasetFilter& filter = {});
|
||||
Dataset get(const std::string& id);
|
||||
Json usage(); // organization_usage (bytes as sent)
|
||||
void remove(const std::string& id); // DELETE, explicit
|
||||
// Polls get() until validated/failed/skipped, at most `max_polls` (<= 60) times; the caller
|
||||
// supplies the wait through `wait` (no sleeping inside the library). Returns the last state.
|
||||
Dataset await_validation(const std::string& id, int max_polls, const std::function<void()>& wait);
|
||||
};
|
||||
|
||||
class Batches : public llm::Service {
|
||||
public:
|
||||
explicit Batches(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
BatchList list(std::optional<int> page_size = std::nullopt, const std::string& page_token = "",
|
||||
const std::string& order_by = "");
|
||||
Batch create(const std::string& name, const std::string& input_dataset_id, const std::string& model);
|
||||
Batch get(const std::string& id);
|
||||
void cancel(const std::string& id); // POST /v2/batches/{id}:cancel, explicit
|
||||
};
|
||||
|
||||
struct EmbedJobParams {
|
||||
std::string model = "embed-v4.0";
|
||||
std::string dataset_id;
|
||||
std::string input_type = "search_document";
|
||||
std::string name;
|
||||
std::vector<std::string> embedding_types;
|
||||
std::string truncate; // START or END
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
class EmbedJobs : public llm::Service {
|
||||
public:
|
||||
explicit EmbedJobs(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
std::string create(const EmbedJobParams& params); // returns job_id
|
||||
std::vector<EmbedJob> list();
|
||||
EmbedJob get(const std::string& id);
|
||||
void cancel(const std::string& id); // explicit
|
||||
};
|
||||
|
||||
struct ConnectorParams {
|
||||
std::string name;
|
||||
std::string url;
|
||||
std::string description;
|
||||
std::vector<std::string> excludes;
|
||||
std::optional<bool> active;
|
||||
std::optional<bool> continue_on_failure;
|
||||
Json oauth; // {client_id, client_secret, authorize_url, token_url, scope}
|
||||
Json service_auth; // {type, token}
|
||||
Json to_json(bool create) const;
|
||||
};
|
||||
|
||||
class Connectors : public llm::Service {
|
||||
public:
|
||||
explicit Connectors(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
std::vector<Connector> list(std::optional<int> limit = std::nullopt, std::optional<int> offset = std::nullopt);
|
||||
Connector create(const ConnectorParams& params);
|
||||
Connector get(const std::string& id);
|
||||
Connector update(const std::string& id, const ConnectorParams& params);
|
||||
void remove(const std::string& id); // explicit
|
||||
std::string oauth_authorize(const std::string& id, const std::string& after_token_redirect = "");
|
||||
};
|
||||
|
||||
class FineTuning : public llm::Service {
|
||||
public:
|
||||
explicit FineTuning(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
struct ModelPage {
|
||||
std::vector<FinetunedModel> models;
|
||||
Page page;
|
||||
};
|
||||
ModelPage list(std::optional<int> page_size = std::nullopt, const std::string& page_token = "",
|
||||
const std::string& order_by = "");
|
||||
// `settings` = {base_model:{base_type,...}, dataset_id, hyperparameters?, wandb?}. Costs money.
|
||||
FinetunedModel create(const std::string& name, const Json& settings);
|
||||
FinetunedModel get(const std::string& id);
|
||||
FinetunedModel update(const std::string& id, const std::string& name, const Json& settings);
|
||||
void remove(const std::string& id); // explicit
|
||||
Json events(const std::string& id, std::optional<int> page_size = std::nullopt, const std::string& page_token = "");
|
||||
Json step_metrics(const std::string& id, std::optional<int> page_size = std::nullopt, const std::string& page_token = "");
|
||||
};
|
||||
|
||||
class Documents : public llm::Service {
|
||||
public:
|
||||
explicit Documents(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
// POST /v2/parse: document is an image URL (http(s) or data URL); output_format markdown|blocks.
|
||||
Json parse(const std::string& model, const std::string& image_url, const std::string& output_format = "");
|
||||
// POST /v2/audio/transcriptions (multipart); returns the text.
|
||||
std::string transcribe(const std::string& model, const std::string& language, const std::string& filename,
|
||||
const std::string& content_type, const std::string& audio,
|
||||
std::optional<double> temperature = std::nullopt);
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+64
@@ -0,0 +1,64 @@
|
||||
#include "vendors/cohere/rules.h"
|
||||
|
||||
#include "vendors/cohere/caps.h"
|
||||
#include "llm/api_client.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
void Rules::fail(const std::string& why) { throw llm::ApiError::invalid("cohere", why); }
|
||||
|
||||
void Rules::model(const std::string& name) {
|
||||
require(!name.empty() && name.size() <= CohereCaps::kMaxModelChars, "model must be 1..128 chars");
|
||||
for (char c : name)
|
||||
require((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
|
||||
c == '_' || c == '.' || c == ':',
|
||||
"model has invalid characters");
|
||||
}
|
||||
|
||||
void Rules::id(const std::string& what, const std::string& id) {
|
||||
require(llm::Service::valid_id(id), "bad " + what + " id");
|
||||
}
|
||||
|
||||
void Rules::range(double v, double lo, double hi, const char* what) {
|
||||
require(v >= lo && v <= hi, std::string(what) + " must be " + std::to_string(lo) + ".." + std::to_string(hi));
|
||||
}
|
||||
|
||||
void Rules::temperature(double v) { range(v, 0.0, 1.0, "temperature"); }
|
||||
void Rules::top_p(double v) { range(v, 0.01, 0.99, "p"); }
|
||||
void Rules::top_k(long v) { require(v >= 0 && v <= CohereCaps::kMaxTopK, "k must be 0..500"); }
|
||||
void Rules::priority(long v) { require(v >= 0 && v <= CohereCaps::kMaxPriority, "priority must be 0..999"); }
|
||||
|
||||
void Rules::stop(const std::vector<std::string>& stop) {
|
||||
require(stop.size() <= CohereCaps::kMaxStopSequences, "at most 5 stop sequences");
|
||||
for (const std::string& s : stop) require(!s.empty(), "stop sequences must not be empty");
|
||||
}
|
||||
|
||||
void Rules::page_size(long v) { require(v >= 1 && v <= CohereCaps::kMaxPageSize, "page_size must be 1..1000"); }
|
||||
|
||||
void Rules::one_of(const std::string& value, const std::vector<std::string>& allowed, const char* what) {
|
||||
for (const std::string& a : allowed)
|
||||
if (a == value) return;
|
||||
std::string list;
|
||||
for (const std::string& a : allowed) list += (list.empty() ? "" : ", ") + a;
|
||||
fail(std::string(what) + " must be one of: " + list);
|
||||
}
|
||||
|
||||
bool Rules::is_http_url(const std::string& url) {
|
||||
if (url.size() > 8192) return false;
|
||||
bool ok = url.rfind("http://", 0) == 0 || url.rfind("https://", 0) == 0;
|
||||
if (!ok) return false;
|
||||
for (char c : url)
|
||||
if (c == ' ' || c == '\r' || c == '\n') return false;
|
||||
return url.find('/', 8) != std::string::npos || url.size() > 8;
|
||||
}
|
||||
|
||||
void Rules::image_url(const std::string& url) {
|
||||
if (url.rfind("data:image/", 0) == 0) {
|
||||
require(url.find(";base64,") != std::string::npos, "data URL must be base64");
|
||||
require(url.size() <= 8u * 1024u * 1024u, "image data URL over 8 MiB");
|
||||
return;
|
||||
}
|
||||
require(is_http_url(url), "image url must be http(s) or a data:image/...;base64, URL");
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+36
@@ -0,0 +1,36 @@
|
||||
#pragma once
|
||||
|
||||
// Input rules shared by every request type. All throw llm::ApiError::invalid
|
||||
// (HTTP 400 INVALID_ARGUMENT, vendor "cohere") before any network traffic.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
struct Rules {
|
||||
[[noreturn]] static void fail(const std::string& why);
|
||||
static void require(bool ok, const std::string& why) {
|
||||
if (!ok) fail(why);
|
||||
}
|
||||
// 1..128 of [A-Za-z0-9._:-]; the server decides which names exist.
|
||||
static void model(const std::string& name);
|
||||
// Single path segment id (llm::Service::valid_id).
|
||||
static void id(const std::string& what, const std::string& id);
|
||||
static void range(double v, double lo, double hi, const char* what);
|
||||
static void temperature(double v); // 0..1
|
||||
static void top_p(double v); // 0.01..0.99
|
||||
static void top_k(long v); // 0..500
|
||||
static void priority(long v); // 0..999
|
||||
static void stop(const std::vector<std::string>& stop); // <= 5, non-empty
|
||||
static void page_size(long v); // 1..1000
|
||||
// One of the listed values.
|
||||
static void one_of(const std::string& value, const std::vector<std::string>& allowed, const char* what);
|
||||
// http(s) URL, or data:...;base64, URL for images, <= 8 MiB
|
||||
static bool is_http_url(const std::string& url);
|
||||
static void image_url(const std::string& url);
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
+424
@@ -0,0 +1,424 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check src/vendors/cohere/calls.cpp against Cohere's published OpenAPI document.
|
||||
|
||||
Cohere publishes cohere-openapi.yaml in the cohere-ai/cohere-developer-experience
|
||||
repository (the file their Fern SDK generation reads). This checker parses that
|
||||
file with a small block-YAML reader (Python 3 standard library only, no PyYAML)
|
||||
and fails (exit 1) when:
|
||||
|
||||
row a declared call's HTTP method + path or operationId is not in the spec;
|
||||
field a request body field or query parameter we send is not in the spec's
|
||||
request body / parameters of that operation;
|
||||
required a field or parameter the spec marks required is not declared in the row;
|
||||
schema a field we read (SCHEMAS table) is not a property of that schema;
|
||||
response a declared response schema is neither a component nor an operationId;
|
||||
coverage a spec operation is neither in the table nor in EXCLUDED_OPERATIONS.
|
||||
|
||||
python3 -I src/vendors/cohere/tools/check_spec.py [--spec path-to-openapi.yaml] [alternative calls.cpp]
|
||||
|
||||
Network required unless --spec is given. No credentials.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
CALLS = os.path.join(HERE, "..", "calls.cpp")
|
||||
SPEC_URL = ("https://raw.githubusercontent.com/cohere-ai/cohere-developer-experience/main/"
|
||||
"cohere-openapi.yaml")
|
||||
# Operations that exist in the spec but are deliberately not wrapped (reason each).
|
||||
EXCLUDED_OPERATIONS = {}
|
||||
STR = re.compile(r'"([^"]*)"')
|
||||
METHODS = ("get", "post", "put", "patch", "delete")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- YAML reader
|
||||
class Yaml:
|
||||
"""Block-style YAML subset: mappings, sequences, plain / quoted scalars (single
|
||||
line or folded continuation), block scalars (| >), empty flow [] and {}."""
|
||||
|
||||
def __init__(self, text):
|
||||
self.lines = []
|
||||
for raw in text.replace("\t", " ").split("\n"):
|
||||
self.lines.append(raw.rstrip("\r"))
|
||||
self.i = 0
|
||||
|
||||
@staticmethod
|
||||
def indent_of(line):
|
||||
return len(line) - len(line.lstrip(" "))
|
||||
|
||||
def skip_blank(self):
|
||||
while self.i < len(self.lines):
|
||||
s = self.lines[self.i].strip()
|
||||
if s == "" or s.startswith("#") or s == "---":
|
||||
self.i += 1
|
||||
else:
|
||||
return
|
||||
|
||||
def parse(self):
|
||||
self.skip_blank()
|
||||
return self.node(self.indent_of(self.lines[self.i]))
|
||||
|
||||
def node(self, indent):
|
||||
self.skip_blank()
|
||||
if self.i >= len(self.lines):
|
||||
return None
|
||||
line = self.lines[self.i]
|
||||
if self.indent_of(line) < indent:
|
||||
return None
|
||||
body = line.strip()
|
||||
if body.startswith("- ") or body == "-":
|
||||
return self.seq(self.indent_of(line))
|
||||
if self.key_split(body) is not None:
|
||||
return self.mapping(self.indent_of(line))
|
||||
self.i += 1
|
||||
return self.scalar(body, self.indent_of(line))
|
||||
|
||||
@staticmethod
|
||||
def key_split(body):
|
||||
"""(key, rest) for `key: rest`, else None. Handles quoted keys."""
|
||||
if body[:1] in ("'", '"'):
|
||||
q = body[0]
|
||||
j = 1
|
||||
while j < len(body):
|
||||
if body[j] == q and (q == "'" or body[j - 1] != "\\"):
|
||||
break
|
||||
j += 1
|
||||
if j + 1 < len(body) and body[j + 1] == ":" and (j + 2 == len(body) or body[j + 2] == " "):
|
||||
return body[1:j], body[j + 2:].strip()
|
||||
return None
|
||||
if body[:1] in ("[", "{", "|", ">", "#", "*", "&", "!"):
|
||||
return None
|
||||
m = re.match(r"^([^:#]+?|[^:#]*?[^:\s#][^#]*?):(?:\s+(.*))?$", body)
|
||||
if not m:
|
||||
return None
|
||||
key = m.group(1)
|
||||
if " #" in key:
|
||||
return None
|
||||
return key.strip(), (m.group(2) or "").strip()
|
||||
|
||||
def mapping(self, indent):
|
||||
out = {}
|
||||
while True:
|
||||
self.skip_blank()
|
||||
if self.i >= len(self.lines):
|
||||
break
|
||||
line = self.lines[self.i]
|
||||
if self.indent_of(line) != indent:
|
||||
break
|
||||
body = line.strip()
|
||||
kv = self.key_split(body)
|
||||
if kv is None or body.startswith("- "):
|
||||
break
|
||||
key, rest = kv
|
||||
self.i += 1
|
||||
out[key] = self.value(rest, indent)
|
||||
return out
|
||||
|
||||
def seq(self, indent):
|
||||
out = []
|
||||
while True:
|
||||
self.skip_blank()
|
||||
if self.i >= len(self.lines):
|
||||
break
|
||||
line = self.lines[self.i]
|
||||
if self.indent_of(line) != indent:
|
||||
break
|
||||
body = line.strip()
|
||||
if not (body.startswith("- ") or body == "-"):
|
||||
break
|
||||
rest = body[1:].strip()
|
||||
if rest == "":
|
||||
self.i += 1
|
||||
out.append(self.node(indent + 1))
|
||||
elif self.key_split(rest) is not None and not rest.startswith(("'", '"')) or \
|
||||
(rest[:1] in ("'", '"') and self.key_split(rest) is not None):
|
||||
# `- key: value`: treat the text after "- " as a mapping line at indent+2
|
||||
self.lines[self.i] = " " * (indent + 2) + rest
|
||||
out.append(self.mapping(indent + 2))
|
||||
else:
|
||||
self.i += 1
|
||||
out.append(self.scalar(rest, indent))
|
||||
return out
|
||||
|
||||
def value(self, rest, indent):
|
||||
if rest == "" or rest.startswith("#"):
|
||||
self.skip_blank()
|
||||
if self.i < len(self.lines):
|
||||
nxt = self.lines[self.i]
|
||||
ni = self.indent_of(nxt)
|
||||
nb = nxt.strip()
|
||||
if ni > indent or (ni == indent and (nb.startswith("- ") or nb == "-")):
|
||||
return self.node(ni)
|
||||
return None
|
||||
if rest[0] in "|>":
|
||||
return self.block_scalar(rest, indent)
|
||||
return self.scalar(rest, indent)
|
||||
|
||||
def block_scalar(self, header, indent):
|
||||
fold = header[0] == ">"
|
||||
buf = []
|
||||
block_indent = None
|
||||
while self.i < len(self.lines):
|
||||
line = self.lines[self.i]
|
||||
if line.strip() == "":
|
||||
buf.append("")
|
||||
self.i += 1
|
||||
continue
|
||||
li = self.indent_of(line)
|
||||
if li <= indent:
|
||||
break
|
||||
if block_indent is None:
|
||||
block_indent = li
|
||||
buf.append(line[block_indent:] if li >= block_indent else line.strip())
|
||||
self.i += 1
|
||||
text = ("\n" if not fold else " ").join(buf).strip()
|
||||
return text
|
||||
|
||||
def scalar(self, text, indent):
|
||||
if text[0] in ("'", '"'):
|
||||
q = text[0]
|
||||
acc = text
|
||||
# continuation lines until the closing quote
|
||||
while not self.closed(acc, q) and self.i < len(self.lines):
|
||||
acc += " " + self.lines[self.i].strip()
|
||||
self.i += 1
|
||||
body = acc[1:Yaml.end_of(acc, q)]
|
||||
if q == '"':
|
||||
body = body.replace('\\"', '"').replace("\\n", "\n").replace("\\\\", "\\")
|
||||
else:
|
||||
body = body.replace("''", "'")
|
||||
return body
|
||||
if text in ("[]", "{}"):
|
||||
return [] if text == "[]" else {}
|
||||
if text[0] in "[{":
|
||||
acc = text
|
||||
while acc.count("[") + acc.count("{") > acc.count("]") + acc.count("}") and self.i < len(self.lines):
|
||||
acc += " " + self.lines[self.i].strip()
|
||||
self.i += 1
|
||||
return acc # flow collections beyond [] and {} are kept as text (not used by the checker)
|
||||
# plain scalar: strip trailing comment, fold continuation lines that are more indented
|
||||
text = re.sub(r"\s+#.*$", "", text)
|
||||
while self.i < len(self.lines):
|
||||
nxt = self.lines[self.i]
|
||||
if nxt.strip() == "" or self.indent_of(nxt) <= indent:
|
||||
break
|
||||
if self.key_split(nxt.strip()) is not None and nxt.strip()[:1] not in ("'", '"'):
|
||||
break
|
||||
text += " " + nxt.strip()
|
||||
self.i += 1
|
||||
return text
|
||||
|
||||
@staticmethod
|
||||
def end_of(acc, q):
|
||||
j = 1
|
||||
while j < len(acc):
|
||||
c = acc[j]
|
||||
if q == '"' and c == "\\":
|
||||
j += 2
|
||||
continue
|
||||
if c == q:
|
||||
if q == "'" and j + 1 < len(acc) and acc[j + 1] == "'":
|
||||
j += 2
|
||||
continue
|
||||
return j
|
||||
j += 1
|
||||
return len(acc)
|
||||
|
||||
@staticmethod
|
||||
def closed(acc, q):
|
||||
"""True when the quoted scalar that starts acc has its closing quote."""
|
||||
j = 1
|
||||
while j < len(acc):
|
||||
c = acc[j]
|
||||
if q == '"' and c == "\\":
|
||||
j += 2
|
||||
continue
|
||||
if c == q:
|
||||
if q == "'" and j + 1 < len(acc) and acc[j + 1] == "'":
|
||||
j += 2
|
||||
continue
|
||||
return True
|
||||
j += 1
|
||||
return False
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ spec model
|
||||
class Spec:
|
||||
def __init__(self, doc):
|
||||
self.doc = doc
|
||||
self.schemas = doc["components"]["schemas"]
|
||||
self.params = doc["components"].get("parameters", {})
|
||||
|
||||
def ref(self, node):
|
||||
seen = 0
|
||||
while isinstance(node, dict) and "$ref" in node and seen < 20:
|
||||
name = node["$ref"].split("/")[-1]
|
||||
kind = node["$ref"].split("/")[-2]
|
||||
node = (self.params if kind == "parameters" else self.schemas).get(name)
|
||||
seen += 1
|
||||
return node
|
||||
|
||||
def props(self, node, depth=0):
|
||||
"""Union of property names (and required names) through $ref / allOf / oneOf / anyOf / items."""
|
||||
props, required = {}, set()
|
||||
node = self.ref(node)
|
||||
if not isinstance(node, dict) or depth > 12:
|
||||
return props, required
|
||||
for key in ("allOf", "oneOf", "anyOf"):
|
||||
for sub in node.get(key) or []:
|
||||
p, r = self.props(sub, depth + 1)
|
||||
props.update(p)
|
||||
if key == "allOf":
|
||||
required |= r
|
||||
if isinstance(node.get("properties"), dict):
|
||||
props.update(node["properties"])
|
||||
if isinstance(node.get("required"), list):
|
||||
required |= set(node["required"])
|
||||
return props, required
|
||||
|
||||
def descend(self, node, path):
|
||||
"""Follow a dotted property path; arrays are entered through items."""
|
||||
for step in [p for p in path.split(".") if p]:
|
||||
node = self.ref(node)
|
||||
if isinstance(node, dict) and node.get("type") == "array" and "items" in node:
|
||||
node = self.ref(node["items"])
|
||||
props, _ = self.props(node)
|
||||
if step not in props:
|
||||
return None
|
||||
node = props[step]
|
||||
node = self.ref(node)
|
||||
if isinstance(node, dict) and node.get("type") == "array" and "items" in node:
|
||||
node = self.ref(node["items"])
|
||||
return node
|
||||
|
||||
def operations(self):
|
||||
for path, item in self.doc["paths"].items():
|
||||
for method, op in item.items():
|
||||
if method in METHODS and isinstance(op, dict):
|
||||
yield method.upper(), path, op
|
||||
|
||||
def op_params(self, op):
|
||||
out = {}
|
||||
for p in op.get("parameters") or []:
|
||||
p = self.ref(p)
|
||||
if isinstance(p, dict) and p.get("in") in ("query", "path"):
|
||||
out[p["name"]] = (p["in"], bool(p.get("required")))
|
||||
return out
|
||||
|
||||
def op_body(self, op):
|
||||
rb = op.get("requestBody")
|
||||
if not rb:
|
||||
return None, set()
|
||||
content = rb.get("content") or {}
|
||||
for ct in ("application/json", "multipart/form-data"):
|
||||
if ct in content:
|
||||
return self.props(content[ct].get("schema"))
|
||||
return {}, set()
|
||||
|
||||
def op_response(self, op):
|
||||
r = (op.get("responses") or {}).get("200") or {}
|
||||
content = r.get("content") or {}
|
||||
if "application/json" in content:
|
||||
return content["application/json"].get("schema")
|
||||
return None
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------ main
|
||||
def rows(text, begin, end):
|
||||
a, b = text.index(begin), text.index(end)
|
||||
out = []
|
||||
for line in text[a:b].splitlines()[1:]:
|
||||
s = line.strip()
|
||||
if s.startswith("{") and '"' in s:
|
||||
out.append(STR.findall(s))
|
||||
return out
|
||||
|
||||
|
||||
def load_spec(argv):
|
||||
if "--spec" in argv:
|
||||
i = argv.index("--spec")
|
||||
path = argv[i + 1]
|
||||
del argv[i:i + 2]
|
||||
with open(path, encoding="utf-8") as f:
|
||||
return f.read()
|
||||
req = urllib.request.Request(SPEC_URL, headers={"User-Agent": "molo-cohere-spec-check/1"})
|
||||
with urllib.request.urlopen(req, timeout=120) as r:
|
||||
return r.read().decode("utf-8", "replace")
|
||||
|
||||
|
||||
def main():
|
||||
argv = sys.argv[1:]
|
||||
spec_text = load_spec(argv)
|
||||
calls_path = argv[0] if argv else CALLS
|
||||
text = open(calls_path).read()
|
||||
calls = rows(text, "// CALLS-BEGIN", "// CALLS-END")
|
||||
schemas = rows(text, "// SCHEMAS-BEGIN", "// SCHEMAS-END")
|
||||
spec = Spec(Yaml(spec_text).parse())
|
||||
ops = {(m, p): op for m, p, op in spec.operations()}
|
||||
by_id = {op.get("operationId"): op for op in ops.values()}
|
||||
bad = []
|
||||
covered = set()
|
||||
for method, path, op_id, body, query, response in calls:
|
||||
op = ops.get((method, path))
|
||||
if op is None:
|
||||
bad.append("%s %s: not a path of the spec" % (method, path))
|
||||
continue
|
||||
covered.add((method, path))
|
||||
if op.get("operationId") != op_id:
|
||||
bad.append("%s %s: operationId is %s in the spec, %s in the table"
|
||||
% (method, path, op.get("operationId"), op_id))
|
||||
params = spec.op_params(op)
|
||||
body_props, body_required = spec.op_body(op)
|
||||
declared_body = {f for f in body.split(",") if f}
|
||||
declared_query = {q for q in query.split(",") if q}
|
||||
for f in sorted(declared_body):
|
||||
if f not in body_props:
|
||||
bad.append("%s %s: body field %s is not in the request body" % (method, path, f))
|
||||
for q in sorted(declared_query):
|
||||
if q not in params or params[q][0] != "query":
|
||||
bad.append("%s %s: query parameter %s is not in the spec" % (method, path, q))
|
||||
for name, (where, required) in params.items():
|
||||
if where == "query" and required and name not in declared_query:
|
||||
bad.append("%s %s: required query parameter %s missing in the row" % (method, path, name))
|
||||
for f in sorted(body_required):
|
||||
if f in body_props and f not in declared_body:
|
||||
bad.append("%s %s: required body field %s missing in the row" % (method, path, f))
|
||||
if response and response not in spec.schemas and response not in by_id:
|
||||
bad.append("%s %s: response schema %s is neither a component nor an operationId" % (method, path, response))
|
||||
for (method, path), op in sorted(ops.items()):
|
||||
if (method, path) not in covered and op.get("operationId") not in EXCLUDED_OPERATIONS:
|
||||
bad.append("unwrapped operation %s %s (%s): add a row or an exclusion"
|
||||
% (method, path, op.get("operationId")))
|
||||
checked_fields = 0
|
||||
for name, fields in schemas:
|
||||
head, _, tail = name.partition(".")
|
||||
if head in spec.schemas:
|
||||
node = spec.schemas[head]
|
||||
elif head in by_id:
|
||||
node = spec.op_response(by_id[head])
|
||||
else:
|
||||
bad.append("schema %s: neither a component nor an operationId" % head)
|
||||
continue
|
||||
target = spec.descend(node, tail) if tail else spec.ref(node)
|
||||
if target is None:
|
||||
bad.append("schema %s: property path not found" % name)
|
||||
continue
|
||||
props, _ = spec.props(target)
|
||||
for f in filter(None, fields.split(",")):
|
||||
checked_fields += 1
|
||||
if f not in props:
|
||||
bad.append("schema %s: field %s is not a property" % (name, f))
|
||||
if bad:
|
||||
print("SPEC CHECK FAILED")
|
||||
for b in bad:
|
||||
print(" " + b)
|
||||
return 1
|
||||
print("cohere spec ok: %d calls of %d spec operations (exclusions: %d), %d schema rows, %d fields"
|
||||
% (len(calls), len(ops), len(EXCLUDED_OPERATIONS), len(schemas), checked_fields))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Vendored
+213
@@ -0,0 +1,213 @@
|
||||
#include "vendors/cohere/types.h"
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
std::string J::str(const Json& doc, const char* key, const std::string& fallback) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : fallback;
|
||||
}
|
||||
long J::num(const Json& doc, const char* key, long fallback) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_number() ? v->as_int() : fallback;
|
||||
}
|
||||
double J::real(const Json& doc, const char* key, double fallback) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_number() ? v->as_number() : fallback;
|
||||
}
|
||||
bool J::flag(const Json& doc, const char* key, bool fallback) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_bool() ? v->as_bool() : fallback;
|
||||
}
|
||||
const Json* J::obj(const Json& doc, const char* key) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_object() ? v : nullptr;
|
||||
}
|
||||
const Json* J::arr(const Json& doc, const char* key) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_array() ? v : nullptr;
|
||||
}
|
||||
std::vector<std::string> J::strings(const Json& doc, const char* key) {
|
||||
std::vector<std::string> out;
|
||||
if (const Json* a = arr(doc, key))
|
||||
for (const Json& v : a->as_array())
|
||||
if (v.is_string()) out.push_back(v.as_string());
|
||||
return out;
|
||||
}
|
||||
|
||||
TokenUsage TokenUsage::parse(const Json* usage) {
|
||||
TokenUsage u;
|
||||
if (!usage || !usage->is_object()) return u;
|
||||
u.present = true;
|
||||
if (const Json* t = J::obj(*usage, "tokens")) {
|
||||
u.input_tokens = J::num(*t, "input_tokens");
|
||||
u.output_tokens = J::num(*t, "output_tokens");
|
||||
}
|
||||
if (const Json* b = J::obj(*usage, "billed_units")) {
|
||||
u.billed_input_tokens = J::num(*b, "input_tokens");
|
||||
u.billed_output_tokens = J::num(*b, "output_tokens");
|
||||
u.search_units = J::num(*b, "search_units");
|
||||
u.classifications = J::num(*b, "classifications");
|
||||
}
|
||||
u.cached_tokens = J::num(*usage, "cached_tokens");
|
||||
return u;
|
||||
}
|
||||
|
||||
static Page page_of(const Json& doc, const char* total_key = nullptr) {
|
||||
Page p;
|
||||
p.next_page_token = J::str(doc, "next_page_token");
|
||||
if (total_key) p.total = J::num(doc, total_key, -1);
|
||||
return p;
|
||||
}
|
||||
|
||||
bool ModelInfo::supports(const std::string& endpoint) const {
|
||||
return std::find(endpoints.begin(), endpoints.end(), endpoint) != endpoints.end();
|
||||
}
|
||||
|
||||
ModelInfo ModelInfo::parse(const Json& doc) {
|
||||
ModelInfo m;
|
||||
m.name = J::str(doc, "name");
|
||||
m.deprecated = J::flag(doc, "is_deprecated");
|
||||
m.finetuned = J::flag(doc, "finetuned");
|
||||
m.context_length = J::real(doc, "context_length");
|
||||
m.tokenizer_url = J::str(doc, "tokenizer_url");
|
||||
m.endpoints = J::strings(doc, "endpoints");
|
||||
m.default_endpoints = J::strings(doc, "default_endpoints");
|
||||
m.features = J::strings(doc, "features");
|
||||
m.raw = doc;
|
||||
return m;
|
||||
}
|
||||
|
||||
ModelList ModelList::parse(const Json& doc) {
|
||||
ModelList l;
|
||||
if (const Json* a = J::arr(doc, "models"))
|
||||
for (const Json& m : a->as_array()) l.models.push_back(ModelInfo::parse(m));
|
||||
l.page = page_of(doc);
|
||||
return l;
|
||||
}
|
||||
|
||||
const ModelInfo* ModelList::find(const std::string& name) const {
|
||||
for (const ModelInfo& m : models)
|
||||
if (m.name == name) return &m;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
ApiKeyCheck ApiKeyCheck::parse(const Json& doc) {
|
||||
ApiKeyCheck c;
|
||||
c.valid = J::flag(doc, "valid");
|
||||
c.organization_id = J::str(doc, "organization_id");
|
||||
c.owner_id = J::str(doc, "owner_id");
|
||||
return c;
|
||||
}
|
||||
|
||||
bool Batch::terminal() const {
|
||||
return status == "BATCH_STATUS_COMPLETED" || status == "BATCH_STATUS_FAILED" ||
|
||||
status == "BATCH_STATUS_CANCELED";
|
||||
}
|
||||
|
||||
Batch Batch::parse(const Json& doc) {
|
||||
Batch b;
|
||||
b.id = J::str(doc, "id");
|
||||
b.name = J::str(doc, "name");
|
||||
b.status = J::str(doc, "status");
|
||||
b.created_at = J::str(doc, "created_at");
|
||||
b.updated_at = J::str(doc, "updated_at");
|
||||
b.input_dataset_id = J::str(doc, "input_dataset_id");
|
||||
b.output_dataset_id = J::str(doc, "output_dataset_id");
|
||||
b.model = J::str(doc, "model");
|
||||
b.status_reason = J::str(doc, "status_reason");
|
||||
b.input_tokens = J::str(doc, "input_tokens");
|
||||
b.output_tokens = J::str(doc, "output_tokens");
|
||||
b.num_records = J::num(doc, "num_records");
|
||||
b.num_successful_records = J::num(doc, "num_successful_records");
|
||||
b.num_failed_records = J::num(doc, "num_failed_records");
|
||||
b.raw = doc;
|
||||
return b;
|
||||
}
|
||||
|
||||
BatchList BatchList::parse(const Json& doc) {
|
||||
BatchList l;
|
||||
if (const Json* a = J::arr(doc, "batches"))
|
||||
for (const Json& b : a->as_array()) l.batches.push_back(Batch::parse(b));
|
||||
l.page = page_of(doc);
|
||||
return l;
|
||||
}
|
||||
|
||||
bool EmbedJob::terminal() const { return status == "complete" || status == "cancelled" || status == "failed"; }
|
||||
|
||||
EmbedJob EmbedJob::parse(const Json& doc) {
|
||||
EmbedJob j;
|
||||
j.job_id = J::str(doc, "job_id");
|
||||
j.name = J::str(doc, "name");
|
||||
j.status = J::str(doc, "status");
|
||||
j.created_at = J::str(doc, "created_at");
|
||||
j.input_dataset_id = J::str(doc, "input_dataset_id");
|
||||
j.output_dataset_id = J::str(doc, "output_dataset_id");
|
||||
j.model = J::str(doc, "model");
|
||||
j.truncate = J::str(doc, "truncate");
|
||||
j.raw = doc;
|
||||
return j;
|
||||
}
|
||||
|
||||
DatasetPart DatasetPart::parse(const Json& doc) {
|
||||
DatasetPart p;
|
||||
p.id = J::str(doc, "id");
|
||||
p.name = J::str(doc, "name");
|
||||
p.url = J::str(doc, "url");
|
||||
p.original_url = J::str(doc, "original_url");
|
||||
p.index = J::num(doc, "index");
|
||||
p.size_bytes = J::num(doc, "size_bytes");
|
||||
p.num_rows = J::num(doc, "num_rows");
|
||||
return p;
|
||||
}
|
||||
|
||||
Dataset Dataset::parse(const Json& doc) {
|
||||
Dataset d;
|
||||
d.id = J::str(doc, "id");
|
||||
d.name = J::str(doc, "name");
|
||||
d.created_at = J::str(doc, "created_at");
|
||||
d.updated_at = J::str(doc, "updated_at");
|
||||
d.dataset_type = J::str(doc, "dataset_type");
|
||||
d.validation_status = J::str(doc, "validation_status");
|
||||
d.validation_error = J::str(doc, "validation_error");
|
||||
d.validation_warnings = J::strings(doc, "validation_warnings");
|
||||
if (const Json* a = J::arr(doc, "dataset_parts"))
|
||||
for (const Json& p : a->as_array()) d.parts.push_back(DatasetPart::parse(p));
|
||||
d.raw = doc;
|
||||
return d;
|
||||
}
|
||||
|
||||
Connector Connector::parse(const Json& doc) {
|
||||
Connector c;
|
||||
c.id = J::str(doc, "id");
|
||||
c.organization_id = J::str(doc, "organization_id");
|
||||
c.name = J::str(doc, "name");
|
||||
c.url = J::str(doc, "url");
|
||||
c.created_at = J::str(doc, "created_at");
|
||||
c.updated_at = J::str(doc, "updated_at");
|
||||
c.auth_type = J::str(doc, "auth_type");
|
||||
c.auth_status = J::str(doc, "auth_status");
|
||||
c.excludes = J::strings(doc, "excludes");
|
||||
c.active = J::flag(doc, "active");
|
||||
c.continue_on_failure = J::flag(doc, "continue_on_failure");
|
||||
c.raw = doc;
|
||||
return c;
|
||||
}
|
||||
|
||||
FinetunedModel FinetunedModel::parse(const Json& doc) {
|
||||
FinetunedModel m;
|
||||
m.id = J::str(doc, "id");
|
||||
m.name = J::str(doc, "name");
|
||||
m.creator_id = J::str(doc, "creator_id");
|
||||
m.organization_id = J::str(doc, "organization_id");
|
||||
m.status = J::str(doc, "status");
|
||||
m.created_at = J::str(doc, "created_at");
|
||||
m.updated_at = J::str(doc, "updated_at");
|
||||
m.completed_at = J::str(doc, "completed_at");
|
||||
m.last_used = J::str(doc, "last_used");
|
||||
if (const Json* s = J::obj(doc, "settings")) m.settings = *s;
|
||||
return m;
|
||||
}
|
||||
|
||||
} // namespace molo::cohere
|
||||
Vendored
+125
@@ -0,0 +1,125 @@
|
||||
#pragma once
|
||||
|
||||
// Typed views of Cohere documents. Every parser reads only fields the call
|
||||
// table's schema rows declare; the full document stays available as `raw`.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::cohere {
|
||||
|
||||
using Json = clients::body::Json;
|
||||
|
||||
// Tolerant readers: a missing or mistyped field gives the fallback.
|
||||
struct J {
|
||||
static std::string str(const Json& doc, const char* key, const std::string& fallback = "");
|
||||
static long num(const Json& doc, const char* key, long fallback = 0);
|
||||
static double real(const Json& doc, const char* key, double fallback = 0);
|
||||
static bool flag(const Json& doc, const char* key, bool fallback = false);
|
||||
static const Json* obj(const Json& doc, const char* key);
|
||||
static const Json* arr(const Json& doc, const char* key);
|
||||
static std::vector<std::string> strings(const Json& doc, const char* key);
|
||||
};
|
||||
|
||||
// Usage (chat) and ApiMeta (everything else) share these fields.
|
||||
struct TokenUsage {
|
||||
long input_tokens = 0; // tokens.input_tokens
|
||||
long output_tokens = 0; // tokens.output_tokens
|
||||
long billed_input_tokens = 0; // billed_units.input_tokens
|
||||
long billed_output_tokens = 0;
|
||||
long search_units = 0;
|
||||
long classifications = 0;
|
||||
long cached_tokens = 0;
|
||||
bool present = false;
|
||||
static TokenUsage parse(const Json* usage);
|
||||
};
|
||||
|
||||
struct Page {
|
||||
std::string next_page_token;
|
||||
long total = -1; // total_size / total_count when sent
|
||||
bool truncated = false; // more pages existed than the walk cap allowed
|
||||
};
|
||||
|
||||
struct ModelInfo {
|
||||
std::string name;
|
||||
bool deprecated = false;
|
||||
bool finetuned = false;
|
||||
double context_length = 0;
|
||||
std::string tokenizer_url;
|
||||
std::vector<std::string> endpoints;
|
||||
std::vector<std::string> default_endpoints;
|
||||
std::vector<std::string> features;
|
||||
Json raw;
|
||||
bool supports(const std::string& endpoint) const;
|
||||
static ModelInfo parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct ModelList {
|
||||
std::vector<ModelInfo> models;
|
||||
Page page;
|
||||
const ModelInfo* find(const std::string& name) const;
|
||||
static ModelList parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct ApiKeyCheck {
|
||||
bool valid = false;
|
||||
std::string organization_id;
|
||||
std::string owner_id;
|
||||
static ApiKeyCheck parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct Batch {
|
||||
std::string id, name, status, created_at, updated_at, input_dataset_id, output_dataset_id, model;
|
||||
std::string status_reason, input_tokens, output_tokens; // token counts are strings in the spec
|
||||
long num_records = 0, num_successful_records = 0, num_failed_records = 0;
|
||||
Json raw;
|
||||
bool terminal() const; // COMPLETED, FAILED or CANCELED
|
||||
static Batch parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct BatchList {
|
||||
std::vector<Batch> batches;
|
||||
Page page;
|
||||
static BatchList parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct EmbedJob {
|
||||
std::string job_id, name, status, created_at, input_dataset_id, output_dataset_id, model, truncate;
|
||||
Json raw;
|
||||
bool terminal() const; // complete, cancelled or failed
|
||||
static EmbedJob parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct DatasetPart {
|
||||
std::string id, name, url, original_url;
|
||||
long index = 0, size_bytes = 0, num_rows = 0;
|
||||
static DatasetPart parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct Dataset {
|
||||
std::string id, name, created_at, updated_at, dataset_type, validation_status, validation_error;
|
||||
std::vector<std::string> validation_warnings;
|
||||
std::vector<DatasetPart> parts;
|
||||
Json raw;
|
||||
bool validated() const { return validation_status == "validated"; }
|
||||
static Dataset parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct Connector {
|
||||
std::string id, organization_id, name, url, created_at, updated_at, auth_type, auth_status;
|
||||
std::vector<std::string> excludes;
|
||||
bool active = false, continue_on_failure = false;
|
||||
Json raw;
|
||||
static Connector parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct FinetunedModel {
|
||||
std::string id, name, creator_id, organization_id, status, created_at, updated_at, completed_at, last_used;
|
||||
Json settings;
|
||||
bool ready() const { return status == "STATUS_READY"; }
|
||||
static FinetunedModel parse(const Json& doc);
|
||||
};
|
||||
|
||||
} // namespace molo::cohere
|
||||
Reference in New Issue
Block a user