Add community node protocol spec, Agent Skills, and make the failing tests hermetic
DevPlace CI / test (push) Failing after 31m42s
DevPlace CI / test (push) Failing after 31m42s
Community nodes (Phase 1 of plan.md): the protocol specification in services/nodes/spec.py as the single source of truth, GET /nodes/spec.json, a generated protocol docs page, the Community nodes API docs group, and the implementation plan plus the residential proxy research. The no-open-proxy rule is enforced in the spec (open_proxy cannot be set). Agent Skills: register the Skills API docs group, cascade skills on account deletion, add the nested CLAUDE.md, README and routes docs, and cover the parser, the fetcher (end to end against a mock origin), the data layer and the routes with 73 unit and 16 api tests. Tests made hermetic with the repo's existing patterns instead of ambient environment: embeddings tests set OPENROUTER_API_KEY via monkeypatch, the workspace tests install FakeBackend via runtime.set_backend, and the skills install tests use run_async. DockerCliBackend now returns 127 (command not found) when the CLI is missing instead of raising FileNotFoundError, so a host without docker answers with a clean error rather than a 500. The Providers tab empty state no longer names the Model routes tab. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -239,3 +239,37 @@ The split it enforces is the whole point:
|
||||
- owner kind `internal` / `key` / `anonymous` -> **platform processing** (news import, the bot fleet, SEO metadata, issue enhancement). Never gated: it is not the user's content.
|
||||
|
||||
`ai_third_party` is **never granted at signup**. The existing `ai_correction_enabled` / `ai_modifier_enabled` user columns survive unchanged as *preferences* subordinate to consent: withdrawing consent turns those features off regardless of the flag, so `ai_modifier_enabled`'s default of `1` is harmless. No existing preference was flipped and no consumer changed - the gate was simply added above them. `USER_CONTENT_OWNER_KINDS` is the single tuple defining the split; widen it only if a new owner kind genuinely carries a user's own content.
|
||||
|
||||
## Responses-protocol providers
|
||||
|
||||
A provider row (or the default upstream) can speak the Responses API
|
||||
instead of Chat Completions. `ProviderIn.protocol` is `chat_completions`
|
||||
(default, forwarded verbatim) or `responses` (transpiled both ways).
|
||||
`chat_overlay` carries the provider choice as `gateway_upstream_protocol`;
|
||||
`gateway_upstream_protocol` is also a service config field for the
|
||||
default upstream. A responses provider's base URL ends in `/responses`
|
||||
(e.g. `https://api.meta.ai/v1/responses`); the models probe tries
|
||||
`/models` then `/muse-code/models` next to it.
|
||||
|
||||
`responses.py` is the transpiler (stdlib only, no gateway imports):
|
||||
`chat_to_responses` / `responses_to_chat` for requests,
|
||||
`responses_object_to_chat_completion` /
|
||||
`chat_completion_to_responses_object` for objects,
|
||||
`ResponsesToChatStream` / `ChatToResponsesStream` for SSE. Every
|
||||
function returns `(converted, warnings)`; lossy mappings warn, never
|
||||
drop silently. `handle_chat` keeps reasoning about a Chat payload
|
||||
through routing/vision/precheck/fallback and transpiles at the send
|
||||
boundary (`_chat_upstream_body`), so fallbacks can cross protocols per
|
||||
attempt. Responses upstreams skip `apply_thinking`; reasoning effort
|
||||
comes from the client (`reasoning_effort`, `reasoning.effort`,
|
||||
thinking flags) or defaults to `{"effort": "minimal"}` when thinking
|
||||
is off - never `"none"`, which is not a value in the Responses
|
||||
reasoning-effort enum (confirmed by a real upstream rejecting it:
|
||||
`minimal/low/medium/high/xhigh/max`). `stream_options.include_usage`
|
||||
is never sent upstream (usage
|
||||
arrives on `response.completed`); the client's flag still controls
|
||||
the relayed usage chunk. Non-200 `{error}` envelopes and 200
|
||||
failed/incomplete Response objects are detected and relayed as
|
||||
Chat-style errors. Passthrough stays raw (no transpiling) and strips
|
||||
a trailing `/responses` base segment so `/openai/v1/responses` routes
|
||||
without doubling the path.
|
||||
|
||||
@@ -20,9 +20,17 @@ from devplacepy.services.openai_gateway.routing import (
|
||||
image_overlay,
|
||||
resolve_fallback,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.responses import (
|
||||
ResponsesToChatStream,
|
||||
chat_to_responses,
|
||||
responses_error_object,
|
||||
responses_failure_message,
|
||||
responses_object_to_chat_completion,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.system_message import apply_system_directives
|
||||
from devplacepy.services.openai_gateway.thinking import (
|
||||
apply_thinking,
|
||||
client_thinking_enabled,
|
||||
upstream_capabilities,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.usage import (
|
||||
@@ -76,7 +84,9 @@ def _should_fallback(resp, exc, timing: dict, body_error: Optional[str] = None)
|
||||
return True
|
||||
|
||||
|
||||
def _embedded_failure_message(resp, stream: bool) -> Optional[str]:
|
||||
def _embedded_failure_message(
|
||||
resp, stream: bool, protocol: str = "chat_completions"
|
||||
) -> Optional[str]:
|
||||
# A 200 with the failure embedded in the JSON body (OpenRouter and the
|
||||
# providers it fronts do this - see usage.embedded_error_message) is
|
||||
# invisible to _call_failed's status-code check. Only safe to peek for a
|
||||
@@ -88,9 +98,35 @@ def _embedded_failure_message(resp, stream: bool) -> Optional[str]:
|
||||
data = resp.json()
|
||||
except (ValueError, httpx.HTTPError):
|
||||
return None
|
||||
if protocol == "responses":
|
||||
return responses_failure_message(data)
|
||||
return embedded_error_message(data)
|
||||
|
||||
|
||||
def _chat_upstream_body(
|
||||
payload: dict, client_body: dict, cfg: dict, protocol: str, log
|
||||
) -> dict:
|
||||
# The handler reasons about a Chat payload throughout (routing, vision,
|
||||
# precheck, fallback); only the bytes on the wire change per protocol.
|
||||
if protocol != "responses":
|
||||
return payload
|
||||
send_body, warnings = chat_to_responses(payload)
|
||||
if warnings:
|
||||
shown = "; ".join(warnings[:3])
|
||||
more = f" (+{len(warnings) - 3} more)" if len(warnings) > 3 else ""
|
||||
log(f"chat->responses transpile warnings: {shown}{more}")
|
||||
if "reasoning" not in send_body:
|
||||
explicit = client_thinking_enabled(client_body)
|
||||
enabled = (
|
||||
explicit
|
||||
if explicit is not None
|
||||
else bool(cfg.get("gateway_thinking", False))
|
||||
)
|
||||
if not enabled:
|
||||
send_body["reasoning"] = {"effort": "minimal"}
|
||||
return send_body
|
||||
|
||||
|
||||
def _apply_served_model(headers: dict, resp) -> dict:
|
||||
# Everything else in `headers` (cost, tokens, latency, context) is our
|
||||
# own measurement and must never be overwritten by an upstream that
|
||||
@@ -366,6 +402,7 @@ class GatewayRuntime:
|
||||
if overlay:
|
||||
cfg = {**cfg, **overlay}
|
||||
log(f"routed model {body.get('model')!r} -> {cfg['gateway_model']!r}")
|
||||
protocol = cfg.get("gateway_upstream_protocol") or "chat_completions"
|
||||
client, sem = self._ensure(cfg)
|
||||
pricing = pricing_from_cfg(cfg)
|
||||
context_map = parse_context_map(cfg.get("gateway_model_context_map"))
|
||||
@@ -430,7 +467,13 @@ class GatewayRuntime:
|
||||
payload["model"] = model
|
||||
payload["messages"] = messages
|
||||
payload["stream"] = stream
|
||||
if stream:
|
||||
if stream and protocol == "responses":
|
||||
# Responses streams carry usage on response.completed natively;
|
||||
# there is no include_usage to request (stream_options only
|
||||
# supports include_obfuscation there). The translator captures it
|
||||
# and the client's own include_usage still controls the relay.
|
||||
payload.pop("stream_options", None)
|
||||
if stream and protocol != "responses":
|
||||
capabilities = upstream_capabilities(
|
||||
cfg.get("gateway_upstream_url", ""),
|
||||
cfg.get("gateway_thinking_dialect", "auto"),
|
||||
@@ -450,12 +493,16 @@ class GatewayRuntime:
|
||||
)
|
||||
else:
|
||||
payload.pop("stream_options", None)
|
||||
apply_thinking(
|
||||
payload,
|
||||
cfg.get("gateway_upstream_url", ""),
|
||||
default_enabled=bool(cfg.get("gateway_thinking", False)),
|
||||
dialect=cfg.get("gateway_thinking_dialect", "auto"),
|
||||
)
|
||||
if protocol != "responses":
|
||||
apply_thinking(
|
||||
payload,
|
||||
cfg.get("gateway_upstream_url", ""),
|
||||
default_enabled=bool(cfg.get("gateway_thinking", False)),
|
||||
dialect=cfg.get("gateway_thinking_dialect", "auto"),
|
||||
)
|
||||
# For responses upstreams the reasoning effort is set at transpile
|
||||
# time instead (see _chat_upstream_body): chat-dialect thinking
|
||||
# fields have no meaning on that wire.
|
||||
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
@@ -493,12 +540,17 @@ class GatewayRuntime:
|
||||
)
|
||||
fallback_used_route = precheck_route.source_model
|
||||
cfg = {**base_cfg, **precheck_overlay}
|
||||
protocol = (
|
||||
cfg.get("gateway_upstream_protocol")
|
||||
or "chat_completions"
|
||||
)
|
||||
model = cfg["gateway_model"]
|
||||
payload = dict(payload)
|
||||
payload["model"] = model
|
||||
headers = _fallback_headers(cfg, "gateway_api_key")
|
||||
|
||||
send_start = time.monotonic()
|
||||
send_body = _chat_upstream_body(payload, body, cfg, protocol, log)
|
||||
resp, exc, timing = await self._send(
|
||||
client,
|
||||
sem,
|
||||
@@ -507,7 +559,7 @@ class GatewayRuntime:
|
||||
headers,
|
||||
cfg,
|
||||
log,
|
||||
json_body=payload,
|
||||
json_body=send_body,
|
||||
stream=stream,
|
||||
)
|
||||
if resp is not None and stream and resp.status_code != 200:
|
||||
@@ -522,7 +574,7 @@ class GatewayRuntime:
|
||||
log(f"failed to read non-200 stream error body: {e}")
|
||||
|
||||
requested_model = requested
|
||||
body_error = _embedded_failure_message(resp, stream)
|
||||
body_error = _embedded_failure_message(resp, stream, protocol)
|
||||
if fallback_used_route is None and _should_fallback(resp, exc, timing, body_error):
|
||||
if body_error:
|
||||
log(f"model {requested!r} returned 200 with an embedded error: {body_error}")
|
||||
@@ -539,11 +591,17 @@ class GatewayRuntime:
|
||||
)
|
||||
fallback_used_route = fallback_route.source_model
|
||||
cfg = {**base_cfg, **fallback_overlay}
|
||||
protocol = (
|
||||
cfg.get("gateway_upstream_protocol") or "chat_completions"
|
||||
)
|
||||
model = cfg["gateway_model"]
|
||||
payload = dict(payload)
|
||||
payload["model"] = model
|
||||
headers = _fallback_headers(cfg, "gateway_api_key")
|
||||
send_start = time.monotonic()
|
||||
send_body = _chat_upstream_body(
|
||||
payload, body, cfg, protocol, log
|
||||
)
|
||||
resp, exc, timing = await self._send(
|
||||
client,
|
||||
sem,
|
||||
@@ -552,7 +610,7 @@ class GatewayRuntime:
|
||||
headers,
|
||||
cfg,
|
||||
log,
|
||||
json_body=payload,
|
||||
json_body=send_body,
|
||||
stream=stream,
|
||||
)
|
||||
if resp is not None and stream and resp.status_code != 200:
|
||||
@@ -560,7 +618,7 @@ class GatewayRuntime:
|
||||
await resp.aread()
|
||||
except httpx.HTTPError as e: # noqa: BLE001
|
||||
log(f"failed to read fallback non-200 stream error body: {e}")
|
||||
body_error = _embedded_failure_message(resp, stream)
|
||||
body_error = _embedded_failure_message(resp, stream, protocol)
|
||||
|
||||
pricing = pricing_from_cfg(cfg)
|
||||
context_map = parse_context_map(cfg.get("gateway_model_context_map"))
|
||||
@@ -636,9 +694,20 @@ class GatewayRuntime:
|
||||
classify_error(resp.status_code, None, resp.text),
|
||||
)
|
||||
log(f"chat upstream POST -> {resp.status_code}: {resp.text[:300]}")
|
||||
content: dict = {
|
||||
"error": {"message": resp.text, "type": "upstream_error"}
|
||||
}
|
||||
if protocol == "responses":
|
||||
try:
|
||||
parsed_error = resp.json()
|
||||
except (ValueError, httpx.HTTPError):
|
||||
parsed_error = None
|
||||
relayed = responses_error_object(parsed_error)
|
||||
if relayed is not None:
|
||||
content = relayed
|
||||
return JSONResponse(
|
||||
status_code=resp.status_code,
|
||||
content={"error": {"message": resp.text, "type": "upstream_error"}},
|
||||
content=content,
|
||||
headers=resp_headers,
|
||||
)
|
||||
if body_error:
|
||||
@@ -661,8 +730,8 @@ class GatewayRuntime:
|
||||
# non-streaming response below, this response carries no
|
||||
# X-Gateway-Cost-USD/token headers (HTTP headers must precede the body).
|
||||
log(f"chat POST -> 200 stream ({timing['upstream_latency_ms']:.0f}ms to first byte)")
|
||||
return StreamingResponse(
|
||||
self._stream_chat_response(
|
||||
if protocol == "responses":
|
||||
body_iter = self._stream_responses_response(
|
||||
resp,
|
||||
base,
|
||||
pricing,
|
||||
@@ -671,7 +740,21 @@ class GatewayRuntime:
|
||||
handle_start,
|
||||
send_start,
|
||||
log,
|
||||
),
|
||||
model,
|
||||
)
|
||||
else:
|
||||
body_iter = self._stream_chat_response(
|
||||
resp,
|
||||
base,
|
||||
pricing,
|
||||
context_map,
|
||||
include_usage,
|
||||
handle_start,
|
||||
send_start,
|
||||
log,
|
||||
)
|
||||
return StreamingResponse(
|
||||
body_iter,
|
||||
media_type="text/event-stream",
|
||||
headers=_apply_served_model(
|
||||
{
|
||||
@@ -698,6 +781,22 @@ class GatewayRuntime:
|
||||
},
|
||||
headers=resp_headers,
|
||||
)
|
||||
if protocol == "responses":
|
||||
converted, convert_warnings = responses_object_to_chat_completion(
|
||||
data if isinstance(data, dict) else {}
|
||||
)
|
||||
if convert_warnings:
|
||||
log(
|
||||
"responses->chat conversion warnings: "
|
||||
+ "; ".join(convert_warnings[:3])
|
||||
)
|
||||
resp_headers = finalize(200, True, None, converted.get("usage"))
|
||||
log(f"chat POST -> 200 ({timing['upstream_latency_ms']:.0f}ms)")
|
||||
return Response(
|
||||
content=json.dumps(converted).encode("utf-8"),
|
||||
media_type="application/json",
|
||||
headers=resp_headers,
|
||||
)
|
||||
resp_headers = finalize(200, True, None, data.get("usage"))
|
||||
log(f"chat POST -> 200 ({timing['upstream_latency_ms']:.0f}ms)")
|
||||
return Response(
|
||||
@@ -706,6 +805,115 @@ class GatewayRuntime:
|
||||
headers=resp_headers,
|
||||
)
|
||||
|
||||
async def _stream_responses_response(
|
||||
self,
|
||||
resp: httpx.Response,
|
||||
base: dict,
|
||||
pricing,
|
||||
context_map: dict,
|
||||
include_usage: bool,
|
||||
handle_start: float,
|
||||
send_start: float,
|
||||
log,
|
||||
model: str,
|
||||
):
|
||||
# Same contract as _stream_chat_response (real-time relay, TTFT /
|
||||
# inter-token metering, usage capture, ledger row in `finally`,
|
||||
# [DONE] termination), but the upstream speaks Responses SSE, so
|
||||
# each event is translated to Chat chunks before relay.
|
||||
translator = ResponsesToChatStream(model=model)
|
||||
first_at: Optional[float] = None
|
||||
last_at: Optional[float] = None
|
||||
content_chunks = 0
|
||||
usage_captured: Optional[dict] = None
|
||||
saw_done = False
|
||||
success = True
|
||||
error_category: Optional[str] = None
|
||||
try:
|
||||
async for raw_line in resp.aiter_lines():
|
||||
line = raw_line.strip()
|
||||
if not line or not line.startswith("data:"):
|
||||
continue
|
||||
now = time.monotonic()
|
||||
if first_at is None:
|
||||
first_at = now
|
||||
last_at = now
|
||||
data_str = line[len("data:") :].strip()
|
||||
if data_str == "[DONE]":
|
||||
saw_done = True
|
||||
yield "data: [DONE]\n\n"
|
||||
break
|
||||
try:
|
||||
event = json.loads(data_str)
|
||||
except ValueError:
|
||||
continue
|
||||
for out in translator.feed(event):
|
||||
if "choices" not in out:
|
||||
success = False
|
||||
error_category = "upstream_error"
|
||||
yield f"data: {json.dumps(out)}\n\n"
|
||||
continue
|
||||
choices = out.get("choices") or []
|
||||
delta = (choices[0].get("delta") if choices else None) or {}
|
||||
if (
|
||||
delta.get("content")
|
||||
or delta.get("refusal")
|
||||
or delta.get("tool_calls")
|
||||
):
|
||||
content_chunks += 1
|
||||
usage = out.get("usage")
|
||||
if usage:
|
||||
usage_captured = usage
|
||||
if not include_usage:
|
||||
continue
|
||||
yield f"data: {json.dumps(out)}\n\n"
|
||||
except GeneratorExit:
|
||||
success = False
|
||||
error_category = "client_disconnected"
|
||||
raise
|
||||
except Exception as exc: # noqa: BLE001 - an upstream stream can drop mid-flight; never crash the response
|
||||
success = False
|
||||
error_category = classify_error(0, exc)
|
||||
log(f"chat stream interrupted: {exc}")
|
||||
finally:
|
||||
try:
|
||||
await resp.aclose()
|
||||
except Exception: # noqa: BLE001 - releasing the connection must never mask the real outcome
|
||||
pass
|
||||
ttft_ms = (
|
||||
round((first_at - send_start) * 1000, 3)
|
||||
if first_at is not None
|
||||
else None
|
||||
)
|
||||
inter_token_ms = (
|
||||
round((last_at - first_at) * 1000 / (content_chunks - 1), 3)
|
||||
if first_at is not None
|
||||
and last_at is not None
|
||||
and content_chunks > 1
|
||||
else None
|
||||
)
|
||||
base["total_latency_ms"] = round(
|
||||
(time.monotonic() - handle_start) * 1000, 3
|
||||
)
|
||||
base["gateway_overhead_ms"] = round(
|
||||
max(
|
||||
base["total_latency_ms"]
|
||||
- base.get("upstream_latency_ms", 0.0)
|
||||
- base.get("queue_wait_ms", 0.0),
|
||||
0.0,
|
||||
),
|
||||
3,
|
||||
)
|
||||
base["status_code"] = 200
|
||||
base["success"] = success
|
||||
base["error_category"] = error_category
|
||||
base["usage"] = usage_captured
|
||||
base["ttft_ms"] = ttft_ms
|
||||
base["inter_token_ms"] = inter_token_ms
|
||||
self._ledger.record(base, pricing, context_map)
|
||||
if not saw_done:
|
||||
yield "data: [DONE]\n\n"
|
||||
|
||||
async def _stream_chat_response(
|
||||
self,
|
||||
resp: httpx.Response,
|
||||
@@ -1307,6 +1515,11 @@ class GatewayRuntime:
|
||||
base_url = cfg["gateway_upstream_url"]
|
||||
if base_url.endswith("/chat/completions"):
|
||||
base_url = base_url[: -len("/chat/completions")]
|
||||
elif base_url.endswith("/responses"):
|
||||
# A responses-protocol base stays raw-passthrough capable, so an
|
||||
# explicit /openai/v1/responses call routes without doubling the
|
||||
# path segment. No transpiling happens on this path by design.
|
||||
base_url = base_url[: -len("/responses")]
|
||||
url = f"{base_url.rstrip('/')}/{subpath}"
|
||||
headers = {}
|
||||
if cfg["gateway_api_key"]:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -36,6 +36,14 @@ CLIENT_PROFILES: dict = {
|
||||
"opencode": opencode_zen.opencode_impersonation_headers,
|
||||
}
|
||||
|
||||
# Upstream wire protocol a provider speaks. `chat_completions` is the
|
||||
# classic OpenAI shape (base_url ends in /chat/completions, forwarded
|
||||
# verbatim). `responses` is the Responses API shape (base_url ends in
|
||||
# /responses); the gateway transpiles inbound Chat requests to it and
|
||||
# converts Response objects / SSE events back to the normal OpenAI surface.
|
||||
PROVIDER_PROTOCOLS = ("chat_completions", "responses")
|
||||
PROTOCOL_DEFAULT = "chat_completions"
|
||||
|
||||
_ROUTING_CACHE: dict = {}
|
||||
|
||||
|
||||
@@ -78,6 +86,20 @@ def _models_url_from_base(base_url: str) -> str:
|
||||
return base_url.rstrip("/") + "/models"
|
||||
|
||||
|
||||
def _responses_models_urls(base_url: str) -> list:
|
||||
# A Responses-protocol provider's model listing is vendor-specific:
|
||||
# OpenAI serves /models next to /responses, Meta serves
|
||||
# /muse-code/models. Probe both, first success wins.
|
||||
base_url = (base_url or "").strip()
|
||||
if not base_url:
|
||||
return []
|
||||
if base_url.endswith("/responses"):
|
||||
root = base_url[: -len("/responses")].rstrip("/")
|
||||
else:
|
||||
root = base_url.rstrip("/")
|
||||
return [root + "/models", root + "/muse-code/models"]
|
||||
|
||||
|
||||
PROVIDER_MODELS_PROBE_TIMEOUT = 8.0
|
||||
|
||||
|
||||
@@ -89,6 +111,14 @@ def _default_provider_credentials() -> tuple[str, str]:
|
||||
return cfg.get("gateway_upstream_url", ""), cfg.get("gateway_api_key", "")
|
||||
|
||||
|
||||
def _default_provider_protocol() -> str:
|
||||
from devplacepy.services.manager import service_manager
|
||||
|
||||
svc = service_manager.get_service("openai")
|
||||
cfg = svc.get_config() if svc is not None else {}
|
||||
return cfg.get("gateway_upstream_protocol", PROTOCOL_DEFAULT) or PROTOCOL_DEFAULT
|
||||
|
||||
|
||||
async def fetch_provider_models(
|
||||
provider_name: str, timeout: float = PROVIDER_MODELS_PROBE_TIMEOUT
|
||||
) -> Optional[list[str]]:
|
||||
@@ -97,33 +127,53 @@ async def fetch_provider_models(
|
||||
provider = provider_store.get(provider_name)
|
||||
if provider is None:
|
||||
return None
|
||||
base_url, api_key = provider.get("base_url", ""), provider.get("api_key", "")
|
||||
base_url = provider.get("base_url", "")
|
||||
api_key = provider.get("api_key", "")
|
||||
protocol = provider.get("protocol", PROTOCOL_DEFAULT) or PROTOCOL_DEFAULT
|
||||
else:
|
||||
base_url, api_key = _default_provider_credentials()
|
||||
models_url = _models_url_from_base(base_url)
|
||||
if not models_url:
|
||||
protocol = _default_provider_protocol()
|
||||
if protocol == "responses":
|
||||
models_urls = _responses_models_urls(base_url)
|
||||
else:
|
||||
single = _models_url_from_base(base_url)
|
||||
models_urls = [single] if single else []
|
||||
if not models_urls:
|
||||
return None
|
||||
|
||||
from devplacepy import stealth
|
||||
|
||||
headers = {"authorization": f"Bearer {api_key}"} if api_key else {}
|
||||
try:
|
||||
async with stealth.stealth_async_client(timeout=timeout, headers=headers) as client:
|
||||
response = await client.get(models_url)
|
||||
except Exception as exc:
|
||||
logger.info("gateway provider model listing failed for %s: %s", models_url, exc)
|
||||
return None
|
||||
if response.status_code != 200:
|
||||
return None
|
||||
try:
|
||||
payload = response.json()
|
||||
except Exception:
|
||||
return None
|
||||
rows = payload.get("data") if isinstance(payload, dict) else None
|
||||
if not isinstance(rows, list):
|
||||
return None
|
||||
ids = [str(row["id"]) for row in rows if isinstance(row, dict) and row.get("id")]
|
||||
return ids or None
|
||||
for models_url in models_urls:
|
||||
try:
|
||||
async with stealth.stealth_async_client(
|
||||
timeout=timeout, headers=headers
|
||||
) as client:
|
||||
response = await client.get(models_url)
|
||||
except Exception as exc:
|
||||
logger.info(
|
||||
"gateway provider model listing failed for %s: %s",
|
||||
models_url,
|
||||
exc,
|
||||
)
|
||||
continue
|
||||
if response.status_code != 200:
|
||||
continue
|
||||
try:
|
||||
payload = response.json()
|
||||
except Exception:
|
||||
continue
|
||||
rows = payload.get("data") if isinstance(payload, dict) else None
|
||||
if not isinstance(rows, list):
|
||||
continue
|
||||
ids = [
|
||||
str(row["id"])
|
||||
for row in rows
|
||||
if isinstance(row, dict) and row.get("id")
|
||||
]
|
||||
if ids:
|
||||
return ids
|
||||
return None
|
||||
|
||||
|
||||
MODEL_TIER2_COLUMNS = (
|
||||
@@ -144,11 +194,16 @@ def ensure_tables() -> None:
|
||||
+ PROVIDERS_TABLE
|
||||
+ " (id INTEGER PRIMARY KEY, name TEXT, base_url TEXT, api_key TEXT, "
|
||||
"is_active INTEGER DEFAULT 1, client_profile TEXT DEFAULT '', "
|
||||
"protocol TEXT DEFAULT 'chat_completions', "
|
||||
"created_at TEXT, updated_at TEXT)"
|
||||
)
|
||||
providers_table = get_table(PROVIDERS_TABLE)
|
||||
if not providers_table.has_column("client_profile"):
|
||||
providers_table.create_column_by_example("client_profile", "")
|
||||
if not providers_table.has_column("protocol"):
|
||||
providers_table.create_column_by_example(
|
||||
"protocol", PROTOCOL_DEFAULT
|
||||
)
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS "
|
||||
+ MODELS_TABLE
|
||||
@@ -196,6 +251,7 @@ class ProviderIn(BaseModel):
|
||||
api_key: str = Field(default="", max_length=400)
|
||||
is_active: bool = True
|
||||
client_profile: str = Field(default="", max_length=32)
|
||||
protocol: str = Field(default=PROTOCOL_DEFAULT, max_length=32)
|
||||
|
||||
@field_validator("name")
|
||||
@classmethod
|
||||
@@ -225,6 +281,16 @@ class ProviderIn(BaseModel):
|
||||
)
|
||||
return value
|
||||
|
||||
@field_validator("protocol")
|
||||
@classmethod
|
||||
def _clean_protocol(cls, value: str) -> str:
|
||||
value = (value or PROTOCOL_DEFAULT).strip().lower()
|
||||
if value not in PROVIDER_PROTOCOLS:
|
||||
raise ValueError(
|
||||
f"protocol must be one of: {', '.join(PROVIDER_PROTOCOLS)}"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
class ModelRouteIn(BaseModel):
|
||||
source_model: str = Field(min_length=1, max_length=128)
|
||||
@@ -366,6 +432,9 @@ def _load() -> dict:
|
||||
"api_key": str(row.get("api_key") or ""),
|
||||
"is_active": _as_bool(row.get("is_active", 1)),
|
||||
"client_profile": str(row.get("client_profile") or ""),
|
||||
"protocol": str(
|
||||
row.get("protocol") or PROTOCOL_DEFAULT
|
||||
),
|
||||
}
|
||||
if MODELS_TABLE in db.tables:
|
||||
for row in get_table(MODELS_TABLE).all():
|
||||
@@ -404,6 +473,7 @@ class ProviderStore:
|
||||
"api_key": payload.api_key,
|
||||
"is_active": 1 if payload.is_active else 0,
|
||||
"client_profile": payload.client_profile,
|
||||
"protocol": payload.protocol,
|
||||
"updated_at": _now(),
|
||||
}
|
||||
if existing:
|
||||
@@ -632,6 +702,11 @@ def chat_overlay(requested_model: Optional[str], base_cfg: dict) -> Optional[dic
|
||||
header_hook = _header_hook_for(route.provider)
|
||||
if header_hook is not None:
|
||||
overlay["gateway_extra_request_headers"] = header_hook()
|
||||
provider = provider_store.get(route.provider)
|
||||
if provider is not None:
|
||||
overlay["gateway_upstream_protocol"] = (
|
||||
provider.get("protocol") or PROTOCOL_DEFAULT
|
||||
)
|
||||
if route.context_window:
|
||||
from devplacepy.services.openai_gateway.usage import parse_context_map
|
||||
|
||||
|
||||
@@ -107,6 +107,21 @@ class GatewayService(BaseService):
|
||||
help="The key currently in use; auto-migrated from DEEPSEEK_API_KEY or OPENROUTER_API_KEY on boot. Editable.",
|
||||
group="Upstream",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_upstream_protocol",
|
||||
"Upstream protocol",
|
||||
type="select",
|
||||
default="chat_completions",
|
||||
options=[
|
||||
{"value": "chat_completions", "label": "Chat Completions"},
|
||||
{"value": "responses", "label": "Responses API"},
|
||||
],
|
||||
help="Wire protocol the default upstream speaks. 'Responses API' "
|
||||
"transpiles inbound chat requests to POST /responses and converts "
|
||||
"the answer back to the normal OpenAI surface. A named provider "
|
||||
"route overrides this per provider.",
|
||||
group="Upstream",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_timeout",
|
||||
"Upstream timeout (seconds)",
|
||||
|
||||
Reference in New Issue
Block a user