feat(clients): sessions+registry+optimize+armor: close gaps, loud list truncation, shared Gemini adapter
sessions: operations get/list/cancel/delete/wait for engine, session, memory and serving profile; servingProfiles CRUD. registry: ListResult with truncated flag, v1alpha twins, gateway/policy operations, authz IAM, deny policy update, boundary patch/bindings. optimize: make_gemini_model over models::VertexModel (DirectModel removed), generateSyntheticData, online evaluator operations, trace/log writers. armor: SecurityAdmin for all SCC v2 methods (attack exposure excluded), documented agent threat categories, floor settings fixed to location global after live run. check_discovery.py of all four enforce coverage. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
+89
-14
@@ -19,6 +19,19 @@ module.
|
||||
Finding resource carries typed `agent`, `agentAnomaly`, `agentSessions`,
|
||||
`agentDataAccessEvents`, `aiModel` and `vulnerability` members, which are
|
||||
read into `Finding`.
|
||||
- Security Command Center administration, `SecurityAdmin` (every v2 method
|
||||
except the attack exposure engine): mute configs, notification configs and
|
||||
BigQuery exports (list, get, create, patch, delete), finding state, mute,
|
||||
patch, security marks, create, external systems, bulk mute, findings export,
|
||||
sources create/patch and source IAM, asset security marks, organization
|
||||
operations (get, list, cancel, delete). Attack paths, valued resources,
|
||||
simulations and resource value configs are excluded on purpose: they are
|
||||
cloud posture risk scoring, not agent security (`EXCLUDED_RE` in
|
||||
`tools/check_discovery.py`).
|
||||
- Agent Threat Detection categories: `AgentCategories` holds the documented
|
||||
category strings (runtime detectors and control plane detectors) with module
|
||||
names. Agent Anomaly Detection has no documented category string: its
|
||||
findings are the ones with the typed `agentAnomaly` member.
|
||||
- Agent Security dashboard: a console product with no dashboard API.
|
||||
`AgentSecurityDashboard` is a LOCAL abstraction that aggregates listed
|
||||
findings (counts by severity, category, class, state, per-asset worst
|
||||
@@ -40,7 +53,10 @@ env: `GOOGLE_CLOUD_PROJECT`, optional `GOOGLE_CLOUD_LOCATION`.
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings (`ArmorCaps`) |
|
||||
| `calls.h/.cpp` | The ONE table of declared calls, schemas, enums; path matcher |
|
||||
| `calls.h/.cpp` | The ONE table of declared calls (144 rows: rpc, verb, path, query, request schema, allowed body fields), schemas, enums; path matcher |
|
||||
| `scc_name.h/.cpp` | `SccName`: a validated SCC resource name; derives the table row (scope, `locations`, `sources`) from the name |
|
||||
| `security_admin.h/.cpp` | `SecurityAdmin : Service`: every SCC v2 administration method, `SccList`, `SccAvailability` probe |
|
||||
| `agent_categories.h/.cpp` | `AgentCategories`: documented Agent Platform Threat Detection categories and module names |
|
||||
| `tools/check_discovery.py` | Fails when `calls.h` and Google's Discovery documents disagree |
|
||||
| `armor_types.h/.cpp` | `SanitizationResult`, `FilterOutcome`, enums |
|
||||
| `template.h/.cpp` | `Template`, `FilterConfig`, `TemplateMetadata`, `FloorSetting`, rule sets |
|
||||
@@ -62,8 +78,8 @@ env: `GOOGLE_CLOUD_PROJECT`, optional `GOOGLE_CLOUD_LOCATION`.
|
||||
| Labels per template | 64 (63 chars each) |
|
||||
| Update mask paths | 16 |
|
||||
| Detail strings per filter result | 50 |
|
||||
| Findings page size (Google max) / pages / results | 1000 / 10 / 5000 |
|
||||
| Categories in a filter | 20 |
|
||||
| Findings page size (Google max) / pages / results | 1000 / 10 / 5000; config and operation lists use 100 / 10 / 5000 and report `truncated` |
|
||||
| Categories in a filter | 100 (all documented agent categories fit) |
|
||||
| Dashboard distinct assets | 500 |
|
||||
|
||||
## 4. Contracts
|
||||
@@ -77,8 +93,19 @@ env: `GOOGLE_CLOUD_PROJECT`, optional `GOOGLE_CLOUD_LOCATION`.
|
||||
`global` (`https://modelarmor.googleapis.com/v1`); templates and sanitize
|
||||
reject `global`. Security Command Center: `https://securitycenter.googleapis.com/v2`.
|
||||
- Destructive: `delete_template` is an explicit method; the demo route also
|
||||
requires `confirm=delete`. Nothing here creates, updates or deletes
|
||||
findings, sources or any Security Command Center resource.
|
||||
requires `confirm=delete`. `SecurityAdmin` mutations are explicit methods
|
||||
(`delete_config`, `bulk_mute`, `set_finding_state`, ...), each checked
|
||||
against its table row (query parameters, top-level body fields, update mask
|
||||
roots) before traffic. `bulk_mute` refuses an empty filter and only mutes or
|
||||
un-mutes (`MUTED`, `UNDEFINED`); its demo route needs `confirm=bulk-mute`.
|
||||
`cancel_operation` / `delete_operation` read the operation first. Floor
|
||||
settings are read and written at location `global` on the global host only
|
||||
(found live, see 6).
|
||||
- Lists never truncate silently: findings (`FindingPage::truncated`), config
|
||||
and operation lists (`SccList::truncated` with `next_page_token`).
|
||||
- Not activated is a state, not a crash: `SecurityAdmin::availability` maps
|
||||
403 API disabled, 404 on `sources` and 400 `FAILED_PRECONDITION` (SCC enabled
|
||||
but not activated for the project) to `SccAvailability`.
|
||||
- `retry ids`: `requestId` is accepted on create, patch and delete as a
|
||||
non-zero UUID.
|
||||
- Sanitization results are typed; an unknown or missing `invocationResult`
|
||||
@@ -97,13 +124,13 @@ env: `GOOGLE_CLOUD_PROJECT`, optional `GOOGLE_CLOUD_LOCATION`.
|
||||
|
||||
## 5. Verified against discovery
|
||||
|
||||
`python3 -I src/google/armor/tools/check_discovery.py` fetches
|
||||
`python3 -I src/google/armor/tools/check_discovery.py` (coverage: every method of both Discovery documents is a table row except the excluded attack exposure engine) fetches
|
||||
`https://modelarmor.googleapis.com/$discovery/rest?version=v1` and
|
||||
`https://securitycenter.googleapis.com/$discovery/rest?version=v2` and checks
|
||||
every row of `calls.h`: 31 calls (rpc id, HTTP method, path, query
|
||||
parameters), 66 schemas (every field this module reads or writes) and 13 enum
|
||||
value sets. Result on 2026-10-24: `OK: 31 calls, 66 schemas, 13 enums
|
||||
checked, 0 problems`. Unit tests additionally prove that every request the
|
||||
every row of `calls.h`: 144 calls (rpc id, HTTP method, path, query
|
||||
parameters, request schema, request fields), 71 schemas (every field this
|
||||
module reads or writes) and 17 enum value sets. Result on 2026-10-24: `OK:
|
||||
144 calls, 71 schemas, 17 enums checked, 0 problems`. Unit tests additionally prove that every request the
|
||||
client puts on the wire matches a declared call and only uses declared query
|
||||
parameters, and that every key of every request body built here is a field of
|
||||
its Discovery schema (nested schemas included).
|
||||
@@ -122,6 +149,34 @@ and messages, `filterVersionSelector`, `modalities`, `dataResidencyCompliant`;
|
||||
Date 2026-10-24, project from `GOOGLE_CLOUD_PROJECT`, no resource created,
|
||||
updated or deleted.
|
||||
|
||||
**Re-run after the owner enabled Model Armor and Security Command Center
|
||||
(service-account token, read-only):**
|
||||
|
||||
| Call | Result |
|
||||
|---|---|
|
||||
| Model Armor `GET https://modelarmor.us-central1.rep.googleapis.com/v1/projects/P/locations/us-central1/templates` | 200 `{}` (no template exists) |
|
||||
| `GET https://modelarmor.googleapis.com/v1/projects/P/locations/global/floorSetting` | 200 `{name, createTime, updateTime}` (no filterConfig: enforcement off) |
|
||||
| `GET .../locations/us-central1/floorSetting` on the regional host | 400 INVALID_ARGUMENT "Malformed name" (floor settings exist only at `global`) |
|
||||
| `GET https://modelarmor.googleapis.com/v1/projects/P/locations?pageSize=5` | 200, a list of locations (`name`, `locationId`) |
|
||||
| `GET https://modelarmor.us-central1.rep.googleapis.com/v1/projects/P/locations` | 403 "Read access to project denied" (list is global-host only) |
|
||||
| `GET .../locations/us-central1` on the regional host / `.../locations/global` on the global host | 200 `{name, locationId}` |
|
||||
| `GET .../locations/us-central1` on the global host | 403 |
|
||||
| `sanitizeUserPrompt` against an absent template id | 400 INVALID_ARGUMENT "The template named '...' was not found" (request shape accepted; absent template is 400, not 404) |
|
||||
| SCC v2 `GET projects/P/sources`, `sources/-/findings`, `sources/-/locations/global/findings` (also v1 sources) | 404 NOT_FOUND "Requested entity was not found." |
|
||||
| SCC v2 `GET projects/P/muteConfigs`, `.../locations/global/muteConfigs`, `.../locations/global/notificationConfigs`, `.../locations/global/bigQueryExports` | 400 FAILED_PRECONDITION "Precondition check failed." |
|
||||
| Cloud Resource Manager `GET v3/projects/P` | 403 API not enabled (so the project's organization could not be read) |
|
||||
|
||||
Reading: the API is enabled but Security Command Center is not activated for
|
||||
this project (no organization or SCC tier reachable): sources do not resolve
|
||||
and every config call fails its precondition. Findings, groups, mute configs,
|
||||
notification configs, exports and every `SecurityAdmin` happy path are
|
||||
therefore NOT seen live; they stay Discovery-verified. `SecurityAdmin::availability`
|
||||
reports `not_activated` for exactly this state, and the opt-in test
|
||||
`google_live_armor.oauth_token_reads_templates_and_findings` prints it.
|
||||
Model Armor code fixed from these runs: floor settings now use location
|
||||
`global` on the global host, `list_locations` uses the global host,
|
||||
`get_location` the location's own host.
|
||||
|
||||
| Call | Credential | Result |
|
||||
|---|---|---|
|
||||
| Model Armor `templates.list` | express key (`AQ.`) | 401 UNAUTHENTICATED (API keys are not supported); `google_live_armor.express_key_is_refused` |
|
||||
@@ -155,8 +210,25 @@ disabled API loudly. Run it once the APIs are enabled.
|
||||
|
||||
Nothing neither Discovery nor docs state:
|
||||
|
||||
- Finding `category` strings for Agent Threat Detection and Agent Anomaly
|
||||
Detection: not read; callers pass categories they see in the console.
|
||||
- Agent Anomaly Detection category strings: neither the anomaly overview page
|
||||
nor the threat pages name one. Settled by documentation reading
|
||||
(2026-10-24): Agent Platform Threat Detection (runtime, Preview) and the AI
|
||||
agent control plane detectors publish the category strings in
|
||||
`agent_categories.cpp` (module names `AGENT_ENGINE_*`); anomaly findings are
|
||||
recognised by the typed `agentAnomaly` member, not by category. The pages
|
||||
disagree on a few spellings (CloudSQL / Cloud SQL, "Evidence of Port
|
||||
Scanning from AI Agent" / "AI Agent Evidence of Port Scanning", "Remove Bulk
|
||||
Data From Disk" / "from Disk", "cryptocurrency" / "crypto", "Abuse of Sudo" /
|
||||
"Attempt to Abuse Sudo"): both are listed and filtered. No finding was ever
|
||||
seen live, so which spelling Google emits is unverified.
|
||||
- `SecurityAdmin` response bodies (configs, operations, marks) are returned as
|
||||
raw Json and were never seen; request fields are checked against Discovery
|
||||
only. Notification configs and BigQuery exports exist only under
|
||||
`locations` in v2 (Discovery), mute configs with and without.
|
||||
- Whether `findings:bulkMute` accepts `UNMUTED` (Discovery's enum for
|
||||
`BulkMuteFindingsRequest.muteState` is `MUTED`, `UNDEFINED`): only those are sent.
|
||||
- Sanitize against a floor setting without a template: not possible (a
|
||||
template name is required); no default template exists.
|
||||
- That agent assets always contain `reasoningEngines` in `resourceName`
|
||||
(dashboard default filter; override with `resource_contains`).
|
||||
- Server-side behavior of the `resourceName:"x"` substring operator (AIP-160
|
||||
@@ -178,6 +250,9 @@ Nothing neither Discovery nor docs state:
|
||||
- Streaming sanitize (`streamingMode`, gRPC `StreamSanitize*`) is not
|
||||
implemented; `GuardedModel` buffers instead.
|
||||
- No asset inventory (Cloud Asset Inventory is a different API).
|
||||
- Findings mutation (mute, state), notification configs, BigQuery exports,
|
||||
mute configs and attack paths are out of scope (read-only module).
|
||||
- Attack exposure engine (attack paths, valued resources, simulations,
|
||||
resource value configs) is excluded on purpose (posture risk scoring).
|
||||
- Security Command Center is not activated for the test project, so no
|
||||
findings, group, config or admin response was ever seen. Owner: whoever
|
||||
activates SCC at the organization; tests `google_live_armor.*`.
|
||||
- Token refresh is the caller's job (foundation gap).
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
#include "google/armor/agent_categories.h"
|
||||
|
||||
namespace molo::google::armor {
|
||||
|
||||
namespace {
|
||||
constexpr AgentFamily R = AgentFamily::kRuntime;
|
||||
constexpr AgentFamily C = AgentFamily::kControlPlane;
|
||||
|
||||
// clang-format off
|
||||
const std::vector<AgentCategory> kTable = {
|
||||
// Control plane (Event Threat Detection), enabled by default.
|
||||
{"Discovery: Evidence of Port Scanning from AI Agent", "Discovery: AI Agent Evidence of Port Scanning", "AGENT_ENGINE_PORT_SCANNING_EVIDENCE", C},
|
||||
{"Discovery: AI Agent Unauthorized Service Account API Call", "", "AGENT_ENGINE_UNAUTHORIZED_SERVICE_ACCOUNT_API_CALL", C},
|
||||
{"Discovery: AI Agent Unauthorized Service Account API Call Affecting Sensitive Data", "", "", C},
|
||||
{"Discovery: AI Agent Service Account Self-Investigation", "", "AGENT_ENGINE_IAM_ANOMALOUS_BEHAVIOR_SERVICE_ACCOUNT_GETS_OWN_IAM_POLICY", C},
|
||||
{"Credential Access: AI Agent Anomalous Access to Metadata Service", "", "AGENT_ENGINE_ANOMALOUS_ACCESS_TO_METADATA_SERVICE", C},
|
||||
{"Exfiltration: AI Agent Initiated BigQuery VPC Perimeter Violation", "", "AGENT_ENGINE_BIG_QUERY_EXFIL_VPC_PERIMETER_VIOLATION", C},
|
||||
{"Exfiltration: AI Agent Initiated BigQuery Data Exfiltration to External Table", "", "AGENT_ENGINE_BIG_QUERY_EXFIL_TO_EXTERNAL_TABLE", C},
|
||||
{"Exfiltration: AI Agent Initiated Sensitive BigQuery Data Exfiltration to External Table", "", "", C},
|
||||
{"Exfiltration: AI Agent Initiated CloudSQL Exfiltration to Public Bucket", "Exfiltration: AI Agent Initiated Cloud SQL Exfiltration to Public Bucket", "AGENT_ENGINE_CLOUDSQL_EXFIL_EXPORT_TO_PUBLIC_GCS", C},
|
||||
{"Exfiltration: AI Agent Initiated CloudSQL Exfiltration to External Bucket", "Exfiltration: AI Agent Initiated Cloud SQL Exfiltration to External Bucket", "AGENT_ENGINE_CLOUDSQL_EXFIL_EXPORT_TO_EXTERNAL_GCS", C},
|
||||
{"Exfiltration: AI Agent Initiated BigQuery Data Extraction", "", "AGENT_ENGINE_BIG_QUERY_EXFIL_TO_CLOUD_STORAGE", C},
|
||||
{"Exfiltration: AI Agent Initiated Sensitive BigQuery Data Extraction", "", "", C},
|
||||
{"Initial Access: AI Agent Identity Excessive Permission Denied Actions", "", "AGENT_ENGINE_EXCESSIVE_FAILED_ATTEMPT", C},
|
||||
{"Privilege Escalation: AI Agent Token Generation Using signJwt", "", "AGENT_ENGINE_SUSPICIOUS_TOKEN_GENERATION_SIGN_JWT", C},
|
||||
{"Privilege Escalation: AI Agent Token Generation Using Implicit Delegation", "", "AGENT_ENGINE_SUSPICIOUS_TOKEN_GENERATION_IMPLICIT_DELEGATION", C},
|
||||
{"Privilege Escalation: AI Agent Cross-Project OpenID Token Generation", "", "AGENT_ENGINE_SUSPICIOUS_TOKEN_GENERATION_CROSS_PROJECT_OPENID", C},
|
||||
{"Privilege Escalation: AI Agent Cross-Project Access Token Generation", "", "AGENT_ENGINE_SUSPICIOUS_TOKEN_GENERATION_CROSS_PROJECT_ACCESS_TOKEN", C},
|
||||
// Runtime (Agent Platform Threat Detection, Preview).
|
||||
{"Command and Control: Steganography Tool Detected", "", "AGENT_ENGINE_STEGANOGRAPHY_TOOL_DETECTED", R},
|
||||
{"Credential Access: Find Google Cloud Credentials", "", "AGENT_ENGINE_FIND_GCP_CREDENTIALS", R},
|
||||
{"Credential Access: GPG Key Reconnaissance", "", "AGENT_ENGINE_GPG_KEY_RECONNAISSANCE", R},
|
||||
{"Credential Access: Search Private Keys or Passwords", "", "AGENT_ENGINE_SEARCH_PRIVATE_KEYS_OR_PASSWORDS", R},
|
||||
{"Defense Evasion: Base64 ELF File Command Line", "", "AGENT_ENGINE_BASE64_ELF_FILE_CMDLINE", R},
|
||||
{"Defense Evasion: Base64 Encoded Python Script Executed", "", "AGENT_ENGINE_BASE64_ENCODED_PYTHON_SCRIPT_EXECUTED", R},
|
||||
{"Defense Evasion: Base64 Encoded Shell Script Executed", "", "AGENT_ENGINE_BASE64_ENCODED_SHELL_SCRIPT_EXECUTED", R},
|
||||
{"Defense Evasion: Launch Code Compiler Tool In Container", "", "AGENT_ENGINE_LAUNCH_CODE_COMPILER_TOOL_IN_CONTAINER", R},
|
||||
{"Execution: Added Malicious Binary Executed", "", "AGENT_ENGINE_ADDED_MALICIOUS_BINARY_EXECUTED", R},
|
||||
{"Execution: Added Malicious Library Loaded", "", "AGENT_ENGINE_ADDED_MALICIOUS_LIBRARY_LOADED", R},
|
||||
{"Execution: Added Malicious Skill Executed", "", "AGENT_ENGINE_ADDED_MALICIOUS_SKILL_EXECUTED", R},
|
||||
{"Execution: Added Malicious Skill Loaded", "", "AGENT_ENGINE_ADDED_MALICIOUS_SKILL_LOADED", R},
|
||||
{"Execution: Built in Malicious Binary Executed", "", "AGENT_ENGINE_BUILT_IN_MALICIOUS_BINARY_EXECUTED", R},
|
||||
{"Execution: Container Escape", "", "AGENT_ENGINE_CONTAINER_ESCAPE", R},
|
||||
{"Execution: Fileless Execution in /memfd:", "", "AGENT_ENGINE_FILELESS_EXECUTION_DETECTION_MEMFD", R},
|
||||
{"Execution: Kubernetes Attack Tool Execution", "", "AGENT_ENGINE_KUBERNETES_ATTACK_TOOL_EXECUTION", R},
|
||||
{"Execution: Local Reconnaissance Tool Execution", "", "AGENT_ENGINE_LOCAL_RECONNAISSANCE_TOOL_EXECUTION", R},
|
||||
{"Execution: Malicious Python Executed", "", "AGENT_ENGINE_MALICIOUS_PYTHON_EXECUTED", R},
|
||||
{"Execution: Modified Malicious Binary Executed", "", "AGENT_ENGINE_MODIFIED_MALICIOUS_BINARY_EXECUTED", R},
|
||||
{"Execution: Modified Malicious Library Loaded", "", "AGENT_ENGINE_MODIFIED_MALICIOUS_LIBRARY_LOADED", R},
|
||||
{"Execution: Modified Malicious Skill Executed", "", "AGENT_ENGINE_MODIFIED_MALICIOUS_SKILL_EXECUTED", R},
|
||||
{"Execution: Modified Malicious Skill Loaded", "", "AGENT_ENGINE_MODIFIED_MALICIOUS_SKILL_LOADED", R},
|
||||
{"Malicious Script Executed", "", "AGENT_ENGINE_MALICIOUS_SCRIPT_EXECUTED", R},
|
||||
{"Malicious URL Observed", "", "AGENT_ENGINE_MALICIOUS_URL_OBSERVED", R},
|
||||
{"Reverse Shell", "", "AGENT_ENGINE_REVERSE_SHELL", R},
|
||||
{"Unexpected Child Shell", "", "AGENT_ENGINE_UNEXPECTED_CHILD_SHELL", R},
|
||||
{"Execution: Netcat Remote Code Execution in Container", "", "AGENT_ENGINE_NETCAT_REMOTE_CODE_EXECUTION_IN_CONTAINER", R},
|
||||
{"Execution: Possible Arbitrary Command Execution through CUPS (CVE-2024-47177)", "", "AGENT_ENGINE_POSSIBLE_ARBITRARY_COMMAND_EXECUTION_THROUGH_CUPS", R},
|
||||
{"Execution: Possible Remote Command Execution Detected", "", "AGENT_ENGINE_POSSIBLE_REMOTE_COMMAND_EXECUTION_DETECTED", R},
|
||||
{"Execution: Program Run with Disallowed HTTP Proxy Env", "", "AGENT_ENGINE_PROGRAM_RUN_WITH_DISALLOWED_HTTP_PROXY_ENV", R},
|
||||
{"Execution: Socat Reverse Shell Detected", "", "AGENT_ENGINE_SOCAT_REVERSE_SHELL_DETECTED", R},
|
||||
{"Execution: Suspicious OpenSSL Shared Object Loaded", "", "AGENT_ENGINE_SUSPICIOUS_OPENSSL_SHARED_OBJECT_LOADED", R},
|
||||
{"Exfiltration: Launch Remote File Copy Tools in Container", "", "AGENT_ENGINE_LAUNCH_REMOTE_FILE_COPY_TOOLS_IN_CONTAINER", R},
|
||||
{"Impact: Detect Malicious Cmdlines", "", "AGENT_ENGINE_DETECT_MALICIOUS_CMDLINES", R},
|
||||
{"Impact: Remove Bulk Data From Disk", "Impact: Remove Bulk Data from Disk", "AGENT_ENGINE_REMOVE_BULK_DATA_FROM_DISK", R},
|
||||
{"Impact: Suspicious cryptocurrency mining activity using the Stratum Protocol", "Impact: Suspicious crypto mining activity using the Stratum Protocol", "AGENT_ENGINE_DETECT_CRYPTO_MINERS_USING_STRATUM_PROTOCOL", R},
|
||||
{"Privilege Escalation: Abuse of Sudo For Privilege Escalation (CVE-2019-14287)", "Privilege Escalation: Attempt to Abuse Sudo For Privilege Escalation (CVE-2019-14287)", "AGENT_ENGINE_ABUSE_SUDO_FOR_PRIVILEGE_ESCALATION", R},
|
||||
{"Privilege Escalation: Polkit Local Privilege Escalation Vulnerability (CVE-2021-4034)", "", "AGENT_ENGINE_POLKIT_LOCAL_PRIVILEGE_ESCALATION_VULNERABILITY", R},
|
||||
{"Privilege Escalation: Sudo Potential Privilege Escalation (CVE-2021-3156)", "", "AGENT_ENGINE_SUDO_POTENTIAL_PRIVILEGE_ESCALATION", R},
|
||||
};
|
||||
// clang-format on
|
||||
} // namespace
|
||||
|
||||
const std::vector<AgentCategory>& AgentCategories::all() { return kTable; }
|
||||
|
||||
std::vector<std::string> AgentCategories::strings(AgentFamily family) {
|
||||
std::vector<std::string> out;
|
||||
for (const AgentCategory& c : kTable) {
|
||||
if (c.family != family) continue;
|
||||
out.push_back(c.category);
|
||||
if (c.alt[0]) out.push_back(c.alt);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<std::string> AgentCategories::strings() {
|
||||
std::vector<std::string> out = strings(AgentFamily::kControlPlane);
|
||||
std::vector<std::string> rt = strings(AgentFamily::kRuntime);
|
||||
out.insert(out.end(), rt.begin(), rt.end());
|
||||
return out;
|
||||
}
|
||||
|
||||
const AgentCategory* AgentCategories::find(const std::string& category) {
|
||||
for (const AgentCategory& c : kTable)
|
||||
if (category == c.category || (c.alt[0] && category == c.alt)) return &c;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
const AgentCategory* AgentCategories::find_module(const std::string& module) {
|
||||
if (module.empty()) return nullptr;
|
||||
for (const AgentCategory& c : kTable)
|
||||
if (module == c.module) return &c;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
} // namespace molo::google::armor
|
||||
@@ -0,0 +1,39 @@
|
||||
#pragma once
|
||||
// Security Command Center finding categories of Agent Platform Threat
|
||||
// Detection (runtime detectors, module names AGENT_ENGINE_*) and of the Event
|
||||
// Threat Detection control plane detectors for AI agents, read from the
|
||||
// official pages (see SPEC section Sources):
|
||||
// https://docs.cloud.google.com/security-command-center/docs/agent-engine-threat-detection-overview
|
||||
// https://docs.cloud.google.com/security-command-center/docs/ai-threats
|
||||
// The two pages spell a few categories differently (for example "CloudSQL"
|
||||
// and "Cloud SQL"); both spellings are listed and both are filtered on.
|
||||
// Agent Anomaly Detection has no documented category string: its findings are
|
||||
// recognised by the typed `agentAnomaly` member of the Finding.
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace molo::google::armor {
|
||||
|
||||
enum class AgentFamily { kRuntime, kControlPlane };
|
||||
|
||||
struct AgentCategory {
|
||||
const char* category; // As on the agent-engine-threat-detection-overview page
|
||||
const char* alt; // Spelling on the ai-threats page when it differs, else ""
|
||||
const char* module; // Event Threat Detection / runtime module name
|
||||
AgentFamily family;
|
||||
};
|
||||
|
||||
class AgentCategories {
|
||||
public:
|
||||
static const std::vector<AgentCategory>& all();
|
||||
// Every category string of a family, both spellings, for FindingQuery.categories.
|
||||
static std::vector<std::string> strings(AgentFamily family);
|
||||
static std::vector<std::string> strings(); // both families
|
||||
// The entry whose category or alt equals `category`, or nullptr.
|
||||
static const AgentCategory* find(const std::string& category);
|
||||
static bool is_agent_threat(const std::string& category) { return find(category) != nullptr; }
|
||||
// Entry by module name (the Finding.moduleName value), or nullptr.
|
||||
static const AgentCategory* find_module(const std::string& module);
|
||||
};
|
||||
|
||||
} // namespace molo::google::armor
|
||||
+155
-31
@@ -18,6 +18,8 @@ struct CallSpec {
|
||||
const char* http; // GET | POST | PATCH | DELETE
|
||||
const char* path; // Discovery `path` (relative to the service root)
|
||||
const char* query; // Comma separated query parameters this module sends
|
||||
const char* request; // Discovery request schema name, "" when no body
|
||||
const char* fields; // Comma separated top-level request fields allowed
|
||||
};
|
||||
|
||||
struct SchemaSpec {
|
||||
@@ -37,37 +39,150 @@ struct EnumSpec {
|
||||
|
||||
// clang-format off
|
||||
inline constexpr CallSpec kCalls[] = {
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.create", "POST", "v1/{+parent}/templates", "templateId,requestId"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.get", "GET", "v1/{+name}", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.list", "GET", "v1/{+parent}/templates", "pageSize,pageToken,filter,orderBy"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.patch", "PATCH", "v1/{+name}", "updateMask,requestId"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.delete", "DELETE", "v1/{+name}", "requestId"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.sanitizeUserPrompt", "POST", "v1/{+name}:sanitizeUserPrompt", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.sanitizeModelResponse", "POST", "v1/{+name}:sanitizeModelResponse", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.getFloorSetting", "GET", "v1/{+name}", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask"},
|
||||
{"modelarmor", "v1", "modelarmor.folders.locations.getFloorSetting", "GET", "v1/{+name}", ""},
|
||||
{"modelarmor", "v1", "modelarmor.folders.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask"},
|
||||
{"modelarmor", "v1", "modelarmor.organizations.locations.getFloorSetting", "GET", "v1/{+name}", ""},
|
||||
{"modelarmor", "v1", "modelarmor.organizations.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.list", "GET", "v1/{+name}/locations", "pageToken"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.get", "GET", "v1/{+name}", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.group", "POST", "v2/{+parent}/findings:group", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.group", "POST", "v2/{+parent}/findings:group", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.group", "POST", "v2/{+parent}/findings:group", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.get", "GET", "v2/{+name}", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.create", "POST", "v1/{+parent}/templates", "templateId,requestId", "Template", "templateMetadata,name,labels,filterConfig"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.get", "GET", "v1/{+name}", "", "", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.list", "GET", "v1/{+parent}/templates", "pageSize,pageToken,filter,orderBy", "", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.patch", "PATCH", "v1/{+name}", "updateMask,requestId", "Template", "templateMetadata,name,labels,filterConfig"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.delete", "DELETE", "v1/{+name}", "requestId", "", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.sanitizeUserPrompt", "POST", "v1/{+name}:sanitizeUserPrompt", "", "SanitizeUserPromptRequest", "userPromptData,streamingMode,multiLanguageDetectionMetadata"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.templates.sanitizeModelResponse", "POST", "v1/{+name}:sanitizeModelResponse", "", "SanitizeModelResponseRequest", "streamingMode,multiLanguageDetectionMetadata,modelResponseData,userPrompt"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.getFloorSetting", "GET", "v1/{+name}", "", "", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask", "FloorSetting", "aiPlatformFloorSetting,name,integratedServices,floorSettingMetadata,enableFloorSettingEnforcement,googleMcpServerFloorSetting,filterConfig"},
|
||||
{"modelarmor", "v1", "modelarmor.folders.locations.getFloorSetting", "GET", "v1/{+name}", "", "", ""},
|
||||
{"modelarmor", "v1", "modelarmor.folders.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask", "FloorSetting", "aiPlatformFloorSetting,name,integratedServices,floorSettingMetadata,enableFloorSettingEnforcement,googleMcpServerFloorSetting,filterConfig"},
|
||||
{"modelarmor", "v1", "modelarmor.organizations.locations.getFloorSetting", "GET", "v1/{+name}", "", "", ""},
|
||||
{"modelarmor", "v1", "modelarmor.organizations.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask", "FloorSetting", "aiPlatformFloorSetting,name,integratedServices,floorSettingMetadata,enableFloorSettingEnforcement,googleMcpServerFloorSetting,filterConfig"},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.list", "GET", "v1/{+name}/locations", "pageToken", "", ""},
|
||||
{"modelarmor", "v1", "modelarmor.projects.locations.get", "GET", "v1/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.group", "POST", "v2/{+parent}/findings:group", "", "GroupFindingsRequest", "pageToken,groupBy,pageSize,filter"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.group", "POST", "v2/{+parent}/findings:group", "", "GroupFindingsRequest", "pageToken,groupBy,pageSize,filter"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.group", "POST", "v2/{+parent}/findings:group", "", "GroupFindingsRequest", "pageToken,groupBy,pageSize,filter"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", "", "GroupFindingsRequest", "pageToken,groupBy,pageSize,filter"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", "", "GroupFindingsRequest", "pageToken,groupBy,pageSize,filter"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", "", "GroupFindingsRequest", "pageToken,groupBy,pageSize,filter"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.assets.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.findings.bulkMute", "POST", "v2/{+parent}/findings:bulkMute", "", "BulkMuteFindingsRequest", "filter,muteState"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.bigQueryExports.create", "POST", "v2/{+parent}/bigQueryExports", "bigQueryExportId", "GoogleCloudSecuritycenterV2BigQueryExport", "name,filter,dataset,deletionNotificationsEnabled,description"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.bigQueryExports.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.bigQueryExports.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.bigQueryExports.list", "GET", "v2/{+parent}/bigQueryExports", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.bigQueryExports.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2BigQueryExport", "name,filter,dataset,deletionNotificationsEnabled,description"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.findings.bulkMute", "POST", "v2/{+parent}/findings:bulkMute", "", "BulkMuteFindingsRequest", "filter,muteState"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.muteConfigs.create", "POST", "v2/{+parent}/muteConfigs", "muteConfigId", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.muteConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.muteConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.muteConfigs.list", "GET", "v2/{+parent}/muteConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.muteConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.notificationConfigs.create", "POST", "v2/{+parent}/notificationConfigs", "configId", "NotificationConfig", "streamingConfig,pubsubTopic,description,name,deletionNotificationsEnabled"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.notificationConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.notificationConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.notificationConfigs.list", "GET", "v2/{+parent}/notificationConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.locations.notificationConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "NotificationConfig", "streamingConfig,pubsubTopic,description,name,deletionNotificationsEnabled"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.muteConfigs.create", "POST", "v2/{+parent}/muteConfigs", "muteConfigId", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.muteConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.muteConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.muteConfigs.list", "GET", "v2/{+parent}/muteConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.folders.muteConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.externalSystems.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2ExternalSystem", "caseUri,name,assignees,externalUid,casePriority,status,caseSla,caseCloseTime,caseCreateTime,externalSystemUpdateTime,ticketInfo"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.setMute", "POST", "v2/{+name}:setMute", "", "SetMuteRequest", "mute"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.setState", "POST", "v2/{+name}:setState", "", "SetFindingStateRequest", "state"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.findings.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.export", "POST", "v2/{+parent}/findings:export", "", "ExportFindingsRequest", "bigQueryDestination"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.externalSystems.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2ExternalSystem", "caseUri,name,assignees,externalUid,casePriority,status,caseSla,caseCloseTime,caseCreateTime,externalSystemUpdateTime,ticketInfo"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.setMute", "POST", "v2/{+name}:setMute", "", "SetMuteRequest", "mute"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.setState", "POST", "v2/{+name}:setState", "", "SetFindingStateRequest", "state"},
|
||||
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.assets.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.findings.bulkMute", "POST", "v2/{+parent}/findings:bulkMute", "", "BulkMuteFindingsRequest", "filter,muteState"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.bigQueryExports.create", "POST", "v2/{+parent}/bigQueryExports", "bigQueryExportId", "GoogleCloudSecuritycenterV2BigQueryExport", "name,filter,dataset,deletionNotificationsEnabled,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.bigQueryExports.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.bigQueryExports.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.bigQueryExports.list", "GET", "v2/{+parent}/bigQueryExports", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.bigQueryExports.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2BigQueryExport", "name,filter,dataset,deletionNotificationsEnabled,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.findings.bulkMute", "POST", "v2/{+parent}/findings:bulkMute", "", "BulkMuteFindingsRequest", "filter,muteState"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.muteConfigs.create", "POST", "v2/{+parent}/muteConfigs", "muteConfigId", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.muteConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.muteConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.muteConfigs.list", "GET", "v2/{+parent}/muteConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.muteConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.notificationConfigs.create", "POST", "v2/{+parent}/notificationConfigs", "configId", "NotificationConfig", "streamingConfig,pubsubTopic,description,name,deletionNotificationsEnabled"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.notificationConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.notificationConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.notificationConfigs.list", "GET", "v2/{+parent}/notificationConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.locations.notificationConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "NotificationConfig", "streamingConfig,pubsubTopic,description,name,deletionNotificationsEnabled"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.muteConfigs.create", "POST", "v2/{+parent}/muteConfigs", "muteConfigId", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.muteConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.muteConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.muteConfigs.list", "GET", "v2/{+parent}/muteConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.muteConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.operations.cancel", "POST", "v2/{+name}:cancel", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.operations.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.operations.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.operations.list", "GET", "v2/{+name}", "filter,pageSize,pageToken,returnPartialSuccess", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.create", "POST", "v2/{+parent}/sources", "", "Source", "canonicalName,name,displayName,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.create", "POST", "v2/{+parent}/findings", "findingId", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.externalSystems.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2ExternalSystem", "caseUri,name,assignees,externalUid,casePriority,status,caseSla,caseCloseTime,caseCreateTime,externalSystemUpdateTime,ticketInfo"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.setMute", "POST", "v2/{+name}:setMute", "", "SetMuteRequest", "mute"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.setState", "POST", "v2/{+name}:setState", "", "SetFindingStateRequest", "state"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "", "GetIamPolicyRequest", "options"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.create", "POST", "v2/{+parent}/findings", "findingId", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.export", "POST", "v2/{+parent}/findings:export", "", "ExportFindingsRequest", "bigQueryDestination"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.externalSystems.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2ExternalSystem", "caseUri,name,assignees,externalUid,casePriority,status,caseSla,caseCloseTime,caseCreateTime,externalSystemUpdateTime,ticketInfo"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.setMute", "POST", "v2/{+name}:setMute", "", "SetMuteRequest", "mute"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.setState", "POST", "v2/{+name}:setState", "", "SetFindingStateRequest", "state"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.patch", "PATCH", "v2/{+name}", "updateMask", "Source", "canonicalName,name,displayName,description"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "", "SetIamPolicyRequest", "updateMask,policy"},
|
||||
{"securitycenter", "v2", "securitycenter.organizations.sources.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "", "TestIamPermissionsRequest", "permissions"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.assets.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.findings.bulkMute", "POST", "v2/{+parent}/findings:bulkMute", "", "BulkMuteFindingsRequest", "filter,muteState"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.bigQueryExports.create", "POST", "v2/{+parent}/bigQueryExports", "bigQueryExportId", "GoogleCloudSecuritycenterV2BigQueryExport", "name,filter,dataset,deletionNotificationsEnabled,description"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.bigQueryExports.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.bigQueryExports.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.bigQueryExports.list", "GET", "v2/{+parent}/bigQueryExports", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.bigQueryExports.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2BigQueryExport", "name,filter,dataset,deletionNotificationsEnabled,description"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.findings.bulkMute", "POST", "v2/{+parent}/findings:bulkMute", "", "BulkMuteFindingsRequest", "filter,muteState"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.muteConfigs.create", "POST", "v2/{+parent}/muteConfigs", "muteConfigId", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.muteConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.muteConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.muteConfigs.list", "GET", "v2/{+parent}/muteConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.muteConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.notificationConfigs.create", "POST", "v2/{+parent}/notificationConfigs", "configId", "NotificationConfig", "streamingConfig,pubsubTopic,description,name,deletionNotificationsEnabled"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.notificationConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.notificationConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.notificationConfigs.list", "GET", "v2/{+parent}/notificationConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.locations.notificationConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "NotificationConfig", "streamingConfig,pubsubTopic,description,name,deletionNotificationsEnabled"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.muteConfigs.create", "POST", "v2/{+parent}/muteConfigs", "muteConfigId", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.muteConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.muteConfigs.get", "GET", "v2/{+name}", "", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.muteConfigs.list", "GET", "v2/{+parent}/muteConfigs", "pageSize,pageToken", "", ""},
|
||||
{"securitycenter", "v2", "securitycenter.projects.muteConfigs.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2MuteConfig", "description,name,type,filter,expiryTime"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.externalSystems.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2ExternalSystem", "caseUri,name,assignees,externalUid,casePriority,status,caseSla,caseCloseTime,caseCreateTime,externalSystemUpdateTime,ticketInfo"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.setMute", "POST", "v2/{+name}:setMute", "", "SetMuteRequest", "mute"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.setState", "POST", "v2/{+name}:setState", "", "SetFindingStateRequest", "state"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.findings.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.export", "POST", "v2/{+parent}/findings:export", "", "ExportFindingsRequest", "bigQueryDestination"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.externalSystems.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2ExternalSystem", "caseUri,name,assignees,externalUid,casePriority,status,caseSla,caseCloseTime,caseCreateTime,externalSystemUpdateTime,ticketInfo"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.patch", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2Finding", "iamBindings,eventTime,loadBalancers,findingClass,secret,database,moduleName,securityPosture,dataRetentionDeletionEvents,indicator,vertexAi,kubernetes,chokepoint,cloudArmor,exfiltration,files,notebook,toxicCombination,iamDetails,agentAnomaly,name,artifactGuardPolicies,compliances,processes,backupDisasterRecovery,parent,severity,externalExposure,connections,groupMemberships,attackExposure,disk,discoveredWorkload,agentDataAccessEvents,policyViolationSummary,logEntries,muteInitiator,vulnerability,externalUri,category,agent,mitreAttack,resourceName,orgPolicies,networks,kernelRootkit,dataAccessEvents,agentSessions,application,ipRules,job,cloudDlpInspection,aiModel,nextSteps,access,mute,cloudDlpDataProfile,affectedResources,complianceDetails,dataFlowEvents,containers,sourceProperties,description"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.setMute", "POST", "v2/{+name}:setMute", "", "SetMuteRequest", "mute"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.setState", "POST", "v2/{+name}:setState", "", "SetFindingStateRequest", "state"},
|
||||
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.updateSecurityMarks", "PATCH", "v2/{+name}", "updateMask", "GoogleCloudSecuritycenterV2SecurityMarks", "name,marks,canonicalName"},
|
||||
};
|
||||
|
||||
// Request bodies (what to_json emits) and response documents (what from_json reads).
|
||||
@@ -138,6 +253,11 @@ inline constexpr SchemaSpec kSchemas[] = {
|
||||
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Cve", "id"},
|
||||
{"securitycenter", "v2", "ListSourcesResponse", "sources,nextPageToken"},
|
||||
{"securitycenter", "v2", "Source", "name,displayName,description,canonicalName"},
|
||||
{"securitycenter", "v2", "ListMuteConfigsResponse", "muteConfigs,nextPageToken"},
|
||||
{"securitycenter", "v2", "ListNotificationConfigsResponse", "notificationConfigs,nextPageToken"},
|
||||
{"securitycenter", "v2", "ListBigQueryExportsResponse", "bigQueryExports,nextPageToken"},
|
||||
{"securitycenter", "v2", "ListOperationsResponse", "operations,nextPageToken"},
|
||||
{"securitycenter", "v2", "Operation", "name,done,error,response,metadata"},
|
||||
};
|
||||
|
||||
inline constexpr EnumSpec kEnums[] = {
|
||||
@@ -153,6 +273,10 @@ inline constexpr EnumSpec kEnums[] = {
|
||||
{"modelarmor", "v1", "ByteDataItem", "byteDataType", "PLAINTEXT_UTF8,PDF,WORD_DOCUMENT,EXCEL_DOCUMENT,POWERPOINT_DOCUMENT,TXT,CSV,IMAGE"},
|
||||
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Finding", "state", "ACTIVE,INACTIVE"},
|
||||
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Finding", "severity", "CRITICAL,HIGH,MEDIUM,LOW"},
|
||||
{"securitycenter", "v2", "SetMuteRequest", "mute", "MUTED,UNMUTED,UNDEFINED"},
|
||||
{"securitycenter", "v2", "SetFindingStateRequest", "state", "ACTIVE,INACTIVE"},
|
||||
{"securitycenter", "v2", "BulkMuteFindingsRequest", "muteState", "MUTED,UNDEFINED"},
|
||||
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2MuteConfig", "type", "STATIC,DYNAMIC"},
|
||||
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Finding", "findingClass", "THREAT,VULNERABILITY,MISCONFIGURATION,OBSERVATION"},
|
||||
};
|
||||
// clang-format on
|
||||
|
||||
@@ -34,7 +34,7 @@ struct ArmorCaps {
|
||||
static constexpr std::size_t kMaxFindingPages = 10;
|
||||
static constexpr std::size_t kMaxFindings = 5000;
|
||||
static constexpr std::size_t kMaxFilterValueChars = 256;
|
||||
static constexpr std::size_t kMaxFilterCategories = 20;
|
||||
static constexpr std::size_t kMaxFilterCategories = 100;
|
||||
static constexpr std::size_t kMaxSources = 100;
|
||||
// Distinct assets reported by the local dashboard.
|
||||
static constexpr std::size_t kMaxDashboardAssets = 500;
|
||||
|
||||
@@ -33,6 +33,17 @@ bool safe_value(const std::string& v) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// Finding categories are display names such as "Execution: Container Escape" or
|
||||
// "Execution: Possible Remote Command Execution Detected": printable ASCII with
|
||||
// spaces, colons and parentheses, never a quote or backslash (the value is
|
||||
// placed inside a quoted filter string).
|
||||
bool safe_category(const std::string& v) {
|
||||
if (v.empty() || v.size() > ArmorCaps::kMaxFilterValueChars) return false;
|
||||
for (unsigned char c : v)
|
||||
if (c < 0x20 || c > 0x7e || c == '"' || c == '\\') return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
// Comma separated field paths: letters, digits, '_' and '.'.
|
||||
bool safe_path_list(const std::string& v) {
|
||||
if (v.empty() || v.size() > ArmorCaps::kMaxFilterValueChars) return false;
|
||||
@@ -166,7 +177,7 @@ void FindingQuery::validate() const {
|
||||
if (categories.size() > ArmorCaps::kMaxFilterCategories)
|
||||
throw GoogleError::invalid("too many categories");
|
||||
for (const auto& c : categories)
|
||||
if (!safe_value(c)) throw GoogleError::invalid("bad category");
|
||||
if (!safe_category(c)) throw GoogleError::invalid("bad category");
|
||||
if (!state.empty() && !one_of(state, {"ACTIVE", "INACTIVE"}))
|
||||
throw GoogleError::invalid("state must be ACTIVE or INACTIVE");
|
||||
if (!severity.empty() && !one_of(severity, {"CRITICAL", "HIGH", "MEDIUM", "LOW"}))
|
||||
|
||||
@@ -104,16 +104,17 @@ std::string ModelArmor::template_path(const std::string& id) const {
|
||||
return "/" + context_.child("templates", id);
|
||||
}
|
||||
|
||||
ServiceEndpoint ModelArmor::floor_endpoint() const {
|
||||
return endpoint_for(context_.location);
|
||||
}
|
||||
// Observed live 2026-10-24: floor settings exist only at location "global"
|
||||
// on the global host (a regional name answers 400 "Malformed name"), and
|
||||
// projects.locations.list answers 403 on a regional host.
|
||||
ServiceEndpoint ModelArmor::floor_endpoint() const { return endpoint_for("global"); }
|
||||
|
||||
std::string ModelArmor::floor_path(const std::string& scope) const {
|
||||
std::string s = scope;
|
||||
context_.parent(); // Validates project and location.
|
||||
if (s.empty()) s = "projects/" + context_.project;
|
||||
if (!valid_scope(s)) throw GoogleError::invalid("bad floor setting scope");
|
||||
return "/" + s + "/locations/" + context_.location + "/floorSetting";
|
||||
return "/" + s + "/locations/global/floorSetting";
|
||||
}
|
||||
|
||||
std::string ModelArmor::request_id_query(const std::string& request_id, char sep) {
|
||||
@@ -216,7 +217,8 @@ std::vector<ArmorLocation> ModelArmor::list_locations() {
|
||||
}
|
||||
|
||||
ArmorLocation ModelArmor::get_location() {
|
||||
Json l = client_->get_json(floor_endpoint(), "/" + context_.parent());
|
||||
// A location is served by its own host (regional or global).
|
||||
Json l = client_->get_json(endpoint_for(context_.location), "/" + context_.parent());
|
||||
return {string_at(l, "name"), string_at(l, "locationId"), string_at(l, "displayName")};
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
#include "google/armor/scc_name.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::armor {
|
||||
|
||||
SccName SccName::parse(const std::string& name) {
|
||||
if (name.empty() || name.size() > 512 || name.front() == '/' ||
|
||||
name.find("..") != std::string::npos || name.find("//") != std::string::npos)
|
||||
throw GoogleError::invalid("bad Security Command Center resource name");
|
||||
SccName out;
|
||||
out.full_ = name;
|
||||
std::string cur;
|
||||
for (char c : name) {
|
||||
if (c == '/') {
|
||||
out.segments_.push_back(cur);
|
||||
cur.clear();
|
||||
} else {
|
||||
cur.push_back(c);
|
||||
}
|
||||
}
|
||||
out.segments_.push_back(cur);
|
||||
const auto& s = out.segments_;
|
||||
if (s.size() < 2 || s.size() > 12)
|
||||
throw GoogleError::invalid("resource name needs 2..12 segments");
|
||||
for (const std::string& seg : s)
|
||||
if (!valid_segment(seg)) throw GoogleError::invalid("resource name has an invalid segment");
|
||||
if (s[0] != "organizations" && s[0] != "folders" && s[0] != "projects")
|
||||
throw GoogleError::invalid("resource name must start with organizations/, folders/ or projects/");
|
||||
out.kind_ = s[0];
|
||||
if (s.size() > 2 && s[2] == "locations") out.has_location_ = true;
|
||||
if (s.size() > 4 && s[2] == "sources" && s[4] == "locations") out.has_location_ = true;
|
||||
return out;
|
||||
}
|
||||
|
||||
bool SccName::names_collection(const std::string& collection) const {
|
||||
for (std::size_t i = 2; i < segments_.size(); ++i)
|
||||
if (segments_[i] == collection) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
bool SccName::ends_with_id_of(const std::string& collection) const {
|
||||
return segments_.size() >= 4 && segments_[segments_.size() - 2] == collection;
|
||||
}
|
||||
|
||||
std::string SccName::rpc_for(const std::string& tail) const {
|
||||
std::string base = "securitycenter." + kind_ + ".";
|
||||
const std::string sources = "sources.";
|
||||
if (tail.compare(0, sources.size(), sources) == 0)
|
||||
return base + sources + (has_location_ ? "locations." : "") + tail.substr(sources.size());
|
||||
return base + (has_location_ ? "locations." : "") + tail;
|
||||
}
|
||||
|
||||
const CallSpec* SccName::row(const std::string& tail) const { return find_rpc(rpc_for(tail)); }
|
||||
|
||||
} // namespace molo::google::armor
|
||||
@@ -0,0 +1,47 @@
|
||||
#pragma once
|
||||
// A validated Security Command Center v2 resource name or parent and the
|
||||
// table row that serves a method for it. Discovery lists each method per
|
||||
// scope (organizations, folders, projects) and, for most, again under
|
||||
// `locations`; the row is derived from the name itself so a caller can never
|
||||
// reach a different scope than the one it named:
|
||||
// projects/p/locations/global/muteConfigs/x -> securitycenter.projects.locations.muteConfigs.*
|
||||
// projects/p/muteConfigs/x -> securitycenter.projects.muteConfigs.*
|
||||
// organizations/1/sources/2/findings/f -> securitycenter.organizations.sources.findings.*
|
||||
// organizations/1/sources/2/locations/l/findings/f -> ...sources.locations.findings.*
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/armor/calls.h"
|
||||
|
||||
namespace molo::google::armor {
|
||||
|
||||
class SccName {
|
||||
public:
|
||||
// Throws GoogleError::invalid when the name is not a well formed resource
|
||||
// name (scope kind, valid id, valid segments, at most 12, no "..").
|
||||
static SccName parse(const std::string& name);
|
||||
|
||||
const std::string& str() const { return full_; }
|
||||
const std::string& kind() const { return kind_; } // organizations|folders|projects
|
||||
const std::vector<std::string>& segments() const { return segments_; }
|
||||
// True when the name goes through a `locations/{l}` segment (either directly
|
||||
// under the scope, or under a source).
|
||||
bool has_location() const { return has_location_; }
|
||||
// True when the last segment is `collection` (a parent names the scope, a
|
||||
// resource name ends with collection/id).
|
||||
bool ends_with_id_of(const std::string& collection) const;
|
||||
bool names_collection(const std::string& collection) const;
|
||||
|
||||
// The declared row for a method tail such as "muteConfigs.get" or
|
||||
// "sources.findings.setState"; nullptr when this scope/location has none.
|
||||
const CallSpec* row(const std::string& tail) const;
|
||||
// The rpc id this name would use for the tail (not checked against the table).
|
||||
std::string rpc_for(const std::string& tail) const;
|
||||
|
||||
private:
|
||||
std::string full_, kind_;
|
||||
std::vector<std::string> segments_;
|
||||
bool has_location_ = false;
|
||||
};
|
||||
|
||||
} // namespace molo::google::armor
|
||||
@@ -0,0 +1,439 @@
|
||||
#include "google/armor/security_admin.h"
|
||||
|
||||
#include <set>
|
||||
#include <sstream>
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "google/armor/caps.h"
|
||||
#include "google/armor/findings.h"
|
||||
#include "google/errors.h"
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::armor {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& m) { throw GoogleError::invalid(m); }
|
||||
|
||||
std::set<std::string> csv(const char* list, char sep = ',') {
|
||||
std::set<std::string> out;
|
||||
std::string cur;
|
||||
for (const char* p = list; *p; ++p) {
|
||||
if (*p == sep) {
|
||||
if (!cur.empty()) out.insert(cur);
|
||||
cur.clear();
|
||||
} else {
|
||||
cur.push_back(*p);
|
||||
}
|
||||
}
|
||||
if (!cur.empty()) out.insert(cur);
|
||||
return out;
|
||||
}
|
||||
|
||||
void check_mask(const CallSpec& row, const std::string& mask) {
|
||||
if (mask.empty() || mask.size() > 256) bad("update_mask is required (max 256 characters)");
|
||||
std::set<std::string> roots = csv(row.fields);
|
||||
std::size_t pos = 0;
|
||||
while (pos <= mask.size()) {
|
||||
std::size_t end = mask.find(',', pos);
|
||||
if (end == std::string::npos) end = mask.size();
|
||||
std::string path = mask.substr(pos, end - pos);
|
||||
if (path.empty()) bad("update_mask has an empty path");
|
||||
for (char c : path)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '_' || c == '.'))
|
||||
bad("update_mask allows letters, digits, '_' and '.'");
|
||||
// Security marks masks name map keys ("marks.k"); other masks name fields.
|
||||
std::string root = path.substr(0, path.find('.'));
|
||||
if (!roots.count(root)) bad("update_mask field not allowed for this call: " + path);
|
||||
pos = end + 1;
|
||||
}
|
||||
}
|
||||
|
||||
void check_id(const std::string& id, const char* what) {
|
||||
if (!valid_segment(id)) bad(std::string(what) + " must be 1-128 of A-Z a-z 0-9 . _ -");
|
||||
}
|
||||
|
||||
const char* id_param(SccCollection c) {
|
||||
switch (c) {
|
||||
case SccCollection::kMuteConfigs: return "muteConfigId";
|
||||
case SccCollection::kNotificationConfigs: return "configId";
|
||||
case SccCollection::kBigQueryExports: return "bigQueryExportId";
|
||||
}
|
||||
return "muteConfigId";
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
const char* scc_collection_name(SccCollection c) {
|
||||
switch (c) {
|
||||
case SccCollection::kMuteConfigs: return "muteConfigs";
|
||||
case SccCollection::kNotificationConfigs: return "notificationConfigs";
|
||||
case SccCollection::kBigQueryExports: return "bigQueryExports";
|
||||
}
|
||||
return "muteConfigs";
|
||||
}
|
||||
|
||||
const char* finding_state_name(FindingState s) {
|
||||
return s == FindingState::kActive ? "ACTIVE" : "INACTIVE";
|
||||
}
|
||||
|
||||
const char* mute_value_name(MuteValue v) {
|
||||
switch (v) {
|
||||
case MuteValue::kMuted: return "MUTED";
|
||||
case MuteValue::kUnmuted: return "UNMUTED";
|
||||
case MuteValue::kUndefined: return "UNDEFINED";
|
||||
}
|
||||
return "UNDEFINED";
|
||||
}
|
||||
|
||||
const char* availability_name(SccAvailability a) {
|
||||
switch (a) {
|
||||
case SccAvailability::kAvailable: return "available";
|
||||
case SccAvailability::kNotActivated: return "not_activated";
|
||||
case SccAvailability::kApiDisabled: return "api_disabled";
|
||||
case SccAvailability::kDenied: return "denied";
|
||||
case SccAvailability::kUnknown: return "unknown";
|
||||
}
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
Json SccList::to_json() const {
|
||||
Json o = Json::object();
|
||||
o.set("count", static_cast<int>(items.as_array().size()));
|
||||
o.set("pages", static_cast<int>(pages));
|
||||
o.set("truncated", truncated);
|
||||
o.set("next_page_token", next_page_token);
|
||||
o.set("items", items);
|
||||
return o;
|
||||
}
|
||||
|
||||
SecurityAdmin::SecurityAdmin(std::shared_ptr<GoogleClient> client)
|
||||
: Service(std::move(client), SecurityFindings::endpoint()) {}
|
||||
|
||||
namespace {
|
||||
|
||||
struct Planned {
|
||||
const CallSpec* row;
|
||||
std::string path; // leading slash, no version, with query
|
||||
clients::Method method;
|
||||
};
|
||||
|
||||
Planned plan(const std::string& tail, const std::string& resource, const Json* body,
|
||||
const std::map<std::string, std::string>& query) {
|
||||
SccName n = SccName::parse(resource);
|
||||
const CallSpec* row = n.row(tail);
|
||||
if (!row)
|
||||
bad(tail + " is not available for " + n.kind() + (n.has_location() ? " with a location" : " without a location"));
|
||||
std::set<std::string> allowed = csv(row->query);
|
||||
std::string q;
|
||||
for (const auto& [k, v] : query) {
|
||||
if (!allowed.count(k)) bad("query parameter " + k + " is not in " + row->rpc);
|
||||
if (v.empty()) continue;
|
||||
if (v.size() > 2048) bad("query value too long");
|
||||
for (char c : v)
|
||||
if (static_cast<unsigned char>(c) < 0x20) bad("query value has control characters");
|
||||
q += (q.empty() ? "?" : "&") + k + "=" + clients::Url::encode_component(v);
|
||||
}
|
||||
std::string request = row->request;
|
||||
if (!request.empty()) {
|
||||
if (!body || !body->is_object()) bad(std::string(row->rpc) + " needs a JSON object body");
|
||||
std::set<std::string> fields = csv(row->fields);
|
||||
for (const auto& [k, v] : body->as_object()) {
|
||||
(void)v;
|
||||
if (!fields.count(k)) bad("field " + k + " is not part of " + request);
|
||||
}
|
||||
if (body->dump().size() > 256 * 1024) bad("request body over 256 KiB");
|
||||
} else if (body) {
|
||||
bad(std::string(row->rpc) + " takes no body");
|
||||
}
|
||||
// Discovery path "v2/{+parent}/muteConfigs": drop the version, fill the one variable.
|
||||
std::string tmpl = row->path;
|
||||
tmpl.erase(0, tmpl.find('/'));
|
||||
std::size_t open = tmpl.find('{'), close = tmpl.find('}');
|
||||
if (open == std::string::npos || close == std::string::npos) bad("bad row path");
|
||||
std::string path = tmpl.substr(0, open) + n.str() + tmpl.substr(close + 1) + q;
|
||||
std::string verb = row->http;
|
||||
clients::Method m = verb == "POST" ? clients::Method::kPost
|
||||
: verb == "PATCH" ? clients::Method::kPatch
|
||||
: verb == "DELETE" ? clients::Method::kDelete
|
||||
: clients::Method::kGet;
|
||||
return {row, path, m};
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Json SecurityAdmin::run(const std::string& tail, const std::string& resource, const Json* body,
|
||||
const std::map<std::string, std::string>& query) const {
|
||||
Planned p = plan(tail, resource, body, query);
|
||||
Json empty = Json::object();
|
||||
const Json* send = body;
|
||||
if (!send && p.method == clients::Method::kPost) send = ∅ // cancel and friends
|
||||
return client_->request(p.method, endpoint_, p.path, send);
|
||||
}
|
||||
|
||||
std::string SecurityAdmin::describe(const std::string& tail, const std::string& resource,
|
||||
const std::map<std::string, std::string>& query) const {
|
||||
Json empty = Json::object();
|
||||
// Describing a call with a body still validates the body shape: use {} and
|
||||
// let callers that need fields pass them through run().
|
||||
Planned p = [&] {
|
||||
try {
|
||||
return plan(tail, resource, nullptr, query);
|
||||
} catch (const GoogleError&) {
|
||||
return plan(tail, resource, &empty, query);
|
||||
}
|
||||
}();
|
||||
return std::string(p.row->http) + " " + p.path;
|
||||
}
|
||||
|
||||
void SecurityAdmin::expect_name(const SccName& n, const std::string& collection, bool resource) {
|
||||
if (resource) {
|
||||
if (!n.ends_with_id_of(collection)) bad("name must end with /" + collection + "/{id}");
|
||||
} else if (n.names_collection(collection)) {
|
||||
bad("parent must not include /" + collection);
|
||||
}
|
||||
}
|
||||
|
||||
SccList SecurityAdmin::list_all_pages(const std::string& tail, const std::string& parent,
|
||||
const std::string& items_key) const {
|
||||
SccList out;
|
||||
std::string token;
|
||||
while (true) {
|
||||
std::map<std::string, std::string> q = {{"pageSize", std::to_string(ArmorCaps::kMaxTemplatePageSize)}};
|
||||
if (!token.empty()) q["pageToken"] = token;
|
||||
Json reply = run(tail, parent, nullptr, q);
|
||||
++out.pages;
|
||||
if (const Json* arr = reply.is_object() ? reply.find(items_key) : nullptr; arr && arr->is_array())
|
||||
for (const Json& item : arr->as_array()) {
|
||||
if (out.items.as_array().size() >= ArmorCaps::kMaxFindings) {
|
||||
out.truncated = true;
|
||||
return out;
|
||||
}
|
||||
out.items.push(item);
|
||||
}
|
||||
const Json* next = reply.is_object() ? reply.find("nextPageToken") : nullptr;
|
||||
token = next && next->is_string() ? next->as_string() : "";
|
||||
if (token.empty()) return out;
|
||||
if (out.pages >= ArmorCaps::kMaxFindingPages) {
|
||||
out.truncated = true;
|
||||
out.next_page_token = token;
|
||||
return out;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Configs
|
||||
|
||||
SccList SecurityAdmin::list_configs(SccCollection c, const std::string& parent) const {
|
||||
expect_name(SccName::parse(parent), scc_collection_name(c), false);
|
||||
return list_all_pages(std::string(scc_collection_name(c)) + ".list", parent,
|
||||
scc_collection_name(c));
|
||||
}
|
||||
|
||||
Json SecurityAdmin::get_config(SccCollection c, const std::string& name) const {
|
||||
expect_name(SccName::parse(name), scc_collection_name(c), true);
|
||||
return run(std::string(scc_collection_name(c)) + ".get", name);
|
||||
}
|
||||
|
||||
Json SecurityAdmin::create_config(SccCollection c, const std::string& parent,
|
||||
const std::string& id, const Json& body) const {
|
||||
check_id(id, "config id");
|
||||
expect_name(SccName::parse(parent), scc_collection_name(c), false);
|
||||
return run(std::string(scc_collection_name(c)) + ".create", parent, &body,
|
||||
{{id_param(c), id}});
|
||||
}
|
||||
|
||||
Json SecurityAdmin::update_config(SccCollection c, const std::string& name, const Json& body,
|
||||
const std::string& update_mask) const {
|
||||
SccName n = SccName::parse(name);
|
||||
expect_name(n, scc_collection_name(c), true);
|
||||
std::string tail = std::string(scc_collection_name(c)) + ".patch";
|
||||
if (const CallSpec* row = n.row(tail)) check_mask(*row, update_mask);
|
||||
return run(tail, name, &body, {{"updateMask", update_mask}});
|
||||
}
|
||||
|
||||
void SecurityAdmin::delete_config(SccCollection c, const std::string& name) const {
|
||||
expect_name(SccName::parse(name), scc_collection_name(c), true);
|
||||
run(std::string(scc_collection_name(c)) + ".delete", name);
|
||||
}
|
||||
|
||||
// ---- Findings
|
||||
|
||||
Json SecurityAdmin::set_finding_state(const std::string& finding, FindingState state) const {
|
||||
expect_name(SccName::parse(finding), "findings", true);
|
||||
Json body = Json::object();
|
||||
body.set("state", finding_state_name(state));
|
||||
return run("sources.findings.setState", finding, &body);
|
||||
}
|
||||
|
||||
Json SecurityAdmin::set_finding_mute(const std::string& finding, MuteValue mute) const {
|
||||
expect_name(SccName::parse(finding), "findings", true);
|
||||
Json body = Json::object();
|
||||
body.set("mute", mute_value_name(mute));
|
||||
return run("sources.findings.setMute", finding, &body);
|
||||
}
|
||||
|
||||
Json SecurityAdmin::patch_finding(const std::string& finding, const Json& body,
|
||||
const std::string& update_mask) const {
|
||||
SccName n = SccName::parse(finding);
|
||||
expect_name(n, "findings", true);
|
||||
if (const CallSpec* row = n.row("sources.findings.patch")) check_mask(*row, update_mask);
|
||||
return run("sources.findings.patch", finding, &body, {{"updateMask", update_mask}});
|
||||
}
|
||||
|
||||
Json SecurityAdmin::update_finding_marks(const std::string& finding, const Json& marks,
|
||||
const std::string& update_mask) const {
|
||||
SccName n = SccName::parse(finding);
|
||||
expect_name(n, "findings", true);
|
||||
// The resource is ".../findings/{f}" and the call patches its securityMarks.
|
||||
if (const CallSpec* row = n.row("sources.findings.updateSecurityMarks")) check_mask(*row, update_mask);
|
||||
return run("sources.findings.updateSecurityMarks", finding + "/securityMarks", &marks,
|
||||
{{"updateMask", update_mask}});
|
||||
}
|
||||
|
||||
Json SecurityAdmin::create_finding(const std::string& source, const std::string& finding_id,
|
||||
const Json& body) const {
|
||||
check_id(finding_id, "finding id");
|
||||
expect_name(SccName::parse(source), "findings", false);
|
||||
return run("sources.findings.create", source, &body, {{"findingId", finding_id}});
|
||||
}
|
||||
|
||||
Json SecurityAdmin::patch_external_system(const std::string& external_system, const Json& body,
|
||||
const std::string& update_mask) const {
|
||||
SccName n = SccName::parse(external_system);
|
||||
expect_name(n, "externalSystems", true);
|
||||
if (const CallSpec* row = n.row("sources.findings.externalSystems.patch")) check_mask(*row, update_mask);
|
||||
return run("sources.findings.externalSystems.patch", external_system, &body,
|
||||
{{"updateMask", update_mask}});
|
||||
}
|
||||
|
||||
Operation SecurityAdmin::bulk_mute(const std::string& parent, const std::string& filter,
|
||||
BulkMuteValue value) const {
|
||||
if (filter.empty() || filter.size() > ArmorCaps::kMaxFilterValueChars * 4)
|
||||
bad("bulk mute needs a filter (1.." + std::to_string(ArmorCaps::kMaxFilterValueChars * 4) +
|
||||
" characters); it never mutes everything");
|
||||
expect_name(SccName::parse(parent), "findings", false);
|
||||
Json body = Json::object();
|
||||
body.set("filter", filter);
|
||||
body.set("muteState", value == BulkMuteValue::kMuted ? "MUTED" : "UNDEFINED");
|
||||
return Operation::from_json(run("findings.bulkMute", parent, &body));
|
||||
}
|
||||
|
||||
Operation SecurityAdmin::export_findings(const std::string& source, const Json& body) const {
|
||||
SccName n = SccName::parse(source);
|
||||
if (!n.has_location()) bad("findings:export needs a source under a location");
|
||||
return Operation::from_json(run("sources.findings.export", source, &body));
|
||||
}
|
||||
|
||||
// ---- Sources
|
||||
|
||||
Json SecurityAdmin::create_source(const std::string& organization, const Json& body) const {
|
||||
SccName n = SccName::parse(organization);
|
||||
if (n.kind() != "organizations" || n.segments().size() != 2)
|
||||
bad("sources are created under organizations/{id}");
|
||||
return run("sources.create", organization, &body);
|
||||
}
|
||||
|
||||
Json SecurityAdmin::update_source(const std::string& source, const Json& body,
|
||||
const std::string& update_mask) const {
|
||||
SccName n = SccName::parse(source);
|
||||
expect_name(n, "sources", true);
|
||||
if (const CallSpec* row = n.row("sources.patch")) check_mask(*row, update_mask);
|
||||
return run("sources.patch", source, &body, {{"updateMask", update_mask}});
|
||||
}
|
||||
|
||||
Json SecurityAdmin::source_iam_policy(const std::string& source) const {
|
||||
expect_name(SccName::parse(source), "sources", true);
|
||||
Json body = Json::object();
|
||||
return run("sources.getIamPolicy", source, &body);
|
||||
}
|
||||
|
||||
Json SecurityAdmin::set_source_iam_policy(const std::string& source, const Json& policy) const {
|
||||
expect_name(SccName::parse(source), "sources", true);
|
||||
if (!policy.is_object()) bad("policy must be a JSON object");
|
||||
source_iam_policy(source); // Read first.
|
||||
Json body = Json::object();
|
||||
body.set("policy", policy);
|
||||
return run("sources.setIamPolicy", source, &body);
|
||||
}
|
||||
|
||||
Json SecurityAdmin::test_source_iam(const std::string& source,
|
||||
const std::vector<std::string>& permissions) const {
|
||||
expect_name(SccName::parse(source), "sources", true);
|
||||
if (permissions.empty() || permissions.size() > 64) bad("1..64 permissions");
|
||||
Json arr = Json::array();
|
||||
for (const std::string& p : permissions) {
|
||||
if (p.empty() || p.size() > 256) bad("permission empty or over 256 characters");
|
||||
arr.push(p);
|
||||
}
|
||||
Json body = Json::object();
|
||||
body.set("permissions", arr);
|
||||
return run("sources.testIamPermissions", source, &body);
|
||||
}
|
||||
|
||||
Json SecurityAdmin::update_asset_marks(const std::string& marks_name, const Json& marks,
|
||||
const std::string& update_mask) const {
|
||||
SccName n = SccName::parse(marks_name);
|
||||
if (!n.names_collection("assets") || marks_name.size() < 14 ||
|
||||
marks_name.compare(marks_name.size() - 14, 14, "/securityMarks") != 0)
|
||||
bad("name must be {scope}/assets/{id}/securityMarks");
|
||||
if (const CallSpec* row = n.row("assets.updateSecurityMarks")) check_mask(*row, update_mask);
|
||||
return run("assets.updateSecurityMarks", marks_name, &marks, {{"updateMask", update_mask}});
|
||||
}
|
||||
|
||||
// ---- Operations
|
||||
|
||||
Json SecurityAdmin::get_operation(const std::string& name) const {
|
||||
if (!SccName::parse(name).ends_with_id_of("operations")) bad("name must be organizations/{id}/operations/{op}");
|
||||
return run("operations.get", name);
|
||||
}
|
||||
|
||||
SccList SecurityAdmin::list_operations(const std::string& organization) const {
|
||||
// Discovery: operations.list takes the collection name itself as {+name}.
|
||||
SccName n = SccName::parse(organization);
|
||||
if (n.kind() != "organizations" || n.segments().size() != 2)
|
||||
bad("operations are listed under organizations/{id}");
|
||||
return list_all_pages("operations.list", organization + "/operations", "operations");
|
||||
}
|
||||
|
||||
void SecurityAdmin::cancel_operation(const std::string& name) const {
|
||||
get_operation(name); // Read first: a missing operation is a clear 404.
|
||||
run("operations.cancel", name);
|
||||
}
|
||||
|
||||
void SecurityAdmin::delete_operation(const std::string& name) const {
|
||||
get_operation(name);
|
||||
run("operations.delete", name);
|
||||
}
|
||||
|
||||
Operation SecurityAdmin::wait_operation_done(const std::string& name) const {
|
||||
if (!SccName::parse(name).ends_with_id_of("operations")) bad("name must be organizations/{id}/operations/{op}");
|
||||
return google::wait_operation(*client_, endpoint_, name);
|
||||
}
|
||||
|
||||
// ---- Probe
|
||||
|
||||
SccAvailability SecurityAdmin::availability(const std::string& scope) const {
|
||||
SccName n = SccName::parse(scope);
|
||||
if (n.segments().size() != 2) bad("scope must be organizations/{id}, folders/{id} or projects/{id}");
|
||||
try {
|
||||
run("sources.list", scope, nullptr, {{"pageSize", "1"}});
|
||||
return SccAvailability::kAvailable;
|
||||
} catch (const NotConfiguredError&) {
|
||||
throw;
|
||||
} catch (const GoogleError& e) {
|
||||
if (e.service().empty() && e.status() == "INVALID_ARGUMENT") throw;
|
||||
const std::string& m = e.message();
|
||||
if (e.http_status() == 403 &&
|
||||
(m.find("has not been used") != std::string::npos || m.find("disabled") != std::string::npos))
|
||||
return SccAvailability::kApiDisabled;
|
||||
if (e.http_status() == 403) return SccAvailability::kDenied;
|
||||
if (e.http_status() == 404 || e.status() == "FAILED_PRECONDITION") return SccAvailability::kNotActivated;
|
||||
return SccAvailability::kUnknown;
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace molo::google::armor
|
||||
@@ -0,0 +1,131 @@
|
||||
#pragma once
|
||||
// Security Command Center v2 beyond reading findings: mute configs,
|
||||
// notification configs, BigQuery exports, finding state/mute/marks, bulk
|
||||
// mute, findings export, sources, asset marks, source IAM and organization
|
||||
// operations. Every method is a row of the call table in calls.h, picked by
|
||||
// SccName from the resource it is given; query parameters and top-level body
|
||||
// fields are checked against the row before any traffic. Every mutation is an
|
||||
// explicit method. Lists report truncation in their result type.
|
||||
// Out of scope on purpose (calls.h / tools/check_discovery.py): the attack
|
||||
// exposure engine (attack paths, valued resources, simulations, resource value
|
||||
// configs), which is cloud posture risk scoring and not agent security.
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "google/armor/scc_name.h"
|
||||
#include "google/operation.h"
|
||||
#include "google/service.h"
|
||||
|
||||
namespace molo::google::armor {
|
||||
|
||||
enum class SccCollection { kMuteConfigs, kNotificationConfigs, kBigQueryExports };
|
||||
enum class FindingState { kActive, kInactive };
|
||||
enum class MuteValue { kMuted, kUnmuted, kUndefined }; // setMute
|
||||
enum class BulkMuteValue { kMuted, kUndefined }; // bulkMute
|
||||
|
||||
const char* scc_collection_name(SccCollection c); // "muteConfigs", ...
|
||||
const char* finding_state_name(FindingState s);
|
||||
const char* mute_value_name(MuteValue v);
|
||||
|
||||
// A paged list. Never silently cut: past the caps `truncated` is true and
|
||||
// `next_page_token` continues.
|
||||
struct SccList {
|
||||
clients::body::Json items = clients::body::Json::array();
|
||||
bool truncated = false;
|
||||
std::string next_page_token;
|
||||
std::size_t pages = 0;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
// Why Security Command Center did not answer, from a real probe (see SPEC):
|
||||
// an enabled API without SCC activation answers 404 (sources) or 400
|
||||
// FAILED_PRECONDITION (configs); a disabled API 403.
|
||||
enum class SccAvailability { kAvailable, kNotActivated, kApiDisabled, kDenied, kUnknown };
|
||||
const char* availability_name(SccAvailability a);
|
||||
|
||||
class SecurityAdmin : public Service {
|
||||
public:
|
||||
explicit SecurityAdmin(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
// ---- Generic table-driven call (tail like "muteConfigs.get"). The public
|
||||
// typed methods below are thin wrappers; this is for tools and tests.
|
||||
clients::body::Json run(const std::string& tail, const std::string& resource,
|
||||
const clients::body::Json* body = nullptr,
|
||||
const std::map<std::string, std::string>& query = {}) const;
|
||||
// The request the typed call would send: "METHOD path", no network.
|
||||
std::string describe(const std::string& tail, const std::string& resource,
|
||||
const std::map<std::string, std::string>& query = {}) const;
|
||||
|
||||
// ---- Configs: parent is "projects/p[/locations/l]" (also folders,
|
||||
// organizations); notification configs and BigQuery exports exist only
|
||||
// with a location. A name is parent + "/" + collection + "/" + id.
|
||||
SccList list_configs(SccCollection c, const std::string& parent) const;
|
||||
clients::body::Json get_config(SccCollection c, const std::string& name) const;
|
||||
clients::body::Json create_config(SccCollection c, const std::string& parent,
|
||||
const std::string& id,
|
||||
const clients::body::Json& body) const;
|
||||
clients::body::Json update_config(SccCollection c, const std::string& name,
|
||||
const clients::body::Json& body,
|
||||
const std::string& update_mask) const;
|
||||
// Destructive: explicit, one config.
|
||||
void delete_config(SccCollection c, const std::string& name) const;
|
||||
|
||||
// ---- Findings. A finding name is ".../sources/{s}[/locations/{l}]/findings/{f}".
|
||||
clients::body::Json set_finding_state(const std::string& finding, FindingState state) const;
|
||||
clients::body::Json set_finding_mute(const std::string& finding, MuteValue mute) const;
|
||||
clients::body::Json patch_finding(const std::string& finding,
|
||||
const clients::body::Json& body,
|
||||
const std::string& update_mask) const;
|
||||
clients::body::Json update_finding_marks(const std::string& finding,
|
||||
const clients::body::Json& marks,
|
||||
const std::string& update_mask) const;
|
||||
clients::body::Json create_finding(const std::string& source, const std::string& finding_id,
|
||||
const clients::body::Json& body) const;
|
||||
clients::body::Json patch_external_system(const std::string& external_system,
|
||||
const clients::body::Json& body,
|
||||
const std::string& update_mask) const;
|
||||
// Mutes or unmutes every finding matching `filter` (required, never empty).
|
||||
// Long-running; returns the operation (not waited).
|
||||
Operation bulk_mute(const std::string& parent, const std::string& filter,
|
||||
BulkMuteValue value) const;
|
||||
// findings:export (locations only), long-running; body {"bigQueryDestination":{...}}.
|
||||
Operation export_findings(const std::string& source, const clients::body::Json& body) const;
|
||||
|
||||
// ---- Sources (organization scope) and IAM on them.
|
||||
clients::body::Json create_source(const std::string& organization,
|
||||
const clients::body::Json& body) const;
|
||||
clients::body::Json update_source(const std::string& source, const clients::body::Json& body,
|
||||
const std::string& update_mask) const;
|
||||
clients::body::Json source_iam_policy(const std::string& source) const;
|
||||
clients::body::Json set_source_iam_policy(const std::string& source,
|
||||
const clients::body::Json& policy) const;
|
||||
clients::body::Json test_source_iam(const std::string& source,
|
||||
const std::vector<std::string>& permissions) const;
|
||||
|
||||
// ---- Asset security marks (assets.updateSecurityMarks).
|
||||
clients::body::Json update_asset_marks(const std::string& marks_name,
|
||||
const clients::body::Json& marks,
|
||||
const std::string& update_mask) const;
|
||||
|
||||
// ---- Organization operations (get, list, cancel, delete, wait by polling).
|
||||
clients::body::Json get_operation(const std::string& name) const;
|
||||
SccList list_operations(const std::string& organization) const;
|
||||
void cancel_operation(const std::string& name) const;
|
||||
void delete_operation(const std::string& name) const; // record only
|
||||
Operation wait_operation_done(const std::string& name) const;
|
||||
|
||||
// ---- Probe: what does Security Command Center say for this scope?
|
||||
// Read-only (sources.list, one page of one). Never throws for a Google
|
||||
// refusal; validation errors still throw.
|
||||
SccAvailability availability(const std::string& scope) const;
|
||||
|
||||
private:
|
||||
SccList list_all_pages(const std::string& tail, const std::string& parent,
|
||||
const std::string& items_key) const;
|
||||
static void expect_name(const SccName& n, const std::string& collection, bool resource);
|
||||
};
|
||||
|
||||
} // namespace molo::google::armor
|
||||
@@ -19,6 +19,13 @@ ROW = re.compile(r'^\s*\{((?:"[^"]*"(?:,\s*)?)+)\},\s*$')
|
||||
STR = re.compile(r'"([^"]*)"')
|
||||
|
||||
|
||||
# Excluded on purpose (scope): the attack exposure engine of Security Command
|
||||
# Center (attack paths, valued resources, simulations, resource value configs)
|
||||
# is cloud posture risk scoring, not agent security. Every other Security
|
||||
# Command Center v2 method and every Model Armor method is a table row.
|
||||
EXCLUDED_RE = re.compile(r"attackPaths|valuedResources|simulations|resourceValueConfigs")
|
||||
|
||||
|
||||
def rows():
|
||||
calls, schemas, enums = [], [], []
|
||||
for line in open(HEADER, encoding="utf-8"):
|
||||
@@ -26,7 +33,7 @@ def rows():
|
||||
if not m:
|
||||
continue
|
||||
cells = STR.findall(m.group(1))
|
||||
{6: calls, 4: schemas, 5: enums}.get(len(cells), []).append(cells)
|
||||
{8: calls, 4: schemas, 5: enums}.get(len(cells), []).append(cells)
|
||||
return calls, schemas, enums
|
||||
|
||||
|
||||
@@ -70,7 +77,7 @@ def main():
|
||||
return docs[key]
|
||||
|
||||
index = {}
|
||||
for service, version, rpc, http, path, query in calls:
|
||||
for service, version, rpc, http, path, query, request, fields in calls:
|
||||
d = doc(service, version)
|
||||
idx = index.setdefault((service, version), methods(d))
|
||||
m = idx.get(rpc)
|
||||
@@ -84,6 +91,25 @@ def main():
|
||||
for q in filter(None, query.split(",")):
|
||||
if q not in m.get("parameters", {}):
|
||||
errors.append("%s: query parameter %s missing" % (rpc, q))
|
||||
ref = (m.get("request") or {}).get("$ref", "")
|
||||
if ref != request:
|
||||
errors.append("%s: request schema %r != declared %r" % (rpc, ref, request))
|
||||
if ref:
|
||||
props = doc(service, version)["schemas"].get(ref, {}).get("properties", {})
|
||||
for f in filter(None, fields.split(",")):
|
||||
if f not in props:
|
||||
errors.append("%s: request field %s not in %s" % (rpc, f, ref))
|
||||
elif fields:
|
||||
errors.append("%s: fields declared for a call without a request body" % rpc)
|
||||
# Coverage: every discovery method must be a table row or be excluded.
|
||||
declared = {(c[0], c[1], c[2]) for c in calls}
|
||||
for (service, version), idx in index.items():
|
||||
for rpc in sorted(idx):
|
||||
if (service, version, rpc) in declared:
|
||||
continue
|
||||
if EXCLUDED_RE.search(rpc):
|
||||
continue
|
||||
errors.append("discovery method not in the table: " + rpc)
|
||||
for service, version, schema, fields in schemas:
|
||||
s = doc(service, version)["schemas"].get(schema)
|
||||
if not s:
|
||||
|
||||
+54
-12
@@ -34,7 +34,9 @@ optimizer; those are local abstractions and are named local in code and here.
|
||||
| `observability.h/.cpp` | `TraceReader`, `LogReader`, `Trace`, `Span`, `LogEntry`, `ConversationBuilder`, `summarize` |
|
||||
| `conversation.h/.cpp` | `Conversation`, `Turn`, transcript parse and render |
|
||||
| `model_json.h/.cpp` | `JsonModelCall`: one JSON object from a `LanguageModel`, one retry |
|
||||
| `direct_model.h/.cpp` | `DirectModel : LanguageModel` over `GoogleClient` (private to this module) |
|
||||
| `model_factory.h/.cpp` | `make_gemini_model`: the one Gemini adapter, `models::VertexModel` from `src/google/models`, target picked from the credentials (the former private `DirectModel` is gone) |
|
||||
| `operations_api.h/.cpp` | `OnlineEvaluatorOperations`: get, list (loud truncation flag), cancel, delete, server-side wait for online evaluator operations |
|
||||
| `telemetry_writer.h/.cpp` | `TraceWriter` (patchTraces, batchWrite, createSpan) and `LogWriter` (entries:write with dryRun, entries:copy) |
|
||||
| `autorater.h/.cpp` | `Autorater`, `Rubric`, `Score` |
|
||||
| `scenario.h/.cpp` | `Scenario` (mirrors Google's `UserScenario`) |
|
||||
| `agent.h/.cpp` | `Agent` seam, `ModelAgent` |
|
||||
@@ -97,23 +99,37 @@ optimizer; those are local abstractions and are named local in code and here.
|
||||
- Simulator loops are bounded by turns, tool rounds and tool calls; the
|
||||
stop reason is reported (`user_done`, `max_turns`, `tool_round_cap`,
|
||||
`tool_call_cap`).
|
||||
- Credentials: this module never reads or prints them. `DirectModel::from_env`
|
||||
picks the endpoint: OAuth token plus project gives the project-scoped
|
||||
Vertex path (global location), an express key the Vertex publisher path,
|
||||
otherwise Generative Language.
|
||||
- Credentials: this module never reads or prints them. `make_gemini_model`
|
||||
builds a `models::VertexModel` (so there is one Gemini adapter in the
|
||||
platform): an OAuth token (`GOOGLE_ACCESS_TOKEN`) with a valid
|
||||
`GOOGLE_CLOUD_PROJECT` gives the project scoped Vertex path in location
|
||||
`global`, anything else Vertex express mode (publisher path). Behavior
|
||||
change from `DirectModel`: the Generative Language endpoint (non Vertex API
|
||||
keys) is no longer used; such a key now fails at Google on the Vertex path.
|
||||
Streaming is real (`?alt=sse`) instead of one chunk.
|
||||
- Write calls are explicit classes (`TraceWriter`, `LogWriter`), bodies are raw
|
||||
Json checked against the table and capped (1 MiB, 1000 log entries, 2000
|
||||
spans). `LogWriter::write` supports `dryRun`; the demo route is a dry run
|
||||
unless `confirm` is `write`. `entries.tail` is excluded: an endless server
|
||||
stream has no cap. `OnlineEvaluatorOperations::list` stops at 20 pages /
|
||||
1000 items and says `truncated` with the next page token.
|
||||
|
||||
## 5. Example routes (`/agent/optimize`)
|
||||
|
||||
GET `/agent/optimize` (page, matrix row), GET `/agent/optimize/status`.
|
||||
POST (503 `google_not_configured` without credentials or a valid
|
||||
`GOOGLE_CLOUD_PROJECT`, 422 on bad input, 502 `google_error` upstream):
|
||||
`evaluate`, `scenarios`, `runs`, `traces`, `logs` (Google APIs) and `score`,
|
||||
`evaluate`, `scenarios`, `runs`, `traces`, `logs`, `synthetic`, `operations/list`, `operations/get`, `operations/wait`, `operations/cancel`, `logs/write` (Google APIs; DELETE `operations?name=` removes an operation record) and `score`,
|
||||
`cluster`, `suggest`, `simulate` (local logic on a Gemini model).
|
||||
|
||||
## 6. Verified against discovery
|
||||
|
||||
`python3 -I src/google/optimize/tools/check_discovery.py`, run 2026-10-24:
|
||||
`OK: 38 calls and 64 schemas match discovery`. It fetches
|
||||
`python3 -I src/google/optimize/tools/check_discovery.py`, run 2026-10-24
|
||||
(gap-closing round): `OK: 46 calls and 64 schemas match discovery`. It also
|
||||
enforces coverage: every aiplatform v1 `evaluat*`, `generate*` and
|
||||
`onlineEvaluators*` method, every Cloud Trace v1 and v2 method and every
|
||||
`logging.entries*` method must be a table row or be listed in `EXCLUDED`
|
||||
(only `logging.entries.tail`, endless stream). It fetches
|
||||
`https://{aiplatform,cloudtrace,logging}.googleapis.com/$discovery/rest`
|
||||
(aiplatform v1, cloudtrace v1 and v2, logging v2) and fails on any declared
|
||||
call absent or different in verb, path, request fields or query parameters,
|
||||
@@ -141,7 +157,21 @@ One run of the OAuth live suite showed 1 failed test of 5 (name not captured);
|
||||
the immediate rerun and a later run passed 5 of 5. Treat as an unexplained
|
||||
transient until reproduced.
|
||||
|
||||
Not run live (would create or change resources): create/patch/delete/import
|
||||
Gap-closing round, 2026-10-24, service-account token and express key,
|
||||
read-only (`optimize_live_gaps.*`, `optimize_live.*`):
|
||||
|
||||
| Call | Result |
|
||||
|---|---|
|
||||
| `GET onlineEvaluators/1/operations?pageSize=1` (absent evaluator) | 200, no operations |
|
||||
| `GET onlineEvaluators/probe/operations` | 400 INVALID_ARGUMENT: the LRO layer wants a numeric evaluator id ("must be a Long") |
|
||||
| `make_gemini_model` (`models::VertexModel`) `generate`, OAuth token, project path, location global | 200, a reply |
|
||||
| `Autorater` through `make_gemini_model` with the express key (no token) | verdicts parsed, score 1 |
|
||||
| `Autorater` through `make_gemini_model` with the token | verdicts parsed, score 1 |
|
||||
| `evaluate_instances` exactMatch / trajectory (key), five resource lists (token) | unchanged from above: 200 |
|
||||
|
||||
Not run live (would create or change resources): `generateSyntheticData`, the
|
||||
online evaluator operation cancel, delete and wait, `patchTraces`, `batchWrite`,
|
||||
`createSpan`, `entries:write`, `entries:copy`, create/patch/delete/import
|
||||
of sets, items, metrics, runs, online evaluators, `evaluateDataset`,
|
||||
`generateInstanceRubrics`, `generateUserScenarios`, `generateLossClusters`.
|
||||
These are verified by discovery only.
|
||||
@@ -164,12 +194,24 @@ These are verified by discovery only.
|
||||
- The default autorater model for LLM-judged metrics is unavailable to the
|
||||
test project (see 7); `AutoraterOptions::model` is the workaround, untested.
|
||||
- v1beta1 differences are not checked.
|
||||
- `generateSyntheticData`: request and reply shapes are raw Json; the reply type
|
||||
is not typed here and was never seen.
|
||||
- Online evaluator operation names: the id of an evaluator in an operation
|
||||
name must be numeric at Google's LRO layer (400 seen); evaluator ids Google
|
||||
assigns were never observed because none exists.
|
||||
- `entries:copy` destination string format and `entries:write` resource labels
|
||||
are passed through, only top-level fields are checked.
|
||||
- The shared adapter is Vertex only: a non Vertex API key (Generative Language
|
||||
`AIza` key) is not routed to the Generative Language endpoint any more.
|
||||
|
||||
## 9. Gaps
|
||||
|
||||
- Cloud Trace v2 `batchWrite` and `createSpan` and Cloud Logging
|
||||
`entries.write/tail` are not wrapped: agents write through OTel exporters.
|
||||
The v2 rows are not in the table because nothing calls them.
|
||||
- `logging.entries.tail` (endless server stream) is not wrapped; the write
|
||||
calls (`TraceWriter`, `LogWriter`) exist for tools, agents normally write
|
||||
through OTel exporters.
|
||||
- Trace and log lists (`TraceReader::list`, `LogReader::entries`) still stop at
|
||||
their caps without a `truncated` flag; only the new operations list reports
|
||||
truncation. Owner: `observability.cpp`.
|
||||
- Dashboards: only numbers (`DashboardData`); no Cloud Monitoring reads.
|
||||
- `evaluateDataset`, `generateInstanceRubrics`, `evaluationMetrics` and item
|
||||
bodies are passed as raw `Json` (checked at the top level against
|
||||
|
||||
@@ -38,6 +38,17 @@ inline constexpr const char* kEvaluateDataset = "aiplatform.projects.locations.e
|
||||
inline constexpr const char* kGenerateInstanceRubrics = "aiplatform.projects.locations.generateInstanceRubrics";
|
||||
inline constexpr const char* kGenerateUserScenarios = "aiplatform.projects.locations.generateUserScenarios";
|
||||
inline constexpr const char* kGenerateLossClusters = "aiplatform.projects.locations.generateLossClusters";
|
||||
inline constexpr const char* kGenerateSyntheticData = "aiplatform.projects.locations.generateSyntheticData";
|
||||
inline constexpr const char* kOnlineEvaluatorOpGet = "aiplatform.projects.locations.onlineEvaluators.operations.get";
|
||||
inline constexpr const char* kOnlineEvaluatorOpList = "aiplatform.projects.locations.onlineEvaluators.operations.list";
|
||||
inline constexpr const char* kOnlineEvaluatorOpCancel = "aiplatform.projects.locations.onlineEvaluators.operations.cancel";
|
||||
inline constexpr const char* kOnlineEvaluatorOpDelete = "aiplatform.projects.locations.onlineEvaluators.operations.delete";
|
||||
inline constexpr const char* kOnlineEvaluatorOpWait = "aiplatform.projects.locations.onlineEvaluators.operations.wait";
|
||||
inline constexpr const char* kTracePatch = "cloudtrace.projects.patchTraces";
|
||||
inline constexpr const char* kTraceBatchWrite = "cloudtrace.projects.traces.batchWrite";
|
||||
inline constexpr const char* kTraceCreateSpan = "cloudtrace.projects.traces.spans.createSpan";
|
||||
inline constexpr const char* kLogEntriesWrite = "logging.entries.write";
|
||||
inline constexpr const char* kLogEntriesCopy = "logging.entries.copy";
|
||||
inline constexpr const char* kOperationGet = "aiplatform.projects.locations.operations.get";
|
||||
inline constexpr const char* kTraceGet = "cloudtrace.projects.traces.get";
|
||||
inline constexpr const char* kTraceList = "cloudtrace.projects.traces.list";
|
||||
|
||||
@@ -115,3 +115,27 @@
|
||||
{"logging.entries.list", "POST", "/entries:list", "logging", "v2",
|
||||
"filter,orderBy,pageSize,pageToken,projectIds,resourceNames",
|
||||
""},
|
||||
{"aiplatform.projects.locations.generateSyntheticData", "POST", "/{+location}:generateSyntheticData", "aiplatform", "v1",
|
||||
"count,examples,outputFieldSpecs,taskDescription",
|
||||
""},
|
||||
{"aiplatform.projects.locations.onlineEvaluators.operations.cancel", "POST", "/{+name}:cancel", "aiplatform", "v1",
|
||||
"",
|
||||
""},
|
||||
{"aiplatform.projects.locations.onlineEvaluators.operations.delete", "DELETE", "/{+name}", "aiplatform", "v1",
|
||||
"",
|
||||
""},
|
||||
{"aiplatform.projects.locations.onlineEvaluators.operations.get", "GET", "/{+name}", "aiplatform", "v1",
|
||||
"",
|
||||
""},
|
||||
{"aiplatform.projects.locations.onlineEvaluators.operations.list", "GET", "/{+name}/operations", "aiplatform", "v1",
|
||||
"",
|
||||
"filter,pageSize,pageToken,returnPartialSuccess"},
|
||||
{"aiplatform.projects.locations.onlineEvaluators.operations.wait", "POST", "/{+name}:wait", "aiplatform", "v1",
|
||||
"",
|
||||
"timeout"},
|
||||
{"logging.entries.write", "POST", "/entries:write", "logging", "v2",
|
||||
"dryRun,entries,labels,logName,partialSuccess,resource",
|
||||
""},
|
||||
{"logging.entries.copy", "POST", "/entries:copy", "logging", "v2",
|
||||
"destination,filter,name",
|
||||
""},
|
||||
|
||||
@@ -36,6 +36,11 @@ struct OptimizeCaps {
|
||||
static constexpr std::size_t kMaxLogPageSize = 100;
|
||||
static constexpr std::size_t kMaxResourceNames = 100; // Cloud Logging limit
|
||||
static constexpr std::size_t kMaxFilterChars = 20000; // Cloud Logging limit
|
||||
// Write side (telemetry, operations).
|
||||
static constexpr std::size_t kMaxWriteEntries = 1000; // log entries per write
|
||||
static constexpr std::size_t kMaxWriteBytes = 1024 * 1024; // serialized write body
|
||||
static constexpr std::size_t kMaxSpansPerWrite = 2000;
|
||||
static constexpr int kMaxServerWaitSeconds = 60; // one POST :wait
|
||||
// Clustering and optimization.
|
||||
static constexpr std::size_t kMaxFailures = 200;
|
||||
static constexpr std::size_t kMaxClusters = 10;
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
#include "google/optimize/direct_model.h"
|
||||
|
||||
#include "google/credentials.h"
|
||||
#include "google/endpoints.h"
|
||||
#include "google/errors.h"
|
||||
#include "google/generative.h"
|
||||
#include "google/resource.h"
|
||||
#include "google/voice_backend.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
DirectModel::DirectModel(std::shared_ptr<GoogleClient> client, std::string model,
|
||||
ServiceEndpoint endpoint)
|
||||
: Service(std::move(client), std::move(endpoint)),
|
||||
model_(std::move(model)) {}
|
||||
|
||||
std::shared_ptr<DirectModel> DirectModel::from_env(std::shared_ptr<GoogleClient> client,
|
||||
std::string model) {
|
||||
// OAuth tokens need the project-scoped Vertex path (express paths refuse them).
|
||||
CloudContext ctx = CloudContext::from_env();
|
||||
if (!Credentials::env("GOOGLE_ACCESS_TOKEN").empty() && ctx.valid()) {
|
||||
auto m = std::make_shared<DirectModel>(std::move(client), std::move(model),
|
||||
endpoints::aiplatform("global"));
|
||||
m->path_prefix_ = "/projects/" + ctx.project + "/locations/global/publishers/google/models/";
|
||||
return m;
|
||||
}
|
||||
ServiceEndpoint ep = voice_backend_from_env() == VoiceBackend::kVertex
|
||||
? endpoints::vertex_generative()
|
||||
: endpoints::generative();
|
||||
return std::make_shared<DirectModel>(std::move(client), std::move(model), std::move(ep));
|
||||
}
|
||||
|
||||
GenerateResponse DirectModel::generate(const GenerateRequest& request) {
|
||||
std::string model = !request.model.empty() ? request.model
|
||||
: !model_.empty() ? model_
|
||||
: Generative::default_model();
|
||||
if (!valid_segment(model)) throw GoogleError::invalid("bad model name");
|
||||
return GenerateResponse::from_json(
|
||||
call(path_prefix_ + model + ":generateContent", request.to_json()));
|
||||
}
|
||||
|
||||
void DirectModel::stream(const GenerateRequest& request,
|
||||
const std::function<bool(const GenerateResponse&)>& on_chunk) {
|
||||
on_chunk(generate(request));
|
||||
}
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -1,38 +0,0 @@
|
||||
#pragma once
|
||||
|
||||
// DirectModel: a LanguageModel over GoogleClient (Gemini generateContent on
|
||||
// the Generative Language endpoint). Private to this module so optimize
|
||||
// needs no other agent-platform module; any LanguageModel can replace it.
|
||||
// Streaming is one chunk (the full reply).
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "google/gemini_types.h"
|
||||
#include "google/service.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
class DirectModel : public Service, public LanguageModel {
|
||||
public:
|
||||
// Empty model: GOOGLE_GENAI_MODEL or the built-in default.
|
||||
explicit DirectModel(std::shared_ptr<GoogleClient> client, std::string model = "",
|
||||
ServiceEndpoint endpoint = endpoints::generative());
|
||||
|
||||
// Endpoint by credential: an OAuth token (GOOGLE_ACCESS_TOKEN) with a valid
|
||||
// GOOGLE_CLOUD_PROJECT uses the project-scoped Vertex path; Vertex express
|
||||
// keys ("AQ.", or GOOGLE_API_BACKEND=vertex) the Vertex publisher endpoint;
|
||||
// everything else the Generative Language endpoint.
|
||||
static std::shared_ptr<DirectModel> from_env(std::shared_ptr<GoogleClient> client,
|
||||
std::string model = "");
|
||||
|
||||
GenerateResponse generate(const GenerateRequest& request) override;
|
||||
void stream(const GenerateRequest& request,
|
||||
const std::function<bool(const GenerateResponse&)>& on_chunk) override;
|
||||
|
||||
private:
|
||||
std::string model_;
|
||||
std::string path_prefix_ = "/models/";
|
||||
};
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -65,6 +65,15 @@ Json EvaluationApi::generate_instance_rubrics(const Json& request) const {
|
||||
&request);
|
||||
}
|
||||
|
||||
Json EvaluationApi::generate_synthetic_data(const Json& request) const {
|
||||
if (!request.is_object() || request.as_object().empty())
|
||||
throw GoogleError::invalid("synthetic data request is empty");
|
||||
if (request.dump().size() > OptimizeCaps::kMaxWriteBytes)
|
||||
throw GoogleError::invalid("synthetic data request is over the size cap");
|
||||
return invoke(call_spec(rpc::kGenerateSyntheticData), {{"location", context_.parent()}}, {},
|
||||
&request);
|
||||
}
|
||||
|
||||
Json EvaluationApi::scenario_body(const ScenarioRequest& r) {
|
||||
if (r.count < 1 || r.count > static_cast<int>(OptimizeCaps::kMaxScenarios))
|
||||
throw GoogleError::invalid("count must be 1.." + std::to_string(OptimizeCaps::kMaxScenarios));
|
||||
|
||||
@@ -48,6 +48,9 @@ class EvaluationApi : public OptimizeService {
|
||||
bool wait = false, const Sleeper& sleeper = nullptr) const;
|
||||
|
||||
clients::body::Json generate_instance_rubrics(const clients::body::Json& request) const;
|
||||
// generateSyntheticData: raw request (top-level fields checked against the
|
||||
// table); the reply is returned as Json, its shape is not typed (Unverified).
|
||||
clients::body::Json generate_synthetic_data(const clients::body::Json& request) const;
|
||||
std::vector<Scenario> generate_user_scenarios(const ScenarioRequest& request) const;
|
||||
|
||||
// Body of generateUserScenarios, exposed so it can be checked offline.
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
#include "google/optimize/model_factory.h"
|
||||
|
||||
#include "google/credentials.h"
|
||||
#include "google/generative.h"
|
||||
#include "google/models/gemini.h"
|
||||
#include "google/models/target.h"
|
||||
#include "google/models/vertex_model.h"
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
std::shared_ptr<LanguageModel> make_gemini_model(std::shared_ptr<GoogleClient> client,
|
||||
std::string model) {
|
||||
if (model.empty()) model = Generative::default_model();
|
||||
CloudContext ctx = CloudContext::from_env();
|
||||
models::VertexTarget target = models::VertexTarget::express();
|
||||
if (!Credentials::env("GOOGLE_ACCESS_TOKEN").empty() && ctx.valid()) {
|
||||
ctx.location = "global";
|
||||
target = models::VertexTarget::project(std::move(ctx));
|
||||
}
|
||||
auto gemini = std::make_shared<models::Gemini>(std::move(client), std::move(target));
|
||||
return std::make_shared<models::VertexModel>(std::move(gemini), std::move(model));
|
||||
}
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
// The one way this module gets a Gemini LanguageModel: models::VertexModel
|
||||
// from src/google/models (the single Gemini adapter of the platform). The
|
||||
// scoring and simulation classes still code against LanguageModel only; this
|
||||
// factory just picks the Vertex target from the credentials.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "google/client.h"
|
||||
#include "google/gemini_types.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
// Target by credential:
|
||||
// - OAuth token (GOOGLE_ACCESS_TOKEN) with a valid GOOGLE_CLOUD_PROJECT: the
|
||||
// project scoped Vertex path in location "global" (what the previous
|
||||
// private adapter used).
|
||||
// - otherwise Vertex express mode (API key, publisher path).
|
||||
// Empty `model`: Generative::default_model() (GOOGLE_GENAI_MODEL or built in).
|
||||
// Nothing is sent here; credential problems surface on the first generate().
|
||||
std::shared_ptr<LanguageModel> make_gemini_model(std::shared_ptr<GoogleClient> client,
|
||||
std::string model = "");
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -0,0 +1,95 @@
|
||||
#include "google/optimize/operations_api.h"
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "google/errors.h"
|
||||
#include "google/optimize/caps.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
OnlineEvaluatorOperations::OnlineEvaluatorOperations(std::shared_ptr<GoogleClient> client,
|
||||
CloudContext context)
|
||||
: OptimizeService(std::move(client), endpoints::aiplatform(context.location)) {
|
||||
(void)context;
|
||||
}
|
||||
|
||||
bool OnlineEvaluatorOperations::is_operation_name(const std::string& name) {
|
||||
// projects/p/locations/l/onlineEvaluators/id/operations/op
|
||||
if (!is_resource_of(name.substr(0, name.rfind("/operations/")), "onlineEvaluators")) return false;
|
||||
std::size_t at = name.rfind("/operations/");
|
||||
return at != std::string::npos && valid_segment(name.substr(at + 12));
|
||||
}
|
||||
|
||||
namespace {
|
||||
void require_operation(const std::string& name) {
|
||||
if (!OnlineEvaluatorOperations::is_operation_name(name))
|
||||
throw GoogleError::invalid(
|
||||
"operation must be projects/p/locations/l/onlineEvaluators/id/operations/op");
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json OnlineEvaluatorOperations::get(const std::string& name) const {
|
||||
require_operation(name);
|
||||
return invoke(call_spec(rpc::kOnlineEvaluatorOpGet), {{"name", name}});
|
||||
}
|
||||
|
||||
OperationPage OnlineEvaluatorOperations::list(const std::string& evaluator_name,
|
||||
const std::string& filter, int page_size) const {
|
||||
if (!is_resource_of(evaluator_name, "onlineEvaluators"))
|
||||
throw GoogleError::invalid("evaluator must be projects/p/locations/l/onlineEvaluators/id");
|
||||
if (filter.size() > 1000) throw GoogleError::invalid("filter over 1000 characters");
|
||||
if (page_size < 0 || page_size > 100) throw GoogleError::invalid("page_size must be 0..100");
|
||||
const CallSpec& spec = call_spec(rpc::kOnlineEvaluatorOpList);
|
||||
OperationPage out;
|
||||
std::string token;
|
||||
for (std::size_t page = 0;; ++page) {
|
||||
Json reply = invoke(spec, {{"name", evaluator_name}},
|
||||
{{"filter", filter},
|
||||
{"pageSize", page_size > 0 ? std::to_string(page_size) : ""},
|
||||
{"pageToken", token}});
|
||||
if (const Json* arr = reply.is_object() ? reply.find("operations") : nullptr;
|
||||
arr && arr->is_array())
|
||||
for (const Json& o : arr->as_array()) {
|
||||
if (out.operations.size() >= OptimizeCaps::kMaxListItems) {
|
||||
out.truncated = true;
|
||||
return out;
|
||||
}
|
||||
out.operations.push_back(Operation::from_json(o));
|
||||
}
|
||||
const Json* next = reply.is_object() ? reply.find("nextPageToken") : nullptr;
|
||||
token = next && next->is_string() ? next->as_string() : "";
|
||||
if (token.empty()) return out;
|
||||
if (page + 1 >= OptimizeCaps::kMaxPages) {
|
||||
out.truncated = true;
|
||||
out.next_page_token = token;
|
||||
return out;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void OnlineEvaluatorOperations::cancel(const std::string& name) const {
|
||||
require_operation(name);
|
||||
invoke(call_spec(rpc::kOnlineEvaluatorOpCancel), {{"name", name}});
|
||||
}
|
||||
|
||||
void OnlineEvaluatorOperations::remove(const std::string& name) const {
|
||||
require_operation(name);
|
||||
invoke(call_spec(rpc::kOnlineEvaluatorOpDelete), {{"name", name}});
|
||||
}
|
||||
|
||||
Operation OnlineEvaluatorOperations::wait_server(const std::string& name,
|
||||
int timeout_seconds) const {
|
||||
require_operation(name);
|
||||
if (timeout_seconds < 1 || timeout_seconds > OptimizeCaps::kMaxServerWaitSeconds)
|
||||
throw GoogleError::invalid("timeout_seconds must be 1.." +
|
||||
std::to_string(OptimizeCaps::kMaxServerWaitSeconds));
|
||||
Operation op = Operation::from_json(invoke(call_spec(rpc::kOnlineEvaluatorOpWait),
|
||||
{{"name", name}},
|
||||
{{"timeout", std::to_string(timeout_seconds) + "s"}}));
|
||||
if (op.done && op.failed())
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, endpoint_.name);
|
||||
return op;
|
||||
}
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -0,0 +1,42 @@
|
||||
#pragma once
|
||||
|
||||
// Operations of online evaluators (aiplatform v1
|
||||
// projects.locations.onlineEvaluators.operations: get, list, cancel, delete,
|
||||
// wait). An online evaluator's activate, suspend and create calls return
|
||||
// long-running operations that live under the evaluator.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/optimize/optimize_service.h"
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
struct OperationPage {
|
||||
std::vector<Operation> operations;
|
||||
bool truncated = false; // More existed past the page/item caps.
|
||||
std::string next_page_token; // Continues a truncated list.
|
||||
};
|
||||
|
||||
class OnlineEvaluatorOperations : public OptimizeService {
|
||||
public:
|
||||
OnlineEvaluatorOperations(std::shared_ptr<GoogleClient> client, CloudContext context);
|
||||
|
||||
// Names: projects/{p}/locations/{l}/onlineEvaluators/{id}/operations/{op}.
|
||||
static bool is_operation_name(const std::string& name);
|
||||
|
||||
clients::body::Json get(const std::string& operation_name) const;
|
||||
// Operations under projects/{p}/locations/{l}/onlineEvaluators/{id}.
|
||||
OperationPage list(const std::string& evaluator_name, const std::string& filter = "",
|
||||
int page_size = 0) const;
|
||||
void cancel(const std::string& operation_name) const;
|
||||
// Destructive: removes the operation record only.
|
||||
void remove(const std::string& operation_name) const;
|
||||
// One server-side wait, 1..OptimizeCaps::kMaxServerWaitSeconds; the reply
|
||||
// may be not done. A failed operation throws.
|
||||
Operation wait_server(const std::string& operation_name, int timeout_seconds) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -6,18 +6,18 @@ namespace molo::google::optimize {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
Json OptimizeService::invoke(const CallSpec& spec,
|
||||
const std::map<std::string, std::string>& path_params,
|
||||
const std::map<std::string, std::string>& query,
|
||||
const Json* body) const {
|
||||
Json OptimizeService::invoke_on(const ServiceEndpoint& endpoint, const CallSpec& spec,
|
||||
const std::map<std::string, std::string>& path_params,
|
||||
const std::map<std::string, std::string>& query,
|
||||
const Json* body) const {
|
||||
std::string path = with_query(spec, expand_path(spec, path_params), query);
|
||||
if (body) check_body(spec, *body);
|
||||
else if (std::string(spec.http) == "POST") {
|
||||
// A POST without fields (cancel, activate, suspend) sends {}.
|
||||
Json empty = Json::object();
|
||||
return client_->request(spec.method(), endpoint_, path, &empty);
|
||||
return client_->request(spec.method(), endpoint, path, &empty);
|
||||
}
|
||||
return client_->request(spec.method(), endpoint_, path, body);
|
||||
return client_->request(spec.method(), endpoint, path, body);
|
||||
}
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
|
||||
@@ -22,7 +22,14 @@ class OptimizeService : public Service {
|
||||
clients::body::Json invoke(const CallSpec& spec,
|
||||
const std::map<std::string, std::string>& path_params,
|
||||
const std::map<std::string, std::string>& query = {},
|
||||
const clients::body::Json* body = nullptr) const;
|
||||
const clients::body::Json* body = nullptr) const {
|
||||
return invoke_on(endpoint_, spec, path_params, query, body);
|
||||
}
|
||||
// Same, on another endpoint (a service with calls in two versions).
|
||||
clients::body::Json invoke_on(const ServiceEndpoint& endpoint, const CallSpec& spec,
|
||||
const std::map<std::string, std::string>& path_params,
|
||||
const std::map<std::string, std::string>& query = {},
|
||||
const clients::body::Json* body = nullptr) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
#include "google/optimize/telemetry_writer.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/optimize/caps.h"
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
void require_project(const std::string& project) {
|
||||
CloudContext c;
|
||||
c.project = project;
|
||||
if (!c.valid()) throw GoogleError::invalid("bad project id");
|
||||
}
|
||||
|
||||
void require_object(const Json& body, const char* what) {
|
||||
if (!body.is_object() || body.as_object().empty())
|
||||
throw GoogleError::invalid(std::string(what) + " must be a non-empty JSON object");
|
||||
if (body.dump().size() > OptimizeCaps::kMaxWriteBytes)
|
||||
throw GoogleError::invalid(std::string(what) + " is over " +
|
||||
std::to_string(OptimizeCaps::kMaxWriteBytes) + " bytes");
|
||||
}
|
||||
|
||||
std::size_t array_size(const Json& body, const char* key) {
|
||||
const Json* arr = body.is_object() ? body.find(key) : nullptr;
|
||||
return arr && arr->is_array() ? arr->as_array().size() : 0;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
TraceWriter::TraceWriter(std::shared_ptr<GoogleClient> client)
|
||||
: OptimizeService(std::move(client),
|
||||
endpoints::custom("cloudtrace", "https://cloudtrace.googleapis.com/v1")),
|
||||
v2_(endpoints::custom("cloudtrace-v2", "https://cloudtrace.googleapis.com/v2")) {}
|
||||
|
||||
void TraceWriter::patch_traces(const std::string& project, const Json& traces) const {
|
||||
require_project(project);
|
||||
require_object(traces, "traces request");
|
||||
const Json* inner = traces.find("traces");
|
||||
std::size_t n = inner && inner->is_object() ? array_size(*inner, "traces") : 0;
|
||||
if (n < 1 || n > OptimizeCaps::kMaxTraces)
|
||||
throw GoogleError::invalid("traces.traces needs 1.." +
|
||||
std::to_string(OptimizeCaps::kMaxTraces) + " traces");
|
||||
invoke(call_spec(rpc::kTracePatch), {{"projectId", project}}, {}, &traces);
|
||||
}
|
||||
|
||||
void TraceWriter::batch_write(const std::string& project, const Json& spans) const {
|
||||
require_project(project);
|
||||
require_object(spans, "spans request");
|
||||
std::size_t n = array_size(spans, "spans");
|
||||
if (n < 1 || n > OptimizeCaps::kMaxSpansPerWrite)
|
||||
throw GoogleError::invalid("spans needs 1.." + std::to_string(OptimizeCaps::kMaxSpansPerWrite) +
|
||||
" entries");
|
||||
invoke_on(v2_, call_spec(rpc::kTraceBatchWrite), {{"name", "projects/" + project}}, {}, &spans);
|
||||
}
|
||||
|
||||
Json TraceWriter::create_span(const std::string& name, const Json& span) const {
|
||||
// projects/{p}/traces/{trace}/spans/{span}
|
||||
std::size_t t = name.find("/traces/"), s = name.find("/spans/");
|
||||
if (name.rfind("projects/", 0) != 0 || t == std::string::npos || s == std::string::npos || s < t)
|
||||
throw GoogleError::invalid("span name must be projects/{p}/traces/{trace}/spans/{span}");
|
||||
require_project(name.substr(9, t - 9));
|
||||
if (!valid_segment(name.substr(t + 8, s - t - 8)) || !valid_segment(name.substr(s + 7)))
|
||||
throw GoogleError::invalid("span name has an invalid trace or span id");
|
||||
require_object(span, "span");
|
||||
const Json* own = span.find("name");
|
||||
if (!own || !own->is_string() || own->as_string() != name)
|
||||
throw GoogleError::invalid("span.name must equal the span name in the path");
|
||||
return invoke_on(v2_, call_spec(rpc::kTraceCreateSpan), {{"name", name}}, {}, &span);
|
||||
}
|
||||
|
||||
LogWriter::LogWriter(std::shared_ptr<GoogleClient> client)
|
||||
: OptimizeService(std::move(client),
|
||||
endpoints::custom("logging", "https://logging.googleapis.com/v2")) {}
|
||||
|
||||
Json LogWriter::write(const Json& request) const {
|
||||
require_object(request, "log write request");
|
||||
std::size_t n = array_size(request, "entries");
|
||||
if (n < 1 || n > OptimizeCaps::kMaxWriteEntries)
|
||||
throw GoogleError::invalid("entries needs 1.." + std::to_string(OptimizeCaps::kMaxWriteEntries) +
|
||||
" items");
|
||||
return invoke(call_spec(rpc::kLogEntriesWrite), {}, {}, &request);
|
||||
}
|
||||
|
||||
Operation LogWriter::copy(const Json& request) const {
|
||||
require_object(request, "log copy request");
|
||||
for (const char* k : {"name", "destination"}) {
|
||||
const Json* v = request.find(k);
|
||||
if (!v || !v->is_string() || v->as_string().empty() || v->as_string().size() > 1024)
|
||||
throw GoogleError::invalid(std::string("copy needs a string ") + k + " (max 1024)");
|
||||
}
|
||||
if (const Json* f = request.find("filter"); f && f->is_string() && f->as_string().size() > OptimizeCaps::kMaxFilterChars)
|
||||
throw GoogleError::invalid("filter over the character cap");
|
||||
return Operation::from_json(invoke(call_spec(rpc::kLogEntriesCopy), {}, {}, &request));
|
||||
}
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -0,0 +1,53 @@
|
||||
#pragma once
|
||||
|
||||
// Write side of Agent Observability, for tests and tools that must push a
|
||||
// span or a log entry themselves (agents normally write through their OTel
|
||||
// exporter). Every method mutates Google state and is explicit; bodies are
|
||||
// raw Json whose top-level fields are checked against the call table and
|
||||
// whose size is capped.
|
||||
//
|
||||
// - TraceWriter: Cloud Trace v1 patchTraces; v2 batchWrite and createSpan.
|
||||
// - LogWriter: Cloud Logging v2 entries.write (dryRun available) and
|
||||
// entries.copy (long-running).
|
||||
// entries.tail is not wrapped: it is an endless server stream (see SPEC).
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/optimize/optimize_service.h"
|
||||
|
||||
namespace molo::google::optimize {
|
||||
|
||||
class TraceWriter : public OptimizeService {
|
||||
public:
|
||||
explicit TraceWriter(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
// PATCH v1 projects/{project}/traces. `traces` is {"traces":[Trace,...]}
|
||||
// (PatchTracesRequest.traces), at most OptimizeCaps::kMaxTraces traces.
|
||||
void patch_traces(const std::string& project, const clients::body::Json& traces) const;
|
||||
// POST v2 projects/{project}/traces:batchWrite with {"spans":[Span,...]},
|
||||
// 1..OptimizeCaps::kMaxSpansPerWrite spans.
|
||||
void batch_write(const std::string& project, const clients::body::Json& spans) const;
|
||||
// POST v2 {name}: name is projects/{p}/traces/{traceId}/spans/{spanId};
|
||||
// `span` is the Span body (its `name` must equal `name`).
|
||||
clients::body::Json create_span(const std::string& name, const clients::body::Json& span) const;
|
||||
|
||||
private:
|
||||
ServiceEndpoint v2_;
|
||||
};
|
||||
|
||||
class LogWriter : public OptimizeService {
|
||||
public:
|
||||
explicit LogWriter(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
// entries:write. `request` carries logName, resource, labels, entries,
|
||||
// partialSuccess, dryRun. 1..kMaxWriteEntries entries. Prefer dryRun=true
|
||||
// first: nothing is stored.
|
||||
clients::body::Json write(const clients::body::Json& request) const;
|
||||
// entries:copy: copies matching entries to a Cloud Storage destination.
|
||||
// `request` carries name (source bucket view), filter, destination.
|
||||
// Returns the long-running operation (not waited).
|
||||
Operation copy(const clients::body::Json& request) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::optimize
|
||||
@@ -111,6 +111,21 @@ ENUM_VALUES = [
|
||||
]
|
||||
|
||||
|
||||
# Coverage: every Discovery method with one of these id prefixes must be a
|
||||
# table row or be listed in EXCLUDED with its reason.
|
||||
COVERED = [
|
||||
('aiplatform', 'v1', ['aiplatform.projects.locations.evaluat',
|
||||
'aiplatform.projects.locations.generate',
|
||||
'aiplatform.projects.locations.onlineEvaluators']),
|
||||
('cloudtrace', 'v1', ['']),
|
||||
('cloudtrace', 'v2', ['']),
|
||||
('logging', 'v2', ['logging.entries']),
|
||||
]
|
||||
EXCLUDED = {
|
||||
'logging.entries.tail': 'server-streaming RPC with no end; this module has no unbounded stream (hard caps rule)',
|
||||
}
|
||||
|
||||
|
||||
def fetch(url):
|
||||
with urllib.request.urlopen(url, timeout=60) as r:
|
||||
return json.load(r)
|
||||
@@ -156,6 +171,12 @@ def main():
|
||||
want_query = sorted(k for k, v in m.get('parameters', {}).items() if v['location'] == 'query')
|
||||
if sorted(x for x in query_csv.split(',') if x) != want_query:
|
||||
errors.append('%s: query params differ from discovery' % rpc_id)
|
||||
declared = {(r[3], r[4], r[0]) for r in rows}
|
||||
for svc, ver, prefixes in COVERED:
|
||||
for m in methods(docs[(svc, ver)]):
|
||||
if any(m['id'].startswith(p) for p in prefixes) and (svc, ver, m['id']) not in declared \
|
||||
and m['id'] not in EXCLUDED:
|
||||
errors.append('discovery method %s is not in the table' % m['id'])
|
||||
for svc, ver, schema, fields in SCHEMA_FIELDS:
|
||||
s = docs[(svc, ver)]['schemas'].get(schema)
|
||||
if s is None:
|
||||
|
||||
@@ -14,8 +14,8 @@ Foundation used: `GoogleClient`, `Service`, `GoogleError`, `CloudContext`,
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings |
|
||||
| `ids.h/.cpp` | `Ids`: validation of ids, asset names, template names, request ids, free text |
|
||||
| `calls.h/.cpp` | `Calls` and THE call table (86 rows): method, path, discovery service/version, rpc id, request schema, query params, request fields |
|
||||
| `call_runner.h/.cpp` | `CallRunner`: renders and validates a table call, paging, mutation + operation wait, `update_mask` |
|
||||
| `calls.h/.cpp` | `Calls` and THE call table (128 rows): method, path, discovery service/version, rpc id, request schema, query params, request fields |
|
||||
| `call_runner.h/.cpp` | `CallRunner`: renders and validates a table call, paging (`ListPager`, `ListResult`), mutation + operation wait, `update_mask` |
|
||||
| `operation_runner.h/.cpp` | `OperationRunner`: finish a long-running operation, 404 existence probe |
|
||||
| `registry_service.h/.cpp` | `RegistryService : Service`: base of the four services |
|
||||
| `service_record.h/.cpp` | `ServiceRecord`, `Interface`: body of `services.create/patch` |
|
||||
@@ -34,7 +34,7 @@ Foundation used: `GoogleClient`, `Service`, `GoogleError`, `CloudContext`,
|
||||
|
||||
| Knob | Value |
|
||||
|---|---|
|
||||
| List page size / pages / items | 100 / 10 / 1000 per list call |
|
||||
| List page size / pages / items | 100 / 10 / 1000 per list call; past a cap the result says `truncated` and carries the next page token (see 3) |
|
||||
| displayName / skill displayName / description | 63 / 128 / 2048 (Google's limits) |
|
||||
| Spec content | 10 KiB (Google's limit) |
|
||||
| Service id | 4..63 of `[a-z0-9-]`, starts with a letter |
|
||||
@@ -57,6 +57,17 @@ Foundation used: `GoogleClient`, `Service`, `GoogleError`, `CloudContext`,
|
||||
call, `pageSize` outside 1..100, a zero or malformed `requestId`, a body on
|
||||
a bodiless call, a missing body, or a top-level body field not declared for
|
||||
the call.
|
||||
- Lists never truncate silently. Every list method returns `ListResult`
|
||||
(`items`, `truncated`, `next_page_token`, `pages`, `to_json()`); `CallRunner`
|
||||
pages with `ListPager` up to 10 pages / 1000 items and sets `truncated` (and
|
||||
the continuation token) when the server still had more. A caller-supplied
|
||||
`pageToken` is refused: the runner owns paging. `list_approved` keeps the
|
||||
flag. The demo routes put `count`, `pages`, `truncated`, `next_page_token`
|
||||
and `cap_items` in every list answer and the page shows a TRUNCATED banner.
|
||||
- Two API versions. Discovery lists every registry resource in v1 and in
|
||||
v1alpha (skills, revisions and publishers in v1alpha only). Both are in the
|
||||
call table (`agentregistry.v1.*` and `agentregistry.v1alpha.*`);
|
||||
`RegistryVersion` picks the rows for `AgentRegistry`, v1 by default.
|
||||
- Read before mutate: `register_service`, `create_skill`, `create_binding`,
|
||||
`create` (gateway), `create_policy`, `create_deny_policy`, `create_boundary`
|
||||
GET first and answer 409 `ALREADY_EXISTS` for an existing id. Update and
|
||||
@@ -97,9 +108,11 @@ Foundation used: `GoogleClient`, `Service`, `GoogleError`, `CloudContext`,
|
||||
GET (no credential needed): `/agent/registry` page, `/status`, `/calls`,
|
||||
`/approvals`, `/audit`. POST: `/list`, `/get`, `/search`, `/approve`,
|
||||
`/deprecate`, `/reject`, `/skills/state`, `/gateways/list`, `/gateways/get`,
|
||||
`/policies/list`, `/identity`, `/identity/audit` (503 `google_not_configured`
|
||||
without a credential, 422 on bad input) and `/identity/principal` (local, no
|
||||
credential). DELETE: `/services/{id}`, `/gateways/{id}`. `/approve` and
|
||||
`/policies/list`, `/identity`, `/identity/audit`, `/operations/list`, `/operations/cancel`, `/deny-policies/list`,
|
||||
`/boundaries/bindings` (503 `google_not_configured` without a credential, 422
|
||||
on bad input) and `/identity/principal` (local, no credential). `/list` takes
|
||||
`version` (`v1` or `v1alpha`). DELETE: `/services/{id}`, `/gateways/{id}`,
|
||||
`/operations?scope=registry|gateways|policies&id=` (the operation record only). `/approve` and
|
||||
`/deprecate` read the asset from Google first.
|
||||
|
||||
## Sources
|
||||
@@ -119,8 +132,15 @@ credential). DELETE: `/services/{id}`, `/gateways/{id}`. `/approve` and
|
||||
|
||||
## Verified against discovery
|
||||
|
||||
`python3 -I src/google/registry/tools/check_discovery.py`, run 2026-10-24:
|
||||
`checked 86 calls across 8 discovery documents` / `OK`. For every row it
|
||||
`python3 -I src/google/registry/tools/check_discovery.py`, run 2026-10-24
|
||||
(gap-closing round): `checked 128 calls across 8 discovery documents, 0
|
||||
discovery methods unwrapped` / `OK`. Coverage is enforced: every discovery
|
||||
method of agentregistry v1 and v1alpha, networkservices
|
||||
`agentGateways` and `operations`, networksecurity `authzPolicies` and
|
||||
`operations`, iam v2 (deny policies) and iam v3 `principalAccessBoundaryPolicies`
|
||||
must be a table row or be listed in `EXCLUDED` (empty). aiplatform and
|
||||
logging are deliberately narrow subsets (reasoningEngines get/list/IAM, `entries:list`)
|
||||
and are not covered. For every row it
|
||||
confirms: rpc method id exists, HTTP method, path (`version + path`), the
|
||||
exact set of query parameters, the request schema name, and that every
|
||||
declared request field is a property of that schema. Discovery findings that
|
||||
@@ -147,6 +167,15 @@ resource created, changed or deleted. Rerun: `MOLO_LIVE_NET=1` plus
|
||||
| networkservices v1 `agentGateways` list | 401 UNAUTHENTICATED | 403 SERVICE_DISABLED: Network Services API not enabled |
|
||||
| networksecurity v1 `authzPolicies` list | 401 UNAUTHENTICATED | 403 SERVICE_DISABLED: Network Security API not enabled |
|
||||
|
||||
Re-probed 2026-10-24 (gap-closing round, service-account token, read-only):
|
||||
agentregistry v1 `agents` and v1alpha `operations`, networkservices
|
||||
`agentGateways` and `operations`, networksecurity `authzPolicies`, and iam v2
|
||||
`denypolicies` (project number 474586702855) all still answer 403
|
||||
PERMISSION_DENIED "... has not been used in project ... or it is disabled".
|
||||
The new v1alpha, operations and deny policy reads in
|
||||
`registry_live.v1alpha_operations_and_deny_policy_reads` are therefore
|
||||
Discovery-verified only, with a loud opt-in test ready.
|
||||
|
||||
What this proves: hosts, versions and collection paths exist (a wrong path
|
||||
would be 404, not an auth or enablement error) and registry/gateway/policy APIs
|
||||
need OAuth. What is NOT live-verified: any registry, gateway or policy
|
||||
@@ -177,7 +206,9 @@ Neither discovery nor the docs state these; each is a choice made here:
|
||||
beyond the shared schema shape.
|
||||
- The deny-policy permission and principal string checks are stricter than
|
||||
Google's (shape checks only).
|
||||
- 1000-item list ceiling: a larger registry is silently truncated at the cap.
|
||||
- v1alpha rows were generated from the v1alpha discovery document, keeping the v1 field order where a v1 twin exists. Their record builders (`ServiceRecord`, `BindingRecord`) emit v1 shapes; a field that v1alpha lacks fails loudly (`field ... is not declared`), a v1alpha-only field cannot be sent. No v1alpha body was ever sent.
|
||||
- `iam.v2.policies.update` (PUT) needs the policy `etag`: `update_deny_policy` reads it first. Whether IAM accepts an empty etag is unread.
|
||||
- `searchPolicyBindings` (iam v3) is listed under key `policyBindings` from the response schema; no live reply was seen (organization scope, no org access).
|
||||
|
||||
## Gaps
|
||||
|
||||
|
||||
@@ -101,7 +101,7 @@ ServiceEndpoint AgentGateway::default_endpoint() {
|
||||
AgentGateway::AgentGateway(std::shared_ptr<GoogleClient> client, CloudContext context)
|
||||
: RegistryService(std::move(client), default_endpoint(), std::move(context)) {}
|
||||
|
||||
Json AgentGateway::list() const {
|
||||
ListResult AgentGateway::list() const {
|
||||
return runner_.list("networkservices.v1.agentGateways.list", "agentGateways",
|
||||
with("parent", context_.parent()));
|
||||
}
|
||||
@@ -148,4 +148,22 @@ Json AgentGateway::operation(const std::string& operation_id) const {
|
||||
with("name", context_.parent() + "/operations/" + operation_id));
|
||||
}
|
||||
|
||||
ListResult AgentGateway::operations() const {
|
||||
return runner_.list("networkservices.v1.operations.list", "operations",
|
||||
with("name", context_.parent()));
|
||||
}
|
||||
|
||||
Json AgentGateway::cancel_operation(const std::string& operation_id) const {
|
||||
operation(operation_id); // Read first: a missing operation is a clear 404.
|
||||
CallArgs a = with("name", context_.parent() + "/operations/" + operation_id);
|
||||
a.body = Json::object();
|
||||
return runner_.invoke("networkservices.v1.operations.cancel", a);
|
||||
}
|
||||
|
||||
Json AgentGateway::delete_operation(const std::string& operation_id) const {
|
||||
operation(operation_id);
|
||||
return runner_.invoke("networkservices.v1.operations.delete",
|
||||
with("name", context_.parent() + "/operations/" + operation_id));
|
||||
}
|
||||
|
||||
} // namespace molo::google::registry
|
||||
|
||||
@@ -41,7 +41,8 @@ class AgentGateway : public RegistryService {
|
||||
static ServiceEndpoint default_endpoint(); // networkservices .../v1
|
||||
|
||||
// Reads first.
|
||||
clients::body::Json list() const;
|
||||
// Paged up to the registry caps; ListResult::truncated is loud.
|
||||
ListResult list() const;
|
||||
clients::body::Json get(const std::string& id) const;
|
||||
|
||||
// Mutations are explicit and read first: create refuses an existing id,
|
||||
@@ -56,7 +57,13 @@ class AgentGateway : public RegistryService {
|
||||
// `etag` guards against a concurrent change; empty sends none.
|
||||
clients::body::Json remove_gateway(const std::string& id, const std::string& etag = "") const;
|
||||
|
||||
// ---- Operations of the networkservices location (get, list, cancel,
|
||||
// delete). cancel and delete read the operation first.
|
||||
clients::body::Json operation(const std::string& operation_id) const;
|
||||
ListResult operations() const;
|
||||
clients::body::Json cancel_operation(const std::string& operation_id) const;
|
||||
// Destructive: removes the operation record, never the gateway.
|
||||
clients::body::Json delete_operation(const std::string& operation_id) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::registry
|
||||
|
||||
@@ -74,11 +74,13 @@ Json DenyPolicySpec::to_json() const {
|
||||
Ids::require_text(display_name, RegistryCaps::kMaxDisplayName, "displayName");
|
||||
if (rules.empty() || rules.size() > RegistryCaps::kMaxPolicyRules)
|
||||
throw GoogleError::invalid("a deny policy needs 1..5 rules");
|
||||
Ids::require_text(etag, 256, "etag");
|
||||
Json j = Json::object();
|
||||
if (!display_name.empty()) j.set("displayName", Json(display_name));
|
||||
Json r = Json::array();
|
||||
for (const auto& rule : rules) r.push(rule.to_json());
|
||||
j.set("rules", r);
|
||||
if (!etag.empty()) j.set("etag", Json(etag));
|
||||
return j;
|
||||
}
|
||||
|
||||
@@ -113,7 +115,7 @@ ServiceEndpoint AgentIdentity::default_endpoint(const std::string& location) {
|
||||
AgentIdentity::AgentIdentity(std::shared_ptr<GoogleClient> client, CloudContext context)
|
||||
: RegistryService(std::move(client), default_endpoint(context.location), context) {}
|
||||
|
||||
Json AgentIdentity::engines() const {
|
||||
ListResult AgentIdentity::engines() const {
|
||||
return runner_.list("aiplatform.v1.reasoningEngines.list", "reasoningEngines",
|
||||
with("parent", context_.parent()));
|
||||
}
|
||||
@@ -185,7 +187,7 @@ std::string AgentIdentity::boundary_parent(const std::string& org_id) {
|
||||
return "organizations/" + org_id + "/locations/global";
|
||||
}
|
||||
|
||||
Json AgentIdentity::deny_policies(const std::string& attachment) const {
|
||||
ListResult AgentIdentity::deny_policies(const std::string& attachment) const {
|
||||
return runner_.list("iam.v2.policies.listPolicies", "policies",
|
||||
with("parent", policy_parent(attachment)));
|
||||
}
|
||||
@@ -214,7 +216,7 @@ Json AgentIdentity::delete_deny_policy(const std::string& attachment, const std:
|
||||
return runner_.mutate("iam.v2.policies.delete", a);
|
||||
}
|
||||
|
||||
Json AgentIdentity::boundaries(const std::string& org_id) const {
|
||||
ListResult AgentIdentity::boundaries(const std::string& org_id) const {
|
||||
return runner_.list("iam.v3.principalAccessBoundaryPolicies.list", "principalAccessBoundaryPolicies",
|
||||
with("parent", boundary_parent(org_id)));
|
||||
}
|
||||
@@ -243,4 +245,52 @@ Json AgentIdentity::delete_boundary(const std::string& org_id, const std::string
|
||||
with("name", boundary_parent(org_id) + "/principalAccessBoundaryPolicies/" + id));
|
||||
}
|
||||
|
||||
Json AgentIdentity::update_deny_policy(const std::string& attachment, const std::string& id,
|
||||
const DenyPolicySpec& spec) const {
|
||||
Ids::require_segment(id, "policy id");
|
||||
Json current = deny_policy(attachment, id); // Must exist; supplies the etag.
|
||||
DenyPolicySpec body = spec;
|
||||
if (body.etag.empty()) body.etag = string_at(current, "etag");
|
||||
CallArgs a = with("name", policy_name(attachment, id));
|
||||
a.body = body.to_json();
|
||||
runner_.mutate("iam.v2.policies.update", a);
|
||||
return deny_policy(attachment, id);
|
||||
}
|
||||
|
||||
Json AgentIdentity::deny_policy_operation(const std::string& operation_name) const {
|
||||
// The name comes from a mutation reply: policies/.../operations/{id}.
|
||||
if (operation_name.rfind("policies/", 0) != 0 ||
|
||||
operation_name.find("/operations/") == std::string::npos)
|
||||
throw GoogleError::invalid("operation name must look like policies/.../operations/{id}");
|
||||
return runner_.invoke("iam.v2.policies.operations.get", with("name", operation_name));
|
||||
}
|
||||
|
||||
Json AgentIdentity::engine_operation(const std::string& operation_id) const {
|
||||
Ids::require_segment(operation_id, "operation id");
|
||||
return runner_.invoke("aiplatform.v1.operations.get",
|
||||
with("name", context_.parent() + "/operations/" + operation_id));
|
||||
}
|
||||
|
||||
Json AgentIdentity::update_boundary(const std::string& org_id, const std::string& id,
|
||||
const AccessBoundarySpec& spec,
|
||||
const std::vector<std::string>& mask) const {
|
||||
Ids::require_segment(id, "policy id");
|
||||
CallArgs a = with("name", boundary_parent(org_id) + "/principalAccessBoundaryPolicies/" + id);
|
||||
a.body = spec.to_json();
|
||||
a.query["updateMask"] =
|
||||
CallRunner::update_mask("iam.v3.principalAccessBoundaryPolicies.patch", mask);
|
||||
boundary(org_id, id); // Must exist (checked after the input, before the write).
|
||||
runner_.mutate("iam.v3.principalAccessBoundaryPolicies.patch", a);
|
||||
return boundary(org_id, id);
|
||||
}
|
||||
|
||||
ListResult AgentIdentity::boundary_bindings(const std::string& org_id,
|
||||
const std::string& id) const {
|
||||
Ids::require_segment(id, "policy id");
|
||||
// searchPolicyBindings: the policy bindings that attach this boundary.
|
||||
return runner_.list("iam.v3.principalAccessBoundaryPolicies.searchPolicyBindings",
|
||||
"policyBindings",
|
||||
with("name", boundary_parent(org_id) + "/principalAccessBoundaryPolicies/" + id));
|
||||
}
|
||||
|
||||
} // namespace molo::google::registry
|
||||
|
||||
@@ -43,6 +43,7 @@ struct DenyRuleSpec {
|
||||
|
||||
struct DenyPolicySpec {
|
||||
std::string display_name;
|
||||
std::string etag; // update only; empty: the current etag is read and used
|
||||
std::vector<DenyRuleSpec> rules; // At least one.
|
||||
|
||||
clients::body::Json to_json() const;
|
||||
@@ -63,7 +64,7 @@ class AgentIdentity : public RegistryService {
|
||||
static ServiceEndpoint default_endpoint(const std::string& location);
|
||||
|
||||
// ---- Reading identities (aiplatform v1; an API key reaches these).
|
||||
clients::body::Json engines() const;
|
||||
ListResult engines() const;
|
||||
IdentityInfo identity(const std::string& engine_id) const;
|
||||
clients::body::Json engine_iam_policy(const std::string& engine_id) const;
|
||||
|
||||
@@ -74,19 +75,34 @@ class AgentIdentity : public RegistryService {
|
||||
// point, e.g. attachment_for_project("123").
|
||||
static std::string attachment_for_project(const std::string& project_number);
|
||||
static std::string attachment_for_organization(const std::string& org_id);
|
||||
clients::body::Json deny_policies(const std::string& attachment) const;
|
||||
ListResult deny_policies(const std::string& attachment) const;
|
||||
clients::body::Json deny_policy(const std::string& attachment, const std::string& id) const;
|
||||
clients::body::Json create_deny_policy(const std::string& attachment, const std::string& id,
|
||||
const DenyPolicySpec& spec) const;
|
||||
// PUT: replaces displayName and rules. Reads the policy first (it must
|
||||
// exist) and sends its etag unless spec.etag is set.
|
||||
clients::body::Json update_deny_policy(const std::string& attachment, const std::string& id,
|
||||
const DenyPolicySpec& spec) const;
|
||||
clients::body::Json delete_deny_policy(const std::string& attachment, const std::string& id,
|
||||
const std::string& etag = "") const;
|
||||
// Long-running operation of the IAM v2 policies API.
|
||||
// `operation_name` is policies/.../operations/{id}, as a mutation returns it.
|
||||
clients::body::Json deny_policy_operation(const std::string& operation_name) const;
|
||||
// Long-running operation of the aiplatform location (reasoning engines).
|
||||
clients::body::Json engine_operation(const std::string& operation_id) const;
|
||||
|
||||
// ---- Principal access boundary policies (iam v3, organization scope).
|
||||
clients::body::Json boundaries(const std::string& org_id) const;
|
||||
ListResult boundaries(const std::string& org_id) const;
|
||||
clients::body::Json boundary(const std::string& org_id, const std::string& id) const;
|
||||
clients::body::Json create_boundary(const std::string& org_id, const std::string& id,
|
||||
const AccessBoundarySpec& spec) const;
|
||||
// PATCH: mask names AccessBoundary fields (displayName, details, ...).
|
||||
clients::body::Json update_boundary(const std::string& org_id, const std::string& id,
|
||||
const AccessBoundarySpec& spec,
|
||||
const std::vector<std::string>& mask) const;
|
||||
clients::body::Json delete_boundary(const std::string& org_id, const std::string& id) const;
|
||||
// searchPolicyBindings: bindings that attach this boundary.
|
||||
ListResult boundary_bindings(const std::string& org_id, const std::string& id) const;
|
||||
|
||||
private:
|
||||
static std::string policy_parent(const std::string& attachment);
|
||||
|
||||
@@ -33,18 +33,27 @@ bool asset_kind_from(const std::string& c, AssetKind& out) {
|
||||
return false;
|
||||
}
|
||||
|
||||
ServiceEndpoint AgentRegistry::default_endpoint() {
|
||||
return endpoints::custom("agentregistry", "https://agentregistry.googleapis.com/v1");
|
||||
ServiceEndpoint AgentRegistry::default_endpoint(RegistryVersion version) {
|
||||
return version == RegistryVersion::kV1Alpha
|
||||
? endpoints::custom("agentregistry-v1alpha",
|
||||
"https://agentregistry.googleapis.com/v1alpha")
|
||||
: endpoints::custom("agentregistry", "https://agentregistry.googleapis.com/v1");
|
||||
}
|
||||
|
||||
AgentRegistry::AgentRegistry(std::shared_ptr<GoogleClient> client, CloudContext context)
|
||||
: RegistryService(std::move(client), default_endpoint(), std::move(context)) {}
|
||||
AgentRegistry::AgentRegistry(std::shared_ptr<GoogleClient> client, CloudContext context,
|
||||
RegistryVersion version)
|
||||
: RegistryService(std::move(client), default_endpoint(version), std::move(context)),
|
||||
version_(version == RegistryVersion::kV1Alpha ? "v1alpha" : "v1") {}
|
||||
|
||||
std::string AgentRegistry::rid(const std::string& tail) const {
|
||||
return "agentregistry." + version_ + "." + tail;
|
||||
}
|
||||
|
||||
std::string AgentRegistry::call_id(AssetKind kind, const char* verb) const {
|
||||
// Skills (CRUD, search, revisions) and publishers exist in v1alpha only.
|
||||
bool alpha = kind == AssetKind::kSkills || kind == AssetKind::kPublishers;
|
||||
return std::string("agentregistry.") + (alpha ? "v1alpha." : "v1.") + asset_collection(kind) +
|
||||
"." + verb;
|
||||
std::string base = alpha ? "agentregistry.v1alpha." : rid("");
|
||||
return base + asset_collection(kind) + "." + verb;
|
||||
}
|
||||
|
||||
std::string AgentRegistry::name_of(AssetKind kind, const std::string& id) const {
|
||||
@@ -55,18 +64,18 @@ CallArgs AgentRegistry::parent_args() const { return with("parent", context_.par
|
||||
|
||||
// ---- Reads
|
||||
|
||||
Json AgentRegistry::locations() const {
|
||||
return runner_.list("agentregistry.v1.locations.list", "locations",
|
||||
ListResult AgentRegistry::locations() const {
|
||||
return runner_.list(rid("locations.list"), "locations",
|
||||
with("name", "projects/" + context_.project));
|
||||
}
|
||||
|
||||
Json AgentRegistry::location(const std::string& id) const {
|
||||
Ids::require_segment(id, "location id");
|
||||
return runner_.invoke("agentregistry.v1.locations.get",
|
||||
return runner_.invoke(rid("locations.get"),
|
||||
with("name", "projects/" + context_.project + "/locations/" + id));
|
||||
}
|
||||
|
||||
Json AgentRegistry::list(AssetKind kind, const std::map<std::string, std::string>& query) const {
|
||||
ListResult AgentRegistry::list(AssetKind kind, const std::map<std::string, std::string>& query) const {
|
||||
CallArgs a = parent_args();
|
||||
for (const auto& kv : query) {
|
||||
if (kv.first == "filter") Ids::require_text(kv.second, RegistryCaps::kMaxFilterChars, "filter");
|
||||
@@ -124,8 +133,10 @@ Json AgentRegistry::tools(const std::string& mcp_server_id) const {
|
||||
return t && t->is_array() ? *t : Json::array();
|
||||
}
|
||||
|
||||
Json AgentRegistry::list_approved(AssetKind kind, const ApprovalLedger& ledger) const {
|
||||
return ledger.restrict_to_approved(list(kind));
|
||||
ListResult AgentRegistry::list_approved(AssetKind kind, const ApprovalLedger& ledger) const {
|
||||
ListResult all = list(kind);
|
||||
all.items = ledger.restrict_to_approved(all.items); // truncated stays as listed
|
||||
return all;
|
||||
}
|
||||
|
||||
// ---- Services
|
||||
@@ -140,7 +151,7 @@ Json AgentRegistry::register_service(const std::string& id, const ServiceRecord&
|
||||
put_query(a, "requestId", request_id);
|
||||
if (exists([&] { return get(AssetKind::kServices, id); }))
|
||||
throw GoogleError(409, "ALREADY_EXISTS", "service " + id + " is already registered");
|
||||
runner_.mutate("agentregistry.v1.services.create", a);
|
||||
runner_.mutate(rid("services.create"), a);
|
||||
return get(AssetKind::kServices, id);
|
||||
}
|
||||
|
||||
@@ -150,17 +161,17 @@ Json AgentRegistry::update_service(const std::string& id, const ServiceRecord& r
|
||||
Ids::require_service_id(id);
|
||||
CallArgs a = with("name", name_of(AssetKind::kServices, id));
|
||||
a.body = record.to_json();
|
||||
a.query["updateMask"] = CallRunner::update_mask("agentregistry.v1.services.patch", mask);
|
||||
a.query["updateMask"] = CallRunner::update_mask(rid("services.patch"), mask);
|
||||
put_query(a, "requestId", request_id);
|
||||
get(AssetKind::kServices, id); // Must exist.
|
||||
runner_.mutate("agentregistry.v1.services.patch", a);
|
||||
runner_.mutate(rid("services.patch"), a);
|
||||
return get(AssetKind::kServices, id);
|
||||
}
|
||||
|
||||
Json AgentRegistry::remove_service(const std::string& id) const {
|
||||
Ids::require_service_id(id);
|
||||
get(AssetKind::kServices, id);
|
||||
return runner_.mutate("agentregistry.v1.services.delete",
|
||||
return runner_.mutate(rid("services.delete"),
|
||||
with("name", name_of(AssetKind::kServices, id)));
|
||||
}
|
||||
|
||||
@@ -197,7 +208,7 @@ Json AgentRegistry::delete_skill(const std::string& id, bool force) const {
|
||||
return runner_.mutate("agentregistry.v1alpha.skills.delete", a);
|
||||
}
|
||||
|
||||
Json AgentRegistry::list_revisions(const std::string& skill_id) const {
|
||||
ListResult AgentRegistry::list_revisions(const std::string& skill_id) const {
|
||||
Ids::require_segment(skill_id, "skill id");
|
||||
return runner_.list("agentregistry.v1alpha.skills.revisions.list", "skillRevisions",
|
||||
with("parent", name_of(AssetKind::kSkills, skill_id)));
|
||||
@@ -237,7 +248,7 @@ Json AgentRegistry::create_binding(const std::string& id, const BindingRecord& b
|
||||
a.query["bindingId"] = id;
|
||||
if (exists([&] { return get(AssetKind::kBindings, id); }))
|
||||
throw GoogleError(409, "ALREADY_EXISTS", "binding " + id + " already exists");
|
||||
runner_.mutate("agentregistry.v1.bindings.create", a);
|
||||
runner_.mutate(rid("bindings.create"), a);
|
||||
return get(AssetKind::kBindings, id);
|
||||
}
|
||||
|
||||
@@ -246,20 +257,20 @@ Json AgentRegistry::update_binding(const std::string& id, const BindingRecord& b
|
||||
Ids::require_segment(id, "binding id");
|
||||
CallArgs a = with("name", name_of(AssetKind::kBindings, id));
|
||||
a.body = binding.to_json();
|
||||
a.query["updateMask"] = CallRunner::update_mask("agentregistry.v1.bindings.patch", mask);
|
||||
a.query["updateMask"] = CallRunner::update_mask(rid("bindings.patch"), mask);
|
||||
get(AssetKind::kBindings, id);
|
||||
runner_.mutate("agentregistry.v1.bindings.patch", a);
|
||||
runner_.mutate(rid("bindings.patch"), a);
|
||||
return get(AssetKind::kBindings, id);
|
||||
}
|
||||
|
||||
Json AgentRegistry::delete_binding(const std::string& id) const {
|
||||
Ids::require_segment(id, "binding id");
|
||||
get(AssetKind::kBindings, id);
|
||||
return runner_.mutate("agentregistry.v1.bindings.delete",
|
||||
return runner_.mutate(rid("bindings.delete"),
|
||||
with("name", name_of(AssetKind::kBindings, id)));
|
||||
}
|
||||
|
||||
Json AgentRegistry::fetch_available_bindings(const std::string& source_id,
|
||||
ListResult AgentRegistry::fetch_available_bindings(const std::string& source_id,
|
||||
const std::string& target_id) const {
|
||||
if (!source_id.empty() && !BindingRecord::valid_identifier(source_id, true))
|
||||
throw GoogleError::invalid("source id must be urn:agent:{publisher}:{namespace}:{name}");
|
||||
@@ -268,15 +279,15 @@ Json AgentRegistry::fetch_available_bindings(const std::string& source_id,
|
||||
CallArgs a = parent_args();
|
||||
put_query(a, "sourceIdentifier", source_id);
|
||||
put_query(a, "targetIdentifier", target_id);
|
||||
return runner_.list("agentregistry.v1.bindings.fetchAvailable", "bindings", a);
|
||||
return runner_.list(rid("bindings.fetchAvailable"), "bindings", a);
|
||||
}
|
||||
|
||||
// ---- AI applications
|
||||
|
||||
Json AgentRegistry::application_assets(const std::string& application_id, AssetKind kind) const {
|
||||
ListResult AgentRegistry::application_assets(const std::string& application_id, AssetKind kind) const {
|
||||
if (kind != AssetKind::kAgents && kind != AssetKind::kMcpServers && kind != AssetKind::kEndpoints)
|
||||
throw GoogleError::invalid("applications hold agents, mcpServers and endpoints");
|
||||
return runner_.list(std::string("agentregistry.v1.aiApplications.") + asset_collection(kind) + ".list",
|
||||
return runner_.list(rid("aiApplications.") + asset_collection(kind) + ".list",
|
||||
asset_collection(kind),
|
||||
with("parent", name_of(AssetKind::kAiApplications, application_id)));
|
||||
}
|
||||
@@ -285,12 +296,12 @@ Json AgentRegistry::application_assets(const std::string& application_id, AssetK
|
||||
|
||||
Json AgentRegistry::operation(const std::string& id) const {
|
||||
Ids::require_segment(id, "operation id");
|
||||
return runner_.invoke("agentregistry.v1.operations.get",
|
||||
return runner_.invoke(rid("operations.get"),
|
||||
with("name", context_.parent() + "/operations/" + id));
|
||||
}
|
||||
|
||||
Json AgentRegistry::operations() const {
|
||||
return runner_.list("agentregistry.v1.operations.list", "operations",
|
||||
ListResult AgentRegistry::operations() const {
|
||||
return runner_.list(rid("operations.list"), "operations",
|
||||
with("name", context_.parent()));
|
||||
}
|
||||
|
||||
@@ -298,12 +309,12 @@ Json AgentRegistry::cancel_operation(const std::string& id) const {
|
||||
operation(id);
|
||||
CallArgs a = with("name", context_.parent() + "/operations/" + id);
|
||||
a.body = Json::object();
|
||||
return runner_.invoke("agentregistry.v1.operations.cancel", a);
|
||||
return runner_.invoke(rid("operations.cancel"), a);
|
||||
}
|
||||
|
||||
Json AgentRegistry::delete_operation(const std::string& id) const {
|
||||
operation(id);
|
||||
return runner_.invoke("agentregistry.v1.operations.delete",
|
||||
return runner_.invoke(rid("operations.delete"),
|
||||
with("name", context_.parent() + "/operations/" + id));
|
||||
}
|
||||
|
||||
@@ -318,8 +329,8 @@ void require_iam_kind(AssetKind k) {
|
||||
|
||||
Json AgentRegistry::iam_policy(AssetKind kind, const std::string& id) const {
|
||||
require_iam_kind(kind);
|
||||
std::string cid = kind == AssetKind::kSkills ? "agentregistry.v1.skills.getIamPolicy"
|
||||
: "agentregistry.v1.aiApplications.getIamPolicy";
|
||||
std::string cid = kind == AssetKind::kSkills ? rid("skills.getIamPolicy")
|
||||
: rid("aiApplications.getIamPolicy");
|
||||
return runner_.invoke(cid, with("resource", name_of(kind, id)));
|
||||
}
|
||||
|
||||
@@ -327,8 +338,8 @@ Json AgentRegistry::set_iam_policy(AssetKind kind, const std::string& id, const
|
||||
require_iam_kind(kind);
|
||||
if (!policy.is_object()) throw GoogleError::invalid("policy must be a JSON object");
|
||||
iam_policy(kind, id); // Read first.
|
||||
std::string cid = kind == AssetKind::kSkills ? "agentregistry.v1.skills.setIamPolicy"
|
||||
: "agentregistry.v1.aiApplications.setIamPolicy";
|
||||
std::string cid = kind == AssetKind::kSkills ? rid("skills.setIamPolicy")
|
||||
: rid("aiApplications.setIamPolicy");
|
||||
CallArgs a = with("resource", name_of(kind, id));
|
||||
a.body = Json::object();
|
||||
a.body.set("policy", policy);
|
||||
@@ -340,8 +351,8 @@ Json AgentRegistry::test_iam_permissions(AssetKind kind, const std::string& id,
|
||||
require_iam_kind(kind);
|
||||
if (permissions.empty() || permissions.size() > RegistryCaps::kMaxPermissions)
|
||||
throw GoogleError::invalid("1..64 permissions");
|
||||
std::string cid = kind == AssetKind::kSkills ? "agentregistry.v1.skills.testIamPermissions"
|
||||
: "agentregistry.v1.aiApplications.testIamPermissions";
|
||||
std::string cid = kind == AssetKind::kSkills ? rid("skills.testIamPermissions")
|
||||
: rid("aiApplications.testIamPermissions");
|
||||
CallArgs a = with("resource", name_of(kind, id));
|
||||
a.body = Json::object();
|
||||
Json arr = Json::array();
|
||||
|
||||
@@ -25,6 +25,11 @@
|
||||
|
||||
namespace molo::google::registry {
|
||||
|
||||
// Discovery lists every registry resource in v1 and in v1alpha (skills and
|
||||
// publishers in v1alpha only). Both versions are in the call table; the
|
||||
// version picks the rows. v1 is the default.
|
||||
enum class RegistryVersion { kV1, kV1Alpha };
|
||||
|
||||
enum class AssetKind { kAgents, kMcpServers, kEndpoints, kServices, kSkills, kBindings, kPublishers, kAiApplications };
|
||||
|
||||
const char* asset_collection(AssetKind kind); // "agents", "mcpServers", ...
|
||||
@@ -37,17 +42,22 @@ struct SearchPage {
|
||||
|
||||
class AgentRegistry : public RegistryService {
|
||||
public:
|
||||
AgentRegistry(std::shared_ptr<GoogleClient> client, CloudContext context);
|
||||
AgentRegistry(std::shared_ptr<GoogleClient> client, CloudContext context,
|
||||
RegistryVersion version = RegistryVersion::kV1);
|
||||
|
||||
static ServiceEndpoint default_endpoint(); // https://agentregistry.googleapis.com/v1
|
||||
// https://agentregistry.googleapis.com/v1 (or /v1alpha)
|
||||
static ServiceEndpoint default_endpoint(RegistryVersion version = RegistryVersion::kV1);
|
||||
const std::string& version() const { return version_; }
|
||||
|
||||
// ---- Reads (always available before any mutation).
|
||||
clients::body::Json locations() const;
|
||||
ListResult locations() const;
|
||||
clients::body::Json location(const std::string& id) const;
|
||||
// `query` holds optional filter / orderBy, accepted only where the
|
||||
// discovery document lists them for that collection.
|
||||
clients::body::Json list(AssetKind kind,
|
||||
const std::map<std::string, std::string>& query = {}) const;
|
||||
// Paged up to RegistryCaps::kMaxPages / kMaxItems; ListResult::truncated
|
||||
// says loudly when more existed.
|
||||
ListResult list(AssetKind kind,
|
||||
const std::map<std::string, std::string>& query = {}) const;
|
||||
clients::body::Json get(AssetKind kind, const std::string& id) const;
|
||||
// agents:search and mcpServers:search (POST body), one page per call.
|
||||
SearchPage search(AssetKind kind, const std::string& query, int page_size = 20,
|
||||
@@ -56,7 +66,7 @@ class AgentRegistry : public RegistryService {
|
||||
SearchPage search_skills(const std::string& query, const std::string& search_type = "",
|
||||
int page_size = 20, const std::string& page_token = "") const;
|
||||
clients::body::Json tools(const std::string& mcp_server_id) const; // McpServer.tools
|
||||
clients::body::Json list_approved(AssetKind kind, const ApprovalLedger& ledger) const;
|
||||
ListResult list_approved(AssetKind kind, const ApprovalLedger& ledger) const;
|
||||
|
||||
// ---- Services (agents, MCP servers, endpoints are registered through
|
||||
// these). Each mutation reads first and waits for the operation.
|
||||
@@ -74,7 +84,7 @@ class AgentRegistry : public RegistryService {
|
||||
clients::body::Json set_skill_state(const std::string& id, SkillState state) const;
|
||||
// force=false unless the caller says otherwise; never defaulted to true.
|
||||
clients::body::Json delete_skill(const std::string& id, bool force = false) const;
|
||||
clients::body::Json list_revisions(const std::string& skill_id) const;
|
||||
ListResult list_revisions(const std::string& skill_id) const;
|
||||
clients::body::Json get_revision(const std::string& skill_id, const std::string& revision) const;
|
||||
clients::body::Json create_revision(const std::string& skill_id, const std::string& revision_id,
|
||||
const SkillRevisionRecord& revision) const;
|
||||
@@ -85,16 +95,15 @@ class AgentRegistry : public RegistryService {
|
||||
clients::body::Json update_binding(const std::string& id, const BindingRecord& binding,
|
||||
const std::vector<std::string>& mask) const;
|
||||
clients::body::Json delete_binding(const std::string& id) const;
|
||||
clients::body::Json fetch_available_bindings(const std::string& source_id,
|
||||
const std::string& target_id) const;
|
||||
ListResult fetch_available_bindings(const std::string& source_id,
|
||||
const std::string& target_id) const;
|
||||
|
||||
// ---- AI applications (read only) and their agents, servers, endpoints.
|
||||
clients::body::Json application_assets(const std::string& application_id,
|
||||
AssetKind kind) const;
|
||||
ListResult application_assets(const std::string& application_id, AssetKind kind) const;
|
||||
|
||||
// ---- Operations (the registry's own long-running operations).
|
||||
clients::body::Json operation(const std::string& operation_id) const;
|
||||
clients::body::Json operations() const;
|
||||
ListResult operations() const;
|
||||
clients::body::Json cancel_operation(const std::string& operation_id) const;
|
||||
clients::body::Json delete_operation(const std::string& operation_id) const;
|
||||
|
||||
@@ -107,8 +116,12 @@ class AgentRegistry : public RegistryService {
|
||||
|
||||
private:
|
||||
std::string call_id(AssetKind kind, const char* verb) const;
|
||||
// "agentregistry.<version>.<tail>": the table id for the chosen version.
|
||||
std::string rid(const std::string& tail) const;
|
||||
std::string name_of(AssetKind kind, const std::string& id) const;
|
||||
CallArgs parent_args() const;
|
||||
|
||||
std::string version_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::registry
|
||||
|
||||
@@ -89,14 +89,70 @@ Json CallRunner::invoke(const std::string& call_id, const CallArgs& args) const
|
||||
Calls::has_body(call) ? &args.body : nullptr);
|
||||
}
|
||||
|
||||
Json CallRunner::list(const std::string& call_id, const std::string& items_key,
|
||||
const CallArgs& args) const {
|
||||
Json ListResult::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("items", items);
|
||||
out.set("count", Json(static_cast<int>(size())));
|
||||
out.set("pages", Json(static_cast<int>(pages)));
|
||||
out.set("truncated", Json(truncated));
|
||||
out.set("next_page_token", Json(next_page_token));
|
||||
out.set("cap_items", Json(static_cast<int>(RegistryCaps::kMaxItems)));
|
||||
return out;
|
||||
}
|
||||
|
||||
ListPager::ListPager(std::string items_key, std::size_t max_pages, std::size_t max_items)
|
||||
: items_key_(std::move(items_key)), max_pages_(max_pages), max_items_(max_items) {}
|
||||
|
||||
bool ListPager::feed(const Json& page) {
|
||||
++result_.pages;
|
||||
if (const Json* arr = page.is_object() ? page.find(items_key_) : nullptr;
|
||||
arr && arr->is_array())
|
||||
for (const Json& item : arr->as_array()) {
|
||||
if (result_.size() >= max_items_) {
|
||||
result_.truncated = true; // This page alone overran the item cap.
|
||||
break;
|
||||
}
|
||||
result_.items.push(item);
|
||||
}
|
||||
const Json* next = page.is_object() ? page.find("nextPageToken") : nullptr;
|
||||
token_ = next && next->is_string() ? next->as_string() : "";
|
||||
if (result_.truncated) {
|
||||
token_.clear(); // Mid-page overrun: no exact continuation exists.
|
||||
return false;
|
||||
}
|
||||
if (token_.empty()) return false;
|
||||
if (result_.pages >= max_pages_ || result_.size() >= max_items_) {
|
||||
result_.truncated = true;
|
||||
result_.next_page_token = token_;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
ListResult CallRunner::list(const std::string& call_id, const std::string& items_key,
|
||||
const CallArgs& args) const {
|
||||
const CallSpec& call = Calls::get(call_id);
|
||||
if (Calls::method_of(call) != clients::Method::kGet)
|
||||
throw GoogleError::invalid("list needs a GET call");
|
||||
if (args.query.count("pageToken"))
|
||||
throw GoogleError::invalid("pageToken is managed by the list call");
|
||||
std::string path = build_path(call, args, true);
|
||||
return list_all(*client_, Calls::endpoint(call, location_), path, items_key,
|
||||
RegistryCaps::kMaxPages);
|
||||
ServiceEndpoint ep = Calls::endpoint(call, location_);
|
||||
bool pageable = Calls::allows_query(call, "pageToken");
|
||||
ListPager pager(items_key, RegistryCaps::kMaxPages, RegistryCaps::kMaxItems);
|
||||
std::string url = path;
|
||||
while (true) {
|
||||
Json reply = client_->request(clients::Method::kGet, ep, url, nullptr);
|
||||
bool more = pager.feed(reply);
|
||||
if (!more || !pageable) {
|
||||
ListResult out = pager.result();
|
||||
// A reply that names a next page the call cannot request is still loud.
|
||||
if (!pageable && !pager.token().empty()) out.truncated = true;
|
||||
return out;
|
||||
}
|
||||
url = path + (path.find('?') == std::string::npos ? '?' : '&') +
|
||||
std::string("pageToken=") + clients::Url::encode_component(pager.token());
|
||||
}
|
||||
}
|
||||
|
||||
Json CallRunner::mutate(const std::string& call_id, const CallArgs& args) const {
|
||||
|
||||
@@ -16,6 +16,41 @@
|
||||
|
||||
namespace molo::google::registry {
|
||||
|
||||
// Result of a paged list. A list never truncates silently: when pages or
|
||||
// items remain after the caps, `truncated` is true and `next_page_token`
|
||||
// continues where the cap stopped.
|
||||
struct ListResult {
|
||||
clients::body::Json items = clients::body::Json::array();
|
||||
bool truncated = false;
|
||||
std::string next_page_token; // Empty unless truncated.
|
||||
std::size_t pages = 0; // Pages fetched.
|
||||
|
||||
std::size_t size() const { return items.as_array().size(); }
|
||||
// {"items":[...],"count":n,"pages":n,"truncated":bool,"next_page_token":"",
|
||||
// "cap_items":RegistryCaps::kMaxItems}
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
// The paging state machine behind CallRunner::list, separate from any network
|
||||
// so the cap logic is testable on its own. Feed it one page document at a
|
||||
// time; it says whether another page is wanted and keeps the loud
|
||||
// `truncated` flag.
|
||||
class ListPager {
|
||||
public:
|
||||
ListPager(std::string items_key, std::size_t max_pages, std::size_t max_items);
|
||||
// Takes the next page document. Returns true when the caller should fetch
|
||||
// the page named by token().
|
||||
bool feed(const clients::body::Json& page);
|
||||
const std::string& token() const { return token_; }
|
||||
const ListResult& result() const { return result_; }
|
||||
|
||||
private:
|
||||
std::string items_key_;
|
||||
std::size_t max_pages_, max_items_;
|
||||
std::string token_;
|
||||
ListResult result_;
|
||||
};
|
||||
|
||||
struct CallArgs {
|
||||
std::map<std::string, std::string> vars; // {+parent}, {+name}, ...
|
||||
std::map<std::string, std::string> query; // updateMask, filter, ...
|
||||
@@ -28,10 +63,12 @@ class CallRunner {
|
||||
|
||||
// One request, one reply.
|
||||
clients::body::Json invoke(const std::string& call_id, const CallArgs& args) const;
|
||||
// Follows nextPageToken (RegistryCaps::kMaxPages pages of kPageSize) and
|
||||
// returns the concatenated `items_key` array.
|
||||
clients::body::Json list(const std::string& call_id, const std::string& items_key,
|
||||
const CallArgs& args) const;
|
||||
// Follows nextPageToken up to RegistryCaps::kMaxPages pages and kMaxItems
|
||||
// items and returns the concatenated `items_key` array. Stopping at a cap
|
||||
// with pages left is reported (ListResult::truncated), never hidden. The
|
||||
// caller cannot pass its own pageToken: the runner owns paging.
|
||||
ListResult list(const std::string& call_id, const std::string& items_key,
|
||||
const CallArgs& args) const;
|
||||
// invoke + wait for the operation the reply describes; returns
|
||||
// {"operation": name}.
|
||||
clients::body::Json mutate(const std::string& call_id, const CallArgs& args) const;
|
||||
|
||||
@@ -51,8 +51,44 @@ const std::vector<CallSpec> kCalls = {
|
||||
{"agentregistry.v1.skills.getIamPolicy", "GET", "/{+resource}:getIamPolicy", "agentregistry", "v1", "agentregistry.projects.locations.skills.getIamPolicy", "-", "options.requestedPolicyVersion", "-"},
|
||||
{"agentregistry.v1.skills.setIamPolicy", "POST", "/{+resource}:setIamPolicy", "agentregistry", "v1", "agentregistry.projects.locations.skills.setIamPolicy", "GoogleIamV1SetIamPolicyRequest", "-", "policy,updateMask"},
|
||||
{"agentregistry.v1.skills.testIamPermissions", "POST", "/{+resource}:testIamPermissions", "agentregistry", "v1", "agentregistry.projects.locations.skills.testIamPermissions", "GoogleIamV1TestIamPermissionsRequest", "-", "permissions"},
|
||||
{"agentregistry.v1alpha.agents.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.agents.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.agents.list", "GET", "/{+parent}/agents", "agentregistry", "v1alpha", "agentregistry.projects.locations.agents.list", "-", "filter,orderBy,pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.agents.search", "POST", "/{+parent}/agents:search", "agentregistry", "v1alpha", "agentregistry.projects.locations.agents.search", "SearchAgentsRequest", "-", "searchString,pageSize,pageToken"},
|
||||
{"agentregistry.v1alpha.aiApplications.agents.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.agents.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.agents.list", "GET", "/{+parent}/agents", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.agents.list", "-", "pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.endpoints.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.endpoints.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.endpoints.list", "GET", "/{+parent}/endpoints", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.endpoints.list", "-", "pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.getIamPolicy", "GET", "/{+resource}:getIamPolicy", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.getIamPolicy", "-", "options.requestedPolicyVersion", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.list", "GET", "/{+parent}/aiApplications", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.list", "-", "filter,orderBy,pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.mcpServers.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.mcpServers.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.mcpServers.list", "GET", "/{+parent}/mcpServers", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.mcpServers.list", "-", "pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.aiApplications.setIamPolicy", "POST", "/{+resource}:setIamPolicy", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.setIamPolicy", "GoogleIamV1SetIamPolicyRequest", "-", "policy,updateMask"},
|
||||
{"agentregistry.v1alpha.aiApplications.testIamPermissions", "POST", "/{+resource}:testIamPermissions", "agentregistry", "v1alpha", "agentregistry.projects.locations.aiApplications.testIamPermissions", "GoogleIamV1TestIamPermissionsRequest", "-", "permissions"},
|
||||
{"agentregistry.v1alpha.bindings.create", "POST", "/{+parent}/bindings", "agentregistry", "v1alpha", "agentregistry.projects.locations.bindings.create", "Binding", "bindingId,requestId", "displayName,description,source,target,authProviderBinding"},
|
||||
{"agentregistry.v1alpha.bindings.delete", "DELETE", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.bindings.delete", "-", "requestId", "-"},
|
||||
{"agentregistry.v1alpha.bindings.fetchAvailable", "GET", "/{+parent}/bindings:fetchAvailable", "agentregistry", "v1alpha", "agentregistry.projects.locations.bindings.fetchAvailable", "-", "pageSize,pageToken,sourceIdentifier,targetIdentifier", "-"},
|
||||
{"agentregistry.v1alpha.bindings.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.bindings.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.bindings.list", "GET", "/{+parent}/bindings", "agentregistry", "v1alpha", "agentregistry.projects.locations.bindings.list", "-", "filter,orderBy,pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.bindings.patch", "PATCH", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.bindings.patch", "Binding", "requestId,updateMask", "displayName,description,source,target,authProviderBinding"},
|
||||
{"agentregistry.v1alpha.endpoints.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.endpoints.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.endpoints.list", "GET", "/{+parent}/endpoints", "agentregistry", "v1alpha", "agentregistry.projects.locations.endpoints.list", "-", "filter,pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.locations.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.locations.list", "GET", "/{+name}/locations", "agentregistry", "v1alpha", "agentregistry.projects.locations.list", "-", "extraLocationTypes,filter,pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.mcpServers.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.mcpServers.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.mcpServers.list", "GET", "/{+parent}/mcpServers", "agentregistry", "v1alpha", "agentregistry.projects.locations.mcpServers.list", "-", "filter,orderBy,pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.mcpServers.search", "POST", "/{+parent}/mcpServers:search", "agentregistry", "v1alpha", "agentregistry.projects.locations.mcpServers.search", "SearchMcpServersRequest", "-", "searchString,pageSize,pageToken"},
|
||||
{"agentregistry.v1alpha.operations.cancel", "POST", "/{+name}:cancel", "agentregistry", "v1alpha", "agentregistry.projects.locations.operations.cancel", "CancelOperationRequest", "-", ""},
|
||||
{"agentregistry.v1alpha.operations.delete", "DELETE", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.operations.delete", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.operations.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.operations.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.operations.list", "GET", "/{+name}/operations", "agentregistry", "v1alpha", "agentregistry.projects.locations.operations.list", "-", "filter,pageSize,pageToken,returnPartialSuccess", "-"},
|
||||
{"agentregistry.v1alpha.publishers.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.publishers.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.publishers.list", "GET", "/{+parent}/publishers", "agentregistry", "v1alpha", "agentregistry.projects.locations.publishers.list", "-", "pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.services.create", "POST", "/{+parent}/services", "agentregistry", "v1alpha", "agentregistry.projects.locations.services.create", "Service", "requestId,serviceId", "displayName,description,interfaces,agentSpec,mcpServerSpec,endpointSpec"},
|
||||
{"agentregistry.v1alpha.services.delete", "DELETE", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.services.delete", "-", "requestId", "-"},
|
||||
{"agentregistry.v1alpha.services.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.services.get", "-", "-", "-"},
|
||||
{"agentregistry.v1alpha.services.list", "GET", "/{+parent}/services", "agentregistry", "v1alpha", "agentregistry.projects.locations.services.list", "-", "filter,pageSize,pageToken", "-"},
|
||||
{"agentregistry.v1alpha.services.patch", "PATCH", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.services.patch", "Service", "requestId,updateMask", "displayName,description,interfaces,agentSpec,mcpServerSpec,endpointSpec"},
|
||||
{"agentregistry.v1alpha.skills.create", "POST", "/{+parent}/skills", "agentregistry", "v1alpha", "agentregistry.projects.locations.skills.create", "Skill", "requestId,skillId", "displayName,description,type,targetState,publisher,defaultRevision,initialRevision"},
|
||||
{"agentregistry.v1alpha.skills.delete", "DELETE", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.skills.delete", "-", "force,requestId", "-"},
|
||||
{"agentregistry.v1alpha.skills.get", "GET", "/{+name}", "agentregistry", "v1alpha", "agentregistry.projects.locations.skills.get", "-", "-", "-"},
|
||||
@@ -91,13 +127,19 @@ const std::vector<CallSpec> kCalls = {
|
||||
{"networksecurity.v1.authzPolicies.patch", "PATCH", "/{+name}", "networksecurity", "v1", "networksecurity.projects.locations.authzPolicies.patch", "AuthzPolicy", "requestId,updateMask", "description,labels,target,httpRules,action,customProvider,policyProfile"},
|
||||
{"networksecurity.v1.authzPolicies.setIamPolicy", "POST", "/{+resource}:setIamPolicy", "networksecurity", "v1", "networksecurity.projects.locations.authzPolicies.setIamPolicy", "GoogleIamV1SetIamPolicyRequest", "-", "policy,updateMask"},
|
||||
{"networksecurity.v1.authzPolicies.testIamPermissions", "POST", "/{+resource}:testIamPermissions", "networksecurity", "v1", "networksecurity.projects.locations.authzPolicies.testIamPermissions", "GoogleIamV1TestIamPermissionsRequest", "-", "permissions"},
|
||||
{"networksecurity.v1.operations.cancel", "POST", "/{+name}:cancel", "networksecurity", "v1", "networksecurity.projects.locations.operations.cancel", "CancelOperationRequest", "-", ""},
|
||||
{"networksecurity.v1.operations.delete", "DELETE", "/{+name}", "networksecurity", "v1", "networksecurity.projects.locations.operations.delete", "-", "-", "-"},
|
||||
{"networksecurity.v1.operations.get", "GET", "/{+name}", "networksecurity", "v1", "networksecurity.projects.locations.operations.get", "-", "-", "-"},
|
||||
{"networksecurity.v1.operations.list", "GET", "/{+name}/operations", "networksecurity", "v1", "networksecurity.projects.locations.operations.list", "-", "filter,pageSize,pageToken,returnPartialSuccess", "-"},
|
||||
{"networkservices.v1.agentGateways.create", "POST", "/{+parent}/agentGateways", "networkservices", "v1", "networkservices.projects.locations.agentGateways.create", "AgentGateway", "agentGatewayId", "description,labels,registries,networkConfig,googleManaged,selfManaged,agentConnectivityTemplate,etag"},
|
||||
{"networkservices.v1.agentGateways.delete", "DELETE", "/{+name}", "networkservices", "v1", "networkservices.projects.locations.agentGateways.delete", "-", "etag", "-"},
|
||||
{"networkservices.v1.agentGateways.get", "GET", "/{+name}", "networkservices", "v1", "networkservices.projects.locations.agentGateways.get", "-", "-", "-"},
|
||||
{"networkservices.v1.agentGateways.list", "GET", "/{+parent}/agentGateways", "networkservices", "v1", "networkservices.projects.locations.agentGateways.list", "-", "filter,pageSize,pageToken,returnPartialSuccess", "-"},
|
||||
{"networkservices.v1.agentGateways.patch", "PATCH", "/{+name}", "networkservices", "v1", "networkservices.projects.locations.agentGateways.patch", "AgentGateway", "updateMask", "description,labels,registries,networkConfig,googleManaged,selfManaged,agentConnectivityTemplate,etag"},
|
||||
{"networkservices.v1.operations.cancel", "POST", "/{+name}:cancel", "networkservices", "v1", "networkservices.projects.locations.operations.cancel", "CancelOperationRequest", "-", ""},
|
||||
{"networkservices.v1.operations.delete", "DELETE", "/{+name}", "networkservices", "v1", "networkservices.projects.locations.operations.delete", "-", "-", "-"},
|
||||
{"networkservices.v1.operations.get", "GET", "/{+name}", "networkservices", "v1", "networkservices.projects.locations.operations.get", "-", "-", "-"},
|
||||
{"networkservices.v1.operations.list", "GET", "/{+name}/operations", "networkservices", "v1", "networkservices.projects.locations.operations.list", "-", "filter,pageSize,pageToken,returnPartialSuccess", "-"},
|
||||
};
|
||||
} // namespace
|
||||
|
||||
|
||||
@@ -126,7 +126,7 @@ ServiceEndpoint GatewayPolicies::default_endpoint() {
|
||||
GatewayPolicies::GatewayPolicies(std::shared_ptr<GoogleClient> client, CloudContext context)
|
||||
: RegistryService(std::move(client), default_endpoint(), std::move(context)) {}
|
||||
|
||||
Json GatewayPolicies::list() const {
|
||||
ListResult GatewayPolicies::list() const {
|
||||
return runner_.list("networksecurity.v1.authzPolicies.list", "authzPolicies",
|
||||
with("parent", context_.parent()));
|
||||
}
|
||||
@@ -184,4 +184,46 @@ Json GatewayPolicies::operation(const std::string& operation_id) const {
|
||||
with("name", context_.parent() + "/operations/" + operation_id));
|
||||
}
|
||||
|
||||
Json GatewayPolicies::set_iam_policy(const std::string& id, const Json& policy) const {
|
||||
if (!policy.is_object()) throw GoogleError::invalid("policy must be a JSON object");
|
||||
get(id); // Read first: a missing policy is a clear 404.
|
||||
CallArgs a = with("resource", context_.child("authzPolicies", id));
|
||||
a.body = Json::object();
|
||||
a.body.set("policy", policy);
|
||||
return runner_.invoke("networksecurity.v1.authzPolicies.setIamPolicy", a);
|
||||
}
|
||||
|
||||
Json GatewayPolicies::test_iam_permissions(const std::string& id,
|
||||
const std::vector<std::string>& permissions) const {
|
||||
if (permissions.empty() || permissions.size() > RegistryCaps::kMaxPermissions)
|
||||
throw GoogleError::invalid("1..64 permissions");
|
||||
CallArgs a = with("resource", context_.child("authzPolicies", id));
|
||||
a.body = Json::object();
|
||||
Json arr = Json::array();
|
||||
for (const auto& p : permissions) {
|
||||
Ids::require_text(p, 256, "permission");
|
||||
arr.push(Json(p));
|
||||
}
|
||||
a.body.set("permissions", arr);
|
||||
return runner_.invoke("networksecurity.v1.authzPolicies.testIamPermissions", a);
|
||||
}
|
||||
|
||||
ListResult GatewayPolicies::operations() const {
|
||||
return runner_.list("networksecurity.v1.operations.list", "operations",
|
||||
with("name", context_.parent()));
|
||||
}
|
||||
|
||||
Json GatewayPolicies::cancel_operation(const std::string& operation_id) const {
|
||||
operation(operation_id);
|
||||
CallArgs a = with("name", context_.parent() + "/operations/" + operation_id);
|
||||
a.body = Json::object();
|
||||
return runner_.invoke("networksecurity.v1.operations.cancel", a);
|
||||
}
|
||||
|
||||
Json GatewayPolicies::delete_operation(const std::string& operation_id) const {
|
||||
operation(operation_id);
|
||||
return runner_.invoke("networksecurity.v1.operations.delete",
|
||||
with("name", context_.parent() + "/operations/" + operation_id));
|
||||
}
|
||||
|
||||
} // namespace molo::google::registry
|
||||
|
||||
@@ -54,7 +54,8 @@ class GatewayPolicies : public RegistryService {
|
||||
|
||||
static ServiceEndpoint default_endpoint(); // networksecurity .../v1
|
||||
|
||||
clients::body::Json list() const;
|
||||
// Paged up to the registry caps; ListResult::truncated is loud.
|
||||
ListResult list() const;
|
||||
clients::body::Json get(const std::string& id) const;
|
||||
|
||||
// Explicit policy changes; each reads first and waits for the operation.
|
||||
@@ -68,7 +69,18 @@ class GatewayPolicies : public RegistryService {
|
||||
clients::body::Json remove_policy(const std::string& id, const std::string& request_id = "") const;
|
||||
|
||||
clients::body::Json iam_policy(const std::string& id) const;
|
||||
// setIamPolicy reads the policy first; the body is {"policy": policy}.
|
||||
clients::body::Json set_iam_policy(const std::string& id,
|
||||
const clients::body::Json& policy) const;
|
||||
clients::body::Json test_iam_permissions(const std::string& id,
|
||||
const std::vector<std::string>& permissions) const;
|
||||
|
||||
// ---- Operations of the networksecurity location (get, list, cancel, delete).
|
||||
clients::body::Json operation(const std::string& operation_id) const;
|
||||
ListResult operations() const;
|
||||
clients::body::Json cancel_operation(const std::string& operation_id) const;
|
||||
// Destructive: removes the operation record, never the policy.
|
||||
clients::body::Json delete_operation(const std::string& operation_id) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::registry
|
||||
|
||||
@@ -36,6 +36,23 @@ def rows():
|
||||
return out
|
||||
|
||||
|
||||
# Coverage: every discovery method whose id starts with one of these prefixes
|
||||
# must be a table row (or be listed in EXCLUDED with a reason). Narrow subsets
|
||||
# of aiplatform and logging are used on purpose and are not covered here.
|
||||
COVERED = [
|
||||
("agentregistry", "v1", "agentregistry."),
|
||||
("agentregistry", "v1alpha", "agentregistry."),
|
||||
("networkservices", "v1", "networkservices.projects.locations.agentGateways."),
|
||||
("networkservices", "v1", "networkservices.projects.locations.operations."),
|
||||
("networksecurity", "v1", "networksecurity.projects.locations.authzPolicies."),
|
||||
("networksecurity", "v1", "networksecurity.projects.locations.operations."),
|
||||
("iam", "v2", "iam."),
|
||||
("iam", "v3", "iam.organizations.locations.principalAccessBoundaryPolicies."),
|
||||
]
|
||||
# rpc id -> reason it is not wrapped (none today).
|
||||
EXCLUDED = {}
|
||||
|
||||
|
||||
def fetch(service, version, cache):
|
||||
name = "%s_%s.json" % (service, version)
|
||||
if cache:
|
||||
@@ -106,7 +123,20 @@ def main():
|
||||
for f in row["fields"].split(","):
|
||||
if f and f not in props:
|
||||
failures.append("%s: field %s not in schema %s" % (row["id"], f, req))
|
||||
print("checked %d calls across %d discovery documents" % (len(table), len(docs)))
|
||||
wrapped = {(r["service"], r["version"], r["rpc"]) for r in table}
|
||||
missing = 0
|
||||
for service, version, prefix in COVERED:
|
||||
if (service, version) not in docs:
|
||||
doc = fetch(service, version, cache)
|
||||
docs[(service, version)] = (doc, methods(doc))
|
||||
for rpc in sorted(docs[(service, version)][1]):
|
||||
if rpc.startswith(prefix) and (service, version, rpc) not in wrapped \
|
||||
and rpc not in EXCLUDED:
|
||||
failures.append("%s %s: discovery method %s is not in the table" %
|
||||
(service, version, rpc))
|
||||
missing += 1
|
||||
print("checked %d calls across %d discovery documents, %d discovery methods unwrapped" %
|
||||
(len(table), len(docs), missing))
|
||||
for f in failures:
|
||||
print("FAIL", f)
|
||||
if failures:
|
||||
|
||||
+59
-18
@@ -17,8 +17,16 @@ Gemini Enterprise Agent Platform, Agent Engine (`reasoningEngines`) children:
|
||||
create, get, list, patch, delete, purge (explicit, guarded), revisions
|
||||
(list, get), rollback, topics (managed and custom), scope maps, metadata.
|
||||
- Long-running operations: every mutating call returns an Operation; the module
|
||||
waits with `wait_operation` (hard caps in `OperationCaps`). `operation()`,
|
||||
`cancel_operation()` and `wait()` expose them for one engine only.
|
||||
waits with `wait_operation` (hard caps in `OperationCaps`). All five
|
||||
Discovery operations methods (get, list, cancel, delete, wait) are wrapped
|
||||
for the four places Discovery puts them: under the engine, under a session,
|
||||
under a memory and under a serving profile (20 table rows). `operation()`,
|
||||
`list_operations()`, `cancel_operation()`, `delete_operation()`, `wait()`
|
||||
(client polling) and `wait_server()` (one POST `:wait?timeout=Ns`, 1..60 s)
|
||||
work for one root only: the engine, or the location for serving profiles.
|
||||
- Serving profiles (`projects.locations.servingProfiles`, location level, not
|
||||
engine scoped): create (waits), get, list, patch, delete, with scope
|
||||
`GEMINI_LIVE`, `INTERACTIONS_API` or `RESPONSE_API`, optional CMEK key.
|
||||
|
||||
Everything is against one API surface: `aiplatform` **v1beta1**. See section 6
|
||||
for why not v1.
|
||||
@@ -29,15 +37,18 @@ for why not v1.
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings (`SessionsCaps`, `MemoryCaps`) |
|
||||
| `calls.h` | The ONE table of every REST call (verb, path, discovery method, request fields, query params) |
|
||||
| `names.h/.cpp` | `EngineName` (validated resource name), `Check` (input validators), wire helpers |
|
||||
| `names.h/.cpp` | `EngineName`, `LocationName` (validated resource names), `Check` (input validators), wire helpers |
|
||||
| `operation_names.h/.cpp` | `OperationName`: validates an operation name against its root, picks the table row for each verb by owner (engine, session, memory, serving profile) |
|
||||
| `operation_service.h/.cpp` | `OperationService : Service`: table-driven `send`, `settle`, loud capped `list_items` (`list_over_limit`), the operations surface |
|
||||
| `serving_profile.h/.cpp` | `ServingProfileSpec`, `ServingProfilePatch`, `ServingProfile`, `ServingScope`, `ServingProfiles : OperationService` |
|
||||
| `session_types.h/.cpp` | `SessionSpec`, `Session`, `SessionPatch`, `CompactRequest`, `SessionEvent`, list options |
|
||||
| `memory_types.h/.cpp` | `Memory`, `MemorySpec`, `MemoryPatch`, `Topic`, `MetadataValue`, `MetadataFilter`, `MemoryGenerateRequest`, `IngestRequest`, `RetrieveRequest`, `PurgeRequest`, `MemoryRevision`, `Profile` |
|
||||
| `engine_service.h/.cpp` | `EngineService : Service`: table-driven `send`, `settle`, capped `list_items`, operation helpers |
|
||||
| `engine_service.h/.cpp` | `EngineService : OperationService`: one validated engine and its v1beta1 endpoint |
|
||||
| `sessions.h/.cpp` | `Sessions : EngineService` |
|
||||
| `memory_bank.h/.cpp` | `MemoryBank : EngineService` |
|
||||
| `tools/check_discovery.py` | Verifies `calls.h` and the typed bodies against Google's Discovery document |
|
||||
|
||||
Tests: `tests/src/google/sessions/test_sessions.cpp`. Example:
|
||||
Tests: `tests/src/google/sessions/test_sessions.cpp` and `test_sessions_gaps.cpp`. Example:
|
||||
`example/app/agent/SessionsDemo.cpp` (routes under `/agent/sessions`), tests in
|
||||
`tests/example/agent/sessions/`.
|
||||
|
||||
@@ -50,7 +61,9 @@ Tests: `tests/src/google/sessions/test_sessions.cpp`. Example:
|
||||
| Session TTL | `Ns` form, 24 h minimum (Google), 3650 days maximum (ours) |
|
||||
| session_state / event | 64 KiB / 256 KiB serialized |
|
||||
| Labels | 20, key and value 64 chars |
|
||||
| List pages / items | 10 pages of 100, 1000 items; more is `RESULT_TOO_LARGE`, narrow the filter |
|
||||
| List pages / items | 10 pages of 100, 1000 items; more items OR a further page after the last allowed one is `RESULT_TOO_LARGE` (502), never a silent cut, narrow the filter |
|
||||
| Server-side operation wait | 1..60 s per call (`kMaxServerWaitSeconds`) |
|
||||
| Serving profile | id 1..63 `[a-z0-9-]`, display name required 128, description 4000, KMS key name 512 |
|
||||
| Scope | 1..10 entries, 128 chars, no `*` |
|
||||
| Fact / description / context | 4000 chars |
|
||||
| Generate sources | 100 events, 64 KiB; direct memories max 5 (Google) |
|
||||
@@ -83,10 +96,14 @@ Tests: `tests/src/google/sessions/test_sessions.cpp`. Example:
|
||||
credential) and fails if a declared call, verb, path, request field, query
|
||||
parameter, enum value, response key or nested request field is absent.
|
||||
|
||||
Result 2026-10-24: `OK 2026-10-24: 23 calls, 7 enums, 25 response keys, 18
|
||||
nested schemas match aiplatform v1beta1 discovery`. A deliberately misspelled
|
||||
field made it exit 1 (checked). Rerun it after any change to `calls.h` or the
|
||||
typed bodies.
|
||||
Result 2026-10-24: `OK 2026-10-24: 46 calls, 8 enums, 34 response keys, 20
|
||||
nested schemas match aiplatform v1beta1 discovery`. The script also checks
|
||||
coverage: every Discovery method under `reasoningEngines.operations`,
|
||||
`reasoningEngines.sessions`, `reasoningEngines.memories` and `servingProfiles`
|
||||
must be a table row or listed in `EXCLUDED` (empty: nothing is excluded), so a
|
||||
method Google adds later fails the check. A deliberately misspelled field made
|
||||
it exit 1 (checked). Rerun it after any change to `calls.h` or the typed
|
||||
bodies.
|
||||
|
||||
From discovery (v1beta1, `projects.locations.reasoningEngines`):
|
||||
sessions create (POST, `sessionId` query, returns Operation), get, list
|
||||
@@ -95,9 +112,12 @@ Operation), appendEvent (POST `:appendEvent`, body `SessionEvent`), compact
|
||||
(POST `:compact`, returns Operation), events.list (`sessionEvents`);
|
||||
memories generate / retrieve / retrieveProfiles / create (`memoryId`) / get /
|
||||
list / patch (`updateMask`) / delete / purge / rollback / ingestEvents,
|
||||
memories.revisions list / get; operations get / cancel. Discovery also lists
|
||||
`wait`, `list`, `delete` on operations, which this module does not wrap
|
||||
(`wait_operation` polls with GET).
|
||||
memories.revisions list / get; operations get / cancel / list / delete / wait
|
||||
under the engine, `sessions.operations.*` and `memories.operations.*` (same
|
||||
five), `locations.servingProfiles` create (`servingProfileId` query, returns
|
||||
Operation) / get / list / patch (`updateMask`) / delete (returns Empty) and
|
||||
`servingProfiles.operations.*`. The Method column of a `calls.h` row is
|
||||
relative to `kDiscoveryPrefix` unless it starts with `aiplatform.`.
|
||||
|
||||
## 6. Verified live
|
||||
|
||||
@@ -127,6 +147,20 @@ an express key and a token; v1 refuses keys. The opt-in test `sessions_live`
|
||||
the real client (sessions list / get / events, memories list / get /
|
||||
revisions) and passed with the key and with the token on 2026-10-24.
|
||||
|
||||
Gap-closing round, same day, service-account token, read-only:
|
||||
|
||||
| Call | Result |
|
||||
|---|---|
|
||||
| GET `v1beta1/.../reasoningEngines/1/operations?pageSize=5`, `.../sessions/abc/operations`, `.../memories/abc/operations` | 200 `{}` (empty list, even though engine 1 does not exist) |
|
||||
| GET `v1beta1/.../locations/us-central1/servingProfiles?pageSize=5` (also `global` host and `europe-west4`) | 501 `UNIMPLEMENTED` "Operation is not implemented, or supported, or enabled." (not available to this project) |
|
||||
| GET `v1beta1/.../servingProfiles/probe` | 501 UNIMPLEMENTED |
|
||||
| GET `v1beta1/.../servingProfiles/probe/operations` | 400 INVALID_ARGUMENT: "the idType servingProfiles is not a documented LRO Parent ID, the associated value must be a Long, but was instead: probe" |
|
||||
| GET `v1beta1/.../servingProfiles/123/operations?pageSize=1` | 200 `{}` |
|
||||
|
||||
The opt-in test `sessions_live.operations_and_serving_profiles_reads` repeats
|
||||
the operations lists and the servingProfiles list through the real client; it
|
||||
accepts 200 or 501/403/404 for servingProfiles and prints which one it saw.
|
||||
|
||||
No Agent Engine exists in the project, so **no happy path was verified
|
||||
live**: create, patch, delete, append, compact, generate, ingest, retrieve,
|
||||
retrieveProfiles, purge, rollback are verified by discovery only. Nothing that
|
||||
@@ -151,18 +185,25 @@ Neither discovery nor the docs pages read state these; they are inferred:
|
||||
search model (docs); the error body when it is not is unread.
|
||||
- Memory Profiles need schema configs on the engine (`MemoryBankConfig` in the
|
||||
runtime module); the shape of `profile` is free-form and kept raw.
|
||||
- Serving profile id charset (`[a-z0-9-]`, 1..63 is our choice) and the real
|
||||
id rule: Google's LRO layer wants a numeric id for operations under a
|
||||
serving profile (400 above), so Google may assign or require numeric ids.
|
||||
The create/patch responses (Operation `response` is `Any`) are parsed
|
||||
tolerantly with a GET fallback. `servingProfiles` itself answers 501 on this
|
||||
project, so no body of it was ever seen.
|
||||
- Whether `operations.wait` answers `done: false` at the timeout or an error
|
||||
(the docs line says "at most the time permitted"): both are handled, a
|
||||
not-done answer is returned to the caller, not retried.
|
||||
- Managed topic names beyond the four in discovery, and `thought`/`audio` parts
|
||||
inside event `content` (typed by `gemini_types.h`, unknown parts dropped).
|
||||
|
||||
## 8. Gaps
|
||||
|
||||
- `servingProfiles` (location-level create/get/list/patch for Gemini Live,
|
||||
Interactions and Response APIs) is listed by discovery next to this product
|
||||
but is not engine-scoped; not implemented here. Owner: the `live` or
|
||||
`models` module.
|
||||
- Engine-level Memory Bank configuration (`contextSpec.memoryBankConfig`,
|
||||
customization, TTL config) is part of the Agent Engine resource: runtime
|
||||
module.
|
||||
- Operations `list`, `delete`, `wait` (server-side) are not wrapped.
|
||||
- `servingProfiles` answers 501 for the test project in three locations, so
|
||||
the happy path is Discovery-verified only. Owner: a project that has the
|
||||
feature; test `sessions_live.operations_and_serving_profiles_reads`.
|
||||
- Session and memory `labels` / `metadata` filters beyond a passthrough string.
|
||||
- Token refresh is the caller's job (foundation gap).
|
||||
|
||||
@@ -9,6 +9,9 @@
|
||||
// "API keys are not supported" for sessions and memories, v1beta1 accepts an
|
||||
// express API key (404 for a missing engine). See SPEC.md.
|
||||
//
|
||||
// Method column: relative to kDiscoveryPrefix, except rows that start with
|
||||
// "aiplatform." (location-level resources such as servingProfiles) which are
|
||||
// absolute Discovery method ids.
|
||||
// Row: {Call, verb, path template, discovery method (under
|
||||
// aiplatform.projects.locations.reasoningEngines.), request schema suffix
|
||||
// (after GoogleCloudAiplatformV1beta1), request fields used, query params used}.
|
||||
@@ -30,7 +33,16 @@ enum class Call {
|
||||
kMemoryGenerate, kMemoryRetrieve, kMemoryRetrieveProfiles, kMemoryCreate,
|
||||
kMemoryGet, kMemoryList, kMemoryPatch, kMemoryDelete, kMemoryPurge,
|
||||
kMemoryRollback, kMemoryIngestEvents, kRevisionList, kRevisionGet,
|
||||
kOperationGet, kOperationCancel,
|
||||
kOperationGet, kOperationCancel, kOperationList, kOperationDelete, kOperationWait,
|
||||
kSessionOperationGet, kSessionOperationCancel, kSessionOperationList,
|
||||
kSessionOperationDelete, kSessionOperationWait,
|
||||
kMemoryOperationGet, kMemoryOperationCancel, kMemoryOperationList,
|
||||
kMemoryOperationDelete, kMemoryOperationWait,
|
||||
kServingProfileCreate, kServingProfileGet, kServingProfileList,
|
||||
kServingProfilePatch, kServingProfileDelete,
|
||||
kServingProfileOperationGet, kServingProfileOperationCancel,
|
||||
kServingProfileOperationList, kServingProfileOperationDelete,
|
||||
kServingProfileOperationWait,
|
||||
kCount
|
||||
};
|
||||
|
||||
@@ -69,6 +81,29 @@ inline constexpr CallSpec kCalls[] = {
|
||||
{Call::kRevisionGet, "GET", "/{name}", "memories.revisions.get", "", "", ""},
|
||||
{Call::kOperationGet, "GET", "/{name}", "operations.get", "", "", ""},
|
||||
{Call::kOperationCancel, "POST", "/{name}:cancel", "operations.cancel", "", "", ""},
|
||||
{Call::kOperationList, "GET", "/{name}/operations", "operations.list", "", "", "filter pageSize pageToken"},
|
||||
{Call::kOperationDelete, "DELETE", "/{name}", "operations.delete", "", "", ""},
|
||||
{Call::kOperationWait, "POST", "/{name}:wait", "operations.wait", "", "", "timeout"},
|
||||
{Call::kSessionOperationGet, "GET", "/{name}", "sessions.operations.get", "", "", ""},
|
||||
{Call::kSessionOperationCancel, "POST", "/{name}:cancel", "sessions.operations.cancel", "", "", ""},
|
||||
{Call::kSessionOperationList, "GET", "/{name}/operations", "sessions.operations.list", "", "", "filter pageSize pageToken"},
|
||||
{Call::kSessionOperationDelete, "DELETE", "/{name}", "sessions.operations.delete", "", "", ""},
|
||||
{Call::kSessionOperationWait, "POST", "/{name}:wait", "sessions.operations.wait", "", "", "timeout"},
|
||||
{Call::kMemoryOperationGet, "GET", "/{name}", "memories.operations.get", "", "", ""},
|
||||
{Call::kMemoryOperationCancel, "POST", "/{name}:cancel", "memories.operations.cancel", "", "", ""},
|
||||
{Call::kMemoryOperationList, "GET", "/{name}/operations", "memories.operations.list", "", "", "filter pageSize pageToken"},
|
||||
{Call::kMemoryOperationDelete, "DELETE", "/{name}", "memories.operations.delete", "", "", ""},
|
||||
{Call::kMemoryOperationWait, "POST", "/{name}:wait", "memories.operations.wait", "", "", "timeout"},
|
||||
{Call::kServingProfileCreate, "POST", "/{parent}/servingProfiles", "aiplatform.projects.locations.servingProfiles.create", "ServingProfile", "displayName description scope cmekConfig", "servingProfileId"},
|
||||
{Call::kServingProfileGet, "GET", "/{name}", "aiplatform.projects.locations.servingProfiles.get", "", "", ""},
|
||||
{Call::kServingProfileList, "GET", "/{parent}/servingProfiles", "aiplatform.projects.locations.servingProfiles.list", "", "", "pageSize pageToken"},
|
||||
{Call::kServingProfilePatch, "PATCH", "/{name}", "aiplatform.projects.locations.servingProfiles.patch", "ServingProfile", "displayName description cmekConfig", "updateMask"},
|
||||
{Call::kServingProfileDelete, "DELETE", "/{name}", "aiplatform.projects.locations.servingProfiles.delete", "", "", ""},
|
||||
{Call::kServingProfileOperationGet, "GET", "/{name}", "aiplatform.projects.locations.servingProfiles.operations.get", "", "", ""},
|
||||
{Call::kServingProfileOperationCancel, "POST", "/{name}:cancel", "aiplatform.projects.locations.servingProfiles.operations.cancel", "", "", ""},
|
||||
{Call::kServingProfileOperationList, "GET", "/{name}/operations", "aiplatform.projects.locations.servingProfiles.operations.list", "", "", "filter pageSize pageToken"},
|
||||
{Call::kServingProfileOperationDelete, "DELETE", "/{name}", "aiplatform.projects.locations.servingProfiles.operations.delete", "", "", ""},
|
||||
{Call::kServingProfileOperationWait, "POST", "/{name}:wait", "aiplatform.projects.locations.servingProfiles.operations.wait", "", "", "timeout"},
|
||||
};
|
||||
// clang-format on
|
||||
|
||||
|
||||
@@ -22,6 +22,9 @@ struct SessionsCaps {
|
||||
static constexpr int kPageSize = 100; // Google ceiling
|
||||
static constexpr std::size_t kMaxListPages = 10;
|
||||
static constexpr std::size_t kMaxListItems = 1000;
|
||||
static constexpr std::size_t kMaxServerWaitSeconds = 60; // one POST :wait
|
||||
static constexpr std::size_t kMaxDescriptionChars = 4000;
|
||||
static constexpr std::size_t kMaxKmsKeyChars = 512;
|
||||
};
|
||||
|
||||
struct MemoryCaps {
|
||||
|
||||
@@ -4,70 +4,10 @@
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
EngineService::EngineService(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper)
|
||||
: Service(std::move(client), endpoints::aiplatform_beta(engine.location())),
|
||||
engine_name_(std::move(engine)),
|
||||
sleeper_(std::move(sleeper)) {}
|
||||
|
||||
Json EngineService::send(Call call, const std::string& resource, const Json* body,
|
||||
const std::string& query) const {
|
||||
const CallSpec& row = spec_of(call);
|
||||
return client_->request(verb_of(row), endpoint_, expand(call, resource, query), body);
|
||||
}
|
||||
|
||||
Operation EngineService::settle(const Json& reply) const {
|
||||
Operation op = Operation::from_json(reply);
|
||||
if (op.done) {
|
||||
if (op.failed())
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, endpoint_.name);
|
||||
return op;
|
||||
}
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "no operation returned", endpoint_.name);
|
||||
return wait(op.name);
|
||||
}
|
||||
|
||||
namespace {
|
||||
void check_operation_name(const EngineName& engine, const std::string& name) {
|
||||
const std::string prefix = engine.str() + "/";
|
||||
bool ok = name.size() > prefix.size() && name.compare(0, prefix.size(), prefix) == 0 &&
|
||||
name.find("/operations/") != std::string::npos &&
|
||||
name.find("..") == std::string::npos && name.find('?') == std::string::npos &&
|
||||
name.size() <= 512;
|
||||
if (!ok) throw GoogleError::invalid("operation name must belong to this engine");
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json EngineService::operation(const std::string& operation_name) const {
|
||||
check_operation_name(engine_name_, operation_name);
|
||||
return send(Call::kOperationGet, operation_name);
|
||||
}
|
||||
|
||||
void EngineService::cancel_operation(const std::string& operation_name) const {
|
||||
check_operation_name(engine_name_, operation_name);
|
||||
Json empty = Json::object();
|
||||
send(Call::kOperationCancel, operation_name, &empty);
|
||||
}
|
||||
|
||||
Operation EngineService::wait(const std::string& operation_name) const {
|
||||
check_operation_name(engine_name_, operation_name);
|
||||
return wait_operation(*client_, endpoint_, operation_name, sleeper_);
|
||||
}
|
||||
|
||||
Json EngineService::list_items(Call call, const std::string& resource,
|
||||
const std::string& query, const std::string& items_key,
|
||||
std::size_t max_pages, std::size_t max_items) const {
|
||||
Json items = list_all(*client_, endpoint_, expand(call, resource, query), items_key,
|
||||
max_pages);
|
||||
if (items.as_array().size() > max_items)
|
||||
throw GoogleError(502, "RESULT_TOO_LARGE",
|
||||
"more than " + std::to_string(max_items) +
|
||||
" items; narrow the filter",
|
||||
endpoint_.name);
|
||||
return items;
|
||||
}
|
||||
: OperationService(std::move(client), endpoints::aiplatform_beta(engine.location()),
|
||||
engine.str(), std::move(sleeper)),
|
||||
engine_name_(std::move(engine)) {}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
|
||||
@@ -1,48 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
// Base of Sessions and MemoryBank: one validated Agent Engine, the v1beta1
|
||||
// aiplatform endpoint of its location, table-driven requests (calls.h) and
|
||||
// long-running operation handling shared by both.
|
||||
// Base of Sessions and MemoryBank: one validated Agent Engine and the v1beta1
|
||||
// aiplatform endpoint of its location. Table-driven requests and the
|
||||
// operations surface come from OperationService.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/service.h"
|
||||
#include "google/sessions/calls.h"
|
||||
#include "google/sessions/names.h"
|
||||
#include "google/sessions/operation_service.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
class EngineService : public Service {
|
||||
class EngineService : public OperationService {
|
||||
public:
|
||||
const EngineName& engine() const { return engine_name_; }
|
||||
|
||||
// Raw operation document (GET) and cancel (POST :cancel). The name must be
|
||||
// an operation of this engine; nothing else is reachable through here.
|
||||
clients::body::Json operation(const std::string& operation_name) const;
|
||||
void cancel_operation(const std::string& operation_name) const;
|
||||
// Polls until done (hard caps in OperationCaps); throws on failure.
|
||||
Operation wait(const std::string& operation_name) const;
|
||||
|
||||
protected:
|
||||
EngineService(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper);
|
||||
|
||||
// One table-driven request. `resource` fills the {parent} / {name} slot.
|
||||
clients::body::Json send(Call call, const std::string& resource,
|
||||
const clients::body::Json* body = nullptr,
|
||||
const std::string& query = "") const;
|
||||
// Follows an Operation reply to completion (done replies are checked).
|
||||
Operation settle(const clients::body::Json& reply) const;
|
||||
// Paged list through the table; caps pages and total items.
|
||||
clients::body::Json list_items(Call call, const std::string& resource,
|
||||
const std::string& query,
|
||||
const std::string& items_key,
|
||||
std::size_t max_pages, std::size_t max_items) const;
|
||||
|
||||
EngineName engine_name_;
|
||||
Sleeper sleeper_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
|
||||
@@ -175,4 +175,9 @@ void MemoryBank::rollback(const std::string& memory_id,
|
||||
settle(send(Call::kMemoryRollback, engine_name_.memory(memory_id), &body));
|
||||
}
|
||||
|
||||
std::vector<Operation> MemoryBank::list_memory_operations(
|
||||
const std::string& memory_id, const OperationListOptions& options) const {
|
||||
return list_operations(engine_name_.memory(memory_id), options);
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
|
||||
@@ -41,6 +41,10 @@ class MemoryBank : public EngineService {
|
||||
MemoryRevision get_revision(const std::string& memory_id,
|
||||
const std::string& revision_id) const;
|
||||
void rollback(const std::string& memory_id, const std::string& revision_id) const;
|
||||
|
||||
// Operations that belong to one memory (update, rollback runs).
|
||||
std::vector<Operation> list_memory_operations(
|
||||
const std::string& memory_id, const OperationListOptions& options = {}) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
|
||||
@@ -59,6 +59,35 @@ EngineName::EngineName(std::string full) : full_(std::move(full)) {
|
||||
if (!valid_segment(engine_)) bad("engine name: bad engine id");
|
||||
}
|
||||
|
||||
LocationName::LocationName(std::string full) : full_(std::move(full)) {
|
||||
if (full_.empty() || full_.size() > SessionsCaps::kMaxNameChars)
|
||||
bad("location name missing or too long");
|
||||
std::vector<std::string> parts;
|
||||
std::string cur;
|
||||
for (char c : full_) {
|
||||
if (c == '/') {
|
||||
parts.push_back(cur);
|
||||
cur.clear();
|
||||
} else {
|
||||
cur.push_back(c);
|
||||
}
|
||||
}
|
||||
parts.push_back(cur);
|
||||
if (parts.size() != 4 || parts[0] != "projects" || parts[2] != "locations")
|
||||
bad("location must look like projects/{p}/locations/{l}");
|
||||
project_ = parts[1];
|
||||
location_ = parts[3];
|
||||
if (!project_ok(project_)) bad("location name: bad project");
|
||||
if (location_.empty() || location_.size() > 64) bad("location name: bad location");
|
||||
for (char c : location_)
|
||||
if (!id_char(c)) bad("location name: bad location");
|
||||
}
|
||||
|
||||
std::string LocationName::serving_profile(const std::string& profile_id) const {
|
||||
Check::serving_profile_id(profile_id);
|
||||
return full_ + "/servingProfiles/" + profile_id;
|
||||
}
|
||||
|
||||
std::string EngineName::session(const std::string& session_id) const {
|
||||
Check::session_id(session_id);
|
||||
return full_ + "/sessions/" + session_id;
|
||||
@@ -77,6 +106,14 @@ void Check::session_id(const std::string& id) {
|
||||
if (!id_char(c)) bad("session id allows only a-z 0-9 -");
|
||||
}
|
||||
|
||||
void Check::serving_profile_id(const std::string& id) {
|
||||
if (id.empty() || id.size() > SessionsCaps::kMaxIdChars)
|
||||
bad("serving profile id must be 1.." + std::to_string(SessionsCaps::kMaxIdChars) +
|
||||
" characters");
|
||||
for (char c : id)
|
||||
if (!id_char(c)) bad("serving profile id allows only a-z 0-9 -");
|
||||
}
|
||||
|
||||
void Check::memory_id(const std::string& id) {
|
||||
if (id.empty() || id.size() > SessionsCaps::kMaxIdChars)
|
||||
bad("memory id must be 1.." + std::to_string(SessionsCaps::kMaxIdChars) +
|
||||
|
||||
@@ -29,6 +29,22 @@ class EngineName {
|
||||
std::string full_, project_, location_, engine_;
|
||||
};
|
||||
|
||||
// "projects/{project}/locations/{location}": the root of location-level
|
||||
// resources (serving profiles).
|
||||
class LocationName {
|
||||
public:
|
||||
explicit LocationName(std::string full); // throws when malformed
|
||||
|
||||
const std::string& str() const { return full_; }
|
||||
const std::string& project() const { return project_; }
|
||||
const std::string& location() const { return location_; }
|
||||
|
||||
std::string serving_profile(const std::string& profile_id) const;
|
||||
|
||||
private:
|
||||
std::string full_, project_, location_;
|
||||
};
|
||||
|
||||
using StringMap = std::map<std::string, std::string>;
|
||||
|
||||
// Input validators.
|
||||
@@ -36,6 +52,9 @@ class Check {
|
||||
public:
|
||||
// Session id: 1..63 of [a-z0-9-].
|
||||
static void session_id(const std::string& id);
|
||||
// Serving profile id: same charset as a session id (Google's exact rule is
|
||||
// not published, see SPEC Unverified).
|
||||
static void serving_profile_id(const std::string& id);
|
||||
// Memory id: 1..63 of [a-z0-9-], starts with a letter, ends alphanumeric.
|
||||
static void memory_id(const std::string& id);
|
||||
static void text(const std::string& field, const std::string& value,
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
#include "google/sessions/operation_names.h"
|
||||
|
||||
#include <vector>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/resource.h"
|
||||
#include "google/sessions/caps.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& message) { throw GoogleError::invalid(message); }
|
||||
|
||||
std::vector<std::string> parts_of(const std::string& s) {
|
||||
std::vector<std::string> out;
|
||||
std::string cur;
|
||||
for (char c : s) {
|
||||
if (c == '/') {
|
||||
out.push_back(cur);
|
||||
cur.clear();
|
||||
} else {
|
||||
cur.push_back(c);
|
||||
}
|
||||
}
|
||||
out.push_back(cur);
|
||||
return out;
|
||||
}
|
||||
|
||||
void check_root_prefix(const std::string& root, const std::string& name) {
|
||||
if (name.size() > SessionsCaps::kMaxNameChars) bad("operation name too long");
|
||||
if (name.size() <= root.size() + 1 || name.compare(0, root.size() + 1, root + "/") != 0)
|
||||
bad("operation name must belong to " + root);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
OperationName OperationName::classify(const std::string& root, const std::string& rest,
|
||||
const std::string& full, bool want_operation) {
|
||||
bool location_root = root.find("/reasoningEngines/") == std::string::npos;
|
||||
std::vector<std::string> p;
|
||||
if (!rest.empty()) p = parts_of(rest);
|
||||
for (const std::string& seg : p)
|
||||
if (!valid_segment(seg)) bad("operation name has an empty or invalid segment");
|
||||
if (want_operation) {
|
||||
std::size_t n = p.size();
|
||||
if (n < 2 || p[n - 2] != "operations") bad("not an operation name");
|
||||
p.resize(n - 2);
|
||||
}
|
||||
std::size_t n = p.size();
|
||||
if (n == 0) {
|
||||
if (location_root) bad("a location has no operations here; name a serving profile");
|
||||
return OperationName(full, OperationOwner::kEngine);
|
||||
}
|
||||
if (n == 2 && !location_root && p[0] == "sessions")
|
||||
return OperationName(full, OperationOwner::kSession);
|
||||
if (n == 2 && !location_root && p[0] == "memories")
|
||||
return OperationName(full, OperationOwner::kMemory);
|
||||
if (n == 2 && location_root && p[0] == "servingProfiles")
|
||||
return OperationName(full, OperationOwner::kServingProfile);
|
||||
bad("operation parent must be the engine, a session, a memory or a serving profile");
|
||||
}
|
||||
|
||||
OperationName OperationName::parse(const std::string& root, const std::string& name) {
|
||||
check_root_prefix(root, name);
|
||||
return classify(root, name.substr(root.size() + 1), name, true);
|
||||
}
|
||||
|
||||
OperationName OperationName::parse_parent(const std::string& root,
|
||||
const std::string& parent) {
|
||||
if (parent == root) return classify(root, "", parent, false);
|
||||
check_root_prefix(root, parent);
|
||||
return classify(root, parent.substr(root.size() + 1), parent, false);
|
||||
}
|
||||
|
||||
Call OperationName::call(OperationVerb verb) const {
|
||||
static constexpr Call kTable[4][5] = {
|
||||
{Call::kOperationGet, Call::kOperationCancel, Call::kOperationList,
|
||||
Call::kOperationDelete, Call::kOperationWait},
|
||||
{Call::kSessionOperationGet, Call::kSessionOperationCancel,
|
||||
Call::kSessionOperationList, Call::kSessionOperationDelete,
|
||||
Call::kSessionOperationWait},
|
||||
{Call::kMemoryOperationGet, Call::kMemoryOperationCancel,
|
||||
Call::kMemoryOperationList, Call::kMemoryOperationDelete,
|
||||
Call::kMemoryOperationWait},
|
||||
{Call::kServingProfileOperationGet, Call::kServingProfileOperationCancel,
|
||||
Call::kServingProfileOperationList, Call::kServingProfileOperationDelete,
|
||||
Call::kServingProfileOperationWait}};
|
||||
return kTable[static_cast<int>(owner_)][static_cast<int>(verb)];
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,44 @@
|
||||
#pragma once
|
||||
|
||||
// A validated long-running operation name and the table row that serves each
|
||||
// verb for it. Discovery lists the same five operations methods (get, list,
|
||||
// cancel, delete, wait) under the Agent Engine, under each session, under
|
||||
// each memory and under each serving profile; the owner is read from the
|
||||
// name so a caller can never reach an operation outside the root it passed.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "google/sessions/calls.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
enum class OperationOwner { kEngine, kSession, kMemory, kServingProfile };
|
||||
enum class OperationVerb { kGet, kCancel, kList, kDelete, kWait };
|
||||
|
||||
class OperationName {
|
||||
public:
|
||||
// `root` is an engine name (owners engine, session, memory) or a location
|
||||
// name "projects/{p}/locations/{l}" (owner serving profile). `name` must be
|
||||
// "{root}[/{sessions|memories|servingProfiles}/{id}]/operations/{op}".
|
||||
// Throws GoogleError::invalid otherwise.
|
||||
static OperationName parse(const std::string& root, const std::string& name);
|
||||
// A list parent: "{root}", "{root}/sessions/{id}", "{root}/memories/{id}" or
|
||||
// "{root}/servingProfiles/{id}". Same validation, no trailing operation.
|
||||
static OperationName parse_parent(const std::string& root, const std::string& parent);
|
||||
|
||||
const std::string& str() const { return full_; }
|
||||
OperationOwner owner() const { return owner_; }
|
||||
// The table row serving a verb for this owner.
|
||||
Call call(OperationVerb verb) const;
|
||||
|
||||
private:
|
||||
OperationName(std::string full, OperationOwner owner)
|
||||
: full_(std::move(full)), owner_(owner) {}
|
||||
static OperationName classify(const std::string& root, const std::string& rest,
|
||||
const std::string& full, bool want_operation);
|
||||
|
||||
std::string full_;
|
||||
OperationOwner owner_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,123 @@
|
||||
#include "google/sessions/operation_service.h"
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "google/errors.h"
|
||||
#include "google/sessions/caps.h"
|
||||
#include "google/sessions/names.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
OperationService::OperationService(std::shared_ptr<GoogleClient> client,
|
||||
ServiceEndpoint endpoint, std::string root,
|
||||
Sleeper sleeper)
|
||||
: Service(std::move(client), std::move(endpoint)),
|
||||
root_(std::move(root)),
|
||||
sleeper_(std::move(sleeper)) {}
|
||||
|
||||
Json OperationService::send(Call call, const std::string& resource, const Json* body,
|
||||
const std::string& query) const {
|
||||
const CallSpec& row = spec_of(call);
|
||||
return client_->request(verb_of(row), endpoint_, expand(call, resource, query), body);
|
||||
}
|
||||
|
||||
Operation OperationService::settle(const Json& reply) const {
|
||||
Operation op = Operation::from_json(reply);
|
||||
if (op.done) {
|
||||
if (op.failed())
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, endpoint_.name);
|
||||
return op;
|
||||
}
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "no operation returned", endpoint_.name);
|
||||
return wait(op.name);
|
||||
}
|
||||
|
||||
bool list_over_limit(std::size_t item_count, bool more_pages, std::size_t page_index,
|
||||
std::size_t max_pages, std::size_t max_items) {
|
||||
return item_count > max_items || (more_pages && page_index + 1 >= max_pages);
|
||||
}
|
||||
|
||||
Json OperationService::operation(const std::string& operation_name) const {
|
||||
OperationName name = OperationName::parse(root_, operation_name);
|
||||
return send(name.call(OperationVerb::kGet), name.str());
|
||||
}
|
||||
|
||||
void OperationService::cancel_operation(const std::string& operation_name) const {
|
||||
OperationName name = OperationName::parse(root_, operation_name);
|
||||
Json empty = Json::object();
|
||||
send(name.call(OperationVerb::kCancel), name.str(), &empty);
|
||||
}
|
||||
|
||||
void OperationService::delete_operation(const std::string& operation_name) const {
|
||||
OperationName name = OperationName::parse(root_, operation_name);
|
||||
send(name.call(OperationVerb::kDelete), name.str());
|
||||
}
|
||||
|
||||
Operation OperationService::wait(const std::string& operation_name) const {
|
||||
OperationName name = OperationName::parse(root_, operation_name);
|
||||
return wait_operation(*client_, endpoint_, name.str(), sleeper_);
|
||||
}
|
||||
|
||||
Operation OperationService::wait_server(const std::string& operation_name,
|
||||
int timeout_seconds) const {
|
||||
OperationName name = OperationName::parse(root_, operation_name);
|
||||
if (timeout_seconds < 1 ||
|
||||
timeout_seconds > static_cast<int>(SessionsCaps::kMaxServerWaitSeconds))
|
||||
throw GoogleError::invalid("timeout_seconds must be 1.." +
|
||||
std::to_string(SessionsCaps::kMaxServerWaitSeconds));
|
||||
Json empty = Json::object();
|
||||
Operation op = Operation::from_json(
|
||||
send(name.call(OperationVerb::kWait), name.str(), &empty,
|
||||
query_of({{"timeout", std::to_string(timeout_seconds) + "s"}})));
|
||||
if (op.done && op.failed())
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, endpoint_.name);
|
||||
return op;
|
||||
}
|
||||
|
||||
std::vector<Operation> OperationService::list_operations(
|
||||
const std::string& parent, const OperationListOptions& options) const {
|
||||
OperationName where = OperationName::parse_parent(root_, parent.empty() ? root_ : parent);
|
||||
Check::expression("filter", options.filter);
|
||||
Check::page_size(options.page_size, SessionsCaps::kPageSize);
|
||||
Json items = list_items(
|
||||
where.call(OperationVerb::kList), where.str(),
|
||||
query_of({{"filter", options.filter},
|
||||
{"pageSize", options.page_size > 0 ? std::to_string(options.page_size) : ""}}),
|
||||
"operations", SessionsCaps::kMaxListPages, SessionsCaps::kMaxListItems);
|
||||
std::vector<Operation> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(Operation::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
Json OperationService::list_items(Call call, const std::string& resource,
|
||||
const std::string& query, const std::string& items_key,
|
||||
std::size_t max_pages, std::size_t max_items) const {
|
||||
// Own loop instead of list_all: list_all stops silently at max_pages, and a
|
||||
// short page size would hide that. Anything left over is a loud error.
|
||||
std::string path = expand(call, resource, query);
|
||||
Json items = Json::array();
|
||||
std::string token;
|
||||
for (std::size_t page = 0;; ++page) {
|
||||
std::string url = path;
|
||||
if (!token.empty())
|
||||
url += (url.find('?') == std::string::npos ? "?" : "&") +
|
||||
std::string("pageToken=") + clients::Url::encode_component(token);
|
||||
Json reply = client_->request(clients::Method::kGet, endpoint_, url, nullptr);
|
||||
if (const Json* arr = reply.is_object() ? reply.find(items_key) : nullptr;
|
||||
arr && arr->is_array())
|
||||
for (const Json& item : arr->as_array()) items.push(item);
|
||||
const Json* next = reply.is_object() ? reply.find("nextPageToken") : nullptr;
|
||||
token = next && next->is_string() ? next->as_string() : "";
|
||||
if (list_over_limit(items.as_array().size(), !token.empty(), page, max_pages,
|
||||
max_items))
|
||||
throw GoogleError(502, "RESULT_TOO_LARGE",
|
||||
"more than " + std::to_string(max_items) + " items or " +
|
||||
std::to_string(max_pages) + " pages; narrow the filter",
|
||||
endpoint_.name);
|
||||
if (token.empty()) return items;
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,72 @@
|
||||
#pragma once
|
||||
|
||||
// Table-driven requests and the whole operations surface (get, list, cancel,
|
||||
// delete, wait on the server, wait by polling) for one validated root: an
|
||||
// Agent Engine (Sessions, MemoryBank) or a location (ServingProfiles). The
|
||||
// root bounds every operation name a caller can reach.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/service.h"
|
||||
#include "google/sessions/calls.h"
|
||||
#include "google/sessions/operation_names.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
struct OperationListOptions {
|
||||
std::string filter; // AIP-160 on name, done, ...
|
||||
int page_size = 0; // 0: Google default
|
||||
};
|
||||
|
||||
// True when a list must stop loudly: more items than max_items, or a further
|
||||
// page exists after page `page_index` (0 based) used up max_pages. Lists never
|
||||
// truncate silently; the caller narrows the filter instead.
|
||||
bool list_over_limit(std::size_t item_count, bool more_pages, std::size_t page_index,
|
||||
std::size_t max_pages, std::size_t max_items);
|
||||
|
||||
class OperationService : public Service {
|
||||
public:
|
||||
// Raw operation document (GET).
|
||||
clients::body::Json operation(const std::string& operation_name) const;
|
||||
// POST :cancel. Best effort at Google; the operation may still finish.
|
||||
void cancel_operation(const std::string& operation_name) const;
|
||||
// Destructive: removes the operation record (DELETE). Explicit method; it
|
||||
// never cancels the work.
|
||||
void delete_operation(const std::string& operation_name) const;
|
||||
// Polls by GET until done (OperationCaps); throws on failure.
|
||||
Operation wait(const std::string& operation_name) const;
|
||||
// One server-side wait (POST :wait?timeout=Ns), 1..kMaxServerWaitSeconds.
|
||||
// Returns the operation as Google answers it: done may still be false when
|
||||
// the timeout passed first. A failed operation throws.
|
||||
Operation wait_server(const std::string& operation_name, int timeout_seconds) const;
|
||||
// Operations under `parent` (default: the root engine; for a location root
|
||||
// a serving profile name is required). Capped by SessionsCaps list limits.
|
||||
std::vector<Operation> list_operations(const std::string& parent = "",
|
||||
const OperationListOptions& options = {}) const;
|
||||
|
||||
const std::string& root() const { return root_; }
|
||||
|
||||
protected:
|
||||
OperationService(std::shared_ptr<GoogleClient> client, ServiceEndpoint endpoint,
|
||||
std::string root, Sleeper sleeper);
|
||||
|
||||
// One table-driven request. `resource` fills the {parent} / {name} slot.
|
||||
clients::body::Json send(Call call, const std::string& resource,
|
||||
const clients::body::Json* body = nullptr,
|
||||
const std::string& query = "") const;
|
||||
// Follows an Operation reply to completion (done replies are checked).
|
||||
Operation settle(const clients::body::Json& reply) const;
|
||||
// Paged list through the table; caps pages and total items.
|
||||
clients::body::Json list_items(Call call, const std::string& resource,
|
||||
const std::string& query,
|
||||
const std::string& items_key, std::size_t max_pages,
|
||||
std::size_t max_items) const;
|
||||
|
||||
std::string root_;
|
||||
Sleeper sleeper_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,165 @@
|
||||
#include "google/sessions/serving_profile.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/resource.h"
|
||||
#include "google/sessions/caps.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
const char* serving_scope_name(ServingScope scope) {
|
||||
switch (scope) {
|
||||
case ServingScope::kGeminiLive: return "GEMINI_LIVE";
|
||||
case ServingScope::kInteractionsApi: return "INTERACTIONS_API";
|
||||
case ServingScope::kResponseApi: return "RESPONSE_API";
|
||||
}
|
||||
return "GEMINI_LIVE";
|
||||
}
|
||||
|
||||
ServingScope serving_scope_from(const std::string& name) {
|
||||
for (ServingScope s : {ServingScope::kGeminiLive, ServingScope::kInteractionsApi,
|
||||
ServingScope::kResponseApi})
|
||||
if (name == serving_scope_name(s)) return s;
|
||||
throw GoogleError::invalid(
|
||||
"scope must be GEMINI_LIVE, INTERACTIONS_API or RESPONSE_API");
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
void check_kms(const std::string& key) {
|
||||
if (key.empty()) return;
|
||||
Check::text("kms_key_name", key, SessionsCaps::kMaxKmsKeyChars);
|
||||
if (key.rfind("projects/", 0) != 0 || key.find("/cryptoKeys/") == std::string::npos ||
|
||||
key.find("..") != std::string::npos || key.find('?') != std::string::npos)
|
||||
throw GoogleError::invalid(
|
||||
"kms_key_name must be projects/{p}/locations/{l}/keyRings/{k}/cryptoKeys/{c}");
|
||||
}
|
||||
|
||||
Json cmek(const std::string& key) {
|
||||
Json spec = Json::object();
|
||||
spec.set("kmsKeyName", Json(key));
|
||||
Json out = Json::object();
|
||||
out.set("encryptionSpec", spec);
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
void ServingProfileSpec::validate() const {
|
||||
Check::serving_profile_id(profile_id);
|
||||
Check::text("display_name", display_name, SessionsCaps::kMaxDisplayChars, true);
|
||||
Check::text("description", description, SessionsCaps::kMaxDescriptionChars);
|
||||
check_kms(kms_key_name);
|
||||
}
|
||||
|
||||
Json ServingProfileSpec::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("displayName", Json(display_name));
|
||||
if (!description.empty()) out.set("description", Json(description));
|
||||
out.set("scope", Json(serving_scope_name(scope)));
|
||||
if (!kms_key_name.empty()) out.set("cmekConfig", cmek(kms_key_name));
|
||||
return out;
|
||||
}
|
||||
|
||||
void ServingProfilePatch::validate() const {
|
||||
Check::serving_profile_id(profile_id);
|
||||
Check::text("update_mask", update_mask, 256, true);
|
||||
for (char c : update_mask) {
|
||||
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == ',' || c == '.';
|
||||
if (!ok) throw GoogleError::invalid("update_mask allows letters, comma, dot");
|
||||
}
|
||||
Check::text("display_name", display_name, SessionsCaps::kMaxDisplayChars);
|
||||
Check::text("description", description, SessionsCaps::kMaxDescriptionChars);
|
||||
check_kms(kms_key_name);
|
||||
}
|
||||
|
||||
Json ServingProfilePatch::to_json() const {
|
||||
Json out = Json::object();
|
||||
if (!display_name.empty()) out.set("displayName", Json(display_name));
|
||||
if (!description.empty()) out.set("description", Json(description));
|
||||
if (!kms_key_name.empty()) out.set("cmekConfig", cmek(kms_key_name));
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string ServingProfile::id() const {
|
||||
std::size_t slash = name.rfind('/');
|
||||
return slash == std::string::npos ? name : name.substr(slash + 1);
|
||||
}
|
||||
|
||||
ServingProfile ServingProfile::from_json(const Json& doc) {
|
||||
ServingProfile p;
|
||||
p.name = str_at(doc, "name");
|
||||
p.display_name = str_at(doc, "displayName");
|
||||
p.description = str_at(doc, "description");
|
||||
p.scope = str_at(doc, "scope");
|
||||
p.create_time = str_at(doc, "createTime");
|
||||
p.update_time = str_at(doc, "updateTime");
|
||||
if (const Json* c = doc.is_object() ? doc.find("cmekConfig") : nullptr)
|
||||
if (const Json* e = c->is_object() ? c->find("encryptionSpec") : nullptr)
|
||||
p.kms_key_name = str_at(*e, "kmsKeyName");
|
||||
return p;
|
||||
}
|
||||
|
||||
Json ServingProfile::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("name", Json(name));
|
||||
out.set("display_name", Json(display_name));
|
||||
out.set("description", Json(description));
|
||||
out.set("scope", Json(scope));
|
||||
out.set("kms_key_name", Json(kms_key_name));
|
||||
out.set("create_time", Json(create_time));
|
||||
out.set("update_time", Json(update_time));
|
||||
return out;
|
||||
}
|
||||
|
||||
ServingProfiles::ServingProfiles(std::shared_ptr<GoogleClient> client,
|
||||
LocationName location, Sleeper sleeper)
|
||||
: OperationService(std::move(client), endpoints::aiplatform_beta(location.location()),
|
||||
location.str(), std::move(sleeper)),
|
||||
location_(std::move(location)) {}
|
||||
|
||||
ServingProfile ServingProfiles::create(const ServingProfileSpec& spec) const {
|
||||
spec.validate();
|
||||
Json body = spec.to_json();
|
||||
Operation op = settle(send(Call::kServingProfileCreate, location_.str(), &body,
|
||||
query_of({{"servingProfileId", spec.profile_id}})));
|
||||
if (op.response.is_object() && !str_at(op.response, "name").empty())
|
||||
return ServingProfile::from_json(op.response);
|
||||
return get(spec.profile_id);
|
||||
}
|
||||
|
||||
ServingProfile ServingProfiles::get(const std::string& profile_id) const {
|
||||
return ServingProfile::from_json(
|
||||
send(Call::kServingProfileGet, location_.serving_profile(profile_id)));
|
||||
}
|
||||
|
||||
std::vector<ServingProfile> ServingProfiles::list(int page_size) const {
|
||||
Check::page_size(page_size, SessionsCaps::kPageSize);
|
||||
Json items = list_items(
|
||||
Call::kServingProfileList, location_.str(),
|
||||
query_of({{"pageSize", page_size > 0 ? std::to_string(page_size) : ""}}),
|
||||
"servingProfiles", SessionsCaps::kMaxListPages, SessionsCaps::kMaxListItems);
|
||||
std::vector<ServingProfile> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(ServingProfile::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
ServingProfile ServingProfiles::patch(const ServingProfilePatch& patch) const {
|
||||
patch.validate();
|
||||
Json body = patch.to_json();
|
||||
return ServingProfile::from_json(
|
||||
send(Call::kServingProfilePatch, location_.serving_profile(patch.profile_id), &body,
|
||||
query_of({{"updateMask", patch.update_mask}})));
|
||||
}
|
||||
|
||||
void ServingProfiles::remove(const std::string& profile_id) const {
|
||||
send(Call::kServingProfileDelete, location_.serving_profile(profile_id));
|
||||
}
|
||||
|
||||
std::vector<Operation> ServingProfiles::list_profile_operations(
|
||||
const std::string& profile_id, const OperationListOptions& options) const {
|
||||
return list_operations(location_.serving_profile(profile_id), options);
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,82 @@
|
||||
#pragma once
|
||||
|
||||
// Serving profiles (aiplatform v1beta1 projects.locations.servingProfiles):
|
||||
// location-level settings for resource-less GenAI serving (Gemini Live,
|
||||
// Interactions API, Response API). Not engine scoped, so ServingProfiles
|
||||
// takes a location, not an Agent Engine.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/sessions/names.h"
|
||||
#include "google/sessions/operation_service.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
// Discovery enum ServingProfile.scope (without the unused UNSPECIFIED).
|
||||
enum class ServingScope { kGeminiLive, kInteractionsApi, kResponseApi };
|
||||
|
||||
const char* serving_scope_name(ServingScope scope);
|
||||
// Throws GoogleError::invalid for any other string.
|
||||
ServingScope serving_scope_from(const std::string& name);
|
||||
|
||||
struct ServingProfileSpec { // Input to create.
|
||||
std::string profile_id; // Required, becomes the servingProfileId query.
|
||||
std::string display_name; // Required, up to 128 characters.
|
||||
std::string description;
|
||||
ServingScope scope = ServingScope::kGeminiLive;
|
||||
std::string kms_key_name; // Optional CMEK: projects/.../cryptoKeys/...
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
// PATCH of one profile; `scope` is not patchable here (not in the patch row).
|
||||
struct ServingProfilePatch {
|
||||
std::string profile_id;
|
||||
std::string display_name;
|
||||
std::string description;
|
||||
std::string kms_key_name;
|
||||
std::string update_mask; // Required: displayName,description,cmekConfig
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct ServingProfile {
|
||||
std::string name;
|
||||
std::string display_name;
|
||||
std::string description;
|
||||
std::string scope; // As Google sent it, kept even when unknown to the enum.
|
||||
std::string kms_key_name;
|
||||
std::string create_time, update_time;
|
||||
|
||||
std::string id() const; // Last component of name.
|
||||
static ServingProfile from_json(const clients::body::Json& doc);
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
class ServingProfiles : public OperationService {
|
||||
public:
|
||||
ServingProfiles(std::shared_ptr<GoogleClient> client, LocationName location,
|
||||
Sleeper sleeper = nullptr);
|
||||
|
||||
const LocationName& location() const { return location_; }
|
||||
|
||||
ServingProfile create(const ServingProfileSpec& spec) const; // waits
|
||||
ServingProfile get(const std::string& profile_id) const;
|
||||
std::vector<ServingProfile> list(int page_size = 0) const;
|
||||
ServingProfile patch(const ServingProfilePatch& patch) const;
|
||||
// Destructive: explicit method.
|
||||
void remove(const std::string& profile_id) const;
|
||||
|
||||
// Operations of one serving profile (name from list_operations or create).
|
||||
std::vector<Operation> list_profile_operations(const std::string& profile_id,
|
||||
const OperationListOptions& o = {}) const;
|
||||
|
||||
private:
|
||||
LocationName location_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -86,4 +86,9 @@ std::vector<SessionEvent> Sessions::list_events(const std::string& session_id,
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<Operation> Sessions::list_session_operations(
|
||||
const std::string& session_id, const OperationListOptions& options) const {
|
||||
return list_operations(engine_name_.session(session_id), options);
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
|
||||
@@ -30,6 +30,10 @@ class Sessions : public EngineService {
|
||||
void append_event(const std::string& session_id, const SessionEvent& event) const;
|
||||
std::vector<SessionEvent> list_events(const std::string& session_id,
|
||||
const EventListOptions& options = {}) const;
|
||||
|
||||
// Operations that belong to one session (create, compact, delete runs).
|
||||
std::vector<Operation> list_session_operations(
|
||||
const std::string& session_id, const OperationListOptions& options = {}) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
|
||||
@@ -34,6 +34,7 @@ ENUMS = [
|
||||
("Memory", "memoryType", ["NATURAL_LANGUAGE_COLLECTION", "STRUCTURED_PROFILE"]),
|
||||
("CompactionConfigLlmSummarizationConfig", "mode", ["AUTO"]),
|
||||
("CompactionConfigEventEditingConfig", "mode", ["AUTO"]),
|
||||
("ServingProfile", "scope", ["GEMINI_LIVE", "INTERACTIONS_API", "RESPONSE_API"]),
|
||||
]
|
||||
|
||||
# (schema suffix, property) read by the response parsers.
|
||||
@@ -51,6 +52,12 @@ RESPONSE_KEYS = [
|
||||
("MemoryRevision", "fact"), ("MemoryRevision", "context"),
|
||||
("MemoryRevision", "labels"), ("MemoryRevision", "createTime"),
|
||||
("MemoryRevision", "expireTime"),
|
||||
("ListServingProfilesResponse", "servingProfiles"),
|
||||
("ListServingProfilesResponse", "nextPageToken"),
|
||||
("ServingProfile", "name"), ("ServingProfile", "displayName"),
|
||||
("ServingProfile", "description"), ("ServingProfile", "scope"),
|
||||
("ServingProfile", "createTime"), ("ServingProfile", "updateTime"),
|
||||
("ServingProfile", "cmekConfig"),
|
||||
("Session", "name"), ("Session", "createTime"), ("Session", "updateTime"),
|
||||
("Session", "expireTime"), ("Memory", "name"), ("Memory", "expireTime"),
|
||||
]
|
||||
@@ -73,12 +80,30 @@ NESTED = [
|
||||
("MemoryFilter", "key op negate value"),
|
||||
("MemoryMetadataValue", "stringValue doubleValue boolValue timestampValue"),
|
||||
("MemoryTopicId", "customMemoryTopicLabel managedMemoryTopic"),
|
||||
("ServingProfileCmekConfig", "encryptionSpec"),
|
||||
("EncryptionSpec", "kmsKeyName"),
|
||||
("CompactionConfig", "summarization eventEditing"),
|
||||
("CompactionConfigLlmSummarizationConfig", "mode"),
|
||||
("CompactionConfigEventEditingConfig", "mode"),
|
||||
]
|
||||
|
||||
|
||||
# Discovery methods this module owns: every one must be a table row.
|
||||
OWNED_PREFIXES = [
|
||||
"aiplatform.projects.locations.reasoningEngines.operations.",
|
||||
"aiplatform.projects.locations.reasoningEngines.sessions.",
|
||||
"aiplatform.projects.locations.reasoningEngines.memories.",
|
||||
"aiplatform.projects.locations.servingProfiles.",
|
||||
]
|
||||
# method id -> reason it is not wrapped (none today).
|
||||
EXCLUDED = {}
|
||||
|
||||
def collect_methods(res, out):
|
||||
out.extend(res.get("methods", {}).values())
|
||||
for r in res.get("resources", {}).values():
|
||||
collect_methods(r, out)
|
||||
|
||||
|
||||
def load(path):
|
||||
if path:
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
@@ -144,7 +169,7 @@ def main():
|
||||
errors.append("calls.h: enum Call and table rows differ")
|
||||
|
||||
for row in rows:
|
||||
mid = prefix + row["method"]
|
||||
mid = row["method"] if row["method"].startswith("aiplatform.") else prefix + row["method"]
|
||||
m = find_method(doc, mid)
|
||||
if m is None:
|
||||
errors.append("%s: method %s absent" % (row["call"], mid))
|
||||
@@ -171,6 +196,25 @@ def main():
|
||||
errors.append("%s: discovery has a request body %s the table omits" %
|
||||
(row["call"], m["request"]["$ref"]))
|
||||
|
||||
# Coverage: every Discovery method of the resources this module owns must
|
||||
# be a table row (or be listed, with a reason, in EXCLUDED).
|
||||
declared_ids = set()
|
||||
for row in rows:
|
||||
declared_ids.add(row["method"] if row["method"].startswith("aiplatform.")
|
||||
else prefix + row["method"])
|
||||
owned = []
|
||||
collect_methods(doc, owned)
|
||||
covered = 0
|
||||
for m in owned:
|
||||
if not any(m["id"].startswith(p) for p in OWNED_PREFIXES):
|
||||
continue
|
||||
if m["id"] in declared_ids:
|
||||
covered += 1
|
||||
elif m["id"] in EXCLUDED:
|
||||
continue
|
||||
else:
|
||||
errors.append("discovery method %s is not in the call table" % m["id"])
|
||||
|
||||
for suffix, prop, values in ENUMS:
|
||||
s = schemas.get(schema_prefix + suffix, {}).get("properties", {}).get(prop)
|
||||
if s is None:
|
||||
@@ -182,6 +226,8 @@ def main():
|
||||
for v in values:
|
||||
if v not in allowed:
|
||||
errors.append("enum value %s absent from %s.%s" % (v, suffix, prop))
|
||||
if "operations" not in schemas.get("GoogleLongrunningListOperationsResponse", {}).get("properties", {}):
|
||||
errors.append("response key GoogleLongrunningListOperationsResponse.operations absent")
|
||||
for suffix, prop in RESPONSE_KEYS:
|
||||
if prop not in schemas.get(schema_prefix + suffix, {}).get("properties", {}):
|
||||
errors.append("response key %s.%s absent" % (suffix, prop))
|
||||
|
||||
Reference in New Issue
Block a user