feat(clients): armor: Model Armor, SCC agent findings, GuardedModel, discovery-checked call table

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 04:38:50 +02:00
co-authored by Claude Sonnet 5.5
parent 6d744a433f
commit 4022be601d
24 changed files with 4237 additions and 3 deletions
+183
View File
@@ -0,0 +1,183 @@
# Armor - Specification
Owned directory: `src/google/armor/`. Namespace `molo::google::armor`.
Depends on the foundation in `src/google/` (`GoogleClient`, `Service`,
`GoogleError`, `CloudContext`, `endpoints::custom`, `LanguageModel`). No other
module.
## 1. Scope
- Model Armor: templates (create, get, list, patch, delete), floor settings
(get, update; project, folder, organization), locations (list, get),
`sanitizeUserPrompt`, `sanitizeModelResponse` with typed filter results
(prompt injection and jailbreak, sensitive data, malicious URIs,
responsible AI, CSAM, virus scan).
- Agent Threat Detection and Agent Anomaly Detection: they have NO API of
their own. They publish findings to Security Command Center. Read-only
access here is `SecurityFindings` over Security Command Center v2
(`findings.list`, `findings:group`, `sources.list`, `sources.get`). The
Finding resource carries typed `agent`, `agentAnomaly`, `agentSessions`,
`agentDataAccessEvents`, `aiModel` and `vulnerability` members, which are
read into `Finding`.
- Agent Security dashboard: a console product with no dashboard API.
`AgentSecurityDashboard` is a LOCAL abstraction that aggregates listed
findings (counts by severity, category, class, state, per-asset worst
severity). Agent assets are matched by the resource name fragment
`reasoningEngines` (local convention, see Unverified).
- Asset discovery: Security Command Center v2 has no assets list method
(Discovery: only `assets.updateSecurityMarks`). `discovered_assets()` is
local: distinct resources that carry findings.
- `GuardedModel : LanguageModel`: sanitizes the prompt before and the response
after any wrapped `LanguageModel`; throws `BlockedError`.
Authentication: Model Armor and Security Command Center REQUIRE an OAuth2
access token (`GOOGLE_ACCESS_TOKEN`, scope `cloud-platform`). API keys,
including express keys, are refused (see Verified live). Required project
env: `GOOGLE_CLOUD_PROJECT`, optional `GOOGLE_CLOUD_LOCATION`.
## 2. Module map
| File | Concern |
|---|---|
| `caps.h` | Static ceilings (`ArmorCaps`) |
| `calls.h/.cpp` | The ONE table of declared calls, schemas, enums; path matcher |
| `tools/check_discovery.py` | Fails when `calls.h` and Google's Discovery documents disagree |
| `armor_types.h/.cpp` | `SanitizationResult`, `FilterOutcome`, enums |
| `template.h/.cpp` | `Template`, `FilterConfig`, `TemplateMetadata`, `FloorSetting`, rule sets |
| `model_armor.h/.cpp` | `ModelArmor : Service`, `DataItem`, `LanguageHint` |
| `armor_util.h/.cpp` | `query_escape`, `join_update_mask`, `valid_scope`, `valid_request_id` |
| `findings.h/.cpp` | `SecurityFindings : Service`, `Finding`, `FindingQuery` |
| `dashboard.h/.cpp` | `AgentSecurityDashboard` (local aggregation) |
| `guarded_model.h/.cpp` | `Sanitizer` seam, `ModelArmorSanitizer`, `GuardedModel`, `BlockedError` |
## 3. Static caps (see caps.h)
| Knob | Value |
|---|---|
| Text per sanitize call | 100000 characters |
| File bytes per sanitize call | 5 MiB (our ceiling) |
| Stream buffer (guarded) | 200000 characters, 4096 chunks |
| Template page size / pages followed | 100 / 10 |
| Rai filters, rule sets, rules per set, words per rule, regex | 4, 16, 64, 1000, 1024 chars |
| Labels per template | 64 (63 chars each) |
| Update mask paths | 16 |
| Detail strings per filter result | 50 |
| Findings page size (Google max) / pages / results | 1000 / 10 / 5000 |
| Categories in a filter | 20 |
| Dashboard distinct assets | 500 |
## 4. Contracts
- Every id, scope, mask, filter value, order_by, enum string and size is
validated before any network traffic (`GoogleError::invalid`, HTTP 400
`INVALID_ARGUMENT`). No credential: `NotConfiguredError` (503) before the
network.
- Hosts: `https://modelarmor.{location}.rep.googleapis.com/v1` for templates,
sanitize and floor settings; floor settings and locations also accept
`global` (`https://modelarmor.googleapis.com/v1`); templates and sanitize
reject `global`. Security Command Center: `https://securitycenter.googleapis.com/v2`.
- Destructive: `delete_template` is an explicit method; the demo route also
requires `confirm=delete`. Nothing here creates, updates or deletes
findings, sources or any Security Command Center resource.
- `retry ids`: `requestId` is accepted on create, patch and delete as a
non-zero UUID.
- Sanitization results are typed; an unknown or missing `invocationResult`
never counts as clean (`fully_executed()` is false).
- `GuardedModel` fails closed: a matched filter blocks (`incomplete() ==
false`); a PARTIAL or FAILURE run, or any skipped filter, also blocks
(`incomplete() == true`) unless `GuardOptions::allow_partial`. An
unreachable sanitizer propagates its error and the model is not called
(prompt stage) or the response is not returned (response stage). Requests
with no text (only files) are counted in `skipped_empty`, not checked.
Streaming buffers the whole stream, sanitizes the full text once, then
replays the chunks; a blocked stream delivers nothing.
- Findings: `FindingQuery` builds an AIP-160 filter from typed fields only
(`category`, `state`, `severity`, `resourceName` substring). `findingClass`
is filtered locally because Discovery lists it for `orderBy` only.
## 5. Verified against discovery
`python3 -I src/google/armor/tools/check_discovery.py` fetches
`https://modelarmor.googleapis.com/$discovery/rest?version=v1` and
`https://securitycenter.googleapis.com/$discovery/rest?version=v2` and checks
every row of `calls.h`: 31 calls (rpc id, HTTP method, path, query
parameters), 66 schemas (every field this module reads or writes) and 13 enum
value sets. Result on 2026-10-24: `OK: 31 calls, 66 schemas, 13 enums
checked, 0 problems`. Unit tests additionally prove that every request the
client puts on the wire matches a declared call and only uses declared query
parameters, and that every key of every request body built here is a field of
its Discovery schema (nested schemas included).
Discovery facts that differ from the prose docs and are implemented as
Discovery states them: `inspectOnly` and `inspectAndBlock` on floor settings
are booleans; `FloorSetting.googleMcpServerFloorSetting` exists; the CSAM
result member is spelled `csamFilterFilterResult`; templates carry
`filterRuleSettings` (exclusion dictionaries and regex), custom error codes
and messages, `filterVersionSelector`, `modalities`, `dataResidencyCompliant`;
`ListTemplatesResponse.unreachable` exists; create, patch and delete accept
`requestId`; the Discovery root URL lists `us-central1` as the sample region.
## 6. Verified live
Date 2026-10-24, project from `GOOGLE_CLOUD_PROJECT`, no resource created,
updated or deleted.
| Call | Credential | Result |
|---|---|---|
| Model Armor `templates.list` | express key (`AQ.`) | 401 UNAUTHENTICATED (API keys are not supported); `google_live_armor.express_key_is_refused` |
| Model Armor `templates.list` | service account OAuth token | 403 PERMISSION_DENIED: "Model Armor API has not been used in project before or it is disabled" |
| Security Command Center v2 `findings.list` | service account OAuth token | 403 PERMISSION_DENIED: API not enabled |
| Security Command Center v2 `sources` | express key | 401 (reported by the coordinator) |
So authentication and routing to both services work with an OAuth token; the
request shapes beyond the 403 gate, and every response parser, are NOT yet
exercised against a real reply. The opt-in test
`google_live_armor.oauth_token_reads_templates_and_findings` (`MOLO_LIVE_NET=1`,
`GOOGLE_ACCESS_TOKEN`, `GOOGLE_CLOUD_PROJECT`; loud SKIP otherwise) is read-only
(list templates, get floor setting, list project findings, and
`sanitizeUserPrompt` only when a template already exists) and reports a
disabled API loudly. Run it once the APIs are enabled.
## 7. Sources
- Discovery documents above (authoritative for paths, methods, fields, enums).
- https://docs.cloud.google.com/security-command-center/docs/reference/model-armor/rest
- https://docs.cloud.google.com/model-armor/reference/rest/v1/ (Template,
FloorSetting, SanitizationResult, DataItem, templates.* method pages)
- https://docs.cloud.google.com/security-command-center/docs/reference/rest/v2/organizations.sources.findings/list
- https://docs.cloud.google.com/security-command-center/docs/reference/rest/v2/ListFindingsResponse
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/agent-anomalies-overview
(anomaly findings are surfaced in Security Command Center)
- https://docs.cloud.google.com/security-command-center/docs/overview-threats
(Agent Platform threat findings)
## 8. Unverified
Nothing neither Discovery nor docs state:
- Finding `category` strings for Agent Threat Detection and Agent Anomaly
Detection: not read; callers pass categories they see in the console.
- That agent assets always contain `reasoningEngines` in `resourceName`
(dashboard default filter; override with `resource_contains`).
- Server-side behavior of the `resourceName:"x"` substring operator (AIP-160
`:` is documented generally, not for this field).
- Real response bodies: parsers follow Discovery schemas and are tested with
documents built from schema field names, not with recorded Google replies.
The FilterResult map keys (`rai`, `pi_and_jailbreak`, ...) are not relied
upon; filters are identified by their member name.
- Google's byte limit for file items in sanitize calls.
- Whether `sources.get` exists for folders and projects (Discovery: only
organizations; `get_source` accepts only organizations).
- Gemini `promptFeedback` and `safetyRatings` were not live-checked; the guard
does not use them.
## 9. Gaps
- `findings:group` is wired; the dashboard still aggregates locally (owner:
`dashboard.cpp`). Server-side counting is the cheaper path for large sets.
- Streaming sanitize (`streamingMode`, gRPC `StreamSanitize*`) is not
implemented; `GuardedModel` buffers instead.
- No asset inventory (Cloud Asset Inventory is a different API).
- Findings mutation (mute, state), notification configs, BigQuery exports,
mute configs and attack paths are out of scope (read-only module).
- Token refresh is the caller's job (foundation gap).
+220
View File
@@ -0,0 +1,220 @@
#include "google/armor/armor_types.h"
#include "google/armor/caps.h"
namespace molo::google::armor {
using clients::body::Json;
namespace {
const Json* member(const Json& doc, const char* key) {
return doc.is_object() ? doc.find(key) : nullptr;
}
std::string text_of(const Json& doc, const char* key) {
const Json* v = member(doc, key);
return v && v->is_string() ? v->as_string() : std::string();
}
MatchState match_from(const std::string& wire) {
if (wire == "MATCH_FOUND") return MatchState::kMatch;
if (wire == "NO_MATCH_FOUND") return MatchState::kNoMatch;
return MatchState::kUnspecified;
}
const char* match_name(MatchState m) {
switch (m) {
case MatchState::kMatch: return "MATCH_FOUND";
case MatchState::kNoMatch: return "NO_MATCH_FOUND";
default: return "";
}
}
void add_detail(FilterOutcome& out, const std::string& d) {
if (out.details.size() < ArmorCaps::kMaxDetailsPerFilter) out.details.push_back(d);
}
void read_common(FilterOutcome& out, const Json& body) {
out.match = match_from(text_of(body, "matchState"));
std::string exec = text_of(body, "executionState");
out.execution = exec == "EXECUTION_SUCCESS" ? ExecutionState::kSuccess
: exec == "EXECUTION_SKIPPED" ? ExecutionState::kSkipped
: ExecutionState::kUnknown;
out.confidence = confidence_from(text_of(body, "confidenceLevel"));
if (const Json* items = member(body, "messageItems"); items && items->is_array())
for (const Json& m : items->as_array()) {
std::string t = text_of(m, "message");
if (!t.empty() && out.messages.size() < ArmorCaps::kMaxDetailsPerFilter)
out.messages.push_back(t);
}
}
// Reads one FilterResult object (exactly one of the *FilterResult members).
bool read_filter(const Json& wrapper, FilterOutcome& out) {
if (const Json* b = member(wrapper, "raiFilterResult")) {
out.kind = FilterKind::kResponsibleAi;
read_common(out, *b);
if (const Json* types = member(*b, "raiFilterTypeResults");
types && types->is_object())
for (const auto& [name, r] : types->as_object())
if (match_from(text_of(r, "matchState")) == MatchState::kMatch)
add_detail(out, name + ":" + text_of(r, "confidenceLevel"));
return true;
}
if (const Json* b = member(wrapper, "sdpFilterResult")) {
out.kind = FilterKind::kSensitiveData;
for (const char* variant : {"inspectResult", "deidentifyResult", "redactResult"}) {
const Json* v = member(*b, variant);
if (!v) continue;
read_common(out, *v);
if (const Json* f = member(*v, "findings"); f && f->is_array())
for (const Json& finding : f->as_array())
add_detail(out, text_of(finding, "infoType"));
if (const Json* t = member(*v, "infoTypes"); t && t->is_array())
for (const Json& i : t->as_array())
if (i.is_string()) add_detail(out, i.as_string());
}
return true;
}
if (const Json* b = member(wrapper, "piAndJailbreakFilterResult")) {
out.kind = FilterKind::kPromptInjectionJailbreak;
read_common(out, *b);
return true;
}
if (const Json* b = member(wrapper, "maliciousUriFilterResult")) {
out.kind = FilterKind::kMaliciousUri;
read_common(out, *b);
if (const Json* u = member(*b, "maliciousUriMatchedItems"); u && u->is_array())
for (const Json& item : u->as_array()) add_detail(out, text_of(item, "uri"));
return true;
}
if (const Json* b = member(wrapper, "csamFilterFilterResult")) {
out.kind = FilterKind::kCsam;
read_common(out, *b);
return true;
}
if (const Json* b = member(wrapper, "virusScanFilterResult")) {
out.kind = FilterKind::kVirusScan;
read_common(out, *b);
if (const Json* d = member(*b, "virusDetails")) {
if (const Json* n = member(*d, "names"); n && n->is_array())
for (const Json& i : n->as_array())
if (i.is_string()) add_detail(out, i.as_string());
}
return true;
}
return false;
}
} // namespace
const char* filter_kind_name(FilterKind kind) {
switch (kind) {
case FilterKind::kResponsibleAi: return "rai";
case FilterKind::kSensitiveData: return "sdp";
case FilterKind::kPromptInjectionJailbreak: return "pi_and_jailbreak";
case FilterKind::kMaliciousUri: return "malicious_uris";
case FilterKind::kCsam: return "csam";
case FilterKind::kVirusScan: return "virus_scan";
}
return "";
}
const char* confidence_name(Confidence c) {
switch (c) {
case Confidence::kLowAndAbove: return "LOW_AND_ABOVE";
case Confidence::kMediumAndAbove: return "MEDIUM_AND_ABOVE";
case Confidence::kHigh: return "HIGH";
default: return "";
}
}
Confidence confidence_from(const std::string& wire) {
if (wire == "LOW_AND_ABOVE") return Confidence::kLowAndAbove;
if (wire == "MEDIUM_AND_ABOVE") return Confidence::kMediumAndAbove;
if (wire == "HIGH") return Confidence::kHigh;
return Confidence::kUnspecified;
}
const char* invocation_name(Invocation i) {
switch (i) {
case Invocation::kSuccess: return "SUCCESS";
case Invocation::kPartial: return "PARTIAL";
case Invocation::kFailure: return "FAILURE";
default: return "";
}
}
Json FilterOutcome::to_json() const {
Json o = Json::object();
o.set("filter", filter_kind_name(kind));
o.set("matched", matched());
o.set("match_state", match_name(match));
o.set("executed", execution == ExecutionState::kSuccess);
if (confidence != Confidence::kUnspecified)
o.set("confidence", confidence_name(confidence));
Json d = Json::array();
for (const auto& s : details) d.push(s);
o.set("details", d);
return o;
}
bool SanitizationResult::fully_executed() const {
if (invocation != Invocation::kSuccess) return false;
for (const auto& f : filters)
if (f.execution == ExecutionState::kSkipped) return false;
return true;
}
std::vector<FilterOutcome> SanitizationResult::matches() const {
std::vector<FilterOutcome> out;
for (const auto& f : filters)
if (f.matched()) out.push_back(f);
return out;
}
const FilterOutcome* SanitizationResult::find(FilterKind kind) const {
for (const auto& f : filters)
if (f.kind == kind) return &f;
return nullptr;
}
Json SanitizationResult::to_json() const {
Json o = Json::object();
o.set("matched", matched());
o.set("invocation", invocation_name(invocation));
o.set("fully_executed", fully_executed());
Json f = Json::array();
for (const auto& x : filters) f.push(x.to_json());
o.set("filters", f);
if (!error_code.empty()) o.set("error_code", error_code);
if (!error_message.empty()) o.set("error_message", error_message);
return o;
}
SanitizationResult SanitizationResult::from_json(const Json& doc) {
SanitizationResult r;
// The response wraps the result in "sanitizationResult".
const Json* body = member(doc, "sanitizationResult");
const Json& src = body ? *body : doc;
r.match = match_from(text_of(src, "filterMatchState"));
std::string inv = text_of(src, "invocationResult");
r.invocation = inv == "SUCCESS" ? Invocation::kSuccess
: inv == "PARTIAL" ? Invocation::kPartial
: inv == "FAILURE" ? Invocation::kFailure
: Invocation::kUnknown;
if (const Json* meta = member(src, "sanitizationMetadata")) {
r.error_code = text_of(*meta, "errorCode");
r.error_message = text_of(*meta, "errorMessage");
}
if (const Json* fr = member(src, "filterResults"); fr && fr->is_object())
for (const auto& [key, wrapper] : fr->as_object()) {
(void)key;
FilterOutcome out;
if (read_filter(wrapper, out)) r.filters.push_back(std::move(out));
}
return r;
}
} // namespace molo::google::armor
+59
View File
@@ -0,0 +1,59 @@
#pragma once
// Typed Model Armor sanitization result. Wire names follow
// SanitizationResult / FilterResult in the Model Armor v1 REST reference.
#include <string>
#include <vector>
#include "clients/body/json.h"
namespace molo::google::armor {
enum class FilterKind {
kResponsibleAi,
kSensitiveData,
kPromptInjectionJailbreak,
kMaliciousUri,
kCsam,
kVirusScan,
};
const char* filter_kind_name(FilterKind kind); // "rai", "sdp", ...
enum class MatchState { kUnspecified, kNoMatch, kMatch };
enum class ExecutionState { kUnknown, kSuccess, kSkipped };
enum class Invocation { kUnknown, kSuccess, kPartial, kFailure };
enum class Confidence { kUnspecified, kLowAndAbove, kMediumAndAbove, kHigh };
const char* confidence_name(Confidence c); // Wire enum string, "" if unset.
Confidence confidence_from(const std::string& wire); // Unknown: unspecified.
const char* invocation_name(Invocation i);
struct FilterOutcome {
FilterKind kind = FilterKind::kResponsibleAi;
MatchState match = MatchState::kUnspecified;
ExecutionState execution = ExecutionState::kUnknown;
Confidence confidence = Confidence::kUnspecified;
// Matched rai types ("hate_speech:HIGH"), sdp info types, malicious uris.
std::vector<std::string> details;
std::vector<std::string> messages; // MessageItem texts.
bool matched() const { return match == MatchState::kMatch; }
clients::body::Json to_json() const;
};
struct SanitizationResult {
MatchState match = MatchState::kUnspecified;
Invocation invocation = Invocation::kUnknown;
std::vector<FilterOutcome> filters;
std::string error_code; // sanitizationMetadata.errorCode
std::string error_message; // sanitizationMetadata.errorMessage
bool matched() const { return match == MatchState::kMatch; }
// Every filter ran: invocation success and no skipped filter.
bool fully_executed() const;
std::vector<FilterOutcome> matches() const;
const FilterOutcome* find(FilterKind kind) const;
clients::body::Json to_json() const;
static SanitizationResult from_json(const clients::body::Json& doc);
};
} // namespace molo::google::armor
+79
View File
@@ -0,0 +1,79 @@
#include "google/armor/armor_util.h"
#include "google/armor/caps.h"
#include "google/errors.h"
#include "google/resource.h"
namespace molo::google::armor {
std::string query_escape(const std::string& value) {
static const char* hex = "0123456789ABCDEF";
std::string out;
for (unsigned char c : value) {
bool plain = (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') ||
(c >= '0' && c <= '9') || c == '-' || c == '_' || c == '.' ||
c == '~';
if (plain) {
out.push_back(static_cast<char>(c));
} else {
out.push_back('%');
out.push_back(hex[c >> 4]);
out.push_back(hex[c & 15]);
}
}
return out;
}
std::string join_update_mask(const std::vector<std::string>& paths) {
if (paths.empty() || paths.size() > ArmorCaps::kMaxUpdateMaskPaths)
throw GoogleError::invalid("update mask needs 1 to 16 paths");
std::string out;
for (const auto& p : paths) {
if (p.empty() || p.size() > 128)
throw GoogleError::invalid("bad update mask path");
for (char c : p)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
(c >= '0' && c <= '9') || c == '_' || c == '.'))
throw GoogleError::invalid("bad update mask path");
if (!out.empty()) out.push_back(',');
out += p;
}
return out;
}
bool valid_request_id(const std::string& id) {
if (id.size() != 36) return false;
bool all_zero = true;
for (std::size_t i = 0; i < id.size(); ++i) {
char c = id[i];
if (i == 8 || i == 13 || i == 18 || i == 23) {
if (c != '-') return false;
continue;
}
bool hex = (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F');
if (!hex) return false;
if (c != '0') all_zero = false;
}
return !all_zero;
}
bool valid_scope(const std::string& scope) {
std::size_t slash = scope.find('/');
if (slash == std::string::npos) return false;
std::string kind = scope.substr(0, slash);
std::string id = scope.substr(slash + 1);
if (kind != "projects" && kind != "folders" && kind != "organizations")
return false;
if (kind == "projects") {
// Project ids may carry ':' (domain-scoped), which valid_segment refuses.
if (id.empty() || id.size() > 128) return false;
for (char c : id)
if (!((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-' ||
c == '.' || c == ':'))
return false;
return true;
}
return valid_segment(id);
}
} // namespace molo::google::armor
+21
View File
@@ -0,0 +1,21 @@
#pragma once
// Small validators and encoders shared by the armor classes.
#include <string>
#include <vector>
namespace molo::google::armor {
// Percent-encodes everything outside the RFC 3986 unreserved set.
std::string query_escape(const std::string& value);
// "a,b.c" style FieldMask: comma separated, each path [A-Za-z0-9_.]+.
// Throws GoogleError::invalid when empty, too long or malformed.
std::string join_update_mask(const std::vector<std::string>& paths);
// 8-4-4-4-12 hex UUID that is not the all-zero UUID (requestId rule).
bool valid_request_id(const std::string& id);
// "projects/x", "folders/1", "organizations/2": kind plus a valid id.
bool valid_scope(const std::string& scope);
} // namespace molo::google::armor
+58
View File
@@ -0,0 +1,58 @@
#include "google/armor/calls.h"
#include <vector>
namespace molo::google::armor {
bool path_matches(const std::string& tmpl, const std::string& path) {
// Split the template into literals around {placeholders}.
std::vector<std::string> literals;
std::string cur;
for (std::size_t i = 0; i < tmpl.size(); ++i) {
if (tmpl[i] == '{') {
literals.push_back(cur);
cur.clear();
i = tmpl.find('}', i);
if (i == std::string::npos) return false;
} else {
cur.push_back(tmpl[i]);
}
}
literals.push_back(cur);
const std::string& first = literals.front();
if (path.compare(0, first.size(), first) != 0) return false;
std::size_t pos = first.size();
if (literals.size() == 1) return pos == path.size();
for (std::size_t i = 1; i < literals.size(); ++i) {
const std::string& lit = literals[i];
bool last = i + 1 == literals.size();
if (pos >= path.size()) return false; // Placeholder needs a character.
if (last) {
if (path.size() < pos + 1 + lit.size()) return false;
return path.compare(path.size() - lit.size(), lit.size(), lit) == 0;
}
std::size_t at = lit.empty() ? std::string::npos : path.find(lit, pos + 1);
if (at == std::string::npos) return false;
pos = at + lit.size();
}
return true;
}
const CallSpec* find_rpc(const std::string& rpc) {
for (const auto& c : kCalls)
if (rpc == c.rpc) return &c;
return nullptr;
}
bool request_matches(const std::string& rpc, const std::string& http,
const std::string& target) {
const CallSpec* c = find_rpc(rpc);
if (!c || http != c->http) return false;
std::string p = target;
if (!p.empty() && p.front() == '/') p.erase(0, 1);
if (std::size_t q = p.find('?'); q != std::string::npos) p.erase(q);
return path_matches(c->path, p);
}
} // namespace molo::google::armor
+177
View File
@@ -0,0 +1,177 @@
#pragma once
// The ONE declaration of every Google call this module makes, and of the
// request/response field names it relies on. tools/check_discovery.py reads
// this file (plain-text rows, one per line) and fails when Google's
// Discovery documents disagree. Rows are copied from the Discovery
// documents, never typed from memory:
// https://modelarmor.googleapis.com/$discovery/rest?version=v1
// https://securitycenter.googleapis.com/$discovery/rest?version=v2
#include <cstddef>
#include <string>
namespace molo::google::armor {
struct CallSpec {
const char* service; // Discovery service name
const char* version; // Discovery version
const char* rpc; // Discovery method id
const char* http; // GET | POST | PATCH | DELETE
const char* path; // Discovery `path` (relative to the service root)
const char* query; // Comma separated query parameters this module sends
};
struct SchemaSpec {
const char* service;
const char* version;
const char* schema; // Discovery schema name
const char* fields; // Comma separated property names this module reads or writes
};
struct EnumSpec {
const char* service;
const char* version;
const char* schema;
const char* field;
const char* values; // Comma separated enum values this module uses
};
// clang-format off
inline constexpr CallSpec kCalls[] = {
{"modelarmor", "v1", "modelarmor.projects.locations.templates.create", "POST", "v1/{+parent}/templates", "templateId,requestId"},
{"modelarmor", "v1", "modelarmor.projects.locations.templates.get", "GET", "v1/{+name}", ""},
{"modelarmor", "v1", "modelarmor.projects.locations.templates.list", "GET", "v1/{+parent}/templates", "pageSize,pageToken,filter,orderBy"},
{"modelarmor", "v1", "modelarmor.projects.locations.templates.patch", "PATCH", "v1/{+name}", "updateMask,requestId"},
{"modelarmor", "v1", "modelarmor.projects.locations.templates.delete", "DELETE", "v1/{+name}", "requestId"},
{"modelarmor", "v1", "modelarmor.projects.locations.templates.sanitizeUserPrompt", "POST", "v1/{+name}:sanitizeUserPrompt", ""},
{"modelarmor", "v1", "modelarmor.projects.locations.templates.sanitizeModelResponse", "POST", "v1/{+name}:sanitizeModelResponse", ""},
{"modelarmor", "v1", "modelarmor.projects.locations.getFloorSetting", "GET", "v1/{+name}", ""},
{"modelarmor", "v1", "modelarmor.projects.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask"},
{"modelarmor", "v1", "modelarmor.folders.locations.getFloorSetting", "GET", "v1/{+name}", ""},
{"modelarmor", "v1", "modelarmor.folders.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask"},
{"modelarmor", "v1", "modelarmor.organizations.locations.getFloorSetting", "GET", "v1/{+name}", ""},
{"modelarmor", "v1", "modelarmor.organizations.locations.updateFloorSetting", "PATCH", "v1/{+name}", "updateMask"},
{"modelarmor", "v1", "modelarmor.projects.locations.list", "GET", "v1/{+name}/locations", "pageToken"},
{"modelarmor", "v1", "modelarmor.projects.locations.get", "GET", "v1/{+name}", ""},
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
{"securitycenter", "v2", "securitycenter.folders.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
{"securitycenter", "v2", "securitycenter.projects.sources.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.list", "GET", "v2/{+parent}/findings", "pageSize,pageToken,filter,orderBy,fieldMask"},
{"securitycenter", "v2", "securitycenter.organizations.sources.findings.group", "POST", "v2/{+parent}/findings:group", ""},
{"securitycenter", "v2", "securitycenter.folders.sources.findings.group", "POST", "v2/{+parent}/findings:group", ""},
{"securitycenter", "v2", "securitycenter.projects.sources.findings.group", "POST", "v2/{+parent}/findings:group", ""},
{"securitycenter", "v2", "securitycenter.organizations.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", ""},
{"securitycenter", "v2", "securitycenter.folders.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", ""},
{"securitycenter", "v2", "securitycenter.projects.sources.locations.findings.group", "POST", "v2/{+parent}/findings:group", ""},
{"securitycenter", "v2", "securitycenter.organizations.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken"},
{"securitycenter", "v2", "securitycenter.folders.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken"},
{"securitycenter", "v2", "securitycenter.projects.sources.list", "GET", "v2/{+parent}/sources", "pageSize,pageToken"},
{"securitycenter", "v2", "securitycenter.organizations.sources.get", "GET", "v2/{+name}", ""},
};
// Request bodies (what to_json emits) and response documents (what from_json reads).
inline constexpr SchemaSpec kSchemas[] = {
{"modelarmor", "v1", "Template", "name,labels,filterConfig,templateMetadata,createTime,updateTime"},
{"modelarmor", "v1", "FilterConfig", "raiSettings,sdpSettings,piAndJailbreakFilterSettings,maliciousUriFilterSettings,filterRuleSettings"},
{"modelarmor", "v1", "TemplateMetadata", "enforcementType,ignorePartialInvocationFailures,logTemplateOperations,logSanitizeOperations,multiLanguageDetection,customPromptSafetyErrorCode,customPromptSafetyErrorMessage,customLlmResponseSafetyErrorCode,customLlmResponseSafetyErrorMessage,filterVersionSelector,modalities,dataResidencyCompliant"},
{"modelarmor", "v1", "MultiLanguageDetection", "enableMultiLanguageDetection"},
{"modelarmor", "v1", "FilterVersionSelector", "version,alias"},
{"modelarmor", "v1", "RaiFilterSettings", "raiFilters"},
{"modelarmor", "v1", "RaiFilter", "filterType,confidenceLevel"},
{"modelarmor", "v1", "FilterRuleSettings", "ruleSets"},
{"modelarmor", "v1", "FilterRuleSet", "filterTypes,rules"},
{"modelarmor", "v1", "FilterRule", "exclusionRule"},
{"modelarmor", "v1", "ExclusionRule", "dictionary,regex,matchingScope"},
{"modelarmor", "v1", "Dictionary", "wordList"},
{"modelarmor", "v1", "WordList", "words"},
{"modelarmor", "v1", "Regex", "pattern"},
{"modelarmor", "v1", "SdpFilterSettings", "basicConfig,advancedConfig"},
{"modelarmor", "v1", "SdpBasicConfig", "filterEnforcement"},
{"modelarmor", "v1", "SdpAdvancedConfig", "inspectTemplate,deidentifyTemplate"},
{"modelarmor", "v1", "PiAndJailbreakFilterSettings", "filterEnforcement,confidenceLevel"},
{"modelarmor", "v1", "MaliciousUriFilterSettings", "filterEnforcement"},
{"modelarmor", "v1", "FloorSetting", "name,updateTime,filterConfig,enableFloorSettingEnforcement,integratedServices,aiPlatformFloorSetting,googleMcpServerFloorSetting,floorSettingMetadata"},
{"modelarmor", "v1", "AiPlatformFloorSetting", "enableCloudLogging,inspectOnly,inspectAndBlock"},
{"modelarmor", "v1", "McpServerFloorSetting", "enableCloudLogging,inspectOnly,inspectAndBlock"},
{"modelarmor", "v1", "FloorSettingMetadata", "multiLanguageDetection"},
{"modelarmor", "v1", "FloorSettingFloorSettingMetadataMultiLanguageDetection", "enableMultiLanguageDetection"},
{"modelarmor", "v1", "ListTemplatesResponse", "templates,nextPageToken,unreachable"},
{"modelarmor", "v1", "ListLocationsResponse", "locations,nextPageToken"},
{"modelarmor", "v1", "Location", "name,locationId,displayName"},
{"modelarmor", "v1", "SanitizeUserPromptRequest", "userPromptData,multiLanguageDetectionMetadata"},
{"modelarmor", "v1", "SanitizeModelResponseRequest", "modelResponseData,userPrompt,multiLanguageDetectionMetadata"},
{"modelarmor", "v1", "MultiLanguageDetectionMetadata", "enableMultiLanguageDetection,sourceLanguage"},
{"modelarmor", "v1", "DataItem", "text,byteItem"},
{"modelarmor", "v1", "ByteDataItem", "byteDataType,byteData,fileLabel"},
{"modelarmor", "v1", "SanitizeUserPromptResponse", "sanitizationResult"},
{"modelarmor", "v1", "SanitizeModelResponseResponse", "sanitizationResult"},
{"modelarmor", "v1", "SanitizationResult", "filterMatchState,filterResults,invocationResult,sanitizationMetadata"},
{"modelarmor", "v1", "SanitizationMetadata", "errorCode,errorMessage"},
{"modelarmor", "v1", "FilterResult", "raiFilterResult,sdpFilterResult,piAndJailbreakFilterResult,maliciousUriFilterResult,csamFilterFilterResult,virusScanFilterResult"},
{"modelarmor", "v1", "RaiFilterResult", "executionState,messageItems,matchState,raiFilterTypeResults"},
{"modelarmor", "v1", "RaiFilterTypeResult", "filterType,confidenceLevel,matchState"},
{"modelarmor", "v1", "SdpFilterResult", "inspectResult,deidentifyResult,redactResult"},
{"modelarmor", "v1", "SdpInspectResult", "executionState,messageItems,matchState,findings"},
{"modelarmor", "v1", "SdpDeidentifyResult", "executionState,messageItems,matchState,infoTypes"},
{"modelarmor", "v1", "SdpRedactResult", "executionState,messageItems,matchState,findings"},
{"modelarmor", "v1", "SdpFinding", "infoType,likelihood"},
{"modelarmor", "v1", "PiAndJailbreakFilterResult", "executionState,messageItems,matchState,confidenceLevel"},
{"modelarmor", "v1", "MaliciousUriFilterResult", "executionState,messageItems,matchState,maliciousUriMatchedItems"},
{"modelarmor", "v1", "MaliciousUriMatchedItem", "uri"},
{"modelarmor", "v1", "CsamFilterResult", "executionState,messageItems,matchState"},
{"modelarmor", "v1", "VirusScanFilterResult", "executionState,messageItems,matchState,virusDetails"},
{"modelarmor", "v1", "VirusDetail", "names"},
{"modelarmor", "v1", "MessageItem", "messageType,message"},
{"securitycenter", "v2", "GroupFindingsRequest", "filter,groupBy,pageSize,pageToken"},
{"securitycenter", "v2", "GroupFindingsResponse", "groupByResults,nextPageToken,totalSize"},
{"securitycenter", "v2", "GroupResult", "count,properties"},
{"securitycenter", "v2", "ListFindingsResponse", "listFindingsResults,nextPageToken,totalSize"},
{"securitycenter", "v2", "ListFindingsResult", "finding,resource"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Finding", "name,parent,category,state,severity,findingClass,mute,resourceName,eventTime,createTime,description,externalUri,moduleName,agent,agentAnomaly,agentSessions,agentDataAccessEvents,aiModel,vulnerability"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Agent", "id,displayName"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2AgentAnomaly", "confidenceScore,invocationReferences,detectorReferences"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2InvocationReference", "invocationId"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2DetectorReference", "detectorId,displayName,severity,explanation,recommendation"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2AgentSession", "sessionId"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Vulnerability", "cve"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Cve", "id"},
{"securitycenter", "v2", "ListSourcesResponse", "sources,nextPageToken"},
{"securitycenter", "v2", "Source", "name,displayName,description,canonicalName"},
};
inline constexpr EnumSpec kEnums[] = {
{"modelarmor", "v1", "RaiFilter", "filterType", "SEXUALLY_EXPLICIT,HATE_SPEECH,HARASSMENT,DANGEROUS"},
{"modelarmor", "v1", "RaiFilter", "confidenceLevel", "LOW_AND_ABOVE,MEDIUM_AND_ABOVE,HIGH"},
{"modelarmor", "v1", "PiAndJailbreakFilterSettings", "filterEnforcement", "ENABLED,DISABLED"},
{"modelarmor", "v1", "MaliciousUriFilterSettings", "filterEnforcement", "ENABLED,DISABLED"},
{"modelarmor", "v1", "SdpBasicConfig", "filterEnforcement", "ENABLED,DISABLED"},
{"modelarmor", "v1", "TemplateMetadata", "enforcementType", "INSPECT_ONLY,INSPECT_AND_BLOCK"},
{"modelarmor", "v1", "SanitizationResult", "filterMatchState", "NO_MATCH_FOUND,MATCH_FOUND"},
{"modelarmor", "v1", "SanitizationResult", "invocationResult", "SUCCESS,PARTIAL,FAILURE"},
{"modelarmor", "v1", "RaiFilterResult", "executionState", "EXECUTION_SUCCESS,EXECUTION_SKIPPED"},
{"modelarmor", "v1", "ByteDataItem", "byteDataType", "PLAINTEXT_UTF8,PDF,WORD_DOCUMENT,EXCEL_DOCUMENT,POWERPOINT_DOCUMENT,TXT,CSV,IMAGE"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Finding", "state", "ACTIVE,INACTIVE"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Finding", "severity", "CRITICAL,HIGH,MEDIUM,LOW"},
{"securitycenter", "v2", "GoogleCloudSecuritycenterV2Finding", "findingClass", "THREAT,VULNERABILITY,MISCONFIGURATION,OBSERVATION"},
};
// clang-format on
inline constexpr std::size_t kCallCount = sizeof(kCalls) / sizeof(kCalls[0]);
inline constexpr std::size_t kSchemaCount = sizeof(kSchemas) / sizeof(kSchemas[0]);
inline constexpr std::size_t kEnumCount = sizeof(kEnums) / sizeof(kEnums[0]);
// Matches a request path (no query, no leading slash) against a Discovery
// path template such as "v1/{+parent}/templates": placeholders match one or
// more characters.
bool path_matches(const std::string& discovery_path, const std::string& path);
// The declared call with this Discovery method id, or nullptr.
const CallSpec* find_rpc(const std::string& rpc);
// True when `http` and the request target (path plus optional query, with or
// without a leading slash) match the declared call `rpc`.
bool request_matches(const std::string& rpc, const std::string& http,
const std::string& target);
} // namespace molo::google::armor
+43
View File
@@ -0,0 +1,43 @@
#pragma once
// Static ceilings for the armor module. Never autotuned.
#include <cstddef>
namespace molo::google::armor {
struct ArmorCaps {
// Text sent to sanitizeUserPrompt / sanitizeModelResponse per call.
static constexpr std::size_t kMaxTextChars = 100000;
// Text held while a stream is buffered for response sanitization.
static constexpr std::size_t kMaxStreamBufferChars = 200000;
// Chunks held while a stream is buffered.
static constexpr std::size_t kMaxStreamChunks = 4096;
// Templates: page size and pages followed per list call.
static constexpr int kMaxTemplatePageSize = 100;
static constexpr std::size_t kMaxTemplatePages = 10;
// File bytes per sanitize call (our ceiling; Google's is not documented).
static constexpr std::size_t kMaxByteItemBytes = 5u * 1024u * 1024u;
static constexpr std::size_t kMaxRaiFilters = 4;
static constexpr std::size_t kMaxRuleSets = 16;
static constexpr std::size_t kMaxRulesPerSet = 64;
static constexpr std::size_t kMaxRuleWords = 1000;
static constexpr std::size_t kMaxRegexChars = 1024;
static constexpr std::size_t kMaxLocations = 100;
static constexpr std::size_t kMaxGroupResults = 1000;
static constexpr std::size_t kMaxLabels = 64;
static constexpr std::size_t kMaxLabelChars = 63;
static constexpr std::size_t kMaxUpdateMaskPaths = 16;
// Result detail lists kept per filter (matched uris, info types).
static constexpr std::size_t kMaxDetailsPerFilter = 50;
// Security Command Center findings.
static constexpr int kMaxFindingsPageSize = 1000; // Google's maximum.
static constexpr int kDefaultFindingsPageSize = 100;
static constexpr std::size_t kMaxFindingPages = 10;
static constexpr std::size_t kMaxFindings = 5000;
static constexpr std::size_t kMaxFilterValueChars = 256;
static constexpr std::size_t kMaxFilterCategories = 20;
static constexpr std::size_t kMaxSources = 100;
// Distinct assets reported by the local dashboard.
static constexpr std::size_t kMaxDashboardAssets = 500;
};
} // namespace molo::google::armor
+109
View File
@@ -0,0 +1,109 @@
#include "google/armor/dashboard.h"
#include <algorithm>
#include "google/armor/caps.h"
namespace molo::google::armor {
using clients::body::Json;
namespace {
int rank(const std::string& severity) {
if (severity == "CRITICAL") return 4;
if (severity == "HIGH") return 3;
if (severity == "MEDIUM") return 2;
if (severity == "LOW") return 1;
return 0;
}
Json counts(const std::map<std::string, std::size_t>& m) {
Json o = Json::object();
for (const auto& [k, v] : m) o.set(k.empty() ? "UNSPECIFIED" : k, static_cast<long>(v));
return o;
}
} // namespace
Json AssetSummary::to_json() const {
Json o = Json::object();
o.set("resource", resource_name);
o.set("findings", static_cast<long>(findings));
o.set("worst_severity", worst_severity);
return o;
}
Json SecuritySummary::to_json() const {
Json o = Json::object();
o.set("total", static_cast<long>(total));
o.set("agent_findings", static_cast<long>(agent_findings));
o.set("anomalies", static_cast<long>(anomalies));
o.set("by_severity", counts(by_severity));
o.set("by_category", counts(by_category));
o.set("by_class", counts(by_class));
o.set("by_state", counts(by_state));
Json a = Json::array();
for (const auto& x : assets) a.push(x.to_json());
o.set("assets", a);
o.set("truncated", truncated);
return o;
}
FindingQuery AgentSecurityDashboard::with_agent_scope(FindingQuery query) {
if (query.resource_contains.empty()) query.resource_contains = "reasoningEngines";
return query;
}
SecuritySummary AgentSecurityDashboard::summarize_findings(const FindingPage& page) {
SecuritySummary s;
s.truncated = page.truncated;
std::map<std::string, AssetSummary> assets;
for (const auto& f : page.findings) {
++s.total;
if (f.is_agent_finding()) ++s.agent_findings;
if (f.has_anomaly) ++s.anomalies;
++s.by_severity[f.severity];
++s.by_category[f.category];
++s.by_class[f.finding_class];
++s.by_state[f.state];
if (f.resource_name.empty()) continue;
auto it = assets.find(f.resource_name);
if (it == assets.end()) {
if (assets.size() >= ArmorCaps::kMaxDashboardAssets) {
s.truncated = true;
continue;
}
it = assets.emplace(f.resource_name, AssetSummary{f.resource_name, 0, ""}).first;
}
++it->second.findings;
if (rank(f.severity) > rank(it->second.worst_severity))
it->second.worst_severity = f.severity;
}
for (auto& [name, a] : assets) {
(void)name;
s.assets.push_back(std::move(a));
}
std::sort(s.assets.begin(), s.assets.end(), [](const AssetSummary& a, const AssetSummary& b) {
if (rank(a.worst_severity) != rank(b.worst_severity))
return rank(a.worst_severity) > rank(b.worst_severity);
if (a.findings != b.findings) return a.findings > b.findings;
return a.resource_name < b.resource_name;
});
return s;
}
SecuritySummary AgentSecurityDashboard::summarize(FindingQuery query) {
return summarize_findings(findings_.list(with_agent_scope(std::move(query))));
}
std::vector<AssetSummary> AgentSecurityDashboard::discovered_assets(FindingQuery query) {
return summarize(std::move(query)).assets;
}
FindingPage AgentSecurityDashboard::vulnerabilities(FindingQuery query) {
query.finding_class = "VULNERABILITY";
return findings_.list(with_agent_scope(std::move(query)));
}
} // namespace molo::google::armor
+53
View File
@@ -0,0 +1,53 @@
#pragma once
// Agent Security dashboard, LOCAL abstraction: Google documents the console
// dashboard but no dashboard API. This class aggregates the findings that
// SecurityFindings lists (agent assets are matched by resource name) into
// the counts the dashboard shows. Read-only.
#include <map>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "google/armor/findings.h"
namespace molo::google::armor {
struct AssetSummary {
std::string resource_name;
std::size_t findings = 0;
std::string worst_severity;
clients::body::Json to_json() const;
};
struct SecuritySummary {
std::size_t total = 0;
std::size_t agent_findings = 0; // Findings carrying agent context.
std::size_t anomalies = 0; // Findings carrying agentAnomaly.
std::map<std::string, std::size_t> by_severity;
std::map<std::string, std::size_t> by_category;
std::map<std::string, std::size_t> by_class;
std::map<std::string, std::size_t> by_state;
std::vector<AssetSummary> assets; // Worst severity first, capped.
bool truncated = false;
clients::body::Json to_json() const;
};
class AgentSecurityDashboard {
public:
explicit AgentSecurityDashboard(SecurityFindings& findings) : findings_(findings) {}
// resource_contains defaults to "reasoningEngines" (Agent Runtime) when empty.
SecuritySummary summarize(FindingQuery query);
// Asset discovery: distinct resources that carry findings.
std::vector<AssetSummary> discovered_assets(FindingQuery query);
// Findings of class VULNERABILITY for agent assets.
FindingPage vulnerabilities(FindingQuery query);
static SecuritySummary summarize_findings(const FindingPage& page);
private:
static FindingQuery with_agent_scope(FindingQuery query);
SecurityFindings& findings_;
};
} // namespace molo::google::armor
+337
View File
@@ -0,0 +1,337 @@
#include "google/armor/findings.h"
#include <algorithm>
#include "google/armor/armor_util.h"
#include "google/armor/caps.h"
#include "google/errors.h"
#include "google/resource.h"
namespace molo::google::armor {
using clients::body::Json;
namespace {
const Json* member(const Json& doc, const char* key) {
return doc.is_object() ? doc.find(key) : nullptr;
}
std::string text_of(const Json& doc, const char* key) {
const Json* v = member(doc, key);
return v && v->is_string() ? v->as_string() : std::string();
}
// Category names, resource fragments: letters, digits and a few separators.
bool safe_value(const std::string& v) {
if (v.empty() || v.size() > ArmorCaps::kMaxFilterValueChars) return false;
for (char c : v)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
(c >= '0' && c <= '9') || c == '_' || c == '-' || c == '.' ||
c == '/' || c == ':'))
return false;
return true;
}
// Comma separated field paths: letters, digits, '_' and '.'.
bool safe_path_list(const std::string& v) {
if (v.empty() || v.size() > ArmorCaps::kMaxFilterValueChars) return false;
for (char c : v)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
(c >= '0' && c <= '9') || c == '_' || c == '.' || c == ','))
return false;
return true;
}
bool one_of(const std::string& v, std::initializer_list<const char*> allowed) {
return std::any_of(allowed.begin(), allowed.end(),
[&](const char* a) { return v == a; });
}
void validate_order_by(const std::string& order_by) {
std::size_t start = 0;
while (start <= order_by.size()) {
std::size_t comma = order_by.find(',', start);
std::string part = order_by.substr(
start, comma == std::string::npos ? std::string::npos : comma - start);
std::size_t space = part.find(' ');
std::string field = part.substr(0, space);
std::string suffix = space == std::string::npos ? "" : part.substr(space + 1);
// Fields the API documents for orderBy (securitycenter v2 findings.list).
if (!one_of(field, {"name", "parent", "state", "category", "resourceName",
"eventTime", "createTime", "severity", "findingClass"}) ||
(!suffix.empty() && suffix != "desc"))
throw GoogleError::invalid("bad order_by");
if (comma == std::string::npos) break;
start = comma + 1;
}
}
std::vector<std::string> strings_of(const Json& doc, const char* key, const char* field) {
std::vector<std::string> out;
const Json* arr = member(doc, key);
if (!arr || !arr->is_array()) return out;
for (const Json& x : arr->as_array())
if (out.size() < ArmorCaps::kMaxDetailsPerFilter) out.push_back(text_of(x, field));
return out;
}
} // namespace
Json Finding::to_json() const {
Json o = Json::object();
o.set("name", name);
o.set("category", category);
o.set("state", state);
o.set("severity", severity);
o.set("class", finding_class);
o.set("mute", mute);
o.set("resource", resource_name);
o.set("event_time", event_time);
o.set("description", description);
o.set("external_uri", external_uri);
if (is_agent_finding()) {
Json a = Json::object();
a.set("id", agent_id);
a.set("display_name", agent_display_name);
a.set("anomaly", has_anomaly);
if (has_anomaly) {
a.set("confidence", anomaly_confidence);
Json d = Json::array();
for (const auto& x : anomaly_detectors) {
Json item = Json::object();
item.set("detector_id", x.detector_id);
item.set("display_name", x.display_name);
item.set("severity", x.severity);
item.set("explanation", x.explanation);
item.set("recommendation", x.recommendation);
d.push(item);
}
a.set("detectors", d);
}
o.set("agent", a);
}
if (!cve_id.empty()) o.set("cve", cve_id);
return o;
}
Finding Finding::from_json(const Json& doc) {
Finding f;
f.name = text_of(doc, "name");
f.parent = text_of(doc, "parent");
f.category = text_of(doc, "category");
f.state = text_of(doc, "state");
f.severity = text_of(doc, "severity");
f.finding_class = text_of(doc, "findingClass");
f.mute = text_of(doc, "mute");
f.resource_name = text_of(doc, "resourceName");
f.event_time = text_of(doc, "eventTime");
f.create_time = text_of(doc, "createTime");
f.description = text_of(doc, "description");
f.external_uri = text_of(doc, "externalUri");
f.module_name = text_of(doc, "moduleName");
if (const Json* a = member(doc, "agent")) {
f.agent_id = text_of(*a, "id");
f.agent_display_name = text_of(*a, "displayName");
}
if (const Json* an = member(doc, "agentAnomaly")) {
f.has_anomaly = true;
if (const Json* c = member(*an, "confidenceScore"); c && c->is_number())
f.anomaly_confidence = c->as_number();
f.anomaly_invocation_ids = strings_of(*an, "invocationReferences", "invocationId");
if (const Json* det = member(*an, "detectorReferences"); det && det->is_array())
for (const Json& d : det->as_array()) {
if (f.anomaly_detectors.size() >= ArmorCaps::kMaxDetailsPerFilter) break;
f.anomaly_detectors.push_back({text_of(d, "detectorId"), text_of(d, "displayName"),
text_of(d, "severity"), text_of(d, "explanation"),
text_of(d, "recommendation")});
}
}
f.agent_session_ids = strings_of(doc, "agentSessions", "sessionId");
if (const Json* v = member(doc, "vulnerability")) {
f.vulnerability = *v;
if (const Json* cve = member(*v, "cve")) f.cve_id = text_of(*cve, "id");
}
if (const Json* m = member(doc, "aiModel")) f.ai_model = *m;
if (const Json* e = member(doc, "agentDataAccessEvents")) f.agent_data_access_events = *e;
return f;
}
void FindingQuery::validate() const {
if (!valid_scope(scope)) throw GoogleError::invalid("bad findings scope");
if (source != "-" && !valid_segment(source))
throw GoogleError::invalid("bad source id");
if (!location.empty() && !valid_segment(location))
throw GoogleError::invalid("bad location");
if (categories.size() > ArmorCaps::kMaxFilterCategories)
throw GoogleError::invalid("too many categories");
for (const auto& c : categories)
if (!safe_value(c)) throw GoogleError::invalid("bad category");
if (!state.empty() && !one_of(state, {"ACTIVE", "INACTIVE"}))
throw GoogleError::invalid("state must be ACTIVE or INACTIVE");
if (!severity.empty() && !one_of(severity, {"CRITICAL", "HIGH", "MEDIUM", "LOW"}))
throw GoogleError::invalid("severity must be CRITICAL, HIGH, MEDIUM or LOW");
if (!resource_contains.empty() && !safe_value(resource_contains))
throw GoogleError::invalid("bad resource filter");
if (!finding_class.empty() && !safe_value(finding_class))
throw GoogleError::invalid("bad finding class");
if (!order_by.empty()) validate_order_by(order_by);
if (!field_mask.empty() && !safe_path_list(field_mask))
throw GoogleError::invalid("bad field mask");
if (page_size < 0 || page_size > ArmorCaps::kMaxFindingsPageSize)
throw GoogleError::invalid("page_size must be 0 to 1000");
if (max_pages == 0 || max_pages > ArmorCaps::kMaxFindingPages)
throw GoogleError::invalid("max_pages must be 1 to 10");
if (max_results > ArmorCaps::kMaxFindings)
throw GoogleError::invalid("max_results must be at most 5000");
}
std::string FindingQuery::parent_path() const {
validate();
std::string path = "/" + scope + "/sources/" + source;
if (!location.empty()) path += "/locations/" + location;
return path;
}
std::string FindingQuery::filter() const {
validate();
std::vector<std::string> terms;
if (!categories.empty()) {
std::string group = "(";
for (std::size_t i = 0; i < categories.size(); ++i) {
if (i) group += " OR ";
group += "category=\"" + categories[i] + "\"";
}
terms.push_back(group + ")");
}
if (!state.empty()) terms.push_back("state=\"" + state + "\"");
if (!severity.empty()) terms.push_back("severity=\"" + severity + "\"");
if (!resource_contains.empty())
terms.push_back("resourceName:\"" + resource_contains + "\"");
std::string out;
for (const auto& t : terms) out += (out.empty() ? "" : " AND ") + t;
return out;
}
Json SecuritySource::to_json() const {
Json o = Json::object();
o.set("name", name);
o.set("display_name", display_name);
o.set("description", description);
o.set("canonical_name", canonical_name);
return o;
}
ServiceEndpoint SecurityFindings::endpoint() {
return endpoints::custom("securitycenter", "https://securitycenter.googleapis.com/v2");
}
SecurityFindings::SecurityFindings(std::shared_ptr<GoogleClient> client)
: Service(std::move(client), endpoint()) {}
FindingPage SecurityFindings::list(const FindingQuery& query) {
std::string filter = query.filter(); // Validates everything.
std::string base = query.parent_path() + "/findings";
int size = query.page_size > 0 ? query.page_size : ArmorCaps::kDefaultFindingsPageSize;
std::string q = "?pageSize=" + std::to_string(size);
if (!filter.empty()) q += "&filter=" + query_escape(filter);
if (!query.order_by.empty()) q += "&orderBy=" + query_escape(query.order_by);
if (!query.field_mask.empty()) q += "&fieldMask=" + query_escape(query.field_mask);
std::size_t cap = query.max_results ? query.max_results : ArmorCaps::kMaxFindings;
FindingPage page;
std::string token;
for (std::size_t pages = 0; pages < query.max_pages; ++pages) {
std::string path = base + q;
if (!token.empty()) path += "&pageToken=" + query_escape(token);
Json doc = fetch(path);
if (pages == 0)
if (const Json* t = doc.find("totalSize"); t && t->is_number())
page.total_size = t->as_int();
if (const Json* items = doc.find("listFindingsResults"); items && items->is_array())
for (const Json& item : items->as_array()) {
const Json* f = member(item, "finding");
if (!f) continue;
Finding finding = Finding::from_json(*f);
if (!query.finding_class.empty() && finding.finding_class != query.finding_class)
continue;
if (page.findings.size() >= cap) {
page.truncated = true;
return page;
}
page.findings.push_back(std::move(finding));
}
const Json* next = doc.find("nextPageToken");
if (!next || !next->is_string() || next->as_string().empty()) return page;
token = next->as_string();
}
page.truncated = true; // A next page existed beyond max_pages.
return page;
}
std::vector<FindingGroup> SecurityFindings::group(const FindingQuery& query,
const std::string& group_by) {
std::string filter = query.filter();
std::string path = query.parent_path() + "/findings:group";
if (!safe_path_list(group_by)) throw GoogleError::invalid("bad group_by");
Json body = Json::object();
body.set("groupBy", group_by);
if (!filter.empty()) body.set("filter", filter);
int size = query.page_size > 0 ? query.page_size : ArmorCaps::kDefaultFindingsPageSize;
body.set("pageSize", size);
std::vector<FindingGroup> out;
std::string token;
for (std::size_t pages = 0; pages < query.max_pages; ++pages) {
if (!token.empty()) body.set("pageToken", token);
Json doc = call(path, body);
if (const Json* rows = doc.find("groupByResults"); rows && rows->is_array())
for (const Json& r : rows->as_array()) {
if (out.size() >= ArmorCaps::kMaxGroupResults) return out;
FindingGroup g;
// count is an int64 string on the wire.
const Json* c = member(r, "count");
if (c && c->is_string()) g.count = std::strtol(c->as_string().c_str(), nullptr, 10);
else if (c && c->is_number()) g.count = c->as_int();
if (const Json* p = member(r, "properties")) g.properties = *p;
out.push_back(std::move(g));
}
const Json* next = doc.find("nextPageToken");
if (!next || !next->is_string() || next->as_string().empty()) break;
token = next->as_string();
}
return out;
}
std::vector<SecuritySource> SecurityFindings::list_sources(const std::string& scope) {
if (!valid_scope(scope)) throw GoogleError::invalid("bad sources scope");
std::vector<SecuritySource> out;
std::string token;
for (int pages = 0; pages < 5; ++pages) {
std::string path = "/" + scope + "/sources?pageSize=100";
if (!token.empty()) path += "&pageToken=" + query_escape(token);
Json doc = fetch(path);
if (const Json* items = doc.find("sources"); items && items->is_array())
for (const Json& s : items->as_array()) {
if (out.size() >= ArmorCaps::kMaxSources) return out;
out.push_back({text_of(s, "name"), text_of(s, "displayName"),
text_of(s, "description"), text_of(s, "canonicalName")});
}
const Json* next = doc.find("nextPageToken");
if (!next || !next->is_string() || next->as_string().empty()) break;
token = next->as_string();
}
return out;
}
SecuritySource SecurityFindings::get_source(const std::string& name) {
// organizations/{id}/sources/{id} (the only scope with a get method).
std::size_t slash = name.find("/sources/");
if (slash == std::string::npos || name.rfind("organizations/", 0) != 0 ||
!valid_scope(name.substr(0, slash)) || !valid_segment(name.substr(slash + 9)))
throw GoogleError::invalid("source name must be organizations/{id}/sources/{id}");
Json s = fetch("/" + name);
return {text_of(s, "name"), text_of(s, "displayName"), text_of(s, "description"),
text_of(s, "canonicalName")};
}
} // namespace molo::google::armor
+123
View File
@@ -0,0 +1,123 @@
#pragma once
// Read-only Security Command Center findings (API v2). Agent Threat
// Detection and Agent Anomaly Detection have no API of their own: they
// publish findings to Security Command Center, whose Finding resource has
// typed `agent`, `agentAnomaly`, `agentSessions`, `agentDataAccessEvents`,
// `aiModel` and `vulnerability` members (Discovery, securitycenter v2).
// Host: https://securitycenter.googleapis.com/v2. Auth: OAuth2 access token.
#include <map>
#include <memory>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "google/service.h"
namespace molo::google::armor {
// agentAnomaly.detectorReferences[]
struct AnomalyDetector {
std::string detector_id;
std::string display_name;
std::string severity;
std::string explanation;
std::string recommendation;
};
struct Finding {
std::string name; // organizations/1/sources/2/locations/global/findings/3
std::string parent;
std::string category;
std::string state; // ACTIVE | INACTIVE
std::string severity; // CRITICAL | HIGH | MEDIUM | LOW
std::string finding_class; // THREAT | VULNERABILITY | MISCONFIGURATION | ...
std::string mute; // MUTED | UNMUTED | UNDEFINED
std::string resource_name;
std::string event_time;
std::string create_time;
std::string description;
std::string external_uri;
std::string module_name;
// Agent context (empty when the finding is not about an agent).
std::string agent_id;
std::string agent_display_name;
bool has_anomaly = false;
double anomaly_confidence = 0.0;
std::vector<std::string> anomaly_invocation_ids;
std::vector<AnomalyDetector> anomaly_detectors;
std::vector<std::string> agent_session_ids;
// Vulnerability context.
std::string cve_id;
// Kept raw: shapes the dashboard does not need typed.
clients::body::Json ai_model;
clients::body::Json agent_data_access_events;
clients::body::Json vulnerability;
bool is_agent_finding() const { return !agent_id.empty() || has_anomaly; }
clients::body::Json to_json() const;
static Finding from_json(const clients::body::Json& doc); // A Finding object.
};
struct FindingQuery {
// "organizations/{id}", "folders/{id}" or "projects/{id}".
std::string scope;
std::string source = "-"; // "-": every source.
std::string location; // Empty: no /locations/ segment.
std::vector<std::string> categories; // ORed.
std::string state; // "", ACTIVE, INACTIVE
std::string severity; // "", CRITICAL, HIGH, MEDIUM, LOW
// Substring of resourceName, e.g. "reasoningEngines" for agent assets.
std::string resource_contains;
// Applied locally after listing (findingClass is orderBy-only in the API).
std::string finding_class;
std::string order_by; // "", "eventTime desc", "severity", ...
std::string field_mask; // "", or comma separated Finding field paths
int page_size = 0; // 0: ArmorCaps::kDefaultFindingsPageSize
std::size_t max_pages = 1;
std::size_t max_results = 0; // 0: ArmorCaps::kMaxFindings
// The AIP-160 filter built from the typed fields; "" when none are set.
// Throws GoogleError::invalid for any malformed value.
std::string filter() const;
void validate() const;
// "/{scope}/sources/{source}[/locations/{l}]" (validated).
std::string parent_path() const;
};
struct FindingPage {
std::vector<Finding> findings;
long total_size = -1; // totalSize of the first page; -1 when absent.
bool truncated = false; // More existed than max_results or max_pages.
};
struct SecuritySource {
std::string name; // organizations/1/sources/2
std::string display_name;
std::string description;
std::string canonical_name;
clients::body::Json to_json() const;
};
// findings:group result row.
struct FindingGroup {
long count = 0;
clients::body::Json properties;
};
class SecurityFindings : public Service {
public:
explicit SecurityFindings(std::shared_ptr<GoogleClient> client);
static ServiceEndpoint endpoint();
FindingPage list(const FindingQuery& query);
// Server side counts (findings:group). group_by: comma separated fields,
// e.g. "severity" or "category,state".
std::vector<FindingGroup> group(const FindingQuery& query, const std::string& group_by);
// Finding sources of a scope (Agent Threat Detection and friends).
std::vector<SecuritySource> list_sources(const std::string& scope);
// organizations/{id}/sources/{id}
SecuritySource get_source(const std::string& name);
};
} // namespace molo::google::armor
+155
View File
@@ -0,0 +1,155 @@
#include "google/armor/guarded_model.h"
#include "google/armor/caps.h"
#include "google/errors.h"
#include "google/resource.h"
namespace molo::google::armor {
namespace {
std::string describe(GuardStage stage, const SanitizationResult& r, bool incomplete) {
std::string out = std::string("model armor blocked the ") +
(stage == GuardStage::kPrompt ? "prompt" : "response");
if (incomplete) return out + ": filters did not all run";
out += ":";
for (const auto& f : r.matches()) out += std::string(" ") + filter_kind_name(f.kind);
return out;
}
} // namespace
BlockedError::BlockedError(GuardStage stage, SanitizationResult result, bool incomplete)
: std::runtime_error(describe(stage, result, incomplete)),
stage_(stage),
result_(std::move(result)),
incomplete_(incomplete) {}
std::vector<FilterKind> BlockedError::filters() const {
std::vector<FilterKind> out;
for (const auto& f : result_.matches()) out.push_back(f.kind);
return out;
}
ModelArmorSanitizer::ModelArmorSanitizer(std::shared_ptr<ModelArmor> armor,
std::string template_id)
: armor_(std::move(armor)), template_id_(std::move(template_id)) {
if (!armor_) throw GoogleError::invalid("sanitizer needs model armor");
if (!valid_segment(template_id_))
throw GoogleError::invalid("guard template id is missing or malformed");
}
SanitizationResult ModelArmorSanitizer::sanitize_prompt(const std::string& text) {
return armor_->sanitize_user_prompt(template_id_, text);
}
SanitizationResult ModelArmorSanitizer::sanitize_response(const std::string& text,
const std::string& prompt) {
return armor_->sanitize_model_response(template_id_, text, prompt);
}
GuardedModel::GuardedModel(std::shared_ptr<LanguageModel> inner,
std::shared_ptr<Sanitizer> sanitizer, GuardOptions options)
: inner_(std::move(inner)), sanitizer_(std::move(sanitizer)), options_(std::move(options)) {
if (!inner_ || !sanitizer_)
throw GoogleError::invalid("guarded model needs a model and a sanitizer");
}
std::shared_ptr<GuardedModel> GuardedModel::with_armor(std::shared_ptr<LanguageModel> inner,
std::shared_ptr<ModelArmor> armor,
const std::string& template_id,
GuardOptions options) {
return std::make_shared<GuardedModel>(
std::move(inner), std::make_shared<ModelArmorSanitizer>(std::move(armor), template_id),
std::move(options));
}
std::string GuardedModel::prompt_text(const GenerateRequest& request) {
std::string out;
auto add = [&](const Content& c) {
std::string t = c.text();
if (t.empty()) return;
if (!out.empty()) out += "\n";
out += t;
};
if (request.system) add(*request.system);
for (const auto& c : request.contents)
if (c.role != "model") add(c);
return out;
}
void GuardedModel::judge(GuardStage stage, const SanitizationResult& result) {
auto& blocked = stage == GuardStage::kPrompt ? prompts_blocked_ : responses_blocked_;
if (result.matched()) {
++blocked;
throw BlockedError(stage, result, false);
}
if (!options_.allow_partial && !result.fully_executed()) {
++blocked;
throw BlockedError(stage, result, true);
}
}
void GuardedModel::check_prompt(const std::string& text) {
if (!options_.check_prompt) return;
if (text.empty()) {
++skipped_empty_;
return;
}
++prompts_checked_;
judge(GuardStage::kPrompt, sanitizer_->sanitize_prompt(text));
}
void GuardedModel::check_response(const std::string& text, const std::string& prompt) {
if (!options_.check_response) return;
if (text.empty()) {
++skipped_empty_;
return;
}
++responses_checked_;
// The prompt is context only; an oversized one is dropped, not an error.
const std::string& context = prompt.size() <= ArmorCaps::kMaxTextChars ? prompt : std::string();
judge(GuardStage::kResponse, sanitizer_->sanitize_response(text, context));
}
GenerateResponse GuardedModel::generate(const GenerateRequest& request) {
std::string prompt = prompt_text(request);
check_prompt(prompt);
GenerateResponse response = inner_->generate(request);
std::string text;
for (const auto& c : response.candidates) {
std::string t = c.content.text();
if (t.empty()) continue;
if (!text.empty()) text += "\n";
text += t;
}
check_response(text, prompt);
return response;
}
void GuardedModel::stream(const GenerateRequest& request,
const std::function<bool(const GenerateResponse&)>& on_chunk) {
std::string prompt = prompt_text(request);
check_prompt(prompt);
std::vector<GenerateResponse> chunks;
std::string text;
inner_->stream(request, [&](const GenerateResponse& chunk) {
if (chunks.size() >= ArmorCaps::kMaxStreamChunks)
throw GoogleError::invalid("guarded stream exceeds the chunk buffer");
text += chunk.text();
if (text.size() > ArmorCaps::kMaxStreamBufferChars)
throw GoogleError::invalid("guarded stream exceeds the text buffer");
chunks.push_back(chunk);
return true;
});
check_response(text, prompt);
for (const auto& chunk : chunks)
if (!on_chunk(chunk)) break;
}
GuardStats GuardedModel::stats() const {
return {prompts_checked_.load(), prompts_blocked_.load(), responses_checked_.load(),
responses_blocked_.load(), skipped_empty_.load()};
}
} // namespace molo::google::armor
+109
View File
@@ -0,0 +1,109 @@
#pragma once
// GuardedModel: a LanguageModel decorator. Sanitizes the prompt through
// Model Armor before the wrapped model runs and the response after, and
// throws BlockedError (carrying the matched filters) instead of returning
// blocked content. Fail closed: an incomplete Model Armor run also blocks
// unless GuardOptions::allow_partial is set.
//
// Streaming: chunks are buffered until the stream ends, the whole text is
// sanitized, then the chunks are replayed. The caller sees nothing of a
// blocked response. Cost: no incremental output while guarded.
#include <atomic>
#include <memory>
#include <stdexcept>
#include <string>
#include <vector>
#include "google/armor/model_armor.h"
#include "google/gemini_types.h"
namespace molo::google::armor {
enum class GuardStage { kPrompt, kResponse };
class BlockedError : public std::runtime_error {
public:
BlockedError(GuardStage stage, SanitizationResult result, bool incomplete);
GuardStage stage() const noexcept { return stage_; }
const SanitizationResult& result() const noexcept { return result_; }
// True: blocked because filters did not all run, not because one matched.
bool incomplete() const noexcept { return incomplete_; }
std::vector<FilterKind> filters() const; // Matched filter kinds.
private:
GuardStage stage_;
SanitizationResult result_;
bool incomplete_;
};
// The sanitizing seam. ModelArmorSanitizer is the production implementation;
// tests use a local fake.
class Sanitizer {
public:
virtual ~Sanitizer() = default;
virtual SanitizationResult sanitize_prompt(const std::string& text) = 0;
virtual SanitizationResult sanitize_response(const std::string& text,
const std::string& prompt) = 0;
};
// Sanitizes through one Model Armor template.
class ModelArmorSanitizer : public Sanitizer {
public:
ModelArmorSanitizer(std::shared_ptr<ModelArmor> armor, std::string template_id);
SanitizationResult sanitize_prompt(const std::string& text) override;
SanitizationResult sanitize_response(const std::string& text,
const std::string& prompt) override;
private:
std::shared_ptr<ModelArmor> armor_;
std::string template_id_;
};
struct GuardOptions {
bool check_prompt = true;
bool check_response = true;
// Accept PARTIAL/skipped runs when no filter matched.
bool allow_partial = false;
};
struct GuardStats {
std::size_t prompts_checked = 0;
std::size_t prompts_blocked = 0;
std::size_t responses_checked = 0;
std::size_t responses_blocked = 0;
std::size_t skipped_empty = 0; // Nothing textual to sanitize.
};
class GuardedModel : public LanguageModel {
public:
GuardedModel(std::shared_ptr<LanguageModel> inner, std::shared_ptr<Sanitizer> sanitizer,
GuardOptions options = {});
// Convenience: guard through a Model Armor template.
static std::shared_ptr<GuardedModel> with_armor(std::shared_ptr<LanguageModel> inner,
std::shared_ptr<ModelArmor> armor,
const std::string& template_id,
GuardOptions options = {});
GenerateResponse generate(const GenerateRequest& request) override;
void stream(const GenerateRequest& request,
const std::function<bool(const GenerateResponse&)>& on_chunk) override;
GuardStats stats() const;
// Text sent for prompt sanitization: system plus user turns.
static std::string prompt_text(const GenerateRequest& request);
private:
void check_prompt(const std::string& text);
void check_response(const std::string& text, const std::string& prompt);
void judge(GuardStage stage, const SanitizationResult& result);
std::shared_ptr<LanguageModel> inner_;
std::shared_ptr<Sanitizer> sanitizer_;
GuardOptions options_;
std::atomic<std::size_t> prompts_checked_{0}, prompts_blocked_{0};
std::atomic<std::size_t> responses_checked_{0}, responses_blocked_{0};
std::atomic<std::size_t> skipped_empty_{0};
};
} // namespace molo::google::armor
+283
View File
@@ -0,0 +1,283 @@
#include "google/armor/model_armor.h"
#include "google/armor/armor_util.h"
#include "google/armor/caps.h"
#include "google/errors.h"
#include "google/operation.h"
#include "security/crypto/Base64.h"
namespace molo::google::armor {
using clients::body::Json;
namespace {
bool valid_byte_type(const std::string& t) {
return t == "PLAINTEXT_UTF8" || t == "PDF" || t == "WORD_DOCUMENT" ||
t == "EXCEL_DOCUMENT" || t == "POWERPOINT_DOCUMENT" || t == "TXT" ||
t == "CSV" || t == "IMAGE";
}
bool valid_filter_text(const std::string& v) {
if (v.size() > ArmorCaps::kMaxFilterValueChars) return false;
for (unsigned char c : v)
if (c < 0x20 || c == 0x7f) return false;
return true;
}
void validate_hint(const LanguageHint& hint) {
if (hint.source_language.size() > 35)
throw GoogleError::invalid("source language too long");
for (char c : hint.source_language)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == '-'))
throw GoogleError::invalid("bad source language");
}
} // namespace
DataItem DataItem::of_text(std::string text) {
DataItem d;
d.text = std::move(text);
return d;
}
DataItem DataItem::of_bytes(std::string byte_type, std::string bytes,
std::string file_label) {
DataItem d;
d.byte_type = std::move(byte_type);
d.bytes = std::move(bytes);
d.file_label = std::move(file_label);
return d;
}
void DataItem::validate(const char* what) const {
bool is_bytes = !byte_type.empty() || !bytes.empty();
if (is_bytes && !text.empty())
throw GoogleError::invalid(std::string(what) + " is text or bytes, not both");
if (!is_bytes) {
if (text.empty()) throw GoogleError::invalid(std::string(what) + " is empty");
if (text.size() > ArmorCaps::kMaxTextChars)
throw GoogleError::invalid(std::string(what) + " exceeds 100000 characters");
return;
}
if (!valid_byte_type(byte_type))
throw GoogleError::invalid("byte type is not a documented ByteItemType");
if (bytes.empty()) throw GoogleError::invalid(std::string(what) + " bytes are empty");
if (bytes.size() > ArmorCaps::kMaxByteItemBytes)
throw GoogleError::invalid(std::string(what) + " bytes exceed 5 MiB");
if (file_label.size() > 128) throw GoogleError::invalid("file label too long");
}
Json DataItem::to_json() const {
Json o = Json::object();
if (byte_type.empty() && bytes.empty()) {
o.set("text", text);
return o;
}
Json b = Json::object();
b.set("byteDataType", byte_type);
b.set("byteData", security::crypto::Base64::encode(bytes));
if (!file_label.empty()) b.set("fileLabel", file_label);
o.set("byteItem", b);
return o;
}
ServiceEndpoint ModelArmor::endpoint_for(const std::string& location) {
if (location == "global")
return endpoints::custom("modelarmor", "https://modelarmor.googleapis.com/v1");
return endpoints::custom(
"modelarmor", "https://modelarmor." + location + ".rep.googleapis.com/v1");
}
ModelArmor::ModelArmor(std::shared_ptr<GoogleClient> client, CloudContext context)
: Service(std::move(client),
endpoint_for(context.valid() ? context.location : "us-central1")),
context_(std::move(context)) {}
void ModelArmor::require_regional() const {
if (context_.location == "global")
throw GoogleError::invalid("templates need a regional location, not global");
}
std::string ModelArmor::template_path(const std::string& id) const {
require_regional();
return "/" + context_.child("templates", id);
}
ServiceEndpoint ModelArmor::floor_endpoint() const {
return endpoint_for(context_.location);
}
std::string ModelArmor::floor_path(const std::string& scope) const {
std::string s = scope;
context_.parent(); // Validates project and location.
if (s.empty()) s = "projects/" + context_.project;
if (!valid_scope(s)) throw GoogleError::invalid("bad floor setting scope");
return "/" + s + "/locations/" + context_.location + "/floorSetting";
}
std::string ModelArmor::request_id_query(const std::string& request_id, char sep) {
if (request_id.empty()) return "";
if (!valid_request_id(request_id))
throw GoogleError::invalid("request id must be a non-zero UUID");
return std::string(1, sep) + "requestId=" + request_id;
}
Template ModelArmor::create_template(const std::string& id, const Template& tmpl,
const std::string& request_id) {
template_path(id); // Validates the id and the location.
std::string rid = request_id_query(request_id, '&');
tmpl.validate();
return Template::from_json(call(
"/" + context_.parent() + "/templates?templateId=" + query_escape(id) + rid,
tmpl.to_json()));
}
Template ModelArmor::get_template(const std::string& id) {
return Template::from_json(fetch(template_path(id)));
}
TemplatePage ModelArmor::list_templates(int page_size, const std::string& filter,
const std::string& order_by) {
require_regional();
if (page_size < 0) throw GoogleError::invalid("page_size must not be negative");
if (page_size > ArmorCaps::kMaxTemplatePageSize)
page_size = ArmorCaps::kMaxTemplatePageSize;
if (!valid_filter_text(filter) || !valid_filter_text(order_by))
throw GoogleError::invalid("bad filter or order_by");
std::string base = "/" + context_.parent() + "/templates";
std::string query;
auto add = [&](const std::string& k, const std::string& v) {
query += (query.empty() ? "?" : "&") + k + "=" + query_escape(v);
};
if (page_size > 0) add("pageSize", std::to_string(page_size));
if (!filter.empty()) add("filter", filter);
if (!order_by.empty()) add("orderBy", order_by);
TemplatePage page;
std::string token;
for (std::size_t pages = 0; pages < ArmorCaps::kMaxTemplatePages; ++pages) {
std::string path = base + query;
if (!token.empty()) path += (query.empty() ? "?" : "&") + std::string("pageToken=") + query_escape(token);
Json doc = fetch(path);
if (const Json* t = doc.find("templates"); t && t->is_array())
for (const Json& item : t->as_array()) page.templates.push_back(Template::from_json(item));
if (const Json* u = doc.find("unreachable"); u && u->is_array())
for (const Json& x : u->as_array())
if (x.is_string() && page.unreachable.size() < ArmorCaps::kMaxLocations)
page.unreachable.push_back(x.as_string());
const Json* next = doc.find("nextPageToken");
if (!next || !next->is_string() || next->as_string().empty()) break;
token = next->as_string();
}
return page;
}
Template ModelArmor::patch_template(const std::string& id, const Template& tmpl,
const std::vector<std::string>& update_mask,
const std::string& request_id) {
std::string path = template_path(id);
std::string mask = join_update_mask(update_mask);
std::string rid = request_id_query(request_id, '&');
tmpl.validate();
return Template::from_json(
patch(path + "?updateMask=" + query_escape(mask) + rid, tmpl.to_json()));
}
void ModelArmor::delete_template(const std::string& id, const std::string& request_id) {
std::string path = template_path(id);
remove(path + request_id_query(request_id, '?'));
}
FloorSetting ModelArmor::get_floor_setting(const std::string& scope) {
return FloorSetting::from_json(client_->get_json(floor_endpoint(), floor_path(scope)));
}
FloorSetting ModelArmor::update_floor_setting(
const FloorSetting& setting, const std::vector<std::string>& update_mask,
const std::string& scope) {
std::string path = floor_path(scope);
std::string mask = join_update_mask(update_mask);
setting.validate();
return FloorSetting::from_json(client_->patch_json(
floor_endpoint(), path + "?updateMask=" + query_escape(mask), setting.to_json()));
}
std::vector<ArmorLocation> ModelArmor::list_locations() {
context_.parent();
Json items = list_all(*client_, floor_endpoint(),
"/projects/" + context_.project + "/locations", "locations", 2);
std::vector<ArmorLocation> out;
for (const Json& l : items.as_array()) {
if (out.size() >= ArmorCaps::kMaxLocations) break;
out.push_back({string_at(l, "name"), string_at(l, "locationId"), string_at(l, "displayName")});
}
return out;
}
ArmorLocation ModelArmor::get_location() {
Json l = client_->get_json(floor_endpoint(), "/" + context_.parent());
return {string_at(l, "name"), string_at(l, "locationId"), string_at(l, "displayName")};
}
Json ModelArmor::user_prompt_body(const DataItem& item, const LanguageHint& hint) {
Json body = Json::object();
body.set("userPromptData", item.to_json());
if (hint.present()) {
Json m = Json::object();
m.set("enableMultiLanguageDetection", hint.enable_detection);
if (!hint.source_language.empty()) m.set("sourceLanguage", hint.source_language);
body.set("multiLanguageDetectionMetadata", m);
}
return body;
}
Json ModelArmor::model_response_body(const DataItem& item, const std::string& user_prompt,
const LanguageHint& hint) {
Json body = Json::object();
body.set("modelResponseData", item.to_json());
if (!user_prompt.empty()) body.set("userPrompt", user_prompt);
if (hint.present()) {
Json m = Json::object();
m.set("enableMultiLanguageDetection", hint.enable_detection);
if (!hint.source_language.empty()) m.set("sourceLanguage", hint.source_language);
body.set("multiLanguageDetectionMetadata", m);
}
return body;
}
SanitizationResult ModelArmor::sanitize_user_prompt(const std::string& template_id,
const std::string& text) {
return sanitize_user_prompt(template_id, DataItem::of_text(text));
}
SanitizationResult ModelArmor::sanitize_user_prompt(const std::string& template_id,
const DataItem& item,
const LanguageHint& hint) {
std::string path = template_path(template_id);
item.validate("prompt");
validate_hint(hint);
return SanitizationResult::from_json(
call(path + ":sanitizeUserPrompt", user_prompt_body(item, hint)));
}
SanitizationResult ModelArmor::sanitize_model_response(const std::string& template_id,
const std::string& text,
const std::string& user_prompt) {
return sanitize_model_response(template_id, DataItem::of_text(text), user_prompt);
}
SanitizationResult ModelArmor::sanitize_model_response(const std::string& template_id,
const DataItem& item,
const std::string& user_prompt,
const LanguageHint& hint) {
std::string path = template_path(template_id);
item.validate("response");
validate_hint(hint);
if (user_prompt.size() > ArmorCaps::kMaxTextChars)
throw GoogleError::invalid("user prompt exceeds 100000 characters");
return SanitizationResult::from_json(
call(path + ":sanitizeModelResponse", model_response_body(item, user_prompt, hint)));
}
} // namespace molo::google::armor
+109
View File
@@ -0,0 +1,109 @@
#pragma once
// Model Armor: templates CRUD, floor settings, locations, sanitizeUserPrompt
// and sanitizeModelResponse. Regional host
// https://modelarmor.{location}.rep.googleapis.com/v1; floor settings also
// accept location "global" (https://modelarmor.googleapis.com/v1).
// Auth: OAuth2 access token only (GOOGLE_ACCESS_TOKEN); API keys are refused
// by this API (verified live, see SPEC.md).
#include <memory>
#include <string>
#include <vector>
#include "google/armor/armor_types.h"
#include "google/armor/template.h"
#include "google/resource.h"
#include "google/service.h"
namespace molo::google::armor {
// DataItem: text, or file bytes with a ByteItemType.
struct DataItem {
std::string text;
std::string bytes; // Raw bytes (base64 on the wire).
std::string byte_type; // PLAINTEXT_UTF8, PDF, WORD_DOCUMENT, EXCEL_DOCUMENT,
// POWERPOINT_DOCUMENT, TXT, CSV, IMAGE
std::string file_label;
static DataItem of_text(std::string text);
static DataItem of_bytes(std::string byte_type, std::string bytes,
std::string file_label = "");
void validate(const char* what) const;
clients::body::Json to_json() const;
};
// multiLanguageDetectionMetadata on both sanitize requests.
struct LanguageHint {
bool enable_detection = false;
std::string source_language; // BCP-47-ish, optional
bool present() const { return enable_detection || !source_language.empty(); }
};
class ModelArmor : public Service {
public:
ModelArmor(std::shared_ptr<GoogleClient> client, CloudContext context);
static ServiceEndpoint endpoint_for(const std::string& location);
const CloudContext& context() const { return context_; }
// Templates (location must be regional, not "global"). request_id is an
// optional retry id (UUID, not all zero); empty: none.
Template create_template(const std::string& id, const Template& tmpl,
const std::string& request_id = "");
Template get_template(const std::string& id);
// page_size 0: server default; capped at ArmorCaps::kMaxTemplatePageSize.
// filter and order_by are passed through after a character check.
TemplatePage list_templates(int page_size = 0, const std::string& filter = "",
const std::string& order_by = "");
// update_mask: Template field paths, e.g. {"filterConfig"}.
Template patch_template(const std::string& id, const Template& tmpl,
const std::vector<std::string>& update_mask,
const std::string& request_id = "");
// Explicit and destructive; there is no default or bulk form.
void delete_template(const std::string& id, const std::string& request_id = "");
// Floor settings. scope is "projects/{id}", "folders/{id}" or
// "organizations/{id}"; empty: the context's project.
FloorSetting get_floor_setting(const std::string& scope = "");
FloorSetting update_floor_setting(const FloorSetting& setting,
const std::vector<std::string>& update_mask,
const std::string& scope = "");
// projects.locations.list / get on the Model Armor host.
std::vector<ArmorLocation> list_locations();
ArmorLocation get_location();
// Sanitization. Text is 1..ArmorCaps::kMaxTextChars characters.
SanitizationResult sanitize_user_prompt(const std::string& template_id,
const std::string& text);
SanitizationResult sanitize_user_prompt(const std::string& template_id,
const DataItem& item,
const LanguageHint& hint = {});
SanitizationResult sanitize_model_response(const std::string& template_id,
const std::string& text,
const std::string& user_prompt = "");
SanitizationResult sanitize_model_response(const std::string& template_id,
const DataItem& item,
const std::string& user_prompt = "",
const LanguageHint& hint = {});
// Request builders, public so tests can compare them to the discovery
// schema without any network.
static clients::body::Json user_prompt_body(const DataItem& item,
const LanguageHint& hint);
static clients::body::Json model_response_body(const DataItem& item,
const std::string& user_prompt,
const LanguageHint& hint);
// Resource paths (leading slash), validated.
std::string template_path(const std::string& id) const;
std::string floor_path(const std::string& scope) const;
private:
ServiceEndpoint floor_endpoint() const;
void require_regional() const;
static std::string request_id_query(const std::string& request_id, char sep);
CloudContext context_;
};
} // namespace molo::google::armor
+398
View File
@@ -0,0 +1,398 @@
#include "google/armor/template.h"
#include "google/armor/caps.h"
#include "google/errors.h"
namespace molo::google::armor {
using clients::body::Json;
namespace {
const Json* member(const Json& doc, const char* key) {
return doc.is_object() ? doc.find(key) : nullptr;
}
std::string text_of(const Json& doc, const char* key) {
const Json* v = member(doc, key);
return v && v->is_string() ? v->as_string() : std::string();
}
bool flag_of(const Json& doc, const char* key) {
const Json* v = member(doc, key);
return v && v->is_bool() && v->as_bool();
}
int int_of(const Json& doc, const char* key) {
const Json* v = member(doc, key);
return v && v->is_number() ? static_cast<int>(v->as_int()) : 0;
}
bool valid_rai_type(const std::string& t) {
return t == "SEXUALLY_EXPLICIT" || t == "HATE_SPEECH" || t == "HARASSMENT" ||
t == "DANGEROUS";
}
bool upper_name(const std::string& v) {
if (v.empty() || v.size() > 64) return false;
for (char c : v)
if (!((c >= 'A' && c <= 'Z') || c == '_')) return false;
return true;
}
Json enabled_json(bool on) {
Json o = Json::object();
o.set("filterEnforcement", on ? "ENABLED" : "DISABLED");
return o;
}
bool enabled(const Json* settings) {
return settings && text_of(*settings, "filterEnforcement") == "ENABLED";
}
Json enforcement_json(const FloorEnforcement& e) {
Json p = Json::object();
p.set("enableCloudLogging", e.cloud_logging);
p.set(e.inspect_only ? "inspectOnly" : "inspectAndBlock", true);
return p;
}
FloorEnforcement enforcement_from(const Json& p) {
FloorEnforcement e;
e.present = true;
e.cloud_logging = flag_of(p, "enableCloudLogging");
e.inspect_only = flag_of(p, "inspectOnly");
return e;
}
Json rule_sets_json(const std::vector<FilterRuleSet>& sets) {
Json arr = Json::array();
for (const auto& rs : sets) {
Json set = Json::object();
Json types = Json::array();
for (const auto& t : rs.filter_types) types.push(t);
set.set("filterTypes", types);
Json rules = Json::array();
for (const auto& r : rs.rules) {
Json ex = Json::object();
if (!r.matching_scope.empty()) ex.set("matchingScope", r.matching_scope);
if (!r.regex.empty()) {
Json re = Json::object();
re.set("pattern", r.regex);
ex.set("regex", re);
} else {
Json words = Json::array();
for (const auto& w : r.words) words.push(w);
Json list = Json::object();
list.set("words", words);
Json dict = Json::object();
dict.set("wordList", list);
ex.set("dictionary", dict);
}
Json rule = Json::object();
rule.set("exclusionRule", ex);
rules.push(rule);
}
set.set("rules", rules);
arr.push(set);
}
Json settings = Json::object();
settings.set("ruleSets", arr);
return settings;
}
std::vector<FilterRuleSet> rule_sets_from(const Json& settings) {
std::vector<FilterRuleSet> out;
const Json* sets = member(settings, "ruleSets");
if (!sets || !sets->is_array()) return out;
for (const Json& set : sets->as_array()) {
if (out.size() >= ArmorCaps::kMaxRuleSets) break;
FilterRuleSet rs;
if (const Json* t = member(set, "filterTypes"); t && t->is_array())
for (const Json& x : t->as_array())
if (x.is_string()) rs.filter_types.push_back(x.as_string());
if (const Json* rules = member(set, "rules"); rules && rules->is_array())
for (const Json& rule : rules->as_array()) {
const Json* ex = member(rule, "exclusionRule");
if (!ex || rs.rules.size() >= ArmorCaps::kMaxRulesPerSet) continue;
ExclusionRule er;
er.matching_scope = text_of(*ex, "matchingScope");
if (const Json* re = member(*ex, "regex")) er.regex = text_of(*re, "pattern");
if (const Json* dict = member(*ex, "dictionary"))
if (const Json* wl = member(*dict, "wordList"))
if (const Json* w = member(*wl, "words"); w && w->is_array())
for (const Json& x : w->as_array())
if (x.is_string() && er.words.size() < ArmorCaps::kMaxRuleWords)
er.words.push_back(x.as_string());
rs.rules.push_back(std::move(er));
}
out.push_back(std::move(rs));
}
return out;
}
} // namespace
void FilterConfig::validate() const {
if (rai.size() > ArmorCaps::kMaxRaiFilters)
throw GoogleError::invalid("too many rai filters");
for (const auto& f : rai)
if (!valid_rai_type(f.type))
throw GoogleError::invalid("rai filter type must be SEXUALLY_EXPLICIT, "
"HATE_SPEECH, HARASSMENT or DANGEROUS");
if (rule_sets.size() > ArmorCaps::kMaxRuleSets)
throw GoogleError::invalid("too many rule sets");
for (const auto& rs : rule_sets) {
if (rs.rules.size() > ArmorCaps::kMaxRulesPerSet)
throw GoogleError::invalid("too many rules in a rule set");
for (const auto& t : rs.filter_types)
if (!upper_name(t)) throw GoogleError::invalid("bad rule set filter type");
for (const auto& r : rs.rules) {
if (r.words.size() > ArmorCaps::kMaxRuleWords ||
r.regex.size() > ArmorCaps::kMaxRegexChars)
throw GoogleError::invalid("exclusion rule too large");
if (!r.matching_scope.empty() && !upper_name(r.matching_scope))
throw GoogleError::invalid("bad matching scope");
if (r.regex.empty() && r.words.empty())
throw GoogleError::invalid("exclusion rule needs words or a regex");
}
}
if (sdp == SdpMode::kAdvanced && sdp_inspect_template.empty())
throw GoogleError::invalid("advanced sdp needs an inspect template");
if (sdp_inspect_template.size() > 512 || sdp_deidentify_template.size() > 512)
throw GoogleError::invalid("sdp template name too long");
}
Json FilterConfig::to_json() const {
Json o = Json::object();
if (!rai.empty()) {
Json filters = Json::array();
for (const auto& f : rai) {
Json item = Json::object();
item.set("filterType", f.type);
if (f.confidence != Confidence::kUnspecified)
item.set("confidenceLevel", confidence_name(f.confidence));
filters.push(item);
}
Json settings = Json::object();
settings.set("raiFilters", filters);
o.set("raiSettings", settings);
}
if (!rule_sets.empty()) o.set("filterRuleSettings", rule_sets_json(rule_sets));
if (sdp == SdpMode::kBasic) {
Json settings = Json::object();
settings.set("basicConfig", enabled_json(true));
o.set("sdpSettings", settings);
} else if (sdp == SdpMode::kAdvanced) {
Json adv = Json::object();
adv.set("inspectTemplate", sdp_inspect_template);
if (!sdp_deidentify_template.empty())
adv.set("deidentifyTemplate", sdp_deidentify_template);
Json settings = Json::object();
settings.set("advancedConfig", adv);
o.set("sdpSettings", settings);
}
if (prompt_injection) {
Json pi = enabled_json(true);
if (prompt_injection_confidence != Confidence::kUnspecified)
pi.set("confidenceLevel", confidence_name(prompt_injection_confidence));
o.set("piAndJailbreakFilterSettings", pi);
}
if (malicious_uri) o.set("maliciousUriFilterSettings", enabled_json(true));
return o;
}
FilterConfig FilterConfig::from_json(const Json& doc) {
FilterConfig c;
if (const Json* r = member(doc, "raiSettings"))
if (const Json* list = member(*r, "raiFilters"); list && list->is_array())
for (const Json& f : list->as_array()) {
if (c.rai.size() >= ArmorCaps::kMaxRaiFilters) break;
c.rai.push_back({text_of(f, "filterType"),
confidence_from(text_of(f, "confidenceLevel"))});
}
if (const Json* r = member(doc, "filterRuleSettings")) c.rule_sets = rule_sets_from(*r);
if (const Json* s = member(doc, "sdpSettings")) {
if (const Json* adv = member(*s, "advancedConfig")) {
c.sdp = SdpMode::kAdvanced;
c.sdp_inspect_template = text_of(*adv, "inspectTemplate");
c.sdp_deidentify_template = text_of(*adv, "deidentifyTemplate");
} else if (const Json* basic = member(*s, "basicConfig")) {
if (enabled(basic)) c.sdp = SdpMode::kBasic;
}
}
if (const Json* pi = member(doc, "piAndJailbreakFilterSettings")) {
c.prompt_injection = enabled(pi);
c.prompt_injection_confidence = confidence_from(text_of(*pi, "confidenceLevel"));
}
c.malicious_uri = enabled(member(doc, "maliciousUriFilterSettings"));
return c;
}
void TemplateMetadata::validate() const {
if (!enforcement.empty() && enforcement != "INSPECT_ONLY" &&
enforcement != "INSPECT_AND_BLOCK")
throw GoogleError::invalid("enforcement must be INSPECT_ONLY or INSPECT_AND_BLOCK");
if (custom_prompt_error_message.size() > 1024 ||
custom_response_error_message.size() > 1024)
throw GoogleError::invalid("custom error message too long");
if (custom_prompt_error_code < 0 || custom_response_error_code < 0)
throw GoogleError::invalid("custom error code must not be negative");
if (!filter_version_alias.empty() && !upper_name(filter_version_alias))
throw GoogleError::invalid("bad filter version alias");
if (!filter_version.empty() && !filter_version_alias.empty())
throw GoogleError::invalid("filter version and alias are mutually exclusive");
if (filter_version.size() > 32) throw GoogleError::invalid("filter version too long");
if (modalities.size() > 2) throw GoogleError::invalid("too many modalities");
for (const auto& m : modalities)
if (m != "MODALITY_TEXT" && m != "MODALITY_IMAGE")
throw GoogleError::invalid("modality must be MODALITY_TEXT or MODALITY_IMAGE");
}
Json TemplateMetadata::to_json() const {
Json o = Json::object();
if (!enforcement.empty()) o.set("enforcementType", enforcement);
if (ignore_partial_failures) o.set("ignorePartialInvocationFailures", true);
if (log_template_operations) o.set("logTemplateOperations", true);
if (log_sanitize_operations) o.set("logSanitizeOperations", true);
if (multi_language_detection) {
Json m = Json::object();
m.set("enableMultiLanguageDetection", true);
o.set("multiLanguageDetection", m);
}
if (custom_prompt_error_code) o.set("customPromptSafetyErrorCode", custom_prompt_error_code);
if (!custom_prompt_error_message.empty())
o.set("customPromptSafetyErrorMessage", custom_prompt_error_message);
if (custom_response_error_code)
o.set("customLlmResponseSafetyErrorCode", custom_response_error_code);
if (!custom_response_error_message.empty())
o.set("customLlmResponseSafetyErrorMessage", custom_response_error_message);
if (!filter_version.empty() || !filter_version_alias.empty()) {
Json sel = Json::object();
if (!filter_version.empty()) sel.set("version", filter_version);
if (!filter_version_alias.empty()) sel.set("alias", filter_version_alias);
o.set("filterVersionSelector", sel);
}
if (!modalities.empty()) {
Json m = Json::array();
for (const auto& x : modalities) m.push(x);
o.set("modalities", m);
}
if (data_residency_set) o.set("dataResidencyCompliant", data_residency_compliant);
return o;
}
TemplateMetadata TemplateMetadata::from_json(const Json& doc) {
TemplateMetadata m;
m.enforcement = text_of(doc, "enforcementType");
if (m.enforcement == "ENFORCEMENT_TYPE_UNSPECIFIED") m.enforcement.clear();
m.ignore_partial_failures = flag_of(doc, "ignorePartialInvocationFailures");
m.log_template_operations = flag_of(doc, "logTemplateOperations");
m.log_sanitize_operations = flag_of(doc, "logSanitizeOperations");
if (const Json* ml = member(doc, "multiLanguageDetection"))
m.multi_language_detection = flag_of(*ml, "enableMultiLanguageDetection");
m.custom_prompt_error_code = int_of(doc, "customPromptSafetyErrorCode");
m.custom_prompt_error_message = text_of(doc, "customPromptSafetyErrorMessage");
m.custom_response_error_code = int_of(doc, "customLlmResponseSafetyErrorCode");
m.custom_response_error_message = text_of(doc, "customLlmResponseSafetyErrorMessage");
if (const Json* sel = member(doc, "filterVersionSelector")) {
m.filter_version = text_of(*sel, "version");
m.filter_version_alias = text_of(*sel, "alias");
}
if (const Json* mods = member(doc, "modalities"); mods && mods->is_array())
for (const Json& x : mods->as_array())
if (x.is_string() && m.modalities.size() < 2) m.modalities.push_back(x.as_string());
if (const Json* r = member(doc, "dataResidencyCompliant"); r && r->is_bool()) {
m.data_residency_set = true;
m.data_residency_compliant = r->as_bool();
}
return m;
}
void Template::validate() const {
filter.validate();
metadata.validate();
if (labels.size() > ArmorCaps::kMaxLabels)
throw GoogleError::invalid("too many labels");
for (const auto& [k, v] : labels)
if (k.empty() || k.size() > ArmorCaps::kMaxLabelChars ||
v.size() > ArmorCaps::kMaxLabelChars)
throw GoogleError::invalid("label key or value too long");
}
Json Template::to_json() const {
Json o = Json::object();
o.set("filterConfig", filter.to_json());
Json meta = metadata.to_json();
if (!meta.as_object().empty()) o.set("templateMetadata", meta);
if (!labels.empty()) {
Json l = Json::object();
for (const auto& [k, v] : labels) l.set(k, v);
o.set("labels", l);
}
return o;
}
Template Template::from_json(const Json& doc) {
Template t;
t.name = text_of(doc, "name");
t.create_time = text_of(doc, "createTime");
t.update_time = text_of(doc, "updateTime");
if (const Json* f = member(doc, "filterConfig")) t.filter = FilterConfig::from_json(*f);
if (const Json* m = member(doc, "templateMetadata"))
t.metadata = TemplateMetadata::from_json(*m);
if (const Json* l = member(doc, "labels"); l && l->is_object())
for (const auto& [k, v] : l->as_object())
if (v.is_string() && t.labels.size() < ArmorCaps::kMaxLabels)
t.labels[k] = v.as_string();
return t;
}
void FloorSetting::validate() const {
filter.validate();
for (const auto& s : integrated_services)
if (s != "AI_PLATFORM" && s != "GOOGLE_MCP_SERVER")
throw GoogleError::invalid(
"integrated service must be AI_PLATFORM or GOOGLE_MCP_SERVER");
if (integrated_services.size() > 2) throw GoogleError::invalid("too many integrated services");
}
Json FloorSetting::to_json() const {
Json o = Json::object();
o.set("filterConfig", filter.to_json());
o.set("enableFloorSettingEnforcement", enforcement_enabled);
if (!integrated_services.empty()) {
Json s = Json::array();
for (const auto& x : integrated_services) s.push(x);
o.set("integratedServices", s);
}
if (ai_platform.present) o.set("aiPlatformFloorSetting", enforcement_json(ai_platform));
if (mcp_server.present) o.set("googleMcpServerFloorSetting", enforcement_json(mcp_server));
if (multi_language_detection) {
Json ml = Json::object();
ml.set("enableMultiLanguageDetection", true);
Json meta = Json::object();
meta.set("multiLanguageDetection", ml);
o.set("floorSettingMetadata", meta);
}
return o;
}
FloorSetting FloorSetting::from_json(const Json& doc) {
FloorSetting f;
f.name = text_of(doc, "name");
f.update_time = text_of(doc, "updateTime");
f.ai_platform = FloorEnforcement{};
if (const Json* c = member(doc, "filterConfig")) f.filter = FilterConfig::from_json(*c);
f.enforcement_enabled = flag_of(doc, "enableFloorSettingEnforcement");
if (const Json* s = member(doc, "integratedServices"); s && s->is_array())
for (const Json& x : s->as_array())
if (x.is_string() && f.integrated_services.size() < 2)
f.integrated_services.push_back(x.as_string());
if (const Json* p = member(doc, "aiPlatformFloorSetting")) f.ai_platform = enforcement_from(*p);
if (const Json* p = member(doc, "googleMcpServerFloorSetting"))
f.mcp_server = enforcement_from(*p);
if (const Json* m = member(doc, "floorSettingMetadata"))
if (const Json* ml = member(*m, "multiLanguageDetection"))
f.multi_language_detection = flag_of(*ml, "enableMultiLanguageDetection");
return f;
}
} // namespace molo::google::armor
+118
View File
@@ -0,0 +1,118 @@
#pragma once
// Model Armor template and floor setting, typed. Wire names are the ones in
// the Model Armor v1 Discovery document (see calls.h and SPEC.md).
#include <map>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "google/armor/armor_types.h"
namespace molo::google::armor {
struct RaiFilter {
std::string type; // SEXUALLY_EXPLICIT | HATE_SPEECH | HARASSMENT | DANGEROUS
Confidence confidence = Confidence::kUnspecified;
};
// Sensitive Data Protection: off, the fixed basic info types, or your own
// inspect (and optional de-identify) templates.
enum class SdpMode { kOff, kBasic, kAdvanced };
// filterRuleSettings.ruleSets[].rules[].exclusionRule
struct ExclusionRule {
std::string matching_scope; // MATCHING_SCOPE_FULL_MATCH | _PARTIAL_MATCH
std::vector<std::string> words; // dictionary.wordList.words
std::string regex; // regex.pattern (instead of words)
};
struct FilterRuleSet {
std::vector<std::string> filter_types; // Enum names as Google spells them.
std::vector<ExclusionRule> rules;
};
struct FilterConfig {
std::vector<RaiFilter> rai;
std::vector<FilterRuleSet> rule_sets;
SdpMode sdp = SdpMode::kOff;
std::string sdp_inspect_template; // advanced only
std::string sdp_deidentify_template; // advanced only, optional
bool prompt_injection = false; // piAndJailbreakFilterSettings
Confidence prompt_injection_confidence = Confidence::kUnspecified;
bool malicious_uri = false;
// Throws GoogleError::invalid on a bad rai type, oversized rules or an
// advanced sdp config without an inspect template.
void validate() const;
clients::body::Json to_json() const;
static FilterConfig from_json(const clients::body::Json& doc);
};
struct TemplateMetadata {
std::string enforcement; // "", INSPECT_ONLY, INSPECT_AND_BLOCK
bool ignore_partial_failures = false;
bool log_template_operations = false;
bool log_sanitize_operations = false;
bool multi_language_detection = false;
int custom_prompt_error_code = 0; // 0: unset
std::string custom_prompt_error_message;
int custom_response_error_code = 0;
std::string custom_response_error_message;
std::string filter_version; // filterVersionSelector.version
std::string filter_version_alias; // filterVersionSelector.alias (enum name)
std::vector<std::string> modalities; // MODALITY_TEXT, MODALITY_IMAGE
bool data_residency_set = false;
bool data_residency_compliant = false;
void validate() const;
clients::body::Json to_json() const;
static TemplateMetadata from_json(const clients::body::Json& doc);
};
struct Template {
std::string name; // projects/{p}/locations/{l}/templates/{id}
std::map<std::string, std::string> labels;
FilterConfig filter;
TemplateMetadata metadata;
std::string create_time;
std::string update_time;
void validate() const;
clients::body::Json to_json() const; // Request body (no output-only fields).
static Template from_json(const clients::body::Json& doc);
};
struct TemplatePage {
std::vector<Template> templates;
std::vector<std::string> unreachable; // Locations Google could not reach.
};
// aiPlatformFloorSetting / googleMcpServerFloorSetting.
struct FloorEnforcement {
bool present = false; // Serialized only when set.
bool cloud_logging = false;
bool inspect_only = false; // else inspectAndBlock
};
struct FloorSetting {
std::string name; // {projects|folders|organizations}/{id}/locations/{l}/floorSetting
FilterConfig filter;
bool enforcement_enabled = false;
std::vector<std::string> integrated_services; // AI_PLATFORM, GOOGLE_MCP_SERVER
FloorEnforcement ai_platform{true, false, false};
FloorEnforcement mcp_server;
bool multi_language_detection = false;
std::string update_time;
void validate() const;
clients::body::Json to_json() const;
static FloorSetting from_json(const clients::body::Json& doc);
};
// One entry of locations.list.
struct ArmorLocation {
std::string name;
std::string location_id;
std::string display_name;
};
} // namespace molo::google::armor
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""Checks src/google/armor/calls.h against Google's Discovery documents.
Fails (exit 1) when a declared call (rpc id, HTTP method, path, query
parameter), a declared schema field, or a declared enum value is absent from
the live Discovery document. Stdlib only, no credentials needed.
python3 -I src/google/armor/tools/check_discovery.py # fetch live
python3 -I src/google/armor/tools/check_discovery.py --cache DIR # read DIR/<service>-<version>.json
"""
import json
import os
import re
import sys
import urllib.request
HEADER = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "calls.h")
ROW = re.compile(r'^\s*\{((?:"[^"]*"(?:,\s*)?)+)\},\s*$')
STR = re.compile(r'"([^"]*)"')
def rows():
calls, schemas, enums = [], [], []
for line in open(HEADER, encoding="utf-8"):
m = ROW.match(line)
if not m:
continue
cells = STR.findall(m.group(1))
{6: calls, 4: schemas, 5: enums}.get(len(cells), []).append(cells)
return calls, schemas, enums
def load(service, version, cache):
if cache:
path = os.path.join(cache, "%s-%s.json" % (service, version))
with open(path, encoding="utf-8") as f:
return json.load(f)
url = "https://%s.googleapis.com/$discovery/rest?version=%s" % (service, version)
with urllib.request.urlopen(url, timeout=60) as r:
return json.load(r)
def methods(doc):
out = {}
def walk(res):
for sub in res.get("resources", {}).values():
walk(sub)
for m in res.get("methods", {}).values():
out[m["id"]] = m
walk(doc)
return out
def main():
cache = None
if "--cache" in sys.argv:
cache = sys.argv[sys.argv.index("--cache") + 1]
calls, schemas, enums = rows()
if not calls or not schemas or not enums:
print("FAIL: no rows parsed from calls.h")
return 1
docs, errors = {}, []
def doc(service, version):
key = (service, version)
if key not in docs:
docs[key] = load(service, version, cache)
return docs[key]
index = {}
for service, version, rpc, http, path, query in calls:
d = doc(service, version)
idx = index.setdefault((service, version), methods(d))
m = idx.get(rpc)
if not m:
errors.append("call missing: " + rpc)
continue
if m["httpMethod"] != http:
errors.append("%s: http %s != declared %s" % (rpc, m["httpMethod"], http))
if m["path"] != path:
errors.append("%s: path %s != declared %s" % (rpc, m["path"], path))
for q in filter(None, query.split(",")):
if q not in m.get("parameters", {}):
errors.append("%s: query parameter %s missing" % (rpc, q))
for service, version, schema, fields in schemas:
s = doc(service, version)["schemas"].get(schema)
if not s:
errors.append("schema missing: " + schema)
continue
for f in fields.split(","):
if f not in s.get("properties", {}):
errors.append("%s: field %s missing" % (schema, f))
for service, version, schema, field, values in enums:
s = doc(service, version)["schemas"].get(schema, {})
p = s.get("properties", {}).get(field)
if not p or "enum" not in p:
errors.append("%s.%s: no enum" % (schema, field))
continue
for v in values.split(","):
if v not in p["enum"]:
errors.append("%s.%s: value %s missing" % (schema, field, v))
for e in errors:
print("FAIL:", e)
print("%s: %d calls, %d schemas, %d enums checked, %d problems"
% ("FAIL" if errors else "OK", len(calls), len(schemas), len(enums), len(errors)))
return 1 if errors else 0
if __name__ == "__main__":
sys.exit(main())