feat(clients): data: BigQuery, Storage media and signed URLs, Pub/Sub, Firestore, Spanner, Bigtable admin, Secret Manager, Dataplex, DLP, KMS, Datastore

Discovery generated call table (918 rows), typed services, /google/data demo.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-25 15:03:43 +01:00
co-authored by Claude Sonnet 5.5
parent 3db67fcf1f
commit 1af14b5d74
47 changed files with 6315 additions and 3 deletions
+186
View File
@@ -0,0 +1,186 @@
# Google data package - Specification
Owned directory: `src/google/data/`. Namespace `molo::google::data`. Depends on
the `src/google` foundation (`GoogleClient`, `Service`, `CloudContext`,
`list_all`), `molo::clients` and wolfCrypt. Plan: `plans/18-breadth.md`.
## 1. Scope
Eleven Google data APIs, every method of each one reachable through one
Discovery-generated call table (918 rows), the common ones through typed
classes:
| API (Discovery) | Class | Typed calls |
|---|---|---|
| BigQuery v2 | `BigQuery` | query, dry run, results, jobs (list, get, insert, cancel), datasets, tables, rows, streaming insert (`tabledata.insertAll`), models, routines |
| Cloud Storage JSON v1 | `Storage`, `StorageMedia`, `SignedUrl` | buckets, objects, compose, copy, patch, delete; download and range download; simple and resumable upload; V4 signed URLs |
| Pub/Sub v1 | `PubSub` | topics, subscriptions, snapshots, publish, pull, acknowledge |
| Firestore v1 | `Firestore` | databases, get, list, collection ids, run query, aggregation, batch get, create, commit, delete |
| Spanner v1 | `Spanner` | instances, databases, DDL, sessions, read-only SQL |
| Bigtable Admin v2 | `BigtableAdmin` | instances, clusters, tables, app profiles, table IAM (admin plane only; row access is gRPC) |
| Secret Manager v1 | `SecretManager` | secrets, versions, access, create, add version, disable, destroy, delete |
| Dataplex v1 | `Dataplex` | locations, lakes, entry groups, entry and aspect types, data scans, entry search and lookup |
| DLP v2 | `Dlp` | inspect, de-identify, infoTypes, templates, triggers, jobs |
| Cloud KMS v1 | `Kms` | locations, key rings, keys, versions, public key, encrypt, decrypt, asymmetric sign |
| Datastore v1 | `Datastore` | lookup, query, aggregation, transactions, commit, allocate and reserve ids |
Anything else (for example `bigquery.rowAccessPolicies.*`, `dataplex.*.glossaries.*`)
is `ApiService::invoke("<rpc id>", args)` with the same validation. `DataPlatform`
holds all typed services over one client and project.
Not here: Bigtable data plane (gRPC), Spanner partitioned/streaming reads, BigQuery
Storage Read/Write API (gRPC), Pub/Sub streaming pull, Firestore listen. They have no
REST surface in Discovery.
## 2. Module map
| File | Concern |
|---|---|
| `caps.h` | Static ceilings (`DataCaps`) |
| `calls.h/.cpp`, `calls_table.inc` | `Catalog`: the ONE table of REST calls (generated), api info, endpoints |
| `ids.h` | `Ids`: project, location, dataset, bucket, object, pubsub and generic id rules |
| `api_service.h/.cpp` | `ApiService : Service`, `CallArgs`, `Prepared`: validate, render, send, page |
| `project_service.h` | `ProjectService`: context bound base with name and paging helpers |
| `bigquery`, `storage`, `pubsub`, `firestore`, `spanner`, `bigtable_admin`, `secret_manager`, `dataplex`, `dlp`, `kms`, `datastore` | one typed class per API |
| `storage_media.h/.cpp` | `StorageMedia`: raw bytes (GoogleClient is JSON only) over its own `clients::Client` |
| `signed_url.h/.cpp` | `SignedUrl`: GOOG4-RSA-SHA256 signing with the service-account key, no network |
| `data.h` | `DataPlatform` facade |
| `tools/gen_calls.py` | regenerates `calls_table.inc` from Discovery |
| `tools/check_discovery.py` | conformance check against Discovery |
## 3. Static caps (`caps.h`)
| Knob | Value |
|---|---|
| path value / resource name | 1024 chars |
| query parameters per call / value chars / array items | 40 / 2048 / 100 |
| JSON request body | 10 MiB |
| list page size / page walk | 1000 / 20 pages |
| BigQuery sql / result rows / query wait | 1 MiB / 10000 / 60000 ms |
| BigQuery insertAll rows / bytes | 10000 / 10 MiB |
| Storage download / simple upload | 16 MiB / 8 MiB |
| Storage resumable chunk / total / granularity | 8 MiB / 64 MiB / 256 KiB |
| Storage compose sources | 32 |
| signed URL lifetime | 1 s to 604800 s (7 days, the V4 maximum) |
| signed headers / extra query | 20 / 20 |
| Pub/Sub publish messages / bytes / pull / ack ids | 1000 / 10 MiB / 100 / 1000 |
| Secret payload / KMS plaintext | 64 KiB / 64 KiB |
| DLP content item | 512 KiB |
| Datastore keys per lookup / writes per commit | 1000 / 500 |
| example route body / response shown / upload / download shown | 3 MiB / 256 KiB / 2 MiB / 64 KiB |
## 4. Contracts
- Every URL is `endpoint base + the table row's path`; no path literal elsewhere.
`{x}` placeholders are one URL-encoded segment (object names with `/` and spaces
are encoded); `{+x}` placeholders are resource names (`[A-Za-z0-9._~:@=,$()*/-]`,
no empty, `.` or `..` segment).
- `prepare()` rejects before any traffic (GoogleError 400 INVALID_ARGUMENT): a call
of another API, unknown or missing path value, unknown query parameter, missing
required query parameter, body on a call without request message, body field not
in the request schema, control characters, caps. Then `invoke()` needs a
credential (`NotConfiguredError`, 503).
- Base URL overrides (regional hosts) must be `https://*.googleapis.com`.
- Destructive and write calls are explicit methods without defaults. The example
routes run them only with `"confirm":true`; reading a secret payload, publishing,
pulling (leases messages), billed BigQuery queries, DLP and KMS operations count as
gated too. A BigQuery `query` is billed at most 1 GB unless `maximum_bytes_billed`
says otherwise.
- `StorageMedia` sends the credential only to `https://storage.googleapis.com`; a
resumable session URL from anywhere else is rejected. Over-cap downloads fail
loudly, never truncate. A chunk that does not end the object must be a multiple of
256 KiB.
- `Spanner::query` always deletes the session it created.
- Signed URL signing needs an RSA service-account key; the lifetime is capped at 7
days. A bad key is rejected loudly. Anyone holding a URL can use it until it expires.
## 5. Example routes (`/google/data`)
GET `/google/data` (page), `/status` (credential state, APIs with call counts, actions),
`/calls[?api=name]` (the table). POST `/bigquery`, `/storage`, `/pubsub`, `/firestore`,
`/spanner`, `/bigtable`, `/secrets`, `/dataplex`, `/dlp`, `/kms`, `/datastore`
(`{"action":...}`; `/status` lists the actions and which need confirm), `/storage/sign`
(signed URL, dry run unless confirm), `/call` (any row by rpc id; non-GET rows are dry
runs unless confirm). Bad input 422, no credential 503 `google_not_configured`, body
over 3 MiB 413. Excluded from locust.
## Required APIs
Enable these on the project (placeholder project `example-214512124`; full list and a one-line `gcloud` command in `src/google/APIS.md`). A disabled API answers 403 `SERVICE_DISABLED`.
- `bigquery.googleapis.com`: https://console.developers.google.com/apis/api/bigquery.googleapis.com/overview?project=example-214512124
- `storage.googleapis.com`: https://console.developers.google.com/apis/api/storage.googleapis.com/overview?project=example-214512124
- `pubsub.googleapis.com`: https://console.developers.google.com/apis/api/pubsub.googleapis.com/overview?project=example-214512124
- `firestore.googleapis.com`: https://console.developers.google.com/apis/api/firestore.googleapis.com/overview?project=example-214512124
- `spanner.googleapis.com`: https://console.developers.google.com/apis/api/spanner.googleapis.com/overview?project=example-214512124
- `bigtableadmin.googleapis.com`: https://console.developers.google.com/apis/api/bigtableadmin.googleapis.com/overview?project=example-214512124
- `secretmanager.googleapis.com`: https://console.developers.google.com/apis/api/secretmanager.googleapis.com/overview?project=example-214512124
- `dataplex.googleapis.com`: https://console.developers.google.com/apis/api/dataplex.googleapis.com/overview?project=example-214512124
- `dlp.googleapis.com`: https://console.developers.google.com/apis/api/dlp.googleapis.com/overview?project=example-214512124
- `cloudkms.googleapis.com`: https://console.developers.google.com/apis/api/cloudkms.googleapis.com/overview?project=example-214512124
- `datastore.googleapis.com`: https://console.developers.google.com/apis/api/datastore.googleapis.com/overview?project=example-214512124
Scope: all calls use `cloud-platform` (the service-account default). Signed URLs need
only the key file.
## Sources
- Discovery documents, fetched 2026-10-25: `https://<api>.googleapis.com/$discovery/rest?version=<v>`
for bigquery v2 (rev 20260922), storage v1 (20261020), pubsub v1 (20261013), firestore v1
(20261013), spanner v1 (20261006), bigtableadmin v2 (20261017), secretmanager v1
(20260925), dataplex v1 (20261013), dlp v2 (20261020), cloudkms v1 (20261016), datastore v1
(20261013).
- Cloud Storage "Signatures V4" and "Resumable uploads" documentation (canonical request,
string to sign, session protocol).
## Verified against discovery
`python3 -I src/google/data/tools/check_discovery.py`, run 2026-10-25:
`discovery ok: 918 rows, 120 call ids used by sources, 36 nested schemas, storage media contract`.
Checks per row: HTTP method, path, request body fields (exactly the request schema's
top level), query parameters, required query parameters, path placeholders; total
coverage (every Discovery method of the 11 APIs is a row and vice versa); every call id
named in the C++ sources is a row; nested schema fields the typed classes write; the
Storage media upload and download contract.
## Verified live
Project `lisa-448004`, service account, 2026-10-25, read only (`make google-live`):
- BigQuery: dry-run `SELECT 1` (200, totalBytesProcessed), datasets.list, jobs.list,
projects.getServiceAccount: 200.
- Storage: buckets.list 200 (the project has no bucket), projects.serviceAccount.get 200.
Download and signed URL against the Google-owned public object
`gcp-public-data-landsat/index.csv.gz`: ranged download of 100 bytes (bearer token) 200;
V4 signed HEAD URL 200; the same URL with one changed signature character 403.
- Secret Manager secrets.list 200. Dataplex locations.list and searchEntries 200.
- Datastore runQuery: 404 NOT_FOUND (no Datastore database in the project; route and auth
accepted).
- API NOT ENABLED on the project (403, "has not been used ... or it is disabled"): Pub/Sub,
Firestore, Spanner, Bigtable Admin, DLP, Cloud KMS. Their typed calls are verified by
Discovery only.
## Unverified
- Every write (create, insert, publish, upload, compose, delete, commit, addVersion):
Discovery only; the live rules forbid writes.
- Simple and resumable upload request shapes and the resumable protocol (308 handling,
`Content-Range`). The resumable session is started on the documented
`/upload/storage/v1/b/{bucket}/o?uploadType=resumable`; Discovery lists
`/resumable/upload/storage/v1/b/{bucket}/o` for the same protocol (both hosts accepted
as session URLs). Not exercised against Google.
- Response shapes: typed methods return Google's JSON unchanged.
- Spanner `query` (session create, `executeSql`, session delete), DLP inspect and
de-identify, KMS encrypt, decrypt and sign, Pub/Sub publish and pull, Firestore queries:
APIs not enabled or no resources; request bodies checked against Discovery schemas only.
- Regional hosts (DLP, Spanner regional endpoints): override accepted, never used live.
## Gaps
- `GoogleClient` is JSON only, so `StorageMedia` owns a second `clients::Client`; a raw
`request_bytes` in the foundation would remove it.
- The google build links `Base64.cpp` only, so `SignedUrl` hashes with wolfCrypt instead
of `security/crypto/Sha256`.
- `api_catalog.*` and `APIS.md` do not list the 11 services yet (owner adds them, see the
Required APIs list).
- Streaming (Pub/Sub streamingPull, Firestore listen, BigQuery Storage API) is gRPC only.
+197
View File
@@ -0,0 +1,197 @@
#include "google/data/api_service.h"
#include <cstring>
#include "clients/url.h"
#include "google/data/caps.h"
#include "google/errors.h"
#include "google/operation.h"
namespace molo::google::data {
using clients::body::Json;
CallArgs& CallArgs::p(const std::string& name, const std::string& value) {
path[name] = value;
return *this;
}
CallArgs& CallArgs::q(const std::string& name, Json value) {
query.set(name, std::move(value));
return *this;
}
CallArgs& CallArgs::with_body(Json b) {
body = std::move(b);
return *this;
}
namespace {
bool control(char c) { return static_cast<unsigned char>(c) < 0x20 || c == 0x7f; }
clients::Method method_of(const char* m) {
if (!std::strcmp(m, "GET")) return clients::Method::kGet;
if (!std::strcmp(m, "POST")) return clients::Method::kPost;
if (!std::strcmp(m, "PUT")) return clients::Method::kPut;
if (!std::strcmp(m, "PATCH")) return clients::Method::kPatch;
if (!std::strcmp(m, "DELETE")) return clients::Method::kDelete;
throw GoogleError::invalid("unsupported http method in table");
}
} // namespace
void ApiService::check_segment(const std::string& name, const std::string& v) {
if (v.empty() || v.size() > DataCaps::kMaxPathValueChars)
throw GoogleError::invalid(name + " must be 1.." +
std::to_string(DataCaps::kMaxPathValueChars) + " characters");
if (v == "." || v == "..") throw GoogleError::invalid(name + " must not be . or ..");
for (char c : v)
if (control(c)) throw GoogleError::invalid(name + " has control characters");
}
void ApiService::check_resource(const std::string& name, const std::string& v) {
if (v.empty() || v.size() > DataCaps::kMaxResourceNameChars)
throw GoogleError::invalid(name + " must be 1.." +
std::to_string(DataCaps::kMaxResourceNameChars) + " characters");
if (v.front() == '/' || v.back() == '/' || v.find("//") != std::string::npos ||
v.find("/../") != std::string::npos || v.rfind("../", 0) == 0 ||
(v.size() >= 3 && v.compare(v.size() - 3, 3, "/..") == 0) || v == "..")
throw GoogleError::invalid(name + " is not a well formed resource name");
for (char c : v) {
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
std::strchr("._~:@=,$()*/-", c) != nullptr;
if (!ok) throw GoogleError::invalid(name + " has characters outside A-Za-z0-9._~:@=,$()*/-");
}
}
std::string ApiService::encode_query_value(const std::string& name, const Json& v) {
auto one = [&](const Json& x) -> std::string {
if (x.is_string()) {
if (x.as_string().size() > DataCaps::kMaxQueryValueChars)
throw GoogleError::invalid("query " + name + " too long");
for (char c : x.as_string())
if (control(c)) throw GoogleError::invalid("query " + name + " has control characters");
return clients::Url::encode_component(x.as_string());
}
if (x.is_bool()) return x.as_bool() ? "true" : "false";
if (x.is_number()) {
double d = x.as_number();
if (d == static_cast<double>(static_cast<long long>(d)))
return std::to_string(static_cast<long long>(d));
return std::to_string(d);
}
throw GoogleError::invalid("query " + name + " must be a string, number or boolean");
};
return one(v);
}
Prepared ApiService::prepare(const std::string& rpc, const CallArgs& args) const {
const CallSpec& spec = Catalog::get(rpc);
if (Catalog::api_of(spec) != api_)
throw GoogleError::invalid("call " + rpc + " belongs to another api");
Prepared out;
out.method = method_of(spec.http);
out.endpoint = endpoint_;
// Path.
std::string path = spec.path;
auto names = Catalog::path_params(spec);
for (const auto& kv : args.path) {
bool known = false;
for (const auto& n : names) known = known || n == kv.first;
if (!known) throw GoogleError::invalid("unknown path value: " + kv.first.substr(0, 64));
}
std::string rendered;
for (std::size_t i = 0; path[i];) {
if (path[i] != '{') {
rendered.push_back(path[i++]);
continue;
}
std::size_t close = path.find('}', i);
std::string key = path.substr(i + 1, close - i - 1);
bool plus = !key.empty() && key[0] == '+';
if (plus) key.erase(0, 1);
auto it = args.path.find(key);
if (it == args.path.end()) throw GoogleError::invalid("missing path value: " + key);
if (plus) {
check_resource(key, it->second);
rendered += it->second;
} else {
check_segment(key, it->second);
rendered += clients::Url::encode_component(it->second);
}
i = close + 1;
}
out.path = "/" + rendered;
// Query.
std::string qs;
std::size_t count = 0;
if (!args.query.is_object()) throw GoogleError::invalid("query must be an object");
for (const auto& kv : args.query.as_object()) {
if (kv.second.is_null()) continue;
if (!Catalog::in_list(spec.query, kv.first))
throw GoogleError::invalid("unknown query parameter for " + rpc + ": " +
kv.first.substr(0, 64));
std::vector<const Json*> values;
if (kv.second.is_array()) {
if (kv.second.as_array().size() > DataCaps::kMaxQueryArrayItems)
throw GoogleError::invalid("query " + kv.first + " has too many items");
for (const Json& x : kv.second.as_array()) values.push_back(&x);
} else {
values.push_back(&kv.second);
}
for (const Json* x : values) {
if (++count > DataCaps::kMaxQueryParams)
throw GoogleError::invalid("too many query parameters");
qs += qs.empty() ? "?" : "&";
qs += kv.first + "=" + encode_query_value(kv.first, *x);
}
}
for (const auto& r : Catalog::split(spec.required)) {
const Json* v = args.query.find(r);
if (!v || v->is_null()) throw GoogleError::invalid("missing required query parameter " + r);
}
out.path += qs;
// Body.
bool body_allowed = spec.body[0] != '\0';
if (!args.body.is_null()) {
if (!body_allowed) throw GoogleError::invalid(rpc + " takes no request body");
if (!args.body.is_object()) throw GoogleError::invalid("body must be a JSON object");
for (const auto& kv : args.body.as_object())
if (!Catalog::in_list(spec.body, kv.first))
throw GoogleError::invalid("unknown body field for " + rpc + ": " +
kv.first.substr(0, 64));
if (args.body.dump().size() > DataCaps::kMaxBodyBytes)
throw GoogleError::invalid("request body too large");
out.body = args.body;
out.has_body = true;
} else if (body_allowed && (out.method == clients::Method::kPost ||
out.method == clients::Method::kPut ||
out.method == clients::Method::kPatch)) {
out.body = Json::object(); // POST with an empty request message
out.has_body = true;
}
return out;
}
ApiService::ApiService(std::shared_ptr<GoogleClient> client, Api api,
const std::string& base_url_override)
: Service(std::move(client), Catalog::endpoint(api, base_url_override)), api_(api) {}
Json ApiService::invoke(const std::string& rpc, const CallArgs& args) const {
Prepared p = prepare(rpc, args);
return client_->request(p.method, p.endpoint, p.path, p.has_body ? &p.body : nullptr);
}
Json ApiService::invoke_paged(const std::string& rpc, const CallArgs& args,
const std::string& items_key, int max_pages) const {
if (max_pages < 1 || max_pages > DataCaps::kMaxListPages)
throw GoogleError::invalid("max_pages must be 1.." + std::to_string(DataCaps::kMaxListPages));
Prepared p = prepare(rpc, args);
if (p.method != clients::Method::kGet)
throw GoogleError::invalid("paged calls must be GET");
return list_all(*client_, p.endpoint, p.path, items_key, static_cast<std::size_t>(max_pages));
}
} // namespace molo::google::data
+67
View File
@@ -0,0 +1,67 @@
#pragma once
// ApiService: the generic, table-driven call engine every typed service of the
// data package derives from. A call is an rpc id from the Catalog plus
// CallArgs; path values, query parameters and body fields are validated
// against the table row before any network traffic.
#include <map>
#include <memory>
#include <string>
#include "clients/body/json.h"
#include "clients/request.h"
#include "google/data/calls.h"
#include "google/service.h"
namespace molo::google::data {
// Arguments of one call.
struct CallArgs {
std::map<std::string, std::string> path; // {x} and {+x} values
clients::body::Json query = clients::body::Json::object(); // string|number|bool|array
clients::body::Json body; // null or object
CallArgs& p(const std::string& name, const std::string& value);
CallArgs& q(const std::string& name, clients::body::Json value);
CallArgs& with_body(clients::body::Json b);
};
// A validated, rendered request (no network yet).
struct Prepared {
clients::Method method = clients::Method::kGet;
ServiceEndpoint endpoint;
std::string path; // "/v1/projects/..." plus "?query"
clients::body::Json body;
bool has_body = false;
std::string url() const { return endpoint.base_url + path; }
};
class ApiService : public Service {
public:
ApiService(std::shared_ptr<GoogleClient> client, Api api,
const std::string& base_url_override = "");
Api api() const { return api_; }
// Validation + rendering only. Throws GoogleError 400 INVALID_ARGUMENT.
Prepared prepare(const std::string& rpc, const CallArgs& args = {}) const;
// prepare + send. NotConfiguredError when no credential.
clients::body::Json invoke(const std::string& rpc, const CallArgs& args = {}) const;
// Follows nextPageToken (pageSize is the caller's query arg) and collects the
// `items_key` arrays. Page walk capped.
clients::body::Json invoke_paged(const std::string& rpc, const CallArgs& args,
const std::string& items_key,
int max_pages = 20) const;
// Value checks shared by the engine and the typed services.
static void check_segment(const std::string& name, const std::string& value);
static void check_resource(const std::string& name, const std::string& value);
static std::string encode_query_value(const std::string& name,
const clients::body::Json& value);
private:
Api api_;
};
} // namespace molo::google::data
+257
View File
@@ -0,0 +1,257 @@
#include "google/data/bigquery.h"
#include "google/data/caps.h"
#include "google/data/ids.h"
namespace molo::google::data {
using clients::body::Json;
void QueryRequest::validate() const {
if (sql.empty() || sql.size() > DataCaps::kMaxSqlChars)
throw GoogleError::invalid("sql must be 1.." + std::to_string(DataCaps::kMaxSqlChars) +
" characters");
if (max_results < 1 || max_results > DataCaps::kMaxResultRows)
throw GoogleError::invalid("max_results must be 1.." +
std::to_string(DataCaps::kMaxResultRows));
if (timeout_ms < 0 || timeout_ms > DataCaps::kMaxQueryTimeoutMs)
throw GoogleError::invalid("timeout_ms must be 0.." +
std::to_string(DataCaps::kMaxQueryTimeoutMs));
if (!location.empty()) Ids::location(location);
if (!default_dataset.empty()) Ids::dataset(default_dataset);
if (!maximum_bytes_billed.empty()) {
if (maximum_bytes_billed.size() > 18)
throw GoogleError::invalid("maximum_bytes_billed must be a decimal string");
for (char c : maximum_bytes_billed)
if (c < '0' || c > '9') throw GoogleError::invalid("maximum_bytes_billed must be digits");
}
if (!query_parameters.is_null()) {
if (!query_parameters.is_array()) throw GoogleError::invalid("query_parameters must be an array");
if (parameter_mode != "NAMED" && parameter_mode != "POSITIONAL")
throw GoogleError::invalid("parameter_mode must be NAMED or POSITIONAL");
}
}
Json QueryRequest::to_json(const std::string& project) const {
validate();
Json b = Json::object();
b.set("query", Json(sql));
b.set("useLegacySql", Json(use_legacy_sql));
b.set("useQueryCache", Json(use_query_cache));
b.set("dryRun", Json(dry_run));
b.set("maxResults", Json(max_results));
b.set("timeoutMs", Json(timeout_ms));
if (!location.empty()) b.set("location", Json(location));
if (!default_dataset.empty()) {
Json d = Json::object();
d.set("projectId", Json(project));
d.set("datasetId", Json(default_dataset));
b.set("defaultDataset", d);
}
if (!maximum_bytes_billed.empty()) b.set("maximumBytesBilled", Json(maximum_bytes_billed));
if (!query_parameters.is_null()) {
b.set("queryParameters", query_parameters);
b.set("parameterMode", Json(parameter_mode));
}
return b;
}
BigQuery::BigQuery(std::shared_ptr<GoogleClient> client, CloudContext context)
: ApiService(std::move(client), Api::kBigquery), context_(std::move(context)) {}
CallArgs BigQuery::base() const {
CallArgs a;
a.p("projectId", Ids::project(context_.project));
return a;
}
CallArgs BigQuery::in_dataset(const std::string& dataset) const {
CallArgs a = base();
a.p("datasetId", Ids::dataset(dataset));
return a;
}
Json BigQuery::page(int max_results) {
if (max_results < 1 || max_results > DataCaps::kMaxPageSize)
throw GoogleError::invalid("max_results must be 1.." + std::to_string(DataCaps::kMaxPageSize));
return Json(max_results);
}
Json BigQuery::query(const QueryRequest& request) const {
CallArgs a = base();
a.with_body(request.to_json(context_.project));
return invoke("bigquery.jobs.query", a);
}
Json BigQuery::dry_run(const std::string& sql) const {
QueryRequest r;
r.sql = sql;
r.dry_run = true;
return query(r);
}
Json BigQuery::query_results(const std::string& job_id, const std::string& location,
int max_results, const std::string& page_token) const {
CallArgs a = base();
a.p("jobId", Ids::bq_id(job_id, "job id"));
a.q("maxResults", page(max_results));
if (!location.empty()) a.q("location", Json(Ids::location(location)));
if (!page_token.empty()) a.q("pageToken", Json(page_token));
return invoke("bigquery.jobs.getQueryResults", a);
}
Json BigQuery::list_jobs(int max_results, const std::string& state_filter) const {
CallArgs a = base();
a.q("maxResults", page(max_results));
if (!state_filter.empty()) {
if (state_filter != "done" && state_filter != "pending" && state_filter != "running")
throw GoogleError::invalid("state_filter must be done, pending or running");
a.q("stateFilter", Json(state_filter));
}
return invoke("bigquery.jobs.list", a);
}
Json BigQuery::get_job(const std::string& job_id, const std::string& location) const {
CallArgs a = base();
a.p("jobId", Ids::bq_id(job_id, "job id"));
if (!location.empty()) a.q("location", Json(Ids::location(location)));
return invoke("bigquery.jobs.get", a);
}
Json BigQuery::insert_job(const Json& job) const {
CallArgs a = base();
a.with_body(job);
return invoke("bigquery.jobs.insert", a);
}
Json BigQuery::cancel_job(const std::string& job_id, const std::string& location) const {
CallArgs a = base();
a.p("jobId", Ids::bq_id(job_id, "job id"));
if (!location.empty()) a.q("location", Json(Ids::location(location)));
return invoke("bigquery.jobs.cancel", a);
}
Json BigQuery::list_datasets(int max_results) const {
CallArgs a = base();
a.q("maxResults", page(max_results));
return invoke("bigquery.datasets.list", a);
}
Json BigQuery::get_dataset(const std::string& dataset) const {
return invoke("bigquery.datasets.get", in_dataset(dataset));
}
Json BigQuery::create_dataset(const std::string& dataset, const std::string& location,
const std::string& description) const {
CallArgs a = base();
Json ref = Json::object();
ref.set("projectId", Json(Ids::project(context_.project)));
ref.set("datasetId", Json(Ids::dataset(dataset)));
Json b = Json::object();
b.set("datasetReference", ref);
if (!location.empty()) b.set("location", Json(Ids::location(location)));
if (description.size() > 1024) throw GoogleError::invalid("description over 1024 characters");
if (!description.empty()) b.set("description", Json(description));
a.with_body(b);
return invoke("bigquery.datasets.insert", a);
}
Json BigQuery::delete_dataset(const std::string& dataset, bool delete_contents) const {
CallArgs a = in_dataset(dataset);
a.q("deleteContents", Json(delete_contents));
return invoke("bigquery.datasets.delete", a);
}
Json BigQuery::list_tables(const std::string& dataset, int max_results) const {
CallArgs a = in_dataset(dataset);
a.q("maxResults", page(max_results));
return invoke("bigquery.tables.list", a);
}
Json BigQuery::get_table(const std::string& dataset, const std::string& table) const {
CallArgs a = in_dataset(dataset);
a.p("tableId", Ids::bq_id(table));
return invoke("bigquery.tables.get", a);
}
Json BigQuery::create_table(const std::string& dataset, const Json& table) const {
CallArgs a = in_dataset(dataset);
a.with_body(table);
return invoke("bigquery.tables.insert", a);
}
Json BigQuery::delete_table(const std::string& dataset, const std::string& table) const {
CallArgs a = in_dataset(dataset);
a.p("tableId", Ids::bq_id(table));
return invoke("bigquery.tables.delete", a);
}
Json BigQuery::list_rows(const std::string& dataset, const std::string& table, int max_results,
const std::string& page_token) const {
CallArgs a = in_dataset(dataset);
a.p("tableId", Ids::bq_id(table));
a.q("maxResults", page(max_results));
if (!page_token.empty()) a.q("pageToken", Json(page_token));
return invoke("bigquery.tabledata.list", a);
}
Json BigQuery::insert_all(const std::string& dataset, const std::string& table, const Json& rows,
bool skip_invalid_rows, bool ignore_unknown_values) const {
if (!rows.is_array() || rows.as_array().empty() ||
rows.as_array().size() > DataCaps::kMaxInsertRows)
throw GoogleError::invalid("rows must be an array of 1.." +
std::to_string(DataCaps::kMaxInsertRows) + " items");
for (const Json& r : rows.as_array())
if (!r.is_object() || !r.find("json") || !r.find("json")->is_object())
throw GoogleError::invalid("every row must be an object with a json object");
if (rows.dump().size() > DataCaps::kMaxInsertBytes)
throw GoogleError::invalid("rows over the insert size cap");
CallArgs a = in_dataset(dataset);
a.p("tableId", Ids::bq_id(table));
Json b = Json::object();
b.set("rows", rows);
b.set("skipInvalidRows", Json(skip_invalid_rows));
b.set("ignoreUnknownValues", Json(ignore_unknown_values));
a.with_body(b);
return invoke("bigquery.tabledata.insertAll", a);
}
Json BigQuery::list_models(const std::string& dataset, int max_results) const {
CallArgs a = in_dataset(dataset);
a.q("maxResults", page(max_results));
return invoke("bigquery.models.list", a);
}
Json BigQuery::get_model(const std::string& dataset, const std::string& model) const {
CallArgs a = in_dataset(dataset);
a.p("modelId", Ids::bq_id(model, "model"));
return invoke("bigquery.models.get", a);
}
Json BigQuery::delete_model(const std::string& dataset, const std::string& model) const {
CallArgs a = in_dataset(dataset);
a.p("modelId", Ids::bq_id(model, "model"));
return invoke("bigquery.models.delete", a);
}
Json BigQuery::list_routines(const std::string& dataset, int max_results) const {
CallArgs a = in_dataset(dataset);
a.q("maxResults", page(max_results));
return invoke("bigquery.routines.list", a);
}
Json BigQuery::get_routine(const std::string& dataset, const std::string& routine) const {
CallArgs a = in_dataset(dataset);
a.p("routineId", Ids::bq_id(routine, "routine"));
return invoke("bigquery.routines.get", a);
}
Json BigQuery::delete_routine(const std::string& dataset, const std::string& routine) const {
CallArgs a = in_dataset(dataset);
a.p("routineId", Ids::bq_id(routine, "routine"));
return invoke("bigquery.routines.delete", a);
}
Json BigQuery::service_account() const { return invoke("bigquery.projects.getServiceAccount", base()); }
} // namespace molo::google::data
+85
View File
@@ -0,0 +1,85 @@
#pragma once
// BigQuery REST API v2: jobs, queries, datasets, tables, routines, models,
// streaming insert (tabledata.insertAll). Every other method of the API is
// reachable with ApiService::invoke("bigquery....").
#include <string>
#include "google/data/api_service.h"
#include "google/resource.h"
namespace molo::google::data {
struct QueryRequest {
std::string sql;
bool dry_run = false; // validate and estimate bytes, run nothing
bool use_legacy_sql = false;
bool use_query_cache = true;
int max_results = 100; // rows in the first response, 1..10000
int timeout_ms = 10000; // wait for completion, 0..60000
std::string location; // processing location, "" = Google decides
std::string default_dataset; // dataset id inside the client project
std::string maximum_bytes_billed; // decimal string, "" = no limit set
clients::body::Json query_parameters; // array of QueryParameter, or null
std::string parameter_mode; // "NAMED" or "POSITIONAL" with parameters
void validate() const;
clients::body::Json to_json(const std::string& project) const;
};
class BigQuery : public ApiService {
public:
explicit BigQuery(std::shared_ptr<GoogleClient> client, CloudContext context);
const CloudContext& context() const { return context_; }
// jobs.query. dry_run=true is free and returns totalBytesProcessed.
clients::body::Json query(const QueryRequest& request) const;
clients::body::Json dry_run(const std::string& sql) const;
clients::body::Json query_results(const std::string& job_id, const std::string& location,
int max_results, const std::string& page_token) const;
clients::body::Json list_jobs(int max_results, const std::string& state_filter) const;
clients::body::Json get_job(const std::string& job_id, const std::string& location) const;
clients::body::Json insert_job(const clients::body::Json& job) const;
clients::body::Json cancel_job(const std::string& job_id, const std::string& location) const;
clients::body::Json list_datasets(int max_results) const;
clients::body::Json get_dataset(const std::string& dataset) const;
clients::body::Json create_dataset(const std::string& dataset, const std::string& location,
const std::string& description) const;
// Destructive, no defaults.
clients::body::Json delete_dataset(const std::string& dataset, bool delete_contents) const;
clients::body::Json list_tables(const std::string& dataset, int max_results) const;
clients::body::Json get_table(const std::string& dataset, const std::string& table) const;
clients::body::Json create_table(const std::string& dataset,
const clients::body::Json& table) const;
clients::body::Json delete_table(const std::string& dataset, const std::string& table) const;
clients::body::Json list_rows(const std::string& dataset, const std::string& table,
int max_results, const std::string& page_token) const;
// tabledata.insertAll: rows is an array of {"insertId"?, "json":{...}}.
clients::body::Json insert_all(const std::string& dataset, const std::string& table,
const clients::body::Json& rows, bool skip_invalid_rows,
bool ignore_unknown_values) const;
clients::body::Json list_models(const std::string& dataset, int max_results) const;
clients::body::Json get_model(const std::string& dataset, const std::string& model) const;
clients::body::Json delete_model(const std::string& dataset, const std::string& model) const;
clients::body::Json list_routines(const std::string& dataset, int max_results) const;
clients::body::Json get_routine(const std::string& dataset, const std::string& routine) const;
clients::body::Json delete_routine(const std::string& dataset,
const std::string& routine) const;
clients::body::Json service_account() const;
private:
CallArgs base() const;
CallArgs in_dataset(const std::string& dataset) const;
static clients::body::Json page(int max_results);
CloudContext context_;
};
} // namespace molo::google::data
+46
View File
@@ -0,0 +1,46 @@
#include "google/data/bigtable_admin.h"
namespace molo::google::data {
using clients::body::Json;
std::string BigtableAdmin::instance_name(const std::string& i) const {
return project_path() + "/instances/" + Ids::id(i, "instance");
}
std::string BigtableAdmin::table_name(const std::string& i, const std::string& t) const {
return instance_name(i) + "/tables/" + Ids::id(t, "table");
}
Json BigtableAdmin::list_instances() const {
return invoke("bigtableadmin.projects.instances.list", on("parent", project_path()));
}
Json BigtableAdmin::get_instance(const std::string& i) const {
return invoke("bigtableadmin.projects.instances.get", on("name", instance_name(i)));
}
Json BigtableAdmin::list_clusters(const std::string& i) const {
return invoke("bigtableadmin.projects.instances.clusters.list", on("parent", instance_name(i)));
}
Json BigtableAdmin::get_cluster(const std::string& i, const std::string& c) const {
return invoke("bigtableadmin.projects.instances.clusters.get",
on("name", instance_name(i) + "/clusters/" + Ids::id(c, "cluster")));
}
Json BigtableAdmin::list_tables(const std::string& i, int n) const {
CallArgs a = on("parent", instance_name(i));
a.q("pageSize", page(n));
return invoke("bigtableadmin.projects.instances.tables.list", a);
}
Json BigtableAdmin::get_table(const std::string& i, const std::string& t) const {
return invoke("bigtableadmin.projects.instances.tables.get", on("name", table_name(i, t)));
}
Json BigtableAdmin::list_app_profiles(const std::string& i, int n) const {
CallArgs a = on("parent", instance_name(i));
a.q("pageSize", page(n));
return invoke("bigtableadmin.projects.instances.appProfiles.list", a);
}
Json BigtableAdmin::table_iam_policy(const std::string& i, const std::string& t) const {
CallArgs a = on("resource", table_name(i, t));
a.with_body(Json::object());
return invoke("bigtableadmin.projects.instances.tables.getIamPolicy", a);
}
} // namespace molo::google::data
+32
View File
@@ -0,0 +1,32 @@
#pragma once
// Cloud Bigtable Admin API v2: instances, clusters, tables, app profiles
// (admin plane only; row reads and writes are the Bigtable data API, which is
// gRPC and not part of this package). Other methods:
// ApiService::invoke("bigtableadmin....").
#include <string>
#include "google/data/project_service.h"
namespace molo::google::data {
class BigtableAdmin : public ProjectService {
public:
explicit BigtableAdmin(std::shared_ptr<GoogleClient> client, CloudContext context)
: ProjectService(std::move(client), Api::kBigtableAdmin, std::move(context)) {}
std::string instance_name(const std::string& instance) const;
std::string table_name(const std::string& instance, const std::string& table) const;
clients::body::Json list_instances() const;
clients::body::Json get_instance(const std::string& instance) const;
clients::body::Json list_clusters(const std::string& instance) const;
clients::body::Json get_cluster(const std::string& instance, const std::string& cluster) const;
clients::body::Json list_tables(const std::string& instance, int page_size) const;
clients::body::Json get_table(const std::string& instance, const std::string& table) const;
clients::body::Json list_app_profiles(const std::string& instance, int page_size) const;
clients::body::Json table_iam_policy(const std::string& instance, const std::string& table) const;
};
} // namespace molo::google::data
+154
View File
@@ -0,0 +1,154 @@
#include "google/data/calls.h"
#include <cstring>
#include <unordered_map>
#include "google/errors.h"
namespace molo::google::data {
namespace {
const ApiInfo kApis[] = {
{Api::kBigquery, "bigquery", "v2", "https://bigquery.googleapis.com/bigquery/v2",
"bigquery.googleapis.com"},
{Api::kStorage, "storage", "v1", "https://storage.googleapis.com/storage/v1",
"storage.googleapis.com"},
{Api::kPubsub, "pubsub", "v1", "https://pubsub.googleapis.com", "pubsub.googleapis.com"},
{Api::kFirestore, "firestore", "v1", "https://firestore.googleapis.com",
"firestore.googleapis.com"},
{Api::kSpanner, "spanner", "v1", "https://spanner.googleapis.com", "spanner.googleapis.com"},
{Api::kBigtableAdmin, "bigtableadmin", "v2", "https://bigtableadmin.googleapis.com",
"bigtableadmin.googleapis.com"},
{Api::kSecretManager, "secretmanager", "v1", "https://secretmanager.googleapis.com",
"secretmanager.googleapis.com"},
{Api::kDataplex, "dataplex", "v1", "https://dataplex.googleapis.com",
"dataplex.googleapis.com"},
{Api::kDlp, "dlp", "v2", "https://dlp.googleapis.com", "dlp.googleapis.com"},
{Api::kKms, "cloudkms", "v1", "https://cloudkms.googleapis.com", "cloudkms.googleapis.com"},
{Api::kDatastore, "datastore", "v1", "https://datastore.googleapis.com",
"datastore.googleapis.com"},
};
// CALLS-BEGIN (parsed by tools/check_discovery.py; one row per line)
const CallSpec kCalls[] = {
#include "google/data/calls_table.inc"
};
// CALLS-END
const std::unordered_map<std::string, const CallSpec*>& index() {
static const auto map = [] {
std::unordered_map<std::string, const CallSpec*> m;
for (const CallSpec& s : kCalls) m.emplace(s.rpc, &s);
return m;
}();
return map;
}
} // namespace
std::size_t Catalog::size() { return sizeof(kCalls) / sizeof(kCalls[0]); }
const CallSpec& Catalog::at(std::size_t i) {
if (i >= size()) throw GoogleError::invalid("call index out of range");
return kCalls[i];
}
const CallSpec* Catalog::find(const std::string& rpc) {
auto it = index().find(rpc);
return it == index().end() ? nullptr : it->second;
}
const CallSpec& Catalog::get(const std::string& rpc) {
const CallSpec* s = find(rpc);
if (!s) throw GoogleError::invalid("unknown call: " + rpc.substr(0, 128));
return *s;
}
const ApiInfo& Catalog::info(Api api) {
for (const ApiInfo& i : kApis)
if (i.api == api) return i;
throw GoogleError::invalid("unknown api");
}
std::size_t Catalog::api_count() { return sizeof(kApis) / sizeof(kApis[0]); }
const ApiInfo& Catalog::info_at(std::size_t i) {
if (i >= api_count()) throw GoogleError::invalid("api index out of range");
return kApis[i];
}
Api Catalog::api_of(const CallSpec& spec) {
const char* dot = std::strchr(spec.rpc, '.');
std::size_t n = dot ? static_cast<std::size_t>(dot - spec.rpc) : std::strlen(spec.rpc);
for (const ApiInfo& i : kApis)
if (std::strlen(i.name) == n && std::strncmp(i.name, spec.rpc, n) == 0) return i.api;
throw GoogleError::invalid("call has no known api prefix");
}
std::size_t Catalog::count_of(Api api) {
std::size_t n = 0;
for (const CallSpec& s : kCalls)
if (api_of(s) == api) ++n;
return n;
}
ServiceEndpoint Catalog::endpoint(Api api, const std::string& base_url_override) {
const ApiInfo& i = info(api);
ServiceEndpoint e;
e.name = i.name;
if (base_url_override.empty()) {
e.base_url = i.base_url;
return e;
}
if (base_url_override.rfind("https://", 0) != 0 || base_url_override.size() > 256 ||
base_url_override.back() == '/')
throw GoogleError::invalid("base url override must be https:// without trailing slash");
// Only Google hosts: a typo must not send a bearer token elsewhere.
auto host_end = base_url_override.find('/', 8);
std::string host = base_url_override.substr(
8, host_end == std::string::npos ? std::string::npos : host_end - 8);
const std::string tail = ".googleapis.com";
if (host.size() <= tail.size() ||
host.compare(host.size() - tail.size(), tail.size(), tail) != 0)
throw GoogleError::invalid("base url override must be on googleapis.com");
e.base_url = base_url_override;
return e;
}
std::vector<std::string> Catalog::split(const char* list) {
std::vector<std::string> out;
std::string cur;
for (const char* p = list; *p; ++p) {
if (*p == ',') {
if (!cur.empty()) out.push_back(cur);
cur.clear();
} else {
cur.push_back(*p);
}
}
if (!cur.empty()) out.push_back(cur);
return out;
}
bool Catalog::in_list(const char* list, const std::string& item) {
for (const std::string& s : split(list))
if (s == item) return true;
return false;
}
std::vector<std::string> Catalog::path_params(const CallSpec& spec) {
std::vector<std::string> out;
const char* p = spec.path;
while ((p = std::strchr(p, '{')) != nullptr) {
const char* e = std::strchr(p, '}');
if (!e) break;
std::string name(p + 1, e);
if (!name.empty() && name[0] == '+') name.erase(0, 1);
out.push_back(name);
p = e + 1;
}
return out;
}
} // namespace molo::google::data
+64
View File
@@ -0,0 +1,64 @@
#pragma once
// The ONE table of REST calls of the data package. Every row is generated from
// Google's Discovery documents (tools/gen_calls.py) and checked by
// tools/check_discovery.py. Code builds every URL from this table, never from
// a literal path.
#include <cstddef>
#include <string>
#include <vector>
#include "google/endpoints.h"
namespace molo::google::data {
enum class Api {
kBigquery, kStorage, kPubsub, kFirestore, kSpanner, kBigtableAdmin,
kSecretManager, kDataplex, kDlp, kKms, kDatastore,
};
struct ApiInfo {
Api api;
const char* name; // Discovery name and rpc id prefix: "bigquery"
const char* version; // "v2"
const char* base_url; // rootUrl + servicePath, no trailing slash
const char* service; // service to enable: "bigquery.googleapis.com"
};
struct CallSpec {
const char* rpc; // discovery method id, e.g. "bigquery.jobs.query"
const char* http; // GET POST PUT PATCH DELETE
const char* path; // discovery "path" relative to base_url
const char* body; // comma list: every top-level field of the request schema
const char* query; // comma list: every query parameter of the method
const char* required; // comma list: required query parameters
};
class Catalog {
public:
static std::size_t size();
static const CallSpec& at(std::size_t index);
// nullptr when absent.
static const CallSpec* find(const std::string& rpc);
// Throws GoogleError (invalid) when absent.
static const CallSpec& get(const std::string& rpc);
static const ApiInfo& info(Api api);
static std::size_t api_count();
static const ApiInfo& info_at(std::size_t index);
static Api api_of(const CallSpec& spec); // from the rpc id prefix
static std::size_t count_of(Api api);
// base_url_override replaces the discovery base (regional hosts); it must
// be https on googleapis.com.
static ServiceEndpoint endpoint(Api api, const std::string& base_url_override = "");
// Comma list helpers.
static std::vector<std::string> split(const char* list);
static bool in_list(const char* list, const std::string& item);
// Names of the {x} and {+x} placeholders of a path, in order, without "+".
static std::vector<std::string> path_params(const CallSpec& spec);
};
} // namespace molo::google::data
+930
View File
@@ -0,0 +1,930 @@
// GENERATED by tools/gen_calls.py from Google Discovery; do not edit by hand.
// bigquery v2 revision 20261009
{"bigquery.datasets.delete", "DELETE", "projects/{+projectId}/datasets/{+datasetId}", "", "deleteContents", ""},
{"bigquery.datasets.get", "GET", "projects/{+projectId}/datasets/{+datasetId}", "", "accessPolicyVersion,datasetView", ""},
{"bigquery.datasets.insert", "POST", "projects/{+projectId}/datasets", "access,catalogSource,creationTime,datasetReference,defaultCollation,defaultEncryptionConfiguration,defaultPartitionExpirationMs,defaultRoundingMode,defaultTableExpirationMs,description,etag,externalCatalogDatasetOptions,externalDatasetReference,friendlyName,id,isCaseInsensitive,kind,labels,lastModifiedTime,linkedDatasetMetadata,linkedDatasetSource,location,maxTimeTravelHours,resourceTags,restrictions,satisfiesPzi,satisfiesPzs,selfLink,storageBillingModel,tags,type", "accessPolicyVersion", ""},
{"bigquery.datasets.list", "GET", "projects/{+projectId}/datasets", "", "all,filter,maxResults,pageToken", ""},
{"bigquery.datasets.patch", "PATCH", "projects/{+projectId}/datasets/{+datasetId}", "access,catalogSource,creationTime,datasetReference,defaultCollation,defaultEncryptionConfiguration,defaultPartitionExpirationMs,defaultRoundingMode,defaultTableExpirationMs,description,etag,externalCatalogDatasetOptions,externalDatasetReference,friendlyName,id,isCaseInsensitive,kind,labels,lastModifiedTime,linkedDatasetMetadata,linkedDatasetSource,location,maxTimeTravelHours,resourceTags,restrictions,satisfiesPzi,satisfiesPzs,selfLink,storageBillingModel,tags,type", "accessPolicyVersion,updateMode", ""},
{"bigquery.datasets.undelete", "POST", "projects/{+projectId}/datasets/{+datasetId}:undelete", "deletionTime", "", ""},
{"bigquery.datasets.update", "PUT", "projects/{+projectId}/datasets/{+datasetId}", "access,catalogSource,creationTime,datasetReference,defaultCollation,defaultEncryptionConfiguration,defaultPartitionExpirationMs,defaultRoundingMode,defaultTableExpirationMs,description,etag,externalCatalogDatasetOptions,externalDatasetReference,friendlyName,id,isCaseInsensitive,kind,labels,lastModifiedTime,linkedDatasetMetadata,linkedDatasetSource,location,maxTimeTravelHours,resourceTags,restrictions,satisfiesPzi,satisfiesPzs,selfLink,storageBillingModel,tags,type", "accessPolicyVersion,updateMode", ""},
{"bigquery.jobs.cancel", "POST", "projects/{+projectId}/jobs/{+jobId}/cancel", "", "location", ""},
{"bigquery.jobs.delete", "DELETE", "projects/{+projectId}/jobs/{+jobId}/delete", "", "location", ""},
{"bigquery.jobs.get", "GET", "projects/{+projectId}/jobs/{+jobId}", "", "location", ""},
{"bigquery.jobs.getQueryResults", "GET", "projects/{+projectId}/queries/{+jobId}", "", "formatOptions.timestampOutputFormat,formatOptions.useInt64Timestamp,location,maxResults,pageToken,startIndex,timeoutMs", ""},
{"bigquery.jobs.insert", "POST", "projects/{+projectId}/jobs", "configuration,etag,id,jobCreationReason,jobReference,kind,principal_subject,selfLink,statistics,status,user_email", "", ""},
{"bigquery.jobs.list", "GET", "projects/{+projectId}/jobs", "", "allUsers,maxCreationTime,maxResults,minCreationTime,pageToken,parentJobId,projection,stateFilter", ""},
{"bigquery.jobs.query", "POST", "projects/{+projectId}/queries", "arrowSerializationOptions,connectionProperties,continuous,createSession,defaultDataset,destinationEncryptionConfiguration,dryRun,formatOptions,jobCreationMode,jobTimeoutMs,kind,labels,location,maxResults,maxSlots,maximumBytesBilled,parameterMode,preserveNulls,query,queryParameters,queryResultsFormat,requestId,reservation,secureContext,timeoutMs,useLegacySql,useQueryCache,writeIncrementalResults", "", ""},
{"bigquery.models.delete", "DELETE", "projects/{+projectId}/datasets/{+datasetId}/models/{+modelId}", "", "", ""},
{"bigquery.models.get", "GET", "projects/{+projectId}/datasets/{+datasetId}/models/{+modelId}", "", "", ""},
{"bigquery.models.list", "GET", "projects/{+projectId}/datasets/{+datasetId}/models", "", "maxResults,pageToken", ""},
{"bigquery.models.patch", "PATCH", "projects/{+projectId}/datasets/{+datasetId}/models/{+modelId}", "bestTrialId,creationTime,defaultTrialId,description,encryptionConfiguration,etag,expirationTime,featureColumns,friendlyName,hparamSearchSpaces,hparamTrials,labelColumns,labels,lastModifiedTime,location,modelReference,modelType,optimalTrialIds,remoteModelInfo,trainingRuns,transformColumns", "", ""},
{"bigquery.projects.getServiceAccount", "GET", "projects/{+projectId}/serviceAccount", "", "", ""},
{"bigquery.projects.list", "GET", "projects", "", "maxResults,pageToken", ""},
{"bigquery.routines.delete", "DELETE", "projects/{+projectId}/datasets/{+datasetId}/routines/{+routineId}", "", "", ""},
{"bigquery.routines.get", "GET", "projects/{+projectId}/datasets/{+datasetId}/routines/{+routineId}", "", "readMask", ""},
{"bigquery.routines.getIamPolicy", "POST", "{+resource}:getIamPolicy", "options", "", ""},
{"bigquery.routines.insert", "POST", "projects/{+projectId}/datasets/{+datasetId}/routines", "arguments,buildStatus,creationTime,dataGovernanceType,definitionBody,description,determinismLevel,etag,externalRuntimeOptions,importedLibraries,language,lastModifiedTime,pythonOptions,remoteFunctionOptions,returnTableType,returnType,routineReference,routineType,securityMode,sparkOptions,strictMode", "", ""},
{"bigquery.routines.list", "GET", "projects/{+projectId}/datasets/{+datasetId}/routines", "", "filter,maxResults,pageToken,readMask", ""},
{"bigquery.routines.setIamPolicy", "POST", "{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigquery.routines.testIamPermissions", "POST", "{+resource}:testIamPermissions", "permissions", "", ""},
{"bigquery.routines.update", "PUT", "projects/{+projectId}/datasets/{+datasetId}/routines/{+routineId}", "arguments,buildStatus,creationTime,dataGovernanceType,definitionBody,description,determinismLevel,etag,externalRuntimeOptions,importedLibraries,language,lastModifiedTime,pythonOptions,remoteFunctionOptions,returnTableType,returnType,routineReference,routineType,securityMode,sparkOptions,strictMode", "", ""},
{"bigquery.rowAccessPolicies.batchDelete", "POST", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/rowAccessPolicies:batchDelete", "force,policyIds", "", ""},
{"bigquery.rowAccessPolicies.delete", "DELETE", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/rowAccessPolicies/{+policyId}", "", "force", ""},
{"bigquery.rowAccessPolicies.get", "GET", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/rowAccessPolicies/{+policyId}", "", "", ""},
{"bigquery.rowAccessPolicies.getIamPolicy", "POST", "{+resource}:getIamPolicy", "options", "", ""},
{"bigquery.rowAccessPolicies.insert", "POST", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/rowAccessPolicies", "creationTime,etag,filterPredicate,grantees,lastModifiedTime,rowAccessPolicyReference", "", ""},
{"bigquery.rowAccessPolicies.list", "GET", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/rowAccessPolicies", "", "pageSize,pageToken", ""},
{"bigquery.rowAccessPolicies.testIamPermissions", "POST", "{+resource}:testIamPermissions", "permissions", "", ""},
{"bigquery.rowAccessPolicies.update", "PUT", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/rowAccessPolicies/{+policyId}", "creationTime,etag,filterPredicate,grantees,lastModifiedTime,rowAccessPolicyReference", "", ""},
{"bigquery.tabledata.insertAll", "POST", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/insertAll", "ignoreUnknownValues,kind,rows,skipInvalidRows,templateSuffix,traceId", "", ""},
{"bigquery.tabledata.list", "GET", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}/data", "", "formatOptions.timestampOutputFormat,formatOptions.useInt64Timestamp,maxResults,pageToken,selectedFields,startIndex", ""},
{"bigquery.tables.delete", "DELETE", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}", "", "", ""},
{"bigquery.tables.get", "GET", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}", "", "selectedFields,view", ""},
{"bigquery.tables.getIamPolicy", "POST", "{+resource}:getIamPolicy", "options", "", ""},
{"bigquery.tables.insert", "POST", "projects/{+projectId}/datasets/{+datasetId}/tables", "biglakeConfiguration,cloneDefinition,clustering,creationTime,defaultCollation,defaultRoundingMode,description,encryptionConfiguration,etag,expirationTime,externalCatalogTableOptions,externalDataConfiguration,friendlyName,id,kind,labels,lastModifiedTime,location,managedTableType,materializedView,materializedViewStatus,maxStaleness,model,numActiveLogicalBytes,numActivePhysicalBytes,numBytes,numCurrentPhysicalBytes,numLongTermBytes,numLongTermLogicalBytes,numLongTermPhysicalBytes,numPartitions,numPhysicalBytes,numRows,numTimeTravelPhysicalBytes,numTotalLogicalBytes,numTotalPhysicalBytes,partitionDefinition,rangePartitioning,replicas,requirePartitionFilter,resourceTags,restrictions,schema,selfLink,snapshotDefinition,streamingBuffer,tableConstraints,tableReference,tableReplicationInfo,timePartitioning,type,view", "", ""},
{"bigquery.tables.list", "GET", "projects/{+projectId}/datasets/{+datasetId}/tables", "", "maxResults,pageToken", ""},
{"bigquery.tables.patch", "PATCH", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}", "biglakeConfiguration,cloneDefinition,clustering,creationTime,defaultCollation,defaultRoundingMode,description,encryptionConfiguration,etag,expirationTime,externalCatalogTableOptions,externalDataConfiguration,friendlyName,id,kind,labels,lastModifiedTime,location,managedTableType,materializedView,materializedViewStatus,maxStaleness,model,numActiveLogicalBytes,numActivePhysicalBytes,numBytes,numCurrentPhysicalBytes,numLongTermBytes,numLongTermLogicalBytes,numLongTermPhysicalBytes,numPartitions,numPhysicalBytes,numRows,numTimeTravelPhysicalBytes,numTotalLogicalBytes,numTotalPhysicalBytes,partitionDefinition,rangePartitioning,replicas,requirePartitionFilter,resourceTags,restrictions,schema,selfLink,snapshotDefinition,streamingBuffer,tableConstraints,tableReference,tableReplicationInfo,timePartitioning,type,view", "autodetect_schema", ""},
{"bigquery.tables.setIamPolicy", "POST", "{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigquery.tables.testIamPermissions", "POST", "{+resource}:testIamPermissions", "permissions", "", ""},
{"bigquery.tables.update", "PUT", "projects/{+projectId}/datasets/{+datasetId}/tables/{+tableId}", "biglakeConfiguration,cloneDefinition,clustering,creationTime,defaultCollation,defaultRoundingMode,description,encryptionConfiguration,etag,expirationTime,externalCatalogTableOptions,externalDataConfiguration,friendlyName,id,kind,labels,lastModifiedTime,location,managedTableType,materializedView,materializedViewStatus,maxStaleness,model,numActiveLogicalBytes,numActivePhysicalBytes,numBytes,numCurrentPhysicalBytes,numLongTermBytes,numLongTermLogicalBytes,numLongTermPhysicalBytes,numPartitions,numPhysicalBytes,numRows,numTimeTravelPhysicalBytes,numTotalLogicalBytes,numTotalPhysicalBytes,partitionDefinition,rangePartitioning,replicas,requirePartitionFilter,resourceTags,restrictions,schema,selfLink,snapshotDefinition,streamingBuffer,tableConstraints,tableReference,tableReplicationInfo,timePartitioning,type,view", "autodetect_schema", ""},
// storage v1 revision 20261020
{"storage.anywhereCaches.disable", "POST", "b/{bucket}/anywhereCaches/{anywhereCacheId}/disable", "", "", ""},
{"storage.anywhereCaches.get", "GET", "b/{bucket}/anywhereCaches/{anywhereCacheId}", "", "", ""},
{"storage.anywhereCaches.insert", "POST", "b/{bucket}/anywhereCaches", "admissionPolicy,anywhereCacheId,bucket,createTime,id,ingestOnWrite,kind,pendingUpdate,selfLink,state,ttl,updateTime,zone", "", ""},
{"storage.anywhereCaches.list", "GET", "b/{bucket}/anywhereCaches", "", "pageSize,pageToken", ""},
{"storage.anywhereCaches.pause", "POST", "b/{bucket}/anywhereCaches/{anywhereCacheId}/pause", "", "", ""},
{"storage.anywhereCaches.resume", "POST", "b/{bucket}/anywhereCaches/{anywhereCacheId}/resume", "", "", ""},
{"storage.anywhereCaches.update", "PATCH", "b/{bucket}/anywhereCaches/{anywhereCacheId}", "admissionPolicy,anywhereCacheId,bucket,createTime,id,ingestOnWrite,kind,pendingUpdate,selfLink,state,ttl,updateTime,zone", "", ""},
{"storage.bucketAccessControls.delete", "DELETE", "b/{bucket}/acl/{entity}", "", "userProject", ""},
{"storage.bucketAccessControls.get", "GET", "b/{bucket}/acl/{entity}", "", "userProject", ""},
{"storage.bucketAccessControls.insert", "POST", "b/{bucket}/acl", "bucket,domain,email,entity,entityId,etag,id,kind,projectTeam,role,selfLink", "userProject", ""},
{"storage.bucketAccessControls.list", "GET", "b/{bucket}/acl", "", "userProject", ""},
{"storage.bucketAccessControls.patch", "PATCH", "b/{bucket}/acl/{entity}", "bucket,domain,email,entity,entityId,etag,id,kind,projectTeam,role,selfLink", "userProject", ""},
{"storage.bucketAccessControls.update", "PUT", "b/{bucket}/acl/{entity}", "bucket,domain,email,entity,entityId,etag,id,kind,projectTeam,role,selfLink", "userProject", ""},
{"storage.buckets.delete", "DELETE", "b/{bucket}", "", "ifMetagenerationMatch,ifMetagenerationNotMatch,userProject", ""},
{"storage.buckets.get", "GET", "b/{bucket}", "", "generation,ifMetagenerationMatch,ifMetagenerationNotMatch,projection,softDeleted,userProject", ""},
{"storage.buckets.getIamPolicy", "GET", "b/{bucket}/iam", "", "optionsRequestedPolicyVersion,userProject", ""},
{"storage.buckets.getStorageLayout", "GET", "b/{bucket}/storageLayout", "", "prefix", ""},
{"storage.buckets.insert", "POST", "b", "acl,autoclass,billing,cors,customPlacementConfig,defaultEventBasedHold,defaultObjectAcl,encryption,etag,generation,hardDeleteTime,hierarchicalNamespace,iamConfiguration,id,ipFilter,kind,labels,lifecycle,location,locationType,logging,metageneration,name,objectRetention,owner,projectNumber,retentionPolicy,rpo,satisfiesPZI,satisfiesPZS,selfLink,softDeletePolicy,softDeleteTime,storageClass,timeCreated,updated,versioning,website", "enableObjectRetention,predefinedAcl,predefinedDefaultObjectAcl,project,projection,userProject", "project"},
{"storage.buckets.list", "GET", "b", "", "maxResults,pageToken,prefix,project,projection,returnPartialSuccess,softDeleted,userProject", "project"},
{"storage.buckets.lockRetentionPolicy", "POST", "b/{bucket}/lockRetentionPolicy", "", "ifMetagenerationMatch,userProject", "ifMetagenerationMatch"},
{"storage.buckets.operations.advanceRelocateBucket", "POST", "b/{bucket}/operations/{operationId}/advanceRelocateBucket", "expireTime,ttl", "", ""},
{"storage.buckets.operations.cancel", "POST", "b/{bucket}/operations/{operationId}/cancel", "", "", ""},
{"storage.buckets.operations.get", "GET", "b/{bucket}/operations/{operationId}", "", "", ""},
{"storage.buckets.operations.list", "GET", "b/{bucket}/operations", "", "filter,pageSize,pageToken", ""},
{"storage.buckets.patch", "PATCH", "b/{bucket}", "acl,autoclass,billing,cors,customPlacementConfig,defaultEventBasedHold,defaultObjectAcl,encryption,etag,generation,hardDeleteTime,hierarchicalNamespace,iamConfiguration,id,ipFilter,kind,labels,lifecycle,location,locationType,logging,metageneration,name,objectRetention,owner,projectNumber,retentionPolicy,rpo,satisfiesPZI,satisfiesPZS,selfLink,softDeletePolicy,softDeleteTime,storageClass,timeCreated,updated,versioning,website", "ifMetagenerationMatch,ifMetagenerationNotMatch,predefinedAcl,predefinedDefaultObjectAcl,projection,userProject", ""},
{"storage.buckets.relocate", "POST", "b/{bucket}/relocate", "destinationCustomPlacementConfig,destinationKmsKeyName,destinationLocation,validateOnly", "", ""},
{"storage.buckets.restore", "POST", "b/{bucket}/restore", "", "generation,projection,userProject", "generation"},
{"storage.buckets.setIamPolicy", "PUT", "b/{bucket}/iam", "bindings,etag,kind,resourceId,version", "userProject", ""},
{"storage.buckets.testIamPermissions", "GET", "b/{bucket}/iam/testPermissions", "", "permissions,userProject", "permissions"},
{"storage.buckets.update", "PUT", "b/{bucket}", "acl,autoclass,billing,cors,customPlacementConfig,defaultEventBasedHold,defaultObjectAcl,encryption,etag,generation,hardDeleteTime,hierarchicalNamespace,iamConfiguration,id,ipFilter,kind,labels,lifecycle,location,locationType,logging,metageneration,name,objectRetention,owner,projectNumber,retentionPolicy,rpo,satisfiesPZI,satisfiesPZS,selfLink,softDeletePolicy,softDeleteTime,storageClass,timeCreated,updated,versioning,website", "ifMetagenerationMatch,ifMetagenerationNotMatch,predefinedAcl,predefinedDefaultObjectAcl,projection,userProject", ""},
{"storage.defaultObjectAccessControls.delete", "DELETE", "b/{bucket}/defaultObjectAcl/{entity}", "", "userProject", ""},
{"storage.defaultObjectAccessControls.get", "GET", "b/{bucket}/defaultObjectAcl/{entity}", "", "userProject", ""},
{"storage.defaultObjectAccessControls.insert", "POST", "b/{bucket}/defaultObjectAcl", "bucket,domain,email,entity,entityId,etag,generation,id,kind,object,projectTeam,role,selfLink", "userProject", ""},
{"storage.defaultObjectAccessControls.list", "GET", "b/{bucket}/defaultObjectAcl", "", "ifMetagenerationMatch,ifMetagenerationNotMatch,userProject", ""},
{"storage.defaultObjectAccessControls.patch", "PATCH", "b/{bucket}/defaultObjectAcl/{entity}", "bucket,domain,email,entity,entityId,etag,generation,id,kind,object,projectTeam,role,selfLink", "userProject", ""},
{"storage.defaultObjectAccessControls.update", "PUT", "b/{bucket}/defaultObjectAcl/{entity}", "bucket,domain,email,entity,entityId,etag,generation,id,kind,object,projectTeam,role,selfLink", "userProject", ""},
{"storage.folders.delete", "DELETE", "b/{bucket}/folders/{folder}", "", "ifMetagenerationMatch,ifMetagenerationNotMatch", ""},
{"storage.folders.deleteRecursive", "POST", "b/{bucket}/folders/{folder}/deleteRecursive", "", "ifMetagenerationMatch,ifMetagenerationNotMatch", ""},
{"storage.folders.get", "GET", "b/{bucket}/folders/{folder}", "", "ifMetagenerationMatch,ifMetagenerationNotMatch", ""},
{"storage.folders.insert", "POST", "b/{bucket}/folders", "bucket,createTime,id,kind,metageneration,name,pendingRenameInfo,selfLink,updateTime", "recursive", ""},
{"storage.folders.list", "GET", "b/{bucket}/folders", "", "delimiter,endOffset,pageSize,pageToken,prefix,startOffset", ""},
{"storage.folders.rename", "POST", "b/{bucket}/folders/{sourceFolder}/renameTo/folders/{destinationFolder}", "", "ifSourceMetagenerationMatch,ifSourceMetagenerationNotMatch", ""},
{"storage.managedFolders.delete", "DELETE", "b/{bucket}/managedFolders/{managedFolder}", "", "allowNonEmpty,ifMetagenerationMatch,ifMetagenerationNotMatch", ""},
{"storage.managedFolders.get", "GET", "b/{bucket}/managedFolders/{managedFolder}", "", "ifMetagenerationMatch,ifMetagenerationNotMatch", ""},
{"storage.managedFolders.getIamPolicy", "GET", "b/{bucket}/managedFolders/{managedFolder}/iam", "", "optionsRequestedPolicyVersion,userProject", ""},
{"storage.managedFolders.insert", "POST", "b/{bucket}/managedFolders", "bucket,createTime,id,kind,metageneration,name,rapidCacheConfig,selfLink,updateTime", "", ""},
{"storage.managedFolders.list", "GET", "b/{bucket}/managedFolders", "", "pageSize,pageToken,prefix", ""},
{"storage.managedFolders.setIamPolicy", "PUT", "b/{bucket}/managedFolders/{managedFolder}/iam", "bindings,etag,kind,resourceId,version", "userProject", ""},
{"storage.managedFolders.testIamPermissions", "GET", "b/{bucket}/managedFolders/{managedFolder}/iam/testPermissions", "", "permissions,userProject", "permissions"},
{"storage.managedFolders.update", "PATCH", "b/{bucket}/managedFolders/{managedFolder}", "bucket,createTime,id,kind,metageneration,name,rapidCacheConfig,selfLink,updateTime", "ifMetagenerationMatch,ifMetagenerationNotMatch", ""},
{"storage.notifications.delete", "DELETE", "b/{bucket}/notificationConfigs/{notification}", "", "userProject", ""},
{"storage.notifications.get", "GET", "b/{bucket}/notificationConfigs/{notification}", "", "userProject", ""},
{"storage.notifications.insert", "POST", "b/{bucket}/notificationConfigs", "custom_attributes,etag,event_types,id,kind,object_name_prefix,payload_format,selfLink,topic", "userProject", ""},
{"storage.notifications.list", "GET", "b/{bucket}/notificationConfigs", "", "userProject", ""},
{"storage.objectAccessControls.delete", "DELETE", "b/{bucket}/o/{object}/acl/{entity}", "", "generation,userProject", ""},
{"storage.objectAccessControls.get", "GET", "b/{bucket}/o/{object}/acl/{entity}", "", "generation,userProject", ""},
{"storage.objectAccessControls.insert", "POST", "b/{bucket}/o/{object}/acl", "bucket,domain,email,entity,entityId,etag,generation,id,kind,object,projectTeam,role,selfLink", "generation,userProject", ""},
{"storage.objectAccessControls.list", "GET", "b/{bucket}/o/{object}/acl", "", "generation,userProject", ""},
{"storage.objectAccessControls.patch", "PATCH", "b/{bucket}/o/{object}/acl/{entity}", "bucket,domain,email,entity,entityId,etag,generation,id,kind,object,projectTeam,role,selfLink", "generation,userProject", ""},
{"storage.objectAccessControls.update", "PUT", "b/{bucket}/o/{object}/acl/{entity}", "bucket,domain,email,entity,entityId,etag,generation,id,kind,object,projectTeam,role,selfLink", "generation,userProject", ""},
{"storage.objects.bulkRestore", "POST", "b/{bucket}/o/bulkRestore", "allowOverwrite,copySourceAcl,createdAfterTime,createdBeforeTime,matchGlobs,softDeletedAfterTime,softDeletedBeforeTime", "", ""},
{"storage.objects.compose", "POST", "b/{destinationBucket}/o/{destinationObject}/compose", "deleteSourceObjects,destination,kind,sourceObjects", "destinationPredefinedAcl,dropContextGroups,ifGenerationMatch,ifMetagenerationMatch,kmsKeyName,userProject", ""},
{"storage.objects.copy", "POST", "b/{sourceBucket}/o/{sourceObject}/copyTo/b/{destinationBucket}/o/{destinationObject}", "acl,bucket,cacheControl,componentCount,contentDisposition,contentEncoding,contentLanguage,contentType,contexts,crc32c,customTime,customerEncryption,etag,eventBasedHold,generation,hardDeleteTime,id,kind,kmsKeyName,md5Hash,mediaLink,metadata,metageneration,name,owner,restoreToken,retention,retentionExpirationTime,selfLink,size,softDeleteTime,storageClass,temporaryHold,timeCreated,timeDeleted,timeFinalized,timeStorageClassUpdated,updated", "destinationKmsKeyName,destinationPredefinedAcl,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,ifSourceGenerationMatch,ifSourceGenerationNotMatch,ifSourceMetagenerationMatch,ifSourceMetagenerationNotMatch,projection,sourceGeneration,userProject", ""},
{"storage.objects.delete", "DELETE", "b/{bucket}/o/{object}", "", "generation,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,userProject", ""},
{"storage.objects.get", "GET", "b/{bucket}/o/{object}", "", "generation,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,projection,restoreToken,softDeleted,userProject", ""},
{"storage.objects.getIamPolicy", "GET", "b/{bucket}/o/{object}/iam", "", "generation,userProject", ""},
{"storage.objects.insert", "POST", "b/{bucket}/o", "acl,bucket,cacheControl,componentCount,contentDisposition,contentEncoding,contentLanguage,contentType,contexts,crc32c,customTime,customerEncryption,etag,eventBasedHold,generation,hardDeleteTime,id,kind,kmsKeyName,md5Hash,mediaLink,metadata,metageneration,name,owner,restoreToken,retention,retentionExpirationTime,selfLink,size,softDeleteTime,storageClass,temporaryHold,timeCreated,timeDeleted,timeFinalized,timeStorageClassUpdated,updated", "contentEncoding,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,kmsKeyName,name,predefinedAcl,projection,userProject", ""},
{"storage.objects.list", "GET", "b/{bucket}/o", "", "delimiter,endOffset,filter,includeFoldersAsPrefixes,includeTrailingDelimiter,matchGlob,maxResults,pageToken,prefix,projection,softDeleted,startOffset,userProject,versions", ""},
{"storage.objects.move", "POST", "b/{bucket}/o/{sourceObject}/moveTo/o/{destinationObject}", "", "ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,ifSourceGenerationMatch,ifSourceGenerationNotMatch,ifSourceMetagenerationMatch,ifSourceMetagenerationNotMatch,projection,userProject", ""},
{"storage.objects.patch", "PATCH", "b/{bucket}/o/{object}", "acl,bucket,cacheControl,componentCount,contentDisposition,contentEncoding,contentLanguage,contentType,contexts,crc32c,customTime,customerEncryption,etag,eventBasedHold,generation,hardDeleteTime,id,kind,kmsKeyName,md5Hash,mediaLink,metadata,metageneration,name,owner,restoreToken,retention,retentionExpirationTime,selfLink,size,softDeleteTime,storageClass,temporaryHold,timeCreated,timeDeleted,timeFinalized,timeStorageClassUpdated,updated", "generation,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,overrideUnlockedRetention,predefinedAcl,projection,userProject", ""},
{"storage.objects.restore", "POST", "b/{bucket}/o/{object}/restore", "", "copySourceAcl,generation,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,projection,restoreToken,userProject", "generation"},
{"storage.objects.rewrite", "POST", "b/{sourceBucket}/o/{sourceObject}/rewriteTo/b/{destinationBucket}/o/{destinationObject}", "acl,bucket,cacheControl,componentCount,contentDisposition,contentEncoding,contentLanguage,contentType,contexts,crc32c,customTime,customerEncryption,etag,eventBasedHold,generation,hardDeleteTime,id,kind,kmsKeyName,md5Hash,mediaLink,metadata,metageneration,name,owner,restoreToken,retention,retentionExpirationTime,selfLink,size,softDeleteTime,storageClass,temporaryHold,timeCreated,timeDeleted,timeFinalized,timeStorageClassUpdated,updated", "destinationKmsKeyName,destinationPredefinedAcl,dropContextGroups,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,ifSourceGenerationMatch,ifSourceGenerationNotMatch,ifSourceMetagenerationMatch,ifSourceMetagenerationNotMatch,maxBytesRewrittenPerCall,projection,rewriteToken,sourceGeneration,userProject", ""},
{"storage.objects.setIamPolicy", "PUT", "b/{bucket}/o/{object}/iam", "bindings,etag,kind,resourceId,version", "generation,userProject", ""},
{"storage.objects.testIamPermissions", "GET", "b/{bucket}/o/{object}/iam/testPermissions", "", "generation,permissions,userProject", "permissions"},
{"storage.objects.update", "PUT", "b/{bucket}/o/{object}", "acl,bucket,cacheControl,componentCount,contentDisposition,contentEncoding,contentLanguage,contentType,contexts,crc32c,customTime,customerEncryption,etag,eventBasedHold,generation,hardDeleteTime,id,kind,kmsKeyName,md5Hash,mediaLink,metadata,metageneration,name,owner,restoreToken,retention,retentionExpirationTime,selfLink,size,softDeleteTime,storageClass,temporaryHold,timeCreated,timeDeleted,timeFinalized,timeStorageClassUpdated,updated", "generation,ifGenerationMatch,ifGenerationNotMatch,ifMetagenerationMatch,ifMetagenerationNotMatch,overrideUnlockedRetention,predefinedAcl,projection,userProject", ""},
{"storage.objects.viewFullContext", "GET", "b/{bucket}/o/{object}/viewFullContext", "", "contextKey,generation,userProject", "contextKey"},
{"storage.projects.hmacKeys.create", "POST", "projects/{projectId}/hmacKeys", "", "serviceAccountEmail,userProject", "serviceAccountEmail"},
{"storage.projects.hmacKeys.delete", "DELETE", "projects/{projectId}/hmacKeys/{accessId}", "", "userProject", ""},
{"storage.projects.hmacKeys.get", "GET", "projects/{projectId}/hmacKeys/{accessId}", "", "userProject", ""},
{"storage.projects.hmacKeys.list", "GET", "projects/{projectId}/hmacKeys", "", "maxResults,pageToken,serviceAccountEmail,showDeletedKeys,userProject", ""},
{"storage.projects.hmacKeys.update", "PUT", "projects/{projectId}/hmacKeys/{accessId}", "accessId,etag,id,kind,projectId,selfLink,serviceAccountEmail,state,timeCreated,updated", "userProject", ""},
{"storage.projects.serviceAccount.get", "GET", "projects/{projectId}/serviceAccount", "", "userProject", ""},
{"storage.rapidCaches.disable", "POST", "b/{bucket}/rapidCaches/{rapidCacheId}/disable", "", "", ""},
{"storage.rapidCaches.get", "GET", "b/{bucket}/rapidCaches/{rapidCacheId}", "", "", ""},
{"storage.rapidCaches.insert", "POST", "b/{bucket}/rapidCaches", "admissionPolicy,bucket,cacheType,createTime,id,ingestOnWrite,kind,pendingUpdate,rapidCacheId,selfLink,state,ttl,updateTime,zone", "", ""},
{"storage.rapidCaches.list", "GET", "b/{bucket}/rapidCaches", "", "pageSize,pageToken", ""},
{"storage.rapidCaches.update", "PATCH", "b/{bucket}/rapidCaches/{rapidCacheId}", "admissionPolicy,bucket,cacheType,createTime,id,ingestOnWrite,kind,pendingUpdate,rapidCacheId,selfLink,state,ttl,updateTime,zone", "", ""},
// pubsub v1 revision 20261013
{"pubsub.projects.schemas.commit", "POST", "v1/{+name}:commit", "schema", "", ""},
{"pubsub.projects.schemas.create", "POST", "v1/{+parent}/schemas", "compiledProtoSchema,definition,name,revisionCreateTime,revisionId,type", "schemaId", ""},
{"pubsub.projects.schemas.delete", "DELETE", "v1/{+name}", "", "", ""},
{"pubsub.projects.schemas.deleteRevision", "DELETE", "v1/{+name}:deleteRevision", "", "revisionId", ""},
{"pubsub.projects.schemas.get", "GET", "v1/{+name}", "", "view", ""},
{"pubsub.projects.schemas.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"pubsub.projects.schemas.list", "GET", "v1/{+parent}/schemas", "", "pageSize,pageToken,view", ""},
{"pubsub.projects.schemas.listRevisions", "GET", "v1/{+name}:listRevisions", "", "pageSize,pageToken,view", ""},
{"pubsub.projects.schemas.rollback", "POST", "v1/{+name}:rollback", "revisionId", "", ""},
{"pubsub.projects.schemas.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"pubsub.projects.schemas.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"pubsub.projects.schemas.validate", "POST", "v1/{+parent}/schemas:validate", "schema", "", ""},
{"pubsub.projects.schemas.validateMessage", "POST", "v1/{+parent}/schemas:validateMessage", "encoding,message,name,schema", "", ""},
{"pubsub.projects.snapshots.create", "PUT", "v1/{+name}", "labels,subscription,tags", "", ""},
{"pubsub.projects.snapshots.delete", "DELETE", "v1/{+snapshot}", "", "", ""},
{"pubsub.projects.snapshots.get", "GET", "v1/{+snapshot}", "", "", ""},
{"pubsub.projects.snapshots.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"pubsub.projects.snapshots.list", "GET", "v1/{+project}/snapshots", "", "pageSize,pageToken", ""},
{"pubsub.projects.snapshots.patch", "PATCH", "v1/{+name}", "snapshot,updateMask", "", ""},
{"pubsub.projects.snapshots.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"pubsub.projects.snapshots.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"pubsub.projects.subscriptions.acknowledge", "POST", "v1/{+subscription}:acknowledge", "ackIds", "", ""},
{"pubsub.projects.subscriptions.create", "PUT", "v1/{+name}", "ackDeadlineSeconds,analyticsHubSubscriptionInfo,bigqueryConfig,bigtableConfig,cloudStorageConfig,deadLetterPolicy,detached,enableExactlyOnceDelivery,enableMessageOrdering,expirationPolicy,filter,labels,messageRetentionDuration,messageTransforms,name,pushConfig,retainAckedMessages,retryPolicy,state,tags,topic,topicMessageRetentionDuration", "", ""},
{"pubsub.projects.subscriptions.delete", "DELETE", "v1/{+subscription}", "", "", ""},
{"pubsub.projects.subscriptions.detach", "POST", "v1/{+subscription}:detach", "", "", ""},
{"pubsub.projects.subscriptions.get", "GET", "v1/{+subscription}", "", "", ""},
{"pubsub.projects.subscriptions.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"pubsub.projects.subscriptions.list", "GET", "v1/{+project}/subscriptions", "", "pageSize,pageToken", ""},
{"pubsub.projects.subscriptions.modifyAckDeadline", "POST", "v1/{+subscription}:modifyAckDeadline", "ackDeadlineSeconds,ackIds", "", ""},
{"pubsub.projects.subscriptions.modifyPushConfig", "POST", "v1/{+subscription}:modifyPushConfig", "pushConfig", "", ""},
{"pubsub.projects.subscriptions.patch", "PATCH", "v1/{+name}", "subscription,updateMask", "", ""},
{"pubsub.projects.subscriptions.pull", "POST", "v1/{+subscription}:pull", "maxMessages,returnImmediately", "", ""},
{"pubsub.projects.subscriptions.seek", "POST", "v1/{+subscription}:seek", "snapshot,time", "", ""},
{"pubsub.projects.subscriptions.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"pubsub.projects.subscriptions.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"pubsub.projects.topics.create", "PUT", "v1/{+name}", "ingestionDataSourceSettings,kmsKeyName,labels,messageRetentionDuration,messageStoragePolicy,messageTransforms,name,satisfiesPzs,schemaSettings,state,tags", "", ""},
{"pubsub.projects.topics.delete", "DELETE", "v1/{+topic}", "", "", ""},
{"pubsub.projects.topics.get", "GET", "v1/{+topic}", "", "", ""},
{"pubsub.projects.topics.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"pubsub.projects.topics.list", "GET", "v1/{+project}/topics", "", "pageSize,pageToken", ""},
{"pubsub.projects.topics.patch", "PATCH", "v1/{+name}", "topic,updateMask", "", ""},
{"pubsub.projects.topics.publish", "POST", "v1/{+topic}:publish", "messages", "", ""},
{"pubsub.projects.topics.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"pubsub.projects.topics.snapshots.list", "GET", "v1/{+topic}/snapshots", "", "pageSize,pageToken", ""},
{"pubsub.projects.topics.subscriptions.list", "GET", "v1/{+topic}/subscriptions", "", "pageSize,pageToken", ""},
{"pubsub.projects.topics.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
// firestore v1 revision 20261013
{"firestore.projects.databases.backupSchedules.create", "POST", "v1/{+parent}/backupSchedules", "createTime,dailyRecurrence,name,retention,updateTime,weeklyRecurrence", "", ""},
{"firestore.projects.databases.backupSchedules.delete", "DELETE", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.backupSchedules.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.backupSchedules.list", "GET", "v1/{+parent}/backupSchedules", "", "", ""},
{"firestore.projects.databases.backupSchedules.patch", "PATCH", "v1/{+name}", "createTime,dailyRecurrence,name,retention,updateTime,weeklyRecurrence", "updateMask", ""},
{"firestore.projects.databases.bulkDeleteDocuments", "POST", "v1/{+name}:bulkDeleteDocuments", "collectionIds,namespaceIds", "", ""},
{"firestore.projects.databases.changeStreams.create", "POST", "v1/{+parent}/changeStreams", "collectionGroupScope,createTime,databaseScope,etag,name,retentionPeriod,startTime,updateTime", "changeStreamId", ""},
{"firestore.projects.databases.changeStreams.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"firestore.projects.databases.changeStreams.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.changeStreams.list", "GET", "v1/{+parent}/changeStreams", "", "", ""},
{"firestore.projects.databases.clone", "POST", "v1/{+parent}/databases:clone", "databaseId,encryptionConfig,pitrSnapshot,tags", "", ""},
{"firestore.projects.databases.collectionGroups.fields.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.collectionGroups.fields.list", "GET", "v1/{+parent}/fields", "", "filter,pageSize,pageToken", ""},
{"firestore.projects.databases.collectionGroups.fields.patch", "PATCH", "v1/{+name}", "indexConfig,name,ttlConfig", "updateMask", ""},
{"firestore.projects.databases.collectionGroups.indexes.create", "POST", "v1/{+parent}/indexes", "apiScope,density,fields,multikey,name,queryScope,searchIndexOptions,shardCount,state,unique", "", ""},
{"firestore.projects.databases.collectionGroups.indexes.delete", "DELETE", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.collectionGroups.indexes.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.collectionGroups.indexes.list", "GET", "v1/{+parent}/indexes", "", "filter,pageSize,pageToken", ""},
{"firestore.projects.databases.create", "POST", "v1/{+parent}/databases", "appEngineIntegrationMode,cmekConfig,concurrencyMode,createTime,databaseEdition,deleteProtectionState,deleteTime,earliestVersionTime,etag,firestoreDataAccessMode,freeTier,keyPrefix,locationId,mongodbCompatibleDataAccessMode,name,pointInTimeRecoveryEnablement,previousId,realtimeUpdatesMode,sourceInfo,tags,type,uid,updateTime,versionRetentionPeriod", "databaseId", ""},
{"firestore.projects.databases.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"firestore.projects.databases.documents.batchGet", "POST", "v1/{+database}/documents:batchGet", "documents,mask,newTransaction,readTime,requestOptions,transaction", "", ""},
{"firestore.projects.databases.documents.batchWrite", "POST", "v1/{+database}/documents:batchWrite", "labels,requestOptions,writes", "", ""},
{"firestore.projects.databases.documents.beginTransaction", "POST", "v1/{+database}/documents:beginTransaction", "options,requestOptions", "", ""},
{"firestore.projects.databases.documents.commit", "POST", "v1/{+database}/documents:commit", "requestOptions,transaction,writes", "", ""},
{"firestore.projects.databases.documents.createDocument", "POST", "v1/{+parent}/{collectionId}", "createTime,fields,name,updateTime", "documentId,mask.fieldPaths,requestOptions.requestTags", ""},
{"firestore.projects.databases.documents.delete", "DELETE", "v1/{+name}", "", "currentDocument.exists,currentDocument.updateTime,requestOptions.requestTags", ""},
{"firestore.projects.databases.documents.executePipeline", "POST", "v1/{+database}/documents:executePipeline", "autoCommitTransaction,newTransaction,readTime,requestOptions,structuredPipeline,transaction", "", ""},
{"firestore.projects.databases.documents.get", "GET", "v1/{+name}", "", "mask.fieldPaths,readTime,requestOptions.requestTags,transaction", ""},
{"firestore.projects.databases.documents.list", "GET", "v1/{+parent}/{collectionId}", "", "mask.fieldPaths,orderBy,pageSize,pageToken,readTime,recursive,requestOptions.requestTags,showMissing,transaction", ""},
{"firestore.projects.databases.documents.listCollectionIds", "POST", "v1/{+parent}:listCollectionIds", "pageSize,pageToken,readTime,requestOptions", "", ""},
{"firestore.projects.databases.documents.listDocuments", "GET", "v1/{+parent}/{collectionId}", "", "mask.fieldPaths,orderBy,pageSize,pageToken,readTime,recursive,requestOptions.requestTags,showMissing,transaction", ""},
{"firestore.projects.databases.documents.listen", "POST", "v1/{+database}/documents:listen", "addTarget,labels,removeTarget,requestOptions", "", ""},
{"firestore.projects.databases.documents.partitionQuery", "POST", "v1/{+parent}:partitionQuery", "pageSize,pageToken,partitionCount,readTime,requestOptions,structuredQuery", "", ""},
{"firestore.projects.databases.documents.patch", "PATCH", "v1/{+name}", "createTime,fields,name,updateTime", "currentDocument.exists,currentDocument.updateTime,mask.fieldPaths,requestOptions.requestTags,updateMask.fieldPaths", ""},
{"firestore.projects.databases.documents.rollback", "POST", "v1/{+database}/documents:rollback", "requestOptions,transaction", "", ""},
{"firestore.projects.databases.documents.runAggregationQuery", "POST", "v1/{+parent}:runAggregationQuery", "explainOptions,newTransaction,readTime,requestOptions,structuredAggregationQuery,transaction", "", ""},
{"firestore.projects.databases.documents.runQuery", "POST", "v1/{+parent}:runQuery", "explainOptions,newTransaction,readTime,requestOptions,structuredQuery,transaction", "", ""},
{"firestore.projects.databases.documents.write", "POST", "v1/{+database}/documents:write", "labels,requestOptions,streamId,streamToken,writes", "", ""},
{"firestore.projects.databases.exportDocuments", "POST", "v1/{+name}:exportDocuments", "collectionIds,namespaceIds,outputUriPrefix,snapshotTime", "", ""},
{"firestore.projects.databases.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.importDocuments", "POST", "v1/{+name}:importDocuments", "collectionIds,inputUriPrefix,namespaceIds", "", ""},
{"firestore.projects.databases.list", "GET", "v1/{+parent}/databases", "", "showDeleted", ""},
{"firestore.projects.databases.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"firestore.projects.databases.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.operations.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.operations.list", "GET", "v1/{+name}/operations", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"firestore.projects.databases.patch", "PATCH", "v1/{+name}", "appEngineIntegrationMode,cmekConfig,concurrencyMode,createTime,databaseEdition,deleteProtectionState,deleteTime,earliestVersionTime,etag,firestoreDataAccessMode,freeTier,keyPrefix,locationId,mongodbCompatibleDataAccessMode,name,pointInTimeRecoveryEnablement,previousId,realtimeUpdatesMode,sourceInfo,tags,type,uid,updateTime,versionRetentionPeriod", "updateMask", ""},
{"firestore.projects.databases.restore", "POST", "v1/{+parent}/databases:restore", "backup,databaseId,encryptionConfig,tags", "", ""},
{"firestore.projects.databases.userCreds.create", "POST", "v1/{+parent}/userCreds", "createTime,name,resourceIdentity,securePassword,state,updateTime", "userCredsId", ""},
{"firestore.projects.databases.userCreds.delete", "DELETE", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.userCreds.disable", "POST", "v1/{+name}:disable", "", "", ""},
{"firestore.projects.databases.userCreds.enable", "POST", "v1/{+name}:enable", "", "", ""},
{"firestore.projects.databases.userCreds.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.databases.userCreds.list", "GET", "v1/{+parent}/userCreds", "", "", ""},
{"firestore.projects.databases.userCreds.resetPassword", "POST", "v1/{+name}:resetPassword", "", "", ""},
{"firestore.projects.locations.backups.delete", "DELETE", "v1/{+name}", "", "", ""},
{"firestore.projects.locations.backups.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.locations.backups.list", "GET", "v1/{+parent}/backups", "", "filter", ""},
{"firestore.projects.locations.get", "GET", "v1/{+name}", "", "", ""},
{"firestore.projects.locations.list", "GET", "v1/{+name}/locations", "", "extraLocationTypes,filter,pageSize,pageToken", ""},
// spanner v1 revision 20261006
{"spanner.projects.instanceConfigOperations.list", "GET", "v1/{+parent}/instanceConfigOperations", "", "filter,pageSize,pageToken", ""},
{"spanner.projects.instanceConfigs.create", "POST", "v1/{+parent}/instanceConfigs", "instanceConfig,instanceConfigId,validateOnly", "", ""},
{"spanner.projects.instanceConfigs.delete", "DELETE", "v1/{+name}", "", "etag,validateOnly", ""},
{"spanner.projects.instanceConfigs.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instanceConfigs.list", "GET", "v1/{+parent}/instanceConfigs", "", "pageSize,pageToken", ""},
{"spanner.projects.instanceConfigs.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"spanner.projects.instanceConfigs.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instanceConfigs.operations.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instanceConfigs.operations.list", "GET", "v1/{+name}", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"spanner.projects.instanceConfigs.patch", "PATCH", "v1/{+name}", "instanceConfig,updateMask,validateOnly", "", ""},
{"spanner.projects.instanceConfigs.ssdCaches.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"spanner.projects.instanceConfigs.ssdCaches.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instanceConfigs.ssdCaches.operations.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instanceConfigs.ssdCaches.operations.list", "GET", "v1/{+name}", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"spanner.projects.instances.backupOperations.list", "GET", "v1/{+parent}/backupOperations", "", "filter,pageSize,pageToken", ""},
{"spanner.projects.instances.backups.copy", "POST", "v1/{+parent}/backups:copy", "backupId,encryptionConfig,expireTime,sourceBackup", "", ""},
{"spanner.projects.instances.backups.create", "POST", "v1/{+parent}/backups", "backupSchedules,createTime,database,databaseDialect,encryptionInfo,encryptionInformation,exclusiveSizeBytes,expireTime,freeableSizeBytes,incrementalBackupChainId,instancePartitions,maxExpireTime,minimumRestorableEdition,name,oldestVersionTime,referencingBackups,referencingDatabases,sizeBytes,state,versionTime", "backupId,encryptionConfig.encryptionType,encryptionConfig.kmsKeyName,encryptionConfig.kmsKeyNames", ""},
{"spanner.projects.instances.backups.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.backups.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.backups.getIamPolicy", "POST", "v1/{+resource}:getIamPolicy", "options", "", ""},
{"spanner.projects.instances.backups.list", "GET", "v1/{+parent}/backups", "", "filter,pageSize,pageToken", ""},
{"spanner.projects.instances.backups.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"spanner.projects.instances.backups.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.backups.operations.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.backups.operations.list", "GET", "v1/{+name}", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"spanner.projects.instances.backups.patch", "PATCH", "v1/{+name}", "backupSchedules,createTime,database,databaseDialect,encryptionInfo,encryptionInformation,exclusiveSizeBytes,expireTime,freeableSizeBytes,incrementalBackupChainId,instancePartitions,maxExpireTime,minimumRestorableEdition,name,oldestVersionTime,referencingBackups,referencingDatabases,sizeBytes,state,versionTime", "updateMask", ""},
{"spanner.projects.instances.backups.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"spanner.projects.instances.backups.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"spanner.projects.instances.create", "POST", "v1/{+parent}/instances", "instance,instanceId", "", ""},
{"spanner.projects.instances.databaseOperations.list", "GET", "v1/{+parent}/databaseOperations", "", "filter,pageSize,pageToken", ""},
{"spanner.projects.instances.databases.addSplitPoints", "POST", "v1/{+database}:addSplitPoints", "initiator,splitPoints", "", ""},
{"spanner.projects.instances.databases.backupSchedules.create", "POST", "v1/{+parent}/backupSchedules", "encryptionConfig,fullBackupSpec,incrementalBackupSpec,name,retentionDuration,spec,updateTime", "backupScheduleId", ""},
{"spanner.projects.instances.databases.backupSchedules.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.databases.backupSchedules.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.databases.backupSchedules.getIamPolicy", "POST", "v1/{+resource}:getIamPolicy", "options", "", ""},
{"spanner.projects.instances.databases.backupSchedules.list", "GET", "v1/{+parent}/backupSchedules", "", "pageSize,pageToken", ""},
{"spanner.projects.instances.databases.backupSchedules.patch", "PATCH", "v1/{+name}", "encryptionConfig,fullBackupSpec,incrementalBackupSpec,name,retentionDuration,spec,updateTime", "updateMask", ""},
{"spanner.projects.instances.databases.backupSchedules.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"spanner.projects.instances.databases.backupSchedules.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"spanner.projects.instances.databases.changequorum", "POST", "v1/{+name}:changequorum", "etag,name,quorumType", "", ""},
{"spanner.projects.instances.databases.create", "POST", "v1/{+parent}/databases", "createStatement,databaseDialect,encryptionConfig,extraStatements,protoDescriptors", "", ""},
{"spanner.projects.instances.databases.databaseRoles.list", "GET", "v1/{+parent}/databaseRoles", "", "pageSize,pageToken", ""},
{"spanner.projects.instances.databases.databaseRoles.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"spanner.projects.instances.databases.dropDatabase", "DELETE", "v1/{+database}", "", "", ""},
{"spanner.projects.instances.databases.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.databases.getDdl", "GET", "v1/{+database}/ddl", "", "", ""},
{"spanner.projects.instances.databases.getIamPolicy", "POST", "v1/{+resource}:getIamPolicy", "options", "", ""},
{"spanner.projects.instances.databases.getScans", "GET", "v1/{+name}/scans", "", "endTime,startTime,view", ""},
{"spanner.projects.instances.databases.list", "GET", "v1/{+parent}/databases", "", "pageSize,pageToken", ""},
{"spanner.projects.instances.databases.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"spanner.projects.instances.databases.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.databases.operations.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.databases.operations.list", "GET", "v1/{+name}", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"spanner.projects.instances.databases.patch", "PATCH", "v1/{+name}", "createTime,databaseDialect,defaultLeader,earliestVersionTime,enableDropProtection,encryptionConfig,encryptionInfo,name,quorumInfo,reconciling,restoreInfo,state,versionRetentionPeriod", "updateMask", ""},
{"spanner.projects.instances.databases.restore", "POST", "v1/{+parent}/databases:restore", "backup,databaseId,encryptionConfig", "", ""},
{"spanner.projects.instances.databases.sessions.adaptMessage", "POST", "v1/{+name}:adaptMessage", "attachments,payload,protocol", "", ""},
{"spanner.projects.instances.databases.sessions.adapter", "POST", "v1/{+parent}/sessions:adapter", "name", "", ""},
{"spanner.projects.instances.databases.sessions.batchCreate", "POST", "v1/{+database}/sessions:batchCreate", "sessionCount,sessionTemplate", "", ""},
{"spanner.projects.instances.databases.sessions.batchWrite", "POST", "v1/{+session}:batchWrite", "excludeTxnFromChangeStreams,mutationGroups,requestOptions", "", ""},
{"spanner.projects.instances.databases.sessions.beginTransaction", "POST", "v1/{+session}:beginTransaction", "mutationKey,options,requestOptions", "", ""},
{"spanner.projects.instances.databases.sessions.commit", "POST", "v1/{+session}:commit", "maxCommitDelay,mutations,precommitToken,requestOptions,returnCommitStats,singleUseTransaction,transactionId", "", ""},
{"spanner.projects.instances.databases.sessions.create", "POST", "v1/{+database}/sessions", "session", "", ""},
{"spanner.projects.instances.databases.sessions.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.databases.sessions.executeBatchDml", "POST", "v1/{+session}:executeBatchDml", "lastStatements,requestOptions,seqno,statements,transaction", "", ""},
{"spanner.projects.instances.databases.sessions.executeSql", "POST", "v1/{+session}:executeSql", "dataBoostEnabled,directedReadOptions,lastStatement,paramTypes,params,partitionToken,queryMode,queryOptions,requestOptions,resumeToken,seqno,sql,transaction", "", ""},
{"spanner.projects.instances.databases.sessions.executeStreamingSql", "POST", "v1/{+session}:executeStreamingSql", "dataBoostEnabled,directedReadOptions,lastStatement,paramTypes,params,partitionToken,queryMode,queryOptions,requestOptions,resumeToken,seqno,sql,transaction", "", ""},
{"spanner.projects.instances.databases.sessions.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.databases.sessions.list", "GET", "v1/{+database}/sessions", "", "filter,pageSize,pageToken", ""},
{"spanner.projects.instances.databases.sessions.partitionQuery", "POST", "v1/{+session}:partitionQuery", "paramTypes,params,partitionOptions,sql,transaction", "", ""},
{"spanner.projects.instances.databases.sessions.partitionRead", "POST", "v1/{+session}:partitionRead", "columns,index,keySet,partitionOptions,table,transaction", "", ""},
{"spanner.projects.instances.databases.sessions.read", "POST", "v1/{+session}:read", "columns,dataBoostEnabled,directedReadOptions,index,keySet,limit,lockHint,orderBy,partitionToken,requestOptions,resumeToken,table,transaction", "", ""},
{"spanner.projects.instances.databases.sessions.rollback", "POST", "v1/{+session}:rollback", "transactionId", "", ""},
{"spanner.projects.instances.databases.sessions.streamingRead", "POST", "v1/{+session}:streamingRead", "columns,dataBoostEnabled,directedReadOptions,index,keySet,limit,lockHint,orderBy,partitionToken,requestOptions,resumeToken,table,transaction", "", ""},
{"spanner.projects.instances.databases.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"spanner.projects.instances.databases.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"spanner.projects.instances.databases.updateDdl", "PATCH", "v1/{+database}/ddl", "operationId,protoDescriptors,statements", "", ""},
{"spanner.projects.instances.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.get", "GET", "v1/{+name}", "", "fieldMask", ""},
{"spanner.projects.instances.getIamPolicy", "POST", "v1/{+resource}:getIamPolicy", "options", "", ""},
{"spanner.projects.instances.instancePartitionOperations.list", "GET", "v1/{+parent}/instancePartitionOperations", "", "filter,instancePartitionDeadline,pageSize,pageToken", ""},
{"spanner.projects.instances.instancePartitions.create", "POST", "v1/{+parent}/instancePartitions", "instancePartition,instancePartitionId", "", ""},
{"spanner.projects.instances.instancePartitions.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"spanner.projects.instances.instancePartitions.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.instancePartitions.list", "GET", "v1/{+parent}/instancePartitions", "", "instancePartitionDeadline,pageSize,pageToken", ""},
{"spanner.projects.instances.instancePartitions.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"spanner.projects.instances.instancePartitions.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.instancePartitions.operations.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.instancePartitions.operations.list", "GET", "v1/{+name}", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"spanner.projects.instances.instancePartitions.patch", "PATCH", "v1/{+name}", "fieldMask,instancePartition", "", ""},
{"spanner.projects.instances.list", "GET", "v1/{+parent}/instances", "", "filter,instanceDeadline,pageSize,pageToken", ""},
{"spanner.projects.instances.move", "POST", "v1/{+name}:move", "targetConfig,targetDatabaseMoveConfigs", "", ""},
{"spanner.projects.instances.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"spanner.projects.instances.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.operations.get", "GET", "v1/{+name}", "", "", ""},
{"spanner.projects.instances.operations.list", "GET", "v1/{+name}", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"spanner.projects.instances.patch", "PATCH", "v1/{+name}", "fieldMask,instance", "", ""},
{"spanner.projects.instances.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy", "", ""},
{"spanner.projects.instances.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"spanner.scans.list", "GET", "v1/{+parent}", "", "filter,pageSize,pageToken,view", ""},
// bigtableadmin v2 revision 20261017
{"bigtableadmin.operations.get", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.operations.projects.operations.list", "GET", "v2/{+name}/operations", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"bigtableadmin.projects.instances.appProfiles.create", "POST", "v2/{+parent}/appProfiles", "dataBoostIsolationReadOnly,description,etag,multiClusterRoutingUseAny,name,priority,singleClusterRouting,standardIsolation", "appProfileId,ignoreWarnings", ""},
{"bigtableadmin.projects.instances.appProfiles.delete", "DELETE", "v2/{+name}", "", "ignoreWarnings", ""},
{"bigtableadmin.projects.instances.appProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.appProfiles.list", "GET", "v2/{+parent}/appProfiles", "", "pageSize,pageToken", ""},
{"bigtableadmin.projects.instances.appProfiles.patch", "PATCH", "v2/{+name}", "dataBoostIsolationReadOnly,description,etag,multiClusterRoutingUseAny,name,priority,singleClusterRouting,standardIsolation", "ignoreWarnings,updateMask", ""},
{"bigtableadmin.projects.instances.clusters.backups.copy", "POST", "v2/{+parent}/backups:copy", "backupId,expireTime,sourceBackup", "", ""},
{"bigtableadmin.projects.instances.clusters.backups.create", "POST", "v2/{+parent}/backups", "backupType,encryptionInfo,endTime,expireTime,hotToStandardTime,name,sizeBytes,sourceBackup,sourceTable,startTime,state", "backupId", ""},
{"bigtableadmin.projects.instances.clusters.backups.delete", "DELETE", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.clusters.backups.get", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.clusters.backups.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "options", "", ""},
{"bigtableadmin.projects.instances.clusters.backups.list", "GET", "v2/{+parent}/backups", "", "filter,orderBy,pageSize,pageToken", ""},
{"bigtableadmin.projects.instances.clusters.backups.patch", "PATCH", "v2/{+name}", "backupType,encryptionInfo,endTime,expireTime,hotToStandardTime,name,sizeBytes,sourceBackup,sourceTable,startTime,state", "updateMask", ""},
{"bigtableadmin.projects.instances.clusters.backups.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigtableadmin.projects.instances.clusters.backups.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "permissions", "", ""},
{"bigtableadmin.projects.instances.clusters.create", "POST", "v2/{+parent}/clusters", "clusterConfig,defaultStorageType,encryptionConfig,location,name,nodeScalingFactor,serveNodes,state", "clusterId", ""},
{"bigtableadmin.projects.instances.clusters.delete", "DELETE", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.clusters.get", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.clusters.getMemoryLayer", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.clusters.hotTablets.list", "GET", "v2/{+parent}/hotTablets", "", "endTime,pageSize,pageToken,startTime", ""},
{"bigtableadmin.projects.instances.clusters.list", "GET", "v2/{+parent}/clusters", "", "pageToken", ""},
{"bigtableadmin.projects.instances.clusters.memoryLayers.list", "GET", "v2/{+parent}/memoryLayers", "", "pageSize,pageToken", ""},
{"bigtableadmin.projects.instances.clusters.partialUpdateCluster", "PATCH", "v2/{+name}", "clusterConfig,defaultStorageType,encryptionConfig,location,name,nodeScalingFactor,serveNodes,state", "updateMask", ""},
{"bigtableadmin.projects.instances.clusters.update", "PUT", "v2/{+name}", "clusterConfig,defaultStorageType,encryptionConfig,location,name,nodeScalingFactor,serveNodes,state", "", ""},
{"bigtableadmin.projects.instances.clusters.updateMemoryLayer", "PATCH", "v2/{+name}", "etag,memoryConfig,name,state", "updateMask", ""},
{"bigtableadmin.projects.instances.create", "POST", "v2/{+parent}/instances", "clusters,instance,instanceId,parent", "", ""},
{"bigtableadmin.projects.instances.delete", "DELETE", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.get", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "options", "", ""},
{"bigtableadmin.projects.instances.list", "GET", "v2/{+parent}/instances", "", "pageToken", ""},
{"bigtableadmin.projects.instances.logicalViews.create", "POST", "v2/{+parent}/logicalViews", "deletionProtection,etag,name,query", "logicalViewId", ""},
{"bigtableadmin.projects.instances.logicalViews.delete", "DELETE", "v2/{+name}", "", "etag", ""},
{"bigtableadmin.projects.instances.logicalViews.get", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.logicalViews.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "options", "", ""},
{"bigtableadmin.projects.instances.logicalViews.list", "GET", "v2/{+parent}/logicalViews", "", "pageSize,pageToken", ""},
{"bigtableadmin.projects.instances.logicalViews.patch", "PATCH", "v2/{+name}", "deletionProtection,etag,name,query", "updateMask", ""},
{"bigtableadmin.projects.instances.logicalViews.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigtableadmin.projects.instances.logicalViews.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "permissions", "", ""},
{"bigtableadmin.projects.instances.materializedViews.create", "POST", "v2/{+parent}/materializedViews", "clusterStates,deletionProtection,etag,name,query", "ignoreWarnings,materializedViewId", ""},
{"bigtableadmin.projects.instances.materializedViews.delete", "DELETE", "v2/{+name}", "", "etag", ""},
{"bigtableadmin.projects.instances.materializedViews.get", "GET", "v2/{+name}", "", "view", ""},
{"bigtableadmin.projects.instances.materializedViews.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "options", "", ""},
{"bigtableadmin.projects.instances.materializedViews.list", "GET", "v2/{+parent}/materializedViews", "", "pageSize,pageToken,view", ""},
{"bigtableadmin.projects.instances.materializedViews.patch", "PATCH", "v2/{+name}", "clusterStates,deletionProtection,etag,name,query", "updateMask", ""},
{"bigtableadmin.projects.instances.materializedViews.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigtableadmin.projects.instances.materializedViews.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "permissions", "", ""},
{"bigtableadmin.projects.instances.partialUpdateInstance", "PATCH", "v2/{+name}", "createTime,displayName,edition,knowledgeCatalogRegion,labels,name,satisfiesPzi,satisfiesPzs,state,tags,type", "updateMask", ""},
{"bigtableadmin.projects.instances.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.create", "POST", "v2/{+parent}/authorizedViews", "deletionProtection,etag,name,subsetView", "authorizedViewId", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.delete", "DELETE", "v2/{+name}", "", "etag", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.get", "GET", "v2/{+name}", "", "view", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "options", "", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.list", "GET", "v2/{+parent}/authorizedViews", "", "pageSize,pageToken,view", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.patch", "PATCH", "v2/{+name}", "deletionProtection,etag,name,subsetView", "ignoreWarnings,updateMask", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigtableadmin.projects.instances.tables.authorizedViews.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "permissions", "", ""},
{"bigtableadmin.projects.instances.tables.checkConsistency", "POST", "v2/{+name}:checkConsistency", "consistencyToken,dataBoostReadLocalWrites,standardReadRemoteWrites", "", ""},
{"bigtableadmin.projects.instances.tables.create", "POST", "v2/{+parent}/tables", "initialSplits,table,tableId", "", ""},
{"bigtableadmin.projects.instances.tables.delete", "DELETE", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.tables.dropRowRange", "POST", "v2/{+name}:dropRowRange", "deleteAllDataFromTable,rowKeyPrefix", "", ""},
{"bigtableadmin.projects.instances.tables.generateConsistencyToken", "POST", "v2/{+name}:generateConsistencyToken", "", "", ""},
{"bigtableadmin.projects.instances.tables.get", "GET", "v2/{+name}", "", "view", ""},
{"bigtableadmin.projects.instances.tables.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "options", "", ""},
{"bigtableadmin.projects.instances.tables.list", "GET", "v2/{+parent}/tables", "", "pageSize,pageToken,view", ""},
{"bigtableadmin.projects.instances.tables.modifyColumnFamilies", "POST", "v2/{+name}:modifyColumnFamilies", "ignoreWarnings,modifications", "", ""},
{"bigtableadmin.projects.instances.tables.patch", "PATCH", "v2/{+name}", "automatedBackupPolicy,changeStreamConfig,clusterStates,columnFamilies,deletionProtection,effectiveAutomatedBackupPolicy,granularity,name,restoreInfo,rowKeySchema,stats,tieredStorageConfig", "ignoreWarnings,updateMask", ""},
{"bigtableadmin.projects.instances.tables.restore", "POST", "v2/{+parent}/tables:restore", "backup,tableId", "", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.create", "POST", "v2/{+parent}/schemaBundles", "avroSchema,etag,name,protoSchema", "schemaBundleId", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.delete", "DELETE", "v2/{+name}", "", "etag", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.get", "GET", "v2/{+name}", "", "", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.getIamPolicy", "POST", "v2/{+resource}:getIamPolicy", "options", "", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.list", "GET", "v2/{+parent}/schemaBundles", "", "pageSize,pageToken,view", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.patch", "PATCH", "v2/{+name}", "avroSchema,etag,name,protoSchema", "ignoreWarnings,updateMask", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigtableadmin.projects.instances.tables.schemaBundles.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "permissions", "", ""},
{"bigtableadmin.projects.instances.tables.setIamPolicy", "POST", "v2/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"bigtableadmin.projects.instances.tables.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "permissions", "", ""},
{"bigtableadmin.projects.instances.tables.undelete", "POST", "v2/{+name}:undelete", "", "", ""},
{"bigtableadmin.projects.instances.testIamPermissions", "POST", "v2/{+resource}:testIamPermissions", "permissions", "", ""},
{"bigtableadmin.projects.instances.update", "PUT", "v2/{+name}", "createTime,displayName,edition,knowledgeCatalogRegion,labels,name,satisfiesPzi,satisfiesPzs,state,tags,type", "", ""},
{"bigtableadmin.projects.locations.list", "GET", "v2/{+name}/locations", "", "extraLocationTypes,filter,pageSize,pageToken", ""},
// secretmanager v1 revision 20260925
{"secretmanager.projects.locations.get", "GET", "v1/{+name}", "", "", ""},
{"secretmanager.projects.locations.list", "GET", "v1/{+name}/locations", "", "extraLocationTypes,filter,pageSize,pageToken", ""},
{"secretmanager.projects.locations.secrets.addVersion", "POST", "v1/{+parent}:addVersion", "payload", "", ""},
{"secretmanager.projects.locations.secrets.create", "POST", "v1/{+parent}/secrets", "annotations,createTime,customerManagedEncryption,etag,expireTime,labels,name,policyMember,replication,rotation,secretType,tags,topics,ttl,versionAliases,versionDestroyTtl", "secretId", ""},
{"secretmanager.projects.locations.secrets.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"secretmanager.projects.locations.secrets.enableManagedRotation", "POST", "v1/{+parent}:enableManagedRotation", "cloudSqlSingleUserCredentials", "", ""},
{"secretmanager.projects.locations.secrets.get", "GET", "v1/{+name}", "", "", ""},
{"secretmanager.projects.locations.secrets.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"secretmanager.projects.locations.secrets.list", "GET", "v1/{+parent}/secrets", "", "filter,pageSize,pageToken", ""},
{"secretmanager.projects.locations.secrets.patch", "PATCH", "v1/{+name}", "annotations,createTime,customerManagedEncryption,etag,expireTime,labels,name,policyMember,replication,rotation,secretType,tags,topics,ttl,versionAliases,versionDestroyTtl", "updateMask", ""},
{"secretmanager.projects.locations.secrets.rotateSecret", "POST", "v1/{+parent}:rotateSecret", "", "", ""},
{"secretmanager.projects.locations.secrets.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"secretmanager.projects.locations.secrets.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"secretmanager.projects.locations.secrets.versions.access", "GET", "v1/{+name}:access", "", "", ""},
{"secretmanager.projects.locations.secrets.versions.destroy", "POST", "v1/{+name}:destroy", "etag", "", ""},
{"secretmanager.projects.locations.secrets.versions.disable", "POST", "v1/{+name}:disable", "etag", "", ""},
{"secretmanager.projects.locations.secrets.versions.enable", "POST", "v1/{+name}:enable", "etag", "", ""},
{"secretmanager.projects.locations.secrets.versions.get", "GET", "v1/{+name}", "", "", ""},
{"secretmanager.projects.locations.secrets.versions.list", "GET", "v1/{+parent}/versions", "", "filter,pageSize,pageToken", ""},
{"secretmanager.projects.secrets.addVersion", "POST", "v1/{+parent}:addVersion", "payload", "", ""},
{"secretmanager.projects.secrets.create", "POST", "v1/{+parent}/secrets", "annotations,createTime,customerManagedEncryption,etag,expireTime,labels,name,policyMember,replication,rotation,secretType,tags,topics,ttl,versionAliases,versionDestroyTtl", "secretId", ""},
{"secretmanager.projects.secrets.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"secretmanager.projects.secrets.enableManagedRotation", "POST", "v1/{+parent}:enableManagedRotation", "cloudSqlSingleUserCredentials", "", ""},
{"secretmanager.projects.secrets.get", "GET", "v1/{+name}", "", "", ""},
{"secretmanager.projects.secrets.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"secretmanager.projects.secrets.list", "GET", "v1/{+parent}/secrets", "", "filter,pageSize,pageToken", ""},
{"secretmanager.projects.secrets.patch", "PATCH", "v1/{+name}", "annotations,createTime,customerManagedEncryption,etag,expireTime,labels,name,policyMember,replication,rotation,secretType,tags,topics,ttl,versionAliases,versionDestroyTtl", "updateMask", ""},
{"secretmanager.projects.secrets.rotateSecret", "POST", "v1/{+parent}:rotateSecret", "", "", ""},
{"secretmanager.projects.secrets.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"secretmanager.projects.secrets.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"secretmanager.projects.secrets.versions.access", "GET", "v1/{+name}:access", "", "", ""},
{"secretmanager.projects.secrets.versions.destroy", "POST", "v1/{+name}:destroy", "etag", "", ""},
{"secretmanager.projects.secrets.versions.disable", "POST", "v1/{+name}:disable", "etag", "", ""},
{"secretmanager.projects.secrets.versions.enable", "POST", "v1/{+name}:enable", "etag", "", ""},
{"secretmanager.projects.secrets.versions.get", "GET", "v1/{+name}", "", "", ""},
{"secretmanager.projects.secrets.versions.list", "GET", "v1/{+parent}/versions", "", "filter,pageSize,pageToken", ""},
// dataplex v1 revision 20261013
{"dataplex.organizations.locations.encryptionConfigs.create", "POST", "v1/{+parent}/encryptionConfigs", "createTime,enableMetastoreEncryption,encryptionState,etag,failureDetails,key,name,updateTime", "encryptionConfigId", ""},
{"dataplex.organizations.locations.encryptionConfigs.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.organizations.locations.encryptionConfigs.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.organizations.locations.encryptionConfigs.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.organizations.locations.encryptionConfigs.list", "GET", "v1/{+parent}/encryptionConfigs", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.organizations.locations.encryptionConfigs.patch", "PATCH", "v1/{+name}", "createTime,enableMetastoreEncryption,encryptionState,etag,failureDetails,key,name,updateTime", "updateMask", ""},
{"dataplex.organizations.locations.encryptionConfigs.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.organizations.locations.encryptionConfigs.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.organizations.locations.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"dataplex.organizations.locations.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.organizations.locations.operations.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.organizations.locations.operations.list", "GET", "v1/{+name}/operations", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"dataplex.projects.locations.agentTasks.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.agentTasks.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.agentTasks.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.aspectTypes.create", "POST", "v1/{+parent}/aspectTypes", "authorization,createTime,dataClassification,description,displayName,etag,labels,metadataTemplate,name,transferStatus,uid,updateTime", "aspectTypeId,validateOnly", ""},
{"dataplex.projects.locations.aspectTypes.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.projects.locations.aspectTypes.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.aspectTypes.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.aspectTypes.list", "GET", "v1/{+parent}/aspectTypes", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.aspectTypes.patch", "PATCH", "v1/{+name}", "authorization,createTime,dataClassification,description,displayName,etag,labels,metadataTemplate,name,transferStatus,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.aspectTypes.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.aspectTypes.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.changeRequests.approve", "POST", "v1/{+name}:approve", "comment,etag", "", ""},
{"dataplex.projects.locations.changeRequests.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.projects.locations.changeRequests.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.changeRequests.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.changeRequests.list", "GET", "v1/{+parent}/changeRequests", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.changeRequests.patch", "PATCH", "v1/{+name}", "approver,author,batchChangeMetadataRequest,batchChangeMetadataResponse,changeType,createEntry,createEntryLink,createGlossary,createGlossaryCategory,createGlossaryTerm,createTime,dataProductAccessRequest,deleteEntry,deleteEntryLink,deleteGlossary,deleteGlossaryCategory,deleteGlossaryTerm,etag,justification,labels,name,rejectionComment,resource,reviewerComment,state,uid,updateEntry,updateGlossary,updateGlossaryCategory,updateGlossaryTerm,updateTime", "updateMask", ""},
{"dataplex.projects.locations.changeRequests.reject", "POST", "v1/{+name}:reject", "comment,etag", "", ""},
{"dataplex.projects.locations.changeRequests.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.changeRequests.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.dataAttributeBindings.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.dataAttributeBindings.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.dataAttributeBindings.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.dataDomains.bindings.create", "POST", "v1/{+parent}/bindings", "createTime,name,resource,uid", "dataDomainBindingId,validateOnly", ""},
{"dataplex.projects.locations.dataDomains.bindings.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.dataDomains.bindings.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.dataDomains.bindings.list", "GET", "v1/{+parent}/bindings", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.dataDomains.create", "POST", "v1/{+parent}/dataDomains", "contacts,createTime,description,displayName,labels,name,parentDataDomain,policyMember,uid,updateTime", "dataDomainId,validateOnly", ""},
{"dataplex.projects.locations.dataDomains.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.dataDomains.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.dataDomains.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.dataDomains.list", "GET", "v1/{+parent}/dataDomains", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.dataDomains.patch", "PATCH", "v1/{+name}", "contacts,createTime,description,displayName,labels,name,parentDataDomain,policyMember,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.dataDomains.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.dataDomains.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.dataProducts.create", "POST", "v1/{+parent}/dataProducts", "accessApprovalConfig,accessGroups,assetCount,createTime,description,displayName,etag,icon,labels,name,ownerEmails,uid,updateTime", "dataProductId,validateOnly", ""},
{"dataplex.projects.locations.dataProducts.dataAssets.create", "POST", "v1/{+parent}/dataAssets", "accessGroupConfigs,createTime,etag,labels,name,resource,uid,updateTime", "dataAssetId,validateOnly", ""},
{"dataplex.projects.locations.dataProducts.dataAssets.delete", "DELETE", "v1/{+name}", "", "etag,validateOnly", ""},
{"dataplex.projects.locations.dataProducts.dataAssets.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.dataProducts.dataAssets.list", "GET", "v1/{+parent}/dataAssets", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.dataProducts.dataAssets.patch", "PATCH", "v1/{+name}", "accessGroupConfigs,createTime,etag,labels,name,resource,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.dataProducts.delete", "DELETE", "v1/{+name}", "", "etag,validateOnly", ""},
{"dataplex.projects.locations.dataProducts.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.dataProducts.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.dataProducts.list", "GET", "v1/{+parent}/dataProducts", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.dataProducts.patch", "PATCH", "v1/{+name}", "accessApprovalConfig,accessGroups,assetCount,createTime,description,displayName,etag,icon,labels,name,ownerEmails,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.dataProducts.requestAccess", "POST", "v1/{+parent}:requestAccess", "changeRequest,validateOnly", "", ""},
{"dataplex.projects.locations.dataProducts.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.dataProducts.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.dataScans.create", "POST", "v1/{+parent}/dataScans", "createTime,data,dataDiscoveryResult,dataDiscoverySpec,dataDocumentationResult,dataDocumentationSpec,dataProfileResult,dataProfileSpec,dataQualityResult,dataQualitySpec,description,displayName,executionIdentity,executionSpec,executionStatus,labels,name,state,type,uid,updateTime", "dataScanId,validateOnly", ""},
{"dataplex.projects.locations.dataScans.delete", "DELETE", "v1/{+name}", "", "force", ""},
{"dataplex.projects.locations.dataScans.generateDataQualityRules", "POST", "v1/{+name}:generateDataQualityRules", "", "", ""},
{"dataplex.projects.locations.dataScans.get", "GET", "v1/{+name}", "", "view", ""},
{"dataplex.projects.locations.dataScans.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.dataScans.jobs.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"dataplex.projects.locations.dataScans.jobs.generateDataQualityRules", "POST", "v1/{+name}:generateDataQualityRules", "", "", ""},
{"dataplex.projects.locations.dataScans.jobs.get", "GET", "v1/{+name}", "", "view", ""},
{"dataplex.projects.locations.dataScans.jobs.list", "GET", "v1/{+parent}/jobs", "", "filter,pageSize,pageToken", ""},
{"dataplex.projects.locations.dataScans.list", "GET", "v1/{+parent}/dataScans", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.dataScans.patch", "PATCH", "v1/{+name}", "createTime,data,dataDiscoveryResult,dataDiscoverySpec,dataDocumentationResult,dataDocumentationSpec,dataProfileResult,dataProfileSpec,dataQualityResult,dataQualitySpec,description,displayName,executionIdentity,executionSpec,executionStatus,labels,name,state,type,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.dataScans.run", "POST", "v1/{+name}:run", "", "", ""},
{"dataplex.projects.locations.dataScans.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.dataScans.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.dataTaxonomies.attributes.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.dataTaxonomies.attributes.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.dataTaxonomies.attributes.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.dataTaxonomies.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.dataTaxonomies.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.dataTaxonomies.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.entryGroups.batchChangeMetadata", "POST", "v1/{+name}:batchChangeMetadata", "changes,description,name", "", ""},
{"dataplex.projects.locations.entryGroups.create", "POST", "v1/{+parent}/entryGroups", "changeRequestConfig,createTime,description,displayName,etag,labels,name,transferStatus,uid,updateTime", "entryGroupId,validateOnly", ""},
{"dataplex.projects.locations.entryGroups.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.projects.locations.entryGroups.entries.create", "POST", "v1/{+parent}/entries", "aspects,createTime,entrySource,entryType,fullyQualifiedName,name,parentEntry,updateTime", "entryId", ""},
{"dataplex.projects.locations.entryGroups.entries.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.entryGroups.entries.get", "GET", "v1/{+name}", "", "aspectTypes,paths,view", ""},
{"dataplex.projects.locations.entryGroups.entries.list", "GET", "v1/{+parent}/entries", "", "filter,pageSize,pageToken", ""},
{"dataplex.projects.locations.entryGroups.entries.patch", "PATCH", "v1/{+name}", "aspects,createTime,entrySource,entryType,fullyQualifiedName,name,parentEntry,updateTime", "allowMissing,aspectKeys,deleteMissingAspects,updateMask", ""},
{"dataplex.projects.locations.entryGroups.entryLinks.create", "POST", "v1/{+parent}/entryLinks", "aspects,createTime,entryLinkType,entryReferences,name,updateTime", "entryLinkId", ""},
{"dataplex.projects.locations.entryGroups.entryLinks.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.entryGroups.entryLinks.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.entryGroups.entryLinks.patch", "PATCH", "v1/{+name}", "aspects,createTime,entryLinkType,entryReferences,name,updateTime", "allowMissing,aspectKeys", ""},
{"dataplex.projects.locations.entryGroups.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.entryGroups.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.entryGroups.list", "GET", "v1/{+parent}/entryGroups", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.entryGroups.patch", "PATCH", "v1/{+name}", "changeRequestConfig,createTime,description,displayName,etag,labels,name,transferStatus,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.entryGroups.requestBatchMetadataChange", "POST", "v1/{+name}:requestBatchMetadataChange", "changePayload,justification,labels", "", ""},
{"dataplex.projects.locations.entryGroups.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.entryGroups.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.entryLinkTypes.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.entryLinkTypes.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.entryLinkTypes.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.entryTypes.create", "POST", "v1/{+parent}/entryTypes", "authorization,createTime,description,displayName,etag,labels,name,platform,requiredAspects,system,typeAliases,uid,updateTime", "entryTypeId,validateOnly", ""},
{"dataplex.projects.locations.entryTypes.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.projects.locations.entryTypes.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.entryTypes.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.entryTypes.list", "GET", "v1/{+parent}/entryTypes", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.entryTypes.patch", "PATCH", "v1/{+name}", "authorization,createTime,description,displayName,etag,labels,name,platform,requiredAspects,system,typeAliases,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.entryTypes.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.entryTypes.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.glossaries.categories.create", "POST", "v1/{+parent}/categories", "createTime,description,displayName,labels,name,parent,uid,updateTime", "categoryId", ""},
{"dataplex.projects.locations.glossaries.categories.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.glossaries.categories.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.glossaries.categories.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.glossaries.categories.list", "GET", "v1/{+parent}/categories", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.glossaries.categories.patch", "PATCH", "v1/{+name}", "createTime,description,displayName,labels,name,parent,uid,updateTime", "updateMask", ""},
{"dataplex.projects.locations.glossaries.categories.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.glossaries.categories.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.glossaries.create", "POST", "v1/{+parent}/glossaries", "categoryCount,createTime,description,displayName,etag,labels,name,termCount,uid,updateTime", "glossaryId,validateOnly", ""},
{"dataplex.projects.locations.glossaries.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.projects.locations.glossaries.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.glossaries.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.glossaries.list", "GET", "v1/{+parent}/glossaries", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.glossaries.patch", "PATCH", "v1/{+name}", "categoryCount,createTime,description,displayName,etag,labels,name,termCount,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.glossaries.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.glossaries.terms.create", "POST", "v1/{+parent}/terms", "createTime,description,displayName,labels,name,parent,uid,updateTime", "termId", ""},
{"dataplex.projects.locations.glossaries.terms.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.glossaries.terms.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.glossaries.terms.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.glossaries.terms.list", "GET", "v1/{+parent}/terms", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.glossaries.terms.patch", "PATCH", "v1/{+name}", "createTime,description,displayName,labels,name,parent,uid,updateTime", "updateMask", ""},
{"dataplex.projects.locations.glossaries.terms.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.glossaries.terms.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.glossaries.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.governanceRules.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.governanceRules.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.governanceRules.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.lakes.actions.list", "GET", "v1/{+parent}/actions", "", "pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.create", "POST", "v1/{+parent}/lakes", "assetStatus,createTime,description,displayName,labels,metastore,metastoreStatus,name,serviceAccount,state,uid,updateTime", "lakeId,validateOnly", ""},
{"dataplex.projects.locations.lakes.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.lakes.list", "GET", "v1/{+parent}/lakes", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.patch", "PATCH", "v1/{+name}", "assetStatus,createTime,description,displayName,labels,metastore,metastoreStatus,name,serviceAccount,state,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.lakes.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.lakes.tasks.create", "POST", "v1/{+parent}/tasks", "createTime,description,displayName,executionSpec,executionStatus,labels,name,notebook,spark,state,triggerSpec,uid,updateTime", "taskId,validateOnly", ""},
{"dataplex.projects.locations.lakes.tasks.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.tasks.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.tasks.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.lakes.tasks.jobs.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"dataplex.projects.locations.lakes.tasks.jobs.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.tasks.jobs.list", "GET", "v1/{+parent}/jobs", "", "pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.tasks.list", "GET", "v1/{+parent}/tasks", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.tasks.patch", "PATCH", "v1/{+name}", "createTime,description,displayName,executionSpec,executionStatus,labels,name,notebook,spark,state,triggerSpec,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.lakes.tasks.run", "POST", "v1/{+name}:run", "args,labels", "", ""},
{"dataplex.projects.locations.lakes.tasks.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.lakes.tasks.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.lakes.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.lakes.zones.actions.list", "GET", "v1/{+parent}/actions", "", "pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.zones.assets.actions.list", "GET", "v1/{+parent}/actions", "", "pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.zones.assets.create", "POST", "v1/{+parent}/assets", "createTime,description,discoverySpec,discoveryStatus,displayName,labels,name,resourceSpec,resourceStatus,securityStatus,state,uid,updateTime", "assetId,validateOnly", ""},
{"dataplex.projects.locations.lakes.zones.assets.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.zones.assets.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.zones.assets.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.lakes.zones.assets.list", "GET", "v1/{+parent}/assets", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.zones.assets.patch", "PATCH", "v1/{+name}", "createTime,description,discoverySpec,discoveryStatus,displayName,labels,name,resourceSpec,resourceStatus,securityStatus,state,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.lakes.zones.assets.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.lakes.zones.assets.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.lakes.zones.create", "POST", "v1/{+parent}/zones", "assetStatus,createTime,description,discoverySpec,displayName,labels,name,resourceSpec,state,type,uid,updateTime", "validateOnly,zoneId", ""},
{"dataplex.projects.locations.lakes.zones.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.zones.entities.create", "POST", "v1/{+parent}/entities", "access,asset,catalogEntry,compatibility,createTime,dataPath,dataPathPattern,description,displayName,etag,format,id,name,schema,system,type,uid,updateTime", "validateOnly", ""},
{"dataplex.projects.locations.lakes.zones.entities.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.projects.locations.lakes.zones.entities.get", "GET", "v1/{+name}", "", "view", ""},
{"dataplex.projects.locations.lakes.zones.entities.list", "GET", "v1/{+parent}/entities", "", "filter,pageSize,pageToken,view", ""},
{"dataplex.projects.locations.lakes.zones.entities.partitions.create", "POST", "v1/{+parent}/partitions", "etag,location,name,values", "validateOnly", ""},
{"dataplex.projects.locations.lakes.zones.entities.partitions.delete", "DELETE", "v1/{+name}", "", "etag", ""},
{"dataplex.projects.locations.lakes.zones.entities.partitions.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.zones.entities.partitions.list", "GET", "v1/{+parent}/partitions", "", "filter,pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.zones.entities.update", "PUT", "v1/{+name}", "access,asset,catalogEntry,compatibility,createTime,dataPath,dataPathPattern,description,displayName,etag,format,id,name,schema,system,type,uid,updateTime", "validateOnly", ""},
{"dataplex.projects.locations.lakes.zones.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.lakes.zones.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"dataplex.projects.locations.lakes.zones.list", "GET", "v1/{+parent}/zones", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.lakes.zones.patch", "PATCH", "v1/{+name}", "assetStatus,createTime,description,discoverySpec,displayName,labels,name,resourceSpec,state,type,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.lakes.zones.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"dataplex.projects.locations.lakes.zones.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"dataplex.projects.locations.list", "GET", "v1/{+name}/locations", "", "extraLocationTypes,filter,pageSize,pageToken", ""},
{"dataplex.projects.locations.lookupContext", "POST", "v1/{+name}:lookupContext", "options,resources", "", ""},
{"dataplex.projects.locations.lookupEntry", "GET", "v1/{+name}:lookupEntry", "", "aspectTypes,entry,paths,view", ""},
{"dataplex.projects.locations.lookupEntryLinks", "GET", "v1/{+name}:lookupEntryLinks", "", "entry,entryLinkTypes,entryMode,pageSize,pageToken", ""},
{"dataplex.projects.locations.metadataFeeds.create", "POST", "v1/{+parent}/metadataFeeds", "createTime,filters,labels,name,pubsubTopic,scope,uid,updateTime", "metadataFeedId,validateOnly", ""},
{"dataplex.projects.locations.metadataFeeds.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.metadataFeeds.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.metadataFeeds.list", "GET", "v1/{+parent}/metadataFeeds", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.metadataFeeds.patch", "PATCH", "v1/{+name}", "createTime,filters,labels,name,pubsubTopic,scope,uid,updateTime", "updateMask,validateOnly", ""},
{"dataplex.projects.locations.metadataJobs.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"dataplex.projects.locations.metadataJobs.create", "POST", "v1/{+parent}/metadataJobs", "createTime,exportResult,exportSpec,importResult,importSpec,labels,name,status,type,uid,updateTime", "metadataJobId,validateOnly", ""},
{"dataplex.projects.locations.metadataJobs.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.metadataJobs.list", "GET", "v1/{+parent}/metadataJobs", "", "filter,orderBy,pageSize,pageToken", ""},
{"dataplex.projects.locations.modifyEntry", "POST", "v1/{+name}:modifyEntry", "aspectKeys,deleteMissingAspects,entry,updateMask", "", ""},
{"dataplex.projects.locations.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"dataplex.projects.locations.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.operations.get", "GET", "v1/{+name}", "", "", ""},
{"dataplex.projects.locations.operations.list", "GET", "v1/{+name}/operations", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"dataplex.projects.locations.searchEntries", "POST", "v1/{+name}:searchEntries", "", "contexts,orderBy,pageSize,pageToken,query,scope,semanticSearch", ""},
// dlp v2 revision 20261020
{"dlp.infoTypes.list", "GET", "v2/infoTypes", "", "filter,languageCode,locationId,parent", ""},
{"dlp.locations.infoTypes.list", "GET", "v2/{+parent}/infoTypes", "", "filter,languageCode,locationId", ""},
{"dlp.organizations.deidentifyTemplates.create", "POST", "v2/{+parent}/deidentifyTemplates", "deidentifyTemplate,locationId,templateId", "", ""},
{"dlp.organizations.deidentifyTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.deidentifyTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.deidentifyTemplates.list", "GET", "v2/{+parent}/deidentifyTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.deidentifyTemplates.patch", "PATCH", "v2/{+name}", "deidentifyTemplate,updateMask", "", ""},
{"dlp.organizations.inspectTemplates.create", "POST", "v2/{+parent}/inspectTemplates", "inspectTemplate,locationId,templateId", "", ""},
{"dlp.organizations.inspectTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.inspectTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.inspectTemplates.list", "GET", "v2/{+parent}/inspectTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.inspectTemplates.patch", "PATCH", "v2/{+name}", "inspectTemplate,updateMask", "", ""},
{"dlp.organizations.locations.columnDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.columnDataProfiles.list", "GET", "v2/{+parent}/columnDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.locations.connections.create", "POST", "v2/{+parent}/connections", "connection", "", ""},
{"dlp.organizations.locations.connections.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.connections.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.connections.list", "GET", "v2/{+parent}/connections", "", "filter,pageSize,pageToken", ""},
{"dlp.organizations.locations.connections.patch", "PATCH", "v2/{+name}", "connection,updateMask", "", ""},
{"dlp.organizations.locations.connections.search", "GET", "v2/{+parent}/connections:search", "", "filter,pageSize,pageToken", ""},
{"dlp.organizations.locations.deidentifyTemplates.create", "POST", "v2/{+parent}/deidentifyTemplates", "deidentifyTemplate,locationId,templateId", "", ""},
{"dlp.organizations.locations.deidentifyTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.deidentifyTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.deidentifyTemplates.list", "GET", "v2/{+parent}/deidentifyTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.locations.deidentifyTemplates.patch", "PATCH", "v2/{+name}", "deidentifyTemplate,updateMask", "", ""},
{"dlp.organizations.locations.discoveryConfigs.create", "POST", "v2/{+parent}/discoveryConfigs", "configId,discoveryConfig", "", ""},
{"dlp.organizations.locations.discoveryConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.discoveryConfigs.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.discoveryConfigs.list", "GET", "v2/{+parent}/discoveryConfigs", "", "orderBy,pageSize,pageToken", ""},
{"dlp.organizations.locations.discoveryConfigs.patch", "PATCH", "v2/{+name}", "discoveryConfig,updateMask", "", ""},
{"dlp.organizations.locations.dlpJobs.list", "GET", "v2/{+parent}/dlpJobs", "", "filter,locationId,orderBy,pageSize,pageToken,type", ""},
{"dlp.organizations.locations.fileStoreDataProfiles.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.fileStoreDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.fileStoreDataProfiles.list", "GET", "v2/{+parent}/fileStoreDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.locations.infoTypes.list", "GET", "v2/{+parent}/infoTypes", "", "filter,languageCode,locationId", ""},
{"dlp.organizations.locations.inspectTemplates.create", "POST", "v2/{+parent}/inspectTemplates", "inspectTemplate,locationId,templateId", "", ""},
{"dlp.organizations.locations.inspectTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.inspectTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.inspectTemplates.list", "GET", "v2/{+parent}/inspectTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.locations.inspectTemplates.patch", "PATCH", "v2/{+name}", "inspectTemplate,updateMask", "", ""},
{"dlp.organizations.locations.jobTriggers.create", "POST", "v2/{+parent}/jobTriggers", "jobTrigger,locationId,triggerId", "", ""},
{"dlp.organizations.locations.jobTriggers.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.jobTriggers.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.jobTriggers.list", "GET", "v2/{+parent}/jobTriggers", "", "filter,locationId,orderBy,pageSize,pageToken,type", ""},
{"dlp.organizations.locations.jobTriggers.patch", "PATCH", "v2/{+name}", "jobTrigger,updateMask", "", ""},
{"dlp.organizations.locations.projectDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.projectDataProfiles.list", "GET", "v2/{+parent}/projectDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.locations.storedInfoTypes.create", "POST", "v2/{+parent}/storedInfoTypes", "config,locationId,storedInfoTypeId", "", ""},
{"dlp.organizations.locations.storedInfoTypes.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.storedInfoTypes.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.storedInfoTypes.list", "GET", "v2/{+parent}/storedInfoTypes", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.locations.storedInfoTypes.patch", "PATCH", "v2/{+name}", "config,updateMask", "", ""},
{"dlp.organizations.locations.tableDataProfiles.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.tableDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.locations.tableDataProfiles.list", "GET", "v2/{+parent}/tableDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.storedInfoTypes.create", "POST", "v2/{+parent}/storedInfoTypes", "config,locationId,storedInfoTypeId", "", ""},
{"dlp.organizations.storedInfoTypes.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.organizations.storedInfoTypes.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.organizations.storedInfoTypes.list", "GET", "v2/{+parent}/storedInfoTypes", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.organizations.storedInfoTypes.patch", "PATCH", "v2/{+name}", "config,updateMask", "", ""},
{"dlp.projects.content.deidentify", "POST", "v2/{+parent}/content:deidentify", "deidentifyConfig,deidentifyTemplateName,inspectConfig,inspectTemplateName,item,locationId", "", ""},
{"dlp.projects.content.inspect", "POST", "v2/{+parent}/content:inspect", "inspectConfig,inspectTemplateName,item,locationId", "", ""},
{"dlp.projects.content.reidentify", "POST", "v2/{+parent}/content:reidentify", "inspectConfig,inspectTemplateName,item,locationId,reidentifyConfig,reidentifyTemplateName", "", ""},
{"dlp.projects.deidentifyTemplates.create", "POST", "v2/{+parent}/deidentifyTemplates", "deidentifyTemplate,locationId,templateId", "", ""},
{"dlp.projects.deidentifyTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.deidentifyTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.deidentifyTemplates.list", "GET", "v2/{+parent}/deidentifyTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.projects.deidentifyTemplates.patch", "PATCH", "v2/{+name}", "deidentifyTemplate,updateMask", "", ""},
{"dlp.projects.dlpJobs.cancel", "POST", "v2/{+name}:cancel", "", "", ""},
{"dlp.projects.dlpJobs.create", "POST", "v2/{+parent}/dlpJobs", "inspectJob,jobId,locationId,riskJob", "", ""},
{"dlp.projects.dlpJobs.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.dlpJobs.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.dlpJobs.list", "GET", "v2/{+parent}/dlpJobs", "", "filter,locationId,orderBy,pageSize,pageToken,type", ""},
{"dlp.projects.image.redact", "POST", "v2/{+parent}/image:redact", "byteItem,deidentifyTemplate,imageRedactionConfigs,includeFindings,inspectConfig,inspectTemplate,locationId", "", ""},
{"dlp.projects.inspectTemplates.create", "POST", "v2/{+parent}/inspectTemplates", "inspectTemplate,locationId,templateId", "", ""},
{"dlp.projects.inspectTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.inspectTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.inspectTemplates.list", "GET", "v2/{+parent}/inspectTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.projects.inspectTemplates.patch", "PATCH", "v2/{+name}", "inspectTemplate,updateMask", "", ""},
{"dlp.projects.jobTriggers.activate", "POST", "v2/{+name}:activate", "", "", ""},
{"dlp.projects.jobTriggers.create", "POST", "v2/{+parent}/jobTriggers", "jobTrigger,locationId,triggerId", "", ""},
{"dlp.projects.jobTriggers.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.jobTriggers.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.jobTriggers.list", "GET", "v2/{+parent}/jobTriggers", "", "filter,locationId,orderBy,pageSize,pageToken,type", ""},
{"dlp.projects.jobTriggers.patch", "PATCH", "v2/{+name}", "jobTrigger,updateMask", "", ""},
{"dlp.projects.locations.columnDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.columnDataProfiles.list", "GET", "v2/{+parent}/columnDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.projects.locations.connections.create", "POST", "v2/{+parent}/connections", "connection", "", ""},
{"dlp.projects.locations.connections.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.connections.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.connections.list", "GET", "v2/{+parent}/connections", "", "filter,pageSize,pageToken", ""},
{"dlp.projects.locations.connections.patch", "PATCH", "v2/{+name}", "connection,updateMask", "", ""},
{"dlp.projects.locations.connections.search", "GET", "v2/{+parent}/connections:search", "", "filter,pageSize,pageToken", ""},
{"dlp.projects.locations.content.deidentify", "POST", "v2/{+parent}/content:deidentify", "deidentifyConfig,deidentifyTemplateName,inspectConfig,inspectTemplateName,item,locationId", "", ""},
{"dlp.projects.locations.content.inspect", "POST", "v2/{+parent}/content:inspect", "inspectConfig,inspectTemplateName,item,locationId", "", ""},
{"dlp.projects.locations.content.reidentify", "POST", "v2/{+parent}/content:reidentify", "inspectConfig,inspectTemplateName,item,locationId,reidentifyConfig,reidentifyTemplateName", "", ""},
{"dlp.projects.locations.contentPolicies.create", "POST", "v2/{+parent}/contentPolicies", "contentPolicy,contentPolicyId", "", ""},
{"dlp.projects.locations.contentPolicies.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.contentPolicies.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.contentPolicies.list", "GET", "v2/{+parent}/contentPolicies", "", "pageSize,pageToken", ""},
{"dlp.projects.locations.contentPolicies.patch", "PATCH", "v2/{+name}", "contentPolicy,updateMask", "", ""},
{"dlp.projects.locations.deidentifyTemplates.create", "POST", "v2/{+parent}/deidentifyTemplates", "deidentifyTemplate,locationId,templateId", "", ""},
{"dlp.projects.locations.deidentifyTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.deidentifyTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.deidentifyTemplates.list", "GET", "v2/{+parent}/deidentifyTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.projects.locations.deidentifyTemplates.patch", "PATCH", "v2/{+name}", "deidentifyTemplate,updateMask", "", ""},
{"dlp.projects.locations.discoveryConfigs.create", "POST", "v2/{+parent}/discoveryConfigs", "configId,discoveryConfig", "", ""},
{"dlp.projects.locations.discoveryConfigs.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.discoveryConfigs.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.discoveryConfigs.list", "GET", "v2/{+parent}/discoveryConfigs", "", "orderBy,pageSize,pageToken", ""},
{"dlp.projects.locations.discoveryConfigs.patch", "PATCH", "v2/{+name}", "discoveryConfig,updateMask", "", ""},
{"dlp.projects.locations.dlpJobs.cancel", "POST", "v2/{+name}:cancel", "", "", ""},
{"dlp.projects.locations.dlpJobs.create", "POST", "v2/{+parent}/dlpJobs", "inspectJob,jobId,locationId,riskJob", "", ""},
{"dlp.projects.locations.dlpJobs.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.dlpJobs.finish", "POST", "v2/{+name}:finish", "", "", ""},
{"dlp.projects.locations.dlpJobs.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.dlpJobs.hybridInspect", "POST", "v2/{+name}:hybridInspect", "hybridItem", "", ""},
{"dlp.projects.locations.dlpJobs.list", "GET", "v2/{+parent}/dlpJobs", "", "filter,locationId,orderBy,pageSize,pageToken,type", ""},
{"dlp.projects.locations.fileStoreDataProfiles.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.fileStoreDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.fileStoreDataProfiles.list", "GET", "v2/{+parent}/fileStoreDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.projects.locations.image.redact", "POST", "v2/{+parent}/image:redact", "byteItem,deidentifyTemplate,imageRedactionConfigs,includeFindings,inspectConfig,inspectTemplate,locationId", "", ""},
{"dlp.projects.locations.infoTypes.list", "GET", "v2/{+parent}/infoTypes", "", "filter,languageCode,locationId", ""},
{"dlp.projects.locations.inspectTemplates.create", "POST", "v2/{+parent}/inspectTemplates", "inspectTemplate,locationId,templateId", "", ""},
{"dlp.projects.locations.inspectTemplates.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.inspectTemplates.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.inspectTemplates.list", "GET", "v2/{+parent}/inspectTemplates", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.projects.locations.inspectTemplates.patch", "PATCH", "v2/{+name}", "inspectTemplate,updateMask", "", ""},
{"dlp.projects.locations.jobTriggers.activate", "POST", "v2/{+name}:activate", "", "", ""},
{"dlp.projects.locations.jobTriggers.create", "POST", "v2/{+parent}/jobTriggers", "jobTrigger,locationId,triggerId", "", ""},
{"dlp.projects.locations.jobTriggers.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.jobTriggers.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.jobTriggers.hybridInspect", "POST", "v2/{+name}:hybridInspect", "hybridItem", "", ""},
{"dlp.projects.locations.jobTriggers.list", "GET", "v2/{+parent}/jobTriggers", "", "filter,locationId,orderBy,pageSize,pageToken,type", ""},
{"dlp.projects.locations.jobTriggers.patch", "PATCH", "v2/{+name}", "jobTrigger,updateMask", "", ""},
{"dlp.projects.locations.projectDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.projectDataProfiles.list", "GET", "v2/{+parent}/projectDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.projects.locations.storedInfoTypes.create", "POST", "v2/{+parent}/storedInfoTypes", "config,locationId,storedInfoTypeId", "", ""},
{"dlp.projects.locations.storedInfoTypes.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.storedInfoTypes.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.storedInfoTypes.list", "GET", "v2/{+parent}/storedInfoTypes", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.projects.locations.storedInfoTypes.patch", "PATCH", "v2/{+name}", "config,updateMask", "", ""},
{"dlp.projects.locations.tableDataProfiles.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.tableDataProfiles.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.locations.tableDataProfiles.list", "GET", "v2/{+parent}/tableDataProfiles", "", "filter,orderBy,pageSize,pageToken", ""},
{"dlp.projects.storedInfoTypes.create", "POST", "v2/{+parent}/storedInfoTypes", "config,locationId,storedInfoTypeId", "", ""},
{"dlp.projects.storedInfoTypes.delete", "DELETE", "v2/{+name}", "", "", ""},
{"dlp.projects.storedInfoTypes.get", "GET", "v2/{+name}", "", "", ""},
{"dlp.projects.storedInfoTypes.list", "GET", "v2/{+parent}/storedInfoTypes", "", "locationId,orderBy,pageSize,pageToken", ""},
{"dlp.projects.storedInfoTypes.patch", "PATCH", "v2/{+name}", "config,updateMask", "", ""},
// cloudkms v1 revision 20261016
{"cloudkms.folders.getAutokeyConfig", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.folders.getKajPolicyConfig", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.folders.showEffectiveAutokeyConfig", "GET", "v1/{+parent}:showEffectiveAutokeyConfig", "", "", ""},
{"cloudkms.folders.updateAutokeyConfig", "PATCH", "v1/{+name}", "etag,keyProject,keyProjectResolutionMode,name,state", "updateMask", ""},
{"cloudkms.folders.updateKajPolicyConfig", "PATCH", "v1/{+name}", "defaultKeyAccessJustificationPolicy,defaultPolicyAvailable,name", "updateMask", ""},
{"cloudkms.organizations.getKajPolicyConfig", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.organizations.updateKajPolicyConfig", "PATCH", "v1/{+name}", "defaultKeyAccessJustificationPolicy,defaultPolicyAvailable,name", "updateMask", ""},
{"cloudkms.projects.getAutokeyConfig", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.getKajPolicyConfig", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.ekmConfig.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"cloudkms.projects.locations.ekmConfig.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"cloudkms.projects.locations.ekmConfig.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"cloudkms.projects.locations.ekmConnections.create", "POST", "v1/{+parent}/ekmConnections", "createTime,cryptoSpacePath,etag,keyManagementMode,name,serviceResolvers", "ekmConnectionId", ""},
{"cloudkms.projects.locations.ekmConnections.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.ekmConnections.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"cloudkms.projects.locations.ekmConnections.list", "GET", "v1/{+parent}/ekmConnections", "", "filter,orderBy,pageSize,pageToken", ""},
{"cloudkms.projects.locations.ekmConnections.patch", "PATCH", "v1/{+name}", "createTime,cryptoSpacePath,etag,keyManagementMode,name,serviceResolvers", "updateMask", ""},
{"cloudkms.projects.locations.ekmConnections.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"cloudkms.projects.locations.ekmConnections.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"cloudkms.projects.locations.ekmConnections.verifyConnectivity", "GET", "v1/{+name}:verifyConnectivity", "", "", ""},
{"cloudkms.projects.locations.generateRandomBytes", "POST", "v1/{+location}:generateRandomBytes", "lengthBytes,protectionLevel", "", ""},
{"cloudkms.projects.locations.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.getEkmConfig", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyHandles.create", "POST", "v1/{+parent}/keyHandles", "kmsKey,name,resourceTypeSelector", "keyHandleId", ""},
{"cloudkms.projects.locations.keyHandles.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyHandles.list", "GET", "v1/{+parent}/keyHandles", "", "filter,pageSize,pageToken", ""},
{"cloudkms.projects.locations.keyRings.create", "POST", "v1/{+parent}/keyRings", "createTime,name", "keyRingId", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.create", "POST", "v1/{+parent}/cryptoKeys", "createTime,cryptoKeyBackend,destroyScheduledDuration,importOnly,keyAccessJustificationsPolicy,labels,name,nextRotationTime,primary,purpose,rotationPeriod,versionTemplate", "cryptoKeyId,skipInitialVersionCreation,trustedWrappingEnabled", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.asymmetricDecrypt", "POST", "v1/{+name}:asymmetricDecrypt", "ciphertext,ciphertextCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.asymmetricSign", "POST", "v1/{+name}:asymmetricSign", "data,dataCrc32c,digest,digestCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.create", "POST", "v1/{+parent}/cryptoKeyVersions", "algorithm,attestation,createTime,destroyEventTime,destroyTime,externalDestructionFailureReason,externalProtectionLevelOptions,generateTime,generationFailureReason,hsmTrusted,importFailureReason,importJob,importTime,name,protectionLevel,reimportEligible,state,trustedWrappingEnabled", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.decapsulate", "POST", "v1/{+name}:decapsulate", "ciphertext,ciphertextCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.delete", "DELETE", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.destroy", "POST", "v1/{+name}:destroy", "", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.exportTrustedKeyWrappedCryptoKeyVersion", "GET", "v1/{+name}:exportTrustedKeyWrappedCryptoKeyVersion", "", "wrappingKey", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.getPublicKey", "GET", "v1/{+name}/publicKey", "", "publicKeyFormat", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.import", "POST", "v1/{+parent}/cryptoKeyVersions:import", "algorithm,cryptoKeyVersion,importJob,rsaAesWrappedKey,trustedWrappingEnabled,wrappedKey", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.importTrustedKeyWrappedCryptoKeyVersion", "POST", "v1/{+parent}/cryptoKeyVersions:importTrustedKeyWrappedCryptoKeyVersion", "algorithm,cryptoKeyVersion,importingKey,wrappedKey", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.list", "GET", "v1/{+parent}/cryptoKeyVersions", "", "filter,orderBy,pageSize,pageToken,view", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.macSign", "POST", "v1/{+name}:macSign", "data,dataCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.macVerify", "POST", "v1/{+name}:macVerify", "data,dataCrc32c,mac,macCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.patch", "PATCH", "v1/{+name}", "algorithm,attestation,createTime,destroyEventTime,destroyTime,externalDestructionFailureReason,externalProtectionLevelOptions,generateTime,generationFailureReason,hsmTrusted,importFailureReason,importJob,importTime,name,protectionLevel,reimportEligible,state,trustedWrappingEnabled", "updateMask", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.rawDecrypt", "POST", "v1/{+name}:rawDecrypt", "additionalAuthenticatedData,additionalAuthenticatedDataCrc32c,ciphertext,ciphertextCrc32c,initializationVector,initializationVectorCrc32c,tagLength", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.rawEncrypt", "POST", "v1/{+name}:rawEncrypt", "additionalAuthenticatedData,additionalAuthenticatedDataCrc32c,initializationVector,initializationVectorCrc32c,plaintext,plaintextCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.restore", "POST", "v1/{+name}:restore", "", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.decrypt", "POST", "v1/{+name}:decrypt", "additionalAuthenticatedData,additionalAuthenticatedDataCrc32c,ciphertext,ciphertextCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.delete", "DELETE", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.encrypt", "POST", "v1/{+name}:encrypt", "additionalAuthenticatedData,additionalAuthenticatedDataCrc32c,plaintext,plaintextCrc32c", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.list", "GET", "v1/{+parent}/cryptoKeys", "", "filter,orderBy,pageSize,pageToken,versionView", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.patch", "PATCH", "v1/{+name}", "createTime,cryptoKeyBackend,destroyScheduledDuration,importOnly,keyAccessJustificationsPolicy,labels,name,nextRotationTime,primary,purpose,rotationPeriod,versionTemplate", "updateMask", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"cloudkms.projects.locations.keyRings.cryptoKeys.updatePrimaryVersion", "POST", "v1/{+name}:updatePrimaryVersion", "cryptoKeyVersionId", "", ""},
{"cloudkms.projects.locations.keyRings.delete", "DELETE", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyRings.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.keyRings.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"cloudkms.projects.locations.keyRings.importJobs.create", "POST", "v1/{+parent}/importJobs", "attestation,createTime,cryptoKeyBackend,expireEventTime,expireTime,generateTime,importMethod,name,protectionLevel,publicKey,publicKeyFormat,state", "importJobId", ""},
{"cloudkms.projects.locations.keyRings.importJobs.get", "GET", "v1/{+name}", "", "publicKeyFormat", ""},
{"cloudkms.projects.locations.keyRings.importJobs.getIamPolicy", "GET", "v1/{+resource}:getIamPolicy", "", "options.requestedPolicyVersion", ""},
{"cloudkms.projects.locations.keyRings.importJobs.list", "GET", "v1/{+parent}/importJobs", "", "filter,orderBy,pageSize,pageToken", ""},
{"cloudkms.projects.locations.keyRings.importJobs.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"cloudkms.projects.locations.keyRings.importJobs.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"cloudkms.projects.locations.keyRings.list", "GET", "v1/{+parent}/keyRings", "", "filter,orderBy,pageSize,pageToken", ""},
{"cloudkms.projects.locations.keyRings.setIamPolicy", "POST", "v1/{+resource}:setIamPolicy", "policy,updateMask", "", ""},
{"cloudkms.projects.locations.keyRings.testIamPermissions", "POST", "v1/{+resource}:testIamPermissions", "permissions", "", ""},
{"cloudkms.projects.locations.list", "GET", "v1/{+name}/locations", "", "extraLocationTypes,filter,pageSize,pageToken", ""},
{"cloudkms.projects.locations.operations.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.retiredResources.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.retiredResources.list", "GET", "v1/{+parent}/retiredResources", "", "pageSize,pageToken", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.create", "POST", "v1/{+parent}/singleTenantHsmInstances", "createTime,deleteTime,disableTime,keyPortabilityEnabled,name,quorumAuth,state,unrefreshedDurationUntilDisable", "singleTenantHsmInstanceId", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.list", "GET", "v1/{+parent}/singleTenantHsmInstances", "", "filter,orderBy,pageSize,pageToken,showDeleted", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.proposals.approve", "POST", "v1/{+name}:approve", "quorumReply,requiredActionQuorumReply", "", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.proposals.create", "POST", "v1/{+parent}/proposals", "addQuorumMember,createTime,deleteSingleTenantHsmInstance,deleteTime,disableSingleTenantHsmInstance,enableSingleTenantHsmInstance,expireTime,failureReason,name,purgeTime,quorumParameters,refreshSingleTenantHsmInstance,registerTwoFactorAuthKeys,removeQuorumMember,requiredActionQuorumParameters,state,ttl,upgradeKeyTrust", "singleTenantHsmInstanceProposalId", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.proposals.delete", "DELETE", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.proposals.execute", "POST", "v1/{+name}:execute", "", "", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.proposals.get", "GET", "v1/{+name}", "", "", ""},
{"cloudkms.projects.locations.singleTenantHsmInstances.proposals.list", "GET", "v1/{+parent}/proposals", "", "filter,orderBy,pageSize,pageToken,showDeleted", ""},
{"cloudkms.projects.locations.updateEkmConfig", "PATCH", "v1/{+name}", "defaultEkmConnection,name", "updateMask", ""},
{"cloudkms.projects.showEffectiveAutokeyConfig", "GET", "v1/{+parent}:showEffectiveAutokeyConfig", "", "", ""},
{"cloudkms.projects.showEffectiveKeyAccessJustificationsEnrollmentConfig", "GET", "v1/{+project}:showEffectiveKeyAccessJustificationsEnrollmentConfig", "", "", ""},
{"cloudkms.projects.showEffectiveKeyAccessJustificationsPolicyConfig", "GET", "v1/{+project}:showEffectiveKeyAccessJustificationsPolicyConfig", "", "", ""},
{"cloudkms.projects.updateAutokeyConfig", "PATCH", "v1/{+name}", "etag,keyProject,keyProjectResolutionMode,name,state", "updateMask", ""},
{"cloudkms.projects.updateKajPolicyConfig", "PATCH", "v1/{+name}", "defaultKeyAccessJustificationPolicy,defaultPolicyAvailable,name", "updateMask", ""},
// datastore v1 revision 20261013
{"datastore.projects.allocateIds", "POST", "v1/projects/{projectId}:allocateIds", "databaseId,keys,requestOptions", "", ""},
{"datastore.projects.beginTransaction", "POST", "v1/projects/{projectId}:beginTransaction", "databaseId,requestOptions,transactionOptions", "", ""},
{"datastore.projects.commit", "POST", "v1/projects/{projectId}:commit", "databaseId,mode,mutations,requestOptions,singleUseTransaction,transaction", "", ""},
{"datastore.projects.export", "POST", "v1/projects/{projectId}:export", "entityFilter,labels,outputUrlPrefix", "", ""},
{"datastore.projects.import", "POST", "v1/projects/{projectId}:import", "entityFilter,inputUrl,labels", "", ""},
{"datastore.projects.indexes.create", "POST", "v1/projects/{projectId}/indexes", "ancestor,indexId,kind,projectId,properties,state", "", ""},
{"datastore.projects.indexes.delete", "DELETE", "v1/projects/{projectId}/indexes/{indexId}", "", "", ""},
{"datastore.projects.indexes.get", "GET", "v1/projects/{projectId}/indexes/{indexId}", "", "", ""},
{"datastore.projects.indexes.list", "GET", "v1/projects/{projectId}/indexes", "", "filter,pageSize,pageToken", ""},
{"datastore.projects.lookup", "POST", "v1/projects/{projectId}:lookup", "databaseId,keys,propertyMask,readOptions,requestOptions", "", ""},
{"datastore.projects.operations.cancel", "POST", "v1/{+name}:cancel", "", "", ""},
{"datastore.projects.operations.delete", "DELETE", "v1/{+name}", "", "", ""},
{"datastore.projects.operations.get", "GET", "v1/{+name}", "", "", ""},
{"datastore.projects.operations.list", "GET", "v1/{+name}/operations", "", "filter,pageSize,pageToken,returnPartialSuccess", ""},
{"datastore.projects.reserveIds", "POST", "v1/projects/{projectId}:reserveIds", "databaseId,keys,requestOptions", "", ""},
{"datastore.projects.rollback", "POST", "v1/projects/{projectId}:rollback", "databaseId,requestOptions,transaction", "", ""},
{"datastore.projects.runAggregationQuery", "POST", "v1/projects/{projectId}:runAggregationQuery", "aggregationQuery,databaseId,explainOptions,gqlQuery,partitionId,readOptions,requestOptions", "", ""},
{"datastore.projects.runQuery", "POST", "v1/projects/{projectId}:runQuery", "databaseId,explainOptions,gqlQuery,partitionId,propertyMask,query,readOptions,requestOptions", "", ""},
+50
View File
@@ -0,0 +1,50 @@
#pragma once
// Static ceilings for the data package. Never autotuned.
#include <cstddef>
namespace molo::google::data {
struct DataCaps {
// Generic call engine.
static constexpr std::size_t kMaxPathValueChars = 1024; // object name limit
static constexpr std::size_t kMaxResourceNameChars = 1024; // {+name} values
static constexpr std::size_t kMaxQueryParams = 40;
static constexpr std::size_t kMaxQueryValueChars = 2048;
static constexpr std::size_t kMaxQueryArrayItems = 100;
static constexpr std::size_t kMaxBodyBytes = 10u * 1024u * 1024u; // JSON request body
static constexpr int kMaxListPages = 20;
static constexpr int kMaxPageSize = 1000;
// BigQuery.
static constexpr std::size_t kMaxSqlChars = 1024u * 1024u; // Google: 1 MB query text
static constexpr int kMaxQueryTimeoutMs = 60000;
static constexpr int kMaxResultRows = 10000;
static constexpr std::size_t kMaxInsertRows = 10000; // Google: 10,000 rows per insertAll
static constexpr std::size_t kMaxInsertBytes = 10u * 1024u * 1024u;
// Cloud Storage media.
static constexpr std::size_t kMaxDownloadBytes = 16u * 1024u * 1024u;
static constexpr std::size_t kMaxSimpleUploadBytes = 8u * 1024u * 1024u;
static constexpr std::size_t kChunkGranularity = 256u * 1024u; // resumable chunk multiple
static constexpr std::size_t kMaxResumableChunk = 8u * 1024u * 1024u;
static constexpr std::size_t kMaxResumableTotal = 64u * 1024u * 1024u;
static constexpr std::size_t kMaxComposeSources = 32; // Google: 32 per compose
static constexpr long kMinSignedUrlSeconds = 1;
static constexpr long kMaxSignedUrlSeconds = 604800; // V4 maximum: 7 days
static constexpr std::size_t kMaxSignedHeaders = 20;
// Pub/Sub.
static constexpr std::size_t kMaxPublishMessages = 1000; // Google: 1000 per publish
static constexpr std::size_t kMaxMessageBytes = 10u * 1024u * 1024u;
static constexpr int kMaxPullMessages = 100;
static constexpr std::size_t kMaxAckIds = 1000;
// Secret Manager / KMS / DLP / Datastore / Firestore.
static constexpr std::size_t kMaxSecretBytes = 64u * 1024u; // Google: 64 KiB payload
static constexpr std::size_t kMaxCryptoBytes = 64u * 1024u; // KMS plaintext limit
static constexpr std::size_t kMaxDlpItemBytes = 512u * 1024u; // DLP content item
static constexpr std::size_t kMaxKeysPerLookup = 1000; // Datastore lookup
static constexpr std::size_t kMaxWritesPerCommit = 500; // Datastore / Firestore commit
static constexpr std::size_t kMaxFilterChars = 1024;
static constexpr std::size_t kMaxIdChars = 1024;
};
} // namespace molo::google::data
+52
View File
@@ -0,0 +1,52 @@
#pragma once
// DataPlatform: one object holding every typed service of the data package
// over a shared client and project context.
#include <memory>
#include "google/data/bigquery.h"
#include "google/data/bigtable_admin.h"
#include "google/data/dataplex.h"
#include "google/data/datastore.h"
#include "google/data/dlp.h"
#include "google/data/firestore.h"
#include "google/data/kms.h"
#include "google/data/pubsub.h"
#include "google/data/secret_manager.h"
#include "google/data/signed_url.h"
#include "google/data/spanner.h"
#include "google/data/storage.h"
#include "google/data/storage_media.h"
namespace molo::google::data {
class DataPlatform {
public:
DataPlatform(std::shared_ptr<GoogleClient> client, CloudContext context)
: bigquery(client, context),
storage(client, context),
pubsub(client, context),
firestore(client, context),
spanner(client, context),
bigtable(client, context),
secrets(client, context),
dataplex(client, context),
dlp(client, context),
kms(client, context),
datastore(client, context) {}
BigQuery bigquery;
Storage storage;
PubSub pubsub;
Firestore firestore;
Spanner spanner;
BigtableAdmin bigtable;
SecretManager secrets;
Dataplex dataplex;
Dlp dlp;
Kms kms;
Datastore datastore;
};
} // namespace molo::google::data
+64
View File
@@ -0,0 +1,64 @@
#include "google/data/dataplex.h"
namespace molo::google::data {
using clients::body::Json;
Json Dataplex::list_locations(int n) const {
CallArgs a = on("name", project_path());
a.q("pageSize", page(n));
return invoke("dataplex.projects.locations.list", a);
}
Json Dataplex::list_lakes(const std::string& l, int n) const {
CallArgs a = on("parent", location_path(l));
a.q("pageSize", page(n));
return invoke("dataplex.projects.locations.lakes.list", a);
}
Json Dataplex::get_lake(const std::string& l, const std::string& lake) const {
return invoke("dataplex.projects.locations.lakes.get",
on("name", location_path(l) + "/lakes/" + Ids::id(lake, "lake")));
}
Json Dataplex::list_entry_groups(const std::string& l, int n) const {
CallArgs a = on("parent", location_path(l));
a.q("pageSize", page(n));
return invoke("dataplex.projects.locations.entryGroups.list", a);
}
Json Dataplex::get_entry_group(const std::string& l, const std::string& g) const {
return invoke("dataplex.projects.locations.entryGroups.get",
on("name", location_path(l) + "/entryGroups/" + Ids::id(g, "entry group")));
}
Json Dataplex::list_entry_types(const std::string& l, int n) const {
CallArgs a = on("parent", location_path(l));
a.q("pageSize", page(n));
return invoke("dataplex.projects.locations.entryTypes.list", a);
}
Json Dataplex::list_aspect_types(const std::string& l, int n) const {
CallArgs a = on("parent", location_path(l));
a.q("pageSize", page(n));
return invoke("dataplex.projects.locations.aspectTypes.list", a);
}
Json Dataplex::list_data_scans(const std::string& l, int n) const {
CallArgs a = on("parent", location_path(l));
a.q("pageSize", page(n));
return invoke("dataplex.projects.locations.dataScans.list", a);
}
Json Dataplex::get_data_scan(const std::string& l, const std::string& s) const {
return invoke("dataplex.projects.locations.dataScans.get",
on("name", location_path(l) + "/dataScans/" + Ids::id(s, "data scan")));
}
Json Dataplex::search_entries(const std::string& l, const std::string& query, int n) const {
need(!query.empty() && query.size() <= DataCaps::kMaxFilterChars,
"query must be 1.." + std::to_string(DataCaps::kMaxFilterChars) + " characters");
CallArgs a = on("name", location_path(l));
a.q("query", Json(query));
a.q("pageSize", page(n));
return invoke("dataplex.projects.locations.searchEntries", a);
}
Json Dataplex::lookup_entry(const std::string& l, const std::string& entry) const {
need(entry.rfind("projects/", 0) == 0, "entry must be a full name starting with projects/");
CallArgs a = on("name", location_path(l));
a.q("entry", Json(entry));
return invoke("dataplex.projects.locations.lookupEntry", a);
}
} // namespace molo::google::data
+34
View File
@@ -0,0 +1,34 @@
#pragma once
// Dataplex Universal Catalog v1: lakes, entry groups, entry types, aspect
// types, data scans, entry search and lookup. Other methods (206 in total):
// ApiService::invoke("dataplex....").
#include <string>
#include "google/data/project_service.h"
namespace molo::google::data {
class Dataplex : public ProjectService {
public:
explicit Dataplex(std::shared_ptr<GoogleClient> client, CloudContext context)
: ProjectService(std::move(client), Api::kDataplex, std::move(context)) {}
clients::body::Json list_locations(int page_size) const;
clients::body::Json list_lakes(const std::string& location, int page_size) const;
clients::body::Json get_lake(const std::string& location, const std::string& lake) const;
clients::body::Json list_entry_groups(const std::string& location, int page_size) const;
clients::body::Json get_entry_group(const std::string& location, const std::string& group) const;
clients::body::Json list_entry_types(const std::string& location, int page_size) const;
clients::body::Json list_aspect_types(const std::string& location, int page_size) const;
clients::body::Json list_data_scans(const std::string& location, int page_size) const;
clients::body::Json get_data_scan(const std::string& location, const std::string& scan) const;
// Catalog search over the project (POST, read only).
clients::body::Json search_entries(const std::string& location, const std::string& query,
int page_size) const;
// entry: full entry name, e.g. projects/p/locations/l/entryGroups/g/entries/e.
clients::body::Json lookup_entry(const std::string& location, const std::string& entry) const;
};
} // namespace molo::google::data
+115
View File
@@ -0,0 +1,115 @@
#include "google/data/datastore.h"
namespace molo::google::data {
using clients::body::Json;
Datastore::Datastore(std::shared_ptr<GoogleClient> client, CloudContext context,
std::string database_id)
: ProjectService(std::move(client), Api::kDatastore, std::move(context)),
database_(std::move(database_id)) {
if (!database_.empty()) Ids::id(database_, "database");
}
CallArgs Datastore::base() const { return on("projectId", Ids::project(context_.project)); }
Json Datastore::body() const {
Json b = Json::object();
if (!database_.empty()) b.set("databaseId", Json(database_));
return b;
}
Json Datastore::lookup(const Json& keys) const {
need(keys.is_array() && !keys.as_array().empty() &&
keys.as_array().size() <= DataCaps::kMaxKeysPerLookup,
"keys must be an array of 1.." + std::to_string(DataCaps::kMaxKeysPerLookup));
Json b = body();
b.set("keys", keys);
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.lookup", a);
}
Json Datastore::run_query(const Json& query, const Json& gql) const {
need(query.is_null() != gql.is_null(), "give exactly one of query and gql");
Json b = body();
if (!query.is_null()) {
need(query.is_object(), "query must be an object");
b.set("query", query);
} else {
need(gql.is_object(), "gql must be an object");
b.set("gqlQuery", gql);
}
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.runQuery", a);
}
Json Datastore::run_aggregation_query(const Json& agg) const {
need(agg.is_object() && !agg.as_object().empty(), "aggregation query must be an object");
Json b = body();
b.set("aggregationQuery", agg);
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.runAggregationQuery", a);
}
Json Datastore::begin_transaction(bool read_only) const {
Json b = body();
Json opts = Json::object();
if (read_only) opts.set("readOnly", Json::object());
else opts.set("readWrite", Json::object());
b.set("transactionOptions", opts);
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.beginTransaction", a);
}
Json Datastore::rollback(const std::string& transaction) const {
need(!transaction.empty() && transaction.size() <= 4096, "transaction handle required");
Json b = body();
b.set("transaction", Json(transaction));
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.rollback", a);
}
Json Datastore::allocate_ids(const Json& keys) const {
need(keys.is_array() && !keys.as_array().empty() &&
keys.as_array().size() <= DataCaps::kMaxKeysPerLookup,
"keys must be an array of 1.." + std::to_string(DataCaps::kMaxKeysPerLookup));
Json b = body();
b.set("keys", keys);
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.allocateIds", a);
}
Json Datastore::reserve_ids(const Json& keys) const {
need(keys.is_array() && !keys.as_array().empty() &&
keys.as_array().size() <= DataCaps::kMaxKeysPerLookup,
"keys must be an array of 1.." + std::to_string(DataCaps::kMaxKeysPerLookup));
Json b = body();
b.set("keys", keys);
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.reserveIds", a);
}
Json Datastore::commit(const Json& mutations, const std::string& mode,
const std::string& transaction) const {
need(mutations.is_array() && !mutations.as_array().empty() &&
mutations.as_array().size() <= DataCaps::kMaxWritesPerCommit,
"mutations must be an array of 1.." + std::to_string(DataCaps::kMaxWritesPerCommit));
need(mode == "TRANSACTIONAL" || mode == "NON_TRANSACTIONAL",
"mode must be TRANSACTIONAL or NON_TRANSACTIONAL");
Json b = body();
b.set("mutations", mutations);
b.set("mode", Json(mode));
if (!transaction.empty()) b.set("transaction", Json(transaction));
CallArgs a = base();
a.with_body(b);
return invoke("datastore.projects.commit", a);
}
} // namespace molo::google::data
+38
View File
@@ -0,0 +1,38 @@
#pragma once
// Cloud Datastore v1 (also Firestore in Datastore mode): lookup, queries,
// aggregation, transactions, commits, id allocation. Entities and keys are
// the Datastore JSON shapes and pass through as Json.
#include <string>
#include "google/data/project_service.h"
namespace molo::google::data {
class Datastore : public ProjectService {
public:
// database_id "" = the default database.
Datastore(std::shared_ptr<GoogleClient> client, CloudContext context,
std::string database_id = "");
clients::body::Json lookup(const clients::body::Json& keys) const;
// query: a Datastore Query object; gql: a GqlQuery object. Exactly one.
clients::body::Json run_query(const clients::body::Json& query,
const clients::body::Json& gql) const;
clients::body::Json run_aggregation_query(const clients::body::Json& aggregation) const;
clients::body::Json begin_transaction(bool read_only) const;
clients::body::Json rollback(const std::string& transaction) const;
clients::body::Json allocate_ids(const clients::body::Json& incomplete_keys) const;
clients::body::Json reserve_ids(const clients::body::Json& keys) const;
// Writes (explicit). mode: TRANSACTIONAL or NON_TRANSACTIONAL.
clients::body::Json commit(const clients::body::Json& mutations, const std::string& mode,
const std::string& transaction) const;
private:
CallArgs base() const;
clients::body::Json body() const;
std::string database_;
};
} // namespace molo::google::data
+110
View File
@@ -0,0 +1,110 @@
#include "google/data/dlp.h"
namespace molo::google::data {
using clients::body::Json;
namespace {
Json item(const std::string& text) {
if (text.empty() || text.size() > DataCaps::kMaxDlpItemBytes)
throw GoogleError::invalid("text must be 1.." + std::to_string(DataCaps::kMaxDlpItemBytes) +
" bytes");
Json i = Json::object();
i.set("value", Json(text));
return i;
}
} // namespace
void InspectRequest::validate() const {
item(text);
static const char* const likelihood[] = {"VERY_UNLIKELY", "UNLIKELY", "POSSIBLE", "LIKELY",
"VERY_LIKELY"};
bool ok = false;
for (const char* l : likelihood) ok = ok || min_likelihood == l;
if (!ok) throw GoogleError::invalid("min_likelihood is not a DLP likelihood");
if (max_findings < 1 || max_findings > 1000)
throw GoogleError::invalid("max_findings must be 1..1000");
if (info_types.size() > 100) throw GoogleError::invalid("at most 100 info types");
for (const auto& t : info_types) Ids::id(t, "info type");
if (!location.empty()) Ids::location(location);
}
Json InspectRequest::to_json() const {
validate();
Json config = Json::object();
if (!info_types.empty()) {
Json arr = Json::array();
for (const auto& t : info_types) {
Json o = Json::object();
o.set("name", Json(t));
arr.push(o);
}
config.set("infoTypes", arr);
}
config.set("minLikelihood", Json(min_likelihood));
config.set("includeQuote", Json(include_quote));
Json limits = Json::object();
limits.set("maxFindingsPerRequest", Json(max_findings));
config.set("limits", limits);
Json b = Json::object();
b.set("item", item(text));
b.set("inspectConfig", config);
return b;
}
std::string Dlp::parent(const std::string& location) const {
return location.empty() ? project_path() : location_path(location);
}
Json Dlp::inspect_content(const InspectRequest& r) const {
CallArgs a = on("parent", parent(r.location));
a.with_body(r.to_json());
return invoke(r.location.empty() ? "dlp.projects.content.inspect"
: "dlp.projects.locations.content.inspect",
a);
}
Json Dlp::deidentify_content(const std::string& text, const Json& config,
const std::string& location) const {
need(config.is_object() && !config.as_object().empty(), "deidentify config must be an object");
Json b = Json::object();
b.set("item", item(text));
b.set("deidentifyConfig", config);
CallArgs a = on("parent", parent(location));
a.with_body(b);
return invoke(location.empty() ? "dlp.projects.content.deidentify"
: "dlp.projects.locations.content.deidentify",
a);
}
Json Dlp::list_info_types(const std::string& language) const {
CallArgs a;
if (!language.empty()) {
need(language.size() <= 16, "language code too long");
a.q("languageCode", Json(language));
}
return invoke("dlp.infoTypes.list", a);
}
Json Dlp::list_inspect_templates(int n) const {
CallArgs a = on("parent", project_path());
a.q("pageSize", page(n));
return invoke("dlp.projects.inspectTemplates.list", a);
}
Json Dlp::list_deidentify_templates(int n) const {
CallArgs a = on("parent", project_path());
a.q("pageSize", page(n));
return invoke("dlp.projects.deidentifyTemplates.list", a);
}
Json Dlp::list_job_triggers(int n) const {
CallArgs a = on("parent", project_path());
a.q("pageSize", page(n));
return invoke("dlp.projects.jobTriggers.list", a);
}
Json Dlp::list_jobs(int n) const {
CallArgs a = on("parent", project_path());
a.q("pageSize", page(n));
return invoke("dlp.projects.dlpJobs.list", a);
}
} // namespace molo::google::data
+46
View File
@@ -0,0 +1,46 @@
#pragma once
// Cloud Data Loss Prevention v2: inspect and de-identify content, infoTypes,
// templates, job triggers, jobs. Other methods (150 in total):
// ApiService::invoke("dlp....").
#include <string>
#include <vector>
#include "google/data/project_service.h"
namespace molo::google::data {
struct InspectRequest {
std::string text; // 1..512 KiB
std::vector<std::string> info_types; // e.g. EMAIL_ADDRESS; empty = Google's default set
std::string min_likelihood = "POSSIBLE";
bool include_quote = false;
int max_findings = 100; // 1..1000
std::string location; // "" = global endpoint path projects/{p}
void validate() const;
clients::body::Json to_json() const;
};
class Dlp : public ProjectService {
public:
explicit Dlp(std::shared_ptr<GoogleClient> client, CloudContext context)
: ProjectService(std::move(client), Api::kDlp, std::move(context)) {}
// content:inspect is a read operation (it returns findings, stores nothing).
clients::body::Json inspect_content(const InspectRequest& request) const;
// content:deidentify: config is a DeidentifyConfig object.
clients::body::Json deidentify_content(const std::string& text,
const clients::body::Json& deidentify_config,
const std::string& location) const;
clients::body::Json list_info_types(const std::string& language_code) const;
clients::body::Json list_inspect_templates(int page_size) const;
clients::body::Json list_deidentify_templates(int page_size) const;
clients::body::Json list_job_triggers(int page_size) const;
clients::body::Json list_jobs(int page_size) const;
private:
std::string parent(const std::string& location) const;
};
} // namespace molo::google::data
+120
View File
@@ -0,0 +1,120 @@
#include "google/data/firestore.h"
namespace molo::google::data {
using clients::body::Json;
Firestore::Firestore(std::shared_ptr<GoogleClient> client, CloudContext context,
std::string database)
: ProjectService(std::move(client), Api::kFirestore, std::move(context)),
database_(std::move(database)) {
if (database_ != "(default)") Ids::id(database_, "database");
}
std::string Firestore::database_name() const {
return project_path() + "/databases/" + database_;
}
std::string Firestore::documents_root() const { return database_name() + "/documents"; }
std::string Firestore::document_name(const std::string& path) const {
if (path.empty()) return documents_root();
need(path.size() <= DataCaps::kMaxResourceNameChars, "document path too long");
std::size_t segments = 1;
for (char c : path) segments += c == '/';
need(segments <= 100, "document path over 100 segments");
return documents_root() + "/" + path; // check_resource validates at render time
}
Json Firestore::list_databases() const {
return invoke("firestore.projects.databases.list", on("parent", project_path()));
}
Json Firestore::get_database() const {
return invoke("firestore.projects.databases.get", on("name", database_name()));
}
Json Firestore::get_document(const std::string& path) const {
need(!path.empty(), "document path is required");
return invoke("firestore.projects.databases.documents.get", on("name", document_name(path)));
}
Json Firestore::list_documents(const std::string& parent, const std::string& collection,
int page_size, const std::string& page_token) const {
need(!collection.empty(), "collection is required");
CallArgs a = on("parent", document_name(parent));
a.p("collectionId", Ids::id(collection, "collection"));
a.q("pageSize", page(page_size));
if (!page_token.empty()) a.q("pageToken", Json(page_token));
return invoke("firestore.projects.databases.documents.list", a);
}
Json Firestore::list_collection_ids(const std::string& parent, int page_size) const {
CallArgs a = on("parent", document_name(parent));
Json b = Json::object();
b.set("pageSize", page(page_size));
a.with_body(b);
return invoke("firestore.projects.databases.documents.listCollectionIds", a);
}
Json Firestore::run_query(const std::string& parent, const Json& q) const {
need(q.is_object() && !q.as_object().empty(), "structured query must be a non-empty object");
CallArgs a = on("parent", document_name(parent));
Json b = Json::object();
b.set("structuredQuery", q);
a.with_body(b);
return invoke("firestore.projects.databases.documents.runQuery", a);
}
Json Firestore::run_aggregation_query(const std::string& parent, const Json& agg) const {
need(agg.is_object() && !agg.as_object().empty(), "aggregation query must be a non-empty object");
CallArgs a = on("parent", document_name(parent));
Json b = Json::object();
b.set("structuredAggregationQuery", agg);
a.with_body(b);
return invoke("firestore.projects.databases.documents.runAggregationQuery", a);
}
Json Firestore::batch_get(const std::vector<std::string>& paths) const {
need(!paths.empty() && paths.size() <= 100, "batch_get takes 1..100 documents");
Json arr = Json::array();
for (const auto& p : paths) {
need(!p.empty(), "empty document path");
std::string name = document_name(p);
check_resource("document", name);
arr.push(Json(name));
}
Json b = Json::object();
b.set("documents", arr);
CallArgs a = on("database", database_name());
a.with_body(b);
return invoke("firestore.projects.databases.documents.batchGet", a);
}
Json Firestore::create_document(const std::string& parent, const std::string& collection,
const std::string& document_id, const Json& fields) const {
need(!collection.empty(), "collection is required");
need(fields.is_object(), "fields must be an object");
CallArgs a = on("parent", document_name(parent));
a.p("collectionId", Ids::id(collection, "collection"));
if (!document_id.empty()) a.q("documentId", Json(Ids::id(document_id, "document id")));
Json b = Json::object();
b.set("fields", fields);
a.with_body(b);
return invoke("firestore.projects.databases.documents.createDocument", a);
}
Json Firestore::commit(const Json& writes) const {
need(writes.is_array() && !writes.as_array().empty() &&
writes.as_array().size() <= DataCaps::kMaxWritesPerCommit,
"writes must be an array of 1.." + std::to_string(DataCaps::kMaxWritesPerCommit));
Json b = Json::object();
b.set("writes", writes);
CallArgs a = on("database", database_name());
a.with_body(b);
return invoke("firestore.projects.databases.documents.commit", a);
}
Json Firestore::delete_document(const std::string& path) const {
need(!path.empty(), "document path is required");
return invoke("firestore.projects.databases.documents.delete", on("name", document_name(path)));
}
} // namespace molo::google::data
+50
View File
@@ -0,0 +1,50 @@
#pragma once
// Cloud Firestore v1 (native mode): documents, queries, collection ids,
// commits, databases. Other methods: ApiService::invoke("firestore....").
// Paths are relative to the database's documents root ("users/alice").
#include <string>
#include "google/data/project_service.h"
namespace molo::google::data {
class Firestore : public ProjectService {
public:
Firestore(std::shared_ptr<GoogleClient> client, CloudContext context,
std::string database = "(default)");
const std::string& database_id() const { return database_; }
std::string database_name() const; // projects/p/databases/(default)
std::string documents_root() const; // .../documents
// Relative path -> full name; segments validated, 1..100 segments.
std::string document_name(const std::string& path) const;
clients::body::Json list_databases() const;
clients::body::Json get_database() const;
clients::body::Json get_document(const std::string& path) const;
// parent "" = documents root; collection is a collection id.
clients::body::Json list_documents(const std::string& parent, const std::string& collection,
int page_size, const std::string& page_token) const;
clients::body::Json list_collection_ids(const std::string& parent, int page_size) const;
// structuredQuery per the Firestore StructuredQuery schema.
clients::body::Json run_query(const std::string& parent,
const clients::body::Json& structured_query) const;
clients::body::Json run_aggregation_query(const std::string& parent,
const clients::body::Json& aggregation) const;
clients::body::Json batch_get(const std::vector<std::string>& paths) const;
// Writes (explicit).
clients::body::Json create_document(const std::string& parent, const std::string& collection,
const std::string& document_id,
const clients::body::Json& fields) const;
clients::body::Json commit(const clients::body::Json& writes) const;
clients::body::Json delete_document(const std::string& path) const;
private:
std::string database_;
};
} // namespace molo::google::data
+77
View File
@@ -0,0 +1,77 @@
#pragma once
// Identifier rules of the data package. Every id is validated here before any
// network traffic; a failure is GoogleError 400 INVALID_ARGUMENT.
#include <string>
#include "google/errors.h"
namespace molo::google::data {
class Ids {
public:
// [a-z0-9.:-], 1..100 (project ids, with domain-scoped "example.com:proj").
static const std::string& project(const std::string& v) {
return check(v, "project", 100, "abcdefghijklmnopqrstuvwxyz0123456789.:-");
}
// [a-z0-9-], 1..40 ("us", "us-central1", "global", "eu").
static const std::string& location(const std::string& v) {
return check(v, "location", 40, "abcdefghijklmnopqrstuvwxyz0123456789-");
}
// BigQuery dataset: letters, digits, underscore, 1..1024.
static const std::string& dataset(const std::string& v) {
return check(v, "dataset", 1024, alnum() + "_");
}
// BigQuery table / model / routine / job id (ASCII subset): letters, digits, _ $ -.
static const std::string& bq_id(const std::string& v, const char* what = "table") {
return check(v, what, 1024, alnum() + "_$-");
}
// Cloud Storage bucket: 3..222 of a-z 0-9 . _ -, alnum at both ends.
static const std::string& bucket(const std::string& v) {
check(v, "bucket", 222, "abcdefghijklmnopqrstuvwxyz0123456789._-");
auto edge = [](char c) { return (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9'); };
if (v.size() < 3 || !edge(v.front()) || !edge(v.back()))
throw GoogleError::invalid("bucket must be 3..222 characters, alphanumeric at both ends");
return v;
}
// Cloud Storage object name: 1..1024 bytes, no control characters, not . or ..
static const std::string& object(const std::string& v) {
if (v.empty() || v.size() > 1024) throw GoogleError::invalid("object must be 1..1024 bytes");
if (v == "." || v == ".." || v.front() == '\r' || v.front() == '\n')
throw GoogleError::invalid("bad object name");
for (char c : v)
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f)
throw GoogleError::invalid("object has control characters");
return v;
}
// Pub/Sub topic / subscription / snapshot id: 3..255, starts with a letter.
static const std::string& pubsub(const std::string& v) {
check(v, "pubsub id", 255, alnum() + "-_.~");
if (v.size() < 3 || !((v[0] >= 'a' && v[0] <= 'z') || (v[0] >= 'A' && v[0] <= 'Z')))
throw GoogleError::invalid("pubsub id must be 3..255 characters and start with a letter");
return v;
}
// Generic single id (secret, key ring, crypto key, instance, ...): 1..255.
static const std::string& id(const std::string& v, const char* what = "id") {
return check(v, what, 255, alnum() + "_-.");
}
static std::string alnum() {
return "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
}
private:
static const std::string& check(const std::string& v, const char* what, std::size_t max,
const std::string& allowed) {
if (v.empty() || v.size() > max)
throw GoogleError::invalid(std::string(what) + " must be 1.." + std::to_string(max) +
" characters");
for (char c : v)
if (allowed.find(c) == std::string::npos)
throw GoogleError::invalid(std::string(what) + " has characters outside " + allowed);
return v;
}
};
} // namespace molo::google::data
+92
View File
@@ -0,0 +1,92 @@
#include "google/data/kms.h"
namespace molo::google::data {
using clients::body::Json;
std::string Kms::key_ring_name(const std::string& l, const std::string& r) const {
return project_path() + "/locations/" + Ids::location(l) + "/keyRings/" + Ids::id(r, "key ring");
}
std::string Kms::key_name(const std::string& l, const std::string& r, const std::string& k) const {
return key_ring_name(l, r) + "/cryptoKeys/" + Ids::id(k, "key");
}
std::string Kms::version_name(const std::string& l, const std::string& r, const std::string& k,
const std::string& v) const {
need(!v.empty() && v.size() <= 10 && v.find_first_not_of("0123456789") == std::string::npos,
"key version must be a number");
return key_name(l, r, k) + "/cryptoKeyVersions/" + v;
}
Json Kms::list_locations(int n) const {
CallArgs a = on("name", project_path());
a.q("pageSize", page(n));
return invoke("cloudkms.projects.locations.list", a);
}
Json Kms::list_key_rings(const std::string& l, int n) const {
CallArgs a = on("parent", project_path() + "/locations/" + Ids::location(l));
a.q("pageSize", page(n));
return invoke("cloudkms.projects.locations.keyRings.list", a);
}
Json Kms::get_key_ring(const std::string& l, const std::string& r) const {
return invoke("cloudkms.projects.locations.keyRings.get", on("name", key_ring_name(l, r)));
}
Json Kms::list_keys(const std::string& l, const std::string& r, int n) const {
CallArgs a = on("parent", key_ring_name(l, r));
a.q("pageSize", page(n));
return invoke("cloudkms.projects.locations.keyRings.cryptoKeys.list", a);
}
Json Kms::get_key(const std::string& l, const std::string& r, const std::string& k) const {
return invoke("cloudkms.projects.locations.keyRings.cryptoKeys.get", on("name", key_name(l, r, k)));
}
Json Kms::list_versions(const std::string& l, const std::string& r, const std::string& k,
int n) const {
CallArgs a = on("parent", key_name(l, r, k));
a.q("pageSize", page(n));
return invoke("cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.list", a);
}
Json Kms::get_public_key(const std::string& l, const std::string& r, const std::string& k,
const std::string& v) const {
return invoke("cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.getPublicKey",
on("name", version_name(l, r, k, v)));
}
Json Kms::encrypt(const std::string& l, const std::string& r, const std::string& k,
const std::string& plaintext, const std::string& aad) const {
need(!plaintext.empty() && plaintext.size() <= DataCaps::kMaxCryptoBytes,
"plaintext must be 1.." + std::to_string(DataCaps::kMaxCryptoBytes) + " bytes");
need(aad.size() <= 64u * 1024u, "aad over 64 KiB");
Json b = Json::object();
b.set("plaintext", Json(b64(plaintext)));
if (!aad.empty()) b.set("additionalAuthenticatedData", Json(b64(aad)));
CallArgs a = on("name", key_name(l, r, k));
a.with_body(b);
return invoke("cloudkms.projects.locations.keyRings.cryptoKeys.encrypt", a);
}
Json Kms::decrypt(const std::string& l, const std::string& r, const std::string& k,
const std::string& ct, const std::string& aad) const {
need(!ct.empty() && ct.size() <= 2 * DataCaps::kMaxCryptoBytes, "ciphertext size out of range");
std::string raw;
need(security::crypto::Base64::decode(ct, raw), "ciphertext must be base64");
need(aad.size() <= 64u * 1024u, "aad over 64 KiB");
Json b = Json::object();
b.set("ciphertext", Json(ct));
if (!aad.empty()) b.set("additionalAuthenticatedData", Json(b64(aad)));
CallArgs a = on("name", key_name(l, r, k));
a.with_body(b);
return invoke("cloudkms.projects.locations.keyRings.cryptoKeys.decrypt", a);
}
Json Kms::sign_digest(const std::string& l, const std::string& r, const std::string& k,
const std::string& v, const std::string& digest) const {
need(digest.size() == 32, "digest must be 32 raw bytes (SHA-256)");
Json d = Json::object();
d.set("sha256", Json(b64(digest)));
Json b = Json::object();
b.set("digest", d);
CallArgs a = on("name", version_name(l, r, k, v));
a.with_body(b);
return invoke("cloudkms.projects.locations.keyRings.cryptoKeys.cryptoKeyVersions.asymmetricSign", a);
}
} // namespace molo::google::data
+50
View File
@@ -0,0 +1,50 @@
#pragma once
// Cloud KMS v1: key rings, crypto keys, versions, public keys, encrypt,
// decrypt, asymmetric sign, MAC sign. Key creation, rotation and destruction
// are not typed here: use ApiService::invoke("cloudkms....") deliberately.
#include <string>
#include "google/data/project_service.h"
namespace molo::google::data {
class Kms : public ProjectService {
public:
explicit Kms(std::shared_ptr<GoogleClient> client, CloudContext context)
: ProjectService(std::move(client), Api::kKms, std::move(context)) {}
std::string key_ring_name(const std::string& location, const std::string& ring) const;
std::string key_name(const std::string& location, const std::string& ring,
const std::string& key) const;
std::string version_name(const std::string& location, const std::string& ring,
const std::string& key, const std::string& version) const;
clients::body::Json list_locations(int page_size) const;
clients::body::Json list_key_rings(const std::string& location, int page_size) const;
clients::body::Json get_key_ring(const std::string& location, const std::string& ring) const;
clients::body::Json list_keys(const std::string& location, const std::string& ring,
int page_size) const;
clients::body::Json get_key(const std::string& location, const std::string& ring,
const std::string& key) const;
clients::body::Json list_versions(const std::string& location, const std::string& ring,
const std::string& key, int page_size) const;
clients::body::Json get_public_key(const std::string& location, const std::string& ring,
const std::string& key, const std::string& version) const;
// Symmetric: plaintext and ciphertext are raw bytes here, base64 on the wire.
// The ciphertext in the response stays base64 (field "ciphertext").
clients::body::Json encrypt(const std::string& location, const std::string& ring,
const std::string& key, const std::string& plaintext,
const std::string& aad) const;
clients::body::Json decrypt(const std::string& location, const std::string& ring,
const std::string& key, const std::string& ciphertext_base64,
const std::string& aad) const;
// Asymmetric sign over a SHA-256 digest (32 raw bytes).
clients::body::Json sign_digest(const std::string& location, const std::string& ring,
const std::string& key, const std::string& version,
const std::string& sha256_digest) const;
};
} // namespace molo::google::data
+50
View File
@@ -0,0 +1,50 @@
#pragma once
// ProjectService: ApiService bound to a CloudContext, with the resource-name
// and paging helpers every project scoped service of the package shares.
#include <string>
#include "google/data/api_service.h"
#include "google/data/caps.h"
#include "google/data/ids.h"
#include "google/resource.h"
#include "security/crypto/Base64.h"
namespace molo::google::data {
class ProjectService : public ApiService {
public:
ProjectService(std::shared_ptr<GoogleClient> client, Api api, CloudContext context,
const std::string& base_url_override = "")
: ApiService(std::move(client), api, base_url_override), context_(std::move(context)) {}
const CloudContext& context() const { return context_; }
protected:
// "projects/{project}"
std::string project_path() const { return "projects/" + Ids::project(context_.project); }
// "projects/{project}/locations/{location}"; empty location uses the context's.
std::string location_path(const std::string& location) const {
return project_path() + "/locations/" +
Ids::location(location.empty() ? context_.location : location);
}
static clients::body::Json page(int size) {
if (size < 1 || size > DataCaps::kMaxPageSize)
throw GoogleError::invalid("page size must be 1.." + std::to_string(DataCaps::kMaxPageSize));
return clients::body::Json(size);
}
static std::string b64(const std::string& raw) { return security::crypto::Base64::encode(raw); }
static void need(bool ok, const std::string& why) {
if (!ok) throw GoogleError::invalid(why);
}
CallArgs on(const char* key, const std::string& value) const {
CallArgs a;
a.p(key, value);
return a;
}
CloudContext context_;
};
} // namespace molo::google::data
+108
View File
@@ -0,0 +1,108 @@
#include "google/data/pubsub.h"
namespace molo::google::data {
using clients::body::Json;
std::string PubSub::topic_name(const std::string& t) const {
return project_path() + "/topics/" + Ids::pubsub(t);
}
std::string PubSub::subscription_name(const std::string& s) const {
return project_path() + "/subscriptions/" + Ids::pubsub(s);
}
Json PubSub::list_topics(int n) const {
CallArgs a = on("project", project_path());
a.q("pageSize", page(n));
return invoke("pubsub.projects.topics.list", a);
}
Json PubSub::get_topic(const std::string& t) const {
return invoke("pubsub.projects.topics.get", on("topic", topic_name(t)));
}
Json PubSub::create_topic(const std::string& t) const {
CallArgs a = on("name", topic_name(t));
a.with_body(Json::object());
return invoke("pubsub.projects.topics.create", a);
}
Json PubSub::delete_topic(const std::string& t) const {
return invoke("pubsub.projects.topics.delete", on("topic", topic_name(t)));
}
Json PubSub::list_subscriptions(int n) const {
CallArgs a = on("project", project_path());
a.q("pageSize", page(n));
return invoke("pubsub.projects.subscriptions.list", a);
}
Json PubSub::get_subscription(const std::string& s) const {
return invoke("pubsub.projects.subscriptions.get", on("subscription", subscription_name(s)));
}
Json PubSub::list_snapshots(int n) const {
CallArgs a = on("project", project_path());
a.q("pageSize", page(n));
return invoke("pubsub.projects.snapshots.list", a);
}
Json PubSub::publish_body(const std::vector<PubsubMessage>& messages) {
need(!messages.empty() && messages.size() <= DataCaps::kMaxPublishMessages,
"publish takes 1.." + std::to_string(DataCaps::kMaxPublishMessages) + " messages");
std::size_t total = 0;
Json arr = Json::array();
for (const PubsubMessage& m : messages) {
need(!m.data.empty() || !m.attributes.empty(), "a message needs data or attributes");
need(m.data.size() <= DataCaps::kMaxMessageBytes, "message over the size cap");
total += m.data.size();
need(total <= DataCaps::kMaxMessageBytes, "publish request over the size cap");
Json o = Json::object();
if (!m.data.empty()) o.set("data", Json(b64(m.data)));
if (!m.attributes.empty()) {
need(m.attributes.size() <= 100, "at most 100 attributes");
Json at = Json::object();
for (const auto& kv : m.attributes) {
need(!kv.first.empty() && kv.first.size() <= 256 && kv.second.size() <= 1024,
"attribute key 1..256, value <= 1024 chars");
at.set(kv.first, Json(kv.second));
}
o.set("attributes", at);
}
if (!m.ordering_key.empty()) {
need(m.ordering_key.size() <= 1024, "ordering key over 1024 bytes");
o.set("orderingKey", Json(m.ordering_key));
}
arr.push(o);
}
Json b = Json::object();
b.set("messages", arr);
return b;
}
Json PubSub::publish(const std::string& t, const std::vector<PubsubMessage>& messages) const {
CallArgs a = on("topic", topic_name(t));
a.with_body(publish_body(messages));
return invoke("pubsub.projects.topics.publish", a);
}
Json PubSub::pull(const std::string& s, int max_messages) const {
need(max_messages >= 1 && max_messages <= DataCaps::kMaxPullMessages,
"max_messages must be 1.." + std::to_string(DataCaps::kMaxPullMessages));
CallArgs a = on("subscription", subscription_name(s));
Json b = Json::object();
b.set("maxMessages", Json(max_messages));
a.with_body(b);
return invoke("pubsub.projects.subscriptions.pull", a);
}
Json PubSub::acknowledge(const std::string& s, const std::vector<std::string>& ack_ids) const {
need(!ack_ids.empty() && ack_ids.size() <= DataCaps::kMaxAckIds,
"ack ids: 1.." + std::to_string(DataCaps::kMaxAckIds));
Json ids = Json::array();
for (const auto& id : ack_ids) {
need(!id.empty() && id.size() <= 512, "ack id 1..512 chars");
ids.push(Json(id));
}
Json b = Json::object();
b.set("ackIds", ids);
CallArgs a = on("subscription", subscription_name(s));
a.with_body(b);
return invoke("pubsub.projects.subscriptions.acknowledge", a);
}
} // namespace molo::google::data
+46
View File
@@ -0,0 +1,46 @@
#pragma once
// Cloud Pub/Sub v1: topics, subscriptions, snapshots, publish, pull,
// acknowledge. Other methods: ApiService::invoke("pubsub....").
#include <string>
#include <vector>
#include "google/data/project_service.h"
namespace molo::google::data {
struct PubsubMessage {
std::string data; // raw bytes, base64 encoded on the wire
std::map<std::string, std::string> attributes;
std::string ordering_key;
};
class PubSub : public ProjectService {
public:
explicit PubSub(std::shared_ptr<GoogleClient> client, CloudContext context)
: ProjectService(std::move(client), Api::kPubsub, std::move(context)) {}
std::string topic_name(const std::string& topic) const;
std::string subscription_name(const std::string& subscription) const;
clients::body::Json list_topics(int page_size) const;
clients::body::Json get_topic(const std::string& topic) const;
clients::body::Json create_topic(const std::string& topic) const;
clients::body::Json delete_topic(const std::string& topic) const;
clients::body::Json list_subscriptions(int page_size) const;
clients::body::Json get_subscription(const std::string& subscription) const;
clients::body::Json list_snapshots(int page_size) const;
// publish: 1..1000 messages, each <= 10 MiB, 10 MiB total.
clients::body::Json publish(const std::string& topic,
const std::vector<PubsubMessage>& messages) const;
static clients::body::Json publish_body(const std::vector<PubsubMessage>& messages);
// pull leases messages: they come back again unless acknowledged.
clients::body::Json pull(const std::string& subscription, int max_messages) const;
clients::body::Json acknowledge(const std::string& subscription,
const std::vector<std::string>& ack_ids) const;
};
} // namespace molo::google::data
+79
View File
@@ -0,0 +1,79 @@
#include "google/data/secret_manager.h"
namespace molo::google::data {
using clients::body::Json;
std::string SecretManager::secret_name(const std::string& s) const {
return project_path() + "/secrets/" + Ids::id(s, "secret");
}
std::string SecretManager::version_name(const std::string& s, const std::string& v) const {
bool ok = v == "latest" || (!v.empty() && v.size() <= 10);
for (char c : v)
if (v != "latest" && (c < '0' || c > '9')) ok = false;
need(ok, "version must be \"latest\" or a number");
return secret_name(s) + "/versions/" + v;
}
Json SecretManager::list_secrets(int n, const std::string& filter) const {
CallArgs a = on("parent", project_path());
a.q("pageSize", page(n));
need(filter.size() <= DataCaps::kMaxFilterChars, "filter too long");
if (!filter.empty()) a.q("filter", Json(filter));
return invoke("secretmanager.projects.secrets.list", a);
}
Json SecretManager::get_secret(const std::string& s) const {
return invoke("secretmanager.projects.secrets.get", on("name", secret_name(s)));
}
Json SecretManager::list_versions(const std::string& s, int n) const {
CallArgs a = on("parent", secret_name(s));
a.q("pageSize", page(n));
return invoke("secretmanager.projects.secrets.versions.list", a);
}
Json SecretManager::get_version(const std::string& s, const std::string& v) const {
return invoke("secretmanager.projects.secrets.versions.get", on("name", version_name(s, v)));
}
Json SecretManager::access_version(const std::string& s, const std::string& v) const {
return invoke("secretmanager.projects.secrets.versions.access", on("name", version_name(s, v)));
}
Json SecretManager::create_secret(const std::string& s) const {
CallArgs a = on("parent", project_path());
a.q("secretId", Json(Ids::id(s, "secret")));
Json automatic = Json::object();
Json replication = Json::object();
replication.set("automatic", automatic);
Json b = Json::object();
b.set("replication", replication);
a.with_body(b);
return invoke("secretmanager.projects.secrets.create", a);
}
Json SecretManager::add_version(const std::string& s, const std::string& payload) const {
need(!payload.empty() && payload.size() <= DataCaps::kMaxSecretBytes,
"payload must be 1.." + std::to_string(DataCaps::kMaxSecretBytes) + " bytes");
Json p = Json::object();
p.set("data", Json(b64(payload)));
Json b = Json::object();
b.set("payload", p);
CallArgs a = on("parent", secret_name(s));
a.with_body(b);
return invoke("secretmanager.projects.secrets.addVersion", a);
}
Json SecretManager::disable_version(const std::string& s, const std::string& v) const {
CallArgs a = on("name", version_name(s, v));
a.with_body(Json::object());
return invoke("secretmanager.projects.secrets.versions.disable", a);
}
Json SecretManager::destroy_version(const std::string& s, const std::string& v) const {
CallArgs a = on("name", version_name(s, v));
a.with_body(Json::object());
return invoke("secretmanager.projects.secrets.versions.destroy", a);
}
Json SecretManager::delete_secret(const std::string& s) const {
return invoke("secretmanager.projects.secrets.delete", on("name", secret_name(s)));
}
} // namespace molo::google::data
+37
View File
@@ -0,0 +1,37 @@
#pragma once
// Secret Manager v1: secrets and versions (global resources; regional secrets
// via ApiService::invoke("secretmanager.projects.locations....")). Reading a
// payload is access_version(); listing and metadata never return payloads.
#include <string>
#include "google/data/project_service.h"
namespace molo::google::data {
class SecretManager : public ProjectService {
public:
explicit SecretManager(std::shared_ptr<GoogleClient> client, CloudContext context)
: ProjectService(std::move(client), Api::kSecretManager, std::move(context)) {}
std::string secret_name(const std::string& secret) const;
// version: "latest" or a positive number.
std::string version_name(const std::string& secret, const std::string& version) const;
clients::body::Json list_secrets(int page_size, const std::string& filter) const;
clients::body::Json get_secret(const std::string& secret) const;
clients::body::Json list_versions(const std::string& secret, int page_size) const;
clients::body::Json get_version(const std::string& secret, const std::string& version) const;
// Returns the payload (base64 in payload.data). Sensitive: callers must not log it.
clients::body::Json access_version(const std::string& secret, const std::string& version) const;
// Writes (explicit).
clients::body::Json create_secret(const std::string& secret) const; // automatic replication
clients::body::Json add_version(const std::string& secret, const std::string& payload) const;
clients::body::Json disable_version(const std::string& secret, const std::string& version) const;
clients::body::Json destroy_version(const std::string& secret, const std::string& version) const;
clients::body::Json delete_secret(const std::string& secret) const;
};
} // namespace molo::google::data
+223
View File
@@ -0,0 +1,223 @@
#include "google/data/signed_url.h"
#include <wolfssl/options.h>
#include <wolfssl/wolfcrypt/asn.h>
#include <wolfssl/wolfcrypt/error-crypt.h>
#include <wolfssl/wolfcrypt/random.h>
#include <wolfssl/wolfcrypt/rsa.h>
#include <wolfssl/wolfcrypt/sha256.h>
#include <wolfssl/wolfcrypt/signature.h>
#include <algorithm>
#include <cctype>
#include <vector>
#include "clients/url.h"
#include "google/credentials.h"
#include "google/data/caps.h"
#include "google/data/ids.h"
#include "google/errors.h"
#include "security/crypto/Base64.h"
namespace molo::google::data {
using security::crypto::Base64;
namespace {
std::string enc(const std::string& s) { return clients::Url::encode_component(s); }
std::string lower(std::string s) {
for (char& c : s) c = static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
return s;
}
std::string trim(const std::string& s) {
std::size_t a = s.find_first_not_of(" \t"), b = s.find_last_not_of(" \t");
return a == std::string::npos ? "" : s.substr(a, b - a + 1);
}
std::string hex(const std::string& raw) {
static const char* k = "0123456789abcdef";
std::string out;
for (unsigned char c : raw) {
out.push_back(k[c >> 4]);
out.push_back(k[c & 15]);
}
return out;
}
std::string pem_to_der(const std::string& pem) {
std::string body, line;
for (char c : pem + "\n") {
if (c != '\n' && c != '\r') {
line.push_back(c);
continue;
}
if (!line.empty() && line.rfind("-----", 0) != 0) body += line;
line.clear();
}
std::string der;
if (!Base64::decode(body, der) || der.empty())
throw GoogleError::invalid("private_key is not valid PEM");
return der;
}
struct RsaGuard {
RsaGuard() { wc_InitRsaKey(&key, nullptr); }
~RsaGuard() { wc_FreeRsaKey(&key); }
RsaGuard(const RsaGuard&) = delete;
RsaGuard& operator=(const RsaGuard&) = delete;
RsaKey key;
};
} // namespace
void SignedUrlRequest::validate() const {
static const char* const ok[] = {"GET", "HEAD", "PUT", "POST", "DELETE"};
bool known = false;
for (const char* m : ok) known = known || method == m;
if (!known) throw GoogleError::invalid("method must be GET, HEAD, PUT, POST or DELETE");
Ids::bucket(bucket);
if (!object.empty()) Ids::object(object);
if (expires_seconds < DataCaps::kMinSignedUrlSeconds ||
expires_seconds > DataCaps::kMaxSignedUrlSeconds)
throw GoogleError::invalid("expires_seconds must be 1.." +
std::to_string(DataCaps::kMaxSignedUrlSeconds));
if (headers.size() > DataCaps::kMaxSignedHeaders || query.size() > DataCaps::kMaxSignedHeaders)
throw GoogleError::invalid("too many signed headers or query parameters");
for (const auto& kv : headers) {
if (kv.first.empty() || kv.first.size() > 128 || kv.second.size() > 512)
throw GoogleError::invalid("bad signed header");
for (char c : kv.first + kv.second)
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f)
throw GoogleError::invalid("signed header has control characters");
if (lower(kv.first) == "host") throw GoogleError::invalid("host is always signed");
}
for (const auto& kv : query) {
if (kv.first.empty() || kv.first.size() > 128 || kv.second.size() > 1024)
throw GoogleError::invalid("bad query parameter");
if (lower(kv.first).rfind("x-goog-", 0) == 0)
throw GoogleError::invalid("x-goog-* query parameters are set by the signer");
}
}
SignedUrl::SignedUrl(ServiceAccountKey key) : key_(std::move(key)) {}
SignedUrl SignedUrl::from_env() {
std::string path = Credentials::env("GOOGLE_APPLICATION_CREDENTIALS");
if (path.empty()) throw NotConfiguredError("storage");
return SignedUrl(ServiceAccountKey::load_file(path));
}
namespace {
struct Stamp {
std::string date, datetime;
};
Stamp stamp(std::time_t t) {
std::tm tm{};
gmtime_r(&t, &tm);
char d[16], dt[32];
std::strftime(d, sizeof d, "%Y%m%d", &tm);
std::strftime(dt, sizeof dt, "%Y%m%dT%H%M%SZ", &tm);
return {d, dt};
}
} // namespace
std::string SignedUrl::canonical_request(const SignedUrlRequest& r, std::time_t now) const {
Stamp s = stamp(now);
std::string uri = "/" + r.bucket;
if (!r.object.empty()) {
uri += "/";
std::string seg;
for (char c : r.object) {
if (c == '/') {
uri += enc(seg) + "/";
seg.clear();
} else {
seg.push_back(c);
}
}
uri += enc(seg);
}
std::map<std::string, std::string> hdr;
hdr["host"] = "storage.googleapis.com";
for (const auto& kv : r.headers) hdr[lower(kv.first)] = trim(kv.second);
std::string signed_headers, canon_headers;
for (const auto& kv : hdr) {
if (!signed_headers.empty()) signed_headers += ";";
signed_headers += kv.first;
canon_headers += kv.first + ":" + kv.second + "\n";
}
std::map<std::string, std::string> q;
for (const auto& kv : r.query) q[enc(kv.first)] = enc(kv.second);
q["X-Goog-Algorithm"] = "GOOG4-RSA-SHA256";
q["X-Goog-Credential"] = enc(key_.client_email + "/" + s.date + "/auto/storage/goog4_request");
q["X-Goog-Date"] = s.datetime;
q["X-Goog-Expires"] = std::to_string(r.expires_seconds);
q["X-Goog-SignedHeaders"] = enc(signed_headers);
std::string canon_query;
for (const auto& kv : q) {
if (!canon_query.empty()) canon_query += "&";
canon_query += kv.first + "=" + kv.second;
}
return r.method + "\n" + uri + "\n" + canon_query + "\n" + canon_headers + "\n" +
signed_headers + "\nUNSIGNED-PAYLOAD";
}
std::string SignedUrl::string_to_sign(const std::string& datetime, const std::string& scope,
const std::string& canonical) {
// wolfCrypt, not security/crypto/Sha256: the google build only links Base64.
wc_Sha256 h;
byte digest[WC_SHA256_DIGEST_SIZE];
if (wc_InitSha256(&h) != 0 ||
wc_Sha256Update(&h, reinterpret_cast<const byte*>(canonical.data()),
static_cast<word32>(canonical.size())) != 0 ||
wc_Sha256Final(&h, digest) != 0)
throw GoogleError(500, "INTERNAL", "sha256 failed");
wc_Sha256Free(&h);
return "GOOG4-RSA-SHA256\n" + datetime + "\n" + scope + "\n" +
hex(std::string(reinterpret_cast<char*>(digest), sizeof digest));
}
std::string SignedUrl::rsa_sha256_hex(const std::string& data) const {
std::string der = pem_to_der(key_.private_key_pem);
RsaGuard rsa;
word32 idx = 0;
int off = wc_GetPkcs8TraditionalOffset(reinterpret_cast<byte*>(der.data()), &idx,
static_cast<word32>(der.size()));
if (off < 0) idx = 0;
if (wc_RsaPrivateKeyDecode(reinterpret_cast<const byte*>(der.data()), &idx, &rsa.key,
static_cast<word32>(der.size())) != 0)
throw GoogleError::invalid("private_key is not a usable RSA key");
WC_RNG rng;
if (wc_InitRng(&rng) != 0) throw GoogleError(500, "INTERNAL", "rng init failed");
word32 len = static_cast<word32>(
wc_SignatureGetSize(WC_SIGNATURE_TYPE_RSA_W_ENC, &rsa.key, sizeof(rsa.key)));
std::string sig(len, '\0');
int rc = wc_SignatureGenerate(WC_HASH_TYPE_SHA256, WC_SIGNATURE_TYPE_RSA_W_ENC,
reinterpret_cast<const byte*>(data.data()),
static_cast<word32>(data.size()),
reinterpret_cast<byte*>(sig.data()), &len, &rsa.key,
sizeof(rsa.key), &rng);
wc_FreeRng(&rng);
if (rc != 0) throw GoogleError(500, "INTERNAL", "signing failed");
sig.resize(len);
return hex(sig);
}
std::string SignedUrl::sign(const SignedUrlRequest& r) const {
r.validate();
std::time_t now = r.now ? r.now : std::time(nullptr);
Stamp s = stamp(now);
std::string canon = canonical_request(r, now);
std::string scope = s.date + "/auto/storage/goog4_request";
std::string signature = rsa_sha256_hex(string_to_sign(s.datetime, scope, canon));
// Rebuild "path?query" from the canonical request (line 2 and 3).
std::size_t a = canon.find('\n'), b = canon.find('\n', a + 1), c = canon.find('\n', b + 1);
std::string uri = canon.substr(a + 1, b - a - 1);
std::string query = canon.substr(b + 1, c - b - 1);
return "https://storage.googleapis.com" + uri + "?" + query + "&X-Goog-Signature=" + signature;
}
} // namespace molo::google::data
+48
View File
@@ -0,0 +1,48 @@
#pragma once
// Cloud Storage V4 signed URLs (GOOG4-RSA-SHA256), signed locally with the
// service-account private key. No network traffic. Anyone holding the URL can
// perform the signed method until it expires, so lifetimes are capped at the
// V4 maximum of 7 days.
#include <ctime>
#include <map>
#include <string>
#include "google/service_account.h"
namespace molo::google::data {
struct SignedUrlRequest {
std::string method = "GET"; // GET, HEAD, PUT, POST, DELETE
std::string bucket;
std::string object; // "" signs a bucket-level URL
long expires_seconds = 900; // 1..604800
std::map<std::string, std::string> headers; // extra signed headers (lowercased on sign)
std::map<std::string, std::string> query; // extra query parameters (e.g. response-content-type)
std::time_t now = 0; // 0: current time (tests inject)
void validate() const;
};
class SignedUrl {
public:
explicit SignedUrl(ServiceAccountKey key);
// Key from GOOGLE_APPLICATION_CREDENTIALS; throws NotConfiguredError if unset.
static SignedUrl from_env();
std::string sign(const SignedUrlRequest& request) const;
// The parts, for tests and for the example page.
std::string canonical_request(const SignedUrlRequest& request, std::time_t now) const;
static std::string string_to_sign(const std::string& datetime, const std::string& scope,
const std::string& canonical_request);
const std::string& email() const { return key_.client_email; }
private:
std::string rsa_sha256_hex(const std::string& data) const;
ServiceAccountKey key_;
};
} // namespace molo::google::data
+89
View File
@@ -0,0 +1,89 @@
#include "google/data/spanner.h"
namespace molo::google::data {
using clients::body::Json;
void SpannerSql::validate() const {
if (sql.empty() || sql.size() > DataCaps::kMaxSqlChars)
throw GoogleError::invalid("sql must be 1.." + std::to_string(DataCaps::kMaxSqlChars) +
" characters");
if (query_mode != "NORMAL" && query_mode != "PLAN" && query_mode != "PROFILE")
throw GoogleError::invalid("query_mode must be NORMAL, PLAN or PROFILE");
if (!params.is_null() && !params.is_object()) throw GoogleError::invalid("params must be an object");
if (!param_types.is_null() && !param_types.is_object())
throw GoogleError::invalid("param_types must be an object");
}
std::string Spanner::instance_name(const std::string& i) const {
return project_path() + "/instances/" + Ids::id(i, "instance");
}
std::string Spanner::database_name(const std::string& i, const std::string& d) const {
return instance_name(i) + "/databases/" + Ids::id(d, "database");
}
Json Spanner::list_instances(int n) const {
CallArgs a = on("parent", project_path());
a.q("pageSize", page(n));
return invoke("spanner.projects.instances.list", a);
}
Json Spanner::get_instance(const std::string& i) const {
return invoke("spanner.projects.instances.get", on("name", instance_name(i)));
}
Json Spanner::list_databases(const std::string& i, int n) const {
CallArgs a = on("parent", instance_name(i));
a.q("pageSize", page(n));
return invoke("spanner.projects.instances.databases.list", a);
}
Json Spanner::get_database(const std::string& i, const std::string& d) const {
return invoke("spanner.projects.instances.databases.get", on("name", database_name(i, d)));
}
Json Spanner::get_ddl(const std::string& i, const std::string& d) const {
return invoke("spanner.projects.instances.databases.getDdl", on("database", database_name(i, d)));
}
Json Spanner::create_session(const std::string& i, const std::string& d) const {
CallArgs a = on("database", database_name(i, d));
a.with_body(Json::object());
return invoke("spanner.projects.instances.databases.sessions.create", a);
}
Json Spanner::delete_session(const std::string& name) const {
return invoke("spanner.projects.instances.databases.sessions.delete", on("name", name));
}
Json Spanner::execute_sql(const std::string& session, const SpannerSql& s) const {
s.validate();
Json read_only = Json::object();
read_only.set("strong", Json(true));
Json opts = Json::object();
opts.set("readOnly", read_only);
Json tx = Json::object();
tx.set("singleUse", opts);
Json b = Json::object();
b.set("sql", Json(s.sql));
b.set("queryMode", Json(s.query_mode));
b.set("transaction", tx);
if (!s.params.is_null()) b.set("params", s.params);
if (!s.param_types.is_null()) b.set("paramTypes", s.param_types);
CallArgs a = on("session", session);
a.with_body(b);
return invoke("spanner.projects.instances.databases.sessions.executeSql", a);
}
Json Spanner::query(const std::string& i, const std::string& d, const SpannerSql& s) const {
s.validate();
Json session = create_session(i, d);
std::string name = string_at(session, "name");
need(!name.empty(), "create session returned no name");
Json result;
try {
result = execute_sql(name, s);
} catch (...) {
try { delete_session(name); } catch (...) {}
throw;
}
delete_session(name);
return result;
}
} // namespace molo::google::data
+45
View File
@@ -0,0 +1,45 @@
#pragma once
// Cloud Spanner v1: instances, databases, DDL, sessions, SQL. Other methods:
// ApiService::invoke("spanner....").
#include <string>
#include "google/data/project_service.h"
namespace molo::google::data {
struct SpannerSql {
std::string sql;
clients::body::Json params; // object name -> value, or null
clients::body::Json param_types; // object name -> Type, or null
std::string query_mode = "NORMAL"; // NORMAL, PLAN (no rows, no cost), PROFILE
void validate() const;
};
class Spanner : public ProjectService {
public:
explicit Spanner(std::shared_ptr<GoogleClient> client, CloudContext context)
: ProjectService(std::move(client), Api::kSpanner, std::move(context)) {}
std::string instance_name(const std::string& instance) const;
std::string database_name(const std::string& instance, const std::string& database) const;
clients::body::Json list_instances(int page_size) const;
clients::body::Json get_instance(const std::string& instance) const;
clients::body::Json list_databases(const std::string& instance, int page_size) const;
clients::body::Json get_database(const std::string& instance, const std::string& database) const;
clients::body::Json get_ddl(const std::string& instance, const std::string& database) const;
// Sessions are cheap server side handles; always delete what you create.
clients::body::Json create_session(const std::string& instance,
const std::string& database) const;
clients::body::Json delete_session(const std::string& session_name) const;
// Read-only, single use strong transaction on an existing session.
clients::body::Json execute_sql(const std::string& session_name, const SpannerSql& sql) const;
// create_session + execute_sql + delete_session (always deletes).
clients::body::Json query(const std::string& instance, const std::string& database,
const SpannerSql& sql) const;
};
} // namespace molo::google::data
+132
View File
@@ -0,0 +1,132 @@
#include "google/data/storage.h"
#include "google/data/caps.h"
#include "google/data/ids.h"
namespace molo::google::data {
using clients::body::Json;
namespace {
Json page(int max_results) {
if (max_results < 1 || max_results > DataCaps::kMaxPageSize)
throw GoogleError::invalid("max_results must be 1.." + std::to_string(DataCaps::kMaxPageSize));
return Json(max_results);
}
} // namespace
Storage::Storage(std::shared_ptr<GoogleClient> client, CloudContext context)
: ApiService(std::move(client), Api::kStorage), context_(std::move(context)) {}
Json Storage::list_buckets(int max_results, const std::string& prefix,
const std::string& page_token) const {
CallArgs a;
a.q("project", Json(Ids::project(context_.project)));
a.q("maxResults", page(max_results));
if (!prefix.empty()) a.q("prefix", Json(prefix));
if (!page_token.empty()) a.q("pageToken", Json(page_token));
return invoke("storage.buckets.list", a);
}
Json Storage::get_bucket(const std::string& bucket) const {
return invoke("storage.buckets.get", CallArgs().p("bucket", Ids::bucket(bucket)));
}
Json Storage::create_bucket(const std::string& bucket, const std::string& location,
const std::string& storage_class) const {
Json b = Json::object();
b.set("name", Json(Ids::bucket(bucket)));
if (!location.empty()) b.set("location", Json(location));
if (!storage_class.empty()) {
if (storage_class != "STANDARD" && storage_class != "NEARLINE" &&
storage_class != "COLDLINE" && storage_class != "ARCHIVE")
throw GoogleError::invalid("storage_class must be STANDARD, NEARLINE, COLDLINE or ARCHIVE");
b.set("storageClass", Json(storage_class));
}
CallArgs a;
a.q("project", Json(Ids::project(context_.project)));
a.with_body(b);
return invoke("storage.buckets.insert", a);
}
Json Storage::delete_bucket(const std::string& bucket) const {
return invoke("storage.buckets.delete", CallArgs().p("bucket", Ids::bucket(bucket)));
}
Json Storage::list_objects(const std::string& bucket, const std::string& prefix,
const std::string& delimiter, int max_results,
const std::string& page_token) const {
CallArgs a;
a.p("bucket", Ids::bucket(bucket));
a.q("maxResults", page(max_results));
if (!prefix.empty()) a.q("prefix", Json(prefix));
if (!delimiter.empty()) a.q("delimiter", Json(delimiter));
if (!page_token.empty()) a.q("pageToken", Json(page_token));
return invoke("storage.objects.list", a);
}
Json Storage::get_object(const std::string& bucket, const std::string& object) const {
return invoke("storage.objects.get",
CallArgs().p("bucket", Ids::bucket(bucket)).p("object", Ids::object(object)));
}
Json Storage::patch_object(const std::string& bucket, const std::string& object,
const Json& fields) const {
CallArgs a;
a.p("bucket", Ids::bucket(bucket)).p("object", Ids::object(object));
a.with_body(fields);
return invoke("storage.objects.patch", a);
}
Json Storage::delete_object(const std::string& bucket, const std::string& object) const {
return invoke("storage.objects.delete",
CallArgs().p("bucket", Ids::bucket(bucket)).p("object", Ids::object(object)));
}
Json Storage::copy_object(const std::string& sb, const std::string& so, const std::string& db,
const std::string& d) const {
CallArgs a;
a.p("sourceBucket", Ids::bucket(sb)).p("sourceObject", Ids::object(so));
a.p("destinationBucket", Ids::bucket(db)).p("destinationObject", Ids::object(d));
return invoke("storage.objects.copy", a);
}
Json Storage::compose_body(const std::vector<ComposeSource>& sources,
const std::string& content_type) {
if (sources.empty() || sources.size() > DataCaps::kMaxComposeSources)
throw GoogleError::invalid("compose takes 1.." + std::to_string(DataCaps::kMaxComposeSources) +
" sources");
Json arr = Json::array();
for (const ComposeSource& s : sources) {
Json o = Json::object();
o.set("name", Json(Ids::object(s.name)));
if (s.generation < 0) throw GoogleError::invalid("generation must be >= 0");
if (s.generation > 0) o.set("generation", Json(std::to_string(s.generation)));
arr.push(o);
}
Json dest = Json::object();
if (!content_type.empty()) dest.set("contentType", Json(content_type));
Json b = Json::object();
b.set("sourceObjects", arr);
b.set("destination", dest);
return b;
}
Json Storage::compose(const std::string& bucket, const std::vector<ComposeSource>& sources,
const std::string& destination, const std::string& content_type) const {
CallArgs a;
a.p("destinationBucket", Ids::bucket(bucket)).p("destinationObject", Ids::object(destination));
a.with_body(compose_body(sources, content_type));
return invoke("storage.objects.compose", a);
}
Json Storage::bucket_iam_policy(const std::string& bucket) const {
return invoke("storage.buckets.getIamPolicy", CallArgs().p("bucket", Ids::bucket(bucket)));
}
Json Storage::service_account() const {
return invoke("storage.projects.serviceAccount.get",
CallArgs().p("projectId", Ids::project(context_.project)));
}
} // namespace molo::google::data
+58
View File
@@ -0,0 +1,58 @@
#pragma once
// Cloud Storage JSON API v1: buckets and objects (metadata, list, compose,
// copy, delete). Media (download, simple and resumable upload) is in
// storage_media.h, signed URLs in signed_url.h. Every other method is
// reachable with ApiService::invoke("storage....").
#include <string>
#include <vector>
#include "google/data/api_service.h"
#include "google/resource.h"
namespace molo::google::data {
struct ComposeSource {
std::string name;
long long generation = 0; // 0: current generation
};
class Storage : public ApiService {
public:
explicit Storage(std::shared_ptr<GoogleClient> client, CloudContext context);
const CloudContext& context() const { return context_; }
clients::body::Json list_buckets(int max_results, const std::string& prefix,
const std::string& page_token) const;
clients::body::Json get_bucket(const std::string& bucket) const;
clients::body::Json create_bucket(const std::string& bucket, const std::string& location,
const std::string& storage_class) const;
clients::body::Json delete_bucket(const std::string& bucket) const;
clients::body::Json list_objects(const std::string& bucket, const std::string& prefix,
const std::string& delimiter, int max_results,
const std::string& page_token) const;
clients::body::Json get_object(const std::string& bucket, const std::string& object) const;
clients::body::Json patch_object(const std::string& bucket, const std::string& object,
const clients::body::Json& fields) const;
clients::body::Json delete_object(const std::string& bucket, const std::string& object) const;
clients::body::Json copy_object(const std::string& source_bucket, const std::string& source,
const std::string& dest_bucket, const std::string& dest) const;
// objects.compose: 1..32 sources in one bucket into `destination`.
clients::body::Json compose(const std::string& bucket, const std::vector<ComposeSource>& sources,
const std::string& destination,
const std::string& content_type) const;
static clients::body::Json compose_body(const std::vector<ComposeSource>& sources,
const std::string& content_type);
clients::body::Json bucket_iam_policy(const std::string& bucket) const;
clients::body::Json service_account() const; // projects.serviceAccount.get
private:
CloudContext context_;
};
} // namespace molo::google::data
+231
View File
@@ -0,0 +1,231 @@
#include "google/data/storage_media.h"
#include "clients/transports/http_transport.h"
#include "clients/url.h"
#include "google/credentials.h"
#include "google/data/caps.h"
#include "google/data/ids.h"
namespace molo::google::data {
using clients::body::Json;
namespace {
clients::ClientConfig config() {
clients::ClientConfig c;
c.persist_cookies = false;
return c;
}
std::string enc(const std::string& s) { return clients::Url::encode_component(s); }
void check_type(const std::string& t) {
if (t.empty() || t.size() > 128) throw GoogleError::invalid("content type must be 1..128 chars");
for (char c : t)
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f)
throw GoogleError::invalid("content type has control characters");
}
Json parse(const clients::ClientResponse& r) {
try {
return r.json();
} catch (const std::exception& e) {
throw GoogleError(502, "BAD_RESPONSE", e.what(), "storage");
}
}
} // namespace
StorageMedia::StorageMedia(std::unique_ptr<clients::Transport> transport,
std::shared_ptr<clients::Auth> auth, std::string host_url)
: client_(std::make_unique<clients::Client>(std::move(transport), config())),
auth_(std::move(auth)),
host_(std::move(host_url)) {}
std::shared_ptr<StorageMedia> StorageMedia::from_env() {
std::shared_ptr<clients::Auth> auth = Credentials::oauth_from_env();
if (!auth) auth = Credentials::api_key_from_env();
return std::make_shared<StorageMedia>(std::make_unique<clients::transports::HttpTransport>(),
auth);
}
std::string StorageMedia::download_url(const std::string& bucket, const std::string& object,
long long generation) const {
if (generation < 0) throw GoogleError::invalid("generation must be >= 0");
std::string u = host_ + "/storage/v1/b/" + enc(Ids::bucket(bucket)) + "/o/" +
enc(Ids::object(object)) + "?alt=media";
if (generation > 0) u += "&generation=" + std::to_string(generation);
return u;
}
std::string StorageMedia::upload_url(const std::string& bucket, const std::string& object,
const char* upload_type) const {
return host_ + "/upload/storage/v1/b/" + enc(Ids::bucket(bucket)) + "/o?uploadType=" +
upload_type + "&name=" + enc(Ids::object(object));
}
std::string StorageMedia::check_session(const std::string& url) {
const std::string prefix = std::string(kHost) + "/upload/storage/v1/b/";
const std::string alt = std::string(kHost) + "/resumable/upload/storage/v1/b/";
if ((url.rfind(prefix, 0) != 0 && url.rfind(alt, 0) != 0) || url.size() > 2048)
throw GoogleError::invalid("session url must be on " + std::string(kHost) +
"/upload/storage/v1/b/");
for (char c : url)
if (static_cast<unsigned char>(c) <= 0x20 || c == 0x7f)
throw GoogleError::invalid("session url has control characters or spaces");
return url;
}
clients::ClientResponse StorageMedia::send(clients::ClientRequest request) const {
if (!auth_) throw NotConfiguredError("storage");
try {
auth_->apply(request);
return client_->send_blocking(std::move(request));
} catch (const GoogleError&) {
throw;
} catch (const std::exception& e) {
throw GoogleError(502, "UNAVAILABLE", e.what(), "storage");
}
}
void StorageMedia::fail(const clients::ClientResponse& r) const {
throw error_from_response("storage", r);
}
Downloaded StorageMedia::download(const std::string& bucket, const std::string& object,
long long generation) const {
clients::ClientRequest req(clients::Method::kGet,
clients::Url::parse(download_url(bucket, object, generation)));
clients::ClientResponse r = send(std::move(req));
if (!r.is_success()) fail(r);
if (r.body().size() > DataCaps::kMaxDownloadBytes)
throw GoogleError(502, "RESPONSE_TOO_LARGE",
"object is larger than the " + std::to_string(DataCaps::kMaxDownloadBytes) +
" byte download cap; use download_range",
"storage");
Downloaded d;
d.data = r.body();
d.content_type = r.content_type();
d.generation = r.headers().get("x-goog-generation").value_or("");
return d;
}
Downloaded StorageMedia::download_range(const std::string& bucket, const std::string& object,
long long first, long long last) const {
if (first < 0 || last < first ||
last - first + 1 > static_cast<long long>(DataCaps::kMaxDownloadBytes))
throw GoogleError::invalid("range must satisfy 0 <= first <= last and span <= " +
std::to_string(DataCaps::kMaxDownloadBytes) + " bytes");
clients::ClientRequest req(clients::Method::kGet,
clients::Url::parse(download_url(bucket, object, 0)));
req.header("Range", "bytes=" + std::to_string(first) + "-" + std::to_string(last));
clients::ClientResponse r = send(std::move(req));
if (!r.is_success()) fail(r);
if (r.body().size() > DataCaps::kMaxDownloadBytes)
throw GoogleError(502, "RESPONSE_TOO_LARGE", "range response over the cap", "storage");
Downloaded d;
d.data = r.body();
d.content_type = r.content_type();
d.generation = r.headers().get("x-goog-generation").value_or("");
return d;
}
Json StorageMedia::upload_simple(const std::string& bucket, const std::string& object,
const std::string& data, const std::string& content_type) const {
if (data.size() > DataCaps::kMaxSimpleUploadBytes)
throw GoogleError::invalid("simple upload over " +
std::to_string(DataCaps::kMaxSimpleUploadBytes) +
" bytes; use the resumable upload");
check_type(content_type);
clients::ClientRequest req(clients::Method::kPost,
clients::Url::parse(upload_url(bucket, object, "media")));
req.bytes(data, content_type);
clients::ClientResponse r = send(std::move(req));
if (!r.is_success()) fail(r);
return parse(r);
}
std::string StorageMedia::resumable_start(const std::string& bucket, const std::string& object,
const std::string& content_type, long long total,
const Json& metadata) const {
if (total < 0 || total > static_cast<long long>(DataCaps::kMaxResumableTotal))
throw GoogleError::invalid("total must be 0.." + std::to_string(DataCaps::kMaxResumableTotal));
check_type(content_type);
if (!metadata.is_null() && !metadata.is_object())
throw GoogleError::invalid("metadata must be a JSON object");
clients::ClientRequest req(clients::Method::kPost,
clients::Url::parse(upload_url(bucket, object, "resumable")));
req.header("X-Upload-Content-Type", content_type);
req.header("X-Upload-Content-Length", std::to_string(total));
req.json(metadata.is_object() ? metadata : Json::object());
clients::ClientResponse r = send(std::move(req));
if (!r.is_success()) fail(r);
std::string loc = r.headers().get("location").value_or("");
if (loc.empty())
throw GoogleError(502, "BAD_RESPONSE", "resumable start returned no Location", "storage");
return check_session(loc);
}
ResumableProgress StorageMedia::resumable_put(const std::string& session_url, long long offset,
const std::string& chunk, long long total) const {
check_session(session_url);
if (total < 0 || total > static_cast<long long>(DataCaps::kMaxResumableTotal))
throw GoogleError::invalid("total out of range");
if (offset < 0 || chunk.empty() || chunk.size() > DataCaps::kMaxResumableChunk)
throw GoogleError::invalid("chunk must be 1.." + std::to_string(DataCaps::kMaxResumableChunk) +
" bytes at a non-negative offset");
long long end = offset + static_cast<long long>(chunk.size());
if (end > total) throw GoogleError::invalid("chunk runs past total");
if (end < total && chunk.size() % DataCaps::kChunkGranularity != 0)
throw GoogleError::invalid("a chunk that does not end the object must be a multiple of 256 KiB");
clients::ClientRequest req(clients::Method::kPut, clients::Url::parse(session_url));
req.header("Content-Range", "bytes " + std::to_string(offset) + "-" + std::to_string(end - 1) +
"/" + std::to_string(total));
req.bytes(chunk, "application/octet-stream");
clients::ClientResponse r = send(std::move(req));
ResumableProgress p;
if (r.status() == 308) {
std::string range = r.headers().get("range").value_or(""); // "bytes=0-N"
auto dash = range.find('-');
p.next_offset = dash == std::string::npos ? 0 : std::stoll(range.substr(dash + 1)) + 1;
return p;
}
if (!r.is_success()) fail(r);
p.done = true;
p.next_offset = total;
p.object = parse(r);
return p;
}
ResumableProgress StorageMedia::resumable_status(const std::string& session_url,
long long total) const {
check_session(session_url);
if (total < 0 || total > static_cast<long long>(DataCaps::kMaxResumableTotal))
throw GoogleError::invalid("total out of range");
clients::ClientRequest req(clients::Method::kPut, clients::Url::parse(session_url));
req.header("Content-Range", "bytes */" + std::to_string(total));
req.bytes("", "application/octet-stream");
clients::ClientResponse r = send(std::move(req));
ResumableProgress p;
if (r.status() == 308) {
std::string range = r.headers().get("range").value_or("");
auto dash = range.find('-');
p.next_offset = dash == std::string::npos ? 0 : std::stoll(range.substr(dash + 1)) + 1;
return p;
}
if (!r.is_success()) fail(r);
p.done = true;
p.next_offset = total;
p.object = parse(r);
return p;
}
void StorageMedia::resumable_cancel(const std::string& session_url) const {
check_session(session_url);
clients::ClientRequest req(clients::Method::kDelete, clients::Url::parse(session_url));
clients::ClientResponse r = send(std::move(req));
if (r.status() != 499 && !r.is_success()) fail(r);
}
} // namespace molo::google::data
+83
View File
@@ -0,0 +1,83 @@
#pragma once
// Cloud Storage media: object download (alt=media), simple upload
// (uploadType=media) and resumable upload (uploadType=resumable). These carry
// raw bytes, which GoogleClient (JSON only) cannot, so this class owns a
// clients::Client with the same credential. The credential is only ever sent to
// storage.googleapis.com: a resumable session URL is checked against that host.
#include <memory>
#include <string>
#include "clients/auth.h"
#include "clients/body/json.h"
#include "clients/client.h"
#include "google/errors.h"
namespace molo::google::data {
struct Downloaded {
std::string data;
std::string content_type;
std::string generation; // x-goog-generation header
};
struct ResumableProgress {
bool done = false;
long long next_offset = 0; // first byte Google does not have yet
clients::body::Json object; // object resource when done
};
class StorageMedia {
public:
// auth may be null: every call then throws NotConfiguredError.
StorageMedia(std::unique_ptr<clients::Transport> transport,
std::shared_ptr<clients::Auth> auth, std::string host_url = kHost);
// HTTPS transport + GOOGLE_ACCESS_TOKEN / service account / API key from env.
static std::shared_ptr<StorageMedia> from_env();
static constexpr const char* kHost = "https://storage.googleapis.com";
bool configured() const { return auth_ != nullptr; }
// GET .../o/{object}?alt=media, optional generation (0 = current). Capped at
// DataCaps::kMaxDownloadBytes; over the cap is an error, never a truncation.
Downloaded download(const std::string& bucket, const std::string& object,
long long generation = 0) const;
// Byte range [first, last] (inclusive), one request.
Downloaded download_range(const std::string& bucket, const std::string& object,
long long first, long long last) const;
// POST /upload/storage/v1/b/{bucket}/o?uploadType=media&name=... (<= 8 MiB).
clients::body::Json upload_simple(const std::string& bucket, const std::string& object,
const std::string& data,
const std::string& content_type) const;
// Resumable: start returns the session URL; put sends one chunk.
// total is the full object size (known up front, <= 64 MiB).
std::string resumable_start(const std::string& bucket, const std::string& object,
const std::string& content_type, long long total,
const clients::body::Json& metadata) const;
// chunk size: multiple of 256 KiB unless it ends the object; <= 8 MiB.
ResumableProgress resumable_put(const std::string& session_url, long long offset,
const std::string& chunk, long long total) const;
ResumableProgress resumable_status(const std::string& session_url, long long total) const;
void resumable_cancel(const std::string& session_url) const;
// Request builders (pure, used by the methods above and by tests).
std::string download_url(const std::string& bucket, const std::string& object,
long long generation) const;
std::string upload_url(const std::string& bucket, const std::string& object,
const char* upload_type) const;
static std::string check_session(const std::string& url);
private:
clients::ClientResponse send(clients::ClientRequest request) const;
[[noreturn]] void fail(const clients::ClientResponse& r) const;
mutable std::unique_ptr<clients::Client> client_;
std::shared_ptr<clients::Auth> auth_;
std::string host_;
};
} // namespace molo::google::data
+185
View File
@@ -0,0 +1,185 @@
#!/usr/bin/env python3
"""Check src/google/data against Google's Discovery documents.
Fails (exit 1) when
* a row of calls_table.inc differs from Discovery (HTTP method, path, request
body fields, query parameters, required query parameters, path placeholders),
* a Discovery method of an owned API has no row, or a row has no method
(coverage is total: every method of the 11 APIs is a row),
* a call id used by the C++ sources (string literal "api.resource.method") is
not a row,
* a nested schema field the typed code writes (SCHEMAS below) is absent,
* the Cloud Storage media contract (upload path, protocols, alt=media) changed.
Python 3 stdlib only, no credentials:
python3 -I src/google/data/tools/check_discovery.py
"""
import json
import os
import re
import sys
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
DATA = os.path.join(HERE, "..")
TABLE = os.path.join(DATA, "calls_table.inc")
APIS = [("bigquery", "v2"), ("storage", "v1"), ("pubsub", "v1"), ("firestore", "v1"),
("spanner", "v1"), ("bigtableadmin", "v2"), ("secretmanager", "v1"),
("dataplex", "v1"), ("dlp", "v2"), ("cloudkms", "v1"), ("datastore", "v1")]
PREFIXES = tuple(n + "." for n, _ in APIS)
# Nested schema fields the typed classes write: (api, schema, fields)
SCHEMAS = [
("bigquery", "QueryRequest", "query,useLegacySql,useQueryCache,dryRun,maxResults,timeoutMs,"
"location,defaultDataset,maximumBytesBilled,queryParameters,parameterMode"),
("bigquery", "DatasetReference", "projectId,datasetId"),
("bigquery", "Dataset", "datasetReference,location,description"),
("bigquery", "TableDataInsertAllRequest", "rows,skipInvalidRows,ignoreUnknownValues"),
("pubsub", "PubsubMessage", "data,attributes,orderingKey"),
("pubsub", "PublishRequest", "messages"),
("pubsub", "PullRequest", "maxMessages"),
("pubsub", "AcknowledgeRequest", "ackIds"),
("dlp", "GooglePrivacyDlpV2InspectConfig", "infoTypes,minLikelihood,includeQuote,limits"),
("dlp", "GooglePrivacyDlpV2FindingLimits", "maxFindingsPerRequest"),
("dlp", "GooglePrivacyDlpV2ContentItem", "value"),
("dlp", "GooglePrivacyDlpV2InfoType", "name"),
("dlp", "GooglePrivacyDlpV2InspectContentRequest", "item,inspectConfig"),
("dlp", "GooglePrivacyDlpV2DeidentifyContentRequest", "item,deidentifyConfig"),
("spanner", "ExecuteSqlRequest", "sql,queryMode,transaction,params,paramTypes"),
("spanner", "TransactionSelector", "singleUse"),
("spanner", "TransactionOptions", "readOnly"),
("spanner", "ReadOnly", "strong"),
("secretmanager", "Secret", "replication"),
("secretmanager", "Replication", "automatic"),
("secretmanager", "SecretPayload", "data"),
("cloudkms", "EncryptRequest", "plaintext,additionalAuthenticatedData"),
("cloudkms", "DecryptRequest", "ciphertext,additionalAuthenticatedData"),
("cloudkms", "AsymmetricSignRequest", "digest"),
("cloudkms", "Digest", "sha256"),
("datastore", "TransactionOptions", "readOnly,readWrite"),
("datastore", "CommitRequest", "mutations,mode,transaction,databaseId"),
("datastore", "LookupRequest", "keys,databaseId"),
("firestore", "BatchGetDocumentsRequest", "documents"),
("firestore", "ListCollectionIdsRequest", "pageSize"),
("firestore", "RunQueryRequest", "structuredQuery"),
("firestore", "RunAggregationQueryRequest", "structuredAggregationQuery"),
("firestore", "CommitRequest", "writes"),
("firestore", "Document", "fields"),
("storage", "ComposeRequest", "sourceObjects,destination"),
("storage", "Bucket", "name,location,storageClass"),
]
ROW = re.compile(r'^\s*\{("(?:[^"\\]|\\.)*"(?:,\s*"(?:[^"\\]|\\.)*"){5})\},\s*$')
STR = re.compile(r'"((?:[^"\\]|\\.)*)"')
def fetch(name, version):
url = "https://%s.googleapis.com/$discovery/rest?version=%s" % (name, version)
with urllib.request.urlopen(url, timeout=90) as r:
return json.load(r)
def methods(node, out):
for m in node.get("methods", {}).values():
out[m["id"]] = m
for child in node.get("resources", {}).values():
methods(child, out)
def main():
rows = []
for line in open(TABLE):
m = ROW.match(line)
if m:
rows.append(STR.findall(m.group(1)))
bad = []
docs, index = {}, {}
for name, version in APIS:
docs[name] = fetch(name, version)
index[name] = {}
methods(docs[name], index[name])
seen = set()
for rpc, http, path, body, query, required in rows:
api = rpc.split(".")[0]
m = index.get(api, {}).get(rpc)
seen.add(rpc)
if not m:
bad.append("missing rpc " + rpc)
continue
if m["httpMethod"] != http:
bad.append("%s: http %s != %s" % (rpc, http, m["httpMethod"]))
if m["path"] != path:
bad.append("%s: path %s != %s" % (rpc, path, m["path"]))
params = m.get("parameters", {})
want_q = sorted(k for k, v in params.items() if v.get("location") == "query")
if sorted(filter(None, query.split(","))) != want_q:
bad.append("%s: query params differ" % rpc)
want_r = sorted(k for k in want_q if params[k].get("required"))
if sorted(filter(None, required.split(","))) != want_r:
bad.append("%s: required query params differ" % rpc)
for p in re.findall(r"\{\+?([A-Za-z0-9_]+)\}", path):
if p not in params or params[p].get("location") != "path":
bad.append("%s: path placeholder %s is not a path parameter" % (rpc, p))
ref = m.get("request", {}).get("$ref")
props = sorted(docs[api]["schemas"].get(ref, {}).get("properties", {})) if ref else []
if sorted(filter(None, body.split(","))) != props:
bad.append("%s: body fields differ from %s" % (rpc, ref))
for api, idx in index.items():
for rpc in idx:
if rpc not in seen:
bad.append("coverage: Discovery method %s has no row" % rpc)
# call ids used by the sources
lit = re.compile(r'"((?:%s)[A-Za-z0-9.]+)"' % "|".join(re.escape(p) for p in PREFIXES))
used = 0
for fn in sorted(os.listdir(DATA)):
if not fn.endswith(".cpp") or fn == "calls.cpp":
continue
for rpc in lit.findall(open(os.path.join(DATA, fn)).read()):
if rpc.endswith(".googleapis.com"):
continue # a host name, not a call id
used += 1
if rpc not in seen:
bad.append("%s uses call id %s which is not a row" % (fn, rpc))
# nested schemas
for api, schema, fields in SCHEMAS:
props = docs[api]["schemas"].get(schema, {}).get("properties")
if props is None:
bad.append("missing schema %s.%s" % (api, schema))
continue
for f in fields.split(","):
if f not in props:
bad.append("schema %s.%s lacks field %s" % (api, schema, f))
# Cloud Storage media contract
st = index["storage"]
ins, get = st["storage.objects.insert"], st["storage.objects.get"]
mu = ins.get("mediaUpload", {})
# simple: Discovery path used verbatim. resumable: Discovery lists
# /resumable/upload/...; the code starts sessions on the documented
# /upload/...?uploadType=resumable (same host, see SPEC Unverified).
proto = mu.get("protocols", {})
if not ins.get("supportsMediaUpload") or \
proto.get("simple", {}).get("path") != "/upload/storage/v1/b/{bucket}/o" or \
proto.get("resumable", {}).get("path") != "/resumable/upload/storage/v1/b/{bucket}/o":
bad.append("storage media upload contract changed")
if mu.get("maxSize") and int(re.sub(r"\D", "", mu["maxSize"]) or 0) < 1:
bad.append("storage upload maxSize unreadable")
alt = get["parameters"].get("alt") or docs["storage"]["parameters"].get("alt", {})
if not get.get("supportsMediaDownload") or "media" not in alt.get("enum", []):
bad.append("storage objects.get media download contract changed")
for q in ("name", "uploadType"):
if q not in ins["parameters"] and q not in docs["storage"]["parameters"]:
bad.append("storage objects.insert lacks parameter %s" % q)
for api, version in APIS:
print("%s %s revision %s: %d methods" % (api, version, docs[api]["revision"], len(index[api])))
if bad:
print("\n".join(bad[:60]))
print("FAILED: %d problems" % len(bad))
return 1
print("discovery ok: %d rows, %d call ids used by sources, %d nested schemas, storage media contract"
% (len(rows), used, len(SCHEMAS)))
return 0
if __name__ == "__main__":
sys.exit(main())
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""Generate the call table of src/google/data/calls.cpp from Discovery.
python3 -I src/google/data/tools/gen_calls.py > src/google/data/calls_table.inc
Every method of the 11 owned APIs becomes one row:
{rpc id, HTTP method, discovery path, body fields, query params, required query params}
body fields are all top-level properties of the request schema; query params
are all parameters with location=query. Python 3 stdlib only, no credentials.
"""
import json
import sys
import urllib.request
APIS = [("bigquery", "v2"), ("storage", "v1"), ("pubsub", "v1"), ("firestore", "v1"),
("spanner", "v1"), ("bigtableadmin", "v2"), ("secretmanager", "v1"),
("dataplex", "v1"), ("dlp", "v2"), ("cloudkms", "v1"), ("datastore", "v1")]
def fetch(name, version):
url = "https://%s.googleapis.com/$discovery/rest?version=%s" % (name, version)
with urllib.request.urlopen(url, timeout=60) as r:
return json.load(r)
def methods(node, out):
for m in node.get("methods", {}).values():
out.append(m)
for child in node.get("resources", {}).values():
methods(child, out)
def main():
print("// GENERATED by tools/gen_calls.py from Google Discovery; do not edit by hand.")
for name, version in APIS:
doc = fetch(name, version)
print("// %s %s revision %s" % (name, version, doc["revision"]))
found = []
methods(doc, found)
for m in sorted(found, key=lambda x: x["id"]):
params = m.get("parameters", {})
query = sorted(k for k, v in params.items() if v.get("location") == "query")
req = sorted(k for k in query if params[k].get("required"))
ref = m.get("request", {}).get("$ref")
body = sorted(doc["schemas"].get(ref, {}).get("properties", {})) if ref else []
print(' {"%s", "%s", "%s", "%s", "%s", "%s"},' % (
m["id"], m["httpMethod"], m["path"], ",".join(body), ",".join(query), ",".join(req)))
if __name__ == "__main__":
sys.exit(main())