Merge sandbox module (Agent Sandbox, Workspaces)

This commit is contained in:
2026-10-24 04:36:57 +02:00
35 changed files with 3659 additions and 3 deletions
+245
View File
@@ -0,0 +1,245 @@
# Agent Sandbox - Specification
Owned directory: `src/google/sandbox/`. Namespace `molo::google::sandbox`.
Depends on the Google foundation (`GoogleClient`, `Service`, `GoogleError`,
`wait_operation`, `list_all`, `CloudContext`) and the Base64 codec. No other
module is included.
Purpose: create, inspect, pause, snapshot and delete Agent Sandbox
environments, run model-generated code, shell commands and browser actions in
them, and move files in and out, with typed results and hard caps.
## 1. Scope and what exists
Google's REST surface for this product is the `sandboxEnvironments` family
under `projects.locations.reasoningEngines` (Discovery service `aiplatform`,
versions `v1` and `v1beta1`). It has three resources: `sandboxEnvironments`
(create, get, list, delete, pause, resume, snapshot, authorizeAccess,
execute, bidiExecute), `sandboxEnvironmentTemplates` (create, get, list,
delete) and `sandboxEnvironmentSnapshots` (get, list, delete). A sandbox's
category is one of `codeExecutionEnvironment` (Python or JavaScript),
`computerUseEnvironment` (browser) or `shellEnvironment` (bash).
**Workspaces.** No resource, method or schema named "workspace" exists in the
`aiplatform` Discovery documents (v1 and v1beta1, checked 2026-10-24) and the
sandbox docs index lists no workspace page. The product-level "Workspaces"
(bash commands and file management isolated from core systems) is what the
docs call the **shell sandbox**: `execute_bash` returns `stdout`, `stderr`,
`returncode`, `duration_ms`, runs as an unprivileged user in `/workspace`, one
fresh shell per command. This module implements that as `ShellRunner`.
File management has no REST surface: `WorkspaceFiles` is a **local
abstraction** that does file work with shell commands through `ShellRunner`.
It is named local everywhere and is not a Google API.
**Computer use.** The documented actions (navigate, click, type, screenshot)
are Chrome DevTools Protocol commands sent to the sandbox data plane with a
service-account JWT (`generate_access_token`, `send_command`, `/cdp`,
`/tabs`, a CDP WebSocket for Playwright, VNC view). The data plane is not in
Discovery. `ComputerUse` carries the same CDP commands through
`sandboxEnvironments.execute`, the only sandbox call in the REST surface; that
mapping is unverified (section 7).
## 2. Module map (one class or concern per file)
| File | Concern |
|---|---|
| `caps.h` | `SandboxCaps`: static ceilings |
| `calls.h` | The one table of every call, schema field and enum used; parsed by `tools/check_discovery.py` |
| `names.h/.cpp` | `SandboxNames` (engine, sandbox, template, snapshot names), file name and workspace path validation |
| `types.h/.cpp` | `SandboxSpec`, `CreateSandboxRequest`, `SandboxEnvironment`, `InputFile`, `ProducedFile`, `ExecutionResult` |
| `chunk.h/.cpp` | `Chunk` wire type and `ChunkCodec` (request and result mapping, output caps) |
| `sandbox_service.h/.cpp` | `SandboxService` base: endpoint per location and version, shared `execute` |
| `sandboxes.h/.cpp` | `Sandboxes`: create, get, list, pause, resume, snapshot, authorize_access, remove |
| `snapshot_types.h`, `snapshots.h/.cpp` | `SandboxSnapshot`, `Snapshots`: get, list, remove |
| `templates.h/.cpp` | `SandboxTemplate`, `Templates`: create, get, list, remove |
| `code_executor.h/.cpp` | `CodeExecutor`: code plus input files in, typed result out |
| `shell_runner.h/.cpp` | `ShellRunner`: bash command with cwd and timeout |
| `workspace_files.h/.cpp` | `WorkspaceFiles` (local): write, read, list, remove_file |
| `computer_use.h/.cpp` | `ComputerUse`: navigate, click, type_text, screenshot |
| `stream_executor.h/.cpp` | `StreamExecutor`: `bidiExecute` (v1beta1), streamed reply |
| `tools/check_discovery.py` | Discovery check for `calls.h` |
## 3. Static caps (see caps.h)
| Knob | Value |
|---|---|
| Code per call | 256 KiB |
| Shell command | 8192 bytes (workspace helpers: 65536, internal only) |
| Input files per call / per file / total | 16 / 4 MiB / 8 MiB |
| File name | 1..255, no leading `/`, no `.` or `..` segment, no backslash or control byte |
| Workspace path | 256 chars of `[A-Za-z0-9._/-]`, under `/workspace`, no `..` |
| Shell file write / read | 32 KiB / 512 KiB (bytes travel in the command line) |
| stdout, stderr each | 1 MiB, truncated and flagged |
| Output chunks per reply | 64 |
| Produced files / per file / total | 16 / 4 MiB / 8 MiB, extras skipped and named in `skipped` |
| Screenshot | 4 MiB |
| Shell timeout | default 30 s, max 300 s |
| Sandbox TTL | 60 s to 14 days (Google keeps state up to 14 days), default 3600 s |
| Display name | 1..128 printable |
| Resource name | 512 chars |
| List page size / pages | 100 (Google default and ceiling) / 10 |
| Template ports / disk | 16 / 1..1024 GB |
| URL / typed text / coordinate | 2048 / 4096 / 0..10000 |
| Stream events | 256 |
| LRO wait | foundation `wait_operation`: 600 s, 400 polls |
Google's own limit is 100 MB of files per request or response. Ours are
tighter because file bytes are base64 inside a JSON payload that is base64
again inside the Chunk, and the codec caps base64 input at 16 MiB.
## 4. Contracts
- Every name and id is validated before any network traffic; bad input
throws `GoogleError` 400 `INVALID_ARGUMENT`. No credential: calls throw
`NotConfiguredError` (503) from the client before any traffic.
- API versions: `v1` for everything it offers; `v1beta1` only where `v1`
lacks the field: creating a computer use sandbox (`computerUseEnvironment`),
templates (`persistentDiskConfig`, computer use category in the documented
flow), and `bidiExecute`.
- Create, pause, resume, snapshot and delete are long-running operations;
they are waited on (capped) and return the typed resource. Delete is an
explicit method (`Sandboxes::remove`, `Templates::remove`,
`Snapshots::remove`, `WorkspaceFiles::remove_file`); nothing deletes as a
side effect. `remove_file` removes one regular file, never recursively.
- Sandboxes are billed while they exist: the TTL is always sent on create
(default 1 hour) so a crashed caller does not leak a sandbox. `execute`
resets the TTL (per docs).
- `ExecutionResult`: `std_out`, `std_err`, `exit_status` (shell only,
otherwise empty), `duration_ms`, `files`, truncation flags, `skipped`.
Over-cap or unsafe files are never dropped silently: each gets a reason in
`skipped`. `ok()` is true when the exit status is absent or 0.
- `Chunk.data` and `Metadata.attributes` values are base64 on the wire
(Discovery: `format: byte`); the codec exposes raw bytes.
- Computer use accepts only `http://` and `https://` URLs.
## 5. Verified against discovery
`python3 -I src/google/sandbox/tools/check_discovery.py` fetches
`https://aiplatform.googleapis.com/$discovery/rest?version=v1` and `v1beta1`
and fails when any row of `calls.h` is absent: 18 calls (rpc id, HTTP method,
path, request body field paths including nested ones), 21 schemas (field
names) and 7 enums (values).
Result on 2026-10-24: `18 calls, 21 schemas, 7 enums, 204 checks, 0 missing`.
Findings from the check that shaped the code: `computerUseEnvironment`,
`useGkeTd` and `persistentDiskConfig` exist only in `v1beta1`; `bidiExecute`
exists only in `v1beta1`; `PauseSandboxEnvironmentRequest`,
`ResumeSandboxEnvironmentRequest` and `AuthorizeSandboxEnvironmentAccessRequest`
have no fields; the snapshot request body is a `SandboxEnvironmentSnapshot`
whose `sourceSandboxEnvironment` is required.
## 6. Verified live
Read-only GETs on 2026-10-24, project from `GOOGLE_CLOUD_PROJECT`, region
us-central1, nothing created. Two credentials were tried.
OAuth bearer token (service account, `GOOGLE_ACCESS_TOKEN`):
| Call | Status and body |
|---|---|
| `GET v1/projects/P/locations/L/reasoningEngines` | 200 `{}` (no engines exist) |
| `GET v1beta1/.../reasoningEngines` | 200 `{}` |
| `GET v1/.../reasoningEngines/{absent}/sandboxEnvironments` | 404 NOT_FOUND "Reasoning Engine [project number: N, reasoning engine id: I] is not found." |
| same path on `v1/.../sandboxEnvironmentSnapshots`, `v1/.../sandboxEnvironmentTemplates`, `v1beta1/.../sandboxEnvironments`, `v1beta1/.../sandboxEnvironmentTemplates` | 404 NOT_FOUND, same message |
| `GET v1/...` and `v1beta1/.../sandboxEnvironments/{absent}` | 404 NOT_FOUND "Sandbox Environment (projectNumber: N, reasoningEngineId: I, sandboxEnvironmentId: S) is not found." |
No SERVICE_DISABLED or PERMISSION_DENIED came back: the Agent Platform API is
enabled on the project and the service account may read these collections.
The 404s are resource-level (Google names the missing engine or sandbox), so
the route, version and name format are real. There is nothing to read beyond
that: the project has no reasoning engine, hence no sandbox, template or
snapshot, and none may be created here.
Express API key (`AQ.`, `x-goog-api-key`): the same probes return the same
statuses and bodies, so the key reaches these read routes too. A made-up
collection (`.../bogusThings`) answers an HTML 404 page, and a non-numeric
engine id (`none`) answers 400 INVALID_ARGUMENT; engine ids are numeric.
Never exercised live (Discovery only): create, pause, resume, snapshot,
delete, execute, bidiExecute, authorizeAccess, template create and delete,
and every response shape of an existing sandbox. Whether a key may mutate
sandboxes is unknown.
Rerun: `MOLO_LIVE_NET=1 GOOGLE_ACCESS_TOKEN=... GOOGLE_CLOUD_PROJECT=... make
google-test` (a key in `GOOGLE_API_KEY` also works). Test
`google_live_sandbox.read_only_probe` prints a loud SKIP otherwise. It lists
sandboxes of an engine id that cannot exist and gets a sandbox that cannot
exist, expecting 404 NOT_FOUND through the real client; it creates nothing.
It passed on 2026-10-24 with the OAuth token and with the key.
Gemini's built-in `codeExecution` tool inside `generateContent` belongs to the
models module and is not used here.
## 7. Unverified
Neither Discovery nor the docs state these; each is inferred and isolated in
`ChunkCodec` so one edit fixes it:
- The payload of an input Chunk. Docs show only the SDK call
(`execute_code(input_data={"code":..., "files":[{"name","content"}]})`) and
an older doc revision says input file content is base64. Assumed: one
`application/json` Chunk holding `{"code", "files":[{"name","content"(base64)}]}`.
- The payload of a shell input Chunk. Assumed `{"command","cwd","timeout"}`
(names from the SDK helper `execute_bash(command, cwd, timeout)`).
- The payload of a computer use input Chunk: `{"command":"Page.navigate",
"params":{...}}` is the documented `/cdp` body; sending it through `execute`
is assumed. Click is `Input.dispatchMouseEvent` pressed then released, typing
is `Input.insertText`, screenshot is `Page.captureScreenshot` (CDP names, not
Google names).
- Output Chunk payloads: docs show an `application/json` Chunk with `msg_out`,
`msg_err`, `output_files[{content (base64)}]`, plus Chunks with a `file_name`
attribute. The `name` key inside `output_files` and the shell keys `stdout`,
`stderr`, `returncode`, `duration_ms` (seen only in the SDK dict) are
assumed. Unknown shapes are skipped loudly, never guessed.
- Whether `attributes` values are really base64 on the wire (Discovery says
`byte`; docs show plain text). The decoder tries base64 and falls back to the
text as written.
- Whether `bidiExecute` over this client's POST stream behaves as a server
stream; it is a bidirectional method and may need a persistent channel.
- Which responses carry which `@type` inside an Operation `response`
(`SandboxEnvironment`, `SandboxEnvironmentSnapshot`, template); parsed by
field name only.
- The `filter` list parameter exists (AIP-160) but its grammar is not
documented; it is not exposed.
## 8. Gaps
- The sandbox data plane (HTTP proxy with JWT, `/tabs`, `/cdp`, CDP
WebSocket, VNC) is not implemented; it needs a service-account JWT signer
(foundation gap in `credentials.cpp`) and a WebSocket client (module `live`).
Owner of the follow-up: this module, `computer_use.cpp`.
- True bidirectional `bidiExecute` input (see Unverified).
- `generate_access_token` and `generate_browser_ws_headers` (SDK helpers) have
no REST method in Discovery.
- Template update, custom container resources/requests, DNS peering, private
service connect ingress, CMEK and VPC-SC settings are not typed. Templates
keep the whole document in `SandboxTemplate::raw`.
- Foundation: `Service::call` always posts to the one endpoint a service was
built with; this module needs a different location and version per call, so
`SandboxService` builds endpoints itself and talks to `GoogleClient`
directly.
- The example server's flat JSON reader does not decode `\n`; the demo uses
the real JSON parser for request bodies.
## 9. Sources
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/code-execution-overview
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/code-execution-quickstart
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/computer-use
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/computer-use-quickstart
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/shell-sandbox-quickstart
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/manage-sandboxes
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1beta1/projects.locations.reasoningEngines.sandboxEnvironments (and its `create`, `execute`, `list`, `delete`, `pause`, `authorizeAccess` pages) and `.../v1beta1/Chunk`
- https://aiplatform.googleapis.com/$discovery/rest?version=v1
- https://aiplatform.googleapis.com/$discovery/rest?version=v1beta1
- Not read (listed on the docs index): `manage-templates`, `manage-snapshots`,
`configure-vpc-sc`, `configure-cmek`, `custom-containers`,
`troubleshooting/code-execution`.
## 10. Example
`example/app/agent/SandboxDemo.cpp`, page script `example/assets/agent_sandbox.js`.
GET `/agent/sandbox` (page, matrix row `agent-sandbox`), GET
`/agent/sandbox/status`. POST `/agent/sandbox/{create,list,get,pause,resume,
snapshot,snapshots,templates,execute,shell,computer,files}`, DELETE
`/agent/sandbox/delete?name=`. Without a credential every Google route is 503
`google_not_configured`; bad input is 422. Tests: `tests/example/agent/sandbox/`.
+97
View File
@@ -0,0 +1,97 @@
#pragma once
// The ONE table of every Google call this module makes, plus the schema
// fields and enum values it reads or writes. tools/check_discovery.py parses
// this file (one brace row per line, double-quoted strings only) and fails if
// the Discovery document of the named service and version lacks any row.
// Keep every row on one line.
namespace molo::google::sandbox {
struct DeclaredCall {
const char* name; // Local name.
const char* http; // HTTP method.
const char* path; // Discovery "path" without the version prefix.
const char* service; // Discovery service host, e.g. aiplatform.
const char* version; // v1 or v1beta1.
const char* rpc; // Discovery method id.
const char* request; // Comma list of request body field paths ("" none).
};
struct DeclaredSchema {
const char* service;
const char* version;
const char* schema; // Discovery schema id without the GoogleCloudAiplatform<Version> prefix.
const char* fields; // Comma list of properties that must exist.
};
struct DeclaredEnum {
const char* service;
const char* version;
const char* schema;
const char* field;
const char* values; // Comma list that must all be listed by the enum.
};
// CALLS-BEGIN
inline constexpr DeclaredCall kDeclaredCalls[] = {
{"sandbox.create", "POST", "{+parent}/sandboxEnvironments", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.create", "displayName,spec,spec.codeExecutionEnvironment,spec.codeExecutionEnvironment.codeLanguage,spec.codeExecutionEnvironment.machineConfig,spec.shellEnvironment,ttl,sandboxEnvironmentTemplate,sandboxEnvironmentSnapshot,owner"},
{"sandbox.create.computer", "POST", "{+parent}/sandboxEnvironments", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.create", "displayName,spec,spec.computerUseEnvironment,ttl,sandboxEnvironmentTemplate,sandboxEnvironmentSnapshot,owner"},
{"sandbox.get", "GET", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.get", ""},
{"sandbox.list", "GET", "{+parent}/sandboxEnvironments", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.list", ""},
{"sandbox.delete", "DELETE", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.delete", ""},
{"sandbox.pause", "POST", "{+name}:pause", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.pause", ""},
{"sandbox.resume", "POST", "{+name}:resume", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.resume", ""},
{"sandbox.snapshot", "POST", "{+name}:snapshot", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.snapshot", "displayName,sourceSandboxEnvironment,ttl,postSnapshotAction,owner"},
{"sandbox.authorizeAccess", "POST", "{+name}:authorizeAccess", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.authorizeAccess", ""},
{"sandbox.execute", "POST", "{+name}:execute", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.execute", "inputs,inputs.mimeType,inputs.data,inputs.metadata,inputs.metadata.attributes"},
{"sandbox.bidiExecute", "POST", "{+name}:bidiExecute", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.bidiExecute", "inputs,inputs.mimeType,inputs.data,inputs.metadata,inputs.metadata.attributes"},
{"template.create", "POST", "{+parent}/sandboxEnvironmentTemplates", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.create", "displayName,defaultContainerEnvironment,defaultContainerEnvironment.defaultContainerCategory,customContainerEnvironment,customContainerEnvironment.customContainerSpec,customContainerEnvironment.customContainerSpec.imageUri,customContainerEnvironment.ports,customContainerEnvironment.ports.protocol,customContainerEnvironment.ports.port,egressControlConfig,egressControlConfig.internetAccess,egressControlConfig.networkAttachment,persistentDiskConfig,persistentDiskConfig.enabled,persistentDiskConfig.mountPath,persistentDiskConfig.sizeGb"},
{"template.get", "GET", "{+name}", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.get", ""},
{"template.list", "GET", "{+parent}/sandboxEnvironmentTemplates", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.list", ""},
{"template.delete", "DELETE", "{+name}", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.delete", ""},
{"snapshot.get", "GET", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentSnapshots.get", ""},
{"snapshot.list", "GET", "{+parent}/sandboxEnvironmentSnapshots", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentSnapshots.list", ""},
{"snapshot.delete", "DELETE", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentSnapshots.delete", ""},
};
// CALLS-END
// SCHEMAS-BEGIN
inline constexpr DeclaredSchema kDeclaredSchemas[] = {
{"aiplatform", "v1", "SandboxEnvironment", "name,displayName,state,createTime,updateTime,expireTime,ttl,spec,sandboxEnvironmentTemplate,sandboxEnvironmentSnapshot,latestSandboxEnvironmentSnapshot,owner,connectionInfo"},
{"aiplatform", "v1beta1", "SandboxEnvironment", "name,displayName,state,createTime,updateTime,expireTime,ttl,spec,sandboxEnvironmentTemplate,connectionInfo"},
{"aiplatform", "v1", "SandboxEnvironmentSpec", "codeExecutionEnvironment,shellEnvironment"},
{"aiplatform", "v1beta1", "SandboxEnvironmentSpec", "codeExecutionEnvironment,shellEnvironment,computerUseEnvironment"},
{"aiplatform", "v1", "SandboxEnvironmentConnectionInfo", "loadBalancerIp,loadBalancerHostname,sandboxInternalIp,routingToken,serviceAttachment"},
{"aiplatform", "v1", "Chunk", "mimeType,data,metadata"},
{"aiplatform", "v1", "Metadata", "attributes"},
{"aiplatform", "v1", "ExecuteSandboxEnvironmentRequest", "inputs"},
{"aiplatform", "v1", "ExecuteSandboxEnvironmentResponse", "outputs"},
{"aiplatform", "v1beta1", "BidiExecuteSandboxEnvironmentResponse", "outputs"},
{"aiplatform", "v1", "ListSandboxEnvironmentsResponse", "sandboxEnvironments,nextPageToken"},
{"aiplatform", "v1", "SandboxEnvironmentSnapshot", "name,displayName,sourceSandboxEnvironment,ttl,postSnapshotAction,owner,parentSnapshot,sizeBytes,createTime,updateTime,expireTime"},
{"aiplatform", "v1", "ListSandboxEnvironmentSnapshotsResponse", "sandboxEnvironmentSnapshots,nextPageToken"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplate", "name,displayName,state,createTime,updateTime,defaultContainerEnvironment,customContainerEnvironment,egressControlConfig,persistentDiskConfig"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateDefaultContainerEnvironment", "defaultContainerCategory"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateCustomContainerEnvironment", "customContainerSpec,ports,resources"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateCustomContainerSpec", "imageUri"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateNetworkPort", "protocol,port"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateEgressControlConfig", "internetAccess,networkAttachment,dnsPeeringConfigs"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplatePersistentDiskConfig", "enabled,mountPath,sizeGb"},
{"aiplatform", "v1beta1", "ListSandboxEnvironmentTemplatesResponse", "sandboxEnvironmentTemplates,nextPageToken"},
};
// SCHEMAS-END
// ENUMS-BEGIN
inline constexpr DeclaredEnum kDeclaredEnums[] = {
{"aiplatform", "v1", "SandboxEnvironmentSpecCodeExecutionEnvironment", "codeLanguage", "LANGUAGE_PYTHON,LANGUAGE_JAVASCRIPT"},
{"aiplatform", "v1", "SandboxEnvironmentSpecCodeExecutionEnvironment", "machineConfig", "MACHINE_CONFIG_VCPU4_RAM4GIB"},
{"aiplatform", "v1", "SandboxEnvironment", "state", "STATE_PROVISIONING,STATE_RUNNING,STATE_DEPROVISIONING,STATE_TERMINATED,STATE_DELETED,STATE_PAUSED,STATE_PAUSING,STATE_RESUMING,STATE_STOPPING"},
{"aiplatform", "v1", "SandboxEnvironmentSnapshot", "postSnapshotAction", "RUNNING,PAUSE"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateDefaultContainerEnvironment", "defaultContainerCategory", "DEFAULT_CONTAINER_CATEGORY_COMPUTER_USE,DEFAULT_CONTAINER_CATEGORY_SHELL_SANDBOX"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateNetworkPort", "protocol", "TCP,UDP"},
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplate", "state", "PROVISIONING,ACTIVE,DEPROVISIONING,DELETED,FAILED"},
};
// ENUMS-END
} // namespace molo::google::sandbox
+54
View File
@@ -0,0 +1,54 @@
#pragma once
// Static ceilings for the sandbox module. Never autotuned. Google's own limit
// is 100 MB of files per request or response; ours are tighter on purpose.
#include <cstddef>
namespace molo::google::sandbox {
struct SandboxCaps {
// Requests.
static constexpr std::size_t kMaxCodeBytes = 256u * 1024u;
static constexpr std::size_t kMaxCommandBytes = 8192;
static constexpr std::size_t kMaxInternalCommandBytes = 65536; // Workspace helpers only.
static constexpr std::size_t kMaxInputFiles = 16;
static constexpr std::size_t kMaxFileBytes = 4u * 1024u * 1024u;
static constexpr std::size_t kMaxInputTotalBytes = 8u * 1024u * 1024u;
static constexpr std::size_t kMaxFileNameChars = 255;
static constexpr std::size_t kMaxPathChars = 256;
// Shell file helpers ship bytes inside the command line, so they stay small.
static constexpr std::size_t kMaxShellWriteBytes = 32u * 1024u;
static constexpr std::size_t kMaxShellReadBytes = 512u * 1024u;
// Responses. Text is truncated (flagged), files over a cap are skipped
// (counted and named in ExecutionResult::skipped).
static constexpr std::size_t kMaxOutputBytes = 1024u * 1024u;
static constexpr std::size_t kMaxOutputChunks = 64;
static constexpr std::size_t kMaxProducedFiles = 16;
static constexpr std::size_t kMaxProducedFileBytes = 4u * 1024u * 1024u;
static constexpr std::size_t kMaxProducedTotalBytes = 8u * 1024u * 1024u;
static constexpr std::size_t kMaxScreenshotBytes = 4u * 1024u * 1024u;
// Time. Execution is synchronous; the shell timeout is sent to the sandbox,
// the HTTP client's own 60 s request timeout bounds everything else.
static constexpr int kDefaultShellTimeoutSeconds = 30;
static constexpr int kMaxShellTimeoutSeconds = 300;
static constexpr long kMinTtlSeconds = 60;
static constexpr long kDefaultTtlSeconds = 3600;
static constexpr long kMaxTtlSeconds = 14L * 24 * 3600; // Google: state kept up to 14 days.
// Names and lists.
static constexpr std::size_t kMaxDisplayNameChars = 128;
static constexpr std::size_t kMaxResourceNameChars = 512;
static constexpr int kDefaultPageSize = 50;
static constexpr int kMaxPageSize = 100; // Google default and ceiling.
static constexpr std::size_t kMaxListPages = 10;
// Computer use.
static constexpr std::size_t kMaxUrlChars = 2048;
static constexpr std::size_t kMaxTypedChars = 4096;
static constexpr int kMaxCoordinate = 10000;
};
} // namespace molo::google::sandbox
+232
View File
@@ -0,0 +1,232 @@
#include "google/sandbox/chunk.h"
#include "google/sandbox/caps.h"
#include "google/sandbox/names.h"
#include "security/crypto/Base64.h"
namespace molo::google::sandbox {
using clients::body::Json;
using security::crypto::Base64;
namespace {
std::string str(const Json& doc, const char* key) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
void append_capped(std::string& dst, const std::string& add, bool& truncated) {
if (dst.size() >= SandboxCaps::kMaxOutputBytes) {
if (!add.empty()) truncated = true;
return;
}
std::size_t room = SandboxCaps::kMaxOutputBytes - dst.size();
if (add.size() > room) {
dst.append(add, 0, room);
truncated = true;
} else {
dst += add;
}
}
// Attribute values are "bytes format" (base64). A value that is not valid
// base64 is kept as written, so a plain-text server still works.
std::string attribute_text(const std::string& wire) {
std::string raw;
if (!wire.empty() && Base64::decode(wire, raw) && !raw.empty()) return raw;
return wire;
}
struct FileBudget {
std::size_t total = 0;
};
void add_file(ExecutionResult& r, FileBudget& budget, std::string name,
std::string mime, std::string content) {
if (!valid_file_name(name)) {
r.skipped.push_back("file with unsafe name rejected");
return;
}
if (r.files.size() >= SandboxCaps::kMaxProducedFiles) {
r.skipped.push_back("file " + name + " skipped: more than " +
std::to_string(SandboxCaps::kMaxProducedFiles) + " files");
return;
}
if (content.size() > SandboxCaps::kMaxProducedFileBytes) {
r.skipped.push_back("file " + name + " skipped: over " +
std::to_string(SandboxCaps::kMaxProducedFileBytes) + " bytes");
return;
}
if (budget.total + content.size() > SandboxCaps::kMaxProducedTotalBytes) {
r.skipped.push_back("file " + name + " skipped: produced total over " +
std::to_string(SandboxCaps::kMaxProducedTotalBytes) + " bytes");
return;
}
budget.total += content.size();
r.files.push_back({std::move(name), std::move(mime), std::move(content)});
}
void apply_json_chunk(ExecutionResult& r, FileBudget& budget, const Json& doc) {
// Code execution: msg_out / msg_err / output_files. Shell: stdout / stderr /
// returncode / duration_ms. Both are read; absent keys are ignored.
append_capped(r.std_out, str(doc, "msg_out"), r.out_truncated);
append_capped(r.std_out, str(doc, "stdout"), r.out_truncated);
append_capped(r.std_err, str(doc, "msg_err"), r.err_truncated);
append_capped(r.std_err, str(doc, "stderr"), r.err_truncated);
for (const char* key : {"returncode", "exit_code", "exit_status"})
if (const Json* v = doc.find(key); v && v->is_number())
r.exit_status = static_cast<int>(v->as_int());
if (const Json* v = doc.find("duration_ms"); v && v->is_number())
r.duration_ms = v->as_int();
if (const Json* files = doc.find("output_files"); files && files->is_array()) {
for (const Json& f : files->as_array()) {
std::string name = str(f, "name");
if (name.empty()) name = str(f, "file_name");
std::string raw;
std::string b64 = str(f, "content");
if (!Base64::decode(b64, raw)) {
r.skipped.push_back("file " + name + " skipped: content is not base64");
continue;
}
std::string mime = str(f, "mime_type");
add_file(r, budget, std::move(name),
mime.empty() ? "application/octet-stream" : mime, std::move(raw));
}
}
}
} // namespace
Json Chunk::to_json() const {
Json o = Json::object();
o.set("mimeType", Json(mime_type));
o.set("data", Json(Base64::encode(data)));
if (!attributes.empty()) {
Json attrs = Json::object();
for (const auto& [k, v] : attributes) attrs.set(k, Json(Base64::encode(v)));
Json meta = Json::object();
meta.set("attributes", attrs);
o.set("metadata", meta);
}
return o;
}
Chunk Chunk::from_json(const Json& doc, bool* invalid) {
Chunk c;
if (invalid) *invalid = false;
c.mime_type = str(doc, "mimeType");
std::string wire = str(doc, "data");
if (!wire.empty() && !Base64::decode(wire, c.data)) {
c.data.clear();
if (invalid) *invalid = true;
}
const Json* meta = doc.is_object() ? doc.find("metadata") : nullptr;
const Json* attrs = meta && meta->is_object() ? meta->find("attributes") : nullptr;
if (attrs && attrs->is_object())
for (const auto& [k, v] : attrs->as_object())
if (v.is_string()) c.attributes[k] = attribute_text(v.as_string());
return c;
}
Chunk Chunk::json_payload(const Json& payload) {
Chunk c;
c.mime_type = "application/json";
c.data = payload.dump();
return c;
}
Json ChunkCodec::request(const std::vector<Chunk>& inputs) {
Json arr = Json::array();
for (const Chunk& c : inputs) arr.push(c.to_json());
Json body = Json::object();
body.set("inputs", arr);
return body;
}
Chunk ChunkCodec::code_input(const std::string& code,
const std::vector<InputFile>& files) {
Json payload = Json::object();
payload.set("code", Json(code));
if (!files.empty()) {
Json arr = Json::array();
for (const InputFile& f : files) {
Json one = Json::object();
one.set("name", Json(f.name));
one.set("content", Json(Base64::encode(f.content)));
arr.push(one);
}
payload.set("files", arr);
}
return Chunk::json_payload(payload);
}
Chunk ChunkCodec::shell_input(const std::string& command, const std::string& cwd,
int timeout_seconds) {
Json payload = Json::object();
payload.set("command", Json(command));
if (!cwd.empty()) payload.set("cwd", Json(cwd));
payload.set("timeout", Json(timeout_seconds));
return Chunk::json_payload(payload);
}
Chunk ChunkCodec::cdp_input(const std::string& command, const Json& params) {
Json payload = Json::object();
payload.set("command", Json(command));
payload.set("params", params);
return Chunk::json_payload(payload);
}
std::vector<Chunk> ChunkCodec::outputs(const Json& reply,
std::vector<std::string>* skipped) {
std::vector<Chunk> out;
const Json* arr = reply.is_object() ? reply.find("outputs") : nullptr;
if (!arr || !arr->is_array()) return out;
std::size_t index = 0;
for (const Json& item : arr->as_array()) {
if (out.size() >= SandboxCaps::kMaxOutputChunks) {
if (skipped)
skipped->push_back("chunks beyond " +
std::to_string(SandboxCaps::kMaxOutputChunks) + " skipped");
break;
}
bool invalid = false;
Chunk c = Chunk::from_json(item, &invalid);
if (invalid) {
if (skipped)
skipped->push_back("chunk " + std::to_string(index) + " skipped: data is not base64");
} else {
out.push_back(std::move(c));
}
++index;
}
return out;
}
ExecutionResult ChunkCodec::decode(const Json& reply) {
ExecutionResult r;
FileBudget budget;
std::size_t index = 0;
for (const Chunk& c : outputs(reply, &r.skipped)) {
auto named = c.attributes.find("file_name");
if (named != c.attributes.end()) {
add_file(r, budget, named->second,
c.mime_type.empty() ? "application/octet-stream" : c.mime_type, c.data);
} else if (c.mime_type == "application/json") {
try {
apply_json_chunk(r, budget, Json::parse(c.data));
} catch (const clients::body::JsonError&) {
r.skipped.push_back("chunk " + std::to_string(index) + " skipped: not valid JSON");
}
} else if (c.mime_type.rfind("text/", 0) == 0) {
append_capped(r.std_out, c.data, r.out_truncated);
} else {
r.skipped.push_back("chunk " + std::to_string(index) + " skipped: unnamed " +
(c.mime_type.empty() ? "untyped" : c.mime_type) + " data");
}
++index;
}
return r;
}
} // namespace molo::google::sandbox
+50
View File
@@ -0,0 +1,50 @@
#pragma once
// The Chunk wire type of sandboxEnvironments.execute plus the codec between
// it and the typed values. Chunk.data and Metadata.attributes values are
// base64 on the wire (REST "bytes format"); here they are raw bytes.
#include <map>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "google/sandbox/types.h"
namespace molo::google::sandbox {
struct Chunk {
std::string mime_type;
std::string data; // Raw bytes.
std::map<std::string, std::string> attributes; // Raw bytes per value.
clients::body::Json to_json() const;
// Tolerant: a chunk whose data is not valid base64 yields data "" and sets
// `invalid` so the decoder can skip it loudly.
static Chunk from_json(const clients::body::Json& doc, bool* invalid = nullptr);
static Chunk json_payload(const clients::body::Json& payload);
};
class ChunkCodec {
public:
// {"inputs":[chunk,...]}
static clients::body::Json request(const std::vector<Chunk>& inputs);
// Input chunks. The payload shapes are inferred, see SPEC Unverified.
static Chunk code_input(const std::string& code,
const std::vector<InputFile>& files);
static Chunk shell_input(const std::string& command, const std::string& cwd,
int timeout_seconds);
static Chunk cdp_input(const std::string& command,
const clients::body::Json& params);
// reply["outputs"] to chunks, at most SandboxCaps::kMaxOutputChunks.
static std::vector<Chunk> outputs(const clients::body::Json& reply,
std::vector<std::string>* skipped = nullptr);
// Typed result with every cap applied.
static ExecutionResult decode(const clients::body::Json& reply);
};
} // namespace molo::google::sandbox
+47
View File
@@ -0,0 +1,47 @@
#include "google/sandbox/code_executor.h"
#include <set>
#include "google/errors.h"
#include "google/sandbox/caps.h"
namespace molo::google::sandbox {
CodeExecutor::CodeExecutor(std::shared_ptr<GoogleClient> client)
: SandboxService(std::move(client)) {}
void CodeExecutor::validate(const std::string& code,
const std::vector<InputFile>& files) {
if (code.empty()) throw GoogleError::invalid("code is empty");
if (code.size() > SandboxCaps::kMaxCodeBytes)
throw GoogleError::invalid("code over " +
std::to_string(SandboxCaps::kMaxCodeBytes) + " bytes");
if (files.size() > SandboxCaps::kMaxInputFiles)
throw GoogleError::invalid("more than " +
std::to_string(SandboxCaps::kMaxInputFiles) + " files");
std::set<std::string> seen;
std::size_t total = 0;
for (const InputFile& f : files) {
if (!valid_file_name(f.name)) throw GoogleError::invalid("bad file name");
if (!seen.insert(f.name).second)
throw GoogleError::invalid("duplicate file name " + f.name);
if (f.content.size() > SandboxCaps::kMaxFileBytes)
throw GoogleError::invalid("file " + f.name + " over " +
std::to_string(SandboxCaps::kMaxFileBytes) + " bytes");
total += f.content.size();
}
if (total > SandboxCaps::kMaxInputTotalBytes)
throw GoogleError::invalid("files over " +
std::to_string(SandboxCaps::kMaxInputTotalBytes) +
" bytes in total");
}
ExecutionResult CodeExecutor::run(const std::string& sandbox_name,
const std::string& code,
const std::vector<InputFile>& files) const {
SandboxNames::parse_sandbox(sandbox_name);
validate(code, files);
return execute(sandbox_name, {ChunkCodec::code_input(code, files)});
}
} // namespace molo::google::sandbox
+26
View File
@@ -0,0 +1,26 @@
#pragma once
// Code execution in a code sandbox (Python or JavaScript): code plus input
// files in, typed stdout / stderr / produced files out. The sandbox keeps its
// state between calls and every call resets its TTL (per Google's docs).
#include <string>
#include <vector>
#include "google/sandbox/sandbox_service.h"
namespace molo::google::sandbox {
class CodeExecutor : public SandboxService {
public:
explicit CodeExecutor(std::shared_ptr<GoogleClient> client);
ExecutionResult run(const std::string& sandbox_name, const std::string& code,
const std::vector<InputFile>& files = {}) const;
// Throws GoogleError::invalid: empty or oversized code, too many files, bad
// names, duplicates, a file or the total over its cap.
static void validate(const std::string& code, const std::vector<InputFile>& files);
};
} // namespace molo::google::sandbox
+126
View File
@@ -0,0 +1,126 @@
#include "google/sandbox/computer_use.h"
#include "google/errors.h"
#include "google/sandbox/caps.h"
#include "security/crypto/Base64.h"
namespace molo::google::sandbox {
using clients::body::Json;
ComputerUse::ComputerUse(std::shared_ptr<GoogleClient> client)
: SandboxService(std::move(client)) {}
void ComputerUse::validate_url(const std::string& url) {
bool http = url.rfind("http://", 0) == 0;
bool https = url.rfind("https://", 0) == 0;
if (!http && !https) throw GoogleError::invalid("url must start with http:// or https://");
if (url.size() > SandboxCaps::kMaxUrlChars)
throw GoogleError::invalid("url over " + std::to_string(SandboxCaps::kMaxUrlChars) +
" characters");
if (url.size() <= (https ? 8u : 7u)) throw GoogleError::invalid("url has no host");
for (char c : url)
if (static_cast<unsigned char>(c) <= 0x20 || c == 0x7f)
throw GoogleError::invalid("url contains spaces or control characters");
}
void ComputerUse::validate_point(int x, int y) {
if (x < 0 || y < 0 || x > SandboxCaps::kMaxCoordinate || y > SandboxCaps::kMaxCoordinate)
throw GoogleError::invalid("coordinates must be 0.." +
std::to_string(SandboxCaps::kMaxCoordinate));
}
void ComputerUse::validate_text(const std::string& text) {
if (text.empty()) throw GoogleError::invalid("text is empty");
if (text.size() > SandboxCaps::kMaxTypedChars)
throw GoogleError::invalid("text over " + std::to_string(SandboxCaps::kMaxTypedChars) +
" bytes");
}
Json ComputerUse::navigate_params(const std::string& url) {
Json p = Json::object();
p.set("url", Json(url));
return p;
}
Json ComputerUse::mouse_params(const char* type, int x, int y) {
Json p = Json::object();
p.set("type", Json(type));
p.set("x", Json(x));
p.set("y", Json(y));
p.set("button", Json("left"));
p.set("clickCount", Json(1));
return p;
}
Json ComputerUse::insert_text_params(const std::string& text) {
Json p = Json::object();
p.set("text", Json(text));
return p;
}
ActionResult ComputerUse::send(const std::string& sandbox_name, const std::string& command,
const Json& params) const {
SandboxNames::parse_sandbox(sandbox_name);
Json reply = execute_raw(sandbox_name, {ChunkCodec::cdp_input(command, params)});
ActionResult out;
out.result = Json::object();
out.exec = ChunkCodec::decode(reply);
for (const Chunk& c : ChunkCodec::outputs(reply)) {
if (c.mime_type == "image/png" && out.screenshot_png.empty() &&
c.data.size() <= SandboxCaps::kMaxScreenshotBytes)
out.screenshot_png = c.data;
if (c.mime_type == "application/json") {
try {
Json doc = Json::parse(c.data);
if (!doc.is_object()) continue;
const Json* r = doc.find("result");
const Json& obj = r && r->is_object() ? *r : doc;
out.result = obj;
// CDP Page.captureScreenshot answers {"data": "<base64 png>"}.
if (const Json* d = obj.find("data"); d && d->is_string() && out.screenshot_png.empty()) {
std::string png;
if (d->as_string().size() <= (SandboxCaps::kMaxScreenshotBytes / 3 + 1) * 4 &&
security::crypto::Base64::decode(d->as_string(), png) &&
png.size() <= SandboxCaps::kMaxScreenshotBytes)
out.screenshot_png = std::move(png);
}
} catch (const clients::body::JsonError&) {
}
}
}
return out;
}
ActionResult ComputerUse::navigate(const std::string& sandbox_name,
const std::string& url) const {
SandboxNames::parse_sandbox(sandbox_name);
validate_url(url);
return send(sandbox_name, "Page.navigate", navigate_params(url));
}
ActionResult ComputerUse::click(const std::string& sandbox_name, int x, int y) const {
SandboxNames::parse_sandbox(sandbox_name);
validate_point(x, y);
send(sandbox_name, "Input.dispatchMouseEvent", mouse_params("mousePressed", x, y));
return send(sandbox_name, "Input.dispatchMouseEvent", mouse_params("mouseReleased", x, y));
}
ActionResult ComputerUse::type_text(const std::string& sandbox_name,
const std::string& text) const {
SandboxNames::parse_sandbox(sandbox_name);
validate_text(text);
return send(sandbox_name, "Input.insertText", insert_text_params(text));
}
ActionResult ComputerUse::screenshot(const std::string& sandbox_name) const {
SandboxNames::parse_sandbox(sandbox_name);
Json p = Json::object();
p.set("format", Json("png"));
ActionResult r = send(sandbox_name, "Page.captureScreenshot", p);
if (r.screenshot_png.empty())
throw GoogleError(502, "EMPTY_RESPONSE", "no screenshot in the reply", endpoint_.name);
return r;
}
} // namespace molo::google::sandbox
+47
View File
@@ -0,0 +1,47 @@
#pragma once
// Computer use: browser actions in a computer use sandbox (navigate, click,
// type, screenshot). Docs describe the action set and that each action is a
// Chrome DevTools Protocol command ({"command","params"}); they send it to
// the sandbox data plane with a signed JWT. This class carries the same CDP
// commands as an application/json Chunk through sandboxEnvironments.execute,
// the only sandbox call in the REST surface. Whether the server routes that
// Chunk to the browser is unverified (SPEC Unverified).
#include <string>
#include "google/sandbox/sandbox_service.h"
namespace molo::google::sandbox {
struct ActionResult {
clients::body::Json result; // The CDP result object ({} when none).
ExecutionResult exec; // Text, files and skip reasons of the reply.
std::string screenshot_png; // Raw PNG bytes, set by screenshot() only.
};
class ComputerUse : public SandboxService {
public:
explicit ComputerUse(std::shared_ptr<GoogleClient> client);
ActionResult navigate(const std::string& sandbox_name, const std::string& url) const;
// Left click at viewport coordinates: mousePressed then mouseReleased.
ActionResult click(const std::string& sandbox_name, int x, int y) const;
ActionResult type_text(const std::string& sandbox_name, const std::string& text) const;
ActionResult screenshot(const std::string& sandbox_name) const;
static void validate_url(const std::string& url); // http(s) only, capped
static void validate_point(int x, int y);
static void validate_text(const std::string& text);
// CDP command rows (compared to the CDP names in tests).
static clients::body::Json navigate_params(const std::string& url);
static clients::body::Json mouse_params(const char* type, int x, int y);
static clients::body::Json insert_text_params(const std::string& text);
private:
ActionResult send(const std::string& sandbox_name, const std::string& command,
const clients::body::Json& params) const;
};
} // namespace molo::google::sandbox
+160
View File
@@ -0,0 +1,160 @@
#include "google/sandbox/names.h"
#include <vector>
#include "google/errors.h"
#include "google/resource.h"
#include "google/sandbox/caps.h"
namespace molo::google::sandbox {
namespace {
std::vector<std::string> split(const std::string& s) {
std::vector<std::string> out;
std::string cur;
for (char c : s) {
if (c == '/') {
out.push_back(cur);
cur.clear();
} else {
cur.push_back(c);
}
}
out.push_back(cur);
return out;
}
bool project_ok(const std::string& p) {
if (p.empty() || p.size() > 128) return false;
for (char c : p)
if (!((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-' ||
c == '.' || c == ':'))
return false;
return true;
}
bool location_ok(const std::string& l) {
if (l.empty() || l.size() > 64) return false;
for (char c : l)
if (!((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-')) return false;
return true;
}
bool parse(const std::string& name, std::size_t segments,
SandboxNames::Parts& out, const char* collection = "sandboxEnvironments") {
if (name.empty() || name.size() > SandboxCaps::kMaxResourceNameChars) return false;
std::vector<std::string> p = split(name);
if (p.size() != segments) return false;
if (p[0] != "projects" || p[2] != "locations" || p[4] != "reasoningEngines")
return false;
if (!project_ok(p[1]) || !location_ok(p[3]) || !valid_segment(p[5])) return false;
if (segments == 8) {
if (p[6] != collection || !valid_segment(p[7])) return false;
out.sandbox = p[7];
}
out.project = p[1];
out.location = p[3];
out.engine = p[5];
return true;
}
// True when no "." or ".." or empty segment exists (separator '/').
bool segments_plain(const std::string& s, bool allow_empty_first) {
std::size_t start = 0;
bool first = true;
while (start <= s.size()) {
std::size_t end = s.find('/', start);
if (end == std::string::npos) end = s.size();
std::string seg = s.substr(start, end - start);
if (seg == ".." || seg == ".") return false;
if (seg.empty() && !(first && allow_empty_first)) return false;
first = false;
start = end + 1;
}
return true;
}
} // namespace
bool SandboxNames::valid_engine(const std::string& name) {
Parts p;
return parse(name, 6, p);
}
bool SandboxNames::valid_sandbox(const std::string& name) {
Parts p;
return parse(name, 8, p);
}
bool SandboxNames::valid_template(const std::string& name) {
Parts p;
return parse(name, 8, p, "sandboxEnvironmentTemplates");
}
bool SandboxNames::valid_snapshot(const std::string& name) {
Parts p;
return parse(name, 8, p, "sandboxEnvironmentSnapshots");
}
SandboxNames::Parts SandboxNames::parse_template(const std::string& name) {
Parts p;
if (!parse(name, 8, p, "sandboxEnvironmentTemplates"))
throw GoogleError::invalid(
"template must look like projects/{p}/locations/{l}/reasoningEngines/{e}"
"/sandboxEnvironmentTemplates/{t}");
return p;
}
SandboxNames::Parts SandboxNames::parse_snapshot(const std::string& name) {
Parts p;
if (!parse(name, 8, p, "sandboxEnvironmentSnapshots"))
throw GoogleError::invalid(
"snapshot must look like projects/{p}/locations/{l}/reasoningEngines/{e}"
"/sandboxEnvironmentSnapshots/{n}");
return p;
}
SandboxNames::Parts SandboxNames::parse_engine(const std::string& name) {
Parts p;
if (!parse(name, 6, p))
throw GoogleError::invalid(
"engine must look like projects/{p}/locations/{l}/reasoningEngines/{e}");
return p;
}
SandboxNames::Parts SandboxNames::parse_sandbox(const std::string& name) {
Parts p;
if (!parse(name, 8, p))
throw GoogleError::invalid(
"sandbox must look like projects/{p}/locations/{l}/reasoningEngines/{e}"
"/sandboxEnvironments/{s}");
return p;
}
std::string SandboxNames::engine_of(const std::string& sandbox_name) {
Parts p = parse_sandbox(sandbox_name);
return "projects/" + p.project + "/locations/" + p.location +
"/reasoningEngines/" + p.engine;
}
bool valid_file_name(const std::string& name) {
if (name.empty() || name.size() > SandboxCaps::kMaxFileNameChars) return false;
if (name[0] == '/') return false;
for (char c : name)
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f || c == '\\') return false;
return segments_plain(name, false);
}
bool valid_workspace_path(const std::string& path) {
if (path.empty() || path.size() > SandboxCaps::kMaxPathChars) return false;
for (char c : path)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
(c >= '0' && c <= '9') || c == '.' || c == '_' || c == '-' || c == '/'))
return false;
// A trailing slash (directory) is fine: "/workspace/".
std::string trimmed = path.size() > 1 && path.back() == '/' ? path.substr(0, path.size() - 1) : path;
return segments_plain(trimmed, true);
}
} // namespace molo::google::sandbox
+44
View File
@@ -0,0 +1,44 @@
#pragma once
// Resource names for the sandbox module. Validated before any network
// traffic. Formats (REST reference, v1beta1):
// projects/{p}/locations/{l}/reasoningEngines/{e}
// projects/{p}/locations/{l}/reasoningEngines/{e}/sandboxEnvironments/{s}
// projects/{p}/locations/{l}/reasoningEngines/{e}/sandboxEnvironmentTemplates/{t}
// projects/{p}/locations/{l}/reasoningEngines/{e}/sandboxEnvironmentSnapshots/{n}
#include <string>
namespace molo::google::sandbox {
struct SandboxNames {
struct Parts {
std::string project;
std::string location;
std::string engine;
std::string sandbox; // Empty for an engine name.
};
static bool valid_engine(const std::string& name);
static bool valid_sandbox(const std::string& name);
static bool valid_template(const std::string& name);
static bool valid_snapshot(const std::string& name);
// Throw GoogleError::invalid on a malformed name.
static Parts parse_engine(const std::string& name);
static Parts parse_sandbox(const std::string& name);
// For templates and snapshots Parts::sandbox holds the child id.
static Parts parse_template(const std::string& name);
static Parts parse_snapshot(const std::string& name);
// "projects/p/locations/l/reasoningEngines/e" from a sandbox name.
static std::string engine_of(const std::string& sandbox_name);
};
// A file name inside a request or response: no leading slash, no "." or ".."
// segment, no backslash or control characters, 1..255 characters.
bool valid_file_name(const std::string& name);
// An absolute or /workspace-relative path of [A-Za-z0-9._/-], no "..".
bool valid_workspace_path(const std::string& path);
} // namespace molo::google::sandbox
+39
View File
@@ -0,0 +1,39 @@
#include "google/sandbox/sandbox_service.h"
#include "google/resource.h"
namespace molo::google::sandbox {
using clients::body::Json;
SandboxService::SandboxService(std::shared_ptr<GoogleClient> client)
: Service(std::move(client), endpoints::aiplatform("us-central1")) {
endpoint_.name = "aiplatform-sandbox";
}
ServiceEndpoint SandboxService::endpoint_in(const std::string& location) {
ServiceEndpoint e = endpoints::aiplatform(location);
e.name = "aiplatform-sandbox";
return e;
}
ServiceEndpoint SandboxService::beta_endpoint_in(const std::string& location) {
ServiceEndpoint e = endpoints::aiplatform_beta(location);
e.name = "aiplatform-sandbox-beta";
return e;
}
Json SandboxService::execute_raw(const std::string& sandbox_name,
const std::vector<Chunk>& inputs) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
Json body = ChunkCodec::request(inputs);
return client_->request(clients::Method::kPost, endpoint_in(parts.location),
"/" + sandbox_name + ":execute", &body);
}
ExecutionResult SandboxService::execute(const std::string& sandbox_name,
const std::vector<Chunk>& inputs) const {
return ChunkCodec::decode(execute_raw(sandbox_name, inputs));
}
} // namespace molo::google::sandbox
+34
View File
@@ -0,0 +1,34 @@
#pragma once
// Base for every sandbox service: the v1beta1 aiplatform endpoint of the
// sandbox's own location, name validation, and the one execute call that
// code, shell, workspace and computer use all share.
#include <memory>
#include <string>
#include <vector>
#include "google/service.h"
#include "google/sandbox/chunk.h"
#include "google/sandbox/names.h"
namespace molo::google::sandbox {
class SandboxService : public Service {
public:
explicit SandboxService(std::shared_ptr<GoogleClient> client);
protected:
// https://{location}-aiplatform.googleapis.com/v1 (every call but bidiExecute).
static ServiceEndpoint endpoint_in(const std::string& location);
// .../v1beta1: bidiExecute exists only there (see calls.h).
static ServiceEndpoint beta_endpoint_in(const std::string& location);
// POST {sandbox}:execute with the given inputs; returns the raw reply.
clients::body::Json execute_raw(const std::string& sandbox_name,
const std::vector<Chunk>& inputs) const;
ExecutionResult execute(const std::string& sandbox_name,
const std::vector<Chunk>& inputs) const;
};
} // namespace molo::google::sandbox
+190
View File
@@ -0,0 +1,190 @@
#include "google/sandbox/sandboxes.h"
#include "google/errors.h"
#include "google/sandbox/caps.h"
#include "google/sandbox/snapshot_types.h"
namespace molo::google::sandbox {
using clients::body::Json;
namespace {
bool printable(const std::string& s) {
for (char c : s)
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f) return false;
return true;
}
} // namespace
Sandboxes::Sandboxes(std::shared_ptr<GoogleClient> client, Sleeper sleeper)
: SandboxService(std::move(client)), sleeper_(std::move(sleeper)) {}
void Sandboxes::validate(const CreateSandboxRequest& r) {
if (r.display_name.empty() ||
r.display_name.size() > SandboxCaps::kMaxDisplayNameChars ||
!printable(r.display_name))
throw GoogleError::invalid("display_name must be 1.." +
std::to_string(SandboxCaps::kMaxDisplayNameChars) +
" printable characters");
if (r.ttl_seconds < SandboxCaps::kMinTtlSeconds ||
r.ttl_seconds > SandboxCaps::kMaxTtlSeconds)
throw GoogleError::invalid("ttl must be " +
std::to_string(SandboxCaps::kMinTtlSeconds) + ".." +
std::to_string(SandboxCaps::kMaxTtlSeconds) + " seconds");
if (r.spec.kind != SandboxKind::kCodeExecution &&
(r.spec.language != CodeLanguage::kUnspecified ||
r.spec.machine != MachineConfig::kUnspecified))
throw GoogleError::invalid("language and machine apply to code sandboxes only");
if (!r.template_name.empty()) SandboxNames::parse_template(r.template_name);
if (!r.snapshot_name.empty()) SandboxNames::parse_snapshot(r.snapshot_name);
if (r.owner.size() > 128 || !printable(r.owner))
throw GoogleError::invalid("owner must be up to 128 printable characters");
if (r.spec.kind == SandboxKind::kComputerUse && r.template_name.empty())
throw GoogleError::invalid("a computer use sandbox needs a template");
}
void Sandboxes::validate(const SnapshotRequest& r) {
if (r.display_name.empty() ||
r.display_name.size() > SandboxCaps::kMaxDisplayNameChars ||
!printable(r.display_name))
throw GoogleError::invalid("display_name must be 1.." +
std::to_string(SandboxCaps::kMaxDisplayNameChars) +
" printable characters");
if (r.ttl_seconds != 0 && (r.ttl_seconds < SandboxCaps::kMinTtlSeconds ||
r.ttl_seconds > SandboxCaps::kMaxTtlSeconds))
throw GoogleError::invalid("ttl must be 0 or " +
std::to_string(SandboxCaps::kMinTtlSeconds) + ".." +
std::to_string(SandboxCaps::kMaxTtlSeconds) + " seconds");
if (!r.post_action.empty() && r.post_action != "RUNNING" && r.post_action != "PAUSE")
throw GoogleError::invalid("post_action must be RUNNING or PAUSE");
if (r.owner.size() > 128 || !printable(r.owner))
throw GoogleError::invalid("owner must be up to 128 printable characters");
}
Json Sandboxes::snapshot_body(const std::string& sandbox_name,
const SnapshotRequest& r) {
Json body = Json::object();
body.set("displayName", Json(r.display_name));
body.set("sourceSandboxEnvironment", Json(sandbox_name));
if (r.ttl_seconds != 0) body.set("ttl", Json(std::to_string(r.ttl_seconds) + "s"));
if (!r.post_action.empty()) body.set("postSnapshotAction", Json(r.post_action));
if (!r.owner.empty()) body.set("owner", Json(r.owner));
return body;
}
SandboxSnapshot Sandboxes::snapshot(const std::string& sandbox_name,
const SnapshotRequest& request) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
validate(request);
ServiceEndpoint ep = endpoint_in(parts.location);
Json body = snapshot_body(sandbox_name, request);
Json reply = client_->request(clients::Method::kPost, ep,
"/" + sandbox_name + ":snapshot", &body);
Operation op = Operation::from_json(reply);
if (!op.done) {
if (op.name.empty())
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
op = wait_operation(*client_, ep, op.name, sleeper_);
} else if (op.failed()) {
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
}
return SandboxSnapshot::from_json(op.response);
}
void Sandboxes::authorize_access(const std::string& sandbox_name) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
Json body = Json::object();
client_->request(clients::Method::kPost, endpoint_in(parts.location),
"/" + sandbox_name + ":authorizeAccess", &body);
}
SandboxEnvironment Sandboxes::finish(const ServiceEndpoint& endpoint,
const Json& reply) const {
Operation op = Operation::from_json(reply);
if (!op.done) {
if (op.name.empty())
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name",
endpoint.name);
op = wait_operation(*client_, endpoint, op.name, sleeper_);
} else if (op.failed()) {
throw GoogleError(502, "OPERATION_FAILED", op.error_message, endpoint.name);
}
return SandboxEnvironment::from_json(op.response);
}
Json Sandboxes::create_body(const CreateSandboxRequest& request) {
Json body = Json::object();
body.set("displayName", Json(request.display_name));
body.set("spec", request.spec.to_json());
body.set("ttl", Json(std::to_string(request.ttl_seconds) + "s"));
if (!request.template_name.empty())
body.set("sandboxEnvironmentTemplate", Json(request.template_name));
if (!request.snapshot_name.empty())
body.set("sandboxEnvironmentSnapshot", Json(request.snapshot_name));
if (!request.owner.empty()) body.set("owner", Json(request.owner));
return body;
}
SandboxEnvironment Sandboxes::create(const std::string& engine_name,
const CreateSandboxRequest& request) const {
SandboxNames::Parts parts = SandboxNames::parse_engine(engine_name);
validate(request);
Json body = create_body(request);
// computerUseEnvironment exists only in v1beta1 (see calls.h).
ServiceEndpoint ep = request.spec.kind == SandboxKind::kComputerUse
? beta_endpoint_in(parts.location)
: endpoint_in(parts.location);
Json reply = client_->request(clients::Method::kPost, ep,
"/" + engine_name + "/sandboxEnvironments", &body);
return finish(ep, reply);
}
SandboxEnvironment Sandboxes::get(const std::string& sandbox_name) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
return SandboxEnvironment::from_json(
client_->get_json(endpoint_in(parts.location), "/" + sandbox_name));
}
std::vector<SandboxEnvironment> Sandboxes::list(const std::string& engine_name,
int page_size) const {
SandboxNames::Parts parts = SandboxNames::parse_engine(engine_name);
if (page_size == 0) page_size = SandboxCaps::kDefaultPageSize;
if (page_size < 1 || page_size > SandboxCaps::kMaxPageSize)
throw GoogleError::invalid("page_size must be 1.." +
std::to_string(SandboxCaps::kMaxPageSize));
Json items = list_all(*client_, endpoint_in(parts.location),
"/" + engine_name + "/sandboxEnvironments?pageSize=" +
std::to_string(page_size),
"sandboxEnvironments", SandboxCaps::kMaxListPages);
std::vector<SandboxEnvironment> out;
for (const Json& item : items.as_array())
out.push_back(SandboxEnvironment::from_json(item));
return out;
}
SandboxEnvironment Sandboxes::pause(const std::string& sandbox_name) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
ServiceEndpoint ep = endpoint_in(parts.location);
Json body = Json::object();
return finish(ep, client_->request(clients::Method::kPost, ep,
"/" + sandbox_name + ":pause", &body));
}
SandboxEnvironment Sandboxes::resume(const std::string& sandbox_name) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
ServiceEndpoint ep = endpoint_in(parts.location);
Json body = Json::object();
return finish(ep, client_->request(clients::Method::kPost, ep,
"/" + sandbox_name + ":resume", &body));
}
void Sandboxes::remove(const std::string& sandbox_name) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
ServiceEndpoint ep = endpoint_in(parts.location);
Json reply = client_->delete_json(ep, "/" + sandbox_name);
finish(ep, reply);
}
} // namespace molo::google::sandbox
+58
View File
@@ -0,0 +1,58 @@
#pragma once
// SandboxEnvironment lifecycle: create, get, list, pause, resume, delete.
// REST: projects.locations.reasoningEngines.sandboxEnvironments (v1).
// Create, pause, resume and delete return long-running operations that this
// class waits on (hard-capped by wait_operation).
#include <string>
#include <vector>
#include "google/operation.h"
#include "google/sandbox/sandbox_service.h"
#include "google/sandbox/snapshot_types.h"
#include "google/sandbox/types.h"
namespace molo::google::sandbox {
class Sandboxes : public SandboxService {
public:
explicit Sandboxes(std::shared_ptr<GoogleClient> client,
Sleeper sleeper = nullptr);
// POST {engine}/sandboxEnvironments
SandboxEnvironment create(const std::string& engine_name,
const CreateSandboxRequest& request) const;
// GET {name}
SandboxEnvironment get(const std::string& sandbox_name) const;
// GET {engine}/sandboxEnvironments, following pages up to the page cap.
std::vector<SandboxEnvironment> list(const std::string& engine_name,
int page_size = 0) const;
// POST {name}:pause and :resume
SandboxEnvironment pause(const std::string& sandbox_name) const;
SandboxEnvironment resume(const std::string& sandbox_name) const;
// POST {name}:snapshot. Returns the new SandboxEnvironmentSnapshot.
SandboxSnapshot snapshot(const std::string& sandbox_name,
const SnapshotRequest& request) const;
// POST {name}:authorizeAccess: succeeds (no value) when the caller holds
// sandboxEnvironments.execute on the sandbox, throws GoogleError otherwise.
void authorize_access(const std::string& sandbox_name) const;
// DELETE {name}. Explicit: sandboxes are billed while they exist, but a
// delete is never implied by any other call.
void remove(const std::string& sandbox_name) const;
static void validate(const CreateSandboxRequest& request);
static void validate(const SnapshotRequest& request);
// The exact request bodies (compared to the Discovery schema in tests).
static clients::body::Json create_body(const CreateSandboxRequest& request);
static clients::body::Json snapshot_body(const std::string& sandbox_name,
const SnapshotRequest& request);
private:
SandboxEnvironment finish(const ServiceEndpoint& endpoint,
const clients::body::Json& reply) const;
Sleeper sleeper_;
};
} // namespace molo::google::sandbox
+44
View File
@@ -0,0 +1,44 @@
#include "google/sandbox/shell_runner.h"
#include "google/errors.h"
namespace molo::google::sandbox {
ShellRunner::ShellRunner(std::shared_ptr<GoogleClient> client)
: SandboxService(std::move(client)) {}
void ShellRunner::validate(const ShellRequest& r, std::size_t max_command) {
if (r.command.empty()) throw GoogleError::invalid("command is empty");
if (r.command.size() > max_command)
throw GoogleError::invalid("command over " + std::to_string(max_command) +
" bytes");
if (r.command.find('\0') != std::string::npos)
throw GoogleError::invalid("command contains a NUL byte");
if (!r.cwd.empty() && !valid_workspace_path(r.cwd))
throw GoogleError::invalid("cwd must be a plain path of [A-Za-z0-9._/-]");
if (r.timeout_seconds < 1 || r.timeout_seconds > SandboxCaps::kMaxShellTimeoutSeconds)
throw GoogleError::invalid("timeout must be 1.." +
std::to_string(SandboxCaps::kMaxShellTimeoutSeconds) +
" seconds");
}
ExecutionResult ShellRunner::run_capped(const std::string& sandbox_name,
const ShellRequest& request,
std::size_t max_command) const {
SandboxNames::parse_sandbox(sandbox_name);
validate(request, max_command);
return execute(sandbox_name, {ChunkCodec::shell_input(
request.command, request.cwd, request.timeout_seconds)});
}
ExecutionResult ShellRunner::run(const std::string& sandbox_name,
const ShellRequest& request) const {
return run_capped(sandbox_name, request, SandboxCaps::kMaxCommandBytes);
}
ExecutionResult ShellRunner::run_internal(const std::string& sandbox_name,
const ShellRequest& request) const {
return run_capped(sandbox_name, request, SandboxCaps::kMaxInternalCommandBytes);
}
} // namespace molo::google::sandbox
+40
View File
@@ -0,0 +1,40 @@
#pragma once
// Bash commands in a shell sandbox ("Workspaces" in the product overview:
// commands and files isolated from core systems). Each command runs in a new
// shell as an unprivileged user; cd and variables do not carry over.
#include <string>
#include "google/sandbox/caps.h"
#include "google/sandbox/sandbox_service.h"
namespace molo::google::sandbox {
struct ShellRequest {
std::string command;
std::string cwd; // Empty: the sandbox default (/workspace).
int timeout_seconds = SandboxCaps::kDefaultShellTimeoutSeconds;
};
class ShellRunner : public SandboxService {
public:
explicit ShellRunner(std::shared_ptr<GoogleClient> client);
ExecutionResult run(const std::string& sandbox_name,
const ShellRequest& request) const;
// Same call with a higher command ceiling, for WorkspaceFiles only (it
// ships file bytes inside the command line). Never exposed to routes.
ExecutionResult run_internal(const std::string& sandbox_name,
const ShellRequest& request) const;
static void validate(const ShellRequest& request, std::size_t max_command);
private:
ExecutionResult run_capped(const std::string& sandbox_name,
const ShellRequest& request,
std::size_t max_command) const;
};
} // namespace molo::google::sandbox
+35
View File
@@ -0,0 +1,35 @@
#pragma once
// SandboxEnvironmentSnapshot (v1): a saved sandbox state a new sandbox can be
// created from. Also the request that takes one.
#include <string>
#include "clients/body/json.h"
namespace molo::google::sandbox {
struct SnapshotRequest {
std::string display_name; // Required, 1..128 characters.
long ttl_seconds = 0; // 0: omit (server default); else 60 s .. 14 days.
std::string post_action; // "", "RUNNING" or "PAUSE" (postSnapshotAction).
std::string owner; // Optional, printable, up to 128 characters.
};
struct SandboxSnapshot {
std::string name;
std::string display_name;
std::string source_sandbox;
std::string parent_snapshot;
std::string owner;
std::string post_action;
std::string create_time;
std::string update_time;
std::string expire_time;
long size_bytes = -1; // sizeBytes is an int64 string; -1 unknown.
static SandboxSnapshot from_json(const clients::body::Json& doc);
clients::body::Json to_json() const;
};
} // namespace molo::google::sandbox
+88
View File
@@ -0,0 +1,88 @@
#include "google/sandbox/snapshots.h"
#include <cstdlib>
#include "google/errors.h"
#include "google/sandbox/caps.h"
namespace molo::google::sandbox {
using clients::body::Json;
namespace {
std::string str(const Json& doc, const char* key) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
} // namespace
SandboxSnapshot SandboxSnapshot::from_json(const Json& doc) {
SandboxSnapshot s;
s.name = str(doc, "name");
s.display_name = str(doc, "displayName");
s.source_sandbox = str(doc, "sourceSandboxEnvironment");
s.parent_snapshot = str(doc, "parentSnapshot");
s.owner = str(doc, "owner");
s.post_action = str(doc, "postSnapshotAction");
s.create_time = str(doc, "createTime");
s.update_time = str(doc, "updateTime");
s.expire_time = str(doc, "expireTime");
std::string size = str(doc, "sizeBytes");
if (!size.empty() && size.find_first_not_of("0123456789") == std::string::npos &&
size.size() < 18)
s.size_bytes = std::strtol(size.c_str(), nullptr, 10);
return s;
}
Json SandboxSnapshot::to_json() const {
Json o = Json::object();
o.set("name", Json(name));
o.set("display_name", Json(display_name));
o.set("source_sandbox", Json(source_sandbox));
o.set("parent_snapshot", Json(parent_snapshot));
o.set("post_action", Json(post_action));
o.set("create_time", Json(create_time));
o.set("expire_time", Json(expire_time));
o.set("size_bytes", Json(size_bytes));
return o;
}
Snapshots::Snapshots(std::shared_ptr<GoogleClient> client, Sleeper sleeper)
: SandboxService(std::move(client)), sleeper_(std::move(sleeper)) {}
SandboxSnapshot Snapshots::get(const std::string& snapshot_name) const {
SandboxNames::Parts p = SandboxNames::parse_snapshot(snapshot_name);
return SandboxSnapshot::from_json(
client_->get_json(endpoint_in(p.location), "/" + snapshot_name));
}
std::vector<SandboxSnapshot> Snapshots::list(const std::string& engine_name,
int page_size) const {
SandboxNames::Parts p = SandboxNames::parse_engine(engine_name);
if (page_size == 0) page_size = SandboxCaps::kDefaultPageSize;
if (page_size < 1 || page_size > SandboxCaps::kMaxPageSize)
throw GoogleError::invalid("page_size must be 1.." +
std::to_string(SandboxCaps::kMaxPageSize));
Json items = list_all(*client_, endpoint_in(p.location),
"/" + engine_name + "/sandboxEnvironmentSnapshots?pageSize=" +
std::to_string(page_size),
"sandboxEnvironmentSnapshots", SandboxCaps::kMaxListPages);
std::vector<SandboxSnapshot> out;
for (const Json& item : items.as_array()) out.push_back(SandboxSnapshot::from_json(item));
return out;
}
void Snapshots::remove(const std::string& snapshot_name) const {
SandboxNames::Parts p = SandboxNames::parse_snapshot(snapshot_name);
ServiceEndpoint ep = endpoint_in(p.location);
Operation op = Operation::from_json(client_->delete_json(ep, "/" + snapshot_name));
if (!op.done) {
if (op.name.empty())
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
wait_operation(*client_, ep, op.name, sleeper_);
} else if (op.failed()) {
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
}
}
} // namespace molo::google::sandbox
+29
View File
@@ -0,0 +1,29 @@
#pragma once
// SandboxEnvironmentSnapshot reads and delete (v1). Taking a snapshot is
// Sandboxes::snapshot (it hangs off a sandbox).
#include <string>
#include <vector>
#include "google/operation.h"
#include "google/sandbox/sandbox_service.h"
#include "google/sandbox/snapshot_types.h"
namespace molo::google::sandbox {
class Snapshots : public SandboxService {
public:
explicit Snapshots(std::shared_ptr<GoogleClient> client, Sleeper sleeper = nullptr);
SandboxSnapshot get(const std::string& snapshot_name) const;
std::vector<SandboxSnapshot> list(const std::string& engine_name,
int page_size = 0) const;
// Explicit delete (long-running; waited on, capped).
void remove(const std::string& snapshot_name) const;
private:
Sleeper sleeper_;
};
} // namespace molo::google::sandbox
+37
View File
@@ -0,0 +1,37 @@
#include "google/sandbox/stream_executor.h"
#include "google/errors.h"
namespace molo::google::sandbox {
using clients::body::Json;
StreamExecutor::StreamExecutor(std::shared_ptr<GoogleClient> client)
: SandboxService(std::move(client)) {}
ExecutionResult StreamExecutor::run(
const std::string& sandbox_name, const std::vector<Chunk>& inputs,
const std::function<bool(const std::vector<Chunk>&)>& on_event) const {
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
if (inputs.empty()) throw GoogleError::invalid("no inputs");
if (inputs.size() > 16) throw GoogleError::invalid("more than 16 input chunks");
Json body = ChunkCodec::request(inputs);
Json merged = Json::array();
std::size_t events = 0;
client_->stream_json(
beta_endpoint_in(parts.location), "/" + sandbox_name + ":bidiExecute", body,
[&](const Json& event) {
if (++events > kMaxEvents) return false;
std::vector<Chunk> chunks = ChunkCodec::outputs(event);
for (const Json& item : (event.find("outputs") && event.find("outputs")->is_array()
? event.find("outputs")->as_array()
: Json::Array{}))
merged.push(item);
return on_event ? on_event(chunks) : true;
});
Json reply = Json::object();
reply.set("outputs", merged);
return ChunkCodec::decode(reply);
}
} // namespace molo::google::sandbox
+29
View File
@@ -0,0 +1,29 @@
#pragma once
// sandboxEnvironments.bidiExecute (v1beta1 only): execute with a streamed
// reply. The REST surface is one POST whose reply is a stream of
// BidiExecuteSandboxEnvironmentResponse events; this class sends the inputs
// once and folds every event's outputs into one typed ExecutionResult.
// True bidirectional input is not expressible over this client (gap).
#include <functional>
#include <string>
#include <vector>
#include "google/sandbox/sandbox_service.h"
namespace molo::google::sandbox {
class StreamExecutor : public SandboxService {
public:
static constexpr std::size_t kMaxEvents = 256;
explicit StreamExecutor(std::shared_ptr<GoogleClient> client);
// `on_event` (optional) sees each event's decoded output chunks as they
// arrive; return false to stop early.
ExecutionResult run(const std::string& sandbox_name, const std::vector<Chunk>& inputs,
const std::function<bool(const std::vector<Chunk>&)>& on_event = nullptr) const;
};
} // namespace molo::google::sandbox
+210
View File
@@ -0,0 +1,210 @@
#include "google/sandbox/templates.h"
#include "google/errors.h"
#include "google/sandbox/caps.h"
namespace molo::google::sandbox {
using clients::body::Json;
namespace {
std::string str(const Json& doc, const char* key) {
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
constexpr std::size_t kMaxPorts = 16;
constexpr long kMaxDiskGb = 1024;
bool printable(const std::string& s) {
for (char c : s)
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f) return false;
return true;
}
// Container image references: [A-Za-z0-9._/:@-], 1..512.
bool valid_image(const std::string& s) {
if (s.empty() || s.size() > 512) return false;
for (char c : s)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '.' || c == '_' || c == '/' || c == ':' || c == '@' || c == '-'))
return false;
return true;
}
} // namespace
SandboxTemplate SandboxTemplate::from_json(const Json& doc) {
SandboxTemplate t;
if (!doc.is_object()) return t;
t.raw = doc;
t.name = str(doc, "name");
t.display_name = str(doc, "displayName");
t.state = str(doc, "state");
t.create_time = str(doc, "createTime");
t.update_time = str(doc, "updateTime");
if (const Json* d = doc.find("defaultContainerEnvironment"); d && d->is_object()) {
std::string c = str(*d, "defaultContainerCategory");
if (c == "DEFAULT_CONTAINER_CATEGORY_COMPUTER_USE") t.category = ContainerCategory::kComputerUse;
else if (c == "DEFAULT_CONTAINER_CATEGORY_SHELL_SANDBOX") t.category = ContainerCategory::kShellSandbox;
}
if (const Json* c = doc.find("customContainerEnvironment"); c && c->is_object())
if (const Json* s = c->find("customContainerSpec"))
t.image_uri = str(*s, "imageUri");
if (const Json* e = doc.find("egressControlConfig"); e && e->is_object())
if (const Json* i = e->find("internetAccess"); i && i->is_bool())
t.internet_access = i->as_bool();
return t;
}
Json SandboxTemplate::to_json() const {
Json o = Json::object();
o.set("name", Json(name));
o.set("display_name", Json(display_name));
o.set("state", Json(state));
o.set("category", Json(category == ContainerCategory::kComputerUse ? "computer"
: category == ContainerCategory::kShellSandbox ? "shell"
: ""));
o.set("image_uri", Json(image_uri));
o.set("internet_access", Json(internet_access));
o.set("create_time", Json(create_time));
return o;
}
Templates::Templates(std::shared_ptr<GoogleClient> client, Sleeper sleeper)
: SandboxService(std::move(client)), sleeper_(std::move(sleeper)) {}
void Templates::validate(const CreateTemplateRequest& r) {
if (r.display_name.empty() ||
r.display_name.size() > SandboxCaps::kMaxDisplayNameChars ||
!printable(r.display_name))
throw GoogleError::invalid("display_name must be 1.." +
std::to_string(SandboxCaps::kMaxDisplayNameChars) +
" printable characters");
bool has_category = r.category != ContainerCategory::kNone;
bool has_image = !r.image_uri.empty();
if (has_category == has_image)
throw GoogleError::invalid("give exactly one of a container category or an image");
if (has_image && !valid_image(r.image_uri))
throw GoogleError::invalid("image_uri has invalid characters");
if (!has_image && !r.ports.empty())
throw GoogleError::invalid("ports apply to custom containers only");
if (r.ports.size() > kMaxPorts)
throw GoogleError::invalid("more than 16 ports");
for (const TemplatePort& p : r.ports) {
if (p.protocol != "TCP" && p.protocol != "UDP")
throw GoogleError::invalid("port protocol must be TCP or UDP");
if (p.port < 1 || p.port > 65535) throw GoogleError::invalid("port must be 1..65535");
}
if (r.network_attachment.size() > SandboxCaps::kMaxResourceNameChars ||
!printable(r.network_attachment) ||
r.network_attachment.find("..") != std::string::npos)
throw GoogleError::invalid("network_attachment is malformed");
if (r.persistent_disk) {
if (!valid_workspace_path(r.disk_mount_path) || r.disk_mount_path[0] != '/')
throw GoogleError::invalid("disk_mount_path must be an absolute plain path");
if (r.disk_size_gb < 1 || r.disk_size_gb > kMaxDiskGb)
throw GoogleError::invalid("disk_size_gb must be 1..1024");
} else if (!r.disk_mount_path.empty() || r.disk_size_gb != 0) {
throw GoogleError::invalid("disk settings need persistent_disk");
}
}
Json Templates::create_body(const CreateTemplateRequest& r) {
Json body = Json::object();
body.set("displayName", Json(r.display_name));
if (r.category != ContainerCategory::kNone) {
Json d = Json::object();
d.set("defaultContainerCategory",
Json(r.category == ContainerCategory::kComputerUse
? "DEFAULT_CONTAINER_CATEGORY_COMPUTER_USE"
: "DEFAULT_CONTAINER_CATEGORY_SHELL_SANDBOX"));
body.set("defaultContainerEnvironment", d);
} else {
Json spec = Json::object();
spec.set("imageUri", Json(r.image_uri));
Json c = Json::object();
c.set("customContainerSpec", spec);
if (!r.ports.empty()) {
Json ports = Json::array();
for (const TemplatePort& p : r.ports) {
Json one = Json::object();
one.set("protocol", Json(p.protocol));
one.set("port", Json(p.port));
ports.push(one);
}
c.set("ports", ports);
}
body.set("customContainerEnvironment", c);
}
Json egress = Json::object();
egress.set("internetAccess", Json(r.internet_access));
if (!r.network_attachment.empty())
egress.set("networkAttachment", Json(r.network_attachment));
body.set("egressControlConfig", egress);
if (r.persistent_disk) {
Json disk = Json::object();
disk.set("enabled", Json(true));
disk.set("mountPath", Json(r.disk_mount_path));
disk.set("sizeGb", Json(std::to_string(r.disk_size_gb))); // int64 is a string.
body.set("persistentDiskConfig", disk);
}
return body;
}
SandboxTemplate Templates::create(const std::string& engine_name,
const CreateTemplateRequest& request) const {
SandboxNames::Parts parts = SandboxNames::parse_engine(engine_name);
validate(request);
ServiceEndpoint ep = beta_endpoint_in(parts.location);
Json body = create_body(request);
Operation op = Operation::from_json(client_->request(
clients::Method::kPost, ep, "/" + engine_name + "/sandboxEnvironmentTemplates",
&body));
if (!op.done) {
if (op.name.empty())
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
op = wait_operation(*client_, ep, op.name, sleeper_);
} else if (op.failed()) {
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
}
return SandboxTemplate::from_json(op.response);
}
SandboxTemplate Templates::get(const std::string& template_name) const {
SandboxNames::Parts p = SandboxNames::parse_template(template_name);
return SandboxTemplate::from_json(
client_->get_json(beta_endpoint_in(p.location), "/" + template_name));
}
std::vector<SandboxTemplate> Templates::list(const std::string& engine_name,
int page_size) const {
SandboxNames::Parts p = SandboxNames::parse_engine(engine_name);
if (page_size == 0) page_size = SandboxCaps::kDefaultPageSize;
if (page_size < 1 || page_size > SandboxCaps::kMaxPageSize)
throw GoogleError::invalid("page_size must be 1.." +
std::to_string(SandboxCaps::kMaxPageSize));
Json items = list_all(*client_, beta_endpoint_in(p.location),
"/" + engine_name + "/sandboxEnvironmentTemplates?pageSize=" +
std::to_string(page_size),
"sandboxEnvironmentTemplates", SandboxCaps::kMaxListPages);
std::vector<SandboxTemplate> out;
for (const Json& item : items.as_array()) out.push_back(SandboxTemplate::from_json(item));
return out;
}
void Templates::remove(const std::string& template_name) const {
SandboxNames::Parts p = SandboxNames::parse_template(template_name);
ServiceEndpoint ep = beta_endpoint_in(p.location);
Operation op = Operation::from_json(client_->delete_json(ep, "/" + template_name));
if (!op.done) {
if (op.name.empty())
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
wait_operation(*client_, ep, op.name, sleeper_);
} else if (op.failed()) {
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
}
}
} // namespace molo::google::sandbox
+68
View File
@@ -0,0 +1,68 @@
#pragma once
// SandboxEnvironmentTemplate (v1beta1): the container recipe a computer use
// or shell sandbox is created from. Create, get, list, delete.
#include <string>
#include <vector>
#include "google/operation.h"
#include "google/sandbox/sandbox_service.h"
namespace molo::google::sandbox {
enum class ContainerCategory { kNone, kComputerUse, kShellSandbox };
struct TemplatePort {
std::string protocol = "TCP"; // TCP or UDP
int port = 0; // 1..65535
};
struct CreateTemplateRequest {
std::string display_name; // Required, 1..128 characters.
// Exactly one of: a default container category, or a custom container image.
ContainerCategory category = ContainerCategory::kNone;
std::string image_uri; // customContainerEnvironment.customContainerSpec.imageUri
std::vector<TemplatePort> ports; // Custom container only, at most 16.
bool internet_access = false; // egressControlConfig.internetAccess
std::string network_attachment; // egressControlConfig.networkAttachment
bool persistent_disk = false; // persistentDiskConfig.enabled
std::string disk_mount_path; // absolute path, plain characters
long disk_size_gb = 0; // 1..1024 when the disk is enabled
};
struct SandboxTemplate {
std::string name;
std::string display_name;
std::string state; // ACTIVE, PROVISIONING, FAILED, ...
std::string create_time;
std::string update_time;
ContainerCategory category = ContainerCategory::kNone;
std::string image_uri;
bool internet_access = false;
clients::body::Json raw; // Whole document, unknown fields kept.
static SandboxTemplate from_json(const clients::body::Json& doc);
clients::body::Json to_json() const;
};
class Templates : public SandboxService {
public:
explicit Templates(std::shared_ptr<GoogleClient> client, Sleeper sleeper = nullptr);
SandboxTemplate create(const std::string& engine_name,
const CreateTemplateRequest& request) const;
SandboxTemplate get(const std::string& template_name) const;
std::vector<SandboxTemplate> list(const std::string& engine_name,
int page_size = 0) const;
// Explicit delete (long-running; waited on, capped).
void remove(const std::string& template_name) const;
static void validate(const CreateTemplateRequest& request);
static clients::body::Json create_body(const CreateTemplateRequest& request);
private:
Sleeper sleeper_;
};
} // namespace molo::google::sandbox
+163
View File
@@ -0,0 +1,163 @@
#!/usr/bin/env python3
"""Fails when a call, request field, schema field or enum value declared in
src/google/sandbox/calls.h is absent from Google's Discovery documents.
Usage: python3 -I check_discovery.py [--cache DIR]
Stdlib only, no credentials (Discovery documents are public). Exit 0: every
declared row exists. Exit 1: at least one row is missing (listed). Exit 2: a
document could not be fetched (loud, never a silent pass).
"""
import json
import os
import re
import sys
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
CALLS_H = os.path.join(HERE, "..", "calls.h")
ROW = re.compile(r'^\s*\{((?:"(?:[^"\\]|\\.)*"\s*,?\s*)+)\},?\s*$')
STR = re.compile(r'"((?:[^"\\]|\\.)*)"')
def rows(text, begin, end):
block = text.split(begin, 1)[1].split(end, 1)[0]
out = []
for line in block.splitlines():
m = ROW.match(line)
if m:
out.append([s for s in STR.findall(m.group(1))])
return out
_docs = {}
def discovery(service, version, cache):
key = (service, version)
if key in _docs:
return _docs[key]
path = os.path.join(cache, f"{service}_{version}.json") if cache else None
if path and os.path.exists(path):
doc = json.load(open(path))
else:
url = f"https://{service}.googleapis.com/$discovery/rest?version={version}"
try:
with urllib.request.urlopen(url, timeout=120) as r:
raw = r.read()
except Exception as e: # noqa: BLE001
print(f"FETCH FAILED {url}: {e}")
sys.exit(2)
doc = json.loads(raw)
if path:
os.makedirs(cache, exist_ok=True)
open(path, "wb").write(raw)
_docs[key] = doc
return doc
def methods(doc):
found = {}
def walk(res):
for r in res.values():
for m in r.get("methods", {}).values():
found[m["id"]] = m
walk(r.get("resources", {}))
walk(doc.get("resources", {}))
return found
def schema_id(service, version, name):
prefix = {"aiplatform": "GoogleCloudAiplatform"}[service]
return prefix + version[0].upper() + version[1:] + name
def resolve(doc, prop):
"""Follow $ref and array items to the object schema of a property."""
if "items" in prop:
prop = prop["items"]
if "$ref" in prop:
return doc["schemas"][prop["$ref"]]
return prop
def has_field_path(doc, schema, path):
cur = schema
for part in path.split("."):
props = cur.get("properties", {})
if part not in props:
return False
cur = resolve(doc, props[part])
return True
def main():
cache = None
if "--cache" in sys.argv:
cache = sys.argv[sys.argv.index("--cache") + 1]
text = open(CALLS_H).read()
calls = rows(text, "// CALLS-BEGIN", "// CALLS-END")
schemas = rows(text, "// SCHEMAS-BEGIN", "// SCHEMAS-END")
enums = rows(text, "// ENUMS-BEGIN", "// ENUMS-END")
bad = []
checked = 0
for name, http, path, service, version, rpc, request in calls:
doc = discovery(service, version, cache)
m = methods(doc).get(rpc)
if m is None:
bad.append(f"{name}: rpc {rpc} missing in {service} {version}")
continue
if m["httpMethod"] != http:
bad.append(f"{name}: http {m['httpMethod']} != declared {http}")
if m["path"] != f"{version}/{path}":
bad.append(f"{name}: path {m['path']} != declared {version}/{path}")
if request:
ref = m.get("request", {}).get("$ref")
if not ref:
bad.append(f"{name}: declares request fields but the method has no body")
else:
for f in request.split(","):
if not has_field_path(doc, doc["schemas"][ref], f):
bad.append(f"{name}: request field {f} missing in {ref}")
checked += 1
else:
checked += 1
elif m.get("request") and doc["schemas"][m["request"]["$ref"]].get("properties"):
bad.append(f"{name}: method takes a body {m['request']['$ref']} but none is declared")
checked += 3
for service, version, name, fields in schemas:
doc = discovery(service, version, cache)
s = doc["schemas"].get(schema_id(service, version, name))
if s is None:
bad.append(f"schema {name} missing in {service} {version}")
continue
for f in fields.split(","):
checked += 1
if f not in s.get("properties", {}):
bad.append(f"schema {name}: field {f} missing in {service} {version}")
for service, version, name, field, values in enums:
doc = discovery(service, version, cache)
s = doc["schemas"].get(schema_id(service, version, name))
prop = s.get("properties", {}).get(field) if s else None
if prop is None:
bad.append(f"enum {name}.{field} missing in {service} {version}")
continue
listed = set(prop.get("enum", []))
for v in values.split(","):
checked += 1
if v not in listed:
bad.append(f"enum {name}.{field}: value {v} not listed in {service} {version}")
for b in bad:
print("MISSING", b)
print(f"{len(calls)} calls, {len(schemas)} schemas, {len(enums)} enums, "
f"{checked} checks, {len(bad)} missing")
sys.exit(1 if bad else 0)
if __name__ == "__main__":
main()
+133
View File
@@ -0,0 +1,133 @@
#include "google/sandbox/types.h"
namespace molo::google::sandbox {
using clients::body::Json;
const char* kind_name(SandboxKind kind) {
switch (kind) {
case SandboxKind::kCodeExecution: return "code";
case SandboxKind::kComputerUse: return "computer";
case SandboxKind::kShell: return "shell";
}
return "code";
}
bool kind_from_name(const std::string& name, SandboxKind& out) {
if (name == "code") out = SandboxKind::kCodeExecution;
else if (name == "computer") out = SandboxKind::kComputerUse;
else if (name == "shell") out = SandboxKind::kShell;
else return false;
return true;
}
const char* language_enum(CodeLanguage language) {
switch (language) {
case CodeLanguage::kPython: return "LANGUAGE_PYTHON";
case CodeLanguage::kJavaScript: return "LANGUAGE_JAVASCRIPT";
case CodeLanguage::kUnspecified: break;
}
return "LANGUAGE_UNSPECIFIED";
}
bool language_from_name(const std::string& name, CodeLanguage& out) {
if (name == "python") out = CodeLanguage::kPython;
else if (name == "javascript") out = CodeLanguage::kJavaScript;
else if (name.empty()) out = CodeLanguage::kUnspecified;
else return false;
return true;
}
Json SandboxSpec::to_json() const {
Json spec = Json::object();
switch (kind) {
case SandboxKind::kCodeExecution: {
Json env = Json::object();
if (language != CodeLanguage::kUnspecified)
env.set("codeLanguage", Json(language_enum(language)));
if (machine == MachineConfig::kVcpu4Ram4Gib)
env.set("machineConfig", Json("MACHINE_CONFIG_VCPU4_RAM4GIB"));
spec.set("codeExecutionEnvironment", env);
break;
}
case SandboxKind::kComputerUse:
spec.set("computerUseEnvironment", Json::object());
break;
case SandboxKind::kShell:
spec.set("shellEnvironment", Json::object());
break;
}
return spec;
}
namespace {
std::string str(const Json& doc, const char* key) {
const Json* v = doc.find(key);
return v && v->is_string() ? v->as_string() : "";
}
} // namespace
SandboxEnvironment SandboxEnvironment::from_json(const Json& doc) {
SandboxEnvironment e;
if (!doc.is_object()) return e;
e.name = str(doc, "name");
e.display_name = str(doc, "displayName");
e.state = str(doc, "state");
e.create_time = str(doc, "createTime");
e.update_time = str(doc, "updateTime");
e.expire_time = str(doc, "expireTime");
e.template_name = str(doc, "sandboxEnvironmentTemplate");
e.snapshot_name = str(doc, "sandboxEnvironmentSnapshot");
e.latest_snapshot_name = str(doc, "latestSandboxEnvironmentSnapshot");
e.owner = str(doc, "owner");
if (const Json* ci = doc.find("connectionInfo"); ci && ci->is_object()) {
e.load_balancer_hostname = str(*ci, "loadBalancerHostname");
e.routing_token = str(*ci, "routingToken");
}
if (const Json* spec = doc.find("spec"); spec && spec->is_object()) {
e.spec = *spec;
if (spec->has("codeExecutionEnvironment")) e.kind = SandboxKind::kCodeExecution;
else if (spec->has("computerUseEnvironment")) e.kind = SandboxKind::kComputerUse;
else if (spec->has("shellEnvironment")) e.kind = SandboxKind::kShell;
}
return e;
}
Json SandboxEnvironment::to_json() const {
Json o = Json::object();
o.set("name", Json(name));
o.set("display_name", Json(display_name));
o.set("state", Json(state));
o.set("kind", Json(kind ? kind_name(*kind) : ""));
o.set("create_time", Json(create_time));
o.set("expire_time", Json(expire_time));
o.set("template", Json(template_name));
o.set("latest_snapshot", Json(latest_snapshot_name));
return o;
}
Json ExecutionResult::to_json() const {
Json o = Json::object();
o.set("stdout", Json(std_out));
o.set("stderr", Json(std_err));
o.set("exit_status", exit_status ? Json(*exit_status) : Json(nullptr));
o.set("duration_ms", Json(duration_ms));
o.set("ok", Json(ok()));
o.set("stdout_truncated", Json(out_truncated));
o.set("stderr_truncated", Json(err_truncated));
Json fs = Json::array();
for (const ProducedFile& f : files) {
Json one = Json::object();
one.set("name", Json(f.name));
one.set("mime_type", Json(f.mime_type));
one.set("bytes", Json(static_cast<long>(f.content.size())));
fs.push(one);
}
o.set("files", fs);
Json sk = Json::array();
for (const std::string& s : skipped) sk.push(Json(s));
o.set("skipped", sk);
return o;
}
} // namespace molo::google::sandbox
+93
View File
@@ -0,0 +1,93 @@
#pragma once
// Typed values of the sandbox module: the SandboxEnvironment resource, create
// request, and the execution result shared by code, shell and computer use.
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
namespace molo::google::sandbox {
// The three mutually exclusive environment categories of SandboxEnvironmentSpec.
enum class SandboxKind { kCodeExecution, kComputerUse, kShell };
const char* kind_name(SandboxKind kind); // "code", "computer", "shell"
bool kind_from_name(const std::string& name, SandboxKind& out);
enum class CodeLanguage { kUnspecified, kPython, kJavaScript };
enum class MachineConfig { kUnspecified, kVcpu4Ram4Gib };
const char* language_enum(CodeLanguage language); // LANGUAGE_*
bool language_from_name(const std::string& name, CodeLanguage& out); // python, javascript
struct SandboxSpec {
SandboxKind kind = SandboxKind::kCodeExecution;
CodeLanguage language = CodeLanguage::kUnspecified; // Code execution only.
MachineConfig machine = MachineConfig::kUnspecified; // Code execution only.
clients::body::Json to_json() const;
};
struct CreateSandboxRequest {
std::string display_name; // Required, 1..128 characters.
SandboxSpec spec;
long ttl_seconds = 3600; // 60 s .. 14 days.
// Optional SandboxEnvironmentTemplate resource name (computer use needs one).
std::string template_name;
// Optional SandboxEnvironmentSnapshot to restore from, and snapshot owner.
std::string snapshot_name;
std::string owner;
};
struct SandboxEnvironment {
std::string name;
std::string display_name;
std::string state; // STATE_RUNNING, STATE_PAUSED, ...
std::string create_time;
std::string update_time;
std::string expire_time;
std::string template_name;
std::string snapshot_name; // sandboxEnvironmentSnapshot
std::string latest_snapshot_name; // latestSandboxEnvironmentSnapshot
std::string owner;
std::string load_balancer_hostname; // connectionInfo (data plane address)
std::string routing_token;
std::optional<SandboxKind> kind;
clients::body::Json spec; // Raw spec, unknown fields kept.
bool running() const { return state == "STATE_RUNNING"; }
static SandboxEnvironment from_json(const clients::body::Json& doc);
clients::body::Json to_json() const;
};
struct InputFile {
std::string name;
std::string content; // Raw bytes.
};
struct ProducedFile {
std::string name;
std::string mime_type;
std::string content; // Raw bytes.
};
// Typed outcome of one execute call. Text is capped and flagged, files over a
// cap are skipped loudly (never silently dropped).
struct ExecutionResult {
std::string std_out;
std::string std_err;
std::optional<int> exit_status; // Shell: returncode. Code execution: unknown.
long duration_ms = -1; // -1 unknown.
std::vector<ProducedFile> files;
bool out_truncated = false;
bool err_truncated = false;
std::vector<std::string> skipped; // One reason per skipped chunk or file.
bool ok() const { return exit_status.value_or(0) == 0; }
clients::body::Json to_json() const; // Files as name, mime, size (no bytes).
};
} // namespace molo::google::sandbox
+115
View File
@@ -0,0 +1,115 @@
#include "google/sandbox/workspace_files.h"
#include "google/errors.h"
#include "google/sandbox/caps.h"
#include "security/crypto/Base64.h"
namespace molo::google::sandbox {
using security::crypto::Base64;
namespace {
// Paths hold only [A-Za-z0-9._/-] (valid_workspace_path), so single quotes
// can never be broken out of.
std::string quoted(const std::string& s) { return "'" + s + "'"; }
void check_ok(const ExecutionResult& r, const std::string& what) {
if (!r.exit_status)
throw GoogleError(502, "BAD_RESPONSE", what + ": no exit status in the reply", "aiplatform-sandbox");
if (*r.exit_status != 0)
throw GoogleError(502, "COMMAND_FAILED",
what + " failed with exit " + std::to_string(*r.exit_status) + ": " +
r.std_err.substr(0, 200),
"aiplatform-sandbox");
}
} // namespace
std::string WorkspaceFiles::resolve(const std::string& path) {
if (!valid_workspace_path(path)) throw GoogleError::invalid("bad path");
std::string full = path[0] == '/' ? path : std::string(kRoot) + "/" + path;
const std::string root = kRoot;
if (full != root && full.rfind(root + "/", 0) != 0)
throw GoogleError::invalid("path must stay under " + root);
while (full.find("//") != std::string::npos) full.erase(full.find("//"), 1);
return full;
}
std::string WorkspaceFiles::write_command(const std::string& p, const std::string& content) {
return "mkdir -p \"$(dirname " + quoted(p) + ")\" && printf %s " +
quoted(Base64::encode(content)) + " | base64 -d > " + quoted(p);
}
std::string WorkspaceFiles::read_command(const std::string& p) {
return "test -f " + quoted(p) + " && { [ \"$(stat -c %s " + quoted(p) + ")\" -le " +
std::to_string(SandboxCaps::kMaxShellReadBytes) +
" ] || { echo too_large >&2; exit 3; }; base64 -w0 < " + quoted(p) + "; }";
}
std::string WorkspaceFiles::list_command(const std::string& p) {
return "ls -1Ap -- " + quoted(p);
}
std::string WorkspaceFiles::remove_command(const std::string& p) {
return "test -f " + quoted(p) + " && rm -f -- " + quoted(p);
}
void WorkspaceFiles::write(const std::string& sandbox_name, const std::string& path,
const std::string& content) const {
std::string p = resolve(path);
if (p == kRoot) throw GoogleError::invalid("path names the workspace root");
if (content.size() > SandboxCaps::kMaxShellWriteBytes)
throw GoogleError::invalid("content over " +
std::to_string(SandboxCaps::kMaxShellWriteBytes) +
" bytes (use a code sandbox for larger files)");
ShellRequest req;
req.command = write_command(p, content);
check_ok(shell_.run_internal(sandbox_name, req), "write");
}
std::string WorkspaceFiles::read(const std::string& sandbox_name,
const std::string& path) const {
std::string p = resolve(path);
ShellRequest req;
req.command = read_command(p);
ExecutionResult r = shell_.run_internal(sandbox_name, req);
check_ok(r, "read");
std::string raw;
if (r.out_truncated || !Base64::decode(r.std_out, raw))
throw GoogleError(502, "BAD_RESPONSE", "file content is not valid base64", "aiplatform-sandbox");
return raw;
}
std::vector<std::string> WorkspaceFiles::list(const std::string& sandbox_name,
const std::string& dir) const {
std::string p = resolve(dir.empty() ? std::string(kRoot) : dir);
ShellRequest req;
req.command = list_command(p);
ExecutionResult r = shell_.run_internal(sandbox_name, req);
check_ok(r, "list");
std::vector<std::string> out;
std::string line;
for (char c : r.std_out) {
if (c == '\n') {
if (!line.empty()) out.push_back(line);
line.clear();
} else {
line.push_back(c);
}
if (out.size() >= 1000) break; // Hard cap on entries.
}
if (!line.empty() && out.size() < 1000) out.push_back(line);
return out;
}
void WorkspaceFiles::remove_file(const std::string& sandbox_name,
const std::string& path) const {
std::string p = resolve(path);
if (p == kRoot) throw GoogleError::invalid("path names the workspace root");
ShellRequest req;
req.command = remove_command(p);
check_ok(shell_.run_internal(sandbox_name, req), "remove");
}
} // namespace molo::google::sandbox
+43
View File
@@ -0,0 +1,43 @@
#pragma once
// File management inside a shell sandbox. LOCAL ABSTRACTION: Google has no
// file REST surface for workspaces (SPEC: Workspaces). Every operation is a
// shell command through ShellRunner, with paths validated and single-quoted.
// Bytes travel base64 in the command line, so sizes are small by cap; large
// files belong in a code sandbox's `files` input/output.
#include <string>
#include <vector>
#include "google/sandbox/shell_runner.h"
namespace molo::google::sandbox {
class WorkspaceFiles {
public:
explicit WorkspaceFiles(const ShellRunner& shell) : shell_(shell) {}
static constexpr const char* kRoot = "/workspace";
// Relative paths resolve under /workspace; absolute paths must stay under it.
static std::string resolve(const std::string& path);
void write(const std::string& sandbox_name, const std::string& path,
const std::string& content) const;
std::string read(const std::string& sandbox_name, const std::string& path) const;
std::vector<std::string> list(const std::string& sandbox_name,
const std::string& dir = "") const;
// Removes one regular file (rm -f, never recursive). Explicit method.
void remove_file(const std::string& sandbox_name, const std::string& path) const;
// The exact commands (compared in tests).
static std::string write_command(const std::string& resolved, const std::string& content);
static std::string read_command(const std::string& resolved);
static std::string list_command(const std::string& resolved);
static std::string remove_command(const std::string& resolved);
private:
const ShellRunner& shell_;
};
} // namespace molo::google::sandbox