Merge sandbox module (Agent Sandbox, Workspaces)
This commit is contained in:
@@ -0,0 +1,245 @@
|
||||
# Agent Sandbox - Specification
|
||||
|
||||
Owned directory: `src/google/sandbox/`. Namespace `molo::google::sandbox`.
|
||||
Depends on the Google foundation (`GoogleClient`, `Service`, `GoogleError`,
|
||||
`wait_operation`, `list_all`, `CloudContext`) and the Base64 codec. No other
|
||||
module is included.
|
||||
|
||||
Purpose: create, inspect, pause, snapshot and delete Agent Sandbox
|
||||
environments, run model-generated code, shell commands and browser actions in
|
||||
them, and move files in and out, with typed results and hard caps.
|
||||
|
||||
## 1. Scope and what exists
|
||||
|
||||
Google's REST surface for this product is the `sandboxEnvironments` family
|
||||
under `projects.locations.reasoningEngines` (Discovery service `aiplatform`,
|
||||
versions `v1` and `v1beta1`). It has three resources: `sandboxEnvironments`
|
||||
(create, get, list, delete, pause, resume, snapshot, authorizeAccess,
|
||||
execute, bidiExecute), `sandboxEnvironmentTemplates` (create, get, list,
|
||||
delete) and `sandboxEnvironmentSnapshots` (get, list, delete). A sandbox's
|
||||
category is one of `codeExecutionEnvironment` (Python or JavaScript),
|
||||
`computerUseEnvironment` (browser) or `shellEnvironment` (bash).
|
||||
|
||||
**Workspaces.** No resource, method or schema named "workspace" exists in the
|
||||
`aiplatform` Discovery documents (v1 and v1beta1, checked 2026-10-24) and the
|
||||
sandbox docs index lists no workspace page. The product-level "Workspaces"
|
||||
(bash commands and file management isolated from core systems) is what the
|
||||
docs call the **shell sandbox**: `execute_bash` returns `stdout`, `stderr`,
|
||||
`returncode`, `duration_ms`, runs as an unprivileged user in `/workspace`, one
|
||||
fresh shell per command. This module implements that as `ShellRunner`.
|
||||
File management has no REST surface: `WorkspaceFiles` is a **local
|
||||
abstraction** that does file work with shell commands through `ShellRunner`.
|
||||
It is named local everywhere and is not a Google API.
|
||||
|
||||
**Computer use.** The documented actions (navigate, click, type, screenshot)
|
||||
are Chrome DevTools Protocol commands sent to the sandbox data plane with a
|
||||
service-account JWT (`generate_access_token`, `send_command`, `/cdp`,
|
||||
`/tabs`, a CDP WebSocket for Playwright, VNC view). The data plane is not in
|
||||
Discovery. `ComputerUse` carries the same CDP commands through
|
||||
`sandboxEnvironments.execute`, the only sandbox call in the REST surface; that
|
||||
mapping is unverified (section 7).
|
||||
|
||||
## 2. Module map (one class or concern per file)
|
||||
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | `SandboxCaps`: static ceilings |
|
||||
| `calls.h` | The one table of every call, schema field and enum used; parsed by `tools/check_discovery.py` |
|
||||
| `names.h/.cpp` | `SandboxNames` (engine, sandbox, template, snapshot names), file name and workspace path validation |
|
||||
| `types.h/.cpp` | `SandboxSpec`, `CreateSandboxRequest`, `SandboxEnvironment`, `InputFile`, `ProducedFile`, `ExecutionResult` |
|
||||
| `chunk.h/.cpp` | `Chunk` wire type and `ChunkCodec` (request and result mapping, output caps) |
|
||||
| `sandbox_service.h/.cpp` | `SandboxService` base: endpoint per location and version, shared `execute` |
|
||||
| `sandboxes.h/.cpp` | `Sandboxes`: create, get, list, pause, resume, snapshot, authorize_access, remove |
|
||||
| `snapshot_types.h`, `snapshots.h/.cpp` | `SandboxSnapshot`, `Snapshots`: get, list, remove |
|
||||
| `templates.h/.cpp` | `SandboxTemplate`, `Templates`: create, get, list, remove |
|
||||
| `code_executor.h/.cpp` | `CodeExecutor`: code plus input files in, typed result out |
|
||||
| `shell_runner.h/.cpp` | `ShellRunner`: bash command with cwd and timeout |
|
||||
| `workspace_files.h/.cpp` | `WorkspaceFiles` (local): write, read, list, remove_file |
|
||||
| `computer_use.h/.cpp` | `ComputerUse`: navigate, click, type_text, screenshot |
|
||||
| `stream_executor.h/.cpp` | `StreamExecutor`: `bidiExecute` (v1beta1), streamed reply |
|
||||
| `tools/check_discovery.py` | Discovery check for `calls.h` |
|
||||
|
||||
## 3. Static caps (see caps.h)
|
||||
|
||||
| Knob | Value |
|
||||
|---|---|
|
||||
| Code per call | 256 KiB |
|
||||
| Shell command | 8192 bytes (workspace helpers: 65536, internal only) |
|
||||
| Input files per call / per file / total | 16 / 4 MiB / 8 MiB |
|
||||
| File name | 1..255, no leading `/`, no `.` or `..` segment, no backslash or control byte |
|
||||
| Workspace path | 256 chars of `[A-Za-z0-9._/-]`, under `/workspace`, no `..` |
|
||||
| Shell file write / read | 32 KiB / 512 KiB (bytes travel in the command line) |
|
||||
| stdout, stderr each | 1 MiB, truncated and flagged |
|
||||
| Output chunks per reply | 64 |
|
||||
| Produced files / per file / total | 16 / 4 MiB / 8 MiB, extras skipped and named in `skipped` |
|
||||
| Screenshot | 4 MiB |
|
||||
| Shell timeout | default 30 s, max 300 s |
|
||||
| Sandbox TTL | 60 s to 14 days (Google keeps state up to 14 days), default 3600 s |
|
||||
| Display name | 1..128 printable |
|
||||
| Resource name | 512 chars |
|
||||
| List page size / pages | 100 (Google default and ceiling) / 10 |
|
||||
| Template ports / disk | 16 / 1..1024 GB |
|
||||
| URL / typed text / coordinate | 2048 / 4096 / 0..10000 |
|
||||
| Stream events | 256 |
|
||||
| LRO wait | foundation `wait_operation`: 600 s, 400 polls |
|
||||
|
||||
Google's own limit is 100 MB of files per request or response. Ours are
|
||||
tighter because file bytes are base64 inside a JSON payload that is base64
|
||||
again inside the Chunk, and the codec caps base64 input at 16 MiB.
|
||||
|
||||
## 4. Contracts
|
||||
|
||||
- Every name and id is validated before any network traffic; bad input
|
||||
throws `GoogleError` 400 `INVALID_ARGUMENT`. No credential: calls throw
|
||||
`NotConfiguredError` (503) from the client before any traffic.
|
||||
- API versions: `v1` for everything it offers; `v1beta1` only where `v1`
|
||||
lacks the field: creating a computer use sandbox (`computerUseEnvironment`),
|
||||
templates (`persistentDiskConfig`, computer use category in the documented
|
||||
flow), and `bidiExecute`.
|
||||
- Create, pause, resume, snapshot and delete are long-running operations;
|
||||
they are waited on (capped) and return the typed resource. Delete is an
|
||||
explicit method (`Sandboxes::remove`, `Templates::remove`,
|
||||
`Snapshots::remove`, `WorkspaceFiles::remove_file`); nothing deletes as a
|
||||
side effect. `remove_file` removes one regular file, never recursively.
|
||||
- Sandboxes are billed while they exist: the TTL is always sent on create
|
||||
(default 1 hour) so a crashed caller does not leak a sandbox. `execute`
|
||||
resets the TTL (per docs).
|
||||
- `ExecutionResult`: `std_out`, `std_err`, `exit_status` (shell only,
|
||||
otherwise empty), `duration_ms`, `files`, truncation flags, `skipped`.
|
||||
Over-cap or unsafe files are never dropped silently: each gets a reason in
|
||||
`skipped`. `ok()` is true when the exit status is absent or 0.
|
||||
- `Chunk.data` and `Metadata.attributes` values are base64 on the wire
|
||||
(Discovery: `format: byte`); the codec exposes raw bytes.
|
||||
- Computer use accepts only `http://` and `https://` URLs.
|
||||
|
||||
## 5. Verified against discovery
|
||||
|
||||
`python3 -I src/google/sandbox/tools/check_discovery.py` fetches
|
||||
`https://aiplatform.googleapis.com/$discovery/rest?version=v1` and `v1beta1`
|
||||
and fails when any row of `calls.h` is absent: 18 calls (rpc id, HTTP method,
|
||||
path, request body field paths including nested ones), 21 schemas (field
|
||||
names) and 7 enums (values).
|
||||
|
||||
Result on 2026-10-24: `18 calls, 21 schemas, 7 enums, 204 checks, 0 missing`.
|
||||
Findings from the check that shaped the code: `computerUseEnvironment`,
|
||||
`useGkeTd` and `persistentDiskConfig` exist only in `v1beta1`; `bidiExecute`
|
||||
exists only in `v1beta1`; `PauseSandboxEnvironmentRequest`,
|
||||
`ResumeSandboxEnvironmentRequest` and `AuthorizeSandboxEnvironmentAccessRequest`
|
||||
have no fields; the snapshot request body is a `SandboxEnvironmentSnapshot`
|
||||
whose `sourceSandboxEnvironment` is required.
|
||||
|
||||
## 6. Verified live
|
||||
|
||||
Read-only GETs on 2026-10-24, project from `GOOGLE_CLOUD_PROJECT`, region
|
||||
us-central1, nothing created. Two credentials were tried.
|
||||
|
||||
OAuth bearer token (service account, `GOOGLE_ACCESS_TOKEN`):
|
||||
|
||||
| Call | Status and body |
|
||||
|---|---|
|
||||
| `GET v1/projects/P/locations/L/reasoningEngines` | 200 `{}` (no engines exist) |
|
||||
| `GET v1beta1/.../reasoningEngines` | 200 `{}` |
|
||||
| `GET v1/.../reasoningEngines/{absent}/sandboxEnvironments` | 404 NOT_FOUND "Reasoning Engine [project number: N, reasoning engine id: I] is not found." |
|
||||
| same path on `v1/.../sandboxEnvironmentSnapshots`, `v1/.../sandboxEnvironmentTemplates`, `v1beta1/.../sandboxEnvironments`, `v1beta1/.../sandboxEnvironmentTemplates` | 404 NOT_FOUND, same message |
|
||||
| `GET v1/...` and `v1beta1/.../sandboxEnvironments/{absent}` | 404 NOT_FOUND "Sandbox Environment (projectNumber: N, reasoningEngineId: I, sandboxEnvironmentId: S) is not found." |
|
||||
|
||||
No SERVICE_DISABLED or PERMISSION_DENIED came back: the Agent Platform API is
|
||||
enabled on the project and the service account may read these collections.
|
||||
The 404s are resource-level (Google names the missing engine or sandbox), so
|
||||
the route, version and name format are real. There is nothing to read beyond
|
||||
that: the project has no reasoning engine, hence no sandbox, template or
|
||||
snapshot, and none may be created here.
|
||||
|
||||
Express API key (`AQ.`, `x-goog-api-key`): the same probes return the same
|
||||
statuses and bodies, so the key reaches these read routes too. A made-up
|
||||
collection (`.../bogusThings`) answers an HTML 404 page, and a non-numeric
|
||||
engine id (`none`) answers 400 INVALID_ARGUMENT; engine ids are numeric.
|
||||
|
||||
Never exercised live (Discovery only): create, pause, resume, snapshot,
|
||||
delete, execute, bidiExecute, authorizeAccess, template create and delete,
|
||||
and every response shape of an existing sandbox. Whether a key may mutate
|
||||
sandboxes is unknown.
|
||||
|
||||
Rerun: `MOLO_LIVE_NET=1 GOOGLE_ACCESS_TOKEN=... GOOGLE_CLOUD_PROJECT=... make
|
||||
google-test` (a key in `GOOGLE_API_KEY` also works). Test
|
||||
`google_live_sandbox.read_only_probe` prints a loud SKIP otherwise. It lists
|
||||
sandboxes of an engine id that cannot exist and gets a sandbox that cannot
|
||||
exist, expecting 404 NOT_FOUND through the real client; it creates nothing.
|
||||
It passed on 2026-10-24 with the OAuth token and with the key.
|
||||
Gemini's built-in `codeExecution` tool inside `generateContent` belongs to the
|
||||
models module and is not used here.
|
||||
|
||||
## 7. Unverified
|
||||
|
||||
Neither Discovery nor the docs state these; each is inferred and isolated in
|
||||
`ChunkCodec` so one edit fixes it:
|
||||
|
||||
- The payload of an input Chunk. Docs show only the SDK call
|
||||
(`execute_code(input_data={"code":..., "files":[{"name","content"}]})`) and
|
||||
an older doc revision says input file content is base64. Assumed: one
|
||||
`application/json` Chunk holding `{"code", "files":[{"name","content"(base64)}]}`.
|
||||
- The payload of a shell input Chunk. Assumed `{"command","cwd","timeout"}`
|
||||
(names from the SDK helper `execute_bash(command, cwd, timeout)`).
|
||||
- The payload of a computer use input Chunk: `{"command":"Page.navigate",
|
||||
"params":{...}}` is the documented `/cdp` body; sending it through `execute`
|
||||
is assumed. Click is `Input.dispatchMouseEvent` pressed then released, typing
|
||||
is `Input.insertText`, screenshot is `Page.captureScreenshot` (CDP names, not
|
||||
Google names).
|
||||
- Output Chunk payloads: docs show an `application/json` Chunk with `msg_out`,
|
||||
`msg_err`, `output_files[{content (base64)}]`, plus Chunks with a `file_name`
|
||||
attribute. The `name` key inside `output_files` and the shell keys `stdout`,
|
||||
`stderr`, `returncode`, `duration_ms` (seen only in the SDK dict) are
|
||||
assumed. Unknown shapes are skipped loudly, never guessed.
|
||||
- Whether `attributes` values are really base64 on the wire (Discovery says
|
||||
`byte`; docs show plain text). The decoder tries base64 and falls back to the
|
||||
text as written.
|
||||
- Whether `bidiExecute` over this client's POST stream behaves as a server
|
||||
stream; it is a bidirectional method and may need a persistent channel.
|
||||
- Which responses carry which `@type` inside an Operation `response`
|
||||
(`SandboxEnvironment`, `SandboxEnvironmentSnapshot`, template); parsed by
|
||||
field name only.
|
||||
- The `filter` list parameter exists (AIP-160) but its grammar is not
|
||||
documented; it is not exposed.
|
||||
|
||||
## 8. Gaps
|
||||
|
||||
- The sandbox data plane (HTTP proxy with JWT, `/tabs`, `/cdp`, CDP
|
||||
WebSocket, VNC) is not implemented; it needs a service-account JWT signer
|
||||
(foundation gap in `credentials.cpp`) and a WebSocket client (module `live`).
|
||||
Owner of the follow-up: this module, `computer_use.cpp`.
|
||||
- True bidirectional `bidiExecute` input (see Unverified).
|
||||
- `generate_access_token` and `generate_browser_ws_headers` (SDK helpers) have
|
||||
no REST method in Discovery.
|
||||
- Template update, custom container resources/requests, DNS peering, private
|
||||
service connect ingress, CMEK and VPC-SC settings are not typed. Templates
|
||||
keep the whole document in `SandboxTemplate::raw`.
|
||||
- Foundation: `Service::call` always posts to the one endpoint a service was
|
||||
built with; this module needs a different location and version per call, so
|
||||
`SandboxService` builds endpoints itself and talks to `GoogleClient`
|
||||
directly.
|
||||
- The example server's flat JSON reader does not decode `\n`; the demo uses
|
||||
the real JSON parser for request bodies.
|
||||
|
||||
## 9. Sources
|
||||
|
||||
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/code-execution-overview
|
||||
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/code-execution-quickstart
|
||||
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/computer-use
|
||||
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/computer-use-quickstart
|
||||
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/shell-sandbox-quickstart
|
||||
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sandbox/manage-sandboxes
|
||||
- https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1beta1/projects.locations.reasoningEngines.sandboxEnvironments (and its `create`, `execute`, `list`, `delete`, `pause`, `authorizeAccess` pages) and `.../v1beta1/Chunk`
|
||||
- https://aiplatform.googleapis.com/$discovery/rest?version=v1
|
||||
- https://aiplatform.googleapis.com/$discovery/rest?version=v1beta1
|
||||
- Not read (listed on the docs index): `manage-templates`, `manage-snapshots`,
|
||||
`configure-vpc-sc`, `configure-cmek`, `custom-containers`,
|
||||
`troubleshooting/code-execution`.
|
||||
|
||||
## 10. Example
|
||||
|
||||
`example/app/agent/SandboxDemo.cpp`, page script `example/assets/agent_sandbox.js`.
|
||||
GET `/agent/sandbox` (page, matrix row `agent-sandbox`), GET
|
||||
`/agent/sandbox/status`. POST `/agent/sandbox/{create,list,get,pause,resume,
|
||||
snapshot,snapshots,templates,execute,shell,computer,files}`, DELETE
|
||||
`/agent/sandbox/delete?name=`. Without a credential every Google route is 503
|
||||
`google_not_configured`; bad input is 422. Tests: `tests/example/agent/sandbox/`.
|
||||
@@ -0,0 +1,97 @@
|
||||
#pragma once
|
||||
|
||||
// The ONE table of every Google call this module makes, plus the schema
|
||||
// fields and enum values it reads or writes. tools/check_discovery.py parses
|
||||
// this file (one brace row per line, double-quoted strings only) and fails if
|
||||
// the Discovery document of the named service and version lacks any row.
|
||||
// Keep every row on one line.
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
struct DeclaredCall {
|
||||
const char* name; // Local name.
|
||||
const char* http; // HTTP method.
|
||||
const char* path; // Discovery "path" without the version prefix.
|
||||
const char* service; // Discovery service host, e.g. aiplatform.
|
||||
const char* version; // v1 or v1beta1.
|
||||
const char* rpc; // Discovery method id.
|
||||
const char* request; // Comma list of request body field paths ("" none).
|
||||
};
|
||||
|
||||
struct DeclaredSchema {
|
||||
const char* service;
|
||||
const char* version;
|
||||
const char* schema; // Discovery schema id without the GoogleCloudAiplatform<Version> prefix.
|
||||
const char* fields; // Comma list of properties that must exist.
|
||||
};
|
||||
|
||||
struct DeclaredEnum {
|
||||
const char* service;
|
||||
const char* version;
|
||||
const char* schema;
|
||||
const char* field;
|
||||
const char* values; // Comma list that must all be listed by the enum.
|
||||
};
|
||||
|
||||
// CALLS-BEGIN
|
||||
inline constexpr DeclaredCall kDeclaredCalls[] = {
|
||||
{"sandbox.create", "POST", "{+parent}/sandboxEnvironments", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.create", "displayName,spec,spec.codeExecutionEnvironment,spec.codeExecutionEnvironment.codeLanguage,spec.codeExecutionEnvironment.machineConfig,spec.shellEnvironment,ttl,sandboxEnvironmentTemplate,sandboxEnvironmentSnapshot,owner"},
|
||||
{"sandbox.create.computer", "POST", "{+parent}/sandboxEnvironments", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.create", "displayName,spec,spec.computerUseEnvironment,ttl,sandboxEnvironmentTemplate,sandboxEnvironmentSnapshot,owner"},
|
||||
{"sandbox.get", "GET", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.get", ""},
|
||||
{"sandbox.list", "GET", "{+parent}/sandboxEnvironments", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.list", ""},
|
||||
{"sandbox.delete", "DELETE", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.delete", ""},
|
||||
{"sandbox.pause", "POST", "{+name}:pause", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.pause", ""},
|
||||
{"sandbox.resume", "POST", "{+name}:resume", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.resume", ""},
|
||||
{"sandbox.snapshot", "POST", "{+name}:snapshot", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.snapshot", "displayName,sourceSandboxEnvironment,ttl,postSnapshotAction,owner"},
|
||||
{"sandbox.authorizeAccess", "POST", "{+name}:authorizeAccess", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.authorizeAccess", ""},
|
||||
{"sandbox.execute", "POST", "{+name}:execute", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.execute", "inputs,inputs.mimeType,inputs.data,inputs.metadata,inputs.metadata.attributes"},
|
||||
{"sandbox.bidiExecute", "POST", "{+name}:bidiExecute", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironments.bidiExecute", "inputs,inputs.mimeType,inputs.data,inputs.metadata,inputs.metadata.attributes"},
|
||||
{"template.create", "POST", "{+parent}/sandboxEnvironmentTemplates", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.create", "displayName,defaultContainerEnvironment,defaultContainerEnvironment.defaultContainerCategory,customContainerEnvironment,customContainerEnvironment.customContainerSpec,customContainerEnvironment.customContainerSpec.imageUri,customContainerEnvironment.ports,customContainerEnvironment.ports.protocol,customContainerEnvironment.ports.port,egressControlConfig,egressControlConfig.internetAccess,egressControlConfig.networkAttachment,persistentDiskConfig,persistentDiskConfig.enabled,persistentDiskConfig.mountPath,persistentDiskConfig.sizeGb"},
|
||||
{"template.get", "GET", "{+name}", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.get", ""},
|
||||
{"template.list", "GET", "{+parent}/sandboxEnvironmentTemplates", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.list", ""},
|
||||
{"template.delete", "DELETE", "{+name}", "aiplatform", "v1beta1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentTemplates.delete", ""},
|
||||
{"snapshot.get", "GET", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentSnapshots.get", ""},
|
||||
{"snapshot.list", "GET", "{+parent}/sandboxEnvironmentSnapshots", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentSnapshots.list", ""},
|
||||
{"snapshot.delete", "DELETE", "{+name}", "aiplatform", "v1", "aiplatform.projects.locations.reasoningEngines.sandboxEnvironmentSnapshots.delete", ""},
|
||||
};
|
||||
// CALLS-END
|
||||
|
||||
// SCHEMAS-BEGIN
|
||||
inline constexpr DeclaredSchema kDeclaredSchemas[] = {
|
||||
{"aiplatform", "v1", "SandboxEnvironment", "name,displayName,state,createTime,updateTime,expireTime,ttl,spec,sandboxEnvironmentTemplate,sandboxEnvironmentSnapshot,latestSandboxEnvironmentSnapshot,owner,connectionInfo"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironment", "name,displayName,state,createTime,updateTime,expireTime,ttl,spec,sandboxEnvironmentTemplate,connectionInfo"},
|
||||
{"aiplatform", "v1", "SandboxEnvironmentSpec", "codeExecutionEnvironment,shellEnvironment"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentSpec", "codeExecutionEnvironment,shellEnvironment,computerUseEnvironment"},
|
||||
{"aiplatform", "v1", "SandboxEnvironmentConnectionInfo", "loadBalancerIp,loadBalancerHostname,sandboxInternalIp,routingToken,serviceAttachment"},
|
||||
{"aiplatform", "v1", "Chunk", "mimeType,data,metadata"},
|
||||
{"aiplatform", "v1", "Metadata", "attributes"},
|
||||
{"aiplatform", "v1", "ExecuteSandboxEnvironmentRequest", "inputs"},
|
||||
{"aiplatform", "v1", "ExecuteSandboxEnvironmentResponse", "outputs"},
|
||||
{"aiplatform", "v1beta1", "BidiExecuteSandboxEnvironmentResponse", "outputs"},
|
||||
{"aiplatform", "v1", "ListSandboxEnvironmentsResponse", "sandboxEnvironments,nextPageToken"},
|
||||
{"aiplatform", "v1", "SandboxEnvironmentSnapshot", "name,displayName,sourceSandboxEnvironment,ttl,postSnapshotAction,owner,parentSnapshot,sizeBytes,createTime,updateTime,expireTime"},
|
||||
{"aiplatform", "v1", "ListSandboxEnvironmentSnapshotsResponse", "sandboxEnvironmentSnapshots,nextPageToken"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplate", "name,displayName,state,createTime,updateTime,defaultContainerEnvironment,customContainerEnvironment,egressControlConfig,persistentDiskConfig"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateDefaultContainerEnvironment", "defaultContainerCategory"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateCustomContainerEnvironment", "customContainerSpec,ports,resources"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateCustomContainerSpec", "imageUri"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateNetworkPort", "protocol,port"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateEgressControlConfig", "internetAccess,networkAttachment,dnsPeeringConfigs"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplatePersistentDiskConfig", "enabled,mountPath,sizeGb"},
|
||||
{"aiplatform", "v1beta1", "ListSandboxEnvironmentTemplatesResponse", "sandboxEnvironmentTemplates,nextPageToken"},
|
||||
};
|
||||
// SCHEMAS-END
|
||||
|
||||
// ENUMS-BEGIN
|
||||
inline constexpr DeclaredEnum kDeclaredEnums[] = {
|
||||
{"aiplatform", "v1", "SandboxEnvironmentSpecCodeExecutionEnvironment", "codeLanguage", "LANGUAGE_PYTHON,LANGUAGE_JAVASCRIPT"},
|
||||
{"aiplatform", "v1", "SandboxEnvironmentSpecCodeExecutionEnvironment", "machineConfig", "MACHINE_CONFIG_VCPU4_RAM4GIB"},
|
||||
{"aiplatform", "v1", "SandboxEnvironment", "state", "STATE_PROVISIONING,STATE_RUNNING,STATE_DEPROVISIONING,STATE_TERMINATED,STATE_DELETED,STATE_PAUSED,STATE_PAUSING,STATE_RESUMING,STATE_STOPPING"},
|
||||
{"aiplatform", "v1", "SandboxEnvironmentSnapshot", "postSnapshotAction", "RUNNING,PAUSE"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateDefaultContainerEnvironment", "defaultContainerCategory", "DEFAULT_CONTAINER_CATEGORY_COMPUTER_USE,DEFAULT_CONTAINER_CATEGORY_SHELL_SANDBOX"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplateNetworkPort", "protocol", "TCP,UDP"},
|
||||
{"aiplatform", "v1beta1", "SandboxEnvironmentTemplate", "state", "PROVISIONING,ACTIVE,DEPROVISIONING,DELETED,FAILED"},
|
||||
};
|
||||
// ENUMS-END
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,54 @@
|
||||
#pragma once
|
||||
|
||||
// Static ceilings for the sandbox module. Never autotuned. Google's own limit
|
||||
// is 100 MB of files per request or response; ours are tighter on purpose.
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
struct SandboxCaps {
|
||||
// Requests.
|
||||
static constexpr std::size_t kMaxCodeBytes = 256u * 1024u;
|
||||
static constexpr std::size_t kMaxCommandBytes = 8192;
|
||||
static constexpr std::size_t kMaxInternalCommandBytes = 65536; // Workspace helpers only.
|
||||
static constexpr std::size_t kMaxInputFiles = 16;
|
||||
static constexpr std::size_t kMaxFileBytes = 4u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxInputTotalBytes = 8u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxFileNameChars = 255;
|
||||
static constexpr std::size_t kMaxPathChars = 256;
|
||||
// Shell file helpers ship bytes inside the command line, so they stay small.
|
||||
static constexpr std::size_t kMaxShellWriteBytes = 32u * 1024u;
|
||||
static constexpr std::size_t kMaxShellReadBytes = 512u * 1024u;
|
||||
|
||||
// Responses. Text is truncated (flagged), files over a cap are skipped
|
||||
// (counted and named in ExecutionResult::skipped).
|
||||
static constexpr std::size_t kMaxOutputBytes = 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxOutputChunks = 64;
|
||||
static constexpr std::size_t kMaxProducedFiles = 16;
|
||||
static constexpr std::size_t kMaxProducedFileBytes = 4u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxProducedTotalBytes = 8u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxScreenshotBytes = 4u * 1024u * 1024u;
|
||||
|
||||
// Time. Execution is synchronous; the shell timeout is sent to the sandbox,
|
||||
// the HTTP client's own 60 s request timeout bounds everything else.
|
||||
static constexpr int kDefaultShellTimeoutSeconds = 30;
|
||||
static constexpr int kMaxShellTimeoutSeconds = 300;
|
||||
static constexpr long kMinTtlSeconds = 60;
|
||||
static constexpr long kDefaultTtlSeconds = 3600;
|
||||
static constexpr long kMaxTtlSeconds = 14L * 24 * 3600; // Google: state kept up to 14 days.
|
||||
|
||||
// Names and lists.
|
||||
static constexpr std::size_t kMaxDisplayNameChars = 128;
|
||||
static constexpr std::size_t kMaxResourceNameChars = 512;
|
||||
static constexpr int kDefaultPageSize = 50;
|
||||
static constexpr int kMaxPageSize = 100; // Google default and ceiling.
|
||||
static constexpr std::size_t kMaxListPages = 10;
|
||||
|
||||
// Computer use.
|
||||
static constexpr std::size_t kMaxUrlChars = 2048;
|
||||
static constexpr std::size_t kMaxTypedChars = 4096;
|
||||
static constexpr int kMaxCoordinate = 10000;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,232 @@
|
||||
#include "google/sandbox/chunk.h"
|
||||
|
||||
#include "google/sandbox/caps.h"
|
||||
#include "google/sandbox/names.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
using security::crypto::Base64;
|
||||
|
||||
namespace {
|
||||
|
||||
std::string str(const Json& doc, const char* key) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
void append_capped(std::string& dst, const std::string& add, bool& truncated) {
|
||||
if (dst.size() >= SandboxCaps::kMaxOutputBytes) {
|
||||
if (!add.empty()) truncated = true;
|
||||
return;
|
||||
}
|
||||
std::size_t room = SandboxCaps::kMaxOutputBytes - dst.size();
|
||||
if (add.size() > room) {
|
||||
dst.append(add, 0, room);
|
||||
truncated = true;
|
||||
} else {
|
||||
dst += add;
|
||||
}
|
||||
}
|
||||
|
||||
// Attribute values are "bytes format" (base64). A value that is not valid
|
||||
// base64 is kept as written, so a plain-text server still works.
|
||||
std::string attribute_text(const std::string& wire) {
|
||||
std::string raw;
|
||||
if (!wire.empty() && Base64::decode(wire, raw) && !raw.empty()) return raw;
|
||||
return wire;
|
||||
}
|
||||
|
||||
struct FileBudget {
|
||||
std::size_t total = 0;
|
||||
};
|
||||
|
||||
void add_file(ExecutionResult& r, FileBudget& budget, std::string name,
|
||||
std::string mime, std::string content) {
|
||||
if (!valid_file_name(name)) {
|
||||
r.skipped.push_back("file with unsafe name rejected");
|
||||
return;
|
||||
}
|
||||
if (r.files.size() >= SandboxCaps::kMaxProducedFiles) {
|
||||
r.skipped.push_back("file " + name + " skipped: more than " +
|
||||
std::to_string(SandboxCaps::kMaxProducedFiles) + " files");
|
||||
return;
|
||||
}
|
||||
if (content.size() > SandboxCaps::kMaxProducedFileBytes) {
|
||||
r.skipped.push_back("file " + name + " skipped: over " +
|
||||
std::to_string(SandboxCaps::kMaxProducedFileBytes) + " bytes");
|
||||
return;
|
||||
}
|
||||
if (budget.total + content.size() > SandboxCaps::kMaxProducedTotalBytes) {
|
||||
r.skipped.push_back("file " + name + " skipped: produced total over " +
|
||||
std::to_string(SandboxCaps::kMaxProducedTotalBytes) + " bytes");
|
||||
return;
|
||||
}
|
||||
budget.total += content.size();
|
||||
r.files.push_back({std::move(name), std::move(mime), std::move(content)});
|
||||
}
|
||||
|
||||
void apply_json_chunk(ExecutionResult& r, FileBudget& budget, const Json& doc) {
|
||||
// Code execution: msg_out / msg_err / output_files. Shell: stdout / stderr /
|
||||
// returncode / duration_ms. Both are read; absent keys are ignored.
|
||||
append_capped(r.std_out, str(doc, "msg_out"), r.out_truncated);
|
||||
append_capped(r.std_out, str(doc, "stdout"), r.out_truncated);
|
||||
append_capped(r.std_err, str(doc, "msg_err"), r.err_truncated);
|
||||
append_capped(r.std_err, str(doc, "stderr"), r.err_truncated);
|
||||
for (const char* key : {"returncode", "exit_code", "exit_status"})
|
||||
if (const Json* v = doc.find(key); v && v->is_number())
|
||||
r.exit_status = static_cast<int>(v->as_int());
|
||||
if (const Json* v = doc.find("duration_ms"); v && v->is_number())
|
||||
r.duration_ms = v->as_int();
|
||||
if (const Json* files = doc.find("output_files"); files && files->is_array()) {
|
||||
for (const Json& f : files->as_array()) {
|
||||
std::string name = str(f, "name");
|
||||
if (name.empty()) name = str(f, "file_name");
|
||||
std::string raw;
|
||||
std::string b64 = str(f, "content");
|
||||
if (!Base64::decode(b64, raw)) {
|
||||
r.skipped.push_back("file " + name + " skipped: content is not base64");
|
||||
continue;
|
||||
}
|
||||
std::string mime = str(f, "mime_type");
|
||||
add_file(r, budget, std::move(name),
|
||||
mime.empty() ? "application/octet-stream" : mime, std::move(raw));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Json Chunk::to_json() const {
|
||||
Json o = Json::object();
|
||||
o.set("mimeType", Json(mime_type));
|
||||
o.set("data", Json(Base64::encode(data)));
|
||||
if (!attributes.empty()) {
|
||||
Json attrs = Json::object();
|
||||
for (const auto& [k, v] : attributes) attrs.set(k, Json(Base64::encode(v)));
|
||||
Json meta = Json::object();
|
||||
meta.set("attributes", attrs);
|
||||
o.set("metadata", meta);
|
||||
}
|
||||
return o;
|
||||
}
|
||||
|
||||
Chunk Chunk::from_json(const Json& doc, bool* invalid) {
|
||||
Chunk c;
|
||||
if (invalid) *invalid = false;
|
||||
c.mime_type = str(doc, "mimeType");
|
||||
std::string wire = str(doc, "data");
|
||||
if (!wire.empty() && !Base64::decode(wire, c.data)) {
|
||||
c.data.clear();
|
||||
if (invalid) *invalid = true;
|
||||
}
|
||||
const Json* meta = doc.is_object() ? doc.find("metadata") : nullptr;
|
||||
const Json* attrs = meta && meta->is_object() ? meta->find("attributes") : nullptr;
|
||||
if (attrs && attrs->is_object())
|
||||
for (const auto& [k, v] : attrs->as_object())
|
||||
if (v.is_string()) c.attributes[k] = attribute_text(v.as_string());
|
||||
return c;
|
||||
}
|
||||
|
||||
Chunk Chunk::json_payload(const Json& payload) {
|
||||
Chunk c;
|
||||
c.mime_type = "application/json";
|
||||
c.data = payload.dump();
|
||||
return c;
|
||||
}
|
||||
|
||||
Json ChunkCodec::request(const std::vector<Chunk>& inputs) {
|
||||
Json arr = Json::array();
|
||||
for (const Chunk& c : inputs) arr.push(c.to_json());
|
||||
Json body = Json::object();
|
||||
body.set("inputs", arr);
|
||||
return body;
|
||||
}
|
||||
|
||||
Chunk ChunkCodec::code_input(const std::string& code,
|
||||
const std::vector<InputFile>& files) {
|
||||
Json payload = Json::object();
|
||||
payload.set("code", Json(code));
|
||||
if (!files.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const InputFile& f : files) {
|
||||
Json one = Json::object();
|
||||
one.set("name", Json(f.name));
|
||||
one.set("content", Json(Base64::encode(f.content)));
|
||||
arr.push(one);
|
||||
}
|
||||
payload.set("files", arr);
|
||||
}
|
||||
return Chunk::json_payload(payload);
|
||||
}
|
||||
|
||||
Chunk ChunkCodec::shell_input(const std::string& command, const std::string& cwd,
|
||||
int timeout_seconds) {
|
||||
Json payload = Json::object();
|
||||
payload.set("command", Json(command));
|
||||
if (!cwd.empty()) payload.set("cwd", Json(cwd));
|
||||
payload.set("timeout", Json(timeout_seconds));
|
||||
return Chunk::json_payload(payload);
|
||||
}
|
||||
|
||||
Chunk ChunkCodec::cdp_input(const std::string& command, const Json& params) {
|
||||
Json payload = Json::object();
|
||||
payload.set("command", Json(command));
|
||||
payload.set("params", params);
|
||||
return Chunk::json_payload(payload);
|
||||
}
|
||||
|
||||
std::vector<Chunk> ChunkCodec::outputs(const Json& reply,
|
||||
std::vector<std::string>* skipped) {
|
||||
std::vector<Chunk> out;
|
||||
const Json* arr = reply.is_object() ? reply.find("outputs") : nullptr;
|
||||
if (!arr || !arr->is_array()) return out;
|
||||
std::size_t index = 0;
|
||||
for (const Json& item : arr->as_array()) {
|
||||
if (out.size() >= SandboxCaps::kMaxOutputChunks) {
|
||||
if (skipped)
|
||||
skipped->push_back("chunks beyond " +
|
||||
std::to_string(SandboxCaps::kMaxOutputChunks) + " skipped");
|
||||
break;
|
||||
}
|
||||
bool invalid = false;
|
||||
Chunk c = Chunk::from_json(item, &invalid);
|
||||
if (invalid) {
|
||||
if (skipped)
|
||||
skipped->push_back("chunk " + std::to_string(index) + " skipped: data is not base64");
|
||||
} else {
|
||||
out.push_back(std::move(c));
|
||||
}
|
||||
++index;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
ExecutionResult ChunkCodec::decode(const Json& reply) {
|
||||
ExecutionResult r;
|
||||
FileBudget budget;
|
||||
std::size_t index = 0;
|
||||
for (const Chunk& c : outputs(reply, &r.skipped)) {
|
||||
auto named = c.attributes.find("file_name");
|
||||
if (named != c.attributes.end()) {
|
||||
add_file(r, budget, named->second,
|
||||
c.mime_type.empty() ? "application/octet-stream" : c.mime_type, c.data);
|
||||
} else if (c.mime_type == "application/json") {
|
||||
try {
|
||||
apply_json_chunk(r, budget, Json::parse(c.data));
|
||||
} catch (const clients::body::JsonError&) {
|
||||
r.skipped.push_back("chunk " + std::to_string(index) + " skipped: not valid JSON");
|
||||
}
|
||||
} else if (c.mime_type.rfind("text/", 0) == 0) {
|
||||
append_capped(r.std_out, c.data, r.out_truncated);
|
||||
} else {
|
||||
r.skipped.push_back("chunk " + std::to_string(index) + " skipped: unnamed " +
|
||||
(c.mime_type.empty() ? "untyped" : c.mime_type) + " data");
|
||||
}
|
||||
++index;
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,50 @@
|
||||
#pragma once
|
||||
|
||||
// The Chunk wire type of sandboxEnvironments.execute plus the codec between
|
||||
// it and the typed values. Chunk.data and Metadata.attributes values are
|
||||
// base64 on the wire (REST "bytes format"); here they are raw bytes.
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "google/sandbox/types.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
struct Chunk {
|
||||
std::string mime_type;
|
||||
std::string data; // Raw bytes.
|
||||
std::map<std::string, std::string> attributes; // Raw bytes per value.
|
||||
|
||||
clients::body::Json to_json() const;
|
||||
// Tolerant: a chunk whose data is not valid base64 yields data "" and sets
|
||||
// `invalid` so the decoder can skip it loudly.
|
||||
static Chunk from_json(const clients::body::Json& doc, bool* invalid = nullptr);
|
||||
|
||||
static Chunk json_payload(const clients::body::Json& payload);
|
||||
};
|
||||
|
||||
class ChunkCodec {
|
||||
public:
|
||||
// {"inputs":[chunk,...]}
|
||||
static clients::body::Json request(const std::vector<Chunk>& inputs);
|
||||
|
||||
// Input chunks. The payload shapes are inferred, see SPEC Unverified.
|
||||
static Chunk code_input(const std::string& code,
|
||||
const std::vector<InputFile>& files);
|
||||
static Chunk shell_input(const std::string& command, const std::string& cwd,
|
||||
int timeout_seconds);
|
||||
static Chunk cdp_input(const std::string& command,
|
||||
const clients::body::Json& params);
|
||||
|
||||
// reply["outputs"] to chunks, at most SandboxCaps::kMaxOutputChunks.
|
||||
static std::vector<Chunk> outputs(const clients::body::Json& reply,
|
||||
std::vector<std::string>* skipped = nullptr);
|
||||
|
||||
// Typed result with every cap applied.
|
||||
static ExecutionResult decode(const clients::body::Json& reply);
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,47 @@
|
||||
#include "google/sandbox/code_executor.h"
|
||||
|
||||
#include <set>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sandbox/caps.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
CodeExecutor::CodeExecutor(std::shared_ptr<GoogleClient> client)
|
||||
: SandboxService(std::move(client)) {}
|
||||
|
||||
void CodeExecutor::validate(const std::string& code,
|
||||
const std::vector<InputFile>& files) {
|
||||
if (code.empty()) throw GoogleError::invalid("code is empty");
|
||||
if (code.size() > SandboxCaps::kMaxCodeBytes)
|
||||
throw GoogleError::invalid("code over " +
|
||||
std::to_string(SandboxCaps::kMaxCodeBytes) + " bytes");
|
||||
if (files.size() > SandboxCaps::kMaxInputFiles)
|
||||
throw GoogleError::invalid("more than " +
|
||||
std::to_string(SandboxCaps::kMaxInputFiles) + " files");
|
||||
std::set<std::string> seen;
|
||||
std::size_t total = 0;
|
||||
for (const InputFile& f : files) {
|
||||
if (!valid_file_name(f.name)) throw GoogleError::invalid("bad file name");
|
||||
if (!seen.insert(f.name).second)
|
||||
throw GoogleError::invalid("duplicate file name " + f.name);
|
||||
if (f.content.size() > SandboxCaps::kMaxFileBytes)
|
||||
throw GoogleError::invalid("file " + f.name + " over " +
|
||||
std::to_string(SandboxCaps::kMaxFileBytes) + " bytes");
|
||||
total += f.content.size();
|
||||
}
|
||||
if (total > SandboxCaps::kMaxInputTotalBytes)
|
||||
throw GoogleError::invalid("files over " +
|
||||
std::to_string(SandboxCaps::kMaxInputTotalBytes) +
|
||||
" bytes in total");
|
||||
}
|
||||
|
||||
ExecutionResult CodeExecutor::run(const std::string& sandbox_name,
|
||||
const std::string& code,
|
||||
const std::vector<InputFile>& files) const {
|
||||
SandboxNames::parse_sandbox(sandbox_name);
|
||||
validate(code, files);
|
||||
return execute(sandbox_name, {ChunkCodec::code_input(code, files)});
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
// Code execution in a code sandbox (Python or JavaScript): code plus input
|
||||
// files in, typed stdout / stderr / produced files out. The sandbox keeps its
|
||||
// state between calls and every call resets its TTL (per Google's docs).
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
class CodeExecutor : public SandboxService {
|
||||
public:
|
||||
explicit CodeExecutor(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
ExecutionResult run(const std::string& sandbox_name, const std::string& code,
|
||||
const std::vector<InputFile>& files = {}) const;
|
||||
|
||||
// Throws GoogleError::invalid: empty or oversized code, too many files, bad
|
||||
// names, duplicates, a file or the total over its cap.
|
||||
static void validate(const std::string& code, const std::vector<InputFile>& files);
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,126 @@
|
||||
#include "google/sandbox/computer_use.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sandbox/caps.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
ComputerUse::ComputerUse(std::shared_ptr<GoogleClient> client)
|
||||
: SandboxService(std::move(client)) {}
|
||||
|
||||
void ComputerUse::validate_url(const std::string& url) {
|
||||
bool http = url.rfind("http://", 0) == 0;
|
||||
bool https = url.rfind("https://", 0) == 0;
|
||||
if (!http && !https) throw GoogleError::invalid("url must start with http:// or https://");
|
||||
if (url.size() > SandboxCaps::kMaxUrlChars)
|
||||
throw GoogleError::invalid("url over " + std::to_string(SandboxCaps::kMaxUrlChars) +
|
||||
" characters");
|
||||
if (url.size() <= (https ? 8u : 7u)) throw GoogleError::invalid("url has no host");
|
||||
for (char c : url)
|
||||
if (static_cast<unsigned char>(c) <= 0x20 || c == 0x7f)
|
||||
throw GoogleError::invalid("url contains spaces or control characters");
|
||||
}
|
||||
|
||||
void ComputerUse::validate_point(int x, int y) {
|
||||
if (x < 0 || y < 0 || x > SandboxCaps::kMaxCoordinate || y > SandboxCaps::kMaxCoordinate)
|
||||
throw GoogleError::invalid("coordinates must be 0.." +
|
||||
std::to_string(SandboxCaps::kMaxCoordinate));
|
||||
}
|
||||
|
||||
void ComputerUse::validate_text(const std::string& text) {
|
||||
if (text.empty()) throw GoogleError::invalid("text is empty");
|
||||
if (text.size() > SandboxCaps::kMaxTypedChars)
|
||||
throw GoogleError::invalid("text over " + std::to_string(SandboxCaps::kMaxTypedChars) +
|
||||
" bytes");
|
||||
}
|
||||
|
||||
Json ComputerUse::navigate_params(const std::string& url) {
|
||||
Json p = Json::object();
|
||||
p.set("url", Json(url));
|
||||
return p;
|
||||
}
|
||||
|
||||
Json ComputerUse::mouse_params(const char* type, int x, int y) {
|
||||
Json p = Json::object();
|
||||
p.set("type", Json(type));
|
||||
p.set("x", Json(x));
|
||||
p.set("y", Json(y));
|
||||
p.set("button", Json("left"));
|
||||
p.set("clickCount", Json(1));
|
||||
return p;
|
||||
}
|
||||
|
||||
Json ComputerUse::insert_text_params(const std::string& text) {
|
||||
Json p = Json::object();
|
||||
p.set("text", Json(text));
|
||||
return p;
|
||||
}
|
||||
|
||||
ActionResult ComputerUse::send(const std::string& sandbox_name, const std::string& command,
|
||||
const Json& params) const {
|
||||
SandboxNames::parse_sandbox(sandbox_name);
|
||||
Json reply = execute_raw(sandbox_name, {ChunkCodec::cdp_input(command, params)});
|
||||
ActionResult out;
|
||||
out.result = Json::object();
|
||||
out.exec = ChunkCodec::decode(reply);
|
||||
for (const Chunk& c : ChunkCodec::outputs(reply)) {
|
||||
if (c.mime_type == "image/png" && out.screenshot_png.empty() &&
|
||||
c.data.size() <= SandboxCaps::kMaxScreenshotBytes)
|
||||
out.screenshot_png = c.data;
|
||||
if (c.mime_type == "application/json") {
|
||||
try {
|
||||
Json doc = Json::parse(c.data);
|
||||
if (!doc.is_object()) continue;
|
||||
const Json* r = doc.find("result");
|
||||
const Json& obj = r && r->is_object() ? *r : doc;
|
||||
out.result = obj;
|
||||
// CDP Page.captureScreenshot answers {"data": "<base64 png>"}.
|
||||
if (const Json* d = obj.find("data"); d && d->is_string() && out.screenshot_png.empty()) {
|
||||
std::string png;
|
||||
if (d->as_string().size() <= (SandboxCaps::kMaxScreenshotBytes / 3 + 1) * 4 &&
|
||||
security::crypto::Base64::decode(d->as_string(), png) &&
|
||||
png.size() <= SandboxCaps::kMaxScreenshotBytes)
|
||||
out.screenshot_png = std::move(png);
|
||||
}
|
||||
} catch (const clients::body::JsonError&) {
|
||||
}
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
ActionResult ComputerUse::navigate(const std::string& sandbox_name,
|
||||
const std::string& url) const {
|
||||
SandboxNames::parse_sandbox(sandbox_name);
|
||||
validate_url(url);
|
||||
return send(sandbox_name, "Page.navigate", navigate_params(url));
|
||||
}
|
||||
|
||||
ActionResult ComputerUse::click(const std::string& sandbox_name, int x, int y) const {
|
||||
SandboxNames::parse_sandbox(sandbox_name);
|
||||
validate_point(x, y);
|
||||
send(sandbox_name, "Input.dispatchMouseEvent", mouse_params("mousePressed", x, y));
|
||||
return send(sandbox_name, "Input.dispatchMouseEvent", mouse_params("mouseReleased", x, y));
|
||||
}
|
||||
|
||||
ActionResult ComputerUse::type_text(const std::string& sandbox_name,
|
||||
const std::string& text) const {
|
||||
SandboxNames::parse_sandbox(sandbox_name);
|
||||
validate_text(text);
|
||||
return send(sandbox_name, "Input.insertText", insert_text_params(text));
|
||||
}
|
||||
|
||||
ActionResult ComputerUse::screenshot(const std::string& sandbox_name) const {
|
||||
SandboxNames::parse_sandbox(sandbox_name);
|
||||
Json p = Json::object();
|
||||
p.set("format", Json("png"));
|
||||
ActionResult r = send(sandbox_name, "Page.captureScreenshot", p);
|
||||
if (r.screenshot_png.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "no screenshot in the reply", endpoint_.name);
|
||||
return r;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,47 @@
|
||||
#pragma once
|
||||
|
||||
// Computer use: browser actions in a computer use sandbox (navigate, click,
|
||||
// type, screenshot). Docs describe the action set and that each action is a
|
||||
// Chrome DevTools Protocol command ({"command","params"}); they send it to
|
||||
// the sandbox data plane with a signed JWT. This class carries the same CDP
|
||||
// commands as an application/json Chunk through sandboxEnvironments.execute,
|
||||
// the only sandbox call in the REST surface. Whether the server routes that
|
||||
// Chunk to the browser is unverified (SPEC Unverified).
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
struct ActionResult {
|
||||
clients::body::Json result; // The CDP result object ({} when none).
|
||||
ExecutionResult exec; // Text, files and skip reasons of the reply.
|
||||
std::string screenshot_png; // Raw PNG bytes, set by screenshot() only.
|
||||
};
|
||||
|
||||
class ComputerUse : public SandboxService {
|
||||
public:
|
||||
explicit ComputerUse(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
ActionResult navigate(const std::string& sandbox_name, const std::string& url) const;
|
||||
// Left click at viewport coordinates: mousePressed then mouseReleased.
|
||||
ActionResult click(const std::string& sandbox_name, int x, int y) const;
|
||||
ActionResult type_text(const std::string& sandbox_name, const std::string& text) const;
|
||||
ActionResult screenshot(const std::string& sandbox_name) const;
|
||||
|
||||
static void validate_url(const std::string& url); // http(s) only, capped
|
||||
static void validate_point(int x, int y);
|
||||
static void validate_text(const std::string& text);
|
||||
|
||||
// CDP command rows (compared to the CDP names in tests).
|
||||
static clients::body::Json navigate_params(const std::string& url);
|
||||
static clients::body::Json mouse_params(const char* type, int x, int y);
|
||||
static clients::body::Json insert_text_params(const std::string& text);
|
||||
|
||||
private:
|
||||
ActionResult send(const std::string& sandbox_name, const std::string& command,
|
||||
const clients::body::Json& params) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,160 @@
|
||||
#include "google/sandbox/names.h"
|
||||
|
||||
#include <vector>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/resource.h"
|
||||
#include "google/sandbox/caps.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
namespace {
|
||||
|
||||
std::vector<std::string> split(const std::string& s) {
|
||||
std::vector<std::string> out;
|
||||
std::string cur;
|
||||
for (char c : s) {
|
||||
if (c == '/') {
|
||||
out.push_back(cur);
|
||||
cur.clear();
|
||||
} else {
|
||||
cur.push_back(c);
|
||||
}
|
||||
}
|
||||
out.push_back(cur);
|
||||
return out;
|
||||
}
|
||||
|
||||
bool project_ok(const std::string& p) {
|
||||
if (p.empty() || p.size() > 128) return false;
|
||||
for (char c : p)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-' ||
|
||||
c == '.' || c == ':'))
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool location_ok(const std::string& l) {
|
||||
if (l.empty() || l.size() > 64) return false;
|
||||
for (char c : l)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-')) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool parse(const std::string& name, std::size_t segments,
|
||||
SandboxNames::Parts& out, const char* collection = "sandboxEnvironments") {
|
||||
if (name.empty() || name.size() > SandboxCaps::kMaxResourceNameChars) return false;
|
||||
std::vector<std::string> p = split(name);
|
||||
if (p.size() != segments) return false;
|
||||
if (p[0] != "projects" || p[2] != "locations" || p[4] != "reasoningEngines")
|
||||
return false;
|
||||
if (!project_ok(p[1]) || !location_ok(p[3]) || !valid_segment(p[5])) return false;
|
||||
if (segments == 8) {
|
||||
if (p[6] != collection || !valid_segment(p[7])) return false;
|
||||
out.sandbox = p[7];
|
||||
}
|
||||
out.project = p[1];
|
||||
out.location = p[3];
|
||||
out.engine = p[5];
|
||||
return true;
|
||||
}
|
||||
|
||||
// True when no "." or ".." or empty segment exists (separator '/').
|
||||
bool segments_plain(const std::string& s, bool allow_empty_first) {
|
||||
std::size_t start = 0;
|
||||
bool first = true;
|
||||
while (start <= s.size()) {
|
||||
std::size_t end = s.find('/', start);
|
||||
if (end == std::string::npos) end = s.size();
|
||||
std::string seg = s.substr(start, end - start);
|
||||
if (seg == ".." || seg == ".") return false;
|
||||
if (seg.empty() && !(first && allow_empty_first)) return false;
|
||||
first = false;
|
||||
start = end + 1;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool SandboxNames::valid_engine(const std::string& name) {
|
||||
Parts p;
|
||||
return parse(name, 6, p);
|
||||
}
|
||||
|
||||
bool SandboxNames::valid_sandbox(const std::string& name) {
|
||||
Parts p;
|
||||
return parse(name, 8, p);
|
||||
}
|
||||
|
||||
bool SandboxNames::valid_template(const std::string& name) {
|
||||
Parts p;
|
||||
return parse(name, 8, p, "sandboxEnvironmentTemplates");
|
||||
}
|
||||
|
||||
bool SandboxNames::valid_snapshot(const std::string& name) {
|
||||
Parts p;
|
||||
return parse(name, 8, p, "sandboxEnvironmentSnapshots");
|
||||
}
|
||||
|
||||
SandboxNames::Parts SandboxNames::parse_template(const std::string& name) {
|
||||
Parts p;
|
||||
if (!parse(name, 8, p, "sandboxEnvironmentTemplates"))
|
||||
throw GoogleError::invalid(
|
||||
"template must look like projects/{p}/locations/{l}/reasoningEngines/{e}"
|
||||
"/sandboxEnvironmentTemplates/{t}");
|
||||
return p;
|
||||
}
|
||||
|
||||
SandboxNames::Parts SandboxNames::parse_snapshot(const std::string& name) {
|
||||
Parts p;
|
||||
if (!parse(name, 8, p, "sandboxEnvironmentSnapshots"))
|
||||
throw GoogleError::invalid(
|
||||
"snapshot must look like projects/{p}/locations/{l}/reasoningEngines/{e}"
|
||||
"/sandboxEnvironmentSnapshots/{n}");
|
||||
return p;
|
||||
}
|
||||
|
||||
SandboxNames::Parts SandboxNames::parse_engine(const std::string& name) {
|
||||
Parts p;
|
||||
if (!parse(name, 6, p))
|
||||
throw GoogleError::invalid(
|
||||
"engine must look like projects/{p}/locations/{l}/reasoningEngines/{e}");
|
||||
return p;
|
||||
}
|
||||
|
||||
SandboxNames::Parts SandboxNames::parse_sandbox(const std::string& name) {
|
||||
Parts p;
|
||||
if (!parse(name, 8, p))
|
||||
throw GoogleError::invalid(
|
||||
"sandbox must look like projects/{p}/locations/{l}/reasoningEngines/{e}"
|
||||
"/sandboxEnvironments/{s}");
|
||||
return p;
|
||||
}
|
||||
|
||||
std::string SandboxNames::engine_of(const std::string& sandbox_name) {
|
||||
Parts p = parse_sandbox(sandbox_name);
|
||||
return "projects/" + p.project + "/locations/" + p.location +
|
||||
"/reasoningEngines/" + p.engine;
|
||||
}
|
||||
|
||||
bool valid_file_name(const std::string& name) {
|
||||
if (name.empty() || name.size() > SandboxCaps::kMaxFileNameChars) return false;
|
||||
if (name[0] == '/') return false;
|
||||
for (char c : name)
|
||||
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f || c == '\\') return false;
|
||||
return segments_plain(name, false);
|
||||
}
|
||||
|
||||
bool valid_workspace_path(const std::string& path) {
|
||||
if (path.empty() || path.size() > SandboxCaps::kMaxPathChars) return false;
|
||||
for (char c : path)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
|
||||
(c >= '0' && c <= '9') || c == '.' || c == '_' || c == '-' || c == '/'))
|
||||
return false;
|
||||
// A trailing slash (directory) is fine: "/workspace/".
|
||||
std::string trimmed = path.size() > 1 && path.back() == '/' ? path.substr(0, path.size() - 1) : path;
|
||||
return segments_plain(trimmed, true);
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,44 @@
|
||||
#pragma once
|
||||
|
||||
// Resource names for the sandbox module. Validated before any network
|
||||
// traffic. Formats (REST reference, v1beta1):
|
||||
// projects/{p}/locations/{l}/reasoningEngines/{e}
|
||||
// projects/{p}/locations/{l}/reasoningEngines/{e}/sandboxEnvironments/{s}
|
||||
// projects/{p}/locations/{l}/reasoningEngines/{e}/sandboxEnvironmentTemplates/{t}
|
||||
// projects/{p}/locations/{l}/reasoningEngines/{e}/sandboxEnvironmentSnapshots/{n}
|
||||
|
||||
#include <string>
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
struct SandboxNames {
|
||||
struct Parts {
|
||||
std::string project;
|
||||
std::string location;
|
||||
std::string engine;
|
||||
std::string sandbox; // Empty for an engine name.
|
||||
};
|
||||
|
||||
static bool valid_engine(const std::string& name);
|
||||
static bool valid_sandbox(const std::string& name);
|
||||
static bool valid_template(const std::string& name);
|
||||
static bool valid_snapshot(const std::string& name);
|
||||
|
||||
// Throw GoogleError::invalid on a malformed name.
|
||||
static Parts parse_engine(const std::string& name);
|
||||
static Parts parse_sandbox(const std::string& name);
|
||||
// For templates and snapshots Parts::sandbox holds the child id.
|
||||
static Parts parse_template(const std::string& name);
|
||||
static Parts parse_snapshot(const std::string& name);
|
||||
|
||||
// "projects/p/locations/l/reasoningEngines/e" from a sandbox name.
|
||||
static std::string engine_of(const std::string& sandbox_name);
|
||||
};
|
||||
|
||||
// A file name inside a request or response: no leading slash, no "." or ".."
|
||||
// segment, no backslash or control characters, 1..255 characters.
|
||||
bool valid_file_name(const std::string& name);
|
||||
// An absolute or /workspace-relative path of [A-Za-z0-9._/-], no "..".
|
||||
bool valid_workspace_path(const std::string& path);
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,39 @@
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
SandboxService::SandboxService(std::shared_ptr<GoogleClient> client)
|
||||
: Service(std::move(client), endpoints::aiplatform("us-central1")) {
|
||||
endpoint_.name = "aiplatform-sandbox";
|
||||
}
|
||||
|
||||
ServiceEndpoint SandboxService::endpoint_in(const std::string& location) {
|
||||
ServiceEndpoint e = endpoints::aiplatform(location);
|
||||
e.name = "aiplatform-sandbox";
|
||||
return e;
|
||||
}
|
||||
|
||||
ServiceEndpoint SandboxService::beta_endpoint_in(const std::string& location) {
|
||||
ServiceEndpoint e = endpoints::aiplatform_beta(location);
|
||||
e.name = "aiplatform-sandbox-beta";
|
||||
return e;
|
||||
}
|
||||
|
||||
Json SandboxService::execute_raw(const std::string& sandbox_name,
|
||||
const std::vector<Chunk>& inputs) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
Json body = ChunkCodec::request(inputs);
|
||||
return client_->request(clients::Method::kPost, endpoint_in(parts.location),
|
||||
"/" + sandbox_name + ":execute", &body);
|
||||
}
|
||||
|
||||
ExecutionResult SandboxService::execute(const std::string& sandbox_name,
|
||||
const std::vector<Chunk>& inputs) const {
|
||||
return ChunkCodec::decode(execute_raw(sandbox_name, inputs));
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,34 @@
|
||||
#pragma once
|
||||
|
||||
// Base for every sandbox service: the v1beta1 aiplatform endpoint of the
|
||||
// sandbox's own location, name validation, and the one execute call that
|
||||
// code, shell, workspace and computer use all share.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/service.h"
|
||||
#include "google/sandbox/chunk.h"
|
||||
#include "google/sandbox/names.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
class SandboxService : public Service {
|
||||
public:
|
||||
explicit SandboxService(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
protected:
|
||||
// https://{location}-aiplatform.googleapis.com/v1 (every call but bidiExecute).
|
||||
static ServiceEndpoint endpoint_in(const std::string& location);
|
||||
// .../v1beta1: bidiExecute exists only there (see calls.h).
|
||||
static ServiceEndpoint beta_endpoint_in(const std::string& location);
|
||||
|
||||
// POST {sandbox}:execute with the given inputs; returns the raw reply.
|
||||
clients::body::Json execute_raw(const std::string& sandbox_name,
|
||||
const std::vector<Chunk>& inputs) const;
|
||||
ExecutionResult execute(const std::string& sandbox_name,
|
||||
const std::vector<Chunk>& inputs) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,190 @@
|
||||
#include "google/sandbox/sandboxes.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sandbox/caps.h"
|
||||
#include "google/sandbox/snapshot_types.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
bool printable(const std::string& s) {
|
||||
for (char c : s)
|
||||
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Sandboxes::Sandboxes(std::shared_ptr<GoogleClient> client, Sleeper sleeper)
|
||||
: SandboxService(std::move(client)), sleeper_(std::move(sleeper)) {}
|
||||
|
||||
void Sandboxes::validate(const CreateSandboxRequest& r) {
|
||||
if (r.display_name.empty() ||
|
||||
r.display_name.size() > SandboxCaps::kMaxDisplayNameChars ||
|
||||
!printable(r.display_name))
|
||||
throw GoogleError::invalid("display_name must be 1.." +
|
||||
std::to_string(SandboxCaps::kMaxDisplayNameChars) +
|
||||
" printable characters");
|
||||
if (r.ttl_seconds < SandboxCaps::kMinTtlSeconds ||
|
||||
r.ttl_seconds > SandboxCaps::kMaxTtlSeconds)
|
||||
throw GoogleError::invalid("ttl must be " +
|
||||
std::to_string(SandboxCaps::kMinTtlSeconds) + ".." +
|
||||
std::to_string(SandboxCaps::kMaxTtlSeconds) + " seconds");
|
||||
if (r.spec.kind != SandboxKind::kCodeExecution &&
|
||||
(r.spec.language != CodeLanguage::kUnspecified ||
|
||||
r.spec.machine != MachineConfig::kUnspecified))
|
||||
throw GoogleError::invalid("language and machine apply to code sandboxes only");
|
||||
if (!r.template_name.empty()) SandboxNames::parse_template(r.template_name);
|
||||
if (!r.snapshot_name.empty()) SandboxNames::parse_snapshot(r.snapshot_name);
|
||||
if (r.owner.size() > 128 || !printable(r.owner))
|
||||
throw GoogleError::invalid("owner must be up to 128 printable characters");
|
||||
if (r.spec.kind == SandboxKind::kComputerUse && r.template_name.empty())
|
||||
throw GoogleError::invalid("a computer use sandbox needs a template");
|
||||
}
|
||||
|
||||
void Sandboxes::validate(const SnapshotRequest& r) {
|
||||
if (r.display_name.empty() ||
|
||||
r.display_name.size() > SandboxCaps::kMaxDisplayNameChars ||
|
||||
!printable(r.display_name))
|
||||
throw GoogleError::invalid("display_name must be 1.." +
|
||||
std::to_string(SandboxCaps::kMaxDisplayNameChars) +
|
||||
" printable characters");
|
||||
if (r.ttl_seconds != 0 && (r.ttl_seconds < SandboxCaps::kMinTtlSeconds ||
|
||||
r.ttl_seconds > SandboxCaps::kMaxTtlSeconds))
|
||||
throw GoogleError::invalid("ttl must be 0 or " +
|
||||
std::to_string(SandboxCaps::kMinTtlSeconds) + ".." +
|
||||
std::to_string(SandboxCaps::kMaxTtlSeconds) + " seconds");
|
||||
if (!r.post_action.empty() && r.post_action != "RUNNING" && r.post_action != "PAUSE")
|
||||
throw GoogleError::invalid("post_action must be RUNNING or PAUSE");
|
||||
if (r.owner.size() > 128 || !printable(r.owner))
|
||||
throw GoogleError::invalid("owner must be up to 128 printable characters");
|
||||
}
|
||||
|
||||
Json Sandboxes::snapshot_body(const std::string& sandbox_name,
|
||||
const SnapshotRequest& r) {
|
||||
Json body = Json::object();
|
||||
body.set("displayName", Json(r.display_name));
|
||||
body.set("sourceSandboxEnvironment", Json(sandbox_name));
|
||||
if (r.ttl_seconds != 0) body.set("ttl", Json(std::to_string(r.ttl_seconds) + "s"));
|
||||
if (!r.post_action.empty()) body.set("postSnapshotAction", Json(r.post_action));
|
||||
if (!r.owner.empty()) body.set("owner", Json(r.owner));
|
||||
return body;
|
||||
}
|
||||
|
||||
SandboxSnapshot Sandboxes::snapshot(const std::string& sandbox_name,
|
||||
const SnapshotRequest& request) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
validate(request);
|
||||
ServiceEndpoint ep = endpoint_in(parts.location);
|
||||
Json body = snapshot_body(sandbox_name, request);
|
||||
Json reply = client_->request(clients::Method::kPost, ep,
|
||||
"/" + sandbox_name + ":snapshot", &body);
|
||||
Operation op = Operation::from_json(reply);
|
||||
if (!op.done) {
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
|
||||
op = wait_operation(*client_, ep, op.name, sleeper_);
|
||||
} else if (op.failed()) {
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
|
||||
}
|
||||
return SandboxSnapshot::from_json(op.response);
|
||||
}
|
||||
|
||||
void Sandboxes::authorize_access(const std::string& sandbox_name) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
Json body = Json::object();
|
||||
client_->request(clients::Method::kPost, endpoint_in(parts.location),
|
||||
"/" + sandbox_name + ":authorizeAccess", &body);
|
||||
}
|
||||
|
||||
SandboxEnvironment Sandboxes::finish(const ServiceEndpoint& endpoint,
|
||||
const Json& reply) const {
|
||||
Operation op = Operation::from_json(reply);
|
||||
if (!op.done) {
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name",
|
||||
endpoint.name);
|
||||
op = wait_operation(*client_, endpoint, op.name, sleeper_);
|
||||
} else if (op.failed()) {
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, endpoint.name);
|
||||
}
|
||||
return SandboxEnvironment::from_json(op.response);
|
||||
}
|
||||
|
||||
Json Sandboxes::create_body(const CreateSandboxRequest& request) {
|
||||
Json body = Json::object();
|
||||
body.set("displayName", Json(request.display_name));
|
||||
body.set("spec", request.spec.to_json());
|
||||
body.set("ttl", Json(std::to_string(request.ttl_seconds) + "s"));
|
||||
if (!request.template_name.empty())
|
||||
body.set("sandboxEnvironmentTemplate", Json(request.template_name));
|
||||
if (!request.snapshot_name.empty())
|
||||
body.set("sandboxEnvironmentSnapshot", Json(request.snapshot_name));
|
||||
if (!request.owner.empty()) body.set("owner", Json(request.owner));
|
||||
return body;
|
||||
}
|
||||
|
||||
SandboxEnvironment Sandboxes::create(const std::string& engine_name,
|
||||
const CreateSandboxRequest& request) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_engine(engine_name);
|
||||
validate(request);
|
||||
Json body = create_body(request);
|
||||
// computerUseEnvironment exists only in v1beta1 (see calls.h).
|
||||
ServiceEndpoint ep = request.spec.kind == SandboxKind::kComputerUse
|
||||
? beta_endpoint_in(parts.location)
|
||||
: endpoint_in(parts.location);
|
||||
Json reply = client_->request(clients::Method::kPost, ep,
|
||||
"/" + engine_name + "/sandboxEnvironments", &body);
|
||||
return finish(ep, reply);
|
||||
}
|
||||
|
||||
SandboxEnvironment Sandboxes::get(const std::string& sandbox_name) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
return SandboxEnvironment::from_json(
|
||||
client_->get_json(endpoint_in(parts.location), "/" + sandbox_name));
|
||||
}
|
||||
|
||||
std::vector<SandboxEnvironment> Sandboxes::list(const std::string& engine_name,
|
||||
int page_size) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_engine(engine_name);
|
||||
if (page_size == 0) page_size = SandboxCaps::kDefaultPageSize;
|
||||
if (page_size < 1 || page_size > SandboxCaps::kMaxPageSize)
|
||||
throw GoogleError::invalid("page_size must be 1.." +
|
||||
std::to_string(SandboxCaps::kMaxPageSize));
|
||||
Json items = list_all(*client_, endpoint_in(parts.location),
|
||||
"/" + engine_name + "/sandboxEnvironments?pageSize=" +
|
||||
std::to_string(page_size),
|
||||
"sandboxEnvironments", SandboxCaps::kMaxListPages);
|
||||
std::vector<SandboxEnvironment> out;
|
||||
for (const Json& item : items.as_array())
|
||||
out.push_back(SandboxEnvironment::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
SandboxEnvironment Sandboxes::pause(const std::string& sandbox_name) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
ServiceEndpoint ep = endpoint_in(parts.location);
|
||||
Json body = Json::object();
|
||||
return finish(ep, client_->request(clients::Method::kPost, ep,
|
||||
"/" + sandbox_name + ":pause", &body));
|
||||
}
|
||||
|
||||
SandboxEnvironment Sandboxes::resume(const std::string& sandbox_name) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
ServiceEndpoint ep = endpoint_in(parts.location);
|
||||
Json body = Json::object();
|
||||
return finish(ep, client_->request(clients::Method::kPost, ep,
|
||||
"/" + sandbox_name + ":resume", &body));
|
||||
}
|
||||
|
||||
void Sandboxes::remove(const std::string& sandbox_name) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
ServiceEndpoint ep = endpoint_in(parts.location);
|
||||
Json reply = client_->delete_json(ep, "/" + sandbox_name);
|
||||
finish(ep, reply);
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,58 @@
|
||||
#pragma once
|
||||
|
||||
// SandboxEnvironment lifecycle: create, get, list, pause, resume, delete.
|
||||
// REST: projects.locations.reasoningEngines.sandboxEnvironments (v1).
|
||||
// Create, pause, resume and delete return long-running operations that this
|
||||
// class waits on (hard-capped by wait_operation).
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
#include "google/sandbox/snapshot_types.h"
|
||||
#include "google/sandbox/types.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
class Sandboxes : public SandboxService {
|
||||
public:
|
||||
explicit Sandboxes(std::shared_ptr<GoogleClient> client,
|
||||
Sleeper sleeper = nullptr);
|
||||
|
||||
// POST {engine}/sandboxEnvironments
|
||||
SandboxEnvironment create(const std::string& engine_name,
|
||||
const CreateSandboxRequest& request) const;
|
||||
// GET {name}
|
||||
SandboxEnvironment get(const std::string& sandbox_name) const;
|
||||
// GET {engine}/sandboxEnvironments, following pages up to the page cap.
|
||||
std::vector<SandboxEnvironment> list(const std::string& engine_name,
|
||||
int page_size = 0) const;
|
||||
// POST {name}:pause and :resume
|
||||
SandboxEnvironment pause(const std::string& sandbox_name) const;
|
||||
SandboxEnvironment resume(const std::string& sandbox_name) const;
|
||||
// POST {name}:snapshot. Returns the new SandboxEnvironmentSnapshot.
|
||||
SandboxSnapshot snapshot(const std::string& sandbox_name,
|
||||
const SnapshotRequest& request) const;
|
||||
// POST {name}:authorizeAccess: succeeds (no value) when the caller holds
|
||||
// sandboxEnvironments.execute on the sandbox, throws GoogleError otherwise.
|
||||
void authorize_access(const std::string& sandbox_name) const;
|
||||
// DELETE {name}. Explicit: sandboxes are billed while they exist, but a
|
||||
// delete is never implied by any other call.
|
||||
void remove(const std::string& sandbox_name) const;
|
||||
|
||||
static void validate(const CreateSandboxRequest& request);
|
||||
static void validate(const SnapshotRequest& request);
|
||||
// The exact request bodies (compared to the Discovery schema in tests).
|
||||
static clients::body::Json create_body(const CreateSandboxRequest& request);
|
||||
static clients::body::Json snapshot_body(const std::string& sandbox_name,
|
||||
const SnapshotRequest& request);
|
||||
|
||||
private:
|
||||
SandboxEnvironment finish(const ServiceEndpoint& endpoint,
|
||||
const clients::body::Json& reply) const;
|
||||
|
||||
Sleeper sleeper_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,44 @@
|
||||
#include "google/sandbox/shell_runner.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
ShellRunner::ShellRunner(std::shared_ptr<GoogleClient> client)
|
||||
: SandboxService(std::move(client)) {}
|
||||
|
||||
void ShellRunner::validate(const ShellRequest& r, std::size_t max_command) {
|
||||
if (r.command.empty()) throw GoogleError::invalid("command is empty");
|
||||
if (r.command.size() > max_command)
|
||||
throw GoogleError::invalid("command over " + std::to_string(max_command) +
|
||||
" bytes");
|
||||
if (r.command.find('\0') != std::string::npos)
|
||||
throw GoogleError::invalid("command contains a NUL byte");
|
||||
if (!r.cwd.empty() && !valid_workspace_path(r.cwd))
|
||||
throw GoogleError::invalid("cwd must be a plain path of [A-Za-z0-9._/-]");
|
||||
if (r.timeout_seconds < 1 || r.timeout_seconds > SandboxCaps::kMaxShellTimeoutSeconds)
|
||||
throw GoogleError::invalid("timeout must be 1.." +
|
||||
std::to_string(SandboxCaps::kMaxShellTimeoutSeconds) +
|
||||
" seconds");
|
||||
}
|
||||
|
||||
ExecutionResult ShellRunner::run_capped(const std::string& sandbox_name,
|
||||
const ShellRequest& request,
|
||||
std::size_t max_command) const {
|
||||
SandboxNames::parse_sandbox(sandbox_name);
|
||||
validate(request, max_command);
|
||||
return execute(sandbox_name, {ChunkCodec::shell_input(
|
||||
request.command, request.cwd, request.timeout_seconds)});
|
||||
}
|
||||
|
||||
ExecutionResult ShellRunner::run(const std::string& sandbox_name,
|
||||
const ShellRequest& request) const {
|
||||
return run_capped(sandbox_name, request, SandboxCaps::kMaxCommandBytes);
|
||||
}
|
||||
|
||||
ExecutionResult ShellRunner::run_internal(const std::string& sandbox_name,
|
||||
const ShellRequest& request) const {
|
||||
return run_capped(sandbox_name, request, SandboxCaps::kMaxInternalCommandBytes);
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,40 @@
|
||||
#pragma once
|
||||
|
||||
// Bash commands in a shell sandbox ("Workspaces" in the product overview:
|
||||
// commands and files isolated from core systems). Each command runs in a new
|
||||
// shell as an unprivileged user; cd and variables do not carry over.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "google/sandbox/caps.h"
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
struct ShellRequest {
|
||||
std::string command;
|
||||
std::string cwd; // Empty: the sandbox default (/workspace).
|
||||
int timeout_seconds = SandboxCaps::kDefaultShellTimeoutSeconds;
|
||||
};
|
||||
|
||||
class ShellRunner : public SandboxService {
|
||||
public:
|
||||
explicit ShellRunner(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
ExecutionResult run(const std::string& sandbox_name,
|
||||
const ShellRequest& request) const;
|
||||
|
||||
// Same call with a higher command ceiling, for WorkspaceFiles only (it
|
||||
// ships file bytes inside the command line). Never exposed to routes.
|
||||
ExecutionResult run_internal(const std::string& sandbox_name,
|
||||
const ShellRequest& request) const;
|
||||
|
||||
static void validate(const ShellRequest& request, std::size_t max_command);
|
||||
|
||||
private:
|
||||
ExecutionResult run_capped(const std::string& sandbox_name,
|
||||
const ShellRequest& request,
|
||||
std::size_t max_command) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,35 @@
|
||||
#pragma once
|
||||
|
||||
// SandboxEnvironmentSnapshot (v1): a saved sandbox state a new sandbox can be
|
||||
// created from. Also the request that takes one.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
struct SnapshotRequest {
|
||||
std::string display_name; // Required, 1..128 characters.
|
||||
long ttl_seconds = 0; // 0: omit (server default); else 60 s .. 14 days.
|
||||
std::string post_action; // "", "RUNNING" or "PAUSE" (postSnapshotAction).
|
||||
std::string owner; // Optional, printable, up to 128 characters.
|
||||
};
|
||||
|
||||
struct SandboxSnapshot {
|
||||
std::string name;
|
||||
std::string display_name;
|
||||
std::string source_sandbox;
|
||||
std::string parent_snapshot;
|
||||
std::string owner;
|
||||
std::string post_action;
|
||||
std::string create_time;
|
||||
std::string update_time;
|
||||
std::string expire_time;
|
||||
long size_bytes = -1; // sizeBytes is an int64 string; -1 unknown.
|
||||
|
||||
static SandboxSnapshot from_json(const clients::body::Json& doc);
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,88 @@
|
||||
#include "google/sandbox/snapshots.h"
|
||||
|
||||
#include <cstdlib>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sandbox/caps.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
std::string str(const Json& doc, const char* key) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
} // namespace
|
||||
|
||||
SandboxSnapshot SandboxSnapshot::from_json(const Json& doc) {
|
||||
SandboxSnapshot s;
|
||||
s.name = str(doc, "name");
|
||||
s.display_name = str(doc, "displayName");
|
||||
s.source_sandbox = str(doc, "sourceSandboxEnvironment");
|
||||
s.parent_snapshot = str(doc, "parentSnapshot");
|
||||
s.owner = str(doc, "owner");
|
||||
s.post_action = str(doc, "postSnapshotAction");
|
||||
s.create_time = str(doc, "createTime");
|
||||
s.update_time = str(doc, "updateTime");
|
||||
s.expire_time = str(doc, "expireTime");
|
||||
std::string size = str(doc, "sizeBytes");
|
||||
if (!size.empty() && size.find_first_not_of("0123456789") == std::string::npos &&
|
||||
size.size() < 18)
|
||||
s.size_bytes = std::strtol(size.c_str(), nullptr, 10);
|
||||
return s;
|
||||
}
|
||||
|
||||
Json SandboxSnapshot::to_json() const {
|
||||
Json o = Json::object();
|
||||
o.set("name", Json(name));
|
||||
o.set("display_name", Json(display_name));
|
||||
o.set("source_sandbox", Json(source_sandbox));
|
||||
o.set("parent_snapshot", Json(parent_snapshot));
|
||||
o.set("post_action", Json(post_action));
|
||||
o.set("create_time", Json(create_time));
|
||||
o.set("expire_time", Json(expire_time));
|
||||
o.set("size_bytes", Json(size_bytes));
|
||||
return o;
|
||||
}
|
||||
|
||||
Snapshots::Snapshots(std::shared_ptr<GoogleClient> client, Sleeper sleeper)
|
||||
: SandboxService(std::move(client)), sleeper_(std::move(sleeper)) {}
|
||||
|
||||
SandboxSnapshot Snapshots::get(const std::string& snapshot_name) const {
|
||||
SandboxNames::Parts p = SandboxNames::parse_snapshot(snapshot_name);
|
||||
return SandboxSnapshot::from_json(
|
||||
client_->get_json(endpoint_in(p.location), "/" + snapshot_name));
|
||||
}
|
||||
|
||||
std::vector<SandboxSnapshot> Snapshots::list(const std::string& engine_name,
|
||||
int page_size) const {
|
||||
SandboxNames::Parts p = SandboxNames::parse_engine(engine_name);
|
||||
if (page_size == 0) page_size = SandboxCaps::kDefaultPageSize;
|
||||
if (page_size < 1 || page_size > SandboxCaps::kMaxPageSize)
|
||||
throw GoogleError::invalid("page_size must be 1.." +
|
||||
std::to_string(SandboxCaps::kMaxPageSize));
|
||||
Json items = list_all(*client_, endpoint_in(p.location),
|
||||
"/" + engine_name + "/sandboxEnvironmentSnapshots?pageSize=" +
|
||||
std::to_string(page_size),
|
||||
"sandboxEnvironmentSnapshots", SandboxCaps::kMaxListPages);
|
||||
std::vector<SandboxSnapshot> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(SandboxSnapshot::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
void Snapshots::remove(const std::string& snapshot_name) const {
|
||||
SandboxNames::Parts p = SandboxNames::parse_snapshot(snapshot_name);
|
||||
ServiceEndpoint ep = endpoint_in(p.location);
|
||||
Operation op = Operation::from_json(client_->delete_json(ep, "/" + snapshot_name));
|
||||
if (!op.done) {
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
|
||||
wait_operation(*client_, ep, op.name, sleeper_);
|
||||
} else if (op.failed()) {
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,29 @@
|
||||
#pragma once
|
||||
|
||||
// SandboxEnvironmentSnapshot reads and delete (v1). Taking a snapshot is
|
||||
// Sandboxes::snapshot (it hangs off a sandbox).
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
#include "google/sandbox/snapshot_types.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
class Snapshots : public SandboxService {
|
||||
public:
|
||||
explicit Snapshots(std::shared_ptr<GoogleClient> client, Sleeper sleeper = nullptr);
|
||||
|
||||
SandboxSnapshot get(const std::string& snapshot_name) const;
|
||||
std::vector<SandboxSnapshot> list(const std::string& engine_name,
|
||||
int page_size = 0) const;
|
||||
// Explicit delete (long-running; waited on, capped).
|
||||
void remove(const std::string& snapshot_name) const;
|
||||
|
||||
private:
|
||||
Sleeper sleeper_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,37 @@
|
||||
#include "google/sandbox/stream_executor.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
StreamExecutor::StreamExecutor(std::shared_ptr<GoogleClient> client)
|
||||
: SandboxService(std::move(client)) {}
|
||||
|
||||
ExecutionResult StreamExecutor::run(
|
||||
const std::string& sandbox_name, const std::vector<Chunk>& inputs,
|
||||
const std::function<bool(const std::vector<Chunk>&)>& on_event) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_sandbox(sandbox_name);
|
||||
if (inputs.empty()) throw GoogleError::invalid("no inputs");
|
||||
if (inputs.size() > 16) throw GoogleError::invalid("more than 16 input chunks");
|
||||
Json body = ChunkCodec::request(inputs);
|
||||
Json merged = Json::array();
|
||||
std::size_t events = 0;
|
||||
client_->stream_json(
|
||||
beta_endpoint_in(parts.location), "/" + sandbox_name + ":bidiExecute", body,
|
||||
[&](const Json& event) {
|
||||
if (++events > kMaxEvents) return false;
|
||||
std::vector<Chunk> chunks = ChunkCodec::outputs(event);
|
||||
for (const Json& item : (event.find("outputs") && event.find("outputs")->is_array()
|
||||
? event.find("outputs")->as_array()
|
||||
: Json::Array{}))
|
||||
merged.push(item);
|
||||
return on_event ? on_event(chunks) : true;
|
||||
});
|
||||
Json reply = Json::object();
|
||||
reply.set("outputs", merged);
|
||||
return ChunkCodec::decode(reply);
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,29 @@
|
||||
#pragma once
|
||||
|
||||
// sandboxEnvironments.bidiExecute (v1beta1 only): execute with a streamed
|
||||
// reply. The REST surface is one POST whose reply is a stream of
|
||||
// BidiExecuteSandboxEnvironmentResponse events; this class sends the inputs
|
||||
// once and folds every event's outputs into one typed ExecutionResult.
|
||||
// True bidirectional input is not expressible over this client (gap).
|
||||
|
||||
#include <functional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
class StreamExecutor : public SandboxService {
|
||||
public:
|
||||
static constexpr std::size_t kMaxEvents = 256;
|
||||
|
||||
explicit StreamExecutor(std::shared_ptr<GoogleClient> client);
|
||||
|
||||
// `on_event` (optional) sees each event's decoded output chunks as they
|
||||
// arrive; return false to stop early.
|
||||
ExecutionResult run(const std::string& sandbox_name, const std::vector<Chunk>& inputs,
|
||||
const std::function<bool(const std::vector<Chunk>&)>& on_event = nullptr) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,210 @@
|
||||
#include "google/sandbox/templates.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sandbox/caps.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
std::string str(const Json& doc, const char* key) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
constexpr std::size_t kMaxPorts = 16;
|
||||
constexpr long kMaxDiskGb = 1024;
|
||||
|
||||
bool printable(const std::string& s) {
|
||||
for (char c : s)
|
||||
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
// Container image references: [A-Za-z0-9._/:@-], 1..512.
|
||||
bool valid_image(const std::string& s) {
|
||||
if (s.empty() || s.size() > 512) return false;
|
||||
for (char c : s)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '.' || c == '_' || c == '/' || c == ':' || c == '@' || c == '-'))
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
SandboxTemplate SandboxTemplate::from_json(const Json& doc) {
|
||||
SandboxTemplate t;
|
||||
if (!doc.is_object()) return t;
|
||||
t.raw = doc;
|
||||
t.name = str(doc, "name");
|
||||
t.display_name = str(doc, "displayName");
|
||||
t.state = str(doc, "state");
|
||||
t.create_time = str(doc, "createTime");
|
||||
t.update_time = str(doc, "updateTime");
|
||||
if (const Json* d = doc.find("defaultContainerEnvironment"); d && d->is_object()) {
|
||||
std::string c = str(*d, "defaultContainerCategory");
|
||||
if (c == "DEFAULT_CONTAINER_CATEGORY_COMPUTER_USE") t.category = ContainerCategory::kComputerUse;
|
||||
else if (c == "DEFAULT_CONTAINER_CATEGORY_SHELL_SANDBOX") t.category = ContainerCategory::kShellSandbox;
|
||||
}
|
||||
if (const Json* c = doc.find("customContainerEnvironment"); c && c->is_object())
|
||||
if (const Json* s = c->find("customContainerSpec"))
|
||||
t.image_uri = str(*s, "imageUri");
|
||||
if (const Json* e = doc.find("egressControlConfig"); e && e->is_object())
|
||||
if (const Json* i = e->find("internetAccess"); i && i->is_bool())
|
||||
t.internet_access = i->as_bool();
|
||||
return t;
|
||||
}
|
||||
|
||||
Json SandboxTemplate::to_json() const {
|
||||
Json o = Json::object();
|
||||
o.set("name", Json(name));
|
||||
o.set("display_name", Json(display_name));
|
||||
o.set("state", Json(state));
|
||||
o.set("category", Json(category == ContainerCategory::kComputerUse ? "computer"
|
||||
: category == ContainerCategory::kShellSandbox ? "shell"
|
||||
: ""));
|
||||
o.set("image_uri", Json(image_uri));
|
||||
o.set("internet_access", Json(internet_access));
|
||||
o.set("create_time", Json(create_time));
|
||||
return o;
|
||||
}
|
||||
|
||||
Templates::Templates(std::shared_ptr<GoogleClient> client, Sleeper sleeper)
|
||||
: SandboxService(std::move(client)), sleeper_(std::move(sleeper)) {}
|
||||
|
||||
void Templates::validate(const CreateTemplateRequest& r) {
|
||||
if (r.display_name.empty() ||
|
||||
r.display_name.size() > SandboxCaps::kMaxDisplayNameChars ||
|
||||
!printable(r.display_name))
|
||||
throw GoogleError::invalid("display_name must be 1.." +
|
||||
std::to_string(SandboxCaps::kMaxDisplayNameChars) +
|
||||
" printable characters");
|
||||
bool has_category = r.category != ContainerCategory::kNone;
|
||||
bool has_image = !r.image_uri.empty();
|
||||
if (has_category == has_image)
|
||||
throw GoogleError::invalid("give exactly one of a container category or an image");
|
||||
if (has_image && !valid_image(r.image_uri))
|
||||
throw GoogleError::invalid("image_uri has invalid characters");
|
||||
if (!has_image && !r.ports.empty())
|
||||
throw GoogleError::invalid("ports apply to custom containers only");
|
||||
if (r.ports.size() > kMaxPorts)
|
||||
throw GoogleError::invalid("more than 16 ports");
|
||||
for (const TemplatePort& p : r.ports) {
|
||||
if (p.protocol != "TCP" && p.protocol != "UDP")
|
||||
throw GoogleError::invalid("port protocol must be TCP or UDP");
|
||||
if (p.port < 1 || p.port > 65535) throw GoogleError::invalid("port must be 1..65535");
|
||||
}
|
||||
if (r.network_attachment.size() > SandboxCaps::kMaxResourceNameChars ||
|
||||
!printable(r.network_attachment) ||
|
||||
r.network_attachment.find("..") != std::string::npos)
|
||||
throw GoogleError::invalid("network_attachment is malformed");
|
||||
if (r.persistent_disk) {
|
||||
if (!valid_workspace_path(r.disk_mount_path) || r.disk_mount_path[0] != '/')
|
||||
throw GoogleError::invalid("disk_mount_path must be an absolute plain path");
|
||||
if (r.disk_size_gb < 1 || r.disk_size_gb > kMaxDiskGb)
|
||||
throw GoogleError::invalid("disk_size_gb must be 1..1024");
|
||||
} else if (!r.disk_mount_path.empty() || r.disk_size_gb != 0) {
|
||||
throw GoogleError::invalid("disk settings need persistent_disk");
|
||||
}
|
||||
}
|
||||
|
||||
Json Templates::create_body(const CreateTemplateRequest& r) {
|
||||
Json body = Json::object();
|
||||
body.set("displayName", Json(r.display_name));
|
||||
if (r.category != ContainerCategory::kNone) {
|
||||
Json d = Json::object();
|
||||
d.set("defaultContainerCategory",
|
||||
Json(r.category == ContainerCategory::kComputerUse
|
||||
? "DEFAULT_CONTAINER_CATEGORY_COMPUTER_USE"
|
||||
: "DEFAULT_CONTAINER_CATEGORY_SHELL_SANDBOX"));
|
||||
body.set("defaultContainerEnvironment", d);
|
||||
} else {
|
||||
Json spec = Json::object();
|
||||
spec.set("imageUri", Json(r.image_uri));
|
||||
Json c = Json::object();
|
||||
c.set("customContainerSpec", spec);
|
||||
if (!r.ports.empty()) {
|
||||
Json ports = Json::array();
|
||||
for (const TemplatePort& p : r.ports) {
|
||||
Json one = Json::object();
|
||||
one.set("protocol", Json(p.protocol));
|
||||
one.set("port", Json(p.port));
|
||||
ports.push(one);
|
||||
}
|
||||
c.set("ports", ports);
|
||||
}
|
||||
body.set("customContainerEnvironment", c);
|
||||
}
|
||||
Json egress = Json::object();
|
||||
egress.set("internetAccess", Json(r.internet_access));
|
||||
if (!r.network_attachment.empty())
|
||||
egress.set("networkAttachment", Json(r.network_attachment));
|
||||
body.set("egressControlConfig", egress);
|
||||
if (r.persistent_disk) {
|
||||
Json disk = Json::object();
|
||||
disk.set("enabled", Json(true));
|
||||
disk.set("mountPath", Json(r.disk_mount_path));
|
||||
disk.set("sizeGb", Json(std::to_string(r.disk_size_gb))); // int64 is a string.
|
||||
body.set("persistentDiskConfig", disk);
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
SandboxTemplate Templates::create(const std::string& engine_name,
|
||||
const CreateTemplateRequest& request) const {
|
||||
SandboxNames::Parts parts = SandboxNames::parse_engine(engine_name);
|
||||
validate(request);
|
||||
ServiceEndpoint ep = beta_endpoint_in(parts.location);
|
||||
Json body = create_body(request);
|
||||
Operation op = Operation::from_json(client_->request(
|
||||
clients::Method::kPost, ep, "/" + engine_name + "/sandboxEnvironmentTemplates",
|
||||
&body));
|
||||
if (!op.done) {
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
|
||||
op = wait_operation(*client_, ep, op.name, sleeper_);
|
||||
} else if (op.failed()) {
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
|
||||
}
|
||||
return SandboxTemplate::from_json(op.response);
|
||||
}
|
||||
|
||||
SandboxTemplate Templates::get(const std::string& template_name) const {
|
||||
SandboxNames::Parts p = SandboxNames::parse_template(template_name);
|
||||
return SandboxTemplate::from_json(
|
||||
client_->get_json(beta_endpoint_in(p.location), "/" + template_name));
|
||||
}
|
||||
|
||||
std::vector<SandboxTemplate> Templates::list(const std::string& engine_name,
|
||||
int page_size) const {
|
||||
SandboxNames::Parts p = SandboxNames::parse_engine(engine_name);
|
||||
if (page_size == 0) page_size = SandboxCaps::kDefaultPageSize;
|
||||
if (page_size < 1 || page_size > SandboxCaps::kMaxPageSize)
|
||||
throw GoogleError::invalid("page_size must be 1.." +
|
||||
std::to_string(SandboxCaps::kMaxPageSize));
|
||||
Json items = list_all(*client_, beta_endpoint_in(p.location),
|
||||
"/" + engine_name + "/sandboxEnvironmentTemplates?pageSize=" +
|
||||
std::to_string(page_size),
|
||||
"sandboxEnvironmentTemplates", SandboxCaps::kMaxListPages);
|
||||
std::vector<SandboxTemplate> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(SandboxTemplate::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
void Templates::remove(const std::string& template_name) const {
|
||||
SandboxNames::Parts p = SandboxNames::parse_template(template_name);
|
||||
ServiceEndpoint ep = beta_endpoint_in(p.location);
|
||||
Operation op = Operation::from_json(client_->delete_json(ep, "/" + template_name));
|
||||
if (!op.done) {
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "operation without a name", ep.name);
|
||||
wait_operation(*client_, ep, op.name, sleeper_);
|
||||
} else if (op.failed()) {
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, ep.name);
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,68 @@
|
||||
#pragma once
|
||||
|
||||
// SandboxEnvironmentTemplate (v1beta1): the container recipe a computer use
|
||||
// or shell sandbox is created from. Create, get, list, delete.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/sandbox/sandbox_service.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
enum class ContainerCategory { kNone, kComputerUse, kShellSandbox };
|
||||
|
||||
struct TemplatePort {
|
||||
std::string protocol = "TCP"; // TCP or UDP
|
||||
int port = 0; // 1..65535
|
||||
};
|
||||
|
||||
struct CreateTemplateRequest {
|
||||
std::string display_name; // Required, 1..128 characters.
|
||||
// Exactly one of: a default container category, or a custom container image.
|
||||
ContainerCategory category = ContainerCategory::kNone;
|
||||
std::string image_uri; // customContainerEnvironment.customContainerSpec.imageUri
|
||||
std::vector<TemplatePort> ports; // Custom container only, at most 16.
|
||||
bool internet_access = false; // egressControlConfig.internetAccess
|
||||
std::string network_attachment; // egressControlConfig.networkAttachment
|
||||
bool persistent_disk = false; // persistentDiskConfig.enabled
|
||||
std::string disk_mount_path; // absolute path, plain characters
|
||||
long disk_size_gb = 0; // 1..1024 when the disk is enabled
|
||||
};
|
||||
|
||||
struct SandboxTemplate {
|
||||
std::string name;
|
||||
std::string display_name;
|
||||
std::string state; // ACTIVE, PROVISIONING, FAILED, ...
|
||||
std::string create_time;
|
||||
std::string update_time;
|
||||
ContainerCategory category = ContainerCategory::kNone;
|
||||
std::string image_uri;
|
||||
bool internet_access = false;
|
||||
clients::body::Json raw; // Whole document, unknown fields kept.
|
||||
|
||||
static SandboxTemplate from_json(const clients::body::Json& doc);
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
class Templates : public SandboxService {
|
||||
public:
|
||||
explicit Templates(std::shared_ptr<GoogleClient> client, Sleeper sleeper = nullptr);
|
||||
|
||||
SandboxTemplate create(const std::string& engine_name,
|
||||
const CreateTemplateRequest& request) const;
|
||||
SandboxTemplate get(const std::string& template_name) const;
|
||||
std::vector<SandboxTemplate> list(const std::string& engine_name,
|
||||
int page_size = 0) const;
|
||||
// Explicit delete (long-running; waited on, capped).
|
||||
void remove(const std::string& template_name) const;
|
||||
|
||||
static void validate(const CreateTemplateRequest& request);
|
||||
static clients::body::Json create_body(const CreateTemplateRequest& request);
|
||||
|
||||
private:
|
||||
Sleeper sleeper_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,163 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fails when a call, request field, schema field or enum value declared in
|
||||
src/google/sandbox/calls.h is absent from Google's Discovery documents.
|
||||
|
||||
Usage: python3 -I check_discovery.py [--cache DIR]
|
||||
Stdlib only, no credentials (Discovery documents are public). Exit 0: every
|
||||
declared row exists. Exit 1: at least one row is missing (listed). Exit 2: a
|
||||
document could not be fetched (loud, never a silent pass).
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
CALLS_H = os.path.join(HERE, "..", "calls.h")
|
||||
ROW = re.compile(r'^\s*\{((?:"(?:[^"\\]|\\.)*"\s*,?\s*)+)\},?\s*$')
|
||||
STR = re.compile(r'"((?:[^"\\]|\\.)*)"')
|
||||
|
||||
|
||||
def rows(text, begin, end):
|
||||
block = text.split(begin, 1)[1].split(end, 1)[0]
|
||||
out = []
|
||||
for line in block.splitlines():
|
||||
m = ROW.match(line)
|
||||
if m:
|
||||
out.append([s for s in STR.findall(m.group(1))])
|
||||
return out
|
||||
|
||||
|
||||
_docs = {}
|
||||
|
||||
|
||||
def discovery(service, version, cache):
|
||||
key = (service, version)
|
||||
if key in _docs:
|
||||
return _docs[key]
|
||||
path = os.path.join(cache, f"{service}_{version}.json") if cache else None
|
||||
if path and os.path.exists(path):
|
||||
doc = json.load(open(path))
|
||||
else:
|
||||
url = f"https://{service}.googleapis.com/$discovery/rest?version={version}"
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=120) as r:
|
||||
raw = r.read()
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"FETCH FAILED {url}: {e}")
|
||||
sys.exit(2)
|
||||
doc = json.loads(raw)
|
||||
if path:
|
||||
os.makedirs(cache, exist_ok=True)
|
||||
open(path, "wb").write(raw)
|
||||
_docs[key] = doc
|
||||
return doc
|
||||
|
||||
|
||||
def methods(doc):
|
||||
found = {}
|
||||
|
||||
def walk(res):
|
||||
for r in res.values():
|
||||
for m in r.get("methods", {}).values():
|
||||
found[m["id"]] = m
|
||||
walk(r.get("resources", {}))
|
||||
|
||||
walk(doc.get("resources", {}))
|
||||
return found
|
||||
|
||||
|
||||
def schema_id(service, version, name):
|
||||
prefix = {"aiplatform": "GoogleCloudAiplatform"}[service]
|
||||
return prefix + version[0].upper() + version[1:] + name
|
||||
|
||||
|
||||
def resolve(doc, prop):
|
||||
"""Follow $ref and array items to the object schema of a property."""
|
||||
if "items" in prop:
|
||||
prop = prop["items"]
|
||||
if "$ref" in prop:
|
||||
return doc["schemas"][prop["$ref"]]
|
||||
return prop
|
||||
|
||||
|
||||
def has_field_path(doc, schema, path):
|
||||
cur = schema
|
||||
for part in path.split("."):
|
||||
props = cur.get("properties", {})
|
||||
if part not in props:
|
||||
return False
|
||||
cur = resolve(doc, props[part])
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
cache = None
|
||||
if "--cache" in sys.argv:
|
||||
cache = sys.argv[sys.argv.index("--cache") + 1]
|
||||
text = open(CALLS_H).read()
|
||||
calls = rows(text, "// CALLS-BEGIN", "// CALLS-END")
|
||||
schemas = rows(text, "// SCHEMAS-BEGIN", "// SCHEMAS-END")
|
||||
enums = rows(text, "// ENUMS-BEGIN", "// ENUMS-END")
|
||||
bad = []
|
||||
checked = 0
|
||||
|
||||
for name, http, path, service, version, rpc, request in calls:
|
||||
doc = discovery(service, version, cache)
|
||||
m = methods(doc).get(rpc)
|
||||
if m is None:
|
||||
bad.append(f"{name}: rpc {rpc} missing in {service} {version}")
|
||||
continue
|
||||
if m["httpMethod"] != http:
|
||||
bad.append(f"{name}: http {m['httpMethod']} != declared {http}")
|
||||
if m["path"] != f"{version}/{path}":
|
||||
bad.append(f"{name}: path {m['path']} != declared {version}/{path}")
|
||||
if request:
|
||||
ref = m.get("request", {}).get("$ref")
|
||||
if not ref:
|
||||
bad.append(f"{name}: declares request fields but the method has no body")
|
||||
else:
|
||||
for f in request.split(","):
|
||||
if not has_field_path(doc, doc["schemas"][ref], f):
|
||||
bad.append(f"{name}: request field {f} missing in {ref}")
|
||||
checked += 1
|
||||
else:
|
||||
checked += 1
|
||||
elif m.get("request") and doc["schemas"][m["request"]["$ref"]].get("properties"):
|
||||
bad.append(f"{name}: method takes a body {m['request']['$ref']} but none is declared")
|
||||
checked += 3
|
||||
|
||||
for service, version, name, fields in schemas:
|
||||
doc = discovery(service, version, cache)
|
||||
s = doc["schemas"].get(schema_id(service, version, name))
|
||||
if s is None:
|
||||
bad.append(f"schema {name} missing in {service} {version}")
|
||||
continue
|
||||
for f in fields.split(","):
|
||||
checked += 1
|
||||
if f not in s.get("properties", {}):
|
||||
bad.append(f"schema {name}: field {f} missing in {service} {version}")
|
||||
|
||||
for service, version, name, field, values in enums:
|
||||
doc = discovery(service, version, cache)
|
||||
s = doc["schemas"].get(schema_id(service, version, name))
|
||||
prop = s.get("properties", {}).get(field) if s else None
|
||||
if prop is None:
|
||||
bad.append(f"enum {name}.{field} missing in {service} {version}")
|
||||
continue
|
||||
listed = set(prop.get("enum", []))
|
||||
for v in values.split(","):
|
||||
checked += 1
|
||||
if v not in listed:
|
||||
bad.append(f"enum {name}.{field}: value {v} not listed in {service} {version}")
|
||||
|
||||
for b in bad:
|
||||
print("MISSING", b)
|
||||
print(f"{len(calls)} calls, {len(schemas)} schemas, {len(enums)} enums, "
|
||||
f"{checked} checks, {len(bad)} missing")
|
||||
sys.exit(1 if bad else 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,133 @@
|
||||
#include "google/sandbox/types.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
const char* kind_name(SandboxKind kind) {
|
||||
switch (kind) {
|
||||
case SandboxKind::kCodeExecution: return "code";
|
||||
case SandboxKind::kComputerUse: return "computer";
|
||||
case SandboxKind::kShell: return "shell";
|
||||
}
|
||||
return "code";
|
||||
}
|
||||
|
||||
bool kind_from_name(const std::string& name, SandboxKind& out) {
|
||||
if (name == "code") out = SandboxKind::kCodeExecution;
|
||||
else if (name == "computer") out = SandboxKind::kComputerUse;
|
||||
else if (name == "shell") out = SandboxKind::kShell;
|
||||
else return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
const char* language_enum(CodeLanguage language) {
|
||||
switch (language) {
|
||||
case CodeLanguage::kPython: return "LANGUAGE_PYTHON";
|
||||
case CodeLanguage::kJavaScript: return "LANGUAGE_JAVASCRIPT";
|
||||
case CodeLanguage::kUnspecified: break;
|
||||
}
|
||||
return "LANGUAGE_UNSPECIFIED";
|
||||
}
|
||||
|
||||
bool language_from_name(const std::string& name, CodeLanguage& out) {
|
||||
if (name == "python") out = CodeLanguage::kPython;
|
||||
else if (name == "javascript") out = CodeLanguage::kJavaScript;
|
||||
else if (name.empty()) out = CodeLanguage::kUnspecified;
|
||||
else return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
Json SandboxSpec::to_json() const {
|
||||
Json spec = Json::object();
|
||||
switch (kind) {
|
||||
case SandboxKind::kCodeExecution: {
|
||||
Json env = Json::object();
|
||||
if (language != CodeLanguage::kUnspecified)
|
||||
env.set("codeLanguage", Json(language_enum(language)));
|
||||
if (machine == MachineConfig::kVcpu4Ram4Gib)
|
||||
env.set("machineConfig", Json("MACHINE_CONFIG_VCPU4_RAM4GIB"));
|
||||
spec.set("codeExecutionEnvironment", env);
|
||||
break;
|
||||
}
|
||||
case SandboxKind::kComputerUse:
|
||||
spec.set("computerUseEnvironment", Json::object());
|
||||
break;
|
||||
case SandboxKind::kShell:
|
||||
spec.set("shellEnvironment", Json::object());
|
||||
break;
|
||||
}
|
||||
return spec;
|
||||
}
|
||||
|
||||
namespace {
|
||||
std::string str(const Json& doc, const char* key) {
|
||||
const Json* v = doc.find(key);
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
} // namespace
|
||||
|
||||
SandboxEnvironment SandboxEnvironment::from_json(const Json& doc) {
|
||||
SandboxEnvironment e;
|
||||
if (!doc.is_object()) return e;
|
||||
e.name = str(doc, "name");
|
||||
e.display_name = str(doc, "displayName");
|
||||
e.state = str(doc, "state");
|
||||
e.create_time = str(doc, "createTime");
|
||||
e.update_time = str(doc, "updateTime");
|
||||
e.expire_time = str(doc, "expireTime");
|
||||
e.template_name = str(doc, "sandboxEnvironmentTemplate");
|
||||
e.snapshot_name = str(doc, "sandboxEnvironmentSnapshot");
|
||||
e.latest_snapshot_name = str(doc, "latestSandboxEnvironmentSnapshot");
|
||||
e.owner = str(doc, "owner");
|
||||
if (const Json* ci = doc.find("connectionInfo"); ci && ci->is_object()) {
|
||||
e.load_balancer_hostname = str(*ci, "loadBalancerHostname");
|
||||
e.routing_token = str(*ci, "routingToken");
|
||||
}
|
||||
if (const Json* spec = doc.find("spec"); spec && spec->is_object()) {
|
||||
e.spec = *spec;
|
||||
if (spec->has("codeExecutionEnvironment")) e.kind = SandboxKind::kCodeExecution;
|
||||
else if (spec->has("computerUseEnvironment")) e.kind = SandboxKind::kComputerUse;
|
||||
else if (spec->has("shellEnvironment")) e.kind = SandboxKind::kShell;
|
||||
}
|
||||
return e;
|
||||
}
|
||||
|
||||
Json SandboxEnvironment::to_json() const {
|
||||
Json o = Json::object();
|
||||
o.set("name", Json(name));
|
||||
o.set("display_name", Json(display_name));
|
||||
o.set("state", Json(state));
|
||||
o.set("kind", Json(kind ? kind_name(*kind) : ""));
|
||||
o.set("create_time", Json(create_time));
|
||||
o.set("expire_time", Json(expire_time));
|
||||
o.set("template", Json(template_name));
|
||||
o.set("latest_snapshot", Json(latest_snapshot_name));
|
||||
return o;
|
||||
}
|
||||
|
||||
Json ExecutionResult::to_json() const {
|
||||
Json o = Json::object();
|
||||
o.set("stdout", Json(std_out));
|
||||
o.set("stderr", Json(std_err));
|
||||
o.set("exit_status", exit_status ? Json(*exit_status) : Json(nullptr));
|
||||
o.set("duration_ms", Json(duration_ms));
|
||||
o.set("ok", Json(ok()));
|
||||
o.set("stdout_truncated", Json(out_truncated));
|
||||
o.set("stderr_truncated", Json(err_truncated));
|
||||
Json fs = Json::array();
|
||||
for (const ProducedFile& f : files) {
|
||||
Json one = Json::object();
|
||||
one.set("name", Json(f.name));
|
||||
one.set("mime_type", Json(f.mime_type));
|
||||
one.set("bytes", Json(static_cast<long>(f.content.size())));
|
||||
fs.push(one);
|
||||
}
|
||||
o.set("files", fs);
|
||||
Json sk = Json::array();
|
||||
for (const std::string& s : skipped) sk.push(Json(s));
|
||||
o.set("skipped", sk);
|
||||
return o;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,93 @@
|
||||
#pragma once
|
||||
|
||||
// Typed values of the sandbox module: the SandboxEnvironment resource, create
|
||||
// request, and the execution result shared by code, shell and computer use.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
// The three mutually exclusive environment categories of SandboxEnvironmentSpec.
|
||||
enum class SandboxKind { kCodeExecution, kComputerUse, kShell };
|
||||
|
||||
const char* kind_name(SandboxKind kind); // "code", "computer", "shell"
|
||||
bool kind_from_name(const std::string& name, SandboxKind& out);
|
||||
|
||||
enum class CodeLanguage { kUnspecified, kPython, kJavaScript };
|
||||
enum class MachineConfig { kUnspecified, kVcpu4Ram4Gib };
|
||||
|
||||
const char* language_enum(CodeLanguage language); // LANGUAGE_*
|
||||
bool language_from_name(const std::string& name, CodeLanguage& out); // python, javascript
|
||||
|
||||
struct SandboxSpec {
|
||||
SandboxKind kind = SandboxKind::kCodeExecution;
|
||||
CodeLanguage language = CodeLanguage::kUnspecified; // Code execution only.
|
||||
MachineConfig machine = MachineConfig::kUnspecified; // Code execution only.
|
||||
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct CreateSandboxRequest {
|
||||
std::string display_name; // Required, 1..128 characters.
|
||||
SandboxSpec spec;
|
||||
long ttl_seconds = 3600; // 60 s .. 14 days.
|
||||
// Optional SandboxEnvironmentTemplate resource name (computer use needs one).
|
||||
std::string template_name;
|
||||
// Optional SandboxEnvironmentSnapshot to restore from, and snapshot owner.
|
||||
std::string snapshot_name;
|
||||
std::string owner;
|
||||
};
|
||||
|
||||
struct SandboxEnvironment {
|
||||
std::string name;
|
||||
std::string display_name;
|
||||
std::string state; // STATE_RUNNING, STATE_PAUSED, ...
|
||||
std::string create_time;
|
||||
std::string update_time;
|
||||
std::string expire_time;
|
||||
std::string template_name;
|
||||
std::string snapshot_name; // sandboxEnvironmentSnapshot
|
||||
std::string latest_snapshot_name; // latestSandboxEnvironmentSnapshot
|
||||
std::string owner;
|
||||
std::string load_balancer_hostname; // connectionInfo (data plane address)
|
||||
std::string routing_token;
|
||||
std::optional<SandboxKind> kind;
|
||||
clients::body::Json spec; // Raw spec, unknown fields kept.
|
||||
|
||||
bool running() const { return state == "STATE_RUNNING"; }
|
||||
static SandboxEnvironment from_json(const clients::body::Json& doc);
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct InputFile {
|
||||
std::string name;
|
||||
std::string content; // Raw bytes.
|
||||
};
|
||||
|
||||
struct ProducedFile {
|
||||
std::string name;
|
||||
std::string mime_type;
|
||||
std::string content; // Raw bytes.
|
||||
};
|
||||
|
||||
// Typed outcome of one execute call. Text is capped and flagged, files over a
|
||||
// cap are skipped loudly (never silently dropped).
|
||||
struct ExecutionResult {
|
||||
std::string std_out;
|
||||
std::string std_err;
|
||||
std::optional<int> exit_status; // Shell: returncode. Code execution: unknown.
|
||||
long duration_ms = -1; // -1 unknown.
|
||||
std::vector<ProducedFile> files;
|
||||
bool out_truncated = false;
|
||||
bool err_truncated = false;
|
||||
std::vector<std::string> skipped; // One reason per skipped chunk or file.
|
||||
|
||||
bool ok() const { return exit_status.value_or(0) == 0; }
|
||||
clients::body::Json to_json() const; // Files as name, mime, size (no bytes).
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,115 @@
|
||||
#include "google/sandbox/workspace_files.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sandbox/caps.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
using security::crypto::Base64;
|
||||
|
||||
namespace {
|
||||
|
||||
// Paths hold only [A-Za-z0-9._/-] (valid_workspace_path), so single quotes
|
||||
// can never be broken out of.
|
||||
std::string quoted(const std::string& s) { return "'" + s + "'"; }
|
||||
|
||||
void check_ok(const ExecutionResult& r, const std::string& what) {
|
||||
if (!r.exit_status)
|
||||
throw GoogleError(502, "BAD_RESPONSE", what + ": no exit status in the reply", "aiplatform-sandbox");
|
||||
if (*r.exit_status != 0)
|
||||
throw GoogleError(502, "COMMAND_FAILED",
|
||||
what + " failed with exit " + std::to_string(*r.exit_status) + ": " +
|
||||
r.std_err.substr(0, 200),
|
||||
"aiplatform-sandbox");
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string WorkspaceFiles::resolve(const std::string& path) {
|
||||
if (!valid_workspace_path(path)) throw GoogleError::invalid("bad path");
|
||||
std::string full = path[0] == '/' ? path : std::string(kRoot) + "/" + path;
|
||||
const std::string root = kRoot;
|
||||
if (full != root && full.rfind(root + "/", 0) != 0)
|
||||
throw GoogleError::invalid("path must stay under " + root);
|
||||
while (full.find("//") != std::string::npos) full.erase(full.find("//"), 1);
|
||||
return full;
|
||||
}
|
||||
|
||||
std::string WorkspaceFiles::write_command(const std::string& p, const std::string& content) {
|
||||
return "mkdir -p \"$(dirname " + quoted(p) + ")\" && printf %s " +
|
||||
quoted(Base64::encode(content)) + " | base64 -d > " + quoted(p);
|
||||
}
|
||||
|
||||
std::string WorkspaceFiles::read_command(const std::string& p) {
|
||||
return "test -f " + quoted(p) + " && { [ \"$(stat -c %s " + quoted(p) + ")\" -le " +
|
||||
std::to_string(SandboxCaps::kMaxShellReadBytes) +
|
||||
" ] || { echo too_large >&2; exit 3; }; base64 -w0 < " + quoted(p) + "; }";
|
||||
}
|
||||
|
||||
std::string WorkspaceFiles::list_command(const std::string& p) {
|
||||
return "ls -1Ap -- " + quoted(p);
|
||||
}
|
||||
|
||||
std::string WorkspaceFiles::remove_command(const std::string& p) {
|
||||
return "test -f " + quoted(p) + " && rm -f -- " + quoted(p);
|
||||
}
|
||||
|
||||
void WorkspaceFiles::write(const std::string& sandbox_name, const std::string& path,
|
||||
const std::string& content) const {
|
||||
std::string p = resolve(path);
|
||||
if (p == kRoot) throw GoogleError::invalid("path names the workspace root");
|
||||
if (content.size() > SandboxCaps::kMaxShellWriteBytes)
|
||||
throw GoogleError::invalid("content over " +
|
||||
std::to_string(SandboxCaps::kMaxShellWriteBytes) +
|
||||
" bytes (use a code sandbox for larger files)");
|
||||
ShellRequest req;
|
||||
req.command = write_command(p, content);
|
||||
check_ok(shell_.run_internal(sandbox_name, req), "write");
|
||||
}
|
||||
|
||||
std::string WorkspaceFiles::read(const std::string& sandbox_name,
|
||||
const std::string& path) const {
|
||||
std::string p = resolve(path);
|
||||
ShellRequest req;
|
||||
req.command = read_command(p);
|
||||
ExecutionResult r = shell_.run_internal(sandbox_name, req);
|
||||
check_ok(r, "read");
|
||||
std::string raw;
|
||||
if (r.out_truncated || !Base64::decode(r.std_out, raw))
|
||||
throw GoogleError(502, "BAD_RESPONSE", "file content is not valid base64", "aiplatform-sandbox");
|
||||
return raw;
|
||||
}
|
||||
|
||||
std::vector<std::string> WorkspaceFiles::list(const std::string& sandbox_name,
|
||||
const std::string& dir) const {
|
||||
std::string p = resolve(dir.empty() ? std::string(kRoot) : dir);
|
||||
ShellRequest req;
|
||||
req.command = list_command(p);
|
||||
ExecutionResult r = shell_.run_internal(sandbox_name, req);
|
||||
check_ok(r, "list");
|
||||
std::vector<std::string> out;
|
||||
std::string line;
|
||||
for (char c : r.std_out) {
|
||||
if (c == '\n') {
|
||||
if (!line.empty()) out.push_back(line);
|
||||
line.clear();
|
||||
} else {
|
||||
line.push_back(c);
|
||||
}
|
||||
if (out.size() >= 1000) break; // Hard cap on entries.
|
||||
}
|
||||
if (!line.empty() && out.size() < 1000) out.push_back(line);
|
||||
return out;
|
||||
}
|
||||
|
||||
void WorkspaceFiles::remove_file(const std::string& sandbox_name,
|
||||
const std::string& path) const {
|
||||
std::string p = resolve(path);
|
||||
if (p == kRoot) throw GoogleError::invalid("path names the workspace root");
|
||||
ShellRequest req;
|
||||
req.command = remove_command(p);
|
||||
check_ok(shell_.run_internal(sandbox_name, req), "remove");
|
||||
}
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
@@ -0,0 +1,43 @@
|
||||
#pragma once
|
||||
|
||||
// File management inside a shell sandbox. LOCAL ABSTRACTION: Google has no
|
||||
// file REST surface for workspaces (SPEC: Workspaces). Every operation is a
|
||||
// shell command through ShellRunner, with paths validated and single-quoted.
|
||||
// Bytes travel base64 in the command line, so sizes are small by cap; large
|
||||
// files belong in a code sandbox's `files` input/output.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/sandbox/shell_runner.h"
|
||||
|
||||
namespace molo::google::sandbox {
|
||||
|
||||
class WorkspaceFiles {
|
||||
public:
|
||||
explicit WorkspaceFiles(const ShellRunner& shell) : shell_(shell) {}
|
||||
|
||||
static constexpr const char* kRoot = "/workspace";
|
||||
|
||||
// Relative paths resolve under /workspace; absolute paths must stay under it.
|
||||
static std::string resolve(const std::string& path);
|
||||
|
||||
void write(const std::string& sandbox_name, const std::string& path,
|
||||
const std::string& content) const;
|
||||
std::string read(const std::string& sandbox_name, const std::string& path) const;
|
||||
std::vector<std::string> list(const std::string& sandbox_name,
|
||||
const std::string& dir = "") const;
|
||||
// Removes one regular file (rm -f, never recursive). Explicit method.
|
||||
void remove_file(const std::string& sandbox_name, const std::string& path) const;
|
||||
|
||||
// The exact commands (compared in tests).
|
||||
static std::string write_command(const std::string& resolved, const std::string& content);
|
||||
static std::string read_command(const std::string& resolved);
|
||||
static std::string list_command(const std::string& resolved);
|
||||
static std::string remove_command(const std::string& resolved);
|
||||
|
||||
private:
|
||||
const ShellRunner& shell_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sandbox
|
||||
Reference in New Issue
Block a user