Merge sessions module (Agent Sessions, Memory Bank)
This commit is contained in:
@@ -0,0 +1,168 @@
|
||||
# Google Agent Sessions and Memory Bank - Specification
|
||||
|
||||
Owned directory: `src/google/sessions/`. Namespace `molo::google::sessions`.
|
||||
Depends on the `src/google` foundation (`GoogleClient`, `Service`, `Operation`,
|
||||
`wait_operation`, `list_all`, `gemini_types.h`). Does not depend on the runtime
|
||||
module: the Agent Engine is accepted as a validated resource-name string.
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Gemini Enterprise Agent Platform, Agent Engine (`reasoningEngines`) children:
|
||||
|
||||
- Agent Sessions: create (custom session id), get, list, patch, delete,
|
||||
compact, appendEvent, list events.
|
||||
- Agent Memory Bank: generate (from a session, a conversation, or given facts),
|
||||
ingestEvents (streaming), retrieve (scope, similarity search, metadata
|
||||
filters), retrieveProfiles (Memory Profiles, low-latency structured recall),
|
||||
create, get, list, patch, delete, purge (explicit, guarded), revisions
|
||||
(list, get), rollback, topics (managed and custom), scope maps, metadata.
|
||||
- Long-running operations: every mutating call returns an Operation; the module
|
||||
waits with `wait_operation` (hard caps in `OperationCaps`). `operation()`,
|
||||
`cancel_operation()` and `wait()` expose them for one engine only.
|
||||
|
||||
Everything is against one API surface: `aiplatform` **v1beta1**. See section 6
|
||||
for why not v1.
|
||||
|
||||
## 2. Module map (one class or concern per file)
|
||||
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings (`SessionsCaps`, `MemoryCaps`) |
|
||||
| `calls.h` | The ONE table of every REST call (verb, path, discovery method, request fields, query params) |
|
||||
| `names.h/.cpp` | `EngineName` (validated resource name), `Check` (input validators), wire helpers |
|
||||
| `session_types.h/.cpp` | `SessionSpec`, `Session`, `SessionPatch`, `CompactRequest`, `SessionEvent`, list options |
|
||||
| `memory_types.h/.cpp` | `Memory`, `MemorySpec`, `MemoryPatch`, `Topic`, `MetadataValue`, `MetadataFilter`, `MemoryGenerateRequest`, `IngestRequest`, `RetrieveRequest`, `PurgeRequest`, `MemoryRevision`, `Profile` |
|
||||
| `engine_service.h/.cpp` | `EngineService : Service`: table-driven `send`, `settle`, capped `list_items`, operation helpers |
|
||||
| `sessions.h/.cpp` | `Sessions : EngineService` |
|
||||
| `memory_bank.h/.cpp` | `MemoryBank : EngineService` |
|
||||
| `tools/check_discovery.py` | Verifies `calls.h` and the typed bodies against Google's Discovery document |
|
||||
|
||||
Tests: `tests/src/google/sessions/test_sessions.cpp`. Example:
|
||||
`example/app/agent/SessionsDemo.cpp` (routes under `/agent/sessions`), tests in
|
||||
`tests/example/agent/sessions/`.
|
||||
|
||||
## 3. Static caps (see caps.h)
|
||||
|
||||
| Knob | Value |
|
||||
|---|---|
|
||||
| Session / memory id | 1..63 chars `[a-z0-9-]`; memory id starts with a letter, ends alphanumeric |
|
||||
| Engine name | `projects/{p}/locations/{l}/reasoningEngines/{e}`, 512 chars max |
|
||||
| Session TTL | `Ns` form, 24 h minimum (Google), 3650 days maximum (ours) |
|
||||
| session_state / event | 64 KiB / 256 KiB serialized |
|
||||
| Labels | 20, key and value 64 chars |
|
||||
| List pages / items | 10 pages of 100, 1000 items; more is `RESULT_TOO_LARGE`, narrow the filter |
|
||||
| Scope | 1..10 entries, 128 chars, no `*` |
|
||||
| Fact / description / context | 4000 chars |
|
||||
| Generate sources | 100 events, 64 KiB; direct memories max 5 (Google) |
|
||||
| Retrieve | query 1000 chars, top_k 0..50 (default 5), page size 0..100 |
|
||||
| Topics / metadata / filter groups | 10 / 20 entries / 10 groups of 10 filters |
|
||||
| Operation wait | `OperationCaps`: 600 s, 2 s interval, 400 polls |
|
||||
|
||||
## 4. Contracts
|
||||
|
||||
- All input and every id is validated before any network traffic and throws
|
||||
`GoogleError::invalid` (HTTP 400 `INVALID_ARGUMENT`, empty service name).
|
||||
- No credential: every call throws `NotConfiguredError` (503) before traffic.
|
||||
- Destructive calls are explicit methods: `delete_session`, `delete_memory`,
|
||||
`purge`. `purge` is a server-side dry run unless `force` is set, and a forced
|
||||
purge additionally needs `confirm` equal to the engine id (local guard).
|
||||
Compaction rewrites history and is its own method.
|
||||
- Request bodies are built only from fields named in the `calls.h` row of the
|
||||
call (tested), and those fields are checked against the discovery schema.
|
||||
- Unknown response parts stay raw `Json` (event `actions`, `eventMetadata`,
|
||||
`rawEvent`, `sessionState`, memory `metadata`, `structuredContent`, profile
|
||||
fields).
|
||||
- Example routes: GET pages answer 2xx without a credential; every route that
|
||||
calls Google is POST or DELETE and answers 503 `google_not_configured`
|
||||
without a credential and 422 on bad input.
|
||||
|
||||
## 5. Verified against discovery
|
||||
|
||||
`python3 -I src/google/sessions/tools/check_discovery.py` fetches
|
||||
`https://aiplatform.googleapis.com/$discovery/rest?version=v1beta1` (no
|
||||
credential) and fails if a declared call, verb, path, request field, query
|
||||
parameter, enum value, response key or nested request field is absent.
|
||||
|
||||
Result 2026-10-24: `OK 2026-10-24: 23 calls, 7 enums, 25 response keys, 18
|
||||
nested schemas match aiplatform v1beta1 discovery`. A deliberately misspelled
|
||||
field made it exit 1 (checked). Rerun it after any change to `calls.h` or the
|
||||
typed bodies.
|
||||
|
||||
From discovery (v1beta1, `projects.locations.reasoningEngines`):
|
||||
sessions create (POST, `sessionId` query, returns Operation), get, list
|
||||
(`filter orderBy pageSize pageToken`), patch (`updateMask`), delete (returns
|
||||
Operation), appendEvent (POST `:appendEvent`, body `SessionEvent`), compact
|
||||
(POST `:compact`, returns Operation), events.list (`sessionEvents`);
|
||||
memories generate / retrieve / retrieveProfiles / create (`memoryId`) / get /
|
||||
list / patch (`updateMask`) / delete / purge / rollback / ingestEvents,
|
||||
memories.revisions list / get; operations get / cancel. Discovery also lists
|
||||
`wait`, `list`, `delete` on operations, which this module does not wrap
|
||||
(`wait_operation` polls with GET).
|
||||
|
||||
## 6. Verified live
|
||||
|
||||
Read-only, 2026-10-24, project read from `GOOGLE_CLOUD_PROJECT`, no resource
|
||||
created, changed or deleted. Two credentials:
|
||||
|
||||
Express API key (`AQ.`):
|
||||
|
||||
| Call | Result |
|
||||
|---|---|
|
||||
| GET `v1/.../reasoningEngines/1/{sessions, sessions/abc, sessions/abc/events, memories, memories/abc}` | 401 `UNAUTHENTICATED`: "API keys are not supported by this API" |
|
||||
| GET `v1beta1/.../reasoningEngines/1/{sessions, sessions/abc, sessions/abc/events, memories, memories/abc, memories/abc/revisions}` | 404 `NOT_FOUND` "The ReasoningEngine does not exist." (key accepted, path routed) |
|
||||
| GET `v1beta1/.../reasoningEngines/1/operations/1` | 404 `NOT_FOUND` "Operation ... not found." |
|
||||
|
||||
Service-account OAuth bearer token (token exchange, never stored):
|
||||
|
||||
| Call | Result |
|
||||
|---|---|
|
||||
| GET `v1` and `v1beta1` `.../reasoningEngines` | 200 `{}` (no engine exists) |
|
||||
| GET `v1` and `v1beta1` `.../reasoningEngines/1/{sessions, sessions/abc, sessions/abc/events, memories, memories/abc, memories/abc/revisions}` | 404 `NOT_FOUND` "The ReasoningEngine does not exist." |
|
||||
| GET `v1` and `v1beta1` `.../reasoningEngines/1/operations/1` | 404 `NOT_FOUND` "Operation ... not found." |
|
||||
|
||||
The module uses v1beta1 only because it is the one surface that accepts both
|
||||
an express key and a token; v1 refuses keys. The opt-in test `sessions_live`
|
||||
(`MOLO_LIVE_NET=1`, `GOOGLE_API_KEY` or `GOOGLE_ACCESS_TOKEN`,
|
||||
`GOOGLE_CLOUD_PROJECT`; loud SKIP otherwise) repeats six of the reads through
|
||||
the real client (sessions list / get / events, memories list / get /
|
||||
revisions) and passed with the key and with the token on 2026-10-24.
|
||||
|
||||
No Agent Engine exists in the project, so **no happy path was verified
|
||||
live**: create, patch, delete, append, compact, generate, ingest, retrieve,
|
||||
retrieveProfiles, purge, rollback are verified by discovery only. Nothing that
|
||||
costs money was called.
|
||||
|
||||
## 7. Unverified
|
||||
|
||||
Neither discovery nor the docs pages read state these; they are inferred:
|
||||
|
||||
- Operation `response` bodies (discovery types them as `Any`): that generate
|
||||
answers `{"generatedMemories":[{"memory":{...},"action":"..."}]}`, that purge
|
||||
answers `{"purgeCount":N}`, that create and patch answer the resource
|
||||
(`name`, ...). Parsers are tolerant: missing keys give empty results, and
|
||||
create/patch fall back to a GET when the id is known.
|
||||
- `GeneratedMemory.action` values (not enumerated anywhere read).
|
||||
- That `delete` Operations are returned `done` or pollable at `operations/{id}`
|
||||
under the engine (the path shape follows discovery's `operations.get`).
|
||||
- Filter grammar (AIP-160) beyond the field names the docs list: `topics`,
|
||||
`scope` as a JSON string with equality, `timestamp` ranges for events,
|
||||
`displayName` / `userId` / `labels.k` for sessions.
|
||||
- Similarity search needs the engine's Memory Bank configured with a similarity
|
||||
search model (docs); the error body when it is not is unread.
|
||||
- Memory Profiles need schema configs on the engine (`MemoryBankConfig` in the
|
||||
runtime module); the shape of `profile` is free-form and kept raw.
|
||||
- Managed topic names beyond the four in discovery, and `thought`/`audio` parts
|
||||
inside event `content` (typed by `gemini_types.h`, unknown parts dropped).
|
||||
|
||||
## 8. Gaps
|
||||
|
||||
- `servingProfiles` (location-level create/get/list/patch for Gemini Live,
|
||||
Interactions and Response APIs) is listed by discovery next to this product
|
||||
but is not engine-scoped; not implemented here. Owner: the `live` or
|
||||
`models` module.
|
||||
- Engine-level Memory Bank configuration (`contextSpec.memoryBankConfig`,
|
||||
customization, TTL config) is part of the Agent Engine resource: runtime
|
||||
module.
|
||||
- Operations `list`, `delete`, `wait` (server-side) are not wrapped.
|
||||
- Session and memory `labels` / `metadata` filters beyond a passthrough string.
|
||||
- Token refresh is the caller's job (foundation gap).
|
||||
@@ -0,0 +1,99 @@
|
||||
#pragma once
|
||||
|
||||
// The ONE table of every REST call this module makes. Each row is checked
|
||||
// against Google's Discovery document by tools/check_discovery.py (method id,
|
||||
// HTTP verb, path, request schema fields, query parameters), and every
|
||||
// request builder in this module takes its verb and path from here.
|
||||
//
|
||||
// Surface: aiplatform v1beta1. Probed live (2026-10-24): v1 answers 401
|
||||
// "API keys are not supported" for sessions and memories, v1beta1 accepts an
|
||||
// express API key (404 for a missing engine). See SPEC.md.
|
||||
//
|
||||
// Row: {Call, verb, path template, discovery method (under
|
||||
// aiplatform.projects.locations.reasoningEngines.), request schema suffix
|
||||
// (after GoogleCloudAiplatformV1beta1), request fields used, query params used}.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "clients/client.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
inline constexpr const char* kDiscoveryService = "aiplatform";
|
||||
inline constexpr const char* kDiscoveryVersion = "v1beta1";
|
||||
inline constexpr const char* kDiscoveryPrefix =
|
||||
"aiplatform.projects.locations.reasoningEngines.";
|
||||
|
||||
enum class Call {
|
||||
kSessionCreate, kSessionGet, kSessionList, kSessionPatch, kSessionDelete,
|
||||
kSessionAppendEvent, kSessionCompact, kEventList,
|
||||
kMemoryGenerate, kMemoryRetrieve, kMemoryRetrieveProfiles, kMemoryCreate,
|
||||
kMemoryGet, kMemoryList, kMemoryPatch, kMemoryDelete, kMemoryPurge,
|
||||
kMemoryRollback, kMemoryIngestEvents, kRevisionList, kRevisionGet,
|
||||
kOperationGet, kOperationCancel,
|
||||
kCount
|
||||
};
|
||||
|
||||
struct CallSpec {
|
||||
Call call;
|
||||
const char* verb;
|
||||
const char* path;
|
||||
const char* method;
|
||||
const char* request;
|
||||
const char* fields;
|
||||
const char* query;
|
||||
};
|
||||
|
||||
// clang-format off
|
||||
inline constexpr CallSpec kCalls[] = {
|
||||
{Call::kSessionCreate, "POST", "/{parent}/sessions", "sessions.create", "Session", "userId displayName labels sessionState ttl", "sessionId"},
|
||||
{Call::kSessionGet, "GET", "/{name}", "sessions.get", "", "", ""},
|
||||
{Call::kSessionList, "GET", "/{parent}/sessions", "sessions.list", "", "", "filter orderBy pageSize pageToken"},
|
||||
{Call::kSessionPatch, "PATCH", "/{name}", "sessions.patch", "Session", "displayName labels sessionState ttl", "updateMask"},
|
||||
{Call::kSessionDelete, "DELETE", "/{name}", "sessions.delete", "", "", ""},
|
||||
{Call::kSessionAppendEvent, "POST", "/{name}:appendEvent", "sessions.appendEvent", "SessionEvent", "author invocationId timestamp content actions errorCode errorMessage eventMetadata rawEvent", ""},
|
||||
{Call::kSessionCompact, "POST", "/{name}:compact", "sessions.compact", "CompactSessionRequest", "compaction", ""},
|
||||
{Call::kEventList, "GET", "/{parent}/events", "sessions.events.list", "", "", "filter orderBy pageSize pageToken"},
|
||||
{Call::kMemoryGenerate, "POST", "/{parent}/memories:generate", "memories.generate", "GenerateMemoriesRequest", "scope disableConsolidation allowedTopics vertexSessionSource directContentsSource directMemoriesSource metadata metadataMergeStrategy timeZone revisionLabels revisionTtl revisionExpireTime disableMemoryRevisions", ""},
|
||||
{Call::kMemoryRetrieve, "POST", "/{parent}/memories:retrieve", "memories.retrieve", "RetrieveMemoriesRequest", "scope filter filterGroups memoryTypes similaritySearchParams simpleRetrievalParams", ""},
|
||||
{Call::kMemoryRetrieveProfiles, "POST", "/{parent}/memories:retrieveProfiles", "memories.retrieveProfiles", "RetrieveProfilesRequest", "scope", ""},
|
||||
{Call::kMemoryCreate, "POST", "/{parent}/memories", "memories.create", "Memory", "fact scope displayName description context topics ttl metadata revisionLabels revisionTtl disableMemoryRevisions memoryType structuredContent", "memoryId"},
|
||||
{Call::kMemoryGet, "GET", "/{name}", "memories.get", "", "", ""},
|
||||
{Call::kMemoryList, "GET", "/{parent}/memories", "memories.list", "", "", "filter orderBy pageSize pageToken"},
|
||||
{Call::kMemoryPatch, "PATCH", "/{name}", "memories.patch", "Memory", "fact displayName description context topics ttl metadata revisionLabels revisionTtl disableMemoryRevisions structuredContent", "updateMask"},
|
||||
{Call::kMemoryDelete, "DELETE", "/{name}", "memories.delete", "", "", ""},
|
||||
{Call::kMemoryPurge, "POST", "/{parent}/memories:purge", "memories.purge", "PurgeMemoriesRequest", "filter filterGroups force", ""},
|
||||
{Call::kMemoryRollback, "POST", "/{name}:rollback", "memories.rollback", "RollbackMemoryRequest", "targetRevisionId", ""},
|
||||
{Call::kMemoryIngestEvents, "POST", "/{parent}/memories:ingestEvents", "memories.ingestEvents", "IngestEventsRequest", "scope streamId forceFlush directContentsSource generationTriggerConfig revisionTtl disableMemoryRevisions revisionLabels metadata metadataMergeStrategy", ""},
|
||||
{Call::kRevisionList, "GET", "/{parent}/revisions", "memories.revisions.list", "", "", "filter pageSize pageToken"},
|
||||
{Call::kRevisionGet, "GET", "/{name}", "memories.revisions.get", "", "", ""},
|
||||
{Call::kOperationGet, "GET", "/{name}", "operations.get", "", "", ""},
|
||||
{Call::kOperationCancel, "POST", "/{name}:cancel", "operations.cancel", "", "", ""},
|
||||
};
|
||||
// clang-format on
|
||||
|
||||
inline const CallSpec& spec_of(Call call) {
|
||||
for (const CallSpec& row : kCalls)
|
||||
if (row.call == call) return row;
|
||||
return kCalls[0]; // Unreachable: every enumerator has a row (tested).
|
||||
}
|
||||
|
||||
inline clients::Method verb_of(const CallSpec& spec) {
|
||||
std::string v = spec.verb;
|
||||
if (v == "POST") return clients::Method::kPost;
|
||||
if (v == "PATCH") return clients::Method::kPatch;
|
||||
if (v == "DELETE") return clients::Method::kDelete;
|
||||
return clients::Method::kGet;
|
||||
}
|
||||
|
||||
// Substitutes the single {parent} or {name} placeholder with a validated
|
||||
// resource name and appends an encoded query string.
|
||||
inline std::string expand(Call call, const std::string& resource,
|
||||
const std::string& query = "") {
|
||||
std::string path = spec_of(call).path;
|
||||
std::size_t open = path.find('{');
|
||||
std::size_t close = path.find('}');
|
||||
return path.substr(0, open) + resource + path.substr(close + 1) + query;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,50 @@
|
||||
#pragma once
|
||||
|
||||
// Static ceilings for Agent Sessions and Agent Memory Bank. Never autotuned.
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
struct SessionsCaps {
|
||||
static constexpr std::size_t kMaxIdChars = 63; // Google: [a-z0-9-], 63
|
||||
static constexpr std::size_t kMaxNameChars = 512; // resource name
|
||||
static constexpr std::size_t kMaxUserIdChars = 128;
|
||||
static constexpr std::size_t kMaxDisplayChars = 128;
|
||||
static constexpr std::size_t kMaxLabels = 20;
|
||||
static constexpr std::size_t kMaxLabelChars = 64; // key and value
|
||||
static constexpr std::size_t kMaxStateBytes = 64 * 1024;
|
||||
static constexpr std::size_t kMaxEventBytes = 256 * 1024;
|
||||
static constexpr std::size_t kMaxFilterChars = 512;
|
||||
static constexpr std::size_t kMaxTimestampChars = 40;
|
||||
static constexpr int kMinTtlSeconds = 24 * 3600; // Google minimum
|
||||
static constexpr long kMaxTtlSeconds = 3650L * 24 * 3600;
|
||||
static constexpr int kPageSize = 100; // Google ceiling
|
||||
static constexpr std::size_t kMaxListPages = 10;
|
||||
static constexpr std::size_t kMaxListItems = 1000;
|
||||
};
|
||||
|
||||
struct MemoryCaps {
|
||||
static constexpr std::size_t kMaxFactChars = 4000;
|
||||
static constexpr std::size_t kMaxQueryChars = 1000;
|
||||
static constexpr std::size_t kMaxScopeEntries = 10;
|
||||
static constexpr std::size_t kMaxScopeChars = 128; // key and value
|
||||
static constexpr std::size_t kMaxTopics = 10;
|
||||
static constexpr std::size_t kMaxTopicLabelChars = 64;
|
||||
static constexpr std::size_t kMaxDirectMemories = 5; // Google
|
||||
static constexpr std::size_t kMaxSourceEvents = 100; // generate from contents
|
||||
static constexpr std::size_t kMaxSourceBytes = 64 * 1024;
|
||||
static constexpr std::size_t kMaxMetadataEntries = 20;
|
||||
static constexpr std::size_t kMaxMetadataChars = 256; // key and string value
|
||||
static constexpr std::size_t kMaxFilterGroups = 10;
|
||||
static constexpr std::size_t kMaxFiltersPerGroup = 10;
|
||||
static constexpr std::size_t kMaxMemoryTypes = 2;
|
||||
static constexpr std::size_t kMaxUpdateMaskChars = 256;
|
||||
static constexpr int kMaxTopK = 50;
|
||||
static constexpr int kDefaultTopK = 5;
|
||||
static constexpr int kPageSize = 100;
|
||||
static constexpr std::size_t kMaxListPages = 10;
|
||||
static constexpr std::size_t kMaxListItems = 1000;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,73 @@
|
||||
#include "google/sessions/engine_service.h"
|
||||
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
EngineService::EngineService(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper)
|
||||
: Service(std::move(client), endpoints::aiplatform_beta(engine.location())),
|
||||
engine_name_(std::move(engine)),
|
||||
sleeper_(std::move(sleeper)) {}
|
||||
|
||||
Json EngineService::send(Call call, const std::string& resource, const Json* body,
|
||||
const std::string& query) const {
|
||||
const CallSpec& row = spec_of(call);
|
||||
return client_->request(verb_of(row), endpoint_, expand(call, resource, query), body);
|
||||
}
|
||||
|
||||
Operation EngineService::settle(const Json& reply) const {
|
||||
Operation op = Operation::from_json(reply);
|
||||
if (op.done) {
|
||||
if (op.failed())
|
||||
throw GoogleError(502, "OPERATION_FAILED", op.error_message, endpoint_.name);
|
||||
return op;
|
||||
}
|
||||
if (op.name.empty())
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "no operation returned", endpoint_.name);
|
||||
return wait(op.name);
|
||||
}
|
||||
|
||||
namespace {
|
||||
void check_operation_name(const EngineName& engine, const std::string& name) {
|
||||
const std::string prefix = engine.str() + "/";
|
||||
bool ok = name.size() > prefix.size() && name.compare(0, prefix.size(), prefix) == 0 &&
|
||||
name.find("/operations/") != std::string::npos &&
|
||||
name.find("..") == std::string::npos && name.find('?') == std::string::npos &&
|
||||
name.size() <= 512;
|
||||
if (!ok) throw GoogleError::invalid("operation name must belong to this engine");
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json EngineService::operation(const std::string& operation_name) const {
|
||||
check_operation_name(engine_name_, operation_name);
|
||||
return send(Call::kOperationGet, operation_name);
|
||||
}
|
||||
|
||||
void EngineService::cancel_operation(const std::string& operation_name) const {
|
||||
check_operation_name(engine_name_, operation_name);
|
||||
Json empty = Json::object();
|
||||
send(Call::kOperationCancel, operation_name, &empty);
|
||||
}
|
||||
|
||||
Operation EngineService::wait(const std::string& operation_name) const {
|
||||
check_operation_name(engine_name_, operation_name);
|
||||
return wait_operation(*client_, endpoint_, operation_name, sleeper_);
|
||||
}
|
||||
|
||||
Json EngineService::list_items(Call call, const std::string& resource,
|
||||
const std::string& query, const std::string& items_key,
|
||||
std::size_t max_pages, std::size_t max_items) const {
|
||||
Json items = list_all(*client_, endpoint_, expand(call, resource, query), items_key,
|
||||
max_pages);
|
||||
if (items.as_array().size() > max_items)
|
||||
throw GoogleError(502, "RESULT_TOO_LARGE",
|
||||
"more than " + std::to_string(max_items) +
|
||||
" items; narrow the filter",
|
||||
endpoint_.name);
|
||||
return items;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,48 @@
|
||||
#pragma once
|
||||
|
||||
// Base of Sessions and MemoryBank: one validated Agent Engine, the v1beta1
|
||||
// aiplatform endpoint of its location, table-driven requests (calls.h) and
|
||||
// long-running operation handling shared by both.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "google/operation.h"
|
||||
#include "google/service.h"
|
||||
#include "google/sessions/calls.h"
|
||||
#include "google/sessions/names.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
class EngineService : public Service {
|
||||
public:
|
||||
const EngineName& engine() const { return engine_name_; }
|
||||
|
||||
// Raw operation document (GET) and cancel (POST :cancel). The name must be
|
||||
// an operation of this engine; nothing else is reachable through here.
|
||||
clients::body::Json operation(const std::string& operation_name) const;
|
||||
void cancel_operation(const std::string& operation_name) const;
|
||||
// Polls until done (hard caps in OperationCaps); throws on failure.
|
||||
Operation wait(const std::string& operation_name) const;
|
||||
|
||||
protected:
|
||||
EngineService(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper);
|
||||
|
||||
// One table-driven request. `resource` fills the {parent} / {name} slot.
|
||||
clients::body::Json send(Call call, const std::string& resource,
|
||||
const clients::body::Json* body = nullptr,
|
||||
const std::string& query = "") const;
|
||||
// Follows an Operation reply to completion (done replies are checked).
|
||||
Operation settle(const clients::body::Json& reply) const;
|
||||
// Paged list through the table; caps pages and total items.
|
||||
clients::body::Json list_items(Call call, const std::string& resource,
|
||||
const std::string& query,
|
||||
const std::string& items_key,
|
||||
std::size_t max_pages, std::size_t max_items) const;
|
||||
|
||||
EngineName engine_name_;
|
||||
Sleeper sleeper_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,178 @@
|
||||
#include "google/sessions/memory_bank.h"
|
||||
|
||||
#include "google/resource.h"
|
||||
#include "google/sessions/caps.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
void check_revision_id(const std::string& id) {
|
||||
if (!valid_segment(id)) throw GoogleError::invalid("bad revision id");
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
MemoryBank::MemoryBank(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper)
|
||||
: EngineService(std::move(client), std::move(engine), std::move(sleeper)) {}
|
||||
|
||||
std::vector<GeneratedMemory> MemoryBank::generate(const MemoryGenerateRequest& request) const {
|
||||
request.validate();
|
||||
Json body = request.to_json(engine_name_.str());
|
||||
Operation op = settle(send(Call::kMemoryGenerate, engine_name_.str(), &body));
|
||||
std::vector<GeneratedMemory> out;
|
||||
const Json* list =
|
||||
op.response.is_object() ? op.response.find("generatedMemories") : nullptr;
|
||||
if (list && list->is_array())
|
||||
for (const Json& item : list->as_array()) {
|
||||
GeneratedMemory g;
|
||||
if (const Json* m = item.find("memory")) g.memory = Memory::from_json(*m);
|
||||
g.action = str_at(item, "action");
|
||||
out.push_back(std::move(g));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
void MemoryBank::ingest_events(const IngestRequest& request) const {
|
||||
request.validate();
|
||||
Json body = request.to_json();
|
||||
settle(send(Call::kMemoryIngestEvents, engine_name_.str(), &body));
|
||||
}
|
||||
|
||||
RetrieveResult MemoryBank::retrieve(const RetrieveRequest& request) const {
|
||||
request.validate();
|
||||
Json body = request.to_json();
|
||||
Json reply = send(Call::kMemoryRetrieve, engine_name_.str(), &body);
|
||||
RetrieveResult out;
|
||||
const Json* list = reply.find("retrievedMemories");
|
||||
if (list && list->is_array()) {
|
||||
if (list->as_array().size() > MemoryCaps::kMaxListItems)
|
||||
throw GoogleError(502, "RESULT_TOO_LARGE", "retrieve returned too many memories",
|
||||
endpoint_.name);
|
||||
for (const Json& item : list->as_array()) {
|
||||
RetrievedMemory r;
|
||||
if (const Json* m = item.find("memory")) r.memory = Memory::from_json(*m);
|
||||
if (const Json* d = item.find("distance"); d && d->is_number()) {
|
||||
r.distance = d->as_number();
|
||||
r.has_distance = true;
|
||||
}
|
||||
out.memories.push_back(std::move(r));
|
||||
}
|
||||
}
|
||||
out.next_page_token = str_at(reply, "nextPageToken");
|
||||
return out;
|
||||
}
|
||||
|
||||
std::map<std::string, Profile> MemoryBank::retrieve_profiles(const StringMap& scope) const {
|
||||
Check::scope(scope);
|
||||
Json body = Json::object();
|
||||
body.set("scope", to_json(scope));
|
||||
Json reply = send(Call::kMemoryRetrieveProfiles, engine_name_.str(), &body);
|
||||
std::map<std::string, Profile> out;
|
||||
const Json* profiles = reply.find("profiles");
|
||||
if (profiles && profiles->is_object())
|
||||
for (const auto& [key, value] : profiles->as_object()) {
|
||||
if (out.size() >= MemoryCaps::kMaxListItems) break;
|
||||
Profile p;
|
||||
p.schema_id = str_at(value, "schemaId");
|
||||
if (p.schema_id.empty()) p.schema_id = key;
|
||||
if (const Json* f = value.find("profile")) p.fields = *f;
|
||||
out[key] = std::move(p);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
Memory MemoryBank::create(const MemorySpec& spec) const {
|
||||
spec.validate();
|
||||
Json body = spec.to_json();
|
||||
Operation op = settle(send(Call::kMemoryCreate, engine_name_.str(), &body,
|
||||
query_of({{"memoryId", spec.memory_id}})));
|
||||
if (op.response.is_object() && !str_at(op.response, "name").empty())
|
||||
return Memory::from_json(op.response);
|
||||
if (!spec.memory_id.empty()) return get(spec.memory_id);
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "create returned no memory", endpoint_.name);
|
||||
}
|
||||
|
||||
Memory MemoryBank::get(const std::string& memory_id) const {
|
||||
return Memory::from_json(send(Call::kMemoryGet, engine_name_.memory(memory_id)));
|
||||
}
|
||||
|
||||
std::vector<Memory> MemoryBank::list(const MemoryListOptions& options) const {
|
||||
Check::expression("filter", options.filter);
|
||||
Check::expression("order_by", options.order_by);
|
||||
Check::page_size(options.page_size, MemoryCaps::kPageSize);
|
||||
Json items = list_items(
|
||||
Call::kMemoryList, engine_name_.str(),
|
||||
query_of({{"filter", options.filter},
|
||||
{"orderBy", options.order_by},
|
||||
{"pageSize", options.page_size > 0 ? std::to_string(options.page_size) : ""}}),
|
||||
"memories", MemoryCaps::kMaxListPages, MemoryCaps::kMaxListItems);
|
||||
std::vector<Memory> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(Memory::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
Memory MemoryBank::patch(const MemoryPatch& patch) const {
|
||||
patch.validate();
|
||||
Json body = patch.to_json();
|
||||
Operation op = settle(send(Call::kMemoryPatch, engine_name_.memory(patch.memory_id),
|
||||
&body, query_of({{"updateMask", patch.update_mask}})));
|
||||
if (op.response.is_object() && !str_at(op.response, "name").empty())
|
||||
return Memory::from_json(op.response);
|
||||
return get(patch.memory_id);
|
||||
}
|
||||
|
||||
void MemoryBank::delete_memory(const std::string& memory_id) const {
|
||||
Json reply = send(Call::kMemoryDelete, engine_name_.memory(memory_id));
|
||||
if (reply.is_object() && !reply.as_object().empty()) settle(reply);
|
||||
}
|
||||
|
||||
PurgeResult MemoryBank::purge(const PurgeRequest& request) const {
|
||||
Check::expression("filter", request.filter);
|
||||
if (request.filter.empty()) throw GoogleError::invalid("purge needs a filter");
|
||||
check_filter_groups(request.filter_groups);
|
||||
if (request.force && request.confirm != engine_name_.engine())
|
||||
throw GoogleError::invalid("forced purge is irreversible: set confirm to the engine id");
|
||||
Json body = Json::object();
|
||||
body.set("filter", Json(request.filter));
|
||||
if (!request.filter_groups.empty())
|
||||
body.set("filterGroups", filter_groups_json(request.filter_groups));
|
||||
body.set("force", Json(request.force));
|
||||
Operation op = settle(send(Call::kMemoryPurge, engine_name_.str(), &body));
|
||||
PurgeResult out;
|
||||
out.forced = request.force;
|
||||
if (op.response.is_object())
|
||||
out.purge_count = static_cast<long>(number_at(op.response, "purgeCount", 0));
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<MemoryRevision> MemoryBank::list_revisions(const std::string& memory_id,
|
||||
const std::string& filter) const {
|
||||
Check::expression("filter", filter);
|
||||
Json items = list_items(Call::kRevisionList, engine_name_.memory(memory_id),
|
||||
query_of({{"filter", filter}}), "memoryRevisions",
|
||||
MemoryCaps::kMaxListPages, MemoryCaps::kMaxListItems);
|
||||
std::vector<MemoryRevision> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(MemoryRevision::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
MemoryRevision MemoryBank::get_revision(const std::string& memory_id,
|
||||
const std::string& revision_id) const {
|
||||
check_revision_id(revision_id);
|
||||
return MemoryRevision::from_json(
|
||||
send(Call::kRevisionGet, engine_name_.memory(memory_id) + "/revisions/" + revision_id));
|
||||
}
|
||||
|
||||
void MemoryBank::rollback(const std::string& memory_id,
|
||||
const std::string& revision_id) const {
|
||||
check_revision_id(revision_id);
|
||||
Json body = Json::object();
|
||||
body.set("targetRevisionId", Json(revision_id));
|
||||
settle(send(Call::kMemoryRollback, engine_name_.memory(memory_id), &body));
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,46 @@
|
||||
#pragma once
|
||||
|
||||
// Agent Memory Bank under one Agent Engine resource. Every call is a row in
|
||||
// calls.h; every Operation is waited with wait_operation.
|
||||
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/sessions/engine_service.h"
|
||||
#include "google/sessions/memory_types.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
class MemoryBank : public EngineService {
|
||||
public:
|
||||
MemoryBank(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper = nullptr);
|
||||
|
||||
// Extract memories from a session, a conversation or given facts.
|
||||
std::vector<GeneratedMemory> generate(const MemoryGenerateRequest& request) const;
|
||||
// Streaming ingestion; generation fires by rule or force_flush.
|
||||
void ingest_events(const IngestRequest& request) const;
|
||||
RetrieveResult retrieve(const RetrieveRequest& request) const;
|
||||
// Low-latency recall of structured profiles (Memory Profiles), keyed by
|
||||
// schema id.
|
||||
std::map<std::string, Profile> retrieve_profiles(const StringMap& scope) const;
|
||||
|
||||
Memory create(const MemorySpec& spec) const;
|
||||
Memory get(const std::string& memory_id) const;
|
||||
std::vector<Memory> list(const MemoryListOptions& options = {}) const;
|
||||
Memory patch(const MemoryPatch& patch) const;
|
||||
// Destructive: explicit method, waits for the delete operation.
|
||||
void delete_memory(const std::string& memory_id) const;
|
||||
// Destructive and irreversible when request.force is true (see PurgeRequest).
|
||||
PurgeResult purge(const PurgeRequest& request) const;
|
||||
|
||||
std::vector<MemoryRevision> list_revisions(const std::string& memory_id,
|
||||
const std::string& filter = "") const;
|
||||
MemoryRevision get_revision(const std::string& memory_id,
|
||||
const std::string& revision_id) const;
|
||||
void rollback(const std::string& memory_id, const std::string& revision_id) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,485 @@
|
||||
#include "google/sessions/memory_types.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sessions/caps.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& message) {
|
||||
throw GoogleError::invalid(message);
|
||||
}
|
||||
|
||||
void check_topics(const std::vector<Topic>& topics) {
|
||||
if (topics.size() > MemoryCaps::kMaxTopics)
|
||||
bad("topics over " + std::to_string(MemoryCaps::kMaxTopics));
|
||||
for (const Topic& t : topics) t.validate();
|
||||
}
|
||||
|
||||
Json topics_json(const std::vector<Topic>& topics) {
|
||||
Json arr = Json::array();
|
||||
for (const Topic& t : topics) arr.push(t.to_json());
|
||||
return arr;
|
||||
}
|
||||
|
||||
void check_metadata(const Metadata& metadata) {
|
||||
if (metadata.size() > MemoryCaps::kMaxMetadataEntries)
|
||||
bad("metadata over " + std::to_string(MemoryCaps::kMaxMetadataEntries) + " entries");
|
||||
for (const auto& [k, v] : metadata) {
|
||||
Check::text("metadata key", k, MemoryCaps::kMaxMetadataChars, true);
|
||||
v.validate();
|
||||
}
|
||||
}
|
||||
|
||||
void check_mask(const std::string& mask) {
|
||||
Check::text("update_mask", mask, MemoryCaps::kMaxUpdateMaskChars, true);
|
||||
for (char c : mask) {
|
||||
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
|
||||
(c >= '0' && c <= '9') || c == ',' || c == '.' || c == '_';
|
||||
if (!ok) bad("update_mask allows letters, digits, comma, dot, underscore");
|
||||
}
|
||||
}
|
||||
|
||||
void check_strategy(const std::string& s) {
|
||||
if (s.empty() || s == "OVERWRITE" || s == "MERGE" || s == "REQUIRE_EXACT_MATCH") return;
|
||||
bad("metadata_merge_strategy must be OVERWRITE, MERGE or REQUIRE_EXACT_MATCH");
|
||||
}
|
||||
|
||||
void put_structured(Json& body, const Json& structured) {
|
||||
if (structured.is_object()) body.set("structuredContent", structured);
|
||||
}
|
||||
|
||||
void check_structured(const Json& structured) {
|
||||
if (structured.is_null()) return;
|
||||
if (!structured.is_object() || !structured.has("schemaId") || !structured.has("data"))
|
||||
bad("structured_content needs schemaId and data");
|
||||
if (structured.dump().size() > MemoryCaps::kMaxFactChars * 4)
|
||||
bad("structured_content too large");
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Topic Topic::managed(std::string name) { return Topic{Kind::kManaged, std::move(name)}; }
|
||||
Topic Topic::custom(std::string label) { return Topic{Kind::kCustom, std::move(label)}; }
|
||||
|
||||
void Topic::validate() const {
|
||||
Check::text("topic", value, MemoryCaps::kMaxTopicLabelChars, true);
|
||||
if (kind == Kind::kManaged)
|
||||
for (char c : value)
|
||||
if (!((c >= 'A' && c <= 'Z') || c == '_'))
|
||||
bad("managed topic must be an enum name like USER_PREFERENCES");
|
||||
}
|
||||
|
||||
Json Topic::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set(kind == Kind::kManaged ? "managedMemoryTopic" : "customMemoryTopicLabel",
|
||||
Json(value));
|
||||
return out;
|
||||
}
|
||||
|
||||
Topic Topic::from_json(const Json& doc) {
|
||||
std::string managed_name = str_at(doc, "managedMemoryTopic");
|
||||
if (!managed_name.empty()) return managed(managed_name);
|
||||
return custom(str_at(doc, "customMemoryTopicLabel"));
|
||||
}
|
||||
|
||||
MetadataValue MetadataValue::of_string(std::string v) {
|
||||
MetadataValue m;
|
||||
m.kind = Kind::kString;
|
||||
m.text = std::move(v);
|
||||
return m;
|
||||
}
|
||||
MetadataValue MetadataValue::of_double(double v) {
|
||||
MetadataValue m;
|
||||
m.kind = Kind::kDouble;
|
||||
m.number = v;
|
||||
return m;
|
||||
}
|
||||
MetadataValue MetadataValue::of_bool(bool v) {
|
||||
MetadataValue m;
|
||||
m.kind = Kind::kBool;
|
||||
m.flag = v;
|
||||
return m;
|
||||
}
|
||||
MetadataValue MetadataValue::of_timestamp(std::string v) {
|
||||
MetadataValue m;
|
||||
m.kind = Kind::kTimestamp;
|
||||
m.text = std::move(v);
|
||||
return m;
|
||||
}
|
||||
|
||||
void MetadataValue::validate() const {
|
||||
if (kind == Kind::kString)
|
||||
Check::text("metadata value", text, MemoryCaps::kMaxMetadataChars);
|
||||
if (kind == Kind::kTimestamp) Check::timestamp("metadata timestamp", text);
|
||||
}
|
||||
|
||||
Json MetadataValue::to_json() const {
|
||||
Json out = Json::object();
|
||||
switch (kind) {
|
||||
case Kind::kString: out.set("stringValue", Json(text)); break;
|
||||
case Kind::kDouble: out.set("doubleValue", Json(number)); break;
|
||||
case Kind::kBool: out.set("boolValue", Json(flag)); break;
|
||||
case Kind::kTimestamp: out.set("timestampValue", Json(text)); break;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
Json metadata_json(const Metadata& metadata) {
|
||||
Json out = Json::object();
|
||||
for (const auto& [k, v] : metadata) out.set(k, v.to_json());
|
||||
return out;
|
||||
}
|
||||
|
||||
void check_filter_groups(const FilterGroups& groups) {
|
||||
if (groups.size() > MemoryCaps::kMaxFilterGroups)
|
||||
bad("filter_groups over " + std::to_string(MemoryCaps::kMaxFilterGroups));
|
||||
for (const auto& group : groups) {
|
||||
if (group.empty()) bad("empty filter group");
|
||||
if (group.size() > MemoryCaps::kMaxFiltersPerGroup)
|
||||
bad("filters per group over " + std::to_string(MemoryCaps::kMaxFiltersPerGroup));
|
||||
for (const MetadataFilter& f : group) {
|
||||
Check::text("filter key", f.key, MemoryCaps::kMaxMetadataChars, true);
|
||||
f.value.validate();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Json filter_groups_json(const FilterGroups& groups) {
|
||||
Json out = Json::array();
|
||||
for (const auto& group : groups) {
|
||||
Json filters = Json::array();
|
||||
for (const MetadataFilter& f : group) {
|
||||
Json item = Json::object();
|
||||
item.set("key", Json(f.key));
|
||||
item.set("op", Json(f.op == MetadataFilter::Op::kEqual ? "EQUAL"
|
||||
: f.op == MetadataFilter::Op::kGreaterThan ? "GREATER_THAN"
|
||||
: "LESS_THAN"));
|
||||
if (f.negate) item.set("negate", Json(true));
|
||||
item.set("value", f.value.to_json());
|
||||
filters.push(item);
|
||||
}
|
||||
Json g = Json::object();
|
||||
g.set("filters", filters);
|
||||
out.push(g);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string Memory::id() const {
|
||||
std::size_t slash = name.rfind('/');
|
||||
return slash == std::string::npos ? name : name.substr(slash + 1);
|
||||
}
|
||||
|
||||
Memory Memory::from_json(const Json& doc) {
|
||||
Memory m;
|
||||
m.name = str_at(doc, "name");
|
||||
m.display_name = str_at(doc, "displayName");
|
||||
m.description = str_at(doc, "description");
|
||||
m.fact = str_at(doc, "fact");
|
||||
m.context = str_at(doc, "context");
|
||||
m.scope = string_map_from(doc.find("scope"));
|
||||
if (const Json* t = doc.find("topics"); t && t->is_array())
|
||||
for (const Json& item : t->as_array()) m.topics.push_back(Topic::from_json(item));
|
||||
m.memory_type = str_at(doc, "memoryType");
|
||||
if (const Json* v = doc.find("structuredContent")) m.structured_content = *v;
|
||||
if (const Json* v = doc.find("metadata")) m.metadata = *v;
|
||||
m.create_time = str_at(doc, "createTime");
|
||||
m.update_time = str_at(doc, "updateTime");
|
||||
m.expire_time = str_at(doc, "expireTime");
|
||||
return m;
|
||||
}
|
||||
|
||||
Json Memory::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("name", Json(name));
|
||||
out.set("displayName", Json(display_name));
|
||||
out.set("description", Json(description));
|
||||
out.set("fact", Json(fact));
|
||||
out.set("context", Json(context));
|
||||
out.set("scope", sessions::to_json(scope));
|
||||
out.set("topics", topics_json(topics));
|
||||
out.set("memoryType", Json(memory_type));
|
||||
if (structured_content.is_object()) out.set("structuredContent", structured_content);
|
||||
if (metadata.is_object()) out.set("metadata", metadata);
|
||||
out.set("createTime", Json(create_time));
|
||||
out.set("updateTime", Json(update_time));
|
||||
out.set("expireTime", Json(expire_time));
|
||||
return out;
|
||||
}
|
||||
|
||||
void RevisionOptions::validate() const {
|
||||
Check::labels(labels);
|
||||
if (!ttl.empty()) Check::ttl(ttl, 1);
|
||||
}
|
||||
|
||||
void RevisionOptions::apply(Json& body) const {
|
||||
if (!labels.empty()) body.set("revisionLabels", sessions::to_json(labels));
|
||||
if (!ttl.empty()) body.set("revisionTtl", Json(ttl));
|
||||
if (disable) body.set("disableMemoryRevisions", Json(true));
|
||||
}
|
||||
|
||||
void MemorySpec::validate() const {
|
||||
if (!memory_id.empty()) Check::memory_id(memory_id);
|
||||
Check::text("display_name", display_name, SessionsCaps::kMaxDisplayChars);
|
||||
Check::text("description", description, MemoryCaps::kMaxFactChars);
|
||||
Check::text("context", context, MemoryCaps::kMaxFactChars);
|
||||
Check::text("fact", fact, MemoryCaps::kMaxFactChars, structured_content.is_null());
|
||||
Check::scope(scope);
|
||||
check_topics(topics);
|
||||
check_metadata(metadata);
|
||||
check_structured(structured_content);
|
||||
if (!ttl.empty()) Check::ttl(ttl, 1);
|
||||
if (!memory_type.empty() && memory_type != "NATURAL_LANGUAGE_COLLECTION" &&
|
||||
memory_type != "STRUCTURED_PROFILE")
|
||||
bad("memory_type must be NATURAL_LANGUAGE_COLLECTION or STRUCTURED_PROFILE");
|
||||
revision.validate();
|
||||
}
|
||||
|
||||
Json MemorySpec::to_json() const {
|
||||
Json out = Json::object();
|
||||
if (!fact.empty()) out.set("fact", Json(fact));
|
||||
out.set("scope", sessions::to_json(scope));
|
||||
if (!display_name.empty()) out.set("displayName", Json(display_name));
|
||||
if (!description.empty()) out.set("description", Json(description));
|
||||
if (!context.empty()) out.set("context", Json(context));
|
||||
if (!topics.empty()) out.set("topics", topics_json(topics));
|
||||
if (!ttl.empty()) out.set("ttl", Json(ttl));
|
||||
if (!metadata.empty()) out.set("metadata", metadata_json(metadata));
|
||||
if (!memory_type.empty()) out.set("memoryType", Json(memory_type));
|
||||
put_structured(out, structured_content);
|
||||
revision.apply(out);
|
||||
return out;
|
||||
}
|
||||
|
||||
void MemoryPatch::validate() const {
|
||||
Check::memory_id(memory_id);
|
||||
check_mask(update_mask);
|
||||
Check::text("fact", fact, MemoryCaps::kMaxFactChars);
|
||||
Check::text("display_name", display_name, SessionsCaps::kMaxDisplayChars);
|
||||
Check::text("description", description, MemoryCaps::kMaxFactChars);
|
||||
Check::text("context", context, MemoryCaps::kMaxFactChars);
|
||||
check_topics(topics);
|
||||
check_metadata(metadata);
|
||||
check_structured(structured_content);
|
||||
if (!ttl.empty()) Check::ttl(ttl, 1);
|
||||
revision.validate();
|
||||
}
|
||||
|
||||
Json MemoryPatch::to_json() const {
|
||||
Json out = Json::object();
|
||||
if (!fact.empty()) out.set("fact", Json(fact));
|
||||
if (!display_name.empty()) out.set("displayName", Json(display_name));
|
||||
if (!description.empty()) out.set("description", Json(description));
|
||||
if (!context.empty()) out.set("context", Json(context));
|
||||
if (!topics.empty()) out.set("topics", topics_json(topics));
|
||||
if (!ttl.empty()) out.set("ttl", Json(ttl));
|
||||
if (!metadata.empty()) out.set("metadata", metadata_json(metadata));
|
||||
put_structured(out, structured_content);
|
||||
revision.apply(out);
|
||||
return out;
|
||||
}
|
||||
|
||||
void MemoryGenerateRequest::validate() const {
|
||||
Check::scope(scope);
|
||||
check_topics(allowed_topics);
|
||||
check_metadata(metadata);
|
||||
check_strategy(metadata_merge_strategy);
|
||||
Check::text("time_zone", time_zone, 64);
|
||||
revision.validate();
|
||||
switch (source) {
|
||||
case Source::kSession:
|
||||
Check::session_id(session_id);
|
||||
if (!start_time.empty()) Check::timestamp("start_time", start_time);
|
||||
if (!end_time.empty()) Check::timestamp("end_time", end_time);
|
||||
break;
|
||||
case Source::kContents: {
|
||||
if (contents.empty()) bad("contents is empty");
|
||||
if (contents.size() > MemoryCaps::kMaxSourceEvents)
|
||||
bad("contents over " + std::to_string(MemoryCaps::kMaxSourceEvents) + " events");
|
||||
std::size_t bytes = 0;
|
||||
for (const Content& c : contents) bytes += c.to_json().dump().size();
|
||||
if (bytes > MemoryCaps::kMaxSourceBytes)
|
||||
bad("contents over " + std::to_string(MemoryCaps::kMaxSourceBytes) + " bytes");
|
||||
break;
|
||||
}
|
||||
case Source::kDirectMemories:
|
||||
if (direct_memories.empty()) bad("direct_memories is empty");
|
||||
if (direct_memories.size() > MemoryCaps::kMaxDirectMemories)
|
||||
bad("direct_memories over " + std::to_string(MemoryCaps::kMaxDirectMemories));
|
||||
for (const DirectMemory& d : direct_memories) {
|
||||
Check::text("fact", d.fact, MemoryCaps::kMaxFactChars, true);
|
||||
check_topics(d.topics);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
Json MemoryGenerateRequest::to_json(const std::string& engine_name) const {
|
||||
Json out = Json::object();
|
||||
out.set("scope", sessions::to_json(scope));
|
||||
if (disable_consolidation) out.set("disableConsolidation", Json(true));
|
||||
if (!allowed_topics.empty()) out.set("allowedTopics", topics_json(allowed_topics));
|
||||
if (!metadata.empty()) out.set("metadata", metadata_json(metadata));
|
||||
if (!metadata_merge_strategy.empty())
|
||||
out.set("metadataMergeStrategy", Json(metadata_merge_strategy));
|
||||
if (!time_zone.empty()) out.set("timeZone", Json(time_zone));
|
||||
revision.apply(out);
|
||||
switch (source) {
|
||||
case Source::kSession: {
|
||||
Json src = Json::object();
|
||||
src.set("session", Json(engine_name + "/sessions/" + session_id));
|
||||
if (!start_time.empty()) src.set("startTime", Json(start_time));
|
||||
if (!end_time.empty()) src.set("endTime", Json(end_time));
|
||||
out.set("vertexSessionSource", src);
|
||||
break;
|
||||
}
|
||||
case Source::kContents: {
|
||||
Json events = Json::array();
|
||||
for (const Content& c : contents) {
|
||||
Json e = Json::object();
|
||||
e.set("content", c.to_json());
|
||||
events.push(e);
|
||||
}
|
||||
Json src = Json::object();
|
||||
src.set("events", events);
|
||||
out.set("directContentsSource", src);
|
||||
break;
|
||||
}
|
||||
case Source::kDirectMemories: {
|
||||
Json list = Json::array();
|
||||
for (const DirectMemory& d : direct_memories) {
|
||||
Json item = Json::object();
|
||||
item.set("fact", Json(d.fact));
|
||||
if (!d.topics.empty()) item.set("topics", topics_json(d.topics));
|
||||
list.push(item);
|
||||
}
|
||||
Json src = Json::object();
|
||||
src.set("directMemories", list);
|
||||
out.set("directMemoriesSource", src);
|
||||
break;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
void IngestRequest::validate() const {
|
||||
Check::scope(scope);
|
||||
if (events.empty()) bad("events is empty");
|
||||
if (events.size() > MemoryCaps::kMaxSourceEvents)
|
||||
bad("events over " + std::to_string(MemoryCaps::kMaxSourceEvents));
|
||||
std::size_t bytes = 0;
|
||||
for (const Event& e : events) {
|
||||
Check::text("event_id", e.event_id, SessionsCaps::kMaxUserIdChars);
|
||||
if (!e.event_time.empty()) Check::timestamp("event_time", e.event_time);
|
||||
bytes += e.content.to_json().dump().size();
|
||||
}
|
||||
if (bytes > MemoryCaps::kMaxSourceBytes)
|
||||
bad("events over " + std::to_string(MemoryCaps::kMaxSourceBytes) + " bytes");
|
||||
Check::text("stream_id", stream_id, SessionsCaps::kMaxUserIdChars);
|
||||
if (trigger_event_count < 0 || trigger_event_count > 1000 ||
|
||||
trigger_overlap_event_count < 0 || trigger_overlap_event_count > 1000)
|
||||
bad("trigger counts outside 0..1000");
|
||||
if (!trigger_fixed_interval.empty()) Check::ttl(trigger_fixed_interval, 60);
|
||||
if (!trigger_idle_duration.empty()) Check::ttl(trigger_idle_duration, 1);
|
||||
check_metadata(metadata);
|
||||
check_strategy(metadata_merge_strategy);
|
||||
revision.validate();
|
||||
}
|
||||
|
||||
Json IngestRequest::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("scope", sessions::to_json(scope));
|
||||
Json list = Json::array();
|
||||
for (const Event& e : events) {
|
||||
Json item = Json::object();
|
||||
if (!e.event_id.empty()) item.set("eventId", Json(e.event_id));
|
||||
if (!e.event_time.empty()) item.set("eventTime", Json(e.event_time));
|
||||
item.set("content", e.content.to_json());
|
||||
list.push(item);
|
||||
}
|
||||
Json src = Json::object();
|
||||
src.set("events", list);
|
||||
out.set("directContentsSource", src);
|
||||
if (!stream_id.empty()) out.set("streamId", Json(stream_id));
|
||||
if (force_flush) out.set("forceFlush", Json(true));
|
||||
Json rule = Json::object();
|
||||
if (trigger_event_count > 0) rule.set("eventCount", Json(trigger_event_count));
|
||||
if (trigger_overlap_event_count > 0)
|
||||
rule.set("overlapEventCount", Json(trigger_overlap_event_count));
|
||||
if (!trigger_fixed_interval.empty()) rule.set("fixedInterval", Json(trigger_fixed_interval));
|
||||
if (!trigger_idle_duration.empty()) rule.set("idleDuration", Json(trigger_idle_duration));
|
||||
if (!rule.as_object().empty()) {
|
||||
Json trigger = Json::object();
|
||||
trigger.set("generationRule", rule);
|
||||
out.set("generationTriggerConfig", trigger);
|
||||
}
|
||||
if (!metadata.empty()) out.set("metadata", metadata_json(metadata));
|
||||
if (!metadata_merge_strategy.empty())
|
||||
out.set("metadataMergeStrategy", Json(metadata_merge_strategy));
|
||||
revision.apply(out);
|
||||
return out;
|
||||
}
|
||||
|
||||
void RetrieveRequest::validate() const {
|
||||
Check::scope(scope);
|
||||
Check::text("search_query", search_query, MemoryCaps::kMaxQueryChars);
|
||||
Check::expression("filter", filter);
|
||||
check_filter_groups(filter_groups);
|
||||
if (memory_types.size() > MemoryCaps::kMaxMemoryTypes) bad("memory_types over 2");
|
||||
for (const std::string& t : memory_types)
|
||||
if (t != "NATURAL_LANGUAGE_COLLECTION" && t != "STRUCTURED_PROFILE")
|
||||
bad("memory_types allows NATURAL_LANGUAGE_COLLECTION, STRUCTURED_PROFILE");
|
||||
if (top_k < 0 || top_k > MemoryCaps::kMaxTopK)
|
||||
bad("top_k outside 0.." + std::to_string(MemoryCaps::kMaxTopK));
|
||||
Check::page_size(page_size, MemoryCaps::kPageSize);
|
||||
if (!search_query.empty() && (page_size > 0 || !page_token.empty()))
|
||||
bad("similarity search takes top_k, not page_size or page_token");
|
||||
if (search_query.empty() && top_k > 0) bad("top_k needs a search_query");
|
||||
Check::text("page_token", page_token, 2048);
|
||||
}
|
||||
|
||||
Json RetrieveRequest::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("scope", sessions::to_json(scope));
|
||||
if (!filter.empty()) out.set("filter", Json(filter));
|
||||
if (!filter_groups.empty()) out.set("filterGroups", filter_groups_json(filter_groups));
|
||||
if (!memory_types.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const std::string& t : memory_types) arr.push(Json(t));
|
||||
out.set("memoryTypes", arr);
|
||||
}
|
||||
if (!search_query.empty()) {
|
||||
Json params = Json::object();
|
||||
params.set("searchQuery", Json(search_query));
|
||||
params.set("topK", Json(top_k > 0 ? top_k : MemoryCaps::kDefaultTopK));
|
||||
out.set("similaritySearchParams", params);
|
||||
} else if (page_size > 0 || !page_token.empty()) {
|
||||
Json params = Json::object();
|
||||
if (page_size > 0) params.set("pageSize", Json(page_size));
|
||||
if (!page_token.empty()) params.set("pageToken", Json(page_token));
|
||||
out.set("simpleRetrievalParams", params);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string MemoryRevision::id() const {
|
||||
std::size_t slash = name.rfind('/');
|
||||
return slash == std::string::npos ? name : name.substr(slash + 1);
|
||||
}
|
||||
|
||||
MemoryRevision MemoryRevision::from_json(const Json& doc) {
|
||||
MemoryRevision r;
|
||||
r.name = str_at(doc, "name");
|
||||
r.fact = str_at(doc, "fact");
|
||||
r.context = str_at(doc, "context");
|
||||
r.labels = string_map_from(doc.find("labels"));
|
||||
r.create_time = str_at(doc, "createTime");
|
||||
r.expire_time = str_at(doc, "expireTime");
|
||||
return r;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,229 @@
|
||||
#pragma once
|
||||
|
||||
// Typed Agent Memory Bank resources and requests. Wire names are the REST
|
||||
// camelCase names of aiplatform v1beta1 (see calls.h for the call table).
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "google/gemini_types.h"
|
||||
#include "google/sessions/names.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
// Memory topic: a Google managed topic (enum name such as
|
||||
// "USER_PREFERENCES") or a custom label.
|
||||
struct Topic {
|
||||
enum class Kind { kManaged, kCustom };
|
||||
Kind kind = Kind::kManaged;
|
||||
std::string value;
|
||||
|
||||
static Topic managed(std::string name);
|
||||
static Topic custom(std::string label);
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
static Topic from_json(const clients::body::Json& doc);
|
||||
};
|
||||
|
||||
// MemoryMetadataValue: exactly one of string, double, bool, timestamp.
|
||||
struct MetadataValue {
|
||||
enum class Kind { kString, kDouble, kBool, kTimestamp };
|
||||
Kind kind = Kind::kString;
|
||||
std::string text; // string or RFC 3339 timestamp
|
||||
double number = 0;
|
||||
bool flag = false;
|
||||
|
||||
static MetadataValue of_string(std::string v);
|
||||
static MetadataValue of_double(double v);
|
||||
static MetadataValue of_bool(bool v);
|
||||
static MetadataValue of_timestamp(std::string v);
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
using Metadata = std::map<std::string, MetadataValue>;
|
||||
|
||||
// One AND-group of metadata filters; groups are OR-ed (Google semantics).
|
||||
struct MetadataFilter {
|
||||
enum class Op { kEqual, kGreaterThan, kLessThan };
|
||||
std::string key;
|
||||
Op op = Op::kEqual;
|
||||
bool negate = false;
|
||||
MetadataValue value;
|
||||
};
|
||||
using FilterGroups = std::vector<std::vector<MetadataFilter>>;
|
||||
|
||||
struct Memory {
|
||||
std::string name;
|
||||
std::string display_name, description, fact, context;
|
||||
StringMap scope;
|
||||
std::vector<Topic> topics;
|
||||
std::string memory_type; // NATURAL_LANGUAGE_COLLECTION, STRUCTURED_PROFILE
|
||||
clients::body::Json structured_content; // {schemaId, data}, raw
|
||||
clients::body::Json metadata; // map of MemoryMetadataValue, raw
|
||||
std::string create_time, update_time, expire_time;
|
||||
|
||||
std::string id() const;
|
||||
static Memory from_json(const clients::body::Json& doc);
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct RevisionOptions { // Shared by create, patch, generate, ingest.
|
||||
StringMap labels;
|
||||
std::string ttl; // "86400s"
|
||||
bool disable = false;
|
||||
void validate() const;
|
||||
void apply(clients::body::Json& body) const; // adds the three wire fields
|
||||
};
|
||||
|
||||
struct MemorySpec { // Input to create (a memory written directly).
|
||||
std::string memory_id; // Optional custom id.
|
||||
std::string display_name, description, context;
|
||||
std::string fact; // Required unless structured_content is set.
|
||||
StringMap scope; // Required, immutable.
|
||||
std::vector<Topic> topics;
|
||||
std::string ttl; // "3600s", empty: never expires.
|
||||
Metadata metadata;
|
||||
std::string memory_type;
|
||||
clients::body::Json structured_content;
|
||||
RevisionOptions revision;
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
// PATCH of one memory. scope and metadata keys cannot be changed by Google
|
||||
// (scope is immutable). update_mask is a comma list of the fields to write.
|
||||
struct MemoryPatch {
|
||||
std::string memory_id;
|
||||
std::string fact, display_name, description, context, ttl;
|
||||
std::vector<Topic> topics;
|
||||
Metadata metadata;
|
||||
clients::body::Json structured_content;
|
||||
RevisionOptions revision;
|
||||
std::string update_mask; // Required, e.g. "fact,displayName".
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
// What generate reads. Exactly one source is used.
|
||||
struct MemoryGenerateRequest {
|
||||
enum class Source { kSession, kContents, kDirectMemories };
|
||||
Source source = Source::kContents;
|
||||
|
||||
std::string session_id; // kSession: an Agent Sessions id under this engine
|
||||
std::string start_time; // kSession, optional RFC 3339
|
||||
std::string end_time; // kSession, optional RFC 3339
|
||||
std::vector<Content> contents; // kContents: the conversation
|
||||
struct DirectMemory {
|
||||
std::string fact;
|
||||
std::vector<Topic> topics;
|
||||
};
|
||||
std::vector<DirectMemory> direct_memories; // kDirectMemories, max 5
|
||||
|
||||
StringMap scope; // Required.
|
||||
bool disable_consolidation = false;
|
||||
std::vector<Topic> allowed_topics;
|
||||
Metadata metadata;
|
||||
std::string metadata_merge_strategy; // OVERWRITE, MERGE, REQUIRE_EXACT_MATCH
|
||||
std::string time_zone; // IANA name
|
||||
RevisionOptions revision;
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json(const std::string& engine_name) const;
|
||||
};
|
||||
|
||||
struct GeneratedMemory {
|
||||
Memory memory;
|
||||
std::string action; // As reported by Google (see SPEC Unverified).
|
||||
};
|
||||
|
||||
// Streaming ingestion into a Memory Bank stream; generation is triggered by
|
||||
// the rule (event count, interval, idle) or by force_flush.
|
||||
struct IngestRequest {
|
||||
struct Event {
|
||||
std::string event_id, event_time; // optional; time is RFC 3339
|
||||
Content content;
|
||||
};
|
||||
std::vector<Event> events; // Required.
|
||||
StringMap scope; // Required.
|
||||
std::string stream_id;
|
||||
bool force_flush = false;
|
||||
int trigger_event_count = 0; // 0: unset
|
||||
int trigger_overlap_event_count = 0;
|
||||
std::string trigger_fixed_interval; // "3600s"
|
||||
std::string trigger_idle_duration; // "600s"
|
||||
Metadata metadata;
|
||||
std::string metadata_merge_strategy;
|
||||
RevisionOptions revision;
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct RetrieveRequest {
|
||||
StringMap scope; // Required.
|
||||
std::string search_query; // Set: similarity search. Empty: simple retrieval.
|
||||
int top_k = 0; // Similarity only; 0: kDefaultTopK.
|
||||
int page_size = 0; // Simple only; 0: Google default.
|
||||
std::string page_token; // Simple only.
|
||||
std::string filter; // fact, createTime, updateTime, topics
|
||||
FilterGroups filter_groups;
|
||||
std::vector<std::string> memory_types;
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct RetrievedMemory {
|
||||
Memory memory;
|
||||
double distance = 0.0; // Similarity only; smaller is closer.
|
||||
bool has_distance = false;
|
||||
};
|
||||
|
||||
struct RetrieveResult {
|
||||
std::vector<RetrievedMemory> memories;
|
||||
std::string next_page_token; // Simple retrieval only.
|
||||
};
|
||||
|
||||
struct MemoryListOptions {
|
||||
std::string filter; // scope (JSON equality) and topics
|
||||
std::string order_by; // createTime / updateTime, "desc"
|
||||
int page_size = 0;
|
||||
};
|
||||
|
||||
struct MemoryRevision {
|
||||
std::string name, fact, context;
|
||||
StringMap labels;
|
||||
std::string create_time, expire_time;
|
||||
static MemoryRevision from_json(const clients::body::Json& doc);
|
||||
std::string id() const;
|
||||
};
|
||||
|
||||
struct Profile {
|
||||
std::string schema_id;
|
||||
clients::body::Json fields; // Free-form profile object.
|
||||
};
|
||||
|
||||
// Purge is irreversible. force=false (the default) is a server-side dry run.
|
||||
// force=true additionally needs `confirm` to equal the engine id.
|
||||
struct PurgeRequest {
|
||||
std::string filter; // Required, AIP-160.
|
||||
FilterGroups filter_groups;
|
||||
bool force = false;
|
||||
std::string confirm;
|
||||
};
|
||||
|
||||
struct PurgeResult {
|
||||
bool forced = false;
|
||||
long purge_count = 0; // Memories matched (dry run) or deleted (force).
|
||||
};
|
||||
|
||||
// Shared helpers (also used by tests).
|
||||
clients::body::Json filter_groups_json(const FilterGroups& groups);
|
||||
void check_filter_groups(const FilterGroups& groups);
|
||||
clients::body::Json metadata_json(const Metadata& metadata);
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,195 @@
|
||||
#include "google/sessions/names.h"
|
||||
|
||||
#include <cctype>
|
||||
#include <cstdlib>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "google/errors.h"
|
||||
#include "google/resource.h"
|
||||
#include "google/sessions/caps.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
bool id_char(char c) {
|
||||
return (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-';
|
||||
}
|
||||
|
||||
bool project_ok(const std::string& p) {
|
||||
if (p.empty() || p.size() > 128) return false;
|
||||
for (char c : p)
|
||||
if (!id_char(c) && c != '.' && c != ':') return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
[[noreturn]] void bad(const std::string& message) {
|
||||
throw GoogleError::invalid(message);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
EngineName::EngineName(std::string full) : full_(std::move(full)) {
|
||||
if (full_.empty() || full_.size() > SessionsCaps::kMaxNameChars)
|
||||
bad("engine name missing or too long");
|
||||
std::vector<std::string> parts;
|
||||
std::string cur;
|
||||
for (char c : full_) {
|
||||
if (c == '/') {
|
||||
parts.push_back(cur);
|
||||
cur.clear();
|
||||
} else {
|
||||
cur.push_back(c);
|
||||
}
|
||||
}
|
||||
parts.push_back(cur);
|
||||
if (parts.size() != 6 || parts[0] != "projects" || parts[2] != "locations" ||
|
||||
parts[4] != "reasoningEngines")
|
||||
bad("engine must look like projects/{p}/locations/{l}/reasoningEngines/{e}");
|
||||
project_ = parts[1];
|
||||
location_ = parts[3];
|
||||
engine_ = parts[5];
|
||||
if (!project_ok(project_)) bad("engine name: bad project");
|
||||
if (location_.empty() || location_.size() > 64) bad("engine name: bad location");
|
||||
for (char c : location_)
|
||||
if (!id_char(c)) bad("engine name: bad location");
|
||||
if (!valid_segment(engine_)) bad("engine name: bad engine id");
|
||||
}
|
||||
|
||||
std::string EngineName::session(const std::string& session_id) const {
|
||||
Check::session_id(session_id);
|
||||
return full_ + "/sessions/" + session_id;
|
||||
}
|
||||
|
||||
std::string EngineName::memory(const std::string& memory_id) const {
|
||||
Check::memory_id(memory_id);
|
||||
return full_ + "/memories/" + memory_id;
|
||||
}
|
||||
|
||||
void Check::session_id(const std::string& id) {
|
||||
if (id.empty() || id.size() > SessionsCaps::kMaxIdChars)
|
||||
bad("session id must be 1.." + std::to_string(SessionsCaps::kMaxIdChars) +
|
||||
" characters");
|
||||
for (char c : id)
|
||||
if (!id_char(c)) bad("session id allows only a-z 0-9 -");
|
||||
}
|
||||
|
||||
void Check::memory_id(const std::string& id) {
|
||||
if (id.empty() || id.size() > SessionsCaps::kMaxIdChars)
|
||||
bad("memory id must be 1.." + std::to_string(SessionsCaps::kMaxIdChars) +
|
||||
" characters");
|
||||
for (char c : id)
|
||||
if (!id_char(c)) bad("memory id allows only a-z 0-9 -");
|
||||
if (!(id.front() >= 'a' && id.front() <= 'z')) bad("memory id must start with a letter");
|
||||
if (id.back() == '-') bad("memory id must end with a letter or number");
|
||||
}
|
||||
|
||||
void Check::text(const std::string& field, const std::string& value,
|
||||
std::size_t max, bool required) {
|
||||
if (required && value.empty()) bad(field + " is required");
|
||||
if (value.size() > max)
|
||||
bad(field + " over " + std::to_string(max) + " characters");
|
||||
}
|
||||
|
||||
void Check::expression(const std::string& field, const std::string& value) {
|
||||
text(field, value, SessionsCaps::kMaxFilterChars);
|
||||
for (char c : value)
|
||||
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7f)
|
||||
bad(field + " has control characters");
|
||||
}
|
||||
|
||||
void Check::scope(const StringMap& scope, bool required) {
|
||||
if (scope.empty()) {
|
||||
if (required) bad("scope is required");
|
||||
return;
|
||||
}
|
||||
if (scope.size() > MemoryCaps::kMaxScopeEntries)
|
||||
bad("scope over " + std::to_string(MemoryCaps::kMaxScopeEntries) + " entries");
|
||||
for (const auto& [k, v] : scope) {
|
||||
if (k.empty() || k.size() > MemoryCaps::kMaxScopeChars ||
|
||||
v.size() > MemoryCaps::kMaxScopeChars)
|
||||
bad("scope key empty or over " + std::to_string(MemoryCaps::kMaxScopeChars) +
|
||||
" characters");
|
||||
if (v.empty()) bad("scope value is empty");
|
||||
if (k.find('*') != std::string::npos || v.find('*') != std::string::npos)
|
||||
bad("scope must not contain the wildcard *");
|
||||
}
|
||||
}
|
||||
|
||||
void Check::labels(const StringMap& labels) {
|
||||
if (labels.size() > SessionsCaps::kMaxLabels)
|
||||
bad("labels over " + std::to_string(SessionsCaps::kMaxLabels));
|
||||
for (const auto& [k, v] : labels)
|
||||
if (k.empty() || k.size() > SessionsCaps::kMaxLabelChars ||
|
||||
v.size() > SessionsCaps::kMaxLabelChars)
|
||||
bad("label key empty or over " + std::to_string(SessionsCaps::kMaxLabelChars) +
|
||||
" characters");
|
||||
}
|
||||
|
||||
void Check::timestamp(const std::string& field, const std::string& value) {
|
||||
if (value.size() < 20 || value.size() > SessionsCaps::kMaxTimestampChars)
|
||||
bad(field + " must be an RFC 3339 timestamp");
|
||||
for (char c : value) {
|
||||
bool ok = std::isdigit(static_cast<unsigned char>(c)) || c == '-' || c == ':' ||
|
||||
c == 'T' || c == 'Z' || c == '.' || c == '+';
|
||||
if (!ok) bad(field + " must be an RFC 3339 timestamp");
|
||||
}
|
||||
if (value[4] != '-' || value[7] != '-' || value[10] != 'T')
|
||||
bad(field + " must be an RFC 3339 timestamp");
|
||||
}
|
||||
|
||||
void Check::ttl(const std::string& value, int min_seconds) {
|
||||
if (value.size() < 2 || value.size() > 24 || value.back() != 's')
|
||||
bad("ttl must look like 86400s");
|
||||
std::string num = value.substr(0, value.size() - 1);
|
||||
int dots = 0;
|
||||
for (char c : num) {
|
||||
if (c == '.') ++dots;
|
||||
else if (!std::isdigit(static_cast<unsigned char>(c))) bad("ttl must look like 86400s");
|
||||
}
|
||||
if (dots > 1 || num.front() == '.' || num.back() == '.')
|
||||
bad("ttl must look like 86400s");
|
||||
double seconds = std::strtod(num.c_str(), nullptr);
|
||||
if (seconds < min_seconds)
|
||||
bad("ttl under the minimum of " + std::to_string(min_seconds) + "s");
|
||||
if (seconds > static_cast<double>(SessionsCaps::kMaxTtlSeconds))
|
||||
bad("ttl over the cap");
|
||||
}
|
||||
|
||||
void Check::page_size(int value, int ceiling) {
|
||||
if (value < 0 || value > ceiling)
|
||||
bad("page_size outside 0.." + std::to_string(ceiling));
|
||||
}
|
||||
|
||||
Json to_json(const StringMap& map) {
|
||||
Json out = Json::object();
|
||||
for (const auto& [k, v] : map) out.set(k, Json(v));
|
||||
return out;
|
||||
}
|
||||
|
||||
StringMap string_map_from(const Json* doc) {
|
||||
StringMap out;
|
||||
if (doc && doc->is_object())
|
||||
for (const auto& [k, v] : doc->as_object())
|
||||
if (v.is_string()) out[k] = v.as_string();
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string str_at(const Json& doc, const std::string& key) {
|
||||
const Json* v = doc.is_object() ? doc.find(key) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
std::string query_of(const std::map<std::string, std::string>& params) {
|
||||
std::string out;
|
||||
for (const auto& [k, v] : params) {
|
||||
if (v.empty()) continue;
|
||||
out += (out.empty() ? "?" : "&") + k + "=" + clients::Url::encode_component(v);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,62 @@
|
||||
#pragma once
|
||||
|
||||
// Validated resource names and shared input checks for the sessions module.
|
||||
// Everything here runs before any network traffic and throws
|
||||
// GoogleError::invalid (HTTP 400 INVALID_ARGUMENT).
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
// "projects/{project}/locations/{location}/reasoningEngines/{engine}".
|
||||
// Accepted as a string so this module does not depend on the runtime module.
|
||||
class EngineName {
|
||||
public:
|
||||
explicit EngineName(std::string full); // throws when malformed
|
||||
|
||||
const std::string& str() const { return full_; }
|
||||
const std::string& project() const { return project_; }
|
||||
const std::string& location() const { return location_; }
|
||||
const std::string& engine() const { return engine_; }
|
||||
|
||||
std::string session(const std::string& session_id) const;
|
||||
std::string memory(const std::string& memory_id) const;
|
||||
|
||||
private:
|
||||
std::string full_, project_, location_, engine_;
|
||||
};
|
||||
|
||||
using StringMap = std::map<std::string, std::string>;
|
||||
|
||||
// Input validators.
|
||||
class Check {
|
||||
public:
|
||||
// Session id: 1..63 of [a-z0-9-].
|
||||
static void session_id(const std::string& id);
|
||||
// Memory id: 1..63 of [a-z0-9-], starts with a letter, ends alphanumeric.
|
||||
static void memory_id(const std::string& id);
|
||||
static void text(const std::string& field, const std::string& value,
|
||||
std::size_t max, bool required = false);
|
||||
// Filter / orderBy expression: bounded, no control characters.
|
||||
static void expression(const std::string& field, const std::string& value);
|
||||
// Scope: 1..N entries, bounded, no '*' (Google forbids the wildcard).
|
||||
static void scope(const StringMap& scope, bool required = true);
|
||||
static void labels(const StringMap& labels);
|
||||
// RFC 3339 shape check (not a calendar check).
|
||||
static void timestamp(const std::string& field, const std::string& value);
|
||||
// Duration "86400s" or "3.5s"; within [min_seconds, kMaxTtlSeconds].
|
||||
static void ttl(const std::string& value, int min_seconds);
|
||||
static void page_size(int value, int ceiling);
|
||||
};
|
||||
|
||||
// Wire helpers.
|
||||
clients::body::Json to_json(const StringMap& map);
|
||||
StringMap string_map_from(const clients::body::Json* doc);
|
||||
std::string str_at(const clients::body::Json& doc, const std::string& key);
|
||||
// Query suffix "?k=v&k2=v2" with encoded values; empty values are skipped.
|
||||
std::string query_of(const std::map<std::string, std::string>& params);
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,148 @@
|
||||
#include "google/sessions/session_types.h"
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/sessions/caps.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
void SessionSpec::validate() const {
|
||||
if (!session_id.empty()) Check::session_id(session_id);
|
||||
Check::text("user_id", user_id, SessionsCaps::kMaxUserIdChars, true);
|
||||
Check::text("display_name", display_name, SessionsCaps::kMaxDisplayChars);
|
||||
Check::labels(labels);
|
||||
if (!session_state.is_null()) {
|
||||
if (!session_state.is_object())
|
||||
throw GoogleError::invalid("session_state must be an object");
|
||||
if (session_state.dump().size() > SessionsCaps::kMaxStateBytes)
|
||||
throw GoogleError::invalid("session_state over " +
|
||||
std::to_string(SessionsCaps::kMaxStateBytes) +
|
||||
" bytes");
|
||||
}
|
||||
if (!ttl.empty()) Check::ttl(ttl, SessionsCaps::kMinTtlSeconds);
|
||||
}
|
||||
|
||||
Json SessionSpec::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("userId", Json(user_id));
|
||||
if (!display_name.empty()) out.set("displayName", Json(display_name));
|
||||
if (!labels.empty()) out.set("labels", sessions::to_json(labels));
|
||||
if (session_state.is_object()) out.set("sessionState", session_state);
|
||||
if (!ttl.empty()) out.set("ttl", Json(ttl));
|
||||
return out;
|
||||
}
|
||||
|
||||
void SessionPatch::validate() const {
|
||||
Check::session_id(session_id);
|
||||
Check::text("update_mask", update_mask, 256, true);
|
||||
for (char c : update_mask) {
|
||||
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == ',' || c == '.';
|
||||
if (!ok) throw GoogleError::invalid("update_mask allows letters, comma, dot");
|
||||
}
|
||||
Check::text("display_name", display_name, SessionsCaps::kMaxDisplayChars);
|
||||
Check::labels(labels);
|
||||
if (!session_state.is_null() && !session_state.is_object())
|
||||
throw GoogleError::invalid("session_state must be an object");
|
||||
if (session_state.dump().size() > SessionsCaps::kMaxStateBytes)
|
||||
throw GoogleError::invalid("session_state over " +
|
||||
std::to_string(SessionsCaps::kMaxStateBytes) + " bytes");
|
||||
if (!ttl.empty()) Check::ttl(ttl, SessionsCaps::kMinTtlSeconds);
|
||||
}
|
||||
|
||||
Json SessionPatch::to_json() const {
|
||||
Json out = Json::object();
|
||||
if (!display_name.empty()) out.set("displayName", Json(display_name));
|
||||
if (!labels.empty()) out.set("labels", sessions::to_json(labels));
|
||||
if (session_state.is_object()) out.set("sessionState", session_state);
|
||||
if (!ttl.empty()) out.set("ttl", Json(ttl));
|
||||
return out;
|
||||
}
|
||||
|
||||
void CompactRequest::validate() const {
|
||||
if (!summarization && !event_editing)
|
||||
throw GoogleError::invalid("compact needs summarization or event_editing");
|
||||
}
|
||||
|
||||
Json CompactRequest::to_json() const {
|
||||
Json mode = Json::object();
|
||||
mode.set("mode", Json("AUTO"));
|
||||
Json compaction = Json::object();
|
||||
if (summarization) compaction.set("summarization", mode);
|
||||
if (event_editing) compaction.set("eventEditing", mode);
|
||||
Json out = Json::object();
|
||||
out.set("compaction", compaction);
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string Session::id() const {
|
||||
std::size_t slash = name.rfind('/');
|
||||
return slash == std::string::npos ? name : name.substr(slash + 1);
|
||||
}
|
||||
|
||||
Session Session::from_json(const Json& doc) {
|
||||
Session s;
|
||||
s.name = str_at(doc, "name");
|
||||
s.display_name = str_at(doc, "displayName");
|
||||
s.user_id = str_at(doc, "userId");
|
||||
s.labels = string_map_from(doc.find("labels"));
|
||||
if (const Json* v = doc.find("sessionState")) s.session_state = *v;
|
||||
s.create_time = str_at(doc, "createTime");
|
||||
s.update_time = str_at(doc, "updateTime");
|
||||
s.expire_time = str_at(doc, "expireTime");
|
||||
return s;
|
||||
}
|
||||
|
||||
Json Session::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("name", Json(name));
|
||||
out.set("displayName", Json(display_name));
|
||||
out.set("userId", Json(user_id));
|
||||
out.set("labels", sessions::to_json(labels));
|
||||
out.set("sessionState", session_state.is_object() ? session_state : Json::object());
|
||||
out.set("createTime", Json(create_time));
|
||||
out.set("updateTime", Json(update_time));
|
||||
out.set("expireTime", Json(expire_time));
|
||||
return out;
|
||||
}
|
||||
|
||||
void SessionEvent::validate() const {
|
||||
Check::text("author", author, SessionsCaps::kMaxUserIdChars, true);
|
||||
Check::text("invocation_id", invocation_id, SessionsCaps::kMaxUserIdChars, true);
|
||||
Check::timestamp("timestamp", timestamp);
|
||||
if (to_json().dump().size() > SessionsCaps::kMaxEventBytes)
|
||||
throw GoogleError::invalid("event over " +
|
||||
std::to_string(SessionsCaps::kMaxEventBytes) + " bytes");
|
||||
}
|
||||
|
||||
Json SessionEvent::to_json() const {
|
||||
Json out = Json::object();
|
||||
out.set("author", Json(author));
|
||||
out.set("invocationId", Json(invocation_id));
|
||||
out.set("timestamp", Json(timestamp));
|
||||
if (content) out.set("content", content->to_json());
|
||||
if (actions.is_object()) out.set("actions", actions);
|
||||
if (!error_code.empty()) out.set("errorCode", Json(error_code));
|
||||
if (!error_message.empty()) out.set("errorMessage", Json(error_message));
|
||||
if (event_metadata.is_object()) out.set("eventMetadata", event_metadata);
|
||||
if (raw_event.is_object()) out.set("rawEvent", raw_event);
|
||||
return out;
|
||||
}
|
||||
|
||||
SessionEvent SessionEvent::from_json(const Json& doc) {
|
||||
SessionEvent e;
|
||||
e.name = str_at(doc, "name");
|
||||
e.author = str_at(doc, "author");
|
||||
e.invocation_id = str_at(doc, "invocationId");
|
||||
e.timestamp = str_at(doc, "timestamp");
|
||||
if (const Json* c = doc.find("content"); c && c->is_object())
|
||||
e.content = Content::from_json(*c);
|
||||
if (const Json* v = doc.find("actions")) e.actions = *v;
|
||||
e.error_code = str_at(doc, "errorCode");
|
||||
e.error_message = str_at(doc, "errorMessage");
|
||||
if (const Json* v = doc.find("eventMetadata")) e.event_metadata = *v;
|
||||
if (const Json* v = doc.find("rawEvent")) e.raw_event = *v;
|
||||
return e;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,92 @@
|
||||
#pragma once
|
||||
|
||||
// Typed Agent Sessions resources (Session, SessionEvent). Wire names are the
|
||||
// REST camelCase names. Free-form parts stay raw Json.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "google/gemini_types.h"
|
||||
#include "google/sessions/names.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
struct SessionSpec { // Input to create.
|
||||
std::string session_id; // Optional custom id, [a-z0-9-], 1..63.
|
||||
std::string user_id; // Required, immutable.
|
||||
std::string display_name;
|
||||
StringMap labels;
|
||||
clients::body::Json session_state; // Object or null.
|
||||
std::string ttl; // "86400s" minimum, empty: none.
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
// PATCH of one session; userId is immutable. update_mask is a comma list
|
||||
// of fields (displayName, labels, sessionState, ttl).
|
||||
struct SessionPatch {
|
||||
std::string session_id;
|
||||
std::string display_name;
|
||||
StringMap labels;
|
||||
clients::body::Json session_state;
|
||||
std::string ttl;
|
||||
std::string update_mask; // Required.
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
// Storage-side rewrite of a session's event history. At least one category.
|
||||
struct CompactRequest {
|
||||
bool summarization = false; // LLM summary, mode AUTO
|
||||
bool event_editing = false; // deterministic edits, mode AUTO
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct Session {
|
||||
std::string name;
|
||||
std::string display_name;
|
||||
std::string user_id;
|
||||
StringMap labels;
|
||||
clients::body::Json session_state;
|
||||
std::string create_time, update_time, expire_time;
|
||||
|
||||
std::string id() const; // Last component of name.
|
||||
static Session from_json(const clients::body::Json& doc);
|
||||
clients::body::Json to_json() const;
|
||||
};
|
||||
|
||||
struct SessionEvent {
|
||||
std::string name; // Output only.
|
||||
std::string author; // Required: "user", an agent name.
|
||||
std::string invocation_id; // Required.
|
||||
std::string timestamp; // Required, RFC 3339.
|
||||
std::optional<Content> content;
|
||||
clients::body::Json actions; // EventActions, raw.
|
||||
std::string error_code, error_message;
|
||||
clients::body::Json event_metadata; // EventMetadata, raw.
|
||||
clients::body::Json raw_event; // Struct, raw.
|
||||
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
static SessionEvent from_json(const clients::body::Json& doc);
|
||||
};
|
||||
|
||||
struct SessionListOptions {
|
||||
std::string filter; // displayName="x", userId="y", labels.k="v"
|
||||
std::string order_by; // "createTime desc"
|
||||
int page_size = 0; // 0: Google default (100)
|
||||
};
|
||||
|
||||
struct EventListOptions {
|
||||
std::string filter; // timestamp>="2025-01-31T11:30:00Z"
|
||||
std::string order_by; // "timestamp desc"
|
||||
int page_size = 0;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,89 @@
|
||||
#include "google/sessions/sessions.h"
|
||||
|
||||
#include "google/sessions/caps.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
std::string list_query(const std::string& filter, const std::string& order_by,
|
||||
int page_size) {
|
||||
Check::expression("filter", filter);
|
||||
Check::expression("order_by", order_by);
|
||||
Check::page_size(page_size, SessionsCaps::kPageSize);
|
||||
return query_of({{"filter", filter},
|
||||
{"orderBy", order_by},
|
||||
{"pageSize", page_size > 0 ? std::to_string(page_size) : ""}});
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Sessions::Sessions(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper)
|
||||
: EngineService(std::move(client), std::move(engine), std::move(sleeper)) {}
|
||||
|
||||
Session Sessions::create(const SessionSpec& spec) const {
|
||||
spec.validate();
|
||||
Json body = spec.to_json();
|
||||
Operation op = settle(send(Call::kSessionCreate, engine_name_.str(), &body,
|
||||
query_of({{"sessionId", spec.session_id}})));
|
||||
if (op.response.is_object() && !str_at(op.response, "name").empty())
|
||||
return Session::from_json(op.response);
|
||||
if (!spec.session_id.empty()) return get(spec.session_id);
|
||||
throw GoogleError(502, "EMPTY_RESPONSE", "create returned no session", endpoint_.name);
|
||||
}
|
||||
|
||||
Session Sessions::get(const std::string& session_id) const {
|
||||
return Session::from_json(send(Call::kSessionGet, engine_name_.session(session_id)));
|
||||
}
|
||||
|
||||
std::vector<Session> Sessions::list(const SessionListOptions& options) const {
|
||||
Json items = list_items(Call::kSessionList, engine_name_.str(),
|
||||
list_query(options.filter, options.order_by, options.page_size),
|
||||
"sessions", SessionsCaps::kMaxListPages,
|
||||
SessionsCaps::kMaxListItems);
|
||||
std::vector<Session> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(Session::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
Session Sessions::patch(const SessionPatch& patch) const {
|
||||
patch.validate();
|
||||
Json body = patch.to_json();
|
||||
return Session::from_json(send(Call::kSessionPatch,
|
||||
engine_name_.session(patch.session_id), &body,
|
||||
query_of({{"updateMask", patch.update_mask}})));
|
||||
}
|
||||
|
||||
void Sessions::delete_session(const std::string& session_id) const {
|
||||
Json reply = send(Call::kSessionDelete, engine_name_.session(session_id));
|
||||
if (reply.is_object() && !reply.as_object().empty()) settle(reply);
|
||||
}
|
||||
|
||||
void Sessions::compact(const std::string& session_id, const CompactRequest& request) const {
|
||||
request.validate();
|
||||
Json body = request.to_json();
|
||||
settle(send(Call::kSessionCompact, engine_name_.session(session_id), &body));
|
||||
}
|
||||
|
||||
void Sessions::append_event(const std::string& session_id,
|
||||
const SessionEvent& event) const {
|
||||
event.validate();
|
||||
Json body = event.to_json();
|
||||
send(Call::kSessionAppendEvent, engine_name_.session(session_id), &body);
|
||||
}
|
||||
|
||||
std::vector<SessionEvent> Sessions::list_events(const std::string& session_id,
|
||||
const EventListOptions& options) const {
|
||||
Json items = list_items(Call::kEventList, engine_name_.session(session_id),
|
||||
list_query(options.filter, options.order_by, options.page_size),
|
||||
"sessionEvents", SessionsCaps::kMaxListPages,
|
||||
SessionsCaps::kMaxListItems);
|
||||
std::vector<SessionEvent> out;
|
||||
for (const Json& item : items.as_array()) out.push_back(SessionEvent::from_json(item));
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,35 @@
|
||||
#pragma once
|
||||
|
||||
// Agent Sessions: sessions and their events under one Agent Engine
|
||||
// (reasoningEngines) resource. Every call is a row in calls.h.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "google/sessions/engine_service.h"
|
||||
#include "google/sessions/session_types.h"
|
||||
|
||||
namespace molo::google::sessions {
|
||||
|
||||
class Sessions : public EngineService {
|
||||
public:
|
||||
// `sleeper` replaces the real sleep between operation polls (tests).
|
||||
Sessions(std::shared_ptr<GoogleClient> client, EngineName engine,
|
||||
Sleeper sleeper = nullptr);
|
||||
|
||||
Session create(const SessionSpec& spec) const; // waits for the operation
|
||||
Session get(const std::string& session_id) const;
|
||||
std::vector<Session> list(const SessionListOptions& options = {}) const;
|
||||
Session patch(const SessionPatch& patch) const;
|
||||
// Destructive: explicit method, waits for the delete operation.
|
||||
void delete_session(const std::string& session_id) const;
|
||||
// Rewrites the event history (summary and/or edits); waits.
|
||||
void compact(const std::string& session_id, const CompactRequest& request) const;
|
||||
|
||||
void append_event(const std::string& session_id, const SessionEvent& event) const;
|
||||
std::vector<SessionEvent> list_events(const std::string& session_id,
|
||||
const EventListOptions& options = {}) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::sessions
|
||||
@@ -0,0 +1,207 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Checks src/google/sessions/calls.h against Google's Discovery document.
|
||||
|
||||
Fetches https://aiplatform.googleapis.com/$discovery/rest?version=<v> (no
|
||||
credential needed) and fails when a declared call, verb, path, request schema
|
||||
field, query parameter, enum value or response key is absent.
|
||||
|
||||
python3 src/google/sessions/tools/check_discovery.py [--file discovery.json]
|
||||
|
||||
Exit 0: everything declared exists. Exit 1: mismatches, listed. Exit 2: the
|
||||
discovery document could not be read. Python 3 stdlib only. Sends no secret.
|
||||
"""
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
CALLS_H = os.path.join(HERE, "..", "calls.h")
|
||||
SCHEMA_PREFIX = "GoogleCloudAiplatform"
|
||||
|
||||
# (schema suffix, property, expected enum values that the C++ code sends)
|
||||
ENUMS = [
|
||||
("MemoryTopicId", "managedMemoryTopic",
|
||||
["USER_PERSONAL_INFO", "USER_PREFERENCES", "KEY_CONVERSATION_DETAILS",
|
||||
"EXPLICIT_INSTRUCTIONS"]),
|
||||
("GenerateMemoriesRequest", "metadataMergeStrategy",
|
||||
["OVERWRITE", "MERGE", "REQUIRE_EXACT_MATCH"]),
|
||||
("IngestEventsRequest", "metadataMergeStrategy",
|
||||
["OVERWRITE", "MERGE", "REQUIRE_EXACT_MATCH"]),
|
||||
("MemoryFilter", "op", ["EQUAL", "GREATER_THAN", "LESS_THAN"]),
|
||||
("Memory", "memoryType", ["NATURAL_LANGUAGE_COLLECTION", "STRUCTURED_PROFILE"]),
|
||||
("CompactionConfigLlmSummarizationConfig", "mode", ["AUTO"]),
|
||||
("CompactionConfigEventEditingConfig", "mode", ["AUTO"]),
|
||||
]
|
||||
|
||||
# (schema suffix, property) read by the response parsers.
|
||||
RESPONSE_KEYS = [
|
||||
("ListSessionsResponse", "sessions"), ("ListSessionsResponse", "nextPageToken"),
|
||||
("ListEventsResponse", "sessionEvents"), ("ListEventsResponse", "nextPageToken"),
|
||||
("ListMemoriesResponse", "memories"), ("ListMemoriesResponse", "nextPageToken"),
|
||||
("ListMemoryRevisionsResponse", "memoryRevisions"),
|
||||
("RetrieveMemoriesResponse", "retrievedMemories"),
|
||||
("RetrieveMemoriesResponse", "nextPageToken"),
|
||||
("RetrieveMemoriesResponseRetrievedMemory", "memory"),
|
||||
("RetrieveMemoriesResponseRetrievedMemory", "distance"),
|
||||
("RetrieveProfilesResponse", "profiles"),
|
||||
("MemoryProfile", "profile"), ("MemoryProfile", "schemaId"),
|
||||
("MemoryRevision", "fact"), ("MemoryRevision", "context"),
|
||||
("MemoryRevision", "labels"), ("MemoryRevision", "createTime"),
|
||||
("MemoryRevision", "expireTime"),
|
||||
("Session", "name"), ("Session", "createTime"), ("Session", "updateTime"),
|
||||
("Session", "expireTime"), ("Memory", "name"), ("Memory", "expireTime"),
|
||||
]
|
||||
|
||||
# Nested request shapes built by the C++ code: (schema suffix, fields).
|
||||
NESTED = [
|
||||
("GenerateMemoriesRequestVertexSessionSource", "session startTime endTime"),
|
||||
("GenerateMemoriesRequestDirectContentsSource", "events"),
|
||||
("GenerateMemoriesRequestDirectContentsSourceEvent", "content"),
|
||||
("GenerateMemoriesRequestDirectMemoriesSource", "directMemories"),
|
||||
("GenerateMemoriesRequestDirectMemoriesSourceDirectMemory", "fact topics"),
|
||||
("RetrieveMemoriesRequestSimilaritySearchParams", "searchQuery topK"),
|
||||
("RetrieveMemoriesRequestSimpleRetrievalParams", "pageSize pageToken"),
|
||||
("IngestionDirectContentsSource", "events"),
|
||||
("IngestionDirectContentsSourceEvent", "eventId eventTime content"),
|
||||
("MemoryGenerationTriggerConfig", "generationRule"),
|
||||
("MemoryGenerationTriggerConfigGenerationTriggerRule",
|
||||
"eventCount overlapEventCount fixedInterval idleDuration"),
|
||||
("MemoryConjunctionFilter", "filters"),
|
||||
("MemoryFilter", "key op negate value"),
|
||||
("MemoryMetadataValue", "stringValue doubleValue boolValue timestampValue"),
|
||||
("MemoryTopicId", "customMemoryTopicLabel managedMemoryTopic"),
|
||||
("CompactionConfig", "summarization eventEditing"),
|
||||
("CompactionConfigLlmSummarizationConfig", "mode"),
|
||||
("CompactionConfigEventEditingConfig", "mode"),
|
||||
]
|
||||
|
||||
|
||||
def load(path):
|
||||
if path:
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
version = parse_header()["version"]
|
||||
url = "https://aiplatform.googleapis.com/$discovery/rest?version=" + version
|
||||
with urllib.request.urlopen(url, timeout=120) as r:
|
||||
return json.load(r)
|
||||
|
||||
|
||||
def parse_header():
|
||||
text = open(CALLS_H, "r", encoding="utf-8").read()
|
||||
version = re.search(r'kDiscoveryVersion\s*=\s*"([^"]+)"', text).group(1)
|
||||
prefix = re.search(r'kDiscoveryPrefix\s*=\s*"([^"]+)"', text).group(1)
|
||||
rows = []
|
||||
row_re = re.compile(
|
||||
r'^\{Call::(k\w+),\s*"(\w+)",\s*"([^"]*)",\s*"([^"]*)",\s*"([^"]*)",'
|
||||
r'\s*"([^"]*)",\s*"([^"]*)"\},\s*$', re.M)
|
||||
for m in row_re.finditer(text):
|
||||
rows.append({"call": m.group(1), "verb": m.group(2), "path": m.group(3),
|
||||
"method": m.group(4), "request": m.group(5),
|
||||
"fields": m.group(6).split(), "query": m.group(7).split()})
|
||||
enum_names = re.search(r"enum class Call \{(.*?)kCount", text, re.S).group(1)
|
||||
declared = re.findall(r"\bk\w+", enum_names)
|
||||
return {"version": version, "prefix": prefix, "rows": rows, "declared": declared}
|
||||
|
||||
|
||||
def find_method(doc, method_id):
|
||||
found = []
|
||||
|
||||
def walk(res):
|
||||
for m in res.get("methods", {}).values():
|
||||
if m["id"] == method_id:
|
||||
found.append(m)
|
||||
for r in res.get("resources", {}).values():
|
||||
walk(r)
|
||||
|
||||
walk(doc)
|
||||
return found[0] if found else None
|
||||
|
||||
|
||||
def normalize(path, version):
|
||||
path = re.sub(r"\{\+(\w+)\}", r"{\1}", path)
|
||||
return "/" + path[len(version) + 1:] if path.startswith(version + "/") else path
|
||||
|
||||
|
||||
def main():
|
||||
path = None
|
||||
if "--file" in sys.argv:
|
||||
path = sys.argv[sys.argv.index("--file") + 1]
|
||||
header = parse_header()
|
||||
try:
|
||||
doc = load(path)
|
||||
except Exception as exc: # network or parse failure is not a mismatch
|
||||
print("cannot read discovery document: %s" % exc)
|
||||
return 2
|
||||
version, prefix = header["version"], header["prefix"]
|
||||
schemas = doc["schemas"]
|
||||
schema_prefix = SCHEMA_PREFIX + ("V1beta1" if version == "v1beta1" else "V1")
|
||||
errors = []
|
||||
rows = header["rows"]
|
||||
if sorted(r["call"] for r in rows) != sorted(header["declared"]):
|
||||
errors.append("calls.h: enum Call and table rows differ")
|
||||
|
||||
for row in rows:
|
||||
mid = prefix + row["method"]
|
||||
m = find_method(doc, mid)
|
||||
if m is None:
|
||||
errors.append("%s: method %s absent" % (row["call"], mid))
|
||||
continue
|
||||
if m["httpMethod"] != row["verb"]:
|
||||
errors.append("%s: verb %s != %s" % (row["call"], row["verb"], m["httpMethod"]))
|
||||
got = normalize(m["path"], version)
|
||||
if got != row["path"]:
|
||||
errors.append("%s: path %s != %s" % (row["call"], row["path"], got))
|
||||
for q in row["query"]:
|
||||
p = m.get("parameters", {}).get(q)
|
||||
if not p or p.get("location") != "query":
|
||||
errors.append("%s: query parameter %s absent" % (row["call"], q))
|
||||
if row["request"]:
|
||||
want = schema_prefix + row["request"]
|
||||
ref = m.get("request", {}).get("$ref")
|
||||
if ref != want:
|
||||
errors.append("%s: request %s != %s" % (row["call"], want, ref))
|
||||
props = schemas.get(want, {}).get("properties", {})
|
||||
for field in row["fields"]:
|
||||
if field not in props:
|
||||
errors.append("%s: field %s absent from %s" % (row["call"], field, want))
|
||||
elif m.get("request"):
|
||||
errors.append("%s: discovery has a request body %s the table omits" %
|
||||
(row["call"], m["request"]["$ref"]))
|
||||
|
||||
for suffix, prop, values in ENUMS:
|
||||
s = schemas.get(schema_prefix + suffix, {}).get("properties", {}).get(prop)
|
||||
if s is None:
|
||||
errors.append("enum %s.%s absent" % (suffix, prop))
|
||||
continue
|
||||
allowed = s.get("enum")
|
||||
if allowed is None: # Free string in discovery: documented, not enforced.
|
||||
continue
|
||||
for v in values:
|
||||
if v not in allowed:
|
||||
errors.append("enum value %s absent from %s.%s" % (v, suffix, prop))
|
||||
for suffix, prop in RESPONSE_KEYS:
|
||||
if prop not in schemas.get(schema_prefix + suffix, {}).get("properties", {}):
|
||||
errors.append("response key %s.%s absent" % (suffix, prop))
|
||||
for suffix, fields in NESTED:
|
||||
props = schemas.get(schema_prefix + suffix, {}).get("properties", {})
|
||||
for field in fields.split():
|
||||
if field not in props:
|
||||
errors.append("nested %s.%s absent" % (suffix, field))
|
||||
|
||||
stamp = datetime.date.today().isoformat()
|
||||
if errors:
|
||||
print("FAIL (%s, discovery %s): %d mismatches" % (stamp, version, len(errors)))
|
||||
for e in errors:
|
||||
print(" - " + e)
|
||||
return 1
|
||||
print("OK %s: %d calls, %d enums, %d response keys, %d nested schemas match "
|
||||
"aiplatform %s discovery" %
|
||||
(stamp, len(rows), len(ENUMS), len(RESPONSE_KEYS), len(NESTED), version))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user