tai/research/agent-memory-recency-tracking/conclusions/last-used-tracking-proposal.md

Conclusion: Last-Used / Use-Count Tracking Proposal for the tai Vault

Metadata

Field	Value
Institute	Molodetz Institute — Research & Retrieval Division
Subject	agent-memory-recency-tracking
Document type	Conclusion
Tier	N/A
Date decided	2026-10-09
Decided by	retoor, advised by tai (agent, Claude Sonnet 5)
Status	Active (proposal recorded, not yet implemented)
Confidence	Medium

Question / problem statement

retoor asked for a way to track when a vault record (and, by extension, a chunk) was last used, or was recently part of a search result, with an eye toward a future LRU-style prioritization or faster-access mechanism. The explicit instruction was to research this thoroughly first — specifically looking at how agent-memory systems already do this, and at systems/database techniques for doing it cheaply — and to produce a concrete proposal, with building anything explicitly forbidden for this task.

Conclusion

The recommended design (not yet built) is:

• Add a last_used INTEGER column (Unix epoch seconds) to both the records and chunks tables. This is a deliberate exception to the vault's otherwise-universal ISO-8601 TEXT timestamp convention, justified specifically because this column is compared, sorted, and range-scanned on a hot path in a way the vault's other timestamp columns are not.
• Add a use_count INTEGER column to chunks. records already has an existing reads counter that serves the same purpose, so no duplicate column is needed there.
• "Used" is triggered by two kinds of event: (a) an explicit record_read/get_record call, and (b) a record or chunk appearing in search results — a chunk hit should also propagate an update to its parent record's last_used, since the parent is "the record" in retoor's own framing of the request.
• Writes must be throttled/conditional, never unconditional on every hit: UPDATE ... SET last_used = ? WHERE last_used IS NULL OR last_used < ? with a coalescing window (a starting estimate of roughly one minute), modeled directly on Linux's relatime mount option and the real, documented GitLab last_downloaded_at production incident.
• No eviction policy and no search-ranking integration is being built in this phase. This is tracking-substrate only — recording the raw signal so a future, separate decision can use it, matching how both the Generative Agents paper and Mem0's product keep recency as an independent signal rather than baking a final blended formula in at write time.

Supporting sources

• arxiv-2304-03442-generative-agents.md (Tier 1) — the foundational precedent for keeping recency, importance, and relevance as three independent, un-collapsed signals combined only at retrieval time.
• arxiv-2310-08560-memgpt.md, letta-agent-memory-blog.md (Tier 1) — establish that even the most prominent "OS-for-LLMs" memory architecture uses FIFO eviction, not LRU, and keeps ranking (recency-aware) separate from eviction/compaction — informing the decision to defer eviction entirely rather than assume a policy.
• mem0-introducing-memory-decay.md, mem0-memory-eviction-forgetting.md, arxiv-2504-19413-mem0-paper.md (Tier 1) — the closest existing commercial precedent: recency as a soft search-time re-ranking multiplier, not a deletion trigger; and an explicit distinction between decay (ranking), TTL (lifecycle), and deletion APIs (user-visible forgetting) that this proposal deliberately mirrors by staying purely in the "raw signal" category.
• arpit-2q-cache-algorithm.md, arxiv-2511-21958-clock2q-plus.md, arxiv-2512-22995-buffer-management-survey.md (Tier 1) — background precedent for low-write-overhead recency tracking (2Q, CLOCK), recorded for a possible future eviction-policy phase, not used directly in the current tracking-only proposal.
• redhat-relatime-drive-access.md, gitlab-last-downloaded-at-write-amplification.md (Tier 1) — the direct justification for the throttled/conditional-UPDATE write pattern: relatime is the exact same tradeoff already running on effectively every Linux filesystem read, and the GitLab incident is concrete, quantified, real-world proof that the naive unconditional-update approach causes genuine production problems.
• sqlite-forum-timestamp-integer-vs-text.md, sqlite-org-bm25-sign-flip-commit.md (Tier 1) — justify the Unix-epoch-INTEGER column choice and flag the BM25 ranking-direction gotcha for whoever eventually builds the ranking-integration phase.
• glama-code-index-mcp-bm25-hybrid.md (Tier 1) — a concrete, reusable BM25-normalization-and-fusion reference for that same future phase.

Alternatives considered and rejected

• Unconditional UPDATE on every access. Rejected outright, per the GitLab incident's own documented production consequences (row-version and index-entry multiplication on a hot read path) — this is the one alternative with direct, quantified evidence against it, not just theoretical concern.
• In-memory batched writes flushed periodically (the lazytime approach). Not rejected — deferred as a second-stage optimization, to be built only if the simple throttled-UPDATE approach proves insufficient under real measured load. Recorded explicitly as "not rejected, just not needed yet," not as a worse option.
• A single blended LRFU-style score computed and stored at write time, instead of separate last_used/use_count columns. Rejected for this phase because collapsing recency and frequency into one opaque number at write time would destroy the ability to use them as independent inputs to whatever ranking or eviction decision gets made later — a decision that has not been made yet. This mirrors the Generative Agents / Mem0 pattern of keeping signals separate until the moment they're actually used.
• ISO-8601 TEXT for last_used, matching every other timestamp column in the vault. Considered, since consistency has its own value and was retoor's own suggested fallback ("if it doesn't matter just use the same date format as the other columns"). Rejected specifically for this one column because it is the one column in the whole schema expected to be compared/range-scanned on a genuine hot path, where the SQLite-forum-documented integer advantages (no text parsing, smaller index entries, no timezone ambiguity) are the kind of thing that could plausibly matter — this is the one deliberate inconsistency in the schema, and it is being flagged as such rather than introduced silently.

Risks / open questions

• The ~one-minute throttle window is an engineering estimate, not a measured value against tai's actual read/search volume — it has not been benchmarked.
• Whether chunk-level use_count should itself decay over time (as in Mem0's or YourMemory's Ebbinghaus-curve approaches) is an open question explicitly deferred to a future decision, not resolved here.
• Whether a chunk hit should always propagate to its parent record's last_used, or only under some condition, is a design detail that was decided one way here (always propagate) but not independently stress-tested against real usage patterns.

Action taken

None yet. retoor's instructions for this research task explicitly forbade building anything — this document records the proposal and its supporting research only. Implementation remains a future decision.

Changelog

Date	Change
2026-10-09	Initial conclusion recorded.