Memory.Statistics / Memory.Prune

Summary#

A corpus grows. Every [Searchable] property, every remembered file, every indexed section carries a vector, and vectors are the expensive part. Memory.Prune reclaims that space from the entries nothing has asked for.

Signature#

MemoryStatistics Memory.Statistics(DateTime unusedSince)   // what it costs, and what pruning would free
int              Memory.Prune(DateTime unusedSince)        // → how many entries lost their vector

Description#

Search records the day each memory was last RETURNED. Memory.Prune takes a date and acts on everything that has not been returned since then.

It drops the vector and keeps the text. An entry without a vector stops being found by meaning — which is what was costing storage — while remaining readable, still matched by exact words, and re-indexed the moment anything changes it. Nothing is deleted.

It never touches memory somebody chose to remember. A [Searchable] property's text is still in the property, so its index can always be rebuilt; a remembered file, a conversation turn, or a link's stated reason exists nowhere else. Only the first kind is eligible, and that rule is the platform's rather than yours to pass.

Nothing prunes on its own. There is no background sweep and no retention setting that quietly forgets things — a corpus that dropped what it had not been asked for lately is one you could not trust with a rare question, and rare questions are what a memory is for. Somebody has to decide, which means somebody has to call this.

Look before you prune#

Memory.Statistics(unusedSince) reports the same corpus from the same rule, so nobody has to prune to find out what pruning does:

• StoredBytes — what your memory ACTUALLY occupies: rows, text, and every index over them, read from the database rather than worked out. It is usually much larger than VectorBytes, and the difference is real — a vector index is often as big again as the vectors it indexes, before the text is counted. null means the database could not say, which is "unknown", not "nothing".
• Entries · Vectors · VectorBytes — how much there is, and how much of that is vectors. VectorBytes is worked out from the count, so it is the floor of what the vectors cost, not the whole bill.
• Derived · Authored — what could ever be reclaimed, and what could not. Authored is the FLOOR: no cutoff, however aggressive, releases any of it.
• NeverUsed — entries no search has ever returned. The strongest signal that a corpus is carrying weight it does not need, with one honest caveat: a rare question nobody has asked yet looks exactly the same.
• Reclaimable · ReclaimableBytes — what a prune at this cutoff would free. Not an estimate: it is counted with the very filter Memory.Prune acts on, so it is the number you get.

Every figure is a count, so asking repeatedly is cheap — trying 30, 90 and 180 days to see the curve is the intended use.

Who may prune? Naming the memory operator#

Both verbs are reserved for the app's memory operator, and you name that operator by declaring a policy:

app.Memory = new MemoryConfig { Operator = IsOperator };

This is not optional, and forgetting it is a compile error rather than a silent refusal. An app that calls Memory.Statistics or Memory.Prune without saying who may run them has not authorized anybody, so the build stops and tells you what to write. The alternative would be a prune button that does nothing for the one person it was built for, with no way to tell "nobody declared this" from "you are not an operator".

Say it once and you are done. The platform applies it at the verb, so the functions you write around it hold no authority code — and neither does the second way in you add later, whether that is an agent's tool, an endpoint, or another function calling the first.

Examples#

using Osyrin.Memory;

[Role] enum AppRole { Member, Operator }

[Principal] entity Person {
  [Required, MaxLength(200), Unique] string Email;
  security { allow read when IsAuthenticated; }
}

entity RoleGrant {
  [Required] Person Grantee;
  [Required] AppRole Level;
  security { allow read when IsAuthenticated; }
}

// Anybody holding an Operator grant. It is an ordinary policy — a role, a flag on the person, membership of a
// team: whatever your app already means by "this is the person who looks after our data".
policy IsOperator => RoleGrant.Any(g => g.Grantee == user && g.Level == AppRole.Operator);

app.Memory = new MemoryConfig { Operator = IsOperator };

using Osyrin.Memory;

string MemoryReport(int olderThanDays) {
  var s = Memory.Statistics(DateTime.UtcNow.AddDays(-olderThanDays));
  return s.Entries + " entries using " + (s.StoredBytes / 1048576) + " MB, " + s.Vectors + " of them indexed. "
    + s.NeverUsed + " have never been returned. Pruning at " + olderThanDays + " days would free "
    + s.Reclaimable + " (" + (s.ReclaimableBytes / 1048576) + " MB). "
    + s.Authored + " were remembered on purpose and are never touched.";
}

using Osyrin.Memory;

// No authority check in here, and none is missing: `app.Memory.Operator` above already said who may, and the
// platform applies it at the verb rather than trusting each caller to remember.
int ReclaimMemorySpace() {
  // Anything derived that nothing has needed for six months gives up its vector.
  return Memory.Prune(DateTime.UtcNow.AddDays(-180));
}

using Osyrin.Memory;

string PruneAndReport(int olderThanDays) {
  var freed = Memory.Prune(DateTime.UtcNow.AddDays(-olderThanDays));
  if (freed == 0) { return "Nothing to reclaim — every memory has been used recently."; }
  return freed + " entries released their search index. Their text is unchanged.";
}

See also#

• [Searchable] — what puts a property into the corpus in the first place
• Memory.Remember / Memory.Forget — memory somebody chose to keep, which this never touches
• using Memory (semantic search) — the reads that record a memory as used