Repository navigation
2.0.20 still truncates internal queries at 500 rows: countPolicies/countEpisodes/countSkills/countWorldModels report 500, and no-limit call sites lose rows (#1954 only changed the ceiling) #2401
Description
Summary
@memtensor/memos-local-plugin 2.0.20 (npm latest, published 2026-09-21) still ships the
500-row clamp on list() that #1954 fixed on dev-v2.0.25, and — independently of the clamp value —
buildPageClauses() still defaults to 500 when the caller omits limit, which silently truncates
the internal "count/list everything" queries that rely on that default.
Two concrete consequences, both observed on a real store:
- Viewer counts are still wrong.
countPolicies/countEpisodes/countSkills/
countWorldModelscompute totals aslist({limit: 100_000}).length. With the clamp at 500 they
can never report more than 500. (countTraceswas already fixed to use real COUNT queries by
Fix #1593: Web UI memory count stuck at 500, actual traces exceed 1400 #1877 — so the same package fixes this pattern in one place and leaves it broken in four.) - Internal consumers silently lose rows. Call sites that pass no
limit— e.g.
l3.jsgathering all active policies for world-model clustering,retrieval-repos.js,
decision-guidance.js, and the L2 candidate sweep — get 500 rows and no warning.
Evidence (2.0.20, dist)
dist/core/storage/repos/_helpers.js
40 export function buildPageClauses(opts, tsColumn) {
42 const limit = clampLimit(opts?.limit ?? 500); // <- default is a page size
46 export function clampLimit(n) {
47 if (!Number.isFinite(n) || n <= 0) return 500;
49 return Math.min(Math.trunc(n), 500); // <- hard ceiling
dist/core/pipeline/memory-core.js
2726 countPolicies -> policies.list({status, limit: 100_000}).filter(...).length
2731 countPolicies -> policies.list({status}) // no limit
(the comment right above line 2731 reads:
"Caller passes no limit/offset so the natural list pages through everything")
2793 countWorldModels -> worldModel.list({limit: 100_000}).filter(...).length
2925 countEpisodes -> episodes.list({sessionId, limit: 100_000}).filter(...).length
3300 countSkills -> skills.list({status, limit: 100_000}).filter(...).length
3111 countTraces -> traces.count()/countTurns() // correct, fixed by #1877
dist/core/memory/l3/l3.js
52 repos.policies.list({ status: "active" }) // no limit -> 500
Measured on a live store (2026-09-21) with the plugin's own repo layer:
| table | rows in DB | repo.list() returned |
viewer reported |
|---|---|---|---|
| policies (active) | 616 | 500 | 500 |
| policies (candidate) | 595 | 500 | 500 |
| episodes | 591 | 500 | 500 |
| skills | 451 | 451 (not yet at the cap) | 451 |
Why #1954 doesn't cover this
#1954 (merged, base dev-v2.0.25) changed only the ceiling:
export function clampLimit(n: number): number {
if (!Number.isFinite(n) || n <= 0) return 50;
return Math.min(Math.trunc(n), 100_000);
}- It did not change
buildPageClauses(opts?.limit ?? 500), so every no-limit call site above is
still truncated at 500 even with that fix applied. - And the fix itself does not appear to have shipped in the local-plugin release line: 2.0.20
(published after that merge) still containsMath.min(Math.trunc(n), 500)and the?? 500default.
Impact beyond display
- L3 world-model clustering only ever sees the newest 500 active policies.
- L2 de-duplication check (
l2.js,policies.list({limit: 5_000})) only sees the newest 500, so
older policies can be re-induced as duplicates. - L2 candidate sweep (
l2.js,policies.list({status:'candidate'})) leaves candidate rows beyond
500 un-swept forever. - Retrieval candidate loading (
retrieval-repos.js) ranks only the newest 500.
(Same class as #2233: an unordered/irrelevant LIMIT applied before ranking. Here the ordering is
updated_at DESC, so a recently re-scored but negative-gain policy can occupy a slot while a
high-gain older policy is dropped. On the store above, the newest-500 cut discarded 73 policies that
belong in the top-500 by gain — best of them gain = 0.591 — while keeping 27 policies with
gain < 0.02, 17 of which have negative gain.)
Suggested fix
clampLimit: raise the ceiling (as fix: remove 500-row cap that truncated viewer count displays #1954 does) — the guard is fine, the value is not.buildPageClauses: do not let the default be a page size for callers that pass no limit, or
make the "give me everything" call sites pass an explicit limit. The comment at
memory-core.js:2729shows the intent is "no limit means everything".- Replace the remaining four
list({limit: 100_000}).lengthcounters with the realcount()
methods that already exist on those repos (policies.js:86,episodes.js:150,skills.js:124,
world_model.js:101) — mirroring what Fix #1593: Web UI memory count stuck at 500, actual traces exceed 1400 #1877 did forcountTraces.
Happy to send a patch if that helps.
Activity
akushonkamen commentedon Oct 8, 2026
I've tracked this down to two semantics in
clampLimit/buildPageClausesinapps/memos-local-plugin/core/storage/repos/_helpers.ts: (1) explicit limits were hard-clamped to 500, and (2) a missinglimitdefaulted to a 500-row page. The fix raises the ceiling to the 100k shape intended by #1954 and omits theLIMIT/OFFSETclauses entirely when no limit is passed, so "no limit" really means all matching rows. A PR is coming shortly — I'll link it here once it's up.akushonkamen commentedon Oct 8, 2026
PR is up: #2464
fix(plugin): treat invalid limits as unbounded and name the query cei…