#340: never show baseline facet counts under an active search - #341
Merged
rdhyee merged 1 commit intoAug 5, 2026
Merged
Conversation
Eric Kansa reported (isamplesorg#340) that searching "pottery Cyprus" returns 1,305 results while facet option counts still show the whole-corpus totals. Reproduced on prod: the counts are the verbatim unfiltered global baseline (SESAR 4,389,231 / OpenContext 1,104,985 / GEOME 291,210 / Smithsonian 240,816), even though all 1,305 hits are OpenContext. Root cause is NOT a missing search predicate. The search-aware count path exists, but under an active search the recompute never reaches a terminal repaint: nothing calls applyFacetCounts / markFacetCountsUnavailable / markFacetCountsPending, and markFacetCountsRecomputing() only adds a CSS class, so the previously-painted global numbers stay on screen forever. Instrumented live: 60/60 .facet-count elements still carried .recomputing 15s+ after the search completed cleanly, and a forced camera move (a fresh request) did not clear it. Reproduced on deep-link AND interactively-typed search, with all three facet trees confirmed rendered. Search-aware cross-filtered counts are PLAN_305 Phase 3, deferred past the grant. Rather than debug the stall during closeout, apply the Honesty rule the plan already commits to: an active search means counts are unavailable. - refreshFacetCounts() bails to markFacetCountsUnavailable() when searchIsActive(), before arming the debounce. Bailing early also bumps facetCountsReqId so a late-settling older query cannot overwrite the dash. - The three search producers (text, concept, substrate) call refreshFacetCounts() immediately after publishing active:true, closing the race where a refresh scheduled while inactive fires after activation and re-enters the bad path. Re-check at debounce fire time as defense in depth. - markFacetCountsUnavailable/-Pending now clear stale .facet-row.zero via a shared clearStaleZeroRows(): an unknown count must not imply zero. Verified against a local render with real R2 data: search -> 60/60 "(—)", 0 recomputing, 0 stale zero rows, search itself unaffected (1,305 hits, pins and side panel intact); clearing search restores real counts; the no-search facet path is unchanged (object_type=artifact still gives SESAR 0 / OpenContext 261,086 / GEOME 0 / Smithsonian 0). tests/test_frontend_derived.py 40 passed. Docs: corrected the stale isamplesorg#158 facet-count contract in EXPLORER_STATE.md §7 (viewport and search rows were both already false in code), flagged the superseded option-(C) description in §6 and the obsolete count-implementation note, and recorded Phase 3 status + a post-grant probe list in PLAN_305_facet_counts.md. Reviewed by Codex over three rounds; it caught the activation race and the overstated claims in my first draft of the docs. Refs isamplesorg#340, isamplesorg#304, isamplesorg#305 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QCCDurpcLzMe7L72y2HDAa
This was referenced Aug 5, 2026
rdhyee
added a commit
that referenced
this pull request
Aug 5, 2026
Follow-up to #341. That fix stopped facet counts showing the unfiltered baseline under an active search, but it removed one TRIGGER, not the hazard class. A read-only audit of every number the Explorer displays — asking "what happens if the thing meant to replace this never returns?" — found the same shape elsewhere. Audit doc added: COHERENCE_AUDIT_2026-08-04.md. F3 (HIGH) — the legacy per-dimension count catch painted applyFacetCounts(d.key, null), i.e. the UNFILTERED GLOBAL BASELINE, when a constrained query threw. The repaint even cleared `.recomputing`, so a wrong number looked settled. Now markFacetCountsUnavailable([d.key]). This is PLAN_305 Phase 4's explicit open item ("remove the misleading active-filter baseline fallback everywhere") for this site. F5 (MEDIUM) — in point mode, when the render cap (POINT_BUDGET 5,000) is hit, a follow-up COUNT establishes the true total. If that COUNT threw, the code set totalCount = data.length and capReached = false, so the UI asserted "5,000 samples in view" when the truth could be 10-100x higher — the same bug shape #206 fixed for the no-failure case. Now tracked via totalCountKnown: stats show an em-dash, the phase message says the total is unavailable, and the a1dbg/console diagnostics report "unknown" rather than the cap. M2 (MEDIUM) — reconcileSettledCamera() refreshed the collapsed-by-default stats panel but not the ALWAYS-VISIBLE phase message, so after exiting point mode an ordinary pan stranded a previous viewport's "N clusters in view" on screen as fact. It now repaints that message — but only behind a freshness guard (!loading && computeTargetMode(alt)==='cluster' && currentRes===targetRes && _clusterFilterSig===desiredClusterSig()). The guard also gates the stats repaint, closing the _clusterFilterSig blindness the audit flagged as F4. Structural (partial) — markFacetCountsRecomputing() added a CSS class and left the old numeric TEXT intact. That was the entire mechanism of #340: "in flight" signalled by opacity alone, so an unsettled recompute displayed confident wrong numbers indefinitely. It now also arms a 400 ms timer; counts still unsettled when it fires have their text replaced with "(Loading...)". The delay keeps short interactions from flickering. Per the audit's structural recommendation this is deliberately NOT a watchdog forcing a terminal state — honesty only requires that invalid data stop looking current. Known-open and documented in the audit doc: legacy per-dimension atomicity; comprehensive pre-await invalidation (the most valuable remaining fix — a hang BEFORE invalidation still strands unmarked numbers); a regression test for the timer; and findings M1/F11/F4/F7/F12 plus the static-prose numeric drift. Two process notes worth preserving: - The first attempt declared the timer state with top-level const/let inside a Quarto {ojs} cell. Those are illegal — each top-level statement is a reactive cell — and it killed the entire Explorer. Caught by tests/test_smoke.py, not by review. State now lives on window.__facetCountsStaleTimer, matching the file's existing window.__searchFilter / window.__facetIndexStatus pattern. - The first M2 attempt had no freshness guard and would have let a stale cluster cache overwrite an honest "Loading H3..." or "Failed to load..." with a green settled count — reintroducing this very failure class while fixing it. Caught by Codex in adversarial review, not by tests. Verified against a local render with real R2 data: tests/test_smoke.py passes; tests/test_frontend_derived.py 40 passed; the #341 behavior is intact (search gives 60/60 dashes with 0 stuck recomputing and 0 stale zero-rows, facet-only still gives real counts — SESAR 0, OpenContext 261,086); no pageerrors. Reviewed by Codex over two rounds; it blocked round 1 on the missing M2 freshness guard and approved after it landed. Refs #340, #304, #305 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QCCDurpcLzMe7L72y2HDAa
rdhyee
added a commit
that referenced
this pull request
Aug 6, 2026
…prose figures #343 (Eric Kansa) — the Guided Tour rendered section headings as "1 Stop 1 —", "2 Stop 2 —" and so on: the site sets number-sections globally, and the headings already carried a hand-written "Stop N". Two fixes, both as requested: - number-sections: false for this page only. The step numbers are written by hand because the numbering is part of the tour's voice, so the automatic numbering is what gets removed, not the manual one. - "Stop" -> "Step" in all seven headings, plus the three prose references, so the page doesn't say "steps" in the titles and "stops" in the body. Coherence audit follow-up — the "How This Demo Works" table asserted file sizes that no longer matched anything published. Corrected against the release manifest, with the cluster counts verified independently against the parquet row counts (38,462 / 112,019 / 176,669 — the manifest doesn't carry row counts). Worth recording WHY they were wrong, because the obvious fix was the wrong one: the old figures (580 KB / 1.6 MB / 2.5 MB) are CORRECT for the 202601 snapshot, which SERIALIZATIONS.md, how-to-use.qmd, data.qmd and tutorials/why_h3.qmd still legitimately document. This page describes the live app, which serves 202608, so it had silently drifted one generation behind. Those four other files were checked against the live 202601 objects (594,541 / 1,627,502 / 2,543,412 B) and are accurate — they are deliberately NOT touched. A "wrong number" in this repo may be a right number for a different release. Also dropped the "4 parquet files" claim, which was false — the published release is 21 objects (18 parquet + 3 JSON search-index files) and the boot manifest check enumerates them. Replaced with the real figure plus a pointer to isamples_202608_release_manifest.json and CANONICAL.md as the authoritative inventory, and a collapsed callout giving the curl/jq and SQL one-liners to re-derive every number on the page without trusting the page. Units are now consistently decimal MB. The old row silently mixed conventions (580 KB was KiB while 1.6/2.5 MB were decimal), which is part of why it read as internally consistent while being a generation stale. Verified: quarto render of both pages clean; tests/test_smoke.py passes; tests/test_frontend_derived.py 40 passed; the #341/#342 count behavior is unaffected (search -> 60/60 dashes, 0 stuck; facet-only -> real counts); no pageerrors. Rendered headings confirmed as "Step 1 ... Step 7" with no duplicate numbering. Refs #343 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QCCDurpcLzMe7L72y2HDAa
rdhyee
added a commit
to rdhyee/isamplesorg.github.io
that referenced
this pull request
Aug 6, 2026
…unable to false-pass Codex blocked round 1 on two findings, both valid: 1. A TERMINAL EXCEPTION COULD STRAND (Loading...). invalidateFacetCountsNow() bumps facetCountsReqId and cancels the pending debounce, but refreshFacetCounts() sat only on the success path. If anything between the invalidate and the end of the globe work threw — updateSourceLegendState, writeQueryState, refreshHeatmap, reconcileGlobeForFilters — nothing would ever schedule a recompute and the counts would sit at (Loading...) until some unrelated later event. A HUNG promise staying Loading is honest (isamplesorg#342); a REJECTED one is terminal, so Loading becomes a different kind of lie. Fixed with an inner try/finally in both handlers, so refreshFacetCounts() runs on success AND on throw. No watchdog, no query issued while work is pending — it only guarantees a successor once the work settles or rejects. The outer finally still owns busyRelease(), and selection revalidation still runs after. Deliberately NOT applied to applySearchFilterChange(): Codex analysed it and it is not a stale-number hole (activation paints the (—) dash synchronously, and clearing leaves that honest dash visible during reconciliation). 2. THE VERIFIER COULD FALSE-PASS. It printed ever_loading without asserting it, checked stale-unmarked only before t=12s while sampling past 18s, would accept a permanently dimmed numeric value, and never proved a successor repaint. All four clauses are now required, the stale check covers every observation, and it un-throttles at the end and requires real numbers to come back — which is precisely the anti-stranding assertion for finding 1. Also took Codex's idempotence hardening: under an active search invalidateFacetCountsNow() now repaints the (—) dash rather than merely returning, so it does not depend on an earlier isamplesorg#340 caller having painted it. Verified on the rebuilt page: verifier passes all four clauses (invalidated within 0.4s / swapped to Loading / 0 stale-unmarked windows / successor repainted); test_smoke.py passes; test_frontend_derived 40 passed; isamplesorg#341/isamplesorg#342 behaviour unaffected. STILL OPEN from the review, deliberately not done here: converting the verifier to a discovered .spec.js that deterministically holds a globe request, and hardening facet-tree.spec.js's fixed 3s sleeps (they parse numeric text and could now read (Loading...) if a globe reload exceeds 3s). Refs isamplesorg#340, isamplesorg#304, isamplesorg#305 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QCCDurpcLzMe7L72y2HDAa
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Eric reported
#340: searching
pottery Cyprusreturns 1,305 results, but facet option counts "greatly exceed the total number of search results."Reproduced on prod. The counts were the verbatim unfiltered global baseline — SESAR (4,389,231) / OpenContext (1,104,985) / GEOME (291,210) / Smithsonian (240,816) — even though all 1,305 hits are OpenContext.
Root cause (not what it looks like)
It is not a missing search predicate. The search-aware count path exists and is wired.
The recompute never reaches a terminal repaint under an active search: nothing calls
applyFacetCounts/markFacetCountsUnavailable/markFacetCountsPending. AndmarkFacetCountsRecomputing()only adds a CSS class — it does not change the text. So the previously painted global numbers stay on screen indefinitely, looking precise and authoritative, with no error.Instrumented live on prod: 60/60
.facet-countelements still carried.recomputing15+ seconds after the search completed cleanly (1,305 hits, no console error), and a forced camera move — a fresh request — did not clear it. Reproduced on both deep-link and interactively-typed search, with all three facet trees confirmed rendered.Two plausible hypotheses were tested and refuted: the mixed tree/flat fallback state (all three trees render: 18/21/14 nodes) and a boot race (reproduces on an interactively typed search long after boot).
What this PR does
Search-aware cross-filtered counts are
PLAN_305Phase 3, deferred past the grant. Rather than debug the stall during closeout, this applies the Honesty rule the plan already commits to — never show the unfiltered baseline under an active filter:refreshFacetCounts()bails tomarkFacetCountsUnavailable()whensearchIsActive(), before arming the debounce. Bailing early also bumpsfacetCountsReqId, so a late-settling older query cannot overwrite the dash.refreshFacetCounts()immediately after publishingactive: true, closing a race where a refresh scheduled while inactive fires after activation and re-enters the bad path. Re-checked at debounce fire time as defense in depth.markFacetCountsUnavailable/-Pendingnow clear stale.facet-row.zerovia a sharedclearStaleZeroRows()— an unknown count must not imply zero.This directly answers Eric's own ask in #304: "make sure the user interface does NOT show inaccurate numbers."
Issue #340 stays open — this stops the misleading numbers but does not deliver the cross-filtered counts Eric asked for. That's Phase 3, and Eric should decide whether the dash is an acceptable resting state.
Verification
Playwright against the deployed staging build (
rdhyee.github.io):?search=pottery+Cyprus(—), 0 recomputing, 0 stale zero-rows?object_type=artifact(no search)(0)for SESAR,261,086OpenContext; 0 dashesAlso:
tests/test_smoke.pypasses against staging;tests/test_frontend_derived.py40 passed; local Quarto render against real R2 data confirmed clearing a search restores real counts.Docs
Corrected genuine doc-vs-code drift found along the way:
EXPLORER_STATE.md§7 still documented the pre-explorer: architectural direction — make filter semantics coherent across all surfaces #234 contract claiming neither viewport nor search affects facet counts — both had been false in code for months. Corrected, with a note on why.PLAN_305_facet_counts.md: recorded Phase 3 status honestly (what was instrumented vs. what was not diagnosed) plus a probe list for whoever picks this up post-grant.Review
Reviewed by Codex over three rounds. It caught the activation race in my first draft, that I had reviewed a 42-commit-stale tree, and that my first doc wording overstated the diagnosis ("the query never settles" — only no terminal repaint was proven). Final verdict: approve.
Refs #340, #304, #305
🤖 Generated with Claude Code
https://claude.ai/code/session_01QCCDurpcLzMe7L72y2HDAa