Repository navigation
feat(reports): support bounded live experiment monitoring - #52
Open
marcfranquesa wants to merge 2 commits into
Open
marcfranquesa wants to merge 2 commits into
marcfranquesa wants to merge 2 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Growing experiment logs previously failed after 10,000 rows and were fully embedded in every live response. Stream and validate the complete CSV prefix while retaining bounded report data: at most 512 sampled rows per plot and the latest 100 recorded rows per table. Samples preserve endpoints and extrema; per-series discontinuity flags prevent lines bridging omitted missing values. Original logs remain untouched, and static builds remain self-contained.
Live viewing now waits for complete newline-terminated records, caches bounded projections, and sends conditional 304 responses after fresh validation. Stable cache admission preserves a warm subset when catalogs exceed capacity. Appends and continuing atomic replacement preserve inspection by x, focus, disclosure, and scroll; invalid completed data keeps the last valid page until recovery. Changed logs still require a full streaming scan.
Extend the existing report guide and CLI help with the schema, limits, sampling/table behavior, failure handling, complete-row/flushing recipes, atomic restarts/publishing, and local/SSH setup. Remove full-history buffering, the row-count rejection, redundant raw-table trimming/fallbacks, unused table break metadata, and FIFO cache bookkeeping. No new dependencies.
Validation:
go test -race ./...,go vet ./..., JavaScript syntax, andgit diff --checkpass.Benchmark medians on Apple M5 Max / macOS / Go 1.27.1, from three process runs of three operations per case:
Unchanged responses were 304 with zero body bytes. Heap observation runs separately from timing and measures Go heap rather than RSS; generated input fixtures stay outside Git. Reports each contain two 100-row raw tables. Timing thresholds are excluded from deterministic regression tests. The experiment record and root project config are excluded from the repository; the Go benchmark tests remain reproducible directly:
Repeat the timing command three times with unchanged source. Both commands construct temporary projects without a checked-in experiment or root config.