Repository navigation
Conversation
… plan The format sentier-diagnostics/1 and its validator, which works every listed class, direction, band and verdict out again and refuses a file that disagrees with its own rule. One counting rule with named classes (missing, near zero, zero reference, zero result, sign change, agrees, within noise, beyond 10x, differs), a floor per indicator and reference unit, and direction and colour from the signed difference only. A chain of states from the published results to the current build: each step held against its claim (must or may), the whole difference split exactly between the steps, and what nothing explains counted. Explanations carry their origin (person, code or model) and never change a count. Breakdowns choose parts by both sides and say what they leave out. A static page with no build step and no network, which works out no class itself; a command (check, build, summary, page, example); a reader of CSV tables by column name. Standard library only, Python 3.10 and later. Tests: the acceptance cases of SPEC.md, one per defect of the tools this replaces, shared by the Python rule and the page script; unit tests and doctests; guards against data in the repository. CI on Python 3.10 to 3.14, the installed package, and Node. SPEC.md, docs/migration.md and docs/runbook.md. Parts adapted from sentier-bafu's before/after version 2 (MIT), credited in NOTICE. Nothing taken from the private backtest repositories. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aintainer The first real step sentier-bafu will show (a flow that names a country takes that country's factor, claim "water use moves; nothing else moves") is a made-up fixture with a test of what it shows. The page says "identical" for two builds that do not differ, never "not measured". Manuel named Laurenz Bougan as the second maintainer; the spec says a state's identity carries the solver. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
log10 differs in its last digit between the C libraries of Linux and macOS, so the exact comparison failed on every CI runner. The rank is still compared exactly; the distance as the page-script test already compares it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the tables fixture From sentier-agribalyse's first use of the reader contract: a reproduction step with the claim "nothing moves" read its residual as "not claimed". The page and the summary now say what step 0's moves are, and the spec says step 0 needs no claim. The tables fixture shows an explanation by a model, with the fields it needs. The measured case for noise per item goes into the open questions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y default From sentier-bafu's review of this pull request (an agent's review, run on its real results and on made-up probes). Each fix has a test. - Validator: counts, attribution and closer/further are worked out again from the listed items. With nothing worth a look left out they must match; otherwise nothing listed may exceed what is counted. Swapped directions, rewritten verdicts, moved bands, missing moved to the other side, rewritten attribution and closer/further on step 0 are refused. A claim whose entries all list their items has their size as `reach`; floors come from the first state, and their basis agrees with their count. Invented agreeing items that are not listed cannot be caught from a file; SPEC 4.3 says so. - Noise is taken only over pairs beyond the floor in both builds. A residue of 4e-16 against 0 made the noise 1.0 and hid a correction by a factor of 580. An indicator without such a pair is "not measured", never "identical"; noise above 1 % is called large, with its size, on the page and in the summary. - A scope entry read from tables without `expect` may move (SPEC 6); it was `must`. - Page: only an https address becomes a link; a file read by ?file= or chosen by a reader says it was not checked; a file it cannot draw draws nothing. Elements marked hidden stay hidden (the local-file banner showed as an empty box). - Missing on both sides counts as missing before and after; check and page refuse a file over 12 MB; scope items given as one text are refused; dumps keeps a file whole whatever its key order. - SPEC 3 and the migration plan: the located-factors claim "water use moves; nothing else moves" was incomplete. sentier-bafu's run found land use moving for 5 datasets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sentier-bafu decided its claim for the countries' factors from its build, not by hand: the install report's coverage.located_by_method names the methods that take a country's own factor, on BAFU:2026 v1 water use and land use. Its first run of this tool had shown land use moving for 5 datasets as "not claimed". The made-up fixture gains a land use column (one dataset moves) and the claim names both indicators; the test holds land use as claimed and the steel's climate change still not claimed. SPEC 3 and the migration plan say how the scope is built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 16, 2026
From sentier-bafu's agent review of this pull request, filed as issues 3 to 19. Each fix has a test that fails without it. Six of them take a decision Manuel is asked to confirm in review (#3, #6, #8, #9, #12, #16). Verdicts and claims - #3: under a `must` entry, a pair that agrees, is within noise or is near zero is "claimed, not moved"; a missing pair is not judged. `chain.build` refuses a claim that names an item no state holds. - #12: a pair that moved is held to the `must` entries that cover it, and to the `may` entries only where no `must` entry does. - #16: a zero reference is never within a relative `largest`; `reach` is the size of the lists the claim holds, `*` entries aside; SPEC and docstrings now say what the code does in all seven places. Classes, numbers and tables - #6: agrees and within noise are tried before the floor classes, for two results of one sign; a pair across the floor that agrees is no longer a zero result listed second worst (cases A26). - #18: the distance of two sizes is taken from their factor where it is a normal float and from the difference of their logs elsewhere, in Python and on the page alike; the median floor cannot overflow (A27). - #10: a table's numbers are plain ASCII decimals; a non-zero text that underflows to zero is refused. - #13: an item of the items table that no results table holds is missing in every state, not dropped. Attribution and the page - #8: the largest step is chosen among the steps in which the item moved, ties to the earlier step; an item that moved over the whole way while no single step did is counted as `spread` (new key). - #9: the page prints the sum of the shares it computed, not a fixed 100 %, and says when steps cancel instead of printing shares. - #7: an item missing in any state gets no share for any step. - #5: a breakdown must be of the item and step that name it: `check` refuses another, the page does not draw it. Validator and reading - #4: the validator returns faults and never raises, whatever type a value has (a fuzz test over every value of the example). - #11: `build` and `check` end in one line on ordinary mistakes. - #19: strict JSON (no key twice), lone surrogates refused, approval words refused as identity keys, the items limit named, numbers from 1e4 written with an exponent. - #17: the summary escapes the file's texts in Markdown. Tests - #14: the page is drawn under Node with a small DOM stand-in (tests/js/dom.js); the Python tests that compared constants or searched source files now test behaviour. - #15: each validator check, broken on its own, must name its own fault (EachRuleOnItsOwn); summary counts, the byte order mark, the 12 MB limit, and the moved, missing and banner texts on the page. A probe of 55 one-line changes to a copy of the code: every one is now caught by a test (28 of 43 went unnoticed before). Also, from sentier-bafu placing a file whose first state is a build: the page says step 0 reproduces the published results only where the chain starts from them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page is now one body for both uses. `page.body(data)` is a single `<div class="sentier-diagnostics">` with the file inside it as JSON: no html, head or body, and no id. A site holds it and loads diagnostics.js and diagnostics.css from `page.assets()`. `page.write` wraps the same body in a page of its own with `standalone.css`. - The style is scoped: every rule is under `.sentier-diagnostics`. It sets no font and nothing on `:root` or `body`, and resets what a site does to bare tables. Colours are `--sd-*` that take the site's tokens (`--sd-ink: var(--ink, ...)`). Up, down and noise keep their own colours. - The script mounts into every `[data-sentier-diagnostics]`. It reads the data from the body, a global or `?file=`, and sets the document title only on a page of its own. - The matrix of every item (`sentier-diagnostics-matrix/1`, new module `matrix`): a local file of items may name one, and `chain.build_with_matrix` writes it. `matrix.against` works every code out again and holds the file's counts, attribution and `items_count` to it exactly. The page loads it after drawing, as `matrix.js` beside the page or as JSON over http, and refuses one of another file. - The grid shows after ÷ before, coloured by direction and shaded by band. It sorts by a heading (furthest first), filters by name, group, class and verdict, and draws 300 rows at a time. - The item opens in a drawer. "Why it differs" comes first: the item's own explanations, then those of a step or indicator it moved in. - CLI: `check` holds a matrix beside its file to it; `page` writes it; `example` also writes the local example. The local example is committed in examples/local/. Taken from the trial dashboard of sentier-bafu-agribalyse: its look and structure only. None of its values, and not its two definitions of agreement, hand-written labels or a "none" over a top-N list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Asked for by sentier-bafu: every writer otherwise drops absent values itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both from sentier-agribalyse's trial on its own data (2,451 products x 19 indicators): - A description may name the matrix (`"matrix": "matrix.json"`, local files of items only). `from_tables.build_with_matrix` returns it, and `sentier-diagnostics build` writes it beside the file. - Once the matrix is loaded, the line under the title says all items are loaded instead of "N are not in this file". Also a node test where a breakdown names the right step and another item. The mutation probe found that removing the page's item check went unseen: the old test was refused by its step alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Asked for by sentier-bafu, whose items are datasets: the matrix's lead, the whole-way note and two aria labels said "item". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sults
Asked for by sentier-bafu, which calls its indicators impact categories and
whose chain starts from a build of its own.
- The format takes an optional `indicator: {one, many}`, checked like `item`.
chain.build takes `indicator_word`, a tables description takes `indicator`.
The page and the summary use the word wherever they said "indicator"; a
file without it reads as before.
- Steps are numbered from 0 only where the first state is `published` (step 0
is then the reproduction); otherwise from 1. Page and summary agree
(`fileformat.first_step`, `stepNo` in the script).
- The page refuses an indicator word that is not {one, many}.
- The near-zero class text no longer names the indicator: "both results are
within their floor".
Tests: Python 182 (from 176), node 45 (from 42). Each of nine reverts of
the change, applied on a copy, makes a test fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…est first From Manuel's look at the page in sentier-bafu's frame (11,947 datasets x 25 impact categories), side by side with sentier-bafu-agribalyse's trial dashboard. Most cells showed "=" or "1 - 3.037e-5", the tint was faint, and the grid opened with the listed items, whose moves were in columns off-screen. - Every cell with a ratio shows it in three significant digits (1.00, 0.984, 2.27, 1.01e-3), agreeing pairs included. The class is the tint and a mark (~ within noise, >> beyond 10x), never in place of the number. Cells without a ratio keep what the class says (from 0, - -> +, ~0, missing). - Up is amber and down teal, as in the dashboard; the tint is stronger (up to 55 %, by band). - The grid opens with the items furthest apart: the largest factor over every indicator among the values that moved with a ratio, then moves without a ratio by severity, then the rest. A second click on a heading goes back. - Compact columns: no Unit column (it is in the name's title and the drawer), hyphenated headers, the group cut to 8rem. - SPEC: section 10.5 says all of this, and section 10 says how a host lets the body be wide when its article caps children at a reading measure. Tests: node 48 (from 45). Seven reverts of the change, applied on a copy, each make a test fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…below From Manuel's second look in sentier-bafu's frame, side by side with the trial dashboard at 1600 px: ours was a report page with a matrix in the middle. A 17-column steps table opened it, its label and claim broke after every word, and the header ran to six blocks with a red box. - Header: title, the file's about, one quiet line of counts; a local file is a quiet part of that line, not a box. No eyebrow. The states move to the steps section. - A row of chips chooses the comparison in place of the select: the whole way, then each step with its number, the values that moved, "!" where nothing explains it and a mark per verdict worth a look. - One control row: name, group, class, verdict and the size filter, with the count, the matrix line and a compact legend on its right. The count says "Showing 300 of 11 947 datasets"; the comparison and the order go to the grid's aria-label. - Then the grid and the drawer. Below them, closed until opened (details): steps and the whole way, distributions, worst, explanations, rule. The steps table gives its label and claim room; numbers stay narrow. - A host's page never offers to open a file, only says why nothing is drawn. Tests: node 50 (from 48). Six reverts, applied on a copy, each fail a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t moved first Built on 7274efd (the chips, the grid first, the rest below), after Manuel's third look in sentier-bafu's frame. What a reviewer still had to hunt for: - What a step claimed. The claim sat in the folded steps table. Under the chips is now one line for the chosen step: the states it goes between, what it is, its claim with who made it, what the claim found ("All 11 124 moves as claimed", or "691 not claimed" in the signal colour), its tolerance and noise. - What a step moved. The columns came in the file's order, so the countries' factors (water use and land use) sat off-screen at the far right. The matrix now puts first the indicators in which the chosen comparison moved most; one in which nothing moved comes last, quietened, and keeps its column. - Where a reviewer is. The address keeps the chosen step with the open item (`#step=<id>&item=<id>`), so a link opens exactly that view; closing an item keeps the step, and an unknown step falls back to the whole way. - The arrow keys move between the chips. - The filters keep one line; the count, the matrix line and the legend take the row under them. Tests: node 54 (from 50). Seven reverts, applied on a copy, each fail a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ress Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ial dashboard has them The distributions were a table of small bars, folded away at the foot of the page. They are now always in view under the matrix, one card per indicator, in the order of the matrix's columns (what the chosen comparison moved most first, the unmoved ones quiet). Each card is a histogram of the file's own counts: across zero, down from far to near, the band around 1, up from near to far, then the values with no ratio; on the scale of its own tallest bar, every count in a bar's title. Under it, how many moved, and the median, 10th and 90th percentile of after ÷ before, in as many digits as keep apart the ones that differ. Those come only from every item (the matrix, or a file that lists all): the listed ones are the furthest apart, and a median of them would say more than the whole holds, so the card gives none and the section says why. A card's title sorts the matrix by its indicator; a heading of the matrix marks its card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each card on its own tallest bar is the default, as the trial dashboard draws them, so a few values that moved are seen beside many that agree; one scale (the share of all values) is a choice beside the legend, so that a bar of one height means as many in each card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…honest percentiles From the review of 752043b and 38d6767: - The line of a step names every verdict on a move with its count, "as claimed" first ("11 124 values moved as claimed"); "Nothing moved, as claimed" only where all are zero. Weight, not green or red: a verdict is not an approval (ADR 0010, A20). - A column or card whose only change is a missing value is not quietened as if nothing moved: missing counts with what moved, and its title says so. - The percentiles say over how many ratios they are taken and how many values have none; each is widened only as far as its neighbour needs, and two that six digits cannot part read "just under the median" / "just over the median". - Each card says what its full height is ("tallest bar: 7", or on one scale the share). - The axis words fit at 375 px (↓ ÷10, ×10 ↑). - SPEC: the distributions are open under the matrix; 10.3 says per card is the default and why, one scale a switch, and that the percentiles are the one number the page works out, from every item only; 10.5 says choosing a comparison sorts furthest-first again; A21. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ring pair apart
Each percentile was widened on its own, so one line could mix precisions:
1.00004, 1.00005, 1.00006 read "1.0000, 1.00005, 1.00006", a rounded 10th
percentile standing as exactly 1 beside a six-digit median. Now the three
share one precision, the smallest (3 to 6) at which every two that differ
read differently. Where six digits cannot part a percentile from the
median, it is said in words and the numbers left take the digits they need
("just under the median, 1.00, 5.95"). Found in review by the
sentier-diagnostics session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review a step where it is chosen: its claim beside its effect, what it moved first
…an its file checkMatrix compared the title, the ids and the count only, and matrixRows then drew the matrix's values and classes even for the items the file lists. A matrix left over from another run of the same chain passed and replaced the file's numbers (found in the review of PR 20: p08's values tripled and every class changed, and the grid showed them). Every listed item's row must now say what the file says of it: name, place, group, unit, results, classes, directions and verdicts. It compares through matrixRows, so it holds for any shorthand the decoder reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A frameless page for a site's own page, and the matrix of every item
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first version of sentier-diagnostics: one open tool for "what changed, before and after, in the impact assessment results, and why". It replaces the backtest dashboards, sentier-bafu's before/after page and the sentier-bafu-agribalyse trial dashboard. Start with
SPEC.md.What is in it
sentier-diagnostics/1and a strict validator (fileformat.py,sentier-diagnostics check). The validator works every listed class, direction, band and verdict out again from the values, the floor and the tolerance in the file, and works the counts out again from the listed items. It refuses a file that disagrees with its own rule, unknown keys (including any status,accepted_byor approval in an identity), NaN, a key written twice and other formats. It returns faults and never raises. There are two tiers (items,counts) and anaudience(localby default, orpublic).rule.py). The classes are missing, near zero, agrees, within noise, zero reference, zero result, sign change, beyond 10x and differs, tried in that order. Nothing is blanked, zeroed or called agreement. There is a floor per indicator and reference unit, written into the file. Direction is the sign of after − before, and colour follows direction only.chain.py). States run from the published results (step 0) to the current build. Each step is held against its claim (mustormay); amustentry is unmet by a pair that did not move. The whole difference is split between the steps, exactly up to rounding. The file counts the step that moved each item most, items no single step moved (spread), what moved in steps nothing explains and the items it cannot split.check,build(from CSV tables read by column name),summary(Markdown, escaped),page,example. Standard library only, Python ≥ 3.10.SPEC.md,docs/migration.md,docs/runbook.md, GLOSSARY, MAINTAINERS and NOTICE.Decision (SPEC §13): sentier-bafu's before/after v2 code is the seed. It is adapted and credited in NOTICE (MIT, sentier.dev). The format is rewritten under a new name, because what its classes and claims mean changes; v1 and v2 files are not read. Nothing was taken from the private backtest repositories. Their defects are described in words as acceptance cases.
The first real step, sentier-bafu's located factors (claim: "water use and land use may move; nothing else moves"), is a made-up fixture in
tests/fixtures/located-factors/, and a test holds what the page will show for it.The requirements the three sessions wrote on 16 November are in SPEC §14, each marked as done, decided or later.
Since the first push
d6a7e0e: from sentier-bafu's first review. The validator holds counts, attribution and closer/further against the listed items. Noise is measured only beyond the floor. A scope entry read from tables withoutexpectmay move.0e1f5be: located factors. The claim for the countries' factors names water use and land use, as sentier-bafu's build decides it.4b7e1c9: the 17 findings of sentier-bafu's second review, filed as Amustclaim is never unmet for a missing result, a near-zero pair, a within-noise pair or an unknown item #3 to Small: number format, lone surrogates, approval keys in identity, duplicate keys, an unnamed limit #19. Each fix has a test that fails without it:mustentry, a pair that agrees, is within noise or is near zero is "claimed, not moved".chain.buildrefuses a claim that names an item no state holdscheckrefuses another one, and the page does not draw itspreadbuildandcheckend in one line on ordinary mistakesmustentries that cover itlargestCloses #3
Closes #4
Closes #5
Closes #6
Closes #7
Closes #8
Closes #9
Closes #10
Closes #11
Closes #12
Closes #13
Closes #14
Closes #15
Closes #16
Closes #17
Closes #18
Closes #19
For Manuel to confirm or change in review: #3, #6, #8, #9, #12 and #16 each take the option their issue proposes. Each issue says what the alternative would be.
A file written before
4b7e1c9is refused by this reader. Its attribution has nospread, and its verdicts may be wrong. The format is stillsentier-diagnostics/1, because no version has been tagged yet; SPEC 4.1 makes any later key change a new major version.How to check
python3 -m unittest discover -s tests -t .Then open
build/example/page/index.html.tests/acceptance/cases.json, run against both the Python rule and the page script.What was not checked
Open, for Manuel (SPEC §15)
largest, and a joint view of two references.A person reviews and merges; the author does not merge it. The tag
v0.1.0is to be made by a person after the merge (runbook).🤖 Generated with Claude Code