From 46866101a4ded7158b7da046a32de100fc4e9b2c Mon Sep 17 00:00:00 2001 From: Manuel Klarmann Date: Tue, 17 Nov 2026 05:47:09 +0100 Subject: [PATCH 1/6] fixes.py: read and check a fixes file; compose, label and count any set The validator, reader and made-up fixtures of sentier-diagnostics-fixes/1 (SPEC 16.12, step 2). errors() checks a file on its own: shape and limits, every fix's counts and verdicts worked out again from its effect and the base, pairs, and every built set the file calls independent against the sum of its effects. against() and beside() hold it to its diagnostics file and matrix; `check` dispatches to it and prints what the file is made of. A listed pair carries the measured size of its cross term (`largest`), as sentier-bafu proposed: an estimated set then says how far it may lie from its sum, and a built pair is held to its stated size. SPEC 16 now says what the code decided: the rule without the chain's total tolerance, closer/further null on a fix, a claim that names its items, a set where the base has no result, the noise of the built-set check, pair terms, measured sizes, and the open questions. Co-Authored-By: Claude Opus 5.5 --- GLOSSARY.md | 1 + SPEC.md | 83 ++- src/sentier_diagnostics/cli.py | 16 +- src/sentier_diagnostics/fixes.py | 1010 ++++++++++++++++++++++++++++++ tests/test_fixes.py | 482 ++++++++++++++ tests/test_modules.py | 3 +- 6 files changed, 1571 insertions(+), 24 deletions(-) create mode 100644 src/sentier_diagnostics/fixes.py create mode 100644 tests/test_fixes.py diff --git a/GLOSSARY.md b/GLOSSARY.md index ea9b02f..1068004 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -36,4 +36,5 @@ The words this repository uses, each in one line. [SPEC.md](SPEC.md) says the re - **Effect**: what a fix does to the base, per item and indicator: its result with the fix minus the base result. - **Set**: the fixes a reader chooses together; its result is the base plus the sum of their effects. It has no verdict. - **Interaction**: two fixes whose joint result is not the sum of their effects. Checked in pairs, in the model. +- **Pair terms**: how far an estimated set may lie from its sum, to first order: the sum of the measured sizes of the listed pairs it holds; not known where one was not measured. - **Built, exact, estimated**: the label of a set. Built where the producer built it; exact where no two of its fixes can interact; estimated otherwise, and marked on every cell. diff --git a/SPEC.md b/SPEC.md index cf16af1..a4ebc1f 100644 --- a/SPEC.md +++ b/SPEC.md @@ -860,8 +860,8 @@ Status: proposed, 18 November 2026. Manuel decided the direction: a chain is to hundreds of fixes, any set of them can be chosen, and the set is **one base with fixes measured on their own**, not a chain of hundreds of states ("yes, one base with independently measured fixes"). He also decided that the fixes live in a file of their own, -and that a set the file cannot show to be exact is shown, marked as estimated. Nothing -here is code yet; the validator follows once this text is agreed. +and that a set the file cannot show to be exact is shown, marked as estimated. The reader +and validator are `fixes.py`; the producer and the page are not written yet (16.12). ### 16.1 Why not a chain of fixes @@ -917,8 +917,8 @@ item. One JSON object, read as strictly as the diagnostics file (section 4.1). | `format` | `"sentier-diagnostics-fixes/1"` | | `of` | `{file, title, base}`: the diagnostics file it belongs to (a file name), that file's title, and the id of the state that is the base | | `title`, `about` | Short text; `about` may be null | -| `item`, `indicator`, `indicators` | As in the diagnostics file, and equal to it | -| `rule` | The floors, equal to the diagnostics file's (they come from the chain's first state) | +| `item`, `indicator`, `indicators` | As in the diagnostics file, and equal to it (`indicator` may be left out where that file leaves it out) | +| `rule` | `{floor_factor, floors_from, beyond, floors}`: the diagnostics file's rule without its `total_tolerance`, which belongs to the chain (a set has its own, 16.5). The floors come from the chain's first state | | `items` | Every item, `[id, name, place, group, unit]`, in the order of the matrix | | `base` | The base result of every item, per indicator: `{indicator: [result or null, ...]}`, in the order of `items` | | `fixes` | `[{id, label, about, kind, explained, link, claim, tolerance, noise, effect, counts}]` | @@ -930,9 +930,12 @@ A fix's `effect` holds, per indicator, the items whose result with the fix is no result bit for bit, as `[[item index, result], ...]`, the result being the one built with the fix alone (`null` where it has none). **Every change is kept, however small**: two changes under the tolerance each can be over it together, and a set is a sum. An entry -equal to the base is refused: it would read as a change. A fix's `counts` is one block per -indicator (section 4.2), its pairs being the base against the result with the fix, judged -with the fix's own tolerance and noise. +equal to the base is refused: it would read as a change. Every indicator has a list, empty +where the fix changes nothing. A fix's `counts` is one block per indicator (section 4.2), +its pairs being the base against the result with the fix, judged with the fix's own +tolerance and noise; `closer` and `further` are null, since a fix has no published results +to come closer to. A fix's claim names its items (no `items: null`): the file is local and +holds every item, and the page works out each item's verdict from the claim. A fix's claim is judged as a step's is (section 6), against the base. Its verdicts do not depend on which other fixes are chosen or in what order: that is what measuring a fix on @@ -940,8 +943,11 @@ its own buys. ### 16.5 A set and its label -The result of a set is the base plus the sum of its fixes' effects. It is judged by the -rule (section 5) against the base: +The result of a set is the base plus the sum of its fixes' effects. A fix without a result +at an item leaves the set none there. Where the base has no result and two of the chosen +fixes give one, their differences have nothing to be added to: the set has no result there +either, and the page says the sum cannot be formed, apart from a result that is missing. It +is judged by the rule (section 5) against the base: - its **tolerance** is the largest of its fixes'; - its **noise** is the sum of its fixes' noise, per indicator. A fix's noise stands for @@ -949,7 +955,9 @@ rule (section 5) against the base: comparisons, and their instabilities add up instead of cancelling, as they would along a run of a chain. Noise not measured for some fixes is said ("noise measured for 3 of 5 fixes"), never read as none; where none is measured the page says "not measured", never - "identical"; + "identical". A fix's noise names its two builds (`between`): two builds in one process on + one solve route measure the engine's determinism and read 0, two routes measure how the + comparison moves with the route; the note says which; - it has **no verdict**: each claim is judged in its own fix. Each set carries one of three labels, and the page shows it wherever the set is drawn: @@ -958,10 +966,14 @@ Each set carries one of three labels, and the page shows it wherever the set is |---|---|---| | built | the set is one fix, or one of `built` | the built results | | exact | `interactions.checked` is true, and no two of its fixes are a listed pair | the sum, as a result | -| estimated | otherwise | the sum, marked on every cell, in the line above the matrix and in every count ("estimated: 153 would differ"), with the listed pairs it holds and their reasons | +| estimated | otherwise | the sum, marked on every cell, in the line above the matrix and in every count ("estimated: 153 would differ"), with the listed pairs it holds, their reasons, and their pair terms where measured ("pair terms up to 0.003 %") | An estimated set is never drawn as a built one: the mark is on the cell, not only in a -heading a reader may have scrolled past. Where interactions were not checked, every set of +heading a reader may have scrolled past. Its **pair terms** are the sum of the measured +sizes (`largest`) of the listed pairs it holds: how far it may lie from its sum, to first +order (a term of three fixes or more needs interacting pairs, 16.3). Where a pair it holds +was not measured, the pair terms are not known, and the page says how many of its pairs were +measured, never 0. Where interactions were not checked, every set of two fixes or more that was not built is estimated. ### 16.6 Interactions @@ -974,8 +986,17 @@ into it. - `checked`: true where every pair of fixes was examined, false where none was. There is no half: a reader must know whether a pair that is not listed was cleared or never looked at. - `method`: what the check did, in a sentence, and `by`: who or what ran it (section 8). -- `pairs`: `[{fixes: [a, b], reason}]`, every pair that can interact, each once, with a - short reason ("both change the market for wheat flour"). +- `pairs`: `[{fixes: [a, b], reason, largest}]`, every pair that can interact, each once, + with a short reason ("both change one market; 412 items reach both changes") and + `largest`: the measured size of its cross term, the largest over items and indicators of + |h_ab − h_a − h_b + h_0| / |h_ab|, or null where it was not measured. A check that only + asks whether A⁻¹ links two changes lists nearly every pair in a database where + electricity and transport reach everything, and every set would be estimated; the size + lets a reader tell a negligible estimate from a large one. A pair whose cross term is + zero by the model's structure is not listed. A pair has no other key, and the check + refuses a reason or a method holding text shaped like an identifier (a UUID, or 32 + hexadecimal digits). Whether a reason may name an activity ("the market for wheat flour") + is open (below); the producer writes counts and kinds of change until it is decided. The validator cannot see the model, so it cannot prove a pair independent. It holds the claim to every built set instead (16.7). @@ -989,13 +1010,19 @@ bit for bit, as an effect is written) and its noise. **A built set tests the claim of independence.** Where `checked` is true and no two of its fixes are a listed pair, the built results must agree with the base plus the sum of the -effects, by the rule with the set's tolerance and noise, for every item. A file where they +effects, by the rule with the set's tolerance and noise, for every item. The noise here is +the sum of its fixes' noise and the built set's own, where measured: the sum stands on one +build of each fix, the built results on one more. A file where they do not is refused, naming the set, the item and the indicator: its producer called two fixes independent that are not. Where the set holds a listed pair, the page shows how far -the built results lie from the sum, as that set's interaction. +the built results lie from the sum, as that set's interaction. Where the set is exactly one +listed pair with a measured `largest`, the built results must lie within it of the sum (by +the rule, with the larger of the set's tolerance and `largest` as the tolerance): a file +that understates a size it could have measured is refused. Building the release with every fix is cheap (it is built anyway) and is the strongest test -the file can carry: it touches every pair at once. +the file can carry: it touches every pair at once. It can refute independence and never +confirm it: cross terms of opposite sign can cancel in the whole and not in a subset. ### 16.8 The page @@ -1031,14 +1058,25 @@ wrong (as section 4.3), and further one where: - a fix's counts or verdicts are not what its effect and the base give, by the rule, with its own tolerance and noise. With every item at hand, the counts must be exact; - a pair names a fix it does not hold, a fix twice, or is listed twice; `checked` is false - and pairs are listed; + and pairs are listed; a reason or method holds text shaped like an identifier; - a built set holds fewer than two fixes, a fix twice, or the same fixes as another; or - disagrees with the sum of its effects where the file says its fixes are independent. + disagrees with the sum of its effects where the file says its fixes are independent, or + lies further from it than the measured size of the one pair it is. + +`check` also prints what the file is made of: how many entries its fixes and built sets +hold, and its largest fixes (a fix to a market everything uses holds about one entry per +item and indicator). ### 16.10 Limits Up to 1,000 fixes, 500 built sets, 200,000 items and 64 MB, as the matrix. A base of 11,947 items in 25 indicators is about 300,000 numbers; an effect holds only what changed. +sentier-bafu measured its fixes on that base: a patch moves tens of entries, a fix to a +characterisation factor about 12,000 (about 250 KB), a fix to a market everything uses up +to about 300,000 (6 to 7 MB). 64 MB holds about ten dense fixes and thousands of sparse +ones; effects move to one file per fix only if dense fixes become common. Checked on a +made-up file of that base with 100 fixes of 500 items each in every indicator (39.5 MB), +the check takes 2.4 seconds. ### 16.11 Acceptance cases @@ -1065,4 +1103,9 @@ Each is a test before the code is written. 4. The page: the list of fixes, the set in the address, the port of the rule: sentier-bafu. Open, for later: a public file of fixes (counts per fix only, no item); noise per item -(section 15); and how a fix becomes a step of the chain once it is released. +(section 15); how a fix becomes a step of the chain once it is released; whether a reason +may name an activity; a fix that needs another applied first (`requires`, its effect +measured on the base and its requirements, which the page adds to any set that chooses it); +a fix that adds items the base does not hold (refused for now: an effect names base items +only); and a built set that names the chain state it equals, so that the release with every +fix is held to that state in the matrix: two builds of one thing. diff --git a/src/sentier_diagnostics/cli.py b/src/sentier_diagnostics/cli.py index 9c732aa..de364c8 100644 --- a/src/sentier_diagnostics/cli.py +++ b/src/sentier_diagnostics/cli.py @@ -1,7 +1,8 @@ """The command `sentier-diagnostics`: check, build, summarise and show diagnostics files. sentier-diagnostics check FILE [FILE ...] exit 1 if any file is not one a page shows - (a matrix beside a file is checked against it) + (a matrix beside a file is checked against it, + a fixes file against the file it names) sentier-diagnostics build DESCRIPTION --out FILE sentier-diagnostics summary FILE sentier-diagnostics page FILE --out FOLDER @@ -14,7 +15,7 @@ import sys from pathlib import Path -from . import __version__, breakdown, chain, example, fileformat, from_tables, matrix, page, summary +from . import __version__, breakdown, chain, example, fileformat, fixes, from_tables, matrix, page, summary def check(paths) -> int: @@ -30,6 +31,15 @@ def check(paths) -> int: errors = breakdown.errors(data) elif isinstance(data, dict) and str(data.get("format", "")).startswith(matrix.FAMILY): errors = matrix.errors(data) # against its file when the file is checked + elif isinstance(data, dict) and str(data.get("format", "")).startswith(fixes.FAMILY): + errors = fixes.errors(data) + size = Path(path).stat().st_size + if size > fixes.MAX_BYTES: + errors.append(f"{size} bytes, and a page loads {fixes.MAX_BYTES} at most") + if not errors: + errors = fixes.beside(data, Path(path).parent) + if not errors: + print(f"note {path}: {fixes.size_note(data)}") else: errors = fileformat.errors(data) size = Path(path).stat().st_size @@ -68,7 +78,7 @@ def main(argv=None) -> int: parser = argparse.ArgumentParser(prog="sentier-diagnostics", description=__doc__.splitlines()[0]) parser.add_argument("--version", action="version", version=f"sentier-diagnostics {__version__}") commands = parser.add_subparsers(dest="command", required=True) - one = commands.add_parser("check", help="check diagnostics and breakdown files") + one = commands.add_parser("check", help="check diagnostics, breakdown, matrix and fixes files") one.add_argument("files", nargs="+", type=Path) one = commands.add_parser("build", help="build a diagnostics file from CSV tables") one.add_argument("description", type=Path) diff --git a/src/sentier_diagnostics/fixes.py b/src/sentier_diagnostics/fixes.py new file mode 100644 index 0000000..ad7652a --- /dev/null +++ b/src/sentier_diagnostics/fixes.py @@ -0,0 +1,1010 @@ +"""Fixes against one base, `sentier-diagnostics-fixes/1` (SPEC 16). + +A chain measures each step with every earlier step applied, so a step's difference belongs +to its place in the order. A fixes file measures each fix on its own against one state of +a chain, the base: a fix has one effect, whichever set it joins and in whatever order. A +reader may then choose any set of fixes; its result is the base plus the sum of their +effects, labelled: + + built one fix, or a set the producer built (`built`) + exact interactions were checked, and no two of its fixes are a listed pair + estimated otherwise: drawn, and marked on every cell and count + +A set has no verdict: each claim is judged in its own fix, against the base. A set's +tolerance is the largest of its fixes'; its noise is the sum of their noise, with the +number of fixes whose noise was measured, never read as none. + +`errors(f)` checks a file on its own: its shape, every fix's counts and verdicts worked out +again from its effect and the base, the pairs of fixes that can interact, and every built +set where the file calls its fixes independent: the built results must agree with the sum. +`against(f, data, full)` holds it to its diagnostics file and that file's matrix; +`beside(f, folder)` finds them next to it, as `sentier-diagnostics check` does. + +`build(...)` makes a file from a checked diagnostics file, its matrix and the results built +with each fix; `label`, `compose` and `counts` give what a page draws for a chosen set. +""" + +from __future__ import annotations + +import json +import re +from collections.abc import Iterable, Mapping +from pathlib import Path + +from . import __version__, chain, claims, fileformat, matrix, rule + +FORMAT = "sentier-diagnostics-fixes/1" +FAMILY = "sentier-diagnostics-fixes/" +KEYS = { + "format", "of", "title", "about", "item", "indicators", "rule", "items", "base", "fixes", + "interactions", "built", "notes", "made_by", +} # fmt: skip +#: A key a file may leave out, as in the diagnostics file. +OPTIONAL_KEYS = {"indicator"} +OF_KEYS = {"file", "title", "base"} +#: The rule of the diagnostics file, without the chain's own total tolerance. +RULE_KEYS = fileformat.RULE_KEYS - {"total_tolerance"} +FIX_KEYS = {"id", "label", "about", "kind", "explained", "link", "claim", "tolerance", "noise", "effect", + "counts"} # fmt: skip +INTERACTION_KEYS = {"checked", "method", "by", "pairs"} +PAIR_KEYS = {"fixes", "reason", "largest"} +BUILT_KEYS = {"id", "label", "fixes", "results", "noise"} + +MAX_FIXES, MAX_BUILT = 1000, 500 +MAX_ITEMS, MAX_BYTES = matrix.MAX_ITEMS, matrix.MAX_BYTES + +BUILT, EXACT, ESTIMATED = "built", "exact", "estimated" +LABELS = (BUILT, EXACT, ESTIMATED) + +#: The shape of an identifier of a model (a UUID, or 32 hexadecimal digits). The reasons of +#: interactions say what two fixes share in words, never by the model's identifiers (SPEC 16.6). +IDENTIFIER = re.compile( + r"\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b|\b[0-9a-fA-F]{32}\b" +) + + +# ------------------------------------------------------------------------ making a file + + +def fix(id: str, label: str, *, kind: str, explained: bool = True, about: str | None = None, + link: str | None = None, claim: dict | None = None, tolerance: dict | None = None, + noise: dict | None = None) -> dict: # fmt: skip + """One fix, as `build` takes it: what a step has (`chain.step`), measured against the base.""" + return chain.step(id, label, kind=kind, explained=explained, about=about, link=link, claim=claim, + tolerance=tolerance, noise=noise) # fmt: skip + + +def interactions(checked: bool = False, *, method: str | None = None, by: dict | None = None, + pairs: Iterable[tuple] = ()) -> dict: # fmt: skip + """The check of pairs: `checked` true where every pair was examined, false where none was. + `pairs` are `(fix, fix, reason)` or `(fix, fix, reason, largest)`, every pair that can + interact, each once; `largest` is the measured size of its cross term, None where it was + not measured.""" + out = [] + for a, b, reason, *largest in pairs: + out.append({"fixes": [a, b], "reason": reason, "largest": largest[0] if largest else None}) + return {"checked": checked, "method": method, "by": by, "pairs": out} + + +def built_set(id: str, label: str, fixes: Iterable[str], results: Mapping, noise: dict | None = None) -> dict: + """A set the producer built: `results` are `{indicator: {item: result or None}}`, an item + left out being the base result.""" + return {"id": id, "label": label, "fixes": list(fixes), "results": results, "noise": noise or chain.noise()} + + +def build( + title: str, + *, + data: Mapping, + full: Mapping, + base: str, + fixes: list[dict], + results: Mapping[str, Mapping], + interactions: dict | None = None, + built: Iterable[dict] = (), + file: str = chain.FILE, + about: str | None = None, + notes: Iterable[str] = (), +) -> dict: + """A fixes file, counted and checked. + + - `data` and `full`: the checked diagnostics file and its matrix; the items and the base + results come from the matrix, in its order. + - `base`: the id of the state every fix was measured against. + - `fixes`: `fix(...)`, in the order the page lists them. + - `results`: `{fix id: {indicator: {item: result or None}}}`, built with that fix alone. + An item left out is the base result; one written equal to it is left out too. + - `interactions`: `interactions(...)`; not checked where it is None. + - `built`: `built_set(...)`. + - `file`: the name of the diagnostics file beside it. + """ + ids = [i["id"] for i in data["indicators"]] + states = [s["id"] for s in data["states"]] + if base not in states: + raise ValueError(f"the base {base!r} is no state of {data['title']!r}") + at, nstates = states.index(base), len(states) + rows = [list(r) for r in full["items"]] + index = {r[0]: k for k, r in enumerate(rows)} + base_values = {ind: [v[x * nstates + at] for v in full["values"]] for x, ind in enumerate(ids)} + + def sparse(given: Mapping, where: str) -> dict: + unknown = set(given) - set(ids) + if unknown: + raise ValueError(f"{where}: results of an indicator the file does not hold: {sorted(unknown)}") + out = {} + for ind in ids: + entries = [] + for item, value in (given.get(ind) or {}).items(): + if item not in index: + raise ValueError(f"{where}: {item!r} is no item of the base") + if value is not None and not rule.is_value(value): + raise ValueError(f"{where}, {ind}, {item}: {value!r} is not a finite number or None") + value = None if value is None else float(value) + if value != base_values[ind][index[item]]: + entries.append([index[item], value]) + out[ind] = sorted(entries) + return out + + known = {one["id"] for one in fixes} + unknown = set(results) - known + if unknown: + raise ValueError(f"results of fixes that are not given: {sorted(unknown)}") + f = { + "format": FORMAT, + "of": {"file": file, "title": data["title"], "base": base}, + "title": title, + "about": about, + "item": dict(data["item"]), + **({"indicator": dict(data["indicator"])} if data.get("indicator") else {}), + "indicators": [dict(i) for i in data["indicators"]], + "rule": {k: data["rule"][k] for k in sorted(RULE_KEYS)}, + "items": rows, + "base": base_values, + "fixes": [], + "interactions": interactions or globals()["interactions"](), + "built": [], + "notes": list(notes), + "made_by": f"sentier-diagnostics {__version__}", + } + for one in fixes: + found_claim = one.get("claim") + f["fixes"].append({ + "id": one["id"], + "label": one["label"], + "about": one.get("about"), + "kind": one["kind"], + "explained": bool(one.get("explained", True)), + "link": one.get("link"), + "claim": claims.published(found_claim, True) if found_claim else None, + "tolerance": one.get("tolerance") or chain.tolerance(), + "noise": one.get("noise") or chain.noise(), + "effect": sparse(results.get(one["id"]) or {}, f"fix {one['id']!r}"), + "counts": None, + }) # fmt: skip + for one in built: + f["built"].append({ + "id": one["id"], + "label": one["label"], + "fixes": list(one["fixes"]), + "results": sparse(one["results"], f"built set {one['id']!r}"), + "noise": one.get("noise") or chain.noise(), + }) # fmt: skip + base_ctx = _Base(f) + for one in f["fixes"]: + one["counts"] = _fix_counts(base_ctx, one, _columns(one["effect"])) + found = errors(f) or against(f, data, full) + if found: + raise ValueError("not a fixes file: " + "; ".join(found[:10])) + return f + + +# ------------------------------------------------------------------------ text and files + + +def _compact(value) -> str: + return json.dumps(value, ensure_ascii=False, allow_nan=False, separators=(",", ":")) + + +def dumps(f: Mapping) -> str: + """The text of a fixes file: one line per item, per base column, per fix and per built set, + so that a diff names them.""" + parts = [] + for key, value in f.items(): + if key in ("items", "fixes", "built") and value: + text = "[\n" + ",\n".join(" " + _compact(v) for v in value) + "\n ]" + elif key == "base" and value: + text = "{\n" + ",\n".join(f" {_compact(k)}:{_compact(v)}" for k, v in value.items()) + "\n }" + else: + text = json.dumps(value, ensure_ascii=False, allow_nan=False) + parts.append(f" {_compact(key)}: {text}") + return "{\n" + ",\n".join(parts) + "\n}\n" + + +def write(f: Mapping, path) -> Path: + """Write a fixes file, checked first; refuse one larger than a page loads.""" + found = errors(f) + if found: + raise ValueError("not a fixes file: " + "; ".join(found[:10])) + text = dumps(f) + size = len(text.encode("utf-8")) + if size > MAX_BYTES: + raise ValueError(f"the fixes file would be {size} bytes, and a page loads {MAX_BYTES} at most") + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + return path + + +def read(path) -> dict: + """A fixes file, read and checked, with the diagnostics file (and its matrix) beside it; + raises ValueError with every error found.""" + path = Path(path) + if path.stat().st_size > MAX_BYTES: + raise ValueError(f"{path}: larger than {MAX_BYTES} bytes") + f = fileformat.load_json(path) + found = errors(f) or beside(f, path.parent) + if found: + raise ValueError(f"{path}: " + "; ".join(found[:10])) + return f + + +def beside(f: Mapping, folder) -> list[str]: + """Everything a checked fixes file and the diagnostics file it names, in `folder`, + disagree on; that file's matrix is used where it is there too.""" + path = Path(folder) / f["of"]["file"] + if not path.is_file(): + return [f"`of` names the diagnostics file {f['of']['file']}, which is not beside it"] + try: + data = fileformat.load_json(path) + except ValueError as error: + return [str(error)] + if fileformat.errors(data): + return [f"{f['of']['file']} is not a diagnostics file a page shows: check it on its own"] + full = None + named = data.get("matrix") + if named and (path.parent / named["file"]).is_file(): + try: + full = matrix.read(path.parent / named["file"], data) + except ValueError as error: + return [str(error)] + return against(f, data, full) + + +def entries_of(f: Mapping) -> list[tuple[str, int]]: + """How many entries each fix's effect holds, the largest first: what the file's size is + made of. A fix to something every item uses holds about one entry per item and indicator.""" + found = [(one["id"], sum(len(v) for v in one["effect"].values())) for one in f["fixes"]] + return sorted(found, key=lambda x: (-x[1], x[0])) + + +def size_note(f: Mapping) -> str: + """One line on what a checked file holds, for `sentier-diagnostics check`.""" + found = entries_of(f) + total = sum(n for _, n in found) + sum(len(v) for b in f["built"] for v in b["results"].values()) + largest = ", ".join(f"{name!r} {n}" for name, n in found[:3]) + return (f"{len(found)} fixes and {len(f['built'])} built sets hold {total} entries over " + f"{len(f['items'])} items; the largest fixes: {largest}") # fmt: skip + + +# ------------------------------------------------------------------------ the base, worked out once + + +class _Base: + """What every fix of a checked file is judged against: the base results, the floor of + every item, and the class of every item against itself (agrees, near zero or missing).""" + + def __init__(self, f: Mapping): + self.ids = [i["id"] for i in f["indicators"]] + self.items = [r[0] for r in f["items"]] + self.index = {item: k for k, item in enumerate(self.items)} + table = {(x["indicator"], x["unit"]): x["value"] for x in f["rule"]["floors"]} + self.floor = {ind: [table[(ind, r[4])] for r in f["items"]] for ind in self.ids} + self.values = f["base"] + self.kinds = { + ind: [rule.classify(v, v, floor=fl) for v, fl in zip(self.values[ind], self.floor[ind])] + for ind in self.ids + } + self.tally = {} + for ind in self.ids: + tally = dict.fromkeys((rule.AGREES, rule.NEAR_ZERO, rule.MISSING), 0) + for kind in self.kinds[ind]: + tally[kind] += 1 + self.tally[ind] = tally + + +def _columns(sparse: Mapping) -> dict[str, dict[int, float | None]]: + """`{indicator: [[item index, result], ...]}` as `{indicator: {item index: result}}`.""" + return {ind: {k: v for k, v in entries} for ind, entries in sparse.items()} + + +def _level(noise: Mapping, indicator: str) -> float | None: + return (noise["by"] or {}).get(indicator) if noise["measured"] else None + + +def _side(before, after) -> str: + return "both" if before is None and after is None else "before" if before is None else "after" + + +def _block(base: _Base, ind: str, changed: Mapping[int, float | None], tolerance: float, + noise: float | None, claim: Mapping | None) -> dict: # fmt: skip + """The counts of one indicator: every item, the base against the base with `changed` + applied, judged by the rule with `tolerance` and `noise`, and against `claim` where there + is one. Only the changed items are judged one by one: every other pair is the base + against itself, counted from the base's own classes.""" + classes = dict.fromkeys(rule.CLASSES, 0) + missing = dict.fromkeys(fileformat.MISSING_SIDES, 0) + ways = dict.fromkeys(rule.DIRECTIONS, 0) + bands = dict.fromkeys(rule.BANDS, 0) + verdicts = dict.fromkeys(claims.VERDICTS, 0) if claim is not None else None + values, floors, kinds = base.values[ind], base.floor[ind], base.kinds[ind] + for kind, n in base.tally[ind].items(): + classes[kind] += n + missing["both"] += base.tally[ind][rule.MISSING] + entries = [] + if claim is not None: + entries = [(e, None if e["items"] == "*" else set(e["items"])) for e in claims.entries_for(claim, ind)] + for k, after in changed.items(): + before, was = values[k], kinds[k] + classes[was] -= 1 + if was == rule.MISSING: + missing["both"] -= 1 + kind, way, band = rule.judge(before, after, floor=floors[k], tolerance=tolerance, noise=noise) + classes[kind] += 1 + if kind == rule.MISSING: + missing[_side(before, after)] += 1 + if way: + ways[way] += 1 + if band: + bands[band] += 1 + if verdicts is not None: + item = base.items[k] + covering = [e for e, listed in entries if listed is None or item in listed] + verdicts[claims.verdict(kind, before, after, way, covering)] += 1 + if verdicts is not None: + # the pairs no change touched: the base against itself + musts = [listed for e, listed in entries if e["expect"] == claims.MUST] + every = any(listed is None for listed in musts) + must_items = set() if every else {base.index[i] for listed in musts for i in listed} + for kind in (rule.AGREES, rule.NEAR_ZERO, rule.MISSING): + untouched = base.tally[ind][kind] - sum(1 for k in changed if kinds[k] == kind) + if kind == rule.MISSING: + verdicts[claims.NOT_JUDGED] += untouched + continue + held = untouched if every else sum(1 for k in must_items if kinds[k] == kind and k not in changed) + verdicts[claims.NOT_MOVED] += held + verdicts[claims.UNCHANGED if kind == rule.AGREES else claims.NOT_JUDGED] += untouched - held + return {"pairs": len(values), "classes": classes, "missing": missing, "directions": ways, "bands": bands, + "verdicts": verdicts, "closer": None, "further": None} # fmt: skip + + +def _fix_counts(base: _Base, one: Mapping, effect: Mapping[str, Mapping[int, float | None]]) -> dict: + """A fix's counts: the base against the base with its effect, with its own tolerance, + noise and claim.""" + tol = one["tolerance"]["value"] + return {ind: _block(base, ind, effect[ind], tol, _level(one["noise"], ind), one["claim"]) for ind in base.ids} + + +def _sum(before: float | None, results: list[float | None]) -> tuple[float | None, bool]: + """The result of a set at one item: the base plus the effects of the fixes that change it, + and whether a sum could be formed. A fix without a result leaves none; a base without a + result leaves nothing to add two fixes' differences to. + + >>> _sum(1.0, [1.5]), _sum(1.0, [1.5, 2.0]), _sum(1.0, [1.5, None]), _sum(None, [1.0, 2.0]) + ((1.5, True), (2.5, True), (None, True), (None, False)) + """ + if len(results) == 1: + return results[0], True + if any(r is None for r in results): + return None, True + if before is None: + return None, False + return before + sum(r - before for r in results), True + + +def _summed(base: _Base, effects: list[Mapping], ind: str) -> dict[int, tuple[float | None, bool]]: + """The items the chosen fixes change in one indicator, with the set's result at each.""" + touched: dict[int, list] = {} + for effect in effects: + for k, value in effect[ind].items(): + touched.setdefault(k, []).append(value) + return {k: _sum(base.values[ind][k], found) for k, found in touched.items()} + + +# ------------------------------------------------------------------------ a chosen set + + +def _chosen(f: Mapping, ids: Iterable[str]) -> list[dict]: + order = {one["id"]: k for k, one in enumerate(f["fixes"])} + ids = list(ids) + if not ids: + raise ValueError("a set holds at least one fix") + for one in ids: + if one not in order: + raise ValueError(f"the file holds no fix {one!r}") + if len(set(ids)) != len(ids): + raise ValueError("a set holds each fix once") + return [f["fixes"][k] for k in sorted(order[i] for i in ids)] + + +def held_pairs(f: Mapping, ids: Iterable[str]) -> list[dict]: + """The listed pairs of a set: two of its fixes that can interact, with the reason.""" + chosen = {one["id"] for one in _chosen(f, ids)} + return [p for p in f["interactions"]["pairs"] if set(p["fixes"]) <= chosen] + + +def built_of(f: Mapping, ids: Iterable[str]) -> dict | None: + """The built set that holds exactly these fixes, or None.""" + chosen = {one["id"] for one in _chosen(f, ids)} + return next((b for b in f["built"] if set(b["fixes"]) == chosen), None) + + +def label(f: Mapping, ids: Iterable[str]) -> str: + """Built, exact or estimated (SPEC 16.5).""" + chosen = _chosen(f, ids) + if len(chosen) == 1 or built_of(f, ids) is not None: + return BUILT + if f["interactions"]["checked"] and not held_pairs(f, ids): + return EXACT + return ESTIMATED + + +def pair_terms(f: Mapping, ids: Iterable[str]) -> dict: + """How far an estimated set may lie from its sum, to first order: `{value, measured, + pairs}`, the value being the sum of the measured sizes of the listed pairs it holds, and + None where some were not measured (a pair not measured is never a pair of size 0).""" + held = held_pairs(f, ids) + sizes = [p["largest"] for p in held if p["largest"] is not None] + value = sum(sizes) if len(sizes) == len(held) else None + return {"value": value, "measured": len(sizes), "pairs": len(held)} + + +def tolerance_of(f: Mapping, ids: Iterable[str]) -> float: + """The tolerance of a set: the largest of its fixes'.""" + return max(one["tolerance"]["value"] for one in _chosen(f, ids)) + + +def noise_of(f: Mapping, ids: Iterable[str]) -> dict[str, dict]: + """The noise of a set, per indicator: `{value, measured, fixes}`, the value being the sum + of the noise of the fixes whose noise was measured there, and None where none was: not + measured is never none.""" + chosen = _chosen(f, ids) + out = {} + for ind in (i["id"] for i in f["indicators"]): + levels = [lv for lv in (_level(one["noise"], ind) for one in chosen) if lv is not None] + out[ind] = {"value": sum(levels) if levels else None, "measured": len(levels), "fixes": len(chosen)} + return out + + +def _changed(f: Mapping, base: _Base, ids) -> tuple[dict, dict]: + """The items a set changes, per indicator, with its result at each; and the items where + no sum can be formed (a missing base under two fixes or more).""" + chosen = _chosen(f, ids) + found = built_of(f, ids) + if len(chosen) == 1 or found is not None: + source = chosen[0]["effect"] if found is None else found["results"] + return _columns(source), {ind: [] for ind in base.ids} + effects = [_columns(one["effect"]) for one in chosen] + changed, cannot = {}, {} + for ind in base.ids: + cells = _summed(base, effects, ind) + changed[ind] = {k: v for k, (v, _) in cells.items()} + cannot[ind] = sorted(k for k, (_, ok) in cells.items() if not ok) + return changed, cannot + + +def compose(f: Mapping, ids: Iterable[str]) -> dict: + """What a page draws for a set: its fixes in the order of the file (each with its own + claim and counts), its label, the listed pairs it holds, its tolerance and noise, and its + result for every item, `{indicator: [result or None, ...]}` in the order of `items`. + `cannot_add` names, per indicator, the item indices whose result is None because the base + has none to add two fixes' differences to, so that it reads apart from a missing result. + A set has no verdict.""" + chosen = _chosen(f, ids) + names = [one["id"] for one in chosen] + base = _Base(f) + changed, cannot = _changed(f, base, names) + results = {} + for ind in base.ids: + column = list(base.values[ind]) + for k, value in changed[ind].items(): + column[k] = value + results[ind] = column + return { + "fixes": [{k: one[k] for k in ("id", "label", "claim", "counts")} for one in chosen], + "label": label(f, names), + "pairs": held_pairs(f, names), + "pair_terms": pair_terms(f, names), + "tolerance": tolerance_of(f, names), + "noise": noise_of(f, names), + "results": results, + "cannot_add": cannot, + } + + +def counts(f: Mapping, ids: Iterable[str]) -> dict: + """The counts of a set against the base, per indicator, with its label beside them: an + estimated set's counts are estimates ("estimated: 153 would differ"). No verdicts.""" + names = [one["id"] for one in _chosen(f, ids)] + base = _Base(f) + changed, _ = _changed(f, base, names) + tol, noise = tolerance_of(f, names), noise_of(f, names) + return { + "label": label(f, names), + "by": {ind: _block(base, ind, changed[ind], tol, noise[ind]["value"], None) for ind in base.ids}, + } + + +# ------------------------------------------------------------------------ the check + + +def errors(f) -> list[str]: + """Everything wrong with a fixes file on its own; nothing for one a page can load. It + never raises: a value it cannot read is a fault like any other.""" + found: list[str] = [] + try: + return _errors(f, found) + except Exception as error: # noqa: BLE001 - the backstop of a check that must not raise + return [*found, f"{fileformat.STOPPED} ({type(error).__name__}: {str(error)[:200]})"] + + +def _errors(f, found: list[str]) -> list[str]: + if not isinstance(f, Mapping): + return ["not a fixes file: not an object"] + said = f.get("format") + if said != FORMAT: + if isinstance(said, str) and said.startswith(FAMILY): + return [f"format {said!r}: this reader knows {FORMAT} only"] + return [f"not a fixes file of format {FORMAT}"] + found += [f"a key this format does not know: {key}" for key in sorted(set(f) - KEYS - OPTIONAL_KEYS)] + found += [f"a key this format requires: {key}" for key in sorted(KEYS - set(f))] + if found: + return found + of = f["of"] + if not ( + isinstance(of, dict) + and set(of) == OF_KEYS + and isinstance(of["file"], str) + and fileformat.FILE_NAME.fullmatch(of["file"]) + and fileformat.is_text(of["title"], 120) + and fileformat.is_text(of["base"], fileformat.MAX_ID) + ): + found.append("`of` is not {file, title, base}: a file name, that file's title and a state's id") + if not fileformat.is_text(f["title"], 120) or not fileformat.is_optional(f["about"]): + found.append("`title` is not a short text, or `about` not a text") + for key in ("item", "indicator"): + word = f.get(key) + if (word is not None or key == "item") and not ( + isinstance(word, dict) and set(word) == {"one", "many"} + and all(fileformat.is_text(w, 30) for w in word.values()) + ): # fmt: skip + found.append(f"`{key}` is not {{one, many}}") + if not fileformat.is_text(f["made_by"], 60): + found.append("`made_by` is not a short text") + notes = f["notes"] + if not isinstance(notes, list) or len(notes) > fileformat.MAX_NOTES or not all(map(fileformat.is_text, notes)): + found.append(f"`notes` is not a list of at most {fileformat.MAX_NOTES} texts") + indicators = fileformat._indicators(f, found) + floors = _rule(f, found, indicators) + if not indicators or floors is None or not _items(f, found, indicators, floors): + return found + n = len(f["items"]) + said_base = f["base"] + if not ( + isinstance(said_base, dict) + and set(said_base) == set(indicators) + and all(isinstance(said_base[i], list) and len(said_base[i]) == n + and all(v is None or rule.is_value(v) for v in said_base[i]) for i in indicators) + ): # fmt: skip + found.append("`base` is not one result or null per item, for every indicator") + if found: + return found # nothing can be judged against a base that cannot be read + base = _Base(f) + effects = _fixes(f, found, base) + if effects is None: + return found + pairs = _interactions(f, found, effects) + if pairs is not None: + _built(f, found, base, effects, pairs) + return found + + +def _rule(f, found, indicators) -> dict | None: + said = f["rule"] + if not isinstance(said, dict) or set(said) != RULE_KEYS: + found.append(f"`rule` is not {{{', '.join(sorted(RULE_KEYS))}}}") + return None + if not fileformat.is_share(said["floor_factor"]): + found.append("`rule`: the floor factor is not a share") + if not fileformat.is_text(said["floors_from"], fileformat.MAX_ID): + found.append("`rule`: the floors come from a state, named by its id") + if said["beyond"] != rule.BEYOND_FACTOR: + found.append(f"`rule.beyond` is {rule.BEYOND_FACTOR:g}: one rule for every file") + if not isinstance(said["floors"], list): + found.append("`rule.floors` is not a list") + return None + table = {} + for one in said["floors"]: + if not ( + isinstance(one, dict) + and set(one) == fileformat.FLOOR_KEYS + and one["indicator"] in indicators + and fileformat.is_optional(one["unit"], fileformat.MAX_WORD) + and fileformat.is_share(one["value"]) + and one["basis"] in fileformat.FLOOR_BASES + and fileformat.is_count(one["n"]) + ): + found.append("a floor is not {indicator, unit, value, basis, n} of a known indicator") + return None + key = (one["indicator"], one["unit"]) + if key in table: + found.append(f"two floors for {one['indicator']!r} in unit {one['unit']!r}") + table[key] = one["value"] + return table + + +def _items(f, found, indicators, floors) -> bool: + rows = f["items"] + if not isinstance(rows, list) or not 0 < len(rows) <= MAX_ITEMS: + found.append(f"`items` is not a list of 1 to {MAX_ITEMS} items") + return False + seen = set() + for at, row in enumerate(rows): + if not matrix._is_row(row): + found.append(f"item {at}: not [id, name, place, group, unit]") + return False + if row[0] in seen: + found.append(f"item {row[0]!r} is in the file twice") + return False + seen.add(row[0]) + units = {row[4] for row in rows} + for ind in indicators: + for unit in sorted(units, key=lambda u: (u is None, u or "")): + if (ind, unit) not in floors: + found.append(f"no floor for {ind!r} in unit {unit!r}") + return False + return True + + +def _sparse(said, where: str, base: _Base, found: list[str]) -> dict | None: + """An effect, or the results of a built set: per indicator, the items whose result is not + the base result, `[[item index, result or null], ...]`.""" + if not isinstance(said, dict) or set(said) != set(base.ids): + found.append(f"{where}: not one list per indicator of the file") + return None + n = len(base.items) + out = {} + for ind in base.ids: + entries = said[ind] + if not isinstance(entries, list) or len(entries) > n: + found.append(f"{where}, {ind}: not a list of at most one entry per item") + return None + column = {} + for entry in entries: + if not ( + isinstance(entry, list) + and len(entry) == 2 + and fileformat.is_count(entry[0]) + and (entry[1] is None or rule.is_value(entry[1])) + ): + found.append(f"{where}, {ind}: an entry is not [item index, result or null]") + return None + k, value = entry + if k >= n: + found.append(f"{where}, {ind}: names item {k}, and the file holds {n}") + return None + if k in column: + found.append(f"{where}, {ind}: holds item {base.items[k]!r} twice") + return None + if value == base.values[ind][k]: + found.append(f"{where}, {ind}: item {base.items[k]!r} is written with the base result, as if it changed") + return None + column[k] = value + out[ind] = column + return out + + +def _noise_errors(noise, where: str, indicators) -> list[str]: + if not ( + isinstance(noise, dict) + and set(noise) == {"measured", "between", "by", "note"} + and isinstance(noise["measured"], bool) + and fileformat.is_optional(noise["note"]) + ): + return [f"{where}: `noise` is not {{measured, between, by, note}}"] + if noise["measured"]: + if not ( + isinstance(noise["between"], list) + and len(noise["between"]) == 2 + and all(fileformat.is_text(n, fileformat.MAX_LABEL) for n in noise["between"]) + and isinstance(noise["by"], dict) + and set(noise["by"]) <= set(indicators) + and all(fileformat.is_share(v) for v in noise["by"].values()) + ): + return [f"{where}: measured noise names the two builds and a size per indicator"] + elif noise["between"] is not None or noise["by"] is not None: + return [f"{where}: noise that was not measured has no builds and no sizes"] + return [] + + +def _fixes(f, found, base: _Base) -> dict | None: + """Every fix, its counts and verdicts worked out again; its effect, by id.""" + said = f["fixes"] + if not isinstance(said, list) or not 0 < len(said) <= MAX_FIXES: + found.append(f"`fixes` is not a list of 1 to {MAX_FIXES} fixes") + return None + effects = {} + for one in said: + if not isinstance(one, dict) or set(one) != FIX_KEYS or not fileformat.is_text(one["id"], fileformat.MAX_ID): + found.append(f"a fix is not {{{', '.join(sorted(FIX_KEYS))}}}") + return None + where = f"fix {one['id']!r}" + if one["id"] in effects: + found.append(f"two fixes have the id {one['id']!r}") + return None + if not fileformat.is_text(one["label"], fileformat.MAX_LABEL) or not fileformat.is_optional(one["about"]): + found.append(f"{where}: `label` or `about` is not a text") + if not fileformat.is_text(one["kind"], fileformat.MAX_WORD) or not isinstance(one["explained"], bool): + found.append(f"{where}: a fix has a `kind` and says whether a cause explains it") + if not fileformat.is_link(one["link"]): + found.append(f"{where}: `link` is not an https address") + if one["claim"] is not None: + shape = fileformat.claim_errors(one["claim"], where, base.ids, "items") + if shape: + found += shape + return None + if any(e["items"] is None for e in one["claim"]["scope"]): + found.append(f"{where}: a fix's claim names its items: a page works out each item's verdict from it") + return None + unknown = {i for e in one["claim"]["scope"] if e["items"] != "*" for i in e["items"]} - set(base.index) + if unknown: + found.append(f"{where}: its claim names {len(unknown)} items the file does not hold") + return None + tol = one["tolerance"] + if not ( + isinstance(tol, dict) + and set(tol) == {"value", "basis", "note"} + and fileformat.is_share(tol["value"]) + and tol["basis"] in fileformat.TOLERANCE_BASES + and fileformat.is_optional(tol["note"]) + ): + found.append(f"{where}: `tolerance` is not {{value, basis, note}}") + return None + noise = _noise_errors(one["noise"], where, base.ids) + if noise: + found += noise + return None + effect = _sparse(one["effect"], f"{where}: its effect", base, found) + if effect is None: + return None + effects[one["id"]] = effect + written = one["counts"] + if not isinstance(written, dict) or set(written) != set(base.ids): + found.append(f"{where}: `counts` is not one block per indicator") + return None + shape = [e for ind in base.ids for e in fileformat._block(written[ind], f"{where}, counts of {ind!r}", one)] + shape += [f"{where}, counts of {ind!r}: `{key}` is null: a fix has no published results to come closer to" + for ind in base.ids for key in ("closer", "further") if written[ind][key] is not None] # fmt: skip + if shape: + found += shape + continue + again = _fix_counts(base, one, effect) + for ind in base.ids: + for key, value in again[ind].items(): + if written[ind][key] != value: + found.append(f"{where}, counts of {ind!r}: `{key}` is not what its effect and the base give") + return effects + + +def _interactions(f, found, effects: Mapping) -> set[frozenset] | None: + """The pairs of fixes that can interact, each once, as sets of two ids.""" + said = f["interactions"] + if not ( + isinstance(said, dict) + and set(said) == INTERACTION_KEYS + and isinstance(said["checked"], bool) + and isinstance(said["pairs"], list) + ): + found.append("`interactions` is not {checked, method, by, pairs}") + return None + if said["checked"]: + if not fileformat.is_text(said["method"]): + found.append("`interactions`: a check says what it did (`method`)") + elif IDENTIFIER.search(said["method"]): + found.append("`interactions.method` holds an identifier of the model: say it in words") + if said["by"] is None: + found.append("`interactions`: a check says who or what ran it (`by`)") + else: + found += fileformat.origin_errors(said["by"], "`interactions`") + else: + if said["method"] is not None or said["by"] is not None: + found.append("`interactions`: no check ran, so it has no `method` and no `by`") + if said["pairs"]: + found.append( + "`interactions`: pairs are listed and `checked` is false: a pair that is not listed would " + "read as cleared" + ) + return None + if len(said["pairs"]) > MAX_FIXES * (MAX_FIXES - 1) // 2: + found.append("`interactions.pairs` lists more pairs than fixes can make") + return None + pairs: set[frozenset] = set() + for one in said["pairs"]: + if not isinstance(one, dict) or set(one) != PAIR_KEYS: + found.append( + "a pair is not {fixes, reason, largest}: it says why in words, not what in the model they share" + ) + return None + named = one["fixes"] + if not (isinstance(named, list) and len(named) == 2 and all(isinstance(x, str) for x in named)): + found.append("a pair does not name two fixes") + return None + where = f"the pair {named[0]!r} and {named[1]!r}" + unknown = [x for x in named if x not in effects] + if unknown: + found.append(f"{where}: the file holds no fix {unknown[0]!r}") + continue + if named[0] == named[1]: + found.append(f"{where}: names one fix twice") + continue + if one["largest"] is not None and not fileformat.is_share(one["largest"]): + found.append(f"{where}: `largest` is not a share, or null where it was not measured") + if not fileformat.is_text(one["reason"], 300): + found.append(f"{where}: its reason is not a short text") + elif IDENTIFIER.search(one["reason"]): + found.append(f"{where}: its reason holds an identifier of the model: say what they share in words") + key = frozenset(named) + if key in pairs: + found.append(f"{where}: is listed twice") + pairs.add(key) + return pairs + + +def _sizes(f) -> dict[frozenset, float]: + """The measured size of every listed pair that has one.""" + return {frozenset(p["fixes"]): p["largest"] for p in f["interactions"]["pairs"] if p["largest"] is not None} + + +def _built(f, found, base: _Base, effects: Mapping, pairs: set[frozenset]) -> None: + said = f["built"] + if not isinstance(said, list) or len(said) > MAX_BUILT: + found.append(f"`built` is not a list of at most {MAX_BUILT} sets") + return + checked = f["interactions"]["checked"] + sizes = _sizes(f) + by_id = {one["id"]: one for one in f["fixes"]} + ids, sets = set(), set() + for one in said: + if not isinstance(one, dict) or set(one) != BUILT_KEYS or not fileformat.is_text(one["id"], fileformat.MAX_ID): + found.append(f"a built set is not {{{', '.join(sorted(BUILT_KEYS))}}}") + return + where = f"built set {one['id']!r}" + if one["id"] in ids: + found.append(f"two built sets have the id {one['id']!r}") + ids.add(one["id"]) + if not fileformat.is_text(one["label"], fileformat.MAX_LABEL): + found.append(f"{where}: `label` is not a short text") + named = one["fixes"] + if not (isinstance(named, list) and all(isinstance(x, str) for x in named)): + found.append(f"{where}: `fixes` is not a list of the ids of fixes") + continue + unknown = [x for x in named if x not in effects] + if unknown: + found.append(f"{where}: the file holds no fix {unknown[0]!r}") + continue + if len(set(named)) != len(named): + found.append(f"{where}: holds a fix twice") + continue + if len(named) < 2: + found.append(f"{where}: holds fewer than two fixes: one fix is built already") + continue + key = frozenset(named) + if key in sets: + found.append(f"{where}: holds the same fixes as another built set") + sets.add(key) + noise = _noise_errors(one["noise"], where, base.ids) + if noise: + found += noise + continue + results = _sparse(one["results"], f"{where}: its results", base, found) + if results is None or not checked: + continue + chosen = [by_id[x] for x in named] + if not any(p <= key for p in pairs): + found += _disagreements(base, where, chosen, [effects[x] for x in named], one, results) + elif key in sizes: # one listed pair, built: its measured size is held to the build + found += _disagreements(base, where, chosen, [effects[x] for x in named], one, results, sizes[key]) + + +#: How many disagreements of one built set are named before the rest are counted. +NAMED = 5 + + +def _disagreements(base: _Base, where: str, chosen: list, effects: list, one: Mapping, results: Mapping, + largest: float | None = None) -> list[str]: # fmt: skip + """Where a built set is further from the sum of its effects than the file says: by the + rule, with the largest tolerance of its fixes and the sum of their noise and its own. A + set the file calls independent must be the sum; a built pair whose cross term the file + measured (`largest`) must lie within it.""" + tol = max(x["tolerance"]["value"] for x in chosen) + if largest is not None: + tol = max(tol, largest) + found, more = [], 0 + for ind in base.ids: + levels = [lv for lv in (_level(x["noise"], ind) for x in [*chosen, one]) if lv is not None] + level = sum(levels) if levels else None + cells = _summed(base, effects, ind) + for k in sorted(set(cells) | set(results[ind])): + before = base.values[ind][k] + summed, ok = cells.get(k, (before, True)) + if not ok: + continue # no sum to hold the build to + got = results[ind].get(k, before) + kind = rule.classify(summed, got, floor=base.floor[ind][k], tolerance=tol, noise=level) + if kind in claims.STILL or (kind == rule.MISSING and summed is None and got is None): + continue + if len(found) < NAMED: + said = ("the file calls its fixes independent, and they are not" if largest is None + else f"the file measured their cross term at {largest:g} at most, and it is larger") # fmt: skip + found.append( + f"{where}, item {base.items[k]!r}, {ind}: built {got!r}, the sum of its effects is {summed!r} " + f"({kind}); {said}" + ) + else: + more += 1 + if more: + found.append(f"{where}: and {more} more pairs where the build is not what the file says") + return found + + +# ------------------------------------------------------------------------ against the diagnostics file + + +def against(f: Mapping, data: Mapping, full: Mapping | None = None) -> list[str]: + """Everything a checked fixes file and its checked diagnostics file disagree on. Its items + and base results are held to the matrix `full` where it is given, and to the listed items + where it is not.""" + try: + return _against(f, data, full) + except Exception as error: # noqa: BLE001 - a backstop, as in `errors` + return [f"{fileformat.STOPPED} ({type(error).__name__}: {str(error)[:200]})"] + + +def _against(f: Mapping, data: Mapping, full: Mapping | None) -> list[str]: + found = [] + of = f["of"] + if of["title"] != data["title"]: + found.append("it belongs to another file: `of.title` is not the diagnostics file's title") + states = [s["id"] for s in data["states"]] + if of["base"] not in states: + return [*found, f"the base {of['base']!r} is no state of the diagnostics file"] + if f["indicators"] != data["indicators"]: + found.append("its indicators differ from the diagnostics file's") + if f["item"] != data["item"] or f.get("indicator") != data.get("indicator"): + found.append("its words for an item or an indicator differ from the diagnostics file's") + if f["rule"] != {k: data["rule"][k] for k in RULE_KEYS}: + found.append("its floors differ from the diagnostics file's") + if found: + return found + ids = [i["id"] for i in data["indicators"]] + at, nstates = states.index(of["base"]), len(states) + if full is not None: + if [list(r) for r in f["items"]] != [list(r) for r in full["items"]]: + return ["its items are not the matrix's, in the matrix's order"] + for x, ind in enumerate(ids): + if [row[x * nstates + at] for row in full["values"]] != f["base"][ind]: + found.append(f"the base results in {ind!r} are not the matrix's results of state {of['base']!r}") + return found + if len(f["items"]) != data["items_count"]["all"]: + found.append(f"it holds {len(f['items'])} items, and the diagnostics file counts {data['items_count']['all']}") + index = {row[0]: k for k, row in enumerate(f["items"])} + for item in data["items"]: + k = index.get(item["id"]) + if k is None: + found.append(f"the listed item {item['id']!r} is not in the fixes file") + continue + if f["items"][k][1:] != [item["name"], item["place"], item["group"], item["unit"]]: + found.append(f"item {item['id']!r}: its name, place, group or unit differ from the diagnostics file") + for ind in ids: + if item["values"][ind][at] != f["base"][ind][k]: + found.append(f"item {item['id']!r}, {ind}: the base result is not the diagnostics file's") + return found diff --git a/tests/test_fixes.py b/tests/test_fixes.py new file mode 100644 index 0000000..c02864a --- /dev/null +++ b/tests/test_fixes.py @@ -0,0 +1,482 @@ +"""Fixes against one base (SPEC 16): the file, its check, and what a page draws for a set. +Every value here is made up.""" + +from __future__ import annotations + +import contextlib +import io +import json +import tempfile +import unittest +from pathlib import Path + +from sentier_diagnostics import chain, claims, cli, fixes, rule + +from . import helpers +from .helpers import INDICATORS, A, B + +FLOORS = {(A, "kg"): 0.01, (B, "kg"): 0.01} + + +def chain_and_matrix(last=None): + """A chain of two states and its matrix; the base is the second state. Item p3 moved in + the step, so it is the one listed item.""" + s1 = last or {A: {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0}, B: {"p1": 3.0, "p2": 5.0, "p3": 7.0, "p4": 1.0}} + s0 = {A: {**s1[A], "p3": 4.0}, B: dict(s1[B])} + items = sorted({i for column in s0.values() for i in column} | {i for column in s1.values() for i in column}) + return chain.build_with_matrix( + "Two states", + indicators=INDICATORS, + states=[chain.state("s0", "Before", kind="published"), chain.state("s1", "After")], + steps=[chain.step("one", "One step", kind="fix")], + values={"s0": s0, "s1": s1}, + units=dict.fromkeys(items, "kg"), + floors=FLOORS, + ) + + +def checked_pairs(*pairs): + return fixes.interactions(True, method="Compared what each fix changes in the model.", + by=claims.origin("code", "pair check"), pairs=pairs) # fmt: skip + + +def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", built=True): + """Three fixes against the base: + + - a and b each move p1 in A by less than their tolerance (1e-3): 1.0 to 1.0006 and 1.0007; + - c moves p3 down in both indicators, as its claim says it must; + - b and c can interact (a listed pair); the set of a and b was built. + """ + data, full = chain_and_matrix() + claim_c = claims.claim("p3 goes down.", [claims.scope([A], {"p3"}, "down", 0.5, claims.MUST)], + by=claims.origin("person", "A. Author")) # fmt: skip + pairs = {"listed": checked_pairs(("b", "c", "both change one market")), "none": checked_pairs(), + "unchecked": fixes.interactions(False)}[interactions] # fmt: skip + f = fixes.build( + "Fixes of two states", + data=data, + full=full, + base="s1", + fixes=[ + fixes.fix("a", "Fix A", kind="mapping", tolerance=chain.tolerance(1e-3), noise=noise_a), + fixes.fix("b", "Fix B", kind="mapping", tolerance=chain.tolerance(1e-3), noise=noise_b), + fixes.fix("c", "Fix C", kind="amount", claim=claim_c), + ], + results={"a": {A: {"p1": 1.0006}}, "b": {A: {"p1": 1.0007}}, "c": {A: {"p3": 2.0}, B: {"p3": 6.0}}}, + interactions=pairs, + built=[fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": built_ab}})] if built else (), + ) + return f, data, full + + +def pair(a, b, reason="r", largest=None): + return {"fixes": [a, b], "reason": reason, "largest": largest} + + +def at(f, item): + return [row[0] for row in f["items"]].index(item) + + +def changed(f, change): + return helpers.mutated(f, change) + + +def said(found, part): + """Whether one of the errors says `part`.""" + return any(part in line for line in found) + + +class File(unittest.TestCase): + def test_a_made_file_is_checked_clean(self): + f, data, full = made() + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + self.assertEqual(fixes.against(f, data, None), []) + self.assertEqual([row[0] for row in f["items"]], [row[0] for row in full["items"]]) + + def test_an_effect_keeps_only_what_changed(self): + f, *_ = made() + self.assertEqual(f["fixes"][0]["effect"], {A: [[at(f, "p1"), 1.0006]], B: []}) + + def test_the_file_survives_its_own_text(self): + f, *_ = made() + self.assertEqual(json.loads(fixes.dumps(f)), f) + + def test_written_and_read_back_beside_its_diagnostics_file(self): + f, data, full = made() + with tempfile.TemporaryDirectory() as folder: + folder = Path(folder) + chain.write(data, folder / chain.FILE) + fixes.write(f, folder / "fixes.json") + self.assertEqual(fixes.read(folder / "fixes.json"), f) # without the matrix: the listed items + from sentier_diagnostics import matrix + + matrix.write(full, folder / "matrix.json") + self.assertEqual(fixes.read(folder / "fixes.json"), f) + + def test_the_limits_are_the_specs(self): + self.assertEqual((fixes.MAX_FIXES, fixes.MAX_BUILT, fixes.MAX_ITEMS, fixes.MAX_BYTES), + (1000, 500, 200_000, 64 * 1024 * 1024)) # fmt: skip + f, *_ = made() + many = changed(f, lambda g: g.__setitem__("fixes", g["fixes"] * 334)) + self.assertTrue(said(fixes.errors(many), "`fixes` is not a list of 1 to 1000")) + + def test_another_major_version_is_refused_by_name(self): + f, *_ = made() + self.assertEqual(fixes.errors({**f, "format": "sentier-diagnostics-fixes/2"}), + ["format 'sentier-diagnostics-fixes/2': this reader knows sentier-diagnostics-fixes/1 only"]) # fmt: skip + self.assertEqual(fixes.errors({**f, "format": "sentier-diagnostics/1"}), + ["not a fixes file of format sentier-diagnostics-fixes/1"]) # fmt: skip + + def test_keys_are_read_strictly(self): + f, *_ = made() + self.assertTrue(said(fixes.errors({**f, "ranges": []}), "a key this format does not know: ranges")) + self.assertTrue(said(fixes.errors({k: v for k, v in f.items() if k != "built"}), "requires: built")) + extra = changed(f, lambda g: g["fixes"][0].__setitem__("verdicts", {})) + self.assertTrue(said(fixes.errors(extra), "a fix is not")) + + def test_shape_faults_never_raise(self): + f, *_ = made() + valid = {("title", '"x"'), ("about", '"x"'), ("about", "null"), ("made_by", '"x"'), ("notes", "[]"), + ("built", "[]")} # fmt: skip + for key in f: + for junk in (None, 1, "x", [], {}, [[None]], {"x": [1]}, True): + with self.subTest(key=key, junk=junk): + found = fixes.errors({**f, key: junk}) + self.assertFalse(said(found, fixes.fileformat.STOPPED), found) + if (key, json.dumps(junk)) not in valid: + self.assertNotEqual(found, [], f"{key} = {junk!r} passed") + for part in ("effect", "counts", "noise", "tolerance", "claim"): + for junk in (1, "x", [1], {"x": 1}): + with self.subTest(fix_key=part, junk=junk): + found = fixes.errors(changed(f, lambda g, k=part, v=junk: g["fixes"][0].__setitem__(k, v))) + self.assertNotEqual(found, []) + self.assertFalse(said(found, fixes.fileformat.STOPPED), found) + self.assertEqual(fixes.errors(None), ["not a fixes file: not an object"]) + + +class Checks(unittest.TestCase): + """SPEC 16.9, each check probed by the change it must find.""" + + def test_of_names_the_file_its_title_and_one_of_its_states(self): + f, data, full = made() + self.assertTrue(said(fixes.against(changed(f, lambda g: g["of"].__setitem__("title", "Another")), data, full), + "belongs to another file")) # fmt: skip + self.assertTrue(said(fixes.against(changed(f, lambda g: g["of"].__setitem__("base", "s9")), data, full), + "no state of the diagnostics file")) # fmt: skip + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["of"].__setitem__("file", "../up.json"))), + "`of` is not {file, title, base}")) # fmt: skip + + def test_items_indicators_floors_and_words_are_the_diagnostics_files(self): + f, data, full = made() + cases = { + "its indicators differ": lambda g: g["indicators"][0].__setitem__("name", "Another"), + "its words for an item": lambda g: g.__setitem__("item", {"one": "product", "many": "products"}), + "its floors differ": lambda g: g["rule"]["floors"][0].__setitem__("value", 0.011), + "not the matrix's, in the matrix's order": lambda g: g["items"].reverse(), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.against(changed(f, change), data, full), part)) + + def test_the_base_is_the_matrix_column_of_the_base_state(self): + f, data, full = made() + moved = changed(f, lambda g: g["base"][B].__setitem__(at(g, "p2"), 5.5)) + self.assertEqual(fixes.errors(moved), []) # no fix touches p2: only the matrix can tell + self.assertTrue(said(fixes.against(moved, data, full), "the base results in 'x:b' are not the matrix's")) + + def test_without_the_matrix_the_base_is_held_to_the_listed_items(self): + f, data, _ = made() + self.assertEqual([i["id"] for i in data["items"]], ["p3"]) + moved = changed(f, lambda g: g["base"][B].__setitem__(at(g, "p3"), 7.5)) + self.assertTrue(said(fixes.against(moved, data, None), "item 'p3', x:b: the base result")) + fewer = changed(f, lambda g: g["items"].pop()) + self.assertTrue(said(fixes.against(fewer, data, None), "it holds 3 items, and the diagnostics file counts 4")) + + def test_an_effect_names_known_items_once_and_never_the_base(self): + f, *_ = made() + k1, k2 = at(f, "p1"), at(f, "p2") + cases = { + "names item 99, and the file holds 4": lambda g: g["fixes"][0]["effect"][A].append([99, 1.0]), + "holds item 'p1' twice": lambda g: g["fixes"][0]["effect"][A].append([k1, 1.5]), + "is written with the base result": lambda g: g["fixes"][0]["effect"][A].append([k2, 2.0]), + "not one list per indicator": lambda g: g["fixes"][0]["effect"].__setitem__("x:z", []), + "an entry is not [item index, result or null]": lambda g: g["fixes"][0]["effect"][B].append([k1]), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + + def test_built_results_are_held_to_the_same_rules(self): + f, *_ = made() + k1 = at(f, "p1") + cases = { + "holds item 'p1' twice": lambda g: g["built"][0]["results"][A].append([k1, 1.5]), + "is written with the base result": lambda g: g["built"][0]["results"][B].append([k1, 3.0]), + "names item 7": lambda g: g["built"][0]["results"][B].append([7, 3.0]), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + + def test_counts_and_verdicts_are_what_the_effect_and_the_base_give(self): + f, *_ = made() + + def move(block, key, a, b): + block[key][a] -= 1 + block[key][b] += 1 + + cases = { + "`classes` is not": lambda g: move(g["fixes"][0]["counts"][B], "classes", rule.AGREES, rule.WITHIN_NOISE), + "`verdicts` is not": lambda g: move(g["fixes"][2]["counts"][A], "verdicts", claims.UNCHANGED, + claims.NOT_JUDGED), + "`bands` is not": lambda g: move(g["fixes"][2]["counts"][A], "bands", "down, factor 1.25-2", + "down, factor 1.1-1.25"), + "is null: a fix has no published results": lambda g: g["fixes"][0]["counts"][A].update(closer=0, further=0), + } # fmt: skip + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + # a changed effect with the counts left as they were + moved = changed(f, lambda g: g["fixes"][0]["effect"][A].__setitem__(0, [at(g, "p1"), 1.5])) + self.assertTrue(said(fixes.errors(moved), "fix 'a', counts of 'x:a': `classes` is not what its effect")) + + def test_a_fix_claim_names_its_items(self): + f, *_ = made() + hidden = changed(f, lambda g: g["fixes"][2]["claim"]["scope"][0].__setitem__("items", None)) + self.assertTrue(said(fixes.errors(hidden), "a fix's claim names its items")) + unknown = changed(f, lambda g: (g["fixes"][2]["claim"]["scope"][0].__setitem__("items", ["p3", "p9"]), + g["fixes"][2]["claim"].__setitem__("reach", 2))) # fmt: skip + self.assertTrue(said(fixes.errors(unknown), "its claim names 1 items the file does not hold")) + + def test_pairs_name_two_known_fixes_once(self): + f, *_ = made() + cases = { + "the file holds no fix 'z'": lambda g: g["interactions"]["pairs"].append(pair("b", "z")), + "names one fix twice": lambda g: g["interactions"]["pairs"].append(pair("a", "a")), + "is listed twice": lambda g: g["interactions"]["pairs"].append(pair("c", "b")), + "does not name two fixes": lambda g: g["interactions"]["pairs"].append({**pair("a", "b"), "fixes": ["a"]}), + "`largest` is not a share": lambda g: g["interactions"]["pairs"][0].__setitem__("largest", -0.1), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + + def test_no_half_check(self): + f, *_ = made() + listed = changed(f, lambda g: g["interactions"].update(checked=False, method=None, by=None)) + self.assertTrue(said(fixes.errors(listed), "pairs are listed and `checked` is false")) + f, *_ = made(interactions="unchecked") + self.assertEqual(fixes.errors(f), []) + ran = changed(f, lambda g: g["interactions"].__setitem__("method", "Looked at some pairs.")) + self.assertTrue(said(fixes.errors(ran), "no check ran, so it has no `method`")) + f, *_ = made() + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["interactions"].__setitem__("by", None))), + "says who or what ran it")) # fmt: skip + + def test_a_built_set_holds_two_or_more_fixes_once_and_is_not_built_twice(self): + f, *_ = made() + empty = {A: [], B: []} + noise = chain.noise() + + def add(names): + return lambda g: g["built"].append({"id": "x", "label": "X", "fixes": names, "results": empty, "noise": noise}) + + cases = { + "fewer than two fixes": add(["a"]), + "holds a fix twice": add(["a", "a"]), + "the same fixes as another built set": add(["b", "a"]), + "the file holds no fix 'z'": add(["a", "z"]), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + twice = changed(f, lambda g: g["built"].append(dict(g["built"][0], fixes=["a", "c"], results=empty))) + self.assertTrue(said(fixes.errors(twice), "two built sets have the id 'ab'")) + + +def dense(f, fix_id): + """A fix's counts worked out item by item, as `chain` counts a step: the reference the + sparse count is held to.""" + one = next(x for x in f["fixes"] if x["id"] == fix_id) + floors = {(x["indicator"], x["unit"]): x["value"] for x in f["rule"]["floors"]} + out = {} + for ind in (i["id"] for i in f["indicators"]): + effect = dict(map(tuple, one["effect"][ind])) + level = (one["noise"]["by"] or {}).get(ind) if one["noise"]["measured"] else None + block = {"classes": dict.fromkeys(rule.CLASSES, 0), "missing": {"before": 0, "after": 0, "both": 0}, + "verdicts": dict.fromkeys(claims.VERDICTS, 0) if one["claim"] else None} # fmt: skip + for k, row in enumerate(f["items"]): + before = f["base"][ind][k] + after = effect.get(k, before) + kind, way, _ = rule.judge(before, after, floor=floors[(ind, row[4])], + tolerance=one["tolerance"]["value"], noise=level) # fmt: skip + block["classes"][kind] += 1 + if kind == rule.MISSING: + block["missing"]["both" if before is None and after is None else "before" if before is None else "after"] += 1 + if one["claim"]: + covering = [e for e in claims.entries_for(one["claim"], ind) if claims.covers(e, row[0], ind)] + block["verdicts"][claims.verdict(kind, before, after, way, covering)] += 1 + out[ind] = block + return out + + +class SparseCounts(unittest.TestCase): + def test_counting_the_changed_items_only_is_counting_every_item(self): + last = {A: {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0, "p5": None, "p6": -2.0}, + B: {"p1": 3.0, "p2": 5.0, "p3": 7.0, "p4": 1.0, "p5": 4.0, "p6": 0.001}} # fmt: skip + data, full = chain_and_matrix(last) + every = claims.claim("All of B must move; p1 and p5 may move in A.", + [claims.scope([B], "*", "up", None, claims.MUST), + claims.scope([A], {"p1", "p5"}, "any", None, claims.MAY), + claims.scope([A], {"p2", "p4"}, "down", 0.1, claims.MUST)]) # fmt: skip + wide = fixes.fix("w", "Wide", kind="amount", claim=every, noise=chain.noise(["one", "two"], {A: 0.05})) + moved = {A: {"p1": 1.02, "p5": 3.0, "p2": None, "p6": 2.0}, B: {"p1": 3.5, "p5": None, "p6": 0.002, "p4": 1.0001}} + f = fixes.build("Sparse", data=data, full=full, base="s1", fixes=[wide], results={"w": moved}) + again = dense(f, "w") + for ind, block in f["fixes"][0]["counts"].items(): + for key in ("classes", "missing", "verdicts"): + with self.subTest(ind=ind, key=key): + self.assertEqual(block[key], again[ind][key]) + self.assertEqual(sum(f["fixes"][0]["counts"][B]["verdicts"].values()), 6) + + +class Acceptance(unittest.TestCase): + """SPEC 16.11. F7 is the page's (sentier-bafu).""" + + def test_f1_two_fixes_within_the_tolerance_differ_together(self): + f, *_ = made(built=False) + for one in ("a", "b"): + self.assertEqual(fixes.counts(f, [one])["by"][A]["classes"][rule.DIFFERS], 0) + together = fixes.counts(f, ["a", "b"]) + self.assertEqual(together["by"][A]["classes"][rule.DIFFERS], 1) + self.assertEqual(together["label"], fixes.EXACT) + self.assertAlmostEqual(fixes.compose(f, ["a", "b"])["results"][A][at(f, "p1")], 1.0013) + + def test_f2_a_set_with_a_listed_pair_is_estimated_everywhere(self): + f, *_ = made() + drawn = fixes.compose(f, ["c", "b"]) + self.assertEqual(drawn["label"], fixes.ESTIMATED) + self.assertEqual(drawn["pairs"], [pair("b", "c", "both change one market")]) + self.assertEqual(drawn["pair_terms"], {"value": None, "measured": 0, "pairs": 1}) + self.assertEqual(fixes.counts(f, ["b", "c"])["label"], fixes.ESTIMATED) + self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.ESTIMATED) # a built subset makes no set built + self.assertEqual(fixes.label(f, ["a", "c"]), fixes.EXACT) + + def test_f3_unchecked_interactions_make_every_set_not_built_estimated(self): + f, *_ = made(interactions="unchecked") + self.assertEqual(fixes.label(f, ["a"]), fixes.BUILT) + self.assertEqual(fixes.label(f, ["a", "b"]), fixes.BUILT) + for names in (["a", "c"], ["b", "c"], ["a", "b", "c"]): + self.assertEqual(fixes.label(f, names), fixes.ESTIMATED) + + def test_f4_a_built_set_that_is_not_the_sum_of_independent_fixes_is_refused(self): + with self.assertRaises(ValueError) as caught: + made(interactions="none", built_ab=1.0030) + self.assertIn("built set 'ab', item 'p1', x:a: built 1.003", str(caught.exception)) + self.assertIn("the file calls its fixes independent, and they are not", str(caught.exception)) + f, *_ = made(interactions="none") # the sum: accepted + self.assertEqual(fixes.errors(f), []) + def interacting(g): + g["built"][0]["results"][A][0] = [at(g, "p1"), 1.0030] + g["interactions"]["pairs"].append(pair("a", "b", "both change one factor")) + + self.assertEqual(fixes.errors(changed(f, interacting)), [], "a listed pair is an interaction, shown, not refused") + f, *_ = made(interactions="unchecked", built_ab=1.0030) + self.assertEqual(fixes.errors(f), [], "without a check the file claims no independence") + + def test_f5_a_fix_is_judged_the_same_whichever_fixes_are_chosen_with_it(self): + f, *_ = made() + own = fixes.compose(f, ["c"])["fixes"][0] + for names in (["a", "c"], ["c", "b"], ["b", "a", "c"]): + drawn = fixes.compose(f, names) + self.assertEqual(next(x for x in drawn["fixes"] if x["id"] == "c"), own) + self.assertNotIn("verdicts", drawn) + self.assertIsNone(fixes.counts(f, names)["by"][A]["verdicts"]) + self.assertEqual(fixes.compose(f, ["b", "a"])["results"], fixes.compose(f, ["a", "b"])["results"]) + + def test_f6_the_noise_of_a_set_is_the_sum_said_with_how_many_were_measured(self): + noise = chain.noise(["one", "two"], {A: 1e-3}) + f, *_ = made(noise_a=noise, noise_b=noise, interactions="none", built=False) + self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 2e-3, "measured": 2, "fixes": 2}) + self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-3, "measured": 1, "fixes": 2}) + self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": None, "measured": 0, "fixes": 1}) + self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "measured": 0, "fixes": 1}) + # the sum, not the largest: 1.3e-3 apart is within 2e-3 and beyond 1e-3 + self.assertEqual(fixes.counts(f, ["a", "b"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) + + def test_f8_a_pair_says_why_in_words_never_by_the_models_identifiers(self): + f, *_ = made() + # joined here, so that the repository itself holds no identifier-shaped text + uuid_shaped = "-".join(["ab12cd34", "ab12", "ab12", "ab12", "ab12cd34ef56"]) # noqa: FLY002 + cases = { + "a pair is not {fixes, reason, largest}": lambda g: g["interactions"]["pairs"][0].__setitem__("activities", ["m"]), + "its reason holds an identifier": lambda g: g["interactions"]["pairs"][0].__setitem__( + "reason", f"both change {uuid_shaped}"), + "`interactions.method` holds an identifier": lambda g: g["interactions"].__setitem__( + "method", "Compared " + "ab" * 16), + } # fmt: skip + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + self.assertIsNone(fixes.IDENTIFIER.search(fixes.dumps(f))) + + +class PairSizes(unittest.TestCase): + """A listed pair may carry the measured size of its cross term (`largest`): an estimated + set then says how far it may lie from its sum, and a built pair holds the size to account.""" + + def test_a_built_pair_lies_within_its_measured_size(self): + f, *_ = made(interactions="unchecked", built_ab=1.0030) # 1.7e-3 from the sum, 1.0013 + f["interactions"] = checked_pairs(("a", "b", "both change one factor")) + self.assertEqual(fixes.errors(f), [], "not measured: the pair is shown, not held to a size") + f["interactions"]["pairs"][-1]["largest"] = 0.002 + self.assertEqual(fixes.errors(f), []) + f["interactions"]["pairs"][-1]["largest"] = 0.001 + found = fixes.errors(f) + self.assertTrue(said(found, "built set 'ab', item 'p1', x:a"), found) + self.assertTrue(said(found, "measured their cross term at 0.001 at most, and it is larger"), found) + + def test_the_pair_terms_of_a_set_are_the_sum_of_its_measured_pairs(self): + f, *_ = made() + f["interactions"]["pairs"] = [pair("b", "c", "one market", 1e-5), pair("a", "c", "one factor", 2e-5)] + self.assertEqual(fixes.errors(f), []) + terms = fixes.pair_terms(f, ["a", "b", "c"]) + self.assertEqual((terms["measured"], terms["pairs"]), (2, 2)) + self.assertAlmostEqual(terms["value"], 3e-5) + f["interactions"]["pairs"][1]["largest"] = None + self.assertEqual(fixes.pair_terms(f, ["a", "b", "c"]), {"value": None, "measured": 1, "pairs": 2}) + self.assertEqual(fixes.pair_terms(f, ["a", "b"]), {"value": 0, "measured": 0, "pairs": 0}) + + +class Command(unittest.TestCase): + def run_cli(self, *args) -> tuple[int, str]: + out = io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(out): + code = cli.main([str(a) for a in args]) + return code, out.getvalue() + + def test_check_holds_a_fixes_file_to_the_file_beside_it(self): + from sentier_diagnostics import matrix + + f, data, full = made() + with tempfile.TemporaryDirectory() as folder: + folder = Path(folder) + chain.write(data, folder / chain.FILE) + matrix.write(full, folder / "matrix.json") + fixes.write(f, folder / "fixes.json") + code, out = self.run_cli("check", folder / "fixes.json") + self.assertEqual(code, 0, out) + self.assertIn("3 fixes and 1 built sets hold 5 entries over 4 items; the largest fixes: 'c' 2", out) + fixes.write(changed(f, lambda g: g["base"][B].__setitem__(at(g, "p2"), 5.5)), folder / "fixes.json") + code, out = self.run_cli("check", folder / "fixes.json") + self.assertEqual(code, 1, out) + self.assertIn("the base results in 'x:b' are not the matrix's", out) + (folder / chain.FILE).unlink() + code, out = self.run_cli("check", folder / "fixes.json") + self.assertEqual(code, 1, out) + self.assertIn("names the diagnostics file diagnostics.json, which is not beside it", out) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_modules.py b/tests/test_modules.py index 67dc943..f6d99ce 100644 --- a/tests/test_modules.py +++ b/tests/test_modules.py @@ -20,7 +20,8 @@ HERE = Path(__file__).parent FIXTURES = HERE / "fixtures" / "tables" -MODULES = ["rule", "claims", "chain", "fileformat", "breakdown", "tables", "summary", "page", "from_tables", "matrix", "together"] +MODULES = ["rule", "claims", "chain", "fileformat", "breakdown", "tables", "summary", "page", "from_tables", "matrix", "together", + "fixes"] def load_tests(loader, tests, pattern): From b1db7350204c1493d164ef2c546ff8ec57f8c259 Mon Sep 17 00:00:00 2001 From: Manuel Klarmann Date: Tue, 17 Nov 2026 06:13:58 +0100 Subject: [PATCH 2/6] Fixes that need others, items a fix brings, a release held to its state, pairs without text MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four questions the fixes format left open, settled before any producer writes a file: - A pair holds no text: {fixes, kind, reach, largest}, its kind one of five places where two changes of h = c·B·A⁻¹·f meet. A reason in words names the model, and a check of text catches identifier shapes, not names. - `requires`: a fix that cannot be applied alone names the fixes it needs, earlier in the file. Its effect and counts are measured against their build (a fix with what it requires, or a built set of exactly those fixes); choosing it chooses them; a fix and one it requires are never a pair. - A fix may bring items no state holds: after the matrix's items, base null, and some fix or built set gives each a result. - A built set may name the state of the chain it is (the release with every fix); the check holds the two builds to each other by the rule. A set's result at an item is now the measured one wherever a build inside the set covers every fix that changes it, and otherwise each effect on top of what it was measured with. The independence check still sums the fixes alone, so tolerances do not stack. 48 fixes tests (was 30); 43 of 43 sabotage probes caught; check of 100 fixes on 11,947 items x 25 indicators: 2.7 s. Co-Authored-By: Claude Opus 5.5 --- GLOSSARY.md | 9 +- SPEC.md | 209 +++++--- src/sentier_diagnostics/fixes.py | 838 +++++++++++++++++++++---------- tests/test_fixes.py | 365 +++++++++++++- 4 files changed, 1054 insertions(+), 367 deletions(-) diff --git a/GLOSSARY.md b/GLOSSARY.md index 1068004..d72f348 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -33,8 +33,9 @@ The words this repository uses, each in one line. [SPEC.md](SPEC.md) says the re - **Drawer**: the panel of one item, at the right of the page, opened from a row. - **Base**: the one state of a chain that every fix is measured against (SPEC section 16). - **Fix**: one change applied to the base alone and built, with a step's kind, claim, tolerance and noise. -- **Effect**: what a fix does to the base, per item and indicator: its result with the fix minus the base result. -- **Set**: the fixes a reader chooses together; its result is the base plus the sum of their effects. It has no verdict. -- **Interaction**: two fixes whose joint result is not the sum of their effects. Checked in pairs, in the model. +- **Requires**: the fixes a fix cannot be applied without. Its effect is measured with them applied, and choosing it chooses them. +- **Effect**: what a fix does, per item and indicator: its result with the fix minus its result without it, both with what it requires. +- **Set**: the fixes a reader chooses together, with what they require; its result is the base plus the sum of their effects. It has no verdict. +- **Interaction**: two fixes whose joint result is not the sum of their effects. Checked in pairs, in the model; a listed pair says its kind, its reach and its size, never words. - **Pair terms**: how far an estimated set may lie from its sum, to first order: the sum of the measured sizes of the listed pairs it holds; not known where one was not measured. -- **Built, exact, estimated**: the label of a set. Built where the producer built it; exact where no two of its fixes can interact; estimated otherwise, and marked on every cell. +- **Built, exact, estimated**: the label of a set. Built where the producer built it (one fix with what it requires, or a built set); exact where no two of its fixes can interact; estimated otherwise, and marked on every cell. diff --git a/SPEC.md b/SPEC.md index a4ebc1f..e896dc3 100644 --- a/SPEC.md +++ b/SPEC.md @@ -860,8 +860,12 @@ Status: proposed, 18 November 2026. Manuel decided the direction: a chain is to hundreds of fixes, any set of them can be chosen, and the set is **one base with fixes measured on their own**, not a chain of hundreds of states ("yes, one base with independently measured fixes"). He also decided that the fixes live in a file of their own, -and that a set the file cannot show to be exact is shown, marked as estimated. The reader -and validator are `fixes.py`; the producer and the page are not written yet (16.12). +and that a set the file cannot show to be exact is shown, marked as estimated. Four +questions were settled in the validator's pull request, before any producer writes a file, +for Manuel to agree there: a pair holds no text (16.6); a fix may require others (16.4); a +fix may bring items the base does not hold (16.4); and a built set may say which state of +the chain it is (16.7). The reader and validator are `fixes.py`; the producer and the page +are not written yet (16.12). ### 16.1 Why not a chain of fixes @@ -881,10 +885,12 @@ the base to a release. A fixes file hangs on one of its states. - **Base**: one state of a chain, the state every fix is measured against. - **Fix**: one change, applied to the base alone and built. It has what a step has: a kind, a claim, a tolerance, a noise, `explained`, a link. -- **Effect**: what a fix does to the base, per item and indicator: its result with the fix - minus the base result. -- **Set**: the fixes a reader chooses together. Its result is the base plus the sum of - their effects. +- **Requires**: the fixes a fix cannot be applied without. Its effect is measured with + them applied, and choosing it chooses them. +- **Effect**: what a fix does, per item and indicator: its result with the fix (and what it + requires) minus its result without the fix (with what it requires). +- **Set**: the fixes a reader chooses together, with everything they require. Its result is + the base plus the sum of their effects. - **Interaction**: two fixes whose joint result is not the sum of their effects. ### 16.3 When a sum is the result @@ -898,7 +904,14 @@ the technosphere A. So: - a fix that changes the factor of a flow and one that changes how much of that flow an activity emits interact; - two fixes that change A interact on every item whose supply chain reaches both changes. - Most fixes of a database (a mapping, an amount, a market share) change A. + Most fixes of a database (a mapping, an amount, a market share) change A; +- a fix that changes A and one that changes B or c interact where the A change shifts how + much of the changed flow or activity an item's chain draws on; +- two fixes that change the same number of A, B or c do not add up whatever kind it is: + one replaces what the other set. + +These are the kinds of pair a file can list (16.6). Two fixes that change different +factors, or different entries of B, are never a pair. An interaction always shows in a pair: every term of A⁻¹ expanded in its changes that holds two fixes holds two of them next to each other. So a set whose fixes are independent @@ -919,35 +932,62 @@ item. One JSON object, read as strictly as the diagnostics file (section 4.1). | `title`, `about` | Short text; `about` may be null | | `item`, `indicator`, `indicators` | As in the diagnostics file, and equal to it (`indicator` may be left out where that file leaves it out) | | `rule` | `{floor_factor, floors_from, beyond, floors}`: the diagnostics file's rule without its `total_tolerance`, which belongs to the chain (a set has its own, 16.5). The floors come from the chain's first state | -| `items` | Every item, `[id, name, place, group, unit]`, in the order of the matrix | -| `base` | The base result of every item, per indicator: `{indicator: [result or null, ...]}`, in the order of `items` | -| `fixes` | `[{id, label, about, kind, explained, link, claim, tolerance, noise, effect, counts}]` | +| `items` | Every item, `[id, name, place, group, unit]`: the matrix's items in the matrix's order, then any item a fix brings that no state of the chain holds | +| `base` | The base result of every item, per indicator: `{indicator: [result or null, ...]}`, in the order of `items`; null for an item a fix brings | +| `fixes` | `[{id, label, about, kind, explained, link, requires, claim, tolerance, noise, effect, counts}]`, a fix's requirements before it | | `interactions` | `{checked, method, by, pairs}` (16.6) | -| `built` | Sets the producer built: `[{id, label, fixes, results, noise}]` (16.7) | +| `built` | Sets the producer built: `[{id, label, fixes, state, results, noise}]` (16.7) | | `notes`, `made_by` | As in the diagnostics file | -A fix's `effect` holds, per indicator, the items whose result with the fix is not the base -result bit for bit, as `[[item index, result], ...]`, the result being the one built with -the fix alone (`null` where it has none). **Every change is kept, however small**: two -changes under the tolerance each can be over it together, and a set is a sum. An entry -equal to the base is refused: it would read as a change. Every indicator has a list, empty -where the fix changes nothing. A fix's `counts` is one block per indicator (section 4.2), -its pairs being the base against the result with the fix, judged with the fix's own -tolerance and noise; `closer` and `further` are null, since a fix has no published results -to come closer to. A fix's claim names its items (no `items: null`): the file is local and -holds every item, and the page works out each item's verdict from the claim. - -A fix's claim is judged as a step's is (section 6), against the base. Its verdicts do not -depend on which other fixes are chosen or in what order: that is what measuring a fix on -its own buys. +A fix's `effect` holds, per indicator, the items whose result with the fix is not the +result it was measured against bit for bit, as `[[item index, result], ...]`, the result +being the one built with the fix (`null` where it has none). **Every change is kept, +however small**: two changes under the tolerance each can be over it together, and a set is +a sum. An entry equal to the result it was measured against is refused: it would read as a +change. Every indicator has a list, empty where the fix changes nothing. A fix's `counts` is +one block per indicator (section 4.2), its pairs being the result it was measured against +and the result with the fix, judged with the fix's own tolerance and noise; `closer` and +`further` are null, since a fix has no published results to come closer to. A fix's claim +names its items (no `items: null`): the file is local and holds every item, and the page +works out each item's verdict from the claim. + +**A fix that cannot be applied alone** (a correction to a market that another fix creates, +a factor for a flow another fix adds) names the fixes it needs in `requires`: fixes that +come before it in `fixes`, each once, never itself. What it requires at any depth is +applied when it is built, and its effect and counts are measured against the build of +those fixes together. Where they are one fix with what it requires, that is that fix's +build (a fix that requires `d`, which requires `a`, is measured against the build of `d`); +otherwise it is a built set of exactly those fixes (16.7). A file that holds no such build +is refused: the effect would have nothing to be measured against. A fix and a fix it requires are never a pair (16.6): +one is measured with the other applied. + +**A fix may bring items** no state of the chain holds (a new product an added activity +makes). They follow the matrix's items in `items`, their base result is null, and some fix +or built set gives each a result. Their unit needs a floor in the rule, as any item's does. + +A fix's claim is judged as a step's is (section 6), against what the fix was measured +against. Its verdicts do not depend on which other fixes are chosen or in what order: that +is what measuring a fix on its own buys. ### 16.5 A set and its label -The result of a set is the base plus the sum of its fixes' effects. A fix without a result -at an item leaves the set none there. Where the base has no result and two of the chosen -fixes give one, their differences have nothing to be added to: the set has no result there -either, and the page says the sum cannot be formed, apart from a result that is missing. It -is judged by the rule (section 5) against the base: +A set holds everything its fixes require: choosing a fix chooses what it requires, at any +depth, and the page ticks those fixes too and says it added them. At each item, its result +is: + +- where one fix of the set changes it, that fix's built result; +- where several do, and they all lie in one build the file holds inside the set, that + build's result: measured, not summed. The builds are looked at in this order: each fix of + the set that requires others, with what it requires, in the order of the file; then each + built set inside the set, in the order of the file. The first that holds them all is + taken; +- otherwise the base plus each fix's difference against the result it was measured + against. A fix measured on top of another adds only its own difference, so what both + build on is counted once. + +Where one of those differences cannot be taken (a missing result with the fix or without +it), the set has no result there, and the page says the sum cannot be formed, apart from a +result that is missing. It is judged by the rule (section 5) against the base: - its **tolerance** is the largest of its fixes'; - its **noise** is the sum of its fixes' noise, per indicator. A fix's noise stands for @@ -964,9 +1004,9 @@ Each set carries one of three labels, and the page shows it wherever the set is | Label | When | What the page draws | |---|---|---| -| built | the set is one fix, or one of `built` | the built results | +| built | the set is one fix with what it requires, or one of `built` | the built results | | exact | `interactions.checked` is true, and no two of its fixes are a listed pair | the sum, as a result | -| estimated | otherwise | the sum, marked on every cell, in the line above the matrix and in every count ("estimated: 153 would differ"), with the listed pairs it holds, their reasons, and their pair terms where measured ("pair terms up to 0.003 %") | +| estimated | otherwise | the sum, marked on every cell, in the line above the matrix and in every count ("estimated: 153 would differ"), with the listed pairs it holds, their kinds, and their pair terms where measured ("pair terms up to 0.003 %") | An estimated set is never drawn as a built one: the mark is on the cell, not only in a heading a reader may have scrolled past. Its **pair terms** are the sum of the measured @@ -986,31 +1026,48 @@ into it. - `checked`: true where every pair of fixes was examined, false where none was. There is no half: a reader must know whether a pair that is not listed was cleared or never looked at. - `method`: what the check did, in a sentence, and `by`: who or what ran it (section 8). -- `pairs`: `[{fixes: [a, b], reason, largest}]`, every pair that can interact, each once, - with a short reason ("both change one market; 412 items reach both changes") and - `largest`: the measured size of its cross term, the largest over items and indicators of - |h_ab − h_a − h_b + h_0| / |h_ab|, or null where it was not measured. A check that only - asks whether A⁻¹ links two changes lists nearly every pair in a database where - electricity and transport reach everything, and every set would be estimated; the size - lets a reader tell a negligible estimate from a large one. A pair whose cross term is - zero by the model's structure is not listed. A pair has no other key, and the check - refuses a reason or a method holding text shaped like an identifier (a UUID, or 32 - hexadecimal digits). Whether a reason may name an activity ("the market for wheat flour") - is open (below); the producer writes counts and kinds of change until it is decided. +- `pairs`: `[{fixes: [a, b], kind, reach, largest}]`, every pair that can interact, each + once: + - `kind`: where the two changes meet (16.3), one of `technosphere` (both change A), + `technosphere and emission` (A, and an amount of B), `technosphere and factor` (A, and + a factor of c), `emission and factor` (an amount of a flow, and that flow's factor), + `same entry` (both change one number); + - `reach`: how many items the cross term can change (1 to the number of items), or null + where it was not counted; + - `largest`: the measured size of its cross term, the largest over items and indicators + of |h_ab − h_a − h_b + h_0| / |h_ab|, or null where it was not measured. + + A check that only asks whether A⁻¹ links two changes lists nearly every pair in a + database where electricity and transport reach everything, and every set would be + estimated; the size lets a reader tell a negligible estimate from a large one. A pair + whose cross term is zero by the model's structure is not listed, and neither is a fix + with a fix it requires. + +**A pair holds no text.** A reason in words ("both change the market for wheat flour") +names the model, and a check of text can only catch what is shaped like an identifier, not +a name. A closed kind, a count and a size say what a reader needs (how the two meet, how +far it spreads, how large it is) and leave nothing to check. The one text the interactions +hold is `method`, and the check refuses one holding text shaped like an identifier (a UUID, +or 32 hexadecimal digits). The validator cannot see the model, so it cannot prove a pair independent. It holds the claim to every built set instead (16.7). ### 16.7 Built sets -The producer may build sets as well as single fixes: the release with every fix, or a pair -it listed, to measure how far it is from the sum. A built set has an id, a label, its fixes -(two or more), its results (per indicator, the items whose result is not the base result -bit for bit, as an effect is written) and its noise. +The producer may build sets as well as single fixes: the release with every fix, a pair +it listed, to measure how far it is from the sum, or the fixes another fix requires, to +measure that fix against. A built set has an id, a label, its fixes (two or more, with +everything they require, and never exactly one fix with what it requires: that is built +already), its results (per indicator, the items whose result is not the base result bit for +bit, as an effect is written), its noise, and `state`: the id of the state of the chain it +is, where it is one, or null. **A built set tests the claim of independence.** Where `checked` is true and no two of its fixes are a listed pair, the built results must agree with the base plus the sum of the -effects, by the rule with the set's tolerance and noise, for every item. The noise here is +effects, by the rule with the set's tolerance and noise, for every item. That sum stands on +the fixes alone (16.5 without its built sets): a set held to a build inside it, itself held +to the sum, would stack two tolerances. The noise here is the sum of its fixes' noise and the built set's own, where measured: the sum stands on one build of each fix, the built results on one more. A file where they do not is refused, naming the set, the item and the indicator: its producer called two @@ -1024,10 +1081,19 @@ Building the release with every fix is cheap (it is built anyway) and is the str the file can carry: it touches every pair at once. It can refute independence and never confirm it: cross terms of opposite sign can cancel in the whole and not in a subset. +**A built set that names a state is two builds of one thing.** The release with every fix +is also the last state of the chain. Its `state` names it, a state after the base, and the +check holds the two to each other: the base with the built results, against the matrix's +results of that state, by the rule with the set's tolerance and noise (the sum of its +fixes' noise and its own), for every item; for the listed items where the matrix is not +beside it. A file whose release does not reproduce the chain's is refused, naming the set, +the item and the indicator. + ### 16.8 The page - A searchable list chooses fixes: by name, kind and whether a fix moved anything, with a - box per fix. Hundreds do not fit a row of chips. + box per fix. Hundreds do not fit a row of chips. Choosing a fix ticks what it requires + and says so; unticking a fix another chosen fix requires unticks that one too. - The address names the set: `#set=a,b,c`, in the order of the file. An address that names a fix the file does not hold opens on the base and says which. - One fix reads as a step does today: its claim, its verdicts, its own counts from the file. @@ -1041,8 +1107,8 @@ confirm it: cross terms of opposite sign can cancel in the whole and not in a su Before it composes a set it works out each chosen fix's own classes again and compares them with the file's; where one differs, it composes nothing and says so. - In the drawer, an item's difference in a set splits into its fixes' effects, exactly - where the set is exact. For a built set holding a listed pair, the rest is shown as the - interaction, never spread over the fixes. + where the set is exact, each fix's against what it was measured with. For a built set + holding a listed pair, the rest is shown as the interaction, never spread over the fixes. ### 16.9 Checks @@ -1051,17 +1117,26 @@ wrong (as section 4.3), and further one where: - `of` does not name the diagnostics file beside it, its title or one of its states, or its items, indicators, floors or words differ from that file's; +- its items are not the matrix's in the matrix's order, followed by items a fix brings, or + an item a fix brings has a base result, or no fix or built set gives it one; - the base results differ from the matrix's results of the base state, where the matrix is beside it, or from the listed items' results, where it is not; +- a fix requires itself, a fix the file does not hold, a fix after it, or one fix twice; or + the file holds no build of what it requires together; - an effect or a built set names an item or indicator it does not hold, holds an entry - equal to the base, or holds one item twice; -- a fix's counts or verdicts are not what its effect and the base give, by the rule, with - its own tolerance and noise. With every item at hand, the counts must be exact; -- a pair names a fix it does not hold, a fix twice, or is listed twice; `checked` is false - and pairs are listed; a reason or method holds text shaped like an identifier; -- a built set holds fewer than two fixes, a fix twice, or the same fixes as another; or - disagrees with the sum of its effects where the file says its fixes are independent, or - lies further from it than the measured size of the one pair it is. + equal to the result it was measured against, or holds one item twice; +- a fix's counts or verdicts are not what its effect and the build it was measured against + give, by the rule, with its own tolerance and noise. With every item at hand, the counts + must be exact; +- a pair names a fix it does not hold, a fix twice, or a fix and one it requires, or is + listed twice; its kind is not one of the five, its reach not a count of items; `checked` + is false and pairs are listed; a pair holds any other key; the method holds text shaped + like an identifier; +- a built set holds fewer than two fixes, a fix twice, a fix without one it requires, + exactly one fix with what it requires, or the same fixes as another; or disagrees with the + sum of its effects where the file says its fixes are independent, or lies further from it + than the measured size of the one pair it is; or names a state that is not after the base, + or that its results do not reproduce. `check` also prints what the file is made of: how many entries its fixes and built sets hold, and its largest fixes (a fix to a market everything uses holds about one entry per @@ -1075,8 +1150,8 @@ sentier-bafu measured its fixes on that base: a patch moves tens of entries, a f characterisation factor about 12,000 (about 250 KB), a fix to a market everything uses up to about 300,000 (6 to 7 MB). 64 MB holds about ten dense fixes and thousands of sparse ones; effects move to one file per fix only if dense fixes become common. Checked on a -made-up file of that base with 100 fixes of 500 items each in every indicator (39.5 MB), -the check takes 2.4 seconds. +made-up file of that base with 100 fixes of 500 items each in every indicator, ten of them +requiring the fix before them (39.5 MB), the check takes 2.7 seconds. ### 16.11 Acceptance cases @@ -1091,7 +1166,10 @@ Each is a test before the code is written. | F5 | A fix's verdicts are the same whichever fixes are chosen with it | | F6 | The noise of a set is the sum of its fixes' noise; noise measured for some fixes only is said, and not measured is never identical | | F7 | The page's rule agrees with the Python rule on every shared case and generated boundary value, and the page refuses to compose where it cannot reproduce a fix's own classes | -| F8 | A file of fixes holds no identifier of the model: a pair has a reason, not the activities it shares | +| F8 | A file of fixes holds nothing of the model in a pair: its kind comes from a closed list, and any text in it is refused | +| F9 | A fix that requires another is measured against their build; choosing it chooses what it requires; in a set, each effect adds on top of what it was measured with, so what two fixes build on counts once; where a build inside the set holds every fix that changes an item, its result is taken, measured, not summed | +| F10 | A built set that names a state of the chain is refused where its results do not reproduce that state | +| F11 | A fix may bring an item no state holds: it follows the matrix's items, with no base result, and a fix gives it one | ### 16.12 Who does what, and in which order @@ -1103,9 +1181,4 @@ Each is a test before the code is written. 4. The page: the list of fixes, the set in the address, the port of the rule: sentier-bafu. Open, for later: a public file of fixes (counts per fix only, no item); noise per item -(section 15); how a fix becomes a step of the chain once it is released; whether a reason -may name an activity; a fix that needs another applied first (`requires`, its effect -measured on the base and its requirements, which the page adds to any set that chooses it); -a fix that adds items the base does not hold (refused for now: an effect names base items -only); and a built set that names the chain state it equals, so that the release with every -fix is held to that state in the matrix: two builds of one thing. +(section 15); and how a fix becomes a step of the chain once it is released. diff --git a/src/sentier_diagnostics/fixes.py b/src/sentier_diagnostics/fixes.py index ad7652a..4b867a2 100644 --- a/src/sentier_diagnostics/fixes.py +++ b/src/sentier_diagnostics/fixes.py @@ -3,25 +3,36 @@ A chain measures each step with every earlier step applied, so a step's difference belongs to its place in the order. A fixes file measures each fix on its own against one state of a chain, the base: a fix has one effect, whichever set it joins and in whatever order. A -reader may then choose any set of fixes; its result is the base plus the sum of their -effects, labelled: +fix that cannot be applied alone names the fixes it needs (`requires`): its effect is +measured with them applied, against a build of them, and choosing it chooses them. - built one fix, or a set the producer built (`built`) +A reader may choose any set of fixes, and gets what they require with them. Its result is +the base plus the sum of their effects, except where a build the file holds inside the set +covers every fix that changes an item: there the build's result is taken. A set is +labelled: + + built the file holds this build: one fix with what it requires, or a set the + producer built (`built`) exact interactions were checked, and no two of its fixes are a listed pair estimated otherwise: drawn, and marked on every cell and count -A set has no verdict: each claim is judged in its own fix, against the base. A set's -tolerance is the largest of its fixes'; its noise is the sum of their noise, with the -number of fixes whose noise was measured, never read as none. +A set has no verdict: each claim is judged in its own fix. A set's tolerance is the largest +of its fixes'; its noise is the sum of their noise, with the number of fixes whose noise was +measured, never read as none. + +A listed pair holds no text: its kind of interaction (one of `PAIR_KINDS`, from where the +two changes meet in h = c·B·A⁻¹·f), how many items it can reach and the measured size of +its cross term. So nothing of the model, no name and no identifier, can go into it. `errors(f)` checks a file on its own: its shape, every fix's counts and verdicts worked out -again from its effect and the base, the pairs of fixes that can interact, and every built -set where the file calls its fixes independent: the built results must agree with the sum. -`against(f, data, full)` holds it to its diagnostics file and that file's matrix; -`beside(f, folder)` finds them next to it, as `sentier-diagnostics check` does. +again from its effect and what it was measured against, the pairs, and every built set the +file calls independent against the sum of its effects. `against(f, data, full)` holds it to +its diagnostics file and that file's matrix, and a built set that names a state of the chain +to that state; `beside(f, folder)` finds them next to it, as `sentier-diagnostics check` +does. `build(...)` makes a file from a checked diagnostics file, its matrix and the results built -with each fix; `label`, `compose` and `counts` give what a page draws for a chosen set. +with each fix; `chosen`, `label`, `compose` and `counts` give what a page draws for a set. """ from __future__ import annotations @@ -44,11 +55,21 @@ OF_KEYS = {"file", "title", "base"} #: The rule of the diagnostics file, without the chain's own total tolerance. RULE_KEYS = fileformat.RULE_KEYS - {"total_tolerance"} -FIX_KEYS = {"id", "label", "about", "kind", "explained", "link", "claim", "tolerance", "noise", "effect", - "counts"} # fmt: skip +FIX_KEYS = {"id", "label", "about", "kind", "explained", "link", "requires", "claim", "tolerance", "noise", + "effect", "counts"} # fmt: skip INTERACTION_KEYS = {"checked", "method", "by", "pairs"} -PAIR_KEYS = {"fixes", "reason", "largest"} -BUILT_KEYS = {"id", "label", "fixes", "results", "noise"} +PAIR_KEYS = {"fixes", "kind", "reach", "largest"} +BUILT_KEYS = {"id", "label", "fixes", "state", "results", "noise"} + +#: Where two changes of h = c·B·A⁻¹·f meet, so that their joint result is not the sum of +#: their effects (SPEC 16.3). Two changes to different factors, or to different entries of +#: B, add up exactly and are never a pair. +TECHNOSPHERE = "technosphere" # both change A, and an item's supply chain reaches both +TECHNOSPHERE_AND_EMISSION = "technosphere and emission" # A, and an amount of B on a chain the A change shifts +TECHNOSPHERE_AND_FACTOR = "technosphere and factor" # A, and the factor of a flow the A change shifts +EMISSION_AND_FACTOR = "emission and factor" # an amount of a flow, and that flow's factor +SAME_ENTRY = "same entry" # both change one number of A, B or c: two edits of one number do not add +PAIR_KINDS = (TECHNOSPHERE, TECHNOSPHERE_AND_EMISSION, TECHNOSPHERE_AND_FACTOR, EMISSION_AND_FACTOR, SAME_ENTRY) MAX_FIXES, MAX_BUILT = 1000, 500 MAX_ITEMS, MAX_BYTES = matrix.MAX_ITEMS, matrix.MAX_BYTES @@ -56,8 +77,8 @@ BUILT, EXACT, ESTIMATED = "built", "exact", "estimated" LABELS = (BUILT, EXACT, ESTIMATED) -#: The shape of an identifier of a model (a UUID, or 32 hexadecimal digits). The reasons of -#: interactions say what two fixes share in words, never by the model's identifiers (SPEC 16.6). +#: The shape of an identifier of a model (a UUID, or 32 hexadecimal digits): refused in the +#: one text the interactions hold, the method of the check. IDENTIFIER = re.compile( r"\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b|\b[0-9a-fA-F]{32}\b" ) @@ -66,30 +87,42 @@ # ------------------------------------------------------------------------ making a file -def fix(id: str, label: str, *, kind: str, explained: bool = True, about: str | None = None, - link: str | None = None, claim: dict | None = None, tolerance: dict | None = None, - noise: dict | None = None) -> dict: # fmt: skip - """One fix, as `build` takes it: what a step has (`chain.step`), measured against the base.""" - return chain.step(id, label, kind=kind, explained=explained, about=about, link=link, claim=claim, - tolerance=tolerance, noise=noise) # fmt: skip +def fix(id: str, label: str, *, kind: str, requires: Iterable[str] = (), explained: bool = True, + about: str | None = None, link: str | None = None, claim: dict | None = None, + tolerance: dict | None = None, noise: dict | None = None) -> dict: # fmt: skip + """One fix, as `build` takes it: what a step has (`chain.step`), and the fixes it needs + applied first.""" + one = chain.step(id, label, kind=kind, explained=explained, about=about, link=link, claim=claim, + tolerance=tolerance, noise=noise) # fmt: skip + return {**one, "requires": list(requires)} -def interactions(checked: bool = False, *, method: str | None = None, by: dict | None = None, - pairs: Iterable[tuple] = ()) -> dict: # fmt: skip - """The check of pairs: `checked` true where every pair was examined, false where none was. - `pairs` are `(fix, fix, reason)` or `(fix, fix, reason, largest)`, every pair that can - interact, each once; `largest` is the measured size of its cross term, None where it was +def pair(a: str, b: str, kind: str, *, reach: int | None = None, largest: float | None = None) -> dict: + """Two fixes that can interact: `kind` is one of `PAIR_KINDS`, `reach` how many items the + cross term can change, `largest` its measured size (SPEC 16.6); None where not counted or not measured.""" - out = [] - for a, b, reason, *largest in pairs: - out.append({"fixes": [a, b], "reason": reason, "largest": largest[0] if largest else None}) - return {"checked": checked, "method": method, "by": by, "pairs": out} + return {"fixes": [a, b], "kind": kind, "reach": reach, "largest": largest} -def built_set(id: str, label: str, fixes: Iterable[str], results: Mapping, noise: dict | None = None) -> dict: +def interactions(checked: bool = False, *, method: str | None = None, by: dict | None = None, + pairs: Iterable[dict] = ()) -> dict: # fmt: skip + """The check of pairs: `checked` true where every pair was examined, false where none was; + `pairs` are `pair(...)`, every pair that can interact, each once.""" + return {"checked": checked, "method": method, "by": by, "pairs": list(pairs)} + + +def built_set(id: str, label: str, fixes: Iterable[str], results: Mapping, *, state: str | None = None, + noise: dict | None = None) -> dict: # fmt: skip """A set the producer built: `results` are `{indicator: {item: result or None}}`, an item - left out being the base result.""" - return {"id": id, "label": label, "fixes": list(fixes), "results": results, "noise": noise or chain.noise()} + left out being the base result; `state` the id of the chain's state it is, where it is + one (the release with every fix).""" + return {"id": id, "label": label, "fixes": list(fixes), "state": state, "results": results, + "noise": noise or chain.noise()} # fmt: skip + + +def _cell(full: Mapping, k: int, x: int, at: int, nstates: int): + """One result of a matrix: item `k`, indicator `x`, state `at`.""" + return full["values"][k][x * nstates + at] def build( @@ -102,6 +135,7 @@ def build( results: Mapping[str, Mapping], interactions: dict | None = None, built: Iterable[dict] = (), + added_items: Iterable = (), file: str = chain.FILE, about: str | None = None, notes: Iterable[str] = (), @@ -110,12 +144,14 @@ def build( - `data` and `full`: the checked diagnostics file and its matrix; the items and the base results come from the matrix, in its order. - - `base`: the id of the state every fix was measured against. - - `fixes`: `fix(...)`, in the order the page lists them. - - `results`: `{fix id: {indicator: {item: result or None}}}`, built with that fix alone. - An item left out is the base result; one written equal to it is left out too. + - `base`: the id of the state every fix is measured from. + - `fixes`: `fix(...)`, in the order the page lists them; a fix's requirements come first. + - `results`: `{fix id: {indicator: {item: result or None}}}`, built with that fix and what + it requires. An item left out is the result without the fix. - `interactions`: `interactions(...)`; not checked where it is None. - `built`: `built_set(...)`. + - `added_items`: `[id, name, place, group, unit]` of items a fix brings that the base + does not hold; the base has no result for them. - `file`: the name of the diagnostics file beside it. """ ids = [i["id"] for i in data["indicators"]] @@ -123,30 +159,17 @@ def build( if base not in states: raise ValueError(f"the base {base!r} is no state of {data['title']!r}") at, nstates = states.index(base), len(states) - rows = [list(r) for r in full["items"]] + rows = [list(r) for r in full["items"]] + [list(r) for r in added_items] + m = len(full["items"]) index = {r[0]: k for k, r in enumerate(rows)} - base_values = {ind: [v[x * nstates + at] for v in full["values"]] for x, ind in enumerate(ids)} - - def sparse(given: Mapping, where: str) -> dict: - unknown = set(given) - set(ids) - if unknown: - raise ValueError(f"{where}: results of an indicator the file does not hold: {sorted(unknown)}") - out = {} - for ind in ids: - entries = [] - for item, value in (given.get(ind) or {}).items(): - if item not in index: - raise ValueError(f"{where}: {item!r} is no item of the base") - if value is not None and not rule.is_value(value): - raise ValueError(f"{where}, {ind}, {item}: {value!r} is not a finite number or None") - value = None if value is None else float(value) - if value != base_values[ind][index[item]]: - entries.append([index[item], value]) - out[ind] = sorted(entries) - return out - - known = {one["id"] for one in fixes} - unknown = set(results) - known + if len(index) != len(rows): + raise ValueError("an added item is an item of the base already") + base_values = { + ind: [_cell(full, k, x, at, nstates) for k in range(m)] + [None] * (len(rows) - m) + for x, ind in enumerate(ids) + } + known = [one["id"] for one in fixes] + unknown = set(results) - set(known) if unknown: raise ValueError(f"results of fixes that are not given: {sorted(unknown)}") f = { @@ -166,32 +189,71 @@ def sparse(given: Mapping, where: str) -> dict: "notes": list(notes), "made_by": f"sentier-diagnostics {__version__}", } + ctx = _Base(f) + + def sparse(given: Mapping, where: str, under: Mapping) -> dict: + unknown = set(given) - set(ids) + if unknown: + raise ValueError(f"{where}: results of an indicator the file does not hold: {sorted(unknown)}") + out = {} + for ind in ids: + entries = [] + for item, value in (given.get(ind) or {}).items(): + if item not in index: + raise ValueError(f"{where}: {item!r} is no item of the base, and not an added item") + if value is not None and not rule.is_value(value): + raise ValueError(f"{where}, {ind}, {item}: {value!r} is not a finite number or None") + value = None if value is None else float(value) + k = index[item] + if value != under[ind].get(k, base_values[ind][k]): + entries.append([k, value]) + out[ind] = sorted(entries, key=lambda e: e[0]) + return out + + closures: dict[str, frozenset] = {} + builds = _Builds(ctx, closures) + nothing = {ind: {} for ind in ids} + for one in built: + results_of = sparse(one["results"], f"built set {one['id']!r}", nothing) + builds.built[frozenset(one["fixes"])] = _columns(results_of) + f["built"].append({ + "id": one["id"], + "label": one["label"], + "fixes": list(one["fixes"]), + "state": one.get("state"), + "results": results_of, + "noise": one.get("noise") or chain.noise(), + }) # fmt: skip for one in fixes: + requires = list(one.get("requires") or ()) + for needed in requires: + if needed not in closures: + raise ValueError(f"fix {one['id']!r} requires {needed!r}, which is not a fix given before it") + closures[one["id"]] = frozenset(requires).union(*(closures[r] for r in requires)) + before = builds.column(closures[one["id"]]) + if before is None: + raise ValueError( + f"fix {one['id']!r}: no build of what it requires ({', '.join(sorted(closures[one['id']]))}); " + "add a built set of exactly those fixes" + ) found_claim = one.get("claim") - f["fixes"].append({ + made = { "id": one["id"], "label": one["label"], "about": one.get("about"), "kind": one["kind"], "explained": bool(one.get("explained", True)), "link": one.get("link"), + "requires": requires, "claim": claims.published(found_claim, True) if found_claim else None, "tolerance": one.get("tolerance") or chain.tolerance(), "noise": one.get("noise") or chain.noise(), - "effect": sparse(results.get(one["id"]) or {}, f"fix {one['id']!r}"), + "effect": sparse(results.get(one["id"]) or {}, f"fix {one['id']!r}", before), "counts": None, - }) # fmt: skip - for one in built: - f["built"].append({ - "id": one["id"], - "label": one["label"], - "fixes": list(one["fixes"]), - "results": sparse(one["results"], f"built set {one['id']!r}"), - "noise": one.get("noise") or chain.noise(), - }) # fmt: skip - base_ctx = _Base(f) - for one in f["fixes"]: - one["counts"] = _fix_counts(base_ctx, one, _columns(one["effect"])) + } + builds.entries[one["id"]] = _columns(made["effect"]) + made["counts"] = _fix_counts(ctx, made, builds.entries[one["id"]], before) + f["fixes"].append(made) found = errors(f) or against(f, data, full) if found: raise ValueError("not a fixes file: " + "; ".join(found[:10])) @@ -286,12 +348,12 @@ def size_note(f: Mapping) -> str: f"{len(f['items'])} items; the largest fixes: {largest}") # fmt: skip -# ------------------------------------------------------------------------ the base, worked out once +# ------------------------------------------------------------------------ the base and the builds class _Base: - """What every fix of a checked file is judged against: the base results, the floor of - every item, and the class of every item against itself (agrees, near zero or missing).""" + """The base of a checked file, worked out once: its results, the floor of every item, and + the class of every item against itself (agrees, near zero or missing).""" def __init__(self, f: Mapping): self.ids = [i["id"] for i in f["indicators"]] @@ -312,11 +374,111 @@ def __init__(self, f: Mapping): self.tally[ind] = tally +class _Builds: + """Every build a file holds, as a column over the base (`{indicator: {item index: result}}`, + the items where it is not the base): a fix with what it requires (its effect laid over the + build of its requirements) and every built set.""" + + def __init__(self, base: _Base, closures: Mapping[str, frozenset]): + self.base = base + self.closures = closures # fix id: every fix it requires, at any depth + self.entries: dict[str, dict] = {} # fix id: its written results, by indicator + self.built: dict[frozenset, dict] = {} # the fixes of a built set: its results + self._memo: dict[frozenset, dict] = {} + + @classmethod + def of(cls, f: Mapping) -> _Builds: + closures: dict[str, frozenset] = {} + for one in f["fixes"]: + closures[one["id"]] = frozenset(one["requires"]).union(*(closures[r] for r in one["requires"])) + found = cls(_Base(f), closures) + found.entries = {one["id"]: _columns(one["effect"]) for one in f["fixes"]} + found.built = {frozenset(b["fixes"]): _columns(b["results"]) for b in f["built"]} + return found + + def top(self, fixes: frozenset) -> str | None: + """The fix that, with what it requires, is exactly these fixes; None where none is.""" + return next((y for y in fixes if y in self.entries and self.closures[y] | {y} == fixes), None) + + def column(self, fixes: Iterable[str]) -> dict | None: + """The build of exactly these fixes, or None where the file holds none.""" + fixes = frozenset(fixes) + if not fixes: + return {ind: {} for ind in self.base.ids} + if fixes in self._memo: + return self._memo[fixes] + found = None + top = self.top(fixes) + if top is not None: + below = self.column(self.closures[top]) + if below is not None: + found = {ind: {**below[ind], **self.entries[top][ind]} for ind in self.base.ids} + if found is None and fixes in self.built: + found = self.built[fixes] + if found is not None: # never remember "none": a build may be added later + self._memo[fixes] = found + return found + + def before(self, fix_id: str, ind: str, k: int): + """What a fix's result at one item was measured against: the build of what it requires.""" + under = self.column(self.closures[fix_id])[ind] + return under[k] if k in under else self.base.values[ind][k] + + def covers(self, fixes: frozenset, *, built: bool = True) -> list[frozenset]: + """The builds inside a closed set that hold two fixes or more, in the order a set's + result looks for them: every fix with what it requires, in the order of the file, then + (where `built`) every built set, in the order of the file; never the set itself.""" + found = [self.closures[y] | {y} for y in self.entries if y in fixes and self.closures[y]] + if built: + found += [b for b in self.built if b < fixes] + return found + + def composed(self, fixes: frozenset, ind: str, *, built: bool = True) -> dict[int, tuple[float | None, bool]]: + """A closed set's result in one indicator, at every item one of its fixes changes: + `{item index: (result, whether it could be formed)}`. Where the fixes that change an + item all lie in one build inside the set (`covers`), the first such build's result is + the set's: measured, not summed. Otherwise the base plus each fix's difference against + what it was measured against. `built=False` leaves the built sets out: the sum a built + set is held to stands on the fixes alone.""" + touching: dict[int, list[str]] = {} + for y in fixes: + for k in self.entries[y][ind]: + touching.setdefault(k, []).append(y) + covers = self.covers(fixes, built=built) + out = {} + for k, ys in touching.items(): + if len(ys) == 1: + out[k] = (self.entries[ys[0]][ind][k], True) + continue + here = set(ys) + cover = next((c for c in covers if here <= c), None) + if cover is not None: + column = self.column(cover)[ind] + out[k] = (column[k] if k in column else self.base.values[ind][k], True) + else: + out[k] = _sum(self.base.values[ind][k], [(self.entries[y][ind][k], self.before(y, ind, k)) for y in ys]) + return out + + def _columns(sparse: Mapping) -> dict[str, dict[int, float | None]]: """`{indicator: [[item index, result], ...]}` as `{indicator: {item index: result}}`.""" return {ind: {k: v for k, v in entries} for ind, entries in sparse.items()} +def _sum(before: float | None, changes: list[tuple[float | None, float | None]]) -> tuple[float | None, bool]: + """The base plus the effects `(result, what it was measured against)` of several fixes at + one item, and whether a sum could be formed: a side without a result has no difference to add. + + >>> _sum(1.0, [(1.5, 1.0), (2.0, 1.0)]), _sum(1.0, [(1.5, 1.0), (None, 1.0)]) + ((2.5, True), (None, False)) + >>> _sum(None, [(1.0, None), (2.0, None)]), _sum(1.0, [(3.0, 2.0), (1.5, 1.0)]) + ((None, False), (2.5, True)) + """ + if before is None or any(a is None or b is None for a, b in changes): + return None, False + return before + sum(a - b for a, b in changes), True + + def _level(noise: Mapping, indicator: str) -> float | None: return (noise["by"] or {}).get(indicator) if noise["measured"] else None @@ -326,25 +488,37 @@ def _side(before, after) -> str: def _block(base: _Base, ind: str, changed: Mapping[int, float | None], tolerance: float, - noise: float | None, claim: Mapping | None) -> dict: # fmt: skip - """The counts of one indicator: every item, the base against the base with `changed` - applied, judged by the rule with `tolerance` and `noise`, and against `claim` where there - is one. Only the changed items are judged one by one: every other pair is the base - against itself, counted from the base's own classes.""" + noise: float | None, claim: Mapping | None, under: Mapping[int, float | None] | None = None) -> dict: # fmt: skip + """The counts of one indicator over every item: the column `under` (the base, with the + items `under` names laid over it) against that column with `changed` laid over it, judged + by the rule with `tolerance` and `noise`, and against `claim` where there is one. Only the + items the two columns name are judged one by one: every other pair is the base against + itself, counted from the base's own classes.""" + under = under or {} classes = dict.fromkeys(rule.CLASSES, 0) missing = dict.fromkeys(fileformat.MISSING_SIDES, 0) ways = dict.fromkeys(rule.DIRECTIONS, 0) bands = dict.fromkeys(rule.BANDS, 0) verdicts = dict.fromkeys(claims.VERDICTS, 0) if claim is not None else None values, floors, kinds = base.values[ind], base.floor[ind], base.kinds[ind] - for kind, n in base.tally[ind].items(): + # the column before, against itself + own = {k: rule.classify(v, v, floor=floors[k]) for k, v in under.items()} + tally = dict(base.tally[ind]) + for k, kind in own.items(): + tally[kinds[k]] -= 1 + tally[kind] += 1 + + def kind_of(k): + return own[k] if k in own else kinds[k] + + for kind, n in tally.items(): classes[kind] += n - missing["both"] += base.tally[ind][rule.MISSING] + missing["both"] += tally[rule.MISSING] entries = [] if claim is not None: entries = [(e, None if e["items"] == "*" else set(e["items"])) for e in claims.entries_for(claim, ind)] for k, after in changed.items(): - before, was = values[k], kinds[k] + before, was = (under[k] if k in under else values[k]), kind_of(k) classes[was] -= 1 if was == rule.MISSING: missing["both"] -= 1 @@ -361,59 +535,39 @@ def _block(base: _Base, ind: str, changed: Mapping[int, float | None], tolerance covering = [e for e, listed in entries if listed is None or item in listed] verdicts[claims.verdict(kind, before, after, way, covering)] += 1 if verdicts is not None: - # the pairs no change touched: the base against itself + # the pairs no change touched: the column before against itself musts = [listed for e, listed in entries if e["expect"] == claims.MUST] every = any(listed is None for listed in musts) must_items = set() if every else {base.index[i] for listed in musts for i in listed} for kind in (rule.AGREES, rule.NEAR_ZERO, rule.MISSING): - untouched = base.tally[ind][kind] - sum(1 for k in changed if kinds[k] == kind) + untouched = tally[kind] - sum(1 for k in changed if kind_of(k) == kind) if kind == rule.MISSING: verdicts[claims.NOT_JUDGED] += untouched continue - held = untouched if every else sum(1 for k in must_items if kinds[k] == kind and k not in changed) + held = untouched if every else sum(1 for k in must_items if kind_of(k) == kind and k not in changed) verdicts[claims.NOT_MOVED] += held verdicts[claims.UNCHANGED if kind == rule.AGREES else claims.NOT_JUDGED] += untouched - held return {"pairs": len(values), "classes": classes, "missing": missing, "directions": ways, "bands": bands, "verdicts": verdicts, "closer": None, "further": None} # fmt: skip -def _fix_counts(base: _Base, one: Mapping, effect: Mapping[str, Mapping[int, float | None]]) -> dict: - """A fix's counts: the base against the base with its effect, with its own tolerance, - noise and claim.""" +def _fix_counts(base: _Base, one: Mapping, effect: Mapping, before: Mapping) -> dict: + """A fix's counts: the build of what it requires against that build with its effect, with + its own tolerance, noise and claim.""" tol = one["tolerance"]["value"] - return {ind: _block(base, ind, effect[ind], tol, _level(one["noise"], ind), one["claim"]) for ind in base.ids} - - -def _sum(before: float | None, results: list[float | None]) -> tuple[float | None, bool]: - """The result of a set at one item: the base plus the effects of the fixes that change it, - and whether a sum could be formed. A fix without a result leaves none; a base without a - result leaves nothing to add two fixes' differences to. - - >>> _sum(1.0, [1.5]), _sum(1.0, [1.5, 2.0]), _sum(1.0, [1.5, None]), _sum(None, [1.0, 2.0]) - ((1.5, True), (2.5, True), (None, True), (None, False)) - """ - if len(results) == 1: - return results[0], True - if any(r is None for r in results): - return None, True - if before is None: - return None, False - return before + sum(r - before for r in results), True - - -def _summed(base: _Base, effects: list[Mapping], ind: str) -> dict[int, tuple[float | None, bool]]: - """The items the chosen fixes change in one indicator, with the set's result at each.""" - touched: dict[int, list] = {} - for effect in effects: - for k, value in effect[ind].items(): - touched.setdefault(k, []).append(value) - return {k: _sum(base.values[ind][k], found) for k, found in touched.items()} + return { + ind: _block(base, ind, effect[ind], tol, _level(one["noise"], ind), one["claim"], before[ind]) + for ind in base.ids + } # ------------------------------------------------------------------------ a chosen set -def _chosen(f: Mapping, ids: Iterable[str]) -> list[dict]: +def chosen(f: Mapping, ids: Iterable[str]) -> tuple[list[str], list[str]]: + """The set a reader's choice makes: the chosen fixes and every fix they require, in the + order of the file; and the fixes it added because a chosen one requires them (the page + says so).""" order = {one["id"]: k for k, one in enumerate(f["fixes"])} ids = list(ids) if not ids: @@ -423,27 +577,29 @@ def _chosen(f: Mapping, ids: Iterable[str]) -> list[dict]: raise ValueError(f"the file holds no fix {one!r}") if len(set(ids)) != len(ids): raise ValueError("a set holds each fix once") - return [f["fixes"][k] for k in sorted(order[i] for i in ids)] + builds = _Builds.of(f) + closed = set(ids).union(*(builds.closures[i] for i in ids)) + names = sorted(closed, key=order.__getitem__) + return names, [x for x in names if x not in ids] -def held_pairs(f: Mapping, ids: Iterable[str]) -> list[dict]: - """The listed pairs of a set: two of its fixes that can interact, with the reason.""" - chosen = {one["id"] for one in _chosen(f, ids)} - return [p for p in f["interactions"]["pairs"] if set(p["fixes"]) <= chosen] +def _fixes_of(f: Mapping, names: Iterable[str]) -> list[dict]: + wanted = set(names) + return [one for one in f["fixes"] if one["id"] in wanted] -def built_of(f: Mapping, ids: Iterable[str]) -> dict | None: - """The built set that holds exactly these fixes, or None.""" - chosen = {one["id"] for one in _chosen(f, ids)} - return next((b for b in f["built"] if set(b["fixes"]) == chosen), None) +def held_pairs(f: Mapping, ids: Iterable[str]) -> list[dict]: + """The listed pairs of a set: two of its fixes that can interact.""" + names = set(chosen(f, ids)[0]) + return [p for p in f["interactions"]["pairs"] if set(p["fixes"]) <= names] def label(f: Mapping, ids: Iterable[str]) -> str: """Built, exact or estimated (SPEC 16.5).""" - chosen = _chosen(f, ids) - if len(chosen) == 1 or built_of(f, ids) is not None: + names = chosen(f, ids)[0] + if _Builds.of(f).column(names) is not None: return BUILT - if f["interactions"]["checked"] and not held_pairs(f, ids): + if f["interactions"]["checked"] and not held_pairs(f, names): return EXACT return ESTIMATED @@ -460,57 +616,54 @@ def pair_terms(f: Mapping, ids: Iterable[str]) -> dict: def tolerance_of(f: Mapping, ids: Iterable[str]) -> float: """The tolerance of a set: the largest of its fixes'.""" - return max(one["tolerance"]["value"] for one in _chosen(f, ids)) + return max(one["tolerance"]["value"] for one in _fixes_of(f, chosen(f, ids)[0])) def noise_of(f: Mapping, ids: Iterable[str]) -> dict[str, dict]: """The noise of a set, per indicator: `{value, measured, fixes}`, the value being the sum of the noise of the fixes whose noise was measured there, and None where none was: not measured is never none.""" - chosen = _chosen(f, ids) + fixes = _fixes_of(f, chosen(f, ids)[0]) out = {} for ind in (i["id"] for i in f["indicators"]): - levels = [lv for lv in (_level(one["noise"], ind) for one in chosen) if lv is not None] - out[ind] = {"value": sum(levels) if levels else None, "measured": len(levels), "fixes": len(chosen)} + levels = [lv for lv in (_level(one["noise"], ind) for one in fixes) if lv is not None] + out[ind] = {"value": sum(levels) if levels else None, "measured": len(levels), "fixes": len(fixes)} return out -def _changed(f: Mapping, base: _Base, ids) -> tuple[dict, dict]: - """The items a set changes, per indicator, with its result at each; and the items where - no sum can be formed (a missing base under two fixes or more).""" - chosen = _chosen(f, ids) - found = built_of(f, ids) - if len(chosen) == 1 or found is not None: - source = chosen[0]["effect"] if found is None else found["results"] - return _columns(source), {ind: [] for ind in base.ids} - effects = [_columns(one["effect"]) for one in chosen] +def _changed(f: Mapping, names: list[str]) -> tuple[_Builds, dict, dict]: + """The items a closed set changes, per indicator, with its result at each; and the items + where no sum can be formed.""" + builds = _Builds.of(f) + column = builds.column(names) + if column is not None: + return builds, column, {ind: [] for ind in builds.base.ids} changed, cannot = {}, {} - for ind in base.ids: - cells = _summed(base, effects, ind) + for ind in builds.base.ids: + cells = builds.composed(frozenset(names), ind) changed[ind] = {k: v for k, (v, _) in cells.items()} cannot[ind] = sorted(k for k, (_, ok) in cells.items() if not ok) - return changed, cannot + return builds, changed, cannot def compose(f: Mapping, ids: Iterable[str]) -> dict: """What a page draws for a set: its fixes in the order of the file (each with its own - claim and counts), its label, the listed pairs it holds, its tolerance and noise, and its - result for every item, `{indicator: [result or None, ...]}` in the order of `items`. - `cannot_add` names, per indicator, the item indices whose result is None because the base - has none to add two fixes' differences to, so that it reads apart from a missing result. - A set has no verdict.""" - chosen = _chosen(f, ids) - names = [one["id"] for one in chosen] - base = _Base(f) - changed, cannot = _changed(f, base, names) + claim and counts), the fixes added because a chosen one requires them, its label, the + listed pairs it holds and their pair terms, its tolerance and noise, and its result for + every item, `{indicator: [result or None, ...]}` in the order of `items`. `cannot_add` + names, per indicator, the item indices whose result is None because no sum can be formed + there, so that it reads apart from a missing result. A set has no verdict.""" + names, added = chosen(f, ids) + builds, changed, cannot = _changed(f, names) results = {} - for ind in base.ids: - column = list(base.values[ind]) + for ind in builds.base.ids: + column = list(builds.base.values[ind]) for k, value in changed[ind].items(): column[k] = value results[ind] = column return { - "fixes": [{k: one[k] for k in ("id", "label", "claim", "counts")} for one in chosen], + "fixes": [{k: one[k] for k in ("id", "label", "requires", "claim", "counts")} for one in _fixes_of(f, names)], + "added": added, "label": label(f, names), "pairs": held_pairs(f, names), "pair_terms": pair_terms(f, names), @@ -524,13 +677,12 @@ def compose(f: Mapping, ids: Iterable[str]) -> dict: def counts(f: Mapping, ids: Iterable[str]) -> dict: """The counts of a set against the base, per indicator, with its label beside them: an estimated set's counts are estimates ("estimated: 153 would differ"). No verdicts.""" - names = [one["id"] for one in _chosen(f, ids)] - base = _Base(f) - changed, _ = _changed(f, base, names) + names = chosen(f, ids)[0] + builds, changed, _ = _changed(f, names) tol, noise = tolerance_of(f, names), noise_of(f, names) return { "label": label(f, names), - "by": {ind: _block(base, ind, changed[ind], tol, noise[ind]["value"], None) for ind in base.ids}, + "by": {ind: _block(builds.base, ind, changed[ind], tol, noise[ind]["value"], None) for ind in builds.base.ids}, } @@ -599,12 +751,16 @@ def _errors(f, found: list[str]) -> list[str]: if found: return found # nothing can be judged against a base that cannot be read base = _Base(f) - effects = _fixes(f, found, base) - if effects is None: + closures = _requirements(f, found) + if closures is None: + return found + builds = _Builds(base, closures) + built = _built_sets(f, found, base, builds) + if built is None or not _fixes(f, found, base, builds): return found - pairs = _interactions(f, found, effects) + pairs = _interactions(f, found, closures, n) if pairs is not None: - _built(f, found, base, effects, pairs) + _built_checks(f, found, builds, built, pairs) return found @@ -665,9 +821,10 @@ def _items(f, found, indicators, floors) -> bool: return True -def _sparse(said, where: str, base: _Base, found: list[str]) -> dict | None: +def _sparse(said, where: str, base: _Base, found: list[str], under: Mapping | None = None) -> dict | None: """An effect, or the results of a built set: per indicator, the items whose result is not - the base result, `[[item index, result or null], ...]`.""" + the one it was measured against (`under`, laid over the base), `[[item index, result or + null], ...]`.""" if not isinstance(said, dict) or set(said) != set(base.ids): found.append(f"{where}: not one list per indicator of the file") return None @@ -679,6 +836,7 @@ def _sparse(said, where: str, base: _Base, found: list[str]) -> dict | None: found.append(f"{where}, {ind}: not a list of at most one entry per item") return None column = {} + against_ = (under or {}).get(ind) or {} for entry in entries: if not ( isinstance(entry, list) @@ -695,8 +853,11 @@ def _sparse(said, where: str, base: _Base, found: list[str]) -> dict | None: if k in column: found.append(f"{where}, {ind}: holds item {base.items[k]!r} twice") return None - if value == base.values[ind][k]: - found.append(f"{where}, {ind}: item {base.items[k]!r} is written with the base result, as if it changed") + if value == (against_[k] if k in against_ else base.values[ind][k]): + found.append( + f"{where}, {ind}: item {base.items[k]!r} is written with the result it was measured " + "against, as if it changed" + ) return None column[k] = value out[ind] = column @@ -726,21 +887,109 @@ def _noise_errors(noise, where: str, indicators) -> list[str]: return [] -def _fixes(f, found, base: _Base) -> dict | None: - """Every fix, its counts and verdicts worked out again; its effect, by id.""" +def _requirements(f, found) -> dict[str, frozenset] | None: + """Every fix's id and what it requires, at any depth. A fix's requirements come before it + in `fixes`, so that no fix requires itself through others.""" said = f["fixes"] if not isinstance(said, list) or not 0 < len(said) <= MAX_FIXES: found.append(f"`fixes` is not a list of 1 to {MAX_FIXES} fixes") return None - effects = {} for one in said: if not isinstance(one, dict) or set(one) != FIX_KEYS or not fileformat.is_text(one["id"], fileformat.MAX_ID): found.append(f"a fix is not {{{', '.join(sorted(FIX_KEYS))}}}") return None + every = [one["id"] for one in said] + if len(set(every)) != len(every): + found.append("two fixes have the same id") + return None + closures: dict[str, frozenset] = {} + for one in said: where = f"fix {one['id']!r}" - if one["id"] in effects: - found.append(f"two fixes have the id {one['id']!r}") + needs = one["requires"] + if not (isinstance(needs, list) and all(isinstance(x, str) for x in needs)): + found.append(f"{where}: `requires` is not a list of the ids of fixes") + return None + if len(set(needs)) != len(needs): + found.append(f"{where}: requires a fix twice") + return None + for x in needs: + if x == one["id"]: + found.append(f"{where}: requires itself") + return None + if x not in every: + found.append(f"{where}: requires {x!r}, and the file holds no such fix") + return None + if x not in closures: + found.append(f"{where}: requires {x!r}, which comes after it: a fix's requirements come first") + return None + closures[one["id"]] = frozenset(needs).union(*(closures[x] for x in needs)) + return closures + + +def _built_sets(f, found, base: _Base, builds: _Builds) -> list | None: + """Every built set, its shape and results; added to `builds`. None where one is wrong: + a fix measured against a build the file does not hold cannot be checked.""" + said = f["built"] + if not isinstance(said, list) or len(said) > MAX_BUILT: + found.append(f"`built` is not a list of at most {MAX_BUILT} sets") + return None + closures = builds.closures + out, ids, sets, before = [], set(), set(), len(found) + for one in said: + if not isinstance(one, dict) or set(one) != BUILT_KEYS or not fileformat.is_text(one["id"], fileformat.MAX_ID): + found.append(f"a built set is not {{{', '.join(sorted(BUILT_KEYS))}}}") return None + where = f"built set {one['id']!r}" + if one["id"] in ids: + found.append(f"two built sets have the id {one['id']!r}") + ids.add(one["id"]) + if not fileformat.is_text(one["label"], fileformat.MAX_LABEL): + found.append(f"{where}: `label` is not a short text") + if not fileformat.is_optional(one["state"], fileformat.MAX_ID): + found.append(f"{where}: `state` is not the id of a state, or null") + named = one["fixes"] + if not (isinstance(named, list) and all(isinstance(x, str) for x in named)): + found.append(f"{where}: `fixes` is not a list of the ids of fixes") + continue + unknown = [x for x in named if x not in closures] + if unknown: + found.append(f"{where}: the file holds no fix {unknown[0]!r}") + continue + if len(set(named)) != len(named): + found.append(f"{where}: holds a fix twice") + continue + if len(named) < 2: + found.append(f"{where}: holds fewer than two fixes: one fix is built already") + continue + key = frozenset(named) + lacking = sorted((x, r) for x in named for r in closures[x] - key) + if lacking: + found.append(f"{where}: holds {lacking[0][0]!r} without {lacking[0][1]!r}, which it requires") + continue + top = next((y for y in named if closures[y] | {y} == key), None) + if top is not None: + found.append(f"{where}: is built already: it is fix {top!r} with what it requires") + continue + if key in sets: + found.append(f"{where}: holds the same fixes as another built set") + sets.add(key) + noise = _noise_errors(one["noise"], where, base.ids) + if noise: + found += noise + continue + results = _sparse(one["results"], f"{where}: its results", base, found) + if results is None: + continue + builds.built[key] = results + out.append((one, key, results)) + return None if len(found) > before else out + + +def _fixes(f, found, base: _Base, builds: _Builds) -> bool: + """Every fix: its texts, claim, tolerance and noise; its effect, measured against the build + of what it requires; and its counts and verdicts, worked out again.""" + for one in f["fixes"]: + where = f"fix {one['id']!r}" if not fileformat.is_text(one["label"], fileformat.MAX_LABEL) or not fileformat.is_optional(one["about"]): found.append(f"{where}: `label` or `about` is not a text") if not fileformat.is_text(one["kind"], fileformat.MAX_WORD) or not isinstance(one["explained"], bool): @@ -751,14 +1000,14 @@ def _fixes(f, found, base: _Base) -> dict | None: shape = fileformat.claim_errors(one["claim"], where, base.ids, "items") if shape: found += shape - return None + return False if any(e["items"] is None for e in one["claim"]["scope"]): found.append(f"{where}: a fix's claim names its items: a page works out each item's verdict from it") - return None + return False unknown = {i for e in one["claim"]["scope"] if e["items"] != "*" for i in e["items"]} - set(base.index) if unknown: found.append(f"{where}: its claim names {len(unknown)} items the file does not hold") - return None + return False tol = one["tolerance"] if not ( isinstance(tol, dict) @@ -768,35 +1017,43 @@ def _fixes(f, found, base: _Base) -> dict | None: and fileformat.is_optional(tol["note"]) ): found.append(f"{where}: `tolerance` is not {{value, basis, note}}") - return None + return False noise = _noise_errors(one["noise"], where, base.ids) if noise: found += noise - return None - effect = _sparse(one["effect"], f"{where}: its effect", base, found) + return False + before = builds.column(builds.closures[one["id"]]) + if before is None: + needs = ", ".join(repr(x) for x in sorted(builds.closures[one["id"]])) + found.append( + f"{where}: it requires {needs}, and the file holds no build of them together (a built set of " + "exactly those fixes): its effect has nothing to be measured against" + ) + return False + effect = _sparse(one["effect"], f"{where}: its effect", base, found, before) if effect is None: - return None - effects[one["id"]] = effect + return False + builds.entries[one["id"]] = effect written = one["counts"] if not isinstance(written, dict) or set(written) != set(base.ids): found.append(f"{where}: `counts` is not one block per indicator") - return None + return False shape = [e for ind in base.ids for e in fileformat._block(written[ind], f"{where}, counts of {ind!r}", one)] shape += [f"{where}, counts of {ind!r}: `{key}` is null: a fix has no published results to come closer to" for ind in base.ids for key in ("closer", "further") if written[ind][key] is not None] # fmt: skip if shape: found += shape continue - again = _fix_counts(base, one, effect) + again = _fix_counts(base, one, effect, before) for ind in base.ids: for key, value in again[ind].items(): if written[ind][key] != value: - found.append(f"{where}, counts of {ind!r}: `{key}` is not what its effect and the base give") - return effects + found.append(f"{where}, counts of {ind!r}: `{key}` is not what its effect gives") + return True -def _interactions(f, found, effects: Mapping) -> set[frozenset] | None: - """The pairs of fixes that can interact, each once, as sets of two ids.""" +def _interactions(f, found, closures: Mapping[str, frozenset], n: int) -> dict[frozenset, dict] | None: + """The pairs of fixes that can interact, each once, by their two ids.""" said = f["interactions"] if not ( isinstance(said, dict) @@ -827,112 +1084,92 @@ def _interactions(f, found, effects: Mapping) -> set[frozenset] | None: if len(said["pairs"]) > MAX_FIXES * (MAX_FIXES - 1) // 2: found.append("`interactions.pairs` lists more pairs than fixes can make") return None - pairs: set[frozenset] = set() + pairs: dict[frozenset, dict] = {} for one in said["pairs"]: if not isinstance(one, dict) or set(one) != PAIR_KEYS: found.append( - "a pair is not {fixes, reason, largest}: it says why in words, not what in the model they share" + "a pair is not {fixes, kind, reach, largest}: it holds no text, so that nothing of the model " + "goes into it" ) return None named = one["fixes"] if not (isinstance(named, list) and len(named) == 2 and all(isinstance(x, str) for x in named)): found.append("a pair does not name two fixes") return None - where = f"the pair {named[0]!r} and {named[1]!r}" - unknown = [x for x in named if x not in effects] + a, b = named + where = f"the pair {a!r} and {b!r}" + unknown = [x for x in named if x not in closures] if unknown: found.append(f"{where}: the file holds no fix {unknown[0]!r}") continue - if named[0] == named[1]: + if a == b: found.append(f"{where}: names one fix twice") continue + if a in closures[b] or b in closures[a]: + found.append(f"{where}: one requires the other, so its effect is measured with the other applied") + continue + if one["kind"] not in PAIR_KINDS: + found.append(f"{where}: `kind` is not one of {', '.join(PAIR_KINDS)}") + if one["reach"] is not None and not (fileformat.is_count(one["reach"]) and 0 < one["reach"] <= n): + found.append(f"{where}: `reach` is not a count of 1 to {n} items, or null") if one["largest"] is not None and not fileformat.is_share(one["largest"]): found.append(f"{where}: `largest` is not a share, or null where it was not measured") - if not fileformat.is_text(one["reason"], 300): - found.append(f"{where}: its reason is not a short text") - elif IDENTIFIER.search(one["reason"]): - found.append(f"{where}: its reason holds an identifier of the model: say what they share in words") key = frozenset(named) if key in pairs: found.append(f"{where}: is listed twice") - pairs.add(key) + pairs[key] = one return pairs -def _sizes(f) -> dict[frozenset, float]: - """The measured size of every listed pair that has one.""" - return {frozenset(p["fixes"]): p["largest"] for p in f["interactions"]["pairs"] if p["largest"] is not None} - - -def _built(f, found, base: _Base, effects: Mapping, pairs: set[frozenset]) -> None: - said = f["built"] - if not isinstance(said, list) or len(said) > MAX_BUILT: - found.append(f"`built` is not a list of at most {MAX_BUILT} sets") +def _built_checks(f, found, builds: _Builds, built: list, pairs: Mapping[frozenset, dict]) -> None: + """A built set the file calls independent must be the sum of its effects; a built pair + whose cross term the file measured must lie within it.""" + if not f["interactions"]["checked"]: return - checked = f["interactions"]["checked"] - sizes = _sizes(f) by_id = {one["id"]: one for one in f["fixes"]} - ids, sets = set(), set() - for one in said: - if not isinstance(one, dict) or set(one) != BUILT_KEYS or not fileformat.is_text(one["id"], fileformat.MAX_ID): - found.append(f"a built set is not {{{', '.join(sorted(BUILT_KEYS))}}}") - return + for one, key, results in built: where = f"built set {one['id']!r}" - if one["id"] in ids: - found.append(f"two built sets have the id {one['id']!r}") - ids.add(one["id"]) - if not fileformat.is_text(one["label"], fileformat.MAX_LABEL): - found.append(f"{where}: `label` is not a short text") - named = one["fixes"] - if not (isinstance(named, list) and all(isinstance(x, str) for x in named)): - found.append(f"{where}: `fixes` is not a list of the ids of fixes") - continue - unknown = [x for x in named if x not in effects] - if unknown: - found.append(f"{where}: the file holds no fix {unknown[0]!r}") - continue - if len(set(named)) != len(named): - found.append(f"{where}: holds a fix twice") - continue - if len(named) < 2: - found.append(f"{where}: holds fewer than two fixes: one fix is built already") - continue - key = frozenset(named) - if key in sets: - found.append(f"{where}: holds the same fixes as another built set") - sets.add(key) - noise = _noise_errors(one["noise"], where, base.ids) - if noise: - found += noise - continue - results = _sparse(one["results"], f"{where}: its results", base, found) - if results is None or not checked: - continue - chosen = [by_id[x] for x in named] - if not any(p <= key for p in pairs): - found += _disagreements(base, where, chosen, [effects[x] for x in named], one, results) - elif key in sizes: # one listed pair, built: its measured size is held to the build - found += _disagreements(base, where, chosen, [effects[x] for x in named], one, results, sizes[key]) + held = [p for p in pairs if p <= key] + if not held: + found += _disagreements(builds, where, key, by_id, one, results) + elif held == [key] and pairs[key]["largest"] is not None: + found += _disagreements(builds, where, key, by_id, one, results, pairs[key]["largest"]) #: How many disagreements of one built set are named before the rest are counted. NAMED = 5 -def _disagreements(base: _Base, where: str, chosen: list, effects: list, one: Mapping, results: Mapping, +def _number(value: float | None) -> str: + """A result in an error, to ten significant digits: the sum of 1.0006 and 1.0007 over + 1.0 reads 1.0013, not 1.0012999999999999. + + >>> _number(1.0012999999999999), _number(None), _number(-2.5e-12) + ('1.0013', 'null', '-2.5e-12') + """ + return "null" if value is None else f"{value:.10g}" + + +def _tolerance_and_noise(fixes: list[Mapping], ind: str, own: Mapping | None = None) -> tuple[float, float | None]: + """The tolerance of a set (the largest of its fixes') and its noise (the sum of theirs, + and of `own` where a built set's own noise is measured); None where none is measured.""" + noises = [x["noise"] for x in fixes] + ([own] if own else []) + levels = [lv for lv in (_level(x, ind) for x in noises) if lv is not None] + return max(x["tolerance"]["value"] for x in fixes), (sum(levels) if levels else None) + + +def _disagreements(builds: _Builds, where: str, key: frozenset, by_id: Mapping, one: Mapping, results: Mapping, largest: float | None = None) -> list[str]: # fmt: skip - """Where a built set is further from the sum of its effects than the file says: by the - rule, with the largest tolerance of its fixes and the sum of their noise and its own. A - set the file calls independent must be the sum; a built pair whose cross term the file - measured (`largest`) must lie within it.""" - tol = max(x["tolerance"]["value"] for x in chosen) - if largest is not None: - tol = max(tol, largest) + """Where a built set is further from the sum of its effects than the file says, by the + rule, with the set's tolerance and the sum of its fixes' noise and its own.""" + base = builds.base + fixes = [by_id[x] for x in sorted(key)] found, more = [], 0 for ind in base.ids: - levels = [lv for lv in (_level(x["noise"], ind) for x in [*chosen, one]) if lv is not None] - level = sum(levels) if levels else None - cells = _summed(base, effects, ind) + tol, level = _tolerance_and_noise(fixes, ind, one["noise"]) + if largest is not None: + tol = max(tol, largest) + cells = builds.composed(key, ind, built=False) for k in sorted(set(cells) | set(results[ind])): before = base.values[ind][k] summed, ok = cells.get(k, (before, True)) @@ -946,7 +1183,8 @@ def _disagreements(base: _Base, where: str, chosen: list, effects: list, one: Ma said = ("the file calls its fixes independent, and they are not" if largest is None else f"the file measured their cross term at {largest:g} at most, and it is larger") # fmt: skip found.append( - f"{where}, item {base.items[k]!r}, {ind}: built {got!r}, the sum of its effects is {summed!r} " + f"{where}, item {base.items[k]!r}, {ind}: built {_number(got)}, the sum of its effects is " + f"{_number(summed)} " f"({kind}); {said}" ) else: @@ -962,7 +1200,7 @@ def _disagreements(base: _Base, where: str, chosen: list, effects: list, one: Ma def against(f: Mapping, data: Mapping, full: Mapping | None = None) -> list[str]: """Everything a checked fixes file and its checked diagnostics file disagree on. Its items and base results are held to the matrix `full` where it is given, and to the listed items - where it is not.""" + where it is not; so is a built set that names a state of the chain.""" try: return _against(f, data, full) except Exception as error: # noqa: BLE001 - a backstop, as in `errors` @@ -983,28 +1221,88 @@ def _against(f: Mapping, data: Mapping, full: Mapping | None) -> list[str]: found.append("its words for an item or an indicator differ from the diagnostics file's") if f["rule"] != {k: data["rule"][k] for k in RULE_KEYS}: found.append("its floors differ from the diagnostics file's") + for one in f["built"]: + later = one["state"] in states and states.index(one["state"]) > states.index(of["base"]) + if one["state"] is not None and not later: + found.append(f"built set {one['id']!r}: `state` {one['state']!r} is not a state of the chain after the base") if found: return found ids = [i["id"] for i in data["indicators"]] at, nstates = states.index(of["base"]), len(states) if full is not None: - if [list(r) for r in f["items"]] != [list(r) for r in full["items"]]: - return ["its items are not the matrix's, in the matrix's order"] + m = len(full["items"]) + if [list(r) for r in f["items"][:m]] != [list(r) for r in full["items"]]: + return ["its items are not the matrix's, in the matrix's order, before any item a fix adds"] for x, ind in enumerate(ids): - if [row[x * nstates + at] for row in full["values"]] != f["base"][ind]: + if [_cell(full, k, x, at, nstates) for k in range(m)] != f["base"][ind][:m]: found.append(f"the base results in {ind!r} are not the matrix's results of state {of['base']!r}") + found += _added_items(f, m) + found += _states(f, data, lambda k, x, s: _cell(full, k, x, states.index(s), nstates) if k < m else None, + range(len(f["items"]))) # fmt: skip return found - if len(f["items"]) != data["items_count"]["all"]: + if len(f["items"]) < data["items_count"]["all"]: found.append(f"it holds {len(f['items'])} items, and the diagnostics file counts {data['items_count']['all']}") index = {row[0]: k for k, row in enumerate(f["items"])} + listed = {} for item in data["items"]: k = index.get(item["id"]) if k is None: found.append(f"the listed item {item['id']!r} is not in the fixes file") continue + listed[k] = item if f["items"][k][1:] != [item["name"], item["place"], item["group"], item["unit"]]: found.append(f"item {item['id']!r}: its name, place, group or unit differ from the diagnostics file") for ind in ids: if item["values"][ind][at] != f["base"][ind][k]: found.append(f"item {item['id']!r}, {ind}: the base result is not the diagnostics file's") + found += _states(f, data, lambda k, x, s: listed[k]["values"][ids[x]][states.index(s)], sorted(listed)) + return found + + +def _added_items(f: Mapping, m: int) -> list[str]: + """The items after the matrix's: a fix brings them, so the base has no result for them and + a fix or a built set gives one.""" + found = [] + given = set() + for one in [*f["fixes"], *f["built"]]: + for entries in (one["effect"] if "effect" in one else one["results"]).values(): + given.update(k for k, v in entries if k >= m and v is not None) + for k in range(m, len(f["items"])): + name = f["items"][k][0] + if any(f["base"][ind][k] is not None for ind in f["base"]): + found.append(f"item {name!r} is not in the matrix, and the base has a result for it") + elif k not in given: + found.append(f"item {name!r} is in no state of the chain, and no fix gives it a result") + return found + + +def _states(f: Mapping, data: Mapping, cell, items) -> list[str]: + """A built set that names a state of the chain is two builds of one thing: by the rule, + with the set's tolerance and the sum of its fixes' noise and its own.""" + found = [] + by_id = {one["id"]: one for one in f["fixes"]} + floors = _Base(f).floor + for one in f["built"]: + if one["state"] is None: + continue + where = f"built set {one['id']!r}" + fixes = [by_id[x] for x in one["fixes"]] + results = _columns(one["results"]) + named = 0 + for x, ind in enumerate(f["base"]): + tol, level = _tolerance_and_noise(fixes, ind, one["noise"]) + for k in items: + got = results[ind].get(k, f["base"][ind][k]) + there = cell(k, x, one["state"]) + kind = rule.classify(there, got, floor=floors[ind][k], tolerance=tol, noise=level) + if kind in claims.STILL or (kind == rule.MISSING and there is None and got is None): + continue + named += 1 + if named <= NAMED: + found.append( + f"{where}, item {f['items'][k][0]!r}, {ind}: built {_number(got)}, and state " + f"{one['state']!r} of the chain, which it says it is, holds {_number(there)} ({kind})" + ) + if named > NAMED: + found.append(f"{where}: and {named - NAMED} more pairs where it is not state {one['state']!r}") return found diff --git a/tests/test_fixes.py b/tests/test_fixes.py index c02864a..77cf3ba 100644 --- a/tests/test_fixes.py +++ b/tests/test_fixes.py @@ -18,18 +18,24 @@ FLOORS = {(A, "kg"): 0.01, (B, "kg"): 0.01} -def chain_and_matrix(last=None): +def chain_and_matrix(last=None, release=None): """A chain of two states and its matrix; the base is the second state. Item p3 moved in - the step, so it is the one listed item.""" + the step, so it is the one listed item. With `release`, a third state after the base.""" s1 = last or {A: {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0}, B: {"p1": 3.0, "p2": 5.0, "p3": 7.0, "p4": 1.0}} s0 = {A: {**s1[A], "p3": 4.0}, B: dict(s1[B])} items = sorted({i for column in s0.values() for i in column} | {i for column in s1.values() for i in column}) + states = [chain.state("s0", "Before", kind="published"), chain.state("s1", "After")] + steps, values = [chain.step("one", "One step", kind="fix")], {"s0": s0, "s1": s1} + if release is not None: + states.append(chain.state("s2", "Release")) + steps.append(chain.step("two", "Every fix", kind="fix")) + values["s2"] = release return chain.build_with_matrix( "Two states", indicators=INDICATORS, - states=[chain.state("s0", "Before", kind="published"), chain.state("s1", "After")], - steps=[chain.step("one", "One step", kind="fix")], - values={"s0": s0, "s1": s1}, + states=states, + steps=steps, + values=values, units=dict.fromkeys(items, "kg"), floors=FLOORS, ) @@ -40,17 +46,23 @@ def checked_pairs(*pairs): by=claims.origin("code", "pair check"), pairs=pairs) # fmt: skip -def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", built=True): +#: The base with every fix of `made`: a and b at p1, c at p3. +RELEASE = {A: {"p1": 1.0013, "p2": 2.0, "p3": 2.0, "p4": 0.0}, B: {"p1": 3.0, "p2": 5.0, "p3": 6.0, "p4": 1.0}} + + +def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", built=True, release=False): """Three fixes against the base: - a and b each move p1 in A by less than their tolerance (1e-3): 1.0 to 1.0006 and 1.0007; - c moves p3 down in both indicators, as its claim says it must; - b and c can interact (a listed pair); the set of a and b was built. - """ - data, full = chain_and_matrix() + + With `release`, the chain has a third state, the base with every fix, and a built set of + every fix says it is that state.""" + data, full = chain_and_matrix(release=RELEASE if release else None) claim_c = claims.claim("p3 goes down.", [claims.scope([A], {"p3"}, "down", 0.5, claims.MUST)], by=claims.origin("person", "A. Author")) # fmt: skip - pairs = {"listed": checked_pairs(("b", "c", "both change one market")), "none": checked_pairs(), + pairs = {"listed": checked_pairs(fixes.pair("b", "c", fixes.TECHNOSPHERE)), "none": checked_pairs(), "unchecked": fixes.interactions(False)}[interactions] # fmt: skip f = fixes.build( "Fixes of two states", @@ -64,19 +76,27 @@ def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", ], results={"a": {A: {"p1": 1.0006}}, "b": {A: {"p1": 1.0007}}, "c": {A: {"p3": 2.0}, B: {"p3": 6.0}}}, interactions=pairs, - built=[fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": built_ab}})] if built else (), + built=[ + *([fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": built_ab}})] if built else ()), + *([fixes.built_set("all", "Every fix", ["a", "b", "c"], RELEASE, state="s2")] if release else ()), + ], ) return f, data, full -def pair(a, b, reason="r", largest=None): - return {"fixes": [a, b], "reason": reason, "largest": largest} +def pair(a, b, kind=fixes.TECHNOSPHERE, largest=None, reach=None): + return fixes.pair(a, b, kind, reach=reach, largest=largest) def at(f, item): return [row[0] for row in f["items"]].index(item) +def named(f, column): + """A column of results by item: `{item: result}`.""" + return {row[0]: v for row, v in zip(f["items"], column)} + + def changed(f, change): return helpers.mutated(f, change) @@ -199,7 +219,7 @@ def test_an_effect_names_known_items_once_and_never_the_base(self): cases = { "names item 99, and the file holds 4": lambda g: g["fixes"][0]["effect"][A].append([99, 1.0]), "holds item 'p1' twice": lambda g: g["fixes"][0]["effect"][A].append([k1, 1.5]), - "is written with the base result": lambda g: g["fixes"][0]["effect"][A].append([k2, 2.0]), + "is written with the result it was measured against": lambda g: g["fixes"][0]["effect"][A].append([k2, 2.0]), "not one list per indicator": lambda g: g["fixes"][0]["effect"].__setitem__("x:z", []), "an entry is not [item index, result or null]": lambda g: g["fixes"][0]["effect"][B].append([k1]), } @@ -212,7 +232,7 @@ def test_built_results_are_held_to_the_same_rules(self): k1 = at(f, "p1") cases = { "holds item 'p1' twice": lambda g: g["built"][0]["results"][A].append([k1, 1.5]), - "is written with the base result": lambda g: g["built"][0]["results"][B].append([k1, 3.0]), + "is written with the result it was measured against": lambda g: g["built"][0]["results"][B].append([k1, 3.0]), "names item 7": lambda g: g["built"][0]["results"][B].append([7, 3.0]), } for part, change in cases.items(): @@ -257,11 +277,19 @@ def test_pairs_name_two_known_fixes_once(self): "is listed twice": lambda g: g["interactions"]["pairs"].append(pair("c", "b")), "does not name two fixes": lambda g: g["interactions"]["pairs"].append({**pair("a", "b"), "fixes": ["a"]}), "`largest` is not a share": lambda g: g["interactions"]["pairs"][0].__setitem__("largest", -0.1), + "`kind` is not one of": lambda g: g["interactions"]["pairs"][0].__setitem__("kind", "market"), + "`reach` is not a count of 1 to 4 items": lambda g: g["interactions"]["pairs"][0].__setitem__("reach", 5), } for part, change in cases.items(): with self.subTest(part): self.assertTrue(said(fixes.errors(changed(f, change)), part)) + def test_a_pair_reaches_at_least_one_item(self): + f, *_ = made() + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["interactions"]["pairs"][0].__setitem__("reach", 0))), + "`reach` is not a count of 1 to 4 items, or null")) # fmt: skip + self.assertEqual(fixes.errors(changed(f, lambda g: g["interactions"]["pairs"][0].__setitem__("reach", 4))), []) + def test_no_half_check(self): f, *_ = made() listed = changed(f, lambda g: g["interactions"].update(checked=False, method=None, by=None)) @@ -280,7 +308,8 @@ def test_a_built_set_holds_two_or_more_fixes_once_and_is_not_built_twice(self): noise = chain.noise() def add(names): - return lambda g: g["built"].append({"id": "x", "label": "X", "fixes": names, "results": empty, "noise": noise}) + return lambda g: g["built"].append({"id": "x", "label": "X", "fixes": names, "state": None, "results": empty, + "noise": noise}) # fmt: skip cases = { "fewer than two fixes": add(["a"]), @@ -292,12 +321,16 @@ def add(names): with self.subTest(part): self.assertTrue(said(fixes.errors(changed(f, change)), part)) twice = changed(f, lambda g: g["built"].append(dict(g["built"][0], fixes=["a", "c"], results=empty))) + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["built"][0].__setitem__("state", 5))), + "`state` is not the id of a state")) # fmt: skip self.assertTrue(said(fixes.errors(twice), "two built sets have the id 'ab'")) -def dense(f, fix_id): +def dense(f, fix_id, under=None): """A fix's counts worked out item by item, as `chain` counts a step: the reference the - sparse count is held to.""" + sparse count is held to. `under` is what the fix was measured against, written out by + hand (`{indicator: {item index: result}}`), the base where it says nothing.""" + under = under or {} one = next(x for x in f["fixes"] if x["id"] == fix_id) floors = {(x["indicator"], x["unit"]): x["value"] for x in f["rule"]["floors"]} out = {} @@ -307,7 +340,7 @@ def dense(f, fix_id): block = {"classes": dict.fromkeys(rule.CLASSES, 0), "missing": {"before": 0, "after": 0, "both": 0}, "verdicts": dict.fromkeys(claims.VERDICTS, 0) if one["claim"] else None} # fmt: skip for k, row in enumerate(f["items"]): - before = f["base"][ind][k] + before = under.get(ind, {}).get(k, f["base"][ind][k]) after = effect.get(k, before) kind, way, _ = rule.judge(before, after, floor=floors[(ind, row[4])], tolerance=one["tolerance"]["value"], noise=level) # fmt: skip @@ -357,7 +390,7 @@ def test_f2_a_set_with_a_listed_pair_is_estimated_everywhere(self): f, *_ = made() drawn = fixes.compose(f, ["c", "b"]) self.assertEqual(drawn["label"], fixes.ESTIMATED) - self.assertEqual(drawn["pairs"], [pair("b", "c", "both change one market")]) + self.assertEqual(drawn["pairs"], [pair("b", "c")]) self.assertEqual(drawn["pair_terms"], {"value": None, "measured": 0, "pairs": 1}) self.assertEqual(fixes.counts(f, ["b", "c"])["label"], fixes.ESTIMATED) self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.ESTIMATED) # a built subset makes no set built @@ -379,12 +412,28 @@ def test_f4_a_built_set_that_is_not_the_sum_of_independent_fixes_is_refused(self self.assertEqual(fixes.errors(f), []) def interacting(g): g["built"][0]["results"][A][0] = [at(g, "p1"), 1.0030] - g["interactions"]["pairs"].append(pair("a", "b", "both change one factor")) + g["interactions"]["pairs"].append(pair("a", "b", fixes.SAME_ENTRY)) self.assertEqual(fixes.errors(changed(f, interacting)), [], "a listed pair is an interaction, shown, not refused") f, *_ = made(interactions="unchecked", built_ab=1.0030) self.assertEqual(fixes.errors(f), [], "without a check the file claims no independence") + def test_f4_a_built_set_is_held_to_the_sum_of_its_fixes_not_to_a_build_inside_it(self): + """Tolerances do not stack: a, b and c built together lie within 1e-3 of the build of a + and b (1.0022), which lies within 1e-3 of the sum (1.0013), and 1.8e-3 from the sum.""" + f, *_ = made(interactions="none", built_ab=1.0022) + self.assertEqual(fixes.errors(f), []) + + def add(g): + p1, p3 = at(g, "p1"), at(g, "p3") + results = {A: sorted([[p1, 1.0031], [p3, 2.0]]), B: [[p3, 6.0]]} + g["built"].append({"id": "abc", "label": "Every fix", "fixes": ["a", "b", "c"], "state": None, + "results": results, "noise": chain.noise()}) # fmt: skip + + found = fixes.errors(changed(f, add)) + self.assertTrue(said(found, "built set 'abc', item 'p1', x:a: built 1.0031, the sum of its effects is 1.0013"), + found) # fmt: skip + def test_f5_a_fix_is_judged_the_same_whichever_fixes_are_chosen_with_it(self): f, *_ = made() own = fixes.compose(f, ["c"])["fixes"][0] @@ -405,14 +454,15 @@ def test_f6_the_noise_of_a_set_is_the_sum_said_with_how_many_were_measured(self) # the sum, not the largest: 1.3e-3 apart is within 2e-3 and beyond 1e-3 self.assertEqual(fixes.counts(f, ["a", "b"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) - def test_f8_a_pair_says_why_in_words_never_by_the_models_identifiers(self): + def test_f8_a_pair_holds_no_text_so_nothing_of_the_model_goes_into_it(self): f, *_ = made() # joined here, so that the repository itself holds no identifier-shaped text uuid_shaped = "-".join(["ab12cd34", "ab12", "ab12", "ab12", "ab12cd34ef56"]) # noqa: FLY002 cases = { - "a pair is not {fixes, reason, largest}": lambda g: g["interactions"]["pairs"][0].__setitem__("activities", ["m"]), - "its reason holds an identifier": lambda g: g["interactions"]["pairs"][0].__setitem__( + "a pair is not {fixes, kind, reach, largest}": lambda g: g["interactions"]["pairs"][0].__setitem__( "reason", f"both change {uuid_shaped}"), + "`kind` is not one of technosphere": lambda g: g["interactions"]["pairs"][0].__setitem__( + "kind", f"both change {uuid_shaped}"), "`interactions.method` holds an identifier": lambda g: g["interactions"].__setitem__( "method", "Compared " + "ab" * 16), } # fmt: skip @@ -428,7 +478,7 @@ class PairSizes(unittest.TestCase): def test_a_built_pair_lies_within_its_measured_size(self): f, *_ = made(interactions="unchecked", built_ab=1.0030) # 1.7e-3 from the sum, 1.0013 - f["interactions"] = checked_pairs(("a", "b", "both change one factor")) + f["interactions"] = checked_pairs(pair("a", "b", fixes.SAME_ENTRY)) self.assertEqual(fixes.errors(f), [], "not measured: the pair is shown, not held to a size") f["interactions"]["pairs"][-1]["largest"] = 0.002 self.assertEqual(fixes.errors(f), []) @@ -439,7 +489,7 @@ def test_a_built_pair_lies_within_its_measured_size(self): def test_the_pair_terms_of_a_set_are_the_sum_of_its_measured_pairs(self): f, *_ = made() - f["interactions"]["pairs"] = [pair("b", "c", "one market", 1e-5), pair("a", "c", "one factor", 2e-5)] + f["interactions"]["pairs"] = [pair("b", "c", largest=1e-5), pair("a", "c", fixes.SAME_ENTRY, 2e-5)] self.assertEqual(fixes.errors(f), []) terms = fixes.pair_terms(f, ["a", "b", "c"]) self.assertEqual((terms["measured"], terms["pairs"]), (2, 2)) @@ -448,6 +498,271 @@ def test_the_pair_terms_of_a_set_are_the_sum_of_its_measured_pairs(self): self.assertEqual(fixes.pair_terms(f, ["a", "b", "c"]), {"value": None, "measured": 1, "pairs": 2}) self.assertEqual(fixes.pair_terms(f, ["a", "b"]), {"value": 0, "measured": 0, "pairs": 0}) +def with_requires(*, built_ab=True): + """Fixes that need others applied first, against the base of `chain_and_matrix`: + + - a moves p1 in A to 1.5; b moves p2 in B to 5.5; both alone; + - d requires a: built with a and d, p1 is 1.8 and p2 is 2.5; + - e requires d (so a too): with a, d and e, p3 is 3.3 as well; + - g requires a and b: with a, b and g, p1 is 1.6, p2 is 2.2 and p4 is 0.5. The file holds + a build of a and b, which g was measured against. + + Each fix's results are written whole, as built; its effect keeps what it changed on top + of what it requires.""" + data, full = chain_and_matrix() + f = fixes.build( + "Fixes that need others", + data=data, + full=full, + base="s1", + fixes=[ + fixes.fix("a", "Fix A", kind="mapping"), + fixes.fix("b", "Fix B", kind="mapping"), + fixes.fix("d", "Fix D", kind="amount", requires=["a"]), + fixes.fix("e", "Fix E", kind="amount", requires=["d"]), + fixes.fix("g", "Fix G", kind="amount", requires=["a", "b"]), + ], + results={ + "a": {A: {"p1": 1.5}}, + "b": {B: {"p2": 5.5}}, + "d": {A: {"p1": 1.8, "p2": 2.5}}, + "e": {A: {"p1": 1.8, "p2": 2.5, "p3": 3.3}}, + "g": {A: {"p1": 1.6, "p2": 2.2, "p4": 0.5}, B: {"p2": 5.5}}, + }, + interactions=checked_pairs(), + built=[fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": 1.5}, B: {"p2": 5.5}})] if built_ab else (), + ) + return f, data, full + + +class Requires(unittest.TestCase): + """A fix that cannot be applied alone names the fixes it needs (SPEC 16.2).""" + + def test_a_fix_is_measured_against_the_build_of_what_it_requires(self): + f, data, full = with_requires() + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + effect = {one["id"]: one["effect"] for one in f["fixes"]} + p1, p2, p3, p4 = (at(f, p) for p in ("p1", "p2", "p3", "p4")) + self.assertEqual(effect["d"], {A: [[p1, 1.8], [p2, 2.5]], B: []}) + self.assertEqual(effect["e"], {A: [[p3, 3.3]], B: []}, "what d already did is d's") + self.assertEqual(effect["g"], {A: [[p1, 1.6], [p2, 2.2], [p4, 0.5]], B: []}, "b's change in B is b's") + # the counts: against that build, item by item + under = {"d": {A: {p1: 1.5}}, "e": {A: {p1: 1.8, p2: 2.5}}, "g": {A: {p1: 1.5}, B: {p2: 5.5}}} + for one in f["fixes"]: + again = dense(f, one["id"], under.get(one["id"])) + for ind in (A, B): + with self.subTest(fix=one["id"], ind=ind): + self.assertEqual(one["counts"][ind]["classes"], again[ind]["classes"]) + e = f["fixes"][3]["counts"][A]["classes"] + self.assertEqual((e[rule.DIFFERS], e[rule.AGREES], e[rule.NEAR_ZERO]), (1, 2, 1), "e changes p3 only") + + def test_an_effect_written_with_the_result_of_its_requirements_is_refused(self): + f, *_ = with_requires() + stale = changed(f, lambda g: g["fixes"][2]["effect"][A].append([at(g, "p4"), 0.0])) + self.assertTrue(said(fixes.errors(stale), "item 'p4' is written with the result it was measured against")) + kept = changed(f, lambda g: g["fixes"][3]["effect"][A].insert(0, [at(g, "p1"), 1.8])) + self.assertTrue(said(fixes.errors(kept), "fix 'e': its effect, x:a: item 'p1' is written with the result")) + + def test_choosing_a_fix_chooses_what_it_requires(self): + f, *_ = with_requires() + self.assertEqual(fixes.chosen(f, ["e"]), (["a", "d", "e"], ["a", "d"])) + drawn = fixes.compose(f, ["e"]) + self.assertEqual((drawn["label"], drawn["added"]), (fixes.BUILT, ["a", "d"])) + self.assertEqual(named(f, drawn["results"][A]), {"p1": 1.8, "p2": 2.5, "p3": 3.3, "p4": 0.0}) + self.assertEqual([x["requires"] for x in drawn["fixes"]], [[], ["a"], ["d"]]) + self.assertEqual(fixes.label(f, ["g"]), fixes.BUILT) + drawn = fixes.compose(f, ["g"])["results"] + self.assertEqual(named(f, drawn[A]), {"p1": 1.6, "p2": 2.2, "p3": 3.0, "p4": 0.5}) + self.assertEqual(named(f, drawn[B]), {"p1": 3.0, "p2": 5.5, "p3": 7.0, "p4": 1.0}) + + def test_effects_add_each_on_top_of_what_it_was_measured_with(self): + f, *_ = with_requires() + # e and b: every item is one fix's with what it requires, so each result is a built one + drawn = fixes.compose(f, ["e", "b"]) + self.assertEqual((drawn["label"], drawn["added"]), (fixes.EXACT, ["a", "d"])) + self.assertEqual(named(f, drawn["results"][A]), {"p1": 1.8, "p2": 2.5, "p3": 3.3, "p4": 0.0}) + self.assertEqual(named(f, drawn["results"][B]), {"p1": 3.0, "p2": 5.5, "p3": 7.0, "p4": 1.0}) + # d and g both build on a: a's effect counts once, each of theirs on top + drawn = fixes.compose(f, ["d", "g"]) + self.assertEqual(drawn["label"], fixes.EXACT) + p1, p2 = at(f, "p1"), at(f, "p2") + self.assertAlmostEqual(drawn["results"][A][p1], 1.0 + 0.5 + 0.3 + 0.1) + self.assertAlmostEqual(drawn["results"][A][p2], 2.0 + 0.5 + 0.2) + self.assertEqual(drawn["cannot_add"], {A: [], B: []}) + + def test_where_one_fix_with_its_requirements_covers_an_item_its_build_is_the_result(self): + """a removes p1's result and d, which requires a, gives it one: no difference can be + taken against a missing result, so only the build of d says what p1 is.""" + data, full = chain_and_matrix() + four = [fixes.fix("a", "A", kind="mapping"), fixes.fix("d", "D", kind="amount", requires=["a"]), + fixes.fix("b", "B", kind="mapping"), fixes.fix("c", "C", kind="mapping")] # fmt: skip + results = {"a": {A: {"p1": None}}, "d": {A: {"p1": 2.0}}, "b": {A: {"p2": 2.5}}, "c": {A: {"p1": 1.2}}} + f = fixes.build("A requirement that removes a result", data=data, full=full, base="s1", fixes=four, + results=results, interactions=checked_pairs()) # fmt: skip + p1 = at(f, "p1") + drawn = fixes.compose(f, ["d", "b"]) + self.assertEqual((drawn["label"], drawn["results"][A][p1], drawn["cannot_add"]), (fixes.EXACT, 2.0, {A: [], B: []})) + for names in (["d", "c"], ["a", "c"]): + with self.subTest(names=names): + drawn = fixes.compose(f, names) + self.assertEqual((drawn["results"][A][p1], drawn["cannot_add"]), (None, {A: [p1], B: []})) + + def test_where_a_build_inside_the_set_covers_an_item_its_result_is_measured_not_summed(self): + """a and b interact at p1 (a listed pair): built together, p1 is 2.9, not their sum 2.7. + g requires both and leaves p1 alone; c is apart.""" + data, full = chain_and_matrix() + four = [fixes.fix("a", "A", kind="mapping"), fixes.fix("b", "B", kind="mapping"), + fixes.fix("g", "G", kind="amount", requires=["a", "b"]), fixes.fix("c", "C", kind="amount")] # fmt: skip + results = {"a": {A: {"p1": 1.5}}, "b": {A: {"p1": 2.2}}, "g": {A: {"p1": 2.9, "p2": 3.0}}, "c": {A: {"p3": 2.0}}} + f = fixes.build("A measured joint", data=data, full=full, base="s1", fixes=four, results=results, + interactions=checked_pairs(pair("a", "b")), + built=[fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": 2.9}})]) # fmt: skip + p1 = at(f, "p1") + for names in (["a", "b", "c"], ["g", "c"]): + with self.subTest(names=names): + drawn = fixes.compose(f, names) + self.assertEqual((drawn["label"], drawn["results"][A][p1]), (fixes.ESTIMATED, 2.9)) + # the sum a built set is held to stands on the fixes alone + cells = fixes._Builds.of(f).composed(frozenset({"a", "b", "c"}), A, built=False) + self.assertAlmostEqual(cells[p1][0], 2.7) + + def test_a_fix_whose_requirements_were_never_built_together_is_refused(self): + with self.assertRaises(ValueError) as caught: + with_requires(built_ab=False) + self.assertIn("fix 'g': no build of what it requires (a, b)", str(caught.exception)) + f, *_ = with_requires() + gone = changed(f, lambda g: g["built"].clear()) + self.assertTrue(said(fixes.errors(gone), "it requires 'a', 'b', and the file holds no build of them together")) + + def test_requires_names_earlier_fixes_once(self): + f, *_ = with_requires() + cases = { + "fix 'd': requires itself": lambda g: g["fixes"][2].__setitem__("requires", ["d"]), + "requires 'z', and the file holds no such fix": lambda g: g["fixes"][2].__setitem__("requires", ["z"]), + "requires 'b', which comes after it": lambda g: g["fixes"][0].__setitem__("requires", ["b"]), + "requires a fix twice": lambda g: g["fixes"][2].__setitem__("requires", ["a", "a"]), + "`requires` is not a list of the ids of fixes": lambda g: g["fixes"][2].__setitem__("requires", "a"), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + with self.assertRaises(ValueError): + fixes.build("x", data=chain_and_matrix()[0], full=chain_and_matrix()[1], base="s1", + fixes=[fixes.fix("d", "D", kind="amount", requires=["a"]), fixes.fix("a", "A", kind="amount")], + results={}) # fmt: skip + + def test_a_fix_and_what_it_requires_are_never_a_pair(self): + f, *_ = with_requires() + for a, b in (("a", "d"), ("e", "a"), ("d", "e")): + with self.subTest(pair=(a, b)): + found = fixes.errors(changed(f, lambda g, a=a, b=b: g["interactions"]["pairs"].append(pair(a, b)))) + self.assertTrue(said(found, "one requires the other"), found) + self.assertEqual(fixes.errors(changed(f, lambda g: g["interactions"]["pairs"].append(pair("d", "g")))), []) + + def test_a_built_set_holds_what_its_fixes_require_and_is_not_a_fix_with_its_requirements(self): + f, *_ = with_requires() + empty, noise = {A: [], B: []}, chain.noise() + + def add(names, results=empty): + return lambda g: g["built"].append({"id": "x", "label": "X", "fixes": names, "state": None, + "results": results, "noise": noise}) # fmt: skip + + self.assertTrue(said(fixes.errors(changed(f, add(["d", "b"]))), "holds 'd' without 'a', which it requires")) + self.assertTrue(said(fixes.errors(changed(f, add(["a", "d"]))), "is built already: it is fix 'd'")) + # a, b and d built together: checked, with no listed pair, so it must be the sum + p1, p2 = at(f, "p1"), at(f, "p2") + summed = {A: [[p1, 1.8], [p2, 2.5]], B: [[p2, 5.5]]} + self.assertEqual(fixes.errors(changed(f, add(["a", "b", "d"], summed))), []) + self.assertEqual(fixes.label(changed(f, add(["a", "b", "d"], summed)), ["d", "b"]), fixes.BUILT) + off = {A: [[p1, 1.8], [p2, 2.9]], B: [[p2, 5.5]]} + found = fixes.errors(changed(f, add(["a", "b", "d"], off))) + self.assertTrue(said(found, "built set 'x', item 'p2', x:a: built 2.9, the sum of its effects is 2.5"), found) + + +def with_added(): + """One fix that brings an item no state of the chain holds: p9, in kg.""" + data, full = chain_and_matrix() + f = fixes.build( + "Fixes that add an item", + data=data, + full=full, + base="s1", + fixes=[fixes.fix("n", "A new item", kind="mapping")], + results={"n": {A: {"p9": 0.7}, B: {"p9": 1.2}}}, + interactions=checked_pairs(), + added_items=[["p9", "A new thing", None, None, "kg"]], + ) + return f, data, full + + +class AddedItems(unittest.TestCase): + """A fix may bring items the base does not hold (SPEC 16.2).""" + + def test_an_added_item_follows_the_matrix_items_with_no_base_result(self): + f, data, full = with_added() + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + self.assertEqual(fixes.against(f, data, None), []) + self.assertEqual([row[0] for row in f["items"]], [row[0] for row in full["items"]] + ["p9"]) + self.assertEqual((f["base"][A][-1], f["base"][B][-1]), (None, None)) + self.assertEqual(named(f, fixes.compose(f, ["n"])["results"][A]), + {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0, "p9": 0.7}) # fmt: skip + self.assertEqual(f["fixes"][0]["counts"][A]["missing"], {"before": 1, "after": 0, "both": 0}) + + def test_an_added_item_has_no_base_result_and_a_result_from_some_fix(self): + f, data, full = with_added() + held = changed(f, lambda g: g["base"][A].__setitem__(4, 0.7)) + self.assertTrue(said(fixes.against(held, data, full), "item 'p9' is not in the matrix, and the base has a result")) + idle = changed(f, lambda g: g["fixes"][0].update(effect={A: [], B: []})) + self.assertTrue(said(fixes.against(idle, data, full), "item 'p9' is in no state of the chain, and no fix gives it")) + with self.assertRaises(ValueError) as caught: + fixes.build("x", data=data, full=full, base="s1", fixes=[fixes.fix("n", "N", kind="mapping")], results={}, + interactions=checked_pairs(), added_items=[["p9", "A new thing", None, None, "kg"]]) # fmt: skip + self.assertIn("item 'p9' is in no state of the chain", str(caught.exception)) + + def test_added_items_come_after_the_matrix_items_and_are_new(self): + f, data, full = with_added() + first = changed(f, lambda g: g["items"].insert(0, g["items"].pop())) + self.assertTrue(said(fixes.against(first, data, full), "before any item a fix adds")) + with self.assertRaises(ValueError) as caught: + fixes.build("x", data=data, full=full, base="s1", fixes=[fixes.fix("n", "N", kind="mapping")], results={}, + added_items=[["p2", "Again", None, None, "kg"]]) # fmt: skip + self.assertIn("an added item is an item of the base already", str(caught.exception)) + + +class ChainState(unittest.TestCase): + """A built set may say it is a state of the chain after the base: the two builds are held + to each other (SPEC 16.4).""" + + def test_a_built_set_that_is_a_state_agrees_with_it(self): + f, data, full = made(release=True) + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + self.assertEqual(fixes.against(f, data, None), []) + self.assertEqual(fixes.label(f, ["c", "b", "a"]), fixes.BUILT) + + def test_a_built_set_that_is_not_the_state_it_names_is_refused(self): + f, data, full = made(release=True) + def nudge(g): + entries = g["built"][-1]["results"][A] + entries[[k for k, _ in entries].index(at(g, "p1"))][1] = 1.01 + + off = changed(f, nudge) + self.assertEqual(fixes.errors(off), [], "on its own the file cannot tell") + found = fixes.against(off, data, full) + self.assertTrue(said(found, "built set 'all', item 'p1', x:a: built 1.01, and state 's2' of the chain"), found) + self.assertEqual([i["id"] for i in data["items"]], ["p3", "p1"]) + found = fixes.against(off, data, None) + self.assertTrue(said(found, "built set 'all', item 'p1', x:a: built 1.01"), "without the matrix: the listed items") + + def test_the_state_comes_after_the_base(self): + f, data, full = made(release=True) + for state in ("s1", "s0", "s9"): + with self.subTest(state=state): + moved = changed(f, lambda g, s=state: g["built"][-1].__setitem__("state", s)) + self.assertTrue(said(fixes.against(moved, data, full), "is not a state of the chain after the base")) + class Command(unittest.TestCase): def run_cli(self, *args) -> tuple[int, str]: From 7a7d566202ccbb12b0e77394d6f5eff013c305ca Mon Sep 17 00:00:00 2001 From: Manuel Klarmann Date: Tue, 17 Nov 2026 06:17:23 +0100 Subject: [PATCH 3/6] fixes: work out the base once per call, and each fix's measured-against build once per indicator counts() of a 50-fix set at 11,947 items x 25 indicators: 2.6 s -> 0.87 s. The base's own classes were worked out six times per call (2 million classify calls); they are now built once and only when a count needs them. Co-Authored-By: Claude Opus 5.5 --- SPEC.md | 3 +- src/sentier_diagnostics/fixes.py | 69 +++++++++++++++++++------------- 2 files changed, 44 insertions(+), 28 deletions(-) diff --git a/SPEC.md b/SPEC.md index e896dc3..266bbd0 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1151,7 +1151,8 @@ characterisation factor about 12,000 (about 250 KB), a fix to a market everythin to about 300,000 (6 to 7 MB). 64 MB holds about ten dense fixes and thousands of sparse ones; effects move to one file per fix only if dense fixes become common. Checked on a made-up file of that base with 100 fixes of 500 items each in every indicator, ten of them -requiring the fix before them (39.5 MB), the check takes 2.7 seconds. +requiring the fix before them (39.5 MB), the check takes about 2.7 seconds, and counting a +set of 50 of them under one. ### 16.11 Acceptance cases diff --git a/src/sentier_diagnostics/fixes.py b/src/sentier_diagnostics/fixes.py index 4b867a2..a9bb18e 100644 --- a/src/sentier_diagnostics/fixes.py +++ b/src/sentier_diagnostics/fixes.py @@ -40,6 +40,7 @@ import json import re from collections.abc import Iterable, Mapping +from functools import cached_property from pathlib import Path from . import __version__, chain, claims, fileformat, matrix, rule @@ -353,7 +354,7 @@ def size_note(f: Mapping) -> str: class _Base: """The base of a checked file, worked out once: its results, the floor of every item, and - the class of every item against itself (agrees, near zero or missing).""" + (when a count asks) the class of every item against itself: agrees, near zero or missing.""" def __init__(self, f: Mapping): self.ids = [i["id"] for i in f["indicators"]] @@ -362,16 +363,23 @@ def __init__(self, f: Mapping): table = {(x["indicator"], x["unit"]): x["value"] for x in f["rule"]["floors"]} self.floor = {ind: [table[(ind, r[4])] for r in f["items"]] for ind in self.ids} self.values = f["base"] - self.kinds = { + + @cached_property + def kinds(self) -> dict[str, list[str]]: + return { ind: [rule.classify(v, v, floor=fl) for v, fl in zip(self.values[ind], self.floor[ind])] for ind in self.ids } - self.tally = {} + + @cached_property + def tally(self) -> dict[str, dict[str, int]]: + out = {} for ind in self.ids: tally = dict.fromkeys((rule.AGREES, rule.NEAR_ZERO, rule.MISSING), 0) for kind in self.kinds[ind]: tally[kind] += 1 - self.tally[ind] = tally + out[ind] = tally + return out class _Builds: @@ -388,10 +396,7 @@ def __init__(self, base: _Base, closures: Mapping[str, frozenset]): @classmethod def of(cls, f: Mapping) -> _Builds: - closures: dict[str, frozenset] = {} - for one in f["fixes"]: - closures[one["id"]] = frozenset(one["requires"]).union(*(closures[r] for r in one["requires"])) - found = cls(_Base(f), closures) + found = cls(_Base(f), _closures(f)) found.entries = {one["id"]: _columns(one["effect"]) for one in f["fixes"]} found.built = {frozenset(b["fixes"]): _columns(b["results"]) for b in f["built"]} return found @@ -419,11 +424,6 @@ def column(self, fixes: Iterable[str]) -> dict | None: self._memo[fixes] = found return found - def before(self, fix_id: str, ind: str, k: int): - """What a fix's result at one item was measured against: the build of what it requires.""" - under = self.column(self.closures[fix_id])[ind] - return under[k] if k in under else self.base.values[ind][k] - def covers(self, fixes: frozenset, *, built: bool = True) -> list[frozenset]: """The builds inside a closed set that hold two fixes or more, in the order a set's result looks for them: every fix with what it requires, in the order of the file, then @@ -445,21 +445,31 @@ def composed(self, fixes: frozenset, ind: str, *, built: bool = True) -> dict[in for k in self.entries[y][ind]: touching.setdefault(k, []).append(y) covers = self.covers(fixes, built=built) + base, mine = self.base.values[ind], {y: self.entries[y][ind] for y in fixes} + under = {y: self.column(self.closures[y])[ind] for y in fixes} # what each was measured against out = {} for k, ys in touching.items(): if len(ys) == 1: - out[k] = (self.entries[ys[0]][ind][k], True) + out[k] = (mine[ys[0]][k], True) continue here = set(ys) cover = next((c for c in covers if here <= c), None) if cover is not None: column = self.column(cover)[ind] - out[k] = (column[k] if k in column else self.base.values[ind][k], True) + out[k] = (column[k] if k in column else base[k], True) else: - out[k] = _sum(self.base.values[ind][k], [(self.entries[y][ind][k], self.before(y, ind, k)) for y in ys]) + out[k] = _sum(base[k], [(mine[y][k], under[y][k] if k in under[y] else base[k]) for y in ys]) return out +def _closures(f: Mapping) -> dict[str, frozenset]: + """Every fix of a checked file, with every fix it requires at any depth.""" + closures: dict[str, frozenset] = {} + for one in f["fixes"]: + closures[one["id"]] = frozenset(one["requires"]).union(*(closures[r] for r in one["requires"])) + return closures + + def _columns(sparse: Mapping) -> dict[str, dict[int, float | None]]: """`{indicator: [[item index, result], ...]}` as `{indicator: {item index: result}}`.""" return {ind: {k: v for k, v in entries} for ind, entries in sparse.items()} @@ -577,8 +587,8 @@ def chosen(f: Mapping, ids: Iterable[str]) -> tuple[list[str], list[str]]: raise ValueError(f"the file holds no fix {one!r}") if len(set(ids)) != len(ids): raise ValueError("a set holds each fix once") - builds = _Builds.of(f) - closed = set(ids).union(*(builds.closures[i] for i in ids)) + closures = _closures(f) + closed = set(ids).union(*(closures[i] for i in ids)) names = sorted(closed, key=order.__getitem__) return names, [x for x in names if x not in ids] @@ -597,7 +607,11 @@ def held_pairs(f: Mapping, ids: Iterable[str]) -> list[dict]: def label(f: Mapping, ids: Iterable[str]) -> str: """Built, exact or estimated (SPEC 16.5).""" names = chosen(f, ids)[0] - if _Builds.of(f).column(names) is not None: + return _label(f, _Builds.of(f), names) + + +def _label(f: Mapping, builds: _Builds, names: list[str]) -> str: + if builds.column(names) is not None: return BUILT if f["interactions"]["checked"] and not held_pairs(f, names): return EXACT @@ -631,19 +645,18 @@ def noise_of(f: Mapping, ids: Iterable[str]) -> dict[str, dict]: return out -def _changed(f: Mapping, names: list[str]) -> tuple[_Builds, dict, dict]: +def _changed(builds: _Builds, names: list[str]) -> tuple[dict, dict]: """The items a closed set changes, per indicator, with its result at each; and the items where no sum can be formed.""" - builds = _Builds.of(f) column = builds.column(names) if column is not None: - return builds, column, {ind: [] for ind in builds.base.ids} + return column, {ind: [] for ind in builds.base.ids} changed, cannot = {}, {} for ind in builds.base.ids: cells = builds.composed(frozenset(names), ind) changed[ind] = {k: v for k, (v, _) in cells.items()} cannot[ind] = sorted(k for k, (_, ok) in cells.items() if not ok) - return builds, changed, cannot + return changed, cannot def compose(f: Mapping, ids: Iterable[str]) -> dict: @@ -654,7 +667,8 @@ def compose(f: Mapping, ids: Iterable[str]) -> dict: names, per indicator, the item indices whose result is None because no sum can be formed there, so that it reads apart from a missing result. A set has no verdict.""" names, added = chosen(f, ids) - builds, changed, cannot = _changed(f, names) + builds = _Builds.of(f) + changed, cannot = _changed(builds, names) results = {} for ind in builds.base.ids: column = list(builds.base.values[ind]) @@ -664,7 +678,7 @@ def compose(f: Mapping, ids: Iterable[str]) -> dict: return { "fixes": [{k: one[k] for k in ("id", "label", "requires", "claim", "counts")} for one in _fixes_of(f, names)], "added": added, - "label": label(f, names), + "label": _label(f, builds, names), "pairs": held_pairs(f, names), "pair_terms": pair_terms(f, names), "tolerance": tolerance_of(f, names), @@ -678,10 +692,11 @@ def counts(f: Mapping, ids: Iterable[str]) -> dict: """The counts of a set against the base, per indicator, with its label beside them: an estimated set's counts are estimates ("estimated: 153 would differ"). No verdicts.""" names = chosen(f, ids)[0] - builds, changed, _ = _changed(f, names) + builds = _Builds.of(f) + changed, _ = _changed(builds, names) tol, noise = tolerance_of(f, names), noise_of(f, names) return { - "label": label(f, names), + "label": _label(f, builds, names), "by": {ind: _block(builds.base, ind, changed[ind], tol, noise[ind]["value"], None) for ind in builds.base.ids}, } From 37b2ceed66dfdca1754516308e52f848bbacbb3c Mon Sep 17 00:00:00 2001 From: Manuel Klarmann Date: Tue, 17 Nov 2026 06:55:47 +0100 Subject: [PATCH 4/6] Fixes on noise per state, #22's matrices, and sentier-bafu's producer review Noise belongs to a build, as it belongs to a state (SPEC 5.3, Manuel: "sum, go ahead"): - the base's noise is its state's (`of.base_noise`, left out where the base is the published results, which are exact), held to the diagnostics file; a diagnostics file whose states carry no noise is refused; - a fix is judged with the noise of its build and of the build it was measured against; - a set takes the sum, over its fixes, of both (the base once for each fix measured against it), or the noise of a build it takes a result from and the base's, where larger; it says at how many builds the noise is known; - a built set is held to its sum with the noise of every build both sides stand on, and to its state with its own noise and the state's. The matrix of #22 writes a result equal to the one before as "=": the base and the state check read it written out. From sentier-bafu's review as producer (12 fixes and the release, written from its chain): - `largest` is the rule's own measure, the smallest tolerance under which a built pair agrees with its sum (`size_of`); "no size" where none is enough, apart from null (not measured); a pair with no size must be built, and its build must have none; - the kinds of pair are in their order of precedence: same entry first; - an item a fix brings in a unit no item of the chain has takes a floor of 0 from nothing; - SPEC 16: effects on the base's route, results read exactly, reach counted on the build with both fixes, the premise of the list of pairs for fixes that edit data. Co-Authored-By: Claude Opus 5.5 --- GLOSSARY.md | 6 +- SPEC.md | 191 +++++++++----- src/sentier_diagnostics/fixes.py | 413 +++++++++++++++++++++++-------- tests/test_fixes.py | 229 +++++++++++++++-- 4 files changed, 644 insertions(+), 195 deletions(-) diff --git a/GLOSSARY.md b/GLOSSARY.md index d72f348..c89d7f4 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -32,10 +32,12 @@ The words this repository uses, each in one line. [SPEC.md](SPEC.md) says the re - **Body**: the page as one element a site holds in its own page; its style stays inside it. - **Drawer**: the panel of one item, at the right of the page, opened from a row. - **Base**: the one state of a chain that every fix is measured against (SPEC section 16). -- **Fix**: one change applied to the base alone and built, with a step's kind, claim, tolerance and noise. +- **Fix**: one change applied to the base alone and built, with a step's kind, claim and tolerance, and the noise of its build. +- **Build** (of fixes): a set of results the producer made, the base, a fix with what it requires, or a built set; noise belongs to it, as to a state. A fix is judged with the noise of its build and of the one it was measured against; a set with the sum, over its fixes, of both. - **Requires**: the fixes a fix cannot be applied without. Its effect is measured with them applied, and choosing it chooses them. - **Effect**: what a fix does, per item and indicator: its result with the fix minus its result without it, both with what it requires. - **Set**: the fixes a reader chooses together, with what they require; its result is the base plus the sum of their effects. It has no verdict. - **Interaction**: two fixes whose joint result is not the sum of their effects. Checked in pairs, in the model; a listed pair says its kind, its reach and its size, never words. -- **Pair terms**: how far an estimated set may lie from its sum, to first order: the sum of the measured sizes of the listed pairs it holds; not known where one was not measured. +- **Size** (of a pair): the smallest tolerance under which the build of two fixes agrees with the sum of their effects. **No size** where no tolerance is enough (two sides of zero, a zero, a missing result): such a pair is built. Not the same as not measured. +- **Pair terms**: how far an estimated set may lie from its sum, to first order: the sum of the sizes of the listed pairs it holds; not known where one was not measured or has no size. - **Built, exact, estimated**: the label of a set. Built where the producer built it (one fix with what it requires, or a built set); exact where no two of its fixes can interact; estimated otherwise, and marked on every cell. diff --git a/SPEC.md b/SPEC.md index 266bbd0..f1ebcf7 100644 --- a/SPEC.md +++ b/SPEC.md @@ -89,7 +89,9 @@ so the claim stays right when the factors change. `tests/fixtures/located-factor that step, made up, and a test holds what it shows. **Order matters.** Where two steps touch the same results, each one's share depends on the -order they were applied in. The file keeps the order; the page says so. +order they were applied in. The file keeps the order; the page says so. A set of changes +chosen freely, not a run of consecutive steps, is not drawn from a chain: section 16 +measures each fix on its own against one base, so that any set can be. **A run of steps.** A file holds one comparison per step and one for the whole way. A run of two or more consecutive steps, short of every step (steps 3 to 9), is not in the file: @@ -535,7 +537,7 @@ of its own.** It reads them from the file, which the validator has checked, and the file's counts up: the classes that moved into "moved", and a step's counts over its indicators. Two things it works out itself, and only from every item: the median and percentiles of a distribution (10.3), and a run of steps (section 3), the one place where -it judges a pair itself. It does so with the rule ported from `rule.py`, held to it by the +it judges a pair itself in a diagnostics file (a set of fixes is the other, 16.8). It does so with the rule ported from `rule.py`, held to it by the shared acceptance cases and by 813 pairs on every edge of the rule that the Python writes (`tests/test_boundaries.py`), after holding each step of the run to the file's own classes. `tests/fixtures/together.json` holds the counts `together.py` gives for every run of the @@ -864,8 +866,12 @@ and that a set the file cannot show to be exact is shown, marked as estimated. F questions were settled in the validator's pull request, before any producer writes a file, for Manuel to agree there: a pair holds no text (16.6); a fix may require others (16.4); a fix may bring items the base does not hold (16.4); and a built set may say which state of -the chain it is (16.7). The reader and validator are `fixes.py`; the producer and the page -are not written yet (16.12). +the chain it is (16.7). Noise follows section 5.3 (Manuel: "sum, go ahead"): it belongs to +a build, and a comparison takes the sum of the noise of the builds it stands on (16.5). +sentier-bafu wrote a file from its own chain to review the format as its producer (12 fixes +and the release, checked clean), and its review defined the size of a pair (16.6), the route +an effect is measured on (16.4), and the premise of the list of pairs (16.3). The reader and +validator are `fixes.py`; the producer and the page are not written yet (16.12). ### 16.1 Why not a chain of fixes @@ -883,8 +889,10 @@ the base to a release. A fixes file hangs on one of its states. ### 16.2 Words - **Base**: one state of a chain, the state every fix is measured against. +- **Build**: a set of results the producer made: the base, a fix with what it requires, or + a built set (16.7). Noise belongs to a build, as it belongs to a state (section 5.3). - **Fix**: one change, applied to the base alone and built. It has what a step has: a - kind, a claim, a tolerance, a noise, `explained`, a link. + kind, a claim, a tolerance, `explained`, a link; and the noise of its build. - **Requires**: the fixes a fix cannot be applied without. Its effect is measured with them applied, and choosing it chooses them. - **Effect**: what a fix does, per item and indicator: its result with the fix (and what it @@ -913,6 +921,16 @@ the technosphere A. So: These are the kinds of pair a file can list (16.6). Two fixes that change different factors, or different entries of B, are never a pair. +All of this holds for fixes that are fixed changes of A, B and c. **A fix that edits the +data a model is imported from can change what another fix puts into the matrices**: an +importer that links an exchange to a flow by its name, compartment or CAS number links it to +whatever another fix calls that flow. Changing which flow an exchange is (its name, +compartment or CAS number, where factors match on them) changes B. Several such edits can +be needed together without any one pair showing it. A producer whose fixes edit data checks +that the release's change of A, B and c is the sum of each fix's change, entry by entry, and +says so in `method` (16.6); where it is not, the fixes that meet before the matrices are a +pair (`same entry`), or one requires the others. + An interaction always shows in a pair: every term of A⁻¹ expanded in its changes that holds two fixes holds two of them next to each other. So a set whose fixes are independent two by two adds up exactly, however many it holds, and a check of pairs is enough. That @@ -928,13 +946,13 @@ item. One JSON object, read as strictly as the diagnostics file (section 4.1). | Key | What it holds | |---|---| | `format` | `"sentier-diagnostics-fixes/1"` | -| `of` | `{file, title, base}`: the diagnostics file it belongs to (a file name), that file's title, and the id of the state that is the base | +| `of` | `{file, title, base, base_noise}`: the diagnostics file it belongs to (a file name), that file's title, the id of the state that is the base, and that state's noise as the diagnostics file says it (5.3). `base_noise` is left out where the base is the published results, which are exact | | `title`, `about` | Short text; `about` may be null | | `item`, `indicator`, `indicators` | As in the diagnostics file, and equal to it (`indicator` may be left out where that file leaves it out) | -| `rule` | `{floor_factor, floors_from, beyond, floors}`: the diagnostics file's rule without its `total_tolerance`, which belongs to the chain (a set has its own, 16.5). The floors come from the chain's first state | +| `rule` | `{floor_factor, floors_from, beyond, floors}`: the diagnostics file's rule without its `total_tolerance`, which belongs to the chain (a set has its own, 16.5). The floors come from the chain's first state, followed by a floor for each unit only an item a fix brings has | | `items` | Every item, `[id, name, place, group, unit]`: the matrix's items in the matrix's order, then any item a fix brings that no state of the chain holds | | `base` | The base result of every item, per indicator: `{indicator: [result or null, ...]}`, in the order of `items`; null for an item a fix brings | -| `fixes` | `[{id, label, about, kind, explained, link, requires, claim, tolerance, noise, effect, counts}]`, a fix's requirements before it | +| `fixes` | `[{id, label, about, kind, explained, link, requires, claim, tolerance, noise, effect, counts}]`, a fix's requirements before it; `noise` is that of the build with the fix | | `interactions` | `{checked, method, by, pairs}` (16.6) | | `built` | Sets the producer built: `[{id, label, fixes, state, results, noise}]` (16.7) | | `notes`, `made_by` | As in the diagnostics file | @@ -944,9 +962,23 @@ result it was measured against bit for bit, as `[[item index, result], ...]`, th being the one built with the fix (`null` where it has none). **Every change is kept, however small**: two changes under the tolerance each can be over it together, and a set is a sum. An entry equal to the result it was measured against is refused: it would read as a -change. Every indicator has a list, empty where the fix changes nothing. A fix's `counts` is -one block per indicator (section 4.2), its pairs being the result it was measured against -and the result with the fix, judged with the fix's own tolerance and noise; `closer` and +change. Every indicator has a list, empty where the fix changes nothing. + +**An effect is measured on the base's route.** The result with the fix is built as the base +was (the same importer, linking and solver), or is the base result plus the difference +between two results of one other route, so that an item the fix does not reach keeps the +base result bit for bit. The base is held to the chain bit for bit and every entry that +differs is kept, so an effect computed another way fills up with the route's own +differences, each read as a change: sentier-bafu built one release twice, a pinned copy +against today's importer with no change meant, and 294,590 of 298,675 results moved, 722 of +them by 1e-5 or more. **Results are read exactly**, as they were written: a reader that +rounds the last bit drops changes and adds others (a default CSV reader misread 134,743 of +the 298,675 results of one build in the last bit, and lost 7 of the 86 changes of one patch; +read round-trip, every result matches its text). + +A fix's `counts` is one block per indicator (section 4.2), its pairs being the result it was +measured against and the result with the fix, judged with the fix's own tolerance and the +sum of the noise of those two builds (16.5); `closer` and `further` are null, since a fix has no published results to come closer to. A fix's claim names its items (no `items: null`): the file is local and holds every item, and the page works out each item's verdict from the claim. @@ -963,7 +995,9 @@ one is measured with the other applied. **A fix may bring items** no state of the chain holds (a new product an added activity makes). They follow the matrix's items in `items`, their base result is null, and some fix -or built set gives each a result. Their unit needs a floor in the rule, as any item's does. +or built set gives each a result. Their unit takes the chain's floor; a unit no item of the +chain has takes the floor the chain gives a unit its first state holds no result in: 0, from +nothing (`basis: "none"`), added to the rule after the chain's floors. A fix's claim is judged as a step's is (section 6), against what the fix was measured against. Its verdicts do not depend on which other fixes are chosen or in what order: that @@ -990,14 +1024,27 @@ it), the set has no result there, and the page says the sum cannot be formed, ap result that is missing. It is judged by the rule (section 5) against the base: - its **tolerance** is the largest of its fixes'; -- its **noise** is the sum of its fixes' noise, per indicator. A fix's noise stands for - how unstable its comparison with the base is; the set's difference is the sum of those - comparisons, and their instabilities add up instead of cancelling, as they would along - a run of a chain. Noise not measured for some fixes is said ("noise measured for 3 of 5 - fixes"), never read as none; where none is measured the page says "not measured", never - "identical". A fix's noise names its two builds (`between`): two builds in one process on - one solve route measure the engine's determinism and read 0, two routes measure how the - comparison moves with the route; the note says which; +- its **noise** follows section 5.3, per indicator: noise belongs to a build, and a + comparison takes the sum of the noise of the builds it stands on. The base's is its + state's (`of.base_noise`; the published results are exact and add nothing), a fix's that + of the build with it and what it requires, a built set's that of its build. + - A set that is a build (one fix with what it requires, or a built set) takes the sum of + its noise and the base's. + - Any other set's difference from the base is a sum of its fixes' differences, each + against what that fix was measured against. Its noise is the sum, over its fixes, of the + noise of each fix's build and of the build it was measured against: the base once for + each fix measured against it, since a rebuild of the base moves every one of those + differences the same way. Where it takes a result from a build inside it, that build's + noise and the base's bound the result there, and the set's noise is the larger of the + two. The sum runs in the order of the file's fixes, each fix's measured-against build + before its own, so that the page's port gives the same number bit for bit. + + A build whose noise was not measured adds nothing, and the page says at how many of the + builds the noise is known ("noise known at 3 of 5 builds"; the published results are + known, and exact), never reading not measured as none; where no build measured it the page + says "not measured", never "identical". A noise names its two builds (`between`): two + builds in one process on one solve route measure the engine's determinism and read 0, two + routes measure how a result moves with the route; the note says which; - it has **no verdict**: each claim is judged in its own fix. Each set carries one of three labels, and the page shows it wherever the set is drawn: @@ -1012,8 +1059,9 @@ An estimated set is never drawn as a built one: the mark is on the cell, not onl heading a reader may have scrolled past. Its **pair terms** are the sum of the measured sizes (`largest`) of the listed pairs it holds: how far it may lie from its sum, to first order (a term of three fixes or more needs interacting pairs, 16.3). Where a pair it holds -was not measured, the pair terms are not known, and the page says how many of its pairs were -measured, never 0. Where interactions were not checked, every set of +was not measured, or has no size (16.6), the pair terms are not known, and the page says how +many of its pairs were measured and how many of those have no size, never 0. Where +interactions were not checked, every set of two fixes or more that was not built is estimated. ### 16.6 Interactions @@ -1028,14 +1076,29 @@ into it. - `method`: what the check did, in a sentence, and `by`: who or what ran it (section 8). - `pairs`: `[{fixes: [a, b], kind, reach, largest}]`, every pair that can interact, each once: - - `kind`: where the two changes meet (16.3), one of `technosphere` (both change A), - `technosphere and emission` (A, and an amount of B), `technosphere and factor` (A, and - a factor of c), `emission and factor` (an amount of a flow, and that flow's factor), - `same entry` (both change one number); - - `reach`: how many items the cross term can change (1 to the number of items), or null - where it was not counted; - - `largest`: the measured size of its cross term, the largest over items and indicators - of |h_ab − h_a − h_b + h_0| / |h_ab|, or null where it was not measured. + - `kind`: where the two changes meet (16.3), the first that applies of `same entry` (both + change one number), `technosphere` (both change A), `technosphere and emission` (A, and + an amount of B), `technosphere and factor` (A, and a factor of c), `emission and factor` + (an amount of a flow, and that flow's factor). A pair that meets in several ways (a fix + to an amount and an emission of one activity, with a fix to A) takes the first; + - `reach`: how many items the cross term can change, counted on the build with both fixes + (1 to the number of items), or null where it was not counted. A fix can open a path (an + exchange finds a supplier it had none for), so the base undercounts; whether an item + reaches one fix's change does not depend on that fix, so the build with both bounds it; + - `largest`: the size of its cross term: the smallest tolerance under which the build of + the two agrees, by the rule, with the base plus their two effects. That is the largest + relative difference of the two (the rule's measure, over the larger size, section 5) + over items and indicators, leaving out the items the rule calls near zero on both sides + (`fixes.size_of`). `"no size"` where no tolerance is enough: at some item the two lie on + two sides of zero, one is zero, or one has no result. Null where it was not measured. + "No size" and "not measured" are kept apart: one is a finding, the other a gap. + + A ratio over |h_ab| has no value where h_ab is 0 and grows without bound for items under + the floor; such a share would swamp the pair terms of every set holding the pair and make + the check of its build (16.7) pass anything. The rule's own measure has neither fault, and + it is the tolerance the check applies. **A pair with no size must be built** (16.7): no set + holding it can be bounded by its sum, and with the build the page takes its result where + only those two change an item. A check that only asks whether A⁻¹ links two changes lists nearly every pair in a database where electricity and transport reach everything, and every set would be @@ -1067,15 +1130,18 @@ is, where it is one, or null. fixes are a listed pair, the built results must agree with the base plus the sum of the effects, by the rule with the set's tolerance and noise, for every item. That sum stands on the fixes alone (16.5 without its built sets): a set held to a build inside it, itself held -to the sum, would stack two tolerances. The noise here is -the sum of its fixes' noise and the built set's own, where measured: the sum stands on one -build of each fix, the built results on one more. A file where they -do not is refused, naming the set, the item and the indicator: its producer called two -fixes independent that are not. Where the set holds a listed pair, the page shows how far -the built results lie from the sum, as that set's interaction. Where the set is exactly one -listed pair with a measured `largest`, the built results must lie within it of the sum (by -the rule, with the larger of the set's tolerance and `largest` as the tolerance): a file -that understates a size it could have measured is refused. +to the sum, would stack two tolerances. The noise here is that of every build the two sides +stand on (16.5): in the sum, the base and each fix's build and the one it was measured +against (the base once more for each fix measured against it); on the other side, the built +set's own. A file where they do not agree is refused, naming the set, the item and the +indicator: its producer called two fixes independent that are not. Where the set holds a +listed pair, the page shows how far the built results lie from the sum, as that set's +interaction. Where the set is exactly one listed pair with a measured `largest`, the built +results must lie within it of the sum (by the rule, with the larger of the set's tolerance +and `largest` as the tolerance): a file that understates a size it could have measured is +refused. Where the file says the pair has no size, its build must have none: a build that +agrees with its sum under some tolerance has that size, and the check refuses the file and +gives it. Building the release with every fix is cheap (it is built anyway) and is the strongest test the file can carry: it touches every pair at once. It can refute independence and never @@ -1084,10 +1150,12 @@ confirm it: cross terms of opposite sign can cancel in the whole and not in a su **A built set that names a state is two builds of one thing.** The release with every fix is also the last state of the chain. Its `state` names it, a state after the base, and the check holds the two to each other: the base with the built results, against the matrix's -results of that state, by the rule with the set's tolerance and noise (the sum of its -fixes' noise and its own), for every item; for the listed items where the matrix is not -beside it. A file whose release does not reproduce the chain's is refused, naming the set, -the item and the indicator. +results of that state, by the rule with the largest tolerance of its fixes and the noise of +the two builds, the built set's and the state's (none where the state is the published +results), for every item; for the listed items where the matrix is not beside it. A file +whose release does not reproduce the chain's is refused, naming the set, the item and the +indicator. A release taken from the same build as the chain's state passes this check by +construction; the check of independence above is the one that can fail it. ### 16.8 The page @@ -1099,11 +1167,11 @@ the item and the indicator. - One fix reads as a step does today: its claim, its verdicts, its own counts from the file. - Any set drives the matrix, the distributions, the worst values and the drawer. Its line says its label, its fixes with their claims, its tolerance and its noise. -- The page works out the classes of a set, the one place where it judges a pair itself - (section 10 says it works out none of its own; a set is the exception). It does so with - a port of `rule.judge` that is held to the Python rule by the shared cases - (`tests/acceptance/cases.json`) and by boundary values the Python generates: every - class edge, the floor, a sign crossing, every band edge, a factor too large for a float. +- The page works out the classes of a set, as it does those of a run of steps (section + 10): with the port of `rule.judge` that the shared cases (`tests/acceptance/cases.json`) + and the boundary pairs the Python writes (`tests/test_boundaries.py`: every class edge, + the floor, a sign crossing, every band edge, a factor too large for a float) hold to the + Python rule. It works out a set's noise as 16.5 says, in the same order. Before it composes a set it works out each chosen fix's own classes again and compares them with the file's; where one differs, it composes nothing and says so. - In the drawer, an item's difference in a set splits into its fixes' effects, exactly @@ -1116,7 +1184,10 @@ the item and the indicator. wrong (as section 4.3), and further one where: - `of` does not name the diagnostics file beside it, its title or one of its states, or its - items, indicators, floors or words differ from that file's; + items, indicators, floors or words differ from that file's (with a floor of 0 from nothing + for each unit only an added item has); the diagnostics file says no noise of its states; + `base_noise` is not the base state's noise, is missing where the base is a build, or is + given where it is the published results; - its items are not the matrix's in the matrix's order, followed by items a fix brings, or an item a fix brings has a base result, or no fix or built set gives it one; - the base results differ from the matrix's results of the base state, where the matrix is @@ -1126,17 +1197,19 @@ wrong (as section 4.3), and further one where: - an effect or a built set names an item or indicator it does not hold, holds an entry equal to the result it was measured against, or holds one item twice; - a fix's counts or verdicts are not what its effect and the build it was measured against - give, by the rule, with its own tolerance and noise. With every item at hand, the counts - must be exact; + give, by the rule, with its own tolerance and the noise of the two builds. With every item + at hand, the counts must be exact; - a pair names a fix it does not hold, a fix twice, or a fix and one it requires, or is - listed twice; its kind is not one of the five, its reach not a count of items; `checked` + listed twice; its kind is not one of the five, its reach not a count of items, its + `largest` not a share, `"no size"` or null; a pair with no size has no build; `checked` is false and pairs are listed; a pair holds any other key; the method holds text shaped like an identifier; - a built set holds fewer than two fixes, a fix twice, a fix without one it requires, exactly one fix with what it requires, or the same fixes as another; or disagrees with the sum of its effects where the file says its fixes are independent, or lies further from it - than the measured size of the one pair it is; or names a state that is not after the base, - or that its results do not reproduce. + than the measured size of the one pair it is, or has a size where the file says that pair + has none; or names a state that is not after the base, or that its results do not + reproduce. `check` also prints what the file is made of: how many entries its fixes and built sets hold, and its largest fixes (a fix to a market everything uses holds about one entry per @@ -1151,8 +1224,8 @@ characterisation factor about 12,000 (about 250 KB), a fix to a market everythin to about 300,000 (6 to 7 MB). 64 MB holds about ten dense fixes and thousands of sparse ones; effects move to one file per fix only if dense fixes become common. Checked on a made-up file of that base with 100 fixes of 500 items each in every indicator, ten of them -requiring the fix before them (39.5 MB), the check takes about 2.7 seconds, and counting a -set of 50 of them under one. +requiring the fix before them, every build's noise measured (39.5 MB), the check takes +about 2.2 seconds, and counting a set of 50 of them under one. ### 16.11 Acceptance cases @@ -1165,12 +1238,14 @@ Each is a test before the code is written. | F3 | Interactions not checked: every set of two fixes or more that was not built is estimated, none exact | | F4 | A built set that disagrees with the sum where the file calls its fixes independent is refused, naming the set, item and indicator | | F5 | A fix's verdicts are the same whichever fixes are chosen with it | -| F6 | The noise of a set is the sum of its fixes' noise; noise measured for some fixes only is said, and not measured is never identical | +| F6 | The noise of a set is the sum, over its fixes, of the noise of each fix's build and of the one it was measured against (the base once for each fix), or that of a build it takes a result from and the base's where larger; the builds whose noise is known are counted, and not measured is never identical | | F7 | The page's rule agrees with the Python rule on every shared case and generated boundary value, and the page refuses to compose where it cannot reproduce a fix's own classes | | F8 | A file of fixes holds nothing of the model in a pair: its kind comes from a closed list, and any text in it is refused | | F9 | A fix that requires another is measured against their build; choosing it chooses what it requires; in a set, each effect adds on top of what it was measured with, so what two fixes build on counts once; where a build inside the set holds every fix that changes an item, its result is taken, measured, not summed | | F10 | A built set that names a state of the chain is refused where its results do not reproduce that state | -| F11 | A fix may bring an item no state holds: it follows the matrix's items, with no base result, and a fix gives it one | +| F11 | A fix may bring an item no state holds: it follows the matrix's items, with no base result, and a fix gives it one; in a unit no item of the chain has, it takes a floor of 0 from nothing | +| F12 | A fix is judged with the noise of its build and of the build it was measured against; a built set is held to its sum with the noise of every build both stand on, and to its state with its noise and the state's; the base's noise is its state's, and the published results are exact | +| F13 | A pair with no size is built, and its build has none; "no size" and "not measured" read apart | ### 16.12 Who does what, and in which order diff --git a/src/sentier_diagnostics/fixes.py b/src/sentier_diagnostics/fixes.py index a9bb18e..0f121db 100644 --- a/src/sentier_diagnostics/fixes.py +++ b/src/sentier_diagnostics/fixes.py @@ -17,8 +17,14 @@ estimated otherwise: drawn, and marked on every cell and count A set has no verdict: each claim is judged in its own fix. A set's tolerance is the largest -of its fixes'; its noise is the sum of their noise, with the number of fixes whose noise was -measured, never read as none. +of its fixes'. Noise belongs to a build, as it belongs to a state of a chain (SPEC 5.3): the +base's is that state's (none where the base is the published results, which are exact), a +fix's is that of the build with it, a built set's that of its build. A fix is judged with the +sum of the noise of its build and of the one it was measured against. A set's result is a +build's, or a sum over its fixes, so its noise is the sum, over its fixes, of the noise of each +fix's build and of the one it was measured against; or that of a build it takes a result from, +and of the base, where that is larger. It says at how many of those builds the noise is known, +never reading not measured as none. A listed pair holds no text: its kind of interaction (one of `PAIR_KINDS`, from where the two changes meet in h = c·B·A⁻¹·f), how many items it can reach and the measured size of @@ -39,7 +45,7 @@ import json import re -from collections.abc import Iterable, Mapping +from collections.abc import Iterable, Mapping, Sequence from functools import cached_property from pathlib import Path @@ -54,6 +60,9 @@ #: A key a file may leave out, as in the diagnostics file. OPTIONAL_KEYS = {"indicator"} OF_KEYS = {"file", "title", "base"} +#: The noise of the base, when the base is a build; left out where it is the published +#: results, which are exact (SPEC 5.3). +OF_OPTIONAL = {"base_noise"} #: The rule of the diagnostics file, without the chain's own total tolerance. RULE_KEYS = fileformat.RULE_KEYS - {"total_tolerance"} FIX_KEYS = {"id", "label", "about", "kind", "explained", "link", "requires", "claim", "tolerance", "noise", @@ -64,13 +73,19 @@ #: Where two changes of h = c·B·A⁻¹·f meet, so that their joint result is not the sum of #: their effects (SPEC 16.3). Two changes to different factors, or to different entries of -#: B, add up exactly and are never a pair. +#: B, add up exactly and are never a pair. In the order of precedence: a pair that meets in +#: several ways is the first that applies. +SAME_ENTRY = "same entry" # both change one number of A, B or c: two edits of one number do not add TECHNOSPHERE = "technosphere" # both change A, and an item's supply chain reaches both TECHNOSPHERE_AND_EMISSION = "technosphere and emission" # A, and an amount of B on a chain the A change shifts TECHNOSPHERE_AND_FACTOR = "technosphere and factor" # A, and the factor of a flow the A change shifts EMISSION_AND_FACTOR = "emission and factor" # an amount of a flow, and that flow's factor -SAME_ENTRY = "same entry" # both change one number of A, B or c: two edits of one number do not add -PAIR_KINDS = (TECHNOSPHERE, TECHNOSPHERE_AND_EMISSION, TECHNOSPHERE_AND_FACTOR, EMISSION_AND_FACTOR, SAME_ENTRY) +PAIR_KINDS = (SAME_ENTRY, TECHNOSPHERE, TECHNOSPHERE_AND_EMISSION, TECHNOSPHERE_AND_FACTOR, EMISSION_AND_FACTOR) + +#: A pair's `largest` where no tolerance makes its build agree with the sum of its effects: +#: at some item the two lie on two sides of zero, one is zero, or one has no result (SPEC +#: 16.6). Not null, which says the size was not measured. +NO_SIZE = "no size" MAX_FIXES, MAX_BUILT = 1000, 500 MAX_ITEMS, MAX_BYTES = matrix.MAX_ITEMS, matrix.MAX_BYTES @@ -91,17 +106,18 @@ def fix(id: str, label: str, *, kind: str, requires: Iterable[str] = (), explained: bool = True, about: str | None = None, link: str | None = None, claim: dict | None = None, tolerance: dict | None = None, noise: dict | None = None) -> dict: # fmt: skip - """One fix, as `build` takes it: what a step has (`chain.step`), and the fixes it needs - applied first.""" + """One fix, as `build` takes it: what a step has (`chain.step`), the fixes it needs applied + first, and the noise of the build with it (`chain.noise`), which belongs to that build as a + state's noise belongs to the state.""" one = chain.step(id, label, kind=kind, explained=explained, about=about, link=link, claim=claim, - tolerance=tolerance, noise=noise) # fmt: skip - return {**one, "requires": list(requires)} + tolerance=tolerance) # fmt: skip + return {**one, "requires": list(requires), "noise": noise} -def pair(a: str, b: str, kind: str, *, reach: int | None = None, largest: float | None = None) -> dict: - """Two fixes that can interact: `kind` is one of `PAIR_KINDS`, `reach` how many items the - cross term can change, `largest` its measured size (SPEC 16.6); None where not counted or - not measured.""" +def pair(a: str, b: str, kind: str, *, reach: int | None = None, largest: float | str | None = None) -> dict: + """Two fixes that can interact: `kind` is the first of `PAIR_KINDS` that applies, `reach` + how many items the cross term can change, counted on the build with both; `largest` its + measured size (`size_of`), or `NO_SIZE`; None where not counted or not measured.""" return {"fixes": [a, b], "kind": kind, "reach": reach, "largest": largest} @@ -121,9 +137,45 @@ def built_set(id: str, label: str, fixes: Iterable[str], results: Mapping, *, st "noise": noise or chain.noise()} # fmt: skip -def _cell(full: Mapping, k: int, x: int, at: int, nstates: int): - """One result of a matrix: item `k`, indicator `x`, state `at`.""" - return full["values"][k][x * nstates + at] +def _rule_of(data: Mapping, added: Iterable) -> dict: + """The rule of a fixes file: its diagnostics file's, without the chain's total tolerance, + and a floor for every unit only an added item has: 0, from nothing, as the chain gives a + unit its first state holds no result in (an added item is in no state of it).""" + out = {k: data["rule"][k] for k in sorted(RULE_KEYS)} + known = {one["unit"] for one in data["rule"]["floors"]} + new = sorted({row[4] for row in added} - known, key=lambda u: (u is None, u or "")) + out["floors"] = [*out["floors"], *({"indicator": i["id"], "unit": unit, "value": 0.0, "basis": "none", "n": 0} + for i in data["indicators"] for unit in new)] # fmt: skip + return out + + +def _expanded(full: Mapping, nstates: int) -> list[list]: + """The results of a matrix, item by item, a result written as the one before it + (`matrix.SAME`) written out again.""" + return [matrix.expand(row, nstates) if matrix.SAME in row else row for row in full["values"]] + + +def _results_of(rows: list[list], x: int, at: int, nstates: int) -> list: + """The results of expanded matrix rows in indicator `x` and state `at`, item by item.""" + col = x * nstates + at + return [row[col] for row in rows] + + +def _base_noise(data: Mapping, at: int) -> dict | None: + """The noise of the state at place `at` of a checked diagnostics file: None where it is the + published results, which are exact. A file whose states carry no noise is refused: it never + says which state its noise was measured on.""" + if not fileformat.states_carry_noise(data): + raise ValueError(NO_STATE_NOISE) + state = data["states"][at] + return None if state["kind"] == "published" else state["noise"] + + +#: Why a fixes file needs the noise of the chain's states. +NO_STATE_NOISE = ( + "the diagnostics file says no noise of its states: it was made before noise belonged to a " + "state (SPEC 5.3), so the base's noise cannot be read from it; make it again" +) def build( @@ -152,8 +204,12 @@ def build( - `interactions`: `interactions(...)`; not checked where it is None. - `built`: `built_set(...)`. - `added_items`: `[id, name, place, group, unit]` of items a fix brings that the base - does not hold; the base has no result for them. + does not hold; the base has no result for them. An added item in a unit no item of the + chain has gets the floor the chain gives a unit with no results: 0, from nothing. - `file`: the name of the diagnostics file beside it. + + The base's noise is its state's (none where it is the published results); a fix's and a + built set's noise are those of their builds. """ ids = [i["id"] for i in data["indicators"]] states = [s["id"] for s in data["states"]] @@ -165,23 +221,25 @@ def build( index = {r[0]: k for k, r in enumerate(rows)} if len(index) != len(rows): raise ValueError("an added item is an item of the base already") - base_values = { - ind: [_cell(full, k, x, at, nstates) for k in range(m)] + [None] * (len(rows) - m) - for x, ind in enumerate(ids) - } + base_noise = _base_noise(data, at) + expanded = _expanded(full, nstates) + base_values = {ind: _results_of(expanded, x, at, nstates) + [None] * (len(rows) - m) for x, ind in enumerate(ids)} known = [one["id"] for one in fixes] unknown = set(results) - set(known) if unknown: raise ValueError(f"results of fixes that are not given: {sorted(unknown)}") + of = {"file": file, "title": data["title"], "base": base} + if base_noise is not None: + of["base_noise"] = base_noise f = { "format": FORMAT, - "of": {"file": file, "title": data["title"], "base": base}, + "of": of, "title": title, "about": about, "item": dict(data["item"]), **({"indicator": dict(data["indicator"])} if data.get("indicator") else {}), "indicators": [dict(i) for i in data["indicators"]], - "rule": {k: data["rule"][k] for k in sorted(RULE_KEYS)}, + "rule": _rule_of(data, rows[m:]), "items": rows, "base": base_values, "fixes": [], @@ -212,11 +270,12 @@ def sparse(given: Mapping, where: str, under: Mapping) -> dict: return out closures: dict[str, frozenset] = {} - builds = _Builds(ctx, closures) + builds = _Builds(ctx, closures, base_noise) nothing = {ind: {} for ind in ids} for one in built: results_of = sparse(one["results"], f"built set {one['id']!r}", nothing) builds.built[frozenset(one["fixes"])] = _columns(results_of) + builds.noise[frozenset(one["fixes"])] = one.get("noise") or chain.noise() f["built"].append({ "id": one["id"], "label": one["label"], @@ -253,7 +312,8 @@ def sparse(given: Mapping, where: str, under: Mapping) -> dict: "counts": None, } builds.entries[one["id"]] = _columns(made["effect"]) - made["counts"] = _fix_counts(ctx, made, builds.entries[one["id"]], before) + builds.noise[one["id"]] = made["noise"] + made["counts"] = _fix_counts(builds, made, builds.entries[one["id"]], before) f["fixes"].append(made) found = errors(f) or against(f, data, full) if found: @@ -385,22 +445,42 @@ def tally(self) -> dict[str, dict[str, int]]: class _Builds: """Every build a file holds, as a column over the base (`{indicator: {item index: result}}`, the items where it is not the base): a fix with what it requires (its effect laid over the - build of its requirements) and every built set.""" + build of its requirements) and every built set; and the noise of each, as an end of a + comparison (`end`).""" - def __init__(self, base: _Base, closures: Mapping[str, frozenset]): + def __init__(self, base: _Base, closures: Mapping[str, frozenset], base_noise: Mapping | None = None): self.base = base self.closures = closures # fix id: every fix it requires, at any depth self.entries: dict[str, dict] = {} # fix id: its written results, by indicator self.built: dict[frozenset, dict] = {} # the fixes of a built set: its results + self.noise: dict[str | frozenset, Mapping] = {} # a fix's id, or a built set's fixes: its noise + # the base as an end: the published results are exact + self.base_end = {"kind": "published"} if base_noise is None else {"kind": "build", "noise": base_noise} self._memo: dict[frozenset, dict] = {} @classmethod def of(cls, f: Mapping) -> _Builds: - found = cls(_Base(f), _closures(f)) + found = cls(_Base(f), _closures(f), f["of"].get("base_noise")) found.entries = {one["id"]: _columns(one["effect"]) for one in f["fixes"]} found.built = {frozenset(b["fixes"]): _columns(b["results"]) for b in f["built"]} + found.noise = {one["id"]: one["noise"] for one in f["fixes"]} + found.noise.update((frozenset(b["fixes"]), b["noise"]) for b in f["built"]) return found + def end(self, fixes: Iterable[str]) -> tuple[str | frozenset, Mapping] | None: + """The build of exactly these fixes that `column` gives, as an end of a comparison: + `(what names it, {kind, noise})`, the base being exact where it is the published + results; None where the file holds no such build.""" + fixes = frozenset(fixes) + if not fixes: + return "", self.base_end + top = self.top(fixes) + if top is not None and self.column(self.closures[top]) is not None: + return top, {"kind": "build", "noise": self.noise[top]} + if fixes in self.built: + return fixes, {"kind": "build", "noise": self.noise[fixes]} + return None + def top(self, fixes: frozenset) -> str | None: """The fix that, with what it requires, is exactly these fixes; None where none is.""" return next((y for y in fixes if y in self.entries and self.closures[y] | {y} == fixes), None) @@ -433,13 +513,15 @@ def covers(self, fixes: frozenset, *, built: bool = True) -> list[frozenset]: found += [b for b in self.built if b < fixes] return found - def composed(self, fixes: frozenset, ind: str, *, built: bool = True) -> dict[int, tuple[float | None, bool]]: + def composed(self, fixes: frozenset, ind: str, *, built: bool = True, + used: set | None = None) -> dict[int, tuple[float | None, bool]]: # fmt: skip """A closed set's result in one indicator, at every item one of its fixes changes: `{item index: (result, whether it could be formed)}`. Where the fixes that change an item all lie in one build inside the set (`covers`), the first such build's result is the set's: measured, not summed. Otherwise the base plus each fix's difference against what it was measured against. `built=False` leaves the built sets out: the sum a built - set is held to stands on the fixes alone.""" + set is held to stands on the fixes alone. `used` gathers the builds a result was taken + from.""" touching: dict[int, list[str]] = {} for y in fixes: for k in self.entries[y][ind]: @@ -455,6 +537,8 @@ def composed(self, fixes: frozenset, ind: str, *, built: bool = True) -> dict[in here = set(ys) cover = next((c for c in covers if here <= c), None) if cover is not None: + if used is not None: + used.add(cover) column = self.column(cover)[ind] out[k] = (column[k] if k in column else base[k], True) else: @@ -489,8 +573,36 @@ def _sum(before: float | None, changes: list[tuple[float | None, float | None]]) return before + sum(a - b for a, b in changes), True -def _level(noise: Mapping, indicator: str) -> float | None: - return (noise["by"] or {}).get(indicator) if noise["measured"] else None +def _level(ends: Iterable[Mapping], indicator: str) -> float | None: + """The noise a comparison is judged with in one indicator: the sum of the noise of its ends + (`{kind, noise}`, SPEC 5.3), each counted as often as it enters it. The published results + are exact and add nothing; an end that did not measure the indicator adds nothing either; + None where no end measured it. For two ends it is `fileformat.noise_of_states`, bit for bit. + + >>> built = lambda by: {"kind": "build", "noise": {"measured": by is not None, "between": by and ["x", "y"], "by": by, "note": None}} + >>> _level([built({"a": 1e-9}), built({"a": 3e-9})], "a"), _level([{"kind": "published"}, built(None)], "a") + (4e-09, None) + >>> _level([built({"a": 0.25})] * 3 + [built({"b": 1.0})], "a") + 0.75 + """ + level = None + for end in ends: + if end["kind"] == "published": + continue + said = end["noise"] + if said["measured"] and indicator in said["by"]: + level = (0.0 if level is None else level) + said["by"][indicator] + return level + + +def _known(end: Mapping, indicator: str) -> bool: + """Whether the noise of an end is known in one indicator: the published results (exact), or + a build that measured it (`fileformat.ends_known`).""" + return end["kind"] == "published" or (end["noise"]["measured"] and indicator in end["noise"]["by"]) + + +def _built(noise: Mapping) -> dict: + return {"kind": "build", "noise": noise} def _side(before, after) -> str: @@ -561,13 +673,14 @@ def kind_of(k): "verdicts": verdicts, "closer": None, "further": None} # fmt: skip -def _fix_counts(base: _Base, one: Mapping, effect: Mapping, before: Mapping) -> dict: +def _fix_counts(builds: _Builds, one: Mapping, effect: Mapping, before: Mapping) -> dict: """A fix's counts: the build of what it requires against that build with its effect, with - its own tolerance, noise and claim.""" + its own tolerance and claim, and the noise of those two builds.""" tol = one["tolerance"]["value"] + ends = [builds.end(builds.closures[one["id"]])[1], _built(one["noise"])] return { - ind: _block(base, ind, effect[ind], tol, _level(one["noise"], ind), one["claim"], before[ind]) - for ind in base.ids + ind: _block(builds.base, ind, effect[ind], tol, _level(ends, ind), one["claim"], before[ind]) + for ind in builds.base.ids } @@ -620,12 +733,41 @@ def _label(f: Mapping, builds: _Builds, names: list[str]) -> str: def pair_terms(f: Mapping, ids: Iterable[str]) -> dict: """How far an estimated set may lie from its sum, to first order: `{value, measured, - pairs}`, the value being the sum of the measured sizes of the listed pairs it holds, and - None where some were not measured (a pair not measured is never a pair of size 0).""" + no_size, pairs}`, the value being the sum of the sizes of the listed pairs it holds; None + where some were not measured (a pair not measured is never a pair of size 0) or have no + size (`NO_SIZE`), which the counts tell apart.""" held = held_pairs(f, ids) - sizes = [p["largest"] for p in held if p["largest"] is not None] + said = [p["largest"] for p in held if p["largest"] is not None] + sizes = [x for x in said if x != NO_SIZE] value = sum(sizes) if len(sizes) == len(held) else None - return {"value": value, "measured": len(sizes), "pairs": len(held)} + return {"value": value, "measured": len(said), "no_size": len(said) - len(sizes), "pairs": len(held)} + + +def size_of(summed: Sequence, built: Sequence, floor: Sequence) -> float | str: + """The size of a pair's cross term in one indicator (SPEC 16.6), item by item: the + smallest tolerance under which its build agrees with the sum of its effects, by the rule. + That is the largest relative difference of the two (`rule.relative_difference`) over the + items the rule does not call near zero on both sides; `NO_SIZE` where no tolerance is + enough: an item where the two lie on two sides of zero, one is zero, or one has no + result. A pair's size is the largest over the indicators. + + >>> size_of([1.5, 2.0], [2.0, 2.0], [0.01, 0.01]) + 0.25 + >>> size_of([1.5], [-0.1], [0.01]), size_of([-0.5], [0.0], [0.01]), size_of([1e-9], [-1e-9], [0.01]) + ('no size', 'no size', 0.0) + """ + largest = 0.0 + for s, b, fl in zip(summed, built, floor): + if s is None and b is None: + continue + if s is None or b is None: + return NO_SIZE + if abs(s) <= fl and abs(b) <= fl: + continue + if not ((s > 0 and b > 0) or (s < 0 and b < 0)): + return NO_SIZE + largest = max(largest, rule.relative_difference(s, b)) + return largest def tolerance_of(f: Mapping, ids: Iterable[str]) -> float: @@ -634,29 +776,50 @@ def tolerance_of(f: Mapping, ids: Iterable[str]) -> float: def noise_of(f: Mapping, ids: Iterable[str]) -> dict[str, dict]: - """The noise of a set, per indicator: `{value, measured, fixes}`, the value being the sum - of the noise of the fixes whose noise was measured there, and None where none was: not - measured is never none.""" - fixes = _fixes_of(f, chosen(f, ids)[0]) + """The noise of a set against the base, per indicator (SPEC 16.5): `{value, known, builds}`. + + A built set's is the sum of the noise of its build and of the base. Any other set's result + at an item is a build's, or the base plus each fix's difference against what it was + measured against: its value is the sum, over its fixes, of the noise of each fix's build and + of the one it was measured against (the base once for each fix measured against it), or, + where larger, that of a build a result is taken from and of the base. `builds` counts the + builds that enter it, `known` those whose noise is known there (the published results are + exact); None where none measured it: not measured is never none.""" + names = chosen(f, ids)[0] + builds = _Builds.of(f) + return _set_noise(builds, names, _changed(builds, names)[2]) + + +def _set_noise(builds: _Builds, names: list[str], used: Mapping[str, set]) -> dict[str, dict]: + whole = builds.end(names) + base = ("", builds.base_end) out = {} - for ind in (i["id"] for i in f["indicators"]): - levels = [lv for lv in (_level(one["noise"], ind) for one in fixes) if lv is not None] - out[ind] = {"value": sum(levels) if levels else None, "measured": len(levels), "fixes": len(fixes)} + for ind in builds.base.ids: + if whole is not None: + sums = [[base, whole]] + else: + terms = [end for y in names for end in (builds.end(builds.closures[y]), (y, _built(builds.noise[y])))] + sums = [terms, *([base, builds.end(cover)] for cover in sorted(used[ind], key=sorted))] + levels = [lv for lv in (_level([end for _, end in one], ind) for one in sums) if lv is not None] + every = {name: end for one in sums for name, end in one} + out[ind] = {"value": max(levels) if levels else None, + "known": sum(_known(end, ind) for end in every.values()), "builds": len(every)} # fmt: skip return out -def _changed(builds: _Builds, names: list[str]) -> tuple[dict, dict]: - """The items a closed set changes, per indicator, with its result at each; and the items - where no sum can be formed.""" +def _changed(builds: _Builds, names: list[str]) -> tuple[dict, dict, dict]: + """The items a closed set changes, per indicator, with its result at each; the items + where no sum can be formed; and the builds inside the set a result was taken from.""" column = builds.column(names) if column is not None: - return column, {ind: [] for ind in builds.base.ids} - changed, cannot = {}, {} + return column, {ind: [] for ind in builds.base.ids}, {ind: set() for ind in builds.base.ids} + changed, cannot, used = {}, {}, {} for ind in builds.base.ids: - cells = builds.composed(frozenset(names), ind) + used[ind] = set() + cells = builds.composed(frozenset(names), ind, used=used[ind]) changed[ind] = {k: v for k, (v, _) in cells.items()} cannot[ind] = sorted(k for k, (_, ok) in cells.items() if not ok) - return changed, cannot + return changed, cannot, used def compose(f: Mapping, ids: Iterable[str]) -> dict: @@ -668,7 +831,7 @@ def compose(f: Mapping, ids: Iterable[str]) -> dict: there, so that it reads apart from a missing result. A set has no verdict.""" names, added = chosen(f, ids) builds = _Builds.of(f) - changed, cannot = _changed(builds, names) + changed, cannot, used = _changed(builds, names) results = {} for ind in builds.base.ids: column = list(builds.base.values[ind]) @@ -682,7 +845,7 @@ def compose(f: Mapping, ids: Iterable[str]) -> dict: "pairs": held_pairs(f, names), "pair_terms": pair_terms(f, names), "tolerance": tolerance_of(f, names), - "noise": noise_of(f, names), + "noise": _set_noise(builds, names, used), "results": results, "cannot_add": cannot, } @@ -693,8 +856,8 @@ def counts(f: Mapping, ids: Iterable[str]) -> dict: estimated set's counts are estimates ("estimated: 153 would differ"). No verdicts.""" names = chosen(f, ids)[0] builds = _Builds.of(f) - changed, _ = _changed(builds, names) - tol, noise = tolerance_of(f, names), noise_of(f, names) + changed, _, used = _changed(builds, names) + tol, noise = tolerance_of(f, names), _set_noise(builds, names, used) return { "label": _label(f, builds, names), "by": {ind: _block(builds.base, ind, changed[ind], tol, noise[ind]["value"], None) for ind in builds.base.ids}, @@ -729,13 +892,16 @@ def _errors(f, found: list[str]) -> list[str]: of = f["of"] if not ( isinstance(of, dict) - and set(of) == OF_KEYS + and OF_KEYS <= set(of) <= OF_KEYS | OF_OPTIONAL and isinstance(of["file"], str) and fileformat.FILE_NAME.fullmatch(of["file"]) and fileformat.is_text(of["title"], 120) and fileformat.is_text(of["base"], fileformat.MAX_ID) ): - found.append("`of` is not {file, title, base}: a file name, that file's title and a state's id") + found.append( + "`of` is not {file, title, base}, with `base_noise` where the base is a build: a file name, that " + "file's title and a state's id" + ) if not fileformat.is_text(f["title"], 120) or not fileformat.is_optional(f["about"]): found.append("`title` is not a short text, or `about` not a text") for key in ("item", "indicator"): @@ -754,6 +920,8 @@ def _errors(f, found: list[str]) -> list[str]: floors = _rule(f, found, indicators) if not indicators or floors is None or not _items(f, found, indicators, floors): return found + if isinstance(of, dict) and "base_noise" in of: + found += fileformat.noise_errors(of["base_noise"], "`of.base_noise`", indicators) n = len(f["items"]) said_base = f["base"] if not ( @@ -769,7 +937,7 @@ def _errors(f, found: list[str]) -> list[str]: closures = _requirements(f, found) if closures is None: return found - builds = _Builds(base, closures) + builds = _Builds(base, closures, f["of"].get("base_noise")) built = _built_sets(f, found, base, builds) if built is None or not _fixes(f, found, base, builds): return found @@ -879,29 +1047,6 @@ def _sparse(said, where: str, base: _Base, found: list[str], under: Mapping | No return out -def _noise_errors(noise, where: str, indicators) -> list[str]: - if not ( - isinstance(noise, dict) - and set(noise) == {"measured", "between", "by", "note"} - and isinstance(noise["measured"], bool) - and fileformat.is_optional(noise["note"]) - ): - return [f"{where}: `noise` is not {{measured, between, by, note}}"] - if noise["measured"]: - if not ( - isinstance(noise["between"], list) - and len(noise["between"]) == 2 - and all(fileformat.is_text(n, fileformat.MAX_LABEL) for n in noise["between"]) - and isinstance(noise["by"], dict) - and set(noise["by"]) <= set(indicators) - and all(fileformat.is_share(v) for v in noise["by"].values()) - ): - return [f"{where}: measured noise names the two builds and a size per indicator"] - elif noise["between"] is not None or noise["by"] is not None: - return [f"{where}: noise that was not measured has no builds and no sizes"] - return [] - - def _requirements(f, found) -> dict[str, frozenset] | None: """Every fix's id and what it requires, at any depth. A fix's requirements come before it in `fixes`, so that no fix requires itself through others.""" @@ -988,7 +1133,7 @@ def _built_sets(f, found, base: _Base, builds: _Builds) -> list | None: if key in sets: found.append(f"{where}: holds the same fixes as another built set") sets.add(key) - noise = _noise_errors(one["noise"], where, base.ids) + noise = fileformat.noise_errors(one["noise"], where, base.ids) if noise: found += noise continue @@ -996,6 +1141,7 @@ def _built_sets(f, found, base: _Base, builds: _Builds) -> list | None: if results is None: continue builds.built[key] = results + builds.noise[key] = one["noise"] out.append((one, key, results)) return None if len(found) > before else out @@ -1033,10 +1179,11 @@ def _fixes(f, found, base: _Base, builds: _Builds) -> bool: ): found.append(f"{where}: `tolerance` is not {{value, basis, note}}") return False - noise = _noise_errors(one["noise"], where, base.ids) + noise = fileformat.noise_errors(one["noise"], where, base.ids) if noise: found += noise return False + builds.noise[one["id"]] = one["noise"] before = builds.column(builds.closures[one["id"]]) if before is None: needs = ", ".join(repr(x) for x in sorted(builds.closures[one["id"]])) @@ -1059,7 +1206,7 @@ def _fixes(f, found, base: _Base, builds: _Builds) -> bool: if shape: found += shape continue - again = _fix_counts(base, one, effect, before) + again = _fix_counts(builds, one, effect, before) for ind in base.ids: for key, value in again[ind].items(): if written[ind][key] != value: @@ -1127,8 +1274,11 @@ def _interactions(f, found, closures: Mapping[str, frozenset], n: int) -> dict[f found.append(f"{where}: `kind` is not one of {', '.join(PAIR_KINDS)}") if one["reach"] is not None and not (fileformat.is_count(one["reach"]) and 0 < one["reach"] <= n): found.append(f"{where}: `reach` is not a count of 1 to {n} items, or null") - if one["largest"] is not None and not fileformat.is_share(one["largest"]): - found.append(f"{where}: `largest` is not a share, or null where it was not measured") + if one["largest"] not in (None, NO_SIZE) and not fileformat.is_share(one["largest"]): + found.append( + f"{where}: `largest` is not a share, {NO_SIZE!r} where no tolerance is enough, or null where it was " + "not measured" + ) key = frozenset(named) if key in pairs: found.append(f"{where}: is listed twice") @@ -1138,19 +1288,48 @@ def _interactions(f, found, closures: Mapping[str, frozenset], n: int) -> dict[f def _built_checks(f, found, builds: _Builds, built: list, pairs: Mapping[frozenset, dict]) -> None: """A built set the file calls independent must be the sum of its effects; a built pair - whose cross term the file measured must lie within it.""" + whose cross term the file measured must lie within it; a pair with no size must be built, + and its build must have none.""" if not f["interactions"]["checked"]: return by_id = {one["id"]: one for one in f["fixes"]} + for key, one in pairs.items(): + a, b = one["fixes"] + if one["largest"] == NO_SIZE and builds.column(key | builds.closures[a] | builds.closures[b]) is None: + found.append( + f"the pair {a!r} and {b!r}: has no size, so no set holding it can be drawn from its sum: the " + "file holds a build of the two with what they require (a built set)" + ) for one, key, results in built: where = f"built set {one['id']!r}" held = [p for p in pairs if p <= key] if not held: found += _disagreements(builds, where, key, by_id, one, results) + elif held == [key] and pairs[key]["largest"] == NO_SIZE: + found += _sized(builds, where, key, results) elif held == [key] and pairs[key]["largest"] is not None: found += _disagreements(builds, where, key, by_id, one, results, pairs[key]["largest"]) +def _sized(builds: _Builds, where: str, key: frozenset, results: Mapping) -> list[str]: + """A built pair the file says has no size must have none: where some tolerance makes its + build agree with its sum, that tolerance is its size, and the file says it.""" + base = builds.base + sizes = [] + for ind in base.ids: + cells, values = builds.composed(key, ind, built=False), base.values[ind] + items = sorted(set(cells) | set(results[ind])) + summed = [cells[k][0] if k in cells else values[k] for k in items] + if not all(cells[k][1] for k in items if k in cells): + return [] # no sum at some item: nothing to measure against + size = size_of(summed, [results[ind].get(k, values[k]) for k in items], [base.floor[ind][k] for k in items]) + if size == NO_SIZE: + return [] + sizes.append(size) + size = _number(max(sizes)) + return [f"{where}: the file says its pair has no size, and its build lies within {size} of the sum of its effects"] + + #: How many disagreements of one built set are named before the rest are counted. NAMED = 5 @@ -1165,23 +1344,23 @@ def _number(value: float | None) -> str: return "null" if value is None else f"{value:.10g}" -def _tolerance_and_noise(fixes: list[Mapping], ind: str, own: Mapping | None = None) -> tuple[float, float | None]: - """The tolerance of a set (the largest of its fixes') and its noise (the sum of theirs, - and of `own` where a built set's own noise is measured); None where none is measured.""" - noises = [x["noise"] for x in fixes] + ([own] if own else []) - levels = [lv for lv in (_level(x, ind) for x in noises) if lv is not None] - return max(x["tolerance"]["value"] for x in fixes), (sum(levels) if levels else None) +def _sum_ends(builds: _Builds, fixes: Iterable[str]) -> list[Mapping]: + """The builds a sum over these fixes stands on, as ends: each fix's build and the one it was + measured against (the base, once for each fix measured against it).""" + return [end for y in sorted(fixes) for end in (builds.end(builds.closures[y])[1], _built(builds.noise[y]))] def _disagreements(builds: _Builds, where: str, key: frozenset, by_id: Mapping, one: Mapping, results: Mapping, largest: float | None = None) -> list[str]: # fmt: skip """Where a built set is further from the sum of its effects than the file says, by the - rule, with the set's tolerance and the sum of its fixes' noise and its own.""" + rule, with the largest tolerance of its fixes and the noise of every build the two sides + stand on: the base and each fix's build and what it was measured against, in the sum; the + built set, on the other side.""" base = builds.base - fixes = [by_id[x] for x in sorted(key)] + ends = [builds.base_end, *_sum_ends(builds, key), _built(one["noise"])] found, more = [], 0 for ind in base.ids: - tol, level = _tolerance_and_noise(fixes, ind, one["noise"]) + tol, level = max(by_id[x]["tolerance"]["value"] for x in key), _level(ends, ind) if largest is not None: tol = max(tol, largest) cells = builds.composed(key, ind, built=False) @@ -1230,12 +1409,26 @@ def _against(f: Mapping, data: Mapping, full: Mapping | None) -> list[str]: states = [s["id"] for s in data["states"]] if of["base"] not in states: return [*found, f"the base {of['base']!r} is no state of the diagnostics file"] + if not fileformat.states_carry_noise(data): + return [*found, NO_STATE_NOISE] + base_state = data["states"][states.index(of["base"])] + if base_state["kind"] == "published": + if "base_noise" in of: + found.append("the base is the published results, which are exact: `of.base_noise` is left out") + elif "base_noise" not in of: + found.append("the base is a build: `of.base_noise` is its noise, as the diagnostics file says it") + elif of["base_noise"] != base_state["noise"]: + found.append(f"`of.base_noise` is not the noise the diagnostics file gives state {of['base']!r}") if f["indicators"] != data["indicators"]: found.append("its indicators differ from the diagnostics file's") if f["item"] != data["item"] or f.get("indicator") != data.get("indicator"): found.append("its words for an item or an indicator differ from the diagnostics file's") - if f["rule"] != {k: data["rule"][k] for k in RULE_KEYS}: - found.append("its floors differ from the diagnostics file's") + m = len(full["items"]) if full is not None else data["items_count"]["all"] + if f["rule"] != _rule_of(data, f["items"][m:]): + found.append( + "its floors differ from the diagnostics file's, with a floor of 0 from nothing for each unit only " + "an added item has" + ) for one in f["built"]: later = one["state"] in states and states.index(one["state"]) > states.index(of["base"]) if one["state"] is not None and not later: @@ -1245,14 +1438,14 @@ def _against(f: Mapping, data: Mapping, full: Mapping | None) -> list[str]: ids = [i["id"] for i in data["indicators"]] at, nstates = states.index(of["base"]), len(states) if full is not None: - m = len(full["items"]) if [list(r) for r in f["items"][:m]] != [list(r) for r in full["items"]]: return ["its items are not the matrix's, in the matrix's order, before any item a fix adds"] + expanded = _expanded(full, nstates) for x, ind in enumerate(ids): - if [_cell(full, k, x, at, nstates) for k in range(m)] != f["base"][ind][:m]: + if _results_of(expanded, x, at, nstates) != f["base"][ind][:m]: found.append(f"the base results in {ind!r} are not the matrix's results of state {of['base']!r}") found += _added_items(f, m) - found += _states(f, data, lambda k, x, s: _cell(full, k, x, states.index(s), nstates) if k < m else None, + found += _states(f, data, lambda k, x, s: expanded[k][x * nstates + states.index(s)] if k < m else None, range(len(f["items"]))) # fmt: skip return found if len(f["items"]) < data["items_count"]["all"]: @@ -1293,19 +1486,23 @@ def _added_items(f: Mapping, m: int) -> list[str]: def _states(f: Mapping, data: Mapping, cell, items) -> list[str]: """A built set that names a state of the chain is two builds of one thing: by the rule, - with the set's tolerance and the sum of its fixes' noise and its own.""" + with the largest tolerance of its fixes and the noise of the two builds, its own and the + state's (none where that is the published results).""" found = [] by_id = {one["id"]: one for one in f["fixes"]} floors = _Base(f).floor + of_state = {s["id"]: s for s in data["states"]} for one in f["built"]: if one["state"] is None: continue where = f"built set {one['id']!r}" - fixes = [by_id[x] for x in one["fixes"]] + state = of_state[one["state"]] + ends = [_built(one["noise"]), state if state["kind"] == "published" else _built(state["noise"])] + tol = max(by_id[x]["tolerance"]["value"] for x in one["fixes"]) results = _columns(one["results"]) named = 0 for x, ind in enumerate(f["base"]): - tol, level = _tolerance_and_noise(fixes, ind, one["noise"]) + level = _level(ends, ind) for k in items: got = results[ind].get(k, f["base"][ind][k]) there = cell(k, x, one["state"]) diff --git a/tests/test_fixes.py b/tests/test_fixes.py index 77cf3ba..3e4e81f 100644 --- a/tests/test_fixes.py +++ b/tests/test_fixes.py @@ -10,7 +10,7 @@ import unittest from pathlib import Path -from sentier_diagnostics import chain, claims, cli, fixes, rule +from sentier_diagnostics import chain, claims, cli, fixes, matrix, rule from . import helpers from .helpers import INDICATORS, A, B @@ -18,16 +18,18 @@ FLOORS = {(A, "kg"): 0.01, (B, "kg"): 0.01} -def chain_and_matrix(last=None, release=None): +def chain_and_matrix(last=None, release=None, noise=None): """A chain of two states and its matrix; the base is the second state. Item p3 moved in - the step, so it is the one listed item. With `release`, a third state after the base.""" + the step, so it is the one listed item. With `release`, a third state after the base. + `noise` gives a built state its noise, by its id.""" + noise = noise or {} s1 = last or {A: {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0}, B: {"p1": 3.0, "p2": 5.0, "p3": 7.0, "p4": 1.0}} s0 = {A: {**s1[A], "p3": 4.0}, B: dict(s1[B])} items = sorted({i for column in s0.values() for i in column} | {i for column in s1.values() for i in column}) - states = [chain.state("s0", "Before", kind="published"), chain.state("s1", "After")] + states = [chain.state("s0", "Before", kind="published"), chain.state("s1", "After", noise=noise.get("s1"))] steps, values = [chain.step("one", "One step", kind="fix")], {"s0": s0, "s1": s1} if release is not None: - states.append(chain.state("s2", "Release")) + states.append(chain.state("s2", "Release", noise=noise.get("s2"))) steps.append(chain.step("two", "Every fix", kind="fix")) values["s2"] = release return chain.build_with_matrix( @@ -50,7 +52,8 @@ def checked_pairs(*pairs): RELEASE = {A: {"p1": 1.0013, "p2": 2.0, "p3": 2.0, "p4": 0.0}, B: {"p1": 3.0, "p2": 5.0, "p3": 6.0, "p4": 1.0}} -def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", built=True, release=False): +def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", built=True, release=False, + state_noise=None, noise_ab=None, noise_all=None): # fmt: skip """Three fixes against the base: - a and b each move p1 in A by less than their tolerance (1e-3): 1.0 to 1.0006 and 1.0007; @@ -58,12 +61,16 @@ def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", - b and c can interact (a listed pair); the set of a and b was built. With `release`, the chain has a third state, the base with every fix, and a built set of - every fix says it is that state.""" - data, full = chain_and_matrix(release=RELEASE if release else None) + every fix says it is that state. `state_noise` gives the chain's built states their noise + (s1 is the base), `noise_ab` and `noise_all` the built sets theirs.""" + data, full = chain_and_matrix(release=RELEASE if release else None, noise=state_noise) claim_c = claims.claim("p3 goes down.", [claims.scope([A], {"p3"}, "down", 0.5, claims.MUST)], by=claims.origin("person", "A. Author")) # fmt: skip pairs = {"listed": checked_pairs(fixes.pair("b", "c", fixes.TECHNOSPHERE)), "none": checked_pairs(), "unchecked": fixes.interactions(False)}[interactions] # fmt: skip + sets = [fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": built_ab}}, noise=noise_ab)] if built else [] + if release: + sets.append(fixes.built_set("all", "Every fix", ["a", "b", "c"], RELEASE, state="s2", noise=noise_all)) f = fixes.build( "Fixes of two states", data=data, @@ -76,10 +83,7 @@ def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", ], results={"a": {A: {"p1": 1.0006}}, "b": {A: {"p1": 1.0007}}, "c": {A: {"p3": 2.0}, B: {"p3": 6.0}}}, interactions=pairs, - built=[ - *([fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": built_ab}})] if built else ()), - *([fixes.built_set("all", "Every fix", ["a", "b", "c"], RELEASE, state="s2")] if release else ()), - ], + built=sets, ) return f, data, full @@ -129,8 +133,6 @@ def test_written_and_read_back_beside_its_diagnostics_file(self): chain.write(data, folder / chain.FILE) fixes.write(f, folder / "fixes.json") self.assertEqual(fixes.read(folder / "fixes.json"), f) # without the matrix: the listed items - from sentier_diagnostics import matrix - matrix.write(full, folder / "matrix.json") self.assertEqual(fixes.read(folder / "fixes.json"), f) @@ -326,17 +328,21 @@ def add(names): self.assertTrue(said(fixes.errors(twice), "two built sets have the id 'ab'")) -def dense(f, fix_id, under=None): +def dense(f, fix_id, under=None, under_noise=None): """A fix's counts worked out item by item, as `chain` counts a step: the reference the sparse count is held to. `under` is what the fix was measured against, written out by - hand (`{indicator: {item index: result}}`), the base where it says nothing.""" + hand (`{indicator: {item index: result}}`), the base where it says nothing; `under_noise` + the noise of that build, the base's where it is None. The noise is the sum of the two + builds' (SPEC 5.3).""" under = under or {} + ends = [under_noise or f["of"].get("base_noise"), next(x for x in f["fixes"] if x["id"] == fix_id)["noise"]] one = next(x for x in f["fixes"] if x["id"] == fix_id) floors = {(x["indicator"], x["unit"]): x["value"] for x in f["rule"]["floors"]} out = {} for ind in (i["id"] for i in f["indicators"]): effect = dict(map(tuple, one["effect"][ind])) - level = (one["noise"]["by"] or {}).get(ind) if one["noise"]["measured"] else None + said = [n["by"][ind] for n in ends if n and n["measured"] and ind in n["by"]] + level = None if not said else said[0] if len(said) == 1 else said[0] + said[1] block = {"classes": dict.fromkeys(rule.CLASSES, 0), "missing": {"before": 0, "after": 0, "both": 0}, "verdicts": dict.fromkeys(claims.VERDICTS, 0) if one["claim"] else None} # fmt: skip for k, row in enumerate(f["items"]): @@ -391,7 +397,7 @@ def test_f2_a_set_with_a_listed_pair_is_estimated_everywhere(self): drawn = fixes.compose(f, ["c", "b"]) self.assertEqual(drawn["label"], fixes.ESTIMATED) self.assertEqual(drawn["pairs"], [pair("b", "c")]) - self.assertEqual(drawn["pair_terms"], {"value": None, "measured": 0, "pairs": 1}) + self.assertEqual(drawn["pair_terms"], {"value": None, "measured": 0, "no_size": 0, "pairs": 1}) self.assertEqual(fixes.counts(f, ["b", "c"])["label"], fixes.ESTIMATED) self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.ESTIMATED) # a built subset makes no set built self.assertEqual(fixes.label(f, ["a", "c"]), fixes.EXACT) @@ -445,12 +451,14 @@ def test_f5_a_fix_is_judged_the_same_whichever_fixes_are_chosen_with_it(self): self.assertEqual(fixes.compose(f, ["b", "a"])["results"], fixes.compose(f, ["a", "b"])["results"]) def test_f6_the_noise_of_a_set_is_the_sum_said_with_how_many_were_measured(self): + """The base (s1) was not measured: it adds nothing, and is one build of three whose + noise is not known.""" noise = chain.noise(["one", "two"], {A: 1e-3}) f, *_ = made(noise_a=noise, noise_b=noise, interactions="none", built=False) - self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 2e-3, "measured": 2, "fixes": 2}) - self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-3, "measured": 1, "fixes": 2}) - self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": None, "measured": 0, "fixes": 1}) - self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "measured": 0, "fixes": 1}) + self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 2e-3, "known": 2, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-3, "known": 1, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": None, "known": 0, "builds": 2}) + self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 0, "builds": 2}) # the sum, not the largest: 1.3e-3 apart is within 2e-3 and beyond 1e-3 self.assertEqual(fixes.counts(f, ["a", "b"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) @@ -461,7 +469,7 @@ def test_f8_a_pair_holds_no_text_so_nothing_of_the_model_goes_into_it(self): cases = { "a pair is not {fixes, kind, reach, largest}": lambda g: g["interactions"]["pairs"][0].__setitem__( "reason", f"both change {uuid_shaped}"), - "`kind` is not one of technosphere": lambda g: g["interactions"]["pairs"][0].__setitem__( + "`kind` is not one of same entry": lambda g: g["interactions"]["pairs"][0].__setitem__( "kind", f"both change {uuid_shaped}"), "`interactions.method` holds an identifier": lambda g: g["interactions"].__setitem__( "method", "Compared " + "ab" * 16), @@ -495,8 +503,31 @@ def test_the_pair_terms_of_a_set_are_the_sum_of_its_measured_pairs(self): self.assertEqual((terms["measured"], terms["pairs"]), (2, 2)) self.assertAlmostEqual(terms["value"], 3e-5) f["interactions"]["pairs"][1]["largest"] = None - self.assertEqual(fixes.pair_terms(f, ["a", "b", "c"]), {"value": None, "measured": 1, "pairs": 2}) - self.assertEqual(fixes.pair_terms(f, ["a", "b"]), {"value": 0, "measured": 0, "pairs": 0}) + self.assertEqual(fixes.pair_terms(f, ["a", "b", "c"]), {"value": None, "measured": 1, "no_size": 0, "pairs": 2}) + self.assertEqual(fixes.pair_terms(f, ["a", "b"]), {"value": 0, "measured": 0, "no_size": 0, "pairs": 0}) + + def test_a_pair_with_no_size_is_built_and_its_build_has_none(self): + """A build of a and b on the other side of zero from their sum: no tolerance makes the + two agree. Said of a build 1.7e-3 from its sum (1.0030 and 1.0013), it is refused: + that is its size.""" + f, *_ = made(interactions="unchecked", built_ab=-0.5) + f["interactions"] = checked_pairs(pair("a", "b", fixes.SAME_ENTRY, fixes.NO_SIZE)) + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.pair_terms(f, ["a", "b", "c"]), {"value": None, "measured": 1, "no_size": 1, "pairs": 1}) + self.assertEqual(fixes.label(f, ["a", "b"]), fixes.BUILT) + self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.ESTIMATED) + self.assertEqual(named(f, fixes.compose(f, ["a", "b", "c"])["results"][A])["p1"], -0.5, "taken from the build") + agrees = changed(f, lambda g: g["built"][0]["results"][A][0].__setitem__(1, 1.0030)) + self.assertTrue(said(fixes.errors(agrees), "built set 'ab': the file says its pair has no size, and its build " + "lies within 0.00169491525")) # fmt: skip + unbuilt = changed(f, lambda g: g["built"].clear()) + self.assertTrue(said(fixes.errors(unbuilt), "the pair 'a' and 'b': has no size, so no set holding it can be drawn")) + other = changed(f, lambda g: g["interactions"]["pairs"][0].__setitem__("largest", "large")) + self.assertTrue(said(fixes.errors(other), "`largest` is not a share, 'no size' where no tolerance is enough")) + + def test_the_kinds_are_listed_in_their_order_of_precedence(self): + self.assertEqual(fixes.PAIR_KINDS, ("same entry", "technosphere", "technosphere and emission", + "technosphere and factor", "emission and factor")) # fmt: skip def with_requires(*, built_ab=True): """Fixes that need others applied first, against the base of `chain_and_matrix`: @@ -721,6 +752,20 @@ def test_an_added_item_has_no_base_result_and_a_result_from_some_fix(self): interactions=checked_pairs(), added_items=[["p9", "A new thing", None, None, "kg"]]) # fmt: skip self.assertIn("item 'p9' is in no state of the chain", str(caught.exception)) + def test_an_added_item_in_a_new_unit_gets_a_floor_from_nothing(self): + """No item of the chain is in litres: the chain would give the unit a floor of 0, from no + result, so the fixes file does.""" + data, full = chain_and_matrix() + f = fixes.build("An item in litres", data=data, full=full, base="s1", fixes=[fixes.fix("n", "N", kind="mapping")], + results={"n": {A: {"p9": 0.7}, B: {"p9": 1.2}}}, interactions=checked_pairs(), + added_items=[["p9", "A new thing", None, None, "l"]]) # fmt: skip + self.assertEqual(f["rule"]["floors"][-2:], [{"indicator": i, "unit": "l", "value": 0.0, "basis": "none", "n": 0} + for i in (A, B)]) # fmt: skip + self.assertEqual((fixes.errors(f), fixes.against(f, data, full), fixes.against(f, data, None)), ([], [], [])) + raised = changed(f, lambda g: g["rule"]["floors"][-1].__setitem__("value", 0.5)) + self.assertEqual(fixes.errors(raised), []) + self.assertTrue(said(fixes.against(raised, data, full), "its floors differ from the diagnostics file's")) + def test_added_items_come_after_the_matrix_items_and_are_new(self): f, data, full = with_added() first = changed(f, lambda g: g["items"].insert(0, g["items"].pop())) @@ -763,6 +808,138 @@ def test_the_state_comes_after_the_base(self): moved = changed(f, lambda g, s=state: g["built"][-1].__setitem__("state", s)) self.assertTrue(said(fixes.against(moved, data, full), "is not a state of the chain after the base")) +def measured(value, ind=A): + return chain.noise(["one", "two"], {ind: value}) + + +class Noise(unittest.TestCase): + """Noise belongs to a build, as it belongs to a state of a chain (SPEC 5.3, 16.5): the + base's is its state's, a fix's that of the build with it, a built set's that of its + build. A comparison takes the sum of the noise of the builds it stands on.""" + + def test_the_base_takes_the_noise_of_its_state(self): + f, data, full = made(state_noise={"s1": measured(1e-4)}) + self.assertEqual(f["of"]["base_noise"], data["states"][1]["noise"]) + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + other = changed(f, lambda g: g["of"]["base_noise"]["by"].__setitem__(A, 2e-4)) + self.assertTrue(said(fixes.against(other, data, full), "`of.base_noise` is not the noise the diagnostics file " + "gives state 's1'")) # fmt: skip + gone = changed(f, lambda g: g["of"].pop("base_noise")) + self.assertEqual(fixes.errors(gone), [], "on its own the file cannot tell a published base") + self.assertTrue(said(fixes.against(gone, data, full), "the base is a build: `of.base_noise` is its noise")) + bad = changed(f, lambda g: g["of"]["base_noise"].__setitem__("between", None)) + self.assertTrue(said(fixes.errors(bad), "`of.base_noise`: measured noise names the two builds")) + + def test_a_published_base_is_exact_and_takes_no_noise(self): + data, full = chain_and_matrix() + f = fixes.build("From the published results", data=data, full=full, base="s0", + fixes=[fixes.fix("a", "Fix A", kind="mapping", noise=measured(1e-3))], + results={"a": {A: {"p1": 1.5}}}) # fmt: skip + self.assertNotIn("base_noise", f["of"]) + self.assertEqual(fixes.noise_of(f, ["a"])[A], {"value": 1e-3, "known": 2, "builds": 2}) + self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 1, "builds": 2}) + noisy = changed(f, lambda g: g["of"].__setitem__("base_noise", chain.noise())) + self.assertTrue(said(fixes.against(noisy, data, full), "the base is the published results, which are exact")) + + def test_a_fix_is_judged_with_the_noise_of_its_build_and_of_the_base(self): + """1.0 to 1.0015 is 1.5e-3 apart: beyond the tolerance (1e-3) and the fix's own noise + (1e-3), within the sum of its and the base's (2e-3).""" + + def judged(base_noise): + data, full = chain_and_matrix(noise={"s1": base_noise}) + f = fixes.build("One fix", data=data, full=full, base="s1", + fixes=[fixes.fix("a", "Fix A", kind="mapping", tolerance=chain.tolerance(1e-3), + noise=measured(1e-3))], + results={"a": {A: {"p1": 1.0015}}}) # fmt: skip + self.assertEqual(f["fixes"][0]["counts"][A]["classes"], dense(f, "a")[A]["classes"]) + return f["fixes"][0]["counts"][A]["classes"][rule.WITHIN_NOISE] + + self.assertEqual(judged(measured(1e-3)), 1) + self.assertEqual(judged(None), 0) + + def test_a_fix_measured_against_its_requirements_takes_their_builds_noise(self): + """d is 1.5e-3 from the build of a: within the noise of a's build and d's, beyond d's + and the base's (not measured).""" + data, full = chain_and_matrix() + f = fixes.build("A fix that needs another", data=data, full=full, base="s1", + fixes=[fixes.fix("a", "Fix A", kind="mapping", noise=measured(1e-3)), + fixes.fix("d", "Fix D", kind="amount", requires=["a"], tolerance=chain.tolerance(1e-3), + noise=measured(1e-3))], + results={"a": {A: {"p1": 1.5}}, "d": {A: {"p1": 1.5 * 1.0015}}}) # fmt: skip + counts = f["fixes"][1]["counts"][A]["classes"] + self.assertEqual(counts[rule.WITHIN_NOISE], 1) + again = dense(f, "d", {A: {at(f, "p1"): 1.5}}, under_noise=measured(1e-3)) + self.assertEqual(counts, again[A]["classes"]) + + def test_a_set_counts_the_base_once_for_each_fix_measured_against_it(self): + """a and b each moved p1 from the base: a rebuild of the base moves both differences.""" + f, *_ = made(noise_a=measured(1e-3), noise_b=measured(1e-3), interactions="none", built=False, + state_noise={"s1": measured(1e-4)}) # fmt: skip + self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 1e-4 + 1e-3 + 1e-4 + 1e-3, "known": 3, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-4 + 1e-3 + 1e-4, "known": 2, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": 1e-4, "known": 1, "builds": 2}, "one fix: its build") + + def test_a_built_set_is_compared_as_its_own_build(self): + f, *_ = made(noise_ab=measured(5e-3), interactions="none", state_noise={"s1": measured(1e-4)}) + self.assertEqual(fixes.label(f, ["a", "b"]), fixes.BUILT) + self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 1e-4 + 5e-3, "known": 2, "builds": 2}) + + def test_a_build_a_result_is_taken_from_counts_where_its_noise_is_larger(self): + """In A the set of a, b and c takes p1 from the build of a and b; in B no build is used.""" + f, *_ = made(noise_ab=measured(5e-2), interactions="none") + self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.EXACT) + noise = fixes.noise_of(f, ["a", "b", "c"]) + self.assertEqual(noise[A], {"value": 5e-2, "known": 1, "builds": 5}) + self.assertEqual(noise[B], {"value": None, "known": 0, "builds": 4}) + self.assertEqual(fixes.compose(f, ["a", "b", "c"])["noise"], noise) + self.assertEqual(fixes.counts(f, ["a", "b", "c"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) + + def test_a_built_set_is_held_to_its_sum_with_the_noise_of_every_build_both_stand_on(self): + """1.00406 is 2.75e-3 from the sum, 1.0013: within the noise of the base three times + (once, and once under each fix), a, b and the built set (6 x 5e-4); beyond it with any + one of them left out (5 x 5e-4), and without the base (3 x 5e-4).""" + kw = {"noise_a": measured(5e-4), "noise_b": measured(5e-4), "noise_ab": measured(5e-4), "interactions": "none", + "built_ab": 1.00406} # fmt: skip + f, *_ = made(state_noise={"s1": measured(5e-4)}, **kw) + self.assertEqual(fixes.errors(f), []) + gone = changed(f, lambda g: g["of"].pop("base_noise")) + self.assertTrue(said(fixes.errors(gone), "built set 'ab', item 'p1', x:a: built 1.00406")) + own = changed(f, lambda g: g["built"][0].__setitem__("noise", chain.noise())) + self.assertTrue(said(fixes.errors(own), "built set 'ab', item 'p1', x:a: built 1.00406")) + + def test_a_release_is_held_to_its_state_with_the_noise_of_both(self): + """1.0028 is 1.5e-3 from the state's 1.0013: within the two builds' noise (2e-3), + beyond the built set's alone.""" + + def nudge(g): + entries = g["built"][-1]["results"][A] + entries[[k for k, _ in entries].index(at(g, "p1"))][1] = 1.0028 + + f, data, full = made(release=True, state_noise={"s2": measured(1e-3)}, noise_all=measured(1e-3)) + self.assertEqual(fixes.against(changed(f, nudge), data, full), []) + f, data, full = made(release=True, noise_all=measured(1e-3)) + self.assertTrue(said(fixes.against(changed(f, nudge), data, full), "built set 'all', item 'p1', x:a: built 1.0028")) + + def test_a_diagnostics_file_whose_states_carry_no_noise_is_refused(self): + f, data, full = made() + old = changed(data, lambda d: [s.pop("noise", None) for s in d["states"]]) + with self.assertRaises(ValueError) as caught: + fixes.build("x", data=old, full=full, base="s1", fixes=[fixes.fix("a", "A", kind="mapping")], results={}) + self.assertIn(fixes.NO_STATE_NOISE, str(caught.exception)) + self.assertEqual(fixes.against(f, old, full), [fixes.NO_STATE_NOISE]) + + +class Matrix(unittest.TestCase): + def test_a_result_written_as_the_one_before_is_read_written_out(self): + f, data, full = made() + row = full["values"][[r[0] for r in full["items"]].index("p1")] + self.assertIn(matrix.SAME, row) + self.assertEqual(f["base"][A][at(f, "p1")], 1.0) + self.assertEqual(fixes.against(f, data, full), []) + written = changed(full, lambda m: m["values"][0].__setitem__(1, matrix.SAME)) # p3 in A: 3.0 is not 4.0 + self.assertTrue(said(fixes.against(f, data, written), "the base results in 'x:a' are not the matrix's")) + class Command(unittest.TestCase): def run_cli(self, *args) -> tuple[int, str]: @@ -772,8 +949,6 @@ def run_cli(self, *args) -> tuple[int, str]: return code, out.getvalue() def test_check_holds_a_fixes_file_to_the_file_beside_it(self): - from sentier_diagnostics import matrix - f, data, full = made() with tempfile.TemporaryDirectory() as folder: folder = Path(folder) From 8c7f2c3f2aa0fd1d64621b24b4d5a571ae577318 Mon Sep 17 00:00:00 2001 From: Manuel Klarmann Date: Tue, 17 Nov 2026 07:46:48 +0100 Subject: [PATCH 5/6] A set's noise item by item: only the builds the item's result stands on sentier-bafu's review as producer (finding 1): a set's noise summed over every ticked fix put an item that moved 2.9 % within noise as soon as an unrelated fix was ticked (A30 again). Each item now takes the noise of the builds its result stands on: a build's item that build and the base; a summed item the fixes that change it, each with what it was measured against, summed in the order of the file so the page's port matches bit for bit. The built-set check takes the same per item. compose gives the noise of each item (item_noise) as the reference for the port. Co-Authored-By: Claude Opus 5.5 --- GLOSSARY.md | 2 +- SPEC.md | 72 ++++++------ src/sentier_diagnostics/fixes.py | 185 ++++++++++++++++++------------- tests/test_fixes.py | 98 ++++++++++++++-- 4 files changed, 239 insertions(+), 118 deletions(-) diff --git a/GLOSSARY.md b/GLOSSARY.md index c89d7f4..21a52a6 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -33,7 +33,7 @@ The words this repository uses, each in one line. [SPEC.md](SPEC.md) says the re - **Drawer**: the panel of one item, at the right of the page, opened from a row. - **Base**: the one state of a chain that every fix is measured against (SPEC section 16). - **Fix**: one change applied to the base alone and built, with a step's kind, claim and tolerance, and the noise of its build. -- **Build** (of fixes): a set of results the producer made, the base, a fix with what it requires, or a built set; noise belongs to it, as to a state. A fix is judged with the noise of its build and of the one it was measured against; a set with the sum, over its fixes, of both. +- **Build** (of fixes): a set of results the producer made, the base, a fix with what it requires, or a built set; noise belongs to it, as to a state. A fix is judged with the noise of its build and of the one it was measured against; a set, item by item, with that of the builds the item's result stands on. - **Requires**: the fixes a fix cannot be applied without. Its effect is measured with them applied, and choosing it chooses them. - **Effect**: what a fix does, per item and indicator: its result with the fix minus its result without it, both with what it requires. - **Set**: the fixes a reader chooses together, with what they require; its result is the base plus the sum of their effects. It has no verdict. diff --git a/SPEC.md b/SPEC.md index f1ebcf7..94c5bb6 100644 --- a/SPEC.md +++ b/SPEC.md @@ -971,10 +971,12 @@ base result bit for bit. The base is held to the chain bit for bit and every ent differs is kept, so an effect computed another way fills up with the route's own differences, each read as a change: sentier-bafu built one release twice, a pinned copy against today's importer with no change meant, and 294,590 of 298,675 results moved, 722 of -them by 1e-5 or more. **Results are read exactly**, as they were written: a reader that -rounds the last bit drops changes and adds others (a default CSV reader misread 134,743 of -the 298,675 results of one build in the last bit, and lost 7 of the 86 changes of one patch; -read round-trip, every result matches its text). +them by 1e-5 or more. **Results are read exactly**, as they were written, and both sides the +same way. A reader that rounds the last bit merges distinct results and drops changes: a +default CSV reader misread 134,743 of the 298,675 results of one build in the last bit, and +lost 7 of the 86 changes of one patch; read round-trip, every result matches its text. Where +the two sides are read differently (effects read exactly, against a base held to a chain whose +results went through a rounding reader), every misread result reads as a change. A fix's `counts` is one block per indicator (section 4.2), its pairs being the result it was measured against and the result with the fix, judged with the fix's own tolerance and the @@ -1024,27 +1026,32 @@ it), the set has no result there, and the page says the sum cannot be formed, ap result that is missing. It is judged by the rule (section 5) against the base: - its **tolerance** is the largest of its fixes'; -- its **noise** follows section 5.3, per indicator: noise belongs to a build, and a - comparison takes the sum of the noise of the builds it stands on. The base's is its +- its **noise** follows section 5.3: noise belongs to a build, and a comparison takes the + sum of the noise of the builds it stands on, never of one it does not. The base's is its state's (`of.base_noise`; the published results are exact and add nothing), a fix's that - of the build with it and what it requires, a built set's that of its build. - - A set that is a build (one fix with what it requires, or a built set) takes the sum of - its noise and the base's. - - Any other set's difference from the base is a sum of its fixes' differences, each - against what that fix was measured against. Its noise is the sum, over its fixes, of the - noise of each fix's build and of the build it was measured against: the base once for - each fix measured against it, since a rebuild of the base moves every one of those - differences the same way. Where it takes a result from a build inside it, that build's - noise and the base's bound the result there, and the set's noise is the larger of the - two. The sum runs in the order of the file's fixes, each fix's measured-against build - before its own, so that the page's port gives the same number bit for bit. - - A build whose noise was not measured adds nothing, and the page says at how many of the - builds the noise is known ("noise known at 3 of 5 builds"; the published results are - known, and exact), never reading not measured as none; where no build measured it the page - says "not measured", never "identical". A noise names its two builds (`between`): two - builds in one process on one solve route measure the engine's determinism and read 0, two - routes measure how a result moves with the route; the note says which; + of the build with it and what it requires, a built set's that of its build. A set is + judged **item by item**, with the builds that item's result stands on: + - where its result is a build's (one fix changes it, or a build inside the set covers + it, or the set is a build), that build's noise and the base's; + - where it is a sum, the sum over the fixes that change the item of the noise of each + fix's build and of the build it was measured against: the base once for each of them + measured against it, since a rebuild of the base moves each of those differences the + same way. The base the sum starts from is the base it is compared with, and cancels. + + A fix that does not change an item adds nothing to it. This is the fixes file's form of + "a state inside a run adds nothing" (5.3): summed over every fix ticked, the noise of + three fixes on three items, each built at 1 %, was 6 %, and ticking an unrelated fix put + an item that moved 2.9 % "within noise" (sentier-bafu's probe; A30 again). The sum runs in + the order of the file's fixes, each fix's measured-against build before its own, so that + the page's port gives the same number bit for bit. + + Per indicator the page says the largest noise an item is judged with, and at how many of + the builds those items stand on the noise is known ("noise known at 3 of 5 builds"; the + published results are known, and exact), never reading not measured as none; where no + build measured it the page says "not measured", never "identical". A noise names its two + builds (`between`): two builds in one process on one solve route measure the engine's + determinism and read 0, two routes measure how a result moves with the route; the note + says which; - it has **no verdict**: each claim is judged in its own fix. Each set carries one of three labels, and the page shows it wherever the set is drawn: @@ -1130,10 +1137,12 @@ is, where it is one, or null. fixes are a listed pair, the built results must agree with the base plus the sum of the effects, by the rule with the set's tolerance and noise, for every item. That sum stands on the fixes alone (16.5 without its built sets): a set held to a build inside it, itself held -to the sum, would stack two tolerances. The noise here is that of every build the two sides -stand on (16.5): in the sum, the base and each fix's build and the one it was measured -against (the base once more for each fix measured against it); on the other side, the built -set's own. A file where they do not agree is refused, naming the set, the item and the +to the sum, would stack two tolerances. The noise is taken item by item, as in 16.5: the +built set's own, and that of the builds the sum stands on at that item (the build it takes +the result from; or the base, with each fix that changes the item, its build and the one it +was measured against; or the base alone where no fix changes it). A fix that does not change +the item adds nothing: otherwise a large built set would pass a real disagreement as within +noise. A file where they do not agree is refused, naming the set, the item and the indicator: its producer called two fixes independent that are not. Where the set holds a listed pair, the page shows how far the built results lie from the sum, as that set's interaction. Where the set is exactly one listed pair with a measured `largest`, the built @@ -1225,7 +1234,8 @@ to about 300,000 (6 to 7 MB). 64 MB holds about ten dense fixes and thousands of ones; effects move to one file per fix only if dense fixes become common. Checked on a made-up file of that base with 100 fixes of 500 items each in every indicator, ten of them requiring the fix before them, every build's noise measured (39.5 MB), the check takes -about 2.2 seconds, and counting a set of 50 of them under one. +about 2.4 seconds, and counting a set of 50 of them, each item with its own noise, about +one. ### 16.11 Acceptance cases @@ -1238,13 +1248,13 @@ Each is a test before the code is written. | F3 | Interactions not checked: every set of two fixes or more that was not built is estimated, none exact | | F4 | A built set that disagrees with the sum where the file calls its fixes independent is refused, naming the set, item and indicator | | F5 | A fix's verdicts are the same whichever fixes are chosen with it | -| F6 | The noise of a set is the sum, over its fixes, of the noise of each fix's build and of the one it was measured against (the base once for each fix), or that of a build it takes a result from and the base's where larger; the builds whose noise is known are counted, and not measured is never identical | +| F6 | A set is judged item by item with the noise of the builds the item's result stands on: a build's and the base's, or for a sum, each summed fix's build and the one it was measured against (the base once for each fix). A fix that does not change the item adds nothing; the builds whose noise is known are counted, and not measured is never identical | | F7 | The page's rule agrees with the Python rule on every shared case and generated boundary value, and the page refuses to compose where it cannot reproduce a fix's own classes | | F8 | A file of fixes holds nothing of the model in a pair: its kind comes from a closed list, and any text in it is refused | | F9 | A fix that requires another is measured against their build; choosing it chooses what it requires; in a set, each effect adds on top of what it was measured with, so what two fixes build on counts once; where a build inside the set holds every fix that changes an item, its result is taken, measured, not summed | | F10 | A built set that names a state of the chain is refused where its results do not reproduce that state | | F11 | A fix may bring an item no state holds: it follows the matrix's items, with no base result, and a fix gives it one; in a unit no item of the chain has, it takes a floor of 0 from nothing | -| F12 | A fix is judged with the noise of its build and of the build it was measured against; a built set is held to its sum with the noise of every build both stand on, and to its state with its noise and the state's; the base's noise is its state's, and the published results are exact | +| F12 | A fix is judged with the noise of its build and of the build it was measured against; a built set is held to its sum, item by item, with the noise of the builds both sides stand on there, and to its state with its noise and the state's; the base's noise is its state's, and the published results are exact | | F13 | A pair with no size is built, and its build has none; "no size" and "not measured" read apart | ### 16.12 Who does what, and in which order diff --git a/src/sentier_diagnostics/fixes.py b/src/sentier_diagnostics/fixes.py index 0f121db..031d5c6 100644 --- a/src/sentier_diagnostics/fixes.py +++ b/src/sentier_diagnostics/fixes.py @@ -20,11 +20,11 @@ of its fixes'. Noise belongs to a build, as it belongs to a state of a chain (SPEC 5.3): the base's is that state's (none where the base is the published results, which are exact), a fix's is that of the build with it, a built set's that of its build. A fix is judged with the -sum of the noise of its build and of the one it was measured against. A set's result is a -build's, or a sum over its fixes, so its noise is the sum, over its fixes, of the noise of each -fix's build and of the one it was measured against; or that of a build it takes a result from, -and of the base, where that is larger. It says at how many of those builds the noise is known, -never reading not measured as none. +sum of the noise of its build and of the one it was measured against. A set is judged item by +item with the noise of the builds the item's result stands on, never with that of a fix that +does not touch it: a build's result, with that build's and the base's; a sum, with each +summed fix's build and the one it was measured against. It says at how many of those builds +the noise is known, never reading not measured as none. A listed pair holds no text: its kind of interaction (one of `PAIR_KINDS`, from where the two changes meet in h = c·B·A⁻¹·f), how many items it can reach and the measured size of @@ -91,6 +91,8 @@ MAX_ITEMS, MAX_BYTES = matrix.MAX_ITEMS, matrix.MAX_BYTES BUILT, EXACT, ESTIMATED = "built", "exact", "estimated" +#: What a set's result at an item stands on (`_Builds.composed`): one build, or a sum of fixes. +BUILD, SUM = "build", "sum" LABELS = (BUILT, EXACT, ESTIMATED) #: The shape of an identifier of a model (a UUID, or 32 hexadecimal digits): refused in the @@ -514,16 +516,17 @@ def covers(self, fixes: frozenset, *, built: bool = True) -> list[frozenset]: return found def composed(self, fixes: frozenset, ind: str, *, built: bool = True, - used: set | None = None) -> dict[int, tuple[float | None, bool]]: # fmt: skip + how: dict | None = None) -> dict[int, tuple[float | None, bool]]: # fmt: skip """A closed set's result in one indicator, at every item one of its fixes changes: - `{item index: (result, whether it could be formed)}`. Where the fixes that change an - item all lie in one build inside the set (`covers`), the first such build's result is - the set's: measured, not summed. Otherwise the base plus each fix's difference against - what it was measured against. `built=False` leaves the built sets out: the sum a built - set is held to stands on the fixes alone. `used` gathers the builds a result was taken - from.""" + `{item index: (result, whether it could be formed)}`. Where one fix changes an item, + its build's result is the set's; where several do and all lie in one build inside the + set (`covers`), the first such build's: measured, not summed. Otherwise the base plus + each fix's difference against what it was measured against. `built=False` leaves the + built sets out: the sum a built set is held to stands on the fixes alone. `how` gathers + what each result stands on, `{item index: ("build", its fixes) or ("sum", the fixes + summed)}`, for its noise (`_item_ends`).""" touching: dict[int, list[str]] = {} - for y in fixes: + for y in (y for y in self.closures if y in fixes): # in the order of the file: one sum, one float for k in self.entries[y][ind]: touching.setdefault(k, []).append(y) covers = self.covers(fixes, built=built) @@ -533,16 +536,19 @@ def composed(self, fixes: frozenset, ind: str, *, built: bool = True, for k, ys in touching.items(): if len(ys) == 1: out[k] = (mine[ys[0]][k], True) - continue - here = set(ys) - cover = next((c for c in covers if here <= c), None) - if cover is not None: - if used is not None: - used.add(cover) - column = self.column(cover)[ind] - out[k] = (column[k] if k in column else base[k], True) + stands = (BUILD, self.closures[ys[0]] | {ys[0]}) else: - out[k] = _sum(base[k], [(mine[y][k], under[y][k] if k in under[y] else base[k]) for y in ys]) + here = set(ys) + cover = next((c for c in covers if here <= c), None) + if cover is not None: + column = self.column(cover)[ind] + out[k] = (column[k] if k in column else base[k], True) + stands = (BUILD, cover) + else: + out[k] = _sum(base[k], [(mine[y][k], under[y][k] if k in under[y] else base[k]) for y in ys]) + stands = (SUM, tuple(ys)) + if how is not None: + how[k] = stands return out @@ -610,10 +616,12 @@ def _side(before, after) -> str: def _block(base: _Base, ind: str, changed: Mapping[int, float | None], tolerance: float, - noise: float | None, claim: Mapping | None, under: Mapping[int, float | None] | None = None) -> dict: # fmt: skip + noise: float | Mapping[int, float | None] | None, claim: Mapping | None, + under: Mapping[int, float | None] | None = None) -> dict: # fmt: skip """The counts of one indicator over every item: the column `under` (the base, with the items `under` names laid over it) against that column with `changed` laid over it, judged - by the rule with `tolerance` and `noise`, and against `claim` where there is one. Only the + by the rule with `tolerance` and `noise` (one for every item, or one per changed item), and + against `claim` where there is one. Only the items the two columns name are judged one by one: every other pair is the base against itself, counted from the base's own classes.""" under = under or {} @@ -644,7 +652,8 @@ def kind_of(k): classes[was] -= 1 if was == rule.MISSING: missing["both"] -= 1 - kind, way, band = rule.judge(before, after, floor=floors[k], tolerance=tolerance, noise=noise) + level = noise.get(k) if isinstance(noise, Mapping) else noise + kind, way, band = rule.judge(before, after, floor=floors[k], tolerance=tolerance, noise=level) classes[kind] += 1 if kind == rule.MISSING: missing[_side(before, after)] += 1 @@ -776,62 +785,82 @@ def tolerance_of(f: Mapping, ids: Iterable[str]) -> float: def noise_of(f: Mapping, ids: Iterable[str]) -> dict[str, dict]: - """The noise of a set against the base, per indicator (SPEC 16.5): `{value, known, builds}`. - - A built set's is the sum of the noise of its build and of the base. Any other set's result - at an item is a build's, or the base plus each fix's difference against what it was - measured against: its value is the sum, over its fixes, of the noise of each fix's build and - of the one it was measured against (the base once for each fix measured against it), or, - where larger, that of a build a result is taken from and of the base. `builds` counts the - builds that enter it, `known` those whose noise is known there (the published results are - exact); None where none measured it: not measured is never none.""" + """The noise of a set against the base, per indicator (SPEC 16.5): `{value, known, + builds}`. Each item it changes is judged with the noise of the builds its result stands on + (`_against_base`), never with that of a fix that does not touch it; `value` is the largest + of those, `builds` counts the builds they stand on and `known` those whose noise is known + there (the published results are exact). None where none measured it: not measured is never + none. A set that changes nothing in an indicator stands on no build there.""" names = chosen(f, ids)[0] builds = _Builds.of(f) - return _set_noise(builds, names, _changed(builds, names)[2]) + return _set_noise(builds, _changed(builds, names)[2])[0] -def _set_noise(builds: _Builds, names: list[str], used: Mapping[str, set]) -> dict[str, dict]: - whole = builds.end(names) - base = ("", builds.base_end) - out = {} +def _sum_side(builds: _Builds, fixes: Iterable[str]) -> list[tuple]: + """The builds a sum over these fixes stands on, `(name, end)`: each fix's build and the one + it was measured against (the base, for a fix that requires nothing).""" + return [end for y in fixes for end in (builds.end(builds.closures[y]), (y, _built(builds.noise[y])))] + + +def _against_base(builds: _Builds, stands: tuple) -> list[tuple]: + """The builds a set's result at one item and the base result there stand on, `(name, end)`: + the build the result is taken from, and the base; for a sum, the builds of the sum alone, + since the base it starts from is the base it is compared with, and cancels.""" + kind, what = stands + if kind == BUILD: + return [("", builds.base_end), builds.end(what)] + return _sum_side(builds, what) + + +def _set_noise(builds: _Builds, how: Mapping[str, Mapping[int, tuple]]) -> tuple[dict, dict]: + """A set's noise per indicator, `{value, known, builds}`, and the noise each item it changes + is judged with, `{indicator: {item index: noise or None}}`.""" + summary, per_item = {}, {} for ind in builds.base.ids: - if whole is not None: - sums = [[base, whole]] - else: - terms = [end for y in names for end in (builds.end(builds.closures[y]), (y, _built(builds.noise[y])))] - sums = [terms, *([base, builds.end(cover)] for cover in sorted(used[ind], key=sorted))] - levels = [lv for lv in (_level([end for _, end in one], ind) for one in sums) if lv is not None] - every = {name: end for one in sums for name, end in one} - out[ind] = {"value": max(levels) if levels else None, - "known": sum(_known(end, ind) for end in every.values()), "builds": len(every)} # fmt: skip - return out + levels, every, seen = {}, {}, {} + for k, stands in how[ind].items(): + if stands not in seen: # most items share what they stand on: one fix's build + ends = _against_base(builds, stands) + seen[stands] = _level([end for _, end in ends], ind) + every.update(ends) + levels[k] = seen[stands] + found = [lv for lv in levels.values() if lv is not None] + summary[ind] = {"value": max(found) if found else None, + "known": sum(_known(end, ind) for end in every.values()), "builds": len(every)} # fmt: skip + per_item[ind] = levels + return summary, per_item def _changed(builds: _Builds, names: list[str]) -> tuple[dict, dict, dict]: """The items a closed set changes, per indicator, with its result at each; the items - where no sum can be formed; and the builds inside the set a result was taken from.""" + where no sum can be formed; and what each result stands on (`_Builds.composed`).""" column = builds.column(names) if column is not None: - return column, {ind: [] for ind in builds.base.ids}, {ind: set() for ind in builds.base.ids} - changed, cannot, used = {}, {}, {} + whole = frozenset(names) + return (column, {ind: [] for ind in builds.base.ids}, + {ind: dict.fromkeys(column[ind], (BUILD, whole)) for ind in builds.base.ids}) # fmt: skip + changed, cannot, how = {}, {}, {} for ind in builds.base.ids: - used[ind] = set() - cells = builds.composed(frozenset(names), ind, used=used[ind]) + how[ind] = {} + cells = builds.composed(frozenset(names), ind, how=how[ind]) changed[ind] = {k: v for k, (v, _) in cells.items()} cannot[ind] = sorted(k for k, (_, ok) in cells.items() if not ok) - return changed, cannot, used + return changed, cannot, how def compose(f: Mapping, ids: Iterable[str]) -> dict: """What a page draws for a set: its fixes in the order of the file (each with its own claim and counts), the fixes added because a chosen one requires them, its label, the - listed pairs it holds and their pair terms, its tolerance and noise, and its result for - every item, `{indicator: [result or None, ...]}` in the order of `items`. `cannot_add` + listed pairs it holds and their pair terms, its tolerance and noise (`noise_of`), the noise + each item it changes is judged with (`item_noise`, `{indicator: {item index: noise or + None}}`), and its result for every item, `{indicator: [result or None, ...]}` in the order + of `items`. `cannot_add` names, per indicator, the item indices whose result is None because no sum can be formed there, so that it reads apart from a missing result. A set has no verdict.""" names, added = chosen(f, ids) builds = _Builds.of(f) - changed, cannot, used = _changed(builds, names) + changed, cannot, how = _changed(builds, names) + noise, item_noise = _set_noise(builds, how) results = {} for ind in builds.base.ids: column = list(builds.base.values[ind]) @@ -845,7 +874,8 @@ def compose(f: Mapping, ids: Iterable[str]) -> dict: "pairs": held_pairs(f, names), "pair_terms": pair_terms(f, names), "tolerance": tolerance_of(f, names), - "noise": _set_noise(builds, names, used), + "noise": noise, + "item_noise": item_noise, "results": results, "cannot_add": cannot, } @@ -856,11 +886,11 @@ def counts(f: Mapping, ids: Iterable[str]) -> dict: estimated set's counts are estimates ("estimated: 153 would differ"). No verdicts.""" names = chosen(f, ids)[0] builds = _Builds.of(f) - changed, _, used = _changed(builds, names) - tol, noise = tolerance_of(f, names), _set_noise(builds, names, used) + changed, _, how = _changed(builds, names) + tol, item_noise = tolerance_of(f, names), _set_noise(builds, how)[1] return { "label": _label(f, builds, names), - "by": {ind: _block(builds.base, ind, changed[ind], tol, noise[ind]["value"], None) for ind in builds.base.ids}, + "by": {ind: _block(builds.base, ind, changed[ind], tol, item_noise[ind], None) for ind in builds.base.ids}, } @@ -1344,32 +1374,35 @@ def _number(value: float | None) -> str: return "null" if value is None else f"{value:.10g}" -def _sum_ends(builds: _Builds, fixes: Iterable[str]) -> list[Mapping]: - """The builds a sum over these fixes stands on, as ends: each fix's build and the one it was - measured against (the base, once for each fix measured against it).""" - return [end for y in sorted(fixes) for end in (builds.end(builds.closures[y])[1], _built(builds.noise[y]))] - - def _disagreements(builds: _Builds, where: str, key: frozenset, by_id: Mapping, one: Mapping, results: Mapping, largest: float | None = None) -> list[str]: # fmt: skip """Where a built set is further from the sum of its effects than the file says, by the - rule, with the largest tolerance of its fixes and the noise of every build the two sides - stand on: the base and each fix's build and what it was measured against, in the sum; the - built set, on the other side.""" + rule, with the largest tolerance of its fixes and, item by item, the noise of the builds + the two sides stand on: the built set's, and the build the sum takes the item from, or the + base with each fix's build and what it was measured against where it adds the fixes that + touch the item, or the base alone where none does.""" base = builds.base - ends = [builds.base_end, *_sum_ends(builds, key), _built(one["noise"])] + own, from_base = _built(one["noise"]), [builds.base_end] + tol = max(by_id[x]["tolerance"]["value"] for x in key) + if largest is not None: + tol = max(tol, largest) found, more = [], 0 for ind in base.ids: - tol, level = max(by_id[x]["tolerance"]["value"] for x in key), _level(ends, ind) - if largest is not None: - tol = max(tol, largest) - cells = builds.composed(key, ind, built=False) + how: dict[int, tuple] = {} + cells = builds.composed(key, ind, built=False, how=how) for k in sorted(set(cells) | set(results[ind])): before = base.values[ind][k] summed, ok = cells.get(k, (before, True)) if not ok: continue # no sum to hold the build to got = results[ind].get(k, before) + if k not in how: + side = from_base + elif how[k][0] == BUILD: + side = [builds.end(how[k][1])[1]] + else: + side = from_base + [end for _, end in _sum_side(builds, how[k][1])] + level = _level([own, *side], ind) kind = rule.classify(summed, got, floor=base.floor[ind][k], tolerance=tol, noise=level) if kind in claims.STILL or (kind == rule.MISSING and summed is None and got is None): continue diff --git a/tests/test_fixes.py b/tests/test_fixes.py index 3e4e81f..db61ca1 100644 --- a/tests/test_fixes.py +++ b/tests/test_fixes.py @@ -452,13 +452,13 @@ def test_f5_a_fix_is_judged_the_same_whichever_fixes_are_chosen_with_it(self): def test_f6_the_noise_of_a_set_is_the_sum_said_with_how_many_were_measured(self): """The base (s1) was not measured: it adds nothing, and is one build of three whose - noise is not known.""" + noise is not known. Where a set changes nothing, it stands on no build.""" noise = chain.noise(["one", "two"], {A: 1e-3}) f, *_ = made(noise_a=noise, noise_b=noise, interactions="none", built=False) self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 2e-3, "known": 2, "builds": 3}) self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-3, "known": 1, "builds": 3}) self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": None, "known": 0, "builds": 2}) - self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 0, "builds": 2}) + self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 0, "builds": 0}) # the sum, not the largest: 1.3e-3 apart is within 2e-3 and beyond 1e-3 self.assertEqual(fixes.counts(f, ["a", "b"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) @@ -838,7 +838,7 @@ def test_a_published_base_is_exact_and_takes_no_noise(self): results={"a": {A: {"p1": 1.5}}}) # fmt: skip self.assertNotIn("base_noise", f["of"]) self.assertEqual(fixes.noise_of(f, ["a"])[A], {"value": 1e-3, "known": 2, "builds": 2}) - self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 1, "builds": 2}) + self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 0, "builds": 0}, "a changes nothing in B") noisy = changed(f, lambda g: g["of"].__setitem__("base_noise", chain.noise())) self.assertTrue(said(fixes.against(noisy, data, full), "the base is the published results, which are exact")) @@ -872,26 +872,75 @@ def test_a_fix_measured_against_its_requirements_takes_their_builds_noise(self): again = dense(f, "d", {A: {at(f, "p1"): 1.5}}, under_noise=measured(1e-3)) self.assertEqual(counts, again[A]["classes"]) - def test_a_set_counts_the_base_once_for_each_fix_measured_against_it(self): - """a and b each moved p1 from the base: a rebuild of the base moves both differences.""" + def test_a_sum_counts_the_base_once_for_each_fix_measured_against_it(self): + """a and b each moved p1 from the base: a rebuild of the base moves both differences. + With a and c, no item is a sum: p1 is a's build and p3 c's, each against the base.""" f, *_ = made(noise_a=measured(1e-3), noise_b=measured(1e-3), interactions="none", built=False, state_noise={"s1": measured(1e-4)}) # fmt: skip self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 1e-4 + 1e-3 + 1e-4 + 1e-3, "known": 3, "builds": 3}) - self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-4 + 1e-3 + 1e-4, "known": 2, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-4 + 1e-3, "known": 2, "builds": 3}) self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": 1e-4, "known": 1, "builds": 2}, "one fix: its build") + drawn = fixes.compose(f, ["a", "c"]) + self.assertEqual(named(f, [drawn["item_noise"][A].get(k) for k in range(len(f["items"]))]), + {"p1": 1e-4 + 1e-3, "p2": None, "p3": 1e-4, "p4": None}) # fmt: skip + + def test_an_item_takes_the_noise_of_the_builds_its_result_stands_on_never_another_fixs(self): + """sentier-bafu's probe: three fixes on three items, each build at 1 %, the base at + 0.5 %; c moves p3 by 2.9 %. At p3 the set's result is c's build, so its noise is the + base's and c's (1.5 %) whichever fixes are ticked with c: p3 differs in every set. A + noise summed over every fix (4.5 % for three) would put it within noise as soon as a + is ticked.""" + f = disjoint() + for names in (["c"], ["a", "c"], ["a", "b", "c"]): + with self.subTest(names=names): + self.assertEqual(fixes.compose(f, names)["item_noise"][A][at(f, "p3")], 0.005 + 0.01) + classes = fixes.counts(f, names)["by"][A]["classes"] + self.assertEqual((classes[rule.DIFFERS], classes[rule.WITHIN_NOISE]), (len(names), 0)) + self.assertEqual(fixes.noise_of(f, ["a", "b", "c"])[A], {"value": 0.005 + 0.01, "known": 4, "builds": 4}) + + def test_a_set_counts_each_item_with_its_own_noise(self): + """a moves p1 by 2.9 % with its build at 1 %; c moves p3 by 10 % with its build at 4 %. + In the set of both, p1 is judged with 1.5 % and differs; the set's largest noise (4.5 %) + would put it within noise.""" + data, full = chain_and_matrix(noise={"s1": measured(0.005)}) + f = fixes.build("Two noises", data=data, full=full, base="s1", + fixes=[fixes.fix("a", "Fix a", kind="amount", noise=measured(0.01)), + fixes.fix("c", "Fix c", kind="amount", noise=measured(0.04))], + results={"a": {A: {"p1": 1.029}}, "c": {A: {"p3": 3.3}}}, + interactions=checked_pairs()) # fmt: skip + self.assertEqual(fixes.errors(f), []) + noise = fixes.compose(f, ["a", "c"])["item_noise"][A] + self.assertEqual((noise[at(f, "p1")], noise[at(f, "p3")]), (0.005 + 0.01, 0.005 + 0.04)) + classes = fixes.counts(f, ["a", "c"])["by"][A]["classes"] + self.assertEqual((classes[rule.DIFFERS], classes[rule.WITHIN_NOISE]), (2, 0)) def test_a_built_set_is_compared_as_its_own_build(self): f, *_ = made(noise_ab=measured(5e-3), interactions="none", state_noise={"s1": measured(1e-4)}) self.assertEqual(fixes.label(f, ["a", "b"]), fixes.BUILT) self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 1e-4 + 5e-3, "known": 2, "builds": 2}) - def test_a_build_a_result_is_taken_from_counts_where_its_noise_is_larger(self): - """In A the set of a, b and c takes p1 from the build of a and b; in B no build is used.""" + def test_a_sum_runs_in_the_order_of_the_file(self): + """Three fixes change p1, their builds' noise 0.1, 0.2 and 0.3: summed in the order of + the file it is 0.6000000000000001, in the other order 0.6. The page's port sums in the + same order, so the two agree bit for bit.""" + data, full = chain_and_matrix() + f = fixes.build("Three on one item", data=data, full=full, base="s1", + fixes=[fixes.fix(x, f"Fix {x}", kind="amount", noise=measured(n)) + for x, n in (("x", 0.1), ("y", 0.2), ("z", 0.3))], + results={"x": {A: {"p1": 1.1}}, "y": {A: {"p1": 1.2}}, "z": {A: {"p1": 1.3}}}, + interactions=checked_pairs()) # fmt: skip + drawn = fixes.compose(f, ["z", "y", "x"]) + self.assertEqual(drawn["item_noise"][A][at(f, "p1")], 0.1 + 0.2 + 0.3) + self.assertNotEqual(0.1 + 0.2 + 0.3, 0.3 + 0.2 + 0.1) + + def test_a_result_taken_from_a_build_inside_the_set_takes_that_builds_noise(self): + """The set of a, b and c takes p1 in A from the build of a and b: p1 stands on that + build and the base, not on a's and b's own builds; p3 on c's and the base.""" f, *_ = made(noise_ab=measured(5e-2), interactions="none") self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.EXACT) noise = fixes.noise_of(f, ["a", "b", "c"]) - self.assertEqual(noise[A], {"value": 5e-2, "known": 1, "builds": 5}) - self.assertEqual(noise[B], {"value": None, "known": 0, "builds": 4}) + self.assertEqual(noise[A], {"value": 5e-2, "known": 1, "builds": 3}) + self.assertEqual(noise[B], {"value": None, "known": 0, "builds": 2}) self.assertEqual(fixes.compose(f, ["a", "b", "c"])["noise"], noise) self.assertEqual(fixes.counts(f, ["a", "b", "c"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) @@ -908,6 +957,23 @@ def test_a_built_set_is_held_to_its_sum_with_the_noise_of_every_build_both_stand own = changed(f, lambda g: g["built"][0].__setitem__("noise", chain.noise())) self.assertTrue(said(fixes.errors(own), "built set 'ab', item 'p1', x:a: built 1.00406")) + def test_a_built_set_is_held_at_each_item_to_the_builds_its_sum_stands_on_there(self): + """The three fixes built together, p3 1.8 % from c's build: within the noise of that + build and the built set's (2 %), not of the base's and the built set's (1.5 %). At + 2.4 % it is refused, where every fix's noise and the base's (6 %) would let it pass.""" + f = disjoint() + self.assertEqual(fixes.errors(f), []) + p1, p2, p3 = (at(f, p) for p in ("p1", "p2", "p3")) + + def add(value): + return lambda g: g["built"].append({"id": "abc", "label": "All three", "fixes": ["a", "b", "c"], "state": None, + "results": {A: sorted([[p1, 1.5], [p2, 2.5], [p3, value]]), B: []}, + "noise": measured(0.01)}) # fmt: skip + + self.assertEqual(fixes.errors(changed(f, add(3.09 * 1.018))), [], "1.8 % from c's build: within 2 %") + found = fixes.errors(changed(f, add(3.09 * 1.025))) + self.assertTrue(said(found, "built set 'abc', item 'p3', x:a"), found) + def test_a_release_is_held_to_its_state_with_the_noise_of_both(self): """1.0028 is 1.5e-3 from the state's 1.0013: within the two builds' noise (2e-3), beyond the built set's alone.""" @@ -930,6 +996,18 @@ def test_a_diagnostics_file_whose_states_carry_no_noise_is_refused(self): self.assertEqual(fixes.against(f, old, full), [fixes.NO_STATE_NOISE]) +def disjoint(): + """Three fixes on three items in A, each built twice and measured at 1 %, the base at + 0.5 %: a moves p1 to 1.5, b p2 to 2.5, c p3 from 3.0 to 3.09 (2.9 %). No pair is listed.""" + data, full = chain_and_matrix(noise={"s1": measured(0.005)}) + return fixes.build( + "Three fixes apart", data=data, full=full, base="s1", + fixes=[fixes.fix(x, f"Fix {x}", kind="amount", noise=measured(0.01)) for x in ("a", "b", "c")], + results={"a": {A: {"p1": 1.5}}, "b": {A: {"p2": 2.5}}, "c": {A: {"p3": 3.09}}}, + interactions=checked_pairs(), + ) # fmt: skip + + class Matrix(unittest.TestCase): def test_a_result_written_as_the_one_before_is_read_written_out(self): f, data, full = made() From 30bbd5beb4a595f6b839df0b12717fbde788aa50 Mon Sep 17 00:00:00 2001 From: Manuel Klarmann Date: Tue, 17 Nov 2026 08:01:28 +0100 Subject: [PATCH 6/6] Tests that tell sum orders apart, and an untouched item in a built set sentier-bafu's review of 8c7f2c3 (nothing blocking) found three mutations the tests let through: - a sum in id order: the three fixes of the order test were already in id order. Six fixes now, out of id order, with noise whose sum in file order only one other order of the 720 gives (its first two swapped); the test asserts that it still tells the orders apart. - a sum in the set's own (hash) order: caught on some seeds only; now on 30 of 30. - the built-set check without the base at an item none of the set's fixes change: a built set of a and b moving p3 by 1.2 % passes with the base's noise and its own (1.5 %), not with its own alone. SPEC 16.5: a set that changes nothing in an indicator stands on no build there; the page says nothing changed, never "not measured". Co-Authored-By: Claude Opus 5.5 --- SPEC.md | 5 ++++- tests/test_fixes.py | 49 ++++++++++++++++++++++++++++++++++++--------- 2 files changed, 43 insertions(+), 11 deletions(-) diff --git a/SPEC.md b/SPEC.md index 94c5bb6..25eae1b 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1048,7 +1048,10 @@ result that is missing. It is judged by the rule (section 5) against the base: Per indicator the page says the largest noise an item is judged with, and at how many of the builds those items stand on the noise is known ("noise known at 3 of 5 builds"; the published results are known, and exact), never reading not measured as none; where no - build measured it the page says "not measured", never "identical". A noise names its two + build measured it the page says "not measured", never "identical". Where the set changes + nothing in an indicator, no item is judged and the noise stands on no build (`builds` 0, + `value` null): the page says nothing changed there, never "not measured", which is a gap + in what was measured and not the absence of anything to measure. A noise names its two builds (`between`): two builds in one process on one solve route measure the engine's determinism and read 0, two routes measure how a result moves with the route; the note says which; diff --git a/tests/test_fixes.py b/tests/test_fixes.py index db61ca1..3159ff3 100644 --- a/tests/test_fixes.py +++ b/tests/test_fixes.py @@ -5,6 +5,7 @@ import contextlib import io +import itertools import json import tempfile import unittest @@ -920,18 +921,29 @@ def test_a_built_set_is_compared_as_its_own_build(self): self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 1e-4 + 5e-3, "known": 2, "builds": 2}) def test_a_sum_runs_in_the_order_of_the_file(self): - """Three fixes change p1, their builds' noise 0.1, 0.2 and 0.3: summed in the order of - the file it is 0.6000000000000001, in the other order 0.6. The page's port sums in the - same order, so the two agree bit for bit.""" + """Six fixes change p1, the base is the published results. Their builds' noise summed in + the order of the file is 0.26099999999999995; of the 720 orders only that one and the + one with its first two swapped give that float, so an order by id, by the set's own + hashing or by anything else is caught: the page's port must sum as the file runs, not + as an object's keys come. (sentier-bafu's review: 3 fixes already in id order let an + id-sorted sum through.)""" + order = (("f", 0.084), ("b", 0.012), ("e", 0.058), ("a", 0.016), ("d", 0.05), ("c", 0.041)) data, full = chain_and_matrix() - f = fixes.build("Three on one item", data=data, full=full, base="s1", - fixes=[fixes.fix(x, f"Fix {x}", kind="amount", noise=measured(n)) - for x, n in (("x", 0.1), ("y", 0.2), ("z", 0.3))], - results={"x": {A: {"p1": 1.1}}, "y": {A: {"p1": 1.2}}, "z": {A: {"p1": 1.3}}}, + f = fixes.build("Six on one item", data=data, full=full, base="s1", + fixes=[fixes.fix(x, f"Fix {x}", kind="amount", noise=measured(n)) for x, n in order], + results={x: {A: {"p1": 1.0 + n}} for x, n in order}, interactions=checked_pairs()) # fmt: skip - drawn = fixes.compose(f, ["z", "y", "x"]) - self.assertEqual(drawn["item_noise"][A][at(f, "p1")], 0.1 + 0.2 + 0.3) - self.assertNotEqual(0.1 + 0.2 + 0.3, 0.3 + 0.2 + 0.1) + + def summed(noises): + level = None + for n in noises: + level = n if level is None else level + n + return level + + in_file = summed(n for _, n in order) + self.assertEqual(fixes.compose(f, sorted(x for x, _ in order))["item_noise"][A][at(f, "p1")], in_file) + alike = [p for p in itertools.permutations(order) if summed(n for _, n in p) == in_file] + self.assertEqual(alike, [order, (order[1], order[0], *order[2:])], "the test no longer tells orders apart") def test_a_result_taken_from_a_build_inside_the_set_takes_that_builds_noise(self): """The set of a, b and c takes p1 in A from the build of a and b: p1 stands on that @@ -974,6 +986,23 @@ def add(value): found = fixes.errors(changed(f, add(3.09 * 1.025))) self.assertTrue(said(found, "built set 'abc', item 'p3', x:a"), found) + def test_a_built_set_is_held_at_an_item_none_of_its_fixes_change_to_the_base(self): + """a and b built together move p3, which neither changes: there the sum is the base, so + the build is held to it with the base's noise and its own (1.5 %). 1.2 % passes; it + would not with the built set's noise alone (1 %). 1.6 % is refused. (sentier-bafu's + review: no test had a built set move an item none of its fixes move.)""" + f = disjoint() + p1, p2, p3 = (at(f, p) for p in ("p1", "p2", "p3")) + + def add(value): + return lambda g: g["built"].append({"id": "ab", "label": "A and B", "fixes": ["a", "b"], "state": None, + "results": {A: sorted([[p1, 1.5], [p2, 2.5], [p3, value]]), B: []}, + "noise": measured(0.01)}) # fmt: skip + + self.assertEqual(fixes.errors(changed(f, add(3.0 * 1.012))), [], "1.2 % from the base: within 1.5 %") + found = fixes.errors(changed(f, add(3.0 * 1.016))) + self.assertTrue(said(found, "built set 'ab', item 'p3', x:a"), found) + def test_a_release_is_held_to_its_state_with_the_noise_of_both(self): """1.0028 is 1.5e-3 from the state's 1.0013: within the two builds' noise (2e-3), beyond the built set's alone."""