diff --git a/GLOSSARY.md b/GLOSSARY.md index ea9b02f..21a52a6 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -32,8 +32,12 @@ The words this repository uses, each in one line. [SPEC.md](SPEC.md) says the re - **Body**: the page as one element a site holds in its own page; its style stays inside it. - **Drawer**: the panel of one item, at the right of the page, opened from a row. - **Base**: the one state of a chain that every fix is measured against (SPEC section 16). -- **Fix**: one change applied to the base alone and built, with a step's kind, claim, tolerance and noise. -- **Effect**: what a fix does to the base, per item and indicator: its result with the fix minus the base result. -- **Set**: the fixes a reader chooses together; its result is the base plus the sum of their effects. It has no verdict. -- **Interaction**: two fixes whose joint result is not the sum of their effects. Checked in pairs, in the model. -- **Built, exact, estimated**: the label of a set. Built where the producer built it; exact where no two of its fixes can interact; estimated otherwise, and marked on every cell. +- **Fix**: one change applied to the base alone and built, with a step's kind, claim and tolerance, and the noise of its build. +- **Build** (of fixes): a set of results the producer made, the base, a fix with what it requires, or a built set; noise belongs to it, as to a state. A fix is judged with the noise of its build and of the one it was measured against; a set, item by item, with that of the builds the item's result stands on. +- **Requires**: the fixes a fix cannot be applied without. Its effect is measured with them applied, and choosing it chooses them. +- **Effect**: what a fix does, per item and indicator: its result with the fix minus its result without it, both with what it requires. +- **Set**: the fixes a reader chooses together, with what they require; its result is the base plus the sum of their effects. It has no verdict. +- **Interaction**: two fixes whose joint result is not the sum of their effects. Checked in pairs, in the model; a listed pair says its kind, its reach and its size, never words. +- **Size** (of a pair): the smallest tolerance under which the build of two fixes agrees with the sum of their effects. **No size** where no tolerance is enough (two sides of zero, a zero, a missing result): such a pair is built. Not the same as not measured. +- **Pair terms**: how far an estimated set may lie from its sum, to first order: the sum of the sizes of the listed pairs it holds; not known where one was not measured or has no size. +- **Built, exact, estimated**: the label of a set. Built where the producer built it (one fix with what it requires, or a built set); exact where no two of its fixes can interact; estimated otherwise, and marked on every cell. diff --git a/SPEC.md b/SPEC.md index cf16af1..25eae1b 100644 --- a/SPEC.md +++ b/SPEC.md @@ -89,7 +89,9 @@ so the claim stays right when the factors change. `tests/fixtures/located-factor that step, made up, and a test holds what it shows. **Order matters.** Where two steps touch the same results, each one's share depends on the -order they were applied in. The file keeps the order; the page says so. +order they were applied in. The file keeps the order; the page says so. A set of changes +chosen freely, not a run of consecutive steps, is not drawn from a chain: section 16 +measures each fix on its own against one base, so that any set can be. **A run of steps.** A file holds one comparison per step and one for the whole way. A run of two or more consecutive steps, short of every step (steps 3 to 9), is not in the file: @@ -535,7 +537,7 @@ of its own.** It reads them from the file, which the validator has checked, and the file's counts up: the classes that moved into "moved", and a step's counts over its indicators. Two things it works out itself, and only from every item: the median and percentiles of a distribution (10.3), and a run of steps (section 3), the one place where -it judges a pair itself. It does so with the rule ported from `rule.py`, held to it by the +it judges a pair itself in a diagnostics file (a set of fixes is the other, 16.8). It does so with the rule ported from `rule.py`, held to it by the shared acceptance cases and by 813 pairs on every edge of the rule that the Python writes (`tests/test_boundaries.py`), after holding each step of the run to the file's own classes. `tests/fixtures/together.json` holds the counts `together.py` gives for every run of the @@ -860,8 +862,16 @@ Status: proposed, 18 November 2026. Manuel decided the direction: a chain is to hundreds of fixes, any set of them can be chosen, and the set is **one base with fixes measured on their own**, not a chain of hundreds of states ("yes, one base with independently measured fixes"). He also decided that the fixes live in a file of their own, -and that a set the file cannot show to be exact is shown, marked as estimated. Nothing -here is code yet; the validator follows once this text is agreed. +and that a set the file cannot show to be exact is shown, marked as estimated. Four +questions were settled in the validator's pull request, before any producer writes a file, +for Manuel to agree there: a pair holds no text (16.6); a fix may require others (16.4); a +fix may bring items the base does not hold (16.4); and a built set may say which state of +the chain it is (16.7). Noise follows section 5.3 (Manuel: "sum, go ahead"): it belongs to +a build, and a comparison takes the sum of the noise of the builds it stands on (16.5). +sentier-bafu wrote a file from its own chain to review the format as its producer (12 fixes +and the release, checked clean), and its review defined the size of a pair (16.6), the route +an effect is measured on (16.4), and the premise of the list of pairs (16.3). The reader and +validator are `fixes.py`; the producer and the page are not written yet (16.12). ### 16.1 Why not a chain of fixes @@ -879,12 +889,16 @@ the base to a release. A fixes file hangs on one of its states. ### 16.2 Words - **Base**: one state of a chain, the state every fix is measured against. +- **Build**: a set of results the producer made: the base, a fix with what it requires, or + a built set (16.7). Noise belongs to a build, as it belongs to a state (section 5.3). - **Fix**: one change, applied to the base alone and built. It has what a step has: a - kind, a claim, a tolerance, a noise, `explained`, a link. -- **Effect**: what a fix does to the base, per item and indicator: its result with the fix - minus the base result. -- **Set**: the fixes a reader chooses together. Its result is the base plus the sum of - their effects. + kind, a claim, a tolerance, `explained`, a link; and the noise of its build. +- **Requires**: the fixes a fix cannot be applied without. Its effect is measured with + them applied, and choosing it chooses them. +- **Effect**: what a fix does, per item and indicator: its result with the fix (and what it + requires) minus its result without the fix (with what it requires). +- **Set**: the fixes a reader chooses together, with everything they require. Its result is + the base plus the sum of their effects. - **Interaction**: two fixes whose joint result is not the sum of their effects. ### 16.3 When a sum is the result @@ -898,7 +912,24 @@ the technosphere A. So: - a fix that changes the factor of a flow and one that changes how much of that flow an activity emits interact; - two fixes that change A interact on every item whose supply chain reaches both changes. - Most fixes of a database (a mapping, an amount, a market share) change A. + Most fixes of a database (a mapping, an amount, a market share) change A; +- a fix that changes A and one that changes B or c interact where the A change shifts how + much of the changed flow or activity an item's chain draws on; +- two fixes that change the same number of A, B or c do not add up whatever kind it is: + one replaces what the other set. + +These are the kinds of pair a file can list (16.6). Two fixes that change different +factors, or different entries of B, are never a pair. + +All of this holds for fixes that are fixed changes of A, B and c. **A fix that edits the +data a model is imported from can change what another fix puts into the matrices**: an +importer that links an exchange to a flow by its name, compartment or CAS number links it to +whatever another fix calls that flow. Changing which flow an exchange is (its name, +compartment or CAS number, where factors match on them) changes B. Several such edits can +be needed together without any one pair showing it. A producer whose fixes edit data checks +that the release's change of A, B and c is the sum of each fix's change, entry by entry, and +says so in `method` (16.6); where it is not, the fixes that meet before the matrices are a +pair (`same entry`), or one requires the others. An interaction always shows in a pair: every term of A⁻¹ expanded in its changes that holds two fixes holds two of them next to each other. So a set whose fixes are independent @@ -915,53 +946,132 @@ item. One JSON object, read as strictly as the diagnostics file (section 4.1). | Key | What it holds | |---|---| | `format` | `"sentier-diagnostics-fixes/1"` | -| `of` | `{file, title, base}`: the diagnostics file it belongs to (a file name), that file's title, and the id of the state that is the base | +| `of` | `{file, title, base, base_noise}`: the diagnostics file it belongs to (a file name), that file's title, the id of the state that is the base, and that state's noise as the diagnostics file says it (5.3). `base_noise` is left out where the base is the published results, which are exact | | `title`, `about` | Short text; `about` may be null | -| `item`, `indicator`, `indicators` | As in the diagnostics file, and equal to it | -| `rule` | The floors, equal to the diagnostics file's (they come from the chain's first state) | -| `items` | Every item, `[id, name, place, group, unit]`, in the order of the matrix | -| `base` | The base result of every item, per indicator: `{indicator: [result or null, ...]}`, in the order of `items` | -| `fixes` | `[{id, label, about, kind, explained, link, claim, tolerance, noise, effect, counts}]` | +| `item`, `indicator`, `indicators` | As in the diagnostics file, and equal to it (`indicator` may be left out where that file leaves it out) | +| `rule` | `{floor_factor, floors_from, beyond, floors}`: the diagnostics file's rule without its `total_tolerance`, which belongs to the chain (a set has its own, 16.5). The floors come from the chain's first state, followed by a floor for each unit only an item a fix brings has | +| `items` | Every item, `[id, name, place, group, unit]`: the matrix's items in the matrix's order, then any item a fix brings that no state of the chain holds | +| `base` | The base result of every item, per indicator: `{indicator: [result or null, ...]}`, in the order of `items`; null for an item a fix brings | +| `fixes` | `[{id, label, about, kind, explained, link, requires, claim, tolerance, noise, effect, counts}]`, a fix's requirements before it; `noise` is that of the build with the fix | | `interactions` | `{checked, method, by, pairs}` (16.6) | -| `built` | Sets the producer built: `[{id, label, fixes, results, noise}]` (16.7) | +| `built` | Sets the producer built: `[{id, label, fixes, state, results, noise}]` (16.7) | | `notes`, `made_by` | As in the diagnostics file | -A fix's `effect` holds, per indicator, the items whose result with the fix is not the base -result bit for bit, as `[[item index, result], ...]`, the result being the one built with -the fix alone (`null` where it has none). **Every change is kept, however small**: two -changes under the tolerance each can be over it together, and a set is a sum. An entry -equal to the base is refused: it would read as a change. A fix's `counts` is one block per -indicator (section 4.2), its pairs being the base against the result with the fix, judged -with the fix's own tolerance and noise. - -A fix's claim is judged as a step's is (section 6), against the base. Its verdicts do not -depend on which other fixes are chosen or in what order: that is what measuring a fix on -its own buys. +A fix's `effect` holds, per indicator, the items whose result with the fix is not the +result it was measured against bit for bit, as `[[item index, result], ...]`, the result +being the one built with the fix (`null` where it has none). **Every change is kept, +however small**: two changes under the tolerance each can be over it together, and a set is +a sum. An entry equal to the result it was measured against is refused: it would read as a +change. Every indicator has a list, empty where the fix changes nothing. + +**An effect is measured on the base's route.** The result with the fix is built as the base +was (the same importer, linking and solver), or is the base result plus the difference +between two results of one other route, so that an item the fix does not reach keeps the +base result bit for bit. The base is held to the chain bit for bit and every entry that +differs is kept, so an effect computed another way fills up with the route's own +differences, each read as a change: sentier-bafu built one release twice, a pinned copy +against today's importer with no change meant, and 294,590 of 298,675 results moved, 722 of +them by 1e-5 or more. **Results are read exactly**, as they were written, and both sides the +same way. A reader that rounds the last bit merges distinct results and drops changes: a +default CSV reader misread 134,743 of the 298,675 results of one build in the last bit, and +lost 7 of the 86 changes of one patch; read round-trip, every result matches its text. Where +the two sides are read differently (effects read exactly, against a base held to a chain whose +results went through a rounding reader), every misread result reads as a change. + +A fix's `counts` is one block per indicator (section 4.2), its pairs being the result it was +measured against and the result with the fix, judged with the fix's own tolerance and the +sum of the noise of those two builds (16.5); `closer` and +`further` are null, since a fix has no published results to come closer to. A fix's claim +names its items (no `items: null`): the file is local and holds every item, and the page +works out each item's verdict from the claim. + +**A fix that cannot be applied alone** (a correction to a market that another fix creates, +a factor for a flow another fix adds) names the fixes it needs in `requires`: fixes that +come before it in `fixes`, each once, never itself. What it requires at any depth is +applied when it is built, and its effect and counts are measured against the build of +those fixes together. Where they are one fix with what it requires, that is that fix's +build (a fix that requires `d`, which requires `a`, is measured against the build of `d`); +otherwise it is a built set of exactly those fixes (16.7). A file that holds no such build +is refused: the effect would have nothing to be measured against. A fix and a fix it requires are never a pair (16.6): +one is measured with the other applied. + +**A fix may bring items** no state of the chain holds (a new product an added activity +makes). They follow the matrix's items in `items`, their base result is null, and some fix +or built set gives each a result. Their unit takes the chain's floor; a unit no item of the +chain has takes the floor the chain gives a unit its first state holds no result in: 0, from +nothing (`basis: "none"`), added to the rule after the chain's floors. + +A fix's claim is judged as a step's is (section 6), against what the fix was measured +against. Its verdicts do not depend on which other fixes are chosen or in what order: that +is what measuring a fix on its own buys. ### 16.5 A set and its label -The result of a set is the base plus the sum of its fixes' effects. It is judged by the -rule (section 5) against the base: +A set holds everything its fixes require: choosing a fix chooses what it requires, at any +depth, and the page ticks those fixes too and says it added them. At each item, its result +is: + +- where one fix of the set changes it, that fix's built result; +- where several do, and they all lie in one build the file holds inside the set, that + build's result: measured, not summed. The builds are looked at in this order: each fix of + the set that requires others, with what it requires, in the order of the file; then each + built set inside the set, in the order of the file. The first that holds them all is + taken; +- otherwise the base plus each fix's difference against the result it was measured + against. A fix measured on top of another adds only its own difference, so what both + build on is counted once. + +Where one of those differences cannot be taken (a missing result with the fix or without +it), the set has no result there, and the page says the sum cannot be formed, apart from a +result that is missing. It is judged by the rule (section 5) against the base: - its **tolerance** is the largest of its fixes'; -- its **noise** is the sum of its fixes' noise, per indicator. A fix's noise stands for - how unstable its comparison with the base is; the set's difference is the sum of those - comparisons, and their instabilities add up instead of cancelling, as they would along - a run of a chain. Noise not measured for some fixes is said ("noise measured for 3 of 5 - fixes"), never read as none; where none is measured the page says "not measured", never - "identical"; +- its **noise** follows section 5.3: noise belongs to a build, and a comparison takes the + sum of the noise of the builds it stands on, never of one it does not. The base's is its + state's (`of.base_noise`; the published results are exact and add nothing), a fix's that + of the build with it and what it requires, a built set's that of its build. A set is + judged **item by item**, with the builds that item's result stands on: + - where its result is a build's (one fix changes it, or a build inside the set covers + it, or the set is a build), that build's noise and the base's; + - where it is a sum, the sum over the fixes that change the item of the noise of each + fix's build and of the build it was measured against: the base once for each of them + measured against it, since a rebuild of the base moves each of those differences the + same way. The base the sum starts from is the base it is compared with, and cancels. + + A fix that does not change an item adds nothing to it. This is the fixes file's form of + "a state inside a run adds nothing" (5.3): summed over every fix ticked, the noise of + three fixes on three items, each built at 1 %, was 6 %, and ticking an unrelated fix put + an item that moved 2.9 % "within noise" (sentier-bafu's probe; A30 again). The sum runs in + the order of the file's fixes, each fix's measured-against build before its own, so that + the page's port gives the same number bit for bit. + + Per indicator the page says the largest noise an item is judged with, and at how many of + the builds those items stand on the noise is known ("noise known at 3 of 5 builds"; the + published results are known, and exact), never reading not measured as none; where no + build measured it the page says "not measured", never "identical". Where the set changes + nothing in an indicator, no item is judged and the noise stands on no build (`builds` 0, + `value` null): the page says nothing changed there, never "not measured", which is a gap + in what was measured and not the absence of anything to measure. A noise names its two + builds (`between`): two builds in one process on one solve route measure the engine's + determinism and read 0, two routes measure how a result moves with the route; the note + says which; - it has **no verdict**: each claim is judged in its own fix. Each set carries one of three labels, and the page shows it wherever the set is drawn: | Label | When | What the page draws | |---|---|---| -| built | the set is one fix, or one of `built` | the built results | +| built | the set is one fix with what it requires, or one of `built` | the built results | | exact | `interactions.checked` is true, and no two of its fixes are a listed pair | the sum, as a result | -| estimated | otherwise | the sum, marked on every cell, in the line above the matrix and in every count ("estimated: 153 would differ"), with the listed pairs it holds and their reasons | +| estimated | otherwise | the sum, marked on every cell, in the line above the matrix and in every count ("estimated: 153 would differ"), with the listed pairs it holds, their kinds, and their pair terms where measured ("pair terms up to 0.003 %") | An estimated set is never drawn as a built one: the mark is on the cell, not only in a -heading a reader may have scrolled past. Where interactions were not checked, every set of +heading a reader may have scrolled past. Its **pair terms** are the sum of the measured +sizes (`largest`) of the listed pairs it holds: how far it may lie from its sum, to first +order (a term of three fixes or more needs interacting pairs, 16.3). Where a pair it holds +was not measured, or has no size (16.6), the pair terms are not known, and the page says how +many of its pairs were measured and how many of those have no size, never 0. Where +interactions were not checked, every set of two fixes or more that was not built is estimated. ### 16.6 Interactions @@ -974,48 +1084,111 @@ into it. - `checked`: true where every pair of fixes was examined, false where none was. There is no half: a reader must know whether a pair that is not listed was cleared or never looked at. - `method`: what the check did, in a sentence, and `by`: who or what ran it (section 8). -- `pairs`: `[{fixes: [a, b], reason}]`, every pair that can interact, each once, with a - short reason ("both change the market for wheat flour"). +- `pairs`: `[{fixes: [a, b], kind, reach, largest}]`, every pair that can interact, each + once: + - `kind`: where the two changes meet (16.3), the first that applies of `same entry` (both + change one number), `technosphere` (both change A), `technosphere and emission` (A, and + an amount of B), `technosphere and factor` (A, and a factor of c), `emission and factor` + (an amount of a flow, and that flow's factor). A pair that meets in several ways (a fix + to an amount and an emission of one activity, with a fix to A) takes the first; + - `reach`: how many items the cross term can change, counted on the build with both fixes + (1 to the number of items), or null where it was not counted. A fix can open a path (an + exchange finds a supplier it had none for), so the base undercounts; whether an item + reaches one fix's change does not depend on that fix, so the build with both bounds it; + - `largest`: the size of its cross term: the smallest tolerance under which the build of + the two agrees, by the rule, with the base plus their two effects. That is the largest + relative difference of the two (the rule's measure, over the larger size, section 5) + over items and indicators, leaving out the items the rule calls near zero on both sides + (`fixes.size_of`). `"no size"` where no tolerance is enough: at some item the two lie on + two sides of zero, one is zero, or one has no result. Null where it was not measured. + "No size" and "not measured" are kept apart: one is a finding, the other a gap. + + A ratio over |h_ab| has no value where h_ab is 0 and grows without bound for items under + the floor; such a share would swamp the pair terms of every set holding the pair and make + the check of its build (16.7) pass anything. The rule's own measure has neither fault, and + it is the tolerance the check applies. **A pair with no size must be built** (16.7): no set + holding it can be bounded by its sum, and with the build the page takes its result where + only those two change an item. + + A check that only asks whether A⁻¹ links two changes lists nearly every pair in a + database where electricity and transport reach everything, and every set would be + estimated; the size lets a reader tell a negligible estimate from a large one. A pair + whose cross term is zero by the model's structure is not listed, and neither is a fix + with a fix it requires. + +**A pair holds no text.** A reason in words ("both change the market for wheat flour") +names the model, and a check of text can only catch what is shaped like an identifier, not +a name. A closed kind, a count and a size say what a reader needs (how the two meet, how +far it spreads, how large it is) and leave nothing to check. The one text the interactions +hold is `method`, and the check refuses one holding text shaped like an identifier (a UUID, +or 32 hexadecimal digits). The validator cannot see the model, so it cannot prove a pair independent. It holds the claim to every built set instead (16.7). ### 16.7 Built sets -The producer may build sets as well as single fixes: the release with every fix, or a pair -it listed, to measure how far it is from the sum. A built set has an id, a label, its fixes -(two or more), its results (per indicator, the items whose result is not the base result -bit for bit, as an effect is written) and its noise. +The producer may build sets as well as single fixes: the release with every fix, a pair +it listed, to measure how far it is from the sum, or the fixes another fix requires, to +measure that fix against. A built set has an id, a label, its fixes (two or more, with +everything they require, and never exactly one fix with what it requires: that is built +already), its results (per indicator, the items whose result is not the base result bit for +bit, as an effect is written), its noise, and `state`: the id of the state of the chain it +is, where it is one, or null. **A built set tests the claim of independence.** Where `checked` is true and no two of its fixes are a listed pair, the built results must agree with the base plus the sum of the -effects, by the rule with the set's tolerance and noise, for every item. A file where they -do not is refused, naming the set, the item and the indicator: its producer called two -fixes independent that are not. Where the set holds a listed pair, the page shows how far -the built results lie from the sum, as that set's interaction. +effects, by the rule with the set's tolerance and noise, for every item. That sum stands on +the fixes alone (16.5 without its built sets): a set held to a build inside it, itself held +to the sum, would stack two tolerances. The noise is taken item by item, as in 16.5: the +built set's own, and that of the builds the sum stands on at that item (the build it takes +the result from; or the base, with each fix that changes the item, its build and the one it +was measured against; or the base alone where no fix changes it). A fix that does not change +the item adds nothing: otherwise a large built set would pass a real disagreement as within +noise. A file where they do not agree is refused, naming the set, the item and the +indicator: its producer called two fixes independent that are not. Where the set holds a +listed pair, the page shows how far the built results lie from the sum, as that set's +interaction. Where the set is exactly one listed pair with a measured `largest`, the built +results must lie within it of the sum (by the rule, with the larger of the set's tolerance +and `largest` as the tolerance): a file that understates a size it could have measured is +refused. Where the file says the pair has no size, its build must have none: a build that +agrees with its sum under some tolerance has that size, and the check refuses the file and +gives it. Building the release with every fix is cheap (it is built anyway) and is the strongest test -the file can carry: it touches every pair at once. +the file can carry: it touches every pair at once. It can refute independence and never +confirm it: cross terms of opposite sign can cancel in the whole and not in a subset. + +**A built set that names a state is two builds of one thing.** The release with every fix +is also the last state of the chain. Its `state` names it, a state after the base, and the +check holds the two to each other: the base with the built results, against the matrix's +results of that state, by the rule with the largest tolerance of its fixes and the noise of +the two builds, the built set's and the state's (none where the state is the published +results), for every item; for the listed items where the matrix is not beside it. A file +whose release does not reproduce the chain's is refused, naming the set, the item and the +indicator. A release taken from the same build as the chain's state passes this check by +construction; the check of independence above is the one that can fail it. ### 16.8 The page - A searchable list chooses fixes: by name, kind and whether a fix moved anything, with a - box per fix. Hundreds do not fit a row of chips. + box per fix. Hundreds do not fit a row of chips. Choosing a fix ticks what it requires + and says so; unticking a fix another chosen fix requires unticks that one too. - The address names the set: `#set=a,b,c`, in the order of the file. An address that names a fix the file does not hold opens on the base and says which. - One fix reads as a step does today: its claim, its verdicts, its own counts from the file. - Any set drives the matrix, the distributions, the worst values and the drawer. Its line says its label, its fixes with their claims, its tolerance and its noise. -- The page works out the classes of a set, the one place where it judges a pair itself - (section 10 says it works out none of its own; a set is the exception). It does so with - a port of `rule.judge` that is held to the Python rule by the shared cases - (`tests/acceptance/cases.json`) and by boundary values the Python generates: every - class edge, the floor, a sign crossing, every band edge, a factor too large for a float. +- The page works out the classes of a set, as it does those of a run of steps (section + 10): with the port of `rule.judge` that the shared cases (`tests/acceptance/cases.json`) + and the boundary pairs the Python writes (`tests/test_boundaries.py`: every class edge, + the floor, a sign crossing, every band edge, a factor too large for a float) hold to the + Python rule. It works out a set's noise as 16.5 says, in the same order. Before it composes a set it works out each chosen fix's own classes again and compares them with the file's; where one differs, it composes nothing and says so. - In the drawer, an item's difference in a set splits into its fixes' effects, exactly - where the set is exact. For a built set holding a listed pair, the rest is shown as the - interaction, never spread over the fixes. + where the set is exact, each fix's against what it was measured with. For a built set + holding a listed pair, the rest is shown as the interaction, never spread over the fixes. ### 16.9 Checks @@ -1023,22 +1196,49 @@ the file can carry: it touches every pair at once. wrong (as section 4.3), and further one where: - `of` does not name the diagnostics file beside it, its title or one of its states, or its - items, indicators, floors or words differ from that file's; + items, indicators, floors or words differ from that file's (with a floor of 0 from nothing + for each unit only an added item has); the diagnostics file says no noise of its states; + `base_noise` is not the base state's noise, is missing where the base is a build, or is + given where it is the published results; +- its items are not the matrix's in the matrix's order, followed by items a fix brings, or + an item a fix brings has a base result, or no fix or built set gives it one; - the base results differ from the matrix's results of the base state, where the matrix is beside it, or from the listed items' results, where it is not; +- a fix requires itself, a fix the file does not hold, a fix after it, or one fix twice; or + the file holds no build of what it requires together; - an effect or a built set names an item or indicator it does not hold, holds an entry - equal to the base, or holds one item twice; -- a fix's counts or verdicts are not what its effect and the base give, by the rule, with - its own tolerance and noise. With every item at hand, the counts must be exact; -- a pair names a fix it does not hold, a fix twice, or is listed twice; `checked` is false - and pairs are listed; -- a built set holds fewer than two fixes, a fix twice, or the same fixes as another; or - disagrees with the sum of its effects where the file says its fixes are independent. + equal to the result it was measured against, or holds one item twice; +- a fix's counts or verdicts are not what its effect and the build it was measured against + give, by the rule, with its own tolerance and the noise of the two builds. With every item + at hand, the counts must be exact; +- a pair names a fix it does not hold, a fix twice, or a fix and one it requires, or is + listed twice; its kind is not one of the five, its reach not a count of items, its + `largest` not a share, `"no size"` or null; a pair with no size has no build; `checked` + is false and pairs are listed; a pair holds any other key; the method holds text shaped + like an identifier; +- a built set holds fewer than two fixes, a fix twice, a fix without one it requires, + exactly one fix with what it requires, or the same fixes as another; or disagrees with the + sum of its effects where the file says its fixes are independent, or lies further from it + than the measured size of the one pair it is, or has a size where the file says that pair + has none; or names a state that is not after the base, or that its results do not + reproduce. + +`check` also prints what the file is made of: how many entries its fixes and built sets +hold, and its largest fixes (a fix to a market everything uses holds about one entry per +item and indicator). ### 16.10 Limits Up to 1,000 fixes, 500 built sets, 200,000 items and 64 MB, as the matrix. A base of 11,947 items in 25 indicators is about 300,000 numbers; an effect holds only what changed. +sentier-bafu measured its fixes on that base: a patch moves tens of entries, a fix to a +characterisation factor about 12,000 (about 250 KB), a fix to a market everything uses up +to about 300,000 (6 to 7 MB). 64 MB holds about ten dense fixes and thousands of sparse +ones; effects move to one file per fix only if dense fixes become common. Checked on a +made-up file of that base with 100 fixes of 500 items each in every indicator, ten of them +requiring the fix before them, every build's noise measured (39.5 MB), the check takes +about 2.4 seconds, and counting a set of 50 of them, each item with its own noise, about +one. ### 16.11 Acceptance cases @@ -1051,9 +1251,14 @@ Each is a test before the code is written. | F3 | Interactions not checked: every set of two fixes or more that was not built is estimated, none exact | | F4 | A built set that disagrees with the sum where the file calls its fixes independent is refused, naming the set, item and indicator | | F5 | A fix's verdicts are the same whichever fixes are chosen with it | -| F6 | The noise of a set is the sum of its fixes' noise; noise measured for some fixes only is said, and not measured is never identical | +| F6 | A set is judged item by item with the noise of the builds the item's result stands on: a build's and the base's, or for a sum, each summed fix's build and the one it was measured against (the base once for each fix). A fix that does not change the item adds nothing; the builds whose noise is known are counted, and not measured is never identical | | F7 | The page's rule agrees with the Python rule on every shared case and generated boundary value, and the page refuses to compose where it cannot reproduce a fix's own classes | -| F8 | A file of fixes holds no identifier of the model: a pair has a reason, not the activities it shares | +| F8 | A file of fixes holds nothing of the model in a pair: its kind comes from a closed list, and any text in it is refused | +| F9 | A fix that requires another is measured against their build; choosing it chooses what it requires; in a set, each effect adds on top of what it was measured with, so what two fixes build on counts once; where a build inside the set holds every fix that changes an item, its result is taken, measured, not summed | +| F10 | A built set that names a state of the chain is refused where its results do not reproduce that state | +| F11 | A fix may bring an item no state holds: it follows the matrix's items, with no base result, and a fix gives it one; in a unit no item of the chain has, it takes a floor of 0 from nothing | +| F12 | A fix is judged with the noise of its build and of the build it was measured against; a built set is held to its sum, item by item, with the noise of the builds both sides stand on there, and to its state with its noise and the state's; the base's noise is its state's, and the published results are exact | +| F13 | A pair with no size is built, and its build has none; "no size" and "not measured" read apart | ### 16.12 Who does what, and in which order diff --git a/src/sentier_diagnostics/cli.py b/src/sentier_diagnostics/cli.py index 9c732aa..de364c8 100644 --- a/src/sentier_diagnostics/cli.py +++ b/src/sentier_diagnostics/cli.py @@ -1,7 +1,8 @@ """The command `sentier-diagnostics`: check, build, summarise and show diagnostics files. sentier-diagnostics check FILE [FILE ...] exit 1 if any file is not one a page shows - (a matrix beside a file is checked against it) + (a matrix beside a file is checked against it, + a fixes file against the file it names) sentier-diagnostics build DESCRIPTION --out FILE sentier-diagnostics summary FILE sentier-diagnostics page FILE --out FOLDER @@ -14,7 +15,7 @@ import sys from pathlib import Path -from . import __version__, breakdown, chain, example, fileformat, from_tables, matrix, page, summary +from . import __version__, breakdown, chain, example, fileformat, fixes, from_tables, matrix, page, summary def check(paths) -> int: @@ -30,6 +31,15 @@ def check(paths) -> int: errors = breakdown.errors(data) elif isinstance(data, dict) and str(data.get("format", "")).startswith(matrix.FAMILY): errors = matrix.errors(data) # against its file when the file is checked + elif isinstance(data, dict) and str(data.get("format", "")).startswith(fixes.FAMILY): + errors = fixes.errors(data) + size = Path(path).stat().st_size + if size > fixes.MAX_BYTES: + errors.append(f"{size} bytes, and a page loads {fixes.MAX_BYTES} at most") + if not errors: + errors = fixes.beside(data, Path(path).parent) + if not errors: + print(f"note {path}: {fixes.size_note(data)}") else: errors = fileformat.errors(data) size = Path(path).stat().st_size @@ -68,7 +78,7 @@ def main(argv=None) -> int: parser = argparse.ArgumentParser(prog="sentier-diagnostics", description=__doc__.splitlines()[0]) parser.add_argument("--version", action="version", version=f"sentier-diagnostics {__version__}") commands = parser.add_subparsers(dest="command", required=True) - one = commands.add_parser("check", help="check diagnostics and breakdown files") + one = commands.add_parser("check", help="check diagnostics, breakdown, matrix and fixes files") one.add_argument("files", nargs="+", type=Path) one = commands.add_parser("build", help="build a diagnostics file from CSV tables") one.add_argument("description", type=Path) diff --git a/src/sentier_diagnostics/fixes.py b/src/sentier_diagnostics/fixes.py new file mode 100644 index 0000000..031d5c6 --- /dev/null +++ b/src/sentier_diagnostics/fixes.py @@ -0,0 +1,1553 @@ +"""Fixes against one base, `sentier-diagnostics-fixes/1` (SPEC 16). + +A chain measures each step with every earlier step applied, so a step's difference belongs +to its place in the order. A fixes file measures each fix on its own against one state of +a chain, the base: a fix has one effect, whichever set it joins and in whatever order. A +fix that cannot be applied alone names the fixes it needs (`requires`): its effect is +measured with them applied, against a build of them, and choosing it chooses them. + +A reader may choose any set of fixes, and gets what they require with them. Its result is +the base plus the sum of their effects, except where a build the file holds inside the set +covers every fix that changes an item: there the build's result is taken. A set is +labelled: + + built the file holds this build: one fix with what it requires, or a set the + producer built (`built`) + exact interactions were checked, and no two of its fixes are a listed pair + estimated otherwise: drawn, and marked on every cell and count + +A set has no verdict: each claim is judged in its own fix. A set's tolerance is the largest +of its fixes'. Noise belongs to a build, as it belongs to a state of a chain (SPEC 5.3): the +base's is that state's (none where the base is the published results, which are exact), a +fix's is that of the build with it, a built set's that of its build. A fix is judged with the +sum of the noise of its build and of the one it was measured against. A set is judged item by +item with the noise of the builds the item's result stands on, never with that of a fix that +does not touch it: a build's result, with that build's and the base's; a sum, with each +summed fix's build and the one it was measured against. It says at how many of those builds +the noise is known, never reading not measured as none. + +A listed pair holds no text: its kind of interaction (one of `PAIR_KINDS`, from where the +two changes meet in h = c·B·A⁻¹·f), how many items it can reach and the measured size of +its cross term. So nothing of the model, no name and no identifier, can go into it. + +`errors(f)` checks a file on its own: its shape, every fix's counts and verdicts worked out +again from its effect and what it was measured against, the pairs, and every built set the +file calls independent against the sum of its effects. `against(f, data, full)` holds it to +its diagnostics file and that file's matrix, and a built set that names a state of the chain +to that state; `beside(f, folder)` finds them next to it, as `sentier-diagnostics check` +does. + +`build(...)` makes a file from a checked diagnostics file, its matrix and the results built +with each fix; `chosen`, `label`, `compose` and `counts` give what a page draws for a set. +""" + +from __future__ import annotations + +import json +import re +from collections.abc import Iterable, Mapping, Sequence +from functools import cached_property +from pathlib import Path + +from . import __version__, chain, claims, fileformat, matrix, rule + +FORMAT = "sentier-diagnostics-fixes/1" +FAMILY = "sentier-diagnostics-fixes/" +KEYS = { + "format", "of", "title", "about", "item", "indicators", "rule", "items", "base", "fixes", + "interactions", "built", "notes", "made_by", +} # fmt: skip +#: A key a file may leave out, as in the diagnostics file. +OPTIONAL_KEYS = {"indicator"} +OF_KEYS = {"file", "title", "base"} +#: The noise of the base, when the base is a build; left out where it is the published +#: results, which are exact (SPEC 5.3). +OF_OPTIONAL = {"base_noise"} +#: The rule of the diagnostics file, without the chain's own total tolerance. +RULE_KEYS = fileformat.RULE_KEYS - {"total_tolerance"} +FIX_KEYS = {"id", "label", "about", "kind", "explained", "link", "requires", "claim", "tolerance", "noise", + "effect", "counts"} # fmt: skip +INTERACTION_KEYS = {"checked", "method", "by", "pairs"} +PAIR_KEYS = {"fixes", "kind", "reach", "largest"} +BUILT_KEYS = {"id", "label", "fixes", "state", "results", "noise"} + +#: Where two changes of h = c·B·A⁻¹·f meet, so that their joint result is not the sum of +#: their effects (SPEC 16.3). Two changes to different factors, or to different entries of +#: B, add up exactly and are never a pair. In the order of precedence: a pair that meets in +#: several ways is the first that applies. +SAME_ENTRY = "same entry" # both change one number of A, B or c: two edits of one number do not add +TECHNOSPHERE = "technosphere" # both change A, and an item's supply chain reaches both +TECHNOSPHERE_AND_EMISSION = "technosphere and emission" # A, and an amount of B on a chain the A change shifts +TECHNOSPHERE_AND_FACTOR = "technosphere and factor" # A, and the factor of a flow the A change shifts +EMISSION_AND_FACTOR = "emission and factor" # an amount of a flow, and that flow's factor +PAIR_KINDS = (SAME_ENTRY, TECHNOSPHERE, TECHNOSPHERE_AND_EMISSION, TECHNOSPHERE_AND_FACTOR, EMISSION_AND_FACTOR) + +#: A pair's `largest` where no tolerance makes its build agree with the sum of its effects: +#: at some item the two lie on two sides of zero, one is zero, or one has no result (SPEC +#: 16.6). Not null, which says the size was not measured. +NO_SIZE = "no size" + +MAX_FIXES, MAX_BUILT = 1000, 500 +MAX_ITEMS, MAX_BYTES = matrix.MAX_ITEMS, matrix.MAX_BYTES + +BUILT, EXACT, ESTIMATED = "built", "exact", "estimated" +#: What a set's result at an item stands on (`_Builds.composed`): one build, or a sum of fixes. +BUILD, SUM = "build", "sum" +LABELS = (BUILT, EXACT, ESTIMATED) + +#: The shape of an identifier of a model (a UUID, or 32 hexadecimal digits): refused in the +#: one text the interactions hold, the method of the check. +IDENTIFIER = re.compile( + r"\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b|\b[0-9a-fA-F]{32}\b" +) + + +# ------------------------------------------------------------------------ making a file + + +def fix(id: str, label: str, *, kind: str, requires: Iterable[str] = (), explained: bool = True, + about: str | None = None, link: str | None = None, claim: dict | None = None, + tolerance: dict | None = None, noise: dict | None = None) -> dict: # fmt: skip + """One fix, as `build` takes it: what a step has (`chain.step`), the fixes it needs applied + first, and the noise of the build with it (`chain.noise`), which belongs to that build as a + state's noise belongs to the state.""" + one = chain.step(id, label, kind=kind, explained=explained, about=about, link=link, claim=claim, + tolerance=tolerance) # fmt: skip + return {**one, "requires": list(requires), "noise": noise} + + +def pair(a: str, b: str, kind: str, *, reach: int | None = None, largest: float | str | None = None) -> dict: + """Two fixes that can interact: `kind` is the first of `PAIR_KINDS` that applies, `reach` + how many items the cross term can change, counted on the build with both; `largest` its + measured size (`size_of`), or `NO_SIZE`; None where not counted or not measured.""" + return {"fixes": [a, b], "kind": kind, "reach": reach, "largest": largest} + + +def interactions(checked: bool = False, *, method: str | None = None, by: dict | None = None, + pairs: Iterable[dict] = ()) -> dict: # fmt: skip + """The check of pairs: `checked` true where every pair was examined, false where none was; + `pairs` are `pair(...)`, every pair that can interact, each once.""" + return {"checked": checked, "method": method, "by": by, "pairs": list(pairs)} + + +def built_set(id: str, label: str, fixes: Iterable[str], results: Mapping, *, state: str | None = None, + noise: dict | None = None) -> dict: # fmt: skip + """A set the producer built: `results` are `{indicator: {item: result or None}}`, an item + left out being the base result; `state` the id of the chain's state it is, where it is + one (the release with every fix).""" + return {"id": id, "label": label, "fixes": list(fixes), "state": state, "results": results, + "noise": noise or chain.noise()} # fmt: skip + + +def _rule_of(data: Mapping, added: Iterable) -> dict: + """The rule of a fixes file: its diagnostics file's, without the chain's total tolerance, + and a floor for every unit only an added item has: 0, from nothing, as the chain gives a + unit its first state holds no result in (an added item is in no state of it).""" + out = {k: data["rule"][k] for k in sorted(RULE_KEYS)} + known = {one["unit"] for one in data["rule"]["floors"]} + new = sorted({row[4] for row in added} - known, key=lambda u: (u is None, u or "")) + out["floors"] = [*out["floors"], *({"indicator": i["id"], "unit": unit, "value": 0.0, "basis": "none", "n": 0} + for i in data["indicators"] for unit in new)] # fmt: skip + return out + + +def _expanded(full: Mapping, nstates: int) -> list[list]: + """The results of a matrix, item by item, a result written as the one before it + (`matrix.SAME`) written out again.""" + return [matrix.expand(row, nstates) if matrix.SAME in row else row for row in full["values"]] + + +def _results_of(rows: list[list], x: int, at: int, nstates: int) -> list: + """The results of expanded matrix rows in indicator `x` and state `at`, item by item.""" + col = x * nstates + at + return [row[col] for row in rows] + + +def _base_noise(data: Mapping, at: int) -> dict | None: + """The noise of the state at place `at` of a checked diagnostics file: None where it is the + published results, which are exact. A file whose states carry no noise is refused: it never + says which state its noise was measured on.""" + if not fileformat.states_carry_noise(data): + raise ValueError(NO_STATE_NOISE) + state = data["states"][at] + return None if state["kind"] == "published" else state["noise"] + + +#: Why a fixes file needs the noise of the chain's states. +NO_STATE_NOISE = ( + "the diagnostics file says no noise of its states: it was made before noise belonged to a " + "state (SPEC 5.3), so the base's noise cannot be read from it; make it again" +) + + +def build( + title: str, + *, + data: Mapping, + full: Mapping, + base: str, + fixes: list[dict], + results: Mapping[str, Mapping], + interactions: dict | None = None, + built: Iterable[dict] = (), + added_items: Iterable = (), + file: str = chain.FILE, + about: str | None = None, + notes: Iterable[str] = (), +) -> dict: + """A fixes file, counted and checked. + + - `data` and `full`: the checked diagnostics file and its matrix; the items and the base + results come from the matrix, in its order. + - `base`: the id of the state every fix is measured from. + - `fixes`: `fix(...)`, in the order the page lists them; a fix's requirements come first. + - `results`: `{fix id: {indicator: {item: result or None}}}`, built with that fix and what + it requires. An item left out is the result without the fix. + - `interactions`: `interactions(...)`; not checked where it is None. + - `built`: `built_set(...)`. + - `added_items`: `[id, name, place, group, unit]` of items a fix brings that the base + does not hold; the base has no result for them. An added item in a unit no item of the + chain has gets the floor the chain gives a unit with no results: 0, from nothing. + - `file`: the name of the diagnostics file beside it. + + The base's noise is its state's (none where it is the published results); a fix's and a + built set's noise are those of their builds. + """ + ids = [i["id"] for i in data["indicators"]] + states = [s["id"] for s in data["states"]] + if base not in states: + raise ValueError(f"the base {base!r} is no state of {data['title']!r}") + at, nstates = states.index(base), len(states) + rows = [list(r) for r in full["items"]] + [list(r) for r in added_items] + m = len(full["items"]) + index = {r[0]: k for k, r in enumerate(rows)} + if len(index) != len(rows): + raise ValueError("an added item is an item of the base already") + base_noise = _base_noise(data, at) + expanded = _expanded(full, nstates) + base_values = {ind: _results_of(expanded, x, at, nstates) + [None] * (len(rows) - m) for x, ind in enumerate(ids)} + known = [one["id"] for one in fixes] + unknown = set(results) - set(known) + if unknown: + raise ValueError(f"results of fixes that are not given: {sorted(unknown)}") + of = {"file": file, "title": data["title"], "base": base} + if base_noise is not None: + of["base_noise"] = base_noise + f = { + "format": FORMAT, + "of": of, + "title": title, + "about": about, + "item": dict(data["item"]), + **({"indicator": dict(data["indicator"])} if data.get("indicator") else {}), + "indicators": [dict(i) for i in data["indicators"]], + "rule": _rule_of(data, rows[m:]), + "items": rows, + "base": base_values, + "fixes": [], + "interactions": interactions or globals()["interactions"](), + "built": [], + "notes": list(notes), + "made_by": f"sentier-diagnostics {__version__}", + } + ctx = _Base(f) + + def sparse(given: Mapping, where: str, under: Mapping) -> dict: + unknown = set(given) - set(ids) + if unknown: + raise ValueError(f"{where}: results of an indicator the file does not hold: {sorted(unknown)}") + out = {} + for ind in ids: + entries = [] + for item, value in (given.get(ind) or {}).items(): + if item not in index: + raise ValueError(f"{where}: {item!r} is no item of the base, and not an added item") + if value is not None and not rule.is_value(value): + raise ValueError(f"{where}, {ind}, {item}: {value!r} is not a finite number or None") + value = None if value is None else float(value) + k = index[item] + if value != under[ind].get(k, base_values[ind][k]): + entries.append([k, value]) + out[ind] = sorted(entries, key=lambda e: e[0]) + return out + + closures: dict[str, frozenset] = {} + builds = _Builds(ctx, closures, base_noise) + nothing = {ind: {} for ind in ids} + for one in built: + results_of = sparse(one["results"], f"built set {one['id']!r}", nothing) + builds.built[frozenset(one["fixes"])] = _columns(results_of) + builds.noise[frozenset(one["fixes"])] = one.get("noise") or chain.noise() + f["built"].append({ + "id": one["id"], + "label": one["label"], + "fixes": list(one["fixes"]), + "state": one.get("state"), + "results": results_of, + "noise": one.get("noise") or chain.noise(), + }) # fmt: skip + for one in fixes: + requires = list(one.get("requires") or ()) + for needed in requires: + if needed not in closures: + raise ValueError(f"fix {one['id']!r} requires {needed!r}, which is not a fix given before it") + closures[one["id"]] = frozenset(requires).union(*(closures[r] for r in requires)) + before = builds.column(closures[one["id"]]) + if before is None: + raise ValueError( + f"fix {one['id']!r}: no build of what it requires ({', '.join(sorted(closures[one['id']]))}); " + "add a built set of exactly those fixes" + ) + found_claim = one.get("claim") + made = { + "id": one["id"], + "label": one["label"], + "about": one.get("about"), + "kind": one["kind"], + "explained": bool(one.get("explained", True)), + "link": one.get("link"), + "requires": requires, + "claim": claims.published(found_claim, True) if found_claim else None, + "tolerance": one.get("tolerance") or chain.tolerance(), + "noise": one.get("noise") or chain.noise(), + "effect": sparse(results.get(one["id"]) or {}, f"fix {one['id']!r}", before), + "counts": None, + } + builds.entries[one["id"]] = _columns(made["effect"]) + builds.noise[one["id"]] = made["noise"] + made["counts"] = _fix_counts(builds, made, builds.entries[one["id"]], before) + f["fixes"].append(made) + found = errors(f) or against(f, data, full) + if found: + raise ValueError("not a fixes file: " + "; ".join(found[:10])) + return f + + +# ------------------------------------------------------------------------ text and files + + +def _compact(value) -> str: + return json.dumps(value, ensure_ascii=False, allow_nan=False, separators=(",", ":")) + + +def dumps(f: Mapping) -> str: + """The text of a fixes file: one line per item, per base column, per fix and per built set, + so that a diff names them.""" + parts = [] + for key, value in f.items(): + if key in ("items", "fixes", "built") and value: + text = "[\n" + ",\n".join(" " + _compact(v) for v in value) + "\n ]" + elif key == "base" and value: + text = "{\n" + ",\n".join(f" {_compact(k)}:{_compact(v)}" for k, v in value.items()) + "\n }" + else: + text = json.dumps(value, ensure_ascii=False, allow_nan=False) + parts.append(f" {_compact(key)}: {text}") + return "{\n" + ",\n".join(parts) + "\n}\n" + + +def write(f: Mapping, path) -> Path: + """Write a fixes file, checked first; refuse one larger than a page loads.""" + found = errors(f) + if found: + raise ValueError("not a fixes file: " + "; ".join(found[:10])) + text = dumps(f) + size = len(text.encode("utf-8")) + if size > MAX_BYTES: + raise ValueError(f"the fixes file would be {size} bytes, and a page loads {MAX_BYTES} at most") + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + return path + + +def read(path) -> dict: + """A fixes file, read and checked, with the diagnostics file (and its matrix) beside it; + raises ValueError with every error found.""" + path = Path(path) + if path.stat().st_size > MAX_BYTES: + raise ValueError(f"{path}: larger than {MAX_BYTES} bytes") + f = fileformat.load_json(path) + found = errors(f) or beside(f, path.parent) + if found: + raise ValueError(f"{path}: " + "; ".join(found[:10])) + return f + + +def beside(f: Mapping, folder) -> list[str]: + """Everything a checked fixes file and the diagnostics file it names, in `folder`, + disagree on; that file's matrix is used where it is there too.""" + path = Path(folder) / f["of"]["file"] + if not path.is_file(): + return [f"`of` names the diagnostics file {f['of']['file']}, which is not beside it"] + try: + data = fileformat.load_json(path) + except ValueError as error: + return [str(error)] + if fileformat.errors(data): + return [f"{f['of']['file']} is not a diagnostics file a page shows: check it on its own"] + full = None + named = data.get("matrix") + if named and (path.parent / named["file"]).is_file(): + try: + full = matrix.read(path.parent / named["file"], data) + except ValueError as error: + return [str(error)] + return against(f, data, full) + + +def entries_of(f: Mapping) -> list[tuple[str, int]]: + """How many entries each fix's effect holds, the largest first: what the file's size is + made of. A fix to something every item uses holds about one entry per item and indicator.""" + found = [(one["id"], sum(len(v) for v in one["effect"].values())) for one in f["fixes"]] + return sorted(found, key=lambda x: (-x[1], x[0])) + + +def size_note(f: Mapping) -> str: + """One line on what a checked file holds, for `sentier-diagnostics check`.""" + found = entries_of(f) + total = sum(n for _, n in found) + sum(len(v) for b in f["built"] for v in b["results"].values()) + largest = ", ".join(f"{name!r} {n}" for name, n in found[:3]) + return (f"{len(found)} fixes and {len(f['built'])} built sets hold {total} entries over " + f"{len(f['items'])} items; the largest fixes: {largest}") # fmt: skip + + +# ------------------------------------------------------------------------ the base and the builds + + +class _Base: + """The base of a checked file, worked out once: its results, the floor of every item, and + (when a count asks) the class of every item against itself: agrees, near zero or missing.""" + + def __init__(self, f: Mapping): + self.ids = [i["id"] for i in f["indicators"]] + self.items = [r[0] for r in f["items"]] + self.index = {item: k for k, item in enumerate(self.items)} + table = {(x["indicator"], x["unit"]): x["value"] for x in f["rule"]["floors"]} + self.floor = {ind: [table[(ind, r[4])] for r in f["items"]] for ind in self.ids} + self.values = f["base"] + + @cached_property + def kinds(self) -> dict[str, list[str]]: + return { + ind: [rule.classify(v, v, floor=fl) for v, fl in zip(self.values[ind], self.floor[ind])] + for ind in self.ids + } + + @cached_property + def tally(self) -> dict[str, dict[str, int]]: + out = {} + for ind in self.ids: + tally = dict.fromkeys((rule.AGREES, rule.NEAR_ZERO, rule.MISSING), 0) + for kind in self.kinds[ind]: + tally[kind] += 1 + out[ind] = tally + return out + + +class _Builds: + """Every build a file holds, as a column over the base (`{indicator: {item index: result}}`, + the items where it is not the base): a fix with what it requires (its effect laid over the + build of its requirements) and every built set; and the noise of each, as an end of a + comparison (`end`).""" + + def __init__(self, base: _Base, closures: Mapping[str, frozenset], base_noise: Mapping | None = None): + self.base = base + self.closures = closures # fix id: every fix it requires, at any depth + self.entries: dict[str, dict] = {} # fix id: its written results, by indicator + self.built: dict[frozenset, dict] = {} # the fixes of a built set: its results + self.noise: dict[str | frozenset, Mapping] = {} # a fix's id, or a built set's fixes: its noise + # the base as an end: the published results are exact + self.base_end = {"kind": "published"} if base_noise is None else {"kind": "build", "noise": base_noise} + self._memo: dict[frozenset, dict] = {} + + @classmethod + def of(cls, f: Mapping) -> _Builds: + found = cls(_Base(f), _closures(f), f["of"].get("base_noise")) + found.entries = {one["id"]: _columns(one["effect"]) for one in f["fixes"]} + found.built = {frozenset(b["fixes"]): _columns(b["results"]) for b in f["built"]} + found.noise = {one["id"]: one["noise"] for one in f["fixes"]} + found.noise.update((frozenset(b["fixes"]), b["noise"]) for b in f["built"]) + return found + + def end(self, fixes: Iterable[str]) -> tuple[str | frozenset, Mapping] | None: + """The build of exactly these fixes that `column` gives, as an end of a comparison: + `(what names it, {kind, noise})`, the base being exact where it is the published + results; None where the file holds no such build.""" + fixes = frozenset(fixes) + if not fixes: + return "", self.base_end + top = self.top(fixes) + if top is not None and self.column(self.closures[top]) is not None: + return top, {"kind": "build", "noise": self.noise[top]} + if fixes in self.built: + return fixes, {"kind": "build", "noise": self.noise[fixes]} + return None + + def top(self, fixes: frozenset) -> str | None: + """The fix that, with what it requires, is exactly these fixes; None where none is.""" + return next((y for y in fixes if y in self.entries and self.closures[y] | {y} == fixes), None) + + def column(self, fixes: Iterable[str]) -> dict | None: + """The build of exactly these fixes, or None where the file holds none.""" + fixes = frozenset(fixes) + if not fixes: + return {ind: {} for ind in self.base.ids} + if fixes in self._memo: + return self._memo[fixes] + found = None + top = self.top(fixes) + if top is not None: + below = self.column(self.closures[top]) + if below is not None: + found = {ind: {**below[ind], **self.entries[top][ind]} for ind in self.base.ids} + if found is None and fixes in self.built: + found = self.built[fixes] + if found is not None: # never remember "none": a build may be added later + self._memo[fixes] = found + return found + + def covers(self, fixes: frozenset, *, built: bool = True) -> list[frozenset]: + """The builds inside a closed set that hold two fixes or more, in the order a set's + result looks for them: every fix with what it requires, in the order of the file, then + (where `built`) every built set, in the order of the file; never the set itself.""" + found = [self.closures[y] | {y} for y in self.entries if y in fixes and self.closures[y]] + if built: + found += [b for b in self.built if b < fixes] + return found + + def composed(self, fixes: frozenset, ind: str, *, built: bool = True, + how: dict | None = None) -> dict[int, tuple[float | None, bool]]: # fmt: skip + """A closed set's result in one indicator, at every item one of its fixes changes: + `{item index: (result, whether it could be formed)}`. Where one fix changes an item, + its build's result is the set's; where several do and all lie in one build inside the + set (`covers`), the first such build's: measured, not summed. Otherwise the base plus + each fix's difference against what it was measured against. `built=False` leaves the + built sets out: the sum a built set is held to stands on the fixes alone. `how` gathers + what each result stands on, `{item index: ("build", its fixes) or ("sum", the fixes + summed)}`, for its noise (`_item_ends`).""" + touching: dict[int, list[str]] = {} + for y in (y for y in self.closures if y in fixes): # in the order of the file: one sum, one float + for k in self.entries[y][ind]: + touching.setdefault(k, []).append(y) + covers = self.covers(fixes, built=built) + base, mine = self.base.values[ind], {y: self.entries[y][ind] for y in fixes} + under = {y: self.column(self.closures[y])[ind] for y in fixes} # what each was measured against + out = {} + for k, ys in touching.items(): + if len(ys) == 1: + out[k] = (mine[ys[0]][k], True) + stands = (BUILD, self.closures[ys[0]] | {ys[0]}) + else: + here = set(ys) + cover = next((c for c in covers if here <= c), None) + if cover is not None: + column = self.column(cover)[ind] + out[k] = (column[k] if k in column else base[k], True) + stands = (BUILD, cover) + else: + out[k] = _sum(base[k], [(mine[y][k], under[y][k] if k in under[y] else base[k]) for y in ys]) + stands = (SUM, tuple(ys)) + if how is not None: + how[k] = stands + return out + + +def _closures(f: Mapping) -> dict[str, frozenset]: + """Every fix of a checked file, with every fix it requires at any depth.""" + closures: dict[str, frozenset] = {} + for one in f["fixes"]: + closures[one["id"]] = frozenset(one["requires"]).union(*(closures[r] for r in one["requires"])) + return closures + + +def _columns(sparse: Mapping) -> dict[str, dict[int, float | None]]: + """`{indicator: [[item index, result], ...]}` as `{indicator: {item index: result}}`.""" + return {ind: {k: v for k, v in entries} for ind, entries in sparse.items()} + + +def _sum(before: float | None, changes: list[tuple[float | None, float | None]]) -> tuple[float | None, bool]: + """The base plus the effects `(result, what it was measured against)` of several fixes at + one item, and whether a sum could be formed: a side without a result has no difference to add. + + >>> _sum(1.0, [(1.5, 1.0), (2.0, 1.0)]), _sum(1.0, [(1.5, 1.0), (None, 1.0)]) + ((2.5, True), (None, False)) + >>> _sum(None, [(1.0, None), (2.0, None)]), _sum(1.0, [(3.0, 2.0), (1.5, 1.0)]) + ((None, False), (2.5, True)) + """ + if before is None or any(a is None or b is None for a, b in changes): + return None, False + return before + sum(a - b for a, b in changes), True + + +def _level(ends: Iterable[Mapping], indicator: str) -> float | None: + """The noise a comparison is judged with in one indicator: the sum of the noise of its ends + (`{kind, noise}`, SPEC 5.3), each counted as often as it enters it. The published results + are exact and add nothing; an end that did not measure the indicator adds nothing either; + None where no end measured it. For two ends it is `fileformat.noise_of_states`, bit for bit. + + >>> built = lambda by: {"kind": "build", "noise": {"measured": by is not None, "between": by and ["x", "y"], "by": by, "note": None}} + >>> _level([built({"a": 1e-9}), built({"a": 3e-9})], "a"), _level([{"kind": "published"}, built(None)], "a") + (4e-09, None) + >>> _level([built({"a": 0.25})] * 3 + [built({"b": 1.0})], "a") + 0.75 + """ + level = None + for end in ends: + if end["kind"] == "published": + continue + said = end["noise"] + if said["measured"] and indicator in said["by"]: + level = (0.0 if level is None else level) + said["by"][indicator] + return level + + +def _known(end: Mapping, indicator: str) -> bool: + """Whether the noise of an end is known in one indicator: the published results (exact), or + a build that measured it (`fileformat.ends_known`).""" + return end["kind"] == "published" or (end["noise"]["measured"] and indicator in end["noise"]["by"]) + + +def _built(noise: Mapping) -> dict: + return {"kind": "build", "noise": noise} + + +def _side(before, after) -> str: + return "both" if before is None and after is None else "before" if before is None else "after" + + +def _block(base: _Base, ind: str, changed: Mapping[int, float | None], tolerance: float, + noise: float | Mapping[int, float | None] | None, claim: Mapping | None, + under: Mapping[int, float | None] | None = None) -> dict: # fmt: skip + """The counts of one indicator over every item: the column `under` (the base, with the + items `under` names laid over it) against that column with `changed` laid over it, judged + by the rule with `tolerance` and `noise` (one for every item, or one per changed item), and + against `claim` where there is one. Only the + items the two columns name are judged one by one: every other pair is the base against + itself, counted from the base's own classes.""" + under = under or {} + classes = dict.fromkeys(rule.CLASSES, 0) + missing = dict.fromkeys(fileformat.MISSING_SIDES, 0) + ways = dict.fromkeys(rule.DIRECTIONS, 0) + bands = dict.fromkeys(rule.BANDS, 0) + verdicts = dict.fromkeys(claims.VERDICTS, 0) if claim is not None else None + values, floors, kinds = base.values[ind], base.floor[ind], base.kinds[ind] + # the column before, against itself + own = {k: rule.classify(v, v, floor=floors[k]) for k, v in under.items()} + tally = dict(base.tally[ind]) + for k, kind in own.items(): + tally[kinds[k]] -= 1 + tally[kind] += 1 + + def kind_of(k): + return own[k] if k in own else kinds[k] + + for kind, n in tally.items(): + classes[kind] += n + missing["both"] += tally[rule.MISSING] + entries = [] + if claim is not None: + entries = [(e, None if e["items"] == "*" else set(e["items"])) for e in claims.entries_for(claim, ind)] + for k, after in changed.items(): + before, was = (under[k] if k in under else values[k]), kind_of(k) + classes[was] -= 1 + if was == rule.MISSING: + missing["both"] -= 1 + level = noise.get(k) if isinstance(noise, Mapping) else noise + kind, way, band = rule.judge(before, after, floor=floors[k], tolerance=tolerance, noise=level) + classes[kind] += 1 + if kind == rule.MISSING: + missing[_side(before, after)] += 1 + if way: + ways[way] += 1 + if band: + bands[band] += 1 + if verdicts is not None: + item = base.items[k] + covering = [e for e, listed in entries if listed is None or item in listed] + verdicts[claims.verdict(kind, before, after, way, covering)] += 1 + if verdicts is not None: + # the pairs no change touched: the column before against itself + musts = [listed for e, listed in entries if e["expect"] == claims.MUST] + every = any(listed is None for listed in musts) + must_items = set() if every else {base.index[i] for listed in musts for i in listed} + for kind in (rule.AGREES, rule.NEAR_ZERO, rule.MISSING): + untouched = tally[kind] - sum(1 for k in changed if kind_of(k) == kind) + if kind == rule.MISSING: + verdicts[claims.NOT_JUDGED] += untouched + continue + held = untouched if every else sum(1 for k in must_items if kind_of(k) == kind and k not in changed) + verdicts[claims.NOT_MOVED] += held + verdicts[claims.UNCHANGED if kind == rule.AGREES else claims.NOT_JUDGED] += untouched - held + return {"pairs": len(values), "classes": classes, "missing": missing, "directions": ways, "bands": bands, + "verdicts": verdicts, "closer": None, "further": None} # fmt: skip + + +def _fix_counts(builds: _Builds, one: Mapping, effect: Mapping, before: Mapping) -> dict: + """A fix's counts: the build of what it requires against that build with its effect, with + its own tolerance and claim, and the noise of those two builds.""" + tol = one["tolerance"]["value"] + ends = [builds.end(builds.closures[one["id"]])[1], _built(one["noise"])] + return { + ind: _block(builds.base, ind, effect[ind], tol, _level(ends, ind), one["claim"], before[ind]) + for ind in builds.base.ids + } + + +# ------------------------------------------------------------------------ a chosen set + + +def chosen(f: Mapping, ids: Iterable[str]) -> tuple[list[str], list[str]]: + """The set a reader's choice makes: the chosen fixes and every fix they require, in the + order of the file; and the fixes it added because a chosen one requires them (the page + says so).""" + order = {one["id"]: k for k, one in enumerate(f["fixes"])} + ids = list(ids) + if not ids: + raise ValueError("a set holds at least one fix") + for one in ids: + if one not in order: + raise ValueError(f"the file holds no fix {one!r}") + if len(set(ids)) != len(ids): + raise ValueError("a set holds each fix once") + closures = _closures(f) + closed = set(ids).union(*(closures[i] for i in ids)) + names = sorted(closed, key=order.__getitem__) + return names, [x for x in names if x not in ids] + + +def _fixes_of(f: Mapping, names: Iterable[str]) -> list[dict]: + wanted = set(names) + return [one for one in f["fixes"] if one["id"] in wanted] + + +def held_pairs(f: Mapping, ids: Iterable[str]) -> list[dict]: + """The listed pairs of a set: two of its fixes that can interact.""" + names = set(chosen(f, ids)[0]) + return [p for p in f["interactions"]["pairs"] if set(p["fixes"]) <= names] + + +def label(f: Mapping, ids: Iterable[str]) -> str: + """Built, exact or estimated (SPEC 16.5).""" + names = chosen(f, ids)[0] + return _label(f, _Builds.of(f), names) + + +def _label(f: Mapping, builds: _Builds, names: list[str]) -> str: + if builds.column(names) is not None: + return BUILT + if f["interactions"]["checked"] and not held_pairs(f, names): + return EXACT + return ESTIMATED + + +def pair_terms(f: Mapping, ids: Iterable[str]) -> dict: + """How far an estimated set may lie from its sum, to first order: `{value, measured, + no_size, pairs}`, the value being the sum of the sizes of the listed pairs it holds; None + where some were not measured (a pair not measured is never a pair of size 0) or have no + size (`NO_SIZE`), which the counts tell apart.""" + held = held_pairs(f, ids) + said = [p["largest"] for p in held if p["largest"] is not None] + sizes = [x for x in said if x != NO_SIZE] + value = sum(sizes) if len(sizes) == len(held) else None + return {"value": value, "measured": len(said), "no_size": len(said) - len(sizes), "pairs": len(held)} + + +def size_of(summed: Sequence, built: Sequence, floor: Sequence) -> float | str: + """The size of a pair's cross term in one indicator (SPEC 16.6), item by item: the + smallest tolerance under which its build agrees with the sum of its effects, by the rule. + That is the largest relative difference of the two (`rule.relative_difference`) over the + items the rule does not call near zero on both sides; `NO_SIZE` where no tolerance is + enough: an item where the two lie on two sides of zero, one is zero, or one has no + result. A pair's size is the largest over the indicators. + + >>> size_of([1.5, 2.0], [2.0, 2.0], [0.01, 0.01]) + 0.25 + >>> size_of([1.5], [-0.1], [0.01]), size_of([-0.5], [0.0], [0.01]), size_of([1e-9], [-1e-9], [0.01]) + ('no size', 'no size', 0.0) + """ + largest = 0.0 + for s, b, fl in zip(summed, built, floor): + if s is None and b is None: + continue + if s is None or b is None: + return NO_SIZE + if abs(s) <= fl and abs(b) <= fl: + continue + if not ((s > 0 and b > 0) or (s < 0 and b < 0)): + return NO_SIZE + largest = max(largest, rule.relative_difference(s, b)) + return largest + + +def tolerance_of(f: Mapping, ids: Iterable[str]) -> float: + """The tolerance of a set: the largest of its fixes'.""" + return max(one["tolerance"]["value"] for one in _fixes_of(f, chosen(f, ids)[0])) + + +def noise_of(f: Mapping, ids: Iterable[str]) -> dict[str, dict]: + """The noise of a set against the base, per indicator (SPEC 16.5): `{value, known, + builds}`. Each item it changes is judged with the noise of the builds its result stands on + (`_against_base`), never with that of a fix that does not touch it; `value` is the largest + of those, `builds` counts the builds they stand on and `known` those whose noise is known + there (the published results are exact). None where none measured it: not measured is never + none. A set that changes nothing in an indicator stands on no build there.""" + names = chosen(f, ids)[0] + builds = _Builds.of(f) + return _set_noise(builds, _changed(builds, names)[2])[0] + + +def _sum_side(builds: _Builds, fixes: Iterable[str]) -> list[tuple]: + """The builds a sum over these fixes stands on, `(name, end)`: each fix's build and the one + it was measured against (the base, for a fix that requires nothing).""" + return [end for y in fixes for end in (builds.end(builds.closures[y]), (y, _built(builds.noise[y])))] + + +def _against_base(builds: _Builds, stands: tuple) -> list[tuple]: + """The builds a set's result at one item and the base result there stand on, `(name, end)`: + the build the result is taken from, and the base; for a sum, the builds of the sum alone, + since the base it starts from is the base it is compared with, and cancels.""" + kind, what = stands + if kind == BUILD: + return [("", builds.base_end), builds.end(what)] + return _sum_side(builds, what) + + +def _set_noise(builds: _Builds, how: Mapping[str, Mapping[int, tuple]]) -> tuple[dict, dict]: + """A set's noise per indicator, `{value, known, builds}`, and the noise each item it changes + is judged with, `{indicator: {item index: noise or None}}`.""" + summary, per_item = {}, {} + for ind in builds.base.ids: + levels, every, seen = {}, {}, {} + for k, stands in how[ind].items(): + if stands not in seen: # most items share what they stand on: one fix's build + ends = _against_base(builds, stands) + seen[stands] = _level([end for _, end in ends], ind) + every.update(ends) + levels[k] = seen[stands] + found = [lv for lv in levels.values() if lv is not None] + summary[ind] = {"value": max(found) if found else None, + "known": sum(_known(end, ind) for end in every.values()), "builds": len(every)} # fmt: skip + per_item[ind] = levels + return summary, per_item + + +def _changed(builds: _Builds, names: list[str]) -> tuple[dict, dict, dict]: + """The items a closed set changes, per indicator, with its result at each; the items + where no sum can be formed; and what each result stands on (`_Builds.composed`).""" + column = builds.column(names) + if column is not None: + whole = frozenset(names) + return (column, {ind: [] for ind in builds.base.ids}, + {ind: dict.fromkeys(column[ind], (BUILD, whole)) for ind in builds.base.ids}) # fmt: skip + changed, cannot, how = {}, {}, {} + for ind in builds.base.ids: + how[ind] = {} + cells = builds.composed(frozenset(names), ind, how=how[ind]) + changed[ind] = {k: v for k, (v, _) in cells.items()} + cannot[ind] = sorted(k for k, (_, ok) in cells.items() if not ok) + return changed, cannot, how + + +def compose(f: Mapping, ids: Iterable[str]) -> dict: + """What a page draws for a set: its fixes in the order of the file (each with its own + claim and counts), the fixes added because a chosen one requires them, its label, the + listed pairs it holds and their pair terms, its tolerance and noise (`noise_of`), the noise + each item it changes is judged with (`item_noise`, `{indicator: {item index: noise or + None}}`), and its result for every item, `{indicator: [result or None, ...]}` in the order + of `items`. `cannot_add` + names, per indicator, the item indices whose result is None because no sum can be formed + there, so that it reads apart from a missing result. A set has no verdict.""" + names, added = chosen(f, ids) + builds = _Builds.of(f) + changed, cannot, how = _changed(builds, names) + noise, item_noise = _set_noise(builds, how) + results = {} + for ind in builds.base.ids: + column = list(builds.base.values[ind]) + for k, value in changed[ind].items(): + column[k] = value + results[ind] = column + return { + "fixes": [{k: one[k] for k in ("id", "label", "requires", "claim", "counts")} for one in _fixes_of(f, names)], + "added": added, + "label": _label(f, builds, names), + "pairs": held_pairs(f, names), + "pair_terms": pair_terms(f, names), + "tolerance": tolerance_of(f, names), + "noise": noise, + "item_noise": item_noise, + "results": results, + "cannot_add": cannot, + } + + +def counts(f: Mapping, ids: Iterable[str]) -> dict: + """The counts of a set against the base, per indicator, with its label beside them: an + estimated set's counts are estimates ("estimated: 153 would differ"). No verdicts.""" + names = chosen(f, ids)[0] + builds = _Builds.of(f) + changed, _, how = _changed(builds, names) + tol, item_noise = tolerance_of(f, names), _set_noise(builds, how)[1] + return { + "label": _label(f, builds, names), + "by": {ind: _block(builds.base, ind, changed[ind], tol, item_noise[ind], None) for ind in builds.base.ids}, + } + + +# ------------------------------------------------------------------------ the check + + +def errors(f) -> list[str]: + """Everything wrong with a fixes file on its own; nothing for one a page can load. It + never raises: a value it cannot read is a fault like any other.""" + found: list[str] = [] + try: + return _errors(f, found) + except Exception as error: # noqa: BLE001 - the backstop of a check that must not raise + return [*found, f"{fileformat.STOPPED} ({type(error).__name__}: {str(error)[:200]})"] + + +def _errors(f, found: list[str]) -> list[str]: + if not isinstance(f, Mapping): + return ["not a fixes file: not an object"] + said = f.get("format") + if said != FORMAT: + if isinstance(said, str) and said.startswith(FAMILY): + return [f"format {said!r}: this reader knows {FORMAT} only"] + return [f"not a fixes file of format {FORMAT}"] + found += [f"a key this format does not know: {key}" for key in sorted(set(f) - KEYS - OPTIONAL_KEYS)] + found += [f"a key this format requires: {key}" for key in sorted(KEYS - set(f))] + if found: + return found + of = f["of"] + if not ( + isinstance(of, dict) + and OF_KEYS <= set(of) <= OF_KEYS | OF_OPTIONAL + and isinstance(of["file"], str) + and fileformat.FILE_NAME.fullmatch(of["file"]) + and fileformat.is_text(of["title"], 120) + and fileformat.is_text(of["base"], fileformat.MAX_ID) + ): + found.append( + "`of` is not {file, title, base}, with `base_noise` where the base is a build: a file name, that " + "file's title and a state's id" + ) + if not fileformat.is_text(f["title"], 120) or not fileformat.is_optional(f["about"]): + found.append("`title` is not a short text, or `about` not a text") + for key in ("item", "indicator"): + word = f.get(key) + if (word is not None or key == "item") and not ( + isinstance(word, dict) and set(word) == {"one", "many"} + and all(fileformat.is_text(w, 30) for w in word.values()) + ): # fmt: skip + found.append(f"`{key}` is not {{one, many}}") + if not fileformat.is_text(f["made_by"], 60): + found.append("`made_by` is not a short text") + notes = f["notes"] + if not isinstance(notes, list) or len(notes) > fileformat.MAX_NOTES or not all(map(fileformat.is_text, notes)): + found.append(f"`notes` is not a list of at most {fileformat.MAX_NOTES} texts") + indicators = fileformat._indicators(f, found) + floors = _rule(f, found, indicators) + if not indicators or floors is None or not _items(f, found, indicators, floors): + return found + if isinstance(of, dict) and "base_noise" in of: + found += fileformat.noise_errors(of["base_noise"], "`of.base_noise`", indicators) + n = len(f["items"]) + said_base = f["base"] + if not ( + isinstance(said_base, dict) + and set(said_base) == set(indicators) + and all(isinstance(said_base[i], list) and len(said_base[i]) == n + and all(v is None or rule.is_value(v) for v in said_base[i]) for i in indicators) + ): # fmt: skip + found.append("`base` is not one result or null per item, for every indicator") + if found: + return found # nothing can be judged against a base that cannot be read + base = _Base(f) + closures = _requirements(f, found) + if closures is None: + return found + builds = _Builds(base, closures, f["of"].get("base_noise")) + built = _built_sets(f, found, base, builds) + if built is None or not _fixes(f, found, base, builds): + return found + pairs = _interactions(f, found, closures, n) + if pairs is not None: + _built_checks(f, found, builds, built, pairs) + return found + + +def _rule(f, found, indicators) -> dict | None: + said = f["rule"] + if not isinstance(said, dict) or set(said) != RULE_KEYS: + found.append(f"`rule` is not {{{', '.join(sorted(RULE_KEYS))}}}") + return None + if not fileformat.is_share(said["floor_factor"]): + found.append("`rule`: the floor factor is not a share") + if not fileformat.is_text(said["floors_from"], fileformat.MAX_ID): + found.append("`rule`: the floors come from a state, named by its id") + if said["beyond"] != rule.BEYOND_FACTOR: + found.append(f"`rule.beyond` is {rule.BEYOND_FACTOR:g}: one rule for every file") + if not isinstance(said["floors"], list): + found.append("`rule.floors` is not a list") + return None + table = {} + for one in said["floors"]: + if not ( + isinstance(one, dict) + and set(one) == fileformat.FLOOR_KEYS + and one["indicator"] in indicators + and fileformat.is_optional(one["unit"], fileformat.MAX_WORD) + and fileformat.is_share(one["value"]) + and one["basis"] in fileformat.FLOOR_BASES + and fileformat.is_count(one["n"]) + ): + found.append("a floor is not {indicator, unit, value, basis, n} of a known indicator") + return None + key = (one["indicator"], one["unit"]) + if key in table: + found.append(f"two floors for {one['indicator']!r} in unit {one['unit']!r}") + table[key] = one["value"] + return table + + +def _items(f, found, indicators, floors) -> bool: + rows = f["items"] + if not isinstance(rows, list) or not 0 < len(rows) <= MAX_ITEMS: + found.append(f"`items` is not a list of 1 to {MAX_ITEMS} items") + return False + seen = set() + for at, row in enumerate(rows): + if not matrix._is_row(row): + found.append(f"item {at}: not [id, name, place, group, unit]") + return False + if row[0] in seen: + found.append(f"item {row[0]!r} is in the file twice") + return False + seen.add(row[0]) + units = {row[4] for row in rows} + for ind in indicators: + for unit in sorted(units, key=lambda u: (u is None, u or "")): + if (ind, unit) not in floors: + found.append(f"no floor for {ind!r} in unit {unit!r}") + return False + return True + + +def _sparse(said, where: str, base: _Base, found: list[str], under: Mapping | None = None) -> dict | None: + """An effect, or the results of a built set: per indicator, the items whose result is not + the one it was measured against (`under`, laid over the base), `[[item index, result or + null], ...]`.""" + if not isinstance(said, dict) or set(said) != set(base.ids): + found.append(f"{where}: not one list per indicator of the file") + return None + n = len(base.items) + out = {} + for ind in base.ids: + entries = said[ind] + if not isinstance(entries, list) or len(entries) > n: + found.append(f"{where}, {ind}: not a list of at most one entry per item") + return None + column = {} + against_ = (under or {}).get(ind) or {} + for entry in entries: + if not ( + isinstance(entry, list) + and len(entry) == 2 + and fileformat.is_count(entry[0]) + and (entry[1] is None or rule.is_value(entry[1])) + ): + found.append(f"{where}, {ind}: an entry is not [item index, result or null]") + return None + k, value = entry + if k >= n: + found.append(f"{where}, {ind}: names item {k}, and the file holds {n}") + return None + if k in column: + found.append(f"{where}, {ind}: holds item {base.items[k]!r} twice") + return None + if value == (against_[k] if k in against_ else base.values[ind][k]): + found.append( + f"{where}, {ind}: item {base.items[k]!r} is written with the result it was measured " + "against, as if it changed" + ) + return None + column[k] = value + out[ind] = column + return out + + +def _requirements(f, found) -> dict[str, frozenset] | None: + """Every fix's id and what it requires, at any depth. A fix's requirements come before it + in `fixes`, so that no fix requires itself through others.""" + said = f["fixes"] + if not isinstance(said, list) or not 0 < len(said) <= MAX_FIXES: + found.append(f"`fixes` is not a list of 1 to {MAX_FIXES} fixes") + return None + for one in said: + if not isinstance(one, dict) or set(one) != FIX_KEYS or not fileformat.is_text(one["id"], fileformat.MAX_ID): + found.append(f"a fix is not {{{', '.join(sorted(FIX_KEYS))}}}") + return None + every = [one["id"] for one in said] + if len(set(every)) != len(every): + found.append("two fixes have the same id") + return None + closures: dict[str, frozenset] = {} + for one in said: + where = f"fix {one['id']!r}" + needs = one["requires"] + if not (isinstance(needs, list) and all(isinstance(x, str) for x in needs)): + found.append(f"{where}: `requires` is not a list of the ids of fixes") + return None + if len(set(needs)) != len(needs): + found.append(f"{where}: requires a fix twice") + return None + for x in needs: + if x == one["id"]: + found.append(f"{where}: requires itself") + return None + if x not in every: + found.append(f"{where}: requires {x!r}, and the file holds no such fix") + return None + if x not in closures: + found.append(f"{where}: requires {x!r}, which comes after it: a fix's requirements come first") + return None + closures[one["id"]] = frozenset(needs).union(*(closures[x] for x in needs)) + return closures + + +def _built_sets(f, found, base: _Base, builds: _Builds) -> list | None: + """Every built set, its shape and results; added to `builds`. None where one is wrong: + a fix measured against a build the file does not hold cannot be checked.""" + said = f["built"] + if not isinstance(said, list) or len(said) > MAX_BUILT: + found.append(f"`built` is not a list of at most {MAX_BUILT} sets") + return None + closures = builds.closures + out, ids, sets, before = [], set(), set(), len(found) + for one in said: + if not isinstance(one, dict) or set(one) != BUILT_KEYS or not fileformat.is_text(one["id"], fileformat.MAX_ID): + found.append(f"a built set is not {{{', '.join(sorted(BUILT_KEYS))}}}") + return None + where = f"built set {one['id']!r}" + if one["id"] in ids: + found.append(f"two built sets have the id {one['id']!r}") + ids.add(one["id"]) + if not fileformat.is_text(one["label"], fileformat.MAX_LABEL): + found.append(f"{where}: `label` is not a short text") + if not fileformat.is_optional(one["state"], fileformat.MAX_ID): + found.append(f"{where}: `state` is not the id of a state, or null") + named = one["fixes"] + if not (isinstance(named, list) and all(isinstance(x, str) for x in named)): + found.append(f"{where}: `fixes` is not a list of the ids of fixes") + continue + unknown = [x for x in named if x not in closures] + if unknown: + found.append(f"{where}: the file holds no fix {unknown[0]!r}") + continue + if len(set(named)) != len(named): + found.append(f"{where}: holds a fix twice") + continue + if len(named) < 2: + found.append(f"{where}: holds fewer than two fixes: one fix is built already") + continue + key = frozenset(named) + lacking = sorted((x, r) for x in named for r in closures[x] - key) + if lacking: + found.append(f"{where}: holds {lacking[0][0]!r} without {lacking[0][1]!r}, which it requires") + continue + top = next((y for y in named if closures[y] | {y} == key), None) + if top is not None: + found.append(f"{where}: is built already: it is fix {top!r} with what it requires") + continue + if key in sets: + found.append(f"{where}: holds the same fixes as another built set") + sets.add(key) + noise = fileformat.noise_errors(one["noise"], where, base.ids) + if noise: + found += noise + continue + results = _sparse(one["results"], f"{where}: its results", base, found) + if results is None: + continue + builds.built[key] = results + builds.noise[key] = one["noise"] + out.append((one, key, results)) + return None if len(found) > before else out + + +def _fixes(f, found, base: _Base, builds: _Builds) -> bool: + """Every fix: its texts, claim, tolerance and noise; its effect, measured against the build + of what it requires; and its counts and verdicts, worked out again.""" + for one in f["fixes"]: + where = f"fix {one['id']!r}" + if not fileformat.is_text(one["label"], fileformat.MAX_LABEL) or not fileformat.is_optional(one["about"]): + found.append(f"{where}: `label` or `about` is not a text") + if not fileformat.is_text(one["kind"], fileformat.MAX_WORD) or not isinstance(one["explained"], bool): + found.append(f"{where}: a fix has a `kind` and says whether a cause explains it") + if not fileformat.is_link(one["link"]): + found.append(f"{where}: `link` is not an https address") + if one["claim"] is not None: + shape = fileformat.claim_errors(one["claim"], where, base.ids, "items") + if shape: + found += shape + return False + if any(e["items"] is None for e in one["claim"]["scope"]): + found.append(f"{where}: a fix's claim names its items: a page works out each item's verdict from it") + return False + unknown = {i for e in one["claim"]["scope"] if e["items"] != "*" for i in e["items"]} - set(base.index) + if unknown: + found.append(f"{where}: its claim names {len(unknown)} items the file does not hold") + return False + tol = one["tolerance"] + if not ( + isinstance(tol, dict) + and set(tol) == {"value", "basis", "note"} + and fileformat.is_share(tol["value"]) + and tol["basis"] in fileformat.TOLERANCE_BASES + and fileformat.is_optional(tol["note"]) + ): + found.append(f"{where}: `tolerance` is not {{value, basis, note}}") + return False + noise = fileformat.noise_errors(one["noise"], where, base.ids) + if noise: + found += noise + return False + builds.noise[one["id"]] = one["noise"] + before = builds.column(builds.closures[one["id"]]) + if before is None: + needs = ", ".join(repr(x) for x in sorted(builds.closures[one["id"]])) + found.append( + f"{where}: it requires {needs}, and the file holds no build of them together (a built set of " + "exactly those fixes): its effect has nothing to be measured against" + ) + return False + effect = _sparse(one["effect"], f"{where}: its effect", base, found, before) + if effect is None: + return False + builds.entries[one["id"]] = effect + written = one["counts"] + if not isinstance(written, dict) or set(written) != set(base.ids): + found.append(f"{where}: `counts` is not one block per indicator") + return False + shape = [e for ind in base.ids for e in fileformat._block(written[ind], f"{where}, counts of {ind!r}", one)] + shape += [f"{where}, counts of {ind!r}: `{key}` is null: a fix has no published results to come closer to" + for ind in base.ids for key in ("closer", "further") if written[ind][key] is not None] # fmt: skip + if shape: + found += shape + continue + again = _fix_counts(builds, one, effect, before) + for ind in base.ids: + for key, value in again[ind].items(): + if written[ind][key] != value: + found.append(f"{where}, counts of {ind!r}: `{key}` is not what its effect gives") + return True + + +def _interactions(f, found, closures: Mapping[str, frozenset], n: int) -> dict[frozenset, dict] | None: + """The pairs of fixes that can interact, each once, by their two ids.""" + said = f["interactions"] + if not ( + isinstance(said, dict) + and set(said) == INTERACTION_KEYS + and isinstance(said["checked"], bool) + and isinstance(said["pairs"], list) + ): + found.append("`interactions` is not {checked, method, by, pairs}") + return None + if said["checked"]: + if not fileformat.is_text(said["method"]): + found.append("`interactions`: a check says what it did (`method`)") + elif IDENTIFIER.search(said["method"]): + found.append("`interactions.method` holds an identifier of the model: say it in words") + if said["by"] is None: + found.append("`interactions`: a check says who or what ran it (`by`)") + else: + found += fileformat.origin_errors(said["by"], "`interactions`") + else: + if said["method"] is not None or said["by"] is not None: + found.append("`interactions`: no check ran, so it has no `method` and no `by`") + if said["pairs"]: + found.append( + "`interactions`: pairs are listed and `checked` is false: a pair that is not listed would " + "read as cleared" + ) + return None + if len(said["pairs"]) > MAX_FIXES * (MAX_FIXES - 1) // 2: + found.append("`interactions.pairs` lists more pairs than fixes can make") + return None + pairs: dict[frozenset, dict] = {} + for one in said["pairs"]: + if not isinstance(one, dict) or set(one) != PAIR_KEYS: + found.append( + "a pair is not {fixes, kind, reach, largest}: it holds no text, so that nothing of the model " + "goes into it" + ) + return None + named = one["fixes"] + if not (isinstance(named, list) and len(named) == 2 and all(isinstance(x, str) for x in named)): + found.append("a pair does not name two fixes") + return None + a, b = named + where = f"the pair {a!r} and {b!r}" + unknown = [x for x in named if x not in closures] + if unknown: + found.append(f"{where}: the file holds no fix {unknown[0]!r}") + continue + if a == b: + found.append(f"{where}: names one fix twice") + continue + if a in closures[b] or b in closures[a]: + found.append(f"{where}: one requires the other, so its effect is measured with the other applied") + continue + if one["kind"] not in PAIR_KINDS: + found.append(f"{where}: `kind` is not one of {', '.join(PAIR_KINDS)}") + if one["reach"] is not None and not (fileformat.is_count(one["reach"]) and 0 < one["reach"] <= n): + found.append(f"{where}: `reach` is not a count of 1 to {n} items, or null") + if one["largest"] not in (None, NO_SIZE) and not fileformat.is_share(one["largest"]): + found.append( + f"{where}: `largest` is not a share, {NO_SIZE!r} where no tolerance is enough, or null where it was " + "not measured" + ) + key = frozenset(named) + if key in pairs: + found.append(f"{where}: is listed twice") + pairs[key] = one + return pairs + + +def _built_checks(f, found, builds: _Builds, built: list, pairs: Mapping[frozenset, dict]) -> None: + """A built set the file calls independent must be the sum of its effects; a built pair + whose cross term the file measured must lie within it; a pair with no size must be built, + and its build must have none.""" + if not f["interactions"]["checked"]: + return + by_id = {one["id"]: one for one in f["fixes"]} + for key, one in pairs.items(): + a, b = one["fixes"] + if one["largest"] == NO_SIZE and builds.column(key | builds.closures[a] | builds.closures[b]) is None: + found.append( + f"the pair {a!r} and {b!r}: has no size, so no set holding it can be drawn from its sum: the " + "file holds a build of the two with what they require (a built set)" + ) + for one, key, results in built: + where = f"built set {one['id']!r}" + held = [p for p in pairs if p <= key] + if not held: + found += _disagreements(builds, where, key, by_id, one, results) + elif held == [key] and pairs[key]["largest"] == NO_SIZE: + found += _sized(builds, where, key, results) + elif held == [key] and pairs[key]["largest"] is not None: + found += _disagreements(builds, where, key, by_id, one, results, pairs[key]["largest"]) + + +def _sized(builds: _Builds, where: str, key: frozenset, results: Mapping) -> list[str]: + """A built pair the file says has no size must have none: where some tolerance makes its + build agree with its sum, that tolerance is its size, and the file says it.""" + base = builds.base + sizes = [] + for ind in base.ids: + cells, values = builds.composed(key, ind, built=False), base.values[ind] + items = sorted(set(cells) | set(results[ind])) + summed = [cells[k][0] if k in cells else values[k] for k in items] + if not all(cells[k][1] for k in items if k in cells): + return [] # no sum at some item: nothing to measure against + size = size_of(summed, [results[ind].get(k, values[k]) for k in items], [base.floor[ind][k] for k in items]) + if size == NO_SIZE: + return [] + sizes.append(size) + size = _number(max(sizes)) + return [f"{where}: the file says its pair has no size, and its build lies within {size} of the sum of its effects"] + + +#: How many disagreements of one built set are named before the rest are counted. +NAMED = 5 + + +def _number(value: float | None) -> str: + """A result in an error, to ten significant digits: the sum of 1.0006 and 1.0007 over + 1.0 reads 1.0013, not 1.0012999999999999. + + >>> _number(1.0012999999999999), _number(None), _number(-2.5e-12) + ('1.0013', 'null', '-2.5e-12') + """ + return "null" if value is None else f"{value:.10g}" + + +def _disagreements(builds: _Builds, where: str, key: frozenset, by_id: Mapping, one: Mapping, results: Mapping, + largest: float | None = None) -> list[str]: # fmt: skip + """Where a built set is further from the sum of its effects than the file says, by the + rule, with the largest tolerance of its fixes and, item by item, the noise of the builds + the two sides stand on: the built set's, and the build the sum takes the item from, or the + base with each fix's build and what it was measured against where it adds the fixes that + touch the item, or the base alone where none does.""" + base = builds.base + own, from_base = _built(one["noise"]), [builds.base_end] + tol = max(by_id[x]["tolerance"]["value"] for x in key) + if largest is not None: + tol = max(tol, largest) + found, more = [], 0 + for ind in base.ids: + how: dict[int, tuple] = {} + cells = builds.composed(key, ind, built=False, how=how) + for k in sorted(set(cells) | set(results[ind])): + before = base.values[ind][k] + summed, ok = cells.get(k, (before, True)) + if not ok: + continue # no sum to hold the build to + got = results[ind].get(k, before) + if k not in how: + side = from_base + elif how[k][0] == BUILD: + side = [builds.end(how[k][1])[1]] + else: + side = from_base + [end for _, end in _sum_side(builds, how[k][1])] + level = _level([own, *side], ind) + kind = rule.classify(summed, got, floor=base.floor[ind][k], tolerance=tol, noise=level) + if kind in claims.STILL or (kind == rule.MISSING and summed is None and got is None): + continue + if len(found) < NAMED: + said = ("the file calls its fixes independent, and they are not" if largest is None + else f"the file measured their cross term at {largest:g} at most, and it is larger") # fmt: skip + found.append( + f"{where}, item {base.items[k]!r}, {ind}: built {_number(got)}, the sum of its effects is " + f"{_number(summed)} " + f"({kind}); {said}" + ) + else: + more += 1 + if more: + found.append(f"{where}: and {more} more pairs where the build is not what the file says") + return found + + +# ------------------------------------------------------------------------ against the diagnostics file + + +def against(f: Mapping, data: Mapping, full: Mapping | None = None) -> list[str]: + """Everything a checked fixes file and its checked diagnostics file disagree on. Its items + and base results are held to the matrix `full` where it is given, and to the listed items + where it is not; so is a built set that names a state of the chain.""" + try: + return _against(f, data, full) + except Exception as error: # noqa: BLE001 - a backstop, as in `errors` + return [f"{fileformat.STOPPED} ({type(error).__name__}: {str(error)[:200]})"] + + +def _against(f: Mapping, data: Mapping, full: Mapping | None) -> list[str]: + found = [] + of = f["of"] + if of["title"] != data["title"]: + found.append("it belongs to another file: `of.title` is not the diagnostics file's title") + states = [s["id"] for s in data["states"]] + if of["base"] not in states: + return [*found, f"the base {of['base']!r} is no state of the diagnostics file"] + if not fileformat.states_carry_noise(data): + return [*found, NO_STATE_NOISE] + base_state = data["states"][states.index(of["base"])] + if base_state["kind"] == "published": + if "base_noise" in of: + found.append("the base is the published results, which are exact: `of.base_noise` is left out") + elif "base_noise" not in of: + found.append("the base is a build: `of.base_noise` is its noise, as the diagnostics file says it") + elif of["base_noise"] != base_state["noise"]: + found.append(f"`of.base_noise` is not the noise the diagnostics file gives state {of['base']!r}") + if f["indicators"] != data["indicators"]: + found.append("its indicators differ from the diagnostics file's") + if f["item"] != data["item"] or f.get("indicator") != data.get("indicator"): + found.append("its words for an item or an indicator differ from the diagnostics file's") + m = len(full["items"]) if full is not None else data["items_count"]["all"] + if f["rule"] != _rule_of(data, f["items"][m:]): + found.append( + "its floors differ from the diagnostics file's, with a floor of 0 from nothing for each unit only " + "an added item has" + ) + for one in f["built"]: + later = one["state"] in states and states.index(one["state"]) > states.index(of["base"]) + if one["state"] is not None and not later: + found.append(f"built set {one['id']!r}: `state` {one['state']!r} is not a state of the chain after the base") + if found: + return found + ids = [i["id"] for i in data["indicators"]] + at, nstates = states.index(of["base"]), len(states) + if full is not None: + if [list(r) for r in f["items"][:m]] != [list(r) for r in full["items"]]: + return ["its items are not the matrix's, in the matrix's order, before any item a fix adds"] + expanded = _expanded(full, nstates) + for x, ind in enumerate(ids): + if _results_of(expanded, x, at, nstates) != f["base"][ind][:m]: + found.append(f"the base results in {ind!r} are not the matrix's results of state {of['base']!r}") + found += _added_items(f, m) + found += _states(f, data, lambda k, x, s: expanded[k][x * nstates + states.index(s)] if k < m else None, + range(len(f["items"]))) # fmt: skip + return found + if len(f["items"]) < data["items_count"]["all"]: + found.append(f"it holds {len(f['items'])} items, and the diagnostics file counts {data['items_count']['all']}") + index = {row[0]: k for k, row in enumerate(f["items"])} + listed = {} + for item in data["items"]: + k = index.get(item["id"]) + if k is None: + found.append(f"the listed item {item['id']!r} is not in the fixes file") + continue + listed[k] = item + if f["items"][k][1:] != [item["name"], item["place"], item["group"], item["unit"]]: + found.append(f"item {item['id']!r}: its name, place, group or unit differ from the diagnostics file") + for ind in ids: + if item["values"][ind][at] != f["base"][ind][k]: + found.append(f"item {item['id']!r}, {ind}: the base result is not the diagnostics file's") + found += _states(f, data, lambda k, x, s: listed[k]["values"][ids[x]][states.index(s)], sorted(listed)) + return found + + +def _added_items(f: Mapping, m: int) -> list[str]: + """The items after the matrix's: a fix brings them, so the base has no result for them and + a fix or a built set gives one.""" + found = [] + given = set() + for one in [*f["fixes"], *f["built"]]: + for entries in (one["effect"] if "effect" in one else one["results"]).values(): + given.update(k for k, v in entries if k >= m and v is not None) + for k in range(m, len(f["items"])): + name = f["items"][k][0] + if any(f["base"][ind][k] is not None for ind in f["base"]): + found.append(f"item {name!r} is not in the matrix, and the base has a result for it") + elif k not in given: + found.append(f"item {name!r} is in no state of the chain, and no fix gives it a result") + return found + + +def _states(f: Mapping, data: Mapping, cell, items) -> list[str]: + """A built set that names a state of the chain is two builds of one thing: by the rule, + with the largest tolerance of its fixes and the noise of the two builds, its own and the + state's (none where that is the published results).""" + found = [] + by_id = {one["id"]: one for one in f["fixes"]} + floors = _Base(f).floor + of_state = {s["id"]: s for s in data["states"]} + for one in f["built"]: + if one["state"] is None: + continue + where = f"built set {one['id']!r}" + state = of_state[one["state"]] + ends = [_built(one["noise"]), state if state["kind"] == "published" else _built(state["noise"])] + tol = max(by_id[x]["tolerance"]["value"] for x in one["fixes"]) + results = _columns(one["results"]) + named = 0 + for x, ind in enumerate(f["base"]): + level = _level(ends, ind) + for k in items: + got = results[ind].get(k, f["base"][ind][k]) + there = cell(k, x, one["state"]) + kind = rule.classify(there, got, floor=floors[ind][k], tolerance=tol, noise=level) + if kind in claims.STILL or (kind == rule.MISSING and there is None and got is None): + continue + named += 1 + if named <= NAMED: + found.append( + f"{where}, item {f['items'][k][0]!r}, {ind}: built {_number(got)}, and state " + f"{one['state']!r} of the chain, which it says it is, holds {_number(there)} ({kind})" + ) + if named > NAMED: + found.append(f"{where}: and {named - NAMED} more pairs where it is not state {one['state']!r}") + return found diff --git a/tests/test_fixes.py b/tests/test_fixes.py new file mode 100644 index 0000000..3159ff3 --- /dev/null +++ b/tests/test_fixes.py @@ -0,0 +1,1079 @@ +"""Fixes against one base (SPEC 16): the file, its check, and what a page draws for a set. +Every value here is made up.""" + +from __future__ import annotations + +import contextlib +import io +import itertools +import json +import tempfile +import unittest +from pathlib import Path + +from sentier_diagnostics import chain, claims, cli, fixes, matrix, rule + +from . import helpers +from .helpers import INDICATORS, A, B + +FLOORS = {(A, "kg"): 0.01, (B, "kg"): 0.01} + + +def chain_and_matrix(last=None, release=None, noise=None): + """A chain of two states and its matrix; the base is the second state. Item p3 moved in + the step, so it is the one listed item. With `release`, a third state after the base. + `noise` gives a built state its noise, by its id.""" + noise = noise or {} + s1 = last or {A: {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0}, B: {"p1": 3.0, "p2": 5.0, "p3": 7.0, "p4": 1.0}} + s0 = {A: {**s1[A], "p3": 4.0}, B: dict(s1[B])} + items = sorted({i for column in s0.values() for i in column} | {i for column in s1.values() for i in column}) + states = [chain.state("s0", "Before", kind="published"), chain.state("s1", "After", noise=noise.get("s1"))] + steps, values = [chain.step("one", "One step", kind="fix")], {"s0": s0, "s1": s1} + if release is not None: + states.append(chain.state("s2", "Release", noise=noise.get("s2"))) + steps.append(chain.step("two", "Every fix", kind="fix")) + values["s2"] = release + return chain.build_with_matrix( + "Two states", + indicators=INDICATORS, + states=states, + steps=steps, + values=values, + units=dict.fromkeys(items, "kg"), + floors=FLOORS, + ) + + +def checked_pairs(*pairs): + return fixes.interactions(True, method="Compared what each fix changes in the model.", + by=claims.origin("code", "pair check"), pairs=pairs) # fmt: skip + + +#: The base with every fix of `made`: a and b at p1, c at p3. +RELEASE = {A: {"p1": 1.0013, "p2": 2.0, "p3": 2.0, "p4": 0.0}, B: {"p1": 3.0, "p2": 5.0, "p3": 6.0, "p4": 1.0}} + + +def made(*, noise_a=None, noise_b=None, built_ab=1.0013, interactions="listed", built=True, release=False, + state_noise=None, noise_ab=None, noise_all=None): # fmt: skip + """Three fixes against the base: + + - a and b each move p1 in A by less than their tolerance (1e-3): 1.0 to 1.0006 and 1.0007; + - c moves p3 down in both indicators, as its claim says it must; + - b and c can interact (a listed pair); the set of a and b was built. + + With `release`, the chain has a third state, the base with every fix, and a built set of + every fix says it is that state. `state_noise` gives the chain's built states their noise + (s1 is the base), `noise_ab` and `noise_all` the built sets theirs.""" + data, full = chain_and_matrix(release=RELEASE if release else None, noise=state_noise) + claim_c = claims.claim("p3 goes down.", [claims.scope([A], {"p3"}, "down", 0.5, claims.MUST)], + by=claims.origin("person", "A. Author")) # fmt: skip + pairs = {"listed": checked_pairs(fixes.pair("b", "c", fixes.TECHNOSPHERE)), "none": checked_pairs(), + "unchecked": fixes.interactions(False)}[interactions] # fmt: skip + sets = [fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": built_ab}}, noise=noise_ab)] if built else [] + if release: + sets.append(fixes.built_set("all", "Every fix", ["a", "b", "c"], RELEASE, state="s2", noise=noise_all)) + f = fixes.build( + "Fixes of two states", + data=data, + full=full, + base="s1", + fixes=[ + fixes.fix("a", "Fix A", kind="mapping", tolerance=chain.tolerance(1e-3), noise=noise_a), + fixes.fix("b", "Fix B", kind="mapping", tolerance=chain.tolerance(1e-3), noise=noise_b), + fixes.fix("c", "Fix C", kind="amount", claim=claim_c), + ], + results={"a": {A: {"p1": 1.0006}}, "b": {A: {"p1": 1.0007}}, "c": {A: {"p3": 2.0}, B: {"p3": 6.0}}}, + interactions=pairs, + built=sets, + ) + return f, data, full + + +def pair(a, b, kind=fixes.TECHNOSPHERE, largest=None, reach=None): + return fixes.pair(a, b, kind, reach=reach, largest=largest) + + +def at(f, item): + return [row[0] for row in f["items"]].index(item) + + +def named(f, column): + """A column of results by item: `{item: result}`.""" + return {row[0]: v for row, v in zip(f["items"], column)} + + +def changed(f, change): + return helpers.mutated(f, change) + + +def said(found, part): + """Whether one of the errors says `part`.""" + return any(part in line for line in found) + + +class File(unittest.TestCase): + def test_a_made_file_is_checked_clean(self): + f, data, full = made() + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + self.assertEqual(fixes.against(f, data, None), []) + self.assertEqual([row[0] for row in f["items"]], [row[0] for row in full["items"]]) + + def test_an_effect_keeps_only_what_changed(self): + f, *_ = made() + self.assertEqual(f["fixes"][0]["effect"], {A: [[at(f, "p1"), 1.0006]], B: []}) + + def test_the_file_survives_its_own_text(self): + f, *_ = made() + self.assertEqual(json.loads(fixes.dumps(f)), f) + + def test_written_and_read_back_beside_its_diagnostics_file(self): + f, data, full = made() + with tempfile.TemporaryDirectory() as folder: + folder = Path(folder) + chain.write(data, folder / chain.FILE) + fixes.write(f, folder / "fixes.json") + self.assertEqual(fixes.read(folder / "fixes.json"), f) # without the matrix: the listed items + matrix.write(full, folder / "matrix.json") + self.assertEqual(fixes.read(folder / "fixes.json"), f) + + def test_the_limits_are_the_specs(self): + self.assertEqual((fixes.MAX_FIXES, fixes.MAX_BUILT, fixes.MAX_ITEMS, fixes.MAX_BYTES), + (1000, 500, 200_000, 64 * 1024 * 1024)) # fmt: skip + f, *_ = made() + many = changed(f, lambda g: g.__setitem__("fixes", g["fixes"] * 334)) + self.assertTrue(said(fixes.errors(many), "`fixes` is not a list of 1 to 1000")) + + def test_another_major_version_is_refused_by_name(self): + f, *_ = made() + self.assertEqual(fixes.errors({**f, "format": "sentier-diagnostics-fixes/2"}), + ["format 'sentier-diagnostics-fixes/2': this reader knows sentier-diagnostics-fixes/1 only"]) # fmt: skip + self.assertEqual(fixes.errors({**f, "format": "sentier-diagnostics/1"}), + ["not a fixes file of format sentier-diagnostics-fixes/1"]) # fmt: skip + + def test_keys_are_read_strictly(self): + f, *_ = made() + self.assertTrue(said(fixes.errors({**f, "ranges": []}), "a key this format does not know: ranges")) + self.assertTrue(said(fixes.errors({k: v for k, v in f.items() if k != "built"}), "requires: built")) + extra = changed(f, lambda g: g["fixes"][0].__setitem__("verdicts", {})) + self.assertTrue(said(fixes.errors(extra), "a fix is not")) + + def test_shape_faults_never_raise(self): + f, *_ = made() + valid = {("title", '"x"'), ("about", '"x"'), ("about", "null"), ("made_by", '"x"'), ("notes", "[]"), + ("built", "[]")} # fmt: skip + for key in f: + for junk in (None, 1, "x", [], {}, [[None]], {"x": [1]}, True): + with self.subTest(key=key, junk=junk): + found = fixes.errors({**f, key: junk}) + self.assertFalse(said(found, fixes.fileformat.STOPPED), found) + if (key, json.dumps(junk)) not in valid: + self.assertNotEqual(found, [], f"{key} = {junk!r} passed") + for part in ("effect", "counts", "noise", "tolerance", "claim"): + for junk in (1, "x", [1], {"x": 1}): + with self.subTest(fix_key=part, junk=junk): + found = fixes.errors(changed(f, lambda g, k=part, v=junk: g["fixes"][0].__setitem__(k, v))) + self.assertNotEqual(found, []) + self.assertFalse(said(found, fixes.fileformat.STOPPED), found) + self.assertEqual(fixes.errors(None), ["not a fixes file: not an object"]) + + +class Checks(unittest.TestCase): + """SPEC 16.9, each check probed by the change it must find.""" + + def test_of_names_the_file_its_title_and_one_of_its_states(self): + f, data, full = made() + self.assertTrue(said(fixes.against(changed(f, lambda g: g["of"].__setitem__("title", "Another")), data, full), + "belongs to another file")) # fmt: skip + self.assertTrue(said(fixes.against(changed(f, lambda g: g["of"].__setitem__("base", "s9")), data, full), + "no state of the diagnostics file")) # fmt: skip + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["of"].__setitem__("file", "../up.json"))), + "`of` is not {file, title, base}")) # fmt: skip + + def test_items_indicators_floors_and_words_are_the_diagnostics_files(self): + f, data, full = made() + cases = { + "its indicators differ": lambda g: g["indicators"][0].__setitem__("name", "Another"), + "its words for an item": lambda g: g.__setitem__("item", {"one": "product", "many": "products"}), + "its floors differ": lambda g: g["rule"]["floors"][0].__setitem__("value", 0.011), + "not the matrix's, in the matrix's order": lambda g: g["items"].reverse(), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.against(changed(f, change), data, full), part)) + + def test_the_base_is_the_matrix_column_of_the_base_state(self): + f, data, full = made() + moved = changed(f, lambda g: g["base"][B].__setitem__(at(g, "p2"), 5.5)) + self.assertEqual(fixes.errors(moved), []) # no fix touches p2: only the matrix can tell + self.assertTrue(said(fixes.against(moved, data, full), "the base results in 'x:b' are not the matrix's")) + + def test_without_the_matrix_the_base_is_held_to_the_listed_items(self): + f, data, _ = made() + self.assertEqual([i["id"] for i in data["items"]], ["p3"]) + moved = changed(f, lambda g: g["base"][B].__setitem__(at(g, "p3"), 7.5)) + self.assertTrue(said(fixes.against(moved, data, None), "item 'p3', x:b: the base result")) + fewer = changed(f, lambda g: g["items"].pop()) + self.assertTrue(said(fixes.against(fewer, data, None), "it holds 3 items, and the diagnostics file counts 4")) + + def test_an_effect_names_known_items_once_and_never_the_base(self): + f, *_ = made() + k1, k2 = at(f, "p1"), at(f, "p2") + cases = { + "names item 99, and the file holds 4": lambda g: g["fixes"][0]["effect"][A].append([99, 1.0]), + "holds item 'p1' twice": lambda g: g["fixes"][0]["effect"][A].append([k1, 1.5]), + "is written with the result it was measured against": lambda g: g["fixes"][0]["effect"][A].append([k2, 2.0]), + "not one list per indicator": lambda g: g["fixes"][0]["effect"].__setitem__("x:z", []), + "an entry is not [item index, result or null]": lambda g: g["fixes"][0]["effect"][B].append([k1]), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + + def test_built_results_are_held_to_the_same_rules(self): + f, *_ = made() + k1 = at(f, "p1") + cases = { + "holds item 'p1' twice": lambda g: g["built"][0]["results"][A].append([k1, 1.5]), + "is written with the result it was measured against": lambda g: g["built"][0]["results"][B].append([k1, 3.0]), + "names item 7": lambda g: g["built"][0]["results"][B].append([7, 3.0]), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + + def test_counts_and_verdicts_are_what_the_effect_and_the_base_give(self): + f, *_ = made() + + def move(block, key, a, b): + block[key][a] -= 1 + block[key][b] += 1 + + cases = { + "`classes` is not": lambda g: move(g["fixes"][0]["counts"][B], "classes", rule.AGREES, rule.WITHIN_NOISE), + "`verdicts` is not": lambda g: move(g["fixes"][2]["counts"][A], "verdicts", claims.UNCHANGED, + claims.NOT_JUDGED), + "`bands` is not": lambda g: move(g["fixes"][2]["counts"][A], "bands", "down, factor 1.25-2", + "down, factor 1.1-1.25"), + "is null: a fix has no published results": lambda g: g["fixes"][0]["counts"][A].update(closer=0, further=0), + } # fmt: skip + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + # a changed effect with the counts left as they were + moved = changed(f, lambda g: g["fixes"][0]["effect"][A].__setitem__(0, [at(g, "p1"), 1.5])) + self.assertTrue(said(fixes.errors(moved), "fix 'a', counts of 'x:a': `classes` is not what its effect")) + + def test_a_fix_claim_names_its_items(self): + f, *_ = made() + hidden = changed(f, lambda g: g["fixes"][2]["claim"]["scope"][0].__setitem__("items", None)) + self.assertTrue(said(fixes.errors(hidden), "a fix's claim names its items")) + unknown = changed(f, lambda g: (g["fixes"][2]["claim"]["scope"][0].__setitem__("items", ["p3", "p9"]), + g["fixes"][2]["claim"].__setitem__("reach", 2))) # fmt: skip + self.assertTrue(said(fixes.errors(unknown), "its claim names 1 items the file does not hold")) + + def test_pairs_name_two_known_fixes_once(self): + f, *_ = made() + cases = { + "the file holds no fix 'z'": lambda g: g["interactions"]["pairs"].append(pair("b", "z")), + "names one fix twice": lambda g: g["interactions"]["pairs"].append(pair("a", "a")), + "is listed twice": lambda g: g["interactions"]["pairs"].append(pair("c", "b")), + "does not name two fixes": lambda g: g["interactions"]["pairs"].append({**pair("a", "b"), "fixes": ["a"]}), + "`largest` is not a share": lambda g: g["interactions"]["pairs"][0].__setitem__("largest", -0.1), + "`kind` is not one of": lambda g: g["interactions"]["pairs"][0].__setitem__("kind", "market"), + "`reach` is not a count of 1 to 4 items": lambda g: g["interactions"]["pairs"][0].__setitem__("reach", 5), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + + def test_a_pair_reaches_at_least_one_item(self): + f, *_ = made() + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["interactions"]["pairs"][0].__setitem__("reach", 0))), + "`reach` is not a count of 1 to 4 items, or null")) # fmt: skip + self.assertEqual(fixes.errors(changed(f, lambda g: g["interactions"]["pairs"][0].__setitem__("reach", 4))), []) + + def test_no_half_check(self): + f, *_ = made() + listed = changed(f, lambda g: g["interactions"].update(checked=False, method=None, by=None)) + self.assertTrue(said(fixes.errors(listed), "pairs are listed and `checked` is false")) + f, *_ = made(interactions="unchecked") + self.assertEqual(fixes.errors(f), []) + ran = changed(f, lambda g: g["interactions"].__setitem__("method", "Looked at some pairs.")) + self.assertTrue(said(fixes.errors(ran), "no check ran, so it has no `method`")) + f, *_ = made() + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["interactions"].__setitem__("by", None))), + "says who or what ran it")) # fmt: skip + + def test_a_built_set_holds_two_or_more_fixes_once_and_is_not_built_twice(self): + f, *_ = made() + empty = {A: [], B: []} + noise = chain.noise() + + def add(names): + return lambda g: g["built"].append({"id": "x", "label": "X", "fixes": names, "state": None, "results": empty, + "noise": noise}) # fmt: skip + + cases = { + "fewer than two fixes": add(["a"]), + "holds a fix twice": add(["a", "a"]), + "the same fixes as another built set": add(["b", "a"]), + "the file holds no fix 'z'": add(["a", "z"]), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + twice = changed(f, lambda g: g["built"].append(dict(g["built"][0], fixes=["a", "c"], results=empty))) + self.assertTrue(said(fixes.errors(changed(f, lambda g: g["built"][0].__setitem__("state", 5))), + "`state` is not the id of a state")) # fmt: skip + self.assertTrue(said(fixes.errors(twice), "two built sets have the id 'ab'")) + + +def dense(f, fix_id, under=None, under_noise=None): + """A fix's counts worked out item by item, as `chain` counts a step: the reference the + sparse count is held to. `under` is what the fix was measured against, written out by + hand (`{indicator: {item index: result}}`), the base where it says nothing; `under_noise` + the noise of that build, the base's where it is None. The noise is the sum of the two + builds' (SPEC 5.3).""" + under = under or {} + ends = [under_noise or f["of"].get("base_noise"), next(x for x in f["fixes"] if x["id"] == fix_id)["noise"]] + one = next(x for x in f["fixes"] if x["id"] == fix_id) + floors = {(x["indicator"], x["unit"]): x["value"] for x in f["rule"]["floors"]} + out = {} + for ind in (i["id"] for i in f["indicators"]): + effect = dict(map(tuple, one["effect"][ind])) + said = [n["by"][ind] for n in ends if n and n["measured"] and ind in n["by"]] + level = None if not said else said[0] if len(said) == 1 else said[0] + said[1] + block = {"classes": dict.fromkeys(rule.CLASSES, 0), "missing": {"before": 0, "after": 0, "both": 0}, + "verdicts": dict.fromkeys(claims.VERDICTS, 0) if one["claim"] else None} # fmt: skip + for k, row in enumerate(f["items"]): + before = under.get(ind, {}).get(k, f["base"][ind][k]) + after = effect.get(k, before) + kind, way, _ = rule.judge(before, after, floor=floors[(ind, row[4])], + tolerance=one["tolerance"]["value"], noise=level) # fmt: skip + block["classes"][kind] += 1 + if kind == rule.MISSING: + block["missing"]["both" if before is None and after is None else "before" if before is None else "after"] += 1 + if one["claim"]: + covering = [e for e in claims.entries_for(one["claim"], ind) if claims.covers(e, row[0], ind)] + block["verdicts"][claims.verdict(kind, before, after, way, covering)] += 1 + out[ind] = block + return out + + +class SparseCounts(unittest.TestCase): + def test_counting_the_changed_items_only_is_counting_every_item(self): + last = {A: {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0, "p5": None, "p6": -2.0}, + B: {"p1": 3.0, "p2": 5.0, "p3": 7.0, "p4": 1.0, "p5": 4.0, "p6": 0.001}} # fmt: skip + data, full = chain_and_matrix(last) + every = claims.claim("All of B must move; p1 and p5 may move in A.", + [claims.scope([B], "*", "up", None, claims.MUST), + claims.scope([A], {"p1", "p5"}, "any", None, claims.MAY), + claims.scope([A], {"p2", "p4"}, "down", 0.1, claims.MUST)]) # fmt: skip + wide = fixes.fix("w", "Wide", kind="amount", claim=every, noise=chain.noise(["one", "two"], {A: 0.05})) + moved = {A: {"p1": 1.02, "p5": 3.0, "p2": None, "p6": 2.0}, B: {"p1": 3.5, "p5": None, "p6": 0.002, "p4": 1.0001}} + f = fixes.build("Sparse", data=data, full=full, base="s1", fixes=[wide], results={"w": moved}) + again = dense(f, "w") + for ind, block in f["fixes"][0]["counts"].items(): + for key in ("classes", "missing", "verdicts"): + with self.subTest(ind=ind, key=key): + self.assertEqual(block[key], again[ind][key]) + self.assertEqual(sum(f["fixes"][0]["counts"][B]["verdicts"].values()), 6) + + +class Acceptance(unittest.TestCase): + """SPEC 16.11. F7 is the page's (sentier-bafu).""" + + def test_f1_two_fixes_within_the_tolerance_differ_together(self): + f, *_ = made(built=False) + for one in ("a", "b"): + self.assertEqual(fixes.counts(f, [one])["by"][A]["classes"][rule.DIFFERS], 0) + together = fixes.counts(f, ["a", "b"]) + self.assertEqual(together["by"][A]["classes"][rule.DIFFERS], 1) + self.assertEqual(together["label"], fixes.EXACT) + self.assertAlmostEqual(fixes.compose(f, ["a", "b"])["results"][A][at(f, "p1")], 1.0013) + + def test_f2_a_set_with_a_listed_pair_is_estimated_everywhere(self): + f, *_ = made() + drawn = fixes.compose(f, ["c", "b"]) + self.assertEqual(drawn["label"], fixes.ESTIMATED) + self.assertEqual(drawn["pairs"], [pair("b", "c")]) + self.assertEqual(drawn["pair_terms"], {"value": None, "measured": 0, "no_size": 0, "pairs": 1}) + self.assertEqual(fixes.counts(f, ["b", "c"])["label"], fixes.ESTIMATED) + self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.ESTIMATED) # a built subset makes no set built + self.assertEqual(fixes.label(f, ["a", "c"]), fixes.EXACT) + + def test_f3_unchecked_interactions_make_every_set_not_built_estimated(self): + f, *_ = made(interactions="unchecked") + self.assertEqual(fixes.label(f, ["a"]), fixes.BUILT) + self.assertEqual(fixes.label(f, ["a", "b"]), fixes.BUILT) + for names in (["a", "c"], ["b", "c"], ["a", "b", "c"]): + self.assertEqual(fixes.label(f, names), fixes.ESTIMATED) + + def test_f4_a_built_set_that_is_not_the_sum_of_independent_fixes_is_refused(self): + with self.assertRaises(ValueError) as caught: + made(interactions="none", built_ab=1.0030) + self.assertIn("built set 'ab', item 'p1', x:a: built 1.003", str(caught.exception)) + self.assertIn("the file calls its fixes independent, and they are not", str(caught.exception)) + f, *_ = made(interactions="none") # the sum: accepted + self.assertEqual(fixes.errors(f), []) + def interacting(g): + g["built"][0]["results"][A][0] = [at(g, "p1"), 1.0030] + g["interactions"]["pairs"].append(pair("a", "b", fixes.SAME_ENTRY)) + + self.assertEqual(fixes.errors(changed(f, interacting)), [], "a listed pair is an interaction, shown, not refused") + f, *_ = made(interactions="unchecked", built_ab=1.0030) + self.assertEqual(fixes.errors(f), [], "without a check the file claims no independence") + + def test_f4_a_built_set_is_held_to_the_sum_of_its_fixes_not_to_a_build_inside_it(self): + """Tolerances do not stack: a, b and c built together lie within 1e-3 of the build of a + and b (1.0022), which lies within 1e-3 of the sum (1.0013), and 1.8e-3 from the sum.""" + f, *_ = made(interactions="none", built_ab=1.0022) + self.assertEqual(fixes.errors(f), []) + + def add(g): + p1, p3 = at(g, "p1"), at(g, "p3") + results = {A: sorted([[p1, 1.0031], [p3, 2.0]]), B: [[p3, 6.0]]} + g["built"].append({"id": "abc", "label": "Every fix", "fixes": ["a", "b", "c"], "state": None, + "results": results, "noise": chain.noise()}) # fmt: skip + + found = fixes.errors(changed(f, add)) + self.assertTrue(said(found, "built set 'abc', item 'p1', x:a: built 1.0031, the sum of its effects is 1.0013"), + found) # fmt: skip + + def test_f5_a_fix_is_judged_the_same_whichever_fixes_are_chosen_with_it(self): + f, *_ = made() + own = fixes.compose(f, ["c"])["fixes"][0] + for names in (["a", "c"], ["c", "b"], ["b", "a", "c"]): + drawn = fixes.compose(f, names) + self.assertEqual(next(x for x in drawn["fixes"] if x["id"] == "c"), own) + self.assertNotIn("verdicts", drawn) + self.assertIsNone(fixes.counts(f, names)["by"][A]["verdicts"]) + self.assertEqual(fixes.compose(f, ["b", "a"])["results"], fixes.compose(f, ["a", "b"])["results"]) + + def test_f6_the_noise_of_a_set_is_the_sum_said_with_how_many_were_measured(self): + """The base (s1) was not measured: it adds nothing, and is one build of three whose + noise is not known. Where a set changes nothing, it stands on no build.""" + noise = chain.noise(["one", "two"], {A: 1e-3}) + f, *_ = made(noise_a=noise, noise_b=noise, interactions="none", built=False) + self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 2e-3, "known": 2, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-3, "known": 1, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": None, "known": 0, "builds": 2}) + self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 0, "builds": 0}) + # the sum, not the largest: 1.3e-3 apart is within 2e-3 and beyond 1e-3 + self.assertEqual(fixes.counts(f, ["a", "b"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) + + def test_f8_a_pair_holds_no_text_so_nothing_of_the_model_goes_into_it(self): + f, *_ = made() + # joined here, so that the repository itself holds no identifier-shaped text + uuid_shaped = "-".join(["ab12cd34", "ab12", "ab12", "ab12", "ab12cd34ef56"]) # noqa: FLY002 + cases = { + "a pair is not {fixes, kind, reach, largest}": lambda g: g["interactions"]["pairs"][0].__setitem__( + "reason", f"both change {uuid_shaped}"), + "`kind` is not one of same entry": lambda g: g["interactions"]["pairs"][0].__setitem__( + "kind", f"both change {uuid_shaped}"), + "`interactions.method` holds an identifier": lambda g: g["interactions"].__setitem__( + "method", "Compared " + "ab" * 16), + } # fmt: skip + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + self.assertIsNone(fixes.IDENTIFIER.search(fixes.dumps(f))) + + +class PairSizes(unittest.TestCase): + """A listed pair may carry the measured size of its cross term (`largest`): an estimated + set then says how far it may lie from its sum, and a built pair holds the size to account.""" + + def test_a_built_pair_lies_within_its_measured_size(self): + f, *_ = made(interactions="unchecked", built_ab=1.0030) # 1.7e-3 from the sum, 1.0013 + f["interactions"] = checked_pairs(pair("a", "b", fixes.SAME_ENTRY)) + self.assertEqual(fixes.errors(f), [], "not measured: the pair is shown, not held to a size") + f["interactions"]["pairs"][-1]["largest"] = 0.002 + self.assertEqual(fixes.errors(f), []) + f["interactions"]["pairs"][-1]["largest"] = 0.001 + found = fixes.errors(f) + self.assertTrue(said(found, "built set 'ab', item 'p1', x:a"), found) + self.assertTrue(said(found, "measured their cross term at 0.001 at most, and it is larger"), found) + + def test_the_pair_terms_of_a_set_are_the_sum_of_its_measured_pairs(self): + f, *_ = made() + f["interactions"]["pairs"] = [pair("b", "c", largest=1e-5), pair("a", "c", fixes.SAME_ENTRY, 2e-5)] + self.assertEqual(fixes.errors(f), []) + terms = fixes.pair_terms(f, ["a", "b", "c"]) + self.assertEqual((terms["measured"], terms["pairs"]), (2, 2)) + self.assertAlmostEqual(terms["value"], 3e-5) + f["interactions"]["pairs"][1]["largest"] = None + self.assertEqual(fixes.pair_terms(f, ["a", "b", "c"]), {"value": None, "measured": 1, "no_size": 0, "pairs": 2}) + self.assertEqual(fixes.pair_terms(f, ["a", "b"]), {"value": 0, "measured": 0, "no_size": 0, "pairs": 0}) + + def test_a_pair_with_no_size_is_built_and_its_build_has_none(self): + """A build of a and b on the other side of zero from their sum: no tolerance makes the + two agree. Said of a build 1.7e-3 from its sum (1.0030 and 1.0013), it is refused: + that is its size.""" + f, *_ = made(interactions="unchecked", built_ab=-0.5) + f["interactions"] = checked_pairs(pair("a", "b", fixes.SAME_ENTRY, fixes.NO_SIZE)) + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.pair_terms(f, ["a", "b", "c"]), {"value": None, "measured": 1, "no_size": 1, "pairs": 1}) + self.assertEqual(fixes.label(f, ["a", "b"]), fixes.BUILT) + self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.ESTIMATED) + self.assertEqual(named(f, fixes.compose(f, ["a", "b", "c"])["results"][A])["p1"], -0.5, "taken from the build") + agrees = changed(f, lambda g: g["built"][0]["results"][A][0].__setitem__(1, 1.0030)) + self.assertTrue(said(fixes.errors(agrees), "built set 'ab': the file says its pair has no size, and its build " + "lies within 0.00169491525")) # fmt: skip + unbuilt = changed(f, lambda g: g["built"].clear()) + self.assertTrue(said(fixes.errors(unbuilt), "the pair 'a' and 'b': has no size, so no set holding it can be drawn")) + other = changed(f, lambda g: g["interactions"]["pairs"][0].__setitem__("largest", "large")) + self.assertTrue(said(fixes.errors(other), "`largest` is not a share, 'no size' where no tolerance is enough")) + + def test_the_kinds_are_listed_in_their_order_of_precedence(self): + self.assertEqual(fixes.PAIR_KINDS, ("same entry", "technosphere", "technosphere and emission", + "technosphere and factor", "emission and factor")) # fmt: skip + +def with_requires(*, built_ab=True): + """Fixes that need others applied first, against the base of `chain_and_matrix`: + + - a moves p1 in A to 1.5; b moves p2 in B to 5.5; both alone; + - d requires a: built with a and d, p1 is 1.8 and p2 is 2.5; + - e requires d (so a too): with a, d and e, p3 is 3.3 as well; + - g requires a and b: with a, b and g, p1 is 1.6, p2 is 2.2 and p4 is 0.5. The file holds + a build of a and b, which g was measured against. + + Each fix's results are written whole, as built; its effect keeps what it changed on top + of what it requires.""" + data, full = chain_and_matrix() + f = fixes.build( + "Fixes that need others", + data=data, + full=full, + base="s1", + fixes=[ + fixes.fix("a", "Fix A", kind="mapping"), + fixes.fix("b", "Fix B", kind="mapping"), + fixes.fix("d", "Fix D", kind="amount", requires=["a"]), + fixes.fix("e", "Fix E", kind="amount", requires=["d"]), + fixes.fix("g", "Fix G", kind="amount", requires=["a", "b"]), + ], + results={ + "a": {A: {"p1": 1.5}}, + "b": {B: {"p2": 5.5}}, + "d": {A: {"p1": 1.8, "p2": 2.5}}, + "e": {A: {"p1": 1.8, "p2": 2.5, "p3": 3.3}}, + "g": {A: {"p1": 1.6, "p2": 2.2, "p4": 0.5}, B: {"p2": 5.5}}, + }, + interactions=checked_pairs(), + built=[fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": 1.5}, B: {"p2": 5.5}})] if built_ab else (), + ) + return f, data, full + + +class Requires(unittest.TestCase): + """A fix that cannot be applied alone names the fixes it needs (SPEC 16.2).""" + + def test_a_fix_is_measured_against_the_build_of_what_it_requires(self): + f, data, full = with_requires() + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + effect = {one["id"]: one["effect"] for one in f["fixes"]} + p1, p2, p3, p4 = (at(f, p) for p in ("p1", "p2", "p3", "p4")) + self.assertEqual(effect["d"], {A: [[p1, 1.8], [p2, 2.5]], B: []}) + self.assertEqual(effect["e"], {A: [[p3, 3.3]], B: []}, "what d already did is d's") + self.assertEqual(effect["g"], {A: [[p1, 1.6], [p2, 2.2], [p4, 0.5]], B: []}, "b's change in B is b's") + # the counts: against that build, item by item + under = {"d": {A: {p1: 1.5}}, "e": {A: {p1: 1.8, p2: 2.5}}, "g": {A: {p1: 1.5}, B: {p2: 5.5}}} + for one in f["fixes"]: + again = dense(f, one["id"], under.get(one["id"])) + for ind in (A, B): + with self.subTest(fix=one["id"], ind=ind): + self.assertEqual(one["counts"][ind]["classes"], again[ind]["classes"]) + e = f["fixes"][3]["counts"][A]["classes"] + self.assertEqual((e[rule.DIFFERS], e[rule.AGREES], e[rule.NEAR_ZERO]), (1, 2, 1), "e changes p3 only") + + def test_an_effect_written_with_the_result_of_its_requirements_is_refused(self): + f, *_ = with_requires() + stale = changed(f, lambda g: g["fixes"][2]["effect"][A].append([at(g, "p4"), 0.0])) + self.assertTrue(said(fixes.errors(stale), "item 'p4' is written with the result it was measured against")) + kept = changed(f, lambda g: g["fixes"][3]["effect"][A].insert(0, [at(g, "p1"), 1.8])) + self.assertTrue(said(fixes.errors(kept), "fix 'e': its effect, x:a: item 'p1' is written with the result")) + + def test_choosing_a_fix_chooses_what_it_requires(self): + f, *_ = with_requires() + self.assertEqual(fixes.chosen(f, ["e"]), (["a", "d", "e"], ["a", "d"])) + drawn = fixes.compose(f, ["e"]) + self.assertEqual((drawn["label"], drawn["added"]), (fixes.BUILT, ["a", "d"])) + self.assertEqual(named(f, drawn["results"][A]), {"p1": 1.8, "p2": 2.5, "p3": 3.3, "p4": 0.0}) + self.assertEqual([x["requires"] for x in drawn["fixes"]], [[], ["a"], ["d"]]) + self.assertEqual(fixes.label(f, ["g"]), fixes.BUILT) + drawn = fixes.compose(f, ["g"])["results"] + self.assertEqual(named(f, drawn[A]), {"p1": 1.6, "p2": 2.2, "p3": 3.0, "p4": 0.5}) + self.assertEqual(named(f, drawn[B]), {"p1": 3.0, "p2": 5.5, "p3": 7.0, "p4": 1.0}) + + def test_effects_add_each_on_top_of_what_it_was_measured_with(self): + f, *_ = with_requires() + # e and b: every item is one fix's with what it requires, so each result is a built one + drawn = fixes.compose(f, ["e", "b"]) + self.assertEqual((drawn["label"], drawn["added"]), (fixes.EXACT, ["a", "d"])) + self.assertEqual(named(f, drawn["results"][A]), {"p1": 1.8, "p2": 2.5, "p3": 3.3, "p4": 0.0}) + self.assertEqual(named(f, drawn["results"][B]), {"p1": 3.0, "p2": 5.5, "p3": 7.0, "p4": 1.0}) + # d and g both build on a: a's effect counts once, each of theirs on top + drawn = fixes.compose(f, ["d", "g"]) + self.assertEqual(drawn["label"], fixes.EXACT) + p1, p2 = at(f, "p1"), at(f, "p2") + self.assertAlmostEqual(drawn["results"][A][p1], 1.0 + 0.5 + 0.3 + 0.1) + self.assertAlmostEqual(drawn["results"][A][p2], 2.0 + 0.5 + 0.2) + self.assertEqual(drawn["cannot_add"], {A: [], B: []}) + + def test_where_one_fix_with_its_requirements_covers_an_item_its_build_is_the_result(self): + """a removes p1's result and d, which requires a, gives it one: no difference can be + taken against a missing result, so only the build of d says what p1 is.""" + data, full = chain_and_matrix() + four = [fixes.fix("a", "A", kind="mapping"), fixes.fix("d", "D", kind="amount", requires=["a"]), + fixes.fix("b", "B", kind="mapping"), fixes.fix("c", "C", kind="mapping")] # fmt: skip + results = {"a": {A: {"p1": None}}, "d": {A: {"p1": 2.0}}, "b": {A: {"p2": 2.5}}, "c": {A: {"p1": 1.2}}} + f = fixes.build("A requirement that removes a result", data=data, full=full, base="s1", fixes=four, + results=results, interactions=checked_pairs()) # fmt: skip + p1 = at(f, "p1") + drawn = fixes.compose(f, ["d", "b"]) + self.assertEqual((drawn["label"], drawn["results"][A][p1], drawn["cannot_add"]), (fixes.EXACT, 2.0, {A: [], B: []})) + for names in (["d", "c"], ["a", "c"]): + with self.subTest(names=names): + drawn = fixes.compose(f, names) + self.assertEqual((drawn["results"][A][p1], drawn["cannot_add"]), (None, {A: [p1], B: []})) + + def test_where_a_build_inside_the_set_covers_an_item_its_result_is_measured_not_summed(self): + """a and b interact at p1 (a listed pair): built together, p1 is 2.9, not their sum 2.7. + g requires both and leaves p1 alone; c is apart.""" + data, full = chain_and_matrix() + four = [fixes.fix("a", "A", kind="mapping"), fixes.fix("b", "B", kind="mapping"), + fixes.fix("g", "G", kind="amount", requires=["a", "b"]), fixes.fix("c", "C", kind="amount")] # fmt: skip + results = {"a": {A: {"p1": 1.5}}, "b": {A: {"p1": 2.2}}, "g": {A: {"p1": 2.9, "p2": 3.0}}, "c": {A: {"p3": 2.0}}} + f = fixes.build("A measured joint", data=data, full=full, base="s1", fixes=four, results=results, + interactions=checked_pairs(pair("a", "b")), + built=[fixes.built_set("ab", "A and B", ["a", "b"], {A: {"p1": 2.9}})]) # fmt: skip + p1 = at(f, "p1") + for names in (["a", "b", "c"], ["g", "c"]): + with self.subTest(names=names): + drawn = fixes.compose(f, names) + self.assertEqual((drawn["label"], drawn["results"][A][p1]), (fixes.ESTIMATED, 2.9)) + # the sum a built set is held to stands on the fixes alone + cells = fixes._Builds.of(f).composed(frozenset({"a", "b", "c"}), A, built=False) + self.assertAlmostEqual(cells[p1][0], 2.7) + + def test_a_fix_whose_requirements_were_never_built_together_is_refused(self): + with self.assertRaises(ValueError) as caught: + with_requires(built_ab=False) + self.assertIn("fix 'g': no build of what it requires (a, b)", str(caught.exception)) + f, *_ = with_requires() + gone = changed(f, lambda g: g["built"].clear()) + self.assertTrue(said(fixes.errors(gone), "it requires 'a', 'b', and the file holds no build of them together")) + + def test_requires_names_earlier_fixes_once(self): + f, *_ = with_requires() + cases = { + "fix 'd': requires itself": lambda g: g["fixes"][2].__setitem__("requires", ["d"]), + "requires 'z', and the file holds no such fix": lambda g: g["fixes"][2].__setitem__("requires", ["z"]), + "requires 'b', which comes after it": lambda g: g["fixes"][0].__setitem__("requires", ["b"]), + "requires a fix twice": lambda g: g["fixes"][2].__setitem__("requires", ["a", "a"]), + "`requires` is not a list of the ids of fixes": lambda g: g["fixes"][2].__setitem__("requires", "a"), + } + for part, change in cases.items(): + with self.subTest(part): + self.assertTrue(said(fixes.errors(changed(f, change)), part)) + with self.assertRaises(ValueError): + fixes.build("x", data=chain_and_matrix()[0], full=chain_and_matrix()[1], base="s1", + fixes=[fixes.fix("d", "D", kind="amount", requires=["a"]), fixes.fix("a", "A", kind="amount")], + results={}) # fmt: skip + + def test_a_fix_and_what_it_requires_are_never_a_pair(self): + f, *_ = with_requires() + for a, b in (("a", "d"), ("e", "a"), ("d", "e")): + with self.subTest(pair=(a, b)): + found = fixes.errors(changed(f, lambda g, a=a, b=b: g["interactions"]["pairs"].append(pair(a, b)))) + self.assertTrue(said(found, "one requires the other"), found) + self.assertEqual(fixes.errors(changed(f, lambda g: g["interactions"]["pairs"].append(pair("d", "g")))), []) + + def test_a_built_set_holds_what_its_fixes_require_and_is_not_a_fix_with_its_requirements(self): + f, *_ = with_requires() + empty, noise = {A: [], B: []}, chain.noise() + + def add(names, results=empty): + return lambda g: g["built"].append({"id": "x", "label": "X", "fixes": names, "state": None, + "results": results, "noise": noise}) # fmt: skip + + self.assertTrue(said(fixes.errors(changed(f, add(["d", "b"]))), "holds 'd' without 'a', which it requires")) + self.assertTrue(said(fixes.errors(changed(f, add(["a", "d"]))), "is built already: it is fix 'd'")) + # a, b and d built together: checked, with no listed pair, so it must be the sum + p1, p2 = at(f, "p1"), at(f, "p2") + summed = {A: [[p1, 1.8], [p2, 2.5]], B: [[p2, 5.5]]} + self.assertEqual(fixes.errors(changed(f, add(["a", "b", "d"], summed))), []) + self.assertEqual(fixes.label(changed(f, add(["a", "b", "d"], summed)), ["d", "b"]), fixes.BUILT) + off = {A: [[p1, 1.8], [p2, 2.9]], B: [[p2, 5.5]]} + found = fixes.errors(changed(f, add(["a", "b", "d"], off))) + self.assertTrue(said(found, "built set 'x', item 'p2', x:a: built 2.9, the sum of its effects is 2.5"), found) + + +def with_added(): + """One fix that brings an item no state of the chain holds: p9, in kg.""" + data, full = chain_and_matrix() + f = fixes.build( + "Fixes that add an item", + data=data, + full=full, + base="s1", + fixes=[fixes.fix("n", "A new item", kind="mapping")], + results={"n": {A: {"p9": 0.7}, B: {"p9": 1.2}}}, + interactions=checked_pairs(), + added_items=[["p9", "A new thing", None, None, "kg"]], + ) + return f, data, full + + +class AddedItems(unittest.TestCase): + """A fix may bring items the base does not hold (SPEC 16.2).""" + + def test_an_added_item_follows_the_matrix_items_with_no_base_result(self): + f, data, full = with_added() + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + self.assertEqual(fixes.against(f, data, None), []) + self.assertEqual([row[0] for row in f["items"]], [row[0] for row in full["items"]] + ["p9"]) + self.assertEqual((f["base"][A][-1], f["base"][B][-1]), (None, None)) + self.assertEqual(named(f, fixes.compose(f, ["n"])["results"][A]), + {"p1": 1.0, "p2": 2.0, "p3": 3.0, "p4": 0.0, "p9": 0.7}) # fmt: skip + self.assertEqual(f["fixes"][0]["counts"][A]["missing"], {"before": 1, "after": 0, "both": 0}) + + def test_an_added_item_has_no_base_result_and_a_result_from_some_fix(self): + f, data, full = with_added() + held = changed(f, lambda g: g["base"][A].__setitem__(4, 0.7)) + self.assertTrue(said(fixes.against(held, data, full), "item 'p9' is not in the matrix, and the base has a result")) + idle = changed(f, lambda g: g["fixes"][0].update(effect={A: [], B: []})) + self.assertTrue(said(fixes.against(idle, data, full), "item 'p9' is in no state of the chain, and no fix gives it")) + with self.assertRaises(ValueError) as caught: + fixes.build("x", data=data, full=full, base="s1", fixes=[fixes.fix("n", "N", kind="mapping")], results={}, + interactions=checked_pairs(), added_items=[["p9", "A new thing", None, None, "kg"]]) # fmt: skip + self.assertIn("item 'p9' is in no state of the chain", str(caught.exception)) + + def test_an_added_item_in_a_new_unit_gets_a_floor_from_nothing(self): + """No item of the chain is in litres: the chain would give the unit a floor of 0, from no + result, so the fixes file does.""" + data, full = chain_and_matrix() + f = fixes.build("An item in litres", data=data, full=full, base="s1", fixes=[fixes.fix("n", "N", kind="mapping")], + results={"n": {A: {"p9": 0.7}, B: {"p9": 1.2}}}, interactions=checked_pairs(), + added_items=[["p9", "A new thing", None, None, "l"]]) # fmt: skip + self.assertEqual(f["rule"]["floors"][-2:], [{"indicator": i, "unit": "l", "value": 0.0, "basis": "none", "n": 0} + for i in (A, B)]) # fmt: skip + self.assertEqual((fixes.errors(f), fixes.against(f, data, full), fixes.against(f, data, None)), ([], [], [])) + raised = changed(f, lambda g: g["rule"]["floors"][-1].__setitem__("value", 0.5)) + self.assertEqual(fixes.errors(raised), []) + self.assertTrue(said(fixes.against(raised, data, full), "its floors differ from the diagnostics file's")) + + def test_added_items_come_after_the_matrix_items_and_are_new(self): + f, data, full = with_added() + first = changed(f, lambda g: g["items"].insert(0, g["items"].pop())) + self.assertTrue(said(fixes.against(first, data, full), "before any item a fix adds")) + with self.assertRaises(ValueError) as caught: + fixes.build("x", data=data, full=full, base="s1", fixes=[fixes.fix("n", "N", kind="mapping")], results={}, + added_items=[["p2", "Again", None, None, "kg"]]) # fmt: skip + self.assertIn("an added item is an item of the base already", str(caught.exception)) + + +class ChainState(unittest.TestCase): + """A built set may say it is a state of the chain after the base: the two builds are held + to each other (SPEC 16.4).""" + + def test_a_built_set_that_is_a_state_agrees_with_it(self): + f, data, full = made(release=True) + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + self.assertEqual(fixes.against(f, data, None), []) + self.assertEqual(fixes.label(f, ["c", "b", "a"]), fixes.BUILT) + + def test_a_built_set_that_is_not_the_state_it_names_is_refused(self): + f, data, full = made(release=True) + def nudge(g): + entries = g["built"][-1]["results"][A] + entries[[k for k, _ in entries].index(at(g, "p1"))][1] = 1.01 + + off = changed(f, nudge) + self.assertEqual(fixes.errors(off), [], "on its own the file cannot tell") + found = fixes.against(off, data, full) + self.assertTrue(said(found, "built set 'all', item 'p1', x:a: built 1.01, and state 's2' of the chain"), found) + self.assertEqual([i["id"] for i in data["items"]], ["p3", "p1"]) + found = fixes.against(off, data, None) + self.assertTrue(said(found, "built set 'all', item 'p1', x:a: built 1.01"), "without the matrix: the listed items") + + def test_the_state_comes_after_the_base(self): + f, data, full = made(release=True) + for state in ("s1", "s0", "s9"): + with self.subTest(state=state): + moved = changed(f, lambda g, s=state: g["built"][-1].__setitem__("state", s)) + self.assertTrue(said(fixes.against(moved, data, full), "is not a state of the chain after the base")) + +def measured(value, ind=A): + return chain.noise(["one", "two"], {ind: value}) + + +class Noise(unittest.TestCase): + """Noise belongs to a build, as it belongs to a state of a chain (SPEC 5.3, 16.5): the + base's is its state's, a fix's that of the build with it, a built set's that of its + build. A comparison takes the sum of the noise of the builds it stands on.""" + + def test_the_base_takes_the_noise_of_its_state(self): + f, data, full = made(state_noise={"s1": measured(1e-4)}) + self.assertEqual(f["of"]["base_noise"], data["states"][1]["noise"]) + self.assertEqual(fixes.errors(f), []) + self.assertEqual(fixes.against(f, data, full), []) + other = changed(f, lambda g: g["of"]["base_noise"]["by"].__setitem__(A, 2e-4)) + self.assertTrue(said(fixes.against(other, data, full), "`of.base_noise` is not the noise the diagnostics file " + "gives state 's1'")) # fmt: skip + gone = changed(f, lambda g: g["of"].pop("base_noise")) + self.assertEqual(fixes.errors(gone), [], "on its own the file cannot tell a published base") + self.assertTrue(said(fixes.against(gone, data, full), "the base is a build: `of.base_noise` is its noise")) + bad = changed(f, lambda g: g["of"]["base_noise"].__setitem__("between", None)) + self.assertTrue(said(fixes.errors(bad), "`of.base_noise`: measured noise names the two builds")) + + def test_a_published_base_is_exact_and_takes_no_noise(self): + data, full = chain_and_matrix() + f = fixes.build("From the published results", data=data, full=full, base="s0", + fixes=[fixes.fix("a", "Fix A", kind="mapping", noise=measured(1e-3))], + results={"a": {A: {"p1": 1.5}}}) # fmt: skip + self.assertNotIn("base_noise", f["of"]) + self.assertEqual(fixes.noise_of(f, ["a"])[A], {"value": 1e-3, "known": 2, "builds": 2}) + self.assertEqual(fixes.noise_of(f, ["a"])[B], {"value": None, "known": 0, "builds": 0}, "a changes nothing in B") + noisy = changed(f, lambda g: g["of"].__setitem__("base_noise", chain.noise())) + self.assertTrue(said(fixes.against(noisy, data, full), "the base is the published results, which are exact")) + + def test_a_fix_is_judged_with_the_noise_of_its_build_and_of_the_base(self): + """1.0 to 1.0015 is 1.5e-3 apart: beyond the tolerance (1e-3) and the fix's own noise + (1e-3), within the sum of its and the base's (2e-3).""" + + def judged(base_noise): + data, full = chain_and_matrix(noise={"s1": base_noise}) + f = fixes.build("One fix", data=data, full=full, base="s1", + fixes=[fixes.fix("a", "Fix A", kind="mapping", tolerance=chain.tolerance(1e-3), + noise=measured(1e-3))], + results={"a": {A: {"p1": 1.0015}}}) # fmt: skip + self.assertEqual(f["fixes"][0]["counts"][A]["classes"], dense(f, "a")[A]["classes"]) + return f["fixes"][0]["counts"][A]["classes"][rule.WITHIN_NOISE] + + self.assertEqual(judged(measured(1e-3)), 1) + self.assertEqual(judged(None), 0) + + def test_a_fix_measured_against_its_requirements_takes_their_builds_noise(self): + """d is 1.5e-3 from the build of a: within the noise of a's build and d's, beyond d's + and the base's (not measured).""" + data, full = chain_and_matrix() + f = fixes.build("A fix that needs another", data=data, full=full, base="s1", + fixes=[fixes.fix("a", "Fix A", kind="mapping", noise=measured(1e-3)), + fixes.fix("d", "Fix D", kind="amount", requires=["a"], tolerance=chain.tolerance(1e-3), + noise=measured(1e-3))], + results={"a": {A: {"p1": 1.5}}, "d": {A: {"p1": 1.5 * 1.0015}}}) # fmt: skip + counts = f["fixes"][1]["counts"][A]["classes"] + self.assertEqual(counts[rule.WITHIN_NOISE], 1) + again = dense(f, "d", {A: {at(f, "p1"): 1.5}}, under_noise=measured(1e-3)) + self.assertEqual(counts, again[A]["classes"]) + + def test_a_sum_counts_the_base_once_for_each_fix_measured_against_it(self): + """a and b each moved p1 from the base: a rebuild of the base moves both differences. + With a and c, no item is a sum: p1 is a's build and p3 c's, each against the base.""" + f, *_ = made(noise_a=measured(1e-3), noise_b=measured(1e-3), interactions="none", built=False, + state_noise={"s1": measured(1e-4)}) # fmt: skip + self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 1e-4 + 1e-3 + 1e-4 + 1e-3, "known": 3, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["a", "c"])[A], {"value": 1e-4 + 1e-3, "known": 2, "builds": 3}) + self.assertEqual(fixes.noise_of(f, ["c"])[A], {"value": 1e-4, "known": 1, "builds": 2}, "one fix: its build") + drawn = fixes.compose(f, ["a", "c"]) + self.assertEqual(named(f, [drawn["item_noise"][A].get(k) for k in range(len(f["items"]))]), + {"p1": 1e-4 + 1e-3, "p2": None, "p3": 1e-4, "p4": None}) # fmt: skip + + def test_an_item_takes_the_noise_of_the_builds_its_result_stands_on_never_another_fixs(self): + """sentier-bafu's probe: three fixes on three items, each build at 1 %, the base at + 0.5 %; c moves p3 by 2.9 %. At p3 the set's result is c's build, so its noise is the + base's and c's (1.5 %) whichever fixes are ticked with c: p3 differs in every set. A + noise summed over every fix (4.5 % for three) would put it within noise as soon as a + is ticked.""" + f = disjoint() + for names in (["c"], ["a", "c"], ["a", "b", "c"]): + with self.subTest(names=names): + self.assertEqual(fixes.compose(f, names)["item_noise"][A][at(f, "p3")], 0.005 + 0.01) + classes = fixes.counts(f, names)["by"][A]["classes"] + self.assertEqual((classes[rule.DIFFERS], classes[rule.WITHIN_NOISE]), (len(names), 0)) + self.assertEqual(fixes.noise_of(f, ["a", "b", "c"])[A], {"value": 0.005 + 0.01, "known": 4, "builds": 4}) + + def test_a_set_counts_each_item_with_its_own_noise(self): + """a moves p1 by 2.9 % with its build at 1 %; c moves p3 by 10 % with its build at 4 %. + In the set of both, p1 is judged with 1.5 % and differs; the set's largest noise (4.5 %) + would put it within noise.""" + data, full = chain_and_matrix(noise={"s1": measured(0.005)}) + f = fixes.build("Two noises", data=data, full=full, base="s1", + fixes=[fixes.fix("a", "Fix a", kind="amount", noise=measured(0.01)), + fixes.fix("c", "Fix c", kind="amount", noise=measured(0.04))], + results={"a": {A: {"p1": 1.029}}, "c": {A: {"p3": 3.3}}}, + interactions=checked_pairs()) # fmt: skip + self.assertEqual(fixes.errors(f), []) + noise = fixes.compose(f, ["a", "c"])["item_noise"][A] + self.assertEqual((noise[at(f, "p1")], noise[at(f, "p3")]), (0.005 + 0.01, 0.005 + 0.04)) + classes = fixes.counts(f, ["a", "c"])["by"][A]["classes"] + self.assertEqual((classes[rule.DIFFERS], classes[rule.WITHIN_NOISE]), (2, 0)) + + def test_a_built_set_is_compared_as_its_own_build(self): + f, *_ = made(noise_ab=measured(5e-3), interactions="none", state_noise={"s1": measured(1e-4)}) + self.assertEqual(fixes.label(f, ["a", "b"]), fixes.BUILT) + self.assertEqual(fixes.noise_of(f, ["a", "b"])[A], {"value": 1e-4 + 5e-3, "known": 2, "builds": 2}) + + def test_a_sum_runs_in_the_order_of_the_file(self): + """Six fixes change p1, the base is the published results. Their builds' noise summed in + the order of the file is 0.26099999999999995; of the 720 orders only that one and the + one with its first two swapped give that float, so an order by id, by the set's own + hashing or by anything else is caught: the page's port must sum as the file runs, not + as an object's keys come. (sentier-bafu's review: 3 fixes already in id order let an + id-sorted sum through.)""" + order = (("f", 0.084), ("b", 0.012), ("e", 0.058), ("a", 0.016), ("d", 0.05), ("c", 0.041)) + data, full = chain_and_matrix() + f = fixes.build("Six on one item", data=data, full=full, base="s1", + fixes=[fixes.fix(x, f"Fix {x}", kind="amount", noise=measured(n)) for x, n in order], + results={x: {A: {"p1": 1.0 + n}} for x, n in order}, + interactions=checked_pairs()) # fmt: skip + + def summed(noises): + level = None + for n in noises: + level = n if level is None else level + n + return level + + in_file = summed(n for _, n in order) + self.assertEqual(fixes.compose(f, sorted(x for x, _ in order))["item_noise"][A][at(f, "p1")], in_file) + alike = [p for p in itertools.permutations(order) if summed(n for _, n in p) == in_file] + self.assertEqual(alike, [order, (order[1], order[0], *order[2:])], "the test no longer tells orders apart") + + def test_a_result_taken_from_a_build_inside_the_set_takes_that_builds_noise(self): + """The set of a, b and c takes p1 in A from the build of a and b: p1 stands on that + build and the base, not on a's and b's own builds; p3 on c's and the base.""" + f, *_ = made(noise_ab=measured(5e-2), interactions="none") + self.assertEqual(fixes.label(f, ["a", "b", "c"]), fixes.EXACT) + noise = fixes.noise_of(f, ["a", "b", "c"]) + self.assertEqual(noise[A], {"value": 5e-2, "known": 1, "builds": 3}) + self.assertEqual(noise[B], {"value": None, "known": 0, "builds": 2}) + self.assertEqual(fixes.compose(f, ["a", "b", "c"])["noise"], noise) + self.assertEqual(fixes.counts(f, ["a", "b", "c"])["by"][A]["classes"][rule.WITHIN_NOISE], 1) + + def test_a_built_set_is_held_to_its_sum_with_the_noise_of_every_build_both_stand_on(self): + """1.00406 is 2.75e-3 from the sum, 1.0013: within the noise of the base three times + (once, and once under each fix), a, b and the built set (6 x 5e-4); beyond it with any + one of them left out (5 x 5e-4), and without the base (3 x 5e-4).""" + kw = {"noise_a": measured(5e-4), "noise_b": measured(5e-4), "noise_ab": measured(5e-4), "interactions": "none", + "built_ab": 1.00406} # fmt: skip + f, *_ = made(state_noise={"s1": measured(5e-4)}, **kw) + self.assertEqual(fixes.errors(f), []) + gone = changed(f, lambda g: g["of"].pop("base_noise")) + self.assertTrue(said(fixes.errors(gone), "built set 'ab', item 'p1', x:a: built 1.00406")) + own = changed(f, lambda g: g["built"][0].__setitem__("noise", chain.noise())) + self.assertTrue(said(fixes.errors(own), "built set 'ab', item 'p1', x:a: built 1.00406")) + + def test_a_built_set_is_held_at_each_item_to_the_builds_its_sum_stands_on_there(self): + """The three fixes built together, p3 1.8 % from c's build: within the noise of that + build and the built set's (2 %), not of the base's and the built set's (1.5 %). At + 2.4 % it is refused, where every fix's noise and the base's (6 %) would let it pass.""" + f = disjoint() + self.assertEqual(fixes.errors(f), []) + p1, p2, p3 = (at(f, p) for p in ("p1", "p2", "p3")) + + def add(value): + return lambda g: g["built"].append({"id": "abc", "label": "All three", "fixes": ["a", "b", "c"], "state": None, + "results": {A: sorted([[p1, 1.5], [p2, 2.5], [p3, value]]), B: []}, + "noise": measured(0.01)}) # fmt: skip + + self.assertEqual(fixes.errors(changed(f, add(3.09 * 1.018))), [], "1.8 % from c's build: within 2 %") + found = fixes.errors(changed(f, add(3.09 * 1.025))) + self.assertTrue(said(found, "built set 'abc', item 'p3', x:a"), found) + + def test_a_built_set_is_held_at_an_item_none_of_its_fixes_change_to_the_base(self): + """a and b built together move p3, which neither changes: there the sum is the base, so + the build is held to it with the base's noise and its own (1.5 %). 1.2 % passes; it + would not with the built set's noise alone (1 %). 1.6 % is refused. (sentier-bafu's + review: no test had a built set move an item none of its fixes move.)""" + f = disjoint() + p1, p2, p3 = (at(f, p) for p in ("p1", "p2", "p3")) + + def add(value): + return lambda g: g["built"].append({"id": "ab", "label": "A and B", "fixes": ["a", "b"], "state": None, + "results": {A: sorted([[p1, 1.5], [p2, 2.5], [p3, value]]), B: []}, + "noise": measured(0.01)}) # fmt: skip + + self.assertEqual(fixes.errors(changed(f, add(3.0 * 1.012))), [], "1.2 % from the base: within 1.5 %") + found = fixes.errors(changed(f, add(3.0 * 1.016))) + self.assertTrue(said(found, "built set 'ab', item 'p3', x:a"), found) + + def test_a_release_is_held_to_its_state_with_the_noise_of_both(self): + """1.0028 is 1.5e-3 from the state's 1.0013: within the two builds' noise (2e-3), + beyond the built set's alone.""" + + def nudge(g): + entries = g["built"][-1]["results"][A] + entries[[k for k, _ in entries].index(at(g, "p1"))][1] = 1.0028 + + f, data, full = made(release=True, state_noise={"s2": measured(1e-3)}, noise_all=measured(1e-3)) + self.assertEqual(fixes.against(changed(f, nudge), data, full), []) + f, data, full = made(release=True, noise_all=measured(1e-3)) + self.assertTrue(said(fixes.against(changed(f, nudge), data, full), "built set 'all', item 'p1', x:a: built 1.0028")) + + def test_a_diagnostics_file_whose_states_carry_no_noise_is_refused(self): + f, data, full = made() + old = changed(data, lambda d: [s.pop("noise", None) for s in d["states"]]) + with self.assertRaises(ValueError) as caught: + fixes.build("x", data=old, full=full, base="s1", fixes=[fixes.fix("a", "A", kind="mapping")], results={}) + self.assertIn(fixes.NO_STATE_NOISE, str(caught.exception)) + self.assertEqual(fixes.against(f, old, full), [fixes.NO_STATE_NOISE]) + + +def disjoint(): + """Three fixes on three items in A, each built twice and measured at 1 %, the base at + 0.5 %: a moves p1 to 1.5, b p2 to 2.5, c p3 from 3.0 to 3.09 (2.9 %). No pair is listed.""" + data, full = chain_and_matrix(noise={"s1": measured(0.005)}) + return fixes.build( + "Three fixes apart", data=data, full=full, base="s1", + fixes=[fixes.fix(x, f"Fix {x}", kind="amount", noise=measured(0.01)) for x in ("a", "b", "c")], + results={"a": {A: {"p1": 1.5}}, "b": {A: {"p2": 2.5}}, "c": {A: {"p3": 3.09}}}, + interactions=checked_pairs(), + ) # fmt: skip + + +class Matrix(unittest.TestCase): + def test_a_result_written_as_the_one_before_is_read_written_out(self): + f, data, full = made() + row = full["values"][[r[0] for r in full["items"]].index("p1")] + self.assertIn(matrix.SAME, row) + self.assertEqual(f["base"][A][at(f, "p1")], 1.0) + self.assertEqual(fixes.against(f, data, full), []) + written = changed(full, lambda m: m["values"][0].__setitem__(1, matrix.SAME)) # p3 in A: 3.0 is not 4.0 + self.assertTrue(said(fixes.against(f, data, written), "the base results in 'x:a' are not the matrix's")) + + +class Command(unittest.TestCase): + def run_cli(self, *args) -> tuple[int, str]: + out = io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(out): + code = cli.main([str(a) for a in args]) + return code, out.getvalue() + + def test_check_holds_a_fixes_file_to_the_file_beside_it(self): + f, data, full = made() + with tempfile.TemporaryDirectory() as folder: + folder = Path(folder) + chain.write(data, folder / chain.FILE) + matrix.write(full, folder / "matrix.json") + fixes.write(f, folder / "fixes.json") + code, out = self.run_cli("check", folder / "fixes.json") + self.assertEqual(code, 0, out) + self.assertIn("3 fixes and 1 built sets hold 5 entries over 4 items; the largest fixes: 'c' 2", out) + fixes.write(changed(f, lambda g: g["base"][B].__setitem__(at(g, "p2"), 5.5)), folder / "fixes.json") + code, out = self.run_cli("check", folder / "fixes.json") + self.assertEqual(code, 1, out) + self.assertIn("the base results in 'x:b' are not the matrix's", out) + (folder / chain.FILE).unlink() + code, out = self.run_cli("check", folder / "fixes.json") + self.assertEqual(code, 1, out) + self.assertIn("names the diagnostics file diagnostics.json, which is not beside it", out) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_modules.py b/tests/test_modules.py index 67dc943..f6d99ce 100644 --- a/tests/test_modules.py +++ b/tests/test_modules.py @@ -20,7 +20,8 @@ HERE = Path(__file__).parent FIXTURES = HERE / "fixtures" / "tables" -MODULES = ["rule", "claims", "chain", "fileformat", "breakdown", "tables", "summary", "page", "from_tables", "matrix", "together"] +MODULES = ["rule", "claims", "chain", "fileformat", "breakdown", "tables", "summary", "page", "from_tables", "matrix", "together", + "fixes"] def load_tests(loader, tests, pattern):