Skip to content

feat(measurement): add calculation-reference provenance to Measurement results - #98

Open
pacharanero wants to merge 2 commits into
livefrom
37-measurement-provenance
Open

feat(measurement): add calculation-reference provenance to Measurement results#98
pacharanero wants to merge 2 commits into
livefrom
37-measurement-provenance

Conversation

@pacharanero

Copy link
Copy Markdown
Member

Summary

Adds a top-level provenance object to every Measurement result:

"provenance": {
  "growth_reference": "uk-who",
  "calculation_engine": {
    "name": "rcpchgrowth",
    "version": "4.6.0",
    "commit": "<40-char SHA, stamped at release build time>"
  }
}

This is step 1 of the sequential mitigation chain for hazard rcpch/digital-growth-charts-documentation#174, where chart curves and a measurement tooltip were found to use different growth references. Downstream steps:

  1. [2/4] Pass Measurement reference provenance through the API schema digital-growth-charts-server#207 - passes this through the API/OpenAPI schema, adding provenance.api_server.
  2. [3/4] Refuse to render measurements from a different growth reference digital-growth-charts-react-component-library#217 - rejects measurements whose provenance doesn't match the intended chart reference.
  3. [4/4] Document and assure the growth-reference mismatch guard digital-growth-charts-documentation#175 - documents the released contract and records QMS/hazard evidence.

Full decision record: spec/growth-reference-provenance-contract.md in the documentation repo (supersedes the flat top-level reference field originally proposed in this issue).

Details

  • growth_reference is the exact value supplied to the calculation (self.reference), not an inferred or age-specific internal table selector.
  • Present on every successful Measurement, including the dates-error early-return path, because it's added once at final self.measurement construction.
  • Propagates automatically through generate_fictional_child_data, which constructs Measurement internally.
  • calculation_engine.version is read via importlib.metadata.version("rcpchgrowth") at import time.
  • calculation_engine.commit defaults to "unknown" (rcpchgrowth/_build_info.py) and is stamped with the release commit SHA by .github/workflows/python-publish.yml immediately before build, so PyPI wheels/sdists carry real provenance while editable/dev installs report "unknown" honestly.
  • Version bumped 4.5.1 -> 4.6.0 (semver-minor, additive).

Testing

New rcpchgrowth/tests/test_provenance.py: all six public references, Trisomy 21 vs Trisomy 21 AAP distinguishability, Turner canonical spelling, provenance survives the dates-error path, and fictional-child propagation for all six references.

Full suite: 145,040 passed, 1,700 skipped, 0 failed.

Closes #37

…t results

Adds a top-level `provenance` object to every Measurement result, containing
the growth reference actually used for calculation and the identity of the
calculation engine that produced it (name, version, build commit).

This is step 1 of the mitigation chain for hazard
rcpch/digital-growth-charts-documentation#174, where chart curves and a
measurement result were found to use different growth references.
Downstream consumers (server, chart component) can compare this provenance
against the intended reference to detect a mismatch before rendering.

- Reports the exact reference supplied to the calculation, not an inferred
  or age-specific table selector.
- Present on every successful Measurement, including the dates-error
  early-return path and fictional-child data (Measurement is the single
  construction point).
- Engine version is read via importlib.metadata at import time; build
  commit defaults to "unknown" and is stamped by the release workflow.

Closes #37
…ronological age

chronological_percentage_median_bmi was computed with age=corrected_age,
identical to corrected_percentage_median_bmi immediately above it. Both
fields returned the same value for every BMI measurement, silently
mislabelling the corrected-age result as chronological.

Invisible whenever corrected and chronological age coincide (term birth,
or a reference that resets/stops gestational correction), which is why
existing fixtures did not catch it. Affects any UK-WHO/Trisomy-21/Turner
child not born at exactly 40+0 weeks gestation, at every age, since those
references retain correction throughout.

Adds a regression test using a preterm UK-WHO child, asserting the two
fields differ and each matches an independently computed expectation -
the previous code could never fail this test.

Hazard raised and scored (severity-minor, likelihood-medium,
risk-level-2-acceptable) for CSO review:
rcpch/digital-growth-charts-documentation#178
@pacharanero

Copy link
Copy Markdown
Member Author

Additional testing: cross-repo, server-boundary evidence

Beyond this branch's own suite, I ran three further checks specifically to answer "does this change the API's observable contract" - relevant given the small additional error-handling fixes now on this branch (returnraise for uk_who_reference(), cdc_reference(), and return_correlation(); the CDC/WHO correction-precedence parenthesisation; the stray WHO_2006_OVER_TWOS_AGES chart-age duplicate) alongside the provenance feature itself.

1. Full suite, current branch state, including the fixes above and their 27 new tests:

145,069 passed, 1,700 skipped, 0 failed

2. Symlinked this exact branch into digital-growth-charts-server in place of the pinned rcpchgrowth==4.5.0, replacing the installed package with a symlink to this working tree (confirmed via os.path.realpath() that it genuinely resolved to this source, not a cached copy). Ran the server's own pytest suite against it:

158 passed, 1 skipped - identical to the pinned 4.5.0 result

3. Built a broad input-sweep regression tool (now in digital-growth-charts-server/regression/) covering all six reference families across every endpoint (calculation, bulk-calculation, chart-coordinates, fictional-child-data, mid-parental-height), both sexes, all four measurement methods, and the clinically significant age boundaries - term, 42 weeks corrected, 2 years, 4 years, the WHO ~5-year data transition, and the ±8/±15 SD validation limits. 878 cases total. Diffed a snapshot taken against the pinned 4.5.0 server against a snapshot taken against this branch.

Result: 16 of 878 cases changed. Every single one is exactly one field, measurement_calculated_values.chronological_percentage_median_bmi - nothing else in any response, on any endpoint, moved. That is precisely the field the corrected-vs-chronological-age fix touches, and nothing beyond it.

Two of those 16 are worth calling out as evidence the fix is strictly safer, not just different: at chronological age exactly 0 (birth, post-term gestation), UK-WHO has no BMI reference data below 2 weeks. Before the fix, that field silently returned the corrected-age value instead of acknowledging the absence. After the fix it correctly returns null. So two of the sixteen changes are "wrong number" → "honestly no data", which is the direction you'd want a contract change to move in even when a hand-wave "it's just a bug fix" wouldn't otherwise reassure a downstream consumer.

No status code, field presence, field type, or any other value changed anywhere in the 878-case sweep. The API's observable contract, outside the one field this PR intentionally corrects, is unchanged.

@eatyourpeas

Copy link
Copy Markdown
Member

I have pulled this branch, run the tests and the jupyter notebook. It all looks good to me.

@mbarton mbarton left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pacharanero. I've also checked it out and made sure I get back the new provenance data in the quickstart notebook.

Just flagging this PR also closes rcpch/digital-growth-charts-documentation#178 as that's a bit lost in the PR thread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[1/4] Add selected growth reference to every Measurement result

3 participants