Skip to content

fix(regression): use absolute targets for MAPE summary values - #1986

Open
anticipate218 wants to merge 1 commit into
evidentlyai:mainfrom
anticipate218:fix/mape-negative-targets
Open

anticipate218 wants to merge 1 commit into
evidentlyai:mainfrom
anticipate218:fix/mape-negative-targets

Conversation

@anticipate218

Copy link
Copy Markdown

For negative regression targets, MAPE summary values currently clip the signed target to epsilon before dividing. For example, a public report with targets [-1, -2, -4] and predictions [-1.1, -1.8, -3.2] reports approximately 1.65e17% instead of 13.33%.

Use the target magnitude in the denominator before applying the existing epsilon floor. This fixes the MAPE summary mean and standard deviation while preserving near-zero handling and existing result units. The separate percentage-error plot calculation is outside this change. Regression tests exercise the public report API with negative and mixed-sign targets, positive controls, zero-handling modes, reference data, and missing/infinite values.

Validation in a clean isolated environment on Windows with Python 3.13.5:

  • New regression module on the original implementation: 4 failed, 4 passed. After the fix: 8 passed.
  • Relevant calculation, legacy regression, modern regression preset and metric tests: 150 passed, including the 8 new cases.
  • Existing MAPE/DummyMAPE field assertions: 48 passed, 1,345 deselected. A temporary pytest plugin disabled only the module's optional test-source rewrite and clipboard side effects. The plugin did not alter test assertions or fixture/source files.
  • Ruff lint and format checks pass for both changed files. All six applicable pre-commit hooks pass; TOML, YAML and Biome hooks have no applicable files. File hashes remain unchanged after hooks.
  • Full mypy checks 585 source files and reports 101 errors in 17 files. Running mypy against an archive of the original base source with the same configuration, interpreter and dependencies produces byte-for-byte identical diagnostics; mypy is not reported as passing.

No full repository runtime suite, notebook smoke tests, live-provider tests, GPU tests, or runtime platform matrix was run. Existing warnings were retained.

AI assistance: OpenAI Codex was used to investigate, implement, validate, and review this change. No separate human code review is claimed.

Preserve the existing epsilon floor and zero-handling policies. Cover negative and mixed targets, reference data, and filtering through the public Report API.

Assisted-by: OpenAI Codex

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant