Skip to content

fix deviation false positives with degenerate zero-area polygons - #209

Closed
mrshannon wants to merge 1 commit into
mapbox:mainfrom
mrshannon:deviation-degenerate-polygon-fix
Closed

fix deviation false positives with degenerate zero-area polygons#209
mrshannon wants to merge 1 commit into
mapbox:mainfrom
mrshannon:deviation-degenerate-polygon-fix

Conversation

@mrshannon

Copy link
Copy Markdown

Problem

deviation() returns a spurious 1 for a correctly-triangulated zero-area (collinear) polygon. It guards the 0/0 case with an exact polygonArea === 0, but the non-robust shoelace leaves a tiny nonzero residual for a non-axis-aligned collinear ring, so the guard is skipped and it computes |0 − ε| / ε = 1.

const p = [0.1, 0.2, 1.3, 2.6, 2.5, 5.0, 3.7, 7.4]; // zero-area, collinear
deviation(p, null, 2, earcut(p)); // 1  → should be 0

Fix

Compute a per-ring floating-point error bound alongside the signed area, and treat |polygonArea| within that bound as zero:

  • New private helper signedAreaWithError() mirrors signedArea() but also accumulates Σ|terms| and returns {sum, error}, with error = Σ|terms| · (n + 2) · u  (u = EPSILON / 2, n = ring vertex count) - the worst-case roundoff bound of the shoelace sum.
  • deviation() accumulates polygonArea and polygonAreaError per ring, then returns 0 when Math.abs(polygonArea) <= polygonAreaError && trianglesArea === 0, otherwise the original relative formula is unchanged.

The test is scale-relative (correct at any coordinate magnitude), unlike a fixed epsilon. signedArea() (on the hot path) is left untouched.

Why it's safe

The new branch only fires when trianglesArea === 0 (earcut produced no triangles). Every polygon that yields triangles takes the identical relative branch as before, so no existing deviation value changes. Verified over the whole fixture suite by computing deviation with and without the guard — they are byte-identical except the five zero-triangle rows:

NOTE: deviation (raw) is with the existing exact 0.0 guard removed as well

fixture edges tris polygonArea trianglesArea polygonAreaError deviation (raw) deviation (guarded)
water-huge3 12864 15470 1.543e+7 1.543e+7 3.656e-5 0.000e+0 0.000e+0
water-huge 5667 5174 5.237e+6 5.246e+6 8.522e-5 1.743e-3 1.743e-3
earcut 548 548 8.142e+7 8.142e+7 3.881e-6 0.000e+0 0.000e+0
issue142 14 4 1.400e+2 1.579e+2 3.553e-12 1.275e-1 1.275e-1
issue17 11 11 3.201e+15 3.201e+15 2.133e+0 1.562e-16 1.562e-16
issue107 14 0 -1.717e-3 0.000e+0 2.357e-14 1.000e+0 1.000e+0
issue83 200 0 0.000e+0 0.000e+0 4.263e-8 NaN 0.000e+0
empty-square 8 0 0.000e+0 0.000e+0 4.263e-8 NaN 0.000e+0
degenerate 6 0 0.000e+0 0.000e+0 7.105e-11 NaN 0.000e+0
infinite-loop-jhl 6 0 0.000e+0 0.000e+0 7.772e-15 NaN 0.000e+0
collinear (inline regression) 4 0 1.776e-15 0.000e+0 3.645e-14 1.000e+0 0.000e+0

Rows where the guard changes the result (all zero-triangle):

issue83, empty-square, degenerate, infinite-loop-jhl:  NaN → 0   (exact 0/0)
collinear (inline regression):                          1  → 0   (the bug)

Takeaways:

  • error / area never exceeds ~1.6e-11 across the 54 triangulated fixtures — the bound is ~10+ orders of magnitude from ever affecting a real polygon.
  • A "large" absolute error is harmless (issue17: error 2.13, area 3.2e15, deviation 1.6e-16) — the error scales with the area, and the row triangulates so the bound is never consulted.
  • issue107 (0 triangles, real area -1.7e-3) is not masked: |area| ≫ error, so it still reports 1. The bound distinguishes "truly zero" from "small but real."
Full table — every fixture + the inline regression case
fixture edges tris polygonArea trianglesArea polygonAreaError deviation (raw) deviation (guarded)
water-huge3 12864 15470 1.543e+7 1.543e+7 3.656e-5 0.000e+0 0.000e+0
water-huge 5667 5174 5.237e+6 5.246e+6 8.522e-5 1.743e-3 1.743e-3
water-huge2 5488 4484 1.561e+7 1.564e+7 5.416e-5 2.246e-3 2.246e-3
touching-holes6 3001 3098 2.689e+7 2.689e+7 8.494e-6 0.000e+0 0.000e+0
rain 2693 2681 1.156e+7 1.156e+7 2.653e-5 0.000e+0 0.000e+0
self-tangent-4 2616 3082 2.361e+6 2.361e+6 1.502e-5 0.000e+0 0.000e+0
water 2523 2482 3.521e+6 3.524e+6 5.479e-5 8.111e-4 8.111e-4
eberly-6 1418 1428 8.488e+1 8.488e+1 6.728e-11 1.674e-15 1.674e-15
water2 1235 1211 1.379e+7 1.379e+7 1.672e-5 0.000e+0 0.000e+0
hilbert 1027 1024 1.054e+3 1.054e+3 1.820e-9 0.000e+0 0.000e+0
issue35 820 844 2.335e+7 2.335e+7 4.215e-6 0.000e+0 0.000e+0
water4 720 705 3.229e+7 3.229e+7 3.076e-6 0.000e+0 0.000e+0
earcut 548 548 8.142e+7 8.142e+7 3.881e-6 0.000e+0 0.000e+0
touching-holes5 213 133 1.318e+3 1.318e+3 2.140e-11 0.000e+0 0.000e+0
water3 212 197 3.279e+7 3.279e+7 3.675e-7 0.000e+0 0.000e+0
issue83 200 0 0.000e+0 0.000e+0 4.263e-8 NaN 0.000e+0
self-tangent-3 147 174 2.966e+4 2.966e+4 1.105e-8 0.000e+0 0.000e+0
issue34 135 138 2.760e+6 2.760e+6 2.157e-8 0.000e+0 0.000e+0
simplified-us-border 126 120 2.831e+5 2.831e+5 6.979e-8 0.000e+0 0.000e+0
self-touching 125 124 7.072e-2 7.072e-2 4.956e-13 3.356e-14 3.356e-14
touching-holes3 109 82 6.590e+2 6.590e+2 1.070e-11 0.000e+0 0.000e+0
touching-holes4 109 55 1.659e+3 1.659e+3 1.148e-11 0.000e+0 0.000e+0
issue52 107 108 4.951e+5 4.951e+5 3.819e-8 0.000e+0 0.000e+0
dude 104 106 2.981e+4 2.981e+4 1.076e-8 1.465e-15 1.465e-15
issue186 83 41 4.000e+4 4.000e+4 2.642e-9 0.000e+0 0.000e+0
hole-touching-outer 81 77 7.570e+6 7.570e+6 3.245e-7 0.000e+0 0.000e+0
eberly-3 76 73 2.422e+5 2.422e+5 7.215e-8 0.000e+0 0.000e+0
outside-ring 69 64 4.459e+4 4.459e+4 7.483e-9 0.000e+0 0.000e+0
boxy 69 58 3.694e+4 3.694e+4 1.717e-8 0.000e+0 0.000e+0
touching-holes 57 57 1.323e+4 1.323e+4 2.523e-9 0.000e+0 0.000e+0
bad-hole 48 42 4.284e+3 4.364e+3 3.694e-9 1.867e-2 1.867e-2
issue29 40 40 3.748e+3 3.748e+3 6.269e-11 6.067e-16 6.067e-16
water3b 32 25 3.771e+7 3.771e+7 2.998e-8 0.000e+0 0.000e+0
issue147 30 30 3.301e+7 3.301e+7 8.703e-8 0.000e+0 0.000e+0
issue111 26 18 2.869e+4 2.869e+4 5.074e-9 0.000e+0 0.000e+0
self-tangent-2 25 25 4.168e+3 4.168e+3 2.851e-9 0.000e+0 0.000e+0
collinear-diagonal 24 14 1.661e+4 1.661e+4 1.245e-8 0.000e+0 0.000e+0
self-tangent-1 23 23 4.668e+3 4.668e+3 2.704e-9 0.000e+0 0.000e+0
filtered-bridge-jhl 22 25 4.370e+2 4.370e+2 7.223e-13 0.000e+0 0.000e+0
touching3 18 15 1.045e+3 1.045e+3 2.786e-10 0.000e+0 0.000e+0
issue119 16 18 3.540e+2 3.540e+2 1.266e-12 0.000e+0 0.000e+0
touching4 16 19 1.680e+2 1.680e+2 3.275e-13 0.000e+0 0.000e+0
issue149 16 2 5.120e+2 5.120e+2 1.881e-9 0.000e+0 0.000e+0
building 15 13 5.214e+3 5.214e+3 8.243e-11 0.000e+0 0.000e+0
issue131 15 12 5.136e+7 5.136e+7 5.370e-8 0.000e+0 0.000e+0
issue107 14 0 -1.717e-3 0.000e+0 2.357e-14 1.000e+0 1.000e+0
issue142 14 4 1.400e+2 1.579e+2 3.553e-12 1.275e-1 1.275e-1
touching-holes2 13 10 9.090e+2 9.090e+2 1.309e-12 0.000e+0 0.000e+0
issue16 12 12 1.288e+3 1.288e+3 9.342e-12 1.766e-16 1.766e-16
issue17 11 11 3.201e+15 3.201e+15 2.133e+0 1.562e-16 1.562e-16
bad-diagonals 11 7 8.384e+3 8.384e+3 6.770e-9 0.000e+0 0.000e+0
issue45 10 10 8.000e+2 8.000e+2 1.971e-12 0.000e+0 0.000e+0
shared-points 10 4 4.352e+3 4.352e+3 5.518e-10 0.000e+0 0.000e+0
touching2 10 8 1.796e+4 1.796e+4 9.180e-10 0.000e+0 0.000e+0
empty-square 8 0 0.000e+0 0.000e+0 4.263e-8 NaN 0.000e+0
steiner 8 9 2.000e+4 2.000e+4 1.332e-11 0.000e+0 0.000e+0
degenerate 6 0 0.000e+0 0.000e+0 7.105e-11 NaN 0.000e+0
hourglass 6 2 3.400e+1 3.400e+1 6.981e-13 0.000e+0 0.000e+0
infinite-loop-jhl 6 0 0.000e+0 0.000e+0 7.772e-15 NaN 0.000e+0
collinear (inline regression) 4 0 1.776e-15 0.000e+0 3.645e-14 1.000e+0 0.000e+0

Testing

  • All 248 existing tests pass unchanged.
  • Added a regression test: deviation is zero for a degenerate collinear polygon.

Notes

  • Bound choice. Uses the tight worst-case bound (n + 2)·u rather than a looser 2n·u. For a correctness check the fail-safe direction is toward flagging, not masking, so the smaller-error bound is preferable — and (n + 2)·u is the minimum that still guarantees catching every genuinely zero-area ring. Both bounds produce identical deviation results across the entire suite; only the internal polygonAreaError differs.

@mrshannon
mrshannon requested a review from a team as a code owner September 3, 2026 18:33
@mrshannon
mrshannon requested review from Xdudu and removed request for a team September 3, 2026 18:33
@mourner
mourner requested review from mourner and removed request for Xdudu September 3, 2026 19:26
@mourner

mourner commented Sep 3, 2026

Copy link
Copy Markdown
Member

Thanks for the contribution! Intuitively, this seems like an overkill for such a minor problem we're never hitting in practice — adding quite a bit of code for minuscule gain. Are there any alternatives that make deviation less prone to catastrophic miscalculation with a much more minimal change?

@mrshannon

mrshannon commented Sep 3, 2026

Copy link
Copy Markdown
Author

this seems like an overkill for such a minor problem we're never hitting in practice — adding quite a bit of code for minuscule gain. Are there any alternatives that make deviation less prone to catastrophic miscalculation with a much more minimal change?

Not without intruding a false negative:

    // current
    return polygonArea === 0 && trianglesArea === 0 ? 0 :
        Math.abs((trianglesArea - polygonArea) / polygonArea);

    // minimal fix (misses zero-area triangulation with non-zero polygon)
    return polygonArea === 0 || trianglesArea === 0 ? 0 :
        Math.abs((trianglesArea - polygonArea) / polygonArea);

This solves the bug, but introduces a new one where an invalid zero-area triangulation would return 0 deviation.

It is possible to run the error calculation only if trianglesArea === 0 to avoid the extra cost.

The only other way I know to solve the issue is to put the decision on the caller by changing the public API shape by allowing the user to get both the triangle and polygon areas so they can use an appropriate error bounds themselves. The MR is the only way I know to determine this error bounds without knowledge of the domain.

never hitting in practice

I don't know about in mapbox but I am hitting this case in practice when using earcut as a library in another project.

@mourner

mourner commented Sep 4, 2026

Copy link
Copy Markdown
Member

Found an easier solution for this in 0302a75 — should be good enough for most cases, right?

@mourner mourner closed this Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants