You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
MANIFEST_SPEC says a row filter takes exactly one operator, but the loader accepts several and keeps NaN rows for the negated ones #999
Where:faircode/manifest.pyRowFilter (__post_init__ and apply()), faircode/MANIFEST_SPEC.md (section row_filters), tests/test_manifest.py.
Current gap: the spec says every filter needs "exactly one of the operators below", but __post_init__ only rejects a filter with none, and apply() ANDs whatever is set, so isin plus equals in one block silently narrows twice. The spec also does not say what happens to a missing value: not_isin and not_equals keep NaN rows (NaN != "A" is true), while isin and equals drop them, which #977's tests now pin down but the document never states.
Repro:
>>> import pandas as pd; from faircode.manifest import RowFilter
>>> df = pd.DataFrame({"race": ["A", "B", "C", None, "A"]})
>>> RowFilter(column="race", not_isin=["C"], equals="A").apply(df).index.tolist() # accepted, two operators
[0, 4]
>>> RowFilter(column="race", not_equals="A").apply(df).index.tolist() # the NaN row (3) is kept
[1, 2, 3]
Suggested approach: decide between "reject more than one operator" and "document AND semantics", state the NaN rule for each operator in MANIFEST_SPEC, and add a validation test for whichever is chosen.
Activity
fix: a manifest row filter takes exactly one operator, and the NaN ru…
yakew7 commentedon Oct 16, 2026
Fixed in 15a52c2 (on main). Full suite green (941 passed).