Repository navigation
lib/cluster: Add pytest ctypes test suite - #7919
Conversation
58ca2ed to
d60b222
Compare
|
@echoix Can you review it? I used the method you suggested this time to review code using better model. |
|
If you want to take a look at what your code coverage looks like, I cherry picked your change on top of my work doing code coverage on the C code, and it shows what was touched and what missed https://app.codecov.io/gh/echoix/grass/pull/698/indirect-changes In c_exec, (I_cluster_exec), I think you might be able to get coverage in the argument validation code paths. Take a look at what you think you could cover easily that would still be in scope. |
d60b222 to
46e282a
Compare
|
@echoix Added 8 tests covering Verified with mutation testing: recompiled |
Summary
lib/cluster, the k-means implementation behindi.cluster, currently has no dedicated test coverage. This PR adds a pytest suite that calls the C API directly through the existing ctypes bindings, continuing the test coverage work from #7871 (lib/datetime) and #7879 (lib/arraystats,lib/rowio).The PR adds 20 tests across 3 files, along with a shared test fixture.
lib_cluster_points_ctypes_test.py6 tests
These tests cover the point-collection API without requiring a GRASS session:
I_cluster_begin()I_cluster_point(), including rejection of points with a NULL bandI_cluster_begin_point_set(),I_cluster_point_part(), andI_cluster_end_point_set()I_cluster_clear()lib_cluster_algorithm_ctypes_test.py12 tests
Each major step of the clustering algorithm is tested independently with expected values derived by hand from the algorithm and then confirmed against the implementation.
The suite covers:
I_cluster_means()I_cluster_assign()I_cluster_reassign(), including both the no-op and point-moving casesI_cluster_sum2()I_cluster_separation(), including the fewer-than-two-points caseI_cluster_distinct()I_cluster_nclasses()I_cluster_merge()I_cluster_reclass()The test data uses two bands deliberately. All of these functions iterate over bands, so testing only one band could allow a loop that accidentally ignores additional bands to pass.
The
I_cluster_reassign()case specifically verifies this. The point(1, 9)is closer to class 0 when considering only the first band, but moves to class 1 when both bands are considered.lib_cluster_exec_ctypes_test.py2 tests
These tests cover
I_cluster_exec()end to end using two well-separated groups.The tests verify:
conftest.pyMost
I_cluster_*()functions callG_debug(), which requiresGISRCto be set even though the clustering code does not actually access a GRASS project or mapset.Since ctypes calls read the real process environment, the fixture uses
monkeypatchto provide the required session environment. This follows the same approach used by thepython/grass/temporaltests and the existinggrass_temporal_gui_support_test.pypattern described inAGENTS.md.Tests that require this fixture are marked
needs_solo_run. The point-collection tests do not require a session and run without the fixture.Testing
The full suite was tested against real GRASS builds in both:
osgeo/grass-gis:main-ubuntuosgeo/grass-gis:main-alpineBoth environments are covered because a platform difference in how a missing GRASS session manifests has previously masked a failure in this test approach.
All 20 tests pass in both environments, including repeated runs with
pytest-xdist.The expected values were also independently reproduced using a small C harness linked directly against the compiled libraries rather than relying only on the ctypes test path. The harness passes cleanly under ASan.
The asserted values were checked against
lib/clusterrebuilt with:-O0-O2-O3-ffast-math-O3 -march=nativeThe results were bit-identical across these builds, so the tests are not relying on floating-point tie behavior.
As an additional regression check, I introduced 15 deliberate mutations into the library, including restricting band processing to band 0 and changing behavior in
means,assign,reassign,sum2,separation,merge, and the NULL/all-zero point handling. The test suite caught all 15 mutations.I then restored the library and rebuilt it from the original source without any mutations. The unmutated build passed the full test suite, confirming that the failures were caused by the intentional mutations rather than the build or test setup.
Formatting and linting were also checked with the pinned
ruffversion from.pre-commit-config.yaml(v0.15.17).Not covered
The following paths are intentionally outside the scope of this PR:
I_cluster_exec()interrupt (-2) pathI_cluster_exec()out-of-memory (-1) pathI_cluster_exec_free()Separate issue found
While writing the tests, I found an issue in
I_cluster_point_part().The function reports a NULL value as rejected, but the point can still be retained by
I_cluster_end_point_set(). The latter only checksall_zero(), which reads a band slot that was never written. This can leave the point with stale or uninitialized data and causeband_sumto become inconsistent withnpoints.The batch API is not currently used by
i.cluster, which usesI_cluster_point(), so this does not affect the current tool behavior.I am keeping this separate from the test PR rather than changing library behavior here and will address it separately.
AI Disclosure
I used AI assistance (Claude) while drafting the tests and during the verification process, including the independent C harness, ASan checks, compiler-flag checks, and mutation testing.
I reviewed the tests and verification results myself and made the final decisions about the coverage, expected behavior, and scope of the changes.