Repository navigation
test(workflow-operator): share Python test discovery - #9406
Conversation
Keep interpreter order, package deadlines, integration tags and each caller’s fail-or-cancel policy in one test utility.
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9406 +/- ##
============================================
- Coverage 91.79% 91.77% -0.02%
- Complexity 5022 5024 +2
============================================
Files 1283 1279 -4
Lines 56874 56215 -659
Branches 7440 7383 -57
============================================
- Hits 52205 51590 -615
+ Misses 2800 2770 -30
+ Partials 1869 1855 -14
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 564 | 0.344 | 17,540/24,994/24,994 us | 🔴 +28.5% / 🔴 +76.3% |
| 🔴 | bs=100 sw=10 sl=64 | 1,178 | 0.719 | 82,677/112,443/112,443 us | 🔴 +10.5% / 🔴 +20.6% |
| 🟢 | bs=1000 sw=10 sl=64 | 1,429 | 0.872 | 699,833/751,484/751,484 us | 🟢 -6.0% / 🟢 -18.4% |
Baseline details
Latest main b83963a from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 564 tuples/sec | 657 tuples/sec | 924.59 tuples/sec | -14.2% | -39.0% |
| bs=10 sw=10 sl=64 | MB/s | 0.344 MB/s | 0.401 MB/s | 0.564 MB/s | -14.2% | -39.0% |
| bs=10 sw=10 sl=64 | p50 | 17,540 us | 16,225 us | 11,019 us | +8.1% | +59.2% |
| bs=10 sw=10 sl=64 | p95 | 24,994 us | 19,456 us | 14,177 us | +28.5% | +76.3% |
| bs=10 sw=10 sl=64 | p99 | 24,994 us | 19,456 us | 17,317 us | +28.5% | +44.3% |
| bs=100 sw=10 sl=64 | throughput | 1,178 tuples/sec | 1,279 tuples/sec | 1,238 tuples/sec | -7.9% | -4.9% |
| bs=100 sw=10 sl=64 | MB/s | 0.719 MB/s | 0.781 MB/s | 0.756 MB/s | -7.9% | -4.9% |
| bs=100 sw=10 sl=64 | p50 | 82,677 us | 74,835 us | 85,949 us | +10.5% | -3.8% |
| bs=100 sw=10 sl=64 | p95 | 112,443 us | 102,053 us | 93,224 us | +10.2% | +20.6% |
| bs=100 sw=10 sl=64 | p99 | 112,443 us | 102,053 us | 115,841 us | +10.2% | -2.9% |
| bs=1000 sw=10 sl=64 | throughput | 1,429 tuples/sec | 1,370 tuples/sec | 1,278 tuples/sec | +4.3% | +11.8% |
| bs=1000 sw=10 sl=64 | MB/s | 0.872 MB/s | 0.836 MB/s | 0.78 MB/s | +4.3% | +11.8% |
| bs=1000 sw=10 sl=64 | p50 | 699,833 us | 735,860 us | 836,081 us | -4.9% | -16.3% |
| bs=1000 sw=10 sl=64 | p95 | 751,484 us | 799,696 us | 878,656 us | -6.0% | -14.5% |
| bs=1000 sw=10 sl=64 | p99 | 751,484 us | 799,696 us | 921,222 us | -6.0% | -18.4% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,354.71,200,128000,564,0.344,17539.82,24994.44,24994.44
1,100,10,64,20,1697.71,2000,1280000,1178,0.719,82677.19,112443.00,112443.00
2,1000,10,64,20,14000.37,20000,12800000,1429,0.872,699833.11,751483.74,751483.74Move the hash-join, interval-join, difference, intersect, symmetric difference and Hugging Face binary-value specs onto PythonTestSupport; each probe keeps its modules. PythonCodeRawInvalidTextSpec uses the shared tag and import probe and keeps its fail-or-cancel policy. configuredPython reads UdfConfig.pythonPath, which UDF_PYTHON_PATH already overrides, so environmentPython goes and the image, 2D histogram and URL visualization specs use configuredPython. Restore the case-insensitive column comments in the regex and substring-search specs, and drop the two resolution comments that no longer describe any code.
bobbai00
left a comment
There was a problem hiding this comment.
🟡 0 must-fix · 2 advisory · 2 polish: a clean, behavior-preserving test refactor. The notes below are optional.
Correctness (1)
PythonTestSupport.scala:50: the probe never reads the child's output, so heavy output reads as a failure (advisory, see inline)
Simplifications (1)
PythonTestSupport.scala:29:maingained a fourth copy inDistinctOpDescSpecafter this branch was cut (advisory, see inline)
Polish: 2 quick touch-ups (see inline comments).
Verification trace
Checked that the refactor preserves behavior. Each of the 29 import probes passes the module list and deadline its removed copy used (60 s, and 120 s in Histogram2DOpDescSpec), and the command string is built the same way. A script comparing base and head for the 33 specs found the same test names in every file and 60 tag registrations on both sides. UdfConfig.pythonPath reads the same key from the same resource the copies parsed. An in-memory merge with main at ed53531 is clean and leaves no call to a removed helper. The pipe finding was reproduced with the same ProcessBuilder calls on JDK 17.
DistinctOpDescSpec gained its own interpreter lookup and import probe on main (apache#8975). It now uses configuredPython and canImport, like the other converted specs, and drops its local NeedsPythonPackages. The shared probe discards the child's output instead of leaving it in a pipe that nothing reads, since a child blocks once that pipe fills. configuredPython documents its lookup order, and the failure message in PythonCodeRawInvalidTextSpec names what was tried.
What changes were proposed in this PR?
Share Python discovery and package probes across 34 operator specs, removing 892 net lines.
PythonTestSupport.configuredPythontries the trimmedUdfConfig.pythonPath, whichUDF_PYTHON_PATHsets, and thenpython3,pythonandpy;PythonWorkerPoolSpeckeeps its PATH-only lookup. Probe deadlines, lazy callers and integration tags are unchanged. Each spec keeps its existing failure or cancellation policy and assertions. The shared probe discards the child's output instead of leaving it in a pipe nothing reads, andPythonCodeRawInvalidTextSpec's failure message lists what the lookup tried.Any related issues, documentation, discussions?
Closes #9401.
How was this PR tested?
With JDK 17 and Python 3.12 (pandas, numpy and plotly installed) first on the PATH:
The 34 suites ran 334 cases: 333 passed, and the existing Windows URL HTML line-ending assertion in
UrlVizOpDescSpecfailed as it does on main. Running main's versions of the same 34 specs gives the same test names, order and outcomes in every suite, and the specs keep the same 61 integration-tag registrations.Apart from
PythonWorkerPoolSpec, every replaced lookup tried the configured interpreter and thenpython3,pythonandpy, asconfiguredPythondoes. The three visualization specs that readUDF_PYTHON_PATHdirectly now get the same value throughudf.conf, trimmed. Every replaced probe imports the same modules within the same deadline. Formatting and lint checks pass.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-6), Claude Code (Claude Opus 5.5, Claude Fable 5.2).