Repository navigation
feat(workflow-operator): volcano plot hover column - #9042
Conversation
Add an optional Hover column property, as on Scatter Plot, used as the hover label of each point. An empty or blank field keeps the first input column as before; a set column missing from the input yields the operator's error page.
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9042 +/- ##
=========================================
Coverage 91.61% 91.61%
- Complexity 5038 5047 +9
=========================================
Files 1283 1283
Lines 57014 57017 +3
Branches 7420 7419 -1
=========================================
+ Hits 52231 52234 +3
Misses 2914 2914
Partials 1869 1869
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 526 | 0.321 | 19,251/21,236/21,236 us | 🔴 +27.6% / 🔴 +76.1% |
| 🔴 | bs=100 sw=10 sl=64 | 1,265 | 0.772 | 79,736/90,521/90,521 us | 🔴 +5.9% / 🟢 -22.1% |
| 🔴 | bs=1000 sw=10 sl=64 | 1,445 | 0.882 | 686,546/834,692/834,692 us | 🔴 +16.2% / 🟢 -17.8% |
Baseline details
Latest main 26f6487 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 526 tuples/sec | 618 tuples/sec | 927.28 tuples/sec | -14.9% | -43.3% |
| bs=10 sw=10 sl=64 | MB/s | 0.321 MB/s | 0.377 MB/s | 0.566 MB/s | -14.9% | -43.3% |
| bs=10 sw=10 sl=64 | p50 | 19,251 us | 15,085 us | 10,932 us | +27.6% | +76.1% |
| bs=10 sw=10 sl=64 | p95 | 21,236 us | 24,805 us | 14,458 us | -14.4% | +46.9% |
| bs=10 sw=10 sl=64 | p99 | 21,236 us | 24,805 us | 17,274 us | -14.4% | +22.9% |
| bs=100 sw=10 sl=64 | throughput | 1,265 tuples/sec | 1,309 tuples/sec | 1,240 tuples/sec | -3.4% | +2.0% |
| bs=100 sw=10 sl=64 | MB/s | 0.772 MB/s | 0.799 MB/s | 0.757 MB/s | -3.4% | +2.0% |
| bs=100 sw=10 sl=64 | p50 | 79,736 us | 75,275 us | 85,781 us | +5.9% | -7.0% |
| bs=100 sw=10 sl=64 | p95 | 90,521 us | 93,130 us | 93,231 us | -2.8% | -2.9% |
| bs=100 sw=10 sl=64 | p99 | 90,521 us | 93,130 us | 116,160 us | -2.8% | -22.1% |
| bs=1000 sw=10 sl=64 | throughput | 1,445 tuples/sec | 1,484 tuples/sec | 1,278 tuples/sec | -2.6% | +13.1% |
| bs=1000 sw=10 sl=64 | MB/s | 0.882 MB/s | 0.906 MB/s | 0.78 MB/s | -2.6% | +13.1% |
| bs=1000 sw=10 sl=64 | p50 | 686,546 us | 670,491 us | 835,630 us | +2.4% | -17.8% |
| bs=1000 sw=10 sl=64 | p95 | 834,692 us | 718,212 us | 877,570 us | +16.2% | -4.9% |
| bs=1000 sw=10 sl=64 | p99 | 834,692 us | 718,212 us | 922,624 us | +16.2% | -9.5% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,380.20,200,128000,526,0.321,19250.62,21236.19,21236.19
1,100,10,64,20,1581.29,2000,1280000,1265,0.772,79736.04,90521.24,90521.24
2,1000,10,64,20,13840.36,20000,12800000,1445,0.882,686546.28,834691.74,834691.74
kz930
left a comment
There was a problem hiding this comment.
LGTM. The empty default keeps saved workflows unchanged, and the missing-column error page is nicer than Scatter Plot's ValueError.
…over-column # Conflicts: # common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/volcanoPlot/VolcanoPlotOpDesc.scala
Remove the export saved-workflow test, compare the engine whitespace and saved-workflow tests against the default operator, and file the two required-field tests back under generatePythonCode.
What changes were proposed in this PR?
The Volcano Plot uses
hover_name=table.columns[0], so the tooltip label is whatever column comes first in the input. That is fine for a DESeq2 table starting withgeneand useless for one starting with a composite id or a number, and there is no way to change it. This PR copies Scatter Plot'sHover columnproperty over, one column only.hover_name=table.columns[0], unchanged for saved workflowshover_name=<that column>Missing hover column in table., like the existing check for the two required columns (Scatter Plot lets plotly raise aValueErrorhere)Since #8504 merged, the operator also exports a standalone script through
generateStandaloneCode(), which hadhover_name=table.columns[0]hard-coded. This PR mirrors the property there: the hover column goes in as an escaped Python literal, and a set column that the input lacks adds one moreelifto the script's column checks, writing the same error page. Whether the field counts as empty is decided by one helper for both paths, so the exported figure and the engine's cannot disagree.The label fix (#9037) has landed and this branch is merged with it. The hover column is independent of it: the merged operator differs from
mainonly by the hover property, its checks and the twohover_namearguments. The PR for the significance thresholds (#9039) touches neighbouring lines ofVolcanoPlotOpDesc.scala, so whichever of the two lands second needs a small merge.Figures below are the operator's generated Python from
mainand from this branch (Hover column =gene), run on the 430-gene example workflow from the issue whose first column istranscript|gene|chr|strand; the same point is hovered in both.Any related issues, documentation, discussions?
Closes #9038
PR #8504 (merged) added the
generateStandaloneCode()export path with the samehover_name=table.columns[0]; the mirroring described above is the follow-up it needed.How was this PR tested?
New tests in
VolcanoPlotOpDescSpec, written before the source change:hover_name=table.columns[0]and emits no hover checkhover_nameplus the presence check, and notable.columns[0]generateStandaloneCode: an empty field keepshover_name=table.columns[0]and adds no check; a set field is emitted as a literal plus an exact five-lineelifbranch between the required-column check and the drawing branch; a name with quotes, non-ASCII text or a backslash comes out escaped rather than decoded; whitespace-only equals emptySmoke run outside Texera (pytexera stubbed, pandas + plotly 5.24.1) on the 430-gene mock table: defaults hover-label the first point as
ENST00000000000|GENE_0000|chr2|+; Hover column =genegivesGENE_0000; Hover column =no_such_columnyields<h1>Volcano Plot failed</h1><p>Missing hover column in table.</p>. The exported script for the default and thegeneconfiguration parses withast.parse.Was this PR authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code (claude-fable-5-1). Co-authored with Claude; the author reviewed the change before submission.