Skip to content

feat(workflow-operator): volcano plot hover column - #9042

Merged
kunwp1 merged 7 commits into
apache:mainfrom
eugenegujing:feat/volcano-plot-hover-column
Nov 16, 2026
Merged

kunwp1 merged 7 commits into
apache:mainfrom
eugenegujing:feat/volcano-plot-hover-column

Conversation

@eugenegujing

@eugenegujing eugenegujing commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

The Volcano Plot uses hover_name=table.columns[0], so the tooltip label is whatever column comes first in the input. That is fine for a DESeq2 table starting with gene and useless for one starting with a composite id or a number, and there is no way to change it. This PR copies Scatter Plot's Hover column property over, one column only.

Hover column Result
empty or whitespace-only (default) hover_name=table.columns[0], unchanged for saved workflows
set, present in the input hover_name=<that column>
set, missing from the input the operator's error page Missing hover column in table., like the existing check for the two required columns (Scatter Plot lets plotly raise a ValueError here)

Since #8504 merged, the operator also exports a standalone script through generateStandaloneCode(), which had hover_name=table.columns[0] hard-coded. This PR mirrors the property there: the hover column goes in as an escaped Python literal, and a set column that the input lacks adds one more elif to the script's column checks, writing the same error page. Whether the field counts as empty is decided by one helper for both paths, so the exported figure and the engine's cannot disagree.

The label fix (#9037) has landed and this branch is merged with it. The hover column is independent of it: the merged operator differs from main only by the hover property, its checks and the two hover_name arguments. The PR for the significance thresholds (#9039) touches neighbouring lines of VolcanoPlotOpDesc.scala, so whichever of the two lands second needs a small merge.

Figures below are the operator's generated Python from main and from this branch (Hover column = gene), run on the 430-gene example workflow from the issue whose first column is transcript|gene|chr|strand; the same point is hovered in both.

Before After (Hover column = gene)
hover-before-first-column hover-after-gene

Any related issues, documentation, discussions?

Closes #9038

PR #8504 (merged) added the generateStandaloneCode() export path with the same hover_name=table.columns[0]; the mirroring described above is the follow-up it needed.

How was this PR tested?

New tests in VolcanoPlotOpDescSpec, written before the source change:

  • empty or whitespace-only field keeps hover_name=table.columns[0] and emits no hover check
  • a set field emits the decoded column as hover_name plus the presence check, and no table.columns[0]
  • a column name with quotes and non-ASCII text reaches the code only as a decode site
  • a workflow JSON saved before this change generates byte-for-byte the same code as a default-constructed operator
  • generateStandaloneCode: an empty field keeps hover_name=table.columns[0] and adds no check; a set field is emitted as a literal plus an exact five-line elif branch between the required-column check and the drawing branch; a name with quotes, non-ASCII text or a backslash comes out escaped rather than decoded; whitespace-only equals empty
sbt -Dsbt.server.forcestart=true "WorkflowOperator/testOnly *VolcanoPlotOpDescSpec"
# Tests: succeeded 24, failed 0, canceled 0, ignored 0, pending 1   (the pending one is pre-existing; the count includes the label tests now on main)

sbt -Dsbt.server.forcestart=true "WorkflowOperator/scalafmtAll" "WorkflowOperator/Test/scalafmtAll"
sbt -Dsbt.server.forcestart=true "WorkflowOperator/scalafixAll --check"
# success

Smoke run outside Texera (pytexera stubbed, pandas + plotly 5.24.1) on the 430-gene mock table: defaults hover-label the first point as ENST00000000000|GENE_0000|chr2|+; Hover column = gene gives GENE_0000; Hover column = no_such_column yields <h1>Volcano Plot failed</h1><p>Missing hover column in table.</p>. The exported script for the default and the gene configuration parses with ast.parse.

Was this PR authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude Code (claude-fable-5-1). Co-authored with Claude; the author reviewed the change before submission.

Add an optional Hover column property, as on Scatter Plot, used as the hover label of each point.
An empty or blank field keeps the first input column as before; a set column missing from the input yields the operator's error page.
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@codecov-commenter

codecov-commenter commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.61%. Comparing base (26f6487) to head (335079c).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #9042   +/-   ##
=========================================
  Coverage     91.61%   91.61%           
- Complexity     5038     5047    +9     
=========================================
  Files          1283     1283           
  Lines         57014    57017    +3     
  Branches       7420     7419    -1     
=========================================
+ Hits          52231    52234    +3     
  Misses         2914     2914           
  Partials       1869     1869           
Flag Coverage Δ *Carryforward flag
access-control-service 77.44% <ø> (ø)
agent-service 99.56% <ø> (ø) Carriedforward from 52093b4
amber 85.34% <100.00%> (+<0.01%) ⬆️
computing-unit-managing-service 64.05% <ø> (ø)
config-service 87.73% <ø> (ø)
file-service 82.31% <ø> (ø) Carriedforward from 52093b4
frontend 96.42% <ø> (+<0.01%) ⬆️ Carriedforward from 52093b4
notebook-migration-service 83.77% <ø> (ø)
pyamber 98.50% <ø> (ø) Carriedforward from 52093b4
workflow-compiling-service 78.30% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 6 worse · ⚪ 7 noise (<±5%) · 0 without baseline

Compared against main 26f6487 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 526 0.321 19,251/21,236/21,236 us 🔴 +27.6% / 🔴 +76.1%
🔴 bs=100 sw=10 sl=64 1,265 0.772 79,736/90,521/90,521 us 🔴 +5.9% / 🟢 -22.1%
🔴 bs=1000 sw=10 sl=64 1,445 0.882 686,546/834,692/834,692 us 🔴 +16.2% / 🟢 -17.8%
Baseline details

Latest main 26f6487 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 526 tuples/sec 618 tuples/sec 927.28 tuples/sec -14.9% -43.3%
bs=10 sw=10 sl=64 MB/s 0.321 MB/s 0.377 MB/s 0.566 MB/s -14.9% -43.3%
bs=10 sw=10 sl=64 p50 19,251 us 15,085 us 10,932 us +27.6% +76.1%
bs=10 sw=10 sl=64 p95 21,236 us 24,805 us 14,458 us -14.4% +46.9%
bs=10 sw=10 sl=64 p99 21,236 us 24,805 us 17,274 us -14.4% +22.9%
bs=100 sw=10 sl=64 throughput 1,265 tuples/sec 1,309 tuples/sec 1,240 tuples/sec -3.4% +2.0%
bs=100 sw=10 sl=64 MB/s 0.772 MB/s 0.799 MB/s 0.757 MB/s -3.4% +2.0%
bs=100 sw=10 sl=64 p50 79,736 us 75,275 us 85,781 us +5.9% -7.0%
bs=100 sw=10 sl=64 p95 90,521 us 93,130 us 93,231 us -2.8% -2.9%
bs=100 sw=10 sl=64 p99 90,521 us 93,130 us 116,160 us -2.8% -22.1%
bs=1000 sw=10 sl=64 throughput 1,445 tuples/sec 1,484 tuples/sec 1,278 tuples/sec -2.6% +13.1%
bs=1000 sw=10 sl=64 MB/s 0.882 MB/s 0.906 MB/s 0.78 MB/s -2.6% +13.1%
bs=1000 sw=10 sl=64 p50 686,546 us 670,491 us 835,630 us +2.4% -17.8%
bs=1000 sw=10 sl=64 p95 834,692 us 718,212 us 877,570 us +16.2% -4.9%
bs=1000 sw=10 sl=64 p99 834,692 us 718,212 us 922,624 us +16.2% -9.5%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,380.20,200,128000,526,0.321,19250.62,21236.19,21236.19
1,100,10,64,20,1581.29,2000,1280000,1265,0.772,79736.04,90521.24,90521.24
2,1000,10,64,20,13840.36,20000,12800000,1445,0.882,686546.28,834691.74,834691.74

@kz930 kz930 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The empty default keeps saved workflows unchanged, and the missing-column error page is nicer than Scatter Plot's ValueError.

…over-column

# Conflicts:
#	common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/volcanoPlot/VolcanoPlotOpDesc.scala

@kunwp1 kunwp1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Remove the export saved-workflow test, compare the engine whitespace and saved-workflow tests against the default operator, and file the two required-field tests back under generatePythonCode.
@kunwp1
kunwp1 added this pull request to the merge queue Nov 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 16, 2026
@kunwp1
kunwp1 added this pull request to the merge queue Nov 16, 2026
Merged via the queue into apache:main with commit f863ad2 Nov 16, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Volcano Plot: hover label is always the first input column; expose a Hover column property like Scatter Plot

4 participants