Repository navigation
refactor(pyamber, amber): remove redundant Python code - #9550
Open
aglinxinyuan wants to merge 1 commit into
Open
aglinxinyuan wants to merge 1 commit into
aglinxinyuan wants to merge 1 commit into
Conversation
Contributor
Automated Reviewer SuggestionsBased on the
|
Codecov Report❌ Patch coverage is
Additional details and impacted files
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 490 | 0.299 | 18,830/39,338/39,338 us | 🔴 +54.9% / 🔴 +207.1% |
| 🟢 | bs=100 sw=10 sl=64 | 1,247 | 0.761 | 82,141/87,858/87,858 us | 🟢 -11.9% / 🟢 -15.3% |
| 🔴 | bs=1000 sw=10 sl=64 | 1,429 | 0.872 | 700,032/825,342/825,342 us | 🔴 +10.8% / 🔴 +8.0% |
Baseline details
Latest main 4db10cf from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 490 tuples/sec | 586 tuples/sec | 1,084 tuples/sec | -16.4% | -54.8% |
| bs=10 sw=10 sl=64 | MB/s | 0.299 MB/s | 0.357 MB/s | 0.662 MB/s | -16.2% | -54.8% |
| bs=10 sw=10 sl=64 | p50 | 18,830 us | 16,817 us | 9,548 us | +12.0% | +97.2% |
| bs=10 sw=10 sl=64 | p95 | 39,338 us | 25,403 us | 12,811 us | +54.9% | +207.1% |
| bs=10 sw=10 sl=64 | p99 | 39,338 us | 25,403 us | 15,432 us | +54.9% | +154.9% |
| bs=100 sw=10 sl=64 | throughput | 1,247 tuples/sec | 1,274 tuples/sec | 1,467 tuples/sec | -2.1% | -15.0% |
| bs=100 sw=10 sl=64 | MB/s | 0.761 MB/s | 0.777 MB/s | 0.895 MB/s | -2.1% | -15.0% |
| bs=100 sw=10 sl=64 | p50 | 82,141 us | 79,151 us | 74,441 us | +3.8% | +10.3% |
| bs=100 sw=10 sl=64 | p95 | 87,858 us | 99,697 us | 81,800 us | -11.9% | +7.4% |
| bs=100 sw=10 sl=64 | p99 | 87,858 us | 99,697 us | 103,712 us | -11.9% | -15.3% |
| bs=1000 sw=10 sl=64 | throughput | 1,429 tuples/sec | 1,425 tuples/sec | 1,517 tuples/sec | +0.3% | -5.8% |
| bs=1000 sw=10 sl=64 | MB/s | 0.872 MB/s | 0.87 MB/s | 0.926 MB/s | +0.2% | -5.8% |
| bs=1000 sw=10 sl=64 | p50 | 700,032 us | 707,138 us | 725,911 us | -1.0% | -3.6% |
| bs=1000 sw=10 sl=64 | p95 | 825,342 us | 744,895 us | 764,152 us | +10.8% | +8.0% |
| bs=1000 sw=10 sl=64 | p99 | 825,342 us | 744,895 us | 808,901 us | +10.8% | +2.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,408.04,200,128000,490,0.299,18829.71,39338.04,39338.04
1,100,10,64,20,1604.03,2000,1280000,1247,0.761,82141.28,87858.36,87858.36
2,1000,10,64,20,13999.70,20000,12800000,1429,0.872,700031.65,825341.88,825341.88
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Remove redundant statements, unused imports, unnecessary control-flow branches, and legacy idioms across
amber/src/main/python/andamber/src/main/scala/org/apache/texera/web/resource/aiassistant/type_annotation_visitor.pywith no runtime behavior change:passstatements in docstring-only abstract methods and marker classes (InternalMarker,BufferedItemWriter,ReadonlyVirtualDocument, andUDFOperatorV2/UDFSourceOperator/UDFTableOperator/UDFBatchOperator), remove trailingreturn NoneinBackpressureHandler, drop unusedimport core, unnecessary dict unpacking, and intermediate return variables iniceberg_utilsandtimestamp, and remove the unusedenumerateindex inTypeAnnotationVisitor.else/elifblocks followingreturnindetect_attribute_type_from_arrow_field,Table.__eq__,DocumentFactory, andIcebergCatalogInstance.get_instance, and merge identicalstaticmethoddecorator checks inTypeAnnotationVisitor.list()/dict()/bytes()with literal forms ([]/{}/b""), replacesuper(ProxyServer, self)withsuper(), replacemap(lambda ...)with a generator expression and simplify the payload conditional inProxyServer, adoptdatetime.UTCand@functools.cacheinattribute_type, and use!r/!sf-strings inSchema,ProxyServer,Heartbeat, andExpressionEvaluator.Any related issues, documentation, discussions?
Closes #9543.
How was this PR tested?
Ran in
amber/on2a319878af(Python 3.12.10,ruff 0.14.7,pytest 7.4.0):ruff checkandruff format --check(218files): passed.test_type_annotation_visitor.py:5 passed.18targetedamberPython unit-test suites:320 passed, 9 skipped(test_attribute_type.pyWindowstime.tzsetskips).test_iceberg_utils_catalog.py(excluding the3s3fs-dependent cases absent locally):19 passed.test_expression_evaluator.py(excluding the2--import-mode=importlibrepr cases):3 passed.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Antigravity (Gemini 4 Neon)