Repository navigation
Refresh benchmark docs, add large-sheet benchmarks, parse decimals faster - #147
Merged
Merged
Conversation
…-api-v7.0.0 chore: promote public API for v7.0.0
Short run of the whole suite on the idle Ryzen 7 5700X (.NET 10.0.12), with the rows whose Short interval exceeded 25% of the mean rerun on MediumRun. OfficeIMO.Excel 3.4.5 reads XLSX several times faster than the figures published before; the conversion-heavy CSV parallel path gained from FastDecimal but is allocator-bound at dop 16.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #147 +/- ##
=======================================
Coverage 91.06% 91.06%
=======================================
Files 251 252 +1
Lines 17472 17501 +29
Branches 3153 3162 +9
=======================================
+ Hits 15911 15938 +27
- Misses 1070 1072 +2
Partials 491 491 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
crates.io has yanked every zip release except 9.0.3. calamine 0.36.1 needs
zip ^8.6 and rust_xlsxwriter 0.99.1 needs zip ^8.3, so a fresh resolution
fails ("version 8.6.0 is yanked") and every Rust CI job with it. Cargo keeps
a yanked version that a lockfile already holds, so committing the lock
restores CI until both crates move to zip 9. They are dev-dependencies of
the comparison benchmarks only; nothing ships with zip.
Benchmark ResultsBaseline: master Moved by at least 10%
57 within ±10%, 44 with no baseline entry (new, renamed, or reparameterized). Full results per groupExcelReader.Benchmarks.ArrowConversionBenchmark
ExcelReader.Benchmarks.ChunkedParseBenchmark
ExcelReader.Benchmarks.ColdStartBenchmark
ExcelReader.Benchmarks.CsvParallelParseBenchmark
ExcelReader.Benchmarks.CsvParallelVsSepBenchmark
ExcelReader.Benchmarks.CsvParseBenchmark
ExcelReader.Benchmarks.CsvReadBenchmark
ExcelReader.Benchmarks.CsvWriteBenchmark
ExcelReader.Benchmarks.DataReaderBenchmark
ExcelReader.Benchmarks.EncryptedWorkbookBenchmark
ExcelReader.Benchmarks.InflateBenchmark
ExcelReader.Benchmarks.LargeSheetParallelBenchmark
ExcelReader.Benchmarks.NativeOpenBenchmark
ExcelReader.Benchmarks.NativeRowReadBenchmark
ExcelReader.Benchmarks.NativeTypedParseBenchmark
ExcelReader.Benchmarks.ParallelRecordWriteBenchmark
ExcelReader.Benchmarks.ParseBenchmark
ExcelReader.Benchmarks.ReadBenchmark
ExcelReader.Benchmarks.RealDataReadBenchmark
ExcelReader.Benchmarks.RealDataTypedParseBenchmark
ExcelReader.Benchmarks.RecordWriteBenchmark
ExcelReader.Benchmarks.SheetParallelBenchmark
ExcelReader.Benchmarks.StringHeavyReadBenchmark
ExcelReader.Benchmarks.WriteBenchmark
ExcelReader.Benchmarks.WritePathBenchmark
ExcelReader.Benchmarks.XlsReadBenchmark
ExcelReader.Benchmarks.XlsWriteBenchmark
ExcelReader.Benchmarks.XlsxSharedStringHotPathBenchmark
|
…+, Python, and Rust implementations
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change?
Documents how 7.0.0 performs, adds benchmarks for the parallel paths at their largest sizes, and speeds up decimal parsing. None of it changes the public API, so the
FastDecimalchange can go out in a patch release.docs/performance/benchmarks.md, the guides that quote it and the README are refreshed from a full--job Shortrun on the Ryzen 7 5700X (.NET 10.0.12). Rows whose Short interval was wider than 25% of the mean were rerun on--job Medium. There is a new "Large sheets" section, and "Parallel record writes" now comes from BenchmarkDotNet instead of a console harness. Every parallel row shows its allocation.ParallelRecordWriteBenchmark: 1,000,000 rows to aMemoryStream, serial,PrefetchWriteandWriteRecordsParallelAsyncat 1 to 16 batches in flight, XLSX and XLSB.LargeSheetParallelBenchmark: sheets generated in memory at the formats' limits (1,000,000 rows for XLSX and XLSB, 4,000,000 for CSV),SheetParallel.AggregateAsyncandParseAsyncat dop 1 to 16. The CSV aggregate shows the allocation-free path next to the typed one.benchmark-groups.json;CsvGeneratorgainsBuildConversionHeavy, sharing code with the file writer.FastDecimal.Cell.TryParse<decimal>first tries a small ASCII parser (up to 19 digits, optional sign and one., when the culture uses.) and falls back to the existing path for anything else. The result is bit-identical todecimal.Parse, scale included (1.50stays1.50). Other types are unaffected: the branch is folded away per generic instantiation.PublicAPI.Unshipped.txtmove toPublicAPI.Shipped.txt, andPackageValidationBaselineVersiongoes from7.0.0-rc.2to7.0.0. No API is added or removed.What the new numbers change in the published claims:
FastDecimal. The ordered typed parse gets slower from dop 8 to 16 because of gen1 collections, and the doc now says so (DOTNET_gcServer=1halves it in a console harness). The aggregate path is unaffected.FastDecimal. The cold-start table has the new figures.Checklist
dotnet build ExcelReader.slnx --configuration Releasebuilds clean (warnings are errors)dotnet test tests/ExcelReader.Tests/ExcelReader.Tests.csproj --configuration ReleasepassesPublicAPI.Unshipped.txtupdated (nothing added; shipped entries moved)FastDecimalTests)Test plan
FastDecimalTests: bit-exact againstdecimal.Parse, plus rejection cases that must fall back.check-benchmark-groups.shreports every benchmark covered. Both new classes ran end to end under BenchmarkDotNet (--job Medium, 44 benchmarks, error 0 to 6%, one at 12%).