Repository navigation
r.sim.water: Add a benchmark for the number of threads - #8002
Merged
Merged
Conversation
Measure r.sim.water with nprocs from 1 to 16 on the elevation of the SECREF North Carolina dataset at 4 m, 2 m and 1 m resolution, using grass.benchmark as the benchmarks of other tools do. The script plots time, speedup and efficiency and saves the results as JSON. Written with the help of Claude Code.
Show the execution time, speedup and efficiency from the benchmark script for 1 to 16 threads on the SECREF elevation at 4 m, 2 m and 1 m, as the documentation of other parallel tools does. The figures were made with the per-walker random number states of OSGeo#7980; with the shared generator on main, more threads make r.sim.water slower, so this belongs after that change. Name the script's output files after the tool, as the figures are. Written with the help of Claude Code.
Plot the mean of the repeated runs with a shaded 95% confidence interval of the mean (Student's t) for time, speedup and efficiency, each band in the color of its line. grass.benchmark.nprocs_plot() draws only a gray min-max band and only for time, so the script plots on its own. The interval of a speedup combines the relative intervals of the two mean times; the speedup at one thread is 1 by definition. Render at 1200 x 900 pixels, twice the 600 pixel width of the documentation, so the figures stay sharp on high density screens; the HTML page shows them at 600 x 450 and links to the full image. The PNGs are quantized to 128 colors and compressed with optipng -o5. The script can redo the plots from its JSON results without rerunning. Written with the help of Claude Code.
With 3 runs per point, a confidence interval of the mean says more about the t quantile than about the runs, so shade the range of the runs instead: from the fastest to the slowest run for time, and over the ratios of every serial run to every parallel run for speedup and efficiency. Drop the sentence that nprocs sets the number of threads, which the parameter description already says. Written with the help of Claude Code.
Follow the documentation of the other parallel tools: a Performance subsection at the end of the notes, opening with the nprocs sentence they share, and the processor named with its clock. Add that the time step, and so the number of iterations, follows from the cell size and the mean flow velocity, so the run time depends on the terrain, as the r.horizon documentation notes for its algorithm. Written with the help of Claude Code.
wenzeslaus
marked this pull request as ready for review
October 7, 2026 04:06
The figures were made with the head of OSGeo#7980 before it was merged. Rerun the benchmark script on this branch merged with main, which now contains OSGeo#7980 (per-walker random numbers) and OSGeo#8013 (the 48-bit multiplication in a helper), with the same sizes, threads and repeats. The times agree with the earlier run within about 2 % (1 m: 113.1 s with 1 thread, 12.2 s with 16, speedup 9.3). Written with the help of Claude Code.
nprocs_plot() shaded the range of the runs in gray and only for time. Shade it in each line's color for time, speedup and efficiency: for speedup, over the ratios of every serial run to every run with the given nprocs, and for efficiency, that divided by nprocs. The range is computed by nprocs_ranges(), which returns None for results without all_times, so such results are plotted again instead of failing. Add figsize and dpi so that figures for the documentation can be made at a higher resolution, and close the figure after saving it. The default title for time was never set because the metric was compared with "times". The subpackage is experimental; say in its documentation that it is changed together with the benchmarks which need more from it. The r.sim.water benchmark now plots with nprocs_plot() instead of its own matplotlib code; its figures are redrawn from the same results. Written with the help of Claude Code.
petrasovaa
approved these changes
Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a benchmark script for r.sim.water with different numbers of threads and its figures in a Performance section of the documentation. The figures show the scaling with the per-walker random number states (#7980, now merged).
It also extends the nprocs plot of grass.benchmark to show the range of the runs for all metrics and to make higher resolution figures.
Details (AI generated)
The script, figures and this description were made with Claude Code and reviewed by the author of the PR.
Benchmark script
raster/r.sim/r.sim.water/benchmark/benchmark_r_sim_water_nprocs.pyruns r.sim.water with nprocs 1 to 16, 3 repeats each in shuffled order (grass.benchmark.benchmark_nprocs), on theelevationraster of the SECREF North Carolina dataset (secref_northcarolina_usa_epsg6542) at 4 m, 2 m and 1 m resolution of the same extent. Coarser than 1 m, the elevation is the average of the 1 m cells (r.resamp.stats); dx and dy come fromr.slope.aspect -e; r.sim.water otherwise runs with its defaults andrandom_seed=1. It plots time, speedup and efficiency withgrass.benchmark.nprocs_plot()at 1200 x 900 pixels. It saves the results as JSON and can redo the plots from that file without rerunning (python benchmark_r_sim_water_nprocs.py r_sim_water_benchmark.json). Resolutions, maximum threads and repeats are constants at the top.Run in a temporary mapset of the dataset:
grass --tmp-mapset ~/grassdata/secref_northcarolina_usa_epsg6542 \ --exec python benchmark_r_sim_water_nprocs.pygrass.benchmark
The subpackage is experimental; its documentation now says that it is changed together with the benchmarks which need more from it.
nprocs_plot()shaded the range of the runs in gray and only for time. It now shades it in each line's color for time, speedup and efficiency: for time between the fastest and the slowest run, for speedup over the ratios of every serial run to every run with the given nprocs, and for efficiency that divided by nprocs. With only a few runs per nprocs, the range of the runs is shown rather than a confidence interval.figsizeanddpiparameters, used here for 1200 x 900 pixel figures.all_timesare plotted again instead of failing (the docstring called it optional), the figure is closed after saving, and the default title for time, which was compared with"times", is set.nprocs_ranges(), tested in the newpython/grass/benchmark/tests/grass_benchmark_plots_test.py(9 tests, under 1 s): the ranges for each metric, the serial runs at the smallest nprocs, results withoutall_times, and the size of the saved image. The existing gunittest suites of the subpackage pass.Figures
The three figures in the documentation come from a run with main including #7980 and the 48-bit multiplication helper of #8013 (c26f5dd), 26 minutes on an Intel Xeon W-2295 (18 cores, 36 logical CPUs) shared with other work (1-minute load average 4 to 6). An earlier run with #7980 at 7c0d776, before it was merged, gave times within about 2 %.
The figures are 1200 x 900 pixels, twice the 600 pixel width of the documentation, so that they stay sharp on high density screens; the HTML page shows them at 600 x 450 and links to the full image. Quantized and compressed as the style guide asks (
pngquant128 colors instead ofpngnq, thenoptipng -o5), 23 to 35 kB each.More threads than 16 were tried separately (3 repeats of 16, 28 and 32 threads in shuffled order): 28 threads took about as long as 16 (1 m: 13.6 s against 14.8 s; 4 m: 0.28 s against 0.19 s) and 32 threads clearly longer (1 m: 19.6 s; 4 m: 2.4 s). The CPU has 18 physical cores and the machine was shared (load average 8 before the run, 30 during it), so beyond 16 the figures would mostly show the load, not r.sim.water; the script stops at 16.
Before #7980, more threads made r.sim.water slower (4 m: 1.3 s with 1 thread, 2.7 s with 4), because all threads shared one random number generator.
Suggested squash commit message