Resource resampling by jaredthomas68 · Pull Request #893 · NatLabRockies/H2Integrate

jaredthomas68 · 2026-09-23T23:40:32Z

Resource-resampling compatible with non-annual resource handling

The work in this PR is also included in #887. This PR splits out the resource-specific changes in #887 that are not already addressed in #897.

This PR contains changes up-sampling and down-sampling data to desired time step (using pandas methods)

Section 1: Type of Contribution

Section 2: Draft PR Checklist

Open draft PR
Describe the feature that will be added
Fill out TODO list steps
Describe requested feedback from reviewers on draft PR
[-] Complete Section 8: New Model Checklist (if applicable)

TODO:

Work with @elenya-grant to merge related work
Make resource resampling non-automatic and warn the user

Type of Reviewer Feedback Requested (on Draft PR)

Structural feedback:

Implementation feedback:

Thoughts on how resampling is being handled

Other feedback:

Section 3: General PR Checklist

Section 4: Related Issues

Resolves #651

Section 5: Impacted Areas of the Software

Section 5.1: New Files

None.

Section 5.2: Modified Files

resource_index.md
- Documents explicit resource resampling settings and warnings.
validation.py
- Retains the one-year simulation-duration restriction while formatting the seconds-per-year constant.
resource_baseclass.py
- Adds opt-in up/downsampling configuration and resamples resource data before timestep clipping.
conftest.py
- Explicitly selects interpolation and aggregation methods in resource integration-test configurations.
test_resource_api_baseclass.py
- Updates test sampling settings and fixes synthetic timestamps to match the intended intervals.
test_time_tools.py
- Covers resampling behavior, leap-day timestamp handling, opt-in errors, and user warnings.
time_tools.py
- Adds timestep resampling with leap-aware timestamp regeneration, explicit method requirements, and warnings.
nlr_developer_wtk_api_models.py
- Accepts filename lists for multi-year wind resource inputs.

Section 6: Additional Supporting Information

Section 7: Test Results, if applicable

Section 8 (Optional): New Model Checklist

jaredthomas68

Lots of what I'm seeing looks very similar to what I have, but with an improved interface and excluding the sampling

elenya-grant

Hi Jared! Thanks for making this PR! I left a few small comments but nothing major. My main questions is why have resample_resource_data_to_dt() called in process_final_resource_data() instead of in get_data_for_year() before the return statements? I see a few possible benefits to calling resample_resource_data_to_dt in get_data_for_year() instead:

If user provided filenames, then the files can have different timesteps. Does your approach work if part of the time index has a timestep of 1 hour and another part has a time index of 30 minutes?
If a user is running non-consecutive years for a multi-year simulation, then would this still work? Like - what happens with the sampling if the 'year' goes from 2012 to 2011 at hour 8761?

I suppose the downside (pretty big one) is that it couldn't interpolate between years. But - maybe that bit could be done in process_final_resource_data()?

(Also - I'm sorry if this suggestion isn't accurate/makes sense - I'm trying to submit this review by EOD so I haven't taken a deep look at the tests you added yet - but I will later and edit this comment as needed)

I'm going to do a deeper-review shortly but wanted to give you my initial thoughts! thanks for making this PR jared!

elenya-grant · 2026-10-08T22:39:28Z

+
+API resource data may have a different native timestep from the simulation, but H2I does not resample it by default. You may explicitly choose a pandas interpolation method for upsampling or a pandas aggregation method for downsampling in the resource's `resource_parameters`.
+
+Set `upsample_method` or `downsample_method` in `resource_parameters` to explicitly select the pandas operation. If the data timestep and simulation timestep already match, neither setting is needed. When resampling is requested, H2I emits a warning naming the source timestep, target timestep, direction, and method. If a mismatch requires resampling but no corresponding method is set, H2I raises an error. See {py:func}`h2integrate.resource.utilities.time_tools.resample_resource_data_to_dt` for method details.


could you add a link to the pandas doc-page for the relevant methods so folks can easily see what sampling methods are available?

elenya-grant · 2026-10-08T22:43:08Z

+    data, start_timestamp, dt_seconds, n, present_time_keys, exclude_leap_day=False
+):
+    """Regenerate time columns as ``n`` steps of ``dt_seconds`` starting at a timestamp.
+


can you add the Args and Returns parts to this doc string?

elenya-grant · 2026-10-08T22:43:59Z

+        - **resource_year_setting** (*str*, optional): How resource years are selected for
+            API datasets. Options include ``start_year``, ``year_order``, and ``filenames``.


remove resource_year_setting (probably left-over from merge)

elenya-grant · 2026-10-08T22:45:49Z

            dict: resource_data after final processing and checks
        """
-        # NOTE: we could up/downsample here also
+        # Remove leap day (if needed) on the native data, resample to the simulation


could you update the doctoring for process_final_resource_data to include the resample step?

elenya-grant · 2026-10-08T22:46:32Z

        """
-        # NOTE: we could up/downsample here also
+        # Remove leap day (if needed) on the native data, resample to the simulation
+        # timestep, then clip to the number of timesteps requested by the simulation.


if you leave this inline comment, can you move "then clip to the number of timesteps requested by the simulation." to the line above clip_data_to_n_timesteps() instead?

I just removed the comment. It is redundant with the doc string

elenya-grant · 2026-10-08T22:56:10Z

+    return data
+
+
+def resample_resource_data_to_dt(


I'm curious how this works for TMY datasets?

idk how it works or if it works as intended but it doesnt raise any errors when I tested it...

This works through the existing exclude_leap_day = spans_leap_year and not has_feb29 check. A TMY whose months come from leap years but has no Feb 29 is treated as "leap day removed", so the regenerated calendar skips it. Year column: for mixed-year TMY (e.g. Jan 2005, Feb 2012, …), the regenerated year column becomes the first sample's year for the whole series. The original per-month years are lost. Anything that relies on the original year labels will no longer see them. I don't think any additional tests are needed.

I also don't think the loss of the per-month years is a problem since we are not using that information anywhere.

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

5 open findings

With resampling now able to produce coarse timesteps (e.g., multi-day dt),… · New
Using round(...) to compute target_n can round down when the ratio is not an integer (or due to… · New
resource_filename is widened to ... | list but the list element type is unspecified, which… · New
resource_filename is widened to ... | list but the list element type is unspecified, which… · New
_make_timeseries is a test helper (not a collected test), but it is marked with… · New

What changed in this PR

This PR adds opt-in resource data resampling so resource timeseries can be upsampled/downsampled to the simulation timestep using pandas interpolation/aggregation, including leap-day-aware timestamp regeneration.

Changes:

Add resample_resource_data_to_dt utility (with warnings + required explicit method selection) and integrate it into resource final-processing.
Update configs/tests/docs to support explicit upsample_method / downsample_method, and expand wind API config to accept filename lists.
Minor validation/doc/changelog updates related to timestep handling and resampling behavior.

File	Description
h2integrate/resource/wind/nlr_developer_wtk_api_models.py	Allow `resource_filename` to be a list for multi-year wind inputs.
h2integrate/resource/utilities/time_tools.py	Implement resampling helper + leap-aware time-column regeneration and warnings.
h2integrate/resource/utilities/test/test_time_tools.py	Add unit tests for resampling behavior, leap-day handling, warnings, and error paths.
h2integrate/resource/test/test_resource_api_baseclass.py	Adjust test configs to explicitly set resampling methods; fix synthetic timestamp minutes.
h2integrate/resource/test/conftest.py	Ensure integration-test configs explicitly set resampling methods.
h2integrate/resource/resource_baseclass.py	Wire resampling into `process_final_resource_data` before clipping and validation.
h2integrate/core/inputs/validation.py	Replace magic number with formatted seconds-per-year constant.
docs/resource/resource_index.md	Document explicit resampling configuration and warning/error behavior.
CHANGELOG.md	Add unreleased entry describing resampling capability.

🧠 Review effort: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…ntain and use ceiling instead of round for determing the number of timesteps required

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open findings

Deriving native_dt from only the first two timestamps can be wrong when the dataset starts… · New
The updated type allows list[Path | str], but the default is still the empty string. This is… · New
The leap-day exclusion branch repeatedly rebuilds pd.date_range in a loop, which can become… · New

5 resolved since last review

🧠 Review effort: Lite

+    # Native timestep is taken from the first two samples so a calendar gap (such as a
+    # removed leap day) does not distort it.
+    native_dt = (calendar_index[1] - calendar_index[0]).total_seconds()


    resource_type: str = field(default="none", init=False)
    resource_data: dict | object = field(default={})
-    resource_filename: Path | str | list = field(default="")
+    resource_filename: Path | str | list[Path | str] = field(default="")


+        periods = n_timesteps
+        new_index = pd.date_range(start=start_timestamp, periods=periods, freq=freq)
+        new_index = new_index[~((new_index.month == 2) & (new_index.day == 29))]
+        while len(new_index) < n_timesteps:
+            periods += (n_timesteps - len(new_index)) + 1
+            new_index = pd.date_range(start=start_timestamp, periods=periods, freq=freq)


enable resource handling for multiple years and non-annual time steps

f1815c6

jaredthomas68 requested a review from elenya-grant September 23, 2026 23:40

Merge branch 'develop' into nonannual-resource

7b6326c

johnjasa changed the title ~~Non-annual resource handling~~ WIP: Non-annual resource handling Sep 24, 2026

jaredthomas68 commented Sep 28, 2026

View reviewed changes

jaredthomas68 added 2 commits October 2, 2026 16:18

Merge branch 'develop' into nonannual-resource

37990b5

resolve conflicts

597f320

jaredthomas68 added the ready for review This PR is ready for input from folks label Oct 5, 2026

jaredthomas68 requested a review from genevievestarke October 5, 2026 17:40

jaredthomas68 changed the title ~~WIP: Non-annual resource handling~~ Resource resampling Oct 6, 2026

Merge branch 'develop' into nonannual-resource

12d2a6f

elenya-grant mentioned this pull request Oct 7, 2026

Nonannual resource models #897

Merged

26 of 41 tasks

johnjasa mentioned this pull request Oct 7, 2026

Support Multi-Year Resource Profiles in Wind and Solar Performance Models #908

Open

johnjasa and others added 6 commits October 7, 2026 16:28

Merging

1329db9

merge from develop

2717b38

fix merge conflicts

80fecee

restore annual simulation only enforment

d439b06

update docs

5c87459

update changelog

ec554c3

jaredthomas68 marked this pull request as ready for review October 8, 2026 19:06

elenya-grant reviewed Oct 8, 2026

View reviewed changes

Comment thread h2integrate/resource/test/test_resource_api_baseclass.py

elenya-grant reviewed Oct 8, 2026

View reviewed changes

jaredthomas68 added 5 commits October 12, 2026 13:21

merge develop

ca8ada4

include pandas docs links for up- and down-sampling of resources

337be0c

update doc string

25087cd

update error message for TMY datasets with filename specs

d167ae4

update doc strings and comment

601e200

jaredthomas68 requested review from elenya-grant and a balanced review from Copilot October 12, 2026 21:59

update class hierarchy

4ad266b

Merge branch 'develop' into nonannual-resource

045410c

Copilot AI reviewed Oct 12, 2026

View reviewed changes

update add_resource_start_end_times to work correctly with dt>24h

a7e83c7

Copilot stopped reviewing on behalf of jaredthomas68 due to an error October 12, 2026 22:37
The job was not acquired by a runner even after multiple attempts.

jaredthomas68 added 2 commits October 12, 2026 16:45

update resource_filename typing to specify what types the list can co…

e310eba

…ntain and use ceiling instead of round for determing the number of timesteps required

remove extraneus test marker

b726a92

jaredthomas68 requested a balanced review from Copilot October 12, 2026 22:48

jaredthomas68 mentioned this pull request Oct 12, 2026

Add core framework support for non-annual simulation durations jaredthomas68/H2Integrate#7

Open

21 of 26 tasks

Copilot AI reviewed Oct 12, 2026

View reviewed changes

jaredthomas68 mentioned this pull request Oct 12, 2026

Nonannual-example jaredthomas68/H2Integrate#6

Open

23 of 28 tasks


		API resource data may have a different native timestep from the simulation, but H2I does not resample it by default. You may explicitly choose a pandas interpolation method for upsampling or a pandas aggregation method for downsampling in the resource's `resource_parameters`.

		Set `upsample_method` or `downsample_method` in `resource_parameters` to explicitly select the pandas operation. If the data timestep and simulation timestep already match, neither setting is needed. When resampling is requested, H2I emits a warning naming the source timestep, target timestep, direction, and method. If a mismatch requires resampling but no corresponding method is set, H2I raises an error. See {py:func}`h2integrate.resource.utilities.time_tools.resample_resource_data_to_dt` for method details.

		- resource_year_setting (str, optional): How resource years are selected for
		API datasets. Options include ``start_year``, ``year_order``, and ``filenames``.

Conversation

jaredthomas68 commented Sep 23, 2026 • edited by johnjasa Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Resource-resampling compatible with non-annual resource handling

Section 1: Type of Contribution

Section 2: Draft PR Checklist

TODO:

Type of Reviewer Feedback Requested (on Draft PR)

Section 3: General PR Checklist

Section 4: Related Issues

Section 5: Impacted Areas of the Software

Section 5.1: New Files

Section 5.2: Modified Files

Section 6: Additional Supporting Information

Section 7: Test Results, if applicable

Section 8 (Optional): New Model Checklist

Uh oh!

jaredthomas68 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elenya-grant left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jaredthomas68 commented Sep 23, 2026 •

edited by johnjasa

Loading