Allow for non-annual simulation.

[jaredthomas68]

Allow for non-annual simulation

This PR adds capabilities to run H2I for non-annual simulations. The approach taken allows for data and simulation on nearly any time step and number of steps. Annual simulation data is then reduced to lists of capacity factors, varopex, and a replacement schedule, using some extrapolation if needed to reach plant life. The resulting lists are then used in the ProFAST cash flow analysis to estimate life-time financial metrics.

The following is an incomplete list of changes

1. General
   • n_timesteps*dt no longer has to equal 1 year
   • Example 24 has been updated to demonstrate non-annual and non-hourly simulation within a single run script.
2. Performance model changes
   • Output capacity factor and replacement schedule are determined based on all simulation time completed
   • Extrapolation to years not simulated can be done using tiling of simulated years, repeating the final year, or using the mean CF.
   • Similar extrapolation approaches can be applied to state-of-health (SOH)/degradation when supplied and will help determine the replacement schedule
   • Adjusted to handle non-hourly simulation more generally
3. Cost models
   • Adjusted to handle non-hourly simulation and use multiple simulation years for varopex
4. Finance models
   • ProFAST wrapper has been adjusted to not assume hourly timesteps or annual simulation.
5. Resource models/api calls
   • Data for multiple years will be downloaded automatically to accommodate the simulation length requested
   • Data for multiple years can be supplied by providing a list of resource filenames where each file is assumed to contain a single year of data and the files are assumed to be given in chronological order
   • Leap can be included or excluded from the data if available in the downloaded or provided data
   • Data may be upsampled or downsampled using pandas resampling methods to meet the requested timestep. The sampling method can be changed by the user.

Section 1: Type of Contribution

• Feature Enhancement
  • Framework
  • New Model
  • Updated Model
  • Tools/Utilities
  • Other (please describe):
• Bug Fix
• Documentation Update
• CI Changes
• Other (please describe):

Section 2: Draft PR Checklist

• Open draft PR
• Describe the feature that will be added
• Fill out TODO list steps
• Describe requested feedback from reviewers on draft PR
• Complete Section 8: New Model Checklist (if applicable)

TODO:

• Make sure user is aware if resampling is done
• Add documentation

Type of Reviewer Feedback Requested (on Draft PR)

Structural feedback:

• Suggestions regarding use of time_tools.py functions vs class methods
• Code reuse or other ideas for reducing the number of changes

Implementation feedback:

• I have used a couple try statements and while loops. I'd love any ideas you have for avoiding those as they can be error prone and hard to read.

Other feedback:

• Suggestions on how/if to split this PR into smaller PRs
• Anything else you want to comment on

Section 3: General PR Checklist

• PR description thoroughly describes the new feature, bug fix, etc.
• Added tests for new functionality or bug fixes
• Tests pass (If not, and this is expected, please elaborate in the Section 6: Test Results)
• Documentation
  • Docstrings are up-to-date
  • Related docs/ files are up-to-date, or added when necessary
  • Documentation has been rebuilt successfully
  • Examples have been updated (if applicable)
• CHANGELOG.md
  • At least one complete sentence has been provided to describe the changes made in this PR
  • After the above, a hyperlink has been provided to the PR using the following format:
    "A complete thought. [PR XYZ]((https://github.com/NatLabRockies/H2Integrate/pull/XYZ)", where
    XYZ should be replaced with the actual number.

Section 4: Related Issues

This PR relates to several issues, I may have missed some:

• Create up sampling and down sampling capabilities #651 - upsampling and downsampling
• Allow variable time step number and length #204 - variable timestep number and length
• Generalize dispatch timesteps (allow different than hourly) #250 - allow non-hourly dispatch timesteps
  • This PR only addresses this for DemandOpenLoopStorageController
• Enable grid model to work with non-hourly timesteps #638 - non-hourly grid model

Section 5: Impacted Areas of the Software

Section 5.1: New Files

• test_model_baseclasses.py
  • calculate_annual_cf_and_replacement_schedule: Adds unit tests for annualization and replacement-schedule projection.
• test_resource_base.py
  • _acquire_resource_data: Adds tests for multi-year resource acquisition and ordered filename-list handling.
• test_time_tools.py
  • process_leap_day, conform_resource_data_to_n_timesteps, concatenate_resource_years, resample_resource_data_to_dt: Adds tests for leap-day handling, horizon conformance, multi-year concatenation, and timestep resampling.
• 30.6617_-101.7096_psmv3_60_2014.csv
  • Resource data: Adds a second solar weather year so the example can run a multi-year case without API calls.

Section 5.2: Modified Files

• examples/24_solar_battery_grid/plant_config.yaml
  • Finance/resource config: Points the example at multi-year GOES resource inputs for non-annual runs.
• run_solar_battery_grid.py
  • Example driver: Runs half-year, annual, and two-year variants programmatically with different time steps and prints a comparison table.
• demand_openloop_storage_controller.py
  • Controller timestep math: Reworks charge/discharge and SOC updates to use dt consistently for non-hourly simulations.
• grid.py
  • compute: Annualizes grid production and variable costs by simulated year so sub-annual and multi-year cases scale correctly.
• atb_utility_pv_cost.py
  • _time_step_bounds: Widens accepted timesteps so the solar cost model can participate in non-hourly runs.
• solar_pysam.py
  • compute: Uses helper-based annualization for non-annual runs while retaining PySAM annual outputs for exact annual cases.
• test_pysam_solar.py
  • Solar tests: Adds non-annual regression coverage and updates grouped assertions for the new behavior.
• validation.py
  • load_plant_yaml: Allows positive non-annual simulation horizons up to plant life instead of assuming a single year.
• model_baseclasses.py
  • calculate_annual_cf_and_replacement_schedule, calculate_annual_varopex: Centralizes annual projection logic for arbitrary simulation lengths and fixes variable OpEx annualization across timesteps.
• test_framework.py
  • Framework tests: Updates validation coverage for finance subgroup commodity-stream timeseries requirements.
• demand_base.py
  • Demand annualization: Uses helper-based annual production and capacity-factor logic for arbitrary simulation lengths.
• profast_base.py
  • populate_profast: Reconstructs ProFAST capacity and annual production from commodity amount/rate units instead of timestep-specific assumptions.
• resource_base.py
  • Resource acquisition: Supports slicing sub-annual data, loading multiple years, and consuming lists of per-year resource files.
• resource_base_hpc.py
  • Leap-day handling: Aligns the HPC resource acquisition with the updated leap-day processing interface.
• openmeteo_solar.py
  • Resource preprocessing: Adopts the updated leap-day utility without fixed annual-length assumptions.
• time_tools.py
  • Time utilities: Adds leap-day-aware slicing, multi-year concatenation, and pandas-based resampling for arbitrary timesteps.
• nlr_developer_wtk_api_models.py
  • Resource loading: Adapts wind API resource handling to the updated time-tool and multi-year acquisition flow.
• openmeteo_wind.py
  • Resource preprocessing: Adopts the updated leap-day utility without fixed annual-length assumptions.
• atb_battery_cost.py
  • _time_step_bounds: Removes hourly-only restrictions because the battery cost model is timestep-independent.
• storage_baseclass.py
  • Storage annualization: Separates discharge-based annual production from net-flow capacity factor for non-annual runs.
• storage_performance_model.py
  • _time_step_bounds: Widens accepted timesteps so storage performance can run outside hourly resolution.

Section 6: Additional Supporting Information

Section 7: Test Results, if applicable

Section 8 (Optional): New Model Checklist

• Model Structure:
  • Follows established naming conventions outlined in docs/developer_guide/coding_guidelines.md
  • Used attrs class to define the Config to load in attributes for the model
    • If applicable: inherit from BaseConfig or CostModelBaseConfig
  • Added: initialize() method, setup() method, compute() method
    • If applicable: inherit from CostModelBaseClass
• Integration: Model has been properly integrated into H2Integrate
  • Add the new model to the appropriate __init__.py file to ensure it is properly imported and used in supported_models.py
  • Added to supported_models.py
  • If a new commodity_type is added, update create_financial_model in h2integrate_model.py
• Tests: Unit tests have been added for the new model
  • Pytest-style unit tests
  • Unit tests are in a "test" folder within the folder a new model was added to
  • If applicable add integration tests
• Example: If applicable, a working example demonstrating the new model has been created
  • Input file comments
  • Run file comments
  • Example has been tested and runs successfully in test_all_examples.py
• Documentation:
  • Write docstrings using the Google style
  • Model added to the main models list in docs/user_guide/model_overview.md
    • Model documentation page added to the appropriate docs/ section
    • <model_name>.md is added to the _toc.yml
  • Run generate_class_hierarchy.py to update the class hierarchy diagram in docs/developer_guide/class_structure.md

[elenya-grant]

Hi Jared! Thanks for this PR! I think that this is a neat functionality add. Thanks for putting all this work!

Responding to your requested feedback from reviewers, I think that this should be split into separate PRs so that each change can get the attention and feedback it deserves. I left some small comments for now but would want to do an in-depth review if this is going to come-in as-is. Since this touches so many baseclasses, I think that splitting out PRs based on base-class changes would be a good approach. I think that the sub-PRs would be:

1. updates to the finance models (if they're needed?)
2. update to resource models (I think that theres a whole lot of different implementation questions that I'd want the group's ideas on)
3. update to performance model baseclass and cost model baseclass and maybe changes to 1 performance model and 1 cost model to showcase usage
4. updates to other performance and cost models

I haven't dug into the changes in the model baseclasses much, I think those methods would benefit from some inline comments to make it easier to follow the logic.
I also think that this PR has changes that are broader than just making models compatible to different simulation lengths, it seems like a lot of updates are based on time-step changes which, to me, is a different category. I also think that all performance and cost models should get an attribute like _time_step_bounds that is something like _simulation_length_bounds so we can better keep track of which models work with different simulation lengths

[elenya-grant]

this change does not seem relevant to non-annual simulations but instead seems more relevant to enabling different timesteps and should perhaps be part of a different PR (given the size of this one)

[elenya-grant]

are you sure this is a valid upper-bound? I thought PvWatts could only for 5min, 15 min, 30 min or hourly?

[elenya-grant]

I agree with this TODO statement. This method is really large, I think it should last least be split up so that the state of health stuff is separate (not many models have degradation included - this is a new concept).

[johnjasa]

Agreed on fully splitting out the SOH contributions outside of this PR. I also think we could separate the CF and replacement schedule computations since they have meaningfully different purposes.

[elenya-grant]

I don't think we should be changing the config value for resource_year - as noted in Issue #870

[jaredthomas68]

Fixed in 708d268

[elenya-grant]

I think the while could be replaced with instead calculating the number of resource years needed to get enough resource data to achieve n_timesteps.

[elenya-grant]

I think a lot of these checks within the while statement could be moved outside of it.

[elenya-grant]

this logic will not work for the three TMY solar resource models. I also don't think we should be changing the config value here. I think that resource_year needs to made as an input to the corresponding methods.

[jaredthomas68]

Resource year is coming from the config and just for the base year. The number of years needed and their values are determined from the dt and n_timesteps. I don't think we should require the user to put in every year. In the future I think we can have a string start and stop date/time format, but for now I lean towards keeping the single year spec.

I have made it not overwrite the base_year. Instead, resource year is determined in the loop and threaded through all the validators when loading multiple years. This created many small changes, but allows us to still reuse the single year loader code.

I also updated the logic slightly so that representative years can be handled (e.g. TMY) and when a representative year is specified, that same data is repeated for all requested simulation time. I thought about throwing an error, but I decided there may be other models or resources in the simulation that we do want to simulate and so I did not restrict the simulation length to match representative year data.

[elenya-grant]

I think that the check could be as simple as making sure value isn't a string, float, int, dictionary or boolean.

[elenya-grant]

I thin that we could just check self.fraction_of_year_simulated instead of is_annual

[jaredthomas68]

Done

[elenya-grant]

I'm nervous about this logic if using lifetime performance results in PySAM

[johnjasa]

Top-level thoughts

Overall, this PR is moving in a useful direction and I'm glad we're finally crossing this bridge!
Thanks for this, @jaredthomas68.

This PR bundles several fairly large changes: annualization, multi-year resource handling, timestep definition expansion, model updates, and finance changes.
I think it would be easier to review and make robust as a sequence of smaller PRs.
I know @elenya-grant feels similarly, though I might have a slightly different suggested sequence for the sub-PRs.

I think the biggest decision points about how to subdivide these efforts revolve around project timeline needs; this will help set the order we tackle annualization, resource handling, model updates, finance changes, and lifecycle logic.
If you said "I absolutely need this before the end of the FY," then I vote you use the battery degradation branch for your project needs, but take more time and stepwise discussion with the team to ensure we're moving in the same decided direction.

Suggested potential broken-out PR order

1. Define annualization behavior at a baseclass model and framework level

   Keep the basic annualization helper and validation changes together. Add tests for partial years, multi-year simulations, leap years, and non-dividing timesteps. We should also document what tile, final_sim_value, and average_sim_value mean for a partial-year run, making that behavior very clear for users.

2. Add resource loading and resampling (could be the first or second sub-PR)

   Heavily influenced by #888 and input from others. Keep the multi-year resource acquisition and time-tools changes together. Add at least one end-to-end test covering load, resample, concatenate, and consumption by a real performance model.

3. Enable models one at a time

   Some of these might need to happen with the sub-PRs above, but otherwise: Update grid/demand first, then solar, storage, and the other technologies. Each model should get focused tests for the timestep ranges we actually support. Finance changes should be handled here, but with special care: verify that annual production, utilization, and variable OpEx match independently calculated timeseries totals for more than just electricity. Once the underlying models are proven, update examples for the supported combinations.

4. Separate SOH and degradation lifecycle logic

   I think this can be split out relatively easily; this PR currently adds generic SOH-based degradation, EOL detection, replacement-cycle extrapolation, and replacement schedules. The initial non-annual work could use the existing no-degradation path and continue returning a zero replacement schedule.

Some open questions worth discussing

I expect that by tackling these as sub-PRs, it'll be easier for folks to identify questions like these and provide higher quality feedback:

• Is the default sub-annual run meant to represent a typical slice that gets scaled to a year, or an actual seasonal slice?
• Are all resource values rates, or do some need different resampling rules? i.e. would they ever be totals or integrated quantities?
• Which models and timestep ranges are intended to be supported in the first version?
• Do we enforce that a model coming into H2I has certain allowable timestep or annualization behavior?
• For the separate SOH follow-up, is replacement always recorded at plant-year boundaries if SOH reaches EOL mid-year?

[johnjasa]

Intentional inclusion of the commented-out years? I'd suggest removing the addition of the 2014 .csv file in this PR if possible

[jaredthomas68]

Good catch, I meant to have the 2013 and 2014 only, but was playing around with running 4 years so I doubled each. I'm inclined to keep the addition of the 2014 csv do clearly demo multi-year data, not just giving one year twice. I'll revert to the intended two years.

[johnjasa]

Agreed on fully splitting out the SOH contributions outside of this PR. I also think we could separate the CF and replacement schedule computations since they have meaningfully different purposes.