Skip to content

Create overview on structure of used forecast source datasets #1124

Open
Open
Create overview on structure of used forecast source datasets#1124
Assignees
Evelyn-MValentinGebhartelianekobler

Description

@chahank
chahank
opened on Dec 1, 2025

Create test datasets for hazards from most commonly used data sources.

  • Make a list of possible different types of forecast datasets and all necessary attributes, coordinates, data var structures:

  • Which different lead times, different ensemble members, different forecast times, different times scales (past, present, future), different meta data, ...?

Minimal suggestions are (separate tasks):

  • One dataset from MeteoSwiss nowcast (INCA); one dasta set from MeteoSwiss forecast (ICON / IFS)
  • One small dataset from Copernicus module
  • One from Glofas
  • One from IBTrACS

Activity

  1. chahank
    added a parent issue on Dec 1, 2025
  2. chahank
    removed a parent issue on Dec 1, 2025
  3. chahank
    added a parent issue on Dec 1, 2025
  4. Evelyn-M
    changed the title [-]Create test datasets[/-] [+]Create overview on structure of used forecast source datasets[/+] on Dec 1, 2025
  5. Evelyn-M
    assigned
    Evelyn-M
    ,
    ValentinGebhart
    and
    elianekobler
    on Dec 9, 2025

  6. Evelyn-M commented on Dec 9, 2025

    @Evelyn-M
    Evelyn-M
    on Dec 9, 2025
    Collaborator

    OGD Coordinate structure (note: z coord is not always a leading coordinate or even present):

    Coordinates:
  * eps         (eps) int64 40B 3 8 13 16 20
  * ref_time    (ref_time) datetime64[ns] 8B 2025-12-08T06:00:00
  * lead_time   (lead_time) timedelta64[ns] 32B 03:00:00 05:00:00 ... 12:00:00
  * z           (z) int64 8B 2
    lon         (cell) float32 0B ...
    lat         (cell) float32 0B ...
    valid_time  (ref_time, lead_time) datetime64[ns] 32B

Data variables:
    __xarray_dataarray_variable__  (eps, ref_time, lead_time, cell) float32 545MB ...

  7. ValentinGebhart commented on Dec 10, 2025

    @ValentinGebhart
    ValentinGebhart
    on Dec 10, 2025
    Collaborator

    Test data for glofas forecasts and copernicus forecasts are ready but their data strucutre is probably not in its final form:

    • glofas forecasts have two coordinates: valid_time for the valid time and time for the init time, while probably the typical data structure includes a leadtime in hours
    • copernicus forecast has a coordinate step that is a string of dates (e.g., "2024-06" or "2023-05-02"), but will probably be adapted to rather be a numpy.timedelta64 as well.

    ECMWF forecasts is ready as well, but only has one leadtime (0) because we would first have to adapt the pipeline how to identify the event and compute the lead time. I suggest to do this at a later point.

  8. Evelyn-M commented on Dec 10, 2025

    @Evelyn-M
    Evelyn-M
    on Dec 10, 2025
    Collaborator

    post-processed forecast data structure (e.g. warn files):

    Coordinates:
  * lead_time                (lead_time) timedelta64[ns] 40B 0 days 00:00:00 ...
  * y                        (y) float32 32B 1.053e+06 1.083e+06 ... 1.263e+06
  * x                        (x) float32 32B 2.471e+06 2.511e+06 ... 2.751e+06
  * realization              (realization) int64 40B 0 1 2 3 4
    forecast_reference_time  datetime64[ns] 8B ...
    time                     (lead_time) datetime64[ns] 40B ...

Data variables:
    severity  (lead_time, y, x, realization) float64 13kB ...
``

  9. peanutfun commented on Jun 10, 2026

    @peanutfun
    peanutfun
    on Jun 10, 2026
    Member

    Can we close this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions