From 0db9b87d3981fa22f3c7066a3b46493a1d39d24b Mon Sep 17 00:00:00 2001 From: Filippo Guerranti Date: Wed, 14 Oct 2026 17:05:28 +0000 Subject: [PATCH 1/6] Add time series forecasting guide --- docs/source/forecasting.md | 270 +++++++++++++++++++++++++++++++++++++ docs/source/index.md | 1 + 2 files changed, 271 insertions(+) create mode 100644 docs/source/forecasting.md diff --git a/docs/source/forecasting.md b/docs/source/forecasting.md new file mode 100644 index 0000000000..c7c9934121 --- /dev/null +++ b/docs/source/forecasting.md @@ -0,0 +1,270 @@ +# Time Series Forecasting + +Time series forecasting predicts future values of a series from its observed history. +The `structured-data-models` package allows forecasting through the same in-context learning interface as all other models (see the [ICL guide](icl)): the observed history is the context, and the future time steps are the queries. +This guide starts with a single series and adds one ingredient at a time: past-only covariates, known future covariates, multiple targets, ensembling, and finally batches of series. + +## The Forecasting Task + +A time series is a sequence of observations ordered in time. +Its variables are either **targets**, which are forecast, or **covariates**, which inform the forecast. +Covariates differ in when they are known: + +- *Past-only covariates* are only observed in the past (*e.g.*, the measured temperature). +- *Known future covariates* are available in the past and the future (*e.g.*, planned promotions or weather forecasts). + +The number of future steps to forecast is the **forecast horizon** `H`. + +Throughout this guide, we forecast the daily sales of ice cream and cold drinks (the targets) for the next three days, *i.e.*, `H = 3`. +The measured temperature is a past-only covariate, and the promotion calendar is a known future covariate. +We start with ice cream alone and add the other variables one at a time. + +## A Single Series + +We begin with a single target, the daily sales of ice cream, observed for five days and forecast for the next three: + +| day | ice_cream | +| --- | --------- | +| 1 | 40 | +| 2 | 55 | +| 3 | 80 | +| 4 | 75 | +| 5 | 50 | +| 6 | ? | +| 7 | ? | +| 8 | ? | + +A time series is a {py:class}`~sdm.tensor.TableTensor` in which each row is one time step, sorted in time, and each column is one variable (see the [tensor guide](tensor)): + +```python +import sdm + +history = sdm.TableTensor.from_columns( + {"ice_cream": [40.0, 55.0, 80.0, 75.0, 50.0]}, + stypes={"ice_cream": "numerical"}, + device="cuda", +) +``` + +Forecasting models implement the {py:class}`~sdm.models.ICLModel` interface, where rows are interpreted as ordered time steps. +Here, we use {py:class}`~sdm.models.TimesFM3`, a time series foundation model that forecasts new series zero-shot, *i.e.*, without training on the series at hand: + +```python +x_context = sdm.TableTensor(size=(5,), device="cuda") # [5, 0] +y_context = history # [5, 1] +x_query = sdm.TableTensor(size=(3,), device="cuda") # [3, 0] + +model = sdm.models.TimesFM3(device="cuda") +out = model(x_context, y_context, x_query) +``` + +The model takes three inputs, each a {py:class}`~sdm.tensor.TableTensor` or {py:class}`torch.Tensor`: + +- `x_context`: the covariates over the observed history, with shape `[R_context, D]`. +- `y_context`: the past values of the targets, with shape `[R_context, Y]`. +- `x_query`: the known future covariates, with shape `[R_query, D_query]`. The number of rows in `x_query` is the forecast horizon, so no further argument is needed to request more or fewer future steps. + +`x_context` and `x_query` carry the covariates. +We have none yet, so both are tables with zero columns, and only the number of rows of `x_query` matters. + +The forecast is returned as a {py:class}`~sdm.tensor.TableTensor` with one row per future step. +Its columns depend on the model: {py:class}`~sdm.models.TimesFM3`, for example, outputs one column per target and quantile, from the 10th to the 90th percentile, named `"__q"`: + +```python +out["ice_cream__q50"].numerical # median forecast, [3, 1] +out["ice_cream__q10"].numerical # lower end of an 80% prediction interval +out["ice_cream__q90"].numerical # upper end of an 80% prediction interval +``` + +## Past-Only Covariates + +We add the measured temperature, a past-only covariate, to the history. +It is unknown for the forecast horizon: + +```python +history = sdm.TableTensor.from_columns( + { + "ice_cream": [40.0, 55.0, 80.0, 75.0, 50.0], + "temperature": [21.0, 24.0, 28.0, 27.0, 22.0], + }, + stypes={"ice_cream": "numerical", "temperature": "numerical"}, + device="cuda", +) +``` + +Past-only covariates need no declaration. +They are the columns of `x_context` that are absent from `x_query`, and are passed to the model as **context-only columns**: + +```python +x_context = history.drop_columns("ice_cream") # [5, 1] +y_context = history.select_columns("ice_cream") # [5, 1] +x_query = sdm.TableTensor(size=(3,), device="cuda") # [3, 0] + +out = model(x_context, y_context, x_query) +``` + +Models declare support for context-only columns via {py:attr}`~sdm.models.ICLModel.supports_context_only_columns` (`True` for {py:class}`~sdm.models.TimesFM3`). + +## Known Future Covariates + +We add the promotion calendar, a known future covariate. +Unlike the temperature, it is known for the next three days: + +| day | ice_cream | temperature | promotion | +| --- | --------- | ----------- | --------- | +| 1 | 40 | 21 | 0 | +| 2 | 55 | 24 | 0 | +| 3 | 80 | 28 | 1 | +| 4 | 75 | 27 | 1 | +| 5 | 50 | 22 | 0 | +| 6 | ? | ? | 1 | +| 7 | ? | ? | 0 | +| 8 | ? | ? | 0 | + +The history gains a `promotion` column, and the future is a table that only holds the known future covariates: + +```python +history = sdm.TableTensor.from_columns( + { + "ice_cream": [40.0, 55.0, 80.0, 75.0, 50.0], + "temperature": [21.0, 24.0, 28.0, 27.0, 22.0], + "promotion": [0.0, 0.0, 1.0, 1.0, 0.0], + }, + stypes=dict.fromkeys(["ice_cream", "temperature", "promotion"], "numerical"), + device="cuda", +) +future = sdm.TableTensor.from_columns( + {"promotion": [1.0, 0.0, 0.0]}, + stypes={"promotion": "numerical"}, + device="cuda", +) +``` + +Known future covariates are the columns that appear in both `x_context` and `x_query`, in the same order: + +```python +x_context = history.drop_columns("ice_cream") # [5, 2] +y_context = history.select_columns("ice_cream") # [5, 1] +x_query = future # [3, 1] + +out = model(x_context, y_context, x_query) +``` + +In summary, the columns that are passed determine which covariates the model sees: + +- **Known future covariates** appear in both `x_context` and `x_query`. +- **Past-only covariates** appear in `x_context` only. +- **Without any covariates**, `x_context` and `x_query` have zero columns. + +To compare several futures for the same history, *e.g.*, alternative promotion plans, call {py:meth}`~sdm.models.ICLModel.fit` once and {py:meth}`~sdm.models.ICLModel.predict` for each plan. +Since there is no query to infer the context-only columns from, they are given explicitly: + +```python +model.fit(x=x_context, y=y_context, context_only_columns=["temperature"]) +for plan in ([1.0, 0.0, 0.0], [0.0, 0.0, 1.0]): + x_query = sdm.TableTensor.from_columns( + {"promotion": plan}, + stypes={"promotion": "numerical"}, + device="cuda", + ) + out = model.predict(x_query) +model.clear() # release the cached context +``` + +Passing `context_only_columns` to {py:meth}`~sdm.models.ICLModel.fit` raises an error for models that do not support them. + +## Multiple Targets + +So far, we forecast a single target. +Models that support multiple targets forecast them jointly, so they can exploit dependencies between them. +Each model declares this via {py:attr}`~sdm.models.ICLModel.supports_multi_target` (`True` for {py:class}`~sdm.models.TimesFM3`), while other models accept a single target column only. + +We add the daily sales of cold drinks as a second target, which completes the example: + +| day | ice_cream | cold_drinks | temperature | promotion | +| --- | --------- | ----------- | ----------- | --------- | +| 1 | 40 | 25 | 21 | 0 | +| 2 | 55 | 30 | 24 | 0 | +| 3 | 80 | 45 | 28 | 1 | +| 4 | 75 | 41 | 27 | 1 | +| 5 | 50 | 28 | 22 | 0 | +| 6 | ? | ? | ? | 1 | +| 7 | ? | ? | ? | 0 | +| 8 | ? | ? | ? | 0 | + +All targets are passed as columns of `y_context`: + +```python +history = sdm.TableTensor.from_columns( + { + "ice_cream": [40.0, 55.0, 80.0, 75.0, 50.0], + "cold_drinks": [25.0, 30.0, 45.0, 41.0, 28.0], + "temperature": [21.0, 24.0, 28.0, 27.0, 22.0], + "promotion": [0.0, 0.0, 1.0, 1.0, 0.0], + }, + stypes=dict.fromkeys(["ice_cream", "cold_drinks", "temperature", "promotion"], "numerical"), + device="cuda", +) + +x_context = history.drop_columns(["ice_cream", "cold_drinks"]) # [5, 2] +y_context = history.select_columns(["ice_cream", "cold_drinks"]) # [5, 2] +x_query = future # [3, 1] + +out = model(x_context, y_context, x_query) +``` + +The forecast contains one column per target and quantile: + +```python +out["ice_cream__q50"].numerical # median forecast, [3, 1] +out["cold_drinks__q10"].numerical # lower end of an 80% prediction interval +out["cold_drinks__q90"].numerical # upper end of an 80% prediction interval +``` + +## Ensembling + +A forecast can be made by several ensemble members, whose forecasts are averaged. +The `num_estimators` argument sets the number of members, and omitting it is the same as `num_estimators=1`. +More members cost proportionally more. +Members may see differently transformed views of the series, so their forecasts can differ (see the [ensembling section](icl) of the ICL guide): + +```python +out = model(x_context, y_context, x_query, num_estimators=4) +``` + +## Batches of Series + +Forecasting often involves many series, *e.g.*, the sales of every store. +To forecast a batch of `B` series in one call, stack them along a leading batch dimension, so that `x_context`, `y_context` and `x_query` have shapes `[B, R_context, D]`, `[B, R_context, Y]` and `[B, R_query, D_query]`. +All series in a batch share the same horizon and number of context rows, and shorter series are left-padded with `NaN`. + +Pass `num_estimators` so that the leading dimension is read as a batch of independent series, each forecast by `num_estimators` ensemble members. + +```{note} +Without `num_estimators`, the leading dimension is read as the ensemble members of a single series, and their forecasts are combined into one. +A batch of series is then averaged into a single forecast, and the batch dimension is lost. +``` + +For example, to forecast the next 24 steps of a batch of `B` series without covariates: + +```python +# values: [B, R_context] tensor of observations, left-padded with NaN +targets = sdm.TableTensor.from_tensor(values.unsqueeze(-1), columns=["target"]) # [B, R_context, 1] + +out = model( + x_context=targets.drop_columns("target"), # [B, R_context, 0] + y_context=targets, # [B, R_context, 1] + x_query=sdm.TableTensor(size=(B, 24), device="cuda"), # [B, 24, 0] + num_estimators=1, +) +``` + +The forecasts keep the batch dimension: + +```python +out["target__q50"].numerical # median forecast, [B, 24, 1] +out["target__q10"].numerical # lower end of an 80% prediction interval, [B, 24, 1] +out["target__q90"].numerical # upper end of an 80% prediction interval, [B, 24, 1] +``` + +See [`examples/timeseries/quickstart.py`](https://github.com/NVIDIA/structured-data-models/blob/main/examples/timeseries/quickstart.py) for runnable examples on real datasets, which cover batches of series as well as known future and past-only covariates. diff --git a/docs/source/index.md b/docs/source/index.md index 5f650b2c16..ad36522f98 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -30,6 +30,7 @@ overview tensor icl +forecasting processing ``` From 38dfc0b0e9941a286e02fa9ef1ca907e8575dd73 Mon Sep 17 00:00:00 2001 From: rusty1s Date: Tue, 27 Oct 2026 14:13:16 +0000 Subject: [PATCH 2/6] update --- docs/source/forecasting.md | 288 ++++++++++++++----------------------- 1 file changed, 106 insertions(+), 182 deletions(-) diff --git a/docs/source/forecasting.md b/docs/source/forecasting.md index c7c9934121..c7c061362c 100644 --- a/docs/source/forecasting.md +++ b/docs/source/forecasting.md @@ -1,40 +1,35 @@ # Time Series Forecasting -Time series forecasting predicts future values of a series from its observed history. -The `structured-data-models` package allows forecasting through the same in-context learning interface as all other models (see the [ICL guide](icl)): the observed history is the context, and the future time steps are the queries. -This guide starts with a single series and adds one ingredient at a time: past-only covariates, known future covariates, multiple targets, ensembling, and finally batches of series. +Time series forecasting models (TSFMs) predict future values of a series from its observed history. +While many forecasting foundation models originally focused on predicting a single variable from its own history (**univariate forecasting**), newer models such as [Chronos-2](https://arxiv.org/abs/2510.15821) and [TimesFM-3](https://research.google/blog/timesfm-3-a-zero-shot-foundation-model-for-multivariate-forecasting) extend this paradigm to multiple targets (**multivariate forecasting**) and **covariates**: additional variables that inform the forecast (*e.g.*, historical temperature or planned promotions). +These capabilities allow models to draw on relationships between targets, past observations of external factors, and known future information. -## The Forecasting Task +Time series forecasting shares a central principle with tabular [in-context learning](icl/): a pretrained model conditions its predictions on observed examples, without requiring training on each new dataset. +In tabular prediction, the context consists of labeled rows and the queries are new rows. +In forecasting, the context is the observed past and the queries are future time steps. +In both settings, attention across columns captures relationships between variables within a row or time step, while attention across rows captures relationships between observations or time steps. +Forecasting thus motivates useful extensions to the in-context learning formulation: -A time series is a sequence of observations ordered in time. -Its variables are either **targets**, which are forecast, or **covariates**, which inform the forecast. -Covariates differ in when they are known: +- **Multiple targets** allow a model to predict several variables together. +- **Known future covariates** provide inputs for both context and query rows. +- **Past-only covariates** provide inputs for context rows even when their query values are unavailable. -- *Past-only covariates* are only observed in the past (*e.g.*, the measured temperature). -- *Known future covariates* are available in the past and the future (*e.g.*, planned promotions or weather forecasts). +This package incorporates these extensions through the same {py:class}`~sdm.models.ICLModel` used by other models. +In particular, `y_context` contains the observed targets, columns shared by `x_context` and `x_query` represent known future covariates, and columns exclusive to `x_context` represent past-only covariates. -The number of future steps to forecast is the **forecast horizon** `H`. - -Throughout this guide, we forecast the daily sales of ice cream and cold drinks (the targets) for the next three days, *i.e.*, `H = 3`. -The measured temperature is a past-only covariate, and the promotion calendar is a known future covariate. -We start with ice cream alone and add the other variables one at a time. - -## A Single Series +## Univariate Forecasting We begin with a single target, the daily sales of ice cream, observed for five days and forecast for the next three: -| day | ice_cream | -| --- | --------- | -| 1 | 40 | -| 2 | 55 | -| 3 | 80 | -| 4 | 75 | -| 5 | 50 | -| 6 | ? | -| 7 | ? | -| 8 | ? | + + + + + + +
day01234567
ice cream4055807550???
-A time series is a {py:class}`~sdm.tensor.TableTensor` in which each row is one time step, sorted in time, and each column is one variable (see the [tensor guide](tensor)): +A time series is a {py:class}`~sdm.tensor.TableTensor` in which each row is one time step, sorted in time, and each column is one variable: ```python import sdm @@ -46,29 +41,24 @@ history = sdm.TableTensor.from_columns( ) ``` -Forecasting models implement the {py:class}`~sdm.models.ICLModel` interface, where rows are interpreted as ordered time steps. -Here, we use {py:class}`~sdm.models.TimesFM3`, a time series foundation model that forecasts new series zero-shot, *i.e.*, without training on the series at hand: +We can forecast with any time series foundation model in [`sdm.models`](api/models). +Here, we use {py:class}`~sdm.models.TimesFM3`: ```python -x_context = sdm.TableTensor(size=(5,), device="cuda") # [5, 0] -y_context = history # [5, 1] -x_query = sdm.TableTensor(size=(3,), device="cuda") # [3, 0] - model = sdm.models.TimesFM3(device="cuda") -out = model(x_context, y_context, x_query) +out = model( + x_context=sdm.TableTensor(size=(5, ), device="cuda"), + y_context=history, + x_query=sdm.TableTensor(size=(3, ), device="cuda"), +) ``` -The model takes three inputs, each a {py:class}`~sdm.tensor.TableTensor` or {py:class}`torch.Tensor`: - -- `x_context`: the covariates over the observed history, with shape `[R_context, D]`. -- `y_context`: the past values of the targets, with shape `[R_context, Y]`. -- `x_query`: the known future covariates, with shape `[R_query, D_query]`. The number of rows in `x_query` is the forecast horizon, so no further argument is needed to request more or fewer future steps. - -`x_context` and `x_query` carry the covariates. -We have none yet, so both are tables with zero columns, and only the number of rows of `x_query` matters. +Note that `x_context` and `x_query` carry the covariates. +Since we have none yet, both tables hold zero columns. +However, we still need to pass them to {py:meth}`~sdm.models.ICLModel.forward`, in particular because the number of rows in `x_query` define the **forecast horizon**: here, we request predictions for the next three time steps. The forecast is returned as a {py:class}`~sdm.tensor.TableTensor` with one row per future step. -Its columns depend on the model: {py:class}`~sdm.models.TimesFM3`, for example, outputs one column per target and quantile, from the 10th to the 90th percentile, named `"__q"`: +Its output is model-specific, *e.g.*, {py:class}`~sdm.models.TimesFM3` outputs one column per target and quantile, from the 10th to the 90th percentile, named `"__q"`: ```python out["ice_cream__q50"].numerical # median forecast, [3, 1] @@ -76,61 +66,76 @@ out["ice_cream__q10"].numerical # lower end of an 80% prediction interval out["ice_cream__q90"].numerical # upper end of an 80% prediction interval ``` -## Past-Only Covariates +## Multivariate Forecasting + +Models with {py:attr}`sdm.models.ICLModel.supports_multi_target` can forecast multiple targets jointly, allowing them to exploit dependencies between them. -We add the measured temperature, a past-only covariate, to the history. -It is unknown for the forecast horizon: +In this case, we can simply add the daily sales of cold drinks as a second target: + + + + + + + + + +
day01234567
ice cream4055807550???
cold drinks2530454128???
```python history = sdm.TableTensor.from_columns( { "ice_cream": [40.0, 55.0, 80.0, 75.0, 50.0], - "temperature": [21.0, 24.0, 28.0, 27.0, 22.0], + "cold_drinks": [25.0, 30.0, 45.0, 41.0, 28.0], }, - stypes={"ice_cream": "numerical", "temperature": "numerical"}, + stypes={"ice_cream": "numerical", "cold_drinks": "numerical"}, device="cuda", ) -``` -Past-only covariates need no declaration. -They are the columns of `x_context` that are absent from `x_query`, and are passed to the model as **context-only columns**: - -```python -x_context = history.drop_columns("ice_cream") # [5, 1] -y_context = history.select_columns("ice_cream") # [5, 1] -x_query = sdm.TableTensor(size=(3,), device="cuda") # [3, 0] - -out = model(x_context, y_context, x_query) +out = model( + x_context=sdm.TableTensor(size=(5, ), device="cuda"), + y_context=history, + x_query=sdm.TableTensor(size=(3, ), device="cuda"), +) ``` -Models declare support for context-only columns via {py:attr}`~sdm.models.ICLModel.supports_context_only_columns` (`True` for {py:class}`~sdm.models.TimesFM3`). - -## Known Future Covariates - -We add the promotion calendar, a known future covariate. -Unlike the temperature, it is known for the next three days: - -| day | ice_cream | temperature | promotion | -| --- | --------- | ----------- | --------- | -| 1 | 40 | 21 | 0 | -| 2 | 55 | 24 | 0 | -| 3 | 80 | 28 | 1 | -| 4 | 75 | 27 | 1 | -| 5 | 50 | 22 | 0 | -| 6 | ? | ? | 1 | -| 7 | ? | ? | 0 | -| 8 | ? | ? | 0 | - -The history gains a `promotion` column, and the future is a table that only holds the known future covariates: +## Covariates + +Lastly, we add known future and past-only covariates to our model. +For example, the measured **temperature** is a past covariate: its values are unavailable for the forecast horizon. +Models with {py:attr}`sdm.models.ICLModel.supports_context_only_columns` can exploit such context-only columns. +In contrast, **promotion** is a known future covariate that indicates whether a promotion was or is scheduled for a given day. + + + + + + + + + + + + + + + +
day01234567
ice cream4055807550???
cold drinks2530454128???
temperature2124282722
promotion00110100
```python history = sdm.TableTensor.from_columns( { "ice_cream": [40.0, 55.0, 80.0, 75.0, 50.0], + "cold_drinks": [25.0, 30.0, 45.0, 41.0, 28.0], "temperature": [21.0, 24.0, 28.0, 27.0, 22.0], "promotion": [0.0, 0.0, 1.0, 1.0, 0.0], }, - stypes=dict.fromkeys(["ice_cream", "temperature", "promotion"], "numerical"), + stypes={ + "ice_cream": "numerical", + "cold_drinks": "numerical" + "temperature": "numerical" + "promotion": "numerical" + }, device="cuda", ) future = sdm.TableTensor.from_columns( @@ -138,16 +143,12 @@ future = sdm.TableTensor.from_columns( stypes={"promotion": "numerical"}, device="cuda", ) -``` -Known future covariates are the columns that appear in both `x_context` and `x_query`, in the same order: - -```python -x_context = history.drop_columns("ice_cream") # [5, 2] -y_context = history.select_columns("ice_cream") # [5, 1] -x_query = future # [3, 1] - -out = model(x_context, y_context, x_query) +out = model( + x_context=history[["temperature", "promotion"]], + y_context=history[["ice_cream", "cold_drinks"]], + x_query=future, +) ``` In summary, the columns that are passed determine which covariates the model sees: @@ -156,115 +157,38 @@ In summary, the columns that are passed determine which covariates the model see - **Past-only covariates** appear in `x_context` only. - **Without any covariates**, `x_context` and `x_query` have zero columns. -To compare several futures for the same history, *e.g.*, alternative promotion plans, call {py:meth}`~sdm.models.ICLModel.fit` once and {py:meth}`~sdm.models.ICLModel.predict` for each plan. -Since there is no query to infer the context-only columns from, they are given explicitly: - -```python -model.fit(x=x_context, y=y_context, context_only_columns=["temperature"]) -for plan in ([1.0, 0.0, 0.0], [0.0, 0.0, 1.0]): - x_query = sdm.TableTensor.from_columns( - {"promotion": plan}, - stypes={"promotion": "numerical"}, - device="cuda", - ) - out = model.predict(x_query) -model.clear() # release the cached context -``` - -Passing `context_only_columns` to {py:meth}`~sdm.models.ICLModel.fit` raises an error for models that do not support them. - -## Multiple Targets - -So far, we forecast a single target. -Models that support multiple targets forecast them jointly, so they can exploit dependencies between them. -Each model declares this via {py:attr}`~sdm.models.ICLModel.supports_multi_target` (`True` for {py:class}`~sdm.models.TimesFM3`), while other models accept a single target column only. - -We add the daily sales of cold drinks as a second target, which completes the example: - -| day | ice_cream | cold_drinks | temperature | promotion | -| --- | --------- | ----------- | ----------- | --------- | -| 1 | 40 | 25 | 21 | 0 | -| 2 | 55 | 30 | 24 | 0 | -| 3 | 80 | 45 | 28 | 1 | -| 4 | 75 | 41 | 27 | 1 | -| 5 | 50 | 28 | 22 | 0 | -| 6 | ? | ? | ? | 1 | -| 7 | ? | ? | ? | 0 | -| 8 | ? | ? | ? | 0 | - -All targets are passed as columns of `y_context`: - -```python -history = sdm.TableTensor.from_columns( - { - "ice_cream": [40.0, 55.0, 80.0, 75.0, 50.0], - "cold_drinks": [25.0, 30.0, 45.0, 41.0, 28.0], - "temperature": [21.0, 24.0, 28.0, 27.0, 22.0], - "promotion": [0.0, 0.0, 1.0, 1.0, 0.0], - }, - stypes=dict.fromkeys(["ice_cream", "cold_drinks", "temperature", "promotion"], "numerical"), - device="cuda", -) - -x_context = history.drop_columns(["ice_cream", "cold_drinks"]) # [5, 2] -y_context = history.select_columns(["ice_cream", "cold_drinks"]) # [5, 2] -x_query = future # [3, 1] - -out = model(x_context, y_context, x_query) -``` - -The forecast contains one column per target and quantile: - -```python -out["ice_cream__q50"].numerical # median forecast, [3, 1] -out["cold_drinks__q10"].numerical # lower end of an 80% prediction interval -out["cold_drinks__q90"].numerical # upper end of an 80% prediction interval -``` - ## Ensembling -A forecast can be made by several ensemble members, whose forecasts are averaged. -The `num_estimators` argument sets the number of members, and omitting it is the same as `num_estimators=1`. -More members cost proportionally more. -Members may see differently transformed views of the series, so their forecasts can differ (see the [ensembling section](icl) of the ICL guide): +As with any other {py:class}`~sdm.models.ICLModel`, the `num_estimators` argument controls how many ensemble members contribute to the forecast. +Each member may receive a differently transformed view of the series, and their predictions are averaged into a single forecast. +For example, the default recipe for {py:class}`~sdm.models.TimesFM3` alternatives between the original series and a sign-flipped version. +Using two estimators includes both views: ```python -out = model(x_context, y_context, x_query, num_estimators=4) +out = model(x_context, y_context, x_query, num_estimators=2) ``` -## Batches of Series +## Batched Forecasting -Forecasting often involves many series, *e.g.*, the sales of every store. -To forecast a batch of `B` series in one call, stack them along a leading batch dimension, so that `x_context`, `y_context` and `x_query` have shapes `[B, R_context, D]`, `[B, R_context, Y]` and `[B, R_query, D_query]`. -All series in a batch share the same horizon and number of context rows, and shorter series are left-padded with `NaN`. +Batching lets us run several independent forecasting tasks in one call. +Each task has its own context and queries, and predictions will not use information from other tasks in the batch. +Each task itself can contain multiple targets and covariates. -Pass `num_estimators` so that the leading dimension is read as a batch of independent series, each forecast by `num_estimators` ensemble members. - -```{note} -Without `num_estimators`, the leading dimension is read as the ensemble members of a single series, and their forecasts are combined into one. -A batch of series is then averaged into a single forecast, and the batch dimension is lost. -``` +For example, store sales can be represented in two ways: as separate univariate tasks in a batch (forecasting each store independently) or as a multivariate task (allowing the model to use relationships between stores). -For example, to forecast the next 24 steps of a batch of `B` series without covariates: +To forecast a batch series in one call, stack them along a leading batch dimension. +All series in a batch share the same horizon and number of context rows, and shorter series should be left-padded with `NaN`. +Set `num_estimators` explicitly so that the leading dimension is interpreted as a batch of independent forecasting tasks rather than as ensemble members: ```python -# values: [B, R_context] tensor of observations, left-padded with NaN -targets = sdm.TableTensor.from_tensor(values.unsqueeze(-1), columns=["target"]) # [B, R_context, 1] +import torch out = model( - x_context=targets.drop_columns("target"), # [B, R_context, 0] - y_context=targets, # [B, R_context, 1] - x_query=sdm.TableTensor(size=(B, 24), device="cuda"), # [B, 24, 0] + x_context=torch.randn(64, 128, 0, device="cuda"), + x_context=torch.randn(64, 128, 2, device="cuda"), + x_query=torch.randn(64, 32, 0, device="cuda"), num_estimators=1, ) ``` -The forecasts keep the batch dimension: - -```python -out["target__q50"].numerical # median forecast, [B, 24, 1] -out["target__q10"].numerical # lower end of an 80% prediction interval, [B, 24, 1] -out["target__q90"].numerical # upper end of an 80% prediction interval, [B, 24, 1] -``` - -See [`examples/timeseries/quickstart.py`](https://github.com/NVIDIA/structured-data-models/blob/main/examples/timeseries/quickstart.py) for runnable examples on real datasets, which cover batches of series as well as known future and past-only covariates. +See [`examples/timeseries/quickstart.py`](https://github.com/NVIDIA/structured-data-models/blob/main/examples/timeseries/quickstart.py) for runnable examples on real-world datasets and tasks. From b066bfd438fe5f17887e4fa9e03e8e79d6f03629 Mon Sep 17 00:00:00 2001 From: rusty1s Date: Tue, 27 Oct 2026 14:33:30 +0000 Subject: [PATCH 3/6] update --- docs/source/forecasting.md | 10 ++++++++++ docs/source/images/forecasting_dark.svg | 4 ++++ docs/source/images/forecasting_light.svg | 4 ++++ 3 files changed, 18 insertions(+) create mode 100644 docs/source/images/forecasting_dark.svg create mode 100644 docs/source/images/forecasting_light.svg diff --git a/docs/source/forecasting.md b/docs/source/forecasting.md index c7c061362c..6048ad2626 100644 --- a/docs/source/forecasting.md +++ b/docs/source/forecasting.md @@ -14,6 +14,16 @@ Forecasting thus motivates useful extensions to the in-context learning formulat - **Known future covariates** provide inputs for both context and query rows. - **Past-only covariates** provide inputs for context rows even when their query values are unavailable. +```{figure} images/forecasting_light.svg +:figclass: light-only +:width: 100% +``` + +```{figure} images/forecasting_dark.svg +:figclass: dark-only +:width: 100% +``` + This package incorporates these extensions through the same {py:class}`~sdm.models.ICLModel` used by other models. In particular, `y_context` contains the observed targets, columns shared by `x_context` and `x_query` represent known future covariates, and columns exclusive to `x_context` represent past-only covariates. diff --git a/docs/source/images/forecasting_dark.svg b/docs/source/images/forecasting_dark.svg new file mode 100644 index 0000000000..e723e41ab5 --- /dev/null +++ b/docs/source/images/forecasting_dark.svg @@ -0,0 +1,4 @@ + + +SeriesTargetsContextPastQueryFuturePast-onlyCovariatesKnown FutureCovariates?????? diff --git a/docs/source/images/forecasting_light.svg b/docs/source/images/forecasting_light.svg new file mode 100644 index 0000000000..641b1e303e --- /dev/null +++ b/docs/source/images/forecasting_light.svg @@ -0,0 +1,4 @@ + + +SeriesTargetsContextPastQueryFuturePast-onlyCovariatesKnown FutureCovariates?????? From 66f396f8137bbf4e452eccd5c692a3c7cb4d3bca Mon Sep 17 00:00:00 2001 From: rusty1s Date: Tue, 27 Oct 2026 14:37:21 +0000 Subject: [PATCH 4/6] update --- docs/source/images/forecasting_dark.svg | 4 ++-- docs/source/images/forecasting_light.svg | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/images/forecasting_dark.svg b/docs/source/images/forecasting_dark.svg index e723e41ab5..8f2b386f67 100644 --- a/docs/source/images/forecasting_dark.svg +++ b/docs/source/images/forecasting_dark.svg @@ -1,4 +1,4 @@ -SeriesTargetsContextPastQueryFuturePast-onlyCovariatesKnown FutureCovariates?????? +SeriesTargetsContextPastQueryFuturePast-onlyCovariatesKnown FutureCovariates?????? diff --git a/docs/source/images/forecasting_light.svg b/docs/source/images/forecasting_light.svg index 641b1e303e..182edce93a 100644 --- a/docs/source/images/forecasting_light.svg +++ b/docs/source/images/forecasting_light.svg @@ -1,4 +1,4 @@ -SeriesTargetsContextPastQueryFuturePast-onlyCovariatesKnown FutureCovariates?????? +SeriesTargetsContextPastQueryFuturePast-onlyCovariatesKnown FutureCovariates?????? From bdc73c7dc1b2fcf2ec804a934765e7aa188ca6de Mon Sep 17 00:00:00 2001 From: rusty1s Date: Tue, 27 Oct 2026 17:08:23 +0000 Subject: [PATCH 5/6] update --- docs/source/forecasting.md | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/docs/source/forecasting.md b/docs/source/forecasting.md index 6048ad2626..03a759583b 100644 --- a/docs/source/forecasting.md +++ b/docs/source/forecasting.md @@ -1,6 +1,6 @@ # Time Series Forecasting -Time series forecasting models (TSFMs) predict future values of a series from its observed history. +Time series foundation models (TSFMs) predict future values of a series from its observed history. While many forecasting foundation models originally focused on predicting a single variable from its own history (**univariate forecasting**), newer models such as [Chronos-2](https://arxiv.org/abs/2510.15821) and [TimesFM-3](https://research.google/blog/timesfm-3-a-zero-shot-foundation-model-for-multivariate-forecasting) extend this paradigm to multiple targets (**multivariate forecasting**) and **covariates**: additional variables that inform the forecast (*e.g.*, historical temperature or planned promotions). These capabilities allow models to draw on relationships between targets, past observations of external factors, and known future information. @@ -39,7 +39,7 @@ We begin with a single target, the daily sales of ice cream, observed for five d -A time series is a {py:class}`~sdm.tensor.TableTensor` in which each row is one time step, sorted in time, and each column is one variable: +We represent a time series as a {py:class}`~sdm.tensor.TableTensor` with one row per time step, sorted in time, and one column per variable: ```python import sdm @@ -51,7 +51,7 @@ history = sdm.TableTensor.from_columns( ) ``` -We can forecast with any time series foundation model in [`sdm.models`](api/models). +We can forecast with any time series forecasting model in [`sdm.models`](api/models). Here, we use {py:class}`~sdm.models.TimesFM3`: ```python @@ -65,7 +65,7 @@ out = model( Note that `x_context` and `x_query` carry the covariates. Since we have none yet, both tables hold zero columns. -However, we still need to pass them to {py:meth}`~sdm.models.ICLModel.forward`, in particular because the number of rows in `x_query` define the **forecast horizon**: here, we request predictions for the next three time steps. +However, we still need to pass them to {py:meth}`~sdm.models.ICLModel.forward`, in particular because the number of rows in `x_query` defines the **forecast horizon**: here, we request predictions for the next three time steps. The forecast is returned as a {py:class}`~sdm.tensor.TableTensor` with one row per future step. Its output is model-specific, *e.g.*, {py:class}`~sdm.models.TimesFM3` outputs one column per target and quantile, from the 10th to the 90th percentile, named `"__q"`: @@ -78,7 +78,7 @@ out["ice_cream__q90"].numerical # upper end of an 80% prediction interval ## Multivariate Forecasting -Models with {py:attr}`sdm.models.ICLModel.supports_multi_target` can forecast multiple targets jointly, allowing them to exploit dependencies between them. +A models that {py:attr}`~sdm.models.ICLModel.supports_multi_target` can forecast multiple targets jointly, allowing them to exploit dependencies between them. In this case, we can simply add the daily sales of cold drinks as a second target: @@ -112,8 +112,8 @@ out = model( ## Covariates Lastly, we add known future and past-only covariates to our model. -For example, the measured **temperature** is a past covariate: its values are unavailable for the forecast horizon. -Models with {py:attr}`sdm.models.ICLModel.supports_context_only_columns` can exploit such context-only columns. +For example, the measured **temperature** is a past-only covariate: its values are unavailable for the forecast horizon. +A model that {py:attr}`~sdm.models.ICLModel.supports_context_only_columns` can exploit such past-only covariates. In contrast, **promotion** is a known future covariate that indicates whether a promotion was or is scheduled for a given day. @@ -123,10 +123,8 @@ In contrast, **promotion** is a known future covariate that indicates whether a - - @@ -142,9 +140,9 @@ history = sdm.TableTensor.from_columns( }, stypes={ "ice_cream": "numerical", - "cold_drinks": "numerical" - "temperature": "numerical" - "promotion": "numerical" + "cold_drinks": "numerical", + "temperature": "numerical", + "promotion": "numerical", }, device="cuda", ) @@ -171,7 +169,7 @@ In summary, the columns that are passed determine which covariates the model see As with any other {py:class}`~sdm.models.ICLModel`, the `num_estimators` argument controls how many ensemble members contribute to the forecast. Each member may receive a differently transformed view of the series, and their predictions are averaged into a single forecast. -For example, the default recipe for {py:class}`~sdm.models.TimesFM3` alternatives between the original series and a sign-flipped version. +For example, the default recipe for {py:class}`~sdm.models.TimesFM3` alternates between the original series and a sign-flipped version. Using two estimators includes both views: ```python @@ -186,7 +184,7 @@ Each task itself can contain multiple targets and covariates. For example, store sales can be represented in two ways: as separate univariate tasks in a batch (forecasting each store independently) or as a multivariate task (allowing the model to use relationships between stores). -To forecast a batch series in one call, stack them along a leading batch dimension. +To forecast a batch of series in one call, stack them along a leading batch dimension. All series in a batch share the same horizon and number of context rows, and shorter series should be left-padded with `NaN`. Set `num_estimators` explicitly so that the leading dimension is interpreted as a batch of independent forecasting tasks rather than as ensemble members: @@ -195,10 +193,10 @@ import torch out = model( x_context=torch.randn(64, 128, 0, device="cuda"), - x_context=torch.randn(64, 128, 2, device="cuda"), + y_context=torch.randn(64, 128, 2, device="cuda"), x_query=torch.randn(64, 32, 0, device="cuda"), num_estimators=1, ) ``` -See [`examples/timeseries/quickstart.py`](https://github.com/NVIDIA/structured-data-models/blob/main/examples/timeseries/quickstart.py) for runnable examples on real-world datasets and tasks. +See [`examples/timeseries/forecast.py`](https://github.com/NVIDIA/structured-data-models/blob/main/examples/timeseries/forecast.py) for runnable examples on real-world datasets and tasks. From 10cea8d15794f5b51573b451fb4239054af7fa5d Mon Sep 17 00:00:00 2001 From: rusty1s Date: Tue, 27 Oct 2026 17:09:52 +0000 Subject: [PATCH 6/6] update --- docs/source/forecasting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/forecasting.md b/docs/source/forecasting.md index 03a759583b..bdde01ed69 100644 --- a/docs/source/forecasting.md +++ b/docs/source/forecasting.md @@ -78,7 +78,7 @@ out["ice_cream__q90"].numerical # upper end of an 80% prediction interval ## Multivariate Forecasting -A models that {py:attr}`~sdm.models.ICLModel.supports_multi_target` can forecast multiple targets jointly, allowing them to exploit dependencies between them. +A model that {py:attr}`~sdm.models.ICLModel.supports_multi_target` can forecast multiple targets jointly, allowing them to exploit dependencies between them. In this case, we can simply add the daily sales of cold drinks as a second target:
ice cream4055807550???
cold drinks2530454128???
temperature2124282722
promotion00110100