fix(KDP): repair silent data defects in seasons, time series options and serialization by piotrlaczkowski · Pull Request #37 · UnicoLab/keras-data-processor

piotrlaczkowski · 2026-09-03T08:01:26Z

Describe your changes shortly

Follow-up to #36, which shipped as 1.12.0. This branch carries 45 further commits fixing defects that were silent — the model built, ran, and returned plausible numbers while quietly losing or fabricating data.

The most consequential:

Defect	What users got
`DateFeature(add_season=True)`	The season layer ran after the cyclic encoding and read its column 1 — `year_cos`, `1.0` for every row — as the month. Every date came out winter: four constant columns.
`MissingValueHandlerLayer`	The documented `extrapolate` option was read nowhere, so the default `forward_fill` returned a leading `NaN` untouched and it reached the model.
`TextPreprocessingLayer`	`get_config` wrote two keys `__init__` rejects, so any saved model containing the layer failed to load.
Empty categorical vocabulary	Replaced with `["<UNK>"]`: an all-out-of-vocabulary constant column for strings, an opaque `invalid literal for int()` for integers.
`AutoLagSelectionLayer`	Lags were chosen from channel 0 alone, so reordering the columns of the same data changed the result.
`WaveletTransformLayer(flatten_output=False)`	Returned `np.zeros(...)`, then raised on a rank-2 `set_shape`. Unusable on every input.
`TSFreshFeatureLayer(window_size=…)`	A window longer than the series declared a negative window count and raised.
`TimeSeriesFeature.get_output_dim()`	Counted wavelet, tsfresh and calendar as columns added to the series. They replace it, so the declared width matched no configuration.
`cyclic_encoding` + 6 more options	Accepted, stored, serialized — and read nowhere. Now implemented or documented as inert.

Behaviour changes

Output values change for add_season, Feature MoE, imputed series edges and the transforms above. docs/getting-started/upgrading.md documents each one and tells users to re-fit anything trained on the old output.

Guards added

Three static checks, each verified non-vacuous by reverting a fix and watching it fail:

every constructor argument must be read somewhere or documented as inert (found 8);
no class may write a get_config key its __init__ rejects (found 1);
a documented call to a method returning a tuple must be unpacked (found 2).

Found by CI on this PR, and fixed here

pre-commit-checks — scripts/generate_docstring_docs.py appended ---\n\n per class, so every generated doc ended with two newlines and end-of-file-fixer stripped them: regenerating the docs always left the tree dirty. Fixed in the generator, so a regeneration is now a no-op.
test-matrix (3.10, layers) — test_distribution_aware.py and test_distribution_aware_encoder.py both saved to the relative path "test_model.keras" and both deleted it. Under pytest -n auto they raced and one failed with File not found. Each now uses its own TemporaryDirectory. Latent in both files; surfaced because this branch changes how xdist distributes the group.
One test I added was fragile: it moved an input from 1.0 to 9.0 and asserted the MoE output changed, but the experts end in a ReLU that can be dead at both points. Measured over 8 seeds × 3 features: one probe misses ~1 in 24, four probes miss none. Now seeded and probed across four values.

Checklist

pre-commits are passing ! — green on CI; pre-commit run --all-files clean locally, hook for hook
unite-tests are passing ! — all 18 checks green: smoke-test, the 3.10/3.11/3.12 test-matrix, keras-latest, doc-examples, coverage. Locally 800 passed / 10 skipped / 2 xfailed / 0 failures, coverage 89%
code is well documented (docstrings in the corresponding format including examples of usage etc ...) — the doc-examples job builds and calls all 79 runnable documentation blocks on real rows, 0 failures; all 6 examples/ scripts run clean (locally — CI does not cover examples/)
code API is connected to the documentetion — mkdocs build --strict clean, generated API reference regenerated from docstrings
all requirements are added using poetry — poetry build succeeds; the wheel was extracted and smoke-tested in isolation (import, build, forward pass, save/load round trip)

Notes for the reviewer

Merge strategy matters for the version. The branch contains one feat(KDP) commit, so preserving individual commits yields 1.13.0. A squash using this PR's fix(KDP) title would yield 1.12.1 instead. fix(KDP): repair core defects, enable the linter and move to the Keras 3 API #36 preserved individual commits, so the default path gives 1.13.0.
All 45 commits were rewritten to the type(KDP): subject form the commit-msg hook and semantic-release's headerPattern require. Nine of them (and one older chore(KDP), a type the release config does not recognise) would otherwise have contributed no version bump and appeared in no release notes. The trees are byte-identical to before the rewrite.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…docs `_add_pipeline_text` defaulted `output_sequence_length` to 35 whenever the caller had not set it. TextVectorization rejects that argument outright for the multi_hot, count and tf_idf output modes, so those modes could never be built -- the default was injected before the layer was constructed and the call raised. It is now applied only for the "int" mode it belongs to, which makes `output_mode="multi_hot"` and `"count"` work. The date and text pages documented large APIs that do not exist. `DateFeature` reads exactly two keyword arguments, `format` and `add_season`; the page advertised ten more (`add_year`, `add_month`, `add_day_of_week`, `add_hour`, `add_is_weekend`, `add_quarter`, `cyclical_encoding`, `add_time_since_reference`, `reference_date`, `time_since_unit`). `TextFeature` forwards to TextVectorization; the page advertised GloVe, word2vec and BERT embeddings plus multi-head attention (`use_pretrained`, `pretrained_name`, `tokenizer`, `use_attention`, `attention_heads`, `attention_dropout`, `max_sequence_length`, `embedding_dim`, `sequence_length`). Both classes take `**kwargs`, so every one of these was accepted in silence and changed nothing -- confirmed by diffing model output with and without each argument. Both pages are rewritten against what the code does, including the encoding widths (8 dims for a date, 12 with `add_season`) and the real output modes, and state plainly that the removed options never worked. Every example on both pages was executed before being written down. Adds test/test_text_output_modes.py (5 tests). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…st the code Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…y name `category_encoding` and the placement options are compared against their constants by plain string equality, so a value that merely looked right matched nothing and the feature it controls was skipped without any error. `CategoricalFeature(category_encoding="hashing")` -- the spelling the docs used -- produced a bare width-1 StringLookup index: no hashing, no embedding, no one-hot. `feature_selection_placement="all"`, also from the docs, meant no feature selection at all. Both looked like working configurations. Both are now normalised case-insensitively and anything unrecognised raises with the valid options listed. The same guard covers `tabular_attention_placement`. The remaining docs are corrected against the code: the phantom `PreprocessingModel` keyword arguments are gone (they raised TypeError -- 19 of them across the optimization, feature-selection, custom-preprocessing and example pages), as are `CrossFeature`, `CompoundFeature`, `FeatureType.FLOAT_ARRAY`, and the date and categorical keyword arguments the source never reads. Adds 7 tests to test/test_text_output_modes.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

The "API Reference" section shipped as a placeholder reading "Auto-generated documentation for KDP classes will appear here" -- `docs/generated/api/` did not exist, so the entire reference was missing from the published site. The generator that fills it only ran again once its `KeyError` was fixed earlier on this branch, and its output was never produced. It now runs, and covers the whole package instead of a hand-maintained list of nine modules: fourteen of the twenty layer modules were absent, including all eleven time series layers. Layer modules are discovered from the package, so the reference stays complete as layers are added. 61 pages, up from 29 that were never published. The index also cut every multi-word module name mid-word -- it split the filename on the first underscore, so `DistributionAwareEncoder` was listed as `aware_encoder_layer_DistributionAwareEncoder`. It strips the module prefix instead. `PassthroughFeature`, `Feature`, `DistributionType` and `FeatureSpaceConverter` are exported from the top level alongside the other feature classes; the first has its own documentation page but could only be reached through `kdp.features`. test/test_docs_match_code.py checks every python block in the docs and the README against the package: that it parses, that imported names and `FeatureType` members exist, that `PreprocessingModel` keyword arguments are real, that feature keyword arguments are read somewhere rather than swallowed by `**kwargs`, and that option strings match a valid constant. Each of the six was failing on the published docs when written, and reintroducing a defect of any class makes it fail again -- verified by injecting one of each. Also documents the four `TimeSeriesFeature` parameters that are stored and serialized but never read by the pipeline, rather than leaving them to look like behaviour. Full suite: 477 passed, 12 skipped, 2 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

… exposed Executing every self-contained example in the docs -- 75 of them, against data synthesised from each block's own feature spec -- turned up four library defects that a symbol check cannot see. `get_feature_importances()` returned a description of each weight tensor (shape, dtype, layer name), so the documented `sorted(importances.items(), key=lambda x: x[1])` raised `TypeError: '<' not supported between instances of 'dict' and 'dict'`. The importances are a per-row softmax and only exist once data is run through the model, so the method now takes an optional batch and returns a float per feature. Note that each feature currently gets its own selector over a single feature, so every weight is 1.0 by construction; the page now says so instead of promising a ranking, and a test records it. `kdp.__version__` did not exist, though the installation page tells the reader to print it as the first check after installing. Every built model was named "preprocessor", and Keras requires unique operation names within a graph, so combining two preprocessors -- the two-tower recommender the integration guide demonstrates -- raised `ValueError: The name "preprocessor" is used 2 times in the model`. The name is now a constructor argument, defaulting to the previous value. The time series formatter rendered a `pd.Timestamp` with `str()`, producing "2023-03-01 00:00:00" where the history held "2023-03-01"; `DateParsingLayer` splits on the separator and calls `tf.strings.to_number`, so the trailing time failed with "Cast string to float is not supported". It also sorted that column with the Timestamp still in place, comparing it against strings, so the new row could land first while callers read the last row. Dates now render in the ISO form the layer parses, and sorting happens on a parsed key. `examples/time_series_inference.py` treated the preprocessor's output as a sales forecast: it wrote a six-element feature block back into the history as a scalar, which turned the column to strings on the next iteration. A preprocessor does not forecast, so the example now puts a small model on top, labelled as the stand-in it is, and no longer calls `.numpy()` on the NumPy arrays `predict()` returns. All six example scripts run to completion. Docs: 34 blocks that leaned on a previous block's imports now carry their own, `is_holiday` is replaced (KDP ships no holiday calendar) and the 17 calendar features that do exist are documented. Adds test/test_feature_importances.py, test/test_multiple_preprocessors.py and test/test_time_series_inference_formatting.py (13 tests). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…mples in CI `FLOAT_NORMALIZED` was documented as "0-1 normalization". It standardises to zero mean and unit variance: the column mean maps to 0 and values below it go negative, which a 0-1 scaling never does. `FLOAT_RESCALED` was documented as "robust scaling". It applies `keras.layers.Rescaling(scale=...)` with `scale` defaulting to 1.0, so the column passes through **unchanged** unless the caller sets the factor. It neither uses the statistics pass nor resists outliers. The page now says so and shows how to set `scale`; the quick-start comments that repeated the claim are corrected too. The passthrough page described the two output modes without stating what actually differs: with `include_passthrough_in_output=False` the model returns `{"processed": ..., "passthrough": {...}}` rather than a single tensor, which is what downstream code has to handle. scripts/verify_doc_examples.py is the harness used to find all of this, promoted from a throwaway: it runs every self-contained python block in docs/ against a CSV synthesised from that block's own feature spec. It runs as a `doc-examples` CI job and a `make verify_doc_examples` target, so a documented example that stops working fails the build rather than waiting for a reader to hit it. Adds test/test_numeric_feature_types.py (5 tests) pinning each numeric type's behaviour and output width. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`test_stats_pandas_vs_tf` built `DatasetStatistics` without `features_stats_path`, which defaults to `features_stats.json` in the working directory. Two assertions in test_public_api_surface.py -- that a numeric feature without `path_data` raises, and that hashing without an explicit bucket size still needs statistics -- then found that file and passed for the wrong reason. They failed only when the suite ran in full, and would have failed in CI where every job starts from a fresh checkout. The test now writes into its own temporary directory, and an autouse fixture removes a stray `features_stats.json` after any test that creates one, so the next test to rely on its absence cannot be misled again. This is the same shared-state trap that hid the `path_data` defect earlier on this branch. Also supplies the `scale` that `keras.layers.Rescaling` requires in the custom-pipeline example, which had no default and raised `TypeError`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…carded Three prominent, documented options had no effect at all. `use_advanced_numerical_embedding` accepts `embedding_dim`, `mlp_hidden_units`, `num_bins`, `init_min`, `init_max`, `dropout_rate` and `use_batch_norm`. They were handed to `add_processing_step`, whose layer creator is `lambda **kwargs: embedding_layer` -- it ignores every keyword and returns a layer already built from the feature's own defaults. The output width was 8 per feature no matter what `embedding_dim` said. The settings now reach the feature, which applies them to whatever it did not set itself, so a value on an individual `NumericalFeature` still wins. In concat mode -- the default -- `FeatureMoE` was constructed with only `num_experts`, `expert_dim` and `routing`, so `feature_moe_hidden_dims`, `feature_moe_sparsity`, `feature_moe_freeze_experts` and `feature_moe_dropout` did nothing, and `feature_moe_routing="predefined"` raised because the feature names and assignments never arrived. The dict-mode branch already passed all of them; concat mode now matches. `freeze_experts` passed `training=False` to each expert, which selects inference behaviour for dropout and batch norm but leaves the weights trainable, so the experts were still learning. They are now marked untrainable, which is what the option says it does. Adds test/test_advanced_embedding_options.py and test/test_feature_moe_options.py (9 tests), and documents that the advanced embedding output is one `embedding_dim`-wide block per numeric feature. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`tabular_attention_dim` is the model's output width with the default placements, because attention projects every feature into one shared space rather than concatenating them; `multi_resolution` concatenates a numeric and a categorical branch and so doubles it. `tabular_attention_embedding_dim` sizes the categorical embedding inside that branch, changing the parameter count but not the width. All four verified against the built model. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`auto_configure("data.csv")` is the call the docstring leads with and the headline "let KDP configure itself" entry point. With no `features_specs` it passed `None` to `DatasetStatistics`, which then had no per-type feature lists, computed nothing, and returned empty recommendations and empty statistics for every dataset -- only a generic code template came back. The earlier fix on this branch covered the case where specs *are* supplied; this is the case where they are not. Feature types are now inferred from the data: a whole-numbered column with few distinct values is a categorical code rather than a quantity, a string column that parses as a date is a date, values of at most three words are labels, and longer ones are text. A high-cardinality single-token column such as an id stays categorical, where hashing keeps it bounded, instead of becoming a text feature with a vocabulary the size of the dataset. `ModelAdvisor` also read date statistics from `date_stats` while `DatasetStatistics` writes them under `date`, so a dataset with a date column came back with no recommendation for that column at all. Both spellings are now accepted. That recommendation described `extract`, `timezone_aware` and `cyclical_encoding`. `DateFeature` reads only `format` and `add_season` -- the components and their cyclical encoding are fixed -- so it was telling the reader to write settings that do nothing. It now offers the two real options and explains the fixed 8-column encoding. The existing advisor test asserted the phantom options and is updated to the corrected contract. Adds test/test_auto_configure_inference.py (12 tests), including one that executes the generated code snippet, and test/layers/test_date_layers_thorough.py (20 tests) covering the parsing, encoding and season layers -- day-of-week is checked against Python's own calendar on leap days and century boundaries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`_process_features_parallel` and `_parallel_setup_inputs` -- 70 lines that group features by type, set up inputs across a thread pool and dispatch each group -- are called from nowhere. Neither appears anywhere else in the package, the tests, or the docs, and no dynamic dispatch reaches them. `_apply_feature_moe` looked for `self.pipeline_<feature>` attributes to pull individual feature outputs from. Nothing ever sets such an attribute, so that branch was always false and the code always fell through to splitting `concat_all`; the split is now the only path. Five helpers in `DistributionTransformLayer` (`_calculate_skewness`, `_calculate_kurtosis`, `_has_negative_values`, `_has_zeros`, `_is_bounded_01`) and one in `DistributionAwareEncoder` are likewise unreferenced -- the `auto` selection computes those statistics inline instead. Adds test/layers/test_distribution_auto_transform.py (9 tests). `auto` had no coverage at all despite choosing the transform for every distribution-aware column: the tests check it stays finite on strictly positive, zero-containing, mixed-sign and bounded data, that it actually reduces skew rather than merely running, that it preserves shape, survives serialization and traces into a functional model. Documents the twelve transformations `transform_type` accepts, three of which (`arcsinh`, `cube-root`, `logit`) appeared nowhere in the documentation, along with what each one requires of its input. Full suite: 550 passed, 12 skipped, 2 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…tureLayer Twelve of the layer's seventeen calendar features had no test. Two of them did not work. `week_of_year` was NaN for every row. `DatetimeIndex.isocalendar()` returns a frame indexed by the timestamps themselves, so assigning `.week` into a column of a RangeIndex frame aligned on the index, matched nothing, and filled the column with NaN -- which then propagated into the model. The values are taken positionally now, and a test checks them against pandas rather than against a hand-written expectation. `hour`, `minute` and `second` are listed as valid and carry cycle lengths, but the non-cyclic path had no branch for them, so asking for them by their plain names raised; only the `_sin` and `_cos` spellings worked. They now read the timestamp, and the documentation notes that a date-only column has no time and so reports zero for all three. Adds test/layers/test_calendar_feature_layer_all.py (11 tests) covering every feature name: that the flags fire on the right dates (month, quarter and year boundaries, and a Sunday), that normalised values stay within [0, 1], that a `_sin`/`_cos` pair lies on the unit circle, that an unknown name is rejected, and that the configuration round-trips. Layer suite: 275 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`drop_na` defaults to True, and in that mode the layer declared its output shape as `rows - max_lag` while `create_lag_features` had already allocated `max(1, rows - max_lag)` rows. Whenever the input had fewer rows than the largest selected lag -- every single-series batch, and so the default configuration on any one series -- `set_shape` was handed a negative dimension and raised. The declared shape now uses the same clamp as the allocation. Both of the file's skipped tests were skipped for the wrong reason and are now enabled and passing: `test_drop_na` was skipped as "requires a negative batch dimension which is not supported in TensorFlow". That negative dimension was the bug above, not a property of the test. `test_call_2d` was skipped as needing "exact lag feature values that are difficult to match". Its shape and value assertions were always correct; what was wrong was the second half, which read column `i` as holding lag `i`. The layer selects its lags from the autocorrelation -- on this fixture it picks 3, 10 and 13 -- so the columns never held lags 1, 2 and 3. The test now asserts against the lags the layer actually selected, which is the property worth checking. Also adds test/layers/test_tsfresh_features_all.py (16 tests). The layer advertises 26 statistics and most had none; every scalar one is now compared with its numpy equivalent rather than merely asserted finite. Two of them, `peak_count` and `valley_count`, return a rate rather than a count, and `first_location_of_max`/`_min` return a fraction of the series length; the docstring described all four as absolute values and is corrected. Full suite: 581 passed, 10 skipped (down from 12), 2 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`test_automatic_detection` ran distribution detection over ten generated shapes and printed the result. It asserted nothing, so a detector returning an out-of-range index or the same answer for every input would have passed. Its fixture was also unseeded, which made detection non-deterministic: the same heavy-tailed shape came back as `heavy_tailed` or as `cauchy` depending on the draw. The fixture is seeded now, and the expected mapping was measured across six seeds rather than taken from one run. Detection identifies `heavy_tailed`, `log_normal`, `discrete`, `periodic`, `sparse` and `beta` correctly and consistently. Four shapes are consistently confused with a neighbour -- `normal` and `uniform` both read as `multimodal`, `multimodal` as `periodic`, and `exponential` as `log_normal`. The test asserts the full mapping as it stands, so a change in either direction is visible, and the documentation states the limitation and points at `preferred_distribution` for columns where the shape matters. `test_add_step_and_chain` had its only assertion commented out under a `# TODO: Fix this test`. It named `mock_step.chain`, but `Pipeline.chain` calls `connect` on each step -- the assertion was wrong, not unfixable. It now checks that `connect` is called once with the input and that its return value is what the pipeline hands back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`auto_configure` presents its code snippet as ready to use, so a keyword the target class silently ignores is worse than an error: the reader believes the setting took effect. It emitted `embedding_dim` on `TextFeature`, which forwards its keyword arguments to `TextVectorization` and has no such argument, and `date_format`, `output_format` and `extract` on `DateFeature`, which reads only `format` and `add_season`. All four were accepted and did nothing. The text recommendation no longer records an embedding dimension at all, and the date branch emits the two real options. test/test_auto_configure_inference.py now checks every keyword the snippet writes against what each class actually honours, across a dataset varied enough to exercise all four feature classes, and executes the result. The existing advisor test asserted the phantom `embedding_dim` was present and now asserts it is absent, with the reason. Also covers the inference edge cases: a directory of CSV shards, an empty directory (which reports clearly rather than failing inside pandas), an all-empty column, columns with gaps, and the rule that a float column of few whole values is read as a code rather than a measurement. Coverage across the package is 85%, up from 77% at the start of this pass; missed lines are down from 743 to 477. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…ranches `TimeSeriesInferenceFormatter.generate_multi_step_forecast` called `self._check_history_requirements(...)`. No such method exists -- the real one is `_check_inference_data_sufficiency` -- so this public, documented entry point raised `AttributeError` on every call, with any input. It had no test, which is why nothing caught it. The documentation showed the call with the right arguments, so the example was correct and the method was not. It also reads as though it returns forecasts. It returns a placeholder frame: one row per future date, sort column filled, every time series feature NaN, for the caller to fill in step by step. The page now says so and points at the example that puts a model on top. The advisor's `uniform` branch recommended `FLOAT_RESCALED` while recording only `min` and `max`, which `NumericalFeature` does not read. `FLOAT_RESCALED` multiplies by `scale`, which defaults to 1.0, so the recommendation left the column untouched while its note claimed rescaling. It now derives `scale` from the observed range, guards a zero span, and the snippet writes it. New tests: - test/test_multi_step_forecast.py (8): the frame's shape and contents, the `steps` limit, both error paths, dict input, and a grouped series. - test/test_model_advisor_branches.py (12): every distribution branch produces a preprocessing choice and an explanation, the categorical cardinality paths, the embedding and bin-count helpers, and -- the point of the file -- that every configuration key the advisor emits is one the target class actually reads, and that its global config names only real `PreprocessingModel` parameters. - test/test_stats_serialization.py (9): each branch of `_custom_serializer`, including the bytes case that used to write `"b'paris'"` into vocabularies, plus a save/reload round trip and the numeric statistics checked against the data. KDP reports the sample variance (ddof=1); the test says so rather than comparing against numpy's population default. Full suite: 609 passed, 10 skipped (all TensorFlow internals), 2 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…tures The predefined assignment matrix doubles as the router's weights, so a feature the caller left out of `feature_moe_assignments` got an all-zero row and its entire representation was multiplied away. The build succeeded, the model ran, and the feature was simply gone. An expert index outside the mixture was the same failure by another route. `FeatureMoE` now validates the map up front and names the problem: missing features, names the mixture never sees, out-of-range or non-integer expert indices. Weighted assignments (`{0: 0.7, 1: 0.3}`), which the matrix builder already supported, are accepted. Also documents the five constructor parameters that no page mentioned -- `feature_moe_assignments`, `feature_moe_dropout`, `feature_moe_freeze_experts`, `feature_moe_use_residual` and `log_to_file` -- and adds a guard so a parameter cannot reach users undocumented again. The `_UNSET` sentinel was a bare `object()`, which rendered in the generated API reference as `<object object at 0x7f37...>`: unreadable, and a fresh address on every regeneration. It is a small class with a `<unset>` repr now, and the generator skips private classes so it does not publish a name nobody can import. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

TransformerBlock had neither `build` nor `get_config`, and no tests: its test file existed and was empty. Keras 3 warned on every model that used it that the layer "does not have a `build()` method implemented and it looks like it has unbuilt state ... which may cause failures down the line". It builds its sub-layers now, reports its configuration, and says plainly when the input width does not match `dim_model` instead of failing inside `Add`. TabularAttention created a dropout, two layer norms and an output projection that `call` never used. They held no weights, so removing them changes no output; it removes four layers that read as part of the computation and are not. `TextVectorizerOutputOptions` existed twice: `kdp.processor` held the strings Keras accepts, `kdp.features` held `auto()` integers. Whichever one a caller imported decided whether their `output_mode` worked or was silently discarded. There is one class now, exported from `kdp`, and `CrossFeatureOutputOptions` is spelled the same way. Six methods the documentation showed did not exist: `get_timing_metrics`, `get_memory_usage`, `plot_model`, `get_feature_importance`, `plot_feature_importance`, `get_top_features`, plus `update_statistics`, `transform` and `fit`. The first three are implemented -- the timing and memory numbers were already being measured by `_monitor_performance` and thrown away -- and the pages using the others now show the working equivalents. Ten cross-links between doc pages pointed at files that do not exist, including the homepage's own link to Feature Selection; `mkdocs --strict` misses these because the docs navigate with raw `<a href>`. The claim that feature selection "identifies your most important features" is corrected wherever it appeared, since every weight it reports is 1.0. Finally, the marker rules assigned `fast`/`slow` *instead of* `unit`, so 26 tests matched no CI group and never ran -- the entire save/load round-trip suite among them, because every one of its names contains "model". Group markers are exhaustive now, and CI fails if a test ever matches none of them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…classes `DistributionType` was defined twice, in `kdp.features` and in the encoder layer, and the copies disagreed: the features one carried a `WEIBULL` the encoder has never known. Asking for it was warned about and silently replaced with "normal". The encoder decides which values are valid, so its class is the one `kdp.features` re-exports, and every member the package exposes is now a distribution the encoder accepts. Two guards go with it: no public class may be defined twice, and every member of the exported enum must be accepted by the encoder. The docstring generator built its index by globbing the output directory, so a page written for a class that was later removed or moved stayed in the published API reference indefinitely -- two such pages were still listed. It prunes each module's pages before regenerating them now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…he advisor `kdp.moe` and `kdp.model_advisor` were absent from the generator's module list, so `FeatureMoE`, `ExpertBlock`, `StackFeaturesLayer`, `UnstackLayer` and `ModelAdvisor` had no API page -- while Feature MoE and `auto_configure` are documented as headline features elsewhere. A guard now fails if any module defining public classes is left off the list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`feature_moe_sparsity` is documented as "use only top k experts" and did nothing: the top-k mask that enforces it sat behind `routing_activation != "softmax"`, a knob `PreprocessingModel` never exposes, so the branch was unreachable and every feature was routed densely to every expert. Forwarding the option, as the previous commit did, was not enough -- the code reading it could not run. Sparsity now applies whenever it is smaller than the number of experts, so the defaults (4 experts, sparsity 2) give the top-2 routing the docs describe. The existing test asserted only the output shape, which is identical either way; it now checks that each feature reaches exactly `sparsity` experts and that the weights still sum to one. `get_expert_assignments()` returned `{}` for learned routing -- the default -- while a pro tip told users to plot those assignments as a heatmap. The router decides from the feature representations rather than from its weights alone, so it takes a batch now and answers for both routing modes as `{feature: {expert_index: weight}}`. Calling it without a batch under learned routing says why instead of returning nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…ct-use layers Missing values were found with `inputs == mask_value`, and NaN compares equal to nothing -- not even itself -- so a series carrying the marker pandas and numpy actually use passed through untouched, and those NaNs then poisoned every statistic computed from it. No value of `mask_value` could select them. Setting it to NaN now selects NaN; a sentinel such as `0.0` behaves exactly as before. `FFTFeatureLayer`, `SeasonalDecompositionLayer`, `MissingValueHandlerLayer` and `AutoLagSelectionLayer` work and are tested, but `TimeSeriesFeature` has no config for them and no written page named them, so they existed only in the generated API reference. The time series page now shows how to use them directly, with examples that run as written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

`kdp.layers` and the model advisor spell this option `prefered_distribution`, with one "r", while `NumericalFeature` takes `preferred_distribution`. The feature swallows `**kwargs`, so the misspelling was discarded and the feature stayed on automatic detection while looking configured. Six tests in test_processor.py used that spelling and passed while asserting nothing about distributions, and a documented example taught it. The advisor made it worse: it recommended the misspelled key, and the code snippet it generates left the distribution out entirely, so following auto_configure() end to end gave none of its distribution advice. The feature now accepts the misspelling as a deprecated alias with a warning, the advisor emits the correct key, and the snippet writes it onto the feature. Two failures from the last full run are fixed here as well: - `tf.nn.top_k` rejects a float `k`, and a config round-trip brought `sparsity` back as one, so a saved mixture-of-experts model failed to rebuild. It is pinned to an int. - The doc-example harness ran blocks from the repository root, so a block naming a relative `features_stats_path` wrote `features_stats.json` there. Every later test asserting "without data this must raise" then found those statistics. The harness runs from a temporary directory now, and the session clears a stray file before the tests start. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

The sweep behind this: perturb one input column and watch the output. A column whose change moves nothing is being dropped somewhere, silently, because the model builds and runs either way. Three of seventeen configurations failed. `processed_features_dims` was written as a flat `{name: dim}` mapping and read back as a nested one keyed by "numeric"/"categorical", so the lookup always missed -- and the loop filling it compared feature *tensors* against input *names*, so it was empty regardless. The mixture then fell back to cutting `concat_all` into equal parts. With features of unequal width that split lands mid-feature: widths 1, 1 and 10 were cut as [4, 4, 4], so every "feature" the router saw spanned several real ones and `feature_moe_assignments` named the wrong slice. Nothing raised. In dict mode the same mismatch raised on the first batch instead of at build time, because `StackFeaturesLayer.compute_output_shape` reported the first feature's width whatever the rest were -- a graph the layer cannot execute. It now refuses to report a width it cannot produce. Both take their widths and names from the tensors actually concatenated, and pad features to a common width before stacking. Padding costs no parameters and leaves a uniform-width model untouched. `use_feature_moe` with `use_global_numerical_embedding` raises, since the global embedding merges every numeric feature into one vector and leaves nothing to route. Separately, `NumericalEmbedding` embeds each column it receives, so a one-column feature came back rank 2 and a discretised one rank 3. Mixing them failed in `Concatenate`, and a model of only discretised features silently produced a three-dimensional output. The embedding is flattened now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

Learned routing fed a Dense layer the batch *mean* of the features, so the routing weights changed with whatever rows shared a batch. Measured on a twelve-row batch: changing one row moved all twelve outputs, and a record scored alone differed from the same record scored in the batch. That is train/serve skew with no configuration explaining it, and one row's values leaking into another's output. Feature-level routing is a property of the feature, not of the data passing through it -- which is what the code comment claimed it was computing. The logits now live in a trainable weight with one row per feature, created in `build`. Routing is identical for every row and every batch, and still learned. `ExpertBlock` gained the `build` its sub-layers needed, so Keras no longer warns that the block "looks like it has unbuilt state". Two new suites hold the line: one perturbs a single feature and fails if the output does not move, the other checks that rows come back in the same number and order and that a row scored alone matches the same row scored in a batch. Both run across concat and dict modes, attention, transformer blocks, feature selection, the embeddings and the mixture. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…-mode defect Extends the save/load matrix from nine configurations to seventeen, and adds a discretised feature so every one of them now runs on a feature set of mixed widths -- the case the mixture used to slice mid-feature and where its padding has to survive the round trip. Sparse routing, predefined routing, multi-resolution attention and dict-mode MoE are covered for the first time. That found a real defect, which this commit records rather than hides: Feature MoE with `output_mode="dict"` does not survive a save/load round trip. Two features come back holding each other's projection kernels -- `score`'s reloaded kernel is byte-identical to `occupation`'s original. Weights, wiring, routing logits and every intermediate tensor up to and including the mixture output match; only the per-feature projections are crossed. Concat mode round-trips exactly, and the model is correct in memory, so the fault is in how the sibling leaves of a dict-output graph are ordered when written and read back. Ruled out along the way: asymmetric graph depth (padding every feature, widest included, changed nothing), layer naming, creation order, and the residual connection. A minimal reproduction with the same stack, mixture and projections round-trips correctly, so the trigger needs more of the real graph than that. It is a strict xfail with the reproduction in its reason, KDP logs a warning when the combination is built, and both the Feature MoE page and the upgrade guide name it with the concat-mode workaround. A separate probe confirmed no value is lost elsewhere: distinct categories, integers, dates, sentences and large ids all stay distinct, normalisation matches (x-mean)/std, and passthrough returns its input unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

Sweeping the options you can put on a *feature* -- as opposed to the model-level switches the earlier passes covered -- found four of twenty-six paths broken, none of them loudly. `output_mode="tf_idf"` could not build. Keras refuses a vocabulary in that mode without matching `idf_weights`, which KDP's statistics never collected, so an exported and documented output mode raised on every use. The vectorizer is adapted on the column now, computing the vocabulary and the weights together from the same data the statistics came from. `NumericalFeature(use_embedding=True, embedding_dim=6)` was accepted and discarded: the flag was assigned in `features.py` and read nowhere, so the feature came through one column wide. It now enables the embedding for that feature alone. `wavelet_transform_config` crashed. The layer computes `min(window, time_steps // 2)`, which is zero for the one-column input a time series feature hands it, and the empty coefficients failed to broadcast; it also wrote the window sizes back onto itself, so the first batch pinned them permanently. Both are fixed, and a wavelet with a single step to work on raises rather than emitting a constant column of zeros -- combine it with lags or rolling statistics so there is a window to decompose. `calendar_feature_config` failed with "Cast string to float is not supported": the pipeline declared the column float and cast it before the calendar layer saw it. Such a feature declares a string column now and skips the numeric front of the pipeline, and combining it with the numeric configs raises, since one column cannot be both a date and a number. `DatasetStatistics._get_csv_file_pattern` is a staticmethod, since the processor needs the same path resolution to adapt a vectorizer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…ic configs Two tests built a `TimeSeriesFeature` with every config at once, calendar included, and asserted only that the attributes were stored. They never built a model from it, so they never surfaced that the combination cannot work: the calendar layer reads date strings where the numeric configs read numbers, out of the same column. They now cover the numeric configs together and the calendar config on its own, and assert that mixing them raises. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

Running an example wrote its PNGs into the repository root, so anyone who tried one came back to a dirty working tree with three untracked images. They go to a gitignored `examples/outputs/` beside the scripts now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…mber Every one of them configured `calendar_feature_config` on the same `TimeSeriesFeature` as `lag_config`, `rolling_stats_config` and the rest. The calendar layer reads date strings where those read numbers out of the same column, so a model built from any of these examples raised "Cast string to float is not supported" the moment it was called. The doc verifier had passed them because it builds each block without calling it. They now declare the calendar features on the date column and the numeric transforms on the numeric one, and a guard fails if an example mixes them again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

The last known defect, and the mechanism turned out to be general enough to be worth naming: `.keras` stores a layer's weights under a name derived from its class and the order it was built -- `dense`, `dense_1`, `dense_2` -- not under the name the layer was given. Keras reorders sibling layers when it rebuilds a functional graph from config, so any group of same-class siblings whose rebuild order differs from their build order comes back with its weights permuted. Dict mode built one `Dense` per feature, four siblings, and two of them crossed: `score` came back holding `occupation`'s kernel and vice versa. What isolated it was that `save_weights`/`load_weights` into a freshly built model is exact while a full `save`/`load_model` is not -- the weight file is right, the rebuilt graph is not. Reordering the layers does not help, because the rebuild order is Keras's to choose. They are one `PerFeatureDense` layer now, holding a kernel and bias per feature and applying them with an einsum. The arithmetic is identical to a Dense per feature, and one layer has no sibling to be confused with. Its round trip is exact, so the xfail, the runtime warning and the documented workaround are all gone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…lled `get_feature_importances()` returned 1.0 for every feature, on every input. Selection wrapped each feature in its own `VariableSelection` with `nr_features=1`, and a softmax over one element is 1.0 by definition, so the gating was real but the numbers ranked nothing -- the documentation had to warn readers not to read a ranking into them. A single softmax now scores every selected feature against the others, and the score scales that feature's output. The scores sum to one and are sortable. Output widths do not change: each selected feature is still `feature_selection_units` wide, so the shape of the model is the same. A model with one selected feature has nothing to compare against and keeps its weight of 1.0. The scores are stacked, multiplied and unstacked with the layers already in the package rather than sliced out with a `Lambda`. A Lambda carries a serialized Python closure, and this release has already spent enough time on layers that do not survive a round trip; the round trip is asserted. The doc-example verifier also builds every model and never called one, which is precisely how four published examples that raised on their first batch went unnoticed until this week. It now feeds each model it built two rows of the same synthesised data. Proved against a model that builds and then rejects its own input: the harness reports the failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…hannel Two paths reached the user as silence or as an error from inside Keras. A categorical feature whose column holds no values came back with an empty vocabulary, which was replaced with ["<UNK>"]. For a string feature that encoded every real value to the out-of-vocabulary slot -- a constant column, produced without a word. For an integer feature it failed inside Keras with "invalid literal for int() with base 10", naming neither the feature nor the cause. Both now raise a ValueError that names the feature and points at CategoryEncodingOptions.HASHING, which needs no vocabulary and does build on a column with no rows. The check stays narrow on purpose: a column of empty strings has the vocabulary [""] and a constant column has one of length one. Neither is empty, both are legitimate, and the tests hold them to building. AutoLagSelectionLayer chose its lags from the autocorrelation of channel 0 and applied them to every channel. With noise in channel 0 and a clean period-10 wave in channel 1 it selected lags [11, 15, 26]; swapping the two columns of the same data gave [5, 10, 15]. The autocorrelation is now averaged over the channels as well as the batch -- the mean across the batch was already how a single set of lags was chosen -- so every channel contributes and the result no longer depends on column order. The 2-D path is untouched. Also replaces three "Single feature for now" comments that read as an unfinished limitation. nr_features=1 is correct there: the selector gates one feature against itself, and the features are scored against each other afterwards in _score_features_against_each_other. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

WaveletTransformLayer(flatten_output=False) returned np.zeros(...) -- every coefficient it had just computed, thrown away -- and then declared a rank-2 shape for the rank-3 tensor it returned, so set_shape rejected it and the option raised on every input before the zeros could reach anyone. It now returns the same coefficients as flatten_output=True, split out per input channel; a test holds the two to being equal under a reshape. TSFreshFeatureLayer(window_size=...) computed the window count from the raw window_size while the extraction clamped it to the length of the series. A window longer than the series declared a negative count and raised "Dimension -1 must be >= 0". call() now sets the shape compute_output_shape reports, which is the one place that arithmetic lives. CalendarFeatureLayer(cyclic_encoding=...) was accepted, stored, serialized and read nowhere: the output is identical whichever value is passed. The sin/cos components are requested by name, and always were, as month_sin and month_cos. The flag is deprecated -- it warns when a caller passes it, stays out of get_config unless it was passed, and still loads. Six documentation examples promised cyclic encoding through the flag and now ask for the components. TimeSeriesFeature.get_output_dim() counted all three of those transforms as columns added to the series. They replace it, so the declared width matched no configuration at all: a wavelet after two lags reported 5 columns against an actual 2, and tsfresh reported 8 against 5. Each layer is now asked for its own width. Sixteen configurations agree, up from six, and the eight that reach a transform are in the parameterized test that compares the declared width to the tensor the stack produces -- which covered none of them before. Inside a feature, a windowed tsfresh and an unflattened wavelet both return rank-3 tensors; both are flattened back into the feature's columns, so what follows still sees (batch, columns). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

MissingValueHandlerLayer has an `extrapolate` option, documented, defaulting to True, naming exactly the case no strategy can reach: a gap at the very start has nothing before it to carry forward, a gap at the end nothing after it to carry back. It was stored, serialized, and read nowhere. With the default forward_fill strategy and mask_value=NaN, a leading NaN came back exactly as it arrived and went into the model; backward_fill and linear_interpolation did the same at the end. Those gaps now take the nearest value that is not itself a gap, so no strategy leaves a missing value behind, and a series missing everywhere becomes zeros rather than NaN. extrapolate=False keeps the old behaviour and now genuinely means it. Six more arguments were stored, round-tripped through get_config, and never read. Each is now documented as inert rather than describing behaviour that does not happen: WaveletTransformLayer(drop_na) nothing carries NaN to drop MovingAverageLayer(pad_value) leading rows keep their own values CalendarFeatureLayer(onehot_categorical) one numeric column per feature DistributionAwareEncoder(handle_sparsity) sparse is a detected distribution DateParsingLayer(date_format) both spellings parse regardless FeatureMoE(routing_activation) routing is always softmax; use sparsity test_no_option_is_silently_ignored.py walks every class in kdp and fails on a constructor argument stored and never read, unless the docstring says it does nothing. It counts reads through subclasses and through getattr, so a feature attribute the processor reads later is not mistaken for a dropped one, and it carries a test that it still catches a dropped argument. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…n guard The guard cleared an argument when the attribute name was read anywhere in the package, which is what a feature class needs -- `NumericalFeature` stores `use_embedding` and the processor reads it much later, sometimes through getattr. A layer is the opposite: it reads its own options in __init__, build or call and nothing reaches in for them, so a matching name on an unrelated object was evidence of nothing and would have let a dead layer option hide behind it. Layers now count only their own reads, their subclasses', and arguments consumed while building sublayers. Still green, so nothing was hiding. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

The heading counted three options that "could not run", but only two of them raised; the third was accepted and did nothing, as were the seven in the table below it. The note about `month` not being cyclic now sits next to the `cyclic_encoding` paragraph rather than after the guard. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

DateFeature(add_season=True) added four columns that were the same on every row. The pipeline ran SeasonLayer after DateEncodingLayer, and the layer reads the month from column 1 of its input -- which by then was year_cos, 1.0 for every date. Cast to int that is 1, which the layer calls January, so every date came out winter. January, April, July and October all encoded to [1, 0, 0, 0]. The season is now read before the encoding replaces the month, and DateEncodingLayer carries through any columns a layer before it appended instead of returning exactly eight and dropping the rest. The output is the same width as before -- twelve columns, as the documentation already said -- and the four season columns now carry the season. Nothing caught this. The layer's own tests feed it [year, month, day, day_of_week] directly, which is what its docstring asks for and works correctly; the processor tests only asserted the output was at least four columns wide. The new test builds a date-only model and checks that four months from four seasons give four different encodings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

… back get_config wrote punctuation_pattern and stop_words_pattern, two values the layer derives from stop_words and which __init__ never accepted. Keras rebuilds a layer by handing the stored config to the constructor, so any saved model containing this layer failed to load outright with "Unrecognized keyword arguments passed to TextPreprocessingLayer". The layer is reachable through PreprocessorLayerFactory.text_preprocessing_layer, so a custom pipeline using it could be saved and never reopened. get_config now writes only stop_words, and from_config drops the two derived keys so files written by earlier releases still load. A test walks every class in kdp and fails when get_config writes a key __init__ will not take. It found this one and nothing else, and it carries a round trip of the layer through save and load plus an old-style config. This came from the coverage report: both methods were among the few uncovered lines in the package, and neither had ever run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

Two blocks stood out in the coverage report as never having run. Neither was broken, but neither was reachable by any test either. add_feature_moe_to_model stacked the feature outputs as they came, and stacking needs them the same width. Preprocessing rarely produces that, so the helper worked only by coincidence -- and after the stack layer started rejecting mismatched widths, it failed outright on the ordinary case. It now pads to the widest, which is what PreprocessingModel does for the same reason; the widest feature is padded by nothing. Tests cover three width patterns including [1, 1, 10], the shape that exposed the earlier slice-routing bug, and check that perturbing one input still moves that feature's own output and that the result survives a save and load. DistributionTransformLayer builds its candidate transforms from the data's own shape only when a caller passes transform_type="auto" with an explicitly empty auto_candidates; the default None makes __init__ fill the list in, so that branch had never executed. Now covered for strictly positive data, positive data with zeros, mixed signs and data bounded in (0, 1). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

PreprocessingModel.load_model returns (model, metadata), and quick-start.md shows that. Two other pages assigned it to a single name and called that name loaded_preprocessor, so a reader following them would write loaded_preprocessor.predict(...) next and get "'tuple' object has no attribute 'predict'". One of them also passed "my_preprocessor.keras" to save_model, which takes a directory, not a file. The runnable-example harness cannot catch either: it skips every block calling load_model, since it has no saved directory to hand it. A guard in test_docs_match_code.py now fails when a documented call to a PreprocessingModel method annotated as returning a tuple is assigned to one name. It is scoped to the class, so keras.models.load_model -- a different function that really does return a model -- is left alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

github-actions · 2026-09-03T08:02:39Z

PR Preview Action v1.8.1
🚀 View preview at https://UnicoLab.github.io/keras-data-processor/pr-preview/pr-37/
Built to branch `gh-pages` at 2026-09-03 08:21 UTC. Preview will be ready when the GitHub Pages deployment is complete.

Every class section is appended with a trailing blank line, so each generated file ended with two newlines and the API index with an extra one. `end-of-file-fixer` strips that, so regenerating the docs always left the tree dirty and the pre-commit CI job failed on all 67 files -- which is exactly what happened on this PR's first run. The generator now writes one trailing newline, so its output is already what the hook wants and a regeneration is a no-op when nothing else changed. `pre-commit run --all-files` passes locally, hook for hook. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

…ng directory test_distribution_aware.py and test_distribution_aware_encoder.py both saved a model to "test_model.keras" -- a relative path, so the same file in the shared working directory -- and both deleted it afterwards. CI runs the suite with `pytest -n auto`, so the two land on different workers and race: whichever finished first removed the file, and the other failed with AssertionError: Model saving/loading failed: File not found: filepath=test_model.keras which is what test-matrix (3.10, layers) hit on this PR. The race is latent in both files and only surfaced because this branch adds tests to the layers group, changing how xdist distributes them; the second test swallowed all exceptions, so only the first ever reported. Each now saves into its own TemporaryDirectory, so there is no shared name left to race over -- and the directory cleans itself up when a test fails, which the manual os.remove in the failure path did by hand. The second test no longer catches and prints its exception either: a save/load failure there was invisible. A scan of every relative path literal in test/ found no other pair sharing a name outside a temporary directory. `pytest -c pytest-fast.ini -m layers` passes: 361 passed, 7 skipped. The whole suite under that config passes too: 800 passed, 10 skipped, 2 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

test_every_feature_still_reaches_its_own_output moved one input from 1.0 to 9.0 and asserted the feature's own output changed. The experts are randomly initialised and end in a ReLU, so for any single value that ReLU can be dead at both the baseline and the perturbation, and the output legitimately does not move -- the test then failed with "True is not false : c" on nothing but an unlucky initialisation. It is a test I added on this branch, and it failed in the unit group under `pytest -n auto`. Measured over eight seeds and three features: one probe misses about once in twenty-four checks; the four now used -- negative, small, large and very large -- miss none. A feature that genuinely did not reach its output would move for none of them, so the assertion still says what it did. The model is also seeded now, and the check runs over three seeds, so a failure is reproducible instead of a fresh model every run. `pytest -c pytest-fast.ini -m unit` passes: 352 passed, 1 skipped, 2 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

## 1.13.0 (2026-09-03) * chore: save last release version for recovery [skip ci] ([302132d](302132d)) * fix: repo maintenance update n40nk0 (#38) ([5f79944](5f79944)), closes [#38](#38) * fix(KDP): build the transformer block, and make the documented API real ([3193f3b](3193f3b)) * fix(KDP): compute the statistics from the right rows, at the right precision ([a76c574](a76c574)) * fix(KDP): deliver the embedding and Feature-MoE options that were discarded ([97cc77e](97cc77e)) * fix(KDP): dict-mode Feature MoE survives a save/load round trip ([ea95742](ea95742)) * fix(KDP): end generated API docs with one newline, not two ([7e7aa19](7e7aa19)) * fix(KDP): fill gaps at the ends of a series and name the inert options ([e22b654](e22b654)) * fix(KDP): honour preferred_distribution however it is spelled ([e4a5b7e](e4a5b7e)) * fix(KDP): let MissingValueHandlerLayer see NaN, and showcase the direct-use layers ([64d2abe](64d2abe)) * fix(KDP): make auto_configure work without features_specs, as documented ([30f86d1](30f86d1)), closes [hi#cardinality](https://github.com/hi/issues/cardinality) * fix(KDP): make AutoLagSelectionLayer work with its default settings ([7c37096](7c37096)) * fix(KDP): make Feature MoE routing independent of the batch ([0f3ca9f](0f3ca9f)) * fix(KDP): make feature_moe_sparsity real and report expert assignments ([a84b82f](a84b82f)) * fix(KDP): make min-max scale onto the range it is given ([883ded3](883ded3)) * fix(KDP): make text output modes reachable and correct the date/text docs ([d9917f2](d9917f2)) * fix(KDP): make the documented examples run, and fix what running them exposed ([32f0e30](32f0e30)) * fix(KDP): make the saved model, the calendar feature and tsfresh work ([ac6bd85](ac6bd85)) * fix(KDP): one DistributionType, and stop publishing docs for deleted classes ([14e9bd5](14e9bd5)) * fix(KDP): pad the features add_feature_moe_to_model stacks ([67c84c3](67c84c3)) * fix(KDP): put the output columns in the order the features were declared ([7cb956f](7cb956f)) * fix(KDP): read the season from the month, not the cosine of the year ([d532c2c](d532c2c)) * fix(KDP): reject option values that silently disabled the feature they name ([820be2c](820be2c)) * fix(KDP): repair four feature-level options that could not work ([4f3e4ea](4f3e4ea)) * fix(KDP): repair generate_multi_step_forecast and cover the advisor branches ([3d29c43](3d29c43)) * fix(KDP): repair options that were accepted and never did what they said ([11085c1](11085c1)) * fix(KDP): repair silent data defects in seasons, time series options and serialization (#37) ([3e8616a](3e8616a)), closes [#37](#37) * fix(KDP): repair three time series options and the widths they report ([1fb8a30](1fb8a30)) * fix(KDP): repair week_of_year and the clock components of CalendarFeatureLayer ([3554af0](3554af0)) * fix(KDP): report an empty categorical vocabulary and read every lag channel ([83eb49a](83eb49a)) * fix(KDP): route real features through the mixture, not arbitrary slices ([faf2c15](faf2c15)) * fix(KDP): stop predefined MoE routing silently erasing unassigned features ([73a9646](73a9646)) * fix(KDP): stop tests leaking a stats file into the working directory ([2e10d69](2e10d69)) * fix(KDP): stop TextPreprocessingLayer writing a config it cannot read back ([d702b02](d702b02)) * fix(KDP): stop the generated snippet writing options the feature ignores ([f82d96a](f82d96a)) * fix(KDP): stop two layer tests racing over one file name in the working directory ([585e366](585e366)) * test(KDP): cover value fidelity and serialization, and pin a MoE dict-mode defect ([20a557e](20a557e)) * test(KDP): hold layer options to their own evidence in the dead-option guard ([f381a6f](f381a6f)) * test(KDP): make the two assertion-free tests actually assert ([baf89af](baf89af)) * test(KDP): probe the MoE reachability check with several values, not one ([972b859](972b859)) * test(KDP): split the time series tests that mixed calendar with numeric configs ([e75a84e](e75a84e)) * docs(KDP): correct the numeric and passthrough pages, and run the examples in CI ([2fa3f84](2fa3f84)) * docs(KDP): publish the API reference and guard the docs against the code ([ec335a3](ec335a3)) * docs(KDP): publish the API reference for the mixture of experts and the advisor ([60a8997](60a8997)) * docs(KDP): rewrite the categorical and distribution-aware pages against the code ([99c0a9f](99c0a9f)) * docs(KDP): say what the time series section covers ([c2cf4cc](c2cf4cc)) * docs(KDP): state the output width tabular attention produces ([9ecc4c7](9ecc4c7)) * docs(KDP): stop four examples asking one column to be a date and a number ([02d1a09](02d1a09)) * docs(KDP): unpack what load_model returns, and guard the pattern ([0f1cb93](0f1cb93)) * feat(KDP): feature importances rank features, and doc examples get called ([a935051](a935051)) * style(KDP): format the new PerFeatureDense tests ([b4602e7](b4602e7)) * maint(KDP): keep the example scripts out of the repository root ([cb57b07](cb57b07)) * refactor(KDP): remove unreachable code and cover the auto transform ([2a38f25](2a38f25))

claude added 30 commits September 2, 2026 13:47

docs(KDP): rewrite the categorical and distribution-aware pages again…

99c0a9f

…st the code Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

claude added 12 commits September 3, 2026 07:55

style(KDP): format the new PerFeatureDense tests

b4602e7

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NQmAjbXCdkBSX1CPAd56H2

claude added 3 commits September 3, 2026 08:03

piotrlaczkowski merged commit 3e8616a into main Sep 3, 2026
36 checks passed

Navigation Menu

Repository navigation

fix(KDP): repair silent data defects in seasons, time series options and serialization - #37

fix(KDP): repair silent data defects in seasons, time series options and serialization#37
piotrlaczkowski merged 45 commits into
mainfrom
claude/repo-maintenance-update-n40nk0

piotrlaczkowski commented Sep 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 3, 2026 •

edited

Loading

Built to branch `gh-pages` at 2026-09-03 08:21 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

piotrlaczkowski commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your changes shortly

Behaviour changes

Guards added

Found by CI on this PR, and fixed here

Checklist

Notes for the reviewer

Uh oh!

github-actions Bot commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Built to branch gh-pages at 2026-09-03 08:21 UTC. Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

piotrlaczkowski commented Sep 3, 2026 •

edited

Loading

github-actions Bot commented Sep 3, 2026 •

edited

Loading

Built to branch `gh-pages` at 2026-09-03 08:21 UTC.
Preview will be ready when the GitHub Pages deployment is complete.