diff --git a/CHANGELOG.md b/CHANGELOG.md index 7516173..608cfc7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,11 +2,19 @@ ## Unreleased +* The user guides (evotune, YAML format, relaxation methods, migration from GECKO MATLAB, kcat sources) moved to [gecko-docs](https://gecko-docs.readthedocs.io/en/latest/). The `docs/` folder is gone; the developer notes are in `dev/`. * `uniprot.id` and `kegg.id` are empty by default instead of the identifiers of Saccharomyces cerevisiae (`"559292"` and `"sce"`), as in GECKO; `adapterParameters.tsv` is the same as GECKO's. `geckopy init` writes both uncommented as `""`, with examples in the comments. `make_ec_model` and `calculate_f_factor` raise `FileNotFoundError` naming `[uniprot] id` when `data/uniprot.tsv` is missing and the identifier is empty, as GECKO's `loadDatabases` raises `loadDatabases:noUniprotID`; an empty `kegg.id` leaves KEGG out with a warning. A project that relied on the yeast defaults sets the identifiers in model_adapter.toml. * `upgrade_project` (`geckopy upgrade`) reports the calls of removed or replaced functions in the project's scripts with their replacement, for example `submit_open_kinetics_predictor`, `fetch_open_kinetics_predictor`, `build_okp_input_csv` and `truncate_values`. * The deprecated aliases with GECKO 3 names are removed from `geckopy`: `run_dlkcat`, `read_dlkcat_output`, `write_dlkcat_input`, `apply_custom_kcats`, `get_kcat_across_isozymes`, `get_standard_kcat`, `select_kcat_value`, `merge_dlkcat_and_fuzzy_kcats`, `get_ec_from_database`, `get_ec_from_gem`, `sigma_fitter` and `constrain_flux_data`, and `find_max_value`, which ports `findMaxValue` that GECKO 4 removed. `upgrade_project` (`geckopy upgrade`) reports each of them in a project's scripts with its replacement, which `geckopy.upgrade.RENAMED` lists. `extract_enzyme_substrate_pairs` stays public. * The `okp` source group of the presets is the OpenKineticsPredictor method KinForm-H alone by default, instead of the eight methods with combination parameters; a project that sets `[kcat_sources.okp] methods` keeps its list. `update_kcat_sources` then gathers `okp_kinform_h.tsv` only. GECKO uses the same default. -* The default of `okp.method` (the OpenKineticsPredictor method of `kcats_from_okp` without `method`) is `KinForm-H` instead of `CataPro`, as in GECKO. +* `okp.method` is removed from model_adapter.toml; `[kcat_sources.okp] methods` is the only setting that names OpenKineticsPredictor methods (default `["KinForm-H"]`). `kcats_from_okp` without `method` uses the first entry of `methods`; it used `okp.method`. `[kcat_sources.okp_] method` is removed: the method of the file `okp_` is the known method whose file name is `okp_` (`okp_` and the method in lower case, each run of other characters than letters and digits as `_`), and `update_kcat_sources` reports any other `okp_` as failed (` is not the file of a known OKP method; the files are ...`). `Unknown OKP methods in [kcat_sources.okp] methods: ...` lists the names comma-separated. `CatPred` and `TurNup` are no longer accepted in `[kcat_sources.okp] methods`: `kcats_from_okp` rejects them, so the configuration now fails early with `Unknown OKP methods`. +* Evotune source groups are one table each, `[evotune.source_groups.]` with `sources` and `sigma0_log`, both required. `[evotune.sigma0_log_source]` and `SourceGroupRule.match_okp` are removed, and so is the `okp_method` argument of `classify_kcat_source`, `classify_kcat_sources`, `screen_kcat_leverage`, `cmaes_kcat_tuning`, `sweep_tunable_mask_size` and the other tuning functions; a group covers OpenKineticsPredictor kcats with a pattern such as `okp_*`. `sigma0_log` is a standard deviation of the natural log of the kcat. A `sources` entry ending in `*` is a prefix pattern on the origin (case-insensitive); exact matches over all groups are tried first, then patterns, the longest prefix first, and groups of equal length in file order. Without `[evotune.source_groups]` the groups are `custom` 0.1, `brenda` 0.2, `ec_tree` 0.3 and `combined` 0.4 (they were `dlkcat` 0.4, `brenda` 0.2 and `custom` 0.1); groups in the file replace the defaults entirely. Tuning warns when more than 20 % of the tunable kcats are in no group: ` of tunable kcats match no evotune source group and get sigma0_log_default (); origins: `. +* `kcat_aggregate_brenda` is removed from model_adapter.toml (and `ModelParameters`); `[kcat_sources.brenda] aggregation` is the only BRENDA aggregation setting. `kcats_from_brenda` and `fuzzy_kcat_matching` without an explicit aggregation use it as the kcat assignment config resolves it: median for the default preset `brenda_ec_tree` and the other median presets, max for `gecko_classic`, and max when neither the preset nor the file sets it. They used `kcat_aggregate_brenda`, which defaulted to `"max"`, so a call without an aggregation under the default preset now takes the median. +* `kcat_aggregate_candidates` and `kcat_aggregate_isozymes` are removed from model_adapter.toml (and `ModelParameters`). `apply_kcat_list` takes `criteria="max"` and `fill_kcats_from_isozymes` takes `aggregate="mean"` as their defaults, as before; they no longer read an adapter value. +* `gr_exp` has no default (it was `0.41`, the yeast growth rate). `sensitivity_tuning`, `fit_sigma`, `flexibilize_enz_concs` and `relax_enz_concs` called without a growth rate raise `ValueError: gr_exp is not set in model_adapter.toml` when the file does not set it; `geckopy init` writes it commented out, noting that the functions that tune kcats or fit sigma require it. A project that relied on `0.41` sets `gr_exp`. +* An unknown key in model_adapter.toml is an error, whatever its place: `ModelAdapter.from_folder` raises `ValueError` with `: unknown key in model_adapter.toml`. This includes keys under `evotune.source_groups.`, the `kcat_assignment.stage` entries and `kcat_sources.`. The keys of `[kcat_sources.]` are `aggregation` and `force_wildcard_level` for `brenda`, `k` for `ec_tree`, `k`, `min_identity` and `coverage` for `homologs`, `methods` for `okp`, and none for `custom`, `dlkcat` and `okp_`; another source name is an unknown key. A malformed evotune group raises `: evotune.source_groups. must be a table with sources and sigma0_log`, `... needs both sources and sigma0_log`, `... .sources must be a list of strings` or `... .sigma0_log must be a positive number`. The removed keys are unknown keys: no alias or warning remains for them. +* `geckopy adapter show [FOLDER]` (default: the current folder) and `show_adapter_params(adapter)` print one line per item in the order of `adapterParameters.tsv`: ` = []` with the origin `file`, `default`, `preset ` or `not set` (` = [not set]`), without a header. The kcat assignment stages (`kcat_assignment.stage[]`), the `kcat_sources..` settings and the evotune source groups with their `sigma0_log` are shown as resolved. GECKO has `showAdapterParams`. +* `ModelAdapter.from_folder` checks more of model_adapter.toml when it loads it: `[kcat_sources.okp] methods` must be a list of strings and name known OKP methods (`Unknown OKP methods in [kcat_sources.okp] methods: ; methods: .`), a `[kcat_sources]` entry must be a table (`: kcat_sources. must be a table`), `[kcat_sources.brenda] aggregation` must be `max` or `median`, an evotune group's `sources` must be a non-empty list of strings, and `sigma0_log` must be finite and positive. A function given another BRENDA aggregation raises `aggregation must be max or median`. The unlabelled-kcat warning of evotune tuning is also issued as a `UserWarning`. `geckopy adapter show` prints whole-valued numbers without decimals and lists the `kcat_sources` settings by source (brenda, ec_tree, homologs, okp, then others alphabetically) and key. * `submit_open_kinetics_predictor`, `fetch_open_kinetics_predictor`, `build_okp_input_csv` and `parse_okp_output` are private (`_submit_open_kinetics_predictor` and so on) and no longer exported from `geckopy`, `geckopy.gather_kcats` or `geckopy.gather_kcats.open_kinetics_predictor`: `kcats_from_okp` and `kcats_from_dlkcat(via="okp")` submit and fetch OpenKineticsPredictor jobs. GECKO moves its OKP engine functions into `kcatsFromOKP` in the same way. `OKPClient` and `OKPError` stay public. * `convert_custom_kcats` (or `geckopy upgrade`) turns `data/customKcats.tsv` into `data/kcats/custom.tsv`. `collect_kcats` and `assign_kcats` do not apply `data/customKcats.tsv`: when a stage lists custom, `data/kcats/custom.tsv` does not exist and `data/customKcats.tsv` does, they raise `ValueError` naming `convert_custom_kcats`, before they change the model. The decisions have no stage `custom`. GECKO's `collectKcats` does the same. * `download_uniprot` and `load_uniprot_tsv` download reviewed and unreviewed entries by default (`reviewed=False`), as the adapter default `uniprot.reviewed = false`; they downloaded reviewed entries only. diff --git a/README.md b/README.md index 7f05af0..008ff7e 100644 --- a/README.md +++ b/README.md @@ -44,11 +44,11 @@ ec_model = make_ec_model(model, adapter, uniprot_db=uniprot) save_ec_model(ec_model, "ecModel.yml", adapter=adapter) ``` -The guide covers each step: [Getting started](https://gecko-docs.readthedocs.io/en/latest/guide/getting-started.html), [Building an ecModel](https://gecko-docs.readthedocs.io/en/latest/guide/building-ec-model.html) and [Gathering kcats](https://gecko-docs.readthedocs.io/en/latest/guide/gathering-kcats.html). Every public function is available as `from geckopy import X`; `geckopy.__all__` lists them, and the [API reference](https://gecko-docs.readthedocs.io/en/latest/api/) documents them. The full protocol, with kcat curation, proteomics and Crabtree-effect simulation, is [`tutorials/full_ecModel/protocol.py`](tutorials/full_ecModel/protocol.py). Fitting kcats to measured growth and fluxes is described in [`docs/evotune_kcat_tuning.md`](docs/evotune_kcat_tuning.md). +The guide covers each step: [Getting started](https://gecko-docs.readthedocs.io/en/latest/guide/getting-started.html), [Building an ecModel](https://gecko-docs.readthedocs.io/en/latest/guide/building-ec-model.html) and [Gathering kcats](https://gecko-docs.readthedocs.io/en/latest/guide/gathering-kcats.html). Every public function is available as `from geckopy import X`; `geckopy.__all__` lists them, and the [API reference](https://gecko-docs.readthedocs.io/en/latest/api/) documents them. The full protocol, with kcat curation, proteomics and Crabtree-effect simulation, is [`tutorials/full_ecModel/protocol.py`](tutorials/full_ecModel/protocol.py). Fitting kcats to measured growth and fluxes is described in [the guide](https://gecko-docs.readthedocs.io/en/latest/guide/evotune.html). ## Documentation and parity with GECKO -The protocol, installation, troubleshooting and API reference for both toolboxes are at [gecko-docs](https://gecko-docs.readthedocs.io/). The ecModel YAML format is described in the [guide](https://gecko-docs.readthedocs.io/en/latest/guide/yaml-format.html) and in [`docs/yaml_format.md`](docs/yaml_format.md). geckopy ports GECKO function by function, and each source file names its MATLAB original. The [parity ledger](https://github.com/CompMetEng/raven-gecko-parity) records, for every public function, whether the two agree and why where they do not. +The protocol, installation, troubleshooting and API reference for both toolboxes are at [gecko-docs](https://gecko-docs.readthedocs.io/). The ecModel YAML format is described in the [guide](https://gecko-docs.readthedocs.io/en/latest/guide/yaml-format.html). geckopy ports GECKO function by function, and each source file names its MATLAB original. The [parity ledger](https://github.com/CompMetEng/raven-gecko-parity) records, for every public function, whether the two agree and why where they do not. ## Cite us diff --git a/dev/README.md b/dev/README.md new file mode 100644 index 0000000..8e0c7e9 --- /dev/null +++ b/dev/README.md @@ -0,0 +1,7 @@ +# Developer notes + +These notes are for developers of geckopy. The user documentation is at [gecko-docs](https://gecko-docs.readthedocs.io/en/latest/). + +* `gecko_light_status.md`: how the full and light ecModels differ and what geckopy implements of the light build. +* `raven_integration.md`: what geckopy takes from raven-toolbox and where the boundary lies. +* `brenda_refresh_plan.md`: the plan for refreshing the bundled BRENDA snapshot. diff --git a/docs/internal/brenda_refresh_plan.md b/dev/brenda_refresh_plan.md similarity index 100% rename from docs/internal/brenda_refresh_plan.md rename to dev/brenda_refresh_plan.md diff --git a/docs/gecko_light_status.md b/dev/gecko_light_status.md similarity index 100% rename from docs/gecko_light_status.md rename to dev/gecko_light_status.md diff --git a/docs/raven_integration.md b/dev/raven_integration.md similarity index 100% rename from docs/raven_integration.md rename to dev/raven_integration.md diff --git a/docs/evotune_kcat_tuning.md b/docs/evotune_kcat_tuning.md deleted file mode 100644 index 79a66cf..0000000 --- a/docs/evotune_kcat_tuning.md +++ /dev/null @@ -1,335 +0,0 @@ -# Tuning kcats against experimental data - -A first, brief protocol for fitting an ecModel's kcats to measured -growth rates and/or fluxes with `cmaes_kcat_tuning`. This covers one -recommended path end to end; it is not a full reference (see the -docstrings in `geckopy.kcat_tuning.evotune` for every -option) and it does not build an ecModel from scratch (see -[migrating_from_gecko_matlab.md](migrating_from_gecko_matlab.md) for -that). - -## What this does - -An ecModel's kcats mostly come from databases (BRENDA), predictors -(DLKcat, OpenKineticsPredictor), or manual curation, and they don't -reproduce measured growth on their own. Three functions, used in -sequence, turn measured data into corrections: - -1. **`screen_kcat_leverage`** reports which kcats the data can - actually speak to -- no optimisation, just "if this kcat were - different, how much would the fit change." Useful for curation on - its own, before any tuning run. -2. **`select_tunable_mask`** turns that report into the set of kcats a - tuning run should search over, automatically if you skip it. -3. **`cmaes_kcat_tuning`** searches that set with CMA-ES for the kcat - vector that best matches the data, weighted by how much each kcat's - source is trusted. - -A fourth function, **`review_assignment`**, is complementary rather -than part of that sequence: it checks the kcats *themselves* for -assignment mistakes -- independent of any measured data -- since the -tuner's largest corrections are usually places the assignment went -wrong rather than discoveries about biology (see Step 4). - -The result is usually a short list of corrections, each with a -before/after value and a source -- not a wholesale rewrite of the -model. A run that changes nearly every kcat by a little has not found -anything; a run that changes a handful by a lot, concentrated in the -least-trusted sources, has. - -## Before you start - -- A working ecModel and `ModelAdapter` project (an existing - `model_adapter.toml` and `models/*.yml`). -- Experimental data: at least one of a set of measured exchange fluxes - per condition, or a set of measured maximum growth rates per carbon - source. Both together works and is preferred -- flux data pins the - *shape* of metabolism, growth rate pins its *rate*. - -## Step 1: Prepare the experimental data - -Three tab-separated files go under `/data/`. All three are -optional -- a missing file is skipped, not an error -- but you need at -least one of the first two. - -| file | contents | required | -|------|----------|----------| -| `evotuneFluxData.tsv` | measured exchange fluxes, one row per condition | at least one of these two | -| `evotuneMaxGrowth.tsv` | measured maximum growth rate, one row per condition, one active carbon source per row (at `-1000`, i.e. unconstrained) | | -| `evotuneZeroExch.tsv` | reaction IDs assumed to carry zero flux in every condition | no | - -Both flux files share one column layout (the same parser geckopy uses -for regular flux data): - -``` -Condition Ptot grRate glucose (r_1714) fructose (r_1709) ... evotuneRMSEweight source -glucose NaN 0.41 -1000 NaN ... 1 DLKcat -fructose NaN 0.338 NaN -1000 ... 1 DLKcat -``` - -`Condition` names the row, `grRate` is the measured growth rate, -`Ptot` is measured protein content (`NaN` if not measured), each -` ()` column is a measured or fixed flux for that exchange -reaction, `evotuneRMSEweight` scales how much that row counts toward -the RMSE (usually `1`), and `source` is free text kept unchanged for -your own bookkeeping. `evotuneZeroExch.tsv` is simpler -- a `Rxns` -header, then one reaction ID per line. - -## Step 2: Configure `[evotune]` in `model_adapter.toml` - -Every field has a default, so this section can be omitted entirely to -start. A configured example, in plain terms: - -```toml -[evotune] -sigma0_log_default = 0.3 # trust for any source not listed below -max_growth_weight = 2.0 # weight the growth-rate data double against flux data - -[evotune.source_groups.okp] -sources = ["OpenKineticsPredictor"] -match_okp = true - -[evotune.source_groups.brenda] -sources = ["brenda"] - -[evotune.source_groups.custom] -sources = ["custom"] - -[evotune.sigma0_log_source] -okp = 0.25 -brenda = 0.2 -custom = 0.1 -``` - -What each knob means: - -- **Trust tiers** (`source_groups` + `sigma0_log_source` + - `sigma0_log_default`). Group your kcats by how much you trust their - origin, and give each group a standard deviation in log-space: - smaller means more trusted, and the search needs stronger evidence - before it moves that kcat far. `custom` (curated by hand) at 0.1 is - three times more trusted than an unlabelled prediction at 0.3. This - is the one section with no universal default -- your model's own - source labels determine it. -- **`max_growth_weight`** (default `1.0`). The combined score is - `(rmse_flux + w * rmse_max_growth) / (w + 1)`. At `2`, growth-rate - error counts double against flux error -- useful when growth rate is - measured across many carbon sources but flux is mostly one. -- **`prior_penalty_weight`** (default `0.03`, already on). Adds - `w * mean(((log k - log k0) / sigma0)^2)` to the score: a charge for - moving a kcat, scaled by how much you trust its prior. Without this - the fit is often flat along many directions -- several very - different kcat vectors fit equally well -- and the search returns an - arbitrary point along that flat direction, which does not reproduce - if you rerun it with a different seed. The default is calibrated so - that large corrections agree in direction and land within a handful - of fold of each other across seeds, at a small cost in raw fit. Set - it to `0` to score on fit alone (also what a run reproducing GECKO - MATLAB, which has no such term, must do). The right strength is - model- and data-specific, since it depends on the scale of - `sigma0_log` and of the objective itself -- - `tune_prior_penalty_weight` runs a candidate sweep and reports fit - cost against cross-seed reproducibility so it doesn't have to be - chosen without information (see its docstring; it costs one full tuning run per - candidate per seed, so treat it as an occasional calibration step, - not part of routine tuning). -- **`tie_isozymes`** (default `true`). Several ecModel reactions are - one enzymatic reaction split across isozyme copies. When copies - share a prior value and a source, no experimental condition can tell - them apart, so an untied search is free to invent a difference - between them that means nothing biologically. Tying gives such - copies one shared kcat. -- **`max_generations` / `rmse_threshold`**. Double as this search's - stopping conditions: run at most this many CMA-ES generations, or - stop earlier once the best RMSE reaches `rmse_threshold` (negative - never stops early). - -## Step 3: Find out what the data can see - -```python -from geckopy import ModelAdapter, load_ec_model -from geckopy.kcat_tuning.evotune import screen_kcat_leverage - -adapter = ModelAdapter.from_folder("path/to/project") -model = load_ec_model(adapter=adapter) # models/ecModel.yml by default - -screen = screen_kcat_leverage(model, adapter=adapter, n_proc=8) -screen.drop(columns="_positions").head(20) -``` - -This perturbs every tunable kcat (isozyme copies moved together, per -`tie_isozymes`) up and down and measures how much the fit changes -- -no tuning happens yet. The result is a table, one row per kcat or tied -group, ranked by that leverage weighted by trust: the reaction ID, how -many isozyme copies it covers, its source group, its current value, -its leverage, and `cum_leverage_share` -- the running fraction of -total leverage accounted for by this row and every row above it. The top -rows are what a curator should look at first, independent of whether -you ever run a tuning search: on ecYeastGEM, a handful of kcats -routinely account for the majority of what any tuning run could achieve, and -they tend to be implausible database values rather than genuine -biology. - -This costs one simulation per condition per kcat group, roughly -comparable in scale to a full tuning run's budget -- expect tens of -minutes on a genome-scale model, not a quick check. `n_proc` -parallelises it the same way tuning does. - -## Step 4: Check the assignments behind the top kcats - -Before spending a tuning run on a kcat, it is worth checking whether -the value was assigned correctly in the first place. -`review_assignment` flags source-level problems: a value with no kcat -row behind it for its EC, one number reused across many unrelated -reactions, an EC's maximum standing in for a whole distribution, an -implausible magnitude, or a curated value that just repeats what the -database already said. These are corrections to the *source* -- fix -them in whatever database or fuzzy-matching step in `gather_kcats` -produced the value -- so they propagate to every model built from the -same data, unlike a value the tuner moves for this one model alone. - -```python -from geckopy.gather_kcats.review import EcStats, review_assignment, findings_tsv - -# One EcStats per EC code queried while gathering kcats -- summarise -# whatever kcat database (usually BRENDA) the assignment came from. -ec_stats = { - "1.1.1.1": EcStats(n_kcat=12, kcat_max=340.0, kcat_median=45.0, - values=frozenset({340.0, 45.0, 12.0}), n_sa=0), - # ... -} - -findings = review_assignment( - model.ec.rxns, model.ec.kcat, model.ec.source, model.ec.eccodes, - ec_stats=ec_stats, -) -print(findings_tsv(findings)) -``` - -Each `Finding` names a reaction, its checks (`no-ec-evidence`, -`repeated-value`, `ec-maximum`, `magnitude`, `custom-duplicate`), and a -human-readable `detail`. Isozyme copies the -assignment could not tell apart (the same grouping `tie_isozymes` in -Step 2 uses) are reported once, not once per copy. Standard fallbacks -and tied groups are deliberate choices, not mistakes, so both stay -quiet unless asked for via `include`. - -Findings rank by `leverage` when you pass it in -- reuse the `screen` -from Step 3, expanded from tie-groups back to individual -`model.ec.rxns` positions via its `_positions` column -- so that -looking odd and actually mattering are not confused; without it, rows -come back in input order, and looking odd is all you get. `coverage` -(a share of flagged leverage) or `top` (a row count) then truncates the -report: on ecYeastGEM, tightening the checks themselves does not work -half as well as this does, since `coverage=0.8` cuts 2575 unfiltered -rows to 33, and the first row alone accounts for half. - -## Step 5: Tune - -```python -from geckopy import save_ec_model -from geckopy.kcat_tuning.evotune import cmaes_kcat_tuning - -result = cmaes_kcat_tuning( - model, adapter=adapter, screen=screen, n_proc=8, seed=0, -) - -save_ec_model(model, "ecModel_tuned.yml", adapter=adapter) -``` - -Passing the screen from Step 3 avoids recomputing it; omit `screen` -entirely and `cmaes_kcat_tuning` runs one itself. Either way, the -tunable set is built by `select_tunable_mask`, which keeps the fewest -highest-ranked kcats whose combined leverage reaches -`target_impact_share` (default `0.9`) of the total -- a cutoff -relative to the model's own achievable improvement, not a fixed count -or an absolute leverage value, so the same setting means a comparable -thing on a different model. Pass `tunable_mask` directly instead if -you want to fix the set yourself. - -`model` is mutated in place: on return, every tunable kcat has the -best value CMA-ES found, and the model's kcat constraints are already -applied -- nothing further is needed before simulating or saving it. -Beyond `target_impact_share`, the only other knobs are `popsize` -(CMA-ES's population size, defaulting to its own dimension-scaled -choice rather than a value tuned for one particular model), `n_proc`, -and `seed`; everything else comes from the same `EvotuneParams` as -Step 2, so there is nothing new to configure once you've read it. - -If growth needs special handling for your organism -- forcing -anaerobic conditions, or scaling the protein pool with a -condition-specific biomass composition -- pass `make_anaerobic` and/or -`change_protein_biomass` callables; geckopy has no organism-agnostic -default for either. See `tutorials/full_ecModel/code/anaerobic.py` in -this repository for a worked example. - -## Step 6: Read the result - -`result` is an `EvotuneResult`: - -- `rmse_trace` / `objective_trace` -- best-so-far plain-fit RMSE and - best-so-far optimised objective, per generation (identical unless - `prior_penalty_weight` is nonzero). A flat trace for many generations - before the run ends means it stalled, not converged. -- `new_kcat` / `old_kcat` / `rxns` / `groups` -- the tuned and prior - kcats over the searched set, parallel arrays, with each kcat's trust - tier. Tied isozyme copies share one value in `new_kcat`. -- `converged` -- whether `rmse_threshold` was reached, or - `max_generations` was hit first. -- `n_generations` -- how many generations actually ran. - -Before trusting a tuned kcat, check more than the final RMSE: - -1. **How many kcats changed**, and by how much -- a result that moves - nearly everything by a little has not identified anything. - `geckopy.kcat_tuning.evotune.parsimony` has the - tools for this: `n_changed`, `fold_change`, `source_movement`. -2. **Impact share** -- `parsimony.impact_share` reports what fraction - of the total achievable improvement the changed kcats actually - account for, distinguishing "few, large, and consequential" from mere - sparsity. This is a different, unweighted quantity from - `screen_kcat_leverage`'s trust-weighted `cum_leverage_share`. -3. **Reproducibility.** Rerun with a different `seed`. Large - corrections (say, beyond two-fold) should land in the same - direction and within a small factor of each other; if they don't, - the search has found a flat direction rather than a real - correction, and `prior_penalty_weight` is the first knob to reach - for. -4. **Whether it generalises**, if you have data you didn't fit on -- - score the tuned model against it and compare to the untuned model. - A large improvement on fitted conditions with no improvement - elsewhere is a sign of overfitting to a thin dataset, not a fixed - model. - -`new_kcat`/`old_kcat` is thousands of numbers, most of them unchanged -- -not something a reviewer can read. `result.corrections()` turns it into -the actual review artifact: one `Correction` row per changed kcat -(reaction, EC code, source, prior/tuned value, fold change), ranked by -*leverage* rather than by how far a kcat moved, so the parameters the -data cannot see (free to drift furthest, least worth trusting) sort to -the bottom instead of the top: - -```python -from geckopy.kcat_tuning import annotate_from_model, corrections_tsv - -# screen is Step 3's table -- reuse it rather than recomputing leverage. -leverage_by_rxn = dict(zip(screen["rxn_id"], screen["leverage"])) -leverage = [leverage_by_rxn.get(r, 0.0) for r in result.rxns] - -annotations = annotate_from_model(model, result.rxns) -rows = result.corrections( - leverage=leverage, - names=annotations["names"], ec_codes=annotations["ec_codes"], -) -open("corrections.tsv", "w").write(corrections_tsv(rows)) -``` - -`annotate_from_model` also returns `sources`, which `corrections()` -takes from `result.groups` itself rather than as a keyword. - -`cumulative_share` on each row is the running fraction of total leverage -the list has accounted for so far, so a reader can stop where it stops -climbing rather than reading every row. This is the tuning-time -counterpart to `review_assignment` in Step 4: that flags kcats worth -curating *before* a run; this reports which ones the run actually -changed, and how much each change is worth trusting. diff --git a/docs/future_improvements.md b/docs/future_improvements.md deleted file mode 100644 index 716c7dd..0000000 --- a/docs/future_improvements.md +++ /dev/null @@ -1,152 +0,0 @@ -# Future improvements - -A running list of ideas the port surfaced that are worth doing but -weren't blocking. Three audiences: - -- **geckopy users / contributors** — sections "Conventions / data - model", "Third-party library gotchas", and "Deferred migrations". -- **MATLAB GECKO maintainers** — section "MATLAB GECKO changes". - Each item is a bug or rough edge in MATLAB GECKO that the port - spotted while comparing the two implementations side by side. For - the current, maintained list of MATLAB ↔ Python behavioural - differences (broader than just what's tracked here), see the - [GECKO vs. geckopy](https://gecko-docs.readthedocs.io/en/latest/gecko-to-geckopy.html) page - on [gecko-docs.readthedocs.io](https://gecko-docs.readthedocs.io/). -- **anyone hitting a strange dependency bug** — section - "Third-party library gotchas". - -## Conventions / data model - -One cross-cutting decision where geckopy and MATLAB GECKO are -still misaligned. Not blocking, but worth doing eventually. - -- **Custom subunit stoichiometry file format.** The `stoicho` - column in `customKcats.tsv` was meant to let users override - subunit counts manually. The MATLAB code never actually reads - it; geckopy also ignores it. Two options if we want to support - it properly: a separate `customComplexes.tsv` file, or a new - "user_overrides" section in the ComplexPortal JSON. Deferred - until someone needs it. - -## Third-party library gotchas - -A scratchpad of brittle behaviours we ran into in geckopy's hard -dependencies. Each is worked around in source today; the entries -exist so the next person hitting the same wall can find the fix -fast, and so we have a list of things to retest when we upgrade. - -- **`multiprocessing.Pool(context="spawn")` deadlocks on some WSL - kernels.** Even a trivial 2-worker pool with a `lambda x: x*2` - task hangs forever — no error, no timeout. The original plan for - parallel `ec_fva` called for `spawn` because it's the only context - available on Windows. Workaround: select the context based on the - platform — `fork` on POSIX (which is also faster, no module - re-import), `spawn` on Windows. See `geckopy.utilities.ec_fva.ec_fva`. - -- **`ruamel.yaml`'s `ScalarInt` / `ScalarFloat` leak out of cobra - YAML loads.** After loading a YAML model with cobra, attributes - like `Metabolite.charge` are ruamel scalar types (subclasses of - int/float that have YAML formatting metadata). The ruamel - safe-dumper has no representer for them, so writing the model - back out fails with `RepresenterError: cannot represent an - object: 0`. Workaround: walk the assembled document and coerce - everything to plain Python primitives just before writing. See - `geckopy.utilities.save_ec_model._to_native`. This affects any - write of a cobra-loaded model back to YAML. - -## Deferred migrations - -Pieces of geckopy that have a clear destination (upstream package -or a different in-tree shape) but whose migration was deferred to -keep an in-flight release on schedule. - -- **`brenda/parse.py` range collapse.** A BRENDA value reported as - a range (e.g. `"0.1-2.5"`) is collapsed to its upper bound - unconditionally in `parse_brenda_json`. After the kcat-aggregation - refactor moved the rest of the pipeline to be configurable - (`kcat_aggregate_brenda = "max" | "median"`), this is the one - residual max-leaning step. Honouring the adapter setting here - would make every layer consistent. Small effect (only rows - reported as ranges) — deferred. - -## MATLAB GECKO changes - -Aimed at MATLAB GECKO maintainers. Each item is something the -geckopy port found while comparing the two implementations: bugs, -dead code, unit-comment slip-ups, or unclear behaviour. - -Notable items: - -- Implement gene-cell splitting in the UniProt loader. -- Apply the `stoicho` column from `customKcats.tsv`, or drop it from - the schema. -- Rename `getKcatAcrossIsozymes` to `fillKcatsFromIsozymes` — the - verb `get` is misleading, the function mutates `ec.kcat` in place. - geckopy renamed its Python counterpart; the MATLAB side is the - remaining half of the coordinated rename. -- Clean up the `applyCustomKcats` Mode B docstring (proteins only, - no rxns) — its docstring and code disagree on whether the - full-match gate fires. Either clarify the docstring or drop the - mode. -- Delete `getECstring` from MATLAB GECKO. After the `findECInDB` - refactor (geckopy works with raw `;`-joined EC tokens throughout) - the function has no remaining callers. It also has three latent - bugs not worth fixing in place: an accumulator footgun (callers - must add a trailing space to `EC_set` before calling, otherwise - tokens collide), an empty-input quirk (returns `"EC"` instead of - `""` because `strsplit("", " ")` returns `{''}`), and no - validation (would happily emit `ECEC1.1.1.1` for already-prefixed - input or `ECnotanec` for junk). -- Fix the validation regex in `getECfromGEM`. The current pattern - `(\d\.(\w|-)+\.(\w|-)+\.(\w|-)+)(;\w+\.(\w|-)+\.(\w|-)+\.(\w|-)+)*(.*)` - substituted with `$3` returns the level-3 character of the first EC - for any valid input (e.g. `"1.2.3.4"` -> `"3"`), and the subsequent - `~cellfun(@isempty, ...)` then flags every non-empty result as - invalid. As written, every non-empty EC string is silently - discarded. Replace with a straightforward - `^TOKEN(;TOKEN)*$` validation where `TOKEN` is the canonical - four-level dotted EC pattern with `-` allowed in any level. -- Dedupe the `intersection` helper output in `findECInDB`. The - helper currently produces duplicates when multiple subunits share - the same EC (e.g. `"1.1.1.1;1.1.1.1"` for a two-subunit complex - whose subunits both map to EC 1.1.1.1). Apply a final - `compare_wild` pass on the intersection result, mirroring what - geckopy does. -- Simplify `loadBRENDAdata` signature: drop the `modelAdapter` arg - and `ModelAdapterManager.getDefault()` fallback; take the folder - path directly. The adapter resolution belongs at the call site. -- Resolve the `'add'` action in `getECfromDatabase`: either implement - the commented-out `addMultipleMatches` branch (and drop the - apologetic `% I don't understand the purpose of this, let's skip - it for now` comment) or remove the option from the documented API. - geckopy currently only supports `'display'` and `'ignore'`. -- Subset by `ecRxns` upfront in `getECfromDatabase` rather than - iterating every reaction and discarding the unwanted results at - the end. The function's own inline comment already calls this out - ("Probably faster to subset with ecRxns in the beginning of the - script, but this was at the moment simpler to implement"). - geckopy already does this. -- Fix the search-order/output-ranking inconsistency in - `fuzzyKcatMatching`. The search order inside `mainMatch` tries - org-SA (output rank 5) BEFORE any-organism-no-substrate-kcat - (output rank 4). When both are available for a reaction, MATLAB - returns the org-SA result and reports its origin as 5, even though - the docstring's origin ranking implies origin 4 should be kept. Either - swap the search order so any-no-subs-kcat is tried before org-SA, - or update the docstring to match the search order. geckopy - replicates the current MATLAB behavior, both within one EC number - and when selecting among the EC numbers of one reaction; see - `SEARCH_LEVELS` and `SEARCH_RANK` in - `fuzzy_kcat_matching/_brenda_query.py`. -- Delete `updateProtPool` from MATLAB GECKO. The function has been - obsolete since GECKO 3.2.0 (all enzymes, measured and unmeasured, - draw from the protein pool); its sole runtime behaviour on a - 3.2.0+ model is to raise an error pointing users to - `setProtPoolSize`. geckopy does not port it; the recommended - replacement is `set_prot_pool_size` in `protein_pool.py`. -- Make `calculateFfactor` raise (or return NaN with a warning) - when no proteome data is provided. Silently returning 0.5 hides - the absence of real data; downstream calculations using this f - factor look fine until they don't. geckopy's split design - (`load_pax_db` raises FileNotFoundError, `calculate_f_factor` - requires pre-loaded data) makes the missing-data case explicit. diff --git a/docs/kcat_aggregation.md b/docs/kcat_aggregation.md deleted file mode 100644 index a28eff1..0000000 --- a/docs/kcat_aggregation.md +++ /dev/null @@ -1,162 +0,0 @@ -# kcat aggregation: how far is `max` from `median` on BRENDA? - -The BRENDA files (toolbox-assets release `brenda-2026.1-1`) contain **both** the per-(ec, substrate, -organism) max and the per-triple median of all raw measurements that -fell into the triple. They live in the same `kcat.tsv` / `sa.tsv` file -distinguished by an `aggregation` column; `load_brenda_data` splits -them into `BrendaData.kcat_max` / `.kcat_median` (and `.sa_*`) so -consumers never see doubled-up rows. The default -(`kcat_aggregate_brenda = "max"` on `ModelParameters`) preserves -MATLAB-GECKO behaviour; flipping that field to `"median"` (or passing -`fuzzy_kcat_matching(aggregate="median")` per call) makes both -aggregation layers — the per-triple snapshot view and the per-EC -runtime collapse — use median together. - -This document quantifies the two layers separately by walking the raw -BRENDA JSON (BRENDA release 2026.1, ~57 900 raw kcat rows, ~39 000 -unique triples after dropping trypsin 3.4.21.4 and the -diffusion-limited filter in `parse_brenda_json`): - -- [`scripts/kcat_aggregation_raw.py`](../scripts/kcat_aggregation_raw.py) - walks the raw JSON and answers both questions below. -- [`scripts/kcat_aggregation_compare.py`](../scripts/kcat_aggregation_compare.py) - reads only the BRENDA TSV (no 710 MB JSON needed) and compares - per-EC max-of-max vs median-of-median over the pre-aggregated views. - -> **3.4.21.4 (trypsin) is excluded** from these analyses: its BRENDA -> entries are known to be unreliable (many non-physiological mutants -> and unrelated substrates). - -## Layer 1 — within-triple aggregation - -When several raw measurements share an (ec, substrate, organism) -triple, the snapshot has to collapse them. The historical `max`-only -snapshot threw the rest away. - -24.5 % of triples (9 549 of 39 053) have ≥ 2 raw measurements. -Across those: - -| | quartiles | mean | p90 | -|---|---:|---:|---:| -| within-triple `log10(max / median)` | 0.05 / 0.16 / 0.28 | **0.26** | 0.58 | - -So within a triple the max is typically ~1.8× above the median; -4.8 % of multi-measurement triples have max ≥ 10× the median, 1.2 % -have max ≥ 100×. Small effect — *within* a triple BRENDA measurements -are usually close enough that picking max vs median doesn't move the -needle much. - -## Layer 2 — per-EC aggregation - -For each EC code with ≥ 5 triples (1 722 EC codes), we collapse -triples to a single per-EC kcat. There are four possible -"snapshot × runtime" combinations: - -| snapshot view | runtime aggregate | shorthand | meaning | -|---|---|---|---| -| max | max | **maxmax** | historical default (MATLAB GECKO) | -| max | median | **maxmed** | the runtime-only median flag we had before | -| median | median | **medmed** | adapter `kcat_aggregate_brenda="median"` | -| median | max | medmax | unused; included for symmetry | - -`log10` spreads: - -| | quartiles | mean | p90 | -|---|---:|---:|---:| -| `maxmax / medmed` (true historical bias) | 0.68 / 1.20 / 1.97 | **1.45** | 2.81 | -| `maxmed / medmed` (snapshot-only swap) | 0.00 / 0.00 / 0.07 | 0.06 | 0.20 | -| `maxmax / maxmed` (runtime-only swap) | 0.63 / 1.14 / 1.90 | 1.38 | 2.77 | - -Fraction of ECs where the historical default is N× above the "true" -median-of-medians: - -| | fraction of ECs | -|---|---:| -| `maxmax ≥ 10× medmed` | **59.5 %** | -| `maxmax ≥ 100× medmed` | 24.2 % | -| `maxmax ≥ 1 000× medmed` | 8.6 % | - -For 60 % of well-measured ECs, the included default kcat is ≥ 10× the -typical BRENDA assay; for 8.6 % it is ≥ 1 000× above. Most of that -comes from the runtime aggregation layer (`maxmax / maxmed` ≈ 1.38), -with a small further bias from the snapshot layer (`maxmed / medmed` -≈ 0.06). Both swap together when the user sets -`kcat_aggregate_brenda = "median"`. - -| EC | n | max [1/s] | median [1/s] | max / median | -|---:|---:|---:|---:|---:| -| 3.5.2.6 | 601 | 7 150 | 20 | 358× | -| 3.1.1.1 | 355 | 431 000 | 29 | **14 862×** | -| 1.1.1.1 | 318 | 3 500 | 3.1 | 1 129× | -| 3.2.1.21 | 289 | 1 214 290 | 14 | **86 735×** | -| 1.10.3.2 | 269 | 158 300 | 42.1 | 3 760× | -| 3.4.21.4 | 265 | 910 000 | 5.1 | **178 431×** | -| 2.3.2.5 | 264 | 220 | 22.95 | 9.6× | -| 3.1.1.73 | 235 | 14 793 | 10 | 1 479× | -| 3.4.21.83 | 213 | 11 300 | 18 | 628× | -| 1.1.99.18 | 203 | 156 | 17 | 9.2× | -| 1.14.18.1 | 181 | 219 000 | 26.7 | 8 202× | -| 4.1.1.39 | 168 | 23.1 | 4.2 | 5.5× | -| 3.4.21.35 | 165 | 1 380 | 0.81 | 1 704× | -| 2.5.1.18 | 161 | 9 600 | 9 | 1 067× | -| 3.4.24.15 | 156 | 179 | 4.75 | 38× | -| 3.2.1.20 | 139 | 1 278 | 26.2 | 49× | -| 3.1.8.1 | 133 | 30 900 | 11.72 | 2 637× | -| 3.1.2.20 | 128 | 190 | 2.15 | 88× | -| 3.4.24.16 | 122 | 11.9 | 1.55 | 7.7× | -| 3.4.11.1 | 112 | 34 500 | 2.94 | **11 735×** | - -| EC | n_triples | maxmax (default) | maxmed | medmed | maxmax / medmed | -|---:|---:|---:|---:|---:|---:| -| 3.5.2.6 | 601 | 7 150 | 20.0 | 15.0 | 477× | -| 3.1.1.1 (esterase) | 355 | 431 000 | 29.0 | **24.0** | **17 958×** | -| 1.1.1.1 (alcohol DH) | 318 | 3 500 | 3.1 | 2.83 | 1 237× | -| 3.2.1.21 | 289 | 1 214 290 | 14.0 | 13.9 | **87 359×** | -| 1.10.3.2 | 269 | 158 300 | 42.1 | 38.0 | 3 760× | -| 2.3.2.5 | 264 | 220 | 22.95 | 22.0 | 10× | -| 3.1.1.73 | 235 | 14 793 | 10.0 | 8.6 | 1 720× | -| 3.4.21.83 | 213 | 11 300 | 18.0 | 17.5 | 646× | -| 1.1.99.18 | 203 | 156 | 17.0 | 13.9 | 11× | -| 1.14.18.1 | 181 | 219 000 | 26.7 | 26.7 | 8 202× | -| 4.1.1.39 (RuBisCO) | 168 | 23.1 | 4.2 | 3.1 | 8× | -| 3.4.21.35 | 165 | 1 380 | 0.81 | 0.81 | 1 704× | -| 2.5.1.18 | 161 | 9 600 | 9.0 | 6.74 | 1 424× | -| 3.4.24.15 | 156 | 179 | 4.75 | 4.0 | 45× | -| 3.2.1.20 | 139 | 1 278 | 26.2 | 25.48 | 50× | -| 3.1.8.1 | 133 | 30 900 | 11.72 | 8.52 | 3 627× | -| 3.1.2.20 | 128 | 190 | 2.15 | 1.68 | 113× | -| 3.4.24.16 | 122 | 11.9 | 1.55 | 1.35 | 9× | -| 3.4.11.1 | 112 | 34 500 | 2.94 | 2.44 | **14 139×** | -| 3.2.1.1 (α-amylase) | 112 | 40 000 | 167.25 | 149.0 | 268× | - -Esterase 3.1.1.1 stays the clearest worry: 355 triples, true median -24/s, MATLAB-default 431 000/s (~18 000× above). - -## File format on disk - -``` -# BRENDA release 2026.1 generated 2026-05-28 - CC BY 4.0 - kcat in 1/s -ec_code substrate organism kcat_max kcat_median n references -1.1.1.1 (2e)-but-2-en-1-ol yokenella sp. 101.0 101.0 1 PMID:24509923 -1.1.1.1 (r)-1-indanol sulfolobus acidocaldarius 7.1 7.1 1 PMID:20049620 -... -``` - -One **wide** row per (ec, substrate, organism) triple. `kcat_max` and -`kcat_median` contain the two statistics computed from the raw -measurements that fell into the triple. `n` is the count of raw -measurements aggregated (so `n=1` rows have max == median, as above). -`sa.tsv` has the same shape with `sa_max` / `sa_median` columns. -`mw.tsv` keeps a single-valued shape (MW is a per-protein physical -property; no aggregation question) — header -`ec_code\tsubstrate\torganism\tmw\tn\treferences`. - -## Reproduce - -```bash -python scripts/kcat_aggregation_raw.py # raw-JSON, both layers -python scripts/kcat_aggregation_compare.py # snapshot-only, per-EC -``` - -Raw-JSON takes ~6 min on the BRENDA 2026.1 dump (limited by 710 MB JSON -load via WSL2 /mnt/c); snapshot-only runs in ~10 s. diff --git a/docs/migrating_from_gecko_matlab.md b/docs/migrating_from_gecko_matlab.md deleted file mode 100644 index 1757d59..0000000 --- a/docs/migrating_from_gecko_matlab.md +++ /dev/null @@ -1,496 +0,0 @@ -# Migrating from GECKO 4 (MATLAB) to geckopy - -geckopy is the Python port of the GECKO toolbox. It builds the same -enzyme-constrained models, follows the same algorithms, and writes the -same on-disk format as **GECKO 4** (MATLAB). This guide is for someone -who knows GECKO 4 in MATLAB and wants to do the same work in Python. - -The port's goal is **behavioural and on-disk fidelity**: same reactions, -metabolites, coefficients, kcats, YAML output. Differences come from -being idiomatic Python and from using cobrapy instead of RAVEN. None of -them require you to change your model files. - -> **Coming from GECKO 3?** GECKO 4 introduced a handful of changes -> (forward-direction protein reactions, several bug fixes, a reformatted -> BRENDA snapshot, the REST-based OpenKineticsPredictor flow) that -> geckopy also follows — see -> [GECKO 3 to GECKO 4](https://gecko-docs.readthedocs.io/en/latest/gecko3-to-gecko4.html) -> for the full list. Throughout this guide, individual functions that -> are affected by those 3→4 changes have a **`[3→4]`** marker; that -> marker is geckopy's signal that the function's behaviour or schema -> differs from what GECKO 3 did, not a Python-vs-MATLAB difference. - ---- - -## 1. At a glance - -| | GECKO 4 (MATLAB) | geckopy | -| ------------------------------- | -------------------------------------- | ------------------------------------------------------- | -| Model toolbox | RAVEN + COBRA Toolbox | cobrapy + raven-toolbox | -| Model object | RAVEN struct + `ec` field | `EcModel(cobra.Model)` with an `.ec` dataclass | -| Per-entity classes | none — parallel cell arrays | `cobra.{Metabolite,Reaction,Gene}` instances | -| Per-enzyme accessor | indexing into `model.ec.enzymes{i}` | `Enzyme` proxy: `model.enzymes.get_by_id("P00350")` | -| Adapter | `ModelAdapter` classdef + `ModelAdapterManager` | `ModelAdapter` + `model_adapter.toml` (pydantic-validated) | -| Default adapter | global `ModelAdapterManager.getDefault()` | none; pass `adapter=` or set `model.adapter` | -| Return style | `[a,b,c] = f(...)` | single return; model mutated in place | -| Naming | `camelCase` | `snake_case` | -| Indexing | 1-based | 0-based | -| Missing data | often a silent default | usually a raised exception | -| Tabular output | cell arrays / structs | `pandas.DataFrame` | -| HTTP (BRENDA/KEGG/UniProt/OKP) | `webread`/`webwrite` | `requests` | -| On-disk format | RAVEN/cobrapy YAML (+ SBML) | RAVEN/cobrapy YAML (SBML ecModel I/O dropped in `0.1.0a2`) | - -The two toolboxes exchange ecModels directly: a model saved by either -side loads in the other (see [§7](#7-file-exchange)). - ---- - -## 2. The model object - -GECKO MATLAB uses a RAVEN model struct with parallel cell arrays plus a -`model.ec` struct and side cells (`model.metSmiles`, `model.eccodes`). -The struct itself isn't a class — everything is an array, indexed by -position. - -geckopy's `EcModel` **subclasses** `cobra.Model`. cobrapy is -class-based: `cobra.Metabolite`, `cobra.Reaction`, `cobra.Gene` are -real classes, so `model.metabolites[17]` returns a `Metabolite` -instance (with `.id`, `.name`, `.formula`, `.annotation`, …), not a -string. - -For the GECKO-specific data, geckopy keeps the **parallel-arrays -layout** rather than a cobrapy-style per-enzyme class, because the -algorithms (`apply_kcat_constraints`, sensitivity tuning) are bulk -vector operations on `kcat` / `mw` and the sparse coupling matrix — -those are several orders of magnitude faster as numpy arrays than as a -loop over N enzyme objects. The layout is in a dataclass on -`model.ec`: - -```python -model.ec.rxns # list[str] parallel to ec.kcat etc. -model.ec.kcat # np.ndarray (per-rxn) — 0 means "unset" -model.ec.source # list[str] per-rxn provenance tag -model.ec.notes # list[str] per-rxn free-text -model.ec.eccodes # list[str] ';' -joined per-rxn -model.ec.enzymes # list[str] per-enzyme accessions -model.ec.genes # list[str] per-enzyme gene name -model.ec.mw # np.ndarray per-enzyme; NaN == "unknown" -model.ec.sequence # list[str] per-enzyme -model.ec.concs # np.ndarray per-enzyme proteomics; NaN == "not measured" -model.ec.rxn_enz_mat # scipy.sparse.csr_matrix shape (n_rxns, n_enzymes), subunit counts -model.ec.gecko_light # bool -``` - -For ergonomic per-enzyme access, geckopy adds a separate **proxy -class** `Enzyme` reached via `model.enzymes.get_by_id("P00350")`. The -proxy holds no state of its own; it forwards reads/writes into the -`model.ec.*` arrays. This gives you cobrapy-style code -(`enz.kcats["R_FOO"] = 30.0`) without paying for N real objects. See -the per-table explanation in §5 for which entry-points read it. - -The `EcData` dataclass itself is owned by **raven-toolbox** -(`raven_toolbox.io.EcData`); geckopy re-exports it as `geckopy.EcData` -so existing imports keep working. Mirrors MATLAB RAVEN's split: -`model.ec` is RAVEN-owned; GECKO is just a consumer. - -Other side-cells move into cobrapy's `annotation` dict: - -| MATLAB | geckopy | -| ----------------------------------------------------------------- | --------------------------------------------------------------- | -| `model.metSmiles{i}` | `metabolite.annotation['smiles']` | -| `model.eccodes{i}` | `reaction.annotation['ec-code']` (`list[str]`) | -| `model.ec.rxnEnzMat` | `model.ec.rxn_enz_mat` (scipy sparse) | -| `model.ec.kcat`, `.mw`, `.sequence`, `.source`, `.notes`, `.concs` | `model.ec.kcat`, `.mw`, `.sequence`, `.source`, `.notes`, `.concs` | -| `model.grRules` (strings) | cobrapy `GPR` objects (parsed AST) | - -Loaders return dataclasses / DataFrames rather than structs: e.g. -`BrendaData`, `ProtData`, `FluxData`, `UniprotDB`, `PhylDist`. Helpers -that produce tables (`fuzzyKcatMatching`, `ecFVA`, -`getReactionsFromEnzyme`, `enzymeUsage`, `reportEnzymeUsage`, -`mapRxnsToConv`) return `pandas.DataFrame`. - ---- - -## 3. Adapters and configuration - -In MATLAB you write an adapter `classdef` and register a default with -`ModelAdapterManager.setDefault`. In geckopy you create an adapter -folder with a `model_adapter.toml`, loaded via -`ModelAdapter.from_folder(path)`. The TOML is parsed into pydantic -models (`ModelParameters` with nested `kegg`, `uniprot`, `okp`, -`evotune`, …) and validated strictly, so config typos are -caught early. - -There is **no global default adapter**. Functions read `model.adapter` -or take an explicit `adapter=` argument. Where a MATLAB function takes -a `modelAdapter` only to derive a path, the geckopy version takes the -path (or pre-loaded data) directly. The pattern is: - -```python -adapter = ModelAdapter.from_folder("my_project") # has params.path, params.bio_rxn, ... -model = make_ec_model(conv_gem, adapter) # adapter stored as model.adapter -# downstream functions read model.adapter implicitly: -apply_complex_data(model) -# ...or take an explicit override: -apply_complex_data(model, path=Path("/custom/complex.json")) -``` - -**Secrets** (e.g. an OpenKineticsPredictor API key) are never stored -in the adapter. Provide them via a function argument, an environment -variable (`OKP_API_KEY`), or a file in the project -`data/` folder (`data/okpApiKey.txt`); `geckopy init` writes a -`.gitignore` that lists it. - ---- - -## 4. Calling conventions - -### Mutate in place, return one thing - -MATLAB functions return several outputs including a copy of the model. -geckopy mutates the model in place and returns at most one value; -diagnostics that MATLAB returned as extra outputs go to the logger and -to `rxn.notes['geckopy_warning']`. - -| MATLAB | geckopy | -| ---------------------------------------------------------- | --------------------------------------------------------------------------------------- | -| `[model, noUniprot] = makeEcModel(...)` | `make_ec_model(...)` mutates; logs unmatched genes, annotates `rxn.notes['geckopy_warning']` | -| `[model, ...] = getStandardKcat(...)` | `assign_standard_kcat(...)` mutates; diagnostics via logging | -| `[minFlux, maxFlux] = ecFVA(...)` | `ec_fva(...)` returns a `DataFrame` indexed by reaction id | -| `[rxns, kcat, name, gpr, idx] = getReactionsFromEnzyme(...)` | `get_reactions_from_enzyme(...)` returns one `DataFrame` | -| `[model, kcatList] = sensitivityTuning(...)` | `sensitivity_tuning(...)` mutates; returns a `TunedKcatsResult` dataclass | -| `[mappedFlux, enzUsageFlux, usageEnz] = mapRxnsToConv(...)` | `map_rxns_to_conv(...)` returns one `MapRxnsResult` dataclass | - -### Errors instead of silent defaults - -Where MATLAB silently substitutes a default, geckopy tends to raise so -problems surface early — e.g. missing proteome data raises -`FileNotFoundError` instead of returning an f-factor of `0.5`; an -unknown protein id raises `ValueError` instead of returning empty. - -### Names and indexing - -`camelCase` → `snake_case` throughout. **Most renames are mechanical** -(`makeEcModel → make_ec_model`); a handful are intentional re-naming -for clarity: - -| MATLAB | geckopy | -| ----------------------- | ------------------------------ | -| `selectKcatValue` | `apply_kcat_list` | -| `getKcatAcrossIsozymes` | `fill_kcats_from_isozymes` | -| `getStandardKcat` | `assign_standard_kcat` | -| `sigmaFitter` | `fit_sigma` | -| `getECfromDatabase` | `fill_eccodes_from_database` | -| `getECfromGEM` | `fill_eccodes_from_gem` | -| `constrainFluxData` | `apply_flux_data_constraints` | -| `mergeDLKcatAndFuzzyKcats` | `merge_kcats` (generalised) | - -geckopy has no alias with a GECKO 3 name. `geckopy upgrade` reports the -calls of the old names in a project's scripts with their replacement. - -MATLAB is 1-indexed; geckopy is 0-indexed, which shows up in arguments -like `data_col` (default `0`) and `condition` (in `apply_flux_data_constraints`). - -### gecko-light - -Implemented end-to-end. `make_ec_model(model, adapter, gecko_light=True)` -produces a working light ecModel; `apply_kcat_constraints`, -`set_kcat_for_reactions`, and the `Enzyme` proxy's read paths all -work on the light layout. The per-enzyme prot-pool analyses (proteomics -integration via `constrain_enz_concs`, `flexibilize_enz_concs`, -`pfba_enzymes`) and the isozyme-aggregating helpers -(`fill_kcats_from_isozymes`) raise `NotImplementedError` on a light -model with a clear message — they have no meaning in the light layout -by design. See [gecko_light_status.md](gecko_light_status.md) for the -full availability table and a worked example. - ---- - -## 5. Function correspondence - -Each row maps a MATLAB function to its geckopy equivalent. The -**Notes** column flags - -- **`[3→4]`** — function affected by GECKO 3→4 changes (direction - flip, bug fix, schema change). Behaviour matches GECKO 4, not - GECKO 3. -- **`[Py]`** — geckopy-specific divergence from MATLAB GECKO 4 (a - port choice, not something flowing back to MATLAB). -- **(new)** — geckopy entry-point with no direct MATLAB counterpart. - -### 5.1 Build / edit the ecModel (`src/change_model`) - -| MATLAB | geckopy | Notes | -| ---------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `makeEcModel` | `make_ec_model` | **`[3→4]`** ec.genes/enzymes/mw/sequence are sorted alphabetically for stable order; KEGG fallback for unmatched genes uses `data/kegg.tsv`, downloaded when missing (an empty `kegg.id` leaves KEGG out), or `kegg_db=`. **`[Py]`** Single return; unmatched genes are logged and annotated on `rxn.notes['geckopy_warning']` instead of being returned. | -| `applyComplexData` | `apply_complex_data` | — | -| `applyCustomKcats` | *(none)* | **`[3→4]`** Deprecated in MATLAB: `convertCustomKcats` / `convert_custom_kcats` turn `data/customKcats.tsv` into `data/kcats/custom.tsv`, which `collectKcats` / `collect_kcats` read; they raise an error when a stage lists custom and only `data/customKcats.tsv` exists. **`[Py]`** No counterpart; use `convert_custom_kcats` to move the file. | -| `applyKcatConstraints` | `apply_kcat_constraints` | **`[3→4]`** Writes coefficients with the forward-direction `usage_prot_*` / `prot_pool_exchange` convention. | -| `findMetSmiles` | `find_met_smiles` | **`[Py]`** Asynchronous PubChem cache via `requests`; uses a TSV cache (`data/smilesDB.tsv`) so re-runs are offline. | -| `getComplexData` | `get_complex_data` | — | -| `getKcatAcrossIsozymes` | `fill_kcats_from_isozymes` | **`[Py]`** Renamed. Aggregation strategy (`max` / `median` / `mean`) is adapter-configurable via `kcat_aggregate_isozymes`. | -| `getReactionsFromEnzyme` | `get_reactions_from_enzyme` | **`[Py]`** Returns one `DataFrame` instead of five parallel cells. | -| `setKcatForReactions` | `set_kcat_for_reactions` | Matches IDs without regard to letter case, ignores an unknown ID and does not apply the constraints unless `apply=True`, as `setKcatForReactions`; a light-model row is addressed by its `###_` ID. | -| `setProtPoolSize` | `set_prot_pool_size` | **`[3→4]`** Sets the **upper** bound of `prot_pool_exchange` (forward direction). | - -### 5.2 Gather kcats (`src/gather_kcats`) - -| MATLAB | geckopy | Notes | -| ----------------------------------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `fuzzyKcatMatching` | `fuzzy_kcat_matching` | **`[3→4]`** Wildcard escalation no longer escalates past the fully-wildcarded form. **`[Py]`** Returns a `DataFrame` (rows: ec / substrate / organism / kcat / source / wildcard\_level …) instead of a struct; takes pre-loaded `BrendaData` + `PhylDist`. | -| `getStandardKcat` | `assign_standard_kcat` | **`[3→4]`** Applies per-subsystem mean kcat (GECKO 3 fell back to the global standard kcat unless every subsystem matched). **`[Py]`** Renamed. | -| `mergeDLKcatAndFuzzyKcats` | `merge_kcats` | **`[Py]`** Generalised to accept any combination of BRENDA / DLKcat / OpenKineticsPredictor / manual rows; records per-row provenance. | -| `readDLKcatOutput` | *(none)* | **`[3→4]`** Deprecated in MATLAB: `kcatsFromDLKcat(..., 'via', 'file')` / `kcats_from_dlkcat(..., via="file")` read the file. Case-insensitive substrate-name matching. | -| `removeStandardKcat` | `remove_standard_kcat` | — | -| `runDLKcat` | *(none)* | **`[3→4]`** Deprecated in MATLAB: `kcatsFromDLKcat(..., 'via', 'docker')` / `kcats_from_dlkcat(..., via="docker")` run DLKcat. | -| `selectKcatValue` | `apply_kcat_list` | **`[Py]`** Renamed. | -| `writeDLKcatInput` | *(none)* | **`[3→4]`** Deprecated in MATLAB: `kcatsFromDLKcat(..., 'via', 'docker')` / `kcats_from_dlkcat(..., via="docker")` write the input. Bounds-safe when `ec_rxns` is a subset. | -| `kcatsFromBrenda` | `kcats_from_brenda` | Fuzzy BRENDA matching, kept as `data/kcats/brenda.tsv`. Loads the BRENDA data and KEGG distances from the adapter unless passed in. | -| `kcatsFromDLKcat` | `kcats_from_dlkcat` | DLKcat, kept as `data/kcats/dlkcat.tsv`. `via` is `"okp"` (the DLKcat method of OpenKineticsPredictor), `"file"` (`data/DLKcat.tsv`) or `"docker"` (a local run, `run=True`); returns `(kcat_list, done)` as `kcats_from_okp`. | -| `kcatsFromOKP` | `kcats_from_okp` | One OpenKineticsPredictor method, kept as `data/kcats/okp_.tsv`; submits a job when there is none, writes the file once it is done. Returns `(kcat_list, done)`. | -| `saveKcatSource`, `loadKcatSource` | `save_kcat_source`, `load_kcat_source`, `read_kcat_source` | Kcat source files: one per source, keyed by conventional reaction and enzyme set, the same file format in both tools. `load_kcat_source` maps a file to the current model and reports rows that fit no reaction. **`[Py]`** `read_kcat_source` reads a file without a model; GECKO gives the same in `loadKcatSource`'s `report.header` and `report.records`. | -| `getEcRxnKeys` | `ec_rxn_keys` | Conventional reaction and enzyme set per `ec.rxns` entry. | -| `kcatsFromECTree` | `kcats_from_ec_tree` | EC median of the BRENDA measurement table, shrunk toward the EC's 3- and 2-digit classes (weight n / (n + k)), kept as `data/kcats/ec_tree.tsv`. | -| `kcatsFromHomologs` | `kcats_from_homologs` | Identity-weighted mean of up to 10 measured homologs (DIAMOND, ≥ 40% identity, 80% coverage) on the reaction's substrate, kept as `data/kcats/homologs.tsv`. | -| `extractEnzymeSubstratePairs` | `extract_enzyme_substrate_pairs` | The enzyme–substrate pairs DLKcat and the homolog search use; geckopy also returns `met_id`. | -| `ecTreeEstimate` | `ECTree.estimate` (`geckopy.gather_kcats.ec_tree`) | The EC-tree value for one or more EC numbers. | -| `loadBrendaMeasurements` | `load_brenda_measurements` | The BRENDA measurement table (`measurements.tsv`): wild-type kcats with an EC number, after the outlier screen. | -| `fetchBrendaAsset` | `fetch_brenda_asset` | Downloads (once) the BRENDA files of the toolbox-assets release `brenda-2026.1-1` and returns their folder; used when a project has no `data/brenda`. | -| `convertCustomKcats` | `convert_custom_kcats` | `data/customKcats.tsv` to the curated source file `data/kcats/custom.tsv`, with the kcats GECKO 3 `applyCustomKcats` gives the ecModel. | -| `upgradeEcModel` | `upgrade_ec_model` | Turns GECKO 3 protein usage and pool reactions forward; `load_ec_model` already does this for the files it reads. | -| `upgradeGECKOproject` | `upgrade_project` (`geckopy upgrade`)| Lists, or carries out, the steps from a GECKO 3 project to the GECKO 4 layout. The adapter-class step is MATLAB only. | -| `collectKcats` | `collect_kcats` | Decides one kcat per reaction from the source files, as `[kcat_assignment]` in model_adapter.toml sets (preset `brenda_ec_tree`, the default, `gecko_median` or `gecko_classic`, or stages), without changing the model. Writes `output/_kcatDecisions.tsv`. | -| `assignKcats` | `assign_kcats` | `collect_kcats`, then sets `ec.kcat` and `ec.source` and applies the kcat constraints; replaces `merge_kcats` → `apply_kcat_list` → the custom kcats → `fill_kcats_from_isozymes` → `assign_standard_kcat` → `apply_kcat_constraints`. In place. | -| `updateKcatSources` | `update_kcat_sources` | Gathers the source files the configuration needs that are missing, or outdated against `[kcat_sources.]`; `dry_run` only reports. | - -### 5.3 Get enzyme data (`src/get_enzyme_data`) - -| MATLAB | geckopy | Notes | -| --------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `calculateMW` | `calculate_mw` | **`[Py]`** Stand-alone function (takes a sequence string, returns the MW); no model argument needed. | -| `copyECtoGEM` | `copy_ec_to_gem` | — | -| `downloadKEGG` | `geckopy.databases.kegg_download.download_kegg` | **`[3→4]`** Standalone helper (was split out in 4). **`[Py]`** Per-gene REST cache; `geckopy brenda-refresh` and friends drive it lazily. | -| `downloadUniProt` | `geckopy.databases.uniprot_download.download_uniprot` | **`[3→4]`** Standalone helper (was split out in 4). | -| `findECInDB` | `find_ec_in_db` | **`[3→4]`** No duplicate EC codes emitted. | -| `getECfromDatabase` | `fill_eccodes_from_database` | **`[3→4]`** Can consult KEGG when UniProt's EC field is empty or ends with `-` (opt-in via `kegg_db=`). **`[Py]`** Renamed. | -| `getECfromGEM` | `fill_eccodes_from_gem` | **`[3→4]`** Actually assigns EC codes (the GECKO 3 validation regex silently discarded every EC string). **`[Py]`** Renamed. | -| `getECstring` | *(internal helper inside `fill_eccodes_*`)* | Not exposed as a public entry-point — folded into the EC-fill helpers. | -| `loadBRENDAdata` | `load_brenda_data` | **`[3→4]`** Reads the new TSV schema (`kcat.tsv` / `sa.tsv` / `mw.tsv`, bare EC codes, plain organism names, `references` column). **`[Py]`** Ships **both** `max` and `median` per (ec, substrate, organism) triple — see [kcat_aggregation.md](kcat_aggregation.md). Returns a `BrendaData` dataclass. | -| `loadDatabases` | *(implicit)* | **`[Py]`** No combined-load entry-point: `make_ec_model` loads `uniprot.tsv` and `kegg.tsv` and downloads a missing one with the adapter's settings, as `makeEcModel` does through `loadDatabases`; elsewhere call `load_uniprot_tsv` and `load_kegg_tsv` directly. | - -### 5.4 Kcat sensitivity analysis and tuning (`src/kcat_tuning`) - -| MATLAB | geckopy | Notes | -| ------------------------------------- | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sensitivityTuning` | `sensitivity_tuning` | **`[Py]`** Returns a `TunedKcatsResult` dataclass (kcat changes table + final flux). | -| `sigmaFitter` | `fit_sigma` | **`[3→4]`** Returns the model fitted to the **optimal** sigma (GECKO 3 returned the last trial value, sigma = 1.0). **`[Py]`** Renamed. Returns a `SigmaFitterResult`. | -| `cmaesKcatTuning` + helpers | `kcat_tuning.evotune.cmaes_kcat_tuning` + helpers | Fits kcats to experimental flux/growth data with CMA-ES; see [evotune_kcat_tuning.md](evotune_kcat_tuning.md). | - -### 5.5 Limit proteins (`src/limit_proteins`) - -| MATLAB | geckopy | Notes | -| --------------------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `calculateFfactor` | `calculate_f_factor` | Without `prot_data` it reads `data/paxDB.tsv` of the project, and without that file it returns `0.5` with a warning, as MATLAB does. | -| `constrainEnzConcs` | `constrain_enz_concs` | **`[3→4]`** Sets the **upper** bound of `usage_prot_*` (forward direction). **`[Py]`** Raises `NotImplementedError` on gecko-light (no per-enzyme constraint to set). | -| `constrainFluxData` | `apply_flux_data_constraints` | **`[Py]`** Renamed. Takes pre-loaded `FluxData`. 0-indexed `condition` (default `0`, MATLAB `1`). | -| `fillEnzConcs` | `fill_enz_concs` | — | -| `flexibilizeEnzConcs` | `flexibilize_enz_concs` | **`[Py]`** Returns a `FlexEnzResult` dataclass. Raises `NotImplementedError` on gecko-light. | -| `getConcControlCoeffs` | `get_conc_control_coeffs` | **`[3→4]`** Probe direction reads the forward `usage_prot_*` upper bound. **`[Py]`** Default path reads the reduced cost of each usage reaction's forward variable (one LP solve plus one check solve per nonzero value, against one solve per enzyme for the finite-difference probe), so an enzyme whose own bound does not bind gets 0 also while the protein pool binds; falls back to finite-difference when the solver exposes no variable duals or returns NaN. Gates on `sol.status == "optimal"`. | -| `getIsozymeCoverage` | `get_isozyme_coverage` | **`[Py]`** Returns the coverage `DataFrame` (columns `rxn`, `n_entries`, `n_measured`, `n_unmeasured`, `measured_enzymes`, `unmeasured_enzymes`, `shared_enzymes`) and the list of zeroable enzymes. Raises `NotImplementedError` on gecko-light. | -| `updateProtPool` | *(not ported)* | Obsolete since GECKO 3.2.0. Use `set_prot_pool_size` instead. | -| `zeroUnmeasuredIsozymes` | `zero_unmeasured_isozymes` | **`[Py]`** Lowers the usage upper bounds in place and returns the changed enzymes as a `DataFrame` (`uniprot`, `new_upper_bound`). Method `'detectionLimit'` is `"detection_limit"`. | -| `relaxProteomicsGreedy` | `relax_proteomics_greedy` | Alternative to `flexibilize_enz_concs`: until growth reaches the target, sets the usage upper bound of the constrained enzyme with the largest absolute shadow price of its `prot_` metabolite to the default bound, one enzyme per iteration. **`[Py]`** Returns a `GreedyRelaxResult` (`converged=False` when no candidate is left) and changes the model in place; raises `RuntimeError` when `max_iterations` relaxations do not reach the target and candidates remain. | - -### 5.6 Utilities (`src/utilities`) - -| MATLAB | geckopy | Notes | -| --------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `addNewRxnsToEC` | `add_new_rxns_to_ec` | **`[Py]`** Returns an `AddNewRxnsResult` dataclass. | -| `ecFSEOF` | `ec_fseof` | **`[Py]`** Thin wrapper over `raven_toolbox.analysis.fseof`. Targets are classified as in MATLAB, with pFBA per step: the correlation of the absolute flux with the enforced production, labelled `OE` / `KD` / `KO`; `nSteps` and `n_steps` both default to 16. Result type is `EcFSEOFResult`, raven's `FSEOFResult` with the gene targets of MATLAB (`gene_targets`: one action per gene, reconciled with the gene essentiality test). No files are written; MATLAB's `rxnTargets` and `transportTargets` are one `targets` table with a `transport` column. | -| `ecFVA` | `ec_fva` | **`[Py]`** Returns a `DataFrame` indexed by reaction id with `minimum` / `maximum` columns; can run in parallel (`n_jobs=`). | -| `enzymeUsage` | `enzyme_usage` | **`[3→4]`** Usage flux read from positive `usage_prot_*` flux. **`[Py]`** Returns an `EnzymeUsageResult` dataclass (`per_enzyme` DataFrame + summary scalars). | -| `findGECKOroot` | *(not needed)* | Python: `pathlib.Path(__file__)` works. | -| `getSubsetEcModel` | `get_subset_ec_model` | **`[Py]`** Returns a new `EcModel`; the input is not mutated. | -| `loadConventionalGEM` | `load_conventional_gem` | — | -| `loadEcModel` | `load_ec_model` | **`[3→4]`** Auto-flips legacy reverse-direction `usage_prot_*` / `prot_pool_exchange` to the forward convention (warns once). Reads the new cobrapy-style YAML. **`[Py]`** Thin wrapper over `raven_toolbox.io.read_yaml_model` (which owns the YAML schema + typed `EcData`); SBML ecModel I/O dropped in `0.1.0a2`. | -| `loadFluxData` | `load_flux_data` | **`[Py]`** Returns a `FluxData` dataclass. | -| `loadProtData` | `load_prot_data` | **`[Py]`** Returns a `ProtData` dataclass. Option `zerosAsMeasured` is `zeros_as_measured`. | -| `mapRxnsToConv` | `map_rxns_to_conv` | **`[Py]`** Returns a `MapRxnsResult` dataclass (was three parallel outputs). | -| `plotEcFVA` | *(not ported)* | Use matplotlib / seaborn against the `ec_fva` DataFrame directly. | -| `reportEnzymeUsage` | `report_enzyme_usage` | **`[3→4]`** Reads positive `usage_prot_*` flux. **`[Py]`** Returns an `EnzymeUsageReport` dataclass. | -| `saveEcModel` | `save_ec_model` | **`[3→4]`** Emits the new cobrapy-style YAML; `usage_prot_*` written in forward convention. **`[Py]`** Thin wrapper over `raven_toolbox.io.write_yaml_model`; YAML only (`.yml` / `.yaml`), SBML ecModel I/O dropped in `0.1.0a2`. Injects a `metaData` provenance block. | -| `startGECKOproject` | `geckopy init` *(CLI)* | **`[Py]`** Project skeleton generator runs from the `geckopy` CLI (`geckopy init my_project`). | -| `updateGECKOdoc` | *(not applicable)* | Python docs are docstring-driven; no MATLAB Toolbox metadata to regenerate. | -| — | `pfba_enzymes` *(new)* | **`[Py]`** pFBA variant that minimises total `usage_prot_*` flux instead of all fluxes. Raises `NotImplementedError` on gecko-light. | -| — | `get_enzyme_bottlenecks` *(new)* | **`[Py]`** Per-enzyme bottleneck ranker; returns a DataFrame sorted by the re-solved change of the objective when the bound of one enzyme at its bound is raised by `fold_change`. The shadow price is a reference column. | - -### 5.7 Model adapter (`src/model_adapter`) - -| MATLAB | geckopy | Notes | -| ----------------------- | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `ModelAdapter` classdef | `ModelAdapter` class + `model_adapter.toml` | **`[Py]`** Config is in TOML, validated against a pydantic `ModelParameters` schema. Override via subclass when you need custom logic (rarely). | -| `ModelAdapterManager` | *(no global default)* | **`[Py]`** Removed by design — pass `adapter=` or set `model.adapter` so adapter use is explicit. | -| `adapterTemplate.m` | `geckopy init` *(CLI)* / `templates/model_adapter.toml` | **`[Py]`** `geckopy init ` scaffolds a project folder with a starter TOML, `data/`, and `models/`. | - ---- - -## 6. A minimal workflow comparison - -**MATLAB** (sketch): - -```matlab -adapter = ModelAdapterManager.setDefault('MyAdapter.m'); -model = makeEcModel(loadConventionalGEM(), false, adapter); -kcatList = fuzzyKcatMatching(model); -model = selectKcatValue(model, kcatList); -model = applyKcatConstraints(model); -model = setProtPoolSize(model); -saveEcModel(model, 'ecModel.yml'); -``` - -**geckopy**: - -```python -from geckopy import ( - ModelAdapter, make_ec_model, load_conventional_gem, - load_brenda_data, load_phyl_dist, - fuzzy_kcat_matching, apply_kcat_list, apply_kcat_constraints, - set_prot_pool_size, save_ec_model, -) -from geckopy.databases import download_phyl_dist - -adapter = ModelAdapter.from_folder("my_model") -model = make_ec_model(load_conventional_gem(adapter), adapter) -brenda = load_brenda_data(adapter.get_brenda_db_folder()) -if not adapter.get_phyl_dist_path().is_file(): - download_phyl_dist(adapter.get_phyl_dist_path()) # RAVEN's keggPhylDist.mat -phyl_dist = load_phyl_dist(adapter.get_phyl_dist_path()) -kcats = fuzzy_kcat_matching(model, brenda, phyl_dist) # returns a DataFrame -apply_kcat_list(model, kcats) # mutates in place -apply_kcat_constraints(model) -set_prot_pool_size(model) -save_ec_model(model, "ecModel.yml", adapter=adapter) -``` - -Three patterns to notice: - -1. **Loaders are explicit.** Where MATLAB's `fuzzyKcatMatching` - re-loads BRENDA + phyl-dist internally on every call, geckopy - passes the pre-loaded `BrendaData` + `PhylDist` so you control - when (and from where) they load. MATLAB reads the phylogenetic - distances from RAVEN's `keggPhylDist.mat`; `download_phyl_dist` - fetches that same file into the project's `data/` folder. -2. **`apply_kcat_list` mutates.** No reassignment to `model`. -3. **`save_ec_model` is YAML-only.** SBML ecModel I/O was dropped in - `0.1.0a2`; `load_conventional_gem` still reads starting GEMs from - SBML (`cobra.io.read_sbml_model`) as well as from RAVEN/cobra YAML - (raven-toolbox's `read_yaml_model`). - -The full tutorial (yeast-GEM, end-to-end, ~600 lines including kcat -curation + proteomics + sensitivity tuning) lives at -[`tutorials/full_ecModel/protocol.py`](../tutorials/full_ecModel/protocol.py). -A scale-down for the light layout (also a working end-to-end script, -~170 lines) is at [`tutorials/light_ecModel/protocol.py`](../tutorials/light_ecModel/protocol.py). - ---- - -## 7. File exchange - -ecModels move between the two tools without conversion: - -- **Writing.** Both tools emit the same cobrapy-style YAML - (cobra-shaped portion as `!!omap`; the GECKO ec sections — - `ec-rxns`, `ec-enzymes`, `gecko_light` — as top-level keys cobrapy - silently ignores). A geckopy `.yml` loads in GECKO 4 and vice versa. -- **Reading.** Backward-compatible: geckopy reads GECKO 3 / older - RAVEN ecModels too, and auto-flips legacy reverse-direction protein - reactions on load (warns once). Save the loaded model and it comes - out as a current model. - -The YAML schema and the legacy normalisations are owned by raven-toolbox -(`raven_toolbox.io.ec_data.EcData`, -`raven_toolbox.io.yaml.{read,write}_yaml_model`); geckopy's -`save_ec_model` / `load_ec_model` are thin wrappers that add geckopy's -application-level concerns (adapter-aware path resolution, -empty-ecModel guard, provenance injection). See -[raven_integration.md](raven_integration.md) for the split, and -[yaml_format.md](yaml_format.md) for the schema reference. - -**BRENDA data files.** **`[3→4]`** GECKO 4 / geckopy include -`kcat.tsv` / `sa.tsv` / `mw.tsv` (refreshed from the BRENDA bulk -JSON; bare EC numbers, plain organism names, `#` release header, -`references` column). GECKO 3's `max_KCAT.txt` etc. are no longer -read. **`[Py]`** geckopy additionally includes both **max** and -**median** rows per (ec, substrate, organism) triple in one file, -distinguished by an `aggregation` column. The default -`kcat_aggregate_brenda = "max"` preserves MATLAB-GECKO behaviour. -See [kcat_aggregation.md](kcat_aggregation.md). - ---- - -## 8. What's available where - -### Same algorithms, aligned results - -The port found and fixed several GECKO 3 bugs (EC-code assignment, -sigma fitting, standard-kcat subsystem means, duplicate EC codes, -…); **`[3→4]`** GECKO 4 includes the same fixes, so the two toolboxes -agree. The full per-function divergence list lives on the -[GECKO vs. geckopy](https://gecko-docs.readthedocs.io/en/latest/gecko-to-geckopy.html) page -of gecko-docs.readthedocs.io, plus [future_improvements.md](future_improvements.md) -for the MATLAB-side items the port surfaced. - -### Available in both (added during the port) - -- **KEGG as a fallback protein/EC source** (`make_ec_model(kegg_db=…)`, - `fill_eccodes_from_database`). -- **OpenKineticsPredictor over its REST API** — - `kcatsFromOKP` / `kcats_from_okp`, which submit a job and fetch its result. -- **Forward-direction protein reactions** (`usage_prot_*` / - `prot_pool_exchange`). -- **Refreshed BRENDA TSV schema** (geckopy adds the max+median - variant; MATLAB includes only max so far). -- **Greedy relaxation of proteomics constraints** - (`relaxProteomicsGreedy` / `relax_proteomics_greedy`), an alternative - to `flexibilize_enz_concs`. - -### geckopy-only (no MATLAB counterpart) - -- `pfba_enzymes` — enzyme-pool-minimising pFBA. -- `get_enzyme_bottlenecks`: bottleneck ranking by the re-solved change of the objective. -- `merge_kcats` — generalised n-source kcat merge (replaces the - DLKcat-only MATLAB helper). -- `geckopy brenda-refresh` CLI — rebuilds the BRENDA TSV from a - bulk-JSON snapshot. -- `get_subset_ec_model` — *(also exists in MATLAB)* but the Python - version returns a fresh model rather than mutating in place. -- The `Enzyme` proxy (`model.enzymes.get_by_id(...)`). -- `geckopy init` — project skeleton CLI. - -### Not yet in geckopy - -- **`plotEcFVA`** — Python users plot from the `ec_fva` DataFrame - with matplotlib / seaborn directly. - ---- - -## 9. Where to look when something differs - -- Each ported function's docstring opens with - `Ported from GECKO MATLAB: .` so you can find the exact - MATLAB original. -- For current MATLAB ↔ Python behavioural differences, see the - [GECKO vs. geckopy](https://gecko-docs.readthedocs.io/en/latest/gecko-to-geckopy.html) page - on [gecko-docs.readthedocs.io](https://gecko-docs.readthedocs.io/), the - shared documentation site for both toolboxes. -- MATLAB-side bugs and rough edges the port found are tracked in - [future_improvements.md](future_improvements.md). -- For the GECKO 3→4 changes as they apply on the MATLAB side, see - [GECKO 3 to GECKO 4](https://gecko-docs.readthedocs.io/en/latest/gecko3-to-gecko4.html) - in the GECKO repository. -- For the YAML schema (shared with MATLAB GECKO 4 / RAVEN), see - [yaml_format.md](yaml_format.md). -- For the gecko-light availability table, see - [gecko_light_status.md](gecko_light_status.md). -- For the raven-toolbox / geckopy responsibility split, see - [raven_integration.md](raven_integration.md). diff --git a/docs/relaxation_methods.md b/docs/relaxation_methods.md deleted file mode 100644 index a2bd6d3..0000000 --- a/docs/relaxation_methods.md +++ /dev/null @@ -1,118 +0,0 @@ -# Relaxing proteomics constraints: `flexibilize_enz_concs`, `relax_proteomics_greedy` and `relax_enz_concs` - -Both functions answer the same question — *the measured proteomics constraints -make the ecModel grow too slowly (or not at all); which enzyme bounds do we -loosen, and by how much, to restore feasible growth?* — but they are **not** -duplicates. They use different selection signals, different relaxation step -sizes, and produce results of very different stringency. This note records the -differences and the recommendation (keep both; do not merge). -`relax_enz_concs`, a third method that solves the same question as one LP, and -the per-complex options for all of them are described at the end. - -## Side by side - -| aspect | `flexibilize_enz_concs` | `relax_proteomics_greedy` | -|---|---|---| -| Origin | GECKO MATLAB `flexibilizeEnzConcs.m` (faithful port) | legacy geckopy, Carrasco et al. 2023 | -| Which enzyme to relax | largest **control coefficient** (how much each measured enzyme's bound limits growth, from `get_conc_control_coeffs`: the reduced cost of its usage bound, or a finite difference where the solver exposes no duals) | largest **\|shadow price\|** (LP dual of the `prot_` metabolite of the enzyme) | -| Solves per selection | one LP solve plus one check solve per enzyme with a nonzero coefficient (one solve per candidate protein where the solver exposes no duals) | a **single** `slim_optimize` exposes all duals (cheap) | -| Relaxation step | **gradual**: usage `ub ← conc·(1 + fold_change·k)`; the same enzyme can be selected repeatedly and loosened a bit more each time | **all-or-nothing**: usage `ub ← default_upper_bound` (≈ unconstrained) in one shot | -| Tighten-back | yes: a refinement pass minimises the protein pool at `exp_growth` and **drops** enzymes whose true usage is below their original conc (un-relaxes over-relaxed ones) | none: each selected enzyme stays fully unconstrained | -| Result stringency | **tight** (minimal, proteomics-faithful relaxation) | **loose** (tends to over-relax) | -| Pool fallback | yes — relaxes `prot_pool_exchange` when no single enzyme helps | no | -| Scope | measured enzymes only (`~isnan(concs)`) | any currently-constrained enzyme (`ub < default`) | -| Non-convergence | returns a partial `FlexEnzResult` + warning (global iteration cap) | returns `converged=False` when no candidate is left; **raises** `RuntimeError` when `max_iterations` relaxations leave growth below the target and candidates remain | -| Result type | `FlexEnzResult` (relaxed enzymes, per-enzyme `frequence`, ratios) | `GreedyRelaxResult` (relaxed→orig-conc dict, step trace, `converged`) | - -## Interpretation - -- **`flexibilize_enz_concs`** is the right default when you want to stay as - faithful to the proteomics data as possible: it loosens enzymes a little at a - time and then tightens back everything that didn't actually need loosening, - so the final model is the *minimal* deviation from the measurements. The cost - is speed — the control-coefficient computation re-solves the LP for every - measured protein on every outer iteration. - -- **`relax_proteomics_greedy`** is the right choice when you want a fast answer - and the infeasibility is dominated by one or two enzymes: a single solve - ranks all enzymes by shadow price, and fully unconstraining the worst offender - usually moves growth a lot. The cost is that it **over-relaxes** (no - tighten-back), so the resulting model is less constrained than necessary, and - it raises when `max_iterations` relaxations do not reach the target while - candidates remain. - -## Recommendation - -Keep both — they are complementary, not redundant. They could share small -helpers (eligible-enzyme selection, result shaping), but their cores differ -enough that merging into one function would obscure the two algorithms. Two -follow-ups would make them more interchangeable: - -1. Give `relax_proteomics_greedy` an optional tighten-back / graded step so its - output is not systematically looser than `flexibilize_enz_concs` (this is - the over-relaxation point in the modeling-intent review, §3.3). -2. Align the failure convention: at its iteration cap `flexibilize_enz_concs` - returns a partial result with a warning and `relax_proteomics_greedy` - raises; select one so callers don't special-case each. - -## `relax_enz_concs`: the least total relaxation, in one LP - -Both methods above relax one enzyme at a time: rank, raise the top one, -re-solve, repeat. Which enzymes end up raised then depends on the ranking -signal and on the order in which enzymes become limiting. On a large model -with noisy proteomics, a small change to the ranking (for example, control -coefficient per mg against gain per fold) can change which enzymes are -raised and, downstream, predictions that depend on the remaining caps. - -`relax_enz_concs` asks the question once. Every measured cap becomes -`usage <= conc * (1 + x)`, the protein pool may grow by a fold too, and one LP -minimises the sum of the `x` subject to `growth >= exp_growth`: - -- `weight="relative"` (default) minimises summed fold increases, so an enzyme - measured at almost nothing is not relaxed freely because its absolute cost is - small; `weight="absolute"` minimises the mg added. -- The pool's fold costs `pool_weight` (default 1000) times an enzyme's, so the - pool grows only where raising enzymes cannot reach the target, the role of - the pool fallback in `flexibilize_enz_concs`. -- Folds count from the current caps, so a cap raised earlier is never lowered. -- If no relaxation reaches the target, it raises `ValueError`: growth is then - not limited by protein. - -It returns a `FlexEnzResult` like `flexibilize_enz_concs` (`frequence` is 1 for -every entry). Use it when the outcome should not depend on the order of -relaxation, and to compare with the iterative methods: if they disagree on -which enzymes need raising, conclusions that hinge on those enzymes are -fragile. - -## Enzyme complexes - -Absolute proteomics recovers the subunits of one complex unevenly: subunits -present 1:1 can come out several-fold apart while their replicates agree within -a few percent. Capped one by one, a complex can then carry no more flux than -its least abundant measurement allows, and relaxation methods that raise one -subunit at a time leave its subunits at unrelated levels, or raise a different -subunit in each condition. - -- `level_complex_concs` (after `fill_enz_concs`, before `constrain_enz_concs`) - gives every complex one abundance: the median per copy over its subunits with an abundance above 0, scaled back to each subunit by its copies in `ec.rxn_enz_mat`; unmeasured subunits are filled, and a subunit measured at 0 stays 0. `level_complex_abundances` does the same on molar data before it is written into the model. -- `complex_families` defines the complexes. Two enzyme sets are one complex when - they share at least two subunits and half the larger set (isozyme variants, - the same complex on several reactions). A protein in three or more complexes, - in two with equal weight, or that also catalyses a reaction alone is *shared* - and keeps its own measurement: electron carriers (cytochrome c), thioredoxin, - or the E3 subunit of the 2-oxo-acid dehydrogenases, which would otherwise - chain unrelated complexes together. A subunit in two complexes belongs to the - one with more reactions (core subunits that also appear in an assembly step). -- `flexibilize_enz_concs(per_complex=True)` and `relax_enz_concs(per_complex=True)` - raise the measured subunits of a complex by one factor, so levelled - concentrations stay levelled. In `flexibilize_enz_concs` complexes are ranked - by the growth gained per fold of raising all their caps (the usage reactions' - reduced costs times the caps), which leaves out the protein pool's price and - does not depend on which of several simultaneously binding subunits the solver - puts the dual on. - -Levelling relies on the stoichiometry in `ec.rxn_enz_mat`; where -`apply_complex_data` found no exact ComplexPortal match, every subunit counts -as one copy. Per-complex relaxation without levelling raises every subunit by -the fold its worst-measured subunit needs, which inflates the well-measured -ones; use the two together. diff --git a/docs/yaml_format.md b/docs/yaml_format.md deleted file mode 100644 index 82201d3..0000000 --- a/docs/yaml_format.md +++ /dev/null @@ -1,193 +0,0 @@ -# geckopy ecModel YAML format - -This document defines the YAML format that geckopy uses to save and -load ecModels. - -## It *is* the cobrapy YAML format, plus a few GECKO keys - -There is no bespoke geckopy format. An ecModel is saved as **exactly -the YAML that cobrapy writes** (`cobra.io.save_yaml_model`), with a -handful of GECKO-specific top-level keys added alongside for the data -cobra doesn't model: kcat values, the enzyme list (MW, sequence, -concentration), and the reaction-enzyme coupling matrix. - -geckopy delegates the I/O to -[raven-toolbox](https://github.com/CompMetEng/raven-toolbox), which -calls cobrapy for the cobra-shaped portion and parses the GECKO -`ec-rxns` / `ec-enzymes` / `gecko_light` sections into a typed -`EcData` substructure attached as `model.ec`. That means: - -- any cobrapy-aware tool (Escher, Memote, plain cobra) loads the file - and simply ignores the extra `ec-*` keys; -- MATLAB GECKO / RAVEN write and read the same format, so ecModels - exchange between the two toolboxes with no translator. - -## Design goals - -1. **Identical to cobrapy.** The cobra-shaped portion is exactly what - `cobra.io.model_to_dict` / `model_from_dict` write and read back; no custom - parser, no custom schema. -2. **Round-trippable with MATLAB GECKO.** RAVEN's `writeYAMLmodel` - and `readYAMLmodel` emit and accept this same format. -3. **Backward compatible.** Both geckopy and RAVEN also load the - legacy MATLAB / RAVEN layout (see the last section). Empty strings - and NaN values can be omitted; the reader fills in the defaults. - -## Top-level structure - -The whole file is one YAML mapping (a top-level dictionary). It -contains the following keys: - -| Key | Required? | Owner | What it holds | -|---|---|---|---| -| `id` | yes | cobra | Model id | -| `name` | optional | cobra | Human-readable name | -| `version` | optional | cobra | cobra schema version (omit if unused) | -| `compartments` | optional | cobra | Mapping of compartment id -> name | -| `metabolites` | yes | cobra | List of metabolite entries (see below) | -| `reactions` | yes | cobra | List of reaction entries (see below) | -| `genes` | yes | cobra | List of gene entries (see below) | -| `ec-rxns` | yes (for ecModel) | raven-toolbox | Per-reaction ec data (see below) | -| `ec-enzymes` | yes (for ecModel) | raven-toolbox | Per-enzyme ec data (see below) | -| `gecko_light` | optional | raven-toolbox | Boolean flag; defaults to `false` | -| `metaData` | optional | raven-toolbox | Free-form provenance: version, date, author, taxonomy, note | - -The "Owner" column says which library defines and parses each field. -Tools that only understand the cobra side (cobrapy, escher, memote, -...) silently ignore the GECKO-specific keys (`ec-rxns`, -`ec-enzymes`, `gecko_light`, `metaData`), so the same file loads -cleanly as a plain cobra model in those tools — you just lose the ec -layer. - -## Cobra-shaped section - -This is exactly the schema produced by -`cobra.io.dict.model_to_dict`. A few conventions worth noting: - -- `compartments` is a flat mapping (`{c: cytosol, e: extracellular}`), - not a list. -- `metabolites`, `reactions`, `genes` are lists of plain mappings - (one per metabolite/reaction/gene). -- A reaction's `metabolites` field is a flat mapping - `met_id -> stoichiometric_coefficient`. -- `annotation` is a mapping `key -> [list, of, strings]`. RAVEN's - per-metabolite `smiles` field lives at - `annotation: {smiles: ["..."]}`. - -Example metabolite: - -```yaml -- id: s_0001 - name: "(1->3)-beta-D-glucan" - compartment: ce - formula: C6H10O5 - charge: 0 - annotation: - bigg.metabolite: ["13BDglcn"] - chebi: ["CHEBI:37671"] - kegg.compound: ["C00965"] - metanetx.chemical: ["MNXM6492"] - sbo: ["SBO:0000247"] - smiles: ["C(C1C(C(C(C(O1)O)O)O)O)O"] -``` - -Example reaction: - -```yaml -- id: r_0001 - name: "(R)-lactate:ferricytochrome-c 2-oxidoreductase" - metabolites: - s_0027: -1.0 - s_0556: 1.0 - lower_bound: 0 - upper_bound: 1000 - gene_reaction_rule: "(YDL174C and YML054C) or (YEL024W and YML054C)" - subsystem: "Oxidative phosphorylation" - annotation: - ec-code: ["1.1.2.4"] - kegg.reaction: ["R00196"] -``` - -## `ec-rxns`: per-reaction ec data - -A list, one entry per catalysed reaction. When the build pipeline -splits a reaction across isozymes (`_EXP_` suffix) or into -forward/reverse pairs (`_REV` suffix), each variant gets its own -entry here. Fields: - -| Field | Type | Required? | Notes | -|---|---|---|---| -| `id` | string | yes | Matches a model reaction id; may have `_EXP_` and/or `_REV` suffixes | -| `kcat` | number | yes | Turnover number in s^-1; write NaN as `.nan` | -| `source` | string | optional | Where the kcat came from (`"brenda"`, `"dlkcat"`, `"manual"`, ...). Default `""` | -| `notes` | string | optional | Free-form note. Default `""` | -| `eccodes` | string \| list | optional | One EC code as a string (`"1.1.2.4"`), or a list when several apply. Default `""` | -| `enzymes` | mapping `enzyme_id -> stoich` | optional | Which enzymes catalyse this reaction, and the subunit count of each. Every key must appear in `ec-enzymes`. A reaction without enzymes (such as a `standard` kcat row) has no `enzymes` entry; the loader also reads the `enzymes: !!omap` and `enzymes: !!omap []` forms that earlier versions of GECKO, RAVEN and raven-toolbox wrote for it | - -Example: - -```yaml -- id: r_0001_EXP_1 - kcat: 1500.0 - source: "brenda" - eccodes: ["1.1.2.4", "1.1.99.40"] - enzymes: - P00045: 1 - P32891: 1 -``` - -## `ec-enzymes`: per-enzyme ec data - -A list, one entry per unique enzyme. Fields: - -| Field | Type | Required? | Notes | -|---|---|---|---| -| `genes` | string | yes | Gene id; must match a `genes[].id` from the cobra section | -| `enzymes` | string | yes | Enzyme id (usually a UniProt accession) | -| `mw` | number | optional | Molecular weight in Da. Write NaN as `.nan`. Default NaN | -| `sequence` | string | optional | Amino acid sequence. Default `""` | -| `concs` | number | optional | Measured concentration in mmol/gDW (from proteomics). Write NaN as `.nan`. Default NaN | - -The `standard` pseudo-gene (added by `assign_standard_kcat` to cover -reactions without a real gene assignment) appears as both `genes: -"standard"` and `enzymes: "standard"`. - -Example: - -```yaml -- genes: "Q0045" - enzymes: "P00401" - mw: 58798.0 - sequence: "MVQRWLYSTNAKDIAVLY..." -``` - -## Sparse defaults - -To keep files compact, writers can omit any field that takes the -documented default (empty string, NaN). Readers fill the defaults -back in when loading. The exception is numeric NaN that you want -to keep explicit (e.g. "this kcat is deliberately unknown"): write -it as `.nan` so the reader sees it. - -## Legacy MATLAB / RAVEN format - -Older RAVEN/GECKO ecModels used a slightly different schema. **Both -geckopy and RAVEN load it** — geckopy's loader normalises it on read -(`_normalize_legacy_layout`), and RAVEN's `readYAMLmodel` auto-detects -it. The schema differences are: - -| Aspect | Legacy MATLAB / RAVEN | Current (cobrapy + GECKO keys) | -|---|---|---| -| Top-level `id` / `name` / `version` | nested under `metaData` | top level (where cobra reads them) | -| `smiles` per metabolite | top-level metabolite key | under `annotation` as `{smiles: ["..."]}` | -| Annotation values | scalar strings | list of strings | -| `metaData.geckoLight` | inside `metaData`, string `"true"`/`"false"` | top-level `gecko_light: ` | -| Outer `---` document marker | present | absent (cobra omits it) | -| `metaData` provenance fields | inside `metaData` | inside `metaData` (unchanged) | - -The normalisation lifts `id`/`name`/`version` to the top level and -moves per-metabolite `smiles` into `annotation`; cobra tolerates the -scalar-vs-list annotation difference on read. A legacy file written as -a bare `---` sequence of single-key maps (no `!!omap`) is merged into a -mapping by geckopy's reader. In short: the two layouts hold the same -information, and the tooling reads both. diff --git a/src/geckopy/__init__.py b/src/geckopy/__init__.py index db639fd..45a58eb 100644 --- a/src/geckopy/__init__.py +++ b/src/geckopy/__init__.py @@ -9,7 +9,7 @@ The most-used names, grouped by stage of a typical workflow: -- **Project setup**: ``ModelAdapter`` +- **Project setup**: ``ModelAdapter``, ``show_adapter_params`` - **Build**: ``load_conventional_gem``, ``make_ec_model`` - **Enrich kcats**: ``update_kcat_sources`` (``kcats_from_brenda``, ``kcats_from_dlkcat``, ``kcats_from_okp``, ...), ``collect_kcats``, @@ -38,7 +38,7 @@ __version__ = "0.0.0+unknown" # Adapter -from geckopy.adapter import ModelAdapter, ModelParameters +from geckopy.adapter import ModelAdapter, ModelParameters, show_adapter_params # Databases (loaders + dataclasses) from geckopy.databases import ( @@ -171,6 +171,7 @@ # Adapter "ModelAdapter", "ModelParameters", + "show_adapter_params", # Core data + builder + per-enzyme accessor "EcData", "EcModel", diff --git a/src/geckopy/adapter/__init__.py b/src/geckopy/adapter/__init__.py index 696e518..084d637 100644 --- a/src/geckopy/adapter/__init__.py +++ b/src/geckopy/adapter/__init__.py @@ -6,7 +6,8 @@ ModelParameters, UniprotParams, ) -from .resolve import resolve_adapter, resolve_param +from .resolve import resolve_adapter, resolve_gr_exp, resolve_param +from .show import format_adapter_params, show_adapter_params __all__ = [ "ComplexParams", @@ -14,6 +15,9 @@ "ModelAdapter", "ModelParameters", "UniprotParams", + "format_adapter_params", "resolve_adapter", + "resolve_gr_exp", "resolve_param", + "show_adapter_params", ] diff --git a/src/geckopy/adapter/adapter.py b/src/geckopy/adapter/adapter.py index e5590d4..f897c80 100644 --- a/src/geckopy/adapter/adapter.py +++ b/src/geckopy/adapter/adapter.py @@ -33,7 +33,7 @@ from pathlib import Path from typing import TYPE_CHECKING -from .params import ModelParameters +from .params import ModelParameters, check_adapter_keys if TYPE_CHECKING: import cobra @@ -82,6 +82,8 @@ def from_folder(cls, folder: str | Path) -> "ModelAdapter": with open(config_path, "rb") as f: data = tomllib.load(f) + check_adapter_keys(data, config_path) + # The path field is injected from the folder location, not the TOML, # so that the TOML file is location-independent. data["path"] = folder diff --git a/src/geckopy/adapter/adapterParameters.tsv b/src/geckopy/adapter/adapterParameters.tsv index dd19cd4..6b9f29b 100644 --- a/src/geckopy/adapter/adapterParameters.tsv +++ b/src/geckopy/adapter/adapterParameters.tsv @@ -5,13 +5,10 @@ org_name org_name Scientific name of the organism sigma sigma 0.5 Average enzyme saturation factor p_tot Ptot 0.5 Total protein content [g protein/gDW] f f 0.5 Fraction of enzymes in the model [g enzyme/g protein] -gr_exp gR_exp 0.41 Growth rate the model should reach when not limited by nutrient uptake [1/h] +gr_exp gR_exp Growth rate the model should reach when not limited by nutrient uptake [1/h]; no default, functions that need it fail when it is not set c_source c_source "" Exchange reaction of the main carbon source bio_rxn bioRxn "" Biomass pseudoreaction enzyme_comp enzyme_comp "cytoplasm" Compartment of the protein pseudometabolites -kcat_aggregate_brenda kcatAggregation "max" BRENDA aggregate read by fuzzyKcatMatching: max or median -kcat_aggregate_candidates "max" Aggregation of several candidate kcats for one reaction (geckopy) -kcat_aggregate_isozymes "mean" Aggregation of isozyme kcats when filling a missing value (geckopy) kegg.id kegg.ID "" KEGG organism code kegg.gene_id kegg.geneID "kegg" Field in the KEGG entry that matches model gene identifiers uniprot.type uniprot.type "taxonomy" Whether uniprot.id is a taxonomy or a proteome identifier @@ -21,15 +18,13 @@ uniprot.reviewed uniprot.reviewed false Use only reviewed (Swiss-Prot) entries complex.taxonomic_id complex.taxonomicID Taxonomy identifier for Complex Portal spontaneous.name_contains spontaneous.name_contains Reactions whose name contains this text are spontaneous spontaneous.file spontaneous.file TSV of spontaneous reactions, relative to the project folder: reaction identifiers, optionally a 0/1 column -evotune.sigma0_log_default evotune.sigma0logDefault 0.5 Prior standard deviation (log) for kcats whose source is in no group -evotune.source_groups evotune.kcatSources Trust tiers: group name to the ec.source values it covers -evotune.sigma0_log_source evotune.sigma0logSource Prior standard deviation (log) per group +evotune.sigma0_log_default evotune.sigma0logDefault 0.5 Prior standard deviation of the natural log of the kcat, for kcats that match no source group +evotune.source_groups evotune.kcatSources Trust tiers, one table per group with sources (ec.source origins, or prefixes ending in *, such as okp_*) and sigma0_log (standard deviation of the natural log of the kcat; the sd of a combined kcat is log10). Default groups: custom 0.1, brenda 0.2, ec_tree 0.3, combined 0.4 evotune.rmse_threshold evotune.rmseThreshold 0.2 Stop when the best RMSE reaches this evotune.max_generations evotune.maxGenerations 150 Maximum number of CMA-ES generations evotune.max_growth_weight evotune.maxGrowthWeight 1.0 Weight of the maximum-growth RMSE against the flux RMSE evotune.prior_penalty_weight evotune.priorPenaltyWeight 0.03 Weight of the prior term in the search objective evotune.tie_isozymes evotune.tieIsozymes true Give isozyme copies of one reaction a single kcat -okp.method okp.method "KinForm-H" OpenKineticsPredictor method okp.handle_long_sequences okp.handleLongSequences "truncate" How OpenKineticsPredictor handles sequences that are too long okp.include_similarity_columns okp.includeSimilarityColumns true Return similarity to the method's training data okp.canonicalize_substrates okp.canonicalizeSubstrates true Canonicalize substrate SMILES before prediction @@ -37,6 +32,6 @@ kcat_assignment.preset Kcat assignment preset: brenda_ec_tree (used when neith kcat_assignment.isozymes true Fill reactions without a kcat from their isozymes after the stages kcat_assignment.standard true Give the remaining reactions the standard kcat kcat_assignment.stage Stages instead of a preset: action (select or combine), sources, criteria and BRENDA limits ([[kcat_assignment.stage]]) -kcat_sources Settings per kcat source file, e.g. [kcat_sources.brenda] aggregation; [kcat_sources.okp] methods lists the OKP methods of the okp source group +kcat_sources Settings per kcat source file: [kcat_sources.brenda] aggregation and force_wildcard_level, [kcat_sources.ec_tree] k, [kcat_sources.homologs] k, min_identity and coverage, [kcat_sources.okp] methods (the OKP methods of the okp source group; default KinForm-H) matlab.make_model_anaerobic matlab.makeModelAnaerobic MATLAB function that constrains the ecModel to anaerobic growth matlab.change_protein_biomass matlab.changeProteinBiomass MATLAB function that changes the protein content of the biomass diff --git a/src/geckopy/adapter/params.py b/src/geckopy/adapter/params.py index 32d257e..e963b4d 100644 --- a/src/geckopy/adapter/params.py +++ b/src/geckopy/adapter/params.py @@ -2,13 +2,14 @@ ``KeggParams``, ``UniprotParams``, ``ComplexParams``, and the core fields of ``ModelParameters`` mirror the ``obj.params.*`` fields set in GECKO -MATLAB's adapter template. ``EvotuneParams`` holds the evotune settings; ``OkpParams`` is a -geckopy-only extension with no MATLAB equivalent. +MATLAB's adapter template. ``EvotuneParams`` holds the evotune settings and +``OkpParams`` the OpenKineticsPredictor request settings. Ported from GECKO MATLAB: src/deprecated/adapterTemplate.m. """ +import math from pathlib import Path -from typing import Any, Literal, Optional +from typing import Any, Literal, Optional, get_args, get_origin from pydantic import BaseModel, ConfigDict, Field, model_validator @@ -160,24 +161,25 @@ def _preset_or_stages(self) -> "KcatAssignmentParams": class SourceGroupRule(BaseModel): """One trust-tier entry in ``EvotuneParams.source_groups``. - ``ec.source`` holds literal strings like ``"dlkcat"``, ``"brenda"``, - and -- for OpenKineticsPredictor kcats -- the raw predictor method - name (e.g. ``"CataPro"``), not a generic ``"okp"`` tag. A rule - matches a given ``ec.source`` value if it's listed in ``sources``, - or (when ``match_okp`` is True) if it equals the project's - configured OKP method (``OkpParams.method``). + ``ec.source`` holds the label of the source of a kcat, such as + ``"custom"``, ``"brenda"``, ``"ec_tree"``, ``"dlkcat"``, ``"combined"`` or + ``"okp_kinform_h"``. An entry of ``sources`` matches the origin of the + label or the whole label, in either case without regard to case. An entry + ending in ``*`` is a prefix pattern on the origin: ``"okp_*"`` covers + ``okp_eitlem_kinetics`` and ``okp_kinform_h``. """ model_config = ConfigDict(extra="forbid") sources: list[str] = Field( - default_factory=list, - description="Literal ec.source strings belonging to this group.", + description=( + "ec.source labels belonging to this group; an entry ending in * " + "is a prefix pattern." + ), ) - match_okp: bool = Field( - default=False, + sigma0_log: float = Field( description=( - "Also match ec.source values equal to the project's " - "configured OpenKineticsPredictor method (OkpParams.method)." + "Prior standard deviation of the natural log of the kcat for this " + "group. Lower is more trusted, and narrows that group's proposal width." ), ) @@ -185,33 +187,30 @@ class SourceGroupRule(BaseModel): class EvotuneParams(BaseModel): """Hyperparameters for kcat tuning against experimental data. - Sources not matched by any ``source_groups`` entry fall back to the - ``*_default`` fields (matching MATLAB's ``noKcatSource`` behaviour: - ``sigma0logDefault`` etc. apply to every kcat first, then only ones - with a recognised source get overridden by their group's value). + A kcat whose source is in no ``source_groups`` entry takes + ``sigma0_log_default``. """ model_config = ConfigDict(extra="forbid") sigma0_log_default: float = Field( default=0.5, description=( - "Prior standard deviation in log-space for kcats whose source " - "matches no source_groups entry." + "Prior standard deviation of the natural log of the kcat for kcats " + "whose source matches no source_groups entry." ), ) source_groups: dict[str, SourceGroupRule] = Field( default_factory=lambda: { - "dlkcat": SourceGroupRule(sources=["dlkcat"]), - "brenda": SourceGroupRule(sources=["brenda"]), - "custom": SourceGroupRule(sources=["custom"]), + "custom": SourceGroupRule(sources=["custom"], sigma0_log=0.1), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "ec_tree": SourceGroupRule(sources=["ec_tree"], sigma0_log=0.3), + "combined": SourceGroupRule(sources=["combined"], sigma0_log=0.4), }, - description="Trust tiers: group name -> which ec.source values it covers.", - ) - sigma0_log_source: dict[str, float] = Field( - default_factory=lambda: {"dlkcat": 0.4, "brenda": 0.2, "custom": 0.1}, description=( - "Prior log-space standard deviation per group. Lower is more " - "trusted, and narrows that group's proposal width." + "Trust tiers: group name -> the ec.source labels it covers (an entry " + "ending in * is a prefix pattern, e.g. okp_*) and its sigma0_log, the " + "standard deviation of the natural log of the kcat. Groups given in " + "the TOML replace the defaults." ), ) @@ -268,39 +267,18 @@ class EvotuneParams(BaseModel): ), ) - @model_validator(mode="after") - def _check_group_keys(self) -> "EvotuneParams": - """The per-source dicts must have exactly ``source_groups``' - keys, otherwise a downstream lookup silently falls back to a - default.""" - group_names = set(self.source_groups) - for name in ("sigma0_log_source",): - keys = set(getattr(self, name)) - if keys != group_names: - raise ValueError( - f"EvotuneParams.{name} keys {sorted(keys)} must match " - f"source_groups keys {sorted(group_names)}." - ) - return self - class OkpParams(BaseModel): """OpenKineticsPredictor settings (used by kcats_from_okp). + The OKP methods are set in ``[kcat_sources.okp] methods``. + The API key is intentionally NOT here (it is a secret): provide it as a function argument, the OKP_API_KEY environment variable, or the file ``/data/okpApiKey.txt``. """ model_config = ConfigDict(extra="forbid") - method: str = Field( - default="KinForm-H", - description=( - "Predictor method: CataPro, CatPred, CatRange, DLKcat, EITLEM, " - "KinForm-H, KinForm-L, OmniESI, OmniESI-O2DENet, TurNup, UniKP " - "(see GET /api/v1/methods/)." - ), - ) handle_long_sequences: str = Field( default="truncate", description="How OKP handles sequences exceeding a method's max length.", @@ -333,7 +311,14 @@ class ModelParameters(BaseModel): sigma: float = Field(default=0.5, description="Average enzyme saturation factor") p_tot: float = Field(default=0.5, description="Total protein content [g/gDw]") f: float = Field(default=0.5, description="Fraction of enzymes in model [g/g]") - gr_exp: float = Field(default=0.41, description="Reference growth rate [1/h]") + gr_exp: Optional[float] = Field( + default=None, + description=( + "Growth rate the model should reach when not limited by nutrient " + "uptake [1/h]. Required by the functions that tune kcats or fit " + "sigma; there is no default." + ), + ) c_source: str = Field( default="", description="Reaction ID for preferred carbon source exchange" @@ -346,38 +331,6 @@ class ModelParameters(BaseModel): description="Compartment where protein pseudometabolites are placed", ) - # kcat-aggregation defaults applied when the matching function is called - # without an explicit aggregate/criteria argument. One field per function - # so each can keep its MATLAB-GECKO default while still being centrally - # overridable on the adapter. See docs/kcat_aggregation.md for the - # empirical rationale for offering `median` as an alternative to `max`. - kcat_aggregate_brenda: Literal["max", "median"] = Field( - default="max", - description=( - "Default aggregation for `fuzzy_kcat_matching` when no `aggregate` " - "is passed: collapse the BRENDA rows matched at one search level " - "to a single kcat. `max` matches MATLAB GECKO; `median` is more " - "robust to engineered-mutant/non-physiological-substrate tails." - ), - ) - kcat_aggregate_candidates: Literal["max", "min", "median", "mean"] = Field( - default="max", - description=( - "Default aggregation for `apply_kcat_list` when no `criteria` is " - "passed: collapse multiple candidate kcats for the same reaction " - "(e.g. several EC tokens per reaction) to one. `max` matches " - "MATLAB GECKO." - ), - ) - kcat_aggregate_isozymes: Literal["max", "median", "mean"] = Field( - default="mean", - description=( - "Default aggregation for `fill_kcats_from_isozymes` when no " - "`aggregate` is passed: combine known kcats across sibling " - "isozymes to fill a missing entry. `mean` matches MATLAB GECKO." - ), - ) - kegg: KeggParams = Field(default_factory=KeggParams) uniprot: UniprotParams = Field(default_factory=UniprotParams) complex: ComplexParams = Field(default_factory=ComplexParams) @@ -390,3 +343,99 @@ class ModelParameters(BaseModel): description="Settings per kcat source file, e.g. kcat_sources.brenda.aggregation.", ) matlab: MatlabParams = Field(default_factory=MatlabParams) + + +#: The keys of ``[kcat_sources.]`` that the kcat source gathering reads +#: (``update_kcat_sources``): the source name and its keys. A source file +#: ``okp_`` has no keys. +KCAT_SOURCE_KEYS: dict[str, tuple[str, ...]] = { + "brenda": ("aggregation", "force_wildcard_level"), + "ec_tree": ("k",), + "homologs": ("k", "min_identity", "coverage"), + "okp": ("methods",), + "dlkcat": (), + "custom": (), +} + + +def _kcat_source_keys(name: str) -> Optional[tuple[str, ...]]: + """The allowed keys of ``[kcat_sources.]``, or None for an unknown source.""" + from ..gather_kcats.kcat_sources import OKP_METHODS, okp_file + + if name in {okp_file(m) for m in OKP_METHODS}: + return () + return KCAT_SOURCE_KEYS.get(name) + + +def _check_table(cls: type[BaseModel], data: dict, prefix: str, file: Path | str) -> None: + for key, value in data.items(): + dotted = f"{prefix}{key}" + field = cls.model_fields.get(key) + if field is None or (cls is ModelParameters and key == "path"): + raise ValueError(f"{file}: unknown key {dotted} in model_adapter.toml") + annotation = field.annotation + args = [a for a in get_args(annotation) if a is not type(None)] + if get_origin(annotation) is None and isinstance(annotation, type) and issubclass(annotation, BaseModel): + if isinstance(value, dict): + _check_table(annotation, value, dotted + ".", file) + elif get_origin(annotation) is list and args and issubclass(args[0], BaseModel): + for item in value if isinstance(value, list) else []: + if isinstance(item, dict): + _check_table(args[0], item, dotted + ".", file) + elif get_origin(annotation) is dict and len(args) == 2 and isinstance(args[1], type) and issubclass(args[1], BaseModel): + for name, item in value.items() if isinstance(value, dict) else []: + if cls is EvotuneParams: + _check_group(item, f"{dotted}.{name}", file) + if isinstance(item, dict): + _check_table(args[1], item, f"{dotted}.{name}.", file) + elif cls is ModelParameters and key == "kcat_sources" and isinstance(value, dict): + for name, table in value.items(): + allowed = _kcat_source_keys(name) + if allowed is None: + raise ValueError(f"{file}: unknown key {dotted}.{name} in model_adapter.toml") + if not isinstance(table, dict): + raise ValueError(f"{file}: {dotted}.{name} must be a table") + for k in table: + if k not in allowed: + raise ValueError(f"{file}: unknown key {dotted}.{name}.{k} in model_adapter.toml") + _check_source_values(name, table, file) + + +def _check_source_values(name: str, table: dict, file: Path | str) -> None: + """The values of ``[kcat_sources.okp] methods`` and ``[kcat_sources.brenda] aggregation``.""" + if name == "okp" and "methods" in table: + methods = table["methods"] + if not isinstance(methods, list) or not all(isinstance(m, str) for m in methods): + raise ValueError(f"{file}: kcat_sources.okp.methods must be a list of strings") + from ..gather_kcats.kcat_sources import check_okp_methods + + check_okp_methods(methods) + if name == "brenda" and "aggregation" in table and table["aggregation"] not in ("max", "median"): + raise ValueError(f"{file}: kcat_sources.brenda.aggregation must be max or median") + + +def _check_group(item: Any, dotted: str, file: Path | str) -> None: + """The shape of one ``[evotune.source_groups.]`` table.""" + if not isinstance(item, dict): + raise ValueError(f"{file}: {dotted} must be a table with sources and sigma0_log") + unknown = [k for k in item if k not in ("sources", "sigma0_log")] + if unknown: + raise ValueError(f"{file}: unknown key {dotted}.{unknown[0]} in model_adapter.toml") + if "sources" not in item or "sigma0_log" not in item: + raise ValueError(f"{file}: {dotted} needs both sources and sigma0_log") + sources, width = item["sources"], item["sigma0_log"] + if not isinstance(sources, list) or not sources or not all(isinstance(s, str) for s in sources): + raise ValueError(f"{file}: {dotted}.sources must be a non-empty list of strings") + if isinstance(width, bool) or not isinstance(width, (int, float)) or not math.isfinite(width) or not width > 0: + raise ValueError(f"{file}: {dotted}.sigma0_log must be a positive number") + + +def check_adapter_keys(data: dict, file: Path | str) -> None: + """Raise ``ValueError`` for a key of model_adapter.toml that no parameter uses. + + The message is ``: unknown key in model_adapter.toml``. + Every table is checked: the sections, ``evotune.source_groups.``, the + ``kcat_assignment.stage`` entries and ``kcat_sources.`` (see + :data:`KCAT_SOURCE_KEYS`). ``matlab.params`` is free-form. + """ + _check_table(ModelParameters, data, "", file) diff --git a/src/geckopy/adapter/resolve.py b/src/geckopy/adapter/resolve.py index 2e93bb8..eb101bd 100644 --- a/src/geckopy/adapter/resolve.py +++ b/src/geckopy/adapter/resolve.py @@ -107,3 +107,27 @@ def resolve_param( return explicit resolved = resolve_adapter(model, adapter, purpose=purpose) return getattr(resolved.params, attr) + + +def resolve_gr_exp( + model, + explicit=None, + *, + adapter: Optional["ModelAdapter"] = None, + purpose: str = "this function", +) -> float: + """Return an explicit growth rate, else ``adapter.params.gr_exp``. + + Raises + ------ + ValueError + If ``explicit`` is ``None`` and the adapter has no ``gr_exp``: + ``gr_exp is not set in model_adapter.toml``. ``resolve_adapter`` + raises if there is no adapter either. + """ + if explicit is not None: + return float(explicit) + gr_exp = resolve_adapter(model, adapter, purpose=purpose).params.gr_exp + if gr_exp is None: + raise ValueError("gr_exp is not set in model_adapter.toml") + return float(gr_exp) diff --git a/src/geckopy/adapter/show.py b/src/geckopy/adapter/show.py new file mode 100644 index 0000000..9c29401 --- /dev/null +++ b/src/geckopy/adapter/show.py @@ -0,0 +1,120 @@ +"""Show the resolved parameters of an adapter and where each value comes from. + +One line per parameter, in the order of ``adapterParameters.tsv``, with the +value in effect and its origin: ``file`` (set in model_adapter.toml), +``preset `` (given by the kcat assignment preset) or ``default``. The +kcat assignment stages, the ``[kcat_sources]`` settings and the evotune source +groups are shown as resolved. GECKO (MATLAB) has the same function as +``showAdapterParams``; the command line is ``geckopy adapter show``. +""" +from __future__ import annotations + +import csv +from pathlib import Path +from typing import TYPE_CHECKING, Any + + +if TYPE_CHECKING: + from .adapter import ModelAdapter + +_TABLE = Path(__file__).with_name("adapterParameters.tsv") + + +def _keys() -> list[str]: + with open(_TABLE, newline="", encoding="utf-8") as f: + return [r["toml_key"] for r in csv.DictReader(f, delimiter="\t", quoting=csv.QUOTE_NONE)] + + +def _text(value: Any) -> str: + """A value as ``adapter show`` prints it: strings bare, booleans true/false, + lists comma-joined, floats with ``%g`` (whole-valued ones without decimals).""" + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, float): + return str(int(value)) if value.is_integer() else f"{value:g}" + if isinstance(value, (list, tuple)): + return ", ".join(_text(v) for v in value) + if isinstance(value, Path): + return value.as_posix() + return str(value) + + +def _line(key: str, value: Any, origin: str) -> str: + if value is None: + return f"{key} = [not set]" + return f"{key} = {_text(value)} [{origin}]" + + +#: The settings of a stage that are shown when set (file) or not at their default (preset). +_STAGE_DEFAULTS = {"criteria": "max", "top_origin_limit": 4, "bottom_origin_limit": 6, "wildcard_limit": 3} + + +def _stage_line(i: int, stage, shown: list[str], origin: str) -> str: + parts = [f"{stage.action}: {', '.join(stage.sources)}"] + parts.extend(f"{k}={_text(getattr(stage, k))}" for k in _STAGE_DEFAULTS if k in shown) + return f"kcat_assignment.stage[{i}] = {'; '.join(parts)} [{origin}]" + + +def format_adapter_params(adapter: "ModelAdapter") -> str: + """The resolved parameters of ``adapter`` as text (see :func:`show_adapter_params`).""" + from ..gather_kcats.kcat_assignment import DEFAULT_OKP_METHODS, kcat_assignment_config + + params = adapter.params + config = kcat_assignment_config(adapter) + preset_origin = f"preset {config.preset}" if config.preset else "file" + out: list[str] = [] + for key in _keys(): + owner: Any = params + *parents, leaf = key.split(".") + for name in parents: + owner = getattr(owner, name) + value = getattr(owner, leaf) + origin = "file" if leaf in owner.model_fields_set else "default" + if key == "kcat_assignment.preset": + if params.kcat_assignment.stage: + out.append("kcat_assignment.preset = none (stages are given) [file]") + else: + out.append(_line(key, config.preset, origin)) + elif key == "kcat_assignment.stage": + for i, stage in enumerate(config.stages, start=1): + if params.kcat_assignment.stage: + shown = [k for k in _STAGE_DEFAULTS if k in params.kcat_assignment.stage[i - 1].model_fields_set] + else: + shown = [k for k, d in _STAGE_DEFAULTS.items() if getattr(stage, k) != d] + out.append(_stage_line(i, stage, shown, "file" if params.kcat_assignment.stage else preset_origin)) + elif key == "kcat_sources": + sources = {name: dict(keys) for name, keys in config.sources.items()} + if "methods" not in sources.get("okp", {}): + sources.setdefault("okp", {})["methods"] = list(DEFAULT_OKP_METHODS) + order = ["brenda", "ec_tree", "homologs", "okp"] + for name in sorted(sources, key=lambda n: (order.index(n), "") if n in order else (len(order), n)): + keys = sources[name] + for k in sorted(keys): + v = keys[k] + if k in params.kcat_sources.get(name, {}): + where = "file" + elif name == "okp" and k == "methods": + where = "default" + else: + where = preset_origin + out.append(_line(f"kcat_sources.{name}.{k}", v, where)) + elif key == "evotune.source_groups": + for name, rule in value.items(): + out.append(f"evotune.source_groups.{name} = {_text(rule.sources)}; sigma0_log {_text(rule.sigma0_log)} [{origin}]") + else: + out.append(_line(key, value, origin)) + return "\n".join(out) + + +def show_adapter_params(adapter: "ModelAdapter") -> None: + """Print every adapter parameter with its value and where it comes from. + + One line per item, in the order of ``adapterParameters.tsv``: + `` = []``. The origin is ``file`` (set in + model_adapter.toml), ``default``, ``preset `` (given by the kcat + assignment preset) or ``not set`` (no value: `` = [not set]``). + The kcat assignment stages (``kcat_assignment.stage[]``), the + ``kcat_sources..`` settings and the evotune source groups, in + matching order with their ``sigma0_log``, are shown as resolved. + """ + print(format_adapter_params(adapter)) diff --git a/src/geckopy/adapter/template.py b/src/geckopy/adapter/template.py index f90cb11..9acd688 100644 --- a/src/geckopy/adapter/template.py +++ b/src/geckopy/adapter/template.py @@ -38,6 +38,24 @@ ], } +# Written last: [kcat_sources.] tables follow every other section's keys. +_KCAT_SOURCES_EXAMPLE = [ + "# Settings per kcat source. A preset sets the keys it needs; a table here", + "# overrides only the keys it sets. The keys each source reads:", + "# [kcat_sources.brenda]", + '# aggregation = "median" # "max" or "median"', + "# force_wildcard_level = 0", + "# [kcat_sources.ec_tree]", + "# k = 5", + "# [kcat_sources.homologs]", + "# k = 5", + "# min_identity = 0.4", + "# coverage = 0.8", + "# [kcat_sources.okp]", + '# methods = ["KinForm-H"] # the first is the default method of kcats_from_okp', + "", +] + def _format_toml_value(value: Any) -> str | None: """Format a Python value as a TOML literal, or None if unrepresentable.""" @@ -138,7 +156,7 @@ def generate_template_toml(*, project_name: str = "my-ecmodel") -> str: sections: list[tuple[str, type[BaseModel]]] = [] for name, field in ModelParameters.model_fields.items(): - if name in _INJECTED_FIELDS: + if name in _INJECTED_FIELDS or name == "kcat_sources": continue if _is_nested_model(field.annotation): sections.append((name, field.annotation)) @@ -164,7 +182,12 @@ def generate_template_toml(*, project_name: str = "my-ecmodel") -> str: for name, model_cls in sections: lines.append(f"[{name}]") instance = model_cls() + tables: list[tuple[str, Any]] = [] for sub_name, sub_field in model_cls.model_fields.items(): + value = getattr(instance, sub_name) + if isinstance(value, dict) and value and all(isinstance(v, BaseModel) for v in value.values()): + tables.append((sub_name, sub_field)) # written last: sub-tables follow the section's keys + continue if (name, sub_name) in _SET_IN_TEMPLATE: value = _format_toml_value(getattr(instance, sub_name)) lines.extend([*_SET_IN_TEMPLATE[(name, sub_name)], f"{sub_name} = {value}", ""]) @@ -175,5 +198,12 @@ def generate_template_toml(*, project_name: str = "my-ecmodel") -> str: default_value=getattr(instance, sub_name), required=False, )) - + for sub_name, sub_field in tables: + lines.append(f"# {sub_field.description}") + for group, rule in getattr(instance, sub_name).items(): + lines.append(f"# [{name}.{sub_name}.{group}]") + lines.extend(f"# {key} = {_format_toml_value(val)}" for key, val in rule.model_dump().items()) + lines.append("") + + lines.extend(_KCAT_SOURCES_EXAMPLE) return "\n".join(lines) + "\n" diff --git a/src/geckopy/cli.py b/src/geckopy/cli.py index 24c73ec..b911b7e 100644 --- a/src/geckopy/cli.py +++ b/src/geckopy/cli.py @@ -143,6 +143,14 @@ def cmd_upgrade(args: argparse.Namespace) -> int: return 0 +def cmd_adapter_show(args: argparse.Namespace) -> int: + """Print every parameter of the project's adapter, with its value and origin.""" + from .adapter import ModelAdapter, show_adapter_params + + show_adapter_params(ModelAdapter.from_folder(args.folder)) + return 0 + + def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser( prog="geckopy", description="Command-line interface for geckopy." @@ -227,6 +235,15 @@ def main(argv: list[str] | None = None) -> int: ) upgrade_parser.set_defaults(func=cmd_upgrade) + adapter_parser = sub.add_parser("adapter", help="Inspect the model adapter of a project") + adapter_sub = adapter_parser.add_subparsers(dest="adapter_command", required=True) + show_parser = adapter_sub.add_parser( + "show", + help="Print every parameter of model_adapter.toml with its value and origin (file, preset or default)", + ) + show_parser.add_argument("folder", nargs="?", default=".", help="Project folder (default: the current folder)") + show_parser.set_defaults(func=cmd_adapter_show) + args = parser.parse_args(argv) return args.func(args) diff --git a/src/geckopy/databases/brenda/__init__.py b/src/geckopy/databases/brenda/__init__.py index 9b409f3..421f4ce 100644 --- a/src/geckopy/databases/brenda/__init__.py +++ b/src/geckopy/databases/brenda/__init__.py @@ -1,6 +1,6 @@ """Refresh the BRENDA-derived kcat / sa / mw TSVs from the BRENDA bulk JSON. -See ``docs/internal/brenda_refresh_plan.md`` for the design. +See ``dev/brenda_refresh_plan.md`` for the design. """ from __future__ import annotations diff --git a/src/geckopy/databases/brenda_loader.py b/src/geckopy/databases/brenda_loader.py index 82aeefa..3838037 100644 --- a/src/geckopy/databases/brenda_loader.py +++ b/src/geckopy/databases/brenda_loader.py @@ -78,9 +78,7 @@ def kcat_for(self, aggregation: Literal["max", "median"]) -> pd.DataFrame: return self.kcat_median if aggregation == "max": return self.kcat_max - raise ValueError( - f"aggregation must be 'max' or 'median', got {aggregation!r}" - ) + raise ValueError("aggregation must be max or median") def sa_for(self, aggregation: Literal["max", "median"]) -> pd.DataFrame: """Return the SA-derived-kcat view for the requested aggregation.""" @@ -88,9 +86,7 @@ def sa_for(self, aggregation: Literal["max", "median"]) -> pd.DataFrame: return self.sa_median if aggregation == "max": return self.sa_max - raise ValueError( - f"aggregation must be 'max' or 'median', got {aggregation!r}" - ) + raise ValueError("aggregation must be max or median") def load_brenda_data( @@ -118,8 +114,8 @@ def load_brenda_data( Both kcat and SA are loaded as separate max and median views over the raw measurements aggregated per (ec, substrate, organism) triple, so a project can opt into median aggregation via - ``adapter.params.kcat_aggregate_brenda`` without regenerating the - snapshot. See ``docs/kcat_aggregation.md`` for the rationale. + ``[kcat_sources.brenda] aggregation`` without regenerating the + snapshot. See https://gecko-docs.readthedocs.io/en/latest/guide/kcat-sources.html for the rationale. Malformed lines (wrong number of fields, non-numeric value column) are skipped with a ``logger.warning``; the rest of the file is diff --git a/src/geckopy/ec_model/make_ec_model.py b/src/geckopy/ec_model/make_ec_model.py index 65d90d0..39e22c5 100644 --- a/src/geckopy/ec_model/make_ec_model.py +++ b/src/geckopy/ec_model/make_ec_model.py @@ -151,7 +151,7 @@ def make_ec_model( in ``ec.rxns`` distinguished by a 3-digit counter prefix (e.g. ``001_RXN1``, ``002_RXN1``). Suitable for large GEMs (HumanGEM, Recon3D) where the full layout's LP size becomes - impractical. See ``docs/gecko_light_status.md``. + impractical. See ``dev/gecko_light_status.md``. uniprot_db Pre-loaded UniprotDB. If None, the function loads ``adapter.params.path / "data" / "uniprot.tsv"``, and first diff --git a/src/geckopy/ec_model/pipeline/fill_kcats.py b/src/geckopy/ec_model/pipeline/fill_kcats.py index 576ea1c..584155f 100644 --- a/src/geckopy/ec_model/pipeline/fill_kcats.py +++ b/src/geckopy/ec_model/pipeline/fill_kcats.py @@ -7,7 +7,7 @@ import logging import re -from typing import TYPE_CHECKING, Optional +from typing import TYPE_CHECKING import numpy as np @@ -32,7 +32,7 @@ def fill_kcats_from_isozymes( model: "EcModel", *, apply: bool = False, - aggregate: Optional[str] = None, + aggregate: str = "mean", ) -> None: """Fill in missing kcats by combining known kcats across sibling isozymes. @@ -67,9 +67,7 @@ def fill_kcats_from_isozymes( aggregate How to combine sibling kcats: ``"mean"`` (matches MATLAB GECKO), ``"median"`` (more robust for log-distributed rate constants), or - ``"max"``. ``None`` (default) reads - ``model.adapter.params.kcat_aggregate_isozymes`` if an adapter is - attached, falling back to ``"mean"`` otherwise. + ``"max"``. Default ``"mean"``. Raises ------ @@ -83,13 +81,6 @@ def fill_kcats_from_isozymes( "fill_kcats_from_isozymes: not applicable to gecko-light models." ) - if aggregate is None: - adapter = getattr(model, "adapter", None) - aggregate = ( - adapter.params.kcat_aggregate_isozymes - if adapter is not None else "mean" - ) - kcat = model.ec.kcat known_mask = kcat > 0 missing_mask = kcat == 0 diff --git a/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py b/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py index 4fc0573..489808a 100644 --- a/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py +++ b/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py @@ -156,15 +156,15 @@ def fuzzy_kcat_matching( How to collapse the BRENDA rows matched at a search level to one kcat: ``"max"`` (the highest reported turnover) or ``"median"`` (more robust to assay outliers / engineered mutants). ``None`` - (default) reads the value from - ``model.adapter.params.kcat_aggregate_brenda``, which itself - defaults to ``"max"``. + (default) reads ``[kcat_sources.brenda] aggregation`` of the adapter + as the kcat assignment resolves it: median for the default preset + ``brenda_ec_tree``, max for ``gecko_classic``. The choice sets both aggregation layers consistently: it selects the matching snapshot view (``brenda.kcat_for(aggregate)`` / ``brenda.sa_for(aggregate)``) and applies the same reduction across whichever rows survive the EC + organism + substrate - gates. See ``docs/kcat_aggregation.md`` for the empirical + gates. See https://gecko-docs.readthedocs.io/en/latest/guide/kcat-sources.html for the empirical rationale. adapter Model adapter for ``org_name`` and the aggregation default @@ -199,7 +199,9 @@ def fuzzy_kcat_matching( ) if aggregate is None: - aggregate = adapter.params.kcat_aggregate_brenda + from ..kcat_assignment import brenda_aggregation + + aggregate = brenda_aggregation(adapter) organism_name = (adapter.params.org_name or "").strip() n = model.ec.n_rxns diff --git a/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py b/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py index 102b89f..2f20edf 100644 --- a/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py +++ b/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py @@ -41,9 +41,7 @@ # 1: org+subs+kcat 2: any+subs+kcat 3: org+!subs+kcat # 4: any+!subs+kcat 5: org+SA 6: any+SA # but the SEARCH order tries org-SA (output 5) BEFORE any-no-subs-kcat -# (output 4), matching MATLAB exactly -- see "Fix the search-order/ -# output-ranking inconsistency in fuzzyKcatMatching" in -# docs/future_improvements.md. +# (output 4), matching MATLAB exactly. SEARCH_LEVELS: list[tuple[bool, bool, bool, int]] = [ (True, True, False, 1), (False, True, False, 2), @@ -125,7 +123,7 @@ def _aggregate_kcats(kcats: np.ndarray, how: str) -> float: return float(np.median(kcats)) if how == "max": return float(np.max(kcats)) - raise ValueError(f"aggregate must be 'max' or 'median', got {how!r}") + raise ValueError("aggregation must be max or median") def match_kcat( diff --git a/src/geckopy/gather_kcats/kcat_assignment.py b/src/geckopy/gather_kcats/kcat_assignment.py index d75530c..37f6293 100644 --- a/src/geckopy/gather_kcats/kcat_assignment.py +++ b/src/geckopy/gather_kcats/kcat_assignment.py @@ -10,7 +10,6 @@ import json import logging -import re from dataclasses import dataclass, field from pathlib import Path from typing import TYPE_CHECKING, Any, Optional @@ -22,7 +21,14 @@ from ..ec_model.pipeline.fill_kcats import fill_kcats_from_isozymes from .get_standard_kcat import assign_standard_kcat from .kcat_source_format import format_kcat_source, is_standard_source -from .kcat_sources import load_kcat_source, read_kcat_source +from .kcat_sources import ( + DEFAULT_OKP_METHODS, + check_okp_methods, + load_kcat_source, + okp_file, + okp_method_of_file, + read_kcat_source, +) from .kcats_from import _brenda_release, _measurements, kcats_from_brenda, kcats_from_dlkcat, kcats_from_ec_tree, kcats_from_okp from .merge_kcats import _strip_okp_prefix, merge_kcats, normalize_source from .select_kcat_value import apply_kcat_list @@ -36,20 +42,6 @@ PRESETS = ("brenda_ec_tree", "ec_tree_homologs", "ec_tree_okp", "okp", "gecko_median", "gecko_classic") _DATABASE_TOKENS = {"brenda", "sabio_rk"} -OKP_METHODS = ("CataPro", "CatPred", "CatRange", "DLKcat", "EITLEM", "KinForm-H", "KinForm-L", - "OmniESI", "OmniESI-O2DENet", "TurNup", "UniKP") -#: OKP methods that read whole reactions (substrate set, and products for TurNup) -#: instead of enzyme-substrate pairs; kcats_from_okp submits pairs only. -REACTION_OKP_METHODS = ("CatPred", "TurNup") -#: The OKP methods of the ``okp`` source group by default; a project lists others in -#: ``[kcat_sources.okp] methods`` (those that read enzyme-substrate pairs and have -#: combination parameters: not CatRange, which predicts classes, nor the reaction methods). -DEFAULT_OKP_METHODS = ("KinForm-H",) - - -def okp_file(method: str) -> str: - """The source file name of an OKP method, e.g. ``okp_kinform_h``.""" - return "okp_" + re.sub(r"[^A-Za-z0-9]+", "_", method).lower().strip("_") @dataclass @@ -126,13 +118,26 @@ def _source_file(token: str) -> str: def okp_methods(config: KcatAssignmentConfig) -> list[str]: """The OKP methods of the ``okp`` source group (``[kcat_sources.okp] methods``).""" methods = config.sources.get("okp", {}).get("methods") or list(DEFAULT_OKP_METHODS) - unknown = [m for m in methods if m not in OKP_METHODS] - if unknown: - raise ValueError(f"Unknown OKP methods in [kcat_sources.okp] methods: {unknown}; " - f"methods: {', '.join(OKP_METHODS)}.") + check_okp_methods(methods) return list(methods) +def brenda_aggregation(adapter: "ModelAdapter") -> str: + """The BRENDA aggregation of the adapter: ``[kcat_sources.brenda] aggregation``. + + As the kcat assignment config resolves it: the preset's value (median for + ``brenda_ec_tree`` and the other median presets, max for ``gecko_classic``) + unless the file sets it. ``max`` when neither the preset nor the file sets it + (the ``okp`` preset, or stages without a ``[kcat_sources.brenda]`` table). + """ + return str(kcat_assignment_config(adapter).sources.get("brenda", {}).get("aggregation", "max")) + + +def default_okp_method(adapter: "ModelAdapter") -> str: + """The OKP method used when none is given: the first of ``[kcat_sources.okp] methods``.""" + return okp_methods(kcat_assignment_config(adapter))[0] + + def _expand(tokens: list[str], config: KcatAssignmentConfig) -> list[str]: """Stage tokens with the ``okp`` group replaced by its methods' files.""" out: list[str] = [] @@ -409,15 +414,6 @@ def assign_kcats( return decisions, report -def _okp_method(name: str, wanted: dict[str, Any]) -> str: - if "method" in wanted: - return str(wanted["method"]) - for m in OKP_METHODS: - if "okp_" + re.sub(r"[^A-Za-z0-9]+", "_", m).lower() == name: - return m - raise ValueError(f'Set the OKP method of {name} in model_adapter.toml: [kcat_sources.{name}] method = "..."') - - def _same(a: Any, b: Any) -> bool: def norm(v: Any) -> Any: return v.lower() if isinstance(v, str) else (int(v) if isinstance(v, bool) else v) @@ -508,7 +504,7 @@ def _gather(model: "EcModel", name: str, wanted: dict[str, Any], adapter: "Model if not done: return "pending", "OKP job for DLKcat still running; call again later" else: - method = _okp_method(name, wanted) + method = okp_method_of_file(name) _, done = kcats_from_okp(model, method=method, save_as=name, adapter=adapter) if not done: return "pending", f"OKP job for {method} still running; call again later" diff --git a/src/geckopy/gather_kcats/kcat_sources.py b/src/geckopy/gather_kcats/kcat_sources.py index 2731024..246a70f 100644 --- a/src/geckopy/gather_kcats/kcat_sources.py +++ b/src/geckopy/gather_kcats/kcat_sources.py @@ -40,6 +40,48 @@ _NUMBER_RE = re.compile(r"^[-+]?([0-9]+\.?[0-9]*|\.[0-9]+)([eE][-+]?[0-9]+)?$|^[-+]?Inf$") _META_RE = re.compile(r"^\s*#\s*([A-Za-z][A-Za-z0-9_]{0,62})\s*:\s?(.*)$", re.ASCII) +#: The OKP methods that ``[kcat_sources.okp] methods`` and ``kcats_from_okp`` accept. +#: The reaction methods CatPred and TurNup are left out: they read whole reactions +#: (substrate set, and products for TurNup), while ``kcats_from_okp`` submits +#: enzyme-substrate pairs. +OKP_METHODS = ("CataPro", "CatRange", "DLKcat", "EITLEM", "KinForm-H", "KinForm-L", + "OmniESI", "OmniESI-O2DENet", "UniKP") +#: The OKP methods of the ``okp`` source group by default; a project lists others in +#: ``[kcat_sources.okp] methods`` (those that have combination parameters: not +#: CatRange, which predicts classes). +DEFAULT_OKP_METHODS = ("KinForm-H",) + + +def okp_file(method: str) -> str: + """The source file name of an OKP method, e.g. ``okp_kinform_h``. + + ``okp_`` and the method in lower case, with each run of characters other + than letters and digits as ``_`` and no ``_`` at either end. + """ + return "okp_" + re.sub(r"[^A-Za-z0-9]+", "_", method).lower().strip("_") + + +def check_okp_methods(methods: list[str]) -> None: + """Raise ``ValueError`` naming the entries of ``methods`` that are no known OKP method. + + The names are in the order given, each once. + """ + unknown = list(dict.fromkeys(m for m in methods if m not in OKP_METHODS)) + if unknown: + raise ValueError(f"Unknown OKP methods in [kcat_sources.okp] methods: {', '.join(unknown)}; " + f"methods: {', '.join(OKP_METHODS)}.") + + +def okp_method_of_file(name: str) -> str: + """The OKP method whose source file is ``name``, e.g. ``KinForm-H`` for ``okp_kinform_h``.""" + for method in OKP_METHODS: + if okp_file(method) == name: + return method + raise ValueError( + f"{name} is not the file of a known OKP method; the files are " + f"{', '.join(okp_file(m) for m in OKP_METHODS)}." + ) + def ec_rxn_keys(model: "EcModel") -> tuple[list[str], list[str]]: """Build-independent keys of the ``model.ec`` reactions. diff --git a/src/geckopy/gather_kcats/kcats_from.py b/src/geckopy/gather_kcats/kcats_from.py index 65e036b..a4f6433 100644 --- a/src/geckopy/gather_kcats/kcats_from.py +++ b/src/geckopy/gather_kcats/kcats_from.py @@ -22,7 +22,7 @@ from ..databases import load_brenda_data, load_brenda_measurements, load_dlkcat_ignore_lists, load_phyl_dist from .ec_tree import ECTree from .fuzzy_kcat_matching import fuzzy_kcat_matching -from .kcat_sources import format_number, kcat_source_file, save_kcat_source +from .kcat_sources import format_number, kcat_source_file, okp_file, save_kcat_source from .open_kinetics_predictor.run import _fetch_open_kinetics_predictor, _submit_open_kinetics_predictor from .open_kinetics_predictor.parse_output import _parse_okp_output from .read_dlkcat_output import _read_dlkcat_output @@ -91,7 +91,9 @@ def kcats_from_brenda( EcModel with EC numbers. aggregation Which BRENDA value per EC number, substrate and organism is used: - ``"max"`` or ``"median"`` (default: ``params.kcat_aggregate_brenda``). + ``"max"`` or ``"median"`` (default: ``[kcat_sources.brenda] aggregation`` + as the kcat assignment resolves it: median for the default preset + ``brenda_ec_tree``, max for ``gecko_classic``). force_wildcard_level Force a minimum wildcard level in the EC numbers. ec_rxns @@ -113,10 +115,12 @@ def kcats_from_brenda( """ adapter = _adapter(model, adapter, "kcats_from_brenda needs the BRENDA folder and params") if aggregation is None: - aggregation = adapter.params.kcat_aggregate_brenda + from .kcat_assignment import brenda_aggregation + + aggregation = brenda_aggregation(adapter) aggregation = aggregation.lower() if aggregation not in ("max", "median"): - raise ValueError("aggregation must be 'max' or 'median'") + raise ValueError("aggregation must be max or median") ec_rxns = _rxn_subset(ec_rxns) if save: path = kcat_source_file(save_as, model=model, adapter=adapter) # fails before the matching @@ -417,8 +421,8 @@ def kcats_from_okp( model EcModel with metabolite SMILES and protein sequences. method - OKP method, e.g. ``"CataPro"``, ``"CatRange"``, ``"UniKP"`` (default: - ``params.okp.method``). + OKP method, e.g. ``"CataPro"``, ``"CatRange"``, ``"UniKP"`` (default: the + first of ``[kcat_sources.okp] methods``, ``"KinForm-H"`` unless set). wait Wait until the job is done. resubmit @@ -442,16 +446,19 @@ def kcats_from_okp( """ adapter = _adapter(model, adapter, "kcats_from_okp needs params.path") if not method: - method = adapter.params.okp.method + from .kcat_assignment import default_okp_method + + method = default_okp_method(adapter) if method in ("CatPred", "TurNup"): raise ValueError(f"{method} reads whole reactions (substrate set{', and products' if method == 'TurNup' else ''}); " "kcats_from_okp submits enzyme-substrate pairs only.") ec_rxns = _rxn_subset(ec_rxns) - tag = re.sub(r"[^A-Za-z0-9]+", "_", method).lower() + name_default = okp_file(method) + tag = name_default[len("okp_"):] data_dir = Path(adapter.params.path) / "data" job_file = data_dir / f"OKP_job_{tag}.txt" out_file = data_dir / f"OKP_output_{tag}.csv" - name = save_as or f"okp_{tag}" + name = save_as or name_default job = _read_job_file(job_file) submitting = not use_stored and (resubmit or "jobId" not in job) if ec_rxns is not None and not submitting: diff --git a/src/geckopy/gather_kcats/open_kinetics_predictor/run.py b/src/geckopy/gather_kcats/open_kinetics_predictor/run.py index 1719d32..c4457bf 100644 --- a/src/geckopy/gather_kcats/open_kinetics_predictor/run.py +++ b/src/geckopy/gather_kcats/open_kinetics_predictor/run.py @@ -70,8 +70,8 @@ def _submit_open_kinetics_predictor( OKP API key. If omitted, resolved from ``OKP_API_KEY`` or ``data/okpApiKey.txt`` (see the module docstring). method - Predictor method. Resolved as: this arg -> ``params.okp.method`` - -> ``'KinForm-H'``. + Predictor method. Resolved as: this arg -> the first of + ``[kcat_sources.okp] methods`` (``'KinForm-H'`` unless set). ec_rxns Optional iterable of reaction IDs to include (default all). only_with_smiles @@ -96,7 +96,10 @@ def _submit_open_kinetics_predictor( ) data_dir = Path(adapter.params.path) / "data" api_key = _resolve_api_key(api_key, adapter) - method = method or adapter.params.okp.method + if not method: + from ..kcat_assignment import default_okp_method + + method = default_okp_method(adapter) okp = adapter.params.okp okp_csv = data_dir / "OKP.csv" diff --git a/src/geckopy/gather_kcats/select_kcat_value.py b/src/geckopy/gather_kcats/select_kcat_value.py index e61fcb1..53eaa0b 100644 --- a/src/geckopy/gather_kcats/select_kcat_value.py +++ b/src/geckopy/gather_kcats/select_kcat_value.py @@ -5,7 +5,7 @@ """ from __future__ import annotations -from typing import TYPE_CHECKING, Literal, Optional, Union +from typing import TYPE_CHECKING, Literal, Union import numpy as np import pandas as pd @@ -38,7 +38,7 @@ def apply_kcat_list( model: "EcModel", kcat_list: pd.DataFrame, *, - criteria: Optional[Literal["max", "min", "median", "mean"]] = None, + criteria: Literal["max", "min", "median", "mean"] = "max", overwrite: _OverwriteMode = True, ) -> list[str]: """Choose one kcat value per reaction and write it to ``model.ec.kcat``. @@ -81,10 +81,7 @@ def apply_kcat_list( ``criteria``. criteria How to aggregate multiple candidates per reaction: - ``"max"``, ``"min"``, ``"median"``, or ``"mean"``. ``None`` - (default) reads - ``model.adapter.params.kcat_aggregate_candidates`` if an adapter - is attached, falling back to ``"max"`` otherwise. + ``"max"`` (default), ``"min"``, ``"median"``, or ``"mean"``. overwrite ``True`` (default) overwrites unconditionally. ``False`` only fills entries currently 0 or NaN. ``"if_higher"`` only @@ -105,12 +102,6 @@ def apply_kcat_list( If ``kcat_list`` is missing the ``rxn_id``, ``kcat``, or ``source`` column. """ - if criteria is None: - adapter = getattr(model, "adapter", None) - criteria = ( - adapter.params.kcat_aggregate_candidates - if adapter is not None else "max" - ) if criteria not in _CRITERIA: raise ValueError( f"criteria must be one of {_CRITERIA}; got {criteria!r}" diff --git a/src/geckopy/kcat_tuning/evotune/__init__.py b/src/geckopy/kcat_tuning/evotune/__init__.py index abb3672..9fc8530 100644 --- a/src/geckopy/kcat_tuning/evotune/__init__.py +++ b/src/geckopy/kcat_tuning/evotune/__init__.py @@ -1,6 +1,6 @@ """Evotune: CMA-ES kcat tuning against experimental data. -See ``docs/evotune_kcat_tuning.md`` for the walkthrough: +See https://gecko-docs.readthedocs.io/en/latest/guide/evotune.html for the walkthrough: :func:`~.tuning.screen_kcat_leverage`, then :func:`~.tuning.select_tunable_mask`, then :func:`~.tuning.cmaes_kcat_tuning`. diff --git a/src/geckopy/kcat_tuning/evotune/mask_size.py b/src/geckopy/kcat_tuning/evotune/mask_size.py index 65cd6d7..76f0ee8 100644 --- a/src/geckopy/kcat_tuning/evotune/mask_size.py +++ b/src/geckopy/kcat_tuning/evotune/mask_size.py @@ -95,7 +95,6 @@ def sweep_tunable_mask_size( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional["EvotuneData"] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -157,7 +156,7 @@ def sweep_tunable_mask_size( screen A precomputed :func:`~.tuning.screen_kcat_leverage` report. Computed once, fresh, if not given. - adapter, params, evotune_data, okp_method, bio_rxn, make_anaerobic, + adapter, params, evotune_data, bio_rxn, make_anaerobic, change_protein_biomass, n_proc, verbose Forwarded to :func:`~.tuning.screen_kcat_leverage` and :func:`~.tuning.cmaes_kcat_tuning`; see their docstrings. @@ -167,13 +166,13 @@ def sweep_tunable_mask_size( list of :class:`MaskSizePoint`, one per share, sorted by ``target_impact_share`` ascending. """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) if screen is None: screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, n_proc=n_proc, ) @@ -187,7 +186,7 @@ def sweep_tunable_mask_size( run_model = model.copy() result = cmaes_kcat_tuning( run_model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, tunable_mask=mask, n_proc=n_proc, seed=seed, verbose=False, ) diff --git a/src/geckopy/kcat_tuning/evotune/priors.py b/src/geckopy/kcat_tuning/evotune/priors.py index 6ae356e..7c2372f 100644 --- a/src/geckopy/kcat_tuning/evotune/priors.py +++ b/src/geckopy/kcat_tuning/evotune/priors.py @@ -2,31 +2,32 @@ """ from __future__ import annotations -from typing import Optional +import logging +import warnings +from collections import Counter import numpy as np from ...adapter.params import EvotuneParams from ...gather_kcats.kcat_source_format import parse_kcat_source +logger = logging.getLogger(__name__) + #: Sentinel group name for an ``ec.source`` value matched by no #: ``source_groups`` entry -- MATLAB's ``noKcatSource``. UNLABELLED_GROUP = "unlabelled" -def classify_kcat_source( - source: str, - params: EvotuneParams, - *, - okp_method: Optional[str] = None, -) -> str: +def classify_kcat_source(source: str, params: EvotuneParams) -> str: """Map one ``ec.source`` string to a trust-tier group name. - Matching is case-insensitive, mirroring MATLAB's - ``strcmpi(ecModel.ec.source, kcatSources{i})``. A ``source_groups`` entry - matches the origin of the source (``"brenda"`` for + Matching is case-insensitive. An entry of ``sources`` without a trailing + ``*`` matches the origin of the source (``"brenda"`` for ``"brenda (wc=0, origin=1) | tuned=cmaes (prior=2)"``, so the detail and - the tuning status are ignored) and also the whole source string. + the tuning status are ignored) or the whole source string; these exact + matches are tried over all groups first. Then the entries ending in + ``*``, prefix patterns on the origin, are tried, the longest prefix first. + Entries equal in length keep the order of the groups. Parameters ---------- @@ -34,76 +35,81 @@ def classify_kcat_source( One ``ec.source`` entry. params Supplies ``source_groups``. - okp_method - The project's configured OpenKineticsPredictor method - (``OkpParams.method``), if any. A group with ``match_okp=True`` - also matches ``source`` when it equals this (case-insensitive). Returns ------- str A key of ``params.source_groups``, or :data:`UNLABELLED_GROUP` if nothing matches -- callers then fall back to - ``params.sigma0_log_default`` / ``shrink_thr_default`` / - ``force_prior_thr_default``. + ``params.sigma0_log_default``. """ source_lower = source.lower() - token = parse_kcat_source(source).origin.lower() - forms = {source_lower, token} - for name, rule in params.source_groups.items(): - if any(s.lower() in forms for s in rule.sources): - return name - if ( - rule.match_okp - and okp_method is not None - and okp_method.lower() in forms - ): + origin = parse_kcat_source(source).origin.lower() + forms = {source_lower, origin} + patterns: list[tuple[int, int, str, str]] = [] + for position, (name, rule) in enumerate(params.source_groups.items()): + for entry in rule.sources: + entry = entry.lower() + if entry.endswith("*"): + patterns.append((-(len(entry) - 1), position, name, entry[:-1])) + elif entry in forms: + return name + for _, _, name, prefix in sorted(patterns): + if origin.startswith(prefix): return name return UNLABELLED_GROUP -def classify_kcat_sources( - ec_sources: list[str], - params: EvotuneParams, - *, - okp_method: Optional[str] = None, -) -> np.ndarray: +def classify_kcat_sources(ec_sources: list[str], params: EvotuneParams) -> np.ndarray: """Vectorised :func:`classify_kcat_source` over ``ec.source``. Returns ------- numpy.ndarray of str, shape ``(len(ec_sources),)`` """ - return np.array( - [ - classify_kcat_source(s, params, okp_method=okp_method) - for s in ec_sources - ], - dtype=object, - ) + return np.array([classify_kcat_source(s, params) for s in ec_sources], dtype=object) def build_sigma0_log(groups: np.ndarray, params: EvotuneParams) -> np.ndarray: """Per-row prior std dev in log-space, from source-group membership. - Mirrors MATLAB's ``sigma0log = sigma0logDefault * ones(...); - sigma0log(uniqKcatParams) = sigma0logSource(kcatSourceIdx(...))``: - every row starts at ``params.sigma0_log_default``, then rows whose - group is a real (non-:data:`UNLABELLED_GROUP`) entry get - overridden by that group's ``sigma0_log_source`` value. + Every row starts at ``params.sigma0_log_default``, then rows whose + group is a real (non-:data:`UNLABELLED_GROUP`) entry get that group's + ``sigma0_log``, a standard deviation of the natural log of the kcat. Parameters ---------- groups Per-row group names from :func:`classify_kcat_sources`. params - Supplies ``sigma0_log_default`` and ``sigma0_log_source``. + Supplies ``sigma0_log_default`` and the groups' ``sigma0_log``. Returns ------- numpy.ndarray of float, same shape as ``groups``. """ sigma0_log = np.full(len(groups), params.sigma0_log_default, dtype=float) - for name, value in params.sigma0_log_source.items(): - sigma0_log[groups == name] = value + for name, rule in params.source_groups.items(): + sigma0_log[groups == name] = rule.sigma0_log return sigma0_log + + +def warn_unlabelled( + groups: np.ndarray, sources: list[str], params: EvotuneParams, *, max_fraction: float = 0.2, +) -> None: + """Warn when more than ``max_fraction`` of the tunable kcats are in no group. + + The kcats then take ``params.sigma0_log_default``. The warning names the + most common origins (up to five) among them, by count descending and then by name. + """ + unlabelled = [s for s, g in zip(sources, groups) if g == UNLABELLED_GROUP] + if len(groups) == 0 or len(unlabelled) <= max_fraction * len(groups): + return + counts = Counter(parse_kcat_source(s).origin for s in unlabelled) + origins = sorted(counts, key=lambda o: (-counts[o], o))[:5] + message = ( + f"{len(unlabelled)} of {len(groups)} tunable kcats match no evotune source group and get " + f"sigma0_log_default ({params.sigma0_log_default:g}); origins: {', '.join(origins)}" + ) + logger.warning(message) + warnings.warn(message, UserWarning, stacklevel=2) diff --git a/src/geckopy/kcat_tuning/evotune/tuning.py b/src/geckopy/kcat_tuning/evotune/tuning.py index a5efd9d..a7ae005 100644 --- a/src/geckopy/kcat_tuning/evotune/tuning.py +++ b/src/geckopy/kcat_tuning/evotune/tuning.py @@ -74,7 +74,7 @@ from .data import EvotuneData, load_evotune_data from .distance import evotune_distance, compute_excarbon from .parsimony import fold_change, n_changed -from .priors import build_sigma0_log, classify_kcat_sources +from .priors import build_sigma0_log, classify_kcat_sources, warn_unlabelled from .simulate import simulate_evotune_dataset from .tying import isozyme_tie_map @@ -159,7 +159,7 @@ def corrections( ) -def _resolve_context(model, adapter, params, evotune_data, bio_rxn, okp_method): +def _resolve_context(model, adapter, params, evotune_data, bio_rxn): """Resolve every optional input against ``model``'s adapter, or raise if it can't be and none was given explicitly.""" if adapter is None: @@ -186,9 +186,7 @@ def _resolve_context(model, adapter, params, evotune_data, bio_rxn, okp_method): if adapter is None: raise ValueError("bio_rxn not given and model has no adapter.") bio_rxn = adapter.params.bio_rxn - if okp_method is None and adapter is not None: - okp_method = adapter.params.okp.method - return params, evotune_data, bio_rxn, okp_method + return params, evotune_data, bio_rxn #: Termination criteria of ``cma`` that a flat objective triggers. @@ -196,7 +194,7 @@ def _resolve_context(model, adapter, params, evotune_data, bio_rxn, okp_method): def _tunable_context( - model, params, evotune_data, bio_rxn, okp_method, tunable_mask, + model, params, evotune_data, bio_rxn, tunable_mask, ): """The tunable subset plus everything derived from it: tie groups, trust weights, and the excarbon table. Shared by every function in @@ -221,7 +219,8 @@ def _tunable_context( ec_rxn_ids_tunable = [model.ec.rxns[i] for i in tunable_idx] kcat0 = model.ec.kcat[tunable_idx].astype(float).copy() sources = [model.ec.source[i] for i in tunable_idx] - groups = classify_kcat_sources(sources, params, okp_method=okp_method) + groups = classify_kcat_sources(sources, params) + warn_unlabelled(groups, sources, params) sigma0_log = build_sigma0_log(groups, params) excarbon_rxn_ids: set[str] = {bio_rxn} @@ -245,7 +244,6 @@ def screen_kcat_leverage( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional[EvotuneData] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -292,11 +290,11 @@ def screen_kcat_leverage( :func:`select_tunable_mask` needs to expand a row back into a mask; drop it before showing the table to a person. """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) (tunable_idx, ec_rxn_ids_tunable, kcat0, groups, sigma0_log, excarbon, tie_map) = _tunable_context( - model, params, evotune_data, bio_rxn, okp_method, tunable_mask) + model, params, evotune_data, bio_rxn, tunable_mask) reps = np.flatnonzero(tie_map == np.arange(len(tie_map))) members_of = {int(r): np.flatnonzero(tie_map == r) for r in reps} @@ -406,7 +404,6 @@ def cmaes_kcat_tuning( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional[EvotuneData] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -485,14 +482,14 @@ def cmaes_kcat_tuning( parameters remain after masking and tying -- too little for CMA-ES to search over. """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) if tunable_mask is None: if screen is None: screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, n_proc=n_proc, @@ -502,7 +499,7 @@ def cmaes_kcat_tuning( (tunable_idx, ec_rxn_ids_tunable, kcat0, groups, sigma0_log, excarbon, tie_map) = _tunable_context( - model, params, evotune_data, bio_rxn, okp_method, tunable_mask) + model, params, evotune_data, bio_rxn, tunable_mask) reps = np.flatnonzero(tie_map == np.arange(len(tie_map))) if len(reps) < 2: @@ -633,7 +630,6 @@ def tune_prior_penalty_weight( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional[EvotuneData] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -687,7 +683,7 @@ def tune_prior_penalty_weight( A kcat counts as "moved" past this fold change from its prior. Matches :func:`~.parsimony.fold_change`'s convention. tunable_mask, screen, target_impact_share, popsize, n_proc, - make_anaerobic, change_protein_biomass, okp_method, bio_rxn + make_anaerobic, change_protein_biomass, bio_rxn As in :func:`cmaes_kcat_tuning`. Returns @@ -704,14 +700,14 @@ def tune_prior_penalty_weight( ``median_fold_spread`` and ``max_fold_spread`` (among movers, the largest fold-change between what the two seeds landed on). """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) if tunable_mask is None: if screen is None: screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, n_proc=n_proc, @@ -738,7 +734,7 @@ def _reset(): _reset() result = cmaes_kcat_tuning( model, adapter=adapter, params=lam_params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, tunable_mask=tunable_mask, popsize=popsize, n_proc=n_proc, diff --git a/src/geckopy/kcat_tuning/sensitivity_tuning.py b/src/geckopy/kcat_tuning/sensitivity_tuning.py index acf18ee..a74e6b1 100644 --- a/src/geckopy/kcat_tuning/sensitivity_tuning.py +++ b/src/geckopy/kcat_tuning/sensitivity_tuning.py @@ -165,7 +165,7 @@ def sensitivity_tuning( installed and a populated ``ec.kcat``. Mutated in place. desired_growth_rate Target growth rate. Defaults to - ``model.adapter.params.gr_exp``. + ``model.adapter.params.gr_exp``; raises ``ValueError`` when neither is set. fold_change Per-iteration kcat multiplier. prot_to_ignore @@ -198,18 +198,12 @@ def sensitivity_tuning( light = bool(model.ec.gecko_light) if desired_growth_rate is None: - from ..adapter import resolve_adapter - adapter = resolve_adapter( + from ..adapter import resolve_gr_exp + desired_growth_rate = resolve_gr_exp( model, purpose="sensitivity_tuning needs `desired_growth_rate` " - "(pass it explicitly, or rely on params.gr_exp from the adapter)", + "(pass it explicitly, or set gr_exp in the adapter)", ) - if adapter.params.gr_exp is None: - raise ValueError( - "desired_growth_rate not provided and " - "model.adapter.params.gr_exp is unset." - ) - desired_growth_rate = float(adapter.params.gr_exp) if prot_to_ignore is None: prot_to_ignore = [] diff --git a/src/geckopy/kcat_tuning/sigma_fitter.py b/src/geckopy/kcat_tuning/sigma_fitter.py index e2aa0b7..c47ecb5 100644 --- a/src/geckopy/kcat_tuning/sigma_fitter.py +++ b/src/geckopy/kcat_tuning/sigma_fitter.py @@ -130,7 +130,7 @@ def fit_sigma( return. growth_rate Experimental growth rate to match. Defaults to - ``model.adapter.params.gr_exp``. + ``model.adapter.params.gr_exp``; raises ``ValueError`` when neither is set. p_tot Total protein content (g/gDCW). Defaults to ``model.adapter.params.p_tot``. @@ -174,7 +174,9 @@ def fit_sigma( params = model.adapter.params if model.adapter is not None else None if growth_rate is None: - growth_rate = float(params.gr_exp) + from ..adapter import resolve_gr_exp + + growth_rate = resolve_gr_exp(model) if not growth_rate > 0: # also NaN raise ValueError(f"growth_rate is {growth_rate}; fit_sigma needs a target growth rate above 0.") if p_tot is None: diff --git a/src/geckopy/limit_proteins/flexibilize_enz_concs.py b/src/geckopy/limit_proteins/flexibilize_enz_concs.py index 9f374bc..a9717fc 100644 --- a/src/geckopy/limit_proteins/flexibilize_enz_concs.py +++ b/src/geckopy/limit_proteins/flexibilize_enz_concs.py @@ -17,8 +17,9 @@ coefficient, relaxes gradually, and runs a tighten-back pass, so it yields the *minimal* (most proteomics-faithful) relaxation but is slower. ``relax_proteomics_greedy`` ranks by shadow price and fully -unconstrains each selected enzyme, which is faster but looser. See -``docs/relaxation_methods.md`` for the full comparison. +unconstrains each selected enzyme, which is faster but looser. The +guide compares the methods: +https://gecko-docs.readthedocs.io/en/latest/guide/relaxing-proteomics.html Ported from GECKO MATLAB: src/limit_proteins/flexibilizeEnzConcs.m. @@ -150,7 +151,7 @@ def flexibilize_enz_concs( itself is left unchanged. exp_growth Target growth rate. Defaults to - ``model.adapter.params.gr_exp``. + ``model.adapter.params.gr_exp``; raises ``ValueError`` when neither is set. fold_change Controls how much the limiting enzyme's usage upper bound grows each time it is selected: on its ``k``-th selection, the @@ -188,18 +189,12 @@ def flexibilize_enz_concs( flexibilize), or if ``model.adapter`` is None and ``exp_growth`` is also None. """ - from ..adapter import resolve_param - if exp_growth is None: - exp_growth = resolve_param( - model, None, "gr_exp", - purpose="flexibilize_enz_concs needs `exp_growth` (pass it " - "explicitly, or rely on params.gr_exp from the adapter)", - ) - if exp_growth is None: - raise ValueError( - "exp_growth not provided and params.gr_exp is unset." - ) - exp_growth = float(exp_growth) + from ..adapter import resolve_gr_exp, resolve_param + exp_growth = resolve_gr_exp( + model, exp_growth, + purpose="flexibilize_enz_concs needs `exp_growth` (pass it " + "explicitly, or set gr_exp in the adapter)", + ) if iter_per_enzyme == 0: iter_per_enzyme = math.inf diff --git a/src/geckopy/limit_proteins/relax_enz_concs.py b/src/geckopy/limit_proteins/relax_enz_concs.py index 703eea8..794cd8f 100644 --- a/src/geckopy/limit_proteins/relax_enz_concs.py +++ b/src/geckopy/limit_proteins/relax_enz_concs.py @@ -60,7 +60,7 @@ def relax_enz_concs( ``ec.concs`` itself is left unchanged, as in ``flexibilize_enz_concs``. exp_growth - Target growth rate. Defaults to ``params.gr_exp``. + Target growth rate. Defaults to ``params.gr_exp``; raises ``ValueError`` when neither is set. bio_rxn Biomass reaction id. Defaults to ``params.bio_rxn``. per_complex @@ -95,15 +95,11 @@ def relax_enz_concs( """ from optlang.symbolics import add - from ..adapter import resolve_param + from ..adapter import resolve_gr_exp, resolve_param if weight not in ("relative", "absolute"): raise ValueError(f"weight must be 'relative' or 'absolute', not {weight!r}") - if exp_growth is None: - exp_growth = resolve_param(model, None, "gr_exp", purpose="relax_enz_concs needs `exp_growth`") - if exp_growth is None: - raise ValueError("exp_growth not provided and params.gr_exp is unset.") - exp_growth = float(exp_growth) + exp_growth = resolve_gr_exp(model, exp_growth, purpose="relax_enz_concs needs `exp_growth`") bio = model.reactions.get_by_id( resolve_param(model, bio_rxn, "bio_rxn", purpose="relax_enz_concs needs the biomass reaction id") ) diff --git a/src/geckopy/limit_proteins/relax_proteomics_greedy.py b/src/geckopy/limit_proteins/relax_proteomics_greedy.py index e7840f0..1406721 100644 --- a/src/geckopy/limit_proteins/relax_proteomics_greedy.py +++ b/src/geckopy/limit_proteins/relax_proteomics_greedy.py @@ -17,8 +17,8 @@ dominate the infeasibility. The trade-off is that it fully unconstrains each selected enzyme with no tighten-back pass, so the result is looser (less proteomics-faithful) than -``flexibilize_enz_concs``. See ``docs/relaxation_methods.md`` for -the full comparison. +``flexibilize_enz_concs``. The guide compares the methods: +https://gecko-docs.readthedocs.io/en/latest/guide/relaxing-proteomics.html """ from __future__ import annotations diff --git a/tests/test_adapter.py b/tests/test_adapter.py index eae193b..ba8c4e5 100644 --- a/tests/test_adapter.py +++ b/tests/test_adapter.py @@ -70,7 +70,7 @@ def test_missing_toml_raises(tmp_path): def test_unknown_field_rejected(tmp_path): bad_toml = MINIMAL_TOML + '\nsgma = 0.3\n' # typo folder = _write_adapter_folder(tmp_path, bad_toml) - with pytest.raises(ValidationError): + with pytest.raises(ValueError, match="unknown key sgma in model_adapter.toml"): ModelAdapter.from_folder(folder) @@ -187,13 +187,11 @@ def test_spontaneous_reactions_from_toml(tmp_path): [evotune.source_groups.dlkcat] sources = ["dlkcat"] +sigma0_log = 0.4 [evotune.source_groups.brenda] sources = ["brenda"] - -[evotune.sigma0_log_source] -dlkcat = 0.4 -brenda = 0.2 +sigma0_log = 0.2 [matlab] make_model_anaerobic = "anaerobicModel_GECKO" @@ -268,7 +266,218 @@ def test_uniprot_table_lookup_reads_quoted_cells_and_first_duplicate(tmp_path): assert result == ["P12345", "P67890"] -def test_default_okp_method_is_kinform_h(): - from geckopy.adapter.params import ModelParameters +def test_okp_section_has_no_method(): + from geckopy.adapter.params import OkpParams + + assert "method" not in OkpParams.model_fields + + +@pytest.mark.parametrize("extra, key", [ + ("typo = 1\n", "typo"), + ("[kegg]\nidd = \"sce\"\n", "kegg.idd"), + ("[okp]\nmethod = \"CataPro\"\n", "okp.method"), + ("[evotune]\nmatch_okp = true\n", "evotune.match_okp"), + ("[evotune.source_groups.brenda]\nsources = [\"brenda\"]\nsigma0_log = 0.2\nmatch_okp = true\n", + "evotune.source_groups.brenda.match_okp"), + ("[evotune.sigma0_log_source]\nbrenda = 0.2\n", "evotune.sigma0_log_source"), + ("[[kcat_assignment.stage]]\nsources = [\"custom\"]\ncriterion = \"max\"\n", "kcat_assignment.stage.criterion"), + ("[kcat_sources.brenda]\naggregate = \"max\"\n", "kcat_sources.brenda.aggregate"), + ("[kcat_sources.okp_kinform_h]\nmethod = \"KinForm-H\"\n", "kcat_sources.okp_kinform_h.method"), + ("[kcat_sources.nosuchsource]\nk = 5\n", "kcat_sources.nosuchsource"), + ("[matlab]\nmake_model_anaerobicc = \"f\"\n", "matlab.make_model_anaerobicc"), + ("kcat_aggregate_brenda = \"max\"\n", "kcat_aggregate_brenda"), +]) +def test_an_unknown_key_is_an_error_naming_file_and_key(tmp_path, extra, key): + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + extra) + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == f"{folder.resolve() / 'model_adapter.toml'}: unknown key {key} in model_adapter.toml" + + +def test_known_kcat_source_keys_and_free_matlab_params_load(tmp_path): + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + """\ +[kcat_sources.brenda] +aggregation = "max" +force_wildcard_level = 1 +[kcat_sources.ec_tree] +k = 5 +[kcat_sources.homologs] +k = 5 +min_identity = 0.4 +coverage = 0.5 +[kcat_sources.okp] +methods = ["KinForm-H"] +[kcat_sources.okp_kinform_h] +[kcat_sources.custom] +[kcat_sources.dlkcat] +[matlab.params] +anything = 1 +""") + assert ModelAdapter.from_folder(folder).params.kcat_sources["homologs"]["coverage"] == 0.5 + + +SHOW_TOML = MINIMAL_TOML + """\ +gr_exp = 0.5 + +[kcat_sources.brenda] +aggregation = "max" + +[evotune.source_groups.okp] +sources = ["okp_*"] +sigma0_log = 0.25 +""" + - assert ModelParameters.model_fields["okp"].get_default(call_default_factory=True).method == "KinForm-H" +def test_show_prints_one_line_per_item_in_the_agreed_format(tmp_path, capsys): + from geckopy.adapter import show_adapter_params + + show_adapter_params(ModelAdapter.from_folder(_write_adapter_folder(tmp_path, SHOW_TOML))) + lines = capsys.readouterr().out.splitlines() + assert lines[0] == "project_format = [not set]" + for expected in [ + "org_name = Testus organismus [file]", + "gr_exp = 0.5 [file]", + "sigma = 0.5 [default]", + "uniprot.reviewed = false [default]", + "kcat_assignment.preset = brenda_ec_tree [default]", + "kcat_assignment.isozymes = true [default]", + "kcat_assignment.stage[1] = select: custom, database_top, ec_tree; top_origin_limit=1 [preset brenda_ec_tree]", + "kcat_assignment.stage[2] = combine: okp [preset brenda_ec_tree]", + "kcat_sources.brenda.aggregation = max [file]", + "kcat_sources.ec_tree.k = 5 [preset brenda_ec_tree]", + "kcat_sources.okp.methods = KinForm-H [default]", + "evotune.source_groups.okp = okp_*; sigma0_log 0.25 [file]", + ]: + assert expected in lines, expected + assert not any(ln.startswith("Adapter parameters") for ln in lines) + # Items in the order of the table; each table key is listed (stage, kcat_sources, groups expanded). + table = Path(__file__).resolve().parent.parent / "src" / "geckopy" / "adapter" / "adapterParameters.tsv" + expected_keys = [ln.split("\t")[0] for ln in table.read_text().splitlines()[1:]] + seen = [] + for ln in lines: + key = ln.split(" =")[0] + base = next((k for k in expected_keys if key == k or key.startswith(k + ".") or key.startswith(k + "[")), key) + if not seen or seen[-1] != base: + seen.append(base) + assert seen == expected_keys + + +def test_show_with_stages_and_file_settings(tmp_path, capsys): + from geckopy.adapter import show_adapter_params + + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + """[[kcat_assignment.stage]] +sources = ["custom", "database_top"] +top_origin_limit = 2 +criteria = "median" +[[kcat_assignment.stage]] +sources = ["okp"] +action = "combine" +""") + show_adapter_params(ModelAdapter.from_folder(folder)) + lines = capsys.readouterr().out.splitlines() + assert "kcat_assignment.preset = none (stages are given) [file]" in lines + assert "kcat_assignment.stage[1] = select: custom, database_top; criteria=median; top_origin_limit=2 [file]" in lines + assert "kcat_assignment.stage[2] = combine: okp [file]" in lines + + +def test_show_cli_prints_the_project_of_the_folder(tmp_path, capsys): + from geckopy.cli import main + + folder = _write_adapter_folder(tmp_path, SHOW_TOML) + assert main(["adapter", "show", str(folder)]) == 0 + out = capsys.readouterr().out + assert out.splitlines()[0] == "project_format = [not set]" + assert "gr_exp = 0.5 [file]" in out + + +def _group_toml(body: str) -> str: + return MINIMAL_TOML + "[evotune.source_groups.g]\n" + body + + +@pytest.mark.parametrize("toml, message", [ + ("[evotune.source_groups]\ng = 1\n", "evotune.source_groups.g must be a table with sources and sigma0_log"), + (_group_toml('sources = ["a"]\n'), "evotune.source_groups.g needs both sources and sigma0_log"), + (_group_toml("sigma0_log = 0.1\n"), "evotune.source_groups.g needs both sources and sigma0_log"), + (_group_toml('sources = "a"\nsigma0_log = 0.1\n'), "evotune.source_groups.g.sources must be a non-empty list of strings"), + (_group_toml('sources = []\nsigma0_log = 0.1\n'), "evotune.source_groups.g.sources must be a non-empty list of strings"), + (_group_toml('sources = ["a"]\nsigma0_log = inf\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a"]\nsigma0_log = nan\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a", 1]\nsigma0_log = 0.1\n'), "evotune.source_groups.g.sources must be a non-empty list of strings"), + (_group_toml('sources = ["a"]\nsigma0_log = "x"\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a"]\nsigma0_log = 0\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a"]\nsigma0_log = true\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), +]) +def test_a_malformed_evotune_group_has_its_own_message(tmp_path, toml, message): + folder = _write_adapter_folder(tmp_path, toml) + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == f"{folder.resolve() / 'model_adapter.toml'}: {message}" + + +def test_okp_source_tables_are_limited_to_known_methods(tmp_path): + ok = _write_adapter_folder(tmp_path, MINIMAL_TOML + "[kcat_sources.okp_unikp]\n") + assert "okp_unikp" in ModelAdapter.from_folder(ok).params.kcat_sources + for name in ("okp_nosuchmethod", "okp_catpred"): + folder = tmp_path / name + folder.mkdir() + (folder / "model_adapter.toml").write_text(MINIMAL_TOML + f"[kcat_sources.{name}]\n") + with pytest.raises(ValueError, match=f"unknown key kcat_sources.{name} in model_adapter.toml"): + ModelAdapter.from_folder(folder) + + +@pytest.mark.parametrize("extra, message", [ + ("[kcat_sources.okp]\nmethods = \"KinForm-H\"\n", "kcat_sources.okp.methods must be a list of strings"), + ("[kcat_sources.okp]\nmethods = [1]\n", "kcat_sources.okp.methods must be a list of strings"), + ("kcat_sources.brenda = 1\n", "kcat_sources.brenda must be a table"), + ("kcat_sources.foo = 1\n", "unknown key kcat_sources.foo in model_adapter.toml"), + ("[kcat_sources.brenda]\naggregation = \"mean\"\n", "kcat_sources.brenda.aggregation must be max or median"), +]) +def test_kcat_source_values_are_checked_at_load(tmp_path, extra, message): + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + extra) + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == f"{folder.resolve() / 'model_adapter.toml'}: {message}" + + +def test_unknown_okp_methods_are_an_error_at_load_in_file_order_without_duplicates(tmp_path): + from geckopy.gather_kcats.kcat_sources import OKP_METHODS + + folder = _write_adapter_folder( + tmp_path, MINIMAL_TOML + '[kcat_sources.okp]\nmethods = ["Zed", "KinForm-H", "Alpha", "Zed", "TurNup"]\n') + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == ( + "Unknown OKP methods in [kcat_sources.okp] methods: Zed, Alpha, TurNup; " + f"methods: {', '.join(OKP_METHODS)}.") + + +def test_show_number_formats_and_kcat_sources_order(tmp_path, capsys): + from geckopy.adapter import show_adapter_params + + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + """\ +sigma = 1e6 +[complex] +taxonomic_id = 5592920 +[kcat_sources.okp] +methods = ["UniKP"] +[kcat_sources.ec_tree] +k = 3 +[kcat_sources.brenda] +force_wildcard_level = 1 +aggregation = "median" +[kcat_assignment] +preset = "gecko_classic" +""") + show_adapter_params(ModelAdapter.from_folder(folder)) + lines = capsys.readouterr().out.splitlines() + assert "sigma = 1000000 [file]" in lines + assert "complex.taxonomic_id = 5592920 [file]" in lines + assert ("kcat_assignment.stage[1] = select: custom, database_top, dlkcat, database_bottom; " + "top_origin_limit=6 [preset gecko_classic]") in lines + ks = [ln for ln in lines if ln.startswith("kcat_sources.")] + assert ks == [ + "kcat_sources.brenda.aggregation = median [file]", + "kcat_sources.brenda.force_wildcard_level = 1 [file]", + "kcat_sources.ec_tree.k = 3 [file]", + "kcat_sources.okp.methods = UniKP [file]", + ] diff --git a/tests/test_cli.py b/tests/test_cli.py index f52c6fb..7c8fe1d 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -25,16 +25,17 @@ def test_template_includes_nested_sections(): assert "[evotune]" in text -def test_template_renders_dict_valued_evotune_fields(): - """The evotune section carries per-source tables, not just scalars.""" +def test_template_renders_source_groups_as_tables(): + """The evotune section lists the default groups as one table each.""" text = generate_template_toml() uncommented = "\n".join( - line.lstrip("# ") for line in text.splitlines() - if line.lstrip("# ").startswith(("sigma0_log_source", "source_groups")) + line[2:] for line in text.splitlines() + if line.startswith("# [evotune.source_groups.") or line.startswith("# sources =") or line.startswith("# sigma0_log =") ) - parsed = tomllib.loads(uncommented) - assert parsed["sigma0_log_source"]["brenda"] == 0.2 - assert parsed["source_groups"]["brenda"]["sources"] == ["brenda"] + parsed = tomllib.loads(uncommented)["evotune"]["source_groups"] + assert parsed["brenda"] == {"sources": ["brenda"], "sigma0_log": 0.2} + assert list(parsed) == ["custom", "brenda", "ec_tree", "combined"] + assert "sigma0_log_source" not in text and "match_okp" not in text def test_template_includes_field_descriptions_as_comments(): @@ -195,3 +196,10 @@ def fake_download(kid, path, *, gene_id_field): assert calls["args"][0] == "sce" assert calls["args"][2] == "kegg" assert out.exists() + + +def test_template_lists_the_kcat_source_keys_after_the_sections(): + text = generate_template_toml() + assert text.index("# [kcat_sources.brenda]") > text.index("# [evotune.source_groups.combined]") + for key in ("aggregation", "force_wildcard_level", "min_identity", "coverage", "methods"): + assert f"# {key} = " in text diff --git a/tests/test_evotune.py b/tests/test_evotune.py index 0b0b845..71f2fca 100644 --- a/tests/test_evotune.py +++ b/tests/test_evotune.py @@ -18,7 +18,7 @@ import cobra from geckopy import EcModel, ModelAdapter -from geckopy.adapter.params import EvotuneParams +from geckopy.adapter.params import EvotuneParams, SourceGroupRule from geckopy.databases.flux_data import FluxData from geckopy.ec_model.ec_data import EcData from geckopy.kcat_tuning.evotune.tuning import ( @@ -30,6 +30,12 @@ from geckopy.kcat_tuning.evotune.data import EvotuneData from geckopy.kcat_tuning.evotune.tuning import EvotuneResult +GROUPS = { + "dlkcat": SourceGroupRule(sources=["dlkcat"], sigma0_log=0.4), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "custom": SourceGroupRule(sources=["custom"], sigma0_log=0.1), +} + _TRUE_KCAT = 2.0 _START_KCAT = 1.0 _MW = 100.0 @@ -271,7 +277,7 @@ def test_cmaes_tuning_moves_kcats_toward_truth(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=15, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=15, rmse_threshold=1e-6) before = model.ec.kcat.copy() result = cmaes_kcat_tuning( @@ -295,7 +301,7 @@ def test_cmaes_tuning_moves_kcats_toward_truth(tmp_path): def test_cmaes_tuning_extends_the_source_of_changed_kcats(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) - params = EvotuneParams(max_generations=15, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=15, rmse_threshold=1e-6) before = model.ec.kcat.copy() origins = ["brenda", "dlkcat"] @@ -320,7 +326,7 @@ def test_cmaes_tuning_extends_the_source_of_changed_kcats(tmp_path): def test_cmaes_tuning_n_proc_matches_serial(tmp_path): - params = EvotuneParams(max_generations=5, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=5, rmse_threshold=-1.0) def _run(n_proc): adapter = _adapter(tmp_path) @@ -346,7 +352,7 @@ def test_tie_isozymes_forces_equal_values(tmp_path): adapter = _adapter(tmp_path) model = _build_tied_toy(adapter) evotune_data = _tied_evotune_data() - params = EvotuneParams(max_generations=8, rmse_threshold=-1.0, + params = EvotuneParams(source_groups=GROUPS, max_generations=8, rmse_threshold=-1.0, tie_isozymes=True) screen = screen_kcat_leverage( @@ -368,7 +374,7 @@ def test_tie_isozymes_false_keeps_groups_separate(tmp_path): adapter = _adapter(tmp_path) model = _build_tied_toy(adapter) evotune_data = _tied_evotune_data() - params = EvotuneParams(tie_isozymes=False) + params = EvotuneParams(source_groups=GROUPS, tie_isozymes=False) screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, n_proc=1, @@ -402,7 +408,7 @@ def test_tune_prior_penalty_weight_shape_and_fit_cost(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=6, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=6, rmse_threshold=-1.0) report = tune_prior_penalty_weight( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -427,7 +433,7 @@ def test_tune_prior_penalty_weight_restores_model(tmp_path): evotune_data = _evotune_data() before = model.ec.kcat.copy() source_before = list(model.ec.source) - params = EvotuneParams(max_generations=4, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=4, rmse_threshold=-1.0) tune_prior_penalty_weight( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -444,7 +450,7 @@ def test_tune_prior_penalty_weight_single_seed_gives_nan_reproducibility(tmp_pat adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=4, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=4, rmse_threshold=-1.0) report = tune_prior_penalty_weight( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -493,7 +499,7 @@ def test_cmaes_tuning_runs_all_generations_on_a_flat_objective(tmp_path, monkeyp monkeypatch.setattr(tuning, "_score_kcat_vector", lambda *a, **k: (1.0, 1.0)) adapter = _adapter(tmp_path) model = _build_toy(adapter) - params = EvotuneParams(max_generations=6, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=6, rmse_threshold=-1.0) result = cmaes_kcat_tuning( model, adapter=adapter, params=params, evotune_data=_evotune_data(), n_proc=1, seed=0, verbose=False, @@ -510,3 +516,26 @@ def test_screen_warns_when_the_model_has_no_formulas(tmp_path): screen_kcat_leverage( model, adapter=adapter, evotune_data=_evotune_data(), n_proc=1, ) + + +def test_default_groups_label_kcats_by_group_name_through_the_tuning_context(tmp_path, caplog): + """With default EvotuneParams, combined and ec_tree kcats get their group's width, + the group name labels the kcat, and the unlabelled warning fires for kcats in no group.""" + from geckopy.kcat_tuning.evotune.tuning import _tunable_context + + model = _build_toy(_adapter(tmp_path)) + model.ec.source = ["combined (sd=0.10, n=2)", "ec_tree"] + with caplog.at_level("WARNING"): + out = _tunable_context(model, EvotuneParams(), _evotune_data(), "biomass", None) + groups, sigma0_log = out[3], out[4] + assert list(groups) == ["combined", "ec_tree"] + assert list(sigma0_log) == [0.4, 0.3] + assert "match no evotune source group" not in caplog.text + + model.ec.source = ["standard", "standard"] + caplog.clear() + with caplog.at_level("WARNING"): + out = _tunable_context(model, EvotuneParams(), _evotune_data(), "biomass", None) + assert list(out[3]) == ["unlabelled", "unlabelled"] + assert list(out[4]) == [0.5, 0.5] + assert "2 of 2 tunable kcats match no evotune source group and get sigma0_log_default (0.5); origins: standard" in caplog.text diff --git a/tests/test_evotune_priors.py b/tests/test_evotune_priors.py index 562cecc..d3a8330 100644 --- a/tests/test_evotune_priors.py +++ b/tests/test_evotune_priors.py @@ -8,6 +8,7 @@ build_sigma0_log, classify_kcat_source, classify_kcat_sources, + warn_unlabelled, ) @@ -15,11 +16,10 @@ def _params() -> EvotuneParams: return EvotuneParams( sigma0_log_default=0.5, source_groups={ - "dlkcat": SourceGroupRule(sources=["dlkcat"]), - "brenda": SourceGroupRule(sources=["brenda"]), - "okp": SourceGroupRule(match_okp=True), + "dlkcat": SourceGroupRule(sources=["dlkcat"], sigma0_log=0.4), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "okp": SourceGroupRule(sources=["okp_*"], sigma0_log=0.3), }, - sigma0_log_source={"dlkcat": 0.4, "brenda": 0.2, "okp": 0.3}, ) @@ -39,23 +39,61 @@ def test_classify_unmatched_source_is_unlabelled(): assert classify_kcat_source("some_other_source", params) == UNLABELLED_GROUP -def test_classify_matches_okp_method(): +def test_classify_matches_prefix_pattern_on_the_origin(): params = _params() - assert classify_kcat_source("CataPro", params, okp_method="CataPro") == "okp" - # Case-insensitive here too. - assert classify_kcat_source("catapro", params, okp_method="CataPro") == "okp" - # No okp_method configured -> falls through to unlabelled. - assert classify_kcat_source("CataPro", params, okp_method=None) == UNLABELLED_GROUP + assert classify_kcat_source("okp_kinform_h", params) == "okp" + assert classify_kcat_source("OKP_EITLEM_Kinetics (similarity=0.9)", params) == "okp" + assert classify_kcat_source("okp", params) == UNLABELLED_GROUP + assert classify_kcat_source("xokp_kinform_h", params) == UNLABELLED_GROUP + + +def test_exact_matches_win_over_patterns_and_longest_prefix_wins(): + params = EvotuneParams(source_groups={ + "wide": SourceGroupRule(sources=["okp_*"], sigma0_log=0.3), + "narrow": SourceGroupRule(sources=["okp_kin*"], sigma0_log=0.2), + "same": SourceGroupRule(sources=["okp_kin*"], sigma0_log=0.25), + "exact": SourceGroupRule(sources=["okp_kinform_h"], sigma0_log=0.1), + }) + assert classify_kcat_source("okp_kinform_h", params) == "exact" # exact over every pattern + assert classify_kcat_source("okp_kinform_l", params) == "narrow" # longest prefix; ties keep group order + assert classify_kcat_source("okp_catapro", params) == "wide" + + +def test_pattern_group_listed_first_does_not_beat_an_exact_match_of_a_later_group(): + params = EvotuneParams(source_groups={ + "all": SourceGroupRule(sources=["*"], sigma0_log=0.5), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + }) + assert classify_kcat_source("brenda", params) == "brenda" + assert classify_kcat_source("anything", params) == "all" def test_classify_kcat_sources_vectorised(): params = _params() - groups = classify_kcat_sources( - ["dlkcat", "brenda", "CataPro", "unknown"], params, okp_method="CataPro", - ) + groups = classify_kcat_sources(["dlkcat", "brenda", "okp_unikp", "unknown"], params) assert list(groups) == ["dlkcat", "brenda", "okp", UNLABELLED_GROUP] +def test_unlabelled_warning_names_count_default_and_common_origins(caplog): + params = _params() + sources = ["brenda"] * 5 + ["standard"] * 3 + ["isozymes"] * 2 + ["a"] + ["b"] + ["c"] + ["d"] + groups = classify_kcat_sources(sources, params) + with caplog.at_level("WARNING"), pytest.warns(UserWarning, match="match no evotune source group"): + warn_unlabelled(groups, sources, params) + assert ( + "9 of 14 tunable kcats match no evotune source group and get sigma0_log_default (0.5); " + "origins: standard, isozymes, a, b, c" + ) in caplog.text + + +def test_unlabelled_warning_only_above_a_fifth(caplog): + params = _params() + sources = ["brenda"] * 4 + ["standard"] # exactly 20 % + with caplog.at_level("WARNING"): + warn_unlabelled(classify_kcat_sources(sources, params), sources, params) + assert "match no evotune source group" not in caplog.text + + # --------------------------------------------------------------------------- # # build_sigma0_log # --------------------------------------------------------------------------- # @@ -71,8 +109,7 @@ def test_build_sigma0_log_uses_group_values_and_default_fallback(): def test_a_tuning_status_does_not_hide_the_source(): params = EvotuneParams( - source_groups={"brenda": SourceGroupRule(sources=["brenda"])}, - sigma0_log_source={"brenda": 0.2}, + source_groups={"brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2)}, sigma0_log_default=0.3, ) for source in ("brenda | tuned=cmaes (prior=6.55)", @@ -85,8 +122,7 @@ def test_a_fuzzy_match_detail_does_not_hide_the_source(): """``ec.source`` carries the wildcard/origin detail; the tier is the source token in front of it.""" params = EvotuneParams( - source_groups={"brenda": SourceGroupRule(sources=["brenda"])}, - sigma0_log_source={"brenda": 0.2}, + source_groups={"brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2)}, sigma0_log_default=0.3, ) assert classify_kcat_source("brenda (wc=0, origin=1)", params) == "brenda" @@ -104,9 +140,8 @@ def test_a_fuzzy_match_detail_does_not_hide_the_source(): def test_a_group_may_still_list_the_whole_source_string(): params = EvotuneParams( source_groups={ - "wildcarded": SourceGroupRule(sources=["brenda (wc=1, origin=3)"]), + "wildcarded": SourceGroupRule(sources=["brenda (wc=1, origin=3)"], sigma0_log=0.5), }, - sigma0_log_source={"wildcarded": 0.5}, ) assert classify_kcat_source( "brenda (wc=1, origin=3)", params, diff --git a/tests/test_expand.py b/tests/test_expand.py index e18a216..e9edd7c 100644 --- a/tests/test_expand.py +++ b/tests/test_expand.py @@ -2,7 +2,7 @@ import cobra from geckopy.ec_model.pipeline import expand_model -# gpr_to_dnf lives in raven-toolbox now (see docs/raven_integration.md). +# gpr_to_dnf lives in raven-toolbox now (see dev/raven_integration.md). from raven_toolbox.manipulation.expand import gpr_to_dnf diff --git a/tests/test_fill_kcats.py b/tests/test_fill_kcats.py index c8d7f51..56b8102 100644 --- a/tests/test_fill_kcats.py +++ b/tests/test_fill_kcats.py @@ -272,47 +272,16 @@ def test_filled_kcats_have_isozymes_source(): # --------------------------------------------------------------------------- # -# Adapter-default aggregate (params.kcat_aggregate_isozymes) +# Aggregate choice # --------------------------------------------------------------------------- # -def _adapter_with_isozyme_aggregate( - tmp_path: Path, aggregate: str | None = None, -) -> ModelAdapter: - body = ['conv_gem = "dummy.xml"', 'org_name = "yeast"'] - if aggregate is not None: - body.append(f'kcat_aggregate_isozymes = "{aggregate}"') - (tmp_path / "model_adapter.toml").write_text("\n".join(body) + "\n") - return ModelAdapter.from_folder(tmp_path) - - -def test_aggregate_default_mean_with_adapter_default(tmp_path): - """Adapter defaults to ``kcat_aggregate_isozymes='mean'``; omitting - ``aggregate`` falls through to that default.""" - model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) - model.adapter = _adapter_with_isozyme_aggregate(tmp_path) - fill_kcats_from_isozymes(model, apply=False) - assert model.ec.kcat[3] == pytest.approx((1.0 + 2.0 + 100.0) / 3.0) - - -def test_aggregate_adapter_median_flips_default(tmp_path): - """With the adapter at ``kcat_aggregate_isozymes='median'``, an - omitted ``aggregate`` falls through to median averaging.""" - model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) - model.adapter = _adapter_with_isozyme_aggregate(tmp_path, "median") - fill_kcats_from_isozymes(model, apply=False) - assert model.ec.kcat[3] == pytest.approx(2.0) - - -def test_aggregate_default_mean_when_no_adapter(): - """Without an adapter attached, the mean default is used.""" +def test_aggregate_default_is_the_mean(): model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) fill_kcats_from_isozymes(model, apply=False) assert model.ec.kcat[3] == pytest.approx((1.0 + 2.0 + 100.0) / 3.0) -def test_explicit_aggregate_overrides_adapter(tmp_path): - """Caller-passed ``aggregate=`` wins over the adapter setting.""" +def test_explicit_aggregate_is_used(): model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) - model.adapter = _adapter_with_isozyme_aggregate(tmp_path, "median") fill_kcats_from_isozymes(model, apply=False, aggregate="max") assert model.ec.kcat[3] == pytest.approx(100.0) diff --git a/tests/test_flexibilize_enz_concs.py b/tests/test_flexibilize_enz_concs.py index 0282e3d..7012090 100644 --- a/tests/test_flexibilize_enz_concs.py +++ b/tests/test_flexibilize_enz_concs.py @@ -297,3 +297,12 @@ def test_infeasible_start_returns_empty_result_when_gurobi_presolves_it(tmp_path model.solver.problem.Params.Presolve = 2 model.reactions.get_by_id("R_AB").lower_bound = 50.0 assert flexibilize_enz_concs(model).uniprot_ids == [] + + +def test_a_missing_gr_exp_is_an_error_naming_the_file_key(tmp_path): + (tmp_path / "model_adapter.toml").write_text( + 'conv_gem = "dummy.xml"\norg_name = "test"\nbio_rxn = "biomass"\n' + ) + model = _build_toy(ModelAdapter.from_folder(tmp_path)) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + flexibilize_enz_concs(model) diff --git a/tests/test_fuzzy_kcat_matching.py b/tests/test_fuzzy_kcat_matching.py index c85947c..dab5deb 100644 --- a/tests/test_fuzzy_kcat_matching.py +++ b/tests/test_fuzzy_kcat_matching.py @@ -454,6 +454,7 @@ def test_force_wildcard_level_skips_exact_match(tmp_path): ]), _phyl_dist(["yeast"]), force_wildcard_level=1, + aggregate="max", ) row = df.iloc[0] # With force_wildcard_level=1, we never try the exact 1.1.1.1 token; @@ -673,56 +674,63 @@ def test_aggregate_kcats_max_or_median(): assert _aggregate_kcats(arr, "median") == pytest.approx(2.5) -def _adapter_with_aggregate( - tmp_path: Path, org_name: str, aggregate: str, -) -> ModelAdapter: +def _adapter_with_aggregate(tmp_path: Path, org_name: str, toml_extra: str) -> ModelAdapter: (tmp_path / "model_adapter.toml").write_text( f'conv_gem = "dummy.xml"\n' f'org_name = "{org_name}"\n' - f'kcat_aggregate_brenda = "{aggregate}"\n' + f'{toml_extra}\n' ) return ModelAdapter.from_folder(tmp_path) -def test_aggregate_default_max_when_no_explicit_arg(tmp_path): - """With the adapter at its default ``kcat_aggregate_brenda='max'``, an - omitted ``aggregate`` arg picks the max of the matched BRENDA rows.""" - adapter = _adapter_with_org(tmp_path, "yeast") # adapter default = max - model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) - brenda = _brenda([ +def _three_values(): + return _brenda([ ("1.1.1.1", "alpha", "yeast", 2.0), ("1.1.1.1", "alpha", "yeast", 8.0), ("1.1.1.1", "alpha", "yeast", 100.0), ]) - df = fuzzy_kcat_matching(model, brenda, _phyl_dist(["yeast"])) - assert df["kcat"].iloc[0] == 100.0 -def test_aggregate_adapter_median_changes_default(tmp_path): - """Adapter ``kcat_aggregate_brenda='median'`` flips the default - behaviour to median without the caller passing ``aggregate=``.""" - adapter = _adapter_with_aggregate(tmp_path, "yeast", "median") +def test_aggregate_default_is_the_median_of_the_default_preset(tmp_path): + """Without an explicit ``aggregate``, the default preset brenda_ec_tree + sets ``[kcat_sources.brenda] aggregation`` to median.""" + adapter = _adapter_with_org(tmp_path, "yeast") model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) - brenda = _brenda([ - ("1.1.1.1", "alpha", "yeast", 2.0), - ("1.1.1.1", "alpha", "yeast", 8.0), - ("1.1.1.1", "alpha", "yeast", 100.0), - ]) - df = fuzzy_kcat_matching(model, brenda, _phyl_dist(["yeast"])) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) assert df["kcat"].iloc[0] == 8.0 +def test_aggregate_default_follows_the_preset(tmp_path): + """The gecko_classic preset aggregates BRENDA by the maximum.""" + adapter = _adapter_with_aggregate(tmp_path, "yeast", '[kcat_assignment]\npreset = "gecko_classic"') + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) + assert df["kcat"].iloc[0] == 100.0 + + +def test_aggregate_default_is_max_for_stages_without_a_brenda_table(tmp_path): + adapter = _adapter_with_aggregate( + tmp_path, "yeast", '[[kcat_assignment.stage]]\nsources = ["custom", "database_top"]') + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) + assert df["kcat"].iloc[0] == 100.0 + + +def test_aggregate_default_follows_kcat_sources_over_the_preset(tmp_path): + adapter = _adapter_with_aggregate( + tmp_path, "yeast", '[kcat_sources.brenda]\naggregation = "max"') + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) + assert df["kcat"].iloc[0] == 100.0 + + def test_explicit_aggregate_overrides_adapter(tmp_path): """Caller-passed ``aggregate=`` wins over the adapter setting.""" - adapter = _adapter_with_aggregate(tmp_path, "yeast", "median") + adapter = _adapter_with_aggregate( + tmp_path, "yeast", '[kcat_sources.brenda]\naggregation = "median"') model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) - brenda = _brenda([ - ("1.1.1.1", "alpha", "yeast", 2.0), - ("1.1.1.1", "alpha", "yeast", 8.0), - ("1.1.1.1", "alpha", "yeast", 100.0), - ]) df = fuzzy_kcat_matching( - model, brenda, _phyl_dist(["yeast"]), aggregate="max", + model, _three_values(), _phyl_dist(["yeast"]), aggregate="max", ) assert df["kcat"].iloc[0] == 100.0 @@ -765,3 +773,10 @@ def test_kcat_is_divided_when_the_matched_substrate_is_taken_more_than_once(tmp_ _phyl_dist(["yeast"]), ) assert df.iloc[0]["kcat"] == pytest.approx(50.0) + + +def test_an_invalid_aggregation_argument_has_the_shared_text(tmp_path): + adapter = _adapter_with_org(tmp_path, "yeast") + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + with pytest.raises(ValueError, match="^aggregation must be max or median$"): + fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"]), aggregate="mean") diff --git a/tests/test_kcat_assignment.py b/tests/test_kcat_assignment.py index ae80beb..67bb809 100644 --- a/tests/test_kcat_assignment.py +++ b/tests/test_kcat_assignment.py @@ -269,3 +269,45 @@ def test_default_okp_methods_are_kinform_h_and_a_project_list_is_kept(tmp_path): assert okp_methods(kcat_assignment_config(adapter)) == ["KinForm-H"] adapter.params.kcat_sources["okp"] = {"methods": ["CataPro", "UniKP"]} assert okp_methods(kcat_assignment_config(adapter)) == ["CataPro", "UniKP"] + + +@pytest.mark.parametrize("method", ["CatPred", "TurNup"]) +def test_reaction_okp_methods_are_unknown_in_methods(tmp_path, method): + from geckopy.gather_kcats.kcat_assignment import kcat_assignment_config, okp_methods + + _, adapter, _ = _project(tmp_path, '[kcat_assignment]\npreset = "okp"\n') + adapter.params.kcat_sources["okp"] = {"methods": [method]} + from geckopy.gather_kcats.kcat_sources import OKP_METHODS + + with pytest.raises(ValueError) as err: + okp_methods(kcat_assignment_config(adapter)) + assert str(err.value) == ( + f"Unknown OKP methods in [kcat_sources.okp] methods: {method}; methods: {', '.join(OKP_METHODS)}.") + + +def test_okp_source_file_of_no_known_method_fails_naming_the_file(tmp_path): + model, adapter, project = _project( + tmp_path, '[[kcat_assignment.stage]]\nsources = ["okp_nosuchmethod"]\n') + report = update_kcat_sources(model, adapter=adapter).set_index("source") + assert report.loc["okp_nosuchmethod", "status"] == "failed" + from geckopy.gather_kcats.kcat_sources import OKP_METHODS, okp_file + + assert report.loc["okp_nosuchmethod", "note"] == ( + "okp_nosuchmethod is not the file of a known OKP method; the files are " + f"{', '.join(okp_file(m) for m in OKP_METHODS)}.") + + +def test_direct_kcats_from_brenda_follows_preset_or_stages(tmp_path): + """Without an aggregation: the median under a preset, the maximum under stages without a brenda table.""" + from geckopy.gather_kcats import read_kcat_source + + model, adapter, project = _project(tmp_path / "preset", '[kcat_assignment]\npreset = "brenda_ec_tree"\n') + kcats_from_brenda(model, adapter=adapter) + _, meta = read_kcat_source(project / "data" / "kcats" / "brenda.tsv") + assert '"aggregation":"median"' in meta["parameters"] + + model, adapter, project = _project( + tmp_path / "stages", '[[kcat_assignment.stage]]\nsources = ["custom", "database_top"]\n') + kcats_from_brenda(model, adapter=adapter) + _, meta = read_kcat_source(project / "data" / "kcats" / "brenda.tsv") + assert '"aggregation":"max"' in meta["parameters"] diff --git a/tests/test_kcat_sources.py b/tests/test_kcat_sources.py index f7cc2e3..6febac1 100644 --- a/tests/test_kcat_sources.py +++ b/tests/test_kcat_sources.py @@ -466,3 +466,46 @@ def test_kcats_from_dlkcat_run_is_docker(tmp_path, monkeypatch): assert done and ran == ["write", "run"] _, meta = read_kcat_source(tmp_path / "data" / "kcats" / "dlkcat.tsv") assert meta["parameters"] == '{"run":true}' + + +def test_kcats_from_okp_default_method_is_first_of_methods(tmp_path, monkeypatch): + """Without a method, kcats_from_okp uses the first entry of [kcat_sources.okp] methods + (KinForm-H when the project lists none).""" + adapter = _adapter(tmp_path) + full = _model(adapter, light=False) + (tmp_path / "data").mkdir() + submitted = [] + + def fake_submit(model, **kw): + submitted.append(kw["method"]) + return "job" + + monkeypatch.setattr(kcats_from_module, "_submit_open_kinetics_predictor", fake_submit) + monkeypatch.setattr(kcats_from_module, "_fetch_open_kinetics_predictor", lambda model, **kw: (False, None)) + assert kcats_from_okp(full, save=False) == (None, False) + adapter.params.kcat_sources["okp"] = {"methods": ["UniKP", "CataPro"]} + kcats_from_okp(full, save=False, resubmit=True) + assert submitted == ["KinForm-H", "UniKP"] + + +@pytest.mark.parametrize("method", ["CatPred", "TurNup"]) +def test_kcats_from_okp_rejects_reaction_methods(tmp_path, method): + full = _model(_adapter(tmp_path), light=False) + with pytest.raises(ValueError, match="enzyme-substrate pairs only"): + kcats_from_okp(full, method=method) + + +@pytest.mark.parametrize("method, file", [ + ("KinForm-H", "okp_kinform_h"), ("OmniESI-O2DENet", "okp_omniesi_o2denet"), ("UniKP", "okp_unikp"), +]) +def test_okp_file_names(method, file): + from geckopy.gather_kcats.kcat_sources import okp_file, okp_method_of_file + + assert okp_file(method) == file + assert okp_method_of_file(file) == method + + +def test_kcats_from_brenda_rejects_an_invalid_aggregation(tmp_path): + full = _model(_adapter(tmp_path), light=False) + with pytest.raises(ValueError, match="^aggregation must be max or median$"): + kcats_from_brenda(full, aggregation="mean") diff --git a/tests/test_mask_size.py b/tests/test_mask_size.py index 0a591e3..a7f5b8a 100644 --- a/tests/test_mask_size.py +++ b/tests/test_mask_size.py @@ -12,7 +12,7 @@ import cobra from geckopy import EcModel, ModelAdapter -from geckopy.adapter.params import EvotuneParams +from geckopy.adapter.params import EvotuneParams, SourceGroupRule from geckopy.databases.flux_data import FluxData from geckopy.ec_model.ec_data import EcData from geckopy.kcat_tuning.evotune.data import EvotuneData @@ -23,6 +23,12 @@ ) from geckopy.kcat_tuning.evotune.tuning import screen_kcat_leverage +GROUPS = { + "dlkcat": SourceGroupRule(sources=["dlkcat"], sigma0_log=0.4), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "custom": SourceGroupRule(sources=["custom"], sigma0_log=0.1), +} + _TRUE_KCAT = 2.0 _START_KCAT = 1.0 _MW = 100.0 @@ -126,7 +132,7 @@ def test_sweep_reports_one_point_per_share_sorted(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) points = sweep_tunable_mask_size( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -145,7 +151,7 @@ def test_sweep_does_not_mutate_input_model(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) before = model.ec.kcat.copy() sweep_tunable_mask_size( @@ -160,7 +166,7 @@ def test_sweep_runs_every_seed_at_every_share(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) points = sweep_tunable_mask_size( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -180,7 +186,7 @@ def test_sweep_reuses_a_given_screen(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) screen = screen_kcat_leverage( model, adapter=adapter, evotune_data=evotune_data, n_proc=1, ) @@ -299,7 +305,7 @@ def test_sweep_uses_rmse_trace_minimum_not_last_entry(tmp_path, monkeypatch): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) # A later generation can improve the (prior-penalised) objective # while its own plain RMSE is worse than an earlier generation's -- diff --git a/tests/test_params.py b/tests/test_params.py index a7b3e72..2ab664c 100644 --- a/tests/test_params.py +++ b/tests/test_params.py @@ -5,40 +5,32 @@ from geckopy.adapter.params import EvotuneParams, SourceGroupRule -def test_defaults_are_consistent(): - # The shipped defaults must satisfy the group-key validator. - bp = EvotuneParams() - assert set(bp.sigma0_log_source) == set(bp.source_groups) - - -def test_mismatched_source_dict_raises(): - with pytest.raises(ValidationError, match="sigma0_log_source"): - EvotuneParams( - source_groups={ - "dlkcat": SourceGroupRule(sources=["dlkcat"]), - "brenda": SourceGroupRule(sources=["brenda"]), - }, - sigma0_log_source={"dlkcat": 0.4}, # missing "brenda" - ) - - -def test_consistent_custom_groups_ok(): - bp = EvotuneParams( - source_groups={ - "a": SourceGroupRule(sources=["src_a"]), - "b": SourceGroupRule(sources=["src_b"], match_okp=True), - }, - sigma0_log_source={"a": 0.1, "b": 0.2}, - ) - assert set(bp.source_groups) == {"a", "b"} - assert bp.source_groups["b"].match_okp is True +def test_default_groups_are_custom_brenda_ec_tree_combined(): + groups = EvotuneParams().source_groups + assert {k: (v.sources, v.sigma0_log) for k, v in groups.items()} == { + "custom": (["custom"], 0.1), + "brenda": (["brenda"], 0.2), + "ec_tree": (["ec_tree"], 0.3), + "combined": (["combined"], 0.4), + } + assert EvotuneParams().sigma0_log_default == 0.5 -def test_dropped_fields_are_rejected(): - """target_accept and variance_cap_* were confirmed dead/cosmetic - in MATLAB and are not part of the schema; extra="forbid" should - reject them outright rather than silently ignoring them.""" +def test_groups_given_replace_the_defaults(): + bp = EvotuneParams(source_groups={"a": SourceGroupRule(sources=["src_a"], sigma0_log=0.1)}) + assert set(bp.source_groups) == {"a"} + + +@pytest.mark.parametrize("kwargs", [{"sources": ["a"]}, {"sigma0_log": 0.1}]) +def test_a_group_needs_sources_and_sigma0_log(kwargs): with pytest.raises(ValidationError): - EvotuneParams(target_accept=10.0) + SourceGroupRule(**kwargs) + + +def test_dropped_fields_are_rejected(): + """Removed settings are unknown keys, not silently ignored.""" + for kwargs in ({"target_accept": 10.0}, {"variance_cap_default": 10.0}, {"sigma0_log_source": {}}): + with pytest.raises(ValidationError): + EvotuneParams(**kwargs) with pytest.raises(ValidationError): - EvotuneParams(variance_cap_default=10.0) + SourceGroupRule(sources=["a"], sigma0_log=0.1, match_okp=True) diff --git a/tests/test_parity_fixes_g1b.py b/tests/test_parity_fixes_g1b.py index 12d21e5..8fe6cd1 100644 --- a/tests/test_parity_fixes_g1b.py +++ b/tests/test_parity_fixes_g1b.py @@ -236,3 +236,10 @@ def test_gecko_light_model_is_tuned(): assert parsed.origin == (parse_kcat_source(source_before[i]).origin or "unknown") else: assert source == source_before[i] + + +def test_sensitivity_tuning_without_gr_exp_is_an_error(tmp_path): + (tmp_path / "model_adapter.toml").write_text('conv_gem = "dummy.xml"\norg_name = "test"\nbio_rxn = "biomass"\n') + model = _chain(ModelAdapter.from_folder(tmp_path), [1.0, 1.0]) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + sensitivity_tuning(model) diff --git a/tests/test_relax_enz_concs.py b/tests/test_relax_enz_concs.py index 2b09e8f..fe8895e 100644 --- a/tests/test_relax_enz_concs.py +++ b/tests/test_relax_enz_concs.py @@ -107,3 +107,12 @@ def test_an_unknown_weighting_is_rejected(tmp_path): model = _two_routes(_adapter(tmp_path, gr_exp=8.0), {"E1": 2.0, "E2": 2.0}) with pytest.raises(ValueError, match="weight"): relax_enz_concs(model, weight="max") + + +def test_a_missing_gr_exp_is_an_error_naming_the_file_key(tmp_path): + (tmp_path / "model_adapter.toml").write_text('conv_gem = "dummy.xml"\norg_name = "test"\nbio_rxn = "biomass"\n') + model = _two_routes(ModelAdapter.from_folder(tmp_path), {"E1": 2.0, "E2": 2.0}) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + relax_enz_concs(model) + # An explicit growth rate needs no gr_exp. + relax_enz_concs(model, exp_growth=8.0) diff --git a/tests/test_select_kcat_value.py b/tests/test_select_kcat_value.py index 45b8447..a2de349 100644 --- a/tests/test_select_kcat_value.py +++ b/tests/test_select_kcat_value.py @@ -1,12 +1,10 @@ """Tests for apply_kcat_list.""" -from pathlib import Path import numpy as np import pandas as pd import pytest from scipy import sparse -from geckopy import ModelAdapter from geckopy.ec_model import EcModel from geckopy.ec_model.ec_data import EcData from geckopy.gather_kcats import apply_kcat_list @@ -395,56 +393,20 @@ def test_source_bare_lowercase_token_without_metadata(): # --------------------------------------------------------------------------- # -# Adapter-default criteria (params.kcat_aggregate_candidates) +# Criteria choice # --------------------------------------------------------------------------- # -def _adapter_with_candidates( - tmp_path: Path, criteria: str | None = None, -) -> ModelAdapter: - body = ['conv_gem = "dummy.xml"', 'org_name = "yeast"'] - if criteria is not None: - body.append(f'kcat_aggregate_candidates = "{criteria}"') - (tmp_path / "model_adapter.toml").write_text("\n".join(body) + "\n") - return ModelAdapter.from_folder(tmp_path) - - -def _ec_model_with_adapter(adapter: ModelAdapter, rxn_ids: list[str]) -> EcModel: - model = _ec_model(rxn_ids) - model.adapter = adapter - return model - - _THREE_CAND = [("r1", 1.0, "src_low"), ("r1", 4.0, "src_mid"), ("r1", 9.0, "src_high")] -def test_criteria_default_max_with_adapter_default(tmp_path): - """Adapter defaults to ``kcat_aggregate_candidates='max'``; omitting - ``criteria`` picks the largest of the candidate kcats.""" - adapter = _adapter_with_candidates(tmp_path) - model = _ec_model_with_adapter(adapter, ["r1"]) - apply_kcat_list(model, _kcat_list(_THREE_CAND)) - assert model.ec.kcat[0] == 9.0 - - -def test_criteria_adapter_median_flips_default(tmp_path): - """When the adapter sets median, an omitted ``criteria`` follows.""" - adapter = _adapter_with_candidates(tmp_path, "median") - model = _ec_model_with_adapter(adapter, ["r1"]) - apply_kcat_list(model, _kcat_list(_THREE_CAND)) - assert model.ec.kcat[0] == 4.0 - - -def test_criteria_default_max_when_no_adapter(): - """Without an adapter attached, ``criteria`` defaults to ``'max'``.""" - model = _ec_model(["r1"]) # no adapter +def test_criteria_default_is_max(): + model = _ec_model(["r1"]) apply_kcat_list(model, _kcat_list(_THREE_CAND)) assert model.ec.kcat[0] == 9.0 -def test_explicit_criteria_overrides_adapter(tmp_path): - """Caller-passed ``criteria=`` wins over the adapter setting.""" - adapter = _adapter_with_candidates(tmp_path, "median") - model = _ec_model_with_adapter(adapter, ["r1"]) +def test_explicit_criteria_is_used(): + model = _ec_model(["r1"]) apply_kcat_list(model, _kcat_list(_THREE_CAND), criteria="min") assert model.ec.kcat[0] == 1.0 diff --git a/tests/test_sigma_fitter.py b/tests/test_sigma_fitter.py index dde8130..841a04c 100644 --- a/tests/test_sigma_fitter.py +++ b/tests/test_sigma_fitter.py @@ -244,3 +244,11 @@ def test_default_method_is_the_full_grid(tmp_path): result = fit_sigma(model) # default == grid assert result.sigma == pytest.approx(0.01, abs=1e-12) assert len(result.sigma_grid) == 100 + + +def test_fit_sigma_without_gr_exp_is_an_error(tmp_path): + (tmp_path / "model_adapter.toml").write_text('conv_gem = "dummy.xml"\norg_name = "test"\np_tot = 1.0\nf = 1.0\n') + model = _build_pool_only_model(ModelAdapter.from_folder(tmp_path)) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + fit_sigma(model) + assert fit_sigma(model, growth_rate=10.0).sigma == pytest.approx(0.01, abs=1e-4) diff --git a/tutorials/full_ecModel/model_adapter.toml b/tutorials/full_ecModel/model_adapter.toml index 63bc8e9..973a474 100644 --- a/tutorials/full_ecModel/model_adapter.toml +++ b/tutorials/full_ecModel/model_adapter.toml @@ -27,9 +27,9 @@ reviewed = true [complex] taxonomic_id = 559292 -# Kcat assignment (assign_kcats): the stages of the gecko_median preset, -# with the BRENDA aggregation that kcats_from_brenda takes from -# kcat_aggregate_brenda (the maximum). +# Kcat assignment (assign_kcats): the stages of the gecko_median preset. There +# is no [kcat_sources.brenda] table, so the stages use the BRENDA maximum, not +# the median of the preset. [[kcat_assignment.stage]] sources = ["custom", "database_top", "dlkcat", "database_bottom"] top_origin_limit = 4 @@ -48,21 +48,19 @@ max_generations = 212 # 30 parts glucose. Same field and value as MATLAB's maxGrowthWeight. max_growth_weight = 2.0 -# Trust tiers, least to most trusted. This model labels predicted kcats -# "OpenKineticsPredictor"; match_okp additionally catches models that label -# them with the predictor method name (OkpParams.method) instead. -[evotune.source_groups.okp] -sources = ["OpenKineticsPredictor"] -match_okp = true +# Trust tiers, each with its prior width (the standard deviation of the +# natural log of the kcat; lower is more trusted). The groups match the labels +# of ec.source that the stages above give: dlkcat (DLKcat predictions), brenda +# and custom. Kcats with another label, such as standard, take +# sigma0_log_default. +[evotune.source_groups.dlkcat] +sources = ["dlkcat"] +sigma0_log = 0.3 [evotune.source_groups.brenda] sources = ["brenda"] +sigma0_log = 0.2 [evotune.source_groups.custom] sources = ["custom"] - -[evotune.sigma0_log_source] -okp = 0.25 -brenda = 0.2 -custom = 0.1 - +sigma0_log = 0.1 diff --git a/tutorials/full_ecModel/protocol.py b/tutorials/full_ecModel/protocol.py index ec8fe4c..6709a6c 100644 --- a/tutorials/full_ecModel/protocol.py +++ b/tutorials/full_ecModel/protocol.py @@ -263,7 +263,7 @@ # %% [markdown] # Besides step-wise sensitivity tuning, kcats can be fitted to # experimental growth rates and exchange fluxes with CMA-ES -# (`geckopy.kcat_tuning.evotune`; see docs/evotune_kcat_tuning.md): +# (`geckopy.kcat_tuning.evotune`; see https://gecko-docs.readthedocs.io/en/latest/guide/evotune.html): # # ```python # from geckopy.kcat_tuning.evotune import (