Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,24 @@
* `get_model_from_homology` with `complex_policy="drop"` removes each complex that has a subunit without an ortholog, and removes the reaction only when no complex remains (was: the reaction was removed as soon as one subunit had no ortholog). For the rule `(A and B) or (C and D)` without an ortholog of `B`, the draft has `C and D`.
* `manipulation.contract_model` is the new name of `remove_duplicate_reactions`, after RAVEN's `contractModel`, and the old name is removed. `contract_model` takes `dist_reverse` (default `True`), the `distReverse` option of `contractModel`: with `False`, reactions with the same metabolites and a different reversibility are duplicates.

### Experiments

* New: the `experiments` package scores a model against measured growth rates and exchange fluxes. `load_experiment_data` reads `fluxData.tsv`, `maxGrowth.tsv` and `zeroFlux.tsv` from a folder, `load_flux_data` parses one flux file into a `FluxData` (with a per-condition `weight`), `simulate_conditions` fixes the uptake of each condition and solves the model, `compute_carbon_weights` gives each exchange the carbon count of its metabolite, `experiment_distance` combines the carbon-weighted RMSE of flux and max-growth conditions into one value, and `apply_flux_data_constraints` sets the exchange bounds of a model from one condition (loose, or a percentage band). The organism-specific steps, an anaerobic switch and a change of the protein content, and the solver call are passed as functions.

### Solver

* New: `utils.solver` holds the support for optlang's scipy (HiGHS) interface, next to `set_raven_tolerance`. `repair_scipy_problem` renumbers the variable columns after a removal and rebuilds a stale coefficient matrix. `switch_to_scipy` and `edit_on_fast_solver` avoid the dense-matrix cost of building or editing a large model. `optimize_model` returns a solution with its status for an infeasible or unbounded LP instead of raising, and `constraint_shadow_prices` reads the row marginals on the scipy interface.

### Biomass

* New: `biomass.fit_maintenance` fits NGAM from a maintenance coefficient and GAM from chemostat substrate uptake, with a study-weighted Huber loss and a bootstrap over studies. The steps are also available separately (`fit_gam`, `bootstrap_gam`, `uptake_grid`, `min_uptake`, `ngam_from_maintenance_coefficient`, `study_weights`, `huber_loss`, `log_ratio`), and `MaintenanceConfig` holds the reaction ids and metabolite names.
* New: `get_gam`, and `set_gam` takes `keep_offsets_of`: the coefficient of that metabolite becomes the value and the other GAM terms shift by the same amount, so a charge-balancing proton offset stays.

### Proteins and annotation

* New: `utils.protein.calculate_mw` computes the molecular weight of a protein from its amino acid sequence (average residue masses plus one water; the codes B, J, O, U, X and Z are recognised; an empty sequence gives NaN).
* New: `annotation.find_met_smiles` fills `annotation["smiles"]` of the metabolites from PubChem, with a local TSV cache (`cache_path`) that is appended to after each lookup, and `annotation.met_smiles` reads the SMILES of a metabolite whether the annotation is a string or a list.

### Data

* The KEGG assets are the `kegg120` release of `toolbox-assets` (was: `kegg118`): the reference model and tables, the taxonomy and the profile HMM libraries of 4246 prokaryotic and 2724 eukaryotic KOs. The baked registry and `data/manifest.json` list them.
Expand Down
5 changes: 4 additions & 1 deletion src/raven_toolbox/annotation/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Annotation helpers: SBO term assignment and ΔG CSV persistence.
"""Annotation helpers: SBO term assignment, ΔG CSV persistence and SMILES lookup.

These are the parts of yeast-GEM's ``missingFields`` module that are
organism-agnostic. Default parameter values
Expand All @@ -7,6 +7,7 @@
overrides.
"""
from raven_toolbox.annotation.delta_g import load_delta_g_csv, save_delta_g_csv
from raven_toolbox.annotation.smiles import find_met_smiles, met_smiles
from raven_toolbox.annotation.sbo import (
DEFAULT_BIOMASS_MET_NAMES,
DEFAULT_BIOMASS_RXN_NAME,
Expand All @@ -19,6 +20,8 @@
"DEFAULT_BIOMASS_RXN_NAME",
"DEFAULT_NGAM_RXN_NAME",
"add_sbo_terms",
"find_met_smiles",
"load_delta_g_csv",
"met_smiles",
"save_delta_g_csv",
]
233 changes: 233 additions & 0 deletions src/raven_toolbox/annotation/smiles.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,233 @@
"""SMILES lookup via PubChem with a local TSV cache.

The cache file is a two-column TSV with no header: ``name<TAB>smiles``.
Each successful lookup is appended immediately so interrupted runs
resume gracefully.
"""
from __future__ import annotations

import logging
import re
import time
from pathlib import Path
from typing import TYPE_CHECKING
from urllib.parse import quote

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

if TYPE_CHECKING:
import cobra


__all__ = ["find_met_smiles", "met_smiles"]

logger = logging.getLogger(__name__)

# PubChem deprecated the `CanonicalSMILES` property in 2025 in favour of
# `SMILES` (its canonical/absolute SMILES); `ConnectivitySMILES` replaces
# the old `IsomericSMILES`.
_PUBCHEM_URL = (
"https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/"
"{name}/property/SMILES/TXT"
)
_TIMEOUT_SECONDS = 30
_NUM_RETRIES = 10
_INTER_REQUEST_SLEEP = 0.05 # seconds; PubChem allows ~5 req/s
_PROT_PATTERN = re.compile(r"^prot_")
_FIRST_LINE_PATTERN = re.compile(r"^(\S*)\n")


def met_smiles(metabolite) -> str:
"""The metabolite's SMILES, whatever shape its annotation is in.

cobra stores an annotation value as the object it was assigned, and
returns a *list* for a model read back from a file: a metabolite
annotated in memory by :func:`find_met_smiles` has ``"CCO"``,
and the same metabolite after it is saved and loaded has
``["CCO"]``. Readers that assume a string get a list that compares
equal to nothing and is not hashable.

Returns the first non-empty entry, or ``""`` when there is none.
"""
value = metabolite.annotation.get("smiles", "")
if isinstance(value, (list, tuple, set)):
return next((str(v) for v in value if v), "")
return str(value) if value else ""


def find_met_smiles(
model: "cobra.Model",
*,
cache_path: str | Path,
) -> None:
"""Populate ``metabolite.annotation['smiles']`` from PubChem.

For every metabolite in the model:

1. Skip if the name starts with ``prot_`` (enzyme pseudometabolite).
2. Skip, with a warning, if the metabolite has no name.
3. Skip if ``annotation['smiles']`` is already set.
4. Look up the metabolite name in the local TSV cache. When the cache
holds a name more than once, the last row is used.
5. If still not found, query PubChem; cache the result.

The same metabolite name may appear on many ``cobra.Metabolite``
instances (different compartments). Lookup is done once per unique
name and the SMILES is applied to every metabolite sharing that name.

Parameters
----------
model
A cobra.Model. Mutated in place: each metabolite gains
``annotation['smiles']`` when a SMILES is found.
cache_path
Path to the SMILES cache TSV. The file is created if missing and
appended to as new SMILES are downloaded.
"""
cache_path = Path(cache_path)
cache_path.parent.mkdir(parents=True, exist_ok=True)

# Build map: metabolite name -> list of metabolite objects sharing it.
name_to_mets: dict[str, list] = {}
unnamed = []
for met in model.metabolites:
name = met.name
if not name:
unnamed.append(met.id)
continue
if _PROT_PATTERN.match(name):
continue
if met.annotation.get("smiles"):
continue
name_to_mets.setdefault(name, []).append(met)

if unnamed:
logger.warning(
"find_met_smiles: %d metabolite(s) have no name and get no SMILES: %s",
len(unnamed), ", ".join(unnamed),
)

if not name_to_mets:
logger.info("find_met_smiles: nothing to look up.")
return

# Load existing cache.
cache = _load_cache(cache_path)
logger.info(
"find_met_smiles: %d unique metabolite name(s) need SMILES; "
"%d already in cache.",
len(name_to_mets),
sum(1 for n in name_to_mets if n in cache),
)

# Apply cached SMILES first (no network).
for name in list(name_to_mets):
if name in cache:
smiles = cache[name]
for met in name_to_mets[name]:
if smiles:
met.annotation["smiles"] = smiles
del name_to_mets[name]

if not name_to_mets:
logger.info("find_met_smiles: all matches resolved from cache.")
return

# Hit PubChem for the rest.
session = _make_session()
n_found = 0
n_missing = 0

for name in sorted(name_to_mets):
smiles = _fetch_one_smiles(session, name)
# Append to cache immediately, including empty results so we
# do not retry next time.
_append_cache(cache_path, name, smiles)

if smiles:
for met in name_to_mets[name]:
met.annotation["smiles"] = smiles
n_found += 1
else:
n_missing += 1
logger.debug("No SMILES for: %s", name)

time.sleep(_INTER_REQUEST_SLEEP)

total = n_found + n_missing
if total > 0:
pct = 100.0 * n_found / total
logger.info(
"find_met_smiles: SMILES found for %d/%d unique names (%.0f%%); "
"%d missing.",
n_found, total, pct, n_missing,
)


def _load_cache(path: Path) -> dict[str, str]:
"""Load the two-column TSV cache. Returns an empty dict if missing."""
cache: dict[str, str] = {}
if not path.is_file():
return cache
with open(path, "r", encoding="utf-8") as f:
for line in f:
parts = line.rstrip("\n").split("\t")
if len(parts) >= 2:
cache[parts[0]] = parts[1]
elif len(parts) == 1 and parts[0]:
cache[parts[0]] = ""
return cache


def _append_cache(path: Path, name: str, smiles: str) -> None:
"""Append one row to the TSV cache. Open and close per call so
interrupted runs leave a consistent file."""
with open(path, "a", encoding="utf-8") as f:
f.write(f"{name}\t{smiles}\n")


def _make_session() -> requests.Session:
"""A requests.Session with retry/backoff for transient errors."""
session = requests.Session()
retry = Retry(
total=_NUM_RETRIES,
backoff_factor=0.5,
status_forcelist=(429, 500, 502, 503, 504),
allowed_methods=("GET",),
)
adapter = HTTPAdapter(max_retries=retry)
session.mount("https://", adapter)
session.mount("http://", adapter)
return session


def _fetch_one_smiles(session: requests.Session, name: str) -> str:
"""Fetch SMILES from PubChem for one metabolite name. Returns
empty string if unfound or unrecoverable error."""
url = _PUBCHEM_URL.format(name=quote(name, safe=""))
try:
response = session.get(url, timeout=_TIMEOUT_SECONDS)
except requests.RequestException as e:
logger.warning("PubChem request failed for %r: %s", name, e)
return ""

if response.status_code in (400, 404, 500):
return ""
if response.status_code != 200:
logger.warning(
"PubChem returned HTTP %d for %r; treating as no match.",
response.status_code, name,
)
return ""

text = response.text
if not text:
return ""

# Multiple SMILES may come on separate lines; take only the first.
match = _FIRST_LINE_PATTERN.match(text)
if match:
return match.group(1).strip()
return text.split("\n", 1)[0].strip()
30 changes: 29 additions & 1 deletion src/raven_toolbox/biomass/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,30 @@
scale_biomass(model, cfg, "protein", 0.50, balance_out="carbohydrate")
set_gam(model, 80, biomass_rxn=cfg.biomass_rxn,
cofactor_met_names=("ATP", "ADP", "H2O", "H+", "phosphate"))

GAM and NGAM are fitted to chemostat uptakes and a maintenance coefficient with
:func:`fit_maintenance` (see :mod:`raven_toolbox.biomass.maintenance`).
"""
from raven_toolbox.biomass.config import BiomassComponent, BiomassConfig
from raven_toolbox.biomass.fit_parameters import (
FitParametersResult,
ParameterPosition,
fit_parameters,
)
from raven_toolbox.biomass.gam import set_gam
from raven_toolbox.biomass.gam import get_gam, set_gam
from raven_toolbox.biomass.maintenance import (
MaintenanceConfig,
MaintenanceFit,
bootstrap_gam,
fit_gam,
fit_maintenance,
huber_loss,
log_ratio,
min_uptake,
ngam_from_maintenance_coefficient,
study_weights,
uptake_grid,
)
from raven_toolbox.biomass.scale import (
rescale_pseudoreaction,
scale_biomass,
Expand All @@ -44,10 +60,22 @@
"BiomassComponent",
"BiomassConfig",
"FitParametersResult",
"MaintenanceConfig",
"MaintenanceFit",
"ParameterPosition",
"bootstrap_gam",
"fit_gam",
"fit_maintenance",
"fit_parameters",
"get_gam",
"huber_loss",
"log_ratio",
"min_uptake",
"ngam_from_maintenance_coefficient",
"rescale_pseudoreaction",
"scale_biomass",
"set_gam",
"study_weights",
"sum_biomass",
"uptake_grid",
]
Loading
Loading