diff --git a/CHANGELOG.md b/CHANGELOG.md index 6892c2d4..280f1194 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ * `getModelFromHomology` with `preferredOrder` treats a gene of a later model that maps to a new gene an earlier model already maps as a subunit without an ortholog, and `complexPolicy` decides what happens to it (was: the gene and every part of the rule that needs it were removed). For the rule `(a and b) or x` with `a` taken by an earlier model, `flag` gives `(OLD_MODELID_a and nb) or nx`, `keep` gives `nb or nx` and `drop` gives `nx`. A reaction that has genes, none of which keeps a mapping, is not included. * `getModelFromHomology` with `complexPolicy` `drop` removes each complex that has a subunit without an ortholog, and removes the reaction only when no complex remains (was: the reaction was removed as soon as one subunit had no ortholog). For the rule `(A and B) or (C and D)` without an ortholog of `B`, the draft has the orthologs of `C` and `D`. +### Experiments + +* New: the `experiments/` folder scores a model against measured growth rates and exchange fluxes. `loadFluxData` reads one flux file, with a `weight` column for the weight of each condition, and `loadExperimentData` reads `fluxData.tsv`, `maxGrowth.tsv` and `zeroFlux.tsv` from a folder. `simulateConditions` fixes the uptake of each condition and solves the model; the anaerobic switch, the change of the protein content and the solver call are given as function handles. `computeCarbonWeights` gives the carbon weight of each reaction, `datasetRmse` and `experimentDistance` combine the carbon-weighted errors of the flux and maximum growth conditions into one value, and `applyFluxDataConstraints` bounds the exchange fluxes of a model to one condition. + ### Fixes * `getModelFromHomology` reads the rule as a tree, in which `and` binds tighter than `or`, when it removes the placeholder genes of isozymes without an ortholog (was: it removed a placeholder that stood beside an `or` token, also when the placeholder belongs to a complex). A placeholder in an `or` group is removed when the group has a member that is not a placeholder, in nested groups as well. For the rule `(A or B and C) and D` without an ortholog of `B`, where `nA`, `nC` and `nD` are the orthologs of `A`, `C` and `D`, the draft rule is `(nA or (OLD_T_B and nC)) and nD` under `flag`, `(nA or nC) and nD` under `keep` and `nA and nD` under `drop`; it was `nA and nC and nD` under all three. The result differs from before only for rules with unbracketed `and` and `or` or with nested `or` groups. diff --git a/src/experiments/applyFluxDataConstraints.m b/src/experiments/applyFluxDataConstraints.m new file mode 100644 index 00000000..32938614 --- /dev/null +++ b/src/experiments/applyFluxDataConstraints.m @@ -0,0 +1,164 @@ +function model = applyFluxDataConstraints(model, fluxData, bioRxnId, varargin) +% applyFluxDataConstraints Constrain the exchange fluxes of a model to one condition. +% +% Sets the bounds of the biomass reaction to the measured growth rate and the +% bounds of each exchange reaction of fluxData to its measured flux in the +% chosen condition. An exchange without a measurement (NaN) is left as it +% is. A flux of -1000 or 1000 means unconstrained, whatever the setting of +% looseStrictFlux: -1000 gives the bounds -1000 and 0 (free uptake), and 1000 +% gives 0 and 1000 (free excretion). +% +% Parameters +% ---------- +% model : struct +% model with the exchange reactions of fluxData. Not changed; the +% constrained model is returned. +% fluxData : struct +% flux data, as returned by loadFluxData. +% bioRxnId : char +% id of the biomass reaction. +% +% Name-Value Arguments +% -------------------- +% condition : double or char +% index of the condition in fluxData.conds (counted from 1), or its name +% (default 1). +% maxMinGrowth : char +% 'max' sets the upper bound of the biomass reaction to the measured +% growth rate and the lower bound to 0; 'min' sets the lower bound to the +% growth rate and the upper bound to 1000, which suits the minimisation +% of a resource such as the protein pool (default 'max'). +% looseStrictFlux : char or double +% how tightly the exchange fluxes are constrained (default 'loose'): +% +% - 'loose' : one bound is 0 and the other the measured flux, which +% allows zero flux. +% - 0-100 : the bounds are set around the measured flux with this +% percentage in total, so 10 gives 0.95 and 1.05 times the flux. This +% forces a flux through the reaction. +% cSource : char +% id of the preferred carbon source uptake reaction. Its bounds are set +% to 0 and 0 before the flux data is applied, so that the data alone +% determines the carbon source. [] or '' blocks no reaction, and a +% reaction that is not in the model is ignored (default []). +% +% Returns +% ------- +% model : struct +% model with the constrained fluxes. +% +% Notes +% ----- +% The function raises applyFluxDataConstraints:invalidMaxMinGrowth, +% applyFluxDataConstraints:invalidLooseStrictFlux, +% applyFluxDataConstraints:unknownCondition (a name that is not in +% fluxData.conds), applyFluxDataConstraints:conditionOutOfRange and +% applyFluxDataConstraints:missingReaction (an exchange reaction of fluxData +% that is not in the model, or the biomass reaction). +% +% Examples +% -------- +% model = applyFluxDataConstraints(model, fluxData, 'growth'); +% model = applyFluxDataConstraints(model, fluxData, 'growth', ... +% 'condition', 'glucose', 'looseStrictFlux', 10, 'cSource', 'EX_glc'); +% +% See also +% -------- +% loadFluxData, simulateConditions + +p = parseRAVENargs(varargin, { ... + 'condition', 1; ... + 'maxMinGrowth', 'max'; ... + 'looseStrictFlux', 'loose'; ... + 'cSource', []}); +condition = p.condition; +maxMinGrowth = p.maxMinGrowth; +looseStrictFlux = p.looseStrictFlux; +cSource = p.cSource; + +if ~(ischar(maxMinGrowth) && any(strcmp(maxMinGrowth, {'max','min'}))) + error('applyFluxDataConstraints:invalidMaxMinGrowth', ... + 'maxMinGrowth must be ''max'' or ''min''.'); +end +isLoose = ischar(looseStrictFlux) && strcmp(looseStrictFlux, 'loose'); +if ~(isLoose || (isnumeric(looseStrictFlux) && isscalar(looseStrictFlux) ... + && looseStrictFlux >= 0 && looseStrictFlux <= 100)) + error('applyFluxDataConstraints:invalidLooseStrictFlux', ... + 'looseStrictFlux must be ''loose'' or a number in [0, 100].'); +end + +if ischar(condition) || isstring(condition) + condIdx = find(strcmp(fluxData.conds, char(condition)), 1); + if isempty(condIdx) + error('applyFluxDataConstraints:unknownCondition', ... + 'Condition %s is not in fluxData.conds.', char(condition)); + end +else + if ~(isscalar(condition) && condition >= 1 && condition <= numel(fluxData.conds) ... + && condition == round(condition)) + error('applyFluxDataConstraints:conditionOutOfRange', ... + 'Condition index %g is out of range (fluxData has %d condition(s)).', ... + condition, numel(fluxData.conds)); + end + condIdx = condition; +end + +[found, exchIdx] = ismember(fluxData.exchRxnIDs, model.rxns); +if ~all(found) + missing = fluxData.exchRxnIDs(~found); + error('applyFluxDataConstraints:missingReaction', ... + '%d exchange reaction(s) of fluxData are not in the model (examples: %s).', ... + numel(missing), strjoin(missing(1:min(5, end)), ', ')); +end +bioIdx = find(strcmp(model.rxns, bioRxnId)); +if isempty(bioIdx) + error('applyFluxDataConstraints:missingReaction', ... + 'The biomass reaction %s is not in the model.', bioRxnId); +end + +if ~isempty(cSource) + cIdx = find(strcmp(model.rxns, cSource)); + model.lb(cIdx) = 0; + model.ub(cIdx) = 0; +end + +growth = fluxData.grRate(condIdx); +if strcmp(maxMinGrowth, 'max') + model.lb(bioIdx) = 0; + model.ub(bioIdx) = growth; +else + model.lb(bioIdx) = growth; + model.ub(bioIdx) = 1000; +end + +fluxes = fluxData.exchFluxes(condIdx, :); +for k = 1:numel(fluxes) + flux = fluxes(k); + if isnan(flux) + continue + end + idx = exchIdx(k); + if abs(flux) == 1000 + if flux == -1000 + model.lb(idx) = -1000; + model.ub(idx) = 0; + else + model.lb(idx) = 0; + model.ub(idx) = 1000; + end + elseif isLoose + if flux < 0 + model.lb(idx) = flux; + model.ub(idx) = 0; + else + model.lb(idx) = 0; + model.ub(idx) = flux; + end + else + lo = flux * (1 - looseStrictFlux / 200); + hi = flux * (1 + looseStrictFlux / 200); + model.lb(idx) = min(lo, hi); + model.ub(idx) = max(lo, hi); + end +end +end diff --git a/src/experiments/computeCarbonWeights.m b/src/experiments/computeCarbonWeights.m new file mode 100644 index 00000000..b9815281 --- /dev/null +++ b/src/experiments/computeCarbonWeights.m @@ -0,0 +1,103 @@ +function carbonWeights = computeCarbonWeights(model, rxnIds, bioRxnId) +% computeCarbonWeights Carbon weight of each reaction for the distance to measurements. +% +% The distance between simulated and measured fluxes compares fluxes of +% different metabolites, so each exchange flux is multiplied by the number of +% carbon atoms of the metabolite it exchanges, which gives Cmmol/gDCWh. The +% growth rate is put on the same scale with the assumed carbon content of +% biomass, 41 Cmmol/gDCW. +% +% Parameters +% ---------- +% model : struct +% model that provides the metabolite formulas (metFormulas). +% rxnIds : cell +% reaction ids to compute weights for, typically the exchange reactions +% of the datasets, the zero-flux reactions and the biomass reaction. +% bioRxnId : char +% id of the biomass reaction. +% +% Returns +% ------- +% carbonWeights : containers.Map +% weight per reaction id. The biomass reaction gets 41. Every other +% reaction gets the number of carbon atoms in the formula of its single +% exchanged metabolite. A reaction with more than one metabolite, a +% metabolite without a formula and a metabolite with 0 carbon atoms (O2, +% water) get the weight 1, so their flux mismatches still count in the +% distance. +% +% Notes +% ----- +% A model without formulas gives every exchange the weight 1 and a warning +% (computeCarbonWeights:noFormulas). A reaction that is not in the model +% raises computeCarbonWeights:missingReaction. +% +% Examples +% -------- +% carbonWeights = computeCarbonWeights(model, {'EX_glc','EX_etoh','growth'}, 'growth'); +% carbonWeights('EX_glc') +% +% See also +% -------- +% datasetRmse, experimentDistance + +rxnIds = cellstr(rxnIds); +biomassCarbon = 41; % Cmmol/gDCW, assumed carbon content of biomass + +if ~isfield(model, 'metFormulas') || all(cellfun(@isempty, model.metFormulas)) + warning('computeCarbonWeights:noFormulas', ... + 'The model has no metabolite formulas; every exchange reaction gets the weight 1.'); + formulas = repmat({''}, numel(model.mets), 1); +else + formulas = model.metFormulas; +end + +[found, rxnIdx] = ismember(rxnIds, model.rxns); +if ~all(found) + error('computeCarbonWeights:missingReaction', ... + 'The reactions %s are not in the model.', strjoin(rxnIds(~found), ', ')); +end + +carbonWeights = containers.Map('KeyType', 'char', 'ValueType', 'double'); +for i = 1:numel(rxnIds) + if strcmp(rxnIds{i}, bioRxnId) + carbonWeights(rxnIds{i}) = biomassCarbon; + continue + end + mets = find(model.S(:, rxnIdx(i))); + value = 1; + if numel(mets) == 1 + carbon = carbonAtoms(formulas{mets}); + if ~isnan(carbon) && carbon ~= 0 + value = carbon; + end + end + carbonWeights(rxnIds{i}) = value; +end +end + +function carbon = carbonAtoms(formula) +% Number of carbon atoms in a formula, NaN when the formula has no carbon or +% cannot be read as a plain list of elements and counts. +carbon = NaN; +if isempty(formula) + return +end +tokens = regexp(formula, '([A-Z][a-z]?)(\d*\.?\d*)', 'tokens'); +if isempty(tokens) || ~strcmp(strjoin(cellfun(@(t) [t{1} t{2}], tokens, 'UniformOutput', false), ''), formula) + return +end +for i = 1:numel(tokens) + if strcmp(tokens{i}{1}, 'C') + count = str2double(tokens{i}{2}); + if isnan(count) + count = 1; + end + if isnan(carbon) + carbon = 0; + end + carbon = carbon + count; + end +end +end diff --git a/src/experiments/datasetRmse.m b/src/experiments/datasetRmse.m new file mode 100644 index 00000000..36b910b2 --- /dev/null +++ b/src/experiments/datasetRmse.m @@ -0,0 +1,112 @@ +function [rmse, rmseList] = datasetRmse(fluxData, sims, constrain, carbonWeights, bioRxnId, varargin) +% datasetRmse Carbon-weighted distance between a simulated and a measured dataset. +% +% For each condition the root mean square error is computed over the +% carbon-weighted growth rate, and if constrain is true also over the +% carbon-weighted measured exchange fluxes and the zero-flux reactions. A +% zero-flux reaction that is simulated at exactly 0 is left out so that +% correct predictions do not dilute the error. An infeasible condition gets +% a fixed penalty. The distance of the dataset is the mean of the errors of +% its conditions, each multiplied by the weight of the condition if the data +% has one. +% +% Parameters +% ---------- +% fluxData : struct +% dataset that was simulated, as returned by loadFluxData. +% sims : struct +% simulation result of the dataset, from simulateConditions. +% constrain : logical +% must be the value that was given to simulateConditions: true includes +% the exchange flux terms (flux data), false scores the growth alone +% (maximum growth data). +% carbonWeights : containers.Map +% weights from computeCarbonWeights. A reaction that is not in the map +% gets the weight 1. +% bioRxnId : char +% id of the biomass reaction. It is not read at the moment; the growth +% rate of the simulation is taken from sims. +% +% Name-Value Arguments +% -------------------- +% penalty : double +% error of an infeasible condition (default 99). It must dominate the +% errors of feasible conditions. +% +% Returns +% ------- +% rmse : double +% weighted mean of the errors of the conditions. +% rmseList : double +% error of each condition, with the penalty but before weighting. +% +% Notes +% ----- +% A sims with another number of elements than fluxData has conditions raises +% datasetRmse:wrongSimCount. +% +% Examples +% -------- +% [rmse, rmseList] = datasetRmse(fluxData, sims, true, carbonWeights, 'growth'); +% +% See also +% -------- +% computeCarbonWeights, simulateConditions, experimentDistance + +p = parseRAVENargs(varargin, {'penalty', 99}); +penalty = p.penalty; +biomassCarbon = 41; % Cmmol/gDCW, assumed carbon content of biomass + +n = numel(fluxData.conds); +if numel(sims) ~= n + error('datasetRmse:wrongSimCount', ... + 'sims has %d elements; expected %d to match fluxData.conds.', numel(sims), n); +end + +rmseList = zeros(n, 1); +for i = 1:n + sim = sims(i); + if ~sim.feasible + rmseList(i) = penalty; + continue + end + bioMeas = fluxData.grRate(i) * biomassCarbon; + bioSim = sim.growth * biomassCarbon; + if ~constrain + rmseList(i) = abs(bioMeas - bioSim); + continue + end + + measured = bioMeas; + simulated = bioSim; + for k = 1:numel(sim.exchRxnIDs) + w = weightOf(carbonWeights, sim.exchRxnIDs{k}); + col = find(strcmp(fluxData.exchRxnIDs, sim.exchRxnIDs{k}), 1); + measured(end+1) = w * fluxData.exchFluxes(i, col); %#ok + simulated(end+1) = w * sim.exchFluxes(k); %#ok + end + for k = 1:numel(sim.blockRxnIDs) + weightedSim = weightOf(carbonWeights, sim.blockRxnIDs{k}) * sim.blockFluxes(k); + if weightedSim == 0 + continue + end + measured(end+1) = 0; %#ok + simulated(end+1) = weightedSim; %#ok + end + rmseList(i) = sqrt(mean((measured - simulated).^2)); +end + +weighted = rmseList; +if isfield(fluxData, 'weight') && ~isempty(fluxData.weight) + weighted = rmseList .* fluxData.weight(:); +end +rmse = mean(weighted); +end + +function w = weightOf(carbonWeights, rxnId) +if isKey(carbonWeights, rxnId) + w = carbonWeights(rxnId); +else + w = 1; +end +end diff --git a/src/experiments/experimentDistance.m b/src/experiments/experimentDistance.m new file mode 100644 index 00000000..e49ef5a1 --- /dev/null +++ b/src/experiments/experimentDistance.m @@ -0,0 +1,91 @@ +function [rmse, detail] = experimentDistance(experimentData, fluxSims, maxGrowthSims, carbonWeights, bioRxnId, varargin) +% experimentDistance Combine the distances of both datasets into one value. +% +% The distance of the flux dataset and of the maximum growth dataset (see +% datasetRmse) are combined as (rmseFlux + w * rmseMaxGrowth) / (1 + w), +% where w is maxGrowthWeight. A dataset that is not in experimentData does +% not contribute. +% +% Parameters +% ---------- +% experimentData : struct +% measured conditions, as returned by loadExperimentData. +% fluxSims : struct +% simulation result of experimentData.fluxData from simulateConditions +% with constrain true. [] if experimentData has no fluxData. +% maxGrowthSims : struct +% simulation result of experimentData.maxGrowth from simulateConditions +% with constrain false. [] if experimentData has no maxGrowth. +% carbonWeights : containers.Map +% weights from computeCarbonWeights. +% bioRxnId : char +% id of the biomass reaction. +% +% Name-Value Arguments +% -------------------- +% penalty : double +% error of an infeasible condition (default 99). +% maxGrowthWeight : double +% relative weight of the maximum growth dataset against the flux dataset +% (default 1). With 2, the maximum growth conditions count twice as much +% as the flux conditions, which is useful when the carbon source +% diversity is in the maximum growth dataset and the flux conditions are +% mostly one carbon source. +% +% Returns +% ------- +% rmse : double +% combined distance, NaN if experimentData has neither dataset. +% detail : struct +% error of each condition, in the fields fluxData and maxGrowth for the +% datasets that were scored. +% +% Notes +% ----- +% A dataset in experimentData without simulation result raises +% experimentDistance:missingSims. +% +% Examples +% -------- +% [rmse, detail] = experimentDistance(experimentData, fluxSims, ... +% maxGrowthSims, carbonWeights, 'growth', 'maxGrowthWeight', 2); +% +% See also +% -------- +% datasetRmse, simulateConditions, computeCarbonWeights + +p = parseRAVENargs(varargin, {'penalty', 99; 'maxGrowthWeight', 1}); +penalty = p.penalty; +maxGrowthWeight = p.maxGrowthWeight; + +parts = []; +weights = []; +detail = struct(); +if ~isempty(experimentData.fluxData) + if isempty(fluxSims) + error('experimentDistance:missingSims', ... + 'experimentData.fluxData is set but fluxSims is empty.'); + end + [r, perCond] = datasetRmse(experimentData.fluxData, fluxSims, true, ... + carbonWeights, bioRxnId, 'penalty', penalty); + parts(end+1) = r; + weights(end+1) = 1; + detail.fluxData = perCond; +end +if ~isempty(experimentData.maxGrowth) + if isempty(maxGrowthSims) + error('experimentDistance:missingSims', ... + 'experimentData.maxGrowth is set but maxGrowthSims is empty.'); + end + [r, perCond] = datasetRmse(experimentData.maxGrowth, maxGrowthSims, false, ... + carbonWeights, bioRxnId, 'penalty', penalty); + parts(end+1) = r; + weights(end+1) = maxGrowthWeight; + detail.maxGrowth = perCond; +end +if isempty(parts) + rmse = NaN; +else + rmse = sum(parts .* weights) / sum(weights); +end +end diff --git a/src/experiments/loadExperimentData.m b/src/experiments/loadExperimentData.m new file mode 100644 index 00000000..19d9c443 --- /dev/null +++ b/src/experiments/loadExperimentData.m @@ -0,0 +1,57 @@ +function experimentData = loadExperimentData(dataDir) +% loadExperimentData Load the measured conditions of a model from a folder. +% +% Reads fluxData.tsv, maxGrowth.tsv and zeroFlux.tsv from a folder. The +% first two have the layout described in loadFluxData; fluxData.tsv holds +% conditions with measured uptake and excretion fluxes, and maxGrowth.tsv +% holds one carbon source per condition, which is opened fully when the +% maximum growth rate is simulated. zeroFlux.tsv has the header Rxns and one +% reaction id per line. Each file is optional: a missing file gives [] (or an +% empty cell array for zeroFlux), not an error. +% +% Parameters +% ---------- +% dataDir : char +% folder that holds the files. +% +% Returns +% ------- +% experimentData : struct +% structure with the fields: +% +% - fluxData : the structure of loadFluxData for fluxData.tsv, or []. +% - maxGrowth : the structure of loadFluxData for maxGrowth.tsv, or []. +% - zeroFlux : cell column with the reaction ids that carry no flux in +% any condition, or {}. +% +% Examples +% -------- +% experimentData = loadExperimentData('data'); +% +% See also +% -------- +% loadFluxData, simulateConditions, experimentDistance + +experimentData.fluxData = loadOptionalFluxData(fullfile(dataDir, 'fluxData.tsv')); +experimentData.maxGrowth = loadOptionalFluxData(fullfile(dataDir, 'maxGrowth.tsv')); +experimentData.zeroFlux = loadZeroFlux(fullfile(dataDir, 'zeroFlux.tsv')); +end + +function fluxData = loadOptionalFluxData(file) +fluxData = []; +if isfile(file) + fluxData = loadFluxData(file); +end +end + +function zeroFlux = loadZeroFlux(file) +zeroFlux = {}; +if ~isfile(file) + return +end +lines = splitlines(string(fileread(file))); +lines = strtrim(erase(lines, char(13))); +lines = lines(2:end); +lines = lines(strlength(lines) > 0); +zeroFlux = cellstr(lines); +end diff --git a/src/experiments/loadFluxData.m b/src/experiments/loadFluxData.m new file mode 100644 index 00000000..82bb923c --- /dev/null +++ b/src/experiments/loadFluxData.m @@ -0,0 +1,172 @@ +function fluxData = loadFluxData(fluxDataFile) +% loadFluxData Load measured growth rates and exchange fluxes of conditions. +% +% Reads a tab-delimited file with one row per condition: the columns +% condition, Ptot and grRate, followed by one column per exchange reaction +% with the header ' ()'. The columns weight and +% source are optional and may stand in any position. +% +% Parameters +% ---------- +% fluxDataFile : char +% path to the tab-delimited file. +% +% Returns +% ------- +% fluxData : struct +% structure with the fields: +% +% - conds : cell column with the names of the conditions. +% - Ptot : total protein content per condition (g/gDCW). +% - grRate : growth rate per condition (1/h). +% - exchFluxes : exchange fluxes, conditions in rows (mmol/gDCWh). NaN +% means that the flux was not measured; -1000 and 1000 mean +% unconstrained uptake and excretion. +% - exchMets : names of the exchanged metabolites, matching the columns +% of exchFluxes. +% - exchRxnIDs : exchange reaction ids, matching the columns of +% exchFluxes. +% - weight : only if the file has the column weight. Weight of each +% condition in the distance to the measurements; 0 leaves the +% condition out. +% - source : only if the file has the column source. Where the data +% comes from. +% +% Notes +% ----- +% An exchange column header is split at the parenthesis that matches the +% final closing one, so a reaction id may contain parentheses ('X (EX_a(e))' +% gives the metabolite 'X' and the reaction 'EX_a(e)'). A header without +% that form is used as both metabolite name and reaction id, with a warning. +% Empty and non-numeric cells are read as NaN, and empty lines are skipped. +% +% The function raises loadFluxData:fileNotFound when the file does not exist, +% loadFluxData:emptyFile when it has no content, loadFluxData:noHeader when +% the header has fewer than three columns, loadFluxData:raggedRow when a line +% has a different number of cells than the header, +% loadFluxData:renamedColumn when the file has the column evotuneRMSEweight +% (now called weight), and loadFluxData:noExchangeColumns when the file has +% no exchange columns. +% +% Examples +% -------- +% fluxData = loadFluxData('data/fluxData.tsv'); +% +% See also +% -------- +% loadExperimentData, applyFluxDataConstraints, simulateConditions + +if ~isfile(fluxDataFile) + error('loadFluxData:fileNotFound', 'The flux data file %s does not exist.', fluxDataFile); +end +lines = splitlines(string(fileread(fluxDataFile))); +lines = erase(lines, char(13)); +lines = lines(strlength(strtrim(lines)) > 0); +if isempty(lines) + error('loadFluxData:emptyFile', 'The flux data file %s is empty.', fluxDataFile); +end + +raw = cell(numel(lines), 1); +for i = 1:numel(lines) + raw{i} = cellstr(strsplit(lines(i), char(9), 'CollapseDelimiters', false)); +end +header = raw{1}; +if numel(header) < 3 + error('loadFluxData:noHeader', ... + '%s: the header has %d column(s); expected at least condition, Ptot and grRate.', ... + fluxDataFile, numel(header)); +end +for i = 2:numel(raw) + if numel(raw{i}) ~= numel(header) + error('loadFluxData:raggedRow', ... + 'Line %d of %s has %d cells, the header has %d.', ... + i, fluxDataFile, numel(raw{i}), numel(header)); + end +end +cells = cell(numel(raw), numel(header)); +for i = 1:numel(raw) + cells(i, :) = raw{i}; +end +header = strtrim(header); + +if any(strcmp(header, 'evotuneRMSEweight')) + error('loadFluxData:renamedColumn', ... + '%s: the column ''evotuneRMSEweight'' is now called ''weight''.', fluxDataFile); +end + +fluxData = struct(); +extra = false(1, numel(header)); +weightIdx = find(strcmp(header, 'weight'), 1); +sourceIdx = find(strcmp(header, 'source'), 1); +if ~isempty(weightIdx) + fluxData.weight = str2double(cells(2:end, weightIdx)); + extra(weightIdx) = true; +end +if ~isempty(sourceIdx) + fluxData.source = cells(2:end, sourceIdx); + extra(sourceIdx) = true; +end + +kept = find(~extra); +if numel(kept) < 4 + error('loadFluxData:noExchangeColumns', ... + '%s has no exchange flux columns after the columns condition, Ptot and grRate.', ... + fluxDataFile); +end +exchHeaders = header(kept(4:end)); +exchMets = exchHeaders; +exchRxns = exchHeaders; +noForm = {}; +for i = 1:numel(exchHeaders) + [exchMets{i}, exchRxns{i}, ok] = splitExchHeader(exchHeaders{i}); + if ~ok + noForm{end+1} = exchHeaders{i}; %#ok + end +end +if ~isempty(noForm) + warning('loadFluxData:headerForm', ... + ['%d exchange column header(s) lack the '' ()'' form; their ' ... + 'reaction id is the full header text: %s'], ... + numel(noForm), strjoin(noForm(1:min(5, end)), ', ')); +end + +fluxData.conds = cells(2:end, kept(1)); +fluxData.Ptot = str2double(cells(2:end, kept(2))); +fluxData.grRate = str2double(cells(2:end, kept(3))); +fluxData.exchFluxes = str2double(cells(2:end, kept(4:end))); +fluxData.exchMets = exchMets(:)'; +fluxData.exchRxnIDs = exchRxns(:)'; +% Order the fields as documented. +order = {'conds','Ptot','grRate','exchFluxes','exchMets','exchRxnIDs','weight','source'}; +order = order(isfield(fluxData, order)); +fluxData = orderfields(fluxData, order); +end + +function [met, rxn, ok] = splitExchHeader(header) +% Split ' ()' at the parenthesis that matches the +% final closing one. A header of another form gives the whole header. +header = strtrim(header); +met = header; +rxn = header; +ok = false; +if isempty(header) || header(end) ~= ')' + return +end +depth = 0; +for k = numel(header):-1:1 + if header(k) == ')' + depth = depth + 1; + elseif header(k) == '(' + depth = depth - 1; + if depth == 0 + candidate = strtrim(header(k+1:end-1)); + if ~isempty(candidate) + met = strtrim(header(1:k-1)); + rxn = candidate; + ok = true; + end + return + end + end +end +end diff --git a/src/experiments/simulateConditions.m b/src/experiments/simulateConditions.m new file mode 100644 index 00000000..35eece38 --- /dev/null +++ b/src/experiments/simulateConditions.m @@ -0,0 +1,162 @@ +function sims = simulateConditions(model, fluxData, constrain, zeroFluxRxns, bioRxnId, varargin) +% simulateConditions Simulate every condition of a dataset in a model. +% +% Performs the simulation half of the distance to measurements: for each +% condition of a flux dataset the uptake of the condition's carbon source is +% set, the model is solved, and the growth rate and the fluxes that are +% compared with the measurements are recorded. datasetRmse and +% experimentDistance turn the result into a distance. +% +% The steps for each condition are: +% +% 1. The lower bound of every exchange reaction that is the carbon source of +% some condition is set to 0, so that the carbon source of one condition +% cannot carry flux in another. +% 2. The lower bound of the carbon source of this condition is set to the +% measured uptake rate if constrain is true, or to -1000 if it is false +% (the best possible growth on the carbon source). +% 3. If the oxygen flux of the condition is exactly 0 and makeAnaerobic is +% given, the model is made anaerobic. If Ptot of the condition is given +% and not 0 and changeProteinBiomass is given, the protein content of the +% model is changed. +% 4. The model is solved. If constrain is true, the simulated flux of every +% measured exchange and of every reaction in zeroFluxRxns (except the +% carbon source of the condition) is recorded. +% +% Parameters +% ---------- +% model : struct +% model with all other constraints already applied. +% fluxData : struct +% dataset to simulate, as returned by loadFluxData. The carbon source of +% a condition is the exchange metabolite with the name of the condition. +% constrain : logical +% true for flux data, which fixes the uptake at the measured rate; false +% for maximum growth data, which opens the uptake fully. +% zeroFluxRxns : cell +% reaction ids that carry no flux in any condition. +% bioRxnId : char +% id of the biomass reaction. +% +% Name-Value Arguments +% -------------------- +% makeAnaerobic : function_handle +% function that takes a model and returns the anaerobic model. These +% steps depend on the organism, and there is no default; without the +% function the anaerobic condition is not applied (default []). +% changeProteinBiomass : function_handle +% function that takes a model and Ptot and returns a model with that +% protein content (default []). +% optimize : function_handle +% function that solves a model and returns a structure with the fields +% x, f and stat, as solveLP does (default @solveLP). +% +% Returns +% ------- +% sims : struct +% structure array with one element per condition of fluxData and the +% fields: +% +% - feasible : false if the model is infeasible or the solution not +% optimal. The other fields have no meaning then. +% - growth : flux of the biomass reaction. +% - exchRxnIDs : ids of the measured exchange reactions (empty if +% constrain is false). +% - exchFluxes : simulated fluxes of exchRxnIDs. +% - blockRxnIDs : ids of the zero-flux reactions that were recorded. +% - blockFluxes : simulated fluxes of blockRxnIDs. +% +% Notes +% ----- +% A condition that cannot be matched by name to an exchange metabolite of +% fluxData raises simulateConditions:unmatchedCondition, and a reaction that +% is not in the model raises simulateConditions:missingReaction. +% +% Examples +% -------- +% sims = simulateConditions(model, experimentData.fluxData, true, ... +% experimentData.zeroFlux, 'growth'); +% +% See also +% -------- +% loadExperimentData, datasetRmse, experimentDistance + +p = parseRAVENargs(varargin, { ... + 'makeAnaerobic', []; ... + 'changeProteinBiomass', []; ... + 'optimize', []}); +makeAnaerobic = p.makeAnaerobic; +changeProteinBiomass = p.changeProteinBiomass; +optimize = p.optimize; +if isempty(optimize) + optimize = @(m) solveLP(m); +end +zeroFluxRxns = cellstr(zeroFluxRxns); + +[condToExch, ~] = matchConditions(fluxData); +conditionRxns = unique(fluxData.exchRxnIDs(unique(condToExch))); +allRxns = [conditionRxns(:); fluxData.exchRxnIDs(:); zeroFluxRxns(:); {bioRxnId}]; +missing = unique(allRxns(~ismember(allRxns, model.rxns))); +if ~isempty(missing) + error('simulateConditions:missingReaction', ... + 'The reactions %s are not in the model.', strjoin(missing(:)', ', ')); +end +o2Col = find(strcmp(fluxData.exchMets, 'oxygen'), 1); +bioIdx = find(strcmp(model.rxns, bioRxnId)); +conditionIdx = find(ismember(model.rxns, conditionRxns)); + +nConds = numel(fluxData.conds); +template = struct('feasible', false, 'growth', NaN, 'exchRxnIDs', {{}}, ... + 'exchFluxes', zeros(0, 1), 'blockRxnIDs', {{}}, 'blockFluxes', zeros(0, 1)); +sims = repmat(template, nConds, 1); + +for i = 1:nConds + carbonRxn = fluxData.exchRxnIDs{condToExch(i)}; + carbonIdx = find(strcmp(model.rxns, carbonRxn)); + m = model; + m.lb(conditionIdx) = 0; + if constrain + m.lb(carbonIdx) = fluxData.exchFluxes(i, condToExch(i)); + else + m.lb(carbonIdx) = -1000; + end + + if ~isempty(o2Col) && fluxData.exchFluxes(i, o2Col) == 0 && ~isempty(makeAnaerobic) + m = makeAnaerobic(m); + end + ptot = fluxData.Ptot(i); + if ~(isnan(ptot) || ptot == 0) && ~isempty(changeProteinBiomass) + m = changeProteinBiomass(m, ptot); + end + + sol = optimize(m); + [~, isOptimal] = checkSolution(sol); + if ~isOptimal || isempty(sol.f) || isnan(sol.f) + continue + end + + sims(i).feasible = true; + sims(i).growth = sol.x(bioIdx); + if constrain + measured = find(~isnan(fluxData.exchFluxes(i, :))); + sims(i).exchRxnIDs = fluxData.exchRxnIDs(measured); + [~, exchIdx] = ismember(sims(i).exchRxnIDs, m.rxns); + sims(i).exchFluxes = sol.x(exchIdx); + blockRxns = zeroFluxRxns(~strcmp(zeroFluxRxns, carbonRxn)); + [~, blockIdx] = ismember(blockRxns, m.rxns); + sims(i).blockRxnIDs = blockRxns; + sims(i).blockFluxes = sol.x(blockIdx); + end +end +end + +function [condToExch, missing] = matchConditions(fluxData) +% Column of exchMets for the name of each condition. +[found, condToExch] = ismember(fluxData.conds, fluxData.exchMets); +missing = unique(fluxData.conds(~found)); +if ~isempty(missing) + error('simulateConditions:unmatchedCondition', ... + ['Carbon source(s) "%s" in the provided fluxData or maxGrowth cannot be ' ... + 'matched by name with an exchange reaction.'], strjoin(missing(:)', '; ')); +end +end diff --git a/tests/function_tests/tExperiments.m b/tests/function_tests/tExperiments.m new file mode 100644 index 00000000..197126ce --- /dev/null +++ b/tests/function_tests/tExperiments.m @@ -0,0 +1,589 @@ +classdef tExperiments < RavenTestCase +% tExperiments Tests for the functions in experiments/. +% +% A toy model with two carbon sources feeds every simulation, so that each +% expected value follows from the stoichiometry. Glucose and ethanol feed the +% pool metabolite A_c, which the reaction R turns into biomass and into a +% byproduct (0.1 per unit of R flux). R needs the enzyme prot_E, whose usage +% is capped by a protein pool of 1 mg/gDCW, so R is capped at +% 3600*kcat/MW = 3600/100 = 36. + + properties (Constant) + enzymeCap = 36 + byproductYield = 0.1 + biomassCarbon = 41 + end + + methods (Test) + + %% loadFluxData + + function loadFluxDataMissingFile(testCase) + testCase.verifyError(@() loadFluxData(tempname), 'loadFluxData:fileNotFound'); + end + + function loadFluxDataEmptyFile(testCase) + f = tExperiments.writeFile(testCase, ''); + testCase.verifyError(@() loadFluxData(f), 'loadFluxData:emptyFile'); + end + + function loadFluxDataShortHeader(testCase) + f = tExperiments.writeRows(testCase, {{'only','two'}}); + testCase.verifyError(@() loadFluxData(f), 'loadFluxData:noHeader'); + end + + function loadFluxDataNoExchangeColumns(testCase) + f = tExperiments.writeRows(testCase, ... + {{'condition','Ptot','grRate'}, {'c1','0.5','0.4'}}); + testCase.verifyError(@() loadFluxData(f), 'loadFluxData:noExchangeColumns'); + end + + function loadFluxDataRaggedRow(testCase) + f = tExperiments.writeRows(testCase, { + {'condition','Ptot','grRate','Glc (EX_a)','O2 (EX_b)'}, ... + {'c1','0.5','0.4','-1','2'}, ... + {'c2','0.5','0.4','-1'}}); + testCase.verifyError(@() loadFluxData(f), 'loadFluxData:raggedRow'); + end + + function loadFluxDataOldWeightColumnIsRejected(testCase) + f = tExperiments.writeRows(testCase, { + {'Condition','Ptot','grRate','glucose (r_1714)','evotuneRMSEweight'}, ... + {'c1','0.5','0.4','-10','1'}}); + testCase.verifyError(@() loadFluxData(f), 'loadFluxData:renamedColumn'); + end + + function loadFluxDataMultipleConditions(testCase) + f = tExperiments.writeRows(testCase, { + {'condition','Ptot','grRate','glucose (EX_glc)','oxygen (EX_O2)','ethanol (EX_eth)'}, ... + {'c1','0.5','0.4','-10.0','-2.0','5.0'}, ... + {'c2','0.6','0.5','-8.5','-3.0','7.0'}}); + fd = loadFluxData(f); + testCase.verifyEqual(fd.conds, {'c1';'c2'}); + testCase.verifyEqual(fd.Ptot, [0.5;0.6]); + testCase.verifyEqual(fd.grRate, [0.4;0.5]); + testCase.verifyEqual(fd.exchFluxes, [-10 -2 5; -8.5 -3 7]); + testCase.verifyEqual(fd.exchMets, {'glucose','oxygen','ethanol'}); + testCase.verifyEqual(fd.exchRxnIDs, {'EX_glc','EX_O2','EX_eth'}); + testCase.verifyFalse(isfield(fd, 'weight')); + testCase.verifyFalse(isfield(fd, 'source')); + end + + function loadFluxDataHeaderForms(testCase) + f = tExperiments.writeRows(testCase, { + {'condition','Ptot','grRate','carbon dioxide (EX_co2)','X (EX_a(e))'}, ... + {'c1','0.5','0.4','5.0','1'}}); + fd = loadFluxData(f); + testCase.verifyEqual(fd.exchMets, {'carbon dioxide','X'}); + testCase.verifyEqual(fd.exchRxnIDs, {'EX_co2','EX_a(e)'}); + end + + function loadFluxDataHeaderWithoutParenthesesWarns(testCase) + f = tExperiments.writeRows(testCase, { + {'condition','Ptot','grRate','EX_glc'}, {'c1','0.5','0.4','-10.0'}}); + testCase.verifyWarning(@() loadFluxData(f), 'loadFluxData:headerForm'); + warning('off', 'loadFluxData:headerForm'); + testCase.addTeardown(@() warning('on', 'loadFluxData:headerForm')); + fd = loadFluxData(f); + testCase.verifyEqual(fd.exchMets, {'EX_glc'}); + testCase.verifyEqual(fd.exchRxnIDs, {'EX_glc'}); + end + + function loadFluxDataEmptyAndNonNumericCellsAreNaN(testCase) + f = tExperiments.writeRows(testCase, { + {'condition','Ptot','grRate','glc (EX_glc)','o2 (EX_O2)'}, ... + {'c1','0.5','0.4','-10.0',''}, ... + {'c2','0.6','0.5','NA','-3.0'}}); + fd = loadFluxData(f); + testCase.verifyTrue(isnan(fd.exchFluxes(1, 2))); + testCase.verifyTrue(isnan(fd.exchFluxes(2, 1))); + testCase.verifyEqual(fd.exchFluxes(1, 1), -10); + testCase.verifyEqual(fd.exchFluxes(2, 2), -3); + end + + function loadFluxDataOptionalColumnsInAnyPosition(testCase) + f = tExperiments.writeRows(testCase, { + {'condition','Ptot','grRate','source','glc (EX_glc)','weight'}, ... + {'c1','0.5','0.4','PMID:1','-10.0','1.5'}, ... + {'c2','0.6','0.5','in-house','-8.5','0'}}); + fd = loadFluxData(f); + testCase.verifyEqual(fd.weight, [1.5;0]); + testCase.verifyEqual(fd.source, {'PMID:1';'in-house'}); + testCase.verifyEqual(fd.exchFluxes, [-10;-8.5]); + testCase.verifyEqual(fd.exchRxnIDs, {'EX_glc'}); + end + + function loadFluxDataSkipsBlankLines(testCase) + f = tExperiments.writeFile(testCase, ... + sprintf('condition\tPtot\tgrRate\tglc (EX_glc)\n\nc1\t0.5\t0.4\t-10.0\n\n')); + fd = loadFluxData(f); + testCase.verifyEqual(fd.conds, {'c1'}); + end + + %% loadExperimentData + + function loadExperimentDataAllFiles(testCase) + d = tempname; mkdir(d); + testCase.addTeardown(@() rmdir(d, 's')); + tExperiments.writeNamed(d, 'fluxData.tsv', sprintf(['Condition\tPtot\tgrRate\tglucose (r_1714)\tethanol (r_1761)\tweight\tsource\n' ... + 'glucose\tNaN\t0.36\t-13.33\t19.83\t1\tDLKcat\nethanol\tNaN\t0.12\tNaN\t-8.5\t1\tbrenda\n'])); + tExperiments.writeNamed(d, 'maxGrowth.tsv', sprintf(['Condition\tPtot\tgrRate\tglucose (r_1714)\tfructose (r_1709)\tweight\tsource\n' ... + 'glucose\tNaN\t0.41\t-1000\tNaN\t1\tDLKcat\nfructose\tNaN\t0.338\tNaN\t-1000\t1\tDLKcat\n'])); + tExperiments.writeNamed(d, 'zeroFlux.tsv', sprintf('Rxns\nr_1542\nr_1545\nr_1546\n')); + ed = loadExperimentData(d); + testCase.verifyEqual(ed.fluxData.conds, {'glucose';'ethanol'}); + testCase.verifyEqual(ed.fluxData.exchRxnIDs, {'r_1714','r_1761'}); + testCase.verifyEqual(ed.fluxData.source, {'DLKcat';'brenda'}); + testCase.verifyEqual(ed.fluxData.weight, [1;1]); + testCase.verifyEqual(ed.maxGrowth.conds, {'glucose';'fructose'}); + testCase.verifyEqual(ed.zeroFlux, {'r_1542';'r_1545';'r_1546'}); + end + + function loadExperimentDataMissingFiles(testCase) + d = tempname; mkdir(d); + testCase.addTeardown(@() rmdir(d, 's')); + ed = loadExperimentData(d); + testCase.verifyEmpty(ed.fluxData); + testCase.verifyEmpty(ed.maxGrowth); + testCase.verifyEqual(ed.zeroFlux, {}); + end + + function loadExperimentDataPartial(testCase) + d = tempname; mkdir(d); + testCase.addTeardown(@() rmdir(d, 's')); + tExperiments.writeNamed(d, 'fluxData.tsv', ... + sprintf('Condition\tPtot\tgrRate\tglucose (r_1714)\nglucose\tNaN\t0.36\t-13.33\n')); + ed = loadExperimentData(d); + testCase.verifyNotEmpty(ed.fluxData); + testCase.verifyEmpty(ed.maxGrowth); + testCase.verifyEqual(ed.zeroFlux, {}); + end + + %% computeCarbonWeights + + function computeCarbonWeightsCountsCarbon(testCase) + model = tExperiments.toyModel(); + w = computeCarbonWeights(model, {'EX_glc','EX_eth','EX_byp','biomass'}, 'biomass'); + testCase.verifyEqual(w('EX_glc'), 6); + testCase.verifyEqual(w('EX_eth'), 2); + testCase.verifyEqual(w('EX_byp'), 1); + testCase.verifyEqual(w('biomass'), tExperiments.biomassCarbon); + end + + function computeCarbonWeightsZeroCarbonIsOne(testCase) + % O2 has no carbon; it still gets weight 1 so that its mismatches count. + model = tExperiments.toyModel(); + w = computeCarbonWeights(model, {'EX_o2'}, 'biomass'); + testCase.verifyEqual(w('EX_o2'), 1); + end + + function computeCarbonWeightsWithoutFormulas(testCase) + model = rmfield(tExperiments.toyModel(), 'metFormulas'); + testCase.verifyWarning(@() computeCarbonWeights(model, {'EX_glc'}, 'biomass'), ... + 'computeCarbonWeights:noFormulas'); + warning('off', 'computeCarbonWeights:noFormulas'); + testCase.addTeardown(@() warning('on', 'computeCarbonWeights:noFormulas')); + w = computeCarbonWeights(model, {'EX_glc'}, 'biomass'); + testCase.verifyEqual(w('EX_glc'), 1); + end + + function computeCarbonWeightsMissingReaction(testCase) + model = tExperiments.toyModel(); + testCase.verifyError(@() computeCarbonWeights(model, {'EX_absent'}, 'biomass'), ... + 'computeCarbonWeights:missingReaction'); + end + + %% simulateConditions + + function simulateConditionsEnzymeVersusCarbonLimited(testCase) + model = tExperiments.toyModel(); + sims = simulateConditions(model, tExperiments.fluxDataToy(20), true, {'EX_byp'}, 'biomass'); + testCase.verifySize(sims, [2 1]); + % Glucose: enzyme-limited growth of 36, the whole uptake is used. + testCase.verifyTrue(sims(1).feasible); + testCase.verifyEqual(sims(1).growth, tExperiments.enzymeCap, 'AbsTol', 1e-6); + testCase.verifyEqual(sims(1).exchFluxes(strcmp(sims(1).exchRxnIDs, 'EX_glc')), ... + -tExperiments.enzymeCap, 'AbsTol', 1e-6); + testCase.verifyEqual(sims(1).blockFluxes, tExperiments.enzymeCap * tExperiments.byproductYield, 'AbsTol', 1e-6); + % Ethanol: carbon-limited by the uptake of 20. + testCase.verifyTrue(sims(2).feasible); + testCase.verifyEqual(sims(2).growth, 20, 'AbsTol', 1e-6); + testCase.verifyEqual(sims(2).exchFluxes(strcmp(sims(2).exchRxnIDs, 'EX_eth')), -20, 'AbsTol', 1e-6); + testCase.verifyEqual(sims(2).blockFluxes, 20 * tExperiments.byproductYield, 'AbsTol', 1e-6); + % The model is returned unchanged to the caller. + testCase.verifyEqual(model.lb(strcmp(model.rxns, 'EX_glc')), -1000); + end + + function simulateConditionsMaxGrowthOpensUptake(testCase) + model = tExperiments.toyModel(); + sims = simulateConditions(model, tExperiments.maxGrowthToy(), false, {}, 'biomass'); + testCase.verifyEqual([sims.growth], [36 36], 'AbsTol', 1e-6); + testCase.verifyEmpty(sims(1).exchRxnIDs); + testCase.verifyEmpty(sims(1).blockRxnIDs); + end + + function simulateConditionsUnmatchedCondition(testCase) + model = tExperiments.toyModel(); + fd = tExperiments.fluxDataToy(20); + fd.conds = {'glucose';'propionate'}; + testCase.verifyError(@() simulateConditions(model, fd, true, {}, 'biomass'), ... + 'simulateConditions:unmatchedCondition'); + end + + function simulateConditionsInfeasibleCondition(testCase) + % A growth lower bound above what the enzyme pool allows. + model = tExperiments.toyModel(); + model.lb(strcmp(model.rxns, 'biomass')) = 50; % above the enzyme cap + sims = simulateConditions(model, tExperiments.maxGrowthToy(), false, {}, 'biomass'); + testCase.verifyFalse(any([sims.feasible])); + end + + function simulateConditionsUsesTheGivenOptimizeFunction(testCase) + model = tExperiments.toyModel(); + calls = containers.Map({'n'}, {0}); % a handle, so the count survives the calls + sims = simulateConditions(model, tExperiments.fluxDataToy(20), true, {}, 'biomass', ... + 'optimize', @(m) tExperiments.countedSolve(m, calls)); + testCase.verifyEqual(calls('n'), 2); + testCase.verifyTrue(all([sims.feasible])); + end + + function simulateConditionsAppliesTheHooks(testCase) + % Oxygen measured at exactly 0 calls makeAnaerobic; a Ptot value + % calls changeProteinBiomass with it. + model = tExperiments.toyModel(); + fd = tExperiments.fluxDataToy(20); + fd.exchMets = [fd.exchMets, {'oxygen'}]; + fd.exchRxnIDs = [fd.exchRxnIDs, {'EX_o2'}]; + fd.exchFluxes = [fd.exchFluxes, [0; NaN]]; + fd.Ptot = [NaN; 0.4]; + anaerobic = @(m) tExperiments.capReaction(m, 'R', 10); + protein = @(m, ptot) tExperiments.capReaction(m, 'biomass', ptot*100); + sims = simulateConditions(model, fd, true, {}, 'biomass', ... + 'makeAnaerobic', anaerobic, 'changeProteinBiomass', protein); + testCase.verifyEqual(sims(1).growth, 10, 'AbsTol', 1e-6); % capped by makeAnaerobic + testCase.verifyEqual(sims(2).growth, 20, 'AbsTol', 1e-6); % 40 cap above the carbon limit + % Without hooks nothing is changed. + sims = simulateConditions(model, fd, true, {}, 'biomass'); + testCase.verifyEqual(sims(1).growth, tExperiments.enzymeCap, 'AbsTol', 1e-6); + end + + %% datasetRmse + + function datasetRmseOnlyTheByproductDeviates(testCase) + model = tExperiments.toyModel(); + fd = tExperiments.fluxDataToy(20); + sims = simulateConditions(model, fd, true, {'EX_byp'}, 'biomass'); + w = computeCarbonWeights(model, {'EX_glc','EX_eth','EX_byp','biomass'}, 'biomass'); + [rmse, perCond] = datasetRmse(fd, sims, true, w, 'biomass'); + % Growth and the own exchange match; the byproduct term is + % measured 0 against the simulated value (carbon weight 1). + expGlc = sqrt(mean([0 0 tExperiments.enzymeCap * tExperiments.byproductYield].^2)); + expEth = sqrt(mean([0 0 20 * tExperiments.byproductYield].^2)); + testCase.verifyEqual(perCond, [expGlc; expEth], 'AbsTol', 1e-6); + testCase.verifyEqual(rmse, mean([expGlc expEth]), 'AbsTol', 1e-6); + end + + function datasetRmseGrowthMismatch(testCase) + model = tExperiments.toyModel(); + fd = tExperiments.fluxDataToy(18); + sims = simulateConditions(model, fd, true, {'EX_byp'}, 'biomass'); + w = computeCarbonWeights(model, {'EX_glc','EX_eth','EX_byp','biomass'}, 'biomass'); + [~, perCond] = datasetRmse(fd, sims, true, w, 'biomass'); + growthDiff = (18 - 20) * tExperiments.biomassCarbon; + bypDiff = 0 - 20 * tExperiments.byproductYield; + testCase.verifyEqual(perCond(2), sqrt(mean([growthDiff 0 bypDiff].^2)), 'AbsTol', 1e-6); + end + + function datasetRmseInfeasibleConditionGetsThePenalty(testCase) + fd = tExperiments.fluxDataToy(20); + sims = repmat(struct('feasible', false, 'growth', NaN, 'exchRxnIDs', {{}}, ... + 'exchFluxes', zeros(0,1), 'blockRxnIDs', {{}}, 'blockFluxes', zeros(0,1)), 2, 1); + [rmse, perCond] = datasetRmse(fd, sims, true, containers.Map(), 'biomass', 'penalty', 99); + testCase.verifyEqual(perCond, [99; 99]); + testCase.verifyEqual(rmse, 99); + end + + function datasetRmseMaxGrowthScoresGrowthOnly(testCase) + fd = tExperiments.maxGrowthToy(); + sims = repmat(struct('feasible', true, 'growth', 0, 'exchRxnIDs', {{}}, ... + 'exchFluxes', zeros(0,1), 'blockRxnIDs', {{}}, 'blockFluxes', zeros(0,1)), 2, 1); + sims(1).growth = tExperiments.enzymeCap; + sims(2).growth = 30; + [rmse, perCond] = datasetRmse(fd, sims, false, containers.Map(), 'biomass'); + second = abs((tExperiments.enzymeCap - 30) * tExperiments.biomassCarbon); + testCase.verifyEqual(perCond, [0; second], 'AbsTol', 1e-9); + testCase.verifyEqual(rmse, mean([0 second]), 'AbsTol', 1e-9); + end + + function datasetRmseMultipliesByTheWeight(testCase) + fd = tExperiments.maxGrowthToy(); + fd.weight = [0; 2]; + sims = repmat(struct('feasible', true, 'growth', 0, 'exchRxnIDs', {{}}, ... + 'exchFluxes', zeros(0,1), 'blockRxnIDs', {{}}, 'blockFluxes', zeros(0,1)), 2, 1); + sims(1).growth = 30; + sims(2).growth = 30; + [rmse, perCond] = datasetRmse(fd, sims, false, containers.Map(), 'biomass'); + testCase.verifyEqual(rmse, mean(perCond .* [0; 2]), 'AbsTol', 1e-9); + end + + function datasetRmseWrongSimCount(testCase) + fd = tExperiments.maxGrowthToy(); + testCase.verifyError(@() datasetRmse(fd, struct('feasible', {}), false, ... + containers.Map(), 'biomass'), 'datasetRmse:wrongSimCount'); + end + + %% experimentDistance + + function experimentDistanceCombinesBothDatasets(testCase) + [ed, fluxSims, maxSims, w, expFlux, expMax] = tExperiments.distanceInputs(); + [rmse, detail] = experimentDistance(ed, fluxSims, maxSims, w, 'biomass'); + testCase.verifyEqual(rmse, mean([expFlux expMax]), 'AbsTol', 1e-9); + testCase.verifyTrue(isfield(detail, 'fluxData') && isfield(detail, 'maxGrowth')); + end + + function experimentDistanceMaxGrowthWeight(testCase) + [ed, fluxSims, maxSims, w, expFlux, expMax] = tExperiments.distanceInputs(); + rmse = experimentDistance(ed, fluxSims, maxSims, w, 'biomass', 'maxGrowthWeight', 2); + testCase.verifyEqual(rmse, (expFlux + 2*expMax) / 3, 'AbsTol', 1e-9); + rmse = experimentDistance(ed, fluxSims, maxSims, w, 'biomass', 'maxGrowthWeight', 0.5); + testCase.verifyEqual(rmse, (2*expFlux + expMax) / 3, 'AbsTol', 1e-9); + % A lone dataset carries the whole score whatever the weight. + solo = ed; solo.maxGrowth = []; + rmse = experimentDistance(solo, fluxSims, [], w, 'biomass', 'maxGrowthWeight', 2); + testCase.verifyEqual(rmse, expFlux, 'AbsTol', 1e-9); + end + + function experimentDistanceMissingDatasetsContributeNothing(testCase) + [ed, fluxSims, ~, w, expFlux] = tExperiments.distanceInputs(); + ed.maxGrowth = []; + [rmse, detail] = experimentDistance(ed, fluxSims, [], w, 'biomass'); + testCase.verifyEqual(rmse, expFlux, 'AbsTol', 1e-9); + testCase.verifyEqual(fieldnames(detail), {'fluxData'}); + none = struct('fluxData', [], 'maxGrowth', [], 'zeroFlux', {{}}); + testCase.verifyTrue(isnan(experimentDistance(none, [], [], w, 'biomass'))); + end + + function experimentDistanceNeedsTheSimulations(testCase) + [ed, ~, maxSims, w] = tExperiments.distanceInputs(); + testCase.verifyError(@() experimentDistance(ed, [], maxSims, w, 'biomass'), ... + 'experimentDistance:missingSims'); + end + + %% applyFluxDataConstraints + + function applyFluxDataConstraintsRejectsInvalidSettings(testCase) + m = tExperiments.simpleModel(); + fd = tExperiments.simpleFluxData({'c1'}, 0.4, -5, {'EX_glc'}); + testCase.verifyError(@() applyFluxDataConstraints(m, fd, 'biomass', 'maxMinGrowth', 'invalid'), ... + 'applyFluxDataConstraints:invalidMaxMinGrowth'); + testCase.verifyError(@() applyFluxDataConstraints(m, fd, 'biomass', 'looseStrictFlux', 200), ... + 'applyFluxDataConstraints:invalidLooseStrictFlux'); + testCase.verifyError(@() applyFluxDataConstraints(m, fd, 'biomass', 'condition', 'nonexistent'), ... + 'applyFluxDataConstraints:unknownCondition'); + testCase.verifyError(@() applyFluxDataConstraints(m, fd, 'biomass', 'condition', 5), ... + 'applyFluxDataConstraints:conditionOutOfRange'); + fd.exchRxnIDs = {'EX_nonexistent'}; + testCase.verifyError(@() applyFluxDataConstraints(m, fd, 'biomass'), ... + 'applyFluxDataConstraints:missingReaction'); + end + + function applyFluxDataConstraintsGrowthBounds(testCase) + m = tExperiments.simpleModel(); + fd = tExperiments.simpleFluxData({'c1'}, 0.4, -5, {'EX_glc'}); + bio = strcmp(m.rxns, 'biomass'); + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'maxMinGrowth', 'max'); + testCase.verifyEqual([m2.lb(bio) m2.ub(bio)], [0 0.4]); + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'maxMinGrowth', 'min'); + testCase.verifyEqual([m2.lb(bio) m2.ub(bio)], [0.4 1000]); + end + + function applyFluxDataConstraintsZeroesTheCarbonSource(testCase) + m = tExperiments.simpleModel(); + fd = tExperiments.simpleFluxData({'c1'}, 0.4, -5, {'EX_glc'}); + eth = strcmp(m.rxns, 'EX_eth'); + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'cSource', 'EX_eth'); + testCase.verifyEqual([m2.lb(eth) m2.ub(eth)], [0 0]); + % A carbon source that is not in the model is ignored, and + % without cSource nothing is blocked. + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'cSource', 'EX_absent'); + testCase.verifyEqual(m2.lb(strcmp(m.rxns, 'EX_glc')), -5); + m2 = applyFluxDataConstraints(m, fd, 'biomass'); + testCase.verifyEqual([m2.lb(eth) m2.ub(eth)], [-1000 1000]); + end + + function applyFluxDataConstraintsLooseMode(testCase) + m = tExperiments.simpleModel(); + fd = tExperiments.simpleFluxData({'c1'}, 0.4, [-5 3 NaN], {'EX_glc','EX_eth','EX_O2'}); + m2 = applyFluxDataConstraints(m, fd, 'biomass'); + testCase.verifyEqual(tExperiments.bounds(m2, 'EX_glc'), [-5 0]); + testCase.verifyEqual(tExperiments.bounds(m2, 'EX_eth'), [0 3]); + % NaN is no measurement: bounds are untouched. + testCase.verifyEqual(tExperiments.bounds(m2, 'EX_O2'), tExperiments.bounds(m, 'EX_O2')); + end + + function applyFluxDataConstraintsPercentageMode(testCase) + m = tExperiments.simpleModel(); + fd = tExperiments.simpleFluxData({'c1'}, 0.4, [3 -5], {'EX_eth','EX_glc'}); + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'looseStrictFlux', 10); + testCase.verifyEqual(tExperiments.bounds(m2, 'EX_eth'), [2.85 3.15], 'AbsTol', 1e-12); + % A negative value: the bounds are swapped to keep lb <= ub. + testCase.verifyEqual(tExperiments.bounds(m2, 'EX_glc'), [-5.25 -4.75], 'AbsTol', 1e-12); + end + + function applyFluxDataConstraintsExtremeValuesAreUnconstrained(testCase) + m = tExperiments.simpleModel(); + fd = tExperiments.simpleFluxData({'c1'}, 0.4, [-1000 1000], {'EX_O2','EX_eth'}); + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'looseStrictFlux', 10); + testCase.verifyEqual(tExperiments.bounds(m2, 'EX_O2'), [-1000 0]); + testCase.verifyEqual(tExperiments.bounds(m2, 'EX_eth'), [0 1000]); + end + + function applyFluxDataConstraintsSelectsTheCondition(testCase) + m = tExperiments.simpleModel(); + fd = tExperiments.simpleFluxData({'c1';'c2'}, [0.4;0.7], [-5;-8], {'EX_glc'}); + bio = strcmp(m.rxns, 'biomass'); + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'condition', 2); + testCase.verifyEqual(m2.ub(bio), 0.7); + testCase.verifyEqual(m2.lb(strcmp(m.rxns, 'EX_glc')), -8); + m2 = applyFluxDataConstraints(m, fd, 'biomass', 'condition', 'c2'); + testCase.verifyEqual(m2.lb(strcmp(m.rxns, 'EX_glc')), -8); + end + + end + + methods (Static, Access = private) + + function file = writeFile(testCase, content) + file = [tempname '.tsv']; + fid = fopen(file, 'w'); + fwrite(fid, content); + fclose(fid); + testCase.addTeardown(@() delete(file)); + end + + function file = writeRows(testCase, rows) + lines = cellfun(@(r) strjoin(r, sprintf('\t')), rows, 'UniformOutput', false); + file = tExperiments.writeFile(testCase, [strjoin(lines, newline) newline]); + end + + function writeNamed(dirName, fileName, content) + fid = fopen(fullfile(dirName, fileName), 'w'); + fwrite(fid, content); + fclose(fid); + end + + function b = bounds(model, rxn) + i = strcmp(model.rxns, rxn); + b = [model.lb(i) model.ub(i)]; + end + + function sol = countedSolve(m, calls) + calls('n') = calls('n') + 1; + sol = solveLP(m); + end + + function m = capReaction(m, rxn, value) + m.ub(strcmp(m.rxns, rxn)) = value; + end + + function fd = fluxDataToy(ethanolGrRate) + % Glucose is enzyme-limited with matching measured uptake, + % ethanol carbon-limited with an uptake of 20. + fd.conds = {'glucose';'ethanol'}; + fd.Ptot = [NaN; NaN]; + fd.grRate = [tExperiments.enzymeCap; ethanolGrRate]; + fd.exchFluxes = [-tExperiments.enzymeCap NaN; NaN -20]; + fd.exchMets = {'glucose','ethanol'}; + fd.exchRxnIDs = {'EX_glc','EX_eth'}; + fd.weight = [1; 1]; + fd.source = {'test';'test'}; + end + + function fd = maxGrowthToy() + fd.conds = {'glucose';'ethanol'}; + fd.Ptot = [NaN; NaN]; + fd.grRate = [tExperiments.enzymeCap; tExperiments.enzymeCap]; + fd.exchFluxes = [-1000 NaN; NaN -1000]; + fd.exchMets = {'glucose','ethanol'}; + fd.exchRxnIDs = {'EX_glc','EX_eth'}; + end + + function [ed, fluxSims, maxSims, w, expFlux, expMax] = distanceInputs() + model = tExperiments.toyModel(); + fd = tExperiments.fluxDataToy(20); + mg = tExperiments.maxGrowthToy(); + ed = struct('fluxData', fd, 'maxGrowth', mg, 'zeroFlux', {{'EX_byp'}}); + fluxSims = simulateConditions(model, fd, true, {'EX_byp'}, 'biomass'); + maxSims = simulateConditions(model, mg, false, {}, 'biomass'); + w = computeCarbonWeights(model, {'EX_glc','EX_eth','EX_byp','biomass'}, 'biomass'); + expFlux = datasetRmse(fd, fluxSims, true, w, 'biomass'); + expMax = datasetRmse(mg, maxSims, false, w, 'biomass'); + end + + function m = toyModel() + rxns = {'EX_glc';'EX_eth';'EX_byp';'EX_o2';'TR_glc';'TR_eth';'R'; ... + 'biomass';'prot_pool_exchange';'usage_prot_E'}; + mets = {'glc_e';'eth_e';'byp_c';'o2_e';'A_c';'pool';'prot_E';'bio_met'}; + coeff = 100 / 3600; % MW / (kcat * 3600), mg per mmol + S = zeros(numel(mets), numel(rxns)); + S(1,1) = -1; % EX_glc + S(2,2) = -1; % EX_eth + S(3,3) = -1; % EX_byp + S(4,4) = -1; % EX_o2 + S(1,5) = -1; S(5,5) = 1; % TR_glc + S(2,6) = -1; S(5,6) = 1; % TR_eth + S(5,7) = -1; S(7,7) = -coeff; S(8,7) = 1; S(3,7) = tExperiments.byproductYield; % R + S(8,8) = -1; % biomass + S(6,9) = 1; % protein pool + S(6,10) = -1; S(7,10) = 1; % usage of prot_E + m = struct(); + m.id = 'toy'; + m.rxns = rxns; + m.rxnNames = rxns; + m.mets = mets; + m.metNames = mets; + m.metComps = ones(numel(mets), 1); + m.comps = {'c'}; + m.compNames = {'cytosol'}; + m.S = sparse(S); + m.lb = [-1000;-1000;0;-1000;0;0;0;0;0;0]; + m.ub = [0;0;1000;0;1000;1000;1000;1000;1;1000]; + m.rev = double(m.lb < 0); + m.c = zeros(numel(rxns), 1); m.c(8) = 1; + m.b = zeros(numel(mets), 1); + m.genes = {}; m.grRules = repmat({''}, numel(rxns), 1); + m.rxnGeneMat = sparse(numel(rxns), 0); + m.metFormulas = {'C6H12O6';'C2H6O';'C1';'O2';'';'';'';''}; + end + + function m = simpleModel() + m = struct(); + m.id = 'simple'; + m.rxns = {'EX_glc';'EX_O2';'EX_eth';'biomass'}; + m.rxnNames = m.rxns; + m.mets = {'glc_e';'O2_e';'eth_e';'bio'}; + m.metNames = m.mets; + m.metComps = ones(4, 1); + m.comps = {'c'}; + m.compNames = {'cytosol'}; + m.S = sparse([-1 0 0 0; 0 -1 0 0; 0 0 -1 0; 0 0 0 1]); + m.lb = [-1000;-1000;-1000;0]; + m.ub = [1000;1000;1000;1000]; + m.rev = [1;1;1;0]; + m.c = [0;0;0;1]; + m.b = zeros(4, 1); + m.genes = {}; m.grRules = repmat({''}, 4, 1); + m.rxnGeneMat = sparse(4, 0); + end + + function fd = simpleFluxData(conds, grRate, fluxes, rxnIds) + fd.conds = conds(:); + fd.Ptot = NaN(numel(conds), 1); + fd.grRate = grRate(:); + fd.exchFluxes = reshape(fluxes, numel(conds), []); + fd.exchMets = rxnIds; + fd.exchRxnIDs = rxnIds; + end + + end +end