Summary
AxoMEME's tokenizer hard-codes the universal genetic code. On an alignment that uses any other
code, it will silently translate codons to the wrong amino acids and feed those to the model. There
is no error and no warning — the run completes and returns plausible-looking per-site rankings built
on wrong inputs.
This cannot be triggered through the UI today, but only by accident (see Why it is latent below),
and the thing preventing it is not the thing that should be preventing it.
Evidence
Found while picking a demo dataset. Two real MEME submissions in our corpus are rejected by DM3's
codon validation for in-frame stop codons:
job seqs sites internal stops sequences affected
69eaba367d06ccca028b0462 74 226 178 74
696c06a3ec632baa7052658c 113 3787 10449 113
Those are not broken alignments. Every stop is TGA, and they cluster in a handful of columns:
69eaba367d06ccca028b0462 -> 178 internal stops in 3 distinct columns
by codon: TGA=178
columns: 48, 68, 109
696c06a3ec632baa7052658c -> 10449 internal stops in 115 distinct columns
by codon: TGA=10449
A column where all 74 sequences carry TGA is not a stop codon. TGA is tryptophan in the
vertebrate mitochondrial code (and several others). These are mitochondrial alignments, and under the
universal code their most conserved tryptophan columns read as stops.
Impact
The model consumes two token streams, codons and amino acids. Under the wrong code the amino-acid
stream is wrong at exactly the positions that are most conserved — and forward() uses the central
site's amino acid to build its synonymous/non-synonymous attention masks, so the error propagates
into the model's view of the substitution pattern, not just into a display column.
Silent wrong answers, on data a researcher has every reason to think is supported.
Why it is latent right now
Two accidents, neither of them a designed guard:
- DM3 rejects these alignments during codon validation before AxoMEME sees them.
- AxoMEME's option panel does not expose a genetic-code selector, because the tokenizer cannot
honour one (deliberate — a control that cannot reach the model is worse than no control).
So today a user cannot select the mitochondrial code and reach AxoMEME. The moment either changes —
codon validation becomes code-aware, or someone re-adds the selector because it looks like an
oversight — this becomes a live silent-wrong-answer bug. Compare #143, where a genetic code was
accepted by the UI and then not passed through to HyPhy.
Proposed fix
AxoMEME should detect and refuse, not mistranslate. The signature is reliable and cheap:
if a codon column translates to a universal-code stop in a large fraction of sequences — and
especially if it is TGA — the alignment is almost certainly not using the universal code
Suggested behaviour: refuse the run with a message naming the affected columns and saying AxoMEME
supports the universal genetic code only. That is honest about the limitation and actionable, and it
is a real guard rather than a side effect of validation ordering.
The alternative — retraining the model with a code-aware tokenizer — is an ML-side change and out of
scope here. Refusing is the correct behaviour until then.
Notes
Summary
AxoMEME's tokenizer hard-codes the universal genetic code. On an alignment that uses any other
code, it will silently translate codons to the wrong amino acids and feed those to the model. There
is no error and no warning — the run completes and returns plausible-looking per-site rankings built
on wrong inputs.
This cannot be triggered through the UI today, but only by accident (see Why it is latent below),
and the thing preventing it is not the thing that should be preventing it.
Evidence
Found while picking a demo dataset. Two real MEME submissions in our corpus are rejected by DM3's
codon validation for in-frame stop codons:
Those are not broken alignments. Every stop is TGA, and they cluster in a handful of columns:
A column where all 74 sequences carry TGA is not a stop codon.
TGAis tryptophan in thevertebrate mitochondrial code (and several others). These are mitochondrial alignments, and under the
universal code their most conserved tryptophan columns read as stops.
Impact
The model consumes two token streams, codons and amino acids. Under the wrong code the amino-acid
stream is wrong at exactly the positions that are most conserved — and
forward()uses the centralsite's amino acid to build its synonymous/non-synonymous attention masks, so the error propagates
into the model's view of the substitution pattern, not just into a display column.
Silent wrong answers, on data a researcher has every reason to think is supported.
Why it is latent right now
Two accidents, neither of them a designed guard:
honour one (deliberate — a control that cannot reach the model is worse than no control).
So today a user cannot select the mitochondrial code and reach AxoMEME. The moment either changes —
codon validation becomes code-aware, or someone re-adds the selector because it looks like an
oversight — this becomes a live silent-wrong-answer bug. Compare #143, where a genetic code was
accepted by the UI and then not passed through to HyPhy.
Proposed fix
AxoMEME should detect and refuse, not mistranslate. The signature is reliable and cheap:
Suggested behaviour: refuse the run with a message naming the affected columns and saying AxoMEME
supports the universal genetic code only. That is honest about the limitation and actionable, and it
is a real guard rather than a side effect of validation ordering.
The alternative — retraining the model with a code-aware tokenizer — is an ML-side change and out of
scope here. Refusing is the correct behaviour until then.
Notes
about DM3 detecting it rather than relying on validation ordering.