Skip to content

AxoMEME silently mistranslates non-universal genetic codes (TGA read as stop, not Trp) #152

Description

@stevenweaver

Summary

AxoMEME's tokenizer hard-codes the universal genetic code. On an alignment that uses any other
code, it will silently translate codons to the wrong amino acids and feed those to the model. There
is no error and no warning — the run completes and returns plausible-looking per-site rankings built
on wrong inputs.

This cannot be triggered through the UI today, but only by accident (see Why it is latent below),
and the thing preventing it is not the thing that should be preventing it.

Evidence

Found while picking a demo dataset. Two real MEME submissions in our corpus are rejected by DM3's
codon validation for in-frame stop codons:

job                        seqs  sites   internal stops   sequences affected
69eaba367d06ccca028b0462     74    226              178                   74
696c06a3ec632baa7052658c    113   3787            10449                  113

Those are not broken alignments. Every stop is TGA, and they cluster in a handful of columns:

69eaba367d06ccca028b0462 -> 178 internal stops in 3 distinct columns
   by codon: TGA=178
   columns: 48, 68, 109

696c06a3ec632baa7052658c -> 10449 internal stops in 115 distinct columns
   by codon: TGA=10449

A column where all 74 sequences carry TGA is not a stop codon. TGA is tryptophan in the
vertebrate mitochondrial code (and several others). These are mitochondrial alignments, and under the
universal code their most conserved tryptophan columns read as stops.

Impact

The model consumes two token streams, codons and amino acids. Under the wrong code the amino-acid
stream is wrong at exactly the positions that are most conserved — and forward() uses the central
site's amino acid to build its synonymous/non-synonymous attention masks, so the error propagates
into the model's view of the substitution pattern, not just into a display column.

Silent wrong answers, on data a researcher has every reason to think is supported.

Why it is latent right now

Two accidents, neither of them a designed guard:

  1. DM3 rejects these alignments during codon validation before AxoMEME sees them.
  2. AxoMEME's option panel does not expose a genetic-code selector, because the tokenizer cannot
    honour one (deliberate — a control that cannot reach the model is worse than no control).

So today a user cannot select the mitochondrial code and reach AxoMEME. The moment either changes —
codon validation becomes code-aware, or someone re-adds the selector because it looks like an
oversight — this becomes a live silent-wrong-answer bug. Compare #143, where a genetic code was
accepted by the UI and then not passed through to HyPhy.

Proposed fix

AxoMEME should detect and refuse, not mistranslate. The signature is reliable and cheap:

if a codon column translates to a universal-code stop in a large fraction of sequences — and
especially if it is TGA — the alignment is almost certainly not using the universal code

Suggested behaviour: refuse the run with a message naming the affected columns and saying AxoMEME
supports the universal genetic code only. That is honest about the limitation and actionable, and it
is a real guard rather than a side effect of validation ordering.

The alternative — retraining the model with a code-aware tokenizer — is an ML-side change and out of
scope here. Refusing is the correct behaviour until then.

Notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: mediumMedium priority - standard issue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions