MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery

[📜 arXiv] · [📖 ACL Anthology] · [🖋️ BibTeX]· [🤗️ Checkpoints]

This repository contains the data processing and training code for the MauBERT paper.

Overview

The HuBERT backbone is exposed as HuBERT. Two task-specific models build on top of it:

• MauBERT-feat: predicts both articulatory features and IPA phone tokens per frame.
• MauBERT-phone: predicts IPA phone tokens per frame only.

Each model owns a HuBERT instance as an attribute, so the backbone can be frozen or fine-tuned.

Layout

src/
├── maubert/                # Library code
│   ├── models/             # HuBERT, HuBERTPretrain, MauBERT-feat, MauBERT-phone
│   ├── data/               # Tokenizers, datasets, IO, samplers
│   ├── config.py           # Configuration classes
│   ├── criterion.py        # feat model loss
│   ├── features.py         # Feature extraction
│   ├── finetune.py         # Monolingual (self-) supervised fine-tuning
│   ├── metric.py           # facc / pacc / per
│   ├── optimizer.py        # Optimizer, scheduler
│   ├── quantise.py         # Quantisation methods for FT
│   ├── train.py            # Multilingual supervised pre-training
│   ├── transcribe.py       # Pseudo-label transcription
│   ├── utils.py            # Miscellaneous tools
│   └── validate.py         # Validation during multilingual supervised pre-training
└── configs/                # YAML configs (data + model + training)

Installation

We recommend using uv:

# Uncomment the last part for training (optional)
uv sync --python 3.12 # --extra train

Alternatively, use conda (also works for mamba/micromamba):

conda create -n maubert python=3.12 -c conda-forge
conda activate maubert
# Replace "." with ".[train]" for training (optional)
uv pip install -e .

Or standard pip:

python3 -m venv .venv
source .venv/bin/activate
# Replace "." with ".[train]" for training (optional)
pip install -e .

Usage

Inference

You can extract features with the first command below. The two other commands are needed to compute pseudo-labels for the monolingual fine-tuning.

# 1. Extract frame-level features from a manifest
python -m maubert.features MANIFEST_FILE FEATURE_DIR \
    --type TYPE --checkpoint CHECKPOINT --layer LAYER_INDEX_OR_NAME

# 2. Fit a quantisation model on those features
python -m maubert.quantise FEATURE_DIR QUANTISER_FILE N_CLUSTERS QUANTISER_METHOD

# 3. Transcribe the features into discrete units
python -m maubert.transcribe FEATURE_DIR QUANTISER_FILE OUTPUT_FILE QUANTISER_METHOD

Training

NOTE: Requires the train extra.

For the multilingual pre-training, use the command below (for the feat variant). There are some unfilled fields that can be completed either directly in the YAML file or with the --override option. Note that the override option takes precedence.

# Train MauBERT-feat (use `configs/train_phone.yaml` for MauBERT-phone)
python -m maubert.train --config configs/train_feat.yaml

Analogously, use the command below for the monolingual fine-tuning.

# Fine-tune MauBERT-feat
python -m maubert.finetune --config configs/finetune.yaml

Citation

@inproceedings{ortiztandazo-etal-2026-maubert,
    title = "{M}au{BERT}: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery",
    author = "Ortiz Tandazo, Angelo  and
      Khentout, Manel  and
      Benchekroun, Youssef  and
      Hueber, Thomas  and
      Dupoux, Emmanuel",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.24/",
    doi = "10.18653/v1/2026.acl-long.24",
    pages = "568--585",
    ISBN = "979-8-89176-390-6",
}

Acknowledgments

minimal_hubert: https://github.com/mxmpl/minimal_hubert
S3PRL: https://github.com/s3prl/s3prl