GLiFormer: Multi-Task Information Extraction

Named Entity Recognition | Text Classification | Relation Extraction | Multi-Level Structuring | Embeddings

Text · Documents · Images · Audio

Quick Start • Structuring • Usage • Training • Architectures • Evaluation

GLiFormer is a framework for training and running models that turn unstructured inputs into labeled spans, relations, classifications, and structured records. Built on GLiNER, it combines a shared encoder with configurable task heads and lets you specify entity types, class labels, relation types, and extraction schemas at inference time.

Alongside text extraction, the codebase includes model variants for document layout, vision, audio, and combined modalities. Available tasks depend on the heads and modalities configured and trained in your checkpoint.

Why GLiFormer?

Multiple Tasks, One Encoder Run entity recognition, classification, relations, and structuring together through a shared backbone.	Labels at Inference Time Describe the entities, classes, and relations you need with readable labels.	Multi-Level Structuring Extract nested records and connect children to their parents: companies → departments → employees, all in one schema.
Beyond Plain Text Use dedicated text, layout, vision, audio, and omni model variants.	Flexible Task Heads Configure individual tasks, shared components, and anchor representations for your workload.	Fine-Tune on Your Data Train with task-specific annotations, freeze selected components, or update only task heads.

Quick Start

Installation

Requires Python 3.10 or newer. Install from the repository:

git clone https://github.com/Bionity/GLiFormer.git
cd GLiFormer
pip install -e .

With uv:

uv pip install -e .

Optional dependencies can be installed for the features you use:

Extra	Purpose
demo	Gradio demos and comparison models
data	Dataset loading with Hugging Face Datasets
pdf	PDF processing with PyMuPDF and pdfplumber
vision	Torchvision utilities
audio	Torchaudio utilities
flash	FlashDeBERTa support
dev	pytest and Ruff
all	All extras listed above

For example:

pip install -e ".[demo,pdf]"

Basic Usage

Load a trained GLiFormer checkpoint containing an NER head. Replace path/to/checkpoint throughout these examples with your checkpoint directory. from_pretrained also accepts a Hugging Face repository ID for a compatible GLiFormer checkpoint.

from gliformer import GLiFormer

model = GLiFormer.from_pretrained("path/to/checkpoint", load_tokenizer=True)

text = "Marie Curie worked at the University of Paris in France."
labels = ["person", "organization", "location"]

entities = model.predict_entities(text, labels, threshold=0.5)

for entity in entities:
    print(entity["text"], "=>", entity["label"])

Entity results contain text, label, start, end, and score. Character offsets use an exclusive end, so text[start:end] gives the extracted mention.

Pass a list of texts for batched inference:

entities = model.predict_entities(
    ["Alice works at Acme.", "Bob lives in Berlin."],
    ["person", "organization", "location"],
    batch_size=8,
)

Per-task methods return one result for a string input and a list of results for a batch. The examples below reuse model; each requires a checkpoint trained for the corresponding task.

Usage

Text Classification

Provide candidate labels to classify text:

predictions = model.classify(
    "The new search feature is fast and easy to use.",
    ["positive", "negative", "neutral"],
    threshold=0.5,
)
print(predictions)

Use a dictionary of label lists to request named classification groups, such as {"sentiment": ["positive", "negative"], "topic": ["product", "support"]}.

Relation Extraction

The open relation head extracts relation endpoints directly from text using the supplied relation labels:

relations = model.predict_relations(
    "Alice works at Acme and lives in London.",
    ["works_at", "lives_in"],
    threshold=0.5,
)

for relation in relations:
    print(
        relation["head"]["text"],
        "=>", relation["relation"], "=>",
        relation["tail"]["text"],
    )

For a checkpoint with a joint relation head, supply both entity and relation types through inference:

results = model.inference(
    "Alice works at Acme.",
    joint_relations={
        "employment": {
            "entities": ["person", "organization"],
            "relations": ["works_at"],
        }
    },
)

Structured Extraction

Define nested Pydantic models to extract company → departments → employees in one call. This example requires a checkpoint trained with multi-level structuring enabled (structuring_config.multi_level: true).

import json
from pydantic import BaseModel


class Employee(BaseModel):
    name: str
    role: str


class Department(BaseModel):
    name: str
    employees: list[Employee]


class Company(BaseModel):
    name: str
    departments: list[Department]


text = (
    "At Acme, Engineering includes Alice, a software engineer, and Bob, "
    "a designer. Sales includes Carol, an account manager."
)
records = model.structure(text, {"company": Company}, validate_output=True)
print(json.dumps(records, indent=2))

Illustrative output:

{
  "company": [{
    "name": "Acme",
    "departments": [
      {
        "name": "Engineering",
        "employees": [
          {"name": "Alice", "role": "software engineer"},
          {"name": "Bob", "role": "designer"}
        ]
      },
      {
        "name": "Sales",
        "employees": [{"name": "Carol", "role": "account manager"}]
      }
    ]
  }]
}

Each employee stays under its department. Results are dictionaries and lists validated against the Pydantic schema; actual predictions depend on the checkpoint and threshold.

Multi-Task Inference

Construct a reusable schema independently of the model:

from gliformer import GLiFormerSchema

schema = GLiFormerSchema(
    entities=["person", "organization"],
    classes=["business", "sports", "technology"],
    structures={"employee": ["name", "company"]},
)

results = model.inference_from_schema(
    ["Alice joined Acme as a software engineer."],
    schema,
    threshold=0.5,
)

print(results["ner"][0])
print(results["classification"][0])
print(results["structuring"][0])

inference and inference_from_schema return a dictionary keyed by task, with one entry per input text under each key. You can also pass entities, classes, relations, and structures directly to model.inference(...).

Document Layout Processing

Use a trained layout checkpoint with an NER head to extract fields from a PDF using both text and word bounding boxes. Install the PDF dependencies first:

pip install "gliformer[pdf]"

This example uses a text-and-box layout checkpoint, such as one trained with the 2D DeBERTa configuration:

from gliformer import GLiFormer

layout_model = GLiFormer.from_pretrained(
    "path/to/layout-checkpoint", load_tokenizer=True
)

results = layout_model.parse_pdf(
    "invoice.pdf",
    entities=["invoice number", "invoice date", "vendor", "total amount"],
    add_image_token=False,
    return_pixel_values=False,
    split_pages=True,
    return_pages=True,
    threshold=0.5,
)

for page_index, entities in zip(results["pages"], results["ner"]):
    print(f"Page {page_index + 1}")
    for entity in entities:
        print(entity["text"], "=>", entity["label"])

parse_pdf extracts embedded PDF text and normalizes word boxes to the 0–1000 coordinate range. Results are grouped by page; pages=[0, 1] selects the first two pages. For a checkpoint that also consumes page images, set add_image_token=True and return_pixel_values=True.

For scanned documents, supply externally extracted OCR tokens as words and matching [x0, y0, x1, y1] boxes as bbox, normalized to the same 0–1000 range. You can also pass structures, classes, or relations when the checkpoint has the corresponding trained heads.

Text Embeddings

A checkpoint with an embedding head can produce vectors for similarity and retrieval:

import torch.nn.functional as F

embeddings = model.embed_text([
    "A scientist is working in a laboratory.",
    "A researcher is conducting an experiment.",
])
similarity = F.cosine_similarity(embeddings[0:1], embeddings[1:2])
print(similarity.item())

embed_text returns a CPU tensor of shape (number_of_texts, embedding_dimension). Bi-encoder configurations also expose model.embed_labels(labels).

Training

Fine-tune a trained checkpoint with annotated examples using train_model. For NER, records use an extraction list containing entity mentions and their labels:

from gliformer import GLiFormer

model = GLiFormer.from_pretrained("path/to/checkpoint", load_tokenizer=True)

train_data = [
    {
        "text": "Alice works at Acme.",
        "extraction": [
            {
                "name": "entities",
                "all_labels": ["person", "organization", "location"],
                "ner": [["Alice", "person"], ["Acme", "organization"]],
            }
        ],
    },
    {
        "text": "Bob lives in Berlin.",
        "extraction": [
            {
                "name": "entities",
                "all_labels": ["person", "organization", "location"],
                "ner": [["Bob", "person"], ["Berlin", "location"]],
            }
        ],
    },
]

trainer = model.train_model(
    train_dataset=train_data,
    output_dir="outputs/ner",
    max_steps=100,
    per_device_train_batch_size=2,
    learning_rate=1e-5,
)
model.save_pretrained("outputs/ner/final")

This small dataset illustrates the format; use a representative training set and pass held-out examples as eval_dataset for your own task. Training records can also carry classification, open_relex, structuring, and embedding annotations for the enabled heads. See the task processors and sample records in the test fixtures for their formats.

Useful training options include freeze_components=["text_encoder"], train_head_only=True, and resume_from_checkpoint="path/to/training-checkpoint".

To initialize a new model with a pretrained backbone and new task heads, use GLiFormer.load_from_config(...). It accepts a configuration object, a model configuration dictionary, or a JSON configuration path. The YAML configurations contain separate model, data, and training sections; when using them programmatically, parse the YAML and pass its model section to load_from_config. Newly initialized heads require training before extraction.

Architectures

GLiFormer selects the appropriate wrapper from the checkpoint configuration:

Variant	Wrapper	Inputs	Example configuration
Text	GLiFormerText	Text and task labels	NER
Layout	GLiFormerLayout	Text with document geometry	Layout
Vision	GLiFormerVision	Images with label prompts	Vision
Audio	GLiFormerAudio	Audio with label prompts	Audio
Omni	GLiFormerOmni	Combined text, image, and audio features	Omni

Text models support joint text/label encoding and separate label encoders. Task heads share encoder representations, while each task has its own processor, neural head, and decoder. Configurable anchors represent entities, relation pairs, or record instances for the corresponding prediction tasks.

The framework also includes instance counting, image and audio classification, object detection, image segmentation, and audio segmentation heads. Use configurations and checkpoints trained for the required modality and task; the text examples above do not activate additional heads in an existing checkpoint.

For implementation details, see the configuration classes, model variants, task modules, and shared layers.

Demo

Launch the Gradio interface with your checkpoint:

pip install -e ".[demo,pdf]"
GLIFORMER_MODEL_ID=path/to/checkpoint python demo.py

The demo includes annotated examples for NER, classification, relations, structured extraction, and embeddings. Select a checkpoint with the heads needed by the tabs you want to use.

Evaluation

Run task-specific evaluators from the repository root. For example, evaluate NER on a prepared CrossNER dataset:

python gliformer_eval/eval_ner.py \
  --model path/to/checkpoint \
  --data path/to/NER \
  --datasets CrossNER_AI CrossNER_literature CrossNER_music CrossNER_politics CrossNER_science \
  --output eval_results/ner.json

The NER data directory must contain one subdirectory per dataset with labels.json and test.json. The evaluator reports strict entity-level precision, recall, and F1 using character spans and entity types.

See the evaluation guide for classification, relation extraction, structuring, and similarity. The benchmark guide documents classification and structuring latency, throughput, memory, and profiling measurements.

Contributing

Bug reports, task examples, and contributions are welcome. For development, install the test and lint tools and run the checks relevant to your changes:

pip install -e ".[dev]"
python -m pytest tests/processing/test_schema.py tests/processing/test_formatting.py

New task implementations follow the processor → head → decoder organization in gliformer/tasks.

Acknowledgements

GLiFormer builds on GLiNER. Its task implementations also draw on ideas from GLiNER2 and GLiClass.