Long-Toolkit
A hub of Python toolkits for longitudinal data, from preparation to explainability

Documentation · CLI · LEAP Group

About The Project

Long-Toolkit (short for Longitudinal-Toolkit) is a hub of Python toolkits for longitudinal data, i.e., data in which the same participants are measured at several waves (time points). It is designed for social, medical and clinical researchers.

First, the hub provides tools to get longitudinal data ready, from two viewpoints. Data preparation produces a usable table, by generating synthetic cohorts or by converting cohort files between formats (CSV, Parquet and Stata). Data preprocessing then makes that table fit for modelling, by cleaning, imputing, harmonising and reshaping the repeated measures.

Second, with the data ready, the hub answers one question: is machine learning, and/or longitudinal machine learning (models that exploit the repeated measures of each variable), worth it compared with the statistical analysis already in place, usually a logistic regression [1][2][3]? The Benchmarking and Modelling Toolkit answers it, by ranking both families of models side by side, all against all, under one shared protocol; it can then train and evaluate one chosen model in depth.

Third, the hub explains the results. The ranking shows whether, and by how much, machine learning improves on the statistical analysis on that dataset; the Explainability Toolkit then shows which features drive the predictions of a saved model, e.g., the best estimator of the benchmark.

Toolkit	Stage	Question it answers
Data Preparation	A	How does a raw or synthetic cohort become a usable table?
Data Preprocessing	B	How are repeated measures cleaned, imputed and reshaped?
Benchmarking and Modelling (main toolkit)	C	Is machine learning, and/or longitudinal machine learning, worth it on this dataset?
Explainability	D	Which features drive the predictions of a chosen model?

The hub ships as two components that run the same tools: the long-toolkit Python package, i.e., this repository, used through its application programming interface (API) in scripts and notebooks, and the long-toolkit command-line interface (CLI), a no-code terminal interface written in Go.

Setup And Launch

Important

Install both components: long-toolkit (Python package) and long-toolkit (CLI). The CLI is the recommended no-code entry point.

Machine Requirements (Python, uv)

Install the following first:

• Python 3.10 only, or uv, the state-of-the-art Python package manager, to manage it
• On macOS, libomp (the OpenMP runtime loaded by XGBoost and LightGBM)
• Go, only to build the CLI from source

# macOS (Homebrew)
brew update
brew install uv libomp

# Linux (Ubuntu/Debian)
curl -LsSf https://astral.sh/uv/install.sh | sh

Install long-toolkit (PyPI Index)

Long-Toolkit supports Python 3.10 only.

Before installing, add the following to your project's pyproject.toml. Scikit-Longitudinal supplies sklearn through its lexicographical fork, so the stock package must be excluded, as described in its installation guide.

[tool.uv]
override-dependencies = ["scikit-learn ; sys_platform == 'never'"]

uv add long-toolkit
# We recommend uv as the Python package manager

Install the long-toolkit CLI (No-Code Terminal Interface)

Homebrew (macOS and Linux):

brew tap Long-Toolkit/homebrew-tap
brew install long-toolkit

Then, from inside the Python project in which long-toolkit is installed, run:

long-toolkit

Warning

The full long-toolkit workflow is not supported natively on Windows yet. As a workaround, run the toolkit inside a Docker Linux environment, or use a macOS or Linux machine. Please open an issue to contribute towards Windows support.

Getting Started With the Toolkits

The documentation walks through every toolkit, with code that runs as is:

• Get Started: what the hub does, and how to install it.
• First Benchmark: standard against longitudinal machine learning on a synthetic cohort, in about thirty lines of Python.
• Toolkits: Data Preparation, Data Preprocessing, Benchmarking and Modelling, and Explainability, in depth.
• Tutorials: the CLI and the Python API, step by step.

Citation

This initiative is developed through research at the University of Edinburgh, under the supervision of Dr Alex Kwong.

Team:

• Simon Provost: Postdoctoral Research @ McGill University (MEDomicsLab).
• Bianca Branco: PhD student, University of Edinburgh.
• Honghui Chen: PhD student, University of Edinburgh.
• Isaac Tassella: PhD student, University of Edinburgh.
• Dr Alex Kwong: University of Edinburgh; Supervisor.

@software{Provost_Long_Toolkit,
    author = {Provost, Simon and Branco, Bianca and Chen, Honghui and Tassella, Isaac and Kwong, Alex},
    title = {{Long-Toolkit: A Hub of Python Toolkits for Longitudinal Data}},
    version = {0.0.4}
}

Use GitHub's "Cite this repository" for citation metadata updates.

License, Data, Security

• Review SECURITY.md for handling expectations.
• Licensed under the MIT License.

Special thanks to @charm.land for their amazing TUI framework!