data-contamination

The Paper List on Data Contamination for Large Language Models Evaluation.

This python code performs an efficient speech reverberation starting from a dataset of close-talking speech signals and a collection of acoustic impulse responses.

Python .pyc decompiler (3.0–3.14) with a contamination-aware benchmark harness. Rule-only pass + one Codex call per module; evaluated on fuzz-synthetic (LLM-naïve) and *-obf (anonymised) corpora to put a number on the memorisation share. Three independent PyPI packages: pychd, pychd-pyfuzz, pychd-pyobf.

The official implementation of the paper "Data Contamination Calibration for Black-box LLMs" (ACL 2024)

The official repository for the paper entitled "Time Travel in LLMs: Tracing Data Contamination in Large Language Models."

Python package developed to evaluate textual overlap (N-Grams) between two volumes of text.

DICE: Detecting In-distribution Data Contamination with LLM's Internal State

The official repository for the paper entitled "Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models."

Point-in-time correct LLM instrumentation — tracing, version pinning and look-ahead-bias protection for research pipelines. pip install traceguard

Reproducible n-gram screening of language-model training data for overlap with evaluation benchmarks.

Zero-dependency CLI + library to detect benchmark/eval data contamination against a training corpus via n-gram overlap, flag leaks, and emit a cleaned dataset.

Detect train-eval data contamination with exact, fuzzy, and semantic matching. Built for LLM fine-tuning, RAG evaluation, and reliable benchmarking.

Zero-CoT Probe (ZCP) for detecting evasive data contamination in LLMs. Paper Link: http://arxiv.org/abs/2605.21856

A contamination-resistant, adversarial rebuild of the BIRD Text-to-SQL benchmark, constructed as the evaluation substrate for a governed data agent.

The official implementation repository for our EMNLP 2024 Findings paper, PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Models.

Contamination-proof LLM benchmark: fresh, verified coding tasks every run, impossible to benchmaxx

TurkishMMLU kirlilik listesi — Türkçe web'de birebir geçen 113 test sorusu (n=13), yöntem ve betik

Do quality filters pull benchmark questions into pretraining corpora? Injects MMLU, GSM8K, GPQA, ARC, HellaSwag, PIQA and TruthfulQA items into a corpus and ranks them with six quality classifiers, including DCLM, FineWeb-Edu and Gaperon.

Agent-generated test suites on Defects4J, with and without the implementation: mutation score does not see where a test's expectations came from.

A contamination-resistant research platform for testing whether public market data carries tradeable information. Ten pre-registered experiments on an explore -> holdout -> live-forward design; none survived.