One place to run, compare, and understand evaluations from Ragas, DeepEval, and scikit-learn, without writing the glue scripts.
Status: early development. Not yet usable.
AI Evaluation Lab is a workbench that sits on top of the evaluation libraries you already use. Connect your system once, pick what to measure from a catalogue, and get a report organised as an investigation instead of a list of numbers.
It does not replace those libraries. It uses them and credits them.
- Connect a system through a guided flow, no integration code
- A catalogue of every metric from every installed library
- Retrieval, generation, efficiency, and classification metrics
- Confidence intervals and honest run-to-run comparison
- Failure analysis that separates retrieval problems from generation problems
- A self-contained HTML report
- A pass/fail regression check for CI pipelines
- Runs locally and free by default, using Ollama
- Specification: what is being built
- Vision: the long-term direction