-
Notifications
You must be signed in to change notification settings - Fork 6
Pull requests: OpenEuroLLM/JudgeArena
Author
Label
Projects
Milestones
Reviews
Assignee
Sort
Pull requests list
Meta-Eval (2/2): Report Model Request Usage for Benchmark Runs
#112
opened Aug 20, 2026 by
kargibora
Member
•
4/4
Loading…
Meta-Eval (1/2): Add Judge Meta-Evaluation Tasks
#111
opened Aug 20, 2026 by
kargibora
Member
•
3/4
Loading…
Official Task Variations (2/2): Add arena-hard, alpaca eval and mt-bench official tasks
#109
opened Aug 20, 2026 by
kargibora
Member
•
2/4
Loading…
Official Task Variations (1/2): Refactor Prompt Parsing into Reusable Components
#108
opened Aug 20, 2026 by
kargibora
Member
•
1/4
Loading…
Meta-eval: (6/7) Measure held-out Elo error against annotation budget
#106
opened Aug 15, 2026 by
ErlisLushtaku
Collaborator
Loading…
Meta-eval: (5/7) Add Arena-Hard and AlpacaEval judge prompt presets
#105
opened Aug 15, 2026 by
ErlisLushtaku
Collaborator
Loading…
Meta-eval: (4/7) Register ranking metrics as the meta-eval scoring adapter
#104
opened Aug 15, 2026 by
ErlisLushtaku
Collaborator
Loading…
Meta-eval: (2/7) Score sampled battles with accuracy and Cohen's kappa
#102
opened Aug 15, 2026 by
ErlisLushtaku
Collaborator
Loading…
Meta-eval: (3/7) Judge both A/B orders with orientation-normalized rows
#103
opened Aug 15, 2026 by
ErlisLushtaku
Collaborator
Loading…
Meta-eval: (1/7) Declare meta-evaluation as a packaged task
#101
opened Aug 15, 2026 by
ErlisLushtaku
Collaborator
Loading…
Cache: WIP (7/7) Remove the unused Parquet cache store
#100
opened Aug 8, 2026 by
ErlisLushtaku
Collaborator
•
Draft
Cache: WIP (6/7) Synchronize inference caches through Hugging Face
#99
opened Aug 8, 2026 by
ErlisLushtaku
Collaborator
•
Draft
Cache: WIP (5/7) Adopt inference caching in MT-Bench
#98
opened Aug 8, 2026 by
ErlisLushtaku
Collaborator
•
Draft
Cache: WIP (4/7) Adopt inference caching in core workflows
#97
opened Aug 8, 2026 by
ErlisLushtaku
Collaborator
•
Draft
Cache: WIP (3/7) Cache inference across model providers
#96
opened Aug 8, 2026 by
ErlisLushtaku
Collaborator
•
Draft
Cache: WIP (2/7) Cache inference at do_inference
#95
opened Aug 8, 2026 by
ErlisLushtaku
Collaborator
•
Draft
Cache: WIP (1/7) Add content-addressed local inference caches
#94
opened Aug 8, 2026 by
ErlisLushtaku
Collaborator
•
Draft
ProTip!
Exclude everything labeled
bug with -label:bug.