MMLU-Pro

Knowledge

Harder version of MMLU with 10 answer choices.

Metrics
Accuracy (%)
Paper Dataset Eval Code

How to Run

pip install lm-eval && lm_eval --model hf --tasks mmlu_pro --batch_size auto

Leaderboard

Rank Model Provider Parameters Score