mmlu

mmlu

MMLU(split: str = 'test', config: str | None = 'all')

Bases: ChatTemplateDataset

MMLU: Massive Multitask Language Understanding (Hendrycks et al., 2021).

Multiple-choice questions across 57 subjects. The config parameter selects a subject (default "all").