pile

pile

Pile perplexity evaluation.

Measures language model perplexity on a sample from the Pile. Returns 1/perplexity (higher is better).

PileEval(num_samples: int = 500)

Bases: PerplexityEvaluation

Perplexity evaluation on the Pile (EleutherAI).