benchmark
¶Benchmark trainers with statically declared curricula and native checkpoints.
STAGES declares each dataset mixture; training/tokens supplies matching budgets. LoRA benchmarks use the same normal DATASET contract as LoRATrainer. Analyses are separate jobs or declarations in ANALYSIS, never model plotting callbacks.
BenchmarkLoRAConfig(batch_size: int = field('training/batch_size', default=512), per_device_batch_size: int = field('training/per_device_batch_size', default=(-1)), total_tokens: int = field('training/tokens', default=1000000000), lr: float = field('optimization/lr', default=0.0003), warmup_pct: float = field('training/warmup_pct', default=0.01), decay_pct: float = field('training/decay_pct', default=0.1), validate: bool = field('training/validation', default=True), evaluate: bool = field('training/evaluate', default=True), analyze: bool = field('training/analyze', default=True), block_size: int = field('architecture/block_size', default=512), param_dtype: str = field('architecture/dtype/param', default='float32'), activation_dtype: str = field('architecture/dtype/activation', default='bfloat16'), report_interval: int = field('logging/report_interval', default=32), checkpoint_interval: int = field('logging/checkpoint_interval', default=1024), validation_interval: int = field('logging/validation_interval', default=512), validation_steps: int = field('training/validation_steps', default=2048), pre_lora_tokens: List[int] = field('training/pre_lora_tokens', default_factory=(lambda: [1000000000])), post_lora_tokens: List[int] = field('training/post_lora_tokens', default_factory=(lambda: [100000000])))
dataclass
¶