llama

llama

FinetuneBackboneLlama(spec: ExecutionSpec)

Bases: BackbonedTrainer

Finetune from a pretrained Llama backbone.

Config keys

architecture/backbone/implementation: "llama" architecture/backbone/weights: e.g. "TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T"