gpt_neox

gpt_neox

FinetuneBackboneGPTNeoX(spec: ExecutionSpec)

Bases: BackbonedTrainer

Finetune from a pretrained GPT-NeoX/Pythia backbone.

Config keys

architecture/backbone/implementation: "gpt_neox" architecture/backbone/weights: e.g. "EleutherAI/pythia-70m-deduped"