pretrain

Pretraining the Fully Looped Transformer, and the standard GPT under the same recipe: Muon on block matrices, Adam on the tied embedding, one global gradient-norm clip (optimization/muon/clip_global_norm), WSD.