Index skipdepth ¶ PretrainSkipDepthGPT(spec: ExecutionSpec, base: Optional[Node] = None) ¶ Bases: PretrainGPT Use the ordinary GPT pretraining pipeline with boundary replacement.