middle_looped

middle_looped

MiddleLoopedGPT

Bases: LoopedGPT

Loop only the blocks between the lexical and output endpoints.

MiddleMoELoopedGPT

Bases: MiddleLoopedGPT

Middle-only looped GPT with routed feed-forward blocks.

new_block() -> Any

Construct one uniquely parameterized MoE block.