Skip to content

theseus

gpt

theseus

Home
Getting Started
Tutorial
Tutorial
How-to Guides
How-to Guides
- Database Migration
- Analysis System
- Evaluation System
- Remote Configuration
  Remote Configuration
- Adding Things
  Adding Things
  - Overview
  - Adding Components
    Adding Components
    
    Overview
    
    Model
    
    Experiment
    
    Dataset
    
    Evaluation
    
    Analysis Job
    
    Optimizer
    
    Learning Rate Schedule
  - Extending Components
    Extending Components
    
    Overview
    
    Trainer
    
    Module
    
    GPT
    
    Transformer Block
    
    Attention
    
    MLP
    
    Mixture of Experts
- Disclosing Code
- Automatic Mirroring
Design
Design
Reference
Reference
- analysis
  analysis
  - Index
  - attention
  - base
  - plots
  - trace
- base
  base
  - Index
  - axis
  - chip
  - dag
  - hardware
  - job
  - topology
- checkpoint
- cli
  cli
  - Index
  - app
  - configurator
    configurator
    
    Index
    
    fields
    
    steps
  - configure
  - interface
    interface
    
    Index
    
    actions
    
    age
    
    chart
    
    data
    
    driver
    
    fields
    
    location
    
    log_data
    
    log_list
    
    log_view
    
    navigation
    
    plot
    
    plot_controls
    
    plot_math
    
    plotly_options
    
    run
    
    state
    
    status
    
    timeline
    
    tree
    
    tree_row
    
    view
    
    views
    
    walk
  - jobs
  - run
  - submit
  - ui
- config
- data
  data
  - Index
  - datasets
    datasets
    
    Index
    
    addition
    
    alpaca
    
    bbq
    
    ccaligned
    
    cfq
    
    clutrr
    
    dataset
    
    dictlearn
    
    fever
    
    fineweb
    
    fineweb_edu
    
    flan
    
    harmfulqa
    
    longbench
    
    longhealth
    
    mmlu
    
    mnli
    
    mtob
    
    openr1_math
    
    pes2o
    
    pg19
    
    pile
    
    pile_detoxify
    
    pile_injected
    
    qqp
    
    reverse_sums
    
    roman_sums
    
    siqa
    
    squad
    
    sst2
    
    truncated_multiplication
    
    winogrande
  - style
  - tokenize
  - tokenizer
- evaluation
  evaluation
  - Index
  - base
  - datasets
    datasets
    
    Index
    
    addition
    
    alpaca
    
    arc_challenge
    
    arithmetic
    
    bbh
    
    bbq
    
    blimp
    
    ccaligned
    
    cfq
    
    clutrr
    
    dictlearn
    
    fever
    
    gsm8k
    
    hellaswag
    
    longbench
    
    longhealth
    
    math
    
    mmlu
    
    mnli
    
    mtob
    
    perplexity_evals
    
    pes2o
    
    pg19
    
    pg19_lengthgen
    
    pile
    
    pile_injected
    
    qqp
    
    reverse_sums
    
    roman_sums
    
    siqa
    
    squad
    
    sst2
    
    tinystories
    
    truncated_multiplication
    
    winogrande
- execute
  execute
  - Index
  - bootstrap
  - bundle
  - combobulator
  - config
  - dispatch
  - provider
    provider
    
    Index
    
    base
    
    slurm
    slurm
    
    Index
    
    availability
    
    provider
    
    ssh
    
    utils
    
    volcano
    volcano
    
    Index
    
    config
    
    manifest
    
    transport
  - run
    run
    
    Index
    
    lineage
    
    run
  - solve
- experiments
  experiments
  - Index
  - benchmark
  - continual
    continual
    
    Index
    
    benchmark
  - data
    data
    
    Index
    
    blockwise
    
    streaming
  - forking
    forking
    
    Index
    
    analysis
    
    diagnostic
    diagnostic
    
    Index
    
    attention
    
    cumulative_scores
    
    sequence
    
    sequence_attention
    
    stream_state
    
    token_attention
    
    evaluation
    
    instrumented
    
    pretrain
    
    reordering
    
    selector
    
    trace
  - loopdepth
    loopdepth
    
    Index
    
    pretrain
    
    projection
  - models
    models
    
    Index
    
    flt
    flt
    
    Index
    
    carved
    
    pretrain
    
    gpt
    
    gpt_neox
    
    lact
    
    llama
    
    looped
    
    moe
    
    qwen
    
    qwen_3_5
    
    skipdepth
    skipdepth
    
    Index
    
    gpt
    
    tiemlp
  - mok
    mok
    
    Index
    
    reward
    
    smoke
- inference
  inference
  - Index
  - base
  - ttt
- job
- model
  model
  - Index
  - activations
    activations
    
    Index
    
    swiglu
  - attention
    attention
    
    Index
    
    base
    
    forking
    
    gated_delta
    
    grouped
    
    injected
    
    rope
    
    scratching
  - axes
  - block
    block
    
    Index
    
    block
    
    flt
    
    forking
    
    gpt_neox
    
    hinted
    
    lact
    
    llama
    
    mamba
    
    moe
    
    qwen
    
    qwen_3_5
    
    scrambling
    
    scratching
    
    tiemlp
  - debug
  - layers
    layers
    
    Index
    
    flt_mlp
    
    lact
    
    layernorm
    
    mlp
    
    mrope
    
    rmsnorm
    
    rope
  - masks
  - models
    models
    
    Index
    
    base
    
    contrib
    contrib
    
    Index
    
    gpt_neox
    
    llama
    
    marin
    
    qwen
    
    qwen_3_5
    
    qwen_3_5_moe
    
    flt
    flt
    
    Index
    
    carved
    
    carved_gpt
    
    looped
    
    hybrid
    
    ignorebubbles
    
    lact
    
    looped
    
    mamba
    
    middle_looped
    
    moe
    
    scramblebubbles
    
    scratchbubbles
    
    skipdepth
    skipdepth
    
    Index
    
    gpt gpt
    On this page
    
    gpt
    
    SkipDepthGPT
    
    thoughtbubbles
    
    tiemlp
  - module
  - moe
    moe
    
    Index
    
    base
    
    bias_balanced
    
    shared
  - utils
- quick
- registry
- store
- training
  training
  - Index
  - backbone
  - base
  - contrastive
  - flywheel
    flywheel
    
    Index
    
    contrastive
    
    dataset
    
    io
    
    padded
    
    pmd
    
    strategy
    
    stream
  - grpo
  - kl_divergence
  - lora
  - optimizers
    optimizers
    
    Index
    
    adamw
    
    base
    
    muon
  - ppo
  - profiler
  - schedules
    schedules
    
    Index
    
    base
    
    cosine_rewarm
    
    wsd
    
    wsds
  - utils
About

On this page

gpt
- SkipDepthGPT

gpt

`gpt` ¶

`SkipDepthGPT` ¶

Bases: GPT

Replace selected boundary states once and restore them before a chosen block.