Skip to content
theseus
moe
Initializing search
theseus
Getting Started
Tutorials
Tutorials
Overview
Running Experiments
Running Experiments
Overview
Local
Remote (SSH & SLURM)
Remote (Volcano / K8s)
Remote (TPU)
Interactive REPL
Adding Things
Adding Things
Overview
Model
Experiment
Dataset
Evaluation
Design
Design
Overview
Dispatch Infrastructure
Config System
Mock System
Plot System
Reference
Reference
base
base
axis
chip
hardware
job
topology
cli
config
data
data
datasets
datasets
alpaca
bbq
ccaligned
cfq
clutrr
dataset
dictlearn
fever
fineweb
harmfulqa
longbench
longhealth
mmlu
mnli
mtob
pes2o
pg19
pile
pile_detoxify
qqp
siqa
squad
sst2
winogrande
tokenize
tokenizer
dispatch
dispatch
bootstrap
config
dispatch
mailbox
mailbox
mailbox
sidecar
slurm
solve
ssh
sync
tpu
volcano
evaluation
evaluation
base
datasets
datasets
bbq
blimp
ccaligned
cfq
clutrr
dictlearn
fever
longbench
longhealth
mmlu
mnli
mtob
perplexity_evals
pes2o
pg19
pile
qqp
siqa
squad
sst2
tinystories
winogrande
experiments
experiments
continual
continual
abcd
models
models
forking
gpt
gpt_neox
llama
moe
qwen
inference
inference
base
job
mock
model
model
activations
activations
swiglu
attention
attention
base
forking
grouped
rope
scratching
axes
block
block
block
forking
gpt_neox
llama
moe
moe
Table of contents
moe
qwen
scratching
layers
layers
layernorm
mlp
rmsnorm
rope
masks
models
models
base
contrib
contrib
gpt_neox
llama
marin
qwen
moe
scratchbubbles
thoughtbubbles
module
moe
moe
base
bias_balanced
plot
quick
registry
training
training
backbone
base
contrastive
flywheel
flywheel
contrastive
padded
pmd
strategy
kl_divergence
optimizers
optimizers
adamw
muon
schedules
schedules
wsd
wsds
utils
web
web
app
auth
generate_password_hash
models
routes
routes
api
auth
views
services
services
cache
checkpoints
logs
status
Table of contents
moe
moe
moe
¶
Mixture-of-experts transformer block.
Back to top