Skip to content
theseus
flt_mlp
Search
Jemoka/thx
theseus
Jemoka/thx
Home
Getting Started
Tutorial
Tutorial
Overview
Local training
Local analysis
Remote training
Your own experiments
What's Next
How-to Guides
How-to Guides
Database Migration
Analysis System
Evaluation System
Remote Configuration
Remote Configuration
Overview
Provider Configuration
Metadata Store
JuiceFS Integration
Adding Things
Adding Things
Overview
Adding Components
Adding Components
Overview
Model
Experiment
Dataset
Evaluation
Analysis Job
Optimizer
Learning Rate Schedule
Extending Components
Extending Components
Overview
Trainer
Module
GPT
Transformer Block
Attention
MLP
Mixture of Experts
Disclosing Code
Automatic Mirroring
Design
Design
Overview
Sharp Edges
Experiment DAG and Storage
Available Chips
Sharding Design
Config System
Analysis and Plotting
Reference
Reference
analysis
analysis
Index
attention
base
plots
trace
base
base
Index
axis
chip
dag
hardware
job
topology
checkpoint
cli
cli
Index
app
configurator
configurator
Index
fields
steps
configure
interface
interface
Index
actions
age
chart
data
driver
fields
location
log_data
log_list
log_view
navigation
plot
plot_controls
plot_math
plotly_options
run
state
status
timeline
tree
tree_row
view
views
walk
jobs
run
submit
ui
config
data
data
Index
datasets
datasets
Index
addition
alpaca
bbq
ccaligned
cfq
clutrr
dataset
dictlearn
fever
fineweb
fineweb_edu
flan
harmfulqa
longbench
longhealth
mmlu
mnli
mtob
openr1_math
pes2o
pg19
pile
pile_detoxify
pile_injected
qqp
reverse_sums
roman_sums
siqa
squad
sst2
truncated_multiplication
winogrande
style
tokenize
tokenizer
evaluation
evaluation
Index
base
datasets
datasets
Index
addition
alpaca
arc_challenge
arithmetic
bbh
bbq
blimp
ccaligned
cfq
clutrr
dictlearn
fever
gsm8k
hellaswag
longbench
longhealth
math
mmlu
mnli
mtob
perplexity_evals
pes2o
pg19
pg19_lengthgen
pile
pile_injected
qqp
reverse_sums
roman_sums
siqa
squad
sst2
tinystories
truncated_multiplication
winogrande
execute
execute
Index
bootstrap
bundle
combobulator
config
dispatch
provider
provider
Index
base
slurm
slurm
Index
availability
provider
ssh
utils
volcano
volcano
Index
config
manifest
transport
run
run
Index
lineage
run
solve
experiments
experiments
Index
benchmark
continual
continual
Index
benchmark
data
data
Index
blockwise
streaming
forking
forking
Index
analysis
diagnostic
diagnostic
Index
attention
cumulative_scores
sequence
sequence_attention
stream_state
token_attention
evaluation
instrumented
pretrain
reordering
selector
trace
loopdepth
loopdepth
Index
pretrain
projection
models
models
Index
flt
flt
Index
carved
pretrain
gpt
gpt_neox
lact
llama
looped
moe
qwen
qwen_3_5
skipdepth
skipdepth
Index
gpt
tiemlp
mok
mok
Index
reward
smoke
inference
inference
Index
base
ttt
job
model
model
Index
activations
activations
Index
swiglu
attention
attention
Index
base
forking
gated_delta
grouped
injected
rope
scratching
axes
block
block
Index
block
flt
forking
gpt_neox
hinted
lact
llama
mamba
moe
qwen
qwen_3_5
scrambling
scratching
tiemlp
debug
layers
layers
Index
flt_mlp
flt_mlp
On this page
flt_mlp
FLTMLP
lact
layernorm
mlp
mrope
rmsnorm
rope
masks
models
models
Index
base
contrib
contrib
Index
gpt_neox
llama
marin
qwen
qwen_3_5
qwen_3_5_moe
flt
flt
Index
carved
carved_gpt
looped
hybrid
ignorebubbles
lact
looped
mamba
middle_looped
moe
scramblebubbles
scratchbubbles
skipdepth
skipdepth
Index
gpt
thoughtbubbles
tiemlp
module
moe
moe
Index
base
bias_balanced
shared
utils
quick
registry
store
training
training
Index
backbone
base
contrastive
flywheel
flywheel
Index
contrastive
dataset
io
padded
pmd
strategy
stream
grpo
kl_divergence
lora
optimizers
optimizers
Index
adamw
base
muon
ppo
profiler
schedules
schedules
Index
base
cosine_rewarm
wsd
wsds
utils
About
On this page
flt_mlp
FLTMLP
flt_mlp
flt_mlp
¶
FLTMLP
¶
Bases:
MLP
ReLU^2 MLP with a fixed 4x expansion and no bias.
Back to top