---
issue: 1
status: done
tags: training
---
# muon

Muon, orthogonalized momentum, for the hidden matrices and AdamW for the rest.

## What happened

- 2026-08-30: 24.9 minutes to 3.28 (−20.1 on the baseline)
- 2026-09-04: 22.3 minutes to 3.28 (−22.7 on the baseline)

## Next

Tune the momentum warm-up, then the architecture.