加入
nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
turing
README.md
| tags | training, model |
|---|
tuned-lr-rotary
A tuned learning rate, rotary embeddings and a trapezoidal schedule.
What happened
- Started.
Next
Try another optimizer for the hidden matrices.