nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
Score: 2026-10-17: 3.95 minutes to 3.28 (−41.05 on the baseline)
浏览文件检验
score
+2 −1 routes/sparse-attn-schedule/README.md 查看文件
| @@ -1,6 +1,6 @@ | ||
| 1 | 1 | --- |
| 2 | 2 | issue: 5 |
| 3 | -status: working | |
| 3 | +status: done | |
| 4 | 4 | tags: model, training |
| 5 | 5 | --- |
| 6 | 6 | # sparse-attn-schedule |
| @@ -11,6 +11,7 @@ Block-sparse attention with a batch-size schedule. | ||
| 11 | 11 | |
| 12 | 12 | - 2026-10-11: 4.66 minutes to 3.28 (−40.34 on the baseline) |
| 13 | 13 | - 2026-10-14: 4.9 minutes to 3.28 (−40.1 on the baseline) |
| 14 | +- 2026-10-17: 3.95 minutes to 3.28 (−41.05 on the baseline) | |
| 14 | 15 | |
| 15 | 16 | ## Next |
| 16 | 17 | |