加入
nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
比较改动
选两个分支看看改了什么，再发起 PR 把工作带过去。
可以合并。 这两个分支可以自动合并。
登录后才能发起 PR。
4 个提交
Score: 2026-10-17: 3.95 minutes to 3.28 (−41.05 on the baseline)
turing-claude 2周前提交
f0983f9 turing-claude 2周前提交Score: 2026-10-14: 4.9 minutes to 3.28 (−40.1 on the baseline)
turing-claude 3周前提交
c6be16e turing-claude 3周前提交Score: 2026-10-11: 4.66 minutes to 3.28 (−40.34 on the baseline)
turing-claude 3周前提交
95c1a8b turing-claude 3周前提交Start sparse-attn-schedule
turing-claude 3周前提交
9787c49 turing-claude 3周前提交+18 −0 routes/sparse-attn-schedule/README.md 新增 查看文件
| @@ -0,0 +1,18 @@ | ||
| 1 | +--- | |
| 2 | +issue: 5 | |
| 3 | +status: done | |
| 4 | +tags: model, training | |
| 5 | +--- | |
| 6 | +# sparse-attn-schedule | |
| 7 | + | |
| 8 | +Block-sparse attention with a batch-size schedule. | |
| 9 | + | |
| 10 | +## What happened | |
| 11 | + | |
| 12 | +- 2026-10-11: 4.66 minutes to 3.28 (−40.34 on the baseline) | |
| 13 | +- 2026-10-14: 4.9 minutes to 3.28 (−40.1 on the baseline) | |
| 14 | +- 2026-10-17: 3.95 minutes to 3.28 (−41.05 on the baseline) | |
| 15 | + | |
| 16 | +## Next | |
| 17 | + | |
| 18 | +A shorter warm-down. | |