nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
比较改动
选两个分支看看改了什么,再发起 PR 把工作带过去。
可以合并。 这两个分支可以自动合并。
登录后才能发起 PR。
2 个提交
+14 −0 routes/adam-mini/README.md 新增 查看文件
| @@ -0,0 +1,14 @@ | ||
| 1 | +--- | |
| 2 | +tags: training | |
| 3 | +--- | |
| 4 | +# adam-mini | |
| 5 | + | |
| 6 | +Adam-mini in place of Muon, for memory. | |
| 7 | + | |
| 8 | +## What happened | |
| 9 | + | |
| 10 | +- 2026-09-06: 26.1 minutes to 3.28 (−18.9 on the baseline) | |
| 11 | + | |
| 12 | +## Next | |
| 13 | + | |
| 14 | +Slower to the target than Muon: a negative result. | |