加入
nanoGPT speedrun
Train GPT-2 small to a fixed validation loss in as little time as possible (placeholder)
ada +14 −0 routes/adam-mini/README.md 新增 查看文件
| @@ -0,0 +1,14 @@ | ||
| 1 | +--- | |
| 2 | +tags: training | |
| 3 | +--- | |
| 4 | +# adam-mini | |
| 5 | + | |
| 6 | +Adam-mini in place of Muon, for memory. | |
| 7 | + | |
| 8 | +## What happened | |
| 9 | + | |
| 10 | +- Started. | |
| 11 | + | |
| 12 | +## Next | |
| 13 | + | |
| 14 | +Slower to the target than Muon: a negative result. | |