---
tags: systems
---
# fp8-head

FP8 matmul for the language-model head.

## What happened

- 2026-10-07: 5.03 minutes to 3.28 (−39.97 on the baseline)
- 2026-10-10: 5.4 minutes to 3.28 (−39.6 on the baseline)

## Next

A sparser attention schedule.