---
issue: 4
status: done
tags: systems
---
# flex-attention-window

FlexAttention with a growing sliding window.

## What happened

- 2026-09-27: 8.2 minutes to 3.28 (−36.8 on the baseline)
- 2026-09-29: 9.1 minutes to 3.28 (−35.9 on the baseline)
- 2026-10-02: 7.2 minutes to 3.28 (−37.8 on the baseline)

## Next

FP8 for the head matmul.