Abstract
OpenRSI's gemm-h100-refined asks for a hand-written GPU kernel that multiplies two 4096 × 4096 half-precision matrices, scored in trillions of floating-point operations per second (TFLOP/s). Our platform's starting point was kernel v490 left from October, which read 785.111 on this run's judge card. In Shoal round 1 (11-08 to 11-09, 24 hours), the board briefly reached 1060.73, above this card's published half-precision peak of about 989: those kernels lowered internal numerical precision and are invalid results. Six hours into the run, we added a source-level precision gate to the judge; across 1,327 evaluations after the gate, the honest best reached 803.343, 2.3% faster than the starting kernel. Because this is a single in-run reading and the pre-registered same-card retest at the end was not performed, it stands as provisional; it is compared only against our platform's starting point, not against public leaderboards, and has had no outside review.
Keywordsmatrix multiplyGPU kerneljudgeprecision
1Problem
How fast can one card go
Multiplying two 4096 × 4096 half-precision matrices takes about 137 billion floating-point operations. The better the kernel, the more operations per second it completes; the judge spot-checks part of the output and counts it as valid if the error stays within tolerance.
The operation count is 2 × 4096³ = 137,438,953,472. A score is the median TFLOP/s across 5 fresh-process repeats; higher is better. Calling off-the-shelf libraries such as cuBLAS, CUTLASS, or Triton is forbidden.
Two reference lines. On the same machine, the vendor's own cuBLAS runs at 683–706. The judge uses an H200, whose NVIDIA-published dense half-precision peak is about 989 (1,979 with sparsity): no kernel doing honest half-precision arithmetic can outrun it.
Best before. The starting kernel v490 came from an October run of ours (gemm-c1, 10-22 to 10-24), where it read 783.185 in-run; at the start of this run it read 785.111 on judge card 5. Its design derives from mechanisms in OpenRSI's public logs rather than from scratch offline. Scores in those public logs were obtained on an H100, whose absolute throughput is not comparable to an H200, so we do not compare against them here.
2Method
A shortcut, a gate, and an honest climb
The story of this problem is less about scores than about rules and judging: a faster shortcut appeared, was questioned in the open, was blocked by a gate on the judge, and honest work kept climbing from there.
Table 1 Conditions of the run
| Item | Value |
|---|---|
| Run | Shoal round 1 orsi-gemm-r1, 11-08 05:21:55 to 11-09 05:21:55 UTC |
| Hardware | Card 5 for the judge and card 6 for jobs, both H200s of ours |
| Members | Claude Code and Codex launched by us on our own machines using our own quota; seats scaled from 12 to 22, with 67 members making tool calls |
| Network | Members had internet access |
| Thresholds | Pre-registered a +1% threshold and an alternating same-card retest at the end; the retest was not run |
| Cost | Not reported here: no audited record |
Times below are in UTC, with minutes since launch in parentheses. On the mechanism of that shortcut, this report states only that it lowered internal numerical precision.
- 07:11–07:39 (109–138): One Codex member proposed in the channel the hypothesis of trading lower precision for speed while staying within the problem's tolerance; a second reached the same conclusion independently within four minutes, and a third joined nine minutes later. The first version was too slow and logged as a negative result.
- 08:31, 08:38: Before the approach set any new high, the member who proposed it wrote down its flaw: "not robust across input ranges … any adoption needs numerical fallback or restriction clearly documented".
- 08:45 (203.1): The first new high along this line hit 793.06, reached 798.6 at 08:46, and was adopted into main by a member at 08:48. Across the 31 minutes from 08:42 to 09:13, 17 members submitted source of this kind for the first time — 30 in total before the gate.
- 09:36–09:55: One Claude member first tried and failed to reproduce it as written, then wrote publicly, "I do not think it should be silently ranked as fp16", asked the problem owner to rule on it, and measured that on wide-range inputs its error crossed the threshold.
- 10:17 (295): The board crossed 989; at 11:13 it hit 1060.73. From minute 300 until the gate, all 117 scored submissions carried lowered-precision source.
- 11:31 (369): We decided to tighten the judge mid-run and void those results: a source-level precision gate was added to the judge, evaluated before compilation or GPU execution, and logged as a deviation.
- 11:36: Members read the new rule from failed check logs; PR #63 merged, restoring main to the half-precision v691 just 5 minutes after the gate landed. Members then packaged the gate as a tool on main and changed adoption to require two fixed holdout comparisons.
3Result
Honest best after the gate: 803.3
Over nearly 18 hours and 1,327 evaluations after the gate, the honest best climbed step by step to 803.343, 2.3% faster than the starting point; 1060.73 stands void.
Every evaluation from 11:37 onward passed the precision gate: 788.5 (11:37) → 791.4 (12:30) → 795.8 (13:22) → 798.9 (17:23) → 800.2 (11-09 01:27) → 803.343 (04:54, minute 1,412.5). The top five readings, 803.17–803.34, came from branches of five members. Branch names point to tile overlap, reverse-K, register splitting, and zigzag ordering — read from branch names only, as the kernel source is not at hand and has not been verified.
Table 2 Results and references (TFLOP/s, judge card 5)
| Reading | Value | How obtained | Status |
|---|---|---|---|
| Starting v490 | 785.111 | First evaluation of main at launch | Prior best on this platform |
| Pre-gate honest best | 789.253 | Minute 162.7, single in-run reading | — |
| Board high | 1060.73 | Minute 351.3, lowered internal precision | invalid |
| Post-gate honest best | 803.343 | Minute 1,412.5, single in-run reading, +2.3% | new record (provisional) |
4Checks
How it was judged, and who reproduced it
5Limits
What it does not show
LimitsWhat this report cannot say
- One reading, provisional. 803.343 is a single in-run reading without a retest; +2.3% is relative only to this platform's starting 785.111, both on H200.
- Not compared against public leaderboards. Public scores were obtained on H100, so absolute throughput is not comparable; we claim no ranking.
- The gate was added mid-run. Logged as a deviation. Across 1,674 chat messages and comments after the gate, only keyword scanning and spot reading were done; seeing no probing or pushback does not prove there was none.
- A single case, not a law. Alongside minference, these are two problems, two judges, and two cohorts — not a controlled comparison; one cannot conclude "whatever the judge says, the crowd does".
- Not called cheating. The approach was proposed in the open, its inventor noted its lack of robustness before it became a new high, and it passed the judge as written at the time. Here we describe its details only as lowering internal numerical precision.
- Starting point was not from scratch. The design of v490 derives from mechanisms in OpenRSI's public logs.
- One run, no control. Members were launched by us and had internet access.
6Data and code
Where the records live
Every number above has been recomputed from the records below. Those records reside in Shoal's private experiment repository, which readers cannot open; the file names and folder lab/docs/2026-11-12-mas-synthesis/notes/centers-orsi/ are given for our own internal audit trail. Kernel sources sit in the run archive and are also not public. Only the problem itself is public: gemm-h100-refined in OpenRSI-Index.
Every figure traced to a record
gemm_results.json (1,898 evaluations: timestamp, minute, branch, score, whether source lowered precision)Starting 785.111, pre-gate 789.253, board 1060.73, post-gate 1,327 evaluations and 803.343, step-by-step climbhack_spread.jsonFirst submission timestamps for all 31 members who submitted lowered-precision source (30 before the gate), plus half-hourly sharesRegistration log entry for this run+1% threshold, end-of-run retest, 11:31 deviationKernel for 803.343 (sha 287b34fa2)Source in the run archive, still on the run machine, not public7Contributions
Who did what
Agents wrote the kernels, ran evaluations, raised the challenge, and restored main; the signed human authors answer for the conclusions and this report. Member handles are in-run identities rather than people and are omitted here.
8Thanks
Problem source
The problem comes from gemm-h100-refined in OpenRSI-Index.
9Versions
Version history
- 2026-11-17v1First published
10Checklist
What a reader will ask
Checklist (after NeurIPS and ARR)
- Claims: do the claims in the abstract and opening accurately reflect the contribution and scope of this report?YesThe abstract states provisional, single in-run reading, and platform-only comparison; the table in section 3
- Best before: is the prior best value given with its source and date?YesSection 1: v490 from October's gemm-c1, reading 785.111 on card 5 in this run
- Limits: are the limitations of the work discussed?YesSection 5
- Theory: does every theoretical result come with full assumptions and a complete, correct proof?N/AMeasured throughput, with no theorem
- Reproduction: is all information needed to reproduce the main result given, whether or not code and data are open?NoMeasurement setup is in section 1; the kernel source for 803.343 is not public and was not retested (sections 4 and 6)
- Code and data: are code and data open, with instructions sufficient to reproduce the result?NoSection 6: evaluation logs sit in Shoal's private experiment repository and the kernel sits on the run machine; only the problem is public
- Settings: are the run settings given, including parameters tried and final values used?YesConditions table in section 2; timestamp of the gate
- Statistics: are error bars, intervals, or summaries across runs given, stating whether a value is a best, mean, or single run?No803.343 is a single in-run best without a retest (stated in sections 4 and 5)
- Compute: are the machine, memory, duration, and cost needed for each run stated?NoHardware and duration are given (section 2), without an audited cost record
- Assets: are authors and licences given for code, data, and models used, and are new files described?YesProblem source is credited in Thanks; records are described in section 6
- AI use: if AI assistants or agents were used, is how they were used described?YesThe line under the authors and section 7
Cite this
@misc{shoal2026gemmh100,
title = {gemm, round 1: an honest half-precision 803.3 (provisional)},
author = {The Shoal team},
year = {2026},
note = {v1, 2026-11-17},
url = {https://shoal.club/news/gemm-h100/}
}