Five published tests

The cases. And the evidence.

Each paper gets a clear answer, a chart, raw numbers and a frank account of what the test could not decide.

5 / published4 / new on 3 Octoberraw data linked

Published / 001–005

Read the answer. Check the work.

All four case groups previously in the lab are now published with limits and downloadable CSVs.

Case 002 · published 3 October 2026

Does the 52-addition 3×3 matrix scheme work?

Original authors: Hugo Møller Nielsen

Yes: all 729 ordered coefficients match, and the scheme works over noncommutative rings. Direct Python is slower than NumPy; historical search claims remain untested.

READ THE REPORT AND DATA ↗
Case 003 · published 3 October 2026

Are new shortest-path algorithms faster than Dijkstra?

Original authors: Ran Duan, Jiayi Mao, Xiao Mao, Xinkai Shu, Longhui Yin; Avi Kadria, Liam Roditty

Not on four US road maps in our implementations: 1,600 certified quiet-server road runs, with median slowdowns about 6–75×. These timings do not test the papers' asymptotic bounds.

READ THE REPORT AND DATA ↗
Case 004 · published 3 October 2026

Does SAMuon beat Muon? A small-scale test

Original authors: Xiaodong Wu, Wenyi Yu, Chao Zhang, Philip Woodland

At 0.056M–0.788M parameters SAMuon had lower median final loss at each size; post-hoc tokens-to-target were mixed, and the spectral stability mechanism did not reproduce.

READ THE REPORT AND DATA ↗
Case 005 · published 3 October 2026

Does tokenizer search matter more than its objective?

Original authors: Ahmetcan Yavuz, Clara Meister, Tiago Pimentel

The published intrinsic measurements mostly reproduce (92/96 Table 1 values), but a tiny 1.32M-parameter test did not show the bottom-up BPB advantage. Paper-scale behavior remains open.

READ THE REPORT AND DATA ↗
Case 001 · published 2 October 2026

Does two-choice linear probing hold up?

Amir Azarmehr, Michael A. Bender, William Kuszmaul, Rose Silver

At 99% load, greedy two-choice used 5.797400 successful probes versus 51.024915 for an ordered one-choice control across five seeds. CPU timing is preliminary; the full-table offline result awaits second review.

READ THE REPORT AND DATA ↗

Reading guide

What these results mean.

Open the notes when you want the comparison boundaries.

Read the publication notes

The case number identifies our test, not a replacement for the paper. Each report links its original authors and arXiv source at the top, then shows an answer, a figure, the measured table and a claim-by-claim verdict. We keep an expert layer behind native disclosure controls. The charts are optional: every finding remains in normal HTML so a reader or crawler can inspect it without JavaScript. Each report also exposes the CSV used for its numbers and the limits that matter most.

Case 002 establishes an exact arithmetic identity, including its ordered noncommutative coefficients. That is a strong mathematical check of a construction. It is separate from the paper's historical priority and search account, which need original artifacts. A direct Python timing comparison also cannot tell us whether a tuned low-level implementation would be useful at large matrix sizes. Read the operation count and basis-boundary notes before comparing the headline 52 with other methods.

Case 003 groups three related shortest-path papers on one page because the road-map experiment used one calibrated Dijkstra family. Every output in the full sweep passed a distance certificate. The implementations were slower on these maps, yet some queues and black-box routines were substituted. The original big-O theorems are about much larger inputs in a comparison-addition model and are not tested by these wall-clock runs. A previous independent study reported a much smaller slowdown for the 2025 method; the magnitude of ours is implementation-sensitive. A re-run on a quiet, single-core server gave the same picture: 6–75× slower than direct Dijkstra, with all 1,600 outputs certified.

Case 004 uses small CPU language models to probe SAMuon. Muon beat AdamW in the calibration, and SAMuon had lower median final loss at each tested size. The separate tokens-to-target question has a mixed answer because the target was chosen after the first results and one size lost to Muon. The spectral stability explanation did not reproduce reliably. The tested models are about a thousand times smaller than the original GPU study, so this is a small-scale observation rather than a verdict on its large-model claims.

Case 005 starts from the authors' shipped tokenizer assets. Most of the paper's intrinsic Table 1 values, along with the fixed vocabulary overlap and illustrative examples, reproduce. A tiny matched downstream model did not show the bottom-up advantage. Its confidence interval describes only this short small-model training setup. The paper's transformer checkpoints, multilingual evaluation corpus and BLiMP materials were unavailable locally, so its main model-scale conclusion remains unresolved by our test.

Our publication method starts with quoted, source-located claims. Implementations must pass independent correctness checks before timing. Baselines are calibrated, run repeatedly, and tied to downloadable data. Suspected paper or code issues receive a human and second review before they become findings. Five shortest-path and two tokenizer issue flags have passed that second review; each case page labels them by severity, and none changes the computed distances or the paper's results. If a result changes, the corrections log provides the public record. The method page gives the full protocol and case 001 shows an internal allegation withdrawn before publication.