Jozef Koval 739077305f Exit 2 on a malformed floors file instead of re-measuring
A broken floors file raised an uncaught exception, so Python exited 1.
The loop read exit 1 as "below floor" and measured twice more before
it reported a real regression. The gate now names the file and exits
2. The loop does not retry exit 2, as its comment says for setup
errors.

Broken means unreadable, not valid JSON, no "floors" object, or a
floor that is not a finite number. Some of these did not crash before,
but gave a wrong result. A NaN floor let every value pass, because a
comparison with NaN is always false. A `true` floor counted as 1, and
a `null` floor was reported as NO-FLOOR with exit 3.
test_malformed_floors_file_fails_without_a_retry covers six cases.

The exit-code comments also called exit 3 "floored benchmark missing".
This branch fails unfloored benchmarks with exit 3 too, so they now
say "floors and benchmarks out of step".
2026-11-01 17:56:30 +01:00
..