Jozef Koval cd8b274f89 Pin the performance floor gate's contract with tests
check-perf-floors.sh is the only check between an order-of-magnitude
regression and main, and it has no tests: scripts/ carries a test file
for validate_georelays.py and two for the workflows around it, and
nothing for the script that decides whether a slow hot path merges. Its
behaviour is a shell retry loop around a Python heredoc that parses
captured test output and picks exit codes -- exactly the kind of logic
that drifts quietly.

This runs the real script against fixture PERF logs and pins the
documented contract:

- every floor met passes;
- the best value across appended attempts is what counts, so one
  retry that clears the floor is enough;
- a below-floor result is re-measured and passes once it clears;
- a below-floor result on every attempt fails (exit 1);
- a floored benchmark that stops reporting fails without a retry
  (exit 3) -- a re-measurement that happened to produce the line would
  paper over a silently dropped benchmark;
- no PERF lines and BITCHAT_SKIP_PERF_BASELINES=1 both skip (exit 0).

The re-measure command is replaced by a stand-in that records each
invocation and appends whatever the test says to the log the gate
points it at, so the retry path runs in milliseconds with no toolchain,
and a test that retries unexpectedly fails fast instead of building the
app.

One test pins a behaviour rather than endorsing it: a benchmark that
reports a PERF line with no floor prints "consider adding a floor" and
passes. That is the gate's one advisory branch -- removing a floored
benchmark fails, adding an unfloored one does not -- and it is left as
found so the follow-up flipping it is a three-line script change with a
one-assertion test diff.

The tests run in the app matrix entry immediately before the gate they
describe. They also fall under the scripts/tests discovery in
validate_georelays.yml, which is harmless.
2026-09-24 22:31:49 +02:00
..