mirror of
https://github.com/permissionlesstech/bitchat.git
synced 2026-11-15 08:12:23 +00:00
check-perf-floors.sh is the only check between an order-of-magnitude regression and main, and it has no tests: scripts/ carries a test file for validate_georelays.py and two for the workflows around it, and nothing for the script that decides whether a slow hot path merges. Its behaviour is a shell retry loop around a Python heredoc that parses captured test output and picks exit codes -- exactly the kind of logic that drifts quietly. This runs the real script against fixture PERF logs and pins the documented contract: - every floor met passes; - the best value across appended attempts is what counts, so one retry that clears the floor is enough; - a below-floor result is re-measured and passes once it clears; - a below-floor result on every attempt fails (exit 1); - a floored benchmark that stops reporting fails without a retry (exit 3) -- a re-measurement that happened to produce the line would paper over a silently dropped benchmark; - no PERF lines and BITCHAT_SKIP_PERF_BASELINES=1 both skip (exit 0). The re-measure command is replaced by a stand-in that records each invocation and appends whatever the test says to the log the gate points it at, so the retry path runs in milliseconds with no toolchain, and a test that retries unexpectedly fails fast instead of building the app. One test pins a behaviour rather than endorsing it: a benchmark that reports a PERF line with no floor prints "consider adding a floor" and passes. That is the gate's one advisory branch -- removing a floored benchmark fails, adding an unfloored one does not -- and it is left as found so the follow-up flipping it is a three-line script change with a one-assertion test diff. The tests run in the app matrix entry immediately before the gate they describe. They also fall under the scripts/tests discovery in validate_georelays.yml, which is harmless.