bitchat-android/docs/release-gate-runbook.md

Physical-device and cross-client release gate

This runbook turns Milestone 10 into a repeatable release procedure. The gate uses a host-side CLI and USB/ADB as its control channel, so control traffic never shares BLE, Wi-Fi Aware, Nostr, or Tor with the system under test.

The gate cannot pass without the required physical devices and counterpart clients. A pending or blocked result is useful diagnostic evidence, but it is not release approval.

Safety and privacy rules

• Use only disposable lab app data, identities, nicknames, messages, and files.
• Never use a personal Nostr account or a production relay.
• Do not put device serials, UDIDs, Bluetooth/MAC/IP addresses, peer IDs, usernames, email addresses, local home paths, or message contents in a result, trace, filename, issue, commit, or release artifact.
• Device selectors may be supplied to ADB commands as ephemeral inputs. The tooling emits only logical aliases such as android-current.
• Models, manufacturer classes, Android API levels, negotiated MTU classes, client versions, commit hashes, aggregate counts, durations, and stable failure reason codes are allowed.
• Do not archive raw logcat. Convert observations to the structured, privacy-checked trace format, and keep any raw diagnostic capture local until it has been reviewed and sanitized.

The validator rejects known identifying fields and values before a passing bundle can be created.

Required lab

Prepare:

• At least three physical Android devices for three-hop relay testing.
• At least two Android API levels and two manufacturer classes.
• BLE central and peripheral support on every Android device.
• At least one Android 13+ device with Wi-Fi Aware.
• One physical device running the current iOS client.
• The last supported Android client.
• The release-candidate APK built from one exact full Git commit.
• A local, disposable Nostr relay/Tor fixture with production network access blocked.

One physical handset may be reused for the legacy-client phase after the current-client evidence for that slot is complete, but the matrix must keep the logical aliases and installed client versions unambiguous.

1. Verify the deterministic gate

From the repository root:

./gradlew clientRewriteContractTest checkChangedLineCoverage lintDebug
python3 tools/release_gate/release_gate.py validate-manifest

Do not begin device work from a dirty tree or a build whose deterministic gate does not pass.

2. Create the device matrix

Copy tools/release_gate/device-matrix.example.json to an ignored working directory under release-gate-results/. Replace every template value and set both current-Android commit fields to the exact full commit under test.

Probe Android capabilities without storing the ADB selector:

python3 tools/release_gate/android_lab.py probe \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --alias android-current

Copy only the returned logical metadata into the matrix. Validate it:

python3 tools/release_gate/release_gate.py validate-matrix \
  --matrix release-gate-results/device-matrix.json \
  --commit "$BITCHAT_RELEASE_COMMIT"

The matrix validator enforces physical devices, three Android participants, two API levels, two manufacturer classes, Wi-Fi Aware, BLE roles, iOS, and explicit current/legacy client versions.

3. Initialize disposable fixtures

python3 tools/release_gate/release_gate.py init \
  --matrix release-gate-results/device-matrix.json \
  --commit "$BITCHAT_RELEASE_COMMIT" \
  --run-id rc-lab-01 \
  --output release-gate-results/rc-lab-01

Initialization pins the scenario and fixture manifests, creates every scenario as pending, and generates deterministic:

• zero-byte and small files;
• a Unicode-named medium file;
• sparse exact-maximum and oversized boundary files.

The fixture manifest records size and SHA-256. The final archive contains the manifest, not the large fixture bodies.

Clear only the disposable app data on each selected lab device:

python3 tools/release_gate/android_lab.py prepare \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --confirm-disposable-app-data

This stops the app and runs package-data cleanup. The explicit confirmation is required because the operation is destructive to that app's local data.

4. Execute scenarios

The canonical scenario list is tools/release_gate/scenarios.json. It contains 27 mandatory scenarios:

• the complete physical transport matrix;
• Android API/manufacturer/permission/background coverage;
• 11 Android-to-Android workflows;
• 8 cross-client/backward-compatibility workflows;
• 6 background and endurance workflows.

For each scenario:

1. Confirm the listed participants and capabilities.
2. Perform the corresponding steps in device-transport-test-matrix.md and the Milestone 10 checklist.
3. Record connection, lifecycle, transport, receipt, resource, and terminal state as aggregate evidence.
4. Append at least one structured trace event.
5. Mark the scenario pass, fail, blocked, or unsupported.

Record evidence with the exact keys declared by the scenario:

python3 tools/release_gate/release_gate.py record \
  --run release-gate-results/rc-lab-01 \
  --scenario A2A-001 \
  --status pass \
  --evidence connection-transitions=4 \
  --evidence packet-correlation-count=6 \
  --evidence failure-reasons=none

Append a privacy-safe trace event:

python3 tools/release_gate/release_gate.py trace \
  --run release-gate-results/rc-lab-01 \
  --scenario A2A-001 \
  --source android-current \
  --event reconnect-terminal \
  --outcome pass \
  --metric reconnect-count=1 \
  --metric duplicate-delivery-count=0

Capture resource snapshots during endurance work:

python3 tools/release_gate/android_lab.py snapshot \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --alias android-current \
  --run release-gate-results/rc-lab-01 \
  --scenario END-003

Use run-local sequential correlation labels while observing packets; archive only aggregate correlation counts. Record failures with a stable reason code, file a regression issue, and preserve the incomplete artifact.

5. Endurance requirements

• END-001 requires at least 240 minutes.
• END-002 requires at least 50 large-transfer/cancellation cycles.
• Sample memory, threads, file descriptors, wake locks, connection counts, and late callbacks at consistent intervals.
• A passing result requires bounded resource behavior and a clean terminal state; merely completing the time window is insufficient.

The validator rejects shorter durations and cycle counts.

6. Inspect progress and validate

During a run:

python3 tools/release_gate/release_gate.py validate \
  --run release-gate-results/rc-lab-01 \
  --allow-incomplete

python3 tools/release_gate/release_gate.py summary \
  --run release-gate-results/rc-lab-01

The release validator, without --allow-incomplete, requires:

• every scenario to be pass;
• every declared evidence field;
• at least one structured trace per scenario;
• the pinned scenario and fixture manifests;
• the exact client commit and complete device matrix;
• endurance minimums;
• a completion timestamp;
• no detected identifying fields or values.

unsupported, blocked, and pending never satisfy release approval.

7. Archive release approval

After the complete validator passes:

python3 tools/release_gate/release_gate.py bundle \
  --run release-gate-results/rc-lab-01 \
  --output release-gate-results/rc-lab-01.zip

The deterministic archive contains the scenario manifest, device/client matrix, results, structured trace, fixture manifest, Markdown summary, and SHA256SUMS. Attach it to the release approval record without renaming fields or adding raw diagnostics.

Finally, clean the disposable app data with the same confirmed cleanup command and stop the local relay/Tor fixture.

Failure handling

• fail: behavior violated a contract. Record a stable reason code, file a bug, add a deterministic regression where possible, fix it, and rerun the affected scenario plus dependent scenarios.
• blocked: required lab infrastructure or counterpart client was unavailable. Preserve the artifact and do not approve release.
• unsupported: the selected device lacks a capability. Because the defined matrix requires Wi-Fi Aware, replace the device or matrix; unsupported does not waive a mandatory scenario.
• A flaky result is a failure until its cause is understood. Never average retries into a pass.

Appendix: mesh lab (ADB test hooks, debug builds)

For day-to-day development there is a lighter-weight harness that drives a debug-only broadcast receiver (app/src/debug/, never shipped in release) exposing mesh operations over ADB: scan, connect, Noise handshake, DMs, public broadcast, announce, file send/receive, BLE toggle, state dumps, and raw packet injection. Results are JSON files in the app sandbox polled by the host (cache/testhook/results/<id>.json, also logged under tag TestHook).

Prerequisites

• A JDK (e.g. the one bundled with Android Studio; set JAVA_HOME) and the Android SDK platform-tools. adb must be on PATH or ANDROID_HOME set.
• Python 3.10+ on the host. No third-party packages are required.
• Two physical Android devices (API 26+, BLE) with USB debugging enabled, both plugged into the host. Emulators are not supported (BLE mesh).
• Verify both are visible: adb devices → note the serials.

Device preparation (important)

Keep both phones unlocked with the screen on for the whole run. A locked or dozing device forces the app into POWER_SAVER (1 s BLE scan per 60 s), which makes discovery and handshakes take minutes and will flake every scenario. The harness runs wake() (dismiss keyguard, stretch screen timeout) during setup, but it cannot defeat a secure lock screen — unlock the devices manually first. Note that svc power stayon only helps while a device is actually charging.

Build and set up

./gradlew assembleDebug
python3 tools/release_gate/mesh_lab.py setup \
  --serial-a <serial-1> --serial-b <serial-2> \
  --apk app/build/outputs/apk/debug/app-arm64-v8a-debug.apk

setup cycles Bluetooth, installs the APK, clears app data, grants all runtime permissions, wakes and launches the app, sets deterministic nicknames (alice/bob), and waits for mutual peer discovery. It is safe (and recommended) to rerun setup before each scenario batch; --apk may be omitted if the current build is already installed.

Run scenarios

python3 tools/release_gate/mesh_lab.py scenario all \
  --serial-a <serial-1> --serial-b <serial-2> --out /tmp/meshlab-evidence

Scenario	What it asserts
dm	Noise handshake both ways, encrypted DM round trips with content match
favorite_verification	favorite signal, orange-outline/filled mutual state, and peer fingerprint verification
broadcast	public mesh message A→B
file	1 KB broadcast file, receiver SHA-256 matches fixture
file_oversize	>256-fragment broadcast file is rejected sender-side, receiver sees nothing
file_private	Noise-encrypted private file, digest match
media_private	private-chat contact ID resolves to the live mesh peer; voice, image, and generic-file digests match
raw	raw packet injection is accepted by the mesh
session_recovery	force-stop B mid-session: identity persists, re-handshake, DMs flow again
identity_reset	pm clear B mid-session: new identity, rediscovery, handshake, DMs
all	every scenario above in sequence

Each run writes <scenario>-evidence.json to --out (digests, timings, session states, logcat excerpts on failure) and exits non-zero on failure. Evidence is a local diagnostic artifact; it may contain lab peer IDs and is not privacy-checked like release-gate bundles — do not publish it.

Ad-hoc commands

Any hook command can be sent to one device directly:

python3 tools/release_gate/mesh_lab.py cmd --serial <serial> scan --extra timeout_ms=30000
python3 tools/release_gate/mesh_lab.py cmd --serial <serial> handshake --extra peer=<peer-id>
python3 tools/release_gate/mesh_lab.py cmd --serial <serial> state   # full mesh dump

See TestHookDriver.kt for the full command set (ping, start, stop, whoami, set_nickname, scan, peers, connect, handshake, session, announce, broadcast_msg, dm_send, dm_recv, msg_recv, favorite_set, favorite_status, verification_set, verification_status, file_send, file_recv, file_cancel, raw_send, ble, state, clear_results).

Troubleshooting

• Discovery/handshake timeouts: almost always a locked or dozing phone — unlock both devices and rerun setup. cmd ... state shows App In Background: true and the BLE duty cycle when this is the cause.
• Stale app state after many churn runs: svc bluetooth disable/enable on both devices (done automatically by setup) clears zombie GATT links.
• Watch the wire: adb -s <serial> logcat -s TestHook MessageHandler FragmentManager BitchatFilePacket shows commands, results, decrypt failures, fragment rejects, and saved incoming files in real time.
• Results also persist on-device at run-as com.bitchat.droid cat cache/testhook/results/<id>.json.

Unlike the release gate, this harness is a development aid: it prints raw diagnostics and does not produce a privacy-checked approval bundle.