bitchat-android/docs/release-gate-runbook.md

Physical-device and cross-client release gate

This runbook turns Milestone 10 into a repeatable release procedure. The gate uses a host-side CLI and USB/ADB as its control channel, so control traffic never shares BLE, Wi-Fi Aware, Nostr, or Tor with the system under test.

The gate cannot pass without the required physical devices and counterpart clients. A pending or blocked result is useful diagnostic evidence, but it is not release approval.

Safety and privacy rules

• Use only disposable lab app data, identities, nicknames, messages, and files.
• Never use a personal Nostr account or a production relay.
• Do not put device serials, UDIDs, Bluetooth/MAC/IP addresses, peer IDs, usernames, email addresses, local home paths, or message contents in a result, trace, filename, issue, commit, or release artifact.
• Device selectors may be supplied to ADB commands as ephemeral inputs. The tooling emits only logical aliases such as android-current.
• Models, manufacturer classes, Android API levels, negotiated MTU classes, client versions, commit hashes, aggregate counts, durations, and stable failure reason codes are allowed.
• Do not archive raw logcat. Convert observations to the structured, privacy-checked trace format, and keep any raw diagnostic capture local until it has been reviewed and sanitized.

The validator rejects known identifying fields and values before a passing bundle can be created.

Required lab

Prepare:

• At least three physical Android devices for three-hop relay testing.
• At least two Android API levels and two manufacturer classes.
• BLE central and peripheral support on every Android device.
• At least one Android 13+ device with Wi-Fi Aware.
• One physical device running the current iOS client.
• The last supported Android client.
• The release-candidate APK built from one exact full Git commit.
• A local, disposable Nostr relay/Tor fixture with production network access blocked.

One physical handset may be reused for the legacy-client phase after the current-client evidence for that slot is complete, but the matrix must keep the logical aliases and installed client versions unambiguous.

1. Verify the deterministic gate

From the repository root:

./gradlew clientRewriteContractTest checkChangedLineCoverage lintDebug
python3 tools/release_gate/release_gate.py validate-manifest

Do not begin device work from a dirty tree or a build whose deterministic gate does not pass.

2. Create the device matrix

Copy tools/release_gate/device-matrix.example.json to an ignored working directory under release-gate-results/. Replace every template value and set both current-Android commit fields to the exact full commit under test.

Probe Android capabilities without storing the ADB selector:

python3 tools/release_gate/android_lab.py probe \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --alias android-current

Copy only the returned logical metadata into the matrix. Validate it:

python3 tools/release_gate/release_gate.py validate-matrix \
  --matrix release-gate-results/device-matrix.json \
  --commit "$BITCHAT_RELEASE_COMMIT"

The matrix validator enforces physical devices, three Android participants, two API levels, two manufacturer classes, Wi-Fi Aware, BLE roles, iOS, and explicit current/legacy client versions.

3. Initialize disposable fixtures

python3 tools/release_gate/release_gate.py init \
  --matrix release-gate-results/device-matrix.json \
  --commit "$BITCHAT_RELEASE_COMMIT" \
  --run-id rc-lab-01 \
  --output release-gate-results/rc-lab-01

Initialization pins the scenario and fixture manifests, creates every scenario as pending, and generates deterministic:

• zero-byte and small files;
• a Unicode-named medium file;
• sparse exact-maximum and oversized boundary files.

The fixture manifest records size and SHA-256. The final archive contains the manifest, not the large fixture bodies.

Clear only the disposable app data on each selected lab device:

python3 tools/release_gate/android_lab.py prepare \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --confirm-disposable-app-data

This stops the app and runs package-data cleanup. The explicit confirmation is required because the operation is destructive to that app's local data.

4. Execute scenarios

The canonical scenario list is tools/release_gate/scenarios.json. It contains 27 mandatory scenarios:

• the complete physical transport matrix;
• Android API/manufacturer/permission/background coverage;
• 11 Android-to-Android workflows;
• 8 cross-client/backward-compatibility workflows;
• 6 background and endurance workflows.

For each scenario:

1. Confirm the listed participants and capabilities.
2. Perform the corresponding steps in device-transport-test-matrix.md and the Milestone 10 checklist.
3. Record connection, lifecycle, transport, receipt, resource, and terminal state as aggregate evidence.
4. Append at least one structured trace event.
5. Mark the scenario pass, fail, blocked, or unsupported.

Record evidence with the exact keys declared by the scenario:

python3 tools/release_gate/release_gate.py record \
  --run release-gate-results/rc-lab-01 \
  --scenario A2A-001 \
  --status pass \
  --evidence connection-transitions=4 \
  --evidence packet-correlation-count=6 \
  --evidence failure-reasons=none

Append a privacy-safe trace event:

python3 tools/release_gate/release_gate.py trace \
  --run release-gate-results/rc-lab-01 \
  --scenario A2A-001 \
  --source android-current \
  --event reconnect-terminal \
  --outcome pass \
  --metric reconnect-count=1 \
  --metric duplicate-delivery-count=0

Capture resource snapshots during endurance work:

python3 tools/release_gate/android_lab.py snapshot \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --alias android-current \
  --run release-gate-results/rc-lab-01 \
  --scenario END-003

Use run-local sequential correlation labels while observing packets; archive only aggregate correlation counts. Record failures with a stable reason code, file a regression issue, and preserve the incomplete artifact.

5. Endurance requirements

• END-001 requires at least 240 minutes.
• END-002 requires at least 50 large-transfer/cancellation cycles.
• Sample memory, threads, file descriptors, wake locks, connection counts, and late callbacks at consistent intervals.
• A passing result requires bounded resource behavior and a clean terminal state; merely completing the time window is insufficient.

The validator rejects shorter durations and cycle counts.

6. Inspect progress and validate

During a run:

python3 tools/release_gate/release_gate.py validate \
  --run release-gate-results/rc-lab-01 \
  --allow-incomplete

python3 tools/release_gate/release_gate.py summary \
  --run release-gate-results/rc-lab-01

The release validator, without --allow-incomplete, requires:

• every scenario to be pass;
• every declared evidence field;
• at least one structured trace per scenario;
• the pinned scenario and fixture manifests;
• the exact client commit and complete device matrix;
• endurance minimums;
• a completion timestamp;
• no detected identifying fields or values.

unsupported, blocked, and pending never satisfy release approval.

7. Archive release approval

After the complete validator passes:

python3 tools/release_gate/release_gate.py bundle \
  --run release-gate-results/rc-lab-01 \
  --output release-gate-results/rc-lab-01.zip

The deterministic archive contains the scenario manifest, device/client matrix, results, structured trace, fixture manifest, Markdown summary, and SHA256SUMS. Attach it to the release approval record without renaming fields or adding raw diagnostics.

Finally, clean the disposable app data with the same confirmed cleanup command and stop the local relay/Tor fixture.

Failure handling

• fail: behavior violated a contract. Record a stable reason code, file a bug, add a deterministic regression where possible, fix it, and rerun the affected scenario plus dependent scenarios.
• blocked: required lab infrastructure or counterpart client was unavailable. Preserve the artifact and do not approve release.
• unsupported: the selected device lacks a capability. Because the defined matrix requires Wi-Fi Aware, replace the device or matrix; unsupported does not waive a mandatory scenario.
• A flaky result is a failure until its cause is understood. Never average retries into a pass.