bitchat-android/docs/release-gate-runbook.md

# Physical-device and cross-client release gate

This runbook turns Milestone 10 into a repeatable release procedure. The gate
uses a host-side CLI and USB/ADB as its control channel, so control traffic
never shares BLE, Wi-Fi Aware, Nostr, or Tor with the system under test.

The gate cannot pass without the required physical devices and counterpart
clients. A pending or blocked result is useful diagnostic evidence, but it is
not release approval.

## Safety and privacy rules

- Use only disposable lab app data, identities, nicknames, messages, and files.
- Never use a personal Nostr account or a production relay.
- Do not put device serials, UDIDs, Bluetooth/MAC/IP addresses, peer IDs,
  usernames, email addresses, local home paths, or message contents in a
  result, trace, filename, issue, commit, or release artifact.
- Device selectors may be supplied to ADB commands as ephemeral inputs. The
  tooling emits only logical aliases such as `android-current`.
- Models, manufacturer classes, Android API levels, negotiated MTU classes,
  client versions, commit hashes, aggregate counts, durations, and stable
  failure reason codes are allowed.
- Do not archive raw logcat. Convert observations to the structured,
  privacy-checked trace format, and keep any raw diagnostic capture local until
  it has been reviewed and sanitized.

The validator rejects known identifying fields and values before a passing
bundle can be created.

## Required lab

Prepare:

- At least three physical Android devices for three-hop relay testing.
- At least two Android API levels and two manufacturer classes.
- BLE central and peripheral support on every Android device.
- At least one Android 13+ device with Wi-Fi Aware.
- One physical device running the current iOS client.
- The last supported Android client.
- The release-candidate APK built from one exact full Git commit.
- A local, disposable Nostr relay/Tor fixture with production network access
  blocked.

One physical handset may be reused for the legacy-client phase after the
current-client evidence for that slot is complete, but the matrix must keep the
logical aliases and installed client versions unambiguous.

## 1. Verify the deterministic gate

From the repository root:

```sh
./gradlew clientRewriteContractTest checkChangedLineCoverage lintDebug
python3 tools/release_gate/release_gate.py validate-manifest
```

Do not begin device work from a dirty tree or a build whose deterministic gate
does not pass.

## 2. Create the device matrix

Copy `tools/release_gate/device-matrix.example.json` to an ignored working
directory under `release-gate-results/`. Replace every template value and set
both current-Android commit fields to the exact full commit under test.

Probe Android capabilities without storing the ADB selector:

```sh
python3 tools/release_gate/android_lab.py probe \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --alias android-current
```

Copy only the returned logical metadata into the matrix. Validate it:

```sh
python3 tools/release_gate/release_gate.py validate-matrix \
  --matrix release-gate-results/device-matrix.json \
  --commit "$BITCHAT_RELEASE_COMMIT"
```

The matrix validator enforces physical devices, three Android participants, two
API levels, two manufacturer classes, Wi-Fi Aware, BLE roles, iOS, and explicit
current/legacy client versions.

## 3. Initialize disposable fixtures

```sh
python3 tools/release_gate/release_gate.py init \
  --matrix release-gate-results/device-matrix.json \
  --commit "$BITCHAT_RELEASE_COMMIT" \
  --run-id rc-lab-01 \
  --output release-gate-results/rc-lab-01
```

Initialization pins the scenario and fixture manifests, creates every scenario
as `pending`, and generates deterministic:

- zero-byte and small files;
- a Unicode-named medium file;
- sparse exact-maximum and oversized boundary files.

The fixture manifest records size and SHA-256. The final archive contains the
manifest, not the large fixture bodies.

Clear only the disposable app data on each selected lab device:

```sh
python3 tools/release_gate/android_lab.py prepare \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --confirm-disposable-app-data
```

This stops the app and runs package-data cleanup. The explicit confirmation is
required because the operation is destructive to that app's local data.

## 4. Execute scenarios

The canonical scenario list is
`tools/release_gate/scenarios.json`. It contains 27 mandatory scenarios:

- the complete physical transport matrix;
- Android API/manufacturer/permission/background coverage;
- 11 Android-to-Android workflows;
- 8 cross-client/backward-compatibility workflows;
- 6 background and endurance workflows.

For each scenario:

1. Confirm the listed participants and capabilities.
2. Perform the corresponding steps in
   [device-transport-test-matrix.md](device-transport-test-matrix.md) and the
   Milestone 10 checklist.
3. Record connection, lifecycle, transport, receipt, resource, and terminal
   state as aggregate evidence.
4. Append at least one structured trace event.
5. Mark the scenario `pass`, `fail`, `blocked`, or `unsupported`.

Record evidence with the exact keys declared by the scenario:

```sh
python3 tools/release_gate/release_gate.py record \
  --run release-gate-results/rc-lab-01 \
  --scenario A2A-001 \
  --status pass \
  --evidence connection-transitions=4 \
  --evidence packet-correlation-count=6 \
  --evidence failure-reasons=none
```

Append a privacy-safe trace event:

```sh
python3 tools/release_gate/release_gate.py trace \
  --run release-gate-results/rc-lab-01 \
  --scenario A2A-001 \
  --source android-current \
  --event reconnect-terminal \
  --outcome pass \
  --metric reconnect-count=1 \
  --metric duplicate-delivery-count=0
```

Capture resource snapshots during endurance work:

```sh
python3 tools/release_gate/android_lab.py snapshot \
  --serial "$BITCHAT_ADB_SELECTOR" \
  --alias android-current \
  --run release-gate-results/rc-lab-01 \
  --scenario END-003
```

Use run-local sequential correlation labels while observing packets; archive
only aggregate correlation counts. Record failures with a stable reason code,
file a regression issue, and preserve the incomplete artifact.

## 5. Endurance requirements

- `END-001` requires at least 240 minutes.
- `END-002` requires at least 50 large-transfer/cancellation cycles.
- Sample memory, threads, file descriptors, wake locks, connection counts, and
  late callbacks at consistent intervals.
- A passing result requires bounded resource behavior and a clean terminal
  state; merely completing the time window is insufficient.

The validator rejects shorter durations and cycle counts.

## 6. Inspect progress and validate

During a run:

```sh
python3 tools/release_gate/release_gate.py validate \
  --run release-gate-results/rc-lab-01 \
  --allow-incomplete

python3 tools/release_gate/release_gate.py summary \
  --run release-gate-results/rc-lab-01
```

The release validator, without `--allow-incomplete`, requires:

- every scenario to be `pass`;
- every declared evidence field;
- at least one structured trace per scenario;
- the pinned scenario and fixture manifests;
- the exact client commit and complete device matrix;
- endurance minimums;
- a completion timestamp;
- no detected identifying fields or values.

`unsupported`, `blocked`, and `pending` never satisfy release approval.

## 7. Archive release approval

After the complete validator passes:

```sh
python3 tools/release_gate/release_gate.py bundle \
  --run release-gate-results/rc-lab-01 \
  --output release-gate-results/rc-lab-01.zip
```

The deterministic archive contains the scenario manifest, device/client matrix,
results, structured trace, fixture manifest, Markdown summary, and
`SHA256SUMS`. Attach it to the release approval record without renaming fields
or adding raw diagnostics.

Finally, clean the disposable app data with the same confirmed `cleanup`
command and stop the local relay/Tor fixture.

## Failure handling

- `fail`: behavior violated a contract. Record a stable reason code, file a bug,
  add a deterministic regression where possible, fix it, and rerun the affected
  scenario plus dependent scenarios.
- `blocked`: required lab infrastructure or counterpart client was unavailable.
  Preserve the artifact and do not approve release.
- `unsupported`: the selected device lacks a capability. Because the defined
  matrix requires Wi-Fi Aware, replace the device or matrix; unsupported does
  not waive a mandatory scenario.
- A flaky result is a failure until its cause is understood. Never average
  retries into a pass.