bitchat/bitchatTests/Simulation/SimulatedMesh.swift
jack e2b409e466
Fix #1538: release stale bindings on rotation instead of leaving a ghost identity (#1554)
* Cohere per-link Noise auth and rebind containment into BLELinkAuthState

The authenticated-link owners, the reconnect revalidation policy, and
the two rebind-containment cooldowns were four loose bleQueue-owned
maps whose invariants lived in call-site discipline: every teardown
path had to remember to retire the proof AND close the revalidation
epoch (the pair appeared seven times), and both cooldowns hand-rolled
the same prune-check-record dance. BLELinkAuthState owns them as whole
transitions — retireLink, retireLinks(ownedBy:), permitRebind,
permitRedundantRetirement — with the ownership question (bleQueue
today, engine after the option-B flip) answered in one place.

No behavior change; the one call-site reordering (redundant retirement
computes the survivor before the cooldown check instead of after) is
outcome-equivalent since the cooldown only ever recorded when a
survivor existed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Split identity-link bindings out of the physical link store

BLELinkStateStore owned two different kinds of truth: what physical
links exist (CB handles, connect lifecycles, characteristics, stream
assemblers) and who each link belongs to (peer bindings in both roles
plus the preferred-peripheral reverse map for directed sends and fanout
collapse). The bindings now live on BLELinkBindings — same bleQueue
ownership, whole-transition methods, direct tests for the rotation
reverse-map cleanup and the preferred-link survivor repair that were
previously only exercised end to end. Composed operations that need
both truths (remove-with-repair, direct link state, the subscribed-
central snapshot, bind-only-live-links) live on the transport as
explicitly bleQueue-confined helpers.

This is the structural half of the option-B boundary flip
(docs/BLE-ARCHITECTURE-V3.md): ownership of the bindings can now move
to the engine without touching what-links-exist. An audit of every
physical clear/remove found three sites (emergency clear, both
unauthorized branches) that needed explicit binding-clear pairing under
the split — each now clears both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix iOS-gated constructors and preserve containment cooldowns on reset

CI caught what the macOS SwiftPM build cannot see: two #if os(iOS)
sites still passed the peerID field that slice B1 removed from
BLEPeripheralLinkState (willRestoreState in BLEService and
armPendingBackgroundConnects in BLERadioController). Both fixed and
verified with a local iOS simulator xcodebuild.

Codex also caught a real regression: BLELinkAuthState.removeAll()
cleared the rebind/retirement cooldown maps, which the original panic
and emergency reset paths deliberately left alive. A stable
CoreBluetooth UUID must not earn a fresh rebind allowance just because
the session state around it was wiped. removeAll() now clears only the
proofs and revalidation epochs, and BLELinkAuthStateTests pins the
survival invariant along with the other auth-state transitions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Link layer slice 3: the option-B domain flip — bindings and link-auth move to the engine

The identity domain (BLELinkBindings + BLELinkAuthState) is now owned
by the engine queue, with a DEBUG dispatchPrecondition trapping any
access from another queue. bleQueue keeps only physical link state.

What changed shape:

- Receive path is sans-I/O: bleQueue decodes frames and hands
  (packet, linkID) up through ingestDecodedPacket (panic lifecycle
  captured at the handoff); attributeAndHandlePacket resolves the
  sender binding, rejects spoofed senders, applies raw-announce
  binding, and records ingress on the engine. Per-link frame order is
  preserved end to end (both queues serial), which supersedes the old
  batch-local TOCTOU binding in the notification path.
- The rotation rebind is one engine slot: containment checks, proof
  retirement, binding flip, reconnect decision, and rotated-identity
  retirement run straight-line; only CoreBluetooth cancels hop to
  bleQueue. The engine->bleQueue->engine ping-pong is gone, along with
  the _test_afterVerifiedDirectRebindEnqueued pause hook — the test
  that used it now asserts the atomicity directly (a paused engine
  wedged the old gate design into a three-queue deadlock).
- Authenticated-send eligibility (notifyOrEnqueueIfAccepted,
  writeOrEnqueueIfAccepted) is checked on the engine, serialized
  against rebinds by construction; only physical admission
  (updateValue/write/backpressure) runs on bleQueue.
- Teardown splits into discardPeripheralLinkPhysical (bleQueue, inline
  in the delegates) + retirePeripheralLinkIdentity (engine hop with
  survivor repair reading liveness via readLinkState). A binding can
  briefly outlive its physical link; liveness queries join against the
  physical store and the queued retirement converges the two.
- Gossip delegate sends enter the engine via onEngine — safe because
  mesh.sync sits above the engine in the sync order (production engine
  code only async-dispatches into the manager).
- checkPeerConnectivity rides an engine slot from the bleQueue
  maintenance tick.

No wire changes. 1,974 tests green (parallel and serial), iOS
simulator build clean, Periphery clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Link layer slice 4: deterministic multi-node mesh simulation — and the panic-announce bug it caught

SimulatedMesh wires real CoreBluetooth-free BLEService engines
edge-to-edge through the outbound packet tap and _test_ingestFrame
(the production attribution path the B2 flip created), with per-edge
synthetic link IDs and manual-scheduler time. Five multi-node tests
run in ~40ms with no wall-clock waits:

- announce exchange binds simulated links and connects peers
- Noise sessions establish end-to-end (real crypto, both directions)
- a public message relays across a line topology inside a TTL/frame
  budget (storm bound asserted)
- an 8x duplicate flood delivers exactly once
- a panic rotation rebinds the survivor's link exactly once and stays
  — the scenario that previously needed two phones and log archaeology

Fidelity boundary (documented in the harness): no physical links, so
fanout planning and backpressure are not exercised; attribution,
binding, dedup, TTL, relay decisions, and sessions are the real
engine code.

The simulator found a real bug on its first run: the forced-announce
throttle's lastSent survived a panic, so a rotation within
bleForceAnnounceMinIntervalSeconds of the last announce silently
swallowed the new identity's announce — leaving it invisible to the
mesh until the next maintenance cycle. Today's device test only
passed because the previous announce happened to be minutes old.
BLEAnnounceThrottle gains reset(), called from the panic slot so the
rotated identity owes no throttle debt; pinned by a unit test and the
mesh rotation test.

New DEBUG seams: _test_ingestFrame (production ingress attribution),
_test_forceAnnounce, _test_fenceEngine.

1,980 tests green, Periphery clean, iOS simulator build clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Link layer slice 5: name the port — BLELinkEvent, one engine entry, delegates in their own files

The upward half of the link-layer port is now a type. BLELinkEvent
enumerates everything the bleQueue link layer tells the engine:
frameDecoded plus the four physical lifecycle transitions
(peripheralLinkEnded, centralLinkEnded, allPeripheralLinksEnded,
allCentralLinksEnded). Every bleQueue→engine crossing goes through
emitLinkEvent into one engine consumer (handleLinkEvent) — the
scattered messageQueue.async identity hops in the delegates collapse
into event emission, and the engine-side retirement/bookkeeping logic
now lives in one switch.

The CoreBluetooth delegate extensions move to their own files as
physical bookkeeping plus event emission:
- BLEService+LinkLayerCentralRole.swift (CBCentralManagerDelegate +
  CBPeripheralDelegate)
- BLEService+LinkLayerPeripheralRole.swift (CBPeripheralManagerDelegate
  + write accumulation)
BLEService.swift drops from 7,836 to ~7,100 lines. The physical-domain
members the role files share flip private→internal; the queue contract
is enforced by the existing DEBUG traps and grep guards, not access
control. (Two of the flips — isAppActive, logBluetoothStatus — only
surfaced on the iOS build; macOS SwiftPM cannot see #if os(iOS) code.
Verified with a local iOS simulator build.)

The simulated mesh now drives lifecycle events through the identical
enum a radio does: linkDropEventRetiresBindingAndReconnectHeals covers
drop → identity retirement → last-link peer bookkeeping → re-announce
heal, entirely through the port. New seam _test_resetAnnounceThrottle
models elapsed wall-clock for the throttle (deliberately separate from
_test_forceAnnounce so the panic-rotation test keeps its regression
value: the production panic path must do its own reset). The panic
test's containment re-announces reset throttles explicitly so those
assertions exercise real delivered announces instead of silently
throttled ones. noiseSessionEstablishesEndToEnd gains a bounded
scheduler-time settle loop after a one-in-many parallel-suite flake
(no wall-clock waits).

Deliberately not done (recorded in docs/BLE-ARCHITECTURE-V3.md): a
formal handle(event)->[Effect] system and further engine-domain file
splits — both would flip the engine's private state to internal for
cosmetic file counts; the effect formalization rides future feature-
module extractions instead.

1,981 tests green, Periphery clean, iOS simulator build clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Baseline logBluetoothStatus for the macOS Periphery scan

Its callers are all inside #if os(iOS) (willRestoreState in both role
files plus the app-state handlers), so the macOS-scheme scan sees the
now-internal declaration with zero callers — the same class as the
baselined candidateCount. Verified 1-USR diff; the previously private
mangled variant was already baselined, which is why the pre-split scan
never flagged it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix #1538: release stale bindings on rotation instead of leaving a ghost

With two live links to one phone, a panic rotation healed only the link
the verified announce arrived on. The second link kept its binding to
the retired identity, so that dead ID stayed in the peer list — and was
kept alive by the NEW identity's own traffic, since a bound link
attributes non-announce frames to its bound peer. It only healed when
the stale link physically dropped.

The issue proposed exempting the containment rule via retiredBy[X] = Y
so the second link could rebind. Two problems: the exemption's stated
precondition (X removed by retireRotatedPeer) can never hold in this
scenario — the retire is gated on X having no remaining links, which is
false precisely because the stale link exists — and it would loosen a
security rule to fix a liveness bug.

Instead the rotation now RELEASES every link still bound to the
rotated-away identity (unbind + retire that link's Noise proof) and
retires the identity. No containment rule changes: unbinding is
strictly less trusting than any binding, and it is correct under both
readings of a second link bound to the retired ID — same physical
device (the field case), or one link is a spoofer holding a forged
binding, since a peer ID is a Noise-key fingerprint and two devices
cannot both legitimately own it. Released links reconverge through the
ordinary unbound-link path: the next raw direct announce binds them to
whoever they actually carry.

Reproduced and fixed under the slice-4 simulator, which is why this
lands as tests rather than another two-phone session:
- duplicateLinkPanicRotationLeavesNoGhostAndHealsBothLinks fails
  without the fix (ghost in both knownPeers and getConnectedPeers,
  duplicate link still bound to the dead ID)
- replayedVerifiedAnnounceCannotStealALinkOrEvictTheVictim pins the
  #1401 containment rule against exactly the attack this fix had to
  avoid re-opening, with a positive control proving the refusal is the
  containment check and not duplicate suppression

Harness gains connectDuplicateLinks (two links to one peer, modelled in
the central role — the links we cannot cancel, and the only role whose
bindings a CB-free harness can form), silence (range loss without a
link event, so a packet can be captured that the far side never saw),
and emittedPackets (the attacker's capture buffer).

Residual, documented at the fix: an attacker who binds their own link
to X by replaying X's raw announce can drive a rebind there and so
evict X's registry entry; X's next announce restores it, and the
per-link rebind cooldown bounds the rate. This is the same class of
capability the containment already accepts, not a new one.

1,983 tests green, Periphery clean, iOS simulator build clean.

Closes #1538

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 15:21:59 +01:00

199 lines
7.7 KiB
Swift

import BitFoundation
import Foundation
@testable import bitchat
/// A deterministic multi-node mesh over real `BLEService` engines and no
/// CoreBluetooth: nodes are wired edge-to-edge through the outbound packet
/// tap and the production ingress-attribution path (`_test_ingestFrame`),
/// so announces bind links, signatures verify, Noise handshakes complete,
/// and rotation rebinds run exactly the engine code a radio would drive.
///
/// Determinism model: outbound packets are buffered under a lock (the tap
/// fires on each sender's engine); the test thread pumps deliveries and
/// fences every engine between rounds. Timer-driven work (relay jitter,
/// deferred flushes) is released explicitly through each node's
/// `BLEEngineManualScheduler` via `advanceTime`.
///
/// Fidelity boundary: there are no physical links, so per-link fanout
/// planning always reports failure to the sender (directed packets spool)
/// every capture happens at the pre-planning tap. Protocol-level
/// behavior (attribution, binding, dedup, TTL, relay decisions, sessions)
/// is faithful; link-selection and backpressure behavior is not exercised.
final class SimulatedMesh {
struct Node {
let service: BLEService
let scheduler: BLEEngineManualScheduler
}
private let lock = NSLock()
private var pendingDeliveries: [(from: Int, packet: BitchatPacket)] = []
/// Total (packet, receiving-node) deliveries pumped the storm bound.
private(set) var deliveredFrameCount = 0
private(set) var nodes: [Node] = []
private var neighbors: [Set<Int>] = []
private var duplicateLinkEdges: Set<String> = []
private var emitted: [[BitchatPacket]] = []
/// Every packet a node has put on the wire the attacker's capture
/// buffer for replay tests.
func emittedPackets(from index: Int) -> [BitchatPacket] {
lock.lock()
defer { lock.unlock() }
return emitted[index]
}
@discardableResult
func addNode(nickname: String) -> Node {
let keychain = MockKeychain()
let identityManager = MockIdentityManager(keychain)
let idBridge = NostrIdentityBridge(keychain: MockKeychainHelper())
let scheduler = BLEEngineManualScheduler()
let service = BLEService(
keychain: keychain,
idBridge: idBridge,
identityManager: identityManager,
initializeBluetoothManagers: false,
engineScheduler: scheduler
)
let index = nodes.count
let node = Node(service: service, scheduler: scheduler)
nodes.append(node)
neighbors.append([])
emitted.append([])
service.setNickname(nickname)
service._test_onOutboundPacket = { [weak self] packet in
// Runs on the sender's engine; only buffer here delivering
// inline would nest one engine inside another.
guard let self else { return }
self.lock.lock()
self.pendingDeliveries.append((from: index, packet: packet))
self.emitted[index].append(packet)
self.lock.unlock()
}
return node
}
func connect(_ a: Int, _ b: Int) {
neighbors[a].insert(b)
neighbors[b].insert(a)
}
/// Radio silence: stops delivering between two nodes without reporting
/// any link event, so existing bindings persist exactly as they do when
/// a peer walks out of range before its link times out. Lets a test
/// capture a packet the far side never received.
func silence(_ a: Int, _ b: Int) {
neighbors[a].remove(b)
neighbors[b].remove(a)
}
/// Models two live links to the same phone (issue #1538): every frame
/// from the neighbour arrives twice, on two link IDs that both bind to
/// the sender.
///
/// Both are central links the remote's connections to our peripheral
/// role. That is deliberate and faithful to the defect: central links
/// are the ones we cannot cancel (they belong to the remote), so they
/// are exactly the links the peripheral-cancel path cannot reach after
/// a rotation. Peripheral-role bindings additionally require physical
/// link state keyed by a real CBPeripheral, which no CB-free harness
/// can fabricate.
func connectDuplicateLinks(_ a: Int, _ b: Int) {
connect(a, b)
duplicateLinkEdges.insert(Self.edgeKey(a, b))
}
/// The synthetic central link a frame from `sender` arrives on at
/// `receiver`. Stable per directed edge, like a CoreBluetooth central
/// UUID.
func linkUUID(from sender: Int, at receiver: Int) -> String {
"SIM-\(sender)-TO-\(receiver)"
}
/// Order-independent edge key.
private static func edgeKey(_ a: Int, _ b: Int) -> String {
"\(min(a, b))-\(max(a, b))"
}
/// The second link of a duplicate-link edge.
func duplicateLinkUUID(from sender: Int, at receiver: Int) -> String {
"SIM-DUP-\(sender)-TO-\(receiver)"
}
private func links(from sender: Int, at receiver: Int) -> [BLEIngressLinkID] {
var links: [BLEIngressLinkID] = [.central(linkUUID(from: sender, at: receiver))]
if duplicateLinkEdges.contains(Self.edgeKey(sender, receiver)) {
links.append(.central(duplicateLinkUUID(from: sender, at: receiver)))
}
return links
}
func forceAnnounce(from index: Int) {
nodes[index].service._test_forceAnnounce()
pump()
}
/// Pumps buffered deliveries until the mesh is quiescent: no pending
/// frames and every engine drained. Timer-deferred work stays pending
/// until `advanceTime`.
func pump(maxRounds: Int = 64) {
for _ in 0..<maxRounds {
lock.lock()
let batch = pendingDeliveries
pendingDeliveries.removeAll()
lock.unlock()
if batch.isEmpty {
// Engines may still be running slots that will emit more.
nodes.forEach { $0.service._test_fenceEngine() }
lock.lock()
let stillEmpty = pendingDeliveries.isEmpty
lock.unlock()
if stillEmpty { return }
continue
}
for (from, packet) in batch {
for receiver in neighbors[from] {
for link in links(from: from, at: receiver) {
deliveredFrameCount += 1
nodes[receiver].service._test_ingestFrame(packet, link: link)
}
}
}
nodes.forEach { $0.service._test_fenceEngine() }
}
fatalError("SimulatedMesh.pump did not quiesce in \(maxRounds) rounds — relay storm?")
}
/// Advances every node's engine clock (releasing relay jitter, retries,
/// deferred flushes) and pumps the resulting traffic.
func advanceTime(by interval: TimeInterval) {
nodes.forEach { $0.scheduler.advance(by: interval) }
pump()
}
/// Full discovery round: every node announces, traffic settles.
func announceAll() {
for node in nodes {
node.service._test_forceAnnounce()
}
pump()
}
/// Advances scheduler time one second per round until `condition`
/// holds (or the round budget runs out the caller's assertion then
/// reports the real failure). Protocol exchanges normally settle in
/// one or two rounds; under a heavily loaded parallel suite, engine
/// slots can interleave with wall-clock-windowed crypto decisions and
/// need a retry cycle or two more. Deterministic: rounds are scheduler
/// time, never sleeps.
func settleUntil(maxRounds: Int = 20, _ condition: () -> Bool) {
for _ in 0..<maxRounds {
if condition() { return }
advanceTime(by: 1)
}
}
}