From cdebdd9347f133d463a2f86e41810766391c44e1 Mon Sep 17 00:00:00 2001 From: jack <212554440+jackjackbits@users.noreply.github.com> Date: Thu, 30 Jul 2026 01:14:19 +0100 Subject: [PATCH] Link layer slice 4: deterministic multi-node mesh simulation (and the panic-announce bug it caught) (#1548) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Cohere per-link Noise auth and rebind containment into BLELinkAuthState The authenticated-link owners, the reconnect revalidation policy, and the two rebind-containment cooldowns were four loose bleQueue-owned maps whose invariants lived in call-site discipline: every teardown path had to remember to retire the proof AND close the revalidation epoch (the pair appeared seven times), and both cooldowns hand-rolled the same prune-check-record dance. BLELinkAuthState owns them as whole transitions — retireLink, retireLinks(ownedBy:), permitRebind, permitRedundantRetirement — with the ownership question (bleQueue today, engine after the option-B flip) answered in one place. No behavior change; the one call-site reordering (redundant retirement computes the survivor before the cooldown check instead of after) is outcome-equivalent since the cooldown only ever recorded when a survivor existed. Co-Authored-By: Claude Fable 5 * Split identity-link bindings out of the physical link store BLELinkStateStore owned two different kinds of truth: what physical links exist (CB handles, connect lifecycles, characteristics, stream assemblers) and who each link belongs to (peer bindings in both roles plus the preferred-peripheral reverse map for directed sends and fanout collapse). The bindings now live on BLELinkBindings — same bleQueue ownership, whole-transition methods, direct tests for the rotation reverse-map cleanup and the preferred-link survivor repair that were previously only exercised end to end. Composed operations that need both truths (remove-with-repair, direct link state, the subscribed- central snapshot, bind-only-live-links) live on the transport as explicitly bleQueue-confined helpers. This is the structural half of the option-B boundary flip (docs/BLE-ARCHITECTURE-V3.md): ownership of the bindings can now move to the engine without touching what-links-exist. An audit of every physical clear/remove found three sites (emergency clear, both unauthorized branches) that needed explicit binding-clear pairing under the split — each now clears both. Co-Authored-By: Claude Fable 5 * Fix iOS-gated constructors and preserve containment cooldowns on reset CI caught what the macOS SwiftPM build cannot see: two #if os(iOS) sites still passed the peerID field that slice B1 removed from BLEPeripheralLinkState (willRestoreState in BLEService and armPendingBackgroundConnects in BLERadioController). Both fixed and verified with a local iOS simulator xcodebuild. Codex also caught a real regression: BLELinkAuthState.removeAll() cleared the rebind/retirement cooldown maps, which the original panic and emergency reset paths deliberately left alive. A stable CoreBluetooth UUID must not earn a fresh rebind allowance just because the session state around it was wiped. removeAll() now clears only the proofs and revalidation epochs, and BLELinkAuthStateTests pins the survival invariant along with the other auth-state transitions. Co-Authored-By: Claude Fable 5 * Link layer slice 3: the option-B domain flip — bindings and link-auth move to the engine The identity domain (BLELinkBindings + BLELinkAuthState) is now owned by the engine queue, with a DEBUG dispatchPrecondition trapping any access from another queue. bleQueue keeps only physical link state. What changed shape: - Receive path is sans-I/O: bleQueue decodes frames and hands (packet, linkID) up through ingestDecodedPacket (panic lifecycle captured at the handoff); attributeAndHandlePacket resolves the sender binding, rejects spoofed senders, applies raw-announce binding, and records ingress on the engine. Per-link frame order is preserved end to end (both queues serial), which supersedes the old batch-local TOCTOU binding in the notification path. - The rotation rebind is one engine slot: containment checks, proof retirement, binding flip, reconnect decision, and rotated-identity retirement run straight-line; only CoreBluetooth cancels hop to bleQueue. The engine->bleQueue->engine ping-pong is gone, along with the _test_afterVerifiedDirectRebindEnqueued pause hook — the test that used it now asserts the atomicity directly (a paused engine wedged the old gate design into a three-queue deadlock). - Authenticated-send eligibility (notifyOrEnqueueIfAccepted, writeOrEnqueueIfAccepted) is checked on the engine, serialized against rebinds by construction; only physical admission (updateValue/write/backpressure) runs on bleQueue. - Teardown splits into discardPeripheralLinkPhysical (bleQueue, inline in the delegates) + retirePeripheralLinkIdentity (engine hop with survivor repair reading liveness via readLinkState). A binding can briefly outlive its physical link; liveness queries join against the physical store and the queued retirement converges the two. - Gossip delegate sends enter the engine via onEngine — safe because mesh.sync sits above the engine in the sync order (production engine code only async-dispatches into the manager). - checkPeerConnectivity rides an engine slot from the bleQueue maintenance tick. No wire changes. 1,974 tests green (parallel and serial), iOS simulator build clean, Periphery clean. Co-Authored-By: Claude Fable 5 * Link layer slice 4: deterministic multi-node mesh simulation — and the panic-announce bug it caught SimulatedMesh wires real CoreBluetooth-free BLEService engines edge-to-edge through the outbound packet tap and _test_ingestFrame (the production attribution path the B2 flip created), with per-edge synthetic link IDs and manual-scheduler time. Five multi-node tests run in ~40ms with no wall-clock waits: - announce exchange binds simulated links and connects peers - Noise sessions establish end-to-end (real crypto, both directions) - a public message relays across a line topology inside a TTL/frame budget (storm bound asserted) - an 8x duplicate flood delivers exactly once - a panic rotation rebinds the survivor's link exactly once and stays — the scenario that previously needed two phones and log archaeology Fidelity boundary (documented in the harness): no physical links, so fanout planning and backpressure are not exercised; attribution, binding, dedup, TTL, relay decisions, and sessions are the real engine code. The simulator found a real bug on its first run: the forced-announce throttle's lastSent survived a panic, so a rotation within bleForceAnnounceMinIntervalSeconds of the last announce silently swallowed the new identity's announce — leaving it invisible to the mesh until the next maintenance cycle. Today's device test only passed because the previous announce happened to be minutes old. BLEAnnounceThrottle gains reset(), called from the panic slot so the rotated identity owes no throttle debt; pinned by a unit test and the mesh rotation test. New DEBUG seams: _test_ingestFrame (production ingress attribution), _test_forceAnnounce, _test_fenceEngine. 1,980 tests green, Periphery clean, iOS simulator build clean. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: jack Co-authored-by: Claude Fable 5 --- bitchat.xcodeproj/project.pbxproj | 13 +- .../Services/BLE/BLEAnnounceThrottle.swift | 9 + bitchat/Services/BLE/BLEService.swift | 26 +++ .../Services/BLEAnnounceThrottleTests.swift | 15 ++ bitchatTests/Simulation/SimulatedMesh.swift | 129 +++++++++++++ .../Simulation/SimulatedMeshTests.swift | 172 ++++++++++++++++++ docs/BLE-ARCHITECTURE-V3.md | 21 +++ 7 files changed, 377 insertions(+), 8 deletions(-) create mode 100644 bitchatTests/Simulation/SimulatedMesh.swift create mode 100644 bitchatTests/Simulation/SimulatedMeshTests.swift diff --git a/bitchat.xcodeproj/project.pbxproj b/bitchat.xcodeproj/project.pbxproj index e0738afc..d9239d66 100644 --- a/bitchat.xcodeproj/project.pbxproj +++ b/bitchat.xcodeproj/project.pbxproj @@ -94,7 +94,6 @@ isa = PBXFileSystemSynchronizedBuildFileExceptionSet; membershipExceptions = ( Info.plist, - bitchatShareExtension.entitlements, ); target = 57CA17A36A2532A6CFF367BB /* bitchatShareExtension */; }; @@ -379,6 +378,11 @@ E0A1B2C3D4E5F6012345678D /* relays/online_relays_gps.csv in Resources */, ); }; + 7E9B64F63F93443FB7BA12DF /* Resources */ = { + isa = PBXResourcesBuildPhase; + files = ( + ); + }; C5E027A42ECCDFD700BD6012 /* Resources */ = { isa = PBXResourcesBuildPhase; files = ( @@ -395,13 +399,6 @@ E0A1B2C3D4E5F6012345678E /* relays/online_relays_gps.csv in Resources */, ); }; - 7E9B64F63F93443FB7BA12DF /* Resources */ = { - isa = PBXResourcesBuildPhase; - buildActionMask = 2147483647; - files = ( - ); - runOnlyForDeploymentPostprocessing = 0; - }; /* End PBXResourcesBuildPhase section */ /* Begin PBXSourcesBuildPhase section */ diff --git a/bitchat/Services/BLE/BLEAnnounceThrottle.swift b/bitchat/Services/BLE/BLEAnnounceThrottle.swift index d6bf5490..4b064136 100644 --- a/bitchat/Services/BLE/BLEAnnounceThrottle.swift +++ b/bitchat/Services/BLE/BLEAnnounceThrottle.swift @@ -37,4 +37,13 @@ final class BLEAnnounceThrottle: @unchecked Sendable { return true } } + + /// Forgets the last-sent timestamp. A panic rotation calls this so the + /// new identity's first announce cannot be swallowed by the old + /// identity's throttle debt — otherwise a panic within the forced + /// minimum interval of the last announce leaves the rotated identity + /// invisible until the next maintenance cycle. + func reset() { + lock.withLock { lastSent = .distantPast } + } } diff --git a/bitchat/Services/BLE/BLEService.swift b/bitchat/Services/BLE/BLEService.swift index cd366608..75775e51 100644 --- a/bitchat/Services/BLE/BLEService.swift +++ b/bitchat/Services/BLE/BLEService.swift @@ -782,6 +782,11 @@ final class BLEService: NSObject { // rebind/retirement cooldowns deliberately survive (see // BLELinkAuthState.removeAll). linkAuth.removeAll() + // The new identity owes no announce-throttle debt: without this, + // a panic within the forced minimum interval of the last + // announce swallows the rotation announce and the new identity + // stays invisible until the next maintenance cycle. + announceThrottle.reset() // These callbacks belong to pre-panic transfer state. Invoking // them would let queued UI work recreate or resend wiped media. privateMediaSessions.panicReset() @@ -3353,6 +3358,27 @@ extension BLEService { } } + /// Simulated-link ingress: the full production attribution path — + /// binding lookup, spoof rejection, raw-announce binding, ingress + /// recording — for a frame arriving on a synthetic link. The + /// SimulatedMesh harness feeds every node through this, so multi-node + /// tests exercise the same engine code as CoreBluetooth ingress. + func _test_ingestFrame(_ packet: BitchatPacket, link: BLEIngressLinkID) { + ingestDecodedPacket(packet, link: link, linkDescription: "Simulated \(link)") + } + + /// Sends an unthrottled announce, exactly like the maintenance forced + /// path. SimulatedMesh uses this as the deterministic discovery step. + func _test_forceAnnounce() { + onEngine { sendAnnounceNow(forceSend: true) } + } + + /// Blocks until every engine slot enqueued so far has run — the + /// deterministic settling fence for simulated-mesh pumping. + func _test_fenceEngine() { + onEngine {} + } + func _test_emitTransportEvent( _ event: TransportEvent, completion: @escaping () -> Void, diff --git a/bitchatTests/Services/BLEAnnounceThrottleTests.swift b/bitchatTests/Services/BLEAnnounceThrottleTests.swift index 0ce23814..0e135a3b 100644 --- a/bitchatTests/Services/BLEAnnounceThrottleTests.swift +++ b/bitchatTests/Services/BLEAnnounceThrottleTests.swift @@ -68,6 +68,21 @@ struct BLEAnnounceThrottleTests { #expect(accepted.value == 1) #expect(throttle.elapsed(since: now.addingTimeInterval(3)) == 3) } + + @Test + func resetForgetsThrottleDebtSoARotationAnnounceIsNeverSwallowed() { + let throttle = BLEAnnounceThrottle( + normalMinimumInterval: 1, + forcedMinimumInterval: 1 + ) + let now = Date() + #expect(throttle.shouldSend(force: true, now: now)) + // A panic inside the forced window would be throttled... + #expect(!throttle.shouldSend(force: true, now: now.addingTimeInterval(0.2))) + // ...so the rotation resets the debt and announces immediately. + throttle.reset() + #expect(throttle.shouldSend(force: true, now: now.addingTimeInterval(0.3))) + } } private final class LockedCounter: @unchecked Sendable { diff --git a/bitchatTests/Simulation/SimulatedMesh.swift b/bitchatTests/Simulation/SimulatedMesh.swift new file mode 100644 index 00000000..47491d34 --- /dev/null +++ b/bitchatTests/Simulation/SimulatedMesh.swift @@ -0,0 +1,129 @@ +import BitFoundation +import Foundation +@testable import bitchat + +/// A deterministic multi-node mesh over real `BLEService` engines and no +/// CoreBluetooth: nodes are wired edge-to-edge through the outbound packet +/// tap and the production ingress-attribution path (`_test_ingestFrame`), +/// so announces bind links, signatures verify, Noise handshakes complete, +/// and rotation rebinds run exactly the engine code a radio would drive. +/// +/// Determinism model: outbound packets are buffered under a lock (the tap +/// fires on each sender's engine); the test thread pumps deliveries and +/// fences every engine between rounds. Timer-driven work (relay jitter, +/// deferred flushes) is released explicitly through each node's +/// `BLEEngineManualScheduler` via `advanceTime`. +/// +/// Fidelity boundary: there are no physical links, so per-link fanout +/// planning always reports failure to the sender (directed packets spool) +/// — every capture happens at the pre-planning tap. Protocol-level +/// behavior (attribution, binding, dedup, TTL, relay decisions, sessions) +/// is faithful; link-selection and backpressure behavior is not exercised. +final class SimulatedMesh { + struct Node { + let service: BLEService + let scheduler: BLEEngineManualScheduler + } + + private let lock = NSLock() + private var pendingDeliveries: [(from: Int, packet: BitchatPacket)] = [] + /// Total (packet, receiving-node) deliveries pumped — the storm bound. + private(set) var deliveredFrameCount = 0 + + private(set) var nodes: [Node] = [] + private var neighbors: [Set] = [] + + @discardableResult + func addNode(nickname: String) -> Node { + let keychain = MockKeychain() + let identityManager = MockIdentityManager(keychain) + let idBridge = NostrIdentityBridge(keychain: MockKeychainHelper()) + let scheduler = BLEEngineManualScheduler() + let service = BLEService( + keychain: keychain, + idBridge: idBridge, + identityManager: identityManager, + initializeBluetoothManagers: false, + engineScheduler: scheduler + ) + let index = nodes.count + let node = Node(service: service, scheduler: scheduler) + nodes.append(node) + neighbors.append([]) + service.setNickname(nickname) + service._test_onOutboundPacket = { [weak self] packet in + // Runs on the sender's engine; only buffer here — delivering + // inline would nest one engine inside another. + guard let self else { return } + self.lock.lock() + self.pendingDeliveries.append((from: index, packet: packet)) + self.lock.unlock() + } + return node + } + + func connect(_ a: Int, _ b: Int) { + neighbors[a].insert(b) + neighbors[b].insert(a) + } + + /// The synthetic link a frame from `sender` arrives on at `receiver`. + /// Stable per directed edge, like a CoreBluetooth central UUID. + func linkUUID(from sender: Int, at receiver: Int) -> String { + "SIM-\(sender)-TO-\(receiver)" + } + + func forceAnnounce(from index: Int) { + nodes[index].service._test_forceAnnounce() + pump() + } + + /// Pumps buffered deliveries until the mesh is quiescent: no pending + /// frames and every engine drained. Timer-deferred work stays pending + /// until `advanceTime`. + func pump(maxRounds: Int = 64) { + for _ in 0.. Int { + lock.lock() + defer { lock.unlock() } + return publicMessages.filter { $0 == content }.count + } + + func drainedPublicMessageCount(content: String, drains: Int = 50) async -> Int { + for _ in 0.. 0 { break } + await MainActor.run {} + } + return count(content: content) + } +} diff --git a/docs/BLE-ARCHITECTURE-V3.md b/docs/BLE-ARCHITECTURE-V3.md index d72df1bf..5dd2fbd8 100644 --- a/docs/BLE-ARCHITECTURE-V3.md +++ b/docs/BLE-ARCHITECTURE-V3.md @@ -201,6 +201,27 @@ throughput is nowhere near what one serial queue sustains. packet switch) ride this seam as handler-registered modules instead of getting closure-environment extractions now. + **The simulator half is done — simulator-first.** Because the B2 + receive path already hands `(packet, linkID)` up through one choke + point, `SimulatedMesh` (bitchatTests/Simulation/) wires real + CB-free `BLEService` engines edge-to-edge through the outbound tap + and `_test_ingestFrame` (the production attribution path), with + per-edge synthetic link IDs and manual-scheduler time. Five + deterministic multi-node tests run in ~40ms: announce/bind + convergence, end-to-end Noise establishment, line-topology relay + within a TTL/frame budget, duplicate-flood dedup, and the panic + rotation single-slot rebind + containment — the scenario that + previously required two phones. Fidelity boundary: no physical + links, so fanout planning/backpressure is not exercised; protocol + behavior is. On its first day the simulator found a real bug: the + forced-announce throttle survived panic, so a rotation within + `bleForceAnnounceMinIntervalSeconds` of the last announce left the + new identity invisible until the next maintenance cycle + (`BLEAnnounceThrottle.reset()` now runs in the panic slot). + Remaining from the original slice-C scope: the mechanical delegate + extraction behind explicit LinkEvent/LinkCommand types, and the + formal `handle(event) -> [Effect]` engine shape. + ## What this is not No wire changes: packet formats, signing (padding is signed), the