Merge 54b78b9ca756f78c3b252e0705d8b6470d23e84d into 1f59e814f90c3f489f48d68262cb1bf640bf6181

2026-08-15 07:06:11 +00:00 · 2026-08-02 12:00:52 +02:00 · 2026-08-02 12:00:52 +02:00 · 7ff7c1bcd7
commit 7ff7c1bcd7
parent 1f59e814f9 54b78b9ca7
9 changed files with 1128 additions and 0 deletions
--- a/README.md
+++ b/README.md
@ -102,6 +102,8 @@ Private messages use **intelligent transport selection**:
   - Automatic delivery when connection established

 For detailed protocol documentation, see the [Technical Whitepaper](WHITEPAPER.md).
+For the byte-exact interoperability contract (wire format, BLE UUIDs, Noise
+mappings), see the versioned [`spec/`](spec/) directory.

 ## Setup

--- a/spec/01-wire-format.md
+++ b/spec/01-wire-format.md
@ -0,0 +1,242 @@
+# 01 — Wire Format
+
+**Spec:** 1.0.1  
+**Canonical source:** `localPackages/BitFoundation/Sources/BitFoundation/BinaryProtocol.swift`
+
+All multi-byte integers on the mesh wire are **network byte order (big-endian)**
+unless a payload chapter explicitly says otherwise (capability bitfields are
+little-endian).
+
+---
+
+## 1. Packet overview
+
+Every mesh PDU is a `BitchatPacket` encoded by `BinaryProtocol`:
+
+```
+----------+------+-----+-----------+-------+---------------+
+| Version  | Type | TTL | Timestamp | Flags | PayloadLength |
+| 1 byte   | 1 B  | 1 B | 8 bytes   | 1 B   | 2 or 4 bytes  |
+----------+------+-----+-----------+-------+---------------+
+| SenderID (8) | [RecipientID (8)] | [Route] | Payload… | [Signature (64)] |
+```
+
+| Version | Fixed header size | `PayloadLength` width |
+|---------|-------------------|------------------------|
+| `0x01`  | 14 bytes          | `uint16` BE            |
+| `0x02`  | 16 bytes          | `uint32` BE            |
+
+Other version bytes **MUST** be rejected.
+
+Minimum valid frame: header + 8-byte sender ID (v1 → 22 bytes before optional
+fields).
+
+---
+
+## 2. Fixed header byte offsets
+
+### Version 1 (14-byte header)
+
+| Offset | Size | Field |
+|--------|------|-------|
+| 0 | 1 | `version` = `0x01` |
+| 1 | 1 | `type` (`MessageType`) |
+| 2 | 1 | `ttl` |
+| 3–10 | 8 | `timestamp` — milliseconds since Unix epoch, `uint64` BE |
+| 11 | 1 | `flags` |
+| 12–13 | 2 | `payloadLength` — `uint16` BE |
+
+### Version 2 (16-byte header)
+
+Same as v1 through the flags byte, then:
+
+| Offset | Size | Field |
+|--------|------|-------|
+| 12–15 | 4 | `payloadLength` — `uint32` BE |
+
+`Flags` always sit at absolute offset **11**
+(`BinaryProtocol.Offsets.flags`).
+
+---
+
+## 3. Flags
+
+| Mask | Name | Meaning |
+|------|------|---------|
+| `0x01` | `hasRecipient` | 8-byte `recipientID` follows `senderID` |
+| `0x02` | `hasSignature` | 64-byte Ed25519 signature trails the payload |
+| `0x04` | `isCompressed` | Payload section is zlib + original-size preamble |
+| `0x08` | `hasRoute` | Source route present (**v2 only**; ignored on v1) |
+| `0x10` | `isRSR` | Reserved/source-routing related marker; **not** covered by packet signature |
+| `0x20`–`0x80` | — | Reserved; leave clear on send; ignore on receive |
+
+---
+
+## 4. Variable sections (in order)
+
+After the fixed header:
+
+1. **`senderID`** — exactly 8 bytes (zero-padded on the right if shorter at encode time).
+2. **`recipientID`** — 8 bytes if `hasRecipient`; omitted otherwise.  
+   Broadcast directed-fragment convention: eight `0xFF` bytes may appear as
+   recipient; receivers treat nil **or** all-`0xFF` as broadcast for fragment
+   assembly.
+3. **`route`** (v2 + `hasRoute` only) — **not** counted inside `payloadLength`:
+   - `uint8` hop count `N` (`1…255`)
+   - `N × 8` bytes of hop peer IDs  
+   Empty hop IDs are illegal at encode time; hops longer than 8 bytes are
+   truncated, shorter hops zero-padded to 8.
+4. **Payload section** — exactly `payloadLength` bytes (see compression).
+5. **`signature`** — 64 bytes if `hasSignature`.
+
+### 4.1 `payloadLength` semantics
+
+`payloadLength` covers **only** the payload section:
+
+- uncompressed: raw payload bytes
+- compressed: `originalSize` field (`uint16`/`uint32` BE matching version) **plus** zlib ciphertext
+
+Route bytes are **excluded**.
+
+Decoders **MUST** reject `payloadLength` values larger than
+`FileTransferLimits.maxFramedFileBytes` (~1 MiB plus TLV/framing headroom).
+
+---
+
+## 5. Compression
+
+Applied automatically at encode when beneficial
+(`CompressionUtil` / `Constants.compressionThresholdBytes = 100`):
+
+1. Candidate payload length ≥ 100 bytes.
+2. Sample entropy check: unique-byte ratio over `min(len, 256)` samples &lt; 0.9.
+3. zlib (`COMPRESSION_ZLIB`) must shrink the buffer; otherwise leave uncompressed.
+4. Set `isCompressed`, prepend original size (2 bytes v1 / 4 bytes v2), then
+   compressed bytes. `payloadLength` = preamble + compressed size.
+
+Decompression:
+
+- Original size **MUST** equal the decompressed length.
+- Compression ratio (original / compressed) **MUST NOT** exceed `50_000:1`
+  (zip-bomb guard).
+
+---
+
+## 6. PKCS#7-style frame padding
+
+`MessagePadding` buckets: **256, 512, 1024, 2048**.
+
+- Pad bytes are all equal to the pad length (1…255).
+- Bucket selection (`optimalBlockSize`): choose the smallest bucket such that
+  `encodedSize + 16 ≤ bucket` (the `+16` accounts for AEAD tag headroom used by
+  the helper even when the frame is not a Noise ciphertext). If no bucket fits,
+  or more than 255 pad bytes would be required, the frame is left **unpadded**.
+- Decode tries the buffer as-is, then strips padding and retries.
+
+Two different call sites use this helper — do not conflate them:
+
+| Path | Padding? |
+|------|----------|
+| **BLE outbound encode** (`padsBLEFrame`) | **Only** `noiseHandshake` / `noiseEncrypted`. All other types travel at natural length on the air (payload length observable). |
+| **Packet signature preimage** (`toBinaryDataForSigning` → `BinaryProtocol.encode` default `padding: true`) | Padding **MAY** be present for *any* signed type (including announces). Verifiers **MUST** use the same padded canonical bytes. |
+
+See §9 for the signature preimage rules.
+
+---
+
+## 7. Message types (`type` byte)
+
+From `MessageType` (`localPackages/BitFoundation/.../MessageType.swift`):
+
+| Value | Name | Notes |
+|-------|------|-------|
+| `0x01` | `announce` | Presence + identity keys (signed) |
+| `0x02` | `message` | Public chat (`BitchatMessage` binary) |
+| `0x03` | `leave` | Departure |
+| `0x04` | `courierEnvelope` | Store-and-forward sealed mail |
+| `0x10` | `noiseHandshake` | Noise XX message blob |
+| `0x11` | `noiseEncrypted` | Noise transport ciphertext |
+| `0x20` | `fragment` | Fragment of a larger encoded packet |
+| `0x21` | `requestSync` | GCS gossip sync request (local) |
+| `0x22` | `fileTransfer` | Public file/audio/image TLV |
+| `0x23` | `boardPost` | Signed geohash board post/tombstone |
+| `0x24` | `prekeyBundle` | Gossiped one-time prekeys |
+| `0x25` | `groupMessage` | Group-encrypted broadcast |
+| `0x26` | `ping` | Mesh diagnostic echo request |
+| `0x27` | `pong` | Mesh diagnostic echo reply |
+| `0x28` | `nostrCarrier` | Signed Nostr event ferry |
+| `0x29` | `voiceFrame` | Public live PTT burst |
+
+Inner private traffic (DMs, receipts, private media, verification) uses type
+`0x11` with a typed plaintext after decrypt — see
+[`03-noise.md`](03-noise.md) and [`04-payloads.md`](04-payloads.md).
+
+---
+
+## 8. Peer identity on the wire
+
+| Concept | Definition |
+|---------|------------|
+| Noise static key | 32-byte Curve25519.KeyAgreement public key |
+| Fingerprint | `SHA-256(noiseStaticPublicKey)` (32 bytes / 64 hex) |
+| Mesh peer ID | **First 8 bytes** of the fingerprint (16 hex chars) |
+| Packet `senderID` / `recipientID` | Those 8 raw bytes |
+
+The 8-byte routing ID is **stable** for the life of the Noise static key. It is
+not a session ephemeral. Panic wipe / identity rotation is the only change
+event.
+
+Ed25519 signing keys (32-byte public) are advertised in announces and used for
+packet signatures; they are distinct from the Noise static key.
+
+---
+
+## 9. Packet signatures
+
+- Algorithm: **Ed25519** (`Curve25519.Signing`), 64-byte signature.
+- Canonical preimage (`BitchatPacket.toBinaryDataForSigning()`):
+  1. Copy the packet with `signature = nil`, `ttl = 0`, `isRSR = false`
+     (TTL and RSR are mutable in flight and excluded from the signed bytes).
+  2. Encode with `BinaryProtocol.encode(..., padding: true)` — the **default**.
+     Apply §6 PKCS#7-style padding / bucket selection to that encoding.
+  3. Sign or verify those exact bytes. An unpadded encode of the same fields
+     **will not** verify against reference-client signatures whenever padding
+     was applied (common for compact announces that land in the 256-byte bucket).
+- Relays **MUST** decrement TTL without recomputing the signature.
+- Announces, leaves, public file transfers, and other authenticated public
+  types set `hasSignature` when the reference clients emit them.
+
+BLE air frames for non-Noise types are often sent **without** this padding
+(§6 table). That is independent of the signature preimage: the signature
+covers the padded canonical encoding, then the on-air frame for that type may
+omit pad bytes. Verifiers rebuild the preimage themselves; they do not require
+the received ATT blob to still carry the pad.
+
+Optional helper `bitchat-announce-v1` binding bytes exist in the Noise service
+for nickname/key binding tests; live mesh announces sign the **full packet**
+canonical form above, verified against the Ed25519 key carried in the announce
+TLV payload.
+
+---
+
+## 10. Size limits (shared)
+
+| Limit | Value | Source |
+|-------|-------|--------|
+| Max file content | 1 MiB | `FileTransferLimits.maxPayloadBytes` |
+| Max voice / image (app policy) | 512 KiB each | same |
+| Max framed file decode ceiling | ~1 MiB + TLV + v2 header/ids/sig | `maxFramedFileBytes` |
+| Compression attempt threshold | 100 bytes | `Constants.compressionThresholdBytes` |
+
+---
+
+## 11. Implementer checklist
+
+- [ ] Round-trip v1 packet with no recipient, no signature, empty payload.
+- [ ] Round-trip v1 with recipient + signature using **padded** canonical bytes.
+- [ ] Confirm an unpadded announce preimage fails verification against a
+      reference signature when padding would have applied.
+- [ ] Round-trip v2 with route hops; confirm route bytes are outside `payloadLength`.
+- [ ] Compress a low-entropy &gt;100 B payload; confirm preamble + flag.
+- [ ] Reject version `0x00` / `0x03`.
+- [ ] Derive peer ID as `SHA256(noisePub)[0..<8]`.
--- a/spec/02-ble-transport.md
+++ b/spec/02-ble-transport.md
@ -0,0 +1,204 @@
+# 02 — BLE Transport
+
+**Spec:** 1.0.1  
+**Canonical source:** `bitchat/Services/BLE/BLEService.swift`,  
+`BLEOutboundFragmentPlanner.swift`, `BLEFragmentAssemblyBuffer.swift`,  
+`TransportConfig.swift`
+
+---
+
+## 1. Role model
+
+Every BitChat node is **simultaneously** a GATT Central and Peripheral
+(dual-role). There is no pairing requirement and no BLE bonding for the mesh
+data path.
+
+---
+
+## 2. GATT UUIDs
+
+| Item | UUID | Notes |
+|------|------|-------|
+| Service (release / “mainnet”) | `F47B5E2D-4A9E-4C5A-9B3F-8E1D2C3A4B5C` | Production builds |
+| Service (debug / “testnet”) | `F47B5E2D-4A9E-4C5A-9B3F-8E1D2C3A4B5A` | `DEBUG` builds only |
+| Characteristic | `A1B2C3D4-E5F6-4A5B-8C9D-0E1F2A3B4C5D` | Single char on the service |
+
+Characteristic properties (reference):
+
+- Notify, Write, Write Without Response, Read  
+- Permissions: readable + writable
+
+iOS restoration identifiers (informational):  
+`chat.bitchat.ble.central`, `chat.bitchat.ble.peripheral`.
+
+Independent clients **MUST** use the release service UUID to interoperate with
+App Store / production Android builds. Debug UUID traffic will not be seen by
+release peers.
+
+---
+
+## 3. Advertising and discovery
+
+Advertisement payload:
+
+```
+CBAdvertisementDataServiceUUIDsKey → [serviceUUID]
+```
+
+| Field | Policy |
+|-------|--------|
+| Local name | **MUST NOT** be included (privacy) |
+| Manufacturer data | Not used |
+| Scan filter | `withServices: [serviceUUID]` |
+
+Discovery may observe a peer-supplied local name if some other stack adds one;
+BitChat itself does not advertise a name.
+
+Typical lifecycle:
+
+1. Add GATT service → start advertising service UUID.
+2. Scan for the same service UUID.
+3. On connect (as central): discover service → discover characteristic →
+   subscribe for notifications; write frames to the characteristic.
+4. As peripheral: accept writes; push frames via notify.
+
+Connection scheduling is RSSI-gated; duty-cycled scan windows conserve battery
+(see §7).
+
+---
+
+## 4. Link framing (ATT → BinaryProtocol)
+
+Each write/notify carries opaque bytes that are assembled into complete
+`BinaryProtocol` frames by a stream assembler (`NotificationStreamAssembler`):
+
+- Accepts leading version byte `1` or `2`.
+- Strips leading PKCS#7-style padding runs when present.
+- Computes expected frame length from header flags + payload length (+
+  recipient / signature / route).
+- Hard cap: **8 MiB** (`bleNotificationAssemblerHardCapBytes`).
+- Incomplete-frame stall reset: **250 ms**.
+
+Preferred write mode: **Write Without Response**, bounded by
+`maximumWriteValueLength` / `maximumUpdateValueLength`. Hard ceiling considered
+by the stack: **512** (`bleMaxMTU`).
+
+---
+
+## 5. Fragmentation
+
+When an encoded `BinaryProtocol` blob exceeds the link chunk size, the sender
+emits one or more packets of type `fragment` (`0x20`).
+
+### 5.1 Chunk sizing
+
+| Knob | Default |
+|------|---------|
+| Default chunk | **469** bytes (`bleDefaultFragmentSize`) |
+| Minimum chunk | **64** bytes |
+| Private-media Android contract | ≤ **256** fragments per transfer |
+
+Link-aware sizing may shrink the chunk using
+`max(64, linkLimit − overhead)` when source routes inflate headers.
+
+### 5.2 Fragment payload layout
+
+Minimum 13 bytes, then chunk data:
+
+```
+ offset  size  field
+ 0       8     fragmentID (random)
+ 8       2     index   (uint16 BE, 0-based)
+ 10      2     total   (uint16 BE, 1…10000)
+ 12      1     originalType (MessageType of the inner packet)
+ 13…           chunk bytes of the original encoded frame
+```
+
+Fragment packet fields:
+
+- `type` = `0x20`
+- `senderID`, `timestamp`, `ttl`, optional `route` / `isRSR` inherited from the
+  original
+- `signature` = **nil** on fragments (inner packet carries its own signature
+  after reassembly)
+- `version` = `1`, or `2` when the original carried a source route
+- `recipientID` = directed peer when applicable; broadcast may use nil or
+  `FF×8`
+
+### 5.3 Reassembly
+
+| Rule | Value |
+|------|-------|
+| Assembly key | `(senderID as u64 BE, fragmentID as u64 BE)` |
+| Max concurrent assemblies | **128** (evict oldest) |
+| Assembly lifetime | **30 s** |
+| Size cap (typical) | 1 MiB payload |
+| Size cap (`fileTransfer` / `noiseEncrypted`) | `maxFramedFileBytes` |
+| Stall → `requestSync` | after **5 s**; retry every **10 s** |
+| Inter-fragment spacing (broadcast) | **30 ms** |
+| Inter-fragment spacing (directed) | **25 ms** |
+| Max concurrent large transfers | **2** |
+
+Validation:
+
+- `total ∈ [1, 10000]`, `index < total`
+- First accepted fragment header is authoritative for `(total, originalType,
+  broadcast scope)`; conflicting later fragments **MUST** be rejected (not
+  stored, not relayed)
+- Duplicate index: do not double-count size toward the assembly budget
+- Oversize fragments **MUST NOT** destroy an assembly they did not create
+
+On completion, concatenate chunks in index order and run `BinaryProtocol.decode`
+on the result. Dispatch using the **decoded inner packet's `type` field**
+(`originalPacket.type`), then re-enter the normal receive path with that
+packet (reference: `BLEFragmentHandler` sets `ttl = 0` and reinjects).
+
+The fragment-header `originalType` byte is **not authenticated** and **MUST
+NOT** be used as the sole dispatch key. Implementations **SHOULD** require
+`originalType == decoded.type` and drop the assembly on mismatch; at minimum
+they **MUST** ignore the header value for parser/policy selection and follow
+the decoded type so a spoofed header cannot steer valid inner bytes into the
+wrong handler.
+
+---
+
+## 6. Default TTL and flood behaviour
+
+| Knob | Default |
+|------|---------|
+| Origination TTL | **7** (`messageTTLDefault`) |
+| Dense-graph broadcast clamp | often **5** when degree ≥ 6 |
+| Dedup | LRU seen-set (~1000 entries, ~5 min) keyed by sender/timestamp/type/digest |
+| Relay jitter | randomized tens–hundreds of ms (wider when dense) |
+| Fanout | deterministic subset (~log₂ degree) for many broadcasts; full fanout for announces/fragments/sync |
+| Directed traffic | TTL−1, tight jitter, never subset |
+
+Full routing prose: `WHITEPAPER.md` §4 and `docs/SOURCE_ROUTING.md`.
+
+---
+
+## 7. Presence and duty cycle (informative)
+
+Reference client behaviour (not hard wire requirements, but useful for
+interoperability timing):
+
+| Behaviour | Typical value |
+|-----------|---------------|
+| Isolated announce interval | ~4 s |
+| Connected announce | ~15–30 s base + jitter |
+| Scan duty on/off (sparse) | 5 s / 10 s |
+| Reachability retention (verified) | 60 s since lastSeen |
+| Initial announce delay after start | ~0.6 s |
+
+---
+
+## 8. Implementer checklist
+
+- [ ] Advertise and scan **only** the release service UUID in production.
+- [ ] Use the shared characteristic UUID with notify + write-without-response.
+- [ ] Do not put a local name in advertisements.
+- [ ] Reassemble ATT notifications into BinaryProtocol frames before parsing.
+- [ ] Fragment at ≤469 B chunks with the 13-byte fragment header.
+- [ ] Cap assemblies (128 / 30 s) and reject conflicting fragment metadata.
+- [ ] After reassembly, dispatch by decoded packet type (not header `originalType`).
+- [ ] Originate mesh packets with TTL 7 unless a documented exception applies.
--- a/spec/03-noise.md
+++ b/spec/03-noise.md
@ -0,0 +1,174 @@
+# 03 — Noise Handshake and Encrypted Transport
+
+**Spec:** 1.0.1  
+**Canonical source:** `bitchat/Noise/NoiseProtocol.swift`,  
+`NoiseSession.swift`, `NoiseEncryptionService.swift`,  
+`bitchat/Protocols/BitchatProtocol.swift` (`NoisePayloadType`)  
+**Background:** `BRING_THE_NOISE.md`, `WHITEPAPER.md` §5
+
+---
+
+## 1. Crypto suite
+
+| Component | Choice |
+|-----------|--------|
+| DH | X25519 (Curve25519.KeyAgreement) |
+| AEAD | ChaCha20-Poly1305 |
+| Hash | SHA-256 |
+| KDF | HKDF-SHA256 |
+| Packet / announce signatures | Ed25519 (separate signing keypair) |
+
+Protocol names used on the wire (Noise naming):
+
+| Use | Name | Prologue |
+|-----|------|----------|
+| Live mesh session | `Noise_XX_25519_ChaChaPoly_SHA256` | empty |
+| Courier seal v1 | `Noise_X_25519_ChaChaPoly_SHA256` | UTF-8 `bitchat-courier-v1` |
+| Prekey seal v2 | `Noise_X_25519_ChaChaPoly_SHA256` | UTF-8 `bitchat-prekey-v1` ‖ `uint32 BE prekeyID` |
+
+`IK` / `NK` appear in the Noise module but are **not** used for live mesh
+sessions in the reference clients.
+
+---
+
+## 2. Live sessions: Noise XX
+
+Pattern:
+
+```
+XX:
+  -> e
+  <- e, ee, s, es
+  -> s, se
+```
+
+Application payloads inside handshake messages are **empty** in production
+(handshake blobs contain only Noise tokens / AEAD wrappers).
+
+### 2.1 Message sizes (empty payload)
+
+| Msg | Direction | Contents (summary) | Typical size |
+|-----|-----------|--------------------|--------------|
+| 1 | Initiator → Responder | raw ephemeral pubkey `e` (32) | **32** |
+| 2 | Responder → Initiator | `e`(32) + `Encrypt(s)`(48) + `Encrypt(∅)`(16) | **96** |
+| 3 | Initiator → Responder | `Encrypt(s)`(48) + `Encrypt(∅)`(16) | **64** |
+
+`Encrypt(s)` is a 32-byte static public key under ChaChaPoly → 32 + 16 tag = 48
+once a cipher key exists. Before a key is mixed, Noise returns plaintext.
+
+### 2.2 Mapping onto mesh packets
+
+| Noise step | `MessageType` | Packet fields |
+|------------|---------------|---------------|
+| XX msg 1/2/3 | `noiseHandshake` (`0x10`) | `payload` = handshake blob; `recipientID` = 8-byte peer; `signature` = nil; TTL default 7 |
+| Transport | `noiseEncrypted` (`0x11`) | `payload` = transport ciphertext (below); directed; typically padded |
+
+Which of the three XX messages a `0x10` packet carries is determined solely by
+session state — there is no extra discriminator byte.
+
+Handshake timeouts in the reference stack: ordinary ~10 s; responder quarantine
+~20 s; max handshake message 2048 bytes.
+
+---
+
+## 3. Transport ciphertext (post-handshake)
+
+Live sessions enable **extracted nonces** (`useExtractedNonce = true`):
+
+```
+[4 bytes nonce BE][ciphertext…][16 bytes Poly1305 tag]
+```
+
+ChaChaPoly nonce construction for the AEAD call: 12-byte buffer with the
+64-bit counter in bytes **4…11 little-endian** (bytes 0…3 zero).
+
+Overhead versus plaintext: **20** bytes (4 nonce + 16 tag).
+
+Receivers apply sliding-window replay protection on the extracted 4-byte
+counter. Nonces beyond `UInt32.max − 1` force rekey / error.
+
+> **Conformance note:** Official Noise explorer vectors in
+> `bitchatTests/Noise/NoiseTestVectors.json` use non-empty prologues/payloads
+> and transport **without** extracted nonces. Those vectors validate the crypto
+> core; they are **not** byte-identical to production XX mesh traffic.
+
+---
+
+## 4. Inner plaintext after decrypt
+
+```
+[1 byte NoisePayloadType][type-specific data…]
+```
+
+| Value | Name | Inner format (summary) |
+|-------|------|------------------------|
+| `0x01` | `privateMessage` | TLV `0x00` messageID, `0x01` content (1-byte lengths, ≤255 each) |
+| `0x02` | `readReceipt` | UTF-8 original message ID |
+| `0x03` | `delivered` | UTF-8 message ID |
+| `0x06` | `groupInvite` | creator-signed group state |
+| `0x07` | `groupKeyUpdate` | creator-signed key/roster update |
+| `0x08` | `voiceFrame` | `VoiceBurstPacket` |
+| `0x10` | `verifyChallenge` | QR verification challenge bytes |
+| `0x11` | `verifyResponse` | QR verification response bytes |
+| `0x12` | `vouch` | vouch attestation batch |
+| `0x20` | `privateFile` | full `BitchatFilePacket` (encrypted *before* outer BLE fragmentation) |
+| `0x21` | `authenticatedPeerState` | versioned TLV peer state (see payloads) |
+| `0x09` | *(legacy alias)* | Decode-only alias for `privateFile`; **MUST NOT** emit |
+
+Unknown payload types **MUST** be ignored without tearing down the session.
+
+Detailed TLV layouts: [`04-payloads.md`](04-payloads.md).
+
+---
+
+## 5. Offline seals: Noise X
+
+### 5.1 Courier v1 (static key)
+
+1. MixHash prologue `bitchat-courier-v1`.
+2. Pre-message: mix recipient static public key into handshake hash.
+3. Single initiator message: `e, es, s, ss` + encrypted application payload.
+4. Ciphertext has **no** 4-byte extracted nonce prefix (handshake AEAD only).
+5. Rough size: `32 + 48 + (payloadLen + 16)` plus any unencrypted token bytes
+   per Noise X.
+
+No forward secrecy: compromise of the recipient static key exposes captured
+sealed mail. Prefer live XX sessions when the peer is reachable.
+
+The ciphertext is wrapped in a `CourierEnvelope` TLV carried as mesh type
+`0x04` — see payloads.
+
+### 5.2 Prekey v2 (forward-secret)
+
+Same X pattern, but the responder static is a **one-time prekey** from a
+gossiped `PrekeyBundle` (`0x24`). Prologue:
+
+```
+"bitchat-prekey-v1" || uint32_be(prekeyID)
+```
+
+Envelope TLV includes optional `prekeyID` (`0x05`) so carriers can forward
+opaquely; old clients that only know v1 still carry the bytes and fail open
+quietly if addressed to them without the matching prekey.
+
+---
+
+## 6. Authenticated peer state
+
+After XX completes, peers exchange `authenticatedPeerState` (`0x21`) inside the
+session to pin capabilities and the Ed25519 signing key under Noise
+authentication. Public announce TLVs remain discovery hints and are not a
+substitute for this proof.
+
+---
+
+## 7. Implementer checklist
+
+- [ ] Implement XX with empty prologue and empty handshake payloads.
+- [ ] Map handshake blobs to `MessageType 0x10` and transport to `0x11`.
+- [ ] Use extracted 4-byte BE nonce prefix on transport frames.
+- [ ] Prefix decrypted plaintext with `NoisePayloadType`.
+- [ ] Seal courier mail with prologue `bitchat-courier-v1` (pattern X).
+- [ ] Seal prekey mail with `bitchat-prekey-v1` ‖ prekeyID.
+- [ ] Pass `NoiseTestVectors.json` for the crypto core (knowing the production
+      divergence above).
--- a/spec/04-payloads.md
+++ b/spec/04-payloads.md
@ -0,0 +1,258 @@
+# 04 — Payload Layouts
+
+**Spec:** 1.0.1  
+
+TLV conventions differ by packet family — do not mix length widths.
+
+| Family | Length field |
+|--------|--------------|
+| Announce / private message / authenticated peer state | `uint8` |
+| Courier envelope / prekey bundle | `uint16` BE |
+| File transfer content TLV | `uint32` BE (canonical); other file TLVs `uint16` BE |
+
+### Unknown-TLV policy (do not over-promise)
+
+Forward-compatible **skip** of unknown TLV types applies only where the
+reference decoder actually skips:
+
+| Family | Unknown TLV behaviour |
+|--------|------------------------|
+| Announce | **Skip** and continue |
+| Authenticated peer state | **Skip** and continue |
+| Courier envelope | **Skip** and continue |
+| Prekey bundle | **Skip** and continue |
+| File transfer | **Skip** and continue |
+| Private message (`PrivateMessagePacket`) | **Reject** entire payload (`nil`) |
+
+Adding a new private-message TLV under a SemVer **minor** bump would break
+current reference DM decoders. Treat private-message TLV extensions as a
+**MAJOR** wire change (or change the decoder first), not as a silent skip.
+
+---
+
+## 1. Announce (`MessageType 0x01`)
+
+**Source:** `bitchat/Protocols/Packets.swift` — `AnnouncementPacket`
+
+Each TLV: `[type:u8][len:u8][value]`.
+
+| Type | Field | Notes |
+|------|-------|-------|
+| `0x01` | nickname | UTF-8, ≤255 bytes (**required**) |
+| `0x02` | noisePublicKey | 32-byte Curve25519 KA public (**required**) |
+| `0x03` | signingPublicKey | 32-byte Ed25519 public (**required**) |
+| `0x04` | directNeighbors | concatenation of ≤10 × 8-byte peer IDs |
+| `0x05` | capabilities | little-endian `PeerCapabilities` bitfield (minimal encoding, ≥1 byte) |
+| `0x06` | bridgeGeohash | UTF-8 cell, ≤12 bytes |
+
+Outer packet is Ed25519-signed (`hasSignature`). Sender ID **MUST** equal
+`SHA256(noisePublicKey)[0..<8]`.
+
+### 1.1 Capability bits (`PeerCapabilities`)
+
+Encoded little-endian, trailing zero bytes dropped; always at least one byte.
+
+| Bit | Name |
+|-----|------|
+| 0 | `prekeys` |
+| 1 | `wifiBulk` |
+| 2 | `gateway` |
+| 3 | `groups` |
+| 4 | `board` |
+| 5 | `vouch` |
+| 6 | `meshDiagnostics` |
+| 7 | `bridge` |
+| 8 | `privateMedia` |
+| 9 | `privateMediaReceipts` |
+| 10 | `nonDestructiveNoiseReplacement` (reserved; do not advertise) |
+
+---
+
+## 2. Public message (`MessageType 0x02`)
+
+**Source:** `BitchatMessage.toBinaryPayload()`
+
+```
+flags:u8
+timestamp_ms:u64 BE
+id_len:u8 | id UTF-8
+sender_len:u8 | sender UTF-8
+content_len:u16 BE | content UTF-8
+[optional fields per flags…]
+```
+
+| Flag bit | Meaning |
+|----------|---------|
+| `0x01` | isRelay |
+| `0x02` | isPrivate (legacy; mesh DMs use Noise instead) |
+| `0x04` | has originalSender (`u8` len + UTF-8) |
+| `0x08` | has recipientNickname |
+| `0x10` | has senderPeerID (`u8` len + UTF-8 peer id string) |
+| `0x20` | has mentions (`u8` count, then each `u8` len + UTF-8) |
+| `0x40` | isBridged |
+
+---
+
+## 3. Leave (`0x03`)
+
+Reference clients send a signed leave with a small/empty payload. Treat unknown
+payload bytes as non-fatal if the outer signature verifies.
+
+---
+
+## 4. Courier envelope (`0x04`)
+
+**Source:** `CourierEnvelope.swift`
+
+TLV: `[type:u8][len:u16 BE][value]`.
+
+| Type | Field | Length |
+|------|-------|--------|
+| `0x01` | recipientTag | 16 bytes — HMAC-SHA256 truncated |
+| `0x02` | expiry | 8 bytes `uint64` BE ms since epoch |
+| `0x03` | ciphertext | 1…16384 bytes Noise X ciphertext |
+| `0x04` | copies | 1 byte spray budget (omitted when `1`) |
+| `0x05` | prekeyID | 4 bytes `uint32` BE (v2 only; omitted for v1) |
+
+Recipient tag:
+
+```
+tag = HMAC-SHA256(key = recipientNoiseStatic,
+                  msg = "bitchat-courier-tag-v1" || uint32_be(utcDay))[0..<16]
+utcDay = floor(unixSeconds / 86400)
+```
+
+Matchers **SHOULD** accept yesterday/today/tomorrow tags for clock skew.
+`copies` is clamped to `1…8`. Max lifetime policy in reference clients: 24 h.
+
+---
+
+## 5. Fragment (`0x20`)
+
+See [`02-ble-transport.md`](02-ble-transport.md) §5 — 13-byte header + chunk.
+
+---
+
+## 6. File transfer (`0x22`) and private files
+
+**Source:** `BitchatFilePacket.swift`
+
+Canonical encode:
+
+| Type | Length width | Value |
+|------|--------------|-------|
+| `0x01` fileName | `u16` BE | UTF-8 |
+| `0x02` fileSize | `u16` BE = 4 | `u32` BE size |
+| `0x03` mimeType | `u16` BE | UTF-8 |
+| `0x04` content | **`u32` BE** | raw bytes |
+
+Decoders **SHOULD** accept legacy `fileSize` length 8 and legacy content length
+width 2 when the canonical parse fails.
+
+Limits: content ≤ 1 MiB; voice/image app caps 512 KiB.
+
+**Public** media uses mesh type `0x22` (signed).  
+**Private** media places the same `BitchatFilePacket` bytes inside Noise as
+`NoisePayloadType.privateFile` (`0x20`), then fragments the outer
+`noiseEncrypted` packet. Peers without capability bit `privateMedia` require a
+consent-gated legacy path (see `docs/PRIVATE-MEDIA-MIGRATION.md`).
+
+---
+
+## 7. Prekey bundle (`0x24`)
+
+**Source:** `PrekeyBundle.swift`
+
+TLV `[type:u8][len:u16 BE][value]`:
+
+| Type | Field |
+|------|-------|
+| `0x01` | noiseStaticPublicKey (32) |
+| `0x02` | prekeys blob: repeated (`id:u32 BE` ‖ `pubkey:32`), 1…8 entries, unique IDs |
+| `0x03` | generatedAt `u64` BE ms |
+| `0x04` | Ed25519 signature (64) |
+
+Signable bytes (domain-separated):
+
+```
+u8(len) || "bitchat-prekey-bundle-v1"
+|| noiseStatic(32)
+|| u8(count) || { id:u32 BE || pubkey:32 }×count
+|| generatedAt:u64 BE
+```
+
+Verified with the owner's announce-bound Ed25519 key.
+
+---
+
+## 8. Ping / pong (`0x26` / `0x27`)
+
+**Source:** `MeshPingPayload.swift`
+
+```
+nonce: 8 random bytes
+originTTL: u8
+```
+
+Pong echoes the nonce. Hop estimate:
+`hopCount = (originTTL - receivedTTL) + 1` when `originTTL ≥ receivedTTL`.
+Unsigned and unencrypted by design. Trailing bytes **MAY** be ignored.
+
+---
+
+## 9. Noise inner: private message
+
+**Source:** `PrivateMessagePacket`
+
+TLV `[type:u8][len:u8][value]`:
+
+| Type | Field |
+|------|-------|
+| `0x00` | messageID UTF-8 ≤255 |
+| `0x01` | content UTF-8 ≤255 |
+
+Prefixed by `NoisePayloadType.privateMessage` (`0x01`) when inside Noise.
+
+Unlike announce/courier TLVs, any unknown type byte causes
+`PrivateMessagePacket.decode` to return `nil` immediately (no skip). Both
+`messageID` and `content` are required.
+
+---
+
+## 10. Noise inner: authenticated peer state
+
+```
+version = 0x01
+then TLV [type:u8][len:u8][value]:
+  0x01 capabilities — canonical little-endian PeerCapabilities (1…8 bytes)
+  0x02 signingPublicKey — 32 bytes Ed25519
+```
+
+Duplicates, non-canonical capability encodings, and unknown versions **MUST**
+be rejected.
+
+---
+
+## 11. Board / group / voice / Nostr carrier
+
+These types have dedicated Swift modules (`BoardPackets`, `GroupProtocol`,
+`VoiceBurstPacket`, bridge carriers). They follow the same outer
+`BinaryProtocol` framing. Full TLV breakdowns for board and groups are
+intentionally deferred to a minor spec revision; implementers should mirror
+the encode/decode in:
+
+- `bitchat/Protocols/BoardPackets.swift`
+- `bitchat/Services/Groups/GroupProtocol.swift`
+- live voice design notes in `docs/PUSH-TO-TALK-DESIGN.md`
+
+---
+
+## 12. Implementer checklist
+
+- [ ] Parse announce TLVs with 1-byte lengths; require nickname + both keys.
+- [ ] Encode capabilities as little-endian with unknown bits preserved.
+- [ ] Courier TLVs use 2-byte lengths; compute rotating recipient tags.
+- [ ] File content TLV uses 4-byte length; tolerate legacy widths on decode.
+- [ ] Prekey bundle signature verifies over domain-prefixed signable bytes.
+- [ ] Private DM content is Noise-typed, not a public `0x02` packet.
+- [ ] Confirm unknown private-message TLVs reject; unknown announce TLVs skip.
--- a/spec/05-nostr-bridge.md
+++ b/spec/05-nostr-bridge.md
@ -0,0 +1,99 @@
+# 05 — Nostr Bridge
+
+**Spec:** 1.0.1  
+**Canonical prose:** `WHITEPAPER.md` §5.3, §6.4; `README.md`;
+`docs/GeohashPresenceSpec.md`
+
+BitChat uses public Nostr relays as an **internet mailbox and geohash
+broadcast** transport. This chapter is intentionally shorter than the mesh
+chapters: relay selection details for external publishers are tracked in
+issue [#1473](https://github.com/permissionlesstech/bitchat/issues/1473).
+
+---
+
+## 1. Compatibility warning (normative)
+
+BitChat private envelopes reuse NIP-17 / NIP-59 **kind numbers** but are
+**not** NIP-17, NIP-44, or NIP-59 compatible.
+
+- Interoperates **only** with BitChat clients.
+- Content fields are **not** NIP-44 ciphertext.
+- Do not expect vanilla Nostr DM clients to decrypt BitChat private mail.
+
+---
+
+## 2. Private envelope sketch
+
+High-level construction (see whitepaper for motivation):
+
+1. Inner unsigned message (kind **14** semantics).
+2. Encrypted into a sender-signed seal (kind **13**).
+3. Seal wrapped again in a public envelope (kind **1059**) signed by a
+   one-time key so relays do not learn the stable sender identity.
+
+Each encrypted content field:
+
+```
+"v2:" || base64url( 24-byte nonce || XChaCha20-Poly1305 ciphertext+tag )
+```
+
+Key schedule: secp256k1 ECDH + HKDF-SHA256. The HKDF info label reuses a
+`nip44-v2` string for historical reasons but is **not** the NIP-44 schedule.
+
+Properties:
+
+- Outer `p` tag exposes the recipient Nostr pubkey to relays.
+- Public timestamps are jittered (±15 minutes); real timestamp is inside
+  ciphertext.
+- **No forward secrecy** on this path: compromise of the recipient Nostr
+  private key can expose stored envelopes.
+
+---
+
+## 3. Geohash public channels
+
+| Kind | Role |
+|------|------|
+| `20000` | Geohash chat message |
+| `20001` | Ephemeral presence heartbeat |
+
+Presence rules (precision caps, heartbeat cadence, participant counting) are
+normative in `docs/GeohashPresenceSpec.md` and should be treated as part of
+the location-channel contract.
+
+Tags typically include `["g", "<geohash>"]`. Presence content is empty and
+omits nickname tags.
+
+---
+
+## 4. Mesh ↔ Nostr carriers
+
+Mesh type `nostrCarrier` (`0x28`) ferries a signed Nostr event between a
+mesh-only peer and an internet gateway peer advertising the `gateway` /
+`bridge` capabilities. Gateway behaviour is an application policy on top of
+the wire type; do not assume every peer will uplink.
+
+---
+
+## 5. Relay interaction (non-normative until #1473 lands)
+
+Reference clients:
+
+- Maintain a geo-proximity relay directory (~300 relays).
+- Subscribe to a small set of relays near the active geohash
+  (`nostrGeoRelayCount` default 5 in `TransportConfig`).
+- Re-subscribe with lookback (DMs ~24 h; geohash chat shorter windows) on
+  reconnect.
+
+External publishers that pin an arbitrary fixed relay set may never intersect
+the subscriber's geo-selected set — replicate proximity selection or publish
+widely enough to overlap.
+
+---
+
+## 6. Implementer checklist
+
+- [ ] Treat BitChat private envelopes as proprietary, not NIP-44.
+- [ ] Implement `v2:` + base64url XChaCha20-Poly1305 content encoding.
+- [ ] Support kinds 20000/20001 for geohash chat/presence.
+- [ ] Do not claim NIP-17/44/59 compatibility in client metadata.
--- a/spec/README.md
+++ b/spec/README.md
@ -0,0 +1,60 @@
+# BitChat Protocol Specification
+
+**Spec version:** [`1.0.1`](VERSION)  
+**Status:** Draft extracted from the reference implementation  
+**Canonical codec:** `localPackages/BitFoundation`  
+**Architecture overview:** [`WHITEPAPER.md`](../WHITEPAPER.md)
+
+This directory is the byte-exact interoperability contract for independent
+clients. It is versioned independently of any one app release. The whitepaper
+describes *why* the system is shaped the way it is; these documents describe
+*what* must be on the wire for two implementations to talk.
+
+## Documents
+
+| # | Document | Contents |
+|---|----------|----------|
+| 1 | [`01-wire-format.md`](01-wire-format.md) | Packet header, flags, compression, padding, peer IDs, message types |
+| 2 | [`02-ble-transport.md`](02-ble-transport.md) | GATT UUIDs, advertising, MTU, fragmentation/reassembly, flood knobs |
+| 3 | [`03-noise.md`](03-noise.md) | XX live sessions, X offline seals, transport frames, payload type map |
+| 4 | [`04-payloads.md`](04-payloads.md) | Per-type TLV layouts (announce, message, file, courier, prekey, ping, …) |
+| 5 | [`05-nostr-bridge.md`](05-nostr-bridge.md) | Proprietary private envelopes and geohash event kinds |
+| — | [`conformance.md`](conformance.md) | Checklist and pointers to existing test vectors |
+
+## Normative language
+
+The key words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are to be
+interpreted as in RFC 2119.
+
+Where this spec and the Swift/Kotlin reference clients disagree, treat the
+codec in `localPackages/BitFoundation` (and the Android mirror of the same
+wire types) as authoritative until this document is amended. Open a PR against
+`/spec` when fixing either side.
+
+## Versioning
+
+- Spec versions use **SemVer** (`MAJOR.MINOR.PATCH`) stored in [`VERSION`](VERSION).
+- **MAJOR** — breaking wire change (header layout, type reassignment, crypto suite).
+- **MINOR** — additive, backward-compatible (new message type, new TLV skipped by old clients).
+- **PATCH** — clarifications, errata, conformance notes with no wire change.
+- Spec version is **independent** of App Store / Android release numbers.
+
+Unknown TLV types and unknown high capability bits are handled
+per-family: most public TLV decoders skip unknowns so older clients can carry
+newer packets opaquely, but some inner payloads (notably private-message TLVs)
+reject unknowns — see [`04-payloads.md`](04-payloads.md).
+
+## Suggested reading order for implementers
+
+1. Wire format → build an encode/decode round-trip for empty announce packets.
+2. BLE transport → discover peers and exchange a single signed announce.
+3. Noise → establish an XX session and send a typed private payload.
+4. Remaining payload chapters as needed (files, courier, Nostr).
+
+## Related work
+
+- Relay-selection / geohash delivery semantics for external publishers: issue
+  [#1473](https://github.com/permissionlesstech/bitchat/issues/1473) and
+  `docs/GeohashPresenceSpec.md`.
+- Formal conformance vectors beyond the Noise explorer set and courier
+  fixtures are tracked as follow-up work; see [`conformance.md`](conformance.md).
--- a/spec/VERSION
+++ b/spec/VERSION
@ -0,0 +1 @@
+1.0.1
--- a/spec/conformance.md
+++ b/spec/conformance.md
@ -0,0 +1,88 @@
+# Conformance
+
+**Spec:** 1.0.1
+
+This file is a living checklist. Golden hex vectors for every mesh type are a
+planned follow-up (see issue
+[#1448](https://github.com/permissionlesstech/bitchat/issues/1448)); until
+then, independent clients SHOULD lock behaviour against the reference tests
+listed below and against byte-identical round-trips with
+`localPackages/BitFoundation`.
+
+---
+
+## 1. Existing vector / test assets
+
+| Asset | Path | Covers |
+|-------|------|--------|
+| Noise explorer XX vectors | `bitchatTests/Noise/NoiseTestVectors.json` | Crypto core for `Noise_XX_25519_ChaChaPoly_SHA256` (prologue/payloads differ from production mesh XX — see [`03-noise.md`](03-noise.md)) |
+| Noise protocol tests | `bitchatTests/Noise/NoiseProtocolTests.swift` | Handshake + cipher behaviour |
+| Binary protocol tests | `localPackages/BitFoundation/Tests/BitFoundationTests/BinaryProtocolTests.swift` | Header, flags, padding |
+| Courier envelope tests | `localPackages/BitFoundation/Tests/BitFoundationTests/CourierEnvelopeTests.swift` | TLV encode/decode, tags |
+| Fragmentation tests | `bitchatTests/Fragmentation/FragmentationTests.swift` | Split/reassembly |
+| Protocol contract tests | `bitchatTests/ProtocolContractTests.swift` | Type surface smoke checks |
+| Private media E2E | `bitchatTests/EndToEnd/PrivateMediaEndToEndTests.swift` | `0x20` private file path |
+| Prekey E2E | `bitchatTests/EndToEnd/PrekeyEndToEndTests.swift` | Prekey seal/open |
+
+Run from repo root:
+
+```bash
+swift test
+# or
+just test
+```
+
+---
+
+## 2. Minimum interoperability checklist
+
+### Wire format
+
+- [ ] Encode/decode v1 and v2 packets with correct endianness.
+- [ ] Honor optional recipient, signature, compression, route flags.
+- [ ] Exclude TTL/`isRSR` from signature canonicalization; include PKCS#7
+      padding from `encode(padding: true)` in the preimage.
+- [ ] Derive 8-byte peer IDs as `SHA256(noiseStatic)[0..<8]`.
+
+### BLE
+
+- [ ] Use release GATT service UUID `…4B5C` and characteristic `…4C5D`.
+- [ ] Advertise service UUID only (no local name).
+- [ ] Fragment with 13-byte header; reassemble with first-wins metadata.
+- [ ] Dispatch reassembled packets by decoded type, not header `originalType`.
+- [ ] Default origination TTL = 7.
+
+### Noise
+
+- [ ] XX live sessions, empty prologue, empty handshake payloads.
+- [ ] Transport frames use 4-byte extracted nonce prefix.
+- [ ] Typed inner payloads; ignore unknown types.
+- [ ] Courier X seals with prologue `bitchat-courier-v1`.
+- [ ] Optional prekey X seals with `bitchat-prekey-v1` ‖ id.
+
+### Payloads
+
+- [ ] Announce TLVs (nickname + Noise + Ed25519) with outer signature.
+- [ ] Capability bitfield little-endian.
+- [ ] File TLV content length 4-byte BE; private files via Noise `0x20`.
+- [ ] Courier recipient tag HMAC construction.
+- [ ] Unknown-TLV skip only where listed; private-message unknown tags reject.
+
+### Nostr
+
+- [ ] Proprietary `v2:` private envelopes (not NIP-44).
+- [ ] Geohash kinds 20000 / 20001 per `docs/GeohashPresenceSpec.md`.
+
+---
+
+## 3. Planned vector work (non-blocking for this draft)
+
+1. Hex fixtures for: empty announce, signed announce, public message, fragment
+   set, XX handshake transcript with production parameters, one courier
+   envelope, one file TLV.
+2. Version the fixture directory under `spec/vectors/` and cite it from this
+   file.
+3. Cross-check fixtures against the Android reference client.
+
+House style for vectors can follow patterns already used by Noise JSON vectors
+and any courier fixtures maintainers add alongside BitFoundation tests.