diff --git a/docs/CALL_TUNNEL.md b/docs/CALL_TUNNEL.md new file mode 100644 index 00000000..a28b136a --- /dev/null +++ b/docs/CALL_TUNNEL.md @@ -0,0 +1,546 @@ +# Media Tunnel Architecture + +## Overview + +signal-cli uses a Rust subprocess called `signal-call-tunnel` to handle voice +calls. The tunnel wraps RingRTC (Signal's WebRTC layer) and exposes a **control +socket** for call signaling. Audio flows through **virtual audio devices** +created by ringrtc's `VirtualAudioDevicePair`, which the tunnel selects via +cubeb. External processes connect using platform audio APIs (PulseAudio on +Linux, CoreAudio on macOS). + +``` + signal-cli (Java) + | + spawn per call, config via stdin + | + v + signal-call-tunnel (Rust) + / \ + ctrl.sock VirtualAudioDevicePair + (JSON control) / \ + | [virtual input] [virtual output] + signal-cli connects (signal_input_XXX) (signal_output_XXX) + (signaling relay) | | + client writes audio client reads audio + (PulseAudio/CoreAudio) (PulseAudio/CoreAudio) +``` + +Each call gets its own tunnel process and control socket inside a temporary +directory (`/tmp/sc-/`). Virtual audio devices are created per call on +Linux (PulseAudio modules) or use pre-installed BlackHole drivers on macOS. +When the call ends, everything is cleaned up. + +--- + +## Components + +### signal-call-tunnel (Rust binary) + +The subprocess that runs one call. Source: `signal-call-tunnel/src/`. + +| File | Role | +|------|------| +| `main.rs` | Entry point, RingRTC initialization, virtual audio setup, event loop | +| `config.rs` | Deserializes startup config from stdin | +| `control.rs` | Control socket server, JSON message parsing/serialization | +| `platform.rs` | RingRTC trait impls (SignalingSender, CallStateHandler) | + +Audio flows through cubeb with virtual audio devices selected by name. The +tunnel creates a `VirtualAudioDevicePair` from ringrtc's `virtual_audio` module, +waits for cubeb to enumerate the devices, then selects them as the recording +and playout devices. + +### CallManager.java (Java parent) + +`lib/src/main/java/org/asamk/signal/manager/helper/CallManager.java` + +Manages the call lifecycle from the Java side: + +1. Creates a temp directory and generates a random auth token +2. Spawns `signal-call-tunnel` with config JSON on stdin +3. Connects to the control socket (retries up to 50x at 200 ms intervals), + authenticates, and relays signaling between the tunnel and the Signal protocol +4. Parses `inputDeviceName` and `outputDeviceName` from the tunnel's `ready` + message and includes them in `CallInfo` +5. Translates tunnel state changes into `CallInfo.State` values and fires + `callEvent` JSON-RPC notifications to connected clients +6. Defers the `accept` message for incoming calls until the tunnel reports + `Ringing` state (sending earlier causes RingRTC to drop it) +7. Schedules a 60-second ring timeout for both incoming and outgoing calls +8. On hangup: sends hangup message, kills the process, deletes the control socket + +### Audio client (external process) + +Any process that sends/receives audio via the virtual audio devices using +platform audio APIs. + +1. Receives `inputDeviceName` and `outputDeviceName` from a `startCall`/ + `acceptCall` JSON-RPC response or a `callEvent` notification +2. Waits for the call to reach `CONNECTED` state +3. **To send audio** (mic input to WebRTC): write to the virtual input device + - Linux: `paplay --device=sink_for_ audio.wav` + - macOS: `sox audio.wav -t coreaudio ` +4. **To receive audio** (WebRTC playout): read from the virtual output device + - Linux: `parecord --device=.monitor output.wav` + - macOS: `sox -t coreaudio output.wav` +5. Disconnects when the call ends + +--- + +## Startup Sequence + +``` +signal-cli signal-call-tunnel + | | + |-- spawn process ------------------> | + | (config JSON on stdin) | + | | parse config + | | create VirtualAudioDevicePair + | | init RingRTC CallManager + | | start control channel + | | bind ctrl.sock + | | queue "ready" message + | | init cubeb AudioDeviceModule + | | wait for device enumeration + | | select virtual devices by name + | | + |-- connect to ctrl.sock ------------->| + | (retries: 50x @ 200ms) | + |<-------- ready -----------------------| + | {"type":"ready", | + | "inputDeviceName":"...", | + | "outputDeviceName":"..."} | + |-- auth ------------------------------>| + | {"type":"auth","token":""} | + | | constant-time token verify + | | +``` + +Config JSON written to stdin before the process starts: + +```json +{ + "call_id": 12345, + "is_outgoing": true, + "control_socket_path": "/tmp/sc-a1b2c3/ctrl.sock", + "control_token": "dG9rZW4...", + "local_device_id": 1, + "input_device_name": "signal_input", + "output_device_name": "signal_output" +} +``` + +The `input_device_name` and `output_device_name` fields are optional. If +omitted, the tunnel generates per-call names like `signal_input_`. +On macOS, these should match the installed BlackHole driver names. + +--- + +## Control Socket Protocol + +Unix SOCK_STREAM at `ctrl.sock`. Newline-delimited JSON messages. + +### Authentication + +The first message from the parent **must** be an auth message. The token is +a random 32-byte value generated per call and passed in the startup config. + +```json +{"type":"auth","token":""} +``` + +### Parent -> Tunnel + +| Type | When | Fields | +|------|------|--------| +| `auth` | First message | `token` | +| `createOutgoingCall` | Outgoing call setup | `callId`, `peerId` | +| `proceed` | After offer/receivedOffer | `callId`, `hideIp`, `iceServers` | +| `receivedOffer` | Incoming call | `callId`, `peerId`, `opaque`, `age`, `senderDeviceId`, `senderIdentityKey`, `receiverIdentityKey` | +| `receivedAnswer` | Outgoing call answered | `opaque`, `senderDeviceId`, `senderIdentityKey`, `receiverIdentityKey` | +| `receivedIce` | ICE candidates arrive | `candidates` (array of base64 opaque blobs) | +| `accept` | User accepts incoming call | *(none)* | +| `hangup` | End the call | *(none)* | + +### Tunnel -> Parent + +| Type | When | Fields | +|------|------|--------| +| `ready` | Control socket bound, virtual devices created | `inputDeviceName`, `outputDeviceName` | +| `sendOffer` | RingRTC generated an offer | `callId`, `opaque`, `callMediaType` | +| `sendAnswer` | RingRTC generated an answer | `callId`, `opaque` | +| `sendIce` | ICE candidates gathered | `callId`, `candidates` (array of `{"opaque":"..."}`) | +| `sendHangup` | RingRTC wants to hang up | `callId`, `hangupType` | +| `sendBusy` | Line is busy | `callId` | +| `stateChange` | Call state transition | `state`, `reason` (optional) | +| `error` | Something went wrong | `message` | + +Opaque blobs and identity keys are base64-encoded. ICE servers use the format: + +```json +{"urls":["turn:example.com"],"username":"u","password":"p"} +``` + +--- + +## Virtual Audio Devices + +The tunnel uses `VirtualAudioDevicePair` from ringrtc's `virtual_audio` module +to create platform-specific virtual audio devices. + +### PCM parameters + +| Parameter | Value | +|-----------|-------| +| Sample rate | 48,000 Hz | +| Channels | 1 (mono) | +| Sample format | 16-bit signed integer, little-endian | + +### Linux (PulseAudio) + +Virtual devices are PulseAudio null sinks/sources created automatically per +call and torn down on drop. No setup required. + +- **Input device** (client -> WebRTC): write to PulseAudio sink `sink_for_` +- **Output device** (WebRTC -> client): read from PulseAudio monitor `.monitor` + +Example: + +```bash +# Send audio to WebRTC +paplay --device=sink_for_signal_input_12345 tone.wav + +# Record from WebRTC +parecord --device=signal_output_12345.monitor --rate=48000 --channels=1 --format=s16le captured.wav +``` + +### macOS (BlackHole) + +Requires one-time root setup to install BlackHole audio drivers: + +```bash +cd third-party/ringrtc +sudo bin/virtual_audio.sh --setup --input-source signal_input --output-sink signal_output +``` + +This installs drivers in `/Library/Audio/Plug-Ins/HAL/` that persist across +reboots. No root is needed after setup. On macOS, use fixed device names +matching the installed drivers via `input_device_name`/`output_device_name` +in the config. + +Example: + +```bash +# Send audio to WebRTC +sox tone.wav -t coreaudio signal_input repeat - + +# Record from WebRTC +sox -t coreaudio signal_output captured.wav trim 0 5 +``` + +--- + +## Call Flows + +### Outgoing call + +``` +signal-cli signal-call-tunnel Remote Phone + | | | + |-- spawn + config ------->| | + |<-- ready ----------------| | + |-- auth ----------------->| | + |-- createOutgoingCall --->| | + |-- proceed (TURN) ------->| | + | | RingRTC creates offer | + |<-- sendOffer ------------| | + |-- offer via Signal -------------------------------->| + |<-- answer via Signal --------------------------------| + |-- receivedAnswer ------->| (+ identity keys) | + | | x25519 DH + HKDF | + |<-- sendIce --------------| | + |-- ICE via Signal -------------------------------> | + |<-- ICE via Signal -------------------------------- | + |-- receivedIce ---------->| | + | | ICE connects, SRTP up | + |<-- stateChange:Connected | | + | | | + | audio client connects to virtual audio devices | + | |<-- audio --- client | + | |--- audio --> client | +``` + +### Incoming call + +``` +signal-cli signal-call-tunnel Remote Phone + | | | + |<-- offer via Signal --------------------------------| + |-- spawn + config ------->| | + |<-- ready ----------------| | + |-- auth ----------------->| | + |-- receivedOffer -------->| (+ identity keys) | + |-- proceed (TURN) ------->| | + | | RingRTC processes offer | + | | x25519 DH + HKDF | + |<-- sendAnswer -----------| | + |-- answer via Signal -------------------------------->| + |<-- sendIce --------------| | + |-- ICE via Signal ------------------------------> | + |<-- ICE via Signal -------------------------------- | + |-- receivedIce ---------->| | + | | ICE connecting... | + | | | + | (user accepts call) | | + | Java defers accept | | + | | | + |<-- stateChange:Ringing --| (tunnel ready to accept)| + |-- accept --------------->| (deferred accept sent) | + | | RingRTC accepts | + |<-- stateChange:Connected | | + | | | + | audio client connects to virtual audio devices | + | |<-- audio --- client | + | |--- audio --> client | +``` + +### JSON-RPC client perspective + +An external application (bot, UI, test script) interacts via JSON-RPC only. +It never touches the control socket directly. + +``` +JSON-RPC Client signal-cli daemon + | | + |-- startCall(recipient) ------------->| + |<-- {callId, state, -| + | inputDeviceName, | + | outputDeviceName} | + | | + |<-- callEvent: RINGING_OUTGOING ------| + | ... remote answers ... | + |<-- callEvent: CONNECTED -------------| + | | + | connect to virtual audio devices | + | (via PulseAudio/CoreAudio) | + | | + |-- hangupCall(callId) --------------->| (or: receive callEvent ENDED) + |<-- callEvent: ENDED -----------------| + | disconnect from audio devices | +``` + +For incoming calls: + +``` +JSON-RPC Client signal-cli daemon + | | + |<-- callEvent: RINGING_INCOMING ------| (includes callId, device names) + | | + |-- acceptCall(callId) --------------->| + |<-- {callId, state, -| + | inputDeviceName, | + | outputDeviceName} | + | | + |<-- callEvent: CONNECTING ------------| + |<-- callEvent: CONNECTED -------------| + | | + | connect to virtual audio devices | + | (via PulseAudio/CoreAudio) | +``` + +--- + +## Audio Client Integration Guide + +### When to connect + +Connect to the virtual audio devices **after** the call reaches `CONNECTED` +state. Device names are returned in the `startCall`/`acceptCall` response and +in `callEvent` notifications. + +You can connect earlier (the devices exist from tunnel startup), but no +meaningful audio will flow until ICE completes and the call is connected. + +### Sending audio (recording path) + +Write audio to the virtual input device using platform APIs. The tunnel's +cubeb recording captures from this device and feeds it to WebRTC. + +- **Linux**: `paplay --device=sink_for_ audio.wav` +- **macOS**: `sox audio.wav -t coreaudio ` + +If you have nothing to send (muted), simply don't write. WebRTC's Opus DTX +will detect silence and send minimal comfort noise packets. + +### Receiving audio (playout path) + +Read audio from the virtual output device using platform APIs. WebRTC receives +and Opus-decodes remote audio, cubeb plays it to the virtual output device, +and you capture it from the monitor/device. + +- **Linux**: `parecord --device=.monitor output.wav` +- **macOS**: `sox -t coreaudio output.wav` + +--- + +## Encryption and Key Derivation + +The tunnel subprocess handles all encryption. Neither the Java parent nor the +audio client ever sees SRTP keys or encrypted media. + +RingRTC uses a custom key derivation scheme (not DTLS-SRTP): + +1. Each side generates an ephemeral x25519 keypair +2. Public keys are embedded in the opaque offer/answer blobs +3. x25519 DH produces a shared secret +4. HKDF-SHA256 derives SRTP keys with info string: + `Signal_Calling_20200807_SignallingDH_SRTPKey_KDF || caller_identity || callee_identity` +5. Keys are injected into WebRTC with DTLS disabled + +The identity keys are **not** inside the opaque blobs. They are passed +separately via the control protocol (`senderIdentityKey`, `receiverIdentityKey`) +and come from the Signal protocol message envelope. + +Identity keys in `senderIdentityKey` and `receiverIdentityKey` must be **raw +32-byte Curve25519 public keys** (without the 0x05 DJB type prefix). Signal +Android strips this prefix via `WebRtcUtil.getPublicKeyBytes()`. If the 33-byte +serialized form is used instead, SRTP key derivation produces different keys on +each side, causing `srtp_err_status_auth_fail`. + +--- + +## Implementation Notes + +### Peer ID consistency + +The `peerId` field in `createOutgoingCall` and `receivedOffer` must be the actual +remote peer UUID (e.g., `senderAddress.toString()`). RingRTC's +`compare_remotes()` rejects ICE candidates if the peer ID doesn't match across +calls, causing "Ignoring peer-reflexive ICE candidate because the ufrag is +unknown." + +### sendHangup semantics + +`sendHangup` from the tunnel is a request to send a hangup message via Signal +protocol. It is **not** a local state change -- local state transitions come +exclusively from `stateChange` events. For single-device clients, ignore +`AcceptedOnAnotherDevice`, `DeclinedOnAnotherDevice`, and +`BusyOnAnotherDevice` hangup types in the `hangupType` field -- sending these to +the remote peer causes it to terminate the call prematurely. + +### Call ID serialization + +Call IDs can exceed `Long.MAX_VALUE` in Java. Use `Long.toUnsignedString()` when +serializing to JSON for the tunnel (which expects `u64`). In the config JSON, +`call_id` should also use unsigned representation. + +### Incoming hangup filtering + +When receiving hangup messages via Signal protocol, only honor `NORMAL` type +hangups. `ACCEPTED`, `DECLINED`, and `BUSY` types are multi-device coordination +messages and should be ignored by single-device clients. + +### JSON-RPC call ID types + +JSON-RPC clients may send call IDs as various numeric types (Long, BigInteger, +Integer). Use `Number.longValue()` rather than direct casting when extracting +call IDs from JSON-RPC parameters. + +### VirtualAudioDevicePair lifecycle + +The `VirtualAudioDevicePair` is kept alive for the duration of the tunnel +process. On drop: +- **Linux**: calls `pactl unload-module` to clean up PulseAudio modules +- **macOS**: logs a warning (can't teardown without root) but BlackHole drivers + persist for the next call, which is the intended behavior + +--- + +## State Machine + +Call states as seen by JSON-RPC clients (mapped from RingRTC internal states): + +``` + startCall() + | + v + +----- RINGING_OUTGOING ----+ RINGING_INCOMING -----+ + | | | | | + | (timeout | (answered) | (rejected) | acceptCall() | (timeout + | ~60s) | | | | ~60s) + v v v v v + ENDED CONNECTED ENDED CONNECTING ENDED + | | + | v + | CONNECTED + | | + | (hangup/error) | (hangup/error) + v v + ENDED ENDED +``` + +For outgoing calls, `CONNECTED` fires directly when the tunnel reports +`Connected` state -- there is no intermediate `CONNECTING` event. + +For incoming calls, `CONNECTING` is set by Java when the user calls +`acceptCall()`, before the tunnel completes ICE negotiation. + +Both directions have a 60-second ring timeout. + +Reconnection (ICE restart): + +``` + CONNECTED --> RECONNECTING --> CONNECTED (ICE restart succeeded) + | + v + ENDED (ICE restart failed) +``` + +`RECONNECTING` maps from the tunnel's `Connecting` state, which RingRTC +emits during ICE restarts (not during initial connection). + +--- + +## File Layout + +``` +/tmp/sc-/ + ctrl.sock control socket (signal-cli <-> tunnel) +``` + +The control socket is created with mode `0700` on the parent directory. The +directory and its contents are deleted when the call ends. + +The `signal-call-tunnel` binary is located by searching (in order): + +1. `SIGNAL_CALL_TUNNEL_BIN` environment variable +2. `/bin/signal-call-tunnel` +3. `signal-call-tunnel` on `PATH` + +--- + +## Building + +```bash +# Build signal-cli +./gradlew installDist + +# Build the Rust call tunnel +cd signal-call-tunnel && cargo build --release && cd .. +``` + +The first Rust build downloads a prebuilt WebRTC library (~100 MB) from +Signal's artifact server. Subsequent builds use the cached copy. + +### Prerequisites + +- **macOS**: Install BlackHole virtual audio drivers (one-time, requires root): + ```bash + cd third-party/ringrtc + sudo bin/virtual_audio.sh --setup --input-source signal_input --output-sink signal_output + ``` + Install `sox` for audio playback/recording in tests: `brew install sox` + +- **Linux**: PulseAudio must be running. Virtual audio modules are created + automatically per call.