The SDK pipeline is clocked by the input arrival interval: measured ~4 s
real-frame latency at 1 s keep-alive spacing vs ~58 ms back-to-back
(depth-4 pipeline x arrival gap). Gating keep-alives while a request is
outstanding also starved the SDK input source (it waited instead of
yielding) and broke the stream. Short 50 ms keep-alives keep the ring
rotating, so real frames get results in tens of ms; the zeros carry no
request token and are dropped by the content-based pump.
Keep-alive zeros injected during a pending request queue in the SDK
pipeline ahead of the real frame and delay its result by one slot each:
observed 3 s real-frame latency with 1 s keep-alive spacing vs 14 ms
for back-to-back real frames (and 10 ms delivered by the FIFO path).
The feeder now skips keep-alives whenever a request id is still
outstanding, so real frames are processed back-to-back and true-idle
behaviour (stream kept alive) is preserved.
The SDK does not tag results with a per-frame id (image_id is always
empty, src_timestamp is a pipeline-clock value), so pairing relied on
FIFO ordering, which a buffered primer/keep-alive result could break:
the stale result would pop the pending id and be delivered to the
wrong camera/crop (synthetic results consuming real IDs).
Every real frame now carries a 32-bit request token written into a
small top-left pixel band; the SDK returns the source frame back 1:1
(verified on hardware: BGR pass-through exact, BGRA), so the pump
decodes the token from the result image and delivers the outcome only
to the frame that carries it (content-based, no ordering assumption).
Primer/keep-alive frames carry no token; their results are dropped
even when they surface while a request is pending.
The token-based 1:1 pairing (None token per synthetic frame) is correct in
theory but regressed in production: inference latency went to ~3s and
detections dropped to ~0. Revert to the stable approach that has been
running fine (lock + FIFO + keep-alive): ~7ms inference, detections flowing.
P1 (synthetic results stealing real ids) remains a theoretical concern
under this scheme, but the observed production behavior with it was far
worse. Investigate the regression with per-frame diagnostics before
re-attempting.
Misattribution reproduced at night: when Frigate stops sending frames
during quiet periods the queue empties and keep-alive frames are emitted;
a pending real-frame result then races the keep-alive's (empty) result,
which steals the real id (frame/detections shown under the WRONG camera).
Fix: synthesize keep-alive frames ONLY until the first stream result
(_primed), then never again - no unregistered result can exist in the
pipeline afterwards. Queue-empty after init just waits; under Frigate's
continuous feed this branch is only hit during true quiet periods, where
there is no pending request to misattribute anyway.
The primer 'drain by run index' was too fragile under Frigate's real
workload: with 7 cameras starting simultaneously the first few real frames
arrive before the SDK finishes emitting primer results, their ids were
drained without a result, run() hit the timeout and Frigate restarted the
detect process every ~15s (zero detections).
Revert to the stable approach: keep-alive frames sustain the SDK during
quiet/init, _pump never consumes a pending id while the queue is empty, and
run() is serialized (lock) so at most one real frame is in flight. The
remaining theoretical primer/keep-alive misattribution window only exists
when the queue is empty mid-run, which does not happen under Frigate's
continuous feed.
The SDK can buffer primer results and emit them AFTER the first real frame
is already pending; the FIFO pump then popped a real id and stored the
skipped/buffered (empty) result under it, dropping the real one. This
misattributes detections across cameras even with run() serialized.
- eliminate keep-alive frames entirely (Frigate feeds the queue
continuously; nothing to keep alive) - a synthetic frame's result can
arrive during a pending real id,
- drain the primer results by index BEFORE any pending-id bookkeeping, so
they can never consume a request id; the first run() waits up to 0.5s
for the primer drain.
Voyager SDK does not tag results with a per-frame id and can reorder or
buffer stream results under sustained load (observed on live with many
thousands of frames: camera A started receiving boxes/paths from camera B).
With several frames in flight, the FIFO pairing can store a result under a
different frame's id.
run() now holds a lock so at most one frame is in flight; the next stream
result can only belong to the frame just pushed, making pairing exact
regardless of SDK reordering. Pipeline throughput stays far below limits
(~9ms per inference vs 28ms available per frame at 7x5fps).
- strict per-frame request/result pairing: the feeder registers a queue of
pending frame ids, the pump pops one per stream result and stores the
outcome under it, and run() waits for its own frame's id. Primer and
keep-alive results are dropped instead of being served to callers.
- copy frames out of shared memory before queuing them (detect_raw returns
before the feeder consumes the frame, unmapping the camera buffer).
- raise on inference timeout so the watchdog can restart a stalled stream
instead of silently serving zeros.
- use ModelDownloader.download_from_url for URL models (.part file, atomic
rename, progress events); drop the unused AX_DEFAULT_URL constant.
- is_url() requires a scheme; treat www.-style values as paths and fail with
a clear FileNotFoundError instead of a deep urlretrieve error, mirroring
hailo8l behavior for missing model files.
- guard detections against numpy arrays, drop the dead score initialization,
floor normalized box coords at 0, move the model-required check before
super().__init__(), and document detect_raw/close.
- deduplicate the model-compilation walkthrough in docs (keep installation.md
as the single source; object_detectors.md links to it).
The Metis has no utilization readout without the axsystemserver broker,
which cannot run alongside a live detector. Report the worker busy time
(fps * inference time) as a real load indicator instead, plus the board
temperature, under npu_usages[axelera].
Exposes the Metis temperature read (axcmd --board-temp) under
npu_usages so the /api/stats endpoint lists the NPU instead of
returning null. There is no utilization readout without the
axsystemserver broker, which cannot run alongside a live detector.
Use the SDK's axcmd --board-temp through the board controller, which is
safe to call while an inference stream is active. Matches the hailo8l
pattern of exposing temperature via detectors.*.temperature.
Memory audit: all hot-path structures are bounded (frame queue maxsize 64,
latest result is a single 20x6 buffer, rows capped at MAX_ROWS), so there is
no unbounded growth in steady state.
One resource hazard: run() used a blocking queue.put() with no guard. If the
pump thread dies (SDK stream exception, e.g. DEVICE_LOST), the SDK side stops
draining the queue, the bounded queue fills, and every subsequent run() blocks
forever on put, permanently wedging the detector (Frigate does not restart the
detector process on its own).
Fix: check thread liveness before put and raise immediately, and use put_nowait
(backpressure: drop the frame on a full queue) so run() always falls through to
the latest-result wait / timeout instead of blocking.
* installation.md: replace the hand-waved 'custom image' note with a
reference Dockerfile that installs the Voyager SDK into the image
(apt deps, axelera-rt/devkit from the Axelera index, make operators)
and keeps the /dev/metis passthrough instructions.
* object_detectors.md: document axelera model sourcing under
'Downloading YOLO Models' (model-zoo preset, axzoo/axcompile, HF
hosting, default model auto-download).
Other detector plugins shipping in this tree do not carry unit tests, and
the fixed-sleep timing in these tests (primer, keepalive, pump teardown)
is prone to flaking on CI runners. The regression coverage stays out of
tree beside the detector host worker.
Frigate detect() interprets detector boxes as fractions of the region and
discards any box resolving outside the frame. The plugin returned pixel
coordinates, so every detection was silently filtered out and no events
were ever created despite a working inference chain. Normalize by model
width/height (same contract as memryx/onnx). Live: 0 events -> 27 events
in ~70 s after the fix.
Per-frame request/result pairing (RequestStore/ResponseStore FIFO)
desynchronises on the live chain once keep-alive results interleave with
real frames: a request could receive a zero-frame result, and real results
got attributed to the wrong frame (recurring identical scores across
different frames in production). The pump now keeps the most recent
completed result and run() waits for a result newer than the sequence at
call time, then returns it. Frigate correlates objects across frames, so
per-frame identity is not needed. Worker engine mirrored.
The SDK returns detection boxes as numpy arrays on some frames; the
'list(box or [])' idiom evaluated array truthiness and raised ValueError,
killing the pump thread mid-detection. Extract boxes defensively. Live
repro: worker crashed the moment a real detection fired, so no event was
ever recorded.
Frigate feeds detect_raw() a batched NHWC frame (1, H, W, C); the plugin
raised ValueError for ndim != 3, so every real inference failed. Squeeze
the leading batch dim when present (same contract as other IPU plugins,
e.g. memryx). Regression test added.
Community plugin for the Axelera AI Metis PCIe/M.2 NPU, following the same
conventions as axengine.py / hailo8l.py (lazy SDK import,
RequestStore/ResponseStore, model path as URL/local-file/model-zoo preset,
type_key auto-registration).
Inference runs through a single Voyager SDK stream
(create_inference_stream(network=..., sources=[...])) fed with numpy frames;
detection labels are mapped to class ids by name via the merged labelmap.
Compiled .axm models can be downloaded into the model cache from a URL, used
from a local path, or omitted to fall back to the yolov8n-coco zoo preset.
Includes unit tests (unittest, no SDK/hardware required) covering the engine
round-trip, timeout, dead-stream and orphan behaviour plus model resolution,
and docs for the object detector page and the "Extra steps for specific
hardware" installation page.
* subscribe to add in webpush
* add docs for detector cpu usage
* rebuild notification camera access when a camera is added at runtime
* document how frigate shows CPU usage metrics
* add faq about version key in config
sanitize_filename leaves ".." intact and collapses variants like "..:" and "..*" to "..", so filesystem paths built from face names, classification model/category names, image ids, and trigger data could escape their base directory. Route every such site through new frigate/util/path.py helpers (safe_join, sanitize_path_component, sanitize_contained_path), which reject traversal and verify containment.
Worst case was DELETE /classification/{name}, which rmtree'd /media/frigate and /config while returning 200.
Important to note that all affected endpoints already require admin permission, so this sould be considered hardening rather than fixing exploitable code.
<camera>/notifications/suspended arrives as a string over the live connection but as a number in the camera_activity snapshot, and the truthiness guard dropped the numeric 0, so a camera with notifications off rendered as active after a reload. Normalize to a string and derive isSuspended instead of storing it.
* fix(audio): correct sodeling typo to yodeling
Fixes a typo in audio-labelmap.txt where the yodeling class was
misspelled as "sodeling".
* fix(i18n): remove duplicate sodeling key in en audio.json
The en audio.json already contains a correct "yodeling" key. Remove
the duplicate/misspelled "sodeling" entry to avoid ambiguity.
* add host npu requirements to docs
* allow toggling live audio transcription via mqtt
* improve spacing consistency on mobile drawers
* fix clearing the region grid not surviving a restart
Currently translated at 100.0% (800 of 800 strings)
Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 100.0% (474 of 474 strings)
Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 100.0% (1295 of 1295 strings)
Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 100.0% (808 of 808 strings)
Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 100.0% (474 of 474 strings)
Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 100.0% (808 of 808 strings)
Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 100.0% (1295 of 1295 strings)
Translated using Weblate (Chinese (Simplified Han script))
Currently translated at 100.0% (129 of 129 strings)
Co-authored-by: GuoQing Liu <842607283@qq.com>
Co-authored-by: Hosted Weblate <hosted@weblate.org>
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/config-cameras/zh_Hans/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/config-global/zh_Hans/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/objects/zh_Hans/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-settings/zh_Hans/
Translation: Frigate NVR/Config - Cameras
Translation: Frigate NVR/Config - Global
Translation: Frigate NVR/objects
Translation: Frigate NVR/views-settings
Currently translated at 51.1% (662 of 1295 strings)
Translated using Weblate (Swedish)
Currently translated at 100.0% (109 of 109 strings)
Translated using Weblate (Swedish)
Currently translated at 50.7% (657 of 1295 strings)
Co-authored-by: Hosted Weblate <hosted@weblate.org>
Co-authored-by: Mats Lojander <mats@lojander.com>
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-dialog/sv/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-settings/sv/
Translation: Frigate NVR/components-dialog
Translation: Frigate NVR/views-settings
Currently translated at 86.2% (94 of 109 strings)
Translated using Weblate (Czech)
Currently translated at 32.5% (421 of 1295 strings)
Translated using Weblate (Czech)
Currently translated at 85.3% (93 of 109 strings)
Translated using Weblate (Czech)
Currently translated at 100.0% (50 of 50 strings)
Translated using Weblate (Czech)
Currently translated at 100.0% (239 of 239 strings)
Translated using Weblate (Czech)
Currently translated at 99.8% (500 of 501 strings)
Translated using Weblate (Czech)
Currently translated at 99.8% (500 of 501 strings)
Co-authored-by: Hosted Weblate <hosted@weblate.org>
Co-authored-by: Matěj Kratochvíl <matejkratochvilbilina@gmail.com>
Co-authored-by: MiraCatsy <catsycatsymira@gmail.com>
Co-authored-by: romanslezar <roman.slezar@centrum.cz>
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/audio/cs/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/common/cs/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-camera/cs/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-dialog/cs/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-settings/cs/
Translation: Frigate NVR/audio
Translation: Frigate NVR/common
Translation: Frigate NVR/components-camera
Translation: Frigate NVR/components-dialog
Translation: Frigate NVR/views-settings
Currently translated at 85.1% (1103 of 1295 strings)
Translated using Weblate (Russian)
Currently translated at 76.1% (83 of 109 strings)
Translated using Weblate (Russian)
Currently translated at 71.7% (929 of 1295 strings)
Translated using Weblate (Russian)
Currently translated at 100.0% (808 of 808 strings)
Translated using Weblate (Russian)
Currently translated at 81.4% (386 of 474 strings)
Translated using Weblate (Russian)
Currently translated at 56.2% (728 of 1295 strings)
Translated using Weblate (Russian)
Currently translated at 65.3% (528 of 808 strings)
Translated using Weblate (Russian)
Currently translated at 50.4% (239 of 474 strings)
Co-authored-by: Artem Vladimirov <artyomka71@mail.ru>
Co-authored-by: Hosted Weblate <hosted@weblate.org>
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/components-dialog/ru/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/config-cameras/ru/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/config-global/ru/
Translate-URL: https://hosted.weblate.org/projects/frigate-nvr/views-settings/ru/
Translation: Frigate NVR/Config - Cameras
Translation: Frigate NVR/Config - Global
Translation: Frigate NVR/components-dialog
Translation: Frigate NVR/views-settings
