mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-10-18 07:29:45 +00:00
docs: Axelera Metis detector (install, config, model catalog, downloads)
This commit is contained in:
patrykorwat
parent
df95482655
commit
950ea9406c
@ -1205,3 +1205,33 @@ axengine:
|
||||
input_dtype: int
|
||||
input_pixel_format: bgr
|
||||
labelmap_path: /labelmap/coco-80.txt
|
||||
axelera:
|
||||
title: Axelera Metis
|
||||
models:
|
||||
- key: yolox
|
||||
label: YOLOX
|
||||
recommended: true
|
||||
download: |-
|
||||
Axelera models must be compiled for the Metis NPU with the Voyager SDK before Frigate can run them (compilation is not supported inside the container). Compile a model with the SDK, place the compiled output directory under `/config`, and point the model's `path` at it. The default recommendation is the Apache-2.0 licensed YOLOX (Megvii YOLOX, e.g. the `yolox-s-coco-onnx` model-zoo network).
|
||||
ui: |-
|
||||
Navigate to **Settings > System > Detection models** and select **Axelera Metis** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:
|
||||
|
||||
| Field | Value |
|
||||
| ---------------------------------------- | ------------------------------------------------------------ |
|
||||
| **Custom object detector model path** | `/config/model_cache/axelera/yolox-s-coco-onnx` |
|
||||
| **Label map for custom object detector** | `/labelmap/coco-80.txt` |
|
||||
| **Object detection model input width** | `704` (the frame size Frigate sends; boxes map back to it) |
|
||||
| **Object detection model input height** | `576` |
|
||||
| **Model Input Pixel Color Format** | `rgb` (Frigate's default value) |
|
||||
| **Model Input Tensor Shape** | `nhwc` (Frigate's default value) |
|
||||
| **Model Input D Type** | `int` (Frigate's default value) |
|
||||
| **Object Detection Model Type** | `yolox` |
|
||||
yaml: |-
|
||||
models:
|
||||
- devices:
|
||||
- axelera:PCIe
|
||||
path: /config/model_cache/axelera/yolox-s-coco-onnx
|
||||
model_type: yolox
|
||||
width: 704
|
||||
height: 576
|
||||
labelmap_path: /labelmap/coco-80.txt
|
||||
|
||||
@ -60,6 +60,10 @@ Frigate supports multiple different detectors that work on different types of ha
|
||||
|
||||
- [AXEngine](#axera): axmodels can run on AXERA AI acceleration.
|
||||
|
||||
**Axelera** <CommunityBadge />
|
||||
|
||||
- [Axelera Metis](#axelera-metis): The Metis AI Accelerator is available in M.2 and PCIe formats, running compiled YOLO models through the vendor runtime.
|
||||
|
||||
**For Testing**
|
||||
|
||||
- [CPU Detector (not recommended for actual use](#cpu-detector-not-recommended): Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results.
|
||||
@ -97,7 +101,7 @@ models:
|
||||
- openvino:GPU
|
||||
```
|
||||
|
||||
Coral EdgeTPU and MemryX accelerators can only be opened by one process, so those devices can not be repeated.
|
||||
Coral EdgeTPU, MemryX, and Axelera Metis accelerators can only be opened by one process, so those devices can not be repeated.
|
||||
|
||||
### Running more than one model
|
||||
|
||||
@ -851,3 +855,45 @@ The AXEngine python package is not part of the Frigate image. It is downloaded a
|
||||
When configuring the AXEngine detector, you have to specify the model name.
|
||||
|
||||
<ModelConfigDropdown detectorTitle="AXEngine" models={objectDetectorsModels.axengine.models} />
|
||||
|
||||
---
|
||||
|
||||
## Axelera Metis
|
||||
|
||||
<CommunityBadge />
|
||||
|
||||
Hardware accelerated object detection is supported on the Axelera Metis AI Accelerator, available in M.2 and PCIe card formats. Inference runs through Axelera's low-level runtime (`axelera.runtime`), which executes compiled YOLO-family models with a flat per-frame latency.
|
||||
|
||||
See the [installation docs](../frigate/installation.md#axelera-metis) for information on configuring the Metis hardware.
|
||||
|
||||
:::info
|
||||
|
||||
The Axelera runtime is not part of the Frigate image. It is downloaded and installed into `/config/.local` the first time an Axelera detector is configured, verified against pinned checksums, and updated automatically when a Frigate release pins a new version. If the container has no internet access, see [Detector runtimes](/frigate/network_requirements#detector-runtimes) for how to provide the files yourself.
|
||||
|
||||
:::
|
||||
|
||||
### Configuration {#configuration-axelera}
|
||||
|
||||
<ModelConfigDropdown detectorTitle="Axelera Metis" models={objectDetectorsModels.axelera.models} />
|
||||
|
||||
#### Using a Custom Model
|
||||
|
||||
Models must be compiled for the Metis NPU with the Axelera Voyager SDK before Frigate can run them. The runtime consumes the compiled output directory, which contains `1/model.json`, `1/manifest.json`, and the baked `1/postprocess_graph.onnx`.
|
||||
|
||||
Compile the model on a machine with the Voyager SDK installed (compilation is not supported inside the Frigate container), for example with the SDK's `deploy.py` against a model-zoo yaml. Then either:
|
||||
|
||||
- place the compiled directory under `/config` and point the model's `path` at it, or
|
||||
- zip the compiled directory and point `path` at the zip (bind-mounted, or at a URL).
|
||||
|
||||
```yaml
|
||||
models:
|
||||
- devices:
|
||||
- axelera:PCIe
|
||||
path: /config/model_cache/axelera/yolox-s-coco-onnx
|
||||
width: 704 # the frame size Frigate sends to the model
|
||||
height: 576
|
||||
model_type: yolox
|
||||
labelmap_path: /labelmap/coco-80.txt
|
||||
```
|
||||
|
||||
A bare preset name (no slash or dot, e.g. `yolox-s-coco-onnx`) is also accepted and resolved under `AXELERA_BUILD_ROOT` (default `/opt/voyager-sdk/build`) for setups that build models into the image.
|
||||
|
||||
@ -503,6 +503,41 @@ If you are using `docker run`, add this option to your command `--device /dev/ax
|
||||
|
||||
Finally, configure [hardware object detection](/configuration/object_detectors#axera) to complete the setup.
|
||||
|
||||
### Axelera Metis
|
||||
|
||||
The Axelera Metis AI Accelerator is available in the M.2 and PCIe card form factors and is supported on x86 (Intel/AMD) PCs and Rock 5B class arm64 boards.
|
||||
|
||||
The Axelera runtime used inside the container is not part of the Frigate image; Frigate downloads and installs it at first start once an Axelera detector is configured. Containers without internet access can provide the files themselves, see [Detector runtimes](/frigate/network_requirements#detector-runtimes). The host side kernel driver still has to be installed as described below.
|
||||
|
||||
#### Installation
|
||||
|
||||
Install the Metis PCIe kernel driver and firmware on the **host** following [Axelera's driver installation guide](https://developer.axelera.ai/axelera-documentation/Getting-Started/GettingStartedWithMetis/InstallingAxeleraMetisDriver). The driver creates the `/dev/metis*` device nodes. Reboot after installing it.
|
||||
|
||||
#### Setup
|
||||
|
||||
To set up Frigate, follow the default installation instructions, for example: `ghcr.io/blakeblackshear/frigate:stable`
|
||||
|
||||
Next, grant Docker permissions to access your hardware by adding the following lines to your `docker-compose.yml` file:
|
||||
|
||||
```yaml
|
||||
devices:
|
||||
- /dev/dma_heap/system
|
||||
# the exact node is printed by `ls /dev/metis*`, e.g. /dev/metis-0-1-0
|
||||
- /dev/metis-0-1-0
|
||||
```
|
||||
|
||||
If you are using `docker run`, add `--device /dev/dma_heap/system --device /dev/metis-0-1-0` to your command.
|
||||
|
||||
:::warning
|
||||
|
||||
DMA-BUF must be available on the host kernel. On Debian/Ubuntu this is provided by the default kernel; on other systems it may need enabling (`CONFIG_DMABUF_HEAPS_SYSTEM`). If Frigate's log reports that no DMA heap can be allocated, check that `/dev/dma_heap/system` exists on the host and is passed into the container.
|
||||
|
||||
:::
|
||||
|
||||
#### Configuration
|
||||
|
||||
Finally, configure [hardware object detection](/configuration/object_detectors#axelera-metis) to complete the setup.
|
||||
|
||||
## Docker
|
||||
|
||||
Running through Docker with Docker Compose is the recommended install method.
|
||||
|
||||
@ -65,6 +65,7 @@ The SDKs for a few hardware detectors are not shipped in the Frigate image. They
|
||||
| [Hailo 8 / 8L](/configuration/object_detectors#hailo-8) | 4.21.0 | `hailort-debian12-amd64.tar.gz` and `hailort-4.21.0-cp311-cp311-linux_x86_64.whl` on x86, `hailort-debian12-arm64.tar.gz` and `hailort-4.21.0-cp311-cp311-linux_aarch64.whl` on arm64 | [GitHub release](https://github.com/frigate-nvr/hailort/releases/tag/v4.21.0) |
|
||||
| [MemryX MX3](/configuration/object_detectors#memryx-mx3) | 2.1.0 | `mx_accl_frigate-2.1.0.zip` (the release source archive, renamed) | [GitHub archive](https://github.com/memryx/mx_accl_frigate/archive/refs/tags/v2.1.0.zip) |
|
||||
| [AXERA AXEngine](/configuration/object_detectors#axera) | 0.1.3 | `axengine-0.1.3-py3-none-any.whl` | [GitHub release](https://github.com/AXERA-TECH/pyaxengine/releases/tag/0.1.3-frigate) |
|
||||
| [Axelera Metis](/configuration/object_detectors#axelera-metis) | 1.8.0 | `axelera_runtime-1.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl` and `axelera_runtime2-0.2.0-cp311-cp311-manylinux_2_28_x86_64.whl` on x86, `axelera_runtime-1.8.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl` and `axelera_runtime2-0.2.0-cp311-cp311-manylinux_2_28_aarch64.whl` on arm64, plus `axelera_firmware-1.8.0-py3-none-any.whl` on both | [Axelera Artifactory PyPI](https://software.axelera.ai/artifactory/#web/ui/redirect/pypi/axelera-pypi/simple) |
|
||||
|
||||
If the container cannot reach GitHub, provide the files yourself:
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user