docs/docs/configuration/face_recognition.md

@@ -24,7 +24,7 @@ Frigate needs to first detect a `person` before it can detect and recognize a fa
 Frigate has support for two face recognition model types:
 
 - **small**: Frigate will run a FaceNet embedding model to recognize faces, which runs locally on the CPU. This model is optimized for efficiency and is not as accurate.
-- **large**: Frigate will run a large ArcFace embedding model that is optimized for accuracy. It is only recommended to be run when an integrated or dedicated GPU / NPU is available.
+- **large**: Frigate will run a large ArcFace embedding model that is optimized for accuracy. It is only recommended to be run when an integrated or dedicated GPU is available.
 
 In both cases, a lightweight face landmark detection model is also used to align faces before running recognition.
 
@@ -34,7 +34,7 @@ All of these features run locally on your system.
 
 The `small` model is optimized for efficiency and runs on the CPU, most CPUs should run the model efficiently.
 
-The `large` model is optimized for accuracy, an integrated or discrete GPU / NPU is required. See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation.
+The `large` model is optimized for accuracy, an integrated or discrete GPU is required. See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_enrichments.md) documentation.
 
 ## Configuration
 

docs/docs/configuration/hardware_acceleration_enrichments.md

@@ -5,11 +5,11 @@ title: Enrichments
 
 # Enrichments
 
-Some of Frigate's enrichments can use a discrete GPU / NPU for accelerated processing.
+Some of Frigate's enrichments can use a discrete GPU for accelerated processing.
 
 ## Requirements
 
-Object detection and enrichments (like Semantic Search, Face Recognition, and License Plate Recognition) are independent features. To use a GPU / NPU for object detection, see the [Object Detectors](/configuration/object_detectors.md) documentation. If you want to use your GPU for any supported enrichments, you must choose the appropriate Frigate Docker image for your GPU / NPU and configure the enrichment according to its specific documentation.
+Object detection and enrichments (like Semantic Search, Face Recognition, and License Plate Recognition) are independent features. To use a GPU for object detection, see the [Object Detectors](/configuration/object_detectors.md) documentation. If you want to use your GPU for any supported enrichments, you must choose the appropriate Frigate Docker image for your GPU and configure the enrichment according to its specific documentation.
 
 - **AMD**
 
@@ -23,9 +23,6 @@ Object detection and enrichments (like Semantic Search, Face Recognition, and Li
   - Nvidia GPUs will automatically be detected and used for enrichments in the `-tensorrt` Frigate image.
   - Jetson devices will automatically be detected and used for enrichments in the `-tensorrt-jp6` Frigate image.
 
-- **RockChip**
-  - RockChip NPU will automatically be detected and used for semantic search v1 and face recognition in the `-rk` Frigate image.
-
 Utilizing a GPU for enrichments does not require you to use the same GPU for object detection. For example, you can run the `tensorrt` Docker image for enrichments and still use other dedicated hardware like a Coral or Hailo for object detection. However, one combination that is not supported is TensorRT for object detection and OpenVINO for enrichments.
 
 :::note

docs/docs/configuration/semantic_search.md

@@ -78,7 +78,7 @@ Switching between V1 and V2 requires reindexing your embeddings. The embeddings
 
 ### GPU Acceleration
 
-The CLIP models are downloaded in ONNX format, and the `large` model can be accelerated using GPU / NPU hardware, when available. This depends on the Docker build that is used. You can also target a specific device in a multi-GPU installation.
+The CLIP models are downloaded in ONNX format, and the `large` model can be accelerated using GPU hardware, when available. This depends on the Docker build that is used. You can also target a specific device in a multi-GPU installation.
 
 ```yaml
 semantic_search:
@@ -90,7 +90,7 @@ semantic_search:
 
 :::info
 
-If the correct build is used for your GPU / NPU and the `large` model is configured, then the GPU / NPU will be detected and used automatically. 
+If the correct build is used for your GPU and the `large` model is configured, then the GPU will be detected and used automatically. 
 Specify the `device` option to target a specific GPU in a multi-GPU system (see [onnxruntime's provider options](https://onnxruntime.ai/docs/execution-providers/)). 
 If you do not specify a device, the first available GPU will be used.
 

frigate/embeddings/onnx/runner.py

@@ -4,12 +4,10 @@ import logging
 import os.path
 from typing import Any
 
-import numpy as np
 import onnxruntime as ort
 
 from frigate.const import MODEL_CACHE_DIR
 from frigate.util.model import get_ort_providers
-from frigate.util.rknn_converter import auto_convert_model, is_rknn_compatible
 
 try:
     import openvino as ov
@@ -27,33 +25,7 @@ class ONNXModelRunner:
         self.model_path = model_path
         self.ort: ort.InferenceSession = None
         self.ov: ov.Core = None
-        self.rknn = None
-        self.type = "ort"
-
-        try:
-            if device != "CPU" and is_rknn_compatible(model_path):
-                # Try to auto-convert to RKNN format
-                rknn_path = auto_convert_model(model_path)
-                if rknn_path:
-                    try:
-                        self.rknn = RKNNModelRunner(rknn_path, device)
-                        self.type = "rknn"
-                        logger.info(f"Using RKNN model: {rknn_path}")
-                        return
-                    except Exception as e:
-                        logger.debug(
-                            f"Failed to load RKNN model, falling back to ONNX: {e}"
-                        )
-                        self.rknn = None
-        except ImportError:
-            pass
-
-        # Fall back to standard ONNX providers
-        providers, options = get_ort_providers(
-            device == "CPU",
-            device,
-            requires_fp16,
-        )
+        providers, options = get_ort_providers(device == "CPU", device, requires_fp16)
         self.interpreter = None
 
         if "OpenVINOExecutionProvider" in providers:
@@ -83,9 +55,7 @@ class ONNXModelRunner:
             )
 
     def get_input_names(self) -> list[str]:
-        if self.type == "rknn":
-            return self.rknn.get_input_names()
-        elif self.type == "ov":
+        if self.type == "ov":
             input_names = []
 
             for input in self.interpreter.inputs:
@@ -97,9 +67,7 @@ class ONNXModelRunner:
 
     def get_input_width(self):
         """Get the input width of the model regardless of backend."""
-        if self.type == "rknn":
-            return self.rknn.get_input_width()
-        elif self.type == "ort":
+        if self.type == "ort":
             return self.ort.get_inputs()[0].shape[3]
         elif self.type == "ov":
             input_info = self.interpreter.inputs
@@ -122,10 +90,8 @@ class ONNXModelRunner:
                     return -1
         return -1
 
-    def run(self, input: dict[str, Any]) -> Any | None:
-        if self.type == "rknn":
-            return self.rknn.run(input)
-        elif self.type == "ov":
+    def run(self, input: dict[str, Any]) -> Any:
+        if self.type == "ov":
             infer_request = self.interpreter.create_infer_request()
 
             try:
@@ -141,103 +107,3 @@ class ONNXModelRunner:
             return outputs
         elif self.type == "ort":
             return self.ort.run(None, input)
-
-
-class RKNNModelRunner:
-    """Run RKNN models for embeddings."""
-
-    def __init__(self, model_path: str, device: str = "AUTO", model_type: str = None):
-        self.model_path = model_path
-        self.device = device
-        self.model_type = model_type
-        self.rknn = None
-        self._load_model()
-
-    def _load_model(self):
-        """Load the RKNN model."""
-        try:
-            from rknnlite.api import RKNNLite
-
-            self.rknn = RKNNLite(verbose=False)
-
-            if self.rknn.load_rknn(self.model_path) != 0:
-                logger.error(f"Failed to load RKNN model: {self.model_path}")
-                raise RuntimeError("Failed to load RKNN model")
-
-            if self.rknn.init_runtime() != 0:
-                logger.error("Failed to initialize RKNN runtime")
-                raise RuntimeError("Failed to initialize RKNN runtime")
-
-            logger.info(f"Successfully loaded RKNN model: {self.model_path}")
-
-        except ImportError:
-            logger.error("RKNN Lite not available")
-            raise ImportError("RKNN Lite not available")
-        except Exception as e:
-            logger.error(f"Error loading RKNN model: {e}")
-            raise
-
-    def get_input_names(self) -> list[str]:
-        """Get input names for the model."""
-        # For CLIP models, we need to determine the model type from the path
-        model_name = os.path.basename(self.model_path).lower()
-
-        if "vision" in model_name:
-            return ["pixel_values"]
-        elif "arcface" in model_name:
-            return ["data"]
-        else:
-            # Default fallback - try to infer from model type
-            if self.model_type and "jina-clip" in self.model_type:
-                if "vision" in self.model_type:
-                    return ["pixel_values"]
-
-            # Generic fallback
-            return ["input"]
-
-    def get_input_width(self) -> int:
-        """Get the input width of the model."""
-        # For CLIP vision models, this is typically 224
-        model_name = os.path.basename(self.model_path).lower()
-        if "vision" in model_name:
-            return 224  # CLIP V1 uses 224x224
-        elif "arcface" in model_name:
-            return 112
-        return -1
-
-    def run(self, inputs: dict[str, Any]) -> Any:
-        """Run inference with the RKNN model."""
-        if not self.rknn:
-            raise RuntimeError("RKNN model not loaded")
-
-        try:
-            input_names = self.get_input_names()
-            rknn_inputs = []
-
-            for name in input_names:
-                if name in inputs:
-                    if name == "pixel_values":
-                        # RKNN expects NHWC format, but ONNX typically provides NCHW
-                        # Transpose from [batch, channels, height, width] to [batch, height, width, channels]
-                        pixel_data = inputs[name]
-                        if len(pixel_data.shape) == 4 and pixel_data.shape[1] == 3:
-                            # Transpose from NCHW to NHWC
-                            pixel_data = np.transpose(pixel_data, (0, 2, 3, 1))
-                        rknn_inputs.append(pixel_data)
-                    else:
-                        rknn_inputs.append(inputs[name])
-
-            outputs = self.rknn.inference(inputs=rknn_inputs)
-            return outputs
-
-        except Exception as e:
-            logger.error(f"Error during RKNN inference: {e}")
-            raise
-
-    def __del__(self):
-        """Cleanup when the runner is destroyed."""
-        if self.rknn:
-            try:
-                self.rknn.release()
-            except Exception:
-                pass

frigate/util/rknn_converter.py

@@ -27,58 +27,9 @@ MODEL_TYPE_CONFIGS = {
         "std_values": [[255, 255, 255]],
         "target_platform": None,  # Will be set dynamically
     },
-    "jina-clip-v1-vision": {
-        "mean_values": [[0.48145466 * 255, 0.4578275 * 255, 0.40821073 * 255]],
-        "std_values": [[0.26862954 * 255, 0.26130258 * 255, 0.27577711 * 255]],
-        "target_platform": None,  # Will be set dynamically
-    },
-    "arcface-r100": {
-        "mean_values": [[127.5, 127.5, 127.5]],
-        "std_values": [[127.5, 127.5, 127.5]],
-        "target_platform": None,  # Will be set dynamically
-    },
 }
 
 
-def get_rknn_model_type(model_path: str) -> str | None:
-    if all(keyword in str(model_path) for keyword in ["jina-clip-v1", "vision"]):
-        return "jina-clip-v1-vision"
-
-    model_name = os.path.basename(str(model_path)).lower()
-
-    if "arcface" in model_name:
-        return "arcface-r100"
-
-    if any(keyword in model_name for keyword in ["yolo", "yolox", "yolonas"]):
-        return model_name
-
-    return None
-
-
-def is_rknn_compatible(model_path: str, model_type: str | None = None) -> bool:
-    """
-    Check if a model is compatible with RKNN conversion.
-
-    Args:
-        model_path: Path to the model file
-        model_type: Type of the model (if known)
-
-    Returns:
-        True if the model is RKNN-compatible, False otherwise
-    """
-    soc = get_soc_type()
-    if soc is None:
-        return False
-
-    if not model_type:
-        model_type = get_rknn_model_type(model_path)
-
-    if model_type and model_type in MODEL_TYPE_CONFIGS:
-        return True
-
-    return False
-
-
 def ensure_torch_dependencies() -> bool:
     """Dynamically install torch dependencies if not available."""
     try:
@@ -116,12 +67,13 @@ def ensure_torch_dependencies() -> bool:
 def ensure_rknn_toolkit() -> bool:
     """Ensure RKNN toolkit is available."""
     try:
+        import rknn  # type: ignore  # noqa: F401
         from rknn.api import RKNN  # type: ignore # noqa: F401
 
         logger.debug("RKNN toolkit is already available")
         return True
-    except ImportError as e:
-        logger.error(f"RKNN toolkit not found. Please ensure it's installed. {e}")
+    except ImportError:
+        logger.error("RKNN toolkit not found. Please ensure it's installed.")
         return False
 
 
@@ -157,11 +109,11 @@ def convert_onnx_to_rknn(
         True if conversion successful, False otherwise
     """
     if not ensure_torch_dependencies():
-        logger.debug("PyTorch dependencies not available")
+        logger.error("PyTorch dependencies not available")
         return False
 
     if not ensure_rknn_toolkit():
-        logger.debug("RKNN toolkit not available")
+        logger.error("RKNN toolkit not available")
         return False
 
     # Get SoC type if not provided
@@ -173,7 +125,7 @@ def convert_onnx_to_rknn(
 
     # Get model config for the specified type
     if model_type not in MODEL_TYPE_CONFIGS:
-        logger.debug(f"Unsupported model type: {model_type}")
+        logger.error(f"Unsupported model type: {model_type}")
         return False
 
     config = MODEL_TYPE_CONFIGS[model_type].copy()
@@ -186,22 +138,7 @@ def convert_onnx_to_rknn(
         rknn = RKNN(verbose=True)
         rknn.config(**config)
 
-        if model_type == "jina-clip-v1-vision":
-            load_output = rknn.load_onnx(
-                model=onnx_path,
-                inputs=["pixel_values"],
-                input_size_list=[[1, 3, 224, 224]],
-            )
-        elif model_type == "arcface-r100":
-            load_output = rknn.load_onnx(
-                model=onnx_path,
-                inputs=["data"],
-                input_size_list=[[1, 3, 112, 112]],
-            )
-        else:
-            load_output = rknn.load_onnx(model=onnx_path)
-
-        if load_output != 0:
+        if rknn.load_onnx(model=onnx_path) != 0:
             logger.error("Failed to load ONNX model")
             return False
 
@@ -328,7 +265,7 @@ def is_lock_stale(lock_file_path: Path, max_age: int = 600) -> bool:
 
 
 def wait_for_conversion_completion(
-    model_type: str, rknn_path: Path, lock_file_path: Path, timeout: int = 300
+    rknn_path: Path, lock_file_path: Path, timeout: int = 300
 ) -> bool:
     """
     Wait for another process to complete the conversion.
@@ -370,7 +307,7 @@ def wait_for_conversion_completion(
                     # Check if RKNN file appeared while waiting
                     if rknn_path.exists():
                         logger.info(f"RKNN model appeared while waiting: {rknn_path}")
-                        return True
+                        return str(rknn_path)
 
                     # Convert ONNX to RKNN
                     logger.info(
@@ -383,12 +320,12 @@ def wait_for_conversion_completion(
 
                     if onnx_path.exists():
                         if convert_onnx_to_rknn(
-                            str(onnx_path), str(rknn_path), model_type, False
+                            str(onnx_path), str(rknn_path), "yolo-generic", False
                         ):
-                            return True
+                            return str(rknn_path)
 
                     logger.error("Failed to convert model after stale lock cleanup")
-                    return False
+                    return None
 
                 finally:
                     release_conversion_lock(lock_file_path)
@@ -401,7 +338,7 @@ def wait_for_conversion_completion(
 
 
 def auto_convert_model(
-    model_path: str, model_type: str | None = None, quantization: bool = False
+    model_path: str, model_type: str, quantization: bool = False
 ) -> Optional[str]:
     """
     Automatically convert a model to RKNN format if needed.
@@ -440,9 +377,6 @@ def auto_convert_model(
                 logger.info(f"Converting {model_path} to RKNN format...")
                 rknn_path.parent.mkdir(parents=True, exist_ok=True)
 
-                if not model_type:
-                    model_type = get_rknn_model_type(base_path)
-
                 if convert_onnx_to_rknn(
                     str(base_path), str(rknn_path), model_type, quantization
                 ):
@@ -458,10 +392,7 @@ def auto_convert_model(
                 f"Another process is converting {model_path}, waiting for completion..."
             )
 
-            if not model_type:
-                model_type = get_rknn_model_type(base_path)
-
-            if wait_for_conversion_completion(model_type, rknn_path, lock_file_path):
+            if wait_for_conversion_completion(rknn_path, lock_file_path):
                 return str(rknn_path)
             else:
                 logger.error(f"Timeout waiting for conversion of {model_path}")

web/src/hooks/use-camera-activity.ts

@@ -144,9 +144,7 @@ export function useCameraActivity(
       return false;
     }
 
-    return (
-      cameras[camera.name]?.camera_fps == 0 && stats["service"].uptime > 60
-    );
+    return cameras[camera.name].camera_fps == 0 && stats["service"].uptime > 60;
   }, [camera, stats]);
 
   const isCameraEnabled = cameraEnabled ? cameraEnabled === "ON" : true;