Face recognition improvements (#17387)

* Increase frequency of updates when internal face detection is used

* Adjust number of required faces based on detection type

* Adjust min_score config to unknown_score

* Only for person

* Improve typing

* Update face rec docs

* Cleanup ui colors

* Cleanup

docker-compose.yml

@@ -1,7 +1,8 @@
 services:
   devcontainer:
     container_name: frigate-devcontainer
-    # add groups from host for render, plugdev, video
+    # Check host system's actual render/video/plugdev group IDs with 'getent group render', 'getent group video', and 'getent group plugdev'
+    # Must add these exact IDs in container's group_add section or OpenVINO GPU acceleration will fail
     group_add:
       - "109" # render
       - "110" # render
@@ -37,4 +38,4 @@ services:
     container_name: mqtt
     image: eclipse-mosquitto:1.6
     ports:
-      - "1883:1883"
+      - "1883:1883"

docker/tensorrt/Dockerfile.base

@@ -20,8 +20,8 @@ RUN --mount=type=bind,source=docker/tensorrt/detector/tensorrt_libyolo.sh,target
 # COPY required individual CUDA deps
 RUN mkdir -p /usr/local/cuda-deps
 RUN if [ "$TARGETARCH" = "amd64" ]; then \
-      cp /usr/local/cuda-12.3/targets/x86_64-linux/lib/libcurand.so.* /usr/local/cuda-deps/ && \
-      cp /usr/local/cuda-12.3/targets/x86_64-linux/lib/libnvrtc.so.* /usr/local/cuda-deps/ ; \
+      cp /usr/local/cuda-12.3/targets/x86_64-linux/lib/libcurand.s* /usr/local/cuda-deps/ && \
+      cp /usr/local/cuda-12.3/targets/x86_64-linux/lib/libnvrtc.s* /usr/local/cuda-deps/ ; \
     fi
 
 # Frigate w/ TensorRT Support as separate image

docs/docs/configuration/face_recognition.md

@@ -22,16 +22,16 @@ Frigate needs to first detect a `face` before it can recognize a face.
 ### Face Recognition
 
 Frigate has support for two face recognition model types:
-- **small**: Frigate will use CV2 Local Binary Pattern Face Recognizer to recognize faces, which runs locally on the CPU.
-- **large**: Frigate will run a face embedding model, this is only recommended to be run when an integrated or dedicated GPU is available.
 
-In both cases a lightweight face landmark detection model is also used to align faces before running them through the face recognizer.
+- **small**: Frigate will use CV2 Local Binary Pattern Face Recognizer to recognize faces, which runs locally on the CPU. This model is optimized for efficiency and is not as accurate.
+- **large**: Frigate will run a face embedding model, this model is optimized for accuracy. It is only recommended to be run when an integrated or dedicated GPU is available.
+
+In both cases a lightweight face landmark detection model is also used to align faces before running the recognition model.
 
 ## Minimum System Requirements
 
-Face recognition is lightweight and runs on the CPU, there are no significantly different system requirements than running Frigate itself when using the `small` model.
-
-When using the `large` model an integrated or discrete GPU is recommended.
+The `small` model is optimized for efficiency and runs on the CPU, there are no significantly different system requirements.
+The `large` model is optimized for accuracy and an integrated or discrete GPU is highly recommended.
 
 ## Configuration
 
@@ -58,6 +58,8 @@ Fine-tune face recognition with these optional parameters:
 ### Recognition
 
 - `model_size`: Which model size to use, options are `small` or `large`
+- `unknown_score`: Min score to mark a person as a potential match, matches below this will be marked as unknown.
+  - Default: `0.8`.
 - `recognition_threshold`: Recognition confidence score required to add the face to the object as a sub label.
   - Default: `0.9`.
 - `blur_confidence_filter`: Enables a filter that calculates how blurry the face is and adjusts the confidence based on this.
@@ -108,13 +110,14 @@ Once straight-on images are performing well, start choosing slightly off-angle i
 
 ## FAQ
 
-### Why is every face tagged as a known face and not unknown?
+### Why can't I bulk upload photos?
 
-Any recognized face with a score >= `min_score` will show in the `Train` tab along with the recognition score. A low scoring face is effectively the same as `unknown`, but includes more information. This does not mean the recognition is not working well, and is part of the importance of choosing the correct `recognition_threshold`.
+It is important to methodically add photos to the library, bulk importing photos (especially from a general photo library) will lead to overfitting in that particular scenario and hurt recognition performance.
 
 ### Why do unknown people score similarly to known people?
 
 This can happen for a few different reasons, but this is usually an indicator that the training set needs to be improved. This is often related to overfitting:
+
 - If you train with only a few images per person, especially if those images are very similar, the recognition model becomes overly specialized to those specific images.
 - When you provide images with different poses, lighting, and expressions, the algorithm extracts features that are consistent across those variations.
 - By training on a diverse set of images, the algorithm becomes less sensitive to minor variations and noise in the input image.

docs/docs/configuration/reference.md

@@ -547,8 +547,8 @@ semantic_search:
 face_recognition:
   # Optional: Enable semantic search (default: shown below)
   enabled: False
-  # Optional: Minimum face distance score required to save the attempt (default: shown below)
-  min_score: 0.8
+  # Optional: Minimum face distance score required to mark as a potential match (default: shown below)
+  unknown_score: 0.8
   # Optional: Minimum face detection score required to detect a face (default: shown below)
   # NOTE: This only applies when not running a Frigate+ model
   detection_threshold: 0.7

frigate/camera/state.py

@@ -5,7 +5,7 @@ import logging
 import os
 import threading
 from collections import defaultdict
-from typing import Callable
+from typing import Any, Callable
 
 import cv2
 import numpy as np
@@ -53,8 +53,19 @@ class CameraState:
         self.callbacks = defaultdict(list)
         self.ptz_autotracker_thread = ptz_autotracker_thread
         self.prev_enabled = self.camera_config.enabled
+        self.requires_face_detection = (
+            self.config.face_recognition.enabled
+            and "face" not in self.config.objects.all_objects
+        )
 
-    def get_current_frame(self, draw_options={}):
+    def get_max_update_frequency(self, obj: TrackedObject) -> int:
+        return (
+            1
+            if self.requires_face_detection and obj.obj_data["label"] == "person"
+            else 5
+        )
+
+    def get_current_frame(self, draw_options: dict[str, Any] = {}):
         with self.current_frame_lock:
             frame_copy = np.copy(self._current_frame)
             frame_time = self.current_frame_time
@@ -283,11 +294,12 @@ class CameraState:
 
                 updated_obj.last_updated = frame_time
 
-            # if it has been more than 5 seconds since the last thumb update
+            # if it has been more than max_update_frequency seconds since the last thumb update
             # and the last update is greater than the last publish or
             # the object has changed significantly
             if (
-                frame_time - updated_obj.last_published > 5
+                frame_time - updated_obj.last_published
+                > self.get_max_update_frequency(updated_obj)
                 and updated_obj.last_updated > updated_obj.last_published
             ) or significant_update:
                 # call event handlers

frigate/config/classification.py

@@ -54,8 +54,8 @@ class FaceRecognitionConfig(FrigateBaseModel):
     model_size: str = Field(
         default="small", title="The size of the embeddings model used."
     )
-    min_score: float = Field(
-        title="Minimum face distance score required to save the attempt.",
+    unknown_score: float = Field(
+        title="Minimum face distance score required to be marked as a potential match.",
         default=0.8,
         gt=0.0,
         le=1.0,

frigate/data_processing/common/face/model.py

@@ -164,9 +164,7 @@ class LBPHRecognizer(FaceRecognizer):
             return
 
         self.recognizer: cv2.face.LBPHFaceRecognizer = (
-            cv2.face.LBPHFaceRecognizer_create(
-                radius=2, threshold=(1 - self.config.face_recognition.min_score) * 1000
-            )
+            cv2.face.LBPHFaceRecognizer_create(radius=2, threshold=400)
         )
         self.recognizer.train(faces, np.array(labels))
 
@@ -243,6 +241,8 @@ class ArcFaceRecognizer(FaceRecognizer):
         for name, embs in face_embeddings_map.items():
             self.mean_embs[name] = stats.trim_mean(embs, 0.15)
 
+        logger.debug("Finished building ArcFace model")
+
     def similarity_to_confidence(
         self, cosine_similarity: float, median=0.3, range_width=0.6, slope_factor=12
     ):
@@ -302,7 +302,4 @@ class ArcFaceRecognizer(FaceRecognizer):
                 score = confidence
                 label = name
 
-        if score < self.config.face_recognition.min_score:
-            return None
-
         return label, round(score * blur_factor, 2)

frigate/data_processing/common/license_plate/mixin.py

@@ -32,10 +32,6 @@ class LicensePlateProcessingMixin:
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
 
-        self.requires_license_plate_detection = (
-            "license_plate" not in self.config.objects.all_objects
-        )
-
         self.event_metadata_publisher = EventMetadataPublisher()
 
         self.ctc_decoder = CTCDecoder()
@@ -312,7 +308,6 @@ class LicensePlateProcessingMixin:
 
             # get minimum bounding box (rotated rectangle) around the contour and the smallest side length.
             points, min_side = self._get_min_boxes(contour)
-            logger.debug(f"min side {index}, {min_side}")
 
             if min_side < self.min_size:
                 continue
@@ -320,7 +315,6 @@ class LicensePlateProcessingMixin:
             points = np.array(points)
 
             score = self._box_score(output, contour)
-            logger.debug(f"box score {index}, {score}")
             if self.box_thresh > score:
                 continue
 
@@ -991,21 +985,21 @@ class LicensePlateProcessingMixin:
             license_plate = self._detect_license_plate(rgb)
 
             logger.debug(
-                f"YOLOv9 LPD inference time: {(datetime.datetime.now().timestamp() - yolov9_start) * 1000:.2f} ms"
+                f"{camera}: YOLOv9 LPD inference time: {(datetime.datetime.now().timestamp() - yolov9_start) * 1000:.2f} ms"
             )
             self.__update_yolov9_metrics(
                 datetime.datetime.now().timestamp() - yolov9_start
             )
 
             if not license_plate:
-                logger.debug("Detected no license plates in full frame.")
+                logger.debug(f"{camera}: Detected no license plates in full frame.")
                 return
 
             license_plate_area = (license_plate[2] - license_plate[0]) * (
                 license_plate[3] - license_plate[1]
             )
             if license_plate_area < self.lpr_config.min_area:
-                logger.debug("License plate area below minimum threshold.")
+                logger.debug(f"{camera}: License plate area below minimum threshold.")
                 return
 
             license_plate_frame = rgb[
@@ -1027,13 +1021,15 @@ class LicensePlateProcessingMixin:
 
             # don't run for non car objects
             if obj_data.get("label") != "car":
-                logger.debug("Not a processing license plate for non car object.")
+                logger.debug(
+                    f"{camera}: Not a processing license plate for non car object."
+                )
                 return
 
             # don't run for stationary car objects
             if obj_data.get("stationary") == True:
                 logger.debug(
-                    "Not a processing license plate for a stationary car object."
+                    f"{camera}: Not a processing license plate for a stationary car object."
                 )
                 return
 
@@ -1041,14 +1037,14 @@ class LicensePlateProcessingMixin:
             # that is not a license plate
             if obj_data.get("sub_label") and id not in self.detected_license_plates:
                 logger.debug(
-                    f"Not processing license plate due to existing sub label: {obj_data.get('sub_label')}."
+                    f"{camera}: Not processing license plate due to existing sub label: {obj_data.get('sub_label')}."
                 )
                 return
 
             license_plate: Optional[dict[str, any]] = None
 
-            if self.requires_license_plate_detection:
-                logger.debug("Running manual license_plate detection.")
+            if "license_plate" not in self.config.cameras[camera].objects.track:
+                logger.debug(f"{camera}: Running manual license_plate detection.")
 
                 car_box = obj_data.get("box")
 
@@ -1071,14 +1067,16 @@ class LicensePlateProcessingMixin:
                 yolov9_start = datetime.datetime.now().timestamp()
                 license_plate = self._detect_license_plate(car)
                 logger.debug(
-                    f"YOLOv9 LPD inference time: {(datetime.datetime.now().timestamp() - yolov9_start) * 1000:.2f} ms"
+                    f"{camera}: YOLOv9 LPD inference time: {(datetime.datetime.now().timestamp() - yolov9_start) * 1000:.2f} ms"
                 )
                 self.__update_yolov9_metrics(
                     datetime.datetime.now().timestamp() - yolov9_start
                 )
 
                 if not license_plate:
-                    logger.debug("Detected no license plates for car object.")
+                    logger.debug(
+                        f"{camera}: Detected no license plates for car object."
+                    )
                     return
 
                 license_plate_area = max(
@@ -1093,7 +1091,7 @@ class LicensePlateProcessingMixin:
                     license_plate_area
                     < self.config.cameras[obj_data["camera"]].lpr.min_area * 2
                 ):
-                    logger.debug("License plate is less than min_area")
+                    logger.debug(f"{camera}: License plate is less than min_area")
                     return
 
                 license_plate_frame = car[
@@ -1103,7 +1101,7 @@ class LicensePlateProcessingMixin:
             else:
                 # don't run for object without attributes
                 if not obj_data.get("current_attributes"):
-                    logger.debug("No attributes to parse.")
+                    logger.debug(f"{camera}: No attributes to parse.")
                     return
 
                 attributes: list[dict[str, any]] = obj_data.get(
@@ -1130,14 +1128,14 @@ class LicensePlateProcessingMixin:
                     or area(license_plate_box)
                     < self.config.cameras[obj_data["camera"]].lpr.min_area
                 ):
-                    logger.debug(f"Invalid license plate box {license_plate}")
+                    logger.debug(f"{camera}: Invalid license plate box {license_plate}")
                     return
 
                 license_plate_frame = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_I420)
 
-                # Expand the license_plate_box by 30%
+                # Expand the license_plate_box by 10%
                 box_array = np.array(license_plate_box)
-                expansion = (box_array[2:] - box_array[:2]) * 0.30
+                expansion = (box_array[2:] - box_array[:2]) * 0.10
                 expanded_box = np.array(
                     [
                         license_plate_box[0] - expansion[0],
@@ -1184,7 +1182,7 @@ class LicensePlateProcessingMixin:
                 )
 
                 logger.debug(
-                    f"Detected text: {plate} (average confidence: {avg_confidence:.2f}, area: {text_area} pixels)"
+                    f"{camera}: Detected text: {plate} (average confidence: {avg_confidence:.2f}, area: {text_area} pixels)"
                 )
         else:
             logger.debug("No text detected")
@@ -1204,7 +1202,7 @@ class LicensePlateProcessingMixin:
         # Check against minimum confidence threshold
         if avg_confidence < self.lpr_config.recognition_threshold:
             logger.debug(
-                f"Average confidence {avg_confidence} is less than threshold ({self.lpr_config.recognition_threshold})"
+                f"{camera}: Average confidence {avg_confidence} is less than threshold ({self.lpr_config.recognition_threshold})"
             )
             return
 
@@ -1223,7 +1221,7 @@ class LicensePlateProcessingMixin:
                     if similarity >= self.similarity_threshold:
                         plate_id = existing_id
                         logger.debug(
-                            f"Matched plate {top_plate} to {data['plate']} (similarity: {similarity:.3f})"
+                            f"{camera}: Matched plate {top_plate} to {data['plate']} (similarity: {similarity:.3f})"
                         )
                         break
             if plate_id is None:
@@ -1231,11 +1229,11 @@ class LicensePlateProcessingMixin:
                     obj_data, top_plate, avg_confidence
                 )
                 logger.debug(
-                    f"New plate event for dedicated LPR camera {plate_id}: {top_plate}"
+                    f"{camera}: New plate event for dedicated LPR camera {plate_id}: {top_plate}"
                 )
             else:
                 logger.debug(
-                    f"Matched existing plate event for dedicated LPR camera {plate_id}: {top_plate}"
+                    f"{camera}: Matched existing plate event for dedicated LPR camera {plate_id}: {top_plate}"
                 )
                 self.detected_license_plates[plate_id]["last_seen"] = current_time
 
@@ -1246,7 +1244,7 @@ class LicensePlateProcessingMixin:
             if self._should_keep_previous_plate(
                 id, top_plate, top_char_confidences, top_area, avg_confidence
             ):
-                logger.debug("Keeping previous plate")
+                logger.debug(f"{camera}: Keeping previous plate")
                 return
 
         # Determine subLabel based on known plates, use regex matching
@@ -1277,7 +1275,9 @@ class LicensePlateProcessingMixin:
 
         if dedicated_lpr:
             # save the best snapshot
-            logger.debug(f"Writing snapshot for {id}, {top_plate}, {current_time}")
+            logger.debug(
+                f"{camera}: Writing snapshot for {id}, {top_plate}, {current_time}"
+            )
             frame_bgr = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_I420)
             self.sub_label_publisher.publish(
                 EventMetadataTypeEnum.save_lpr_snapshot,

frigate/data_processing/post/license_plate.py

@@ -139,7 +139,7 @@ class LicensePlatePostProcessor(LicensePlateProcessingMixin, PostProcessorApi):
         scale_y = image.shape[0] / detect_height
 
         # Determine which box to enlarge based on detection mode
-        if self.requires_license_plate_detection:
+        if "license_plate" not in self.config.cameras[camera_name].objects.track:
             # Scale and enlarge the car box
             box = obj_data.get("box")
             if not box:
@@ -189,7 +189,7 @@ class LicensePlatePostProcessor(LicensePlateProcessingMixin, PostProcessorApi):
         )
 
         keyframe_obj_data = obj_data.copy()
-        if self.requires_license_plate_detection:
+        if "license_plate" not in self.config.cameras[camera_name].objects.track:
             # car box
             keyframe_obj_data["box"] = [new_left, new_top, new_right, new_bottom]
         else:

frigate/data_processing/real_time/face.py

@@ -36,36 +36,6 @@ MAX_DETECTION_HEIGHT = 1080
 MIN_MATCHING_FACES = 2
 
 
-def weighted_average_by_area(results_list: list[tuple[str, float, int]]):
-    if len(results_list) < 3:
-        return "unknown", 0.0
-
-    score_count = {}
-    weighted_scores = {}
-    total_face_areas = {}
-
-    for name, score, face_area in results_list:
-        if name not in weighted_scores:
-            score_count[name] = 1
-            weighted_scores[name] = 0.0
-            total_face_areas[name] = 0.0
-        else:
-            score_count[name] += 1
-
-        weighted_scores[name] += score * face_area
-        total_face_areas[name] += face_area
-
-    prominent_name = max(score_count)
-
-    # if a single name is not prominent in the history then we are not confident
-    if score_count[prominent_name] / len(results_list) < 0.65:
-        return "unknown", 0.0
-
-    return prominent_name, weighted_scores[prominent_name] / total_face_areas[
-        prominent_name
-    ]
-
-
 class FaceRealTimeProcessor(RealTimeProcessorApi):
     def __init__(
         self,
@@ -271,6 +241,9 @@ class FaceRealTimeProcessor(RealTimeProcessorApi):
 
         sub_label, score = res
 
+        if score < self.face_config.unknown_score:
+            sub_label = "unknown"
+
         logger.debug(
             f"Detected best face for person as: {sub_label} with probability {score}"
         )
@@ -288,7 +261,7 @@ class FaceRealTimeProcessor(RealTimeProcessorApi):
         self.person_face_history[id].append(
             (sub_label, score, face_frame.shape[0] * face_frame.shape[1])
         )
-        (weighted_sub_label, weighted_score) = weighted_average_by_area(
+        (weighted_sub_label, weighted_score) = self.weighted_average_by_area(
             self.person_face_history[id]
         )
 
@@ -415,3 +388,34 @@ class FaceRealTimeProcessor(RealTimeProcessorApi):
     def expire_object(self, object_id: str):
         if object_id in self.person_face_history:
             self.person_face_history.pop(object_id)
+
+    def weighted_average_by_area(self, results_list: list[tuple[str, float, int]]):
+        min_faces = 1 if self.requires_face_detection else 3
+
+        if len(results_list) < min_faces:
+            return "unknown", 0.0
+
+        score_count = {}
+        weighted_scores = {}
+        total_face_areas = {}
+
+        for name, score, face_area in results_list:
+            if name not in weighted_scores:
+                score_count[name] = 1
+                weighted_scores[name] = 0.0
+                total_face_areas[name] = 0.0
+            else:
+                score_count[name] += 1
+
+            weighted_scores[name] += score * face_area
+            total_face_areas[name] += face_area
+
+        prominent_name = max(score_count)
+
+        # if a single name is not prominent in the history then we are not confident
+        if score_count[prominent_name] / len(results_list) < 0.65:
+            return "unknown", 0.0
+
+        return prominent_name, weighted_scores[prominent_name] / total_face_areas[
+            prominent_name
+        ]

frigate/embeddings/onnx/lpr_embedding.py

@@ -261,8 +261,8 @@ class LicensePlateDetector(BaseEmbedding):
     def _preprocess_inputs(self, raw_inputs):
         if isinstance(raw_inputs, list):
             raise ValueError("License plate embedding does not support batch inputs.")
-        # Get image as numpy array
-        img = self._process_image(raw_inputs)
+
+        img = raw_inputs
         height, width, channels = img.shape
 
         # Resize maintaining aspect ratio

web/public/locales/zh-CN/views/settings.json

@@ -7,13 +7,8 @@
     "masksAndZones": "遮罩和区域编辑器 - Frigate",
     "motionTuner": "运动调整器 - Frigate",
     "object": "对象设置 - Frigate",
-    "general": "常规设置 - Frigate"
-  },
-  "dialog": {
-    "unsavedChanges": {
-      "title": "你有未保存的更改。",
-      "desc": "是否要在继续之前保存更改？"
-    }
+    "general": "常规设置 - Frigate",
+    "frigatePlus": "Frigate+ 设置 - Frigate"
   },
   "menu": {
     "uiSettings": "界面设置",
@@ -23,7 +18,14 @@
     "motionTuner": "运动调整器",
     "debug": "调试",
     "users": "用户",
-    "notifications": "通知"
+    "notifications": "通知",
+    "frigateplus": "Frigate+"
+  },
+  "dialog": {
+    "unsavedChanges": {
+      "title": "你有未保存的更改。",
+      "desc": "是否要在继续之前保存更改？"
+    }
   },
   "cameraSetting": {
     "camera": "摄像头",
@@ -105,7 +107,19 @@
     "faceRecognition": {
       "title": "人脸识别",
       "desc": "人脸识别功能允许为人物分配名称，当识别到他们的面孔时，Frigate 会将人物的名字作为子标签进行分配。这些信息会显示在界面、过滤器以及通知中。",
-      "readTheDocumentation": "阅读文档（英文）"
+      "readTheDocumentation": "阅读文档（英文）",
+      "modelSize": {
+          "label": "模型大小",
+          "desc": "用于人脸识别的模型尺寸。",
+          "small": {
+              "title": "小模型",
+              "desc": "使用<em>小模型</em>将采用OpenCV的局部二值模式直方图(LBPH)算法，可在大多数CPU上高效运行。"
+          },
+          "large": {
+              "title": "大模型",
+              "desc": "使用<em>大模型</em>将采用ArcFace人脸嵌入模型，若适用将自动在GPU上运行。"
+          }
+      }
     },
     "licensePlateRecognition": {
       "title": "车牌识别",
@@ -113,7 +127,7 @@
       "readTheDocumentation": "阅读文档（英文）"
     },
     "toast": {
-      "success": "分类设置已保存。",
+      "success": "分类设置已保存，请重启 Frigate 以应用更改。",
       "error": "保存配置更改失败：{{errorMessage}}"
     }
   },
@@ -542,9 +556,17 @@
       "trainDate": "训练日期",
       "baseModel": "基础模型",
       "supportedDetectors": "支持的检测器",
+      "dimensions": "大小",
       "cameras": "摄像头",
       "loading": "正在加载模型信息...",
-      "error": "加载模型信息失败"
+      "error": "加载模型信息失败",
+      "availableModels": "可用模型",
+      "loadingAvailableModels": "正在加载可用模型...",
+      "modelSelect": "您可以在Frigate+上选择可用的模型。请注意，只能选择与当前探测器配置兼容的模型。"
+    },
+    "toast": {
+        "success": "Frigate+ 设置已保存。请重启 Frigate 以应用更改。",
+        "error": "配置更改保存失败：{{errorMessage}}"
     }
   }
 }

web/src/pages/FaceLibrary.tsx

@@ -33,7 +33,7 @@ import useKeyboardListener from "@/hooks/use-keyboard-listener";
 import useOptimisticState from "@/hooks/use-optimistic-state";
 import { cn } from "@/lib/utils";
 import { FaceLibraryData, RecognizedFaceData } from "@/types/face";
-import { FrigateConfig } from "@/types/frigateConfig";
+import { FaceRecognitionConfig, FrigateConfig } from "@/types/frigateConfig";
 import axios from "axios";
 import { useCallback, useEffect, useMemo, useRef, useState } from "react";
 import { isDesktop, isMobile } from "react-device-detect";
@@ -451,7 +451,7 @@ function TrainingGrid({
             key={image}
             image={image}
             faceNames={faceNames}
-            threshold={config.face_recognition.recognition_threshold}
+            recognitionConfig={config.face_recognition}
             selected={selectedFaces.includes(image)}
             onClick={(data, meta) => {
               if (meta) {
@@ -471,7 +471,7 @@ function TrainingGrid({
 type FaceAttemptProps = {
   image: string;
   faceNames: string[];
-  threshold: number;
+  recognitionConfig: FaceRecognitionConfig;
   selected: boolean;
   onClick: (data: RecognizedFaceData, meta: boolean) => void;
   onRefresh: () => void;
@@ -479,7 +479,7 @@ type FaceAttemptProps = {
 function FaceAttempt({
   image,
   faceNames,
-  threshold,
+  recognitionConfig,
   selected,
   onClick,
   onRefresh,
@@ -496,6 +496,16 @@ function FaceAttempt({
     };
   }, [image]);
 
+  const scoreStatus = useMemo(() => {
+    if (data.score >= recognitionConfig.recognition_threshold) {
+      return "match";
+    } else if (data.score >= recognitionConfig.unknown_score) {
+      return "potential";
+    } else {
+      return "unknown";
+    }
+  }, [data, recognitionConfig]);
+
   // interaction
 
   const imgRef = useRef<HTMLImageElement | null>(null);
@@ -579,10 +589,13 @@ function FaceAttempt({
             <div className="capitalize">{data.name}</div>
             <div
               className={cn(
-                data.score >= threshold ? "text-success" : "text-danger",
+                "",
+                scoreStatus == "match" && "text-success",
+                scoreStatus == "potential" && "text-orange-400",
+                scoreStatus == "unknown" && "text-danger",
               )}
             >
-              {data.score * 100}%
+              {Math.round(data.score * 100)}%
             </div>
           </div>
           <div className="flex flex-row items-start justify-end gap-5 md:gap-4">

web/src/types/frigateConfig.ts

@@ -20,6 +20,14 @@ export interface BirdseyeConfig {
   width: number;
 }
 
+export interface FaceRecognitionConfig {
+  enabled: boolean;
+  model_size: SearchModelSize;
+  unknown_score: number;
+  detection_threshold: number;
+  recognition_threshold: number;
+}
+
 export type SearchModel = "jinav1" | "jinav2";
 export type SearchModelSize = "small" | "large";
 
@@ -331,12 +339,7 @@ export interface FrigateConfig {
 
   environment_vars: Record<string, unknown>;
 
-  face_recognition: {
-    enabled: boolean;
-    model_size: SearchModelSize;
-    detection_threshold: number;
-    recognition_threshold: number;
-  };
+  face_recognition: FaceRecognitionConfig;
 
   ffmpeg: {
     global_args: string[];

web/src/views/settings/ClassificationSettingsView.tsx

@@ -424,7 +424,7 @@ export default function ClassificationSettingsView({
               }
             >
               <SelectTrigger className="w-20">
-                {classificationSettings.search.model_size}
+                {classificationSettings.face.model_size}
               </SelectTrigger>
               <SelectContent>
                 <SelectGroup>