Classification train updates (#19173)

* Improve model train button

* Add filters for classification

* Cleanup

* Don't run classification on false positives

* Cleanup filter

* Fix icon color

Makefile

@@ -1,7 +1,7 @@
 default_target: local
 
 COMMIT_HASH := $(shell git log -1 --pretty=format:"%h"|tail -1)
-VERSION = 0.16.0
+VERSION = 0.17.0
 IMAGE_REPO ?= ghcr.io/blakeblackshear/frigate
 GITHUB_REF_NAME ?= $(shell git rev-parse --abbrev-ref HEAD)
 BOARDS= #Initialized empty

docker/main/Dockerfile

@@ -224,6 +224,9 @@ ENV TRANSFORMERS_NO_ADVISORY_WARNINGS=1
 # Set OpenCV ffmpeg loglevel to fatal: https://ffmpeg.org/doxygen/trunk/log_8h.html
 ENV OPENCV_FFMPEG_LOGLEVEL=8
 
+# Set NumPy to ignore getlimits warning
+ENV PYTHONWARNINGS="ignore:::numpy.core.getlimits"
+
 # Set HailoRT to disable logging
 ENV HAILORT_LOGGER_PATH=NONE
 

docs/docs/configuration/genai.md

@@ -27,6 +27,8 @@ cameras:
       enabled: False
 ```
 
+Generative AI can also be toggled dynamically for a camera via MQTT with the topic `frigate/<camera_name>/genai/set`. See the [MQTT documentation](/integrations/mqtt/#frigatecamera_namegenaiset).
+
 ## Ollama
 
 :::warning

docs/docs/configuration/reference.md

@@ -840,6 +840,23 @@ cameras:
       # By default the cameras are sorted alphabetically.
       order: 0
 
+    # Optional: Configuration for triggers to automate actions based on semantic search results.
+    triggers:
+      # Required: Unique identifier for the trigger (generated automatically from nickname if not specified).
+      trigger_name:
+        # Required: Enable or disable the trigger. (default: shown below)
+        enabled: true
+        # Type of trigger, either `thumbnail` for image-based matching or `description` for text-based matching. (default: none)
+        type: thumbnail
+        # Reference data for matching, either an event ID for `thumbnail` or a text string for `description`. (default: none)
+        data: 1751565549.853251-b69j73
+        # Similarity threshold for triggering. (default: none)
+        threshold: 0.7
+        # List of actions to perform when the trigger fires. (default: none)
+        # Available options: `notification` (send a webpush notification)
+        actions:
+          - notification
+
     # Optional: Configuration for AI generated tracked object descriptions
     genai:
       # Optional: Enable AI description generation (default: shown below)

docs/docs/configuration/semantic_search.md

@@ -102,3 +102,41 @@ See the [Hardware Accelerated Enrichments](/configuration/hardware_acceleration_
 4. Make your search language and tone closely match exactly what you're looking for. If you are using thumbnail search, **phrase your query as an image caption**. Searching for "red car" may not work as well as "red sedan driving down a residential street on a sunny day".
 5. Semantic search on thumbnails tends to return better results when matching large subjects that take up most of the frame. Small things like "cat" tend to not work well.
 6. Experiment! Find a tracked object you want to test and start typing keywords and phrases to see what works for you.
+
+## Triggers
+
+Triggers utilize semantic search to automate actions when a tracked object matches a specified image or description. Triggers can be configured so that Frigate executes a specific actions when a tracked object's image or description matches a predefined image or text, based on a similarity threshold. Triggers are managed per camera and can be configured via the Frigate UI in the Settings page under the Triggers tab.
+
+### Configuration
+
+Triggers are defined within the `semantic_search` configuration for each camera in your Frigate configuration file or through the UI. Each trigger consists of a `type` (either `thumbnail` or `description`), a `data` field (the reference image event ID or text), a `threshold` for similarity matching, and a list of `actions` to perform when the trigger fires.
+
+#### Managing Triggers in the UI
+
+1. Navigate to the **Settings** page and select the **Triggers** tab.
+2. Choose a camera from the dropdown menu to view or manage its triggers.
+3. Click **Add Trigger** to create a new trigger or use the pencil icon to edit an existing one.
+4. In the **Create Trigger** dialog:
+   - Enter a **Name** for the trigger (e.g., "red_car_alert").
+   - Select the **Type** (`Thumbnail` or `Description`).
+   - For `Thumbnail`, select an image to trigger this action when a similar thumbnail image is detected, based on the threshold.
+   - For `Description`, enter text to trigger this action when a similar tracked object description is detected.
+   - Set the **Threshold** for similarity matching.
+   - Select **Actions** to perform when the trigger fires.
+5. Save the trigger to update the configuration and store the embedding in the database.
+
+When a trigger fires, the UI highlights the trigger with a blue outline for 3 seconds for easy identification.
+
+### Usage and Best Practices
+
+1. **Thumbnail Triggers**: Select a representative image (event ID) from the Explore page that closely matches the object you want to detect. For best results, choose images where the object is prominent and fills most of the frame.
+2. **Description Triggers**: Write concise, specific text descriptions (e.g., "Person in a red jacket") that align with the tracked object’s description. Avoid vague terms to improve matching accuracy.
+3. **Threshold Tuning**: Adjust the threshold to balance sensitivity and specificity. A higher threshold (e.g., 0.8) requires closer matches, reducing false positives but potentially missing similar objects. A lower threshold (e.g., 0.6) is more inclusive but may trigger more often.
+4. **Using Explore**: Use the context menu or right-click / long-press on a tracked object in the Grid View in Explore to quickly add a trigger based on the tracked object's thumbnail.
+5. **Editing triggers**: For the best experience, triggers should be edited via the UI. However, Frigate will ensure triggers edited in the config will be synced with triggers created and edited in the UI.
+
+### Notes
+
+- Triggers rely on the same Jina AI CLIP models (V1 or V2) used for semantic search. Ensure `semantic_search` is enabled and properly configured.
+- Reindexing embeddings (via the UI or `reindex: True`) does not affect trigger configurations but may update the embeddings used for matching.
+- For optimal performance, use a system with sufficient RAM (8GB minimum, 16GB recommended) and a GPU for `large` model configurations, as described in the Semantic Search requirements.

docs/docs/integrations/mqtt.md

@@ -192,6 +192,20 @@ Message published for each changed review item. The first message is published w
 }
 ```
 
+### `frigate/triggers`
+
+Message published when a trigger defined in a camera's `semantic_search` configuration fires.
+
+```json
+{
+  "name": "car_trigger",
+  "camera": "driveway",
+  "event_id": "1751565549.853251-b69j73",
+  "type": "thumbnail",
+  "score": 0.85
+}
+```
+
 ### `frigate/stats`
 
 Same data available at `/api/stats` published at a configurable interval.
@@ -383,6 +397,14 @@ Topic to turn review detections for a camera on or off. Expected values are `ON`
 
 Topic with current state of review detections for a camera. Published values are `ON` and `OFF`.
 
+### `frigate/<camera_name>/genai/set`
+
+Topic to turn generative AI for a camera on or off. Expected values are `ON` and `OFF`.
+
+### `frigate/<camera_name>/genai/state`
+
+Topic with current state of generative AI for a camera. Published values are `ON` and `OFF`.
+
 ### `frigate/<camera_name>/birdseye/set`
 
 Topic to turn Birdseye for a camera on and off. Expected values are `ON` and `OFF`. Birdseye mode

frigate/__main__.py

@@ -23,6 +23,10 @@ def main() -> None:
     setup_logging(manager)
 
     threading.current_thread().name = "frigate"
+    stop_event = mp.Event()
+
+    # send stop event on SIGINT
+    signal.signal(signal.SIGINT, lambda sig, frame: stop_event.set())
 
     # Make sure we exit cleanly on SIGTERM.
     signal.signal(signal.SIGTERM, lambda sig, frame: sys.exit())
@@ -110,9 +114,23 @@ def main() -> None:
         sys.exit(0)
 
     # Run the main application.
-    FrigateApp(config, manager).start()
+    FrigateApp(config, manager, stop_event).start()
 
 
 if __name__ == "__main__":
+    mp.set_forkserver_preload(
+        [
+            # Standard library and core dependencies
+            "sqlite3",
+            # Third-party libraries commonly used in Frigate
+            "numpy",
+            "cv2",
+            "peewee",
+            "zmq",
+            "ruamel.yaml",
+            # Frigate core modules
+            "frigate.camera.maintainer",
+        ]
+    )
     mp.set_start_method("forkserver", force=True)
     main()

frigate/api/defs/query/media_query_parameters.py

@@ -18,6 +18,7 @@ class MediaLatestFrameQueryParams(BaseModel):
     zones: Optional[int] = None
     mask: Optional[int] = None
     motion: Optional[int] = None
+    paths: Optional[int] = None
     regions: Optional[int] = None
     quality: Optional[int] = 70
     height: Optional[int] = None

frigate/api/defs/request/events_body.py

@@ -2,6 +2,8 @@ from typing import List, Optional, Union
 
 from pydantic import BaseModel, Field
 
+from frigate.config.classification import TriggerType
+
 
 class EventsSubLabelBody(BaseModel):
     subLabel: str = Field(title="Sub label", max_length=100)
@@ -45,3 +47,9 @@ class EventsDeleteBody(BaseModel):
 
 class SubmitPlusBody(BaseModel):
     include_annotation: int = Field(default=1)
+
+
+class TriggerEmbeddingBody(BaseModel):
+    type: TriggerType
+    data: str
+    threshold: float = Field(default=0.5, ge=0.0, le=1.0)

frigate/api/event.py

@@ -1,5 +1,6 @@
 """Event apis."""
 
+import base64
 import datetime
 import logging
 import os
@@ -10,6 +11,7 @@ from pathlib import Path
 from urllib.parse import unquote
 
 import cv2
+import numpy as np
 from fastapi import APIRouter, Request
 from fastapi.params import Depends
 from fastapi.responses import JSONResponse
@@ -34,6 +36,7 @@ from frigate.api.defs.request.events_body import (
     EventsLPRBody,
     EventsSubLabelBody,
     SubmitPlusBody,
+    TriggerEmbeddingBody,
 )
 from frigate.api.defs.response.event_response import (
     EventCreateResponse,
@@ -44,11 +47,12 @@ from frigate.api.defs.response.event_response import (
 from frigate.api.defs.response.generic_response import GenericResponse
 from frigate.api.defs.tags import Tags
 from frigate.comms.event_metadata_updater import EventMetadataTypeEnum
-from frigate.const import CLIPS_DIR
+from frigate.const import CLIPS_DIR, TRIGGER_DIR
 from frigate.embeddings import EmbeddingsContext
-from frigate.models import Event, ReviewSegment, Timeline
+from frigate.models import Event, ReviewSegment, Timeline, Trigger
 from frigate.track.object_processing import TrackedObject
 from frigate.util.builtin import get_tz_modifiers
+from frigate.util.path import get_event_thumbnail_bytes
 
 logger = logging.getLogger(__name__)
 
@@ -1255,6 +1259,38 @@ def regenerate_description(
     )
 
 
+@router.post(
+    "/description/generate",
+    response_model=GenericResponse,
+    # dependencies=[Depends(require_role(["admin"]))],
+)
+def generate_description_embedding(
+    request: Request,
+    body: EventsDescriptionBody,
+):
+    new_description = body.description
+
+    # If semantic search is enabled, update the index
+    if request.app.frigate_config.semantic_search.enabled:
+        context: EmbeddingsContext = request.app.embeddings
+        if len(new_description) > 0:
+            result = context.generate_description_embedding(
+                new_description,
+            )
+
+    return JSONResponse(
+        content=(
+            {
+                "success": True,
+                "message": f"Embedding for description is {result}"
+                if result
+                else "Failed to generate embedding",
+            }
+        ),
+        status_code=200,
+    )
+
+
 def delete_single_event(event_id: str, request: Request) -> dict:
     try:
         event = Event.get(Event.id == event_id)
@@ -1403,3 +1439,397 @@ def end_event(request: Request, event_id: str, body: EventsEndBody):
         content=({"success": True, "message": "Event successfully ended."}),
         status_code=200,
     )
+
+
+@router.post(
+    "/trigger/embedding",
+    response_model=dict,
+    dependencies=[Depends(require_role(["admin"]))],
+)
+def create_trigger_embedding(
+    request: Request,
+    body: TriggerEmbeddingBody,
+    camera: str,
+    name: str,
+):
+    try:
+        if not request.app.frigate_config.semantic_search.enabled:
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": "Semantic search is not enabled",
+                },
+                status_code=400,
+            )
+
+        # Check if trigger already exists
+        if (
+            Trigger.select()
+            .where(Trigger.camera == camera, Trigger.name == name)
+            .exists()
+        ):
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": f"Trigger {camera}:{name} already exists",
+                },
+                status_code=400,
+            )
+
+        context: EmbeddingsContext = request.app.embeddings
+        # Generate embedding based on type
+        embedding = None
+        if body.type == "description":
+            embedding = context.generate_description_embedding(body.data)
+        elif body.type == "thumbnail":
+            try:
+                event: Event = Event.get(Event.id == body.data)
+            except DoesNotExist:
+                # TODO: check triggers directory for image
+                return JSONResponse(
+                    content={
+                        "success": False,
+                        "message": f"Failed to fetch event for {body.type} trigger",
+                    },
+                    status_code=400,
+                )
+
+            # Skip the event if not an object
+            if event.data.get("type") != "object":
+                return
+
+            if thumbnail := get_event_thumbnail_bytes(event):
+                cursor = context.db.execute_sql(
+                    """
+                    SELECT thumbnail_embedding FROM vec_thumbnails WHERE id = ?
+                    """,
+                    [body.data],
+                )
+
+                row = cursor.fetchone() if cursor else None
+
+                if row:
+                    query_embedding = row[0]
+                    embedding = np.frombuffer(query_embedding, dtype=np.float32)
+            else:
+                # Extract valid thumbnail
+                thumbnail = get_event_thumbnail_bytes(event)
+
+                if thumbnail is None:
+                    return JSONResponse(
+                        content={
+                            "success": False,
+                            "message": f"Failed to get thumbnail for {body.data} for {body.type} trigger",
+                        },
+                        status_code=400,
+                    )
+
+                embedding = context.generate_image_embedding(
+                    body.data, (base64.b64encode(thumbnail).decode("ASCII"))
+                )
+
+        if embedding is None:
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": f"Failed to generate embedding for {body.type} trigger",
+                },
+                status_code=400,
+            )
+
+        if body.type == "thumbnail":
+            # Save image to the triggers directory
+            try:
+                os.makedirs(os.path.join(TRIGGER_DIR, camera), exist_ok=True)
+                with open(
+                    os.path.join(TRIGGER_DIR, camera, f"{body.data}.webp"), "wb"
+                ) as f:
+                    f.write(thumbnail)
+                logger.debug(
+                    f"Writing thumbnail for trigger with data {body.data} in {camera}."
+                )
+            except Exception as e:
+                logger.error(
+                    f"Failed to write thumbnail for trigger with data {body.data} in {camera}: {e}"
+                )
+
+        Trigger.create(
+            camera=camera,
+            name=name,
+            type=body.type,
+            data=body.data,
+            threshold=body.threshold,
+            model=request.app.frigate_config.semantic_search.model,
+            embedding=np.array(embedding, dtype=np.float32).tobytes(),
+            triggering_event_id="",
+            last_triggered=None,
+        )
+
+        return JSONResponse(
+            content={
+                "success": True,
+                "message": f"Trigger created successfully for {camera}:{name}",
+            },
+            status_code=200,
+        )
+
+    except Exception as e:
+        return JSONResponse(
+            content={
+                "success": False,
+                "message": f"Error creating trigger embedding: {str(e)}",
+            },
+            status_code=500,
+        )
+
+
+@router.put(
+    "/trigger/embedding/{camera}/{name}",
+    response_model=dict,
+    dependencies=[Depends(require_role(["admin"]))],
+)
+def update_trigger_embedding(
+    request: Request,
+    camera: str,
+    name: str,
+    body: TriggerEmbeddingBody,
+):
+    try:
+        if not request.app.frigate_config.semantic_search.enabled:
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": "Semantic search is not enabled",
+                },
+                status_code=400,
+            )
+
+        context: EmbeddingsContext = request.app.embeddings
+        # Generate embedding based on type
+        embedding = None
+        if body.type == "description":
+            embedding = context.generate_description_embedding(body.data)
+        elif body.type == "thumbnail":
+            webp_file = body.data + ".webp"
+            webp_path = os.path.join(TRIGGER_DIR, camera, webp_file)
+
+            try:
+                event: Event = Event.get(Event.id == body.data)
+                # Skip the event if not an object
+                if event.data.get("type") != "object":
+                    return JSONResponse(
+                        content={
+                            "success": False,
+                            "message": f"Event {body.data} is not a tracked object for {body.type} trigger",
+                        },
+                        status_code=400,
+                    )
+                # Extract valid thumbnail
+                thumbnail = get_event_thumbnail_bytes(event)
+
+                with open(webp_path, "wb") as f:
+                    f.write(thumbnail)
+            except DoesNotExist:
+                # check triggers directory for image
+                if not os.path.exists(webp_path):
+                    return JSONResponse(
+                        content={
+                            "success": False,
+                            "message": f"Failed to fetch event for {body.type} trigger",
+                        },
+                        status_code=400,
+                    )
+                else:
+                    # Load the image from the triggers directory
+                    with open(webp_path, "rb") as f:
+                        thumbnail = f.read()
+
+            embedding = context.generate_image_embedding(
+                body.data, (base64.b64encode(thumbnail).decode("ASCII"))
+            )
+
+        if embedding is None:
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": f"Failed to generate embedding for {body.type} trigger",
+                },
+                status_code=400,
+            )
+
+        # Check if trigger exists for upsert
+        trigger = Trigger.get_or_none(Trigger.camera == camera, Trigger.name == name)
+
+        if trigger:
+            # Update existing trigger
+            if trigger.data != body.data:  # Delete old thumbnail only if data changes
+                try:
+                    os.remove(os.path.join(TRIGGER_DIR, camera, f"{trigger.data}.webp"))
+                    logger.debug(
+                        f"Deleted thumbnail for trigger with data {trigger.data} in {camera}."
+                    )
+                except Exception as e:
+                    logger.error(
+                        f"Failed to delete thumbnail for trigger with data {trigger.data} in {camera}: {e}"
+                    )
+
+            Trigger.update(
+                data=body.data,
+                model=request.app.frigate_config.semantic_search.model,
+                embedding=np.array(embedding, dtype=np.float32).tobytes(),
+                threshold=body.threshold,
+                triggering_event_id="",
+                last_triggered=None,
+            ).where(Trigger.camera == camera, Trigger.name == name).execute()
+        else:
+            # Create new trigger (for rename case)
+            Trigger.create(
+                camera=camera,
+                name=name,
+                type=body.type,
+                data=body.data,
+                threshold=body.threshold,
+                model=request.app.frigate_config.semantic_search.model,
+                embedding=np.array(embedding, dtype=np.float32).tobytes(),
+                triggering_event_id="",
+                last_triggered=None,
+            )
+
+        if body.type == "thumbnail":
+            # Save image to the triggers directory
+            try:
+                os.makedirs(os.path.join(TRIGGER_DIR, camera), exist_ok=True)
+                with open(
+                    os.path.join(TRIGGER_DIR, camera, f"{body.data}.webp"), "wb"
+                ) as f:
+                    f.write(thumbnail)
+                logger.debug(
+                    f"Writing thumbnail for trigger with data {body.data} in {camera}."
+                )
+            except Exception as e:
+                logger.error(
+                    f"Failed to write thumbnail for trigger with data {body.data} in {camera}: {e}"
+                )
+
+        return JSONResponse(
+            content={
+                "success": True,
+                "message": f"Trigger updated successfully for {camera}:{name}",
+            },
+            status_code=200,
+        )
+
+    except Exception as e:
+        return JSONResponse(
+            content={
+                "success": False,
+                "message": f"Error updating trigger embedding: {str(e)}",
+            },
+            status_code=500,
+        )
+
+
+@router.delete(
+    "/trigger/embedding/{camera}/{name}",
+    response_model=dict,
+    dependencies=[Depends(require_role(["admin"]))],
+)
+def delete_trigger_embedding(
+    request: Request,
+    camera: str,
+    name: str,
+):
+    try:
+        trigger = Trigger.get_or_none(Trigger.camera == camera, Trigger.name == name)
+        if trigger is None:
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": f"Trigger {camera}:{name} not found",
+                },
+                status_code=500,
+            )
+
+        deleted = (
+            Trigger.delete()
+            .where(Trigger.camera == camera, Trigger.name == name)
+            .execute()
+        )
+        if deleted == 0:
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": f"Error deleting trigger {camera}:{name}",
+                },
+                status_code=401,
+            )
+
+        try:
+            os.remove(os.path.join(TRIGGER_DIR, camera, f"{trigger.data}.webp"))
+            logger.debug(
+                f"Deleted thumbnail for trigger with data {trigger.data} in {camera}."
+            )
+        except Exception as e:
+            logger.error(
+                f"Failed to delete thumbnail for trigger with data {trigger.data} in {camera}: {e}"
+            )
+
+        return JSONResponse(
+            content={
+                "success": True,
+                "message": f"Trigger deleted successfully for {camera}:{name}",
+            },
+            status_code=200,
+        )
+
+    except Exception as e:
+        return JSONResponse(
+            content={
+                "success": False,
+                "message": f"Error deleting trigger embedding: {str(e)}",
+            },
+            status_code=500,
+        )
+
+
+@router.get(
+    "/triggers/status/{camera_name}",
+    response_model=dict,
+    dependencies=[Depends(require_role(["admin"]))],
+)
+def get_triggers_status(
+    camera_name: str,
+):
+    try:
+        # Fetch all triggers for the specified camera
+        triggers = Trigger.select().where(Trigger.camera == camera_name)
+
+        # Prepare the response with trigger status
+        status = {
+            trigger.name: {
+                "last_triggered": trigger.last_triggered.timestamp()
+                if trigger.last_triggered
+                else None,
+                "triggering_event_id": trigger.triggering_event_id
+                if trigger.triggering_event_id
+                else None,
+            }
+            for trigger in triggers
+        }
+
+        if not status:
+            return JSONResponse(
+                content={
+                    "success": False,
+                    "message": f"No triggers found for camera {camera_name}",
+                },
+                status_code=404,
+            )
+
+        return {"success": True, "triggers": status}
+    except Exception as ex:
+        logger.exception(ex)
+        return JSONResponse(
+            content=({"success": False, "message": "Error fetching trigger status"}),
+            status_code=400,
+        )

frigate/api/media.py

@@ -141,6 +141,7 @@ def latest_frame(
         "zones": params.zones,
         "mask": params.mask,
         "motion_boxes": params.motion,
+        "paths": params.paths,
         "regions": params.regions,
     }
     quality = params.quality

frigate/app.py

@@ -38,6 +38,7 @@ from frigate.const import (
     MODEL_CACHE_DIR,
     RECORD_DIR,
     THUMB_DIR,
+    TRIGGER_DIR,
 )
 from frigate.data_processing.types import DataProcessorMetrics
 from frigate.db.sqlitevecq import SqliteVecQueueDatabase
@@ -55,6 +56,7 @@ from frigate.models import (
     Regions,
     ReviewSegment,
     Timeline,
+    Trigger,
     User,
 )
 from frigate.object_detection.base import ObjectDetectProcess
@@ -79,10 +81,12 @@ logger = logging.getLogger(__name__)
 
 
 class FrigateApp:
-    def __init__(self, config: FrigateConfig, manager: SyncManager) -> None:
+    def __init__(
+        self, config: FrigateConfig, manager: SyncManager, stop_event: MpEvent
+    ) -> None:
         self.metrics_manager = manager
         self.audio_process: Optional[mp.Process] = None
-        self.stop_event: MpEvent = mp.Event()
+        self.stop_event = stop_event
         self.detection_queue: Queue = mp.Queue()
         self.detectors: dict[str, ObjectDetectProcess] = {}
         self.detection_shms: list[mp.shared_memory.SharedMemory] = []
@@ -119,6 +123,9 @@ class FrigateApp:
         if self.config.face_recognition.enabled:
             dirs.append(FACE_DIR)
 
+        if self.config.semantic_search.enabled:
+            dirs.append(TRIGGER_DIR)
+
         for d in dirs:
             if not os.path.exists(d) and not os.path.islink(d):
                 logger.info(f"Creating directory: {d}")
@@ -223,14 +230,14 @@ class FrigateApp:
                 self.processes["go2rtc"] = proc.info["pid"]
 
     def init_recording_manager(self) -> None:
-        recording_process = RecordProcess(self.config)
+        recording_process = RecordProcess(self.config, self.stop_event)
         self.recording_process = recording_process
         recording_process.start()
         self.processes["recording"] = recording_process.pid or 0
         logger.info(f"Recording process started: {recording_process.pid}")
 
     def init_review_segment_manager(self) -> None:
-        review_segment_process = ReviewProcess(self.config)
+        review_segment_process = ReviewProcess(self.config, self.stop_event)
         self.review_segment_process = review_segment_process
         review_segment_process.start()
         self.processes["review_segment"] = review_segment_process.pid or 0
@@ -250,8 +257,7 @@ class FrigateApp:
             return
 
         embedding_process = EmbeddingProcess(
-            self.config,
-            self.embeddings_metrics,
+            self.config, self.embeddings_metrics, self.stop_event
         )
         self.embedding_process = embedding_process
         embedding_process.start()
@@ -285,6 +291,7 @@ class FrigateApp:
             ReviewSegment,
             Timeline,
             User,
+            Trigger,
         ]
         self.db.bind(models)
 
@@ -387,6 +394,7 @@ class FrigateApp:
                 list(self.config.cameras.keys()),
                 self.config,
                 detector_config,
+                self.stop_event,
             )
 
     def start_ptz_autotracker(self) -> None:
@@ -410,7 +418,7 @@ class FrigateApp:
         self.detected_frames_processor.start()
 
     def start_video_output_processor(self) -> None:
-        output_processor = OutputProcess(self.config)
+        output_processor = OutputProcess(self.config, self.stop_event)
         self.output_processor = output_processor
         output_processor.start()
         logger.info(f"Output process started: {output_processor.pid}")
@@ -436,7 +444,7 @@ class FrigateApp:
 
         if audio_cameras:
             self.audio_process = AudioProcessor(
-                self.config, audio_cameras, self.camera_metrics
+                self.config, audio_cameras, self.camera_metrics, self.stop_event
             )
             self.audio_process.start()
             self.processes["audio_detector"] = self.audio_process.pid or 0
@@ -658,4 +666,3 @@ class FrigateApp:
 
         _stop_logging()
         self.metrics_manager.shutdown()
-        os._exit(os.EX_OK)

frigate/camera/maintainer.py

@@ -165,6 +165,7 @@ class CameraMaintainer(threading.Thread):
             self.camera_metrics[name],
             self.ptz_metrics[name],
             self.region_grids[name],
+            self.stop_event,
         )
         self.camera_processes[config.name] = camera_process
         camera_process.start()
@@ -184,7 +185,9 @@ class CameraMaintainer(threading.Thread):
             frame_size = config.frame_shape_yuv[0] * config.frame_shape_yuv[1]
             self.frame_manager.create(f"{config.name}_frame{i}", frame_size)
 
-        capture_process = CameraCapture(config, count, self.camera_metrics[name])
+        capture_process = CameraCapture(
+            config, count, self.camera_metrics[name], self.stop_event
+        )
         capture_process.daemon = True
         self.capture_processes[name] = capture_process
         capture_process.start()

frigate/camera/state.py

@@ -228,6 +228,45 @@ class CameraState:
                 position=self.camera_config.timestamp_style.position,
             )
 
+        if draw_options.get("paths"):
+            for obj in tracked_objects.values():
+                if obj["frame_time"] == frame_time and obj["path_data"]:
+                    color = self.config.model.colormap.get(
+                        obj["label"], (255, 255, 255)
+                    )
+
+                    path_points = [
+                        (
+                            int(point[0][0] * self.camera_config.detect.width),
+                            int(point[0][1] * self.camera_config.detect.height),
+                        )
+                        for point in obj["path_data"]
+                    ]
+
+                    for point in path_points:
+                        cv2.circle(frame_copy, point, 5, color, -1)
+
+                    for i in range(1, len(path_points)):
+                        cv2.line(
+                            frame_copy,
+                            path_points[i - 1],
+                            path_points[i],
+                            color,
+                            2,
+                        )
+
+                    bottom_center = (
+                        int((obj["box"][0] + obj["box"][2]) / 2),
+                        int(obj["box"][3]),
+                    )
+                    cv2.line(
+                        frame_copy,
+                        path_points[-1],
+                        bottom_center,
+                        color,
+                        2,
+                    )
+
         return frame_copy
 
     def finished(self, obj_id):

frigate/comms/dispatcher.py

@@ -75,6 +75,7 @@ class Dispatcher:
             "birdseye_mode": self._on_birdseye_mode_command,
             "review_alerts": self._on_alerts_command,
             "review_detections": self._on_detections_command,
+            "genai": self._on_genai_command,
         }
         self._global_settings_handlers: dict[str, Callable] = {
             "notifications": self._on_global_notification_command,
@@ -207,6 +208,7 @@ class Dispatcher:
                     ].onvif.autotracking.enabled,
                     "alerts": self.config.cameras[camera].review.alerts.enabled,
                     "detections": self.config.cameras[camera].review.detections.enabled,
+                    "genai": self.config.cameras[camera].genai.enabled,
                 }
 
             self.publish("camera_activity", json.dumps(camera_status))
@@ -737,3 +739,28 @@ class Dispatcher:
             review_settings,
         )
         self.publish(f"{camera_name}/review_detections/state", payload, retain=True)
+
+    def _on_genai_command(self, camera_name: str, payload: str) -> None:
+        """Callback for GenAI topic."""
+        genai_settings = self.config.cameras[camera_name].genai
+
+        if payload == "ON":
+            if not self.config.cameras[camera_name].genai.enabled_in_config:
+                logger.error(
+                    "GenAI must be enabled in the config to be turned on via MQTT."
+                )
+                return
+
+            if not genai_settings.enabled:
+                logger.info(f"Turning on GenAI for {camera_name}")
+                genai_settings.enabled = True
+        elif payload == "OFF":
+            if genai_settings.enabled:
+                logger.info(f"Turning off GenAI for {camera_name}")
+                genai_settings.enabled = False
+
+        self.config_updater.publish_update(
+            CameraConfigUpdateTopic(CameraConfigUpdateEnum.genai, camera_name),
+            genai_settings,
+        )
+        self.publish(f"{camera_name}/genai/state", payload, retain=True)

frigate/comms/mqtt.py

@@ -122,6 +122,11 @@ class MqttClient(Communicator):  # type: ignore[misc]
                 "ON" if camera.review.detections.enabled_in_config else "OFF",
                 retain=True,
             )
+            self.publish(
+                f"{camera_name}/genai/state",
+                "ON" if camera.genai.enabled_in_config else "OFF",
+                retain=True,
+            )
 
         if self.config.notifications.enabled_in_config:
             self.publish(
@@ -215,6 +220,7 @@ class MqttClient(Communicator):  # type: ignore[misc]
             "birdseye_mode",
             "review_alerts",
             "review_detections",
+            "genai",
         ]
 
         for name in self.config.cameras.keys():

frigate/comms/webpush.py

@@ -186,6 +186,28 @@ class WebPushClient(Communicator):  # type: ignore[misc]
                 logger.debug(f"Notifications for {camera} are currently suspended.")
                 return
             self.send_alert(decoded)
+        if topic == "triggers":
+            decoded = json.loads(payload)
+
+            camera = decoded["camera"]
+            name = decoded["name"]
+
+            # ensure notifications are enabled and the specific trigger has
+            # notification action enabled
+            if (
+                not self.config.cameras[camera].notifications.enabled
+                or name not in self.config.cameras[camera].semantic_search.triggers
+                or "notification"
+                not in self.config.cameras[camera]
+                .semantic_search.triggers[name]
+                .actions
+            ):
+                return
+
+            if self.is_camera_suspended(camera):
+                logger.debug(f"Notifications for {camera} are currently suspended.")
+                return
+            self.send_trigger(decoded)
         elif topic == "notification_test":
             if not self.config.notifications.enabled and not any(
                 cam.notifications.enabled for cam in self.config.cameras.values()
@@ -264,6 +286,23 @@ class WebPushClient(Communicator):  # type: ignore[misc]
             except Exception as e:
                 logger.error(f"Error processing notification: {str(e)}")
 
+    def _within_cooldown(self, camera: str) -> bool:
+        now = datetime.datetime.now().timestamp()
+        if now - self.last_notification_time < self.config.notifications.cooldown:
+            logger.debug(
+                f"Skipping notification for {camera} - in global cooldown period"
+            )
+            return True
+        if (
+            now - self.last_camera_notification_time[camera]
+            < self.config.cameras[camera].notifications.cooldown
+        ):
+            logger.debug(
+                f"Skipping notification for {camera} - in camera-specific cooldown period"
+            )
+            return True
+        return False
+
     def send_notification_test(self) -> None:
         if not self.config.notifications.email:
             return
@@ -290,24 +329,7 @@ class WebPushClient(Communicator):  # type: ignore[misc]
         camera: str = payload["after"]["camera"]
         current_time = datetime.datetime.now().timestamp()
 
-        # Check global cooldown period
-        if (
-            current_time - self.last_notification_time
-            < self.config.notifications.cooldown
-        ):
-            logger.debug(
-                f"Skipping notification for {camera} - in global cooldown period"
-            )
-            return
-
-        # Check camera-specific cooldown period
-        if (
-            current_time - self.last_camera_notification_time[camera]
-            < self.config.cameras[camera].notifications.cooldown
-        ):
-            logger.debug(
-                f"Skipping notification for {camera} - in camera-specific cooldown period"
-            )
+        if self._within_cooldown(camera):
             return
 
         self.check_registrations()
@@ -362,6 +384,48 @@ class WebPushClient(Communicator):  # type: ignore[misc]
 
         self.cleanup_registrations()
 
+    def send_trigger(self, payload: dict[str, Any]) -> None:
+        if not self.config.notifications.email:
+            return
+
+        camera: str = payload["camera"]
+        current_time = datetime.datetime.now().timestamp()
+
+        if self._within_cooldown(camera):
+            return
+
+        self.check_registrations()
+
+        self.last_camera_notification_time[camera] = current_time
+        self.last_notification_time = current_time
+
+        trigger_type = payload["type"]
+        event_id = payload["event_id"]
+        name = payload["name"]
+        score = payload["score"]
+
+        title = f"{name.replace('_', ' ')} triggered on {titlecase(camera.replace('_', ' '))}"
+        message = f"{titlecase(trigger_type)} trigger fired for {titlecase(camera.replace('_', ' '))} with score {score:.2f}"
+        image = f"clips/triggers/{camera}/{event_id}.webp"
+
+        direct_url = f"/explore?event_id={event_id}"
+        ttl = 0
+
+        logger.debug(f"Sending push notification for {camera}, trigger name {name}")
+
+        for user in self.web_pushers:
+            self.send_push_notification(
+                user=user,
+                payload=payload,
+                title=title,
+                message=message,
+                direct_url=direct_url,
+                image=image,
+                ttl=ttl,
+            )
+
+        self.cleanup_registrations()
+
     def stop(self) -> None:
         logger.info("Closing notification queue")
         self.notification_thread.join()

frigate/config/camera/camera.py

@@ -22,6 +22,7 @@ from ..classification import (
     AudioTranscriptionConfig,
     CameraFaceRecognitionConfig,
     CameraLicensePlateRecognitionConfig,
+    CameraSemanticSearchConfig,
 )
 from .audio import AudioConfig
 from .birdseye import BirdseyeCameraConfig
@@ -91,6 +92,10 @@ class CameraConfig(FrigateBaseModel):
     review: ReviewConfig = Field(
         default_factory=ReviewConfig, title="Review configuration."
     )
+    semantic_search: CameraSemanticSearchConfig = Field(
+        default_factory=CameraSemanticSearchConfig,
+        title="Semantic search configuration.",
+    )
     snapshots: SnapshotsConfig = Field(
         default_factory=SnapshotsConfig, title="Snapshot configuration."
     )

frigate/config/camera/genai.py

@@ -58,6 +58,10 @@ class GenAICameraConfig(BaseModel):
         title="What triggers to use to send frames to generative AI for a tracked object.",
     )
 
+    enabled_in_config: Optional[bool] = Field(
+        default=None, title="Keep track of original state of generative AI."
+    )
+
     @field_validator("required_zones", mode="before")
     @classmethod
     def validate_required_zones(cls, v):

frigate/config/camera/updater.py

@@ -17,12 +17,14 @@ class CameraConfigUpdateEnum(str, Enum):
     birdseye = "birdseye"
     detect = "detect"
     enabled = "enabled"
+    genai = "genai"
     motion = "motion"  # includes motion and motion masks
     notifications = "notifications"
     objects = "objects"
     record = "record"
     remove = "remove"  # for removing a camera
     review = "review"
+    semantic_search = "semantic_search"  # for semantic search triggers
     snapshots = "snapshots"
     zones = "zones"
 
@@ -96,6 +98,8 @@ class CameraConfigUpdateSubscriber:
             config.detect = updated_config
         elif update_type == CameraConfigUpdateEnum.enabled:
             config.enabled = updated_config
+        elif update_type == CameraConfigUpdateEnum.genai:
+            config.genai = updated_config
         elif update_type == CameraConfigUpdateEnum.motion:
             config.motion = updated_config
         elif update_type == CameraConfigUpdateEnum.notifications:
@@ -106,6 +110,8 @@ class CameraConfigUpdateSubscriber:
             config.record = updated_config
         elif update_type == CameraConfigUpdateEnum.review:
             config.review = updated_config
+        elif update_type == CameraConfigUpdateEnum.semantic_search:
+            config.semantic_search = updated_config
         elif update_type == CameraConfigUpdateEnum.snapshots:
             config.snapshots = updated_config
         elif update_type == CameraConfigUpdateEnum.zones:

frigate/config/classification.py

@@ -10,6 +10,7 @@ __all__ = [
     "CameraLicensePlateRecognitionConfig",
     "FaceRecognitionConfig",
     "SemanticSearchConfig",
+    "CameraSemanticSearchConfig",
     "LicensePlateRecognitionConfig",
 ]
 
@@ -24,6 +25,15 @@ class EnrichmentsDeviceEnum(str, Enum):
     CPU = "CPU"
 
 
+class TriggerType(str, Enum):
+    THUMBNAIL = "thumbnail"
+    DESCRIPTION = "description"
+
+
+class TriggerAction(str, Enum):
+    NOTIFICATION = "notification"
+
+
 class AudioTranscriptionConfig(FrigateBaseModel):
     enabled: bool = Field(default=False, title="Enable audio transcription.")
     language: str = Field(
@@ -59,9 +69,6 @@ class CustomClassificationStateCameraConfig(FrigateBaseModel):
     crop: list[int, int, int, int] = Field(
         title="Crop of image frame on this camera to run classification on."
     )
-    threshold: float = Field(
-        default=0.8, title="Classification score threshold to change the state."
-    )
 
 
 class CustomClassificationStateConfig(FrigateBaseModel):
@@ -86,6 +93,9 @@ class CustomClassificationObjectConfig(FrigateBaseModel):
 class CustomClassificationConfig(FrigateBaseModel):
     enabled: bool = Field(default=True, title="Enable running the model.")
     name: str | None = Field(default=None, title="Name of classification model.")
+    threshold: float = Field(
+        default=0.8, title="Classification score threshold to change the state."
+    )
     object_config: CustomClassificationObjectConfig | None = Field(default=None)
     state_config: CustomClassificationStateConfig | None = Field(default=None)
 
@@ -113,6 +123,32 @@ class SemanticSearchConfig(FrigateBaseModel):
     )
 
 
+class TriggerConfig(FrigateBaseModel):
+    enabled: bool = Field(default=True, title="Enable this trigger")
+    type: TriggerType = Field(default=TriggerType.DESCRIPTION, title="Type of trigger")
+    data: str = Field(title="Trigger content (text phrase or image ID)")
+    threshold: float = Field(
+        title="Confidence score required to run the trigger",
+        default=0.8,
+        gt=0.0,
+        le=1.0,
+    )
+    actions: Optional[List[TriggerAction]] = Field(
+        default=[], title="Actions to perform when trigger is matched"
+    )
+
+    model_config = ConfigDict(extra="forbid", protected_namespaces=())
+
+
+class CameraSemanticSearchConfig(FrigateBaseModel):
+    triggers: Optional[Dict[str, TriggerConfig]] = Field(
+        default=None,
+        title="Trigger actions on tracked objects that match existing thumbnails or descriptions",
+    )
+
+    model_config = ConfigDict(extra="forbid", protected_namespaces=())
+
+
 class FaceRecognitionConfig(FrigateBaseModel):
     enabled: bool = Field(default=False, title="Enable face recognition.")
     model_size: str = Field(

frigate/config/config.py

@@ -606,6 +606,7 @@ class FrigateConfig(FrigateBaseModel):
             camera_config.review.detections.enabled_in_config = (
                 camera_config.review.detections.enabled
             )
+            camera_config.genai.enabled_in_config = camera_config.genai.enabled
 
             # Add default filters
             object_keys = camera_config.objects.track

frigate/const.py

@@ -11,6 +11,7 @@ EXPORT_DIR = f"{BASE_DIR}/exports"
 FACE_DIR = f"{CLIPS_DIR}/faces"
 THUMB_DIR = f"{CLIPS_DIR}/thumbs"
 RECORD_DIR = f"{BASE_DIR}/recordings"
+TRIGGER_DIR = f"{CLIPS_DIR}/triggers"
 BIRDSEYE_PIPE = "/tmp/cache/birdseye"
 CACHE_DIR = "/tmp/cache"
 FRIGATE_LOCALHOST = "http://127.0.0.1:5000"

frigate/data_processing/common/face/model.py

@@ -11,6 +11,7 @@ from scipy import stats
 from frigate.config import FrigateConfig
 from frigate.const import MODEL_CACHE_DIR
 from frigate.embeddings.onnx.face_embedding import ArcfaceEmbedding, FaceNetEmbedding
+from frigate.log import redirect_output_to_logger
 
 logger = logging.getLogger(__name__)
 
@@ -37,6 +38,7 @@ class FaceRecognizer(ABC):
     def classify(self, face_image: np.ndarray) -> tuple[str, float] | None:
         pass
 
+    @redirect_output_to_logger(logger, logging.DEBUG)
     def init_landmark_detector(self) -> None:
         landmark_model = os.path.join(MODEL_CACHE_DIR, "facedet/landmarkdet.yaml")
 

frigate/data_processing/post/semantic_trigger.py

@@ -0,0 +1,233 @@
+"""Post time processor to trigger actions based on similar embeddings."""
+
+import datetime
+import json
+import logging
+import os
+from typing import Any
+
+import cv2
+import numpy as np
+from peewee import DoesNotExist
+
+from frigate.comms.inter_process import InterProcessRequestor
+from frigate.config import FrigateConfig
+from frigate.const import CONFIG_DIR
+from frigate.data_processing.types import PostProcessDataEnum
+from frigate.db.sqlitevecq import SqliteVecQueueDatabase
+from frigate.embeddings.util import ZScoreNormalization
+from frigate.models import Event, Trigger
+from frigate.util.builtin import cosine_distance
+from frigate.util.path import get_event_thumbnail_bytes
+
+from ..post.api import PostProcessorApi
+from ..types import DataProcessorMetrics
+
+logger = logging.getLogger(__name__)
+
+WRITE_DEBUG_IMAGES = False
+
+
+class SemanticTriggerProcessor(PostProcessorApi):
+    def __init__(
+        self,
+        db: SqliteVecQueueDatabase,
+        config: FrigateConfig,
+        requestor: InterProcessRequestor,
+        metrics: DataProcessorMetrics,
+        embeddings,
+    ):
+        super().__init__(config, metrics, None)
+        self.db = db
+        self.embeddings = embeddings
+        self.requestor = requestor
+        self.trigger_embeddings: list[np.ndarray] = []
+
+        self.thumb_stats = ZScoreNormalization()
+        self.desc_stats = ZScoreNormalization()
+
+        # load stats from disk
+        try:
+            with open(os.path.join(CONFIG_DIR, ".search_stats.json"), "r") as f:
+                data = json.loads(f.read())
+                self.thumb_stats.from_dict(data["thumb_stats"])
+                self.desc_stats.from_dict(data["desc_stats"])
+        except FileNotFoundError:
+            pass
+
+    def process_data(
+        self, data: dict[str, Any], data_type: PostProcessDataEnum
+    ) -> None:
+        event_id = data["event_id"]
+        camera = data["camera"]
+        process_type = data["type"]
+
+        if self.config.cameras[camera].semantic_search.triggers is None:
+            return
+
+        triggers = (
+            Trigger.select(
+                Trigger.camera,
+                Trigger.name,
+                Trigger.data,
+                Trigger.type,
+                Trigger.embedding,
+                Trigger.threshold,
+            )
+            .where(Trigger.camera == camera)
+            .dicts()
+            .iterator()
+        )
+
+        for trigger in triggers:
+            if (
+                trigger["name"]
+                not in self.config.cameras[camera].semantic_search.triggers
+                or not self.config.cameras[camera]
+                .semantic_search.triggers[trigger["name"]]
+                .enabled
+            ):
+                logger.debug(
+                    f"Trigger {trigger['name']} is disabled for camera {camera}"
+                )
+                continue
+
+            logger.debug(
+                f"Processing {trigger['type']} trigger for {event_id} on {trigger['camera']}: {trigger['name']}"
+            )
+
+            trigger_embedding = np.frombuffer(trigger["embedding"], dtype=np.float32)
+
+            # Get embeddings based on type
+            thumbnail_embedding = None
+            description_embedding = None
+
+            if process_type == "image":
+                cursor = self.db.execute_sql(
+                    """
+                    SELECT thumbnail_embedding FROM vec_thumbnails WHERE id = ?
+                    """,
+                    [event_id],
+                )
+                row = cursor.fetchone() if cursor else None
+                if row:
+                    thumbnail_embedding = np.frombuffer(row[0], dtype=np.float32)
+
+            if process_type == "text":
+                cursor = self.db.execute_sql(
+                    """
+                    SELECT description_embedding FROM vec_descriptions WHERE id = ?
+                    """,
+                    [event_id],
+                )
+                row = cursor.fetchone() if cursor else None
+                if row:
+                    description_embedding = np.frombuffer(row[0], dtype=np.float32)
+
+            # Skip processing if we don't have any embeddings
+            if thumbnail_embedding is None and description_embedding is None:
+                logger.debug(f"No embeddings found for {event_id}")
+                return
+
+            # Determine which embedding to compare based on trigger type
+            if (
+                trigger["type"] in ["text", "thumbnail"]
+                and thumbnail_embedding is not None
+            ):
+                data_embedding = thumbnail_embedding
+                normalized_distance = self.thumb_stats.normalize(
+                    [cosine_distance(data_embedding, trigger_embedding)],
+                    save_stats=False,
+                )[0]
+            elif trigger["type"] == "description" and description_embedding is not None:
+                data_embedding = description_embedding
+                normalized_distance = self.desc_stats.normalize(
+                    [cosine_distance(data_embedding, trigger_embedding)],
+                    save_stats=False,
+                )[0]
+
+            else:
+                continue
+
+            similarity = 1 - normalized_distance
+
+            logger.debug(
+                f"Trigger {trigger['name']} ({trigger['data'] if trigger['type'] == 'text' or trigger['type'] == 'description' else 'image'}): "
+                f"normalized distance: {normalized_distance:.4f}, "
+                f"similarity: {similarity:.4f}, threshold: {trigger['threshold']}"
+            )
+
+            # Check if similarity meets threshold
+            if similarity >= trigger["threshold"]:
+                logger.info(
+                    f"Trigger {trigger['name']} activated with similarity {similarity:.4f}"
+                )
+
+                # Update the trigger's last_triggered and triggering_event_id
+                Trigger.update(
+                    last_triggered=datetime.datetime.now(), triggering_event_id=event_id
+                ).where(
+                    Trigger.camera == camera, Trigger.name == trigger["name"]
+                ).execute()
+
+                # Always publish MQTT message
+                self.requestor.send_data(
+                    "triggers",
+                    json.dumps(
+                        {
+                            "name": trigger["name"],
+                            "camera": camera,
+                            "event_id": event_id,
+                            "type": trigger["type"],
+                            "score": similarity,
+                        }
+                    ),
+                )
+
+                if (
+                    self.config.cameras[camera]
+                    .semantic_search.triggers[trigger["name"]]
+                    .actions
+                ):
+                    # TODO: handle actions for the trigger
+                    # notifications already handled by webpush
+                    pass
+
+            if WRITE_DEBUG_IMAGES:
+                try:
+                    event: Event = Event.get(Event.id == event_id)
+                except DoesNotExist:
+                    return
+
+                # Skip the event if not an object
+                if event.data.get("type") != "object":
+                    return
+
+                thumbnail_bytes = get_event_thumbnail_bytes(event)
+
+                nparr = np.frombuffer(thumbnail_bytes, np.uint8)
+                thumbnail = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
+
+                font_scale = 0.5
+                font = cv2.FONT_HERSHEY_SIMPLEX
+                cv2.putText(
+                    thumbnail,
+                    f"{similarity:.4f}",
+                    (10, 30),
+                    font,
+                    fontScale=font_scale,
+                    color=(0, 255, 0),
+                    thickness=2,
+                )
+
+                current_time = int(datetime.datetime.now().timestamp())
+                cv2.imwrite(
+                    f"debug/frames/trigger-{event_id}_{current_time}.jpg",
+                    thumbnail,
+                )
+
+    def handle_request(self, topic, request_data):
+        return None
+
+    def expire_object(self, object_id, camera):
+        pass

frigate/data_processing/real_time/bird.py

@@ -13,6 +13,7 @@ from frigate.comms.event_metadata_updater import (
 )
 from frigate.config import FrigateConfig
 from frigate.const import MODEL_CACHE_DIR
+from frigate.log import redirect_output_to_logger
 from frigate.util.object import calculate_region
 
 from ..types import DataProcessorMetrics
@@ -76,6 +77,7 @@ class BirdRealTimeProcessor(RealTimeProcessorApi):
         except Exception as e:
             logger.error(f"Failed to download {path}: {e}")
 
+    @redirect_output_to_logger(logger, logging.DEBUG)
     def __build_detector(self) -> None:
         self.interpreter = Interpreter(
             model_path=os.path.join(MODEL_CACHE_DIR, "bird/bird.tflite"),

frigate/data_processing/real_time/custom_classification.py

@@ -17,6 +17,7 @@ from frigate.comms.inter_process import InterProcessRequestor
 from frigate.config import FrigateConfig
 from frigate.config.classification import CustomClassificationConfig
 from frigate.const import CLIPS_DIR, MODEL_CACHE_DIR
+from frigate.log import redirect_output_to_logger
 from frigate.util.builtin import EventsPerSecond, InferenceSpeed, load_labels
 from frigate.util.object import box_overlaps, calculate_region
 
@@ -55,6 +56,7 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
         self.last_run = datetime.datetime.now().timestamp()
         self.__build_detector()
 
+    @redirect_output_to_logger(logger, logging.DEBUG)
     def __build_detector(self) -> None:
         self.interpreter = Interpreter(
             model_path=os.path.join(self.model_dir, "model.tflite"),
@@ -150,7 +152,7 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
             score,
         )
 
-        if score >= camera_config.threshold:
+        if score >= self.model_config.threshold:
             self.requestor.send_data(
                 f"{camera}/classification/{self.model_config.name}",
                 self.labelmap[best_id],
@@ -187,7 +189,7 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
         super().__init__(config, metrics)
         self.model_config = model_config
         self.model_dir = os.path.join(MODEL_CACHE_DIR, self.model_config.name)
-        self.train_dir = os.path.join(self.model_dir, "train")
+        self.train_dir = os.path.join(CLIPS_DIR, self.model_config.name, "train")
         self.interpreter: Interpreter = None
         self.sub_label_publisher = sub_label_publisher
         self.tensor_input_details: dict[str, Any] = None
@@ -200,6 +202,7 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
         )
         self.__build_detector()
 
+    @redirect_output_to_logger(logger, logging.DEBUG)
     def __build_detector(self) -> None:
         self.interpreter = Interpreter(
             model_path=os.path.join(self.model_dir, "model.tflite"),
@@ -222,6 +225,9 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
             self.model_config.name
         ].value = self.classifications_per_second.eps()
 
+        if obj_data["false_positive"]:
+            return
+
         if obj_data["label"] not in self.model_config.object_config.objects:
             return
 
@@ -232,20 +238,23 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
             obj_data["box"][1],
             obj_data["box"][2],
             obj_data["box"][3],
-            224,
+            max(
+                obj_data["box"][1] - obj_data["box"][0],
+                obj_data["box"][3] - obj_data["box"][2],
+            ),
             1.0,
         )
 
         rgb = cv2.cvtColor(frame, cv2.COLOR_YUV2RGB_I420)
-        input = rgb[
+        crop = rgb[
             y:y2,
             x:x2,
         ]
 
-        if input.shape != (224, 224):
-            input = cv2.resize(input, (224, 224))
+        if crop.shape != (224, 224):
+            crop = cv2.resize(crop, (224, 224))
 
-        input = np.expand_dims(input, axis=0)
+        input = np.expand_dims(crop, axis=0)
         self.interpreter.set_tensor(self.tensor_input_details[0]["index"], input)
         self.interpreter.invoke()
         res: np.ndarray = self.interpreter.get_tensor(
@@ -259,22 +268,29 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
 
         write_classification_attempt(
             self.train_dir,
-            cv2.cvtColor(frame, cv2.COLOR_RGB2BGR),
+            cv2.cvtColor(crop, cv2.COLOR_RGB2BGR),
             now,
             self.labelmap[best_id],
             score,
         )
 
+        if score < self.model_config.threshold:
+            logger.debug(f"Score {score} is less than threshold.")
+            return
+
         if score <= previous_score:
             logger.debug(f"Score {score} is worse than previous score {previous_score}")
             return
 
-        self.sub_label_publisher.publish(
-            EventMetadataTypeEnum.sub_label,
-            (obj_data["id"], self.labelmap[best_id], score),
-        )
+        sub_label = self.labelmap[best_id]
         self.detected_objects[obj_data["id"]] = score
 
+        if sub_label != "none":
+            self.sub_label_publisher.publish(
+                EventMetadataTypeEnum.sub_label,
+                (obj_data["id"], sub_label, score),
+            )
+
     def handle_request(self, topic, request_data):
         if topic == EmbeddingsRequestEnum.reload_classification_model.value:
             if request_data.get("model_name") == self.model_config.name:

frigate/detectors/plugins/cpu_tfl.py

@@ -5,6 +5,7 @@ from typing_extensions import Literal
 
 from frigate.detectors.detection_api import DetectionApi
 from frigate.detectors.detector_config import BaseDetectorConfig
+from frigate.log import redirect_output_to_logger
 
 from ..detector_utils import tflite_detect_raw, tflite_init
 
@@ -27,6 +28,7 @@ class CpuDetectorConfig(BaseDetectorConfig):
 class CpuTfl(DetectionApi):
     type_key = DETECTOR_KEY
 
+    @redirect_output_to_logger(logger, logging.DEBUG)
     def __init__(self, detector_config: CpuDetectorConfig):
         interpreter = Interpreter(
             model_path=detector_config.model.path,

frigate/embeddings/__init__.py

@@ -5,6 +5,7 @@ import json
 import logging
 import os
 import threading
+from multiprocessing.synchronize import Event as MpEvent
 from typing import Any, Union
 
 import regex
@@ -28,9 +29,12 @@ logger = logging.getLogger(__name__)
 
 class EmbeddingProcess(FrigateProcess):
     def __init__(
-        self, config: FrigateConfig, metrics: DataProcessorMetrics | None
+        self,
+        config: FrigateConfig,
+        metrics: DataProcessorMetrics | None,
+        stop_event: MpEvent,
     ) -> None:
-        super().__init__(name="frigate.embeddings_manager", daemon=True)
+        super().__init__(stop_event, name="frigate.embeddings_manager", daemon=True)
         self.config = config
         self.metrics = metrics
 
@@ -283,3 +287,15 @@ class EmbeddingsContext:
         return self.requestor.send_data(
             EmbeddingsRequestEnum.transcribe_audio.value, {"event": event}
         )
+
+    def generate_description_embedding(self, text: str) -> None:
+        return self.requestor.send_data(
+            EmbeddingsRequestEnum.embed_description.value,
+            {"id": None, "description": text, "upsert": False},
+        )
+
+    def generate_image_embedding(self, event_id: str, thumbnail: bytes) -> None:
+        return self.requestor.send_data(
+            EmbeddingsRequestEnum.embed_thumbnail.value,
+            {"id": str(event_id), "thumbnail": str(thumbnail), "upsert": False},
+        )

frigate/embeddings/embeddings.py

@@ -6,20 +6,25 @@ import os
 import threading
 import time
 
-from numpy import ndarray
+import numpy as np
+from peewee import DoesNotExist, IntegrityError
 from playhouse.shortcuts import model_to_dict
 
+from frigate.comms.embeddings_updater import (
+    EmbeddingsRequestEnum,
+)
 from frigate.comms.inter_process import InterProcessRequestor
 from frigate.config import FrigateConfig
 from frigate.config.classification import SemanticSearchModelEnum
 from frigate.const import (
     CONFIG_DIR,
+    TRIGGER_DIR,
     UPDATE_EMBEDDINGS_REINDEX_PROGRESS,
     UPDATE_MODEL_STATE,
 )
 from frigate.data_processing.types import DataProcessorMetrics
 from frigate.db.sqlitevecq import SqliteVecQueueDatabase
-from frigate.models import Event
+from frigate.models import Event, Trigger
 from frigate.types import ModelStatusTypesEnum
 from frigate.util.builtin import EventsPerSecond, InferenceSpeed, serialize
 from frigate.util.path import get_event_thumbnail_bytes
@@ -165,7 +170,7 @@ class Embeddings:
 
     def embed_thumbnail(
         self, event_id: str, thumbnail: bytes, upsert: bool = True
-    ) -> ndarray:
+    ) -> np.ndarray:
         """Embed thumbnail and optionally insert into DB.
 
         @param: event_id in Events DB
@@ -192,7 +197,7 @@ class Embeddings:
 
     def batch_embed_thumbnail(
         self, event_thumbs: dict[str, bytes], upsert: bool = True
-    ) -> list[ndarray]:
+    ) -> list[np.ndarray]:
         """Embed thumbnails and optionally insert into DB.
 
         @param: event_thumbs Map of Event IDs in DB to thumbnail bytes in jpg format
@@ -225,7 +230,7 @@ class Embeddings:
 
     def embed_description(
         self, event_id: str, description: str, upsert: bool = True
-    ) -> ndarray:
+    ) -> np.ndarray:
         start = datetime.datetime.now().timestamp()
         embedding = self.text_embedding([description])[0]
 
@@ -245,7 +250,7 @@ class Embeddings:
 
     def batch_embed_description(
         self, event_descriptions: dict[str, str], upsert: bool = True
-    ) -> ndarray:
+    ) -> np.ndarray:
         start = datetime.datetime.now().timestamp()
         # upsert embeddings one by one to avoid token limit
         embeddings = []
@@ -401,3 +406,224 @@ class Embeddings:
             with self.reindex_lock:
                 self.reindex_running = False
                 self.reindex_thread = None
+
+    def sync_triggers(self) -> None:
+        for camera in self.config.cameras.values():
+            # Get all existing triggers for this camera
+            existing_triggers = {
+                trigger.name: trigger
+                for trigger in Trigger.select().where(Trigger.camera == camera.name)
+            }
+
+            # Get all configured trigger names
+            configured_trigger_names = set(camera.semantic_search.triggers or {})
+
+            # Create or update triggers from config
+            for trigger_name, trigger in (
+                camera.semantic_search.triggers or {}
+            ).items():
+                if trigger_name in existing_triggers:
+                    existing_trigger = existing_triggers[trigger_name]
+                    needs_embedding_update = False
+                    thumbnail_missing = False
+
+                    # Check if data has changed or thumbnail is missing for thumbnail type
+                    if trigger.type == "thumbnail":
+                        thumbnail_path = os.path.join(
+                            TRIGGER_DIR, camera.name, f"{trigger.data}.webp"
+                        )
+                        try:
+                            event = Event.get(Event.id == trigger.data)
+                            if event.data.get("type") != "object":
+                                logger.warning(
+                                    f"Event {trigger.data} is not a tracked object for {trigger.type} trigger"
+                                )
+                                continue  # Skip if not an object
+
+                            # Check if thumbnail needs to be updated (data changed or missing)
+                            if (
+                                existing_trigger.data != trigger.data
+                                or not os.path.exists(thumbnail_path)
+                            ):
+                                thumbnail = get_event_thumbnail_bytes(event)
+                                if not thumbnail:
+                                    logger.warning(
+                                        f"Unable to retrieve thumbnail for event ID {trigger.data} for {trigger_name}."
+                                    )
+                                    continue
+                                self.write_trigger_thumbnail(
+                                    camera.name, trigger.data, thumbnail
+                                )
+                                thumbnail_missing = True
+                        except DoesNotExist:
+                            logger.warning(
+                                f"Event ID {trigger.data} for trigger {trigger_name} does not exist."
+                            )
+                            continue
+
+                    # Update existing trigger if data has changed
+                    if (
+                        existing_trigger.type != trigger.type
+                        or existing_trigger.data != trigger.data
+                        or existing_trigger.threshold != trigger.threshold
+                    ):
+                        existing_trigger.type = trigger.type
+                        existing_trigger.data = trigger.data
+                        existing_trigger.threshold = trigger.threshold
+                        needs_embedding_update = True
+
+                    # Check if embedding is missing or needs update
+                    if (
+                        not existing_trigger.embedding
+                        or needs_embedding_update
+                        or thumbnail_missing
+                    ):
+                        existing_trigger.embedding = self._calculate_trigger_embedding(
+                            trigger
+                        )
+                        needs_embedding_update = True
+
+                    if needs_embedding_update:
+                        existing_trigger.save()
+                else:
+                    # Create new trigger
+                    try:
+                        try:
+                            event: Event = Event.get(Event.id == trigger.data)
+                        except DoesNotExist:
+                            logger.warning(
+                                f"Event ID {trigger.data} for trigger {trigger_name} does not exist."
+                            )
+                            continue
+
+                        # Skip the event if not an object
+                        if event.data.get("type") != "object":
+                            logger.warning(
+                                f"Event ID {trigger.data} for trigger {trigger_name} is not a tracked object."
+                            )
+                            continue
+
+                        thumbnail = get_event_thumbnail_bytes(event)
+
+                        if not thumbnail:
+                            logger.warning(
+                                f"Unable to retrieve thumbnail for event ID {trigger.data} for {trigger_name}."
+                            )
+                            continue
+
+                        self.write_trigger_thumbnail(
+                            camera.name, trigger.data, thumbnail
+                        )
+
+                        # Calculate embedding for new trigger
+                        embedding = self._calculate_trigger_embedding(trigger)
+
+                        Trigger.create(
+                            camera=camera.name,
+                            name=trigger_name,
+                            type=trigger.type,
+                            data=trigger.data,
+                            threshold=trigger.threshold,
+                            model=self.config.semantic_search.model,
+                            embedding=embedding,
+                            triggering_event_id="",
+                            last_triggered=None,
+                        )
+
+                    except IntegrityError:
+                        pass  # Handle duplicate creation attempts
+
+            # Remove triggers that are no longer in config
+            triggers_to_remove = (
+                set(existing_triggers.keys()) - configured_trigger_names
+            )
+            if triggers_to_remove:
+                Trigger.delete().where(
+                    Trigger.camera == camera.name, Trigger.name.in_(triggers_to_remove)
+                ).execute()
+                for trigger_name in triggers_to_remove:
+                    self.remove_trigger_thumbnail(camera.name, trigger_name)
+
+    def write_trigger_thumbnail(
+        self, camera: str, event_id: str, thumbnail: bytes
+    ) -> None:
+        """Write the thumbnail to the trigger directory."""
+        try:
+            os.makedirs(os.path.join(TRIGGER_DIR, camera), exist_ok=True)
+            with open(os.path.join(TRIGGER_DIR, camera, f"{event_id}.webp"), "wb") as f:
+                f.write(thumbnail)
+            logger.debug(
+                f"Writing thumbnail for trigger with data {event_id} in {camera}."
+            )
+        except Exception as e:
+            logger.error(
+                f"Failed to write thumbnail for trigger with data {event_id} in {camera}: {e}"
+            )
+
+    def remove_trigger_thumbnail(self, camera: str, event_id: str) -> None:
+        """Write the thumbnail to the trigger directory."""
+        try:
+            os.remove(os.path.join(TRIGGER_DIR, camera, f"{event_id}.webp"))
+            logger.debug(
+                f"Deleted thumbnail for trigger with data {event_id} in {camera}."
+            )
+        except Exception as e:
+            logger.error(
+                f"Failed to delete thumbnail for trigger with data {event_id} in {camera}: {e}"
+            )
+
+    def _calculate_trigger_embedding(self, trigger) -> bytes:
+        """Calculate embedding for a trigger based on its type and data."""
+        if trigger.type == "description":
+            logger.debug(f"Generating embedding for trigger description {trigger.name}")
+            embedding = self.requestor.send_data(
+                EmbeddingsRequestEnum.embed_description.value,
+                {"id": None, "description": trigger.data, "upsert": False},
+            )
+            return embedding.astype(np.float32).tobytes()
+
+        elif trigger.type == "thumbnail":
+            # For image triggers, trigger.data should be an image ID
+            # Try to get embedding from vec_thumbnails table first
+            cursor = self.db.execute_sql(
+                "SELECT thumbnail_embedding FROM vec_thumbnails WHERE id = ?",
+                [trigger.data],
+            )
+            row = cursor.fetchone() if cursor else None
+            if row:
+                return row[0]  # Already in bytes format
+            else:
+                logger.debug(
+                    f"No thumbnail embedding found for image ID: {trigger.data}, generating from saved trigger thumbnail"
+                )
+
+                try:
+                    with open(
+                        os.path.join(
+                            TRIGGER_DIR, trigger.camera, f"{trigger.data}.webp"
+                        ),
+                        "rb",
+                    ) as f:
+                        thumbnail = f.read()
+                except Exception as e:
+                    logger.error(
+                        f"Failed to read thumbnail for trigger {trigger.name} with ID {trigger.data}: {e}"
+                    )
+                    return b""
+
+                logger.debug(
+                    f"Generating embedding for trigger thumbnail {trigger.name} with ID {trigger.data}"
+                )
+                embedding = self.requestor.send_data(
+                    EmbeddingsRequestEnum.embed_thumbnail.value,
+                    {
+                        "id": str(trigger.data),
+                        "thumbnail": str(thumbnail),
+                        "upsert": False,
+                    },
+                )
+                return embedding.astype(np.float32).tobytes()
+
+        else:
+            logger.warning(f"Unknown trigger type: {trigger.type}")
+            return b""

frigate/embeddings/maintainer.py

@@ -14,7 +14,10 @@ import numpy as np
 from peewee import DoesNotExist
 
 from frigate.comms.detections_updater import DetectionSubscriber, DetectionTypeEnum
-from frigate.comms.embeddings_updater import EmbeddingsRequestEnum, EmbeddingsResponder
+from frigate.comms.embeddings_updater import (
+    EmbeddingsRequestEnum,
+    EmbeddingsResponder,
+)
 from frigate.comms.event_metadata_updater import (
     EventMetadataPublisher,
     EventMetadataSubscriber,
@@ -46,6 +49,7 @@ from frigate.data_processing.post.audio_transcription import (
 from frigate.data_processing.post.license_plate import (
     LicensePlatePostProcessor,
 )
+from frigate.data_processing.post.semantic_trigger import SemanticTriggerProcessor
 from frigate.data_processing.real_time.api import RealTimeProcessorApi
 from frigate.data_processing.real_time.bird import BirdRealTimeProcessor
 from frigate.data_processing.real_time.custom_classification import (
@@ -60,7 +64,7 @@ from frigate.data_processing.types import DataProcessorMetrics, PostProcessDataE
 from frigate.db.sqlitevecq import SqliteVecQueueDatabase
 from frigate.events.types import EventTypeEnum, RegenerateDescriptionEnum
 from frigate.genai import get_genai_client
-from frigate.models import Event, Recordings
+from frigate.models import Event, Recordings, Trigger
 from frigate.types import TrackedObjectUpdateTypesEnum
 from frigate.util.builtin import serialize
 from frigate.util.image import (
@@ -93,7 +97,12 @@ class EmbeddingMaintainer(threading.Thread):
         self.config_updater = CameraConfigUpdateSubscriber(
             self.config,
             self.config.cameras,
-            [CameraConfigUpdateEnum.add, CameraConfigUpdateEnum.remove],
+            [
+                CameraConfigUpdateEnum.add,
+                CameraConfigUpdateEnum.remove,
+                CameraConfigUpdateEnum.genai,
+                CameraConfigUpdateEnum.semantic_search,
+            ],
         )
 
         # Configure Frigate DB
@@ -109,7 +118,7 @@ class EmbeddingMaintainer(threading.Thread):
             ),
             load_vec_extension=True,
         )
-        models = [Event, Recordings]
+        models = [Event, Recordings, Trigger]
         db.bind(models)
 
         if config.semantic_search.enabled:
@@ -119,6 +128,9 @@ class EmbeddingMaintainer(threading.Thread):
             if config.semantic_search.reindex:
                 self.embeddings.reindex()
 
+            # Sync semantic search triggers in db with config
+            self.embeddings.sync_triggers()
+
         # create communication for updating event descriptions
         self.requestor = InterProcessRequestor()
 
@@ -211,6 +223,17 @@ class EmbeddingMaintainer(threading.Thread):
                 AudioTranscriptionPostProcessor(self.config, self.requestor, metrics)
             )
 
+        if self.config.semantic_search.enabled:
+            self.post_processors.append(
+                SemanticTriggerProcessor(
+                    db,
+                    self.config,
+                    self.requestor,
+                    metrics,
+                    self.embeddings,
+                )
+            )
+
         self.stop_event = stop_event
         self.tracked_events: dict[str, list[Any]] = {}
         self.early_request_sent: dict[str, bool] = {}
@@ -387,33 +410,6 @@ class EmbeddingMaintainer(threading.Thread):
             event_id, camera, updated_db = ended
             camera_config = self.config.cameras[camera]
 
-            # call any defined post processors
-            for processor in self.post_processors:
-                if isinstance(processor, LicensePlatePostProcessor):
-                    recordings_available = self.recordings_available_through.get(camera)
-                    if (
-                        recordings_available is not None
-                        and event_id in self.detected_license_plates
-                        and self.config.cameras[camera].type != "lpr"
-                    ):
-                        processor.process_data(
-                            {
-                                "event_id": event_id,
-                                "camera": camera,
-                                "recordings_available": self.recordings_available_through[
-                                    camera
-                                ],
-                                "obj_data": self.detected_license_plates[event_id][
-                                    "obj_data"
-                                ],
-                            },
-                            PostProcessDataEnum.recording,
-                        )
-                elif isinstance(processor, AudioTranscriptionPostProcessor):
-                    continue
-                else:
-                    processor.process_data(event_id, PostProcessDataEnum.event_id)
-
             # expire in realtime processors
             for processor in self.realtime_processors:
                 processor.expire_object(event_id, camera)
@@ -450,6 +446,41 @@ class EmbeddingMaintainer(threading.Thread):
                 ):
                     self._process_genai_description(event, camera_config, thumbnail)
 
+            # call any defined post processors
+            for processor in self.post_processors:
+                if isinstance(processor, LicensePlatePostProcessor):
+                    recordings_available = self.recordings_available_through.get(camera)
+                    if (
+                        recordings_available is not None
+                        and event_id in self.detected_license_plates
+                        and self.config.cameras[camera].type != "lpr"
+                    ):
+                        processor.process_data(
+                            {
+                                "event_id": event_id,
+                                "camera": camera,
+                                "recordings_available": self.recordings_available_through[
+                                    camera
+                                ],
+                                "obj_data": self.detected_license_plates[event_id][
+                                    "obj_data"
+                                ],
+                            },
+                            PostProcessDataEnum.recording,
+                        )
+                elif isinstance(processor, AudioTranscriptionPostProcessor):
+                    continue
+                elif isinstance(processor, SemanticTriggerProcessor):
+                    processor.process_data(
+                        {"event_id": event_id, "camera": camera, "type": "image"},
+                        PostProcessDataEnum.tracked_object,
+                    )
+                else:
+                    processor.process_data(
+                        {"event_id": event_id, "camera": camera},
+                        PostProcessDataEnum.tracked_object,
+                    )
+
             # Delete tracked events based on the event_id
             if event_id in self.tracked_events:
                 del self.tracked_events[event_id]
@@ -658,6 +689,16 @@ class EmbeddingMaintainer(threading.Thread):
         if self.config.semantic_search.enabled:
             self.embeddings.embed_description(event.id, description)
 
+        # Check semantic trigger for this description
+        for processor in self.post_processors:
+            if isinstance(processor, SemanticTriggerProcessor):
+                processor.process_data(
+                    {"event_id": event.id, "camera": event.camera, "type": "text"},
+                    PostProcessDataEnum.tracked_object,
+                )
+            else:
+                continue
+
         logger.debug(
             "Generated description for %s (%d images): %s",
             event.id,

frigate/embeddings/onnx/face_embedding.py

@@ -6,6 +6,7 @@ import os
 import numpy as np
 
 from frigate.const import MODEL_CACHE_DIR
+from frigate.log import redirect_output_to_logger
 from frigate.util.downloader import ModelDownloader
 
 from .base_embedding import BaseEmbedding
@@ -53,6 +54,7 @@ class FaceNetEmbedding(BaseEmbedding):
             self._load_model_and_utils()
             logger.debug(f"models are already downloaded for {self.model_name}")
 
+    @redirect_output_to_logger(logger, logging.DEBUG)
     def _load_model_and_utils(self):
         if self.runner is None:
             if self.downloader:

frigate/events/audio.py

@@ -7,6 +7,7 @@ import string
 import threading
 import time
 from multiprocessing.managers import DictProxy
+from multiprocessing.synchronize import Event as MpEvent
 from typing import Any, Tuple
 
 import numpy as np
@@ -36,7 +37,7 @@ from frigate.data_processing.real_time.audio_transcription import (
     AudioTranscriptionRealTimeProcessor,
 )
 from frigate.ffmpeg_presets import parse_preset_input
-from frigate.log import LogPipe
+from frigate.log import LogPipe, redirect_output_to_logger
 from frigate.object_detection.base import load_labels
 from frigate.util.builtin import get_ffmpeg_arg_list
 from frigate.util.process import FrigateProcess
@@ -48,6 +49,9 @@ except ModuleNotFoundError:
     from tensorflow.lite.python.interpreter import Interpreter
 
 
+logger = logging.getLogger(__name__)
+
+
 def get_ffmpeg_command(ffmpeg: FfmpegConfig) -> list[str]:
     ffmpeg_input: CameraInput = [i for i in ffmpeg.inputs if "audio" in i.roles][0]
     input_args = get_ffmpeg_arg_list(ffmpeg.global_args) + (
@@ -84,8 +88,9 @@ class AudioProcessor(FrigateProcess):
         config: FrigateConfig,
         cameras: list[CameraConfig],
         camera_metrics: DictProxy,
+        stop_event: MpEvent,
     ):
-        super().__init__(name="frigate.audio_manager", daemon=True)
+        super().__init__(stop_event, name="frigate.audio_manager", daemon=True)
 
         self.camera_metrics = camera_metrics
         self.cameras = cameras
@@ -421,6 +426,7 @@ class AudioEventMaintainer(threading.Thread):
 
 
 class AudioTfl:
+    @redirect_output_to_logger(logger, logging.DEBUG)
     def __init__(self, stop_event: threading.Event, num_threads=2):
         self.stop_event = stop_event
         self.num_threads = num_threads

frigate/log.py

@@ -1,15 +1,18 @@
 # In log.py
 import atexit
+import io
 import logging
 import os
 import sys
 import threading
 from collections import deque
+from contextlib import contextmanager
 from enum import Enum
+from functools import wraps
 from logging.handlers import QueueHandler, QueueListener
 from multiprocessing.managers import SyncManager
-from queue import Queue
-from typing import Deque, Optional
+from queue import Empty, Queue
+from typing import Any, Callable, Deque, Generator, Optional
 
 from frigate.util.builtin import clean_camera_user_pass
 
@@ -77,6 +80,7 @@ def apply_log_levels(default: str, log_levels: dict[str, LogLevel]) -> None:
     log_levels = {
         "absl": LogLevel.error,
         "httpx": LogLevel.error,
+        "matplotlib": LogLevel.error,
         "tensorflow": LogLevel.error,
         "werkzeug": LogLevel.error,
         "ws4py": LogLevel.error,
@@ -102,11 +106,11 @@ os.register_at_fork(after_in_child=reopen_std_streams)
 
 # based on https://codereview.stackexchange.com/a/17959
 class LogPipe(threading.Thread):
-    def __init__(self, log_name: str):
+    def __init__(self, log_name: str, level: int = logging.ERROR):
         """Setup the object with a logger and start the thread"""
         super().__init__(daemon=False)
         self.logger = logging.getLogger(log_name)
-        self.level = logging.ERROR
+        self.level = level
         self.deque: Deque[str] = deque(maxlen=100)
         self.fdRead, self.fdWrite = os.pipe()
         self.pipeReader = os.fdopen(self.fdRead)
@@ -135,3 +139,182 @@ class LogPipe(threading.Thread):
     def close(self) -> None:
         """Close the write end of the pipe."""
         os.close(self.fdWrite)
+
+
+class LogRedirect(io.StringIO):
+    """
+    A custom file-like object to capture stdout and process it.
+    It extends io.StringIO to capture output and then processes it
+    line by line.
+    """
+
+    def __init__(self, logger_instance: logging.Logger, level: int):
+        super().__init__()
+        self.logger = logger_instance
+        self.log_level = level
+        self._line_buffer: list[str] = []
+
+    def write(self, s: Any) -> int:
+        if not isinstance(s, str):
+            s = str(s)
+
+        self._line_buffer.append(s)
+
+        # Process output line by line if a newline is present
+        if "\n" in s:
+            full_output = "".join(self._line_buffer)
+            lines = full_output.splitlines(keepends=True)
+            self._line_buffer = []
+
+            for line in lines:
+                if line.endswith("\n"):
+                    self._process_line(line.rstrip("\n"))
+                else:
+                    self._line_buffer.append(line)
+
+        return len(s)
+
+    def _process_line(self, line: str) -> None:
+        self.logger.log(self.log_level, line)
+
+    def flush(self) -> None:
+        if self._line_buffer:
+            full_output = "".join(self._line_buffer)
+            self._line_buffer = []
+            if full_output:  # Only process if there's content
+                self._process_line(full_output)
+
+    def __enter__(self) -> "LogRedirect":
+        """Context manager entry point."""
+        return self
+
+    def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
+        """Context manager exit point. Ensures buffered content is flushed."""
+        self.flush()
+
+
+@contextmanager
+def __redirect_fd_to_queue(queue: Queue[str]) -> Generator[None, None, None]:
+    """Redirect file descriptor 1 (stdout) to a pipe and capture output in a queue."""
+    stdout_fd = os.dup(1)
+    read_fd, write_fd = os.pipe()
+    os.dup2(write_fd, 1)
+    os.close(write_fd)
+
+    stop_event = threading.Event()
+
+    def reader() -> None:
+        """Read from pipe and put lines in queue until stop_event is set."""
+        try:
+            with os.fdopen(read_fd, "r") as pipe:
+                while not stop_event.is_set():
+                    line = pipe.readline()
+                    if not line:  # EOF
+                        break
+                    queue.put(line.strip())
+        except OSError as e:
+            queue.put(f"Reader error: {e}")
+        finally:
+            if not stop_event.is_set():
+                stop_event.set()
+
+    reader_thread = threading.Thread(target=reader, daemon=False)
+    reader_thread.start()
+
+    try:
+        yield
+    finally:
+        os.dup2(stdout_fd, 1)
+        os.close(stdout_fd)
+        stop_event.set()
+        reader_thread.join(timeout=1.0)
+        try:
+            os.close(read_fd)
+        except OSError:
+            pass
+
+
+def redirect_output_to_logger(logger: logging.Logger, level: int) -> Any:
+    """Decorator to redirect both Python sys.stdout/stderr and C-level stdout to logger."""
+
+    def decorator(func: Callable) -> Callable:
+        @wraps(func)
+        def wrapper(*args: Any, **kwargs: Any) -> Any:
+            queue: Queue[str] = Queue()
+
+            log_redirect = LogRedirect(logger, level)
+            old_stdout = sys.stdout
+            old_stderr = sys.stderr
+            sys.stdout = log_redirect
+            sys.stderr = log_redirect
+
+            try:
+                # Redirect C-level stdout
+                with __redirect_fd_to_queue(queue):
+                    result = func(*args, **kwargs)
+            finally:
+                # Restore Python stdout/stderr
+                sys.stdout = old_stdout
+                sys.stderr = old_stderr
+                log_redirect.flush()
+
+                # Log C-level output from queue
+                while True:
+                    try:
+                        logger.log(level, queue.get_nowait())
+                    except Empty:
+                        break
+
+            return result
+
+        return wrapper
+
+    return decorator
+
+
+def suppress_os_output(func: Callable) -> Callable:
+    """
+    A decorator that suppresses all output (stdout and stderr)
+    at the operating system file descriptor level for the decorated function.
+    This is useful for silencing noisy C/C++ libraries.
+    Note: This is a Unix-specific solution using os.dup2 and os.pipe.
+    It temporarily redirects file descriptors 1 (stdout) and 2 (stderr)
+    to a non-read pipe, effectively discarding their output.
+    """
+
+    @wraps(func)
+    def wrapper(*args: tuple, **kwargs: dict[str, Any]) -> Any:
+        # Save the original file descriptors for stdout (1) and stderr (2)
+        original_stdout_fd = os.dup(1)
+        original_stderr_fd = os.dup(2)
+
+        # Create dummy pipes. We only need the write ends to redirect to.
+        # The data written to these pipes will be discarded as nothing
+        # will read from the read ends.
+        devnull_read_fd, devnull_write_fd = os.pipe()
+
+        try:
+            # Redirect stdout (FD 1) and stderr (FD 2) to the write end of our dummy pipe
+            os.dup2(devnull_write_fd, 1)  # Redirect stdout to devnull pipe
+            os.dup2(devnull_write_fd, 2)  # Redirect stderr to devnull pipe
+
+            # Execute the original function
+            result = func(*args, **kwargs)
+
+        finally:
+            # Restore original stdout and stderr file descriptors (1 and 2)
+            # This is crucial to ensure normal printing resumes after the decorated function.
+            os.dup2(original_stdout_fd, 1)
+            os.dup2(original_stderr_fd, 2)
+
+            # Close all duplicated and pipe file descriptors to prevent resource leaks.
+            # It's important to close the read end of the dummy pipe too,
+            # as nothing is explicitly reading from it.
+            os.close(original_stdout_fd)
+            os.close(original_stderr_fd)
+            os.close(devnull_read_fd)
+            os.close(devnull_write_fd)
+
+        return result
+
+    return wrapper

frigate/models.py

@@ -1,6 +1,8 @@
 from peewee import (
+    BlobField,
     BooleanField,
     CharField,
+    CompositeKey,
     DateTimeField,
     FloatField,
     ForeignKeyField,
@@ -132,3 +134,18 @@ class User(Model):  # type: ignore[misc]
     )
     password_hash = CharField(null=False, max_length=120)
     notification_tokens = JSONField()
+
+
+class Trigger(Model):  # type: ignore[misc]
+    camera = CharField(max_length=20)
+    name = CharField()
+    type = CharField(max_length=10)
+    data = TextField()
+    threshold = FloatField()
+    model = CharField(max_length=30)
+    embedding = BlobField()
+    triggering_event_id = CharField(max_length=30)
+    last_triggered = DateTimeField()
+
+    class Meta:
+        primary_key = CompositeKey("camera", "name")

frigate/object_detection/base.py

@@ -95,8 +95,9 @@ class DetectorRunner(FrigateProcess):
         start_time: Value,
         config: FrigateConfig,
         detector_config: BaseDetectorConfig,
+        stop_event: MpEvent,
     ) -> None:
-        super().__init__(name=name, daemon=True)
+        super().__init__(stop_event, name=name, daemon=True)
         self.detection_queue = detection_queue
         self.cameras = cameras
         self.avg_speed = avg_speed
@@ -166,6 +167,7 @@ class ObjectDetectProcess:
         cameras: list[str],
         config: FrigateConfig,
         detector_config: BaseDetectorConfig,
+        stop_event: MpEvent,
     ):
         self.name = name
         self.cameras = cameras
@@ -175,6 +177,7 @@ class ObjectDetectProcess:
         self.detect_process: FrigateProcess | None = None
         self.config = config
         self.detector_config = detector_config
+        self.stop_event = stop_event
         self.start_or_restart()
 
     def stop(self):
@@ -202,6 +205,7 @@ class ObjectDetectProcess:
             self.detection_start,
             self.config,
             self.detector_config,
+            self.stop_event,
         )
         self.detect_process.start()
 

frigate/output/output.py

@@ -5,6 +5,7 @@ import logging
 import os
 import shutil
 import threading
+from multiprocessing.synchronize import Event as MpEvent
 from wsgiref.simple_server import make_server
 
 from ws4py.server.wsgirefserver import (
@@ -72,8 +73,8 @@ def check_disabled_camera_update(
 
 
 class OutputProcess(FrigateProcess):
-    def __init__(self, config: FrigateConfig) -> None:
-        super().__init__(name="frigate.output", daemon=True)
+    def __init__(self, config: FrigateConfig, stop_event: MpEvent) -> None:
+        super().__init__(stop_event, name="frigate.output", daemon=True)
         self.config = config
 
     def run(self) -> None:

frigate/ptz/onvif.py

@@ -33,6 +33,8 @@ class OnvifCommandEnum(str, Enum):
     stop = "stop"
     zoom_in = "zoom_in"
     zoom_out = "zoom_out"
+    focus_in = "focus_in"
+    focus_out = "focus_out"
 
 
 class OnvifController:
@@ -185,6 +187,16 @@ class OnvifController:
         ptz: ONVIFService = await onvif.create_ptz_service()
         self.cams[camera_name]["ptz"] = ptz
 
+        imaging: ONVIFService = await onvif.create_imaging_service()
+        self.cams[camera_name]["imaging"] = imaging
+        try:
+            video_sources = await media.GetVideoSources()
+            if video_sources and len(video_sources) > 0:
+                self.cams[camera_name]["video_source_token"] = video_sources[0].token
+        except (Fault, ONVIFError, TransportError, Exception) as e:
+            logger.debug(f"Unable to get video sources for {camera_name}: {e}")
+            self.cams[camera_name]["video_source_token"] = None
+
         # setup continuous moving request
         move_request = ptz.create_type("ContinuousMove")
         move_request.ProfileToken = profile.token
@@ -265,9 +277,15 @@ class OnvifController:
                             "RelativeZoomTranslationSpace"
                         ][zoom_space_id]["URI"]
                 else:
-                    if "Zoom" in move_request["Translation"]:
+                    if (
+                        move_request["Translation"] is not None
+                        and "Zoom" in move_request["Translation"]
+                    ):
                         del move_request["Translation"]["Zoom"]
-                    if "Zoom" in move_request["Speed"]:
+                    if (
+                        move_request["Speed"] is not None
+                        and "Zoom" in move_request["Speed"]
+                    ):
                         del move_request["Speed"]["Zoom"]
                     logger.debug(
                         f"{camera_name}: Relative move request after deleting zoom: {move_request}"
@@ -360,7 +378,19 @@ class OnvifController:
                             f"Disabling autotracking zooming for {camera_name}: Absolute zoom not supported. Exception: {e}"
                         )
 
-        # set relative pan/tilt space for autotracker
+        if self.cams[camera_name]["video_source_token"] is not None:
+            try:
+                imaging_capabilities = await imaging.GetImagingSettings(
+                    {"VideoSourceToken": self.cams[camera_name]["video_source_token"]}
+                )
+                if (
+                    hasattr(imaging_capabilities, "Focus")
+                    and imaging_capabilities.Focus
+                ):
+                    supported_features.append("focus")
+            except (Fault, ONVIFError, TransportError, Exception) as e:
+                logger.debug(f"Focus not supported for {camera_name}: {e}")
+
         if (
             self.config.cameras[camera_name].onvif.autotracking.enabled_in_config
             and self.config.cameras[camera_name].onvif.autotracking.enabled
@@ -385,6 +415,18 @@ class OnvifController:
                 "Zoom": True,
             }
         )
+        if (
+            "focus" in self.cams[camera_name]["features"]
+            and self.cams[camera_name]["video_source_token"]
+        ):
+            try:
+                stop_request = self.cams[camera_name]["imaging"].create_type("Stop")
+                stop_request.VideoSourceToken = self.cams[camera_name][
+                    "video_source_token"
+                ]
+                await self.cams[camera_name]["imaging"].Stop(stop_request)
+            except (Fault, ONVIFError, TransportError, Exception) as e:
+                logger.warning(f"Failed to stop focus for {camera_name}: {e}")
         self.cams[camera_name]["active"] = False
 
     async def _move(self, camera_name: str, command: OnvifCommandEnum) -> None:
@@ -593,6 +635,35 @@ class OnvifController:
 
         self.cams[camera_name]["active"] = False
 
+    async def _focus(self, camera_name: str, command: OnvifCommandEnum) -> None:
+        if self.cams[camera_name]["active"]:
+            logger.warning(
+                f"{camera_name} is already performing an action, not moving..."
+            )
+            await self._stop(camera_name)
+
+        if (
+            "focus" not in self.cams[camera_name]["features"]
+            or not self.cams[camera_name]["video_source_token"]
+        ):
+            logger.error(f"{camera_name} does not support ONVIF continuous focus.")
+            return
+
+        self.cams[camera_name]["active"] = True
+        move_request = self.cams[camera_name]["imaging"].create_type("Move")
+        move_request.VideoSourceToken = self.cams[camera_name]["video_source_token"]
+        move_request.Focus = {
+            "Continuous": {
+                "Speed": 0.5 if command == OnvifCommandEnum.focus_in else -0.5
+            }
+        }
+
+        try:
+            await self.cams[camera_name]["imaging"].Move(move_request)
+        except (Fault, ONVIFError, TransportError, Exception) as e:
+            logger.warning(f"Onvif sending focus request to {camera_name} failed: {e}")
+            self.cams[camera_name]["active"] = False
+
     async def handle_command_async(
         self, camera_name: str, command: OnvifCommandEnum, param: str = ""
     ) -> None:
@@ -616,11 +687,10 @@ class OnvifController:
             elif command == OnvifCommandEnum.move_relative:
                 _, pan, tilt = param.split("_")
                 await self._move_relative(camera_name, float(pan), float(tilt), 0, 1)
-            elif (
-                command == OnvifCommandEnum.zoom_in
-                or command == OnvifCommandEnum.zoom_out
-            ):
+            elif command in (OnvifCommandEnum.zoom_in, OnvifCommandEnum.zoom_out):
                 await self._zoom(camera_name, command)
+            elif command in (OnvifCommandEnum.focus_in, OnvifCommandEnum.focus_out):
+                await self._focus(camera_name, command)
             else:
                 await self._move(camera_name, command)
         except (Fault, ONVIFError, TransportError, Exception) as e:
@@ -631,7 +701,6 @@ class OnvifController:
     ) -> None:
         """
         Handle ONVIF commands by scheduling them in the event loop.
-        This is the synchronous interface that schedules async work.
         """
         future = asyncio.run_coroutine_threadsafe(
             self.handle_command_async(camera_name, command, param), self.loop

frigate/record/record.py

@@ -1,6 +1,7 @@
 """Run recording maintainer and cleanup."""
 
 import logging
+from multiprocessing.synchronize import Event as MpEvent
 
 from playhouse.sqliteq import SqliteQueueDatabase
 
@@ -13,8 +14,8 @@ logger = logging.getLogger(__name__)
 
 
 class RecordProcess(FrigateProcess):
-    def __init__(self, config: FrigateConfig) -> None:
-        super().__init__(name="frigate.recording_manager", daemon=True)
+    def __init__(self, config: FrigateConfig, stop_event: MpEvent) -> None:
+        super().__init__(stop_event, name="frigate.recording_manager", daemon=True)
         self.config = config
 
     def run(self) -> None:

frigate/review/review.py

@@ -1,6 +1,7 @@
 """Run recording maintainer and cleanup."""
 
 import logging
+from multiprocessing.synchronize import Event as MpEvent
 
 from frigate.config import FrigateConfig
 from frigate.review.maintainer import ReviewSegmentMaintainer
@@ -10,8 +11,8 @@ logger = logging.getLogger(__name__)
 
 
 class ReviewProcess(FrigateProcess):
-    def __init__(self, config: FrigateConfig) -> None:
-        super().__init__(name="frigate.review_segment_manager", daemon=True)
+    def __init__(self, config: FrigateConfig, stop_event: MpEvent) -> None:
+        super().__init__(stop_event, name="frigate.review_segment_manager", daemon=True)
         self.config = config
 
     def run(self) -> None:

frigate/util/builtin.py

@@ -5,7 +5,7 @@ import copy
 import datetime
 import logging
 import math
-import multiprocessing as mp
+import multiprocessing.queues
 import queue
 import re
 import shlex
@@ -338,16 +338,23 @@ def clear_and_unlink(file: Path, missing_ok: bool = True) -> None:
     file.unlink(missing_ok=missing_ok)
 
 
-def empty_and_close_queue(q: mp.Queue):
+def empty_and_close_queue(q):
     while True:
         try:
-            try:
-                q.get(block=True, timeout=0.5)
-            except (queue.Empty, EOFError):
-                q.close()
-                q.join_thread()
-                return
-        except AttributeError:
+            q.get(block=True, timeout=0.5)
+        except (queue.Empty, EOFError):
+            break
+        except Exception as e:
+            logger.debug(f"Error while emptying queue: {e}")
+            break
+
+    # close the queue if it is a multiprocessing queue
+    # manager proxy queues do not have close or join_thread method
+    if isinstance(q, multiprocessing.queues.Queue):
+        try:
+            q.close()
+            q.join_thread()
+        except Exception:
             pass
 
 
@@ -421,3 +428,19 @@ def sanitize_float(value):
     if isinstance(value, (int, float)) and not math.isfinite(value):
         return 0.0
     return value
+
+
+def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
+    return 1 - cosine_distance(a, b)
+
+
+def cosine_distance(a: np.ndarray, b: np.ndarray) -> float:
+    """Returns cosine distance to match sqlite-vec's calculation."""
+    dot = np.dot(a, b)
+    a_mag = np.dot(a, a)  # ||a||^2
+    b_mag = np.dot(b, b)  # ||b||^2
+
+    if a_mag == 0 or b_mag == 0:
+        return 1.0
+
+    return 1.0 - (dot / (np.sqrt(a_mag) * np.sqrt(b_mag)))

frigate/util/classification.py

@@ -1,7 +1,7 @@
 """Util for classification models."""
 
+import logging
 import os
-import sys
 
 import cv2
 import numpy as np
@@ -9,6 +9,7 @@ import numpy as np
 from frigate.comms.embeddings_updater import EmbeddingsRequestEnum, EmbeddingsRequestor
 from frigate.comms.inter_process import InterProcessRequestor
 from frigate.const import CLIPS_DIR, MODEL_CACHE_DIR, UPDATE_MODEL_STATE
+from frigate.log import redirect_output_to_logger
 from frigate.types import ModelStatusTypesEnum
 from frigate.util.process import FrigateProcess
 
@@ -16,117 +17,120 @@ BATCH_SIZE = 16
 EPOCHS = 50
 LEARNING_RATE = 0.001
 
-
-def __generate_representative_dataset_factory(dataset_dir: str):
-    def generate_representative_dataset():
-        image_paths = []
-        for root, dirs, files in os.walk(dataset_dir):
-            for file in files:
-                if file.lower().endswith((".jpg", ".jpeg", ".png")):
-                    image_paths.append(os.path.join(root, file))
-
-        for path in image_paths[:300]:
-            img = cv2.imread(path)
-            img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
-            img = cv2.resize(img, (224, 224))
-            img_array = np.array(img, dtype=np.float32) / 255.0
-            img_array = img_array[None, ...]
-            yield [img_array]
-
-    return generate_representative_dataset
+logger = logging.getLogger(__name__)
 
 
-def __train_classification_model(model_name: str) -> bool:
-    """Train a classification model."""
+class ClassificationTrainingProcess(FrigateProcess):
+    def __init__(self, model_name: str) -> None:
+        super().__init__(
+            stop_event=None,
+            name=f"model_training:{model_name}",
+        )
+        self.model_name = model_name
 
-    # import in the function so that tensorflow is not initialized multiple times
-    import tensorflow as tf
-    from tensorflow.keras import layers, models, optimizers
-    from tensorflow.keras.applications import MobileNetV2
-    from tensorflow.keras.preprocessing.image import ImageDataGenerator
+    def run(self) -> None:
+        self.pre_run_setup()
+        self.__train_classification_model()
 
-    dataset_dir = os.path.join(CLIPS_DIR, model_name, "dataset")
-    model_dir = os.path.join(MODEL_CACHE_DIR, model_name)
-    num_classes = len(
-        [
-            d
-            for d in os.listdir(dataset_dir)
-            if os.path.isdir(os.path.join(dataset_dir, d))
-        ]
-    )
+    def __generate_representative_dataset_factory(self, dataset_dir: str):
+        def generate_representative_dataset():
+            image_paths = []
+            for root, dirs, files in os.walk(dataset_dir):
+                for file in files:
+                    if file.lower().endswith((".jpg", ".jpeg", ".png")):
+                        image_paths.append(os.path.join(root, file))
 
-    # TF and Keras are very loud with logging
-    # we want to avoid these logs so we
-    # temporarily redirect stdout / stderr
-    original_stdout = sys.stdout
-    original_stderr = sys.stderr
-    sys.stdout = open(os.devnull, "w")
-    sys.stderr = open(os.devnull, "w")
+            for path in image_paths[:300]:
+                img = cv2.imread(path)
+                img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
+                img = cv2.resize(img, (224, 224))
+                img_array = np.array(img, dtype=np.float32) / 255.0
+                img_array = img_array[None, ...]
+                yield [img_array]
 
-    # Start with imagenet base model with 35% of channels in each layer
-    base_model = MobileNetV2(
-        input_shape=(224, 224, 3),
-        include_top=False,
-        weights="imagenet",
-        alpha=0.35,
-    )
-    base_model.trainable = False  # Freeze pre-trained layers
+        return generate_representative_dataset
 
-    model = models.Sequential(
-        [
-            base_model,
-            layers.GlobalAveragePooling2D(),
-            layers.Dense(128, activation="relu"),
-            layers.Dropout(0.3),
-            layers.Dense(num_classes, activation="softmax"),
-        ]
-    )
+    @redirect_output_to_logger(logger, logging.DEBUG)
+    def __train_classification_model(self) -> bool:
+        """Train a classification model."""
 
-    model.compile(
-        optimizer=optimizers.Adam(learning_rate=LEARNING_RATE),
-        loss="categorical_crossentropy",
-        metrics=["accuracy"],
-    )
+        # import in the function so that tensorflow is not initialized multiple times
+        import tensorflow as tf
+        from tensorflow.keras import layers, models, optimizers
+        from tensorflow.keras.applications import MobileNetV2
+        from tensorflow.keras.preprocessing.image import ImageDataGenerator
 
-    # create training set
-    datagen = ImageDataGenerator(rescale=1.0 / 255, validation_split=0.2)
-    train_gen = datagen.flow_from_directory(
-        dataset_dir,
-        target_size=(224, 224),
-        batch_size=BATCH_SIZE,
-        class_mode="categorical",
-        subset="training",
-    )
+        logger.info(f"Kicking off classification training for {self.model_name}.")
+        dataset_dir = os.path.join(CLIPS_DIR, self.model_name, "dataset")
+        model_dir = os.path.join(MODEL_CACHE_DIR, self.model_name)
+        num_classes = len(
+            [
+                d
+                for d in os.listdir(dataset_dir)
+                if os.path.isdir(os.path.join(dataset_dir, d))
+            ]
+        )
 
-    # write labelmap
-    class_indices = train_gen.class_indices
-    index_to_class = {v: k for k, v in class_indices.items()}
-    sorted_classes = [index_to_class[i] for i in range(len(index_to_class))]
-    with open(os.path.join(model_dir, "labelmap.txt"), "w") as f:
-        for class_name in sorted_classes:
-            f.write(f"{class_name}\n")
+        # Start with imagenet base model with 35% of channels in each layer
+        base_model = MobileNetV2(
+            input_shape=(224, 224, 3),
+            include_top=False,
+            weights="imagenet",
+            alpha=0.35,
+        )
+        base_model.trainable = False  # Freeze pre-trained layers
 
-    # train the model
-    model.fit(train_gen, epochs=EPOCHS, verbose=0)
+        model = models.Sequential(
+            [
+                base_model,
+                layers.GlobalAveragePooling2D(),
+                layers.Dense(128, activation="relu"),
+                layers.Dropout(0.3),
+                layers.Dense(num_classes, activation="softmax"),
+            ]
+        )
 
-    # convert model to tflite
-    converter = tf.lite.TFLiteConverter.from_keras_model(model)
-    converter.optimizations = [tf.lite.Optimize.DEFAULT]
-    converter.representative_dataset = __generate_representative_dataset_factory(
-        dataset_dir
-    )
-    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
-    converter.inference_input_type = tf.uint8
-    converter.inference_output_type = tf.uint8
-    tflite_model = converter.convert()
+        model.compile(
+            optimizer=optimizers.Adam(learning_rate=LEARNING_RATE),
+            loss="categorical_crossentropy",
+            metrics=["accuracy"],
+        )
 
-    # write model
-    with open(os.path.join(model_dir, "model.tflite"), "wb") as f:
-        f.write(tflite_model)
+        # create training set
+        datagen = ImageDataGenerator(rescale=1.0 / 255, validation_split=0.2)
+        train_gen = datagen.flow_from_directory(
+            dataset_dir,
+            target_size=(224, 224),
+            batch_size=BATCH_SIZE,
+            class_mode="categorical",
+            subset="training",
+        )
 
-    # restore original stdout / stderr
-    sys.stdout = original_stdout
-    sys.stderr = original_stderr
+        # write labelmap
+        class_indices = train_gen.class_indices
+        index_to_class = {v: k for k, v in class_indices.items()}
+        sorted_classes = [index_to_class[i] for i in range(len(index_to_class))]
+        with open(os.path.join(model_dir, "labelmap.txt"), "w") as f:
+            for class_name in sorted_classes:
+                f.write(f"{class_name}\n")
+
+        # train the model
+        model.fit(train_gen, epochs=EPOCHS, verbose=0)
+
+        # convert model to tflite
+        converter = tf.lite.TFLiteConverter.from_keras_model(model)
+        converter.optimizations = [tf.lite.Optimize.DEFAULT]
+        converter.representative_dataset = (
+            self.__generate_representative_dataset_factory(dataset_dir)
+        )
+        converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
+        converter.inference_input_type = tf.uint8
+        converter.inference_output_type = tf.uint8
+        tflite_model = converter.convert()
+
+        # write model
+        with open(os.path.join(model_dir, "model.tflite"), "wb") as f:
+            f.write(tflite_model)
 
 
 @staticmethod
@@ -145,11 +149,7 @@ def kickoff_model_training(
     # run training in sub process so that
     # tensorflow will free CPU / GPU memory
     # upon training completion
-    training_process = FrigateProcess(
-        target=__train_classification_model,
-        name=f"model_training:{model_name}",
-        args=(model_name,),
-    )
+    training_process = ClassificationTrainingProcess(model_name)
     training_process.start()
     training_process.join()
 

frigate/util/process.py

@@ -1,10 +1,9 @@
 import faulthandler
 import logging
 import multiprocessing as mp
-import signal
-import sys
 import threading
 from logging.handlers import QueueHandler
+from multiprocessing.synchronize import Event as MpEvent
 from typing import Callable, Optional
 
 from setproctitle import setproctitle
@@ -16,6 +15,7 @@ from frigate.config.logger import LoggerConfig
 class BaseProcess(mp.Process):
     def __init__(
         self,
+        stop_event: MpEvent,
         *,
         name: Optional[str] = None,
         target: Optional[Callable] = None,
@@ -23,6 +23,7 @@ class BaseProcess(mp.Process):
         kwargs: dict = {},
         daemon: Optional[bool] = None,
     ):
+        self.stop_event = stop_event
         super().__init__(
             name=name, target=target, args=args, kwargs=kwargs, daemon=daemon
         )
@@ -42,14 +43,6 @@ class BaseProcess(mp.Process):
 class FrigateProcess(BaseProcess):
     logger: logging.Logger
 
-    @property
-    def stop_event(self) -> threading.Event:
-        # Lazily create the stop_event. This allows the signal handler to tell if anyone is
-        # monitoring the stop event, and to raise a SystemExit if not.
-        if "stop_event" not in self.__dict__:
-            self.__dict__["stop_event"] = threading.Event()
-        return self.__dict__["stop_event"]
-
     def before_start(self) -> None:
         self.__log_queue = frigate.log.log_listener.queue
 
@@ -58,18 +51,7 @@ class FrigateProcess(BaseProcess):
         threading.current_thread().name = f"process:{self.name}"
         faulthandler.enable()
 
-        def receiveSignal(signalNumber, frame):
-            # Get the stop_event through the dict to bypass lazy initialization.
-            stop_event = self.__dict__.get("stop_event")
-            if stop_event is not None:
-                # Someone is monitoring stop_event. We should set it.
-                stop_event.set()
-            else:
-                # Nobody is monitoring stop_event. We should raise SystemExit.
-                sys.exit()
-
-        signal.signal(signal.SIGTERM, receiveSignal)
-        signal.signal(signal.SIGINT, receiveSignal)
+        # setup logging
         self.logger = logging.getLogger(self.name)
         logging.basicConfig(handlers=[], force=True)
         logging.getLogger().addHandler(QueueHandler(self.__log_queue))

frigate/video.py

@@ -439,9 +439,13 @@ class CameraCaptureRunner(threading.Thread):
 
 class CameraCapture(FrigateProcess):
     def __init__(
-        self, config: CameraConfig, shm_frame_count: int, camera_metrics: CameraMetrics
+        self,
+        config: CameraConfig,
+        shm_frame_count: int,
+        camera_metrics: CameraMetrics,
+        stop_event: MpEvent,
     ) -> None:
-        super().__init__(name=f"frigate.capture:{config.name}", daemon=True)
+        super().__init__(stop_event, name=f"frigate.capture:{config.name}", daemon=True)
         self.config = config
         self.shm_frame_count = shm_frame_count
         self.camera_metrics = camera_metrics
@@ -472,8 +476,9 @@ class CameraTracker(FrigateProcess):
         camera_metrics: CameraMetrics,
         ptz_metrics: PTZMetrics,
         region_grid: list[list[dict[str, Any]]],
+        stop_event: MpEvent,
     ) -> None:
-        super().__init__(name=f"frigate.process:{config.name}", daemon=True)
+        super().__init__(stop_event, name=f"frigate.process:{config.name}", daemon=True)
         self.config = config
         self.model_config = model_config
         self.labelmap = labelmap

migrations/031_create_trigger_table.py

@@ -0,0 +1,50 @@
+"""Peewee migrations -- 031_create_trigger_table.py.
+
+This migration creates the Trigger table to track semantic search triggers for cameras.
+
+Some examples (model - class or model_name)::
+
+    > Model = migrator.orm['model_name']            # Return model in current state by name
+    > migrator.sql(sql)                             # Run custom SQL
+    > migrator.python(func, *args, **kwargs)        # Run python code
+    > migrator.create_model(Model)                  # Create a model (could be used as decorator)
+    > migrator.remove_model(model, cascade=True)    # Remove a model
+    > migrator.add_fields(model, **fields)          # Add fields to a model
+    > migrator.change_fields(model, **fields)       # Change fields
+    > migrator.remove_fields(model, *field_names, cascade=True)
+    > migrator.rename_field(model, old_field_name, new_field_name)
+    > migrator.rename_table(model, new_table_name)
+    > migrator.add_index(model, *col_names, unique=False)
+    > migrator.drop_index(model, *col_names)
+    > migrator.add_not_null(model, *field_names)
+    > migrator.drop_not_null(model, *field_names)
+    > migrator.add_default(model, field_name, default)
+
+"""
+
+import peewee as pw
+
+SQL = pw.SQL
+
+
+def migrate(migrator, database, fake=False, **kwargs):
+    migrator.sql(
+        """
+        CREATE TABLE IF NOT EXISTS trigger (
+            camera VARCHAR(20) NOT NULL,
+            name VARCHAR NOT NULL,
+            type VARCHAR(10) NOT NULL,
+            model VARCHAR(30) NOT NULL,
+            data TEXT NOT NULL,
+            threshold REAL,
+            embedding BLOB,
+            triggering_event_id VARCHAR(30),
+            last_triggered DATETIME,
+            PRIMARY KEY (camera, name)
+        )
+        """
+    )
+
+
+def rollback(migrator, database, fake=False, **kwargs):
+    migrator.sql("DROP TABLE IF EXISTS trigger")

web/public/locales/en/components/dialog.json

@@ -109,5 +109,12 @@
       "markAsReviewed": "Mark as reviewed",
       "deleteNow": "Delete Now"
     }
+  },
+  "imagePicker": {
+    "selectImage": "Select a tracked object's thumbnail",
+    "search": {
+      "placeholder": "Search by label or sub label..."
+    },
+    "noImages": "No thumbnails found for this camera"
   }
 }

web/public/locales/en/components/filter.json

@@ -1,5 +1,11 @@
 {
   "filter": "Filter",
+  "classes": {
+    "label": "Classes",
+    "all": { "title": "All Classes" },
+    "count_one": "{{count}} Class",
+    "count_other": "{{count}} Classes"
+  },
   "labels": {
     "label": "Labels",
     "all": {

web/public/locales/en/views/classificationModel.json

@@ -3,7 +3,8 @@
     "deleteClassificationAttempts": "Delete Classification Images",
     "renameCategory": "Rename Class",
     "deleteCategory": "Delete Class",
-    "deleteImages": "Delete Images"
+    "deleteImages": "Delete Images",
+    "trainModel": "Train Model"
   },
   "toast": {
     "success": {

web/public/locales/en/views/explore.json

@@ -175,6 +175,10 @@
       "label": "Find similar",
       "aria": "Find similar tracked objects"
     },
+    "addTrigger": {
+      "label": "Add trigger",
+      "aria": "Add a trigger for this tracked object"
+    },
     "audioTranscription": {
       "label": "Transcribe",
       "aria": "Request audio transcription"

web/public/locales/en/views/live.json

@@ -38,6 +38,14 @@
         "label": "Zoom PTZ camera out"
       }
     },
+    "focus": {
+      "in": {
+        "label": "Focus PTZ camera in"
+      },
+      "out": {
+        "label": "Focus PTZ camera out"
+      }
+    },
     "frame": {
       "center": {
         "label": "Click in the frame to center the PTZ camera"

web/public/locales/en/views/settings.json

@@ -150,6 +150,10 @@
       "title": "Streams",
       "desc": "Disabling a camera completely stops Frigate's processing of this camera's streams. Detection, recording, and debugging will be unavailable.<br /> <em>Note: This does not disable go2rtc restreams.</em>"
     },
+    "genai": {
+      "title": "Generative AI",
+      "desc": "Temporarily enable/disable Generative AI for this camera. When disabled, AI generated descriptions will not be requested for tracked objects on this camera."
+    },
     "review": {
       "title": "Review",
       "desc": "Enable/disable alerts and detections for this camera. When disabled, no new review items will be generated.",
@@ -439,6 +443,11 @@
       "desc": "Show a box of the region of interest sent to the object detector",
       "tips": "<p><strong>Region Boxes</strong></p><br><p>Bright green boxes will be overlaid on areas of interest in the frame that are being sent to the object detector.</p>"
     },
+    "paths": {
+      "title": "Paths",
+      "desc": "Show significant points of the tracked object's path",
+      "tips": "<p><strong>Paths</strong></p><br><p>Lines and circles will indicate significant points the tracked object has moved during its lifecycle.</p>"
+    },
     "objectShapeFilterDrawing": {
       "title": "Object Shape Filter Drawing",
       "desc": "Draw a rectangle on the image to view area and ratio details",
@@ -644,5 +653,100 @@
       "success": "Frigate+ settings have been saved. Restart Frigate to apply changes.",
       "error": "Failed to save config changes: {{errorMessage}}"
     }
+  },
+  "triggers": {
+    "documentTitle": "Triggers",
+    "management": {
+      "title": "Trigger Management",
+      "desc": "Manage triggers for {{camera}}. Use the thumbnail type to trigger on similar thumbnails to your selected tracked object, and the description type to trigger on similar descriptions to text you specify."
+    },
+    "addTrigger": "Add Trigger",
+    "table": {
+      "name": "Name",
+      "type": "Type",
+      "content": "Content",
+      "threshold": "Threshold",
+      "actions": "Actions",
+      "noTriggers": "No triggers configured for this camera.",
+      "edit": "Edit",
+      "deleteTrigger": "Delete Trigger",
+      "lastTriggered": "Last triggered"
+    },
+    "type": {
+      "thumbnail": "Thumbnail",
+      "description": "Description"
+    },
+    "actions": {
+      "alert": "Mark as Alert",
+      "notification": "Send Notification"
+    },
+    "dialog": {
+      "createTrigger": {
+        "title": "Create Trigger",
+        "desc": "Create a trigger for camera {{camera}}"
+      },
+      "editTrigger": {
+        "title": "Edit Trigger",
+        "desc": "Edit the settings for trigger on camera {{camera}}"
+      },
+      "deleteTrigger": {
+        "title": "Delete Trigger",
+        "desc": "Are you sure you want to delete the trigger <strong>{{triggerName}}</strong>? This action cannot be undone."
+      },
+      "form": {
+        "name": {
+          "title": "Name",
+          "placeholder": "Enter trigger name",
+          "error": {
+            "minLength": "Name must be at least 2 characters long.",
+            "invalidCharacters": "Name can only contain letters, numbers, underscores, and hyphens.",
+            "alreadyExists": "A trigger with this name already exists for this camera."
+          }
+        },
+        "enabled": {
+          "description": "Enable or disable this trigger"
+        },
+        "type": {
+          "title": "Type",
+          "placeholder": "Select trigger type"
+        },
+        "content": {
+          "title": "Content",
+          "imagePlaceholder": "Select an image",
+          "textPlaceholder": "Enter text content",
+          "imageDesc": "Select an image to trigger this action when a similar image is detected.",
+          "textDesc": "Enter text to trigger this action when a similar tracked object description is detected.",
+          "error": {
+            "required": "Content is required."
+          }
+        },
+        "threshold": {
+          "title": "Threshold",
+          "error": {
+            "min": "Threshold must be at least 0",
+            "max": "Threshold must be at most 1"
+          }
+        },
+        "actions": {
+          "title": "Actions",
+          "desc": "By default, Frigate fires an MQTT message for all triggers. Choose an additional action to perform when this trigger fires.",
+          "error": {
+            "min": "At least one action must be selected."
+          }
+        }
+      }
+    },
+    "toast": {
+      "success": {
+        "createTrigger": "Trigger {{name}} created successfully.",
+        "updateTrigger": "Trigger {{name}} updated successfully.",
+        "deleteTrigger": "Trigger {{name}} deleted successfully."
+      },
+      "error": {
+        "createTriggerFailed": "Failed to create trigger: {{errorMessage}}",
+        "updateTriggerFailed": "Failed to update trigger: {{errorMessage}}",
+        "deleteTriggerFailed": "Failed to delete trigger: {{errorMessage}}"
+      }
+    }
   }
 }

web/src/api/ws.tsx

@@ -9,6 +9,7 @@ import {
   ModelState,
   ToggleableSetting,
   TrackedObjectUpdateReturnType,
+  TriggerStatus,
 } from "@/types/ws";
 import { FrigateStats } from "@/types/stats";
 import { createContainer } from "react-tracked";
@@ -67,6 +68,7 @@ function useValue(): useValueReturn {
         autotracking,
         alerts,
         detections,
+        genai,
       } = state["config"];
       cameraStates[`${name}/recordings/state`] = record ? "ON" : "OFF";
       cameraStates[`${name}/enabled/state`] = enabled ? "ON" : "OFF";
@@ -88,6 +90,7 @@ function useValue(): useValueReturn {
       cameraStates[`${name}/review_detections/state`] = detections
         ? "ON"
         : "OFF";
+      cameraStates[`${name}/genai/state`] = genai ? "ON" : "OFF";
     });
 
     setWsState((prevState) => ({
@@ -275,6 +278,17 @@ export function useDetectionsState(camera: string): {
   return { payload: payload as ToggleableSetting, send };
 }
 
+export function useGenAIState(camera: string): {
+  payload: ToggleableSetting;
+  send: (payload: ToggleableSetting, retain?: boolean) => void;
+} {
+  const {
+    value: { payload },
+    send,
+  } = useWs(`${camera}/genai/state`, `${camera}/genai/set`);
+  return { payload: payload as ToggleableSetting, send };
+}
+
 export function usePtzCommand(camera: string): {
   payload: string;
   send: (payload: string, retain?: boolean) => void;
@@ -572,3 +586,13 @@ export function useNotificationTest(): {
   } = useWs("notification_test", "notification_test");
   return { payload: payload as string, send };
 }
+
+export function useTriggers(): { payload: TriggerStatus } {
+  const {
+    value: { payload },
+  } = useWs("triggers", "");
+  const parsed = payload
+    ? JSON.parse(payload as string)
+    : { name: "", camera: "", event_id: "", type: "", score: 0 };
+  return { payload: useDeepMemo(parsed) };
+}

web/src/components/camera/DebugCameraImage.tsx

@@ -158,6 +158,16 @@ function DebugSettings({ handleSetOption, options }: DebugSettingsProps) {
         />
         <Label htmlFor="regions">{t("debug.regions")}</Label>
       </div>
+      <div className="flex items-center space-x-2">
+        <Switch
+          id="paths"
+          checked={options["paths"]}
+          onCheckedChange={(isChecked) => {
+            handleSetOption("paths", isChecked);
+          }}
+        />
+        <Label htmlFor="paths">{t("debug.paths")}</Label>
+      </div>
     </div>
   );
 }

web/src/components/card/SearchThumbnailFooter.tsx

@@ -15,6 +15,7 @@ type SearchThumbnailProps = {
   refreshResults: () => void;
   showObjectLifecycle: () => void;
   showSnapshot: () => void;
+  addTrigger: () => void;
 };
 
 export default function SearchThumbnailFooter({
@@ -24,6 +25,7 @@ export default function SearchThumbnailFooter({
   refreshResults,
   showObjectLifecycle,
   showSnapshot,
+  addTrigger,
 }: SearchThumbnailProps) {
   const { t } = useTranslation(["views/search"]);
   const { data: config } = useSWR<FrigateConfig>("config");
@@ -61,6 +63,7 @@ export default function SearchThumbnailFooter({
           refreshResults={refreshResults}
           showObjectLifecycle={showObjectLifecycle}
           showSnapshot={showSnapshot}
+          addTrigger={addTrigger}
         />
       </div>
     </div>

web/src/components/menu/SearchResultActions.tsx

@@ -41,6 +41,7 @@ import {
 import useSWR from "swr";
 
 import { Trans, useTranslation } from "react-i18next";
+import { BsFillLightningFill } from "react-icons/bs";
 
 type SearchResultActionsProps = {
   searchResult: SearchResult;
@@ -48,6 +49,7 @@ type SearchResultActionsProps = {
   refreshResults: () => void;
   showObjectLifecycle: () => void;
   showSnapshot: () => void;
+  addTrigger: () => void;
   isContextMenu?: boolean;
   children?: ReactNode;
 };
@@ -58,6 +60,7 @@ export default function SearchResultActions({
   refreshResults,
   showObjectLifecycle,
   showSnapshot,
+  addTrigger,
   isContextMenu = false,
   children,
 }: SearchResultActionsProps) {
@@ -138,6 +141,16 @@ export default function SearchResultActions({
           <span>{t("itemMenu.findSimilar.label")}</span>
         </MenuItem>
       )}
+      {config?.semantic_search?.enabled &&
+        searchResult.data.type == "object" && (
+          <MenuItem
+            aria-label={t("itemMenu.addTrigger.aria")}
+            onClick={addTrigger}
+          >
+            <BsFillLightningFill className="mr-2 size-4" />
+            <span>{t("itemMenu.addTrigger.label")}</span>
+          </MenuItem>
+        )}
       {isMobileOnly &&
         config?.plus?.enabled &&
         searchResult.has_snapshot &&

web/src/components/overlay/ClassificationSelectionDialog.tsx

@@ -82,7 +82,7 @@ export default function ClassificationSelectionDialog({
   );
 
   // control
-  const [newFace, setNewFace] = useState(false);
+  const [newClass, setNewClass] = useState(false);
 
   // components
   const Selector = isDesktop ? DropdownMenu : Drawer;
@@ -98,10 +98,10 @@ export default function ClassificationSelectionDialog({
 
   return (
     <div className={className ?? ""}>
-      {newFace && (
+      {newClass && (
         <TextEntryDialog
           open={true}
-          setOpen={setNewFace}
+          setOpen={setNewClass}
           title={t("createCategory.new")}
           onSave={(newCat) => onCategorizeImage(newCat)}
         />
@@ -130,7 +130,7 @@ export default function ClassificationSelectionDialog({
             >
               <SelectorItem
                 className="flex cursor-pointer gap-2 smart-capitalize"
-                onClick={() => setNewFace(true)}
+                onClick={() => setNewClass(true)}
               >
                 <LuPlus />
                 {t("createCategory.new")}
@@ -142,7 +142,7 @@ export default function ClassificationSelectionDialog({
                   onClick={() => onCategorizeImage(category)}
                 >
                   <MdCategory />
-                  {category}
+                  {category.replaceAll("_", " ")}
                 </SelectorItem>
               ))}
             </div>

web/src/components/overlay/CreateTriggerDialog.tsx

@@ -0,0 +1,416 @@
+import { useEffect, useMemo } from "react";
+import { useTranslation } from "react-i18next";
+import { useForm } from "react-hook-form";
+import { zodResolver } from "@hookform/resolvers/zod";
+import { z } from "zod";
+import useSWR from "swr";
+import {
+  Dialog,
+  DialogContent,
+  DialogDescription,
+  DialogFooter,
+  DialogHeader,
+  DialogTitle,
+} from "@/components/ui/dialog";
+import {
+  Form,
+  FormControl,
+  FormDescription,
+  FormField,
+  FormItem,
+  FormLabel,
+  FormMessage,
+} from "@/components/ui/form";
+import { Input } from "@/components/ui/input";
+import {
+  Select,
+  SelectContent,
+  SelectItem,
+  SelectTrigger,
+  SelectValue,
+} from "@/components/ui/select";
+import { Checkbox } from "@/components/ui/checkbox";
+import { Button } from "@/components/ui/button";
+import ActivityIndicator from "@/components/indicators/activity-indicator";
+import { FrigateConfig } from "@/types/frigateConfig";
+import ImagePicker from "@/components/overlay/ImagePicker";
+import { Trigger, TriggerAction, TriggerType } from "@/types/trigger";
+import { Switch } from "@/components/ui/switch";
+import { Textarea } from "../ui/textarea";
+
+type CreateTriggerDialogProps = {
+  show: boolean;
+  trigger: Trigger | null;
+  selectedCamera: string;
+  isLoading: boolean;
+  onCreate: (
+    enabled: boolean,
+    name: string,
+    type: TriggerType,
+    data: string,
+    threshold: number,
+    actions: TriggerAction[],
+  ) => void;
+  onEdit: (trigger: Trigger) => void;
+  onCancel: () => void;
+};
+
+export default function CreateTriggerDialog({
+  show,
+  trigger,
+  selectedCamera,
+  isLoading,
+  onCreate,
+  onEdit,
+  onCancel,
+}: CreateTriggerDialogProps) {
+  const { t } = useTranslation("views/settings");
+  const { data: config } = useSWR<FrigateConfig>("config");
+
+  const existingTriggerNames = useMemo(() => {
+    if (
+      !config ||
+      !selectedCamera ||
+      !config.cameras[selectedCamera]?.semantic_search?.triggers
+    ) {
+      return [];
+    }
+    return Object.keys(config.cameras[selectedCamera].semantic_search.triggers);
+  }, [config, selectedCamera]);
+
+  const formSchema = z.object({
+    enabled: z.boolean(),
+    name: z
+      .string()
+      .min(2, t("triggers.dialog.form.name.error.minLength"))
+      .regex(
+        /^[a-zA-Z0-9_-]+$/,
+        t("triggers.dialog.form.name.error.invalidCharacters"),
+      )
+      .refine(
+        (value) =>
+          !existingTriggerNames.includes(value) || value === trigger?.name,
+        t("triggers.dialog.form.name.error.alreadyExists"),
+      ),
+    type: z.enum(["thumbnail", "description"]),
+    data: z.string().min(1, t("triggers.dialog.form.content.error.required")),
+    threshold: z
+      .number()
+      .min(0, t("triggers.dialog.form.threshold.error.min"))
+      .max(1, t("triggers.dialog.form.threshold.error.max")),
+    actions: z.array(z.enum(["notification"])),
+  });
+
+  const form = useForm<z.infer<typeof formSchema>>({
+    resolver: zodResolver(formSchema),
+    mode: "onChange",
+    defaultValues: {
+      enabled: trigger?.enabled ?? true,
+      name: trigger?.name ?? "",
+      type: trigger?.type ?? "description",
+      data: trigger?.data ?? "",
+      threshold: trigger?.threshold ?? 0.5,
+      actions: trigger?.actions ?? [],
+    },
+  });
+
+  const onSubmit = async (values: z.infer<typeof formSchema>) => {
+    if (trigger) {
+      onEdit({ ...values });
+    } else {
+      onCreate(
+        values.enabled,
+        values.name,
+        values.type,
+        values.data,
+        values.threshold,
+        values.actions,
+      );
+    }
+  };
+
+  useEffect(() => {
+    if (!show) {
+      form.reset({
+        enabled: true,
+        name: "",
+        type: "description",
+        data: "",
+        threshold: 0.5,
+        actions: [],
+      });
+    } else if (trigger) {
+      form.reset(
+        {
+          enabled: trigger.enabled,
+          name: trigger.name,
+          type: trigger.type,
+          data: trigger.data,
+          threshold: trigger.threshold,
+          actions: trigger.actions,
+        },
+        { keepDirty: false, keepTouched: false }, // Reset validation state
+      );
+      // Trigger validation to ensure isValid updates
+      // form.trigger();
+    }
+  }, [show, trigger, form]);
+
+  const handleCancel = () => {
+    form.reset();
+    onCancel();
+  };
+
+  return (
+    <Dialog open={show} onOpenChange={onCancel}>
+      <DialogContent className="sm:max-w-[425px]">
+        <DialogHeader>
+          <DialogTitle>
+            {t(
+              trigger
+                ? "triggers.dialog.editTrigger.title"
+                : "triggers.dialog.createTrigger.title",
+            )}
+          </DialogTitle>
+          <DialogDescription>
+            {t(
+              trigger
+                ? "triggers.dialog.editTrigger.desc"
+                : "triggers.dialog.createTrigger.desc",
+              { camera: selectedCamera },
+            )}
+          </DialogDescription>
+        </DialogHeader>
+
+        <Form {...form}>
+          <form
+            onSubmit={form.handleSubmit(onSubmit)}
+            className="space-y-5 py-4"
+          >
+            <FormField
+              control={form.control}
+              name="name"
+              render={({ field }) => (
+                <FormItem>
+                  <FormLabel>{t("triggers.dialog.form.name.title")}</FormLabel>
+                  <FormControl>
+                    <Input
+                      placeholder={t("triggers.dialog.form.name.placeholder")}
+                      className="h-10"
+                      {...field}
+                    />
+                  </FormControl>
+                  <FormMessage />
+                </FormItem>
+              )}
+            />
+
+            <FormField
+              control={form.control}
+              name="enabled"
+              render={({ field }) => (
+                <FormItem className="flex flex-row items-center justify-between">
+                  <div className="space-y-0.5">
+                    <FormLabel className="text-base">
+                      {t("enabled", { ns: "common" })}
+                    </FormLabel>
+                    <div className="text-sm text-muted-foreground">
+                      {t("triggers.dialog.form.enabled.description")}
+                    </div>
+                  </div>
+                  <FormControl>
+                    <Switch
+                      checked={field.value}
+                      onCheckedChange={field.onChange}
+                    />
+                  </FormControl>
+                </FormItem>
+              )}
+            />
+
+            <FormField
+              control={form.control}
+              name="type"
+              render={({ field }) => (
+                <FormItem>
+                  <FormLabel>{t("triggers.dialog.form.type.title")}</FormLabel>
+                  <Select
+                    onValueChange={field.onChange}
+                    defaultValue={field.value}
+                  >
+                    <FormControl>
+                      <SelectTrigger className="h-10">
+                        <SelectValue
+                          placeholder={t(
+                            "triggers.dialog.form.type.placeholder",
+                          )}
+                        />
+                      </SelectTrigger>
+                    </FormControl>
+                    <SelectContent>
+                      <SelectItem value="thumbnail">
+                        {t("triggers.type.thumbnail")}
+                      </SelectItem>
+                      <SelectItem value="description">
+                        {t("triggers.type.description")}
+                      </SelectItem>
+                    </SelectContent>
+                  </Select>
+                  <FormMessage />
+                </FormItem>
+              )}
+            />
+
+            <FormField
+              control={form.control}
+              name="data"
+              render={({ field }) => (
+                <FormItem>
+                  <FormLabel>
+                    {t("triggers.dialog.form.content.title")}
+                  </FormLabel>
+                  {form.watch("type") === "thumbnail" ? (
+                    <>
+                      <FormControl>
+                        <ImagePicker
+                          selectedImageId={field.value}
+                          setSelectedImageId={field.onChange}
+                          camera={selectedCamera}
+                        />
+                      </FormControl>
+                      <FormDescription>
+                        {t("triggers.dialog.form.content.imageDesc")}
+                      </FormDescription>
+                    </>
+                  ) : (
+                    <>
+                      <FormControl>
+                        <Textarea
+                          placeholder={t(
+                            "triggers.dialog.form.content.textPlaceholder",
+                          )}
+                          {...field}
+                        />
+                      </FormControl>
+                      <FormDescription>
+                        {t("triggers.dialog.form.content.textDesc")}
+                      </FormDescription>
+                    </>
+                  )}
+
+                  <FormMessage />
+                </FormItem>
+              )}
+            />
+
+            <FormField
+              control={form.control}
+              name="threshold"
+              render={({ field }) => (
+                <FormItem>
+                  <FormLabel>
+                    {t("triggers.dialog.form.threshold.title")}
+                  </FormLabel>
+                  <FormControl>
+                    <Input
+                      type="number"
+                      step="0.01"
+                      min="0"
+                      max="1"
+                      placeholder="0.50"
+                      className="h-10"
+                      {...field}
+                      onChange={(e) => {
+                        const value = parseFloat(e.target.value);
+                        field.onChange(isNaN(value) ? 0 : value);
+                      }}
+                    />
+                  </FormControl>
+                  <FormMessage />
+                </FormItem>
+              )}
+            />
+
+            <FormField
+              control={form.control}
+              name="actions"
+              render={() => (
+                <FormItem>
+                  <FormLabel>
+                    {t("triggers.dialog.form.actions.title")}
+                  </FormLabel>
+                  <div className="space-y-2">
+                    {["notification"].map((action) => (
+                      <div key={action} className="flex items-center space-x-2">
+                        <FormControl>
+                          <Checkbox
+                            checked={form
+                              .watch("actions")
+                              .includes(action as TriggerAction)}
+                            onCheckedChange={(checked) => {
+                              const currentActions = form.getValues("actions");
+                              if (checked) {
+                                form.setValue("actions", [
+                                  ...currentActions,
+                                  action as TriggerAction,
+                                ]);
+                              } else {
+                                form.setValue(
+                                  "actions",
+                                  currentActions.filter((a) => a !== action),
+                                );
+                              }
+                            }}
+                          />
+                        </FormControl>
+                        <FormLabel className="text-sm font-normal">
+                          {t(`triggers.actions.${action}`)}
+                        </FormLabel>
+                      </div>
+                    ))}
+                  </div>
+                  <FormDescription>
+                    {t("triggers.dialog.form.actions.desc")}
+                  </FormDescription>
+                  <FormMessage />
+                </FormItem>
+              )}
+            />
+
+            <DialogFooter className="flex gap-2 pt-2 sm:justify-end">
+              <div className="flex flex-1 flex-col justify-end">
+                <div className="flex flex-row gap-2 pt-5">
+                  <Button
+                    className="flex flex-1"
+                    aria-label={t("button.cancel", { ns: "common" })}
+                    disabled={isLoading}
+                    onClick={handleCancel}
+                    type="button"
+                  >
+                    {t("button.cancel", { ns: "common" })}
+                  </Button>
+                  <Button
+                    variant="select"
+                    aria-label={t("button.save", { ns: "common" })}
+                    disabled={isLoading || !form.formState.isValid}
+                    className="flex flex-1"
+                    type="submit"
+                  >
+                    {isLoading ? (
+                      <div className="flex flex-row items-center gap-2">
+                        <ActivityIndicator />
+                        <span>{t("button.saving", { ns: "common" })}</span>
+                      </div>
+                    ) : (
+                      t("button.save", { ns: "common" })
+                    )}
+                  </Button>
+                </div>
+              </div>
+            </DialogFooter>
+          </form>
+        </Form>
+      </DialogContent>
+    </Dialog>
+  );
+}

web/src/components/overlay/DeleteTriggerDialog.tsx

@@ -0,0 +1,80 @@
+import { useTranslation } from "react-i18next";
+import {
+  Dialog,
+  DialogContent,
+  DialogDescription,
+  DialogFooter,
+  DialogHeader,
+  DialogTitle,
+} from "@/components/ui/dialog";
+import { Button } from "@/components/ui/button";
+import { Trans } from "react-i18next";
+import ActivityIndicator from "@/components/indicators/activity-indicator";
+
+type DeleteTriggerDialogProps = {
+  show: boolean;
+  triggerName: string;
+  isLoading: boolean;
+  onCancel: () => void;
+  onDelete: () => void;
+};
+
+export default function DeleteTriggerDialog({
+  show,
+  triggerName,
+  isLoading,
+  onCancel,
+  onDelete,
+}: DeleteTriggerDialogProps) {
+  const { t } = useTranslation("views/settings");
+
+  return (
+    <Dialog open={show} onOpenChange={onCancel}>
+      <DialogContent className="sm:max-w-[425px]">
+        <DialogHeader>
+          <DialogTitle>{t("triggers.dialog.deleteTrigger.title")}</DialogTitle>
+          <DialogDescription>
+            <Trans
+              ns={"views/settings"}
+              values={{ triggerName }}
+              components={{ strong: <span className="font-medium" /> }}
+            >
+              triggers.dialog.deleteTrigger.desc
+            </Trans>
+          </DialogDescription>
+        </DialogHeader>
+        <DialogFooter className="flex gap-3 sm:justify-end">
+          <div className="flex flex-1 flex-col justify-end">
+            <div className="flex flex-row gap-2 pt-5">
+              <Button
+                className="flex flex-1"
+                aria-label={t("button.cancel", { ns: "common" })}
+                onClick={onCancel}
+                type="button"
+                disabled={isLoading}
+              >
+                {t("button.cancel", { ns: "common" })}
+              </Button>
+              <Button
+                variant="destructive"
+                aria-label={t("button.delete", { ns: "common" })}
+                className="flex flex-1 text-white"
+                onClick={onDelete}
+                disabled={isLoading}
+              >
+                {isLoading ? (
+                  <div className="flex flex-row items-center gap-2">
+                    <ActivityIndicator />
+                    <span>{t("button.delete", { ns: "common" })}</span>
+                  </div>
+                ) : (
+                  t("button.delete", { ns: "common" })
+                )}
+              </Button>
+            </div>
+          </div>
+        </DialogFooter>
+      </DialogContent>
+    </Dialog>
+  );
+}

web/src/components/overlay/DeleteUserDialog.tsx

@@ -60,7 +60,7 @@ export default function DeleteUserDialog({
               <Button
                 variant="destructive"
                 aria-label={t("button.delete", { ns: "common" })}
-                className="flex flex-1"
+                className="flex flex-1 text-white"
                 onClick={onDelete}
               >
                 {t("button.delete", { ns: "common" })}

web/src/components/overlay/ImagePicker.tsx

@@ -0,0 +1,172 @@
+import { useCallback, useMemo, useRef, useState } from "react";
+import { useTranslation } from "react-i18next";
+import useSWR from "swr";
+import {
+  Dialog,
+  DialogContent,
+  DialogTitle,
+  DialogTrigger,
+} from "@/components/ui/dialog";
+import { IoClose } from "react-icons/io5";
+import { Input } from "@/components/ui/input";
+import { Button } from "@/components/ui/button";
+import Heading from "@/components/ui/heading";
+import { cn } from "@/lib/utils";
+import { Event } from "@/types/event";
+import { useApiHost } from "@/api";
+import { isDesktop, isMobile } from "react-device-detect";
+
+type ImagePickerProps = {
+  selectedImageId?: string;
+  setSelectedImageId?: (id: string) => void;
+  camera: string;
+};
+
+export default function ImagePicker({
+  selectedImageId,
+  setSelectedImageId,
+  camera,
+}: ImagePickerProps) {
+  const { t } = useTranslation(["components/dialog"]);
+  const [open, setOpen] = useState(false);
+  const containerRef = useRef<HTMLDivElement>(null);
+  const [searchTerm, setSearchTerm] = useState("");
+
+  const { data: events } = useSWR<Event[]>(
+    `events?camera=${camera}&limit=100`,
+    {
+      revalidateOnFocus: false,
+    },
+  );
+  const apiHost = useApiHost();
+
+  const images = useMemo(() => {
+    if (!events) return [];
+    return events.filter(
+      (event) =>
+        (event.label.toLowerCase().includes(searchTerm.toLowerCase()) ||
+          (event.sub_label &&
+            event.sub_label.toLowerCase().includes(searchTerm.toLowerCase())) ||
+          searchTerm === "") &&
+        event.camera === camera,
+    );
+  }, [events, searchTerm, camera]);
+
+  const selectedImage = useMemo(
+    () => images.find((img) => img.id === selectedImageId),
+    [images, selectedImageId],
+  );
+
+  const handleImageSelect = useCallback(
+    (id: string) => {
+      if (setSelectedImageId) {
+        setSelectedImageId(id);
+      }
+      setSearchTerm("");
+      setOpen(false);
+    },
+    [setSelectedImageId],
+  );
+
+  return (
+    <div ref={containerRef}>
+      <Dialog
+        open={open}
+        onOpenChange={(open) => {
+          setOpen(open);
+        }}
+      >
+        <DialogTrigger asChild>
+          {!selectedImageId ? (
+            <Button
+              className="mt-2 w-full text-muted-foreground"
+              aria-label={t("imagePicker.selectImage")}
+            >
+              {t("imagePicker.selectImage")}
+            </Button>
+          ) : (
+            <div className="hover:cursor-pointer">
+              <div className="my-3 flex w-full flex-row items-center justify-between gap-2">
+                <div className="flex flex-row items-center gap-2">
+                  <img
+                    src={
+                      selectedImage
+                        ? `${apiHost}api/events/${selectedImage.id}/thumbnail.webp`
+                        : `${apiHost}clips/triggers/${camera}/${selectedImageId}.webp`
+                    }
+                    alt={selectedImage?.label || "Selected image"}
+                    className="h-8 w-8 rounded object-cover"
+                  />
+                  <div className="text-sm smart-capitalize">
+                    {selectedImage?.label || selectedImageId}
+                    {selectedImage?.sub_label
+                      ? ` (${selectedImage.sub_label})`
+                      : ""}
+                  </div>
+                </div>
+                <IoClose
+                  className="mx-2 hover:cursor-pointer"
+                  onClick={() => {
+                    if (setSelectedImageId) {
+                      setSelectedImageId("");
+                    }
+                  }}
+                />
+              </div>
+            </div>
+          )}
+        </DialogTrigger>
+        <DialogTitle className="sr-only">
+          {t("imagePicker.selectImage")}
+        </DialogTitle>
+        <DialogContent
+          className={cn(
+            "scrollbar-container overflow-y-auto",
+            isDesktop && "max-h-[75dvh] sm:max-w-xl md:max-w-3xl",
+            isMobile && "px-4",
+          )}
+        >
+          <div className="mb-3 flex flex-row items-center justify-between">
+            <Heading as="h4">{t("imagePicker.selectImage")}</Heading>
+            <span tabIndex={0} className="sr-only" />
+          </div>
+          <Input
+            type="text"
+            placeholder={t("imagePicker.search.placeholder")}
+            className="text-md mb-3 md:text-sm"
+            value={searchTerm}
+            onChange={(e) => setSearchTerm(e.target.value)}
+          />
+          <div className="scrollbar-container flex h-full flex-col overflow-y-auto">
+            <div className="grid grid-cols-3 gap-2 pr-1">
+              {images.length === 0 ? (
+                <div className="col-span-3 text-center text-sm text-muted-foreground">
+                  {t("imagePicker.noImages")}
+                </div>
+              ) : (
+                images.map((image) => (
+                  <div
+                    key={image.id}
+                    className={cn(
+                      "flex flex-row items-center justify-center rounded-lg p-1 hover:cursor-pointer",
+                      selectedImageId === image.id
+                        ? "bg-selected text-white"
+                        : "hover:bg-secondary-foreground",
+                    )}
+                  >
+                    <img
+                      src={`${apiHost}api/events/${image.id}/thumbnail.webp`}
+                      alt={image.label}
+                      className="rounded object-cover"
+                      onClick={() => handleImageSelect(image.id)}
+                    />
+                  </div>
+                ))
+              )}
+            </div>
+          </div>
+        </DialogContent>
+      </Dialog>
+    </div>
+  );
+}

web/src/components/overlay/dialog/TrainFilterDialog.tsx

@@ -0,0 +1,262 @@
+import { FaFilter } from "react-icons/fa";
+
+import { useEffect, useMemo, useState } from "react";
+import { PlatformAwareSheet } from "./PlatformAwareDialog";
+import { Button } from "@/components/ui/button";
+import { isDesktop, isMobile } from "react-device-detect";
+import FilterSwitch from "@/components/filter/FilterSwitch";
+import { Switch } from "@/components/ui/switch";
+import { Label } from "@/components/ui/label";
+import { DropdownMenuSeparator } from "@/components/ui/dropdown-menu";
+import { cn } from "@/lib/utils";
+import { DualThumbSlider } from "@/components/ui/slider";
+import { Input } from "@/components/ui/input";
+import { useTranslation } from "react-i18next";
+import { TrainFilter } from "@/types/classification";
+
+type TrainFilterDialogProps = {
+  filter?: TrainFilter;
+  filterValues: {
+    classes: string[];
+  };
+  onUpdateFilter: (filter: TrainFilter) => void;
+};
+export default function TrainFilterDialog({
+  filter,
+  filterValues,
+  onUpdateFilter,
+}: TrainFilterDialogProps) {
+  // data
+  const { t } = useTranslation(["components/filter"]);
+  const [currentFilter, setCurrentFilter] = useState(filter ?? {});
+
+  useEffect(() => {
+    if (filter) {
+      setCurrentFilter(filter);
+    }
+  }, [filter]);
+
+  // state
+
+  const [open, setOpen] = useState(false);
+
+  const moreFiltersSelected = useMemo(
+    () =>
+      currentFilter &&
+      (currentFilter.classes ||
+        (currentFilter.min_score ?? 0) > 0.5 ||
+        (currentFilter.max_score ?? 1) < 1),
+    [currentFilter],
+  );
+
+  const trigger = (
+    <Button
+      className="flex items-center gap-2"
+      aria-label={t("more")}
+      variant={moreFiltersSelected ? "select" : "default"}
+    >
+      <FaFilter
+        className={cn(
+          moreFiltersSelected ? "text-white" : "text-secondary-foreground",
+        )}
+      />
+      {t("more")}
+    </Button>
+  );
+  const content = (
+    <div className="space-y-3">
+      <ClassFilterContent
+        allClasses={filterValues.classes}
+        classes={currentFilter.classes}
+        updateClasses={(newClasses) =>
+          setCurrentFilter({ ...currentFilter, classes: newClasses })
+        }
+      />
+      <ScoreFilterContent
+        minScore={currentFilter.min_score}
+        maxScore={currentFilter.max_score}
+        setScoreRange={(min, max) =>
+          setCurrentFilter({ ...currentFilter, min_score: min, max_score: max })
+        }
+      />
+      {isDesktop && <DropdownMenuSeparator />}
+      <div className="flex items-center justify-evenly p-2">
+        <Button
+          variant="select"
+          aria-label={t("button.apply", { ns: "common" })}
+          onClick={() => {
+            if (currentFilter != filter) {
+              onUpdateFilter(currentFilter);
+            }
+
+            setOpen(false);
+          }}
+        >
+          {t("button.apply", { ns: "common" })}
+        </Button>
+        <Button
+          aria-label={t("reset.label")}
+          onClick={() => {
+            setCurrentFilter((prevFilter) => ({
+              ...prevFilter,
+              time_range: undefined,
+              zones: undefined,
+              sub_labels: undefined,
+              search_type: undefined,
+              min_score: undefined,
+              max_score: undefined,
+              min_speed: undefined,
+              max_speed: undefined,
+              has_snapshot: undefined,
+              has_clip: undefined,
+              recognized_license_plate: undefined,
+            }));
+          }}
+        >
+          {t("button.reset", { ns: "common" })}
+        </Button>
+      </div>
+    </div>
+  );
+
+  return (
+    <PlatformAwareSheet
+      trigger={trigger}
+      title={t("more")}
+      content={content}
+      contentClassName={cn(
+        "w-auto lg:min-w-[275px] scrollbar-container h-full overflow-auto px-4",
+        isMobile && "pb-20",
+      )}
+      open={open}
+      onOpenChange={(open) => {
+        if (!open) {
+          setCurrentFilter(filter ?? {});
+        }
+
+        setOpen(open);
+      }}
+    />
+  );
+}
+
+type ClassFilterContentProps = {
+  allClasses?: string[];
+  classes?: string[];
+  updateClasses: (classes: string[] | undefined) => void;
+};
+export function ClassFilterContent({
+  allClasses,
+  classes,
+  updateClasses,
+}: ClassFilterContentProps) {
+  const { t } = useTranslation(["components/filter"]);
+  return (
+    <>
+      <div className="overflow-x-hidden">
+        <DropdownMenuSeparator className="mb-3" />
+        <div className="text-lg">{t("classes.label")}</div>
+        {allClasses && (
+          <>
+            <div className="mb-5 mt-2.5 flex items-center justify-between">
+              <Label
+                className="mx-2 cursor-pointer text-primary"
+                htmlFor="allClasses"
+              >
+                {t("classes.all.title")}
+              </Label>
+              <Switch
+                className="ml-1"
+                id="allClasses"
+                checked={classes == undefined}
+                onCheckedChange={(isChecked) => {
+                  if (isChecked) {
+                    updateClasses(undefined);
+                  }
+                }}
+              />
+            </div>
+            <div className="mt-2.5 flex flex-col gap-2.5">
+              {allClasses.map((item) => (
+                <FilterSwitch
+                  key={item}
+                  label={item.replaceAll("_", " ")}
+                  isChecked={classes?.includes(item) ?? false}
+                  onCheckedChange={(isChecked) => {
+                    if (isChecked) {
+                      const updatedClasses = classes ? [...classes] : [];
+
+                      updatedClasses.push(item);
+                      updateClasses(updatedClasses);
+                    } else {
+                      const updatedClasses = classes ? [...classes] : [];
+
+                      // can not deselect the last item
+                      if (updatedClasses.length > 1) {
+                        updatedClasses.splice(updatedClasses.indexOf(item), 1);
+                        updateClasses(updatedClasses);
+                      }
+                    }
+                  }}
+                />
+              ))}
+            </div>
+          </>
+        )}
+      </div>
+    </>
+  );
+}
+
+type ScoreFilterContentProps = {
+  minScore: number | undefined;
+  maxScore: number | undefined;
+  setScoreRange: (min: number | undefined, max: number | undefined) => void;
+};
+export function ScoreFilterContent({
+  minScore,
+  maxScore,
+  setScoreRange,
+}: ScoreFilterContentProps) {
+  const { t } = useTranslation(["components/filter"]);
+  return (
+    <div className="overflow-x-hidden">
+      <DropdownMenuSeparator className="mb-3" />
+      <div className="mb-3 text-lg">{t("score")}</div>
+      <div className="flex items-center gap-1">
+        <Input
+          className="w-14 text-center"
+          inputMode="numeric"
+          value={Math.round((minScore ?? 0.5) * 100)}
+          onChange={(e) => {
+            const value = e.target.value;
+
+            if (value) {
+              setScoreRange(parseInt(value) / 100.0, maxScore ?? 1.0);
+            }
+          }}
+        />
+        <DualThumbSlider
+          className="mx-2 w-full"
+          min={0.5}
+          max={1.0}
+          step={0.01}
+          value={[minScore ?? 0.5, maxScore ?? 1.0]}
+          onValueChange={([min, max]) => setScoreRange(min, max)}
+        />
+        <Input
+          className="w-14 text-center"
+          inputMode="numeric"
+          value={Math.round((maxScore ?? 1.0) * 100)}
+          onChange={(e) => {
+            const value = e.target.value;
+
+            if (value) {
+              setScoreRange(minScore ?? 0.5, parseInt(value) / 100.0);
+            }
+          }}
+        />
+      </div>
+    </div>
+  );
+}

web/src/pages/FaceLibrary.tsx

@@ -476,6 +476,7 @@ function LibrarySelector({
             </Button>
             <Button
               variant="destructive"
+              className="text-white"
               onClick={() => {
                 if (confirmDelete) {
                   handleDeleteFace(confirmDelete);

web/src/pages/Settings.tsx

@@ -45,6 +45,7 @@ import { isInIframe } from "@/utils/isIFrame";
 import { isPWA } from "@/utils/isPWA";
 import { useIsAdmin } from "@/hooks/use-is-admin";
 import { useTranslation } from "react-i18next";
+import TriggerView from "@/views/settings/TriggerView";
 
 const allSettingsViews = [
   "ui",
@@ -52,6 +53,7 @@ const allSettingsViews = [
   "cameras",
   "masksAndZones",
   "motionTuner",
+  "triggers",
   "debug",
   "users",
   "notifications",
@@ -171,7 +173,7 @@ export default function Settings() {
       }
     }
     // don't clear url params if we're creating a new object mask
-    return !searchParams.has("object_mask");
+    return !(searchParams.has("object_mask") || searchParams.has("event_id"));
   });
 
   useSearchEffect("camera", (camera: string) => {
@@ -179,8 +181,8 @@ export default function Settings() {
     if (cameraNames.includes(camera)) {
       setSelectedCamera(camera);
     }
-    // don't clear url params if we're creating a new object mask
-    return !searchParams.has("object_mask");
+    // don't clear url params if we're creating a new object mask or trigger
+    return !(searchParams.has("object_mask") || searchParams.has("event_id"));
   });
 
   useEffect(() => {
@@ -229,7 +231,8 @@ export default function Settings() {
         {(page == "debug" ||
           page == "cameras" ||
           page == "masksAndZones" ||
-          page == "motionTuner") && (
+          page == "motionTuner" ||
+          page == "triggers") && (
           <div className="ml-2 flex flex-shrink-0 items-center gap-2">
             {page == "masksAndZones" && (
               <ZoneMaskFilterButton
@@ -274,6 +277,12 @@ export default function Settings() {
             setUnsavedChanges={setUnsavedChanges}
           />
         )}
+        {page === "triggers" && (
+          <TriggerView
+            selectedCamera={selectedCamera}
+            setUnsavedChanges={setUnsavedChanges}
+          />
+        )}
         {page == "users" && <AuthenticationView />}
         {page == "notifications" && (
           <NotificationView setUnsavedChanges={setUnsavedChanges} />

web/src/types/classification.ts

@@ -0,0 +1,8 @@
+const TRAIN_FILTERS = ["class", "score"] as const;
+export type TrainFilters = (typeof TRAIN_FILTERS)[number];
+
+export type TrainFilter = {
+  classes?: string[];
+  min_score?: number;
+  max_score?: number;
+};

web/src/types/frigateConfig.ts

@@ -1,4 +1,5 @@
 import { IconName } from "@/components/icons/IconPicker";
+import { TriggerAction, TriggerType } from "./trigger";
 
 export interface UiConfig {
   timezone?: string;
@@ -220,6 +221,17 @@ export interface CameraConfig {
   rtmp: {
     enabled: boolean;
   };
+  semantic_search: {
+    triggers: {
+      [triggerName: string]: {
+        enabled: boolean;
+        type: TriggerType;
+        data: string;
+        threshold: number;
+        actions: TriggerAction[];
+      };
+    };
+  };
   snapshots: {
     bounding_box: boolean;
     clean_copy: boolean;
@@ -282,6 +294,7 @@ export type CameraStreamingSettings = {
 export type CustomClassificationModelConfig = {
   enabled: boolean;
   name: string;
+  threshold: number;
   object_config: null | {
     objects: string[];
   };
@@ -289,7 +302,6 @@ export type CustomClassificationModelConfig = {
     cameras: {
       [cameraName: string]: {
         crop: [number, number, number, number];
-        threshold: number;
       };
     };
     motion: boolean;

web/src/types/ptz.ts

@@ -1,4 +1,11 @@
-type PtzFeature = "pt" | "zoom" | "pt-r" | "zoom-r" | "zoom-a" | "pt-r-fov";
+type PtzFeature =
+  | "pt"
+  | "zoom"
+  | "pt-r"
+  | "zoom-r"
+  | "zoom-a"
+  | "pt-r-fov"
+  | "focus";
 
 export type CameraPtzInfo = {
   name: string;

web/src/types/trigger.ts

@@ -0,0 +1,11 @@
+export type TriggerType = "thumbnail" | "description";
+export type TriggerAction = "notification";
+
+export type Trigger = {
+  enabled: boolean;
+  name: string;
+  type: TriggerType;
+  data: string;
+  threshold: number;
+  actions: TriggerAction[];
+};

web/src/types/ws.ts

@@ -64,6 +64,7 @@ export interface FrigateCameraState {
     autotracking: boolean;
     alerts: boolean;
     detections: boolean;
+    genai: boolean;
   };
   motion: boolean;
   objects: ObjectType[];
@@ -105,3 +106,11 @@ export type TrackedObjectUpdateReturnType = {
   timestamp?: number;
   text?: string;
 } | null;
+
+export type TriggerStatus = {
+  name: string;
+  camera: string;
+  event_id: string;
+  type: string;
+  score: number;
+};

web/src/views/classification/ModelTrainingView.tsx

@@ -48,12 +48,19 @@ import { TbCategoryPlus } from "react-icons/tb";
 import { useModelState } from "@/api/ws";
 import { ModelState } from "@/types/ws";
 import ActivityIndicator from "@/components/indicators/activity-indicator";
+import { useNavigate } from "react-router-dom";
+import { IoMdArrowRoundBack } from "react-icons/io";
+import { MdAutoFixHigh } from "react-icons/md";
+import TrainFilterDialog from "@/components/overlay/dialog/TrainFilterDialog";
+import useApiFilter from "@/hooks/use-api-filter";
+import { TrainFilter } from "@/types/classification";
 
 type ModelTrainingViewProps = {
   model: CustomClassificationModelConfig;
 };
 export default function ModelTrainingView({ model }: ModelTrainingViewProps) {
   const { t } = useTranslation(["views/classificationModel"]);
+  const navigate = useNavigate();
   const [page, setPage] = useState<string>("train");
   const [pageToggle, setPageToggle] = useOptimisticState(page, setPage, 100);
 
@@ -93,6 +100,8 @@ export default function ModelTrainingView({ model }: ModelTrainingViewProps) {
     [id: string]: string[];
   }>(`classification/${model.name}/dataset`);
 
+  const [trainFilter, setTrainFilter] = useApiFilter<TrainFilter>();
+
   // image multiselect
 
   const [selectedImages, setSelectedImages] = useState<string[]>([]);
@@ -294,14 +303,28 @@ export default function ModelTrainingView({ model }: ModelTrainingViewProps) {
       </AlertDialog>
 
       <div className="flex flex-row justify-between gap-2 p-2 align-middle">
-        <LibrarySelector
-          pageToggle={pageToggle}
-          dataset={dataset || {}}
-          trainImages={trainImages || []}
-          setPageToggle={setPageToggle}
-          onDelete={onDelete}
-          onRename={() => {}}
-        />
+        <div className="flex flex-row items-center justify-center gap-2">
+          <Button
+            className="flex items-center gap-2.5 rounded-lg"
+            aria-label={t("label.back", { ns: "common" })}
+            onClick={() => navigate(-1)}
+          >
+            <IoMdArrowRoundBack className="size-5 text-secondary-foreground" />
+            {isDesktop && (
+              <div className="text-primary">
+                {t("button.back", { ns: "common" })}
+              </div>
+            )}
+          </Button>
+          <LibrarySelector
+            pageToggle={pageToggle}
+            dataset={dataset || {}}
+            trainImages={trainImages || []}
+            setPageToggle={setPageToggle}
+            onDelete={onDelete}
+            onRename={() => {}}
+          />
+        </div>
         {selectedImages?.length > 0 ? (
           <div className="flex items-center justify-center gap-2">
             <div className="mx-1 flex w-48 items-center justify-center text-sm text-muted-foreground">
@@ -323,14 +346,25 @@ export default function ModelTrainingView({ model }: ModelTrainingViewProps) {
             </Button>
           </div>
         ) : (
-          <Button
-            className="flex justify-center gap-2"
-            onClick={trainModel}
-            disabled={modelState != "complete"}
-          >
-            Train Model
-            {modelState == "training" && <ActivityIndicator size={20} />}
-          </Button>
+          <div className="flex flex-row gap-2">
+            <TrainFilterDialog
+              filter={trainFilter}
+              filterValues={{ classes: Object.keys(dataset || {}) }}
+              onUpdateFilter={setTrainFilter}
+            />
+            <Button
+              className="flex justify-center gap-2"
+              onClick={trainModel}
+              disabled={modelState != "complete"}
+            >
+              {modelState == "training" ? (
+                <ActivityIndicator size={20} />
+              ) : (
+                <MdAutoFixHigh className="text-secondary-foreground" />
+              )}
+              {t("button.trainModel")}
+            </Button>
+          </div>
         )}
       </div>
       {pageToggle == "train" ? (
@@ -338,6 +372,7 @@ export default function ModelTrainingView({ model }: ModelTrainingViewProps) {
           model={model}
           classes={Object.keys(dataset || {})}
           trainImages={trainImages || []}
+          trainFilter={trainFilter}
           selectedImages={selectedImages}
           onRefresh={refreshTrain}
           onClickImages={onClickImages}
@@ -375,7 +410,7 @@ function LibrarySelector({
 }: LibrarySelectorProps) {
   const { t } = useTranslation(["views/classificationModel"]);
   const [confirmDelete, setConfirmDelete] = useState<string | null>(null);
-  const [renameFace, setRenameFace] = useState<string | null>(null);
+  const [renameClass, setRenameFace] = useState<string | null>(null);
 
   const handleDeleteFace = useCallback(
     (name: string) => {
@@ -390,9 +425,9 @@ function LibrarySelector({
 
   const handleSetOpen = useCallback(
     (open: boolean) => {
-      setRenameFace(open ? renameFace : null);
+      setRenameFace(open ? renameClass : null);
     },
-    [renameFace],
+    [renameClass],
   );
 
   return (
@@ -414,6 +449,7 @@ function LibrarySelector({
             </Button>
             <Button
               variant="destructive"
+              className="text-white"
               onClick={() => {
                 if (confirmDelete) {
                   handleDeleteFace(confirmDelete);
@@ -428,15 +464,15 @@ function LibrarySelector({
       </Dialog>
 
       <TextEntryDialog
-        open={!!renameFace}
+        open={!!renameClass}
         setOpen={handleSetOpen}
         title={t("renameCategory.title")}
-        description={t("renameCategory.desc", { name: renameFace })}
+        description={t("renameCategory.desc", { name: renameClass })}
         onSave={(newName) => {
-          onRename(renameFace!, newName);
+          onRename(renameClass!, newName);
           setRenameFace(null);
         }}
-        defaultValue={renameFace || ""}
+        defaultValue={renameClass || ""}
         regexPattern={/^[\p{L}\p{N}\s'_-]{1,50}$/u}
         regexErrorMessage={t("description.invalidName")}
       />
@@ -484,10 +520,10 @@ function LibrarySelector({
               className="group flex items-center justify-between"
             >
               <div
-                className="flex-grow cursor-pointer"
+                className="flex-grow cursor-pointer capitalize"
                 onClick={() => setPageToggle(id)}
               >
-                {id}
+                {id.replaceAll("_", " ")}
                 <span className="ml-2 text-muted-foreground">
                   ({dataset?.[id].length})
                 </span>
@@ -560,9 +596,14 @@ function DatasetGrid({
 }: DatasetGridProps) {
   const { t } = useTranslation(["views/classificationModel"]);
 
+  const classData = useMemo(
+    () => images.sort((a, b) => a.localeCompare(b)),
+    [images],
+  );
+
   return (
     <div className="flex flex-wrap gap-2 overflow-y-auto p-2">
-      {images.map((image) => (
+      {classData.map((image) => (
         <div
           className={cn(
             "flex w-60 cursor-pointer flex-col gap-2 rounded-lg bg-card outline outline-[3px]",
@@ -619,6 +660,7 @@ type TrainGridProps = {
   model: CustomClassificationModelConfig;
   classes: string[];
   trainImages: string[];
+  trainFilter?: TrainFilter;
   selectedImages: string[];
   onClickImages: (images: string[], ctrl: boolean) => void;
   onRefresh: () => void;
@@ -628,6 +670,7 @@ function TrainGrid({
   model,
   classes,
   trainImages,
+  trainFilter,
   selectedImages,
   onClickImages,
   onRefresh,
@@ -640,15 +683,45 @@ function TrainGrid({
       trainImages
         .map((raw) => {
           const parts = raw.replaceAll(".webp", "").split("-");
+          const rawScore = Number.parseFloat(parts[2]);
           return {
             raw,
             timestamp: parts[0],
             label: parts[1],
-            score: Number.parseFloat(parts[2]) * 100,
+            score: rawScore * 100,
+            truePositive: rawScore >= model.threshold,
           };
         })
+        .filter((data) => {
+          if (!trainFilter) {
+            return true;
+          }
+
+          if (
+            trainFilter.classes &&
+            !trainFilter.classes.includes(data.label)
+          ) {
+            return false;
+          }
+
+          if (
+            trainFilter.min_score &&
+            trainFilter.min_score > data.score / 100.0
+          ) {
+            return false;
+          }
+
+          if (
+            trainFilter.max_score &&
+            trainFilter.max_score <= data.score / 100.0
+          ) {
+            return false;
+          }
+
+          return true;
+        })
         .sort((a, b) => b.timestamp.localeCompare(a.timestamp)),
-    [trainImages],
+    [model, trainImages, trainFilter],
   );
 
   return (
@@ -681,8 +754,17 @@ function TrainGrid({
           <div className="rounded-b-lg bg-card p-3">
             <div className="flex w-full flex-row items-center justify-between gap-2">
               <div className="flex flex-col items-start text-xs text-primary-variant">
-                <div className="smart-capitalize">{data.label}</div>
-                <div>{data.score}%</div>
+                <div className="smart-capitalize">
+                  {data.label.replaceAll("_", " ")}
+                </div>
+                <div
+                  className={cn(
+                    "",
+                    data.truePositive ? "text-success" : "text-danger",
+                  )}
+                >
+                  {data.score}%
+                </div>
               </div>
               <div className="flex flex-row items-start justify-end gap-5 md:gap-4">
                 <ClassificationSelectionDialog

web/src/views/explore/ExploreView.tsx

@@ -218,6 +218,7 @@ function ExploreThumbnailImage({
   const apiHost = useApiHost();
   const { data: config } = useSWR<FrigateConfig>("config");
   const [imgRef, imgLoaded, onImgLoad] = useImageLoaded();
+  const navigate = useNavigate();
 
   const handleFindSimilar = () => {
     if (config?.semantic_search.enabled) {
@@ -233,6 +234,12 @@ function ExploreThumbnailImage({
     onSelectSearch(event, false, "snapshot");
   };
 
+  const handleAddTrigger = () => {
+    navigate(
+      `/settings?page=triggers&camera=${event.camera}&event_id=${event.id}`,
+    );
+  };
+
   return (
     <SearchResultActions
       searchResult={event}
@@ -240,6 +247,7 @@ function ExploreThumbnailImage({
       refreshResults={mutate}
       showObjectLifecycle={handleShowObjectLifecycle}
       showSnapshot={handleShowSnapshot}
+      addTrigger={handleAddTrigger}
       isContextMenu={true}
     >
       <div className="relative size-full">

web/src/views/live/LiveCameraView.tsx

@@ -92,6 +92,8 @@ import {
   LuX,
 } from "react-icons/lu";
 import {
+  MdCenterFocusStrong,
+  MdCenterFocusWeak,
   MdClosedCaption,
   MdClosedCaptionDisabled,
   MdNoPhotography,
@@ -808,10 +810,10 @@ function PtzControlPanel({
           sendPtz("MOVE_DOWN");
           break;
         case "+":
-          sendPtz("ZOOM_IN");
+          sendPtz(modifiers.shift ? "FOCUS_IN" : "ZOOM_IN");
           break;
         case "-":
-          sendPtz("ZOOM_OUT");
+          sendPtz(modifiers.shift ? "FOCUS_OUT" : "ZOOM_OUT");
           break;
       }
     },
@@ -922,6 +924,40 @@ function PtzControlPanel({
           </TooltipButton>
         </>
       )}
+      {ptz?.features?.includes("focus") && (
+        <>
+          <TooltipButton
+            label={t("ptz.focus.in.label")}
+            onMouseDown={(e) => {
+              e.preventDefault();
+              sendPtz("FOCUS_IN");
+            }}
+            onTouchStart={(e) => {
+              e.preventDefault();
+              sendPtz("FOCUS_IN");
+            }}
+            onMouseUp={onStop}
+            onTouchEnd={onStop}
+          >
+            <MdCenterFocusStrong />
+          </TooltipButton>
+          <TooltipButton
+            label={t("ptz.focus.out.label")}
+            onMouseDown={(e) => {
+              e.preventDefault();
+              sendPtz("FOCUS_OUT");
+            }}
+            onTouchStart={(e) => {
+              e.preventDefault();
+              sendPtz("FOCUS_OUT");
+            }}
+            onMouseUp={onStop}
+            onTouchEnd={onStop}
+          >
+            <MdCenterFocusWeak />
+          </TooltipButton>
+        </>
+      )}
 
       {ptz?.features?.includes("pt-r-fov") && (
         <TooltipProvider>

web/src/views/search/SearchView.tsx

@@ -32,6 +32,7 @@ import Chip from "@/components/indicators/Chip";
 import { TooltipPortal } from "@radix-ui/react-tooltip";
 import SearchActionGroup from "@/components/filter/SearchActionGroup";
 import { Trans, useTranslation } from "react-i18next";
+import { useNavigate } from "react-router-dom";
 
 type SearchViewProps = {
   search: string;
@@ -76,6 +77,7 @@ export default function SearchView({
   const { data: config } = useSWR<FrigateConfig>("config", {
     revalidateOnFocus: false,
   });
+  const navigate = useNavigate();
 
   // grid
 
@@ -648,6 +650,16 @@ export default function SearchView({
                         showSnapshot={() =>
                           onSelectSearch(value, false, "snapshot")
                         }
+                        addTrigger={() => {
+                          if (
+                            config?.semantic_search.enabled &&
+                            value.data.type == "object"
+                          ) {
+                            navigate(
+                              `/settings?page=triggers&camera=${value.camera}&event_id=${value.id}`,
+                            );
+                          }
+                        }}
                       />
                     </div>
                   </div>

web/src/views/settings/CameraSettingsView.tsx

@@ -29,7 +29,12 @@ import { cn } from "@/lib/utils";
 import { Trans, useTranslation } from "react-i18next";
 import { Switch } from "@/components/ui/switch";
 import { Label } from "@/components/ui/label";
-import { useAlertsState, useDetectionsState, useEnabledState } from "@/api/ws";
+import {
+  useAlertsState,
+  useDetectionsState,
+  useEnabledState,
+  useGenAIState,
+} from "@/api/ws";
 import CameraEditForm from "@/components/settings/CameraEditForm";
 import { LuPlus } from "react-icons/lu";
 import {
@@ -142,6 +147,9 @@ export default function CameraSettingsView({
   const { payload: detectionsState, send: sendDetections } =
     useDetectionsState(selectedCamera);
 
+  const { payload: genAIState, send: sendGenAI } =
+    useGenAIState(selectedCamera);
+
   const handleCheckedChange = useCallback(
     (isChecked: boolean) => {
       if (!isChecked) {
@@ -402,6 +410,36 @@ export default function CameraSettingsView({
                   </div>
                 </div>
               </div>
+              {config?.genai?.enabled && (
+                <>
+                  <Separator className="my-2 flex bg-secondary" />
+
+                  <Heading as="h4" className="my-2">
+                    <Trans ns="views/settings">camera.genai.title</Trans>
+                  </Heading>
+
+                  <div className="mb-5 mt-2 flex max-w-5xl flex-col gap-2 space-y-3 text-sm text-primary-variant">
+                    <div className="flex flex-row items-center">
+                      <Switch
+                        id="alerts-enabled"
+                        className="mr-3"
+                        checked={genAIState == "ON"}
+                        onCheckedChange={(isChecked) => {
+                          sendGenAI(isChecked ? "ON" : "OFF");
+                        }}
+                      />
+                      <div className="space-y-0.5">
+                        <Label htmlFor="genai-enabled">
+                          <Trans>button.enabled</Trans>
+                        </Label>
+                      </div>
+                    </div>
+                    <div className="mt-3 text-sm text-muted-foreground">
+                      <Trans ns="views/settings">camera.genai.desc</Trans>
+                    </div>
+                  </div>
+                </>
+              )}
 
               <Separator className="my-2 flex bg-secondary" />
 

web/src/views/settings/ObjectSettingsView.tsx

@@ -89,6 +89,12 @@ export default function ObjectSettingsView({
       description: t("debug.regions.desc"),
       info: <Trans ns="views/settings">debug.regions.tips</Trans>,
     },
+    {
+      param: "paths",
+      title: t("debug.paths.title"),
+      description: t("debug.paths.desc"),
+      info: <Trans ns="views/settings">debug.paths.tips</Trans>,
+    },
   ];
 
   const [options, setOptions, optionsLoaded] = usePersistence<Options>(

web/src/views/settings/TriggerView.tsx

@@ -0,0 +1,595 @@
+import { useCallback, useEffect, useMemo, useState } from "react";
+import { useTranslation } from "react-i18next";
+import { Toaster, toast } from "sonner";
+import useSWR from "swr";
+import axios from "axios";
+import { Button } from "@/components/ui/button";
+import Heading from "@/components/ui/heading";
+import { Badge } from "@/components/ui/badge";
+import {
+  Tooltip,
+  TooltipContent,
+  TooltipProvider,
+  TooltipTrigger,
+} from "@/components/ui/tooltip";
+import { LuPlus, LuTrash, LuPencil, LuSearch } from "react-icons/lu";
+import ActivityIndicator from "@/components/indicators/activity-indicator";
+import CreateTriggerDialog from "@/components/overlay/CreateTriggerDialog";
+import DeleteTriggerDialog from "@/components/overlay/DeleteTriggerDialog";
+import { FrigateConfig } from "@/types/frigateConfig";
+import { Trigger, TriggerAction, TriggerType } from "@/types/trigger";
+import { useSearchEffect } from "@/hooks/use-overlay-state";
+import { cn } from "@/lib/utils";
+import { formatUnixTimestampToDateTime } from "@/utils/dateUtil";
+import { Link } from "react-router-dom";
+import { useTriggers } from "@/api/ws";
+
+type ConfigSetBody = {
+  requires_restart: number;
+  config_data: {
+    cameras: {
+      [key: string]: {
+        semantic_search?: {
+          triggers?: {
+            [key: string]:
+              | {
+                  enabled: boolean;
+                  type: string;
+                  data: string;
+                  threshold: number;
+                  actions: string[];
+                }
+              | "";
+          };
+        };
+      };
+    };
+  };
+  update_topic?: string;
+};
+
+type TriggerEmbeddingBody = {
+  type: TriggerType;
+  data: string;
+  threshold: number;
+};
+
+type TriggerViewProps = {
+  selectedCamera: string;
+  setUnsavedChanges: React.Dispatch<React.SetStateAction<boolean>>;
+};
+
+export default function TriggerView({
+  selectedCamera,
+  setUnsavedChanges,
+}: TriggerViewProps) {
+  const { t } = useTranslation("views/settings");
+  const { data: config, mutate: updateConfig } =
+    useSWR<FrigateConfig>("config");
+  const { data: trigger_status, mutate } = useSWR(
+    `/triggers/status/${selectedCamera}`,
+    {
+      revalidateOnFocus: false,
+    },
+  );
+  const [showCreate, setShowCreate] = useState(false);
+  const [showDelete, setShowDelete] = useState(false);
+  const [selectedTrigger, setSelectedTrigger] = useState<Trigger | null>(null);
+  const [triggeredTrigger, setTriggeredTrigger] = useState<string>();
+  const [isLoading, setIsLoading] = useState(false);
+
+  const triggers = useMemo(() => {
+    if (
+      !config ||
+      !selectedCamera ||
+      !config.cameras[selectedCamera]?.semantic_search?.triggers
+    ) {
+      return [];
+    }
+    return Object.entries(
+      config.cameras[selectedCamera].semantic_search.triggers,
+    ).map(([name, trigger]) => ({
+      enabled: trigger.enabled,
+      name,
+      type: trigger.type,
+      data: trigger.data,
+      threshold: trigger.threshold,
+      actions: trigger.actions,
+    }));
+  }, [config, selectedCamera]);
+
+  // watch websocket for updates
+  const { payload: triggers_status_ws } = useTriggers();
+
+  useEffect(() => {
+    if (!triggers_status_ws) return;
+
+    mutate();
+
+    setTriggeredTrigger(triggers_status_ws.name);
+    const target = document.querySelector(
+      `#trigger-${triggers_status_ws.name}`,
+    );
+    if (target) {
+      target.scrollIntoView({
+        block: "center",
+        behavior: "smooth",
+        inline: "nearest",
+      });
+      const ring = target.querySelector(".trigger-ring");
+      if (ring) {
+        ring.classList.add(`outline-selected`);
+        ring.classList.remove("outline-transparent");
+
+        const timeout = setTimeout(() => {
+          ring.classList.remove(`outline-selected`);
+          ring.classList.add("outline-transparent");
+        }, 3000);
+        return () => clearTimeout(timeout);
+      }
+    }
+  }, [triggers_status_ws, selectedCamera, mutate]);
+
+  useEffect(() => {
+    document.title = t("triggers.documentTitle");
+  }, [t]);
+
+  const saveToConfig = useCallback(
+    (trigger: Trigger, isEdit: boolean) => {
+      setIsLoading(true);
+      const { enabled, name, type, data, threshold, actions } = trigger;
+      const embeddingBody: TriggerEmbeddingBody = { type, data, threshold };
+      const embeddingUrl = isEdit
+        ? `/trigger/embedding/${selectedCamera}/${name}`
+        : `/trigger/embedding?camera=${selectedCamera}&name=${name}`;
+      const embeddingMethod = isEdit ? axios.put : axios.post;
+
+      embeddingMethod(embeddingUrl, embeddingBody)
+        .then((embeddingResponse) => {
+          if (embeddingResponse.data.success) {
+            const configBody: ConfigSetBody = {
+              requires_restart: 0,
+              config_data: {
+                cameras: {
+                  [selectedCamera]: {
+                    semantic_search: {
+                      triggers: {
+                        [name]: {
+                          enabled,
+                          type,
+                          data,
+                          threshold,
+                          actions,
+                        },
+                      },
+                    },
+                  },
+                },
+              },
+              update_topic: `config/cameras/${selectedCamera}/semantic_search`,
+            };
+
+            return axios
+              .put("config/set", configBody)
+              .then((configResponse) => {
+                if (configResponse.status === 200) {
+                  updateConfig();
+                  toast.success(
+                    t(
+                      isEdit
+                        ? "triggers.toast.success.updateTrigger"
+                        : "triggers.toast.success.createTrigger",
+                      { name },
+                    ),
+                    { position: "top-center" },
+                  );
+                  setUnsavedChanges(false);
+                } else {
+                  throw new Error(configResponse.statusText);
+                }
+              });
+          } else {
+            throw new Error(embeddingResponse.data.message);
+          }
+        })
+        .catch((error) => {
+          const errorMessage =
+            error.response?.data?.message ||
+            error.response?.data?.detail ||
+            "Unknown error";
+          toast.error(
+            t("toast.save.error.title", { errorMessage, ns: "common" }),
+            { position: "top-center" },
+          );
+        })
+        .finally(() => {
+          setIsLoading(false);
+          setShowCreate(false);
+        });
+    },
+    [t, updateConfig, selectedCamera, setUnsavedChanges],
+  );
+
+  const onCreate = useCallback(
+    (
+      enabled: boolean,
+      name: string,
+      type: TriggerType,
+      data: string,
+      threshold: number,
+      actions: TriggerAction[],
+    ) => {
+      setUnsavedChanges(true);
+      saveToConfig({ enabled, name, type, data, threshold, actions }, false);
+    },
+    [saveToConfig, setUnsavedChanges],
+  );
+
+  const onEdit = useCallback(
+    (trigger: Trigger) => {
+      setUnsavedChanges(true);
+      setIsLoading(true);
+      if (selectedTrigger?.name && selectedTrigger.name !== trigger.name) {
+        // Handle rename: delete old trigger, update config, then save new trigger
+        axios
+          .delete(
+            `/trigger/embedding/${selectedCamera}/${selectedTrigger.name}`,
+          )
+          .then((embeddingResponse) => {
+            if (!embeddingResponse.data.success) {
+              throw new Error(embeddingResponse.data.message);
+            }
+            const deleteConfigBody: ConfigSetBody = {
+              requires_restart: 0,
+              config_data: {
+                cameras: {
+                  [selectedCamera]: {
+                    semantic_search: {
+                      triggers: {
+                        [selectedTrigger.name]: "",
+                      },
+                    },
+                  },
+                },
+              },
+              update_topic: `config/cameras/${selectedCamera}/semantic_search`,
+            };
+            return axios.put("config/set", deleteConfigBody);
+          })
+          .then((configResponse) => {
+            if (configResponse.status !== 200) {
+              throw new Error(configResponse.statusText);
+            }
+            // Save new trigger
+            saveToConfig(trigger, false);
+          })
+          .catch((error) => {
+            const errorMessage =
+              error.response?.data?.message ||
+              error.response?.data?.detail ||
+              "Unknown error";
+            toast.error(
+              t("toast.save.error.title", { errorMessage, ns: "common" }),
+              { position: "top-center" },
+            );
+            setIsLoading(false);
+          });
+      } else {
+        // Regular update without rename
+        saveToConfig(trigger, true);
+      }
+      setSelectedTrigger(null);
+    },
+    [t, saveToConfig, selectedCamera, selectedTrigger, setUnsavedChanges],
+  );
+
+  const onDelete = useCallback(
+    (name: string) => {
+      setUnsavedChanges(true);
+      setIsLoading(true);
+      axios
+        .delete(`/trigger/embedding/${selectedCamera}/${name}`)
+        .then((embeddingResponse) => {
+          if (embeddingResponse.data.success) {
+            const configBody: ConfigSetBody = {
+              requires_restart: 0,
+              config_data: {
+                cameras: {
+                  [selectedCamera]: {
+                    semantic_search: {
+                      triggers: {
+                        [name]: "",
+                      },
+                    },
+                  },
+                },
+              },
+              update_topic: `config/cameras/${selectedCamera}/semantic_search`,
+            };
+
+            return axios
+              .put("config/set", configBody)
+              .then((configResponse) => {
+                if (configResponse.status === 200) {
+                  updateConfig();
+                  toast.success(
+                    t("triggers.toast.success.deleteTrigger", { name }),
+                    {
+                      position: "top-center",
+                    },
+                  );
+                  setUnsavedChanges(false);
+                } else {
+                  throw new Error(configResponse.statusText);
+                }
+              });
+          } else {
+            throw new Error(embeddingResponse.data.message);
+          }
+        })
+        .catch((error) => {
+          const errorMessage =
+            error.response?.data?.message ||
+            error.response?.data?.detail ||
+            "Unknown error";
+          toast.error(
+            t("triggers.toast.error.deleteTriggerFailed", { errorMessage }),
+            { position: "top-center" },
+          );
+        })
+        .finally(() => {
+          setShowDelete(false);
+          setIsLoading(false);
+        });
+    },
+    [t, updateConfig, selectedCamera, setUnsavedChanges],
+  );
+
+  useEffect(() => {
+    if (selectedCamera) {
+      setSelectedTrigger(null);
+      setShowCreate(false);
+      setShowDelete(false);
+      setUnsavedChanges(false);
+    }
+  }, [selectedCamera, setUnsavedChanges]);
+
+  // for adding a trigger with event id via explore context menu
+
+  useSearchEffect("event_id", (eventId: string) => {
+    if (!config || isLoading) {
+      return false;
+    }
+    setShowCreate(true);
+    setSelectedTrigger({
+      enabled: true,
+      name: "",
+      type: "thumbnail",
+      data: eventId,
+      threshold: 0.5,
+      actions: [],
+    });
+    return true;
+  });
+
+  if (!config || !selectedCamera) {
+    return (
+      <div className="flex h-full w-full items-center justify-center">
+        <ActivityIndicator />
+      </div>
+    );
+  }
+
+  return (
+    <div className="flex size-full flex-col md:flex-row">
+      <Toaster position="top-center" closeButton={true} />
+      <div className="scrollbar-container order-last mb-10 mt-2 flex h-full w-full flex-col overflow-y-auto rounded-lg border-[1px] border-secondary-foreground bg-background_alt p-2 md:order-none md:mb-0 md:mr-2 md:mt-0">
+        <div className="mb-5 flex flex-row items-center justify-between gap-2">
+          <div className="flex flex-col items-start">
+            <Heading as="h3" className="my-2">
+              {t("triggers.management.title")}
+            </Heading>
+            <p className="text-sm text-muted-foreground">
+              {t("triggers.management.desc", { camera: selectedCamera })}
+            </p>
+          </div>
+          <Button
+            className="flex items-center gap-2 self-start sm:self-auto"
+            aria-label={t("triggers.addTrigger")}
+            variant="default"
+            onClick={() => {
+              setSelectedTrigger(null);
+              setShowCreate(true);
+            }}
+            disabled={isLoading}
+          >
+            <LuPlus className="size-4" />
+            {t("triggers.addTrigger")}
+          </Button>
+        </div>
+        <div className="mb-6 flex flex-col gap-2 sm:flex-row sm:items-center sm:justify-between">
+          <div className="scrollbar-container flex-1 overflow-hidden rounded-lg border border-border bg-background_alt">
+            <div className="h-full overflow-auto p-0">
+              {triggers.length === 0 ? (
+                <div className="flex h-24 items-center justify-center">
+                  <p className="text-center text-muted-foreground">
+                    {t("triggers.table.noTriggers")}
+                  </p>
+                </div>
+              ) : (
+                <div className="space-y-2">
+                  {triggers.map((trigger) => (
+                    <div
+                      key={trigger.name}
+                      id={`trigger-${trigger.name}`}
+                      className="relative flex items-center justify-between rounded-lg border border-border bg-background p-4 transition-all"
+                    >
+                      <div
+                        className={cn(
+                          "trigger-ring pointer-events-none absolute inset-0 z-10 size-full rounded-md outline outline-[3px] -outline-offset-[2.8px] duration-500",
+                          triggeredTrigger === trigger.name
+                            ? "shadow-selected outline-selected"
+                            : "outline-transparent duration-500",
+                        )}
+                      />
+                      <div className="min-w-0 flex-1">
+                        <h3
+                          className={cn(
+                            "truncate text-lg font-medium",
+                            !trigger.enabled && "opacity-60",
+                          )}
+                        >
+                          {trigger.name}
+                        </h3>
+                        <div
+                          className={cn(
+                            "mt-1 flex flex-col gap-1 text-sm text-muted-foreground md:flex-row md:items-center md:gap-3",
+                            !trigger.enabled && "opacity-60",
+                          )}
+                        >
+                          <div>
+                            <Badge
+                              variant={
+                                trigger.type === "thumbnail"
+                                  ? "default"
+                                  : "outline"
+                              }
+                              className={
+                                trigger.type === "thumbnail"
+                                  ? "bg-primary/20 text-primary hover:bg-primary/30"
+                                  : ""
+                              }
+                            >
+                              {t(`triggers.type.${trigger.type}`)}
+                            </Badge>
+                          </div>
+
+                          <Link
+                            to={`/explore?event_id=${trigger_status?.triggers[trigger.name]?.triggering_event_id || ""}`}
+                            className={cn(
+                              "text-sm",
+                              !trigger_status?.triggers[trigger.name]
+                                ?.triggering_event_id && "pointer-events-none",
+                            )}
+                          >
+                            <div className="flex flex-row items-center">
+                              {t("triggers.table.lastTriggered")}:{" "}
+                              {trigger_status &&
+                              trigger_status.triggers[trigger.name]
+                                ?.last_triggered
+                                ? formatUnixTimestampToDateTime(
+                                    trigger_status.triggers[trigger.name]
+                                      ?.last_triggered,
+                                    {
+                                      timezone: config.ui.timezone,
+                                      date_format:
+                                        config.ui.time_format == "24hour"
+                                          ? t(
+                                              "time.formattedTimestamp2.24hour",
+                                              {
+                                                ns: "common",
+                                              },
+                                            )
+                                          : t(
+                                              "time.formattedTimestamp2.12hour",
+                                              {
+                                                ns: "common",
+                                              },
+                                            ),
+                                      time_style: "medium",
+                                      date_style: "medium",
+                                    },
+                                  )
+                                : "Never"}
+                              <Tooltip>
+                                <TooltipTrigger>
+                                  <LuSearch className="ml-2 size-3.5" />
+                                </TooltipTrigger>
+                                <TooltipContent>
+                                  {t("details.item.button.viewInExplore", {
+                                    ns: "views/explore",
+                                  })}
+                                </TooltipContent>
+                              </Tooltip>
+                            </div>
+                          </Link>
+                        </div>
+                      </div>
+
+                      <div className="flex items-center gap-2">
+                        <TooltipProvider>
+                          <Tooltip>
+                            <TooltipTrigger asChild>
+                              <Button
+                                size="sm"
+                                variant="outline"
+                                className="h-8 w-8 p-0"
+                                onClick={() => {
+                                  setSelectedTrigger(trigger);
+                                  setShowCreate(true);
+                                }}
+                                disabled={isLoading}
+                              >
+                                <LuPencil className="size-3.5" />
+                              </Button>
+                            </TooltipTrigger>
+                            <TooltipContent>
+                              <p>{t("triggers.table.edit")}</p>
+                            </TooltipContent>
+                          </Tooltip>
+                          <Tooltip>
+                            <TooltipTrigger asChild>
+                              <Button
+                                size="sm"
+                                variant="destructive"
+                                className="h-8 w-8 p-0 text-white"
+                                onClick={() => {
+                                  setSelectedTrigger(trigger);
+                                  setShowDelete(true);
+                                }}
+                                disabled={isLoading}
+                              >
+                                <LuTrash className="size-3.5" />
+                              </Button>
+                            </TooltipTrigger>
+                            <TooltipContent>
+                              <p>{t("triggers.table.deleteTrigger")}</p>
+                            </TooltipContent>
+                          </Tooltip>
+                        </TooltipProvider>
+                      </div>
+                    </div>
+                  ))}
+                </div>
+              )}
+            </div>
+          </div>
+        </div>
+      </div>
+      <CreateTriggerDialog
+        show={showCreate}
+        trigger={selectedTrigger}
+        selectedCamera={selectedCamera}
+        isLoading={isLoading}
+        onCreate={onCreate}
+        onEdit={onEdit}
+        onCancel={() => {
+          setShowCreate(false);
+          setSelectedTrigger(null);
+          setUnsavedChanges(false);
+        }}
+      />
+      <DeleteTriggerDialog
+        show={showDelete}
+        triggerName={selectedTrigger?.name ?? ""}
+        isLoading={isLoading}
+        onCancel={() => {
+          setShowDelete(false);
+          setSelectedTrigger(null);
+          setUnsavedChanges(false);
+        }}
+        onDelete={() => onDelete(selectedTrigger?.name ?? "")}
+      />
+    </div>
+  );
+}