Compare commits

...

12 Commits

Author SHA1 Message Date
Josh Hawkins
4ff61a77e7 add ability to download clean snapshot 2025-12-01 18:32:18 -06:00
Josh Hawkins
bf5e0c76fe fix trigger logic 2025-12-01 15:00:20 -06:00
Nicolas Mowen
e957f8d9f9 Fix incorrect averaging of the segments so it correctly only uses the most recent segments 2025-12-01 13:43:44 -07:00
Nicolas Mowen
cac29f96e7 Add bird to classification reference 2025-12-01 12:51:58 -07:00
Nicolas Mowen
755f51f1ad Make number of classification images to be kept configurable 2025-12-01 12:49:13 -07:00
Nicolas Mowen
d5f5e93f4f Update classification docs for training recommendations 2025-12-01 10:51:03 -07:00
Josh Hawkins
475ab146b4 update transcription docs 2025-12-01 11:13:59 -06:00
Josh Hawkins
0d614b5a3e tweak tracking details layout for small desktop sizes 2025-12-01 11:02:58 -06:00
Josh Hawkins
949c426df1 ensure audio events display timeline entries in tracking details 2025-11-30 12:32:32 -06:00
Nicolas Mowen
97b29d177a
Miscellaneous Fixes (#21072)
* Implement renaming in model editing dialog

* add transcription faq

* remove incorrect constraint for viewer as username

should be able to change anyone's role other than admin

* Don't save redundant state changes

* prevent crash when a camera doesn't support onvif imaging service required for focus support

* Fine tune behavior

* Stop redundant go2rtc stream metadata requests and defer audio information to allow bandwidth for image requests

* Improve cleanup logic for capture process

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-11-30 06:54:42 -06:00
Ryan Hass
1a75251ffb
Add yolov9 inference speeds for UHD 730 GPU. (#21090)
This adds the inference speeds measured on an i5-11400T with a UHD 730
GPU running at nominal temperatures.
2025-11-29 07:32:16 -06:00
Josh Hawkins
048475e750
API admin exemptions and route guard updates (#21094)
* update exempt paths and add missing guard to api endpoints

* admin only frigate+ submission
2025-11-29 07:30:04 -06:00
34 changed files with 656 additions and 314 deletions

View File

@ -191,6 +191,7 @@ ONVIF
openai
opencv
openvino
overfitting
OWASP
paddleocr
paho

View File

@ -157,3 +157,21 @@ Only one `speech` event may be transcribed at a time. Frigate does not automatic
:::
Recorded `speech` events will always use a `whisper` model, regardless of the `model_size` config setting. Without a supported Nvidia GPU, generating transcriptions for longer `speech` events may take a fair amount of time, so be patient.
#### FAQ
1. Why doesn't Frigate automatically transcribe all `speech` events?
Frigate does not implement a queue mechanism for speech transcription, and adding one is not trivial. A proper queue would need backpressure, prioritization, memory/disk buffering, retry logic, crash recovery, and safeguards to prevent unbounded growth when events outpace processing. Thats a significant amount of complexity for a feature that, in most real-world environments, would mostly just churn through low-value noise.
Because transcription is **serialized (one event at a time)** and speech events can be generated far faster than they can be processed, an auto-transcribe toggle would very quickly create an ever-growing backlog and degrade core functionality. For the amount of engineering and risk involved, it adds **very little practical value** for the majority of deployments, which are often on low-powered, edge hardware.
If you hear speech thats actually important and worth saving/indexing for the future, **just press the transcribe button in Explore** on that specific `speech` event - that keeps things explicit, reliable, and under your control.
Other options are being considered for future versions of Frigate to add transcription options that support external `whisper` Docker containers. A single transcription service could then be shared by Frigate and other applications (for example, Home Assistant Voice), and run on more powerful machines when available.
2. Why don't you save live transcription text and use that for `speech` events?
Theres no guarantee that a `speech` event is even created from the exact audio that went through the transcription model. Live transcription and `speech` event creation are **separate, asynchronous processes**. Even when both are correctly configured, trying to align the **precise start and end time of a speech event** with whatever audio the model happened to be processing at that moment is unreliable.
Automatically persisting that data would often result in **misaligned, partial, or irrelevant transcripts**, while still incurring all of the CPU, storage, and privacy costs of transcription. Thats why Frigate treats transcription as an **explicit, user-initiated action** rather than an automatic side-effect of every `speech` event.

View File

@ -69,4 +69,6 @@ Once all images are assigned, training will begin automatically.
### Improving the Model
- **Problem framing**: Keep classes visually distinct and state-focused (e.g., `open`, `closed`, `unknown`). Avoid combining object identity with state in a single model unless necessary.
- **Data collection**: Use the models Recent Classifications tab to gather balanced examples across times of day and weather.
- **Data collection**: Use the model's Recent Classifications tab to gather balanced examples across times of day and weather.
- **When to train**: Focus on cases where the model is entirely incorrect or flips between states when it should not. There's no need to train additional images when the model is already working consistently.
- **Selecting training images**: Images scoring below 100% due to new conditions (e.g., first snow of the year, seasonal changes) or variations (e.g., objects temporarily in view, insects at night) are good candidates for training, as they represent scenarios different from the default state. Training these lower-scoring images that differ from existing training data helps prevent overfitting. Avoid training large quantities of images that look very similar, especially if they already score 100% as this can lead to overfitting.

View File

@ -710,6 +710,44 @@ audio_transcription:
# List of language codes: https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L10
language: en
# Optional: Configuration for classification models
classification:
# Optional: Configuration for bird classification
bird:
# Optional: Enable bird classification (default: shown below)
enabled: False
# Optional: Minimum classification score required to be considered a match (default: shown below)
threshold: 0.9
custom:
# Required: name of the classification model
model_name:
# Optional: Enable running the model (default: shown below)
enabled: True
# Optional: Name of classification model (default: shown below)
name: None
# Optional: Classification score threshold to change the state (default: shown below)
threshold: 0.8
# Optional: Number of classification attempts to save in the recent classifications tab (default: shown below)
# NOTE: Defaults to 200 for object classification and 100 for state classification if not specified
save_attempts: None
# Optional: Object classification configuration
object_config:
# Required: Object types to classify
objects: [dog]
# Optional: Type of classification that is applied (default: shown below)
classification_type: sub_label
# Optional: State classification configuration
state_config:
# Required: Cameras to run classification on
cameras:
camera_name:
# Required: Crop of image frame on this camera to run classification on
crop: [0, 180, 220, 400]
# Optional: If classification should be run when motion is detected in the crop (default: shown below)
motion: False
# Optional: Interval to run classification on in seconds (default: shown below)
interval: None
# Optional: Restream configuration
# Uses https://github.com/AlexxIT/go2rtc (v1.9.10)
# NOTE: The default go2rtc API port (1984) must be used,

View File

@ -159,7 +159,7 @@ Inference speeds vary greatly depending on the CPU or GPU used, some known examp
| Intel HD 530 | 15 - 35 ms | | | | Can only run one detector instance |
| Intel HD 620 | 15 - 25 ms | | 320: ~ 35 ms | | |
| Intel HD 630 | ~ 15 ms | | 320: ~ 30 ms | | |
| Intel UHD 730 | ~ 10 ms | | 320: ~ 19 ms 640: ~ 54 ms | | |
| Intel UHD 730 | ~ 10 ms | t-320: 14ms s-320: 24ms t-640: 34ms s-640: 65ms | 320: ~ 19 ms 640: ~ 54 ms | | |
| Intel UHD 770 | ~ 15 ms | t-320: ~ 16 ms s-320: ~ 20 ms s-640: ~ 40 ms | 320: ~ 20 ms 640: ~ 46 ms | | |
| Intel N100 | ~ 15 ms | s-320: 30 ms | 320: ~ 25 ms | | Can only run one detector instance |
| Intel N150 | ~ 15 ms | t-320: 16 ms s-320: 24 ms | | | |

View File

@ -62,8 +62,8 @@ def require_admin_by_default():
"/",
"/version",
"/config/schema.json",
"/metrics",
# Authenticated user endpoints (allow_any_authenticated)
"/metrics",
"/stats",
"/stats/history",
"/config",
@ -76,22 +76,28 @@ def require_admin_by_default():
"/recognized_license_plates",
"/timeline",
"/timeline/hourly",
"/events/summary",
"/recordings/storage",
"/recordings/summary",
"/recordings/unavailable",
"/go2rtc/streams",
"/event_ids",
"/events",
"/exports",
}
# Path prefixes that should be exempt (for paths with parameters)
EXEMPT_PREFIXES = (
"/logs/", # /logs/{service}
"/review", # /review, /review/{id}, /review_ids, /review/summary, etc.
"/review", # /review, /review/{id}, /review/summary, /review_ids, etc.
"/reviews/", # /reviews/viewed, /reviews/delete
"/events/", # /events/{id}/thumbnail, etc. (camera-scoped)
"/events/", # /events/{id}/thumbnail, /events/summary, etc. (camera-scoped)
"/export/", # /export/{camera}/start/..., /export/{id}/rename, /export/{id}
"/go2rtc/streams/", # /go2rtc/streams/{camera}
"/users/", # /users/{username}/password (has own auth)
"/preview/", # /preview/{file}/thumbnail.jpg
"/exports/", # /exports/{export_id}
"/vod/", # /vod/{camera_name}/...
"/notifications/", # /notifications/pubkey, /notifications/register
)
async def admin_checker(request: Request):
@ -105,6 +111,24 @@ def require_admin_by_default():
if path.startswith(EXEMPT_PREFIXES):
return
# Dynamic camera path exemption:
# Any path whose first segment matches a configured camera name should
# bypass the global admin requirement. These endpoints enforce access
# via route-level dependencies (e.g. require_camera_access) to ensure
# per-camera authorization. This allows non-admin authenticated users
# (e.g. viewer role) to access camera-specific resources without
# needing admin privileges.
try:
if path.startswith("/"):
first_segment = path.split("/", 2)[1]
if (
first_segment
and first_segment in request.app.frigate_config.cameras
):
return
except Exception:
pass
# For all other paths, require admin role
# Port 5000 (internal) requests have admin role set automatically
role = request.headers.get("remote-role")
@ -113,7 +137,7 @@ def require_admin_by_default():
raise HTTPException(
status_code=403,
detail="Admin role required for this endpoint",
detail="Access denied. A user with the admin role is required.",
)
return admin_checker

View File

@ -70,6 +70,7 @@ router = APIRouter(tags=[Tags.events])
@router.get(
"/events",
response_model=list[EventResponse],
dependencies=[Depends(allow_any_authenticated())],
summary="Get events",
description="Returns a list of events.",
)
@ -344,6 +345,7 @@ def events(
@router.get(
"/events/explore",
response_model=list[EventResponse],
dependencies=[Depends(allow_any_authenticated())],
summary="Get summary of objects.",
description="""Gets a summary of objects from the database.
Returns a list of objects with a max of `limit` objects for each label.
@ -436,6 +438,7 @@ def events_explore(
@router.get(
"/event_ids",
response_model=list[EventResponse],
dependencies=[Depends(allow_any_authenticated())],
summary="Get events by ids.",
description="""Gets events by a list of ids.
Returns a list of events.
@ -469,6 +472,7 @@ async def event_ids(ids: str, request: Request):
@router.get(
"/events/search",
dependencies=[Depends(allow_any_authenticated())],
summary="Search events.",
description="""Searches for events in the database.
Returns a list of events.
@ -919,6 +923,7 @@ def events_summary(
@router.get(
"/events/{event_id}",
response_model=EventResponse,
dependencies=[Depends(allow_any_authenticated())],
summary="Get event by id.",
description="Gets an event by its id.",
)
@ -962,6 +967,7 @@ def set_retain(event_id: str):
@router.post(
"/events/{event_id}/plus",
response_model=EventUploadPlusResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Send event to Frigate+.",
description="""Sends an event to Frigate+.
Returns a success message or an error if the event is not found.
@ -1102,6 +1108,7 @@ async def send_to_plus(request: Request, event_id: str, body: SubmitPlusBody = N
@router.put(
"/events/{event_id}/false_positive",
response_model=EventUploadPlusResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Submit false positive to Frigate+",
description="""Submit an event as a false positive to Frigate+.
This endpoint is the same as the standard Frigate+ submission endpoint,
@ -1724,37 +1731,40 @@ def create_trigger_embedding(
if event.data.get("type") != "object":
return
if thumbnail := get_event_thumbnail_bytes(event):
cursor = context.db.execute_sql(
"""
SELECT thumbnail_embedding FROM vec_thumbnails WHERE id = ?
""",
[body.data],
# Get the thumbnail
thumbnail = get_event_thumbnail_bytes(event)
if thumbnail is None:
return JSONResponse(
content={
"success": False,
"message": f"Failed to get thumbnail for {body.data} for {body.type} trigger",
},
status_code=400,
)
row = cursor.fetchone() if cursor else None
# Try to reuse existing embedding from database
cursor = context.db.execute_sql(
"""
SELECT thumbnail_embedding FROM vec_thumbnails WHERE id = ?
""",
[body.data],
)
if row:
query_embedding = row[0]
embedding = np.frombuffer(query_embedding, dtype=np.float32)
row = cursor.fetchone() if cursor else None
if row:
query_embedding = row[0]
embedding = np.frombuffer(query_embedding, dtype=np.float32)
else:
# Extract valid thumbnail
thumbnail = get_event_thumbnail_bytes(event)
if thumbnail is None:
return JSONResponse(
content={
"success": False,
"message": f"Failed to get thumbnail for {body.data} for {body.type} trigger",
},
status_code=400,
)
# Generate new embedding
embedding = context.generate_image_embedding(
body.data, (base64.b64encode(thumbnail).decode("ASCII"))
)
if not embedding:
if embedding is None or (
isinstance(embedding, (list, np.ndarray)) and len(embedding) == 0
):
return JSONResponse(
content={
"success": False,
@ -1889,7 +1899,9 @@ def update_trigger_embedding(
body.data, (base64.b64encode(thumbnail).decode("ASCII"))
)
if not embedding:
if embedding is None or (
isinstance(embedding, (list, np.ndarray)) and len(embedding) == 0
):
return JSONResponse(
content={
"success": False,

View File

@ -14,6 +14,7 @@ from peewee import DoesNotExist
from playhouse.shortcuts import model_to_dict
from frigate.api.auth import (
allow_any_authenticated,
get_allowed_cameras_for_filter,
require_camera_access,
require_role,
@ -44,6 +45,7 @@ router = APIRouter(tags=[Tags.export])
@router.get(
"/exports",
response_model=ExportsResponse,
dependencies=[Depends(allow_any_authenticated())],
summary="Get exports",
description="""Gets all exports from the database for cameras the user has access to.
Returns a list of exports ordered by date (most recent first).""",
@ -272,6 +274,7 @@ async def export_delete(event_id: str, request: Request):
@router.get(
"/exports/{export_id}",
response_model=ExportModel,
dependencies=[Depends(allow_any_authenticated())],
summary="Get a single export",
description="""Gets a specific export by ID. The user must have access to the camera
associated with the export.""",

View File

@ -945,6 +945,7 @@ async def vod_hour(
@router.get(
"/vod/event/{event_id}",
dependencies=[Depends(allow_any_authenticated())],
description="Returns an HLS playlist for the specified object. Append /master.m3u8 or /index.m3u8 for HLS playback.",
)
async def vod_event(

View File

@ -5,11 +5,12 @@ import os
from typing import Any
from cryptography.hazmat.primitives import serialization
from fastapi import APIRouter, Request
from fastapi import APIRouter, Depends, Request
from fastapi.responses import JSONResponse
from peewee import DoesNotExist
from py_vapid import Vapid01, utils
from frigate.api.auth import allow_any_authenticated
from frigate.api.defs.tags import Tags
from frigate.const import CONFIG_DIR
from frigate.models import User
@ -21,6 +22,7 @@ router = APIRouter(tags=[Tags.notifications])
@router.get(
"/notifications/pubkey",
dependencies=[Depends(allow_any_authenticated())],
summary="Get VAPID public key",
description="""Gets the VAPID public key for the notifications.
Returns the public key or an error if notifications are not enabled.
@ -47,6 +49,7 @@ def get_vapid_pub_key(request: Request):
@router.post(
"/notifications/register",
dependencies=[Depends(allow_any_authenticated())],
summary="Register notifications",
description="""Registers a notifications subscription.
Returns a success message or an error if the subscription is not provided.

View File

@ -577,7 +577,9 @@ def delete_reviews(body: ReviewModifyMultipleBody):
@router.get(
"/review/activity/motion", response_model=list[ReviewActivityMotionResponse]
"/review/activity/motion",
response_model=list[ReviewActivityMotionResponse],
dependencies=[Depends(allow_any_authenticated())],
)
def motion_activity(
params: ReviewActivityMotionQueryParams = Depends(),
@ -739,6 +741,7 @@ async def set_not_reviewed(
@router.post(
"/review/summarize/start/{start_ts}/end/{end_ts}",
dependencies=[Depends(allow_any_authenticated())],
description="Use GenAI to summarize review items over a period of time.",
)
def generate_review_summary(request: Request, start_ts: float, end_ts: float):

View File

@ -105,6 +105,11 @@ class CustomClassificationConfig(FrigateBaseModel):
threshold: float = Field(
default=0.8, title="Classification score threshold to change the state."
)
save_attempts: int | None = Field(
default=None,
title="Number of classification attempts to save in the recent classifications tab. If not specified, defaults to 200 for object classification and 100 for state classification.",
ge=0,
)
object_config: CustomClassificationObjectConfig | None = Field(default=None)
state_config: CustomClassificationStateConfig | None = Field(default=None)

View File

@ -99,6 +99,42 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
if self.inference_speed:
self.inference_speed.update(duration)
def _should_save_image(
self, camera: str, detected_state: str, score: float = 1.0
) -> bool:
"""
Determine if we should save the image for training.
Save when:
- State is changing or being verified (regardless of score)
- Score is less than 100% (even if state matches, useful for training)
Don't save when:
- State is stable (matches current_state) AND score is 100%
"""
if camera not in self.state_history:
# First detection for this camera, save it
return True
verification = self.state_history[camera]
current_state = verification.get("current_state")
pending_state = verification.get("pending_state")
# Save if there's a pending state change being verified
if pending_state is not None:
return True
# Save if the detected state differs from the current verified state
# (state is changing)
if current_state is not None and detected_state != current_state:
return True
# If score is less than 100%, save even if state matches
# (useful for training to improve confidence)
if score < 1.0:
return True
# Don't save if state is stable (detected_state == current_state) AND score is 100%
return False
def verify_state_change(self, camera: str, detected_state: str) -> str | None:
"""
Verify state change requires 3 consecutive identical states before publishing.
@ -212,14 +248,22 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
return
if self.interpreter is None:
write_classification_attempt(
self.train_dir,
cv2.cvtColor(frame, cv2.COLOR_RGB2BGR),
"none-none",
now,
"unknown",
0.0,
)
# When interpreter is None, always save (score is 0.0, which is < 1.0)
if self._should_save_image(camera, "unknown", 0.0):
save_attempts = (
self.model_config.save_attempts
if self.model_config.save_attempts is not None
else 100
)
write_classification_attempt(
self.train_dir,
cv2.cvtColor(frame, cv2.COLOR_RGB2BGR),
"none-none",
now,
"unknown",
0.0,
max_files=save_attempts,
)
return
input = np.expand_dims(resized_frame, axis=0)
@ -236,14 +280,23 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
score = round(probs[best_id], 2)
self.__update_metrics(datetime.datetime.now().timestamp() - now)
write_classification_attempt(
self.train_dir,
cv2.cvtColor(frame, cv2.COLOR_RGB2BGR),
"none-none",
now,
self.labelmap[best_id],
score,
)
detected_state = self.labelmap[best_id]
if self._should_save_image(camera, detected_state, score):
save_attempts = (
self.model_config.save_attempts
if self.model_config.save_attempts is not None
else 100
)
write_classification_attempt(
self.train_dir,
cv2.cvtColor(frame, cv2.COLOR_RGB2BGR),
"none-none",
now,
detected_state,
score,
max_files=save_attempts,
)
if score < self.model_config.threshold:
logger.debug(
@ -251,7 +304,6 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
)
return
detected_state = self.labelmap[best_id]
verified_state = self.verify_state_change(camera, detected_state)
if verified_state is not None:
@ -442,6 +494,11 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
return
if self.interpreter is None:
save_attempts = (
self.model_config.save_attempts
if self.model_config.save_attempts is not None
else 200
)
write_classification_attempt(
self.train_dir,
cv2.cvtColor(crop, cv2.COLOR_RGB2BGR),
@ -449,6 +506,7 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
now,
"unknown",
0.0,
max_files=save_attempts,
)
return
@ -466,6 +524,11 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
score = round(probs[best_id], 2)
self.__update_metrics(datetime.datetime.now().timestamp() - now)
save_attempts = (
self.model_config.save_attempts
if self.model_config.save_attempts is not None
else 200
)
write_classification_attempt(
self.train_dir,
cv2.cvtColor(crop, cv2.COLOR_RGB2BGR),
@ -473,7 +536,7 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
now,
self.labelmap[best_id],
score,
max_files=200,
max_files=save_attempts,
)
if score < self.model_config.threshold:

View File

@ -190,7 +190,11 @@ class OnvifController:
ptz: ONVIFService = await onvif.create_ptz_service()
self.cams[camera_name]["ptz"] = ptz
imaging: ONVIFService = await onvif.create_imaging_service()
try:
imaging: ONVIFService = await onvif.create_imaging_service()
except (Fault, ONVIFError, TransportError, Exception) as e:
logger.debug(f"Imaging service not supported for {camera_name}: {e}")
imaging = None
self.cams[camera_name]["imaging"] = imaging
try:
video_sources = await media.GetVideoSources()
@ -381,7 +385,10 @@ class OnvifController:
f"Disabling autotracking zooming for {camera_name}: Absolute zoom not supported. Exception: {e}"
)
if self.cams[camera_name]["video_source_token"] is not None:
if (
self.cams[camera_name]["video_source_token"] is not None
and imaging is not None
):
try:
imaging_capabilities = await imaging.GetImagingSettings(
{"VideoSourceToken": self.cams[camera_name]["video_source_token"]}
@ -421,6 +428,7 @@ class OnvifController:
if (
"focus" in self.cams[camera_name]["features"]
and self.cams[camera_name]["video_source_token"]
and self.cams[camera_name]["imaging"] is not None
):
try:
stop_request = self.cams[camera_name]["imaging"].create_type("Stop")
@ -648,6 +656,7 @@ class OnvifController:
if (
"focus" not in self.cams[camera_name]["features"]
or not self.cams[camera_name]["video_source_token"]
or self.cams[camera_name]["imaging"] is None
):
logger.error(f"{camera_name} does not support ONVIF continuous focus.")
return

View File

@ -5,7 +5,7 @@ import shutil
import threading
from pathlib import Path
from peewee import fn
from peewee import SQL, fn
from frigate.config import FrigateConfig
from frigate.const import RECORD_DIR
@ -44,13 +44,19 @@ class StorageMaintainer(threading.Thread):
)
}
# calculate MB/hr
# calculate MB/hr from last 100 segments
try:
bandwidth = round(
Recordings.select(fn.AVG(bandwidth_equation))
# Subquery to get last 100 segments, then average their bandwidth
last_100 = (
Recordings.select(bandwidth_equation.alias("bw"))
.where(Recordings.camera == camera, Recordings.segment_size > 0)
.order_by(Recordings.start_time.desc())
.limit(100)
.scalar()
.alias("recent")
)
bandwidth = round(
Recordings.select(fn.AVG(SQL("bw"))).from_(last_100).scalar()
* 3600,
2,
)

View File

@ -330,7 +330,7 @@ def collect_state_classification_examples(
1. Queries review items from specified cameras
2. Selects 100 balanced timestamps across the data
3. Extracts keyframes from recordings (cropped to specified regions)
4. Selects 20 most visually distinct images
4. Selects 24 most visually distinct images
5. Saves them to the dataset directory
Args:
@ -660,7 +660,6 @@ def collect_object_classification_examples(
Args:
model_name: Name of the classification model
label: Object label to collect (e.g., "person", "car")
cameras: List of camera names to collect examples from
"""
dataset_dir = os.path.join(CLIPS_DIR, model_name, "dataset")
temp_dir = os.path.join(dataset_dir, "temp")

View File

@ -124,45 +124,50 @@ def capture_frames(
config_subscriber.check_for_updates()
return config.enabled
while not stop_event.is_set():
if not get_enabled_state():
logger.debug(f"Stopping capture thread for disabled {config.name}")
break
fps.value = frame_rate.eps()
skipped_fps.value = skipped_eps.eps()
current_frame.value = datetime.now().timestamp()
frame_name = f"{config.name}_frame{frame_index}"
frame_buffer = frame_manager.write(frame_name)
try:
frame_buffer[:] = ffmpeg_process.stdout.read(frame_size)
except Exception:
# shutdown has been initiated
if stop_event.is_set():
try:
while not stop_event.is_set():
if not get_enabled_state():
logger.debug(f"Stopping capture thread for disabled {config.name}")
break
logger.error(f"{config.name}: Unable to read frames from ffmpeg process.")
fps.value = frame_rate.eps()
skipped_fps.value = skipped_eps.eps()
current_frame.value = datetime.now().timestamp()
frame_name = f"{config.name}_frame{frame_index}"
frame_buffer = frame_manager.write(frame_name)
try:
frame_buffer[:] = ffmpeg_process.stdout.read(frame_size)
except Exception:
# shutdown has been initiated
if stop_event.is_set():
break
if ffmpeg_process.poll() is not None:
logger.error(
f"{config.name}: ffmpeg process is not running. exiting capture thread..."
f"{config.name}: Unable to read frames from ffmpeg process."
)
break
continue
if ffmpeg_process.poll() is not None:
logger.error(
f"{config.name}: ffmpeg process is not running. exiting capture thread..."
)
break
frame_rate.update()
continue
# don't lock the queue to check, just try since it should rarely be full
try:
# add to the queue
frame_queue.put((frame_name, current_frame.value), False)
frame_manager.close(frame_name)
except queue.Full:
# if the queue is full, skip this frame
skipped_eps.update()
frame_rate.update()
frame_index = 0 if frame_index == shm_frame_count - 1 else frame_index + 1
# don't lock the queue to check, just try since it should rarely be full
try:
# add to the queue
frame_queue.put((frame_name, current_frame.value), False)
frame_manager.close(frame_name)
except queue.Full:
# if the queue is full, skip this frame
skipped_eps.update()
frame_index = 0 if frame_index == shm_frame_count - 1 else frame_index + 1
finally:
config_subscriber.stop()
class CameraWatchdog(threading.Thread):
@ -234,6 +239,16 @@ class CameraWatchdog(threading.Thread):
else:
self.ffmpeg_detect_process.wait()
# Wait for old capture thread to fully exit before starting a new one
if self.capture_thread is not None and self.capture_thread.is_alive():
self.logger.info("Waiting for capture thread to exit...")
self.capture_thread.join(timeout=5)
if self.capture_thread.is_alive():
self.logger.warning(
f"Capture thread for {self.config.name} did not exit in time"
)
self.logger.error(
"The following ffmpeg logs include the last 100 lines prior to exit."
)

View File

@ -170,6 +170,10 @@
"label": "Download snapshot",
"aria": "Download snapshot"
},
"downloadCleanSnapshot": {
"label": "Download clean snapshot",
"aria": "Download clean snapshot"
},
"viewTrackingDetails": {
"label": "View tracking details",
"aria": "Show the tracking details"

View File

@ -37,7 +37,7 @@ import { useForm } from "react-hook-form";
import { useTranslation } from "react-i18next";
import { LuPlus, LuX } from "react-icons/lu";
import { toast } from "sonner";
import useSWR from "swr";
import useSWR, { mutate } from "swr";
import { z } from "zod";
type ClassificationModelEditDialogProps = {
@ -240,15 +240,61 @@ export default function ClassificationModelEditDialog({
position: "top-center",
});
} else {
// State model - update classes
// Note: For state models, updating classes requires renaming categories
// which is handled through the dataset API, not the config API
// We'll need to implement this by calling the rename endpoint for each class
// For now, we just show a message that this requires retraining
const stateData = data as StateFormData;
const newClasses = stateData.classes.filter(
(c) => c.trim().length > 0,
);
const oldClasses = dataset?.categories
? Object.keys(dataset.categories).filter((key) => key !== "none")
: [];
toast.info(t("edit.stateClassesInfo"), {
position: "top-center",
});
const renameMap = new Map<string, string>();
const maxLength = Math.max(oldClasses.length, newClasses.length);
for (let i = 0; i < maxLength; i++) {
const oldClass = oldClasses[i];
const newClass = newClasses[i];
if (oldClass && newClass && oldClass !== newClass) {
renameMap.set(oldClass, newClass);
}
}
const renamePromises = Array.from(renameMap.entries()).map(
async ([oldName, newName]) => {
try {
await axios.put(
`/classification/${model.name}/dataset/${oldName}/rename`,
{
new_category: newName,
},
);
} catch (err) {
const error = err as {
response?: { data?: { message?: string; detail?: string } };
};
const errorMessage =
error.response?.data?.message ||
error.response?.data?.detail ||
"Unknown error";
throw new Error(
`Failed to rename ${oldName} to ${newName}: ${errorMessage}`,
);
}
},
);
if (renamePromises.length > 0) {
await Promise.all(renamePromises);
await mutate(`classification/${model.name}/dataset`);
toast.success(t("toast.success.updatedModel"), {
position: "top-center",
});
} else {
toast.info(t("edit.stateClassesInfo"), {
position: "top-center",
});
}
}
onSuccess();
@ -256,8 +302,10 @@ export default function ClassificationModelEditDialog({
} catch (err) {
const error = err as {
response?: { data?: { message?: string; detail?: string } };
message?: string;
};
const errorMessage =
error.message ||
error.response?.data?.message ||
error.response?.data?.detail ||
"Unknown error";
@ -268,7 +316,7 @@ export default function ClassificationModelEditDialog({
setIsSaving(false);
}
},
[isObjectModel, model, t, onSuccess, onClose],
[isObjectModel, model, dataset, t, onSuccess, onClose],
);
const handleCancel = useCallback(() => {

View File

@ -48,6 +48,7 @@ import { useTranslation } from "react-i18next";
import { useDateLocale } from "@/hooks/use-date-locale";
import { useIsAdmin } from "@/hooks/use-is-admin";
import { CameraNameLabel } from "../camera/FriendlyNameLabel";
import { LiveStreamMetadata } from "@/types/live";
type LiveContextMenuProps = {
className?: string;
@ -68,6 +69,7 @@ type LiveContextMenuProps = {
resetPreferredLiveMode: () => void;
config?: FrigateConfig;
children?: ReactNode;
streamMetadata?: { [key: string]: LiveStreamMetadata };
};
export default function LiveContextMenu({
className,
@ -88,6 +90,7 @@ export default function LiveContextMenu({
resetPreferredLiveMode,
config,
children,
streamMetadata,
}: LiveContextMenuProps) {
const { t } = useTranslation("views/live");
const [showSettings, setShowSettings] = useState(false);
@ -558,6 +561,7 @@ export default function LiveContextMenu({
setGroupStreamingSettings={setGroupStreamingSettings}
setIsDialogOpen={setShowSettings}
onSave={onSave}
streamMetadata={streamMetadata}
/>
</Dialog>
</div>

View File

@ -108,6 +108,18 @@ export default function SearchResultActions({
</a>
</MenuItem>
)}
{searchResult.has_snapshot &&
config?.cameras[searchResult.camera].snapshots.clean_copy && (
<MenuItem aria-label={t("itemMenu.downloadCleanSnapshot.aria")}>
<a
className="flex items-center"
href={`${baseUrl}api/events/${searchResult.id}/snapshot-clean.webp`}
download={`${searchResult.camera}_${searchResult.label}-clean.webp`}
>
<span>{t("itemMenu.downloadCleanSnapshot.label")}</span>
</a>
</MenuItem>
)}
{searchResult.data.type == "object" && (
<MenuItem
aria-label={t("itemMenu.viewTrackingDetails.aria")}

View File

@ -69,6 +69,20 @@ export default function DetailActionsMenu({
</a>
</DropdownMenuItem>
)}
{search.has_snapshot &&
config?.cameras[search.camera].snapshots.clean_copy && (
<DropdownMenuItem>
<a
className="w-full"
href={`${baseUrl}api/events/${search.id}/snapshot-clean.webp`}
download={`${search.camera}_${search.label}-clean.webp`}
>
<div className="flex cursor-pointer items-center gap-2">
<span>{t("itemMenu.downloadCleanSnapshot.label")}</span>
</div>
</a>
</DropdownMenuItem>
)}
{search.has_clip && (
<DropdownMenuItem>
<a

View File

@ -498,7 +498,7 @@ export default function SearchDetailDialog({
const views = [...SEARCH_TABS];
if (search.data.type != "object" || !search.has_clip) {
if (!search.has_clip) {
const index = views.indexOf("tracking_details");
views.splice(index, 1);
}
@ -548,7 +548,7 @@ export default function SearchDetailDialog({
"relative flex items-center justify-between",
"w-full",
// match dialog's max-width classes
"sm:max-w-xl md:max-w-4xl lg:max-w-[70%]",
"max-h-[95dvh] max-w-[85%] xl:max-w-[70%]",
)}
>
<Tooltip>
@ -594,8 +594,7 @@ export default function SearchDetailDialog({
ref={isDesktop ? dialogContentRef : undefined}
className={cn(
"scrollbar-container overflow-y-auto",
isDesktop &&
"max-h-[95dvh] sm:max-w-xl md:max-w-4xl lg:max-w-[70%]",
isDesktop && "max-h-[95dvh] max-w-[85%] xl:max-w-[70%]",
isMobile && "flex h-full flex-col px-4",
)}
onEscapeKeyDown={(event) => {
@ -1299,7 +1298,8 @@ function ObjectDetailsTab({
</div>
</div>
{search.data.type === "object" &&
{isAdmin &&
search.data.type === "object" &&
config?.plus?.enabled &&
search.end_time != undefined &&
search.has_snapshot && (

View File

@ -38,6 +38,7 @@ import { isDesktop, isIOS, isMobileOnly, isSafari } from "react-device-detect";
import { useApiHost } from "@/api";
import ImageLoadingIndicator from "@/components/indicators/ImageLoadingIndicator";
import ObjectTrackOverlay from "../ObjectTrackOverlay";
import { useIsAdmin } from "@/hooks/use-is-admin";
type TrackingDetailsProps = {
className?: string;
@ -621,7 +622,7 @@ export function TrackingDetails({
<div
className={cn(
isDesktop && "justify-between overflow-hidden md:basis-2/5",
isDesktop && "justify-between overflow-hidden lg:basis-2/5",
)}
>
{isDesktop && tabs && (
@ -777,6 +778,7 @@ function LifecycleIconRow({
const { data: config } = useSWR<FrigateConfig>("config");
const [isOpen, setIsOpen] = useState(false);
const navigate = useNavigate();
const isAdmin = useIsAdmin();
const aspectRatio = useMemo(() => {
if (!config) {
@ -898,102 +900,105 @@ function LifecycleIconRow({
<div className="text-md flex items-start break-words text-left">
{getLifecycleItemDescription(item)}
</div>
<div className="my-2 ml-2 flex flex-col flex-wrap items-start gap-1.5 text-xs text-secondary-foreground">
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.score")}
</span>
<span className="font-medium text-primary">{score}</span>
</div>
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.ratio")}
</span>
<span className="font-medium text-primary">{ratio}</span>
</div>
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.area")}{" "}
{attributeAreaPx !== undefined &&
attributeAreaPct !== undefined && (
<span className="text-primary-variant">
({getTranslatedLabel(item.data.label)})
</span>
)}
</span>
{areaPx !== undefined && areaPct !== undefined ? (
<span className="font-medium text-primary">
{t("information.pixels", { ns: "common", area: areaPx })} ·{" "}
{areaPct}%
{/* Only show Score/Ratio/Area for object events, not for audio (heard) or manual API (external) events */}
{item.class_type !== "heard" && item.class_type !== "external" && (
<div className="my-2 ml-2 flex flex-col flex-wrap items-start gap-1.5 text-xs text-secondary-foreground">
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.score")}
</span>
) : (
<span>N/A</span>
)}
</div>
{attributeAreaPx !== undefined &&
attributeAreaPct !== undefined && (
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.area")} (
{getTranslatedLabel(item.data.attribute)})
</span>
<span className="font-medium text-primary">
{t("information.pixels", {
ns: "common",
area: attributeAreaPx,
})}{" "}
· {attributeAreaPct}%
</span>
</div>
)}
{item.data?.zones && item.data.zones.length > 0 && (
<div className="mt-1 flex flex-wrap items-center gap-2">
{item.data.zones.map((zone, zidx) => {
const color = getZoneColor(zone)?.join(",") ?? "0,0,0";
return (
<Badge
key={`${zone}-${zidx}`}
variant="outline"
className="inline-flex cursor-pointer items-center gap-2"
onClick={(e: React.MouseEvent) => {
e.stopPropagation();
setSelectedZone(zone);
}}
style={{
borderColor: `rgba(${color}, 0.6)`,
background: `rgba(${color}, 0.08)`,
}}
>
<span
className="size-1 rounded-full"
style={{
display: "inline-block",
width: 10,
height: 10,
backgroundColor: `rgb(${color})`,
}}
/>
<span
className={cn(
item.data?.zones_friendly_names?.[zidx] === zone &&
"smart-capitalize",
)}
>
{item.data?.zones_friendly_names?.[zidx]}
</span>
</Badge>
);
})}
<span className="font-medium text-primary">{score}</span>
</div>
)}
</div>
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.ratio")}
</span>
<span className="font-medium text-primary">{ratio}</span>
</div>
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.area")}{" "}
{attributeAreaPx !== undefined &&
attributeAreaPct !== undefined && (
<span className="text-primary-variant">
({getTranslatedLabel(item.data.label)})
</span>
)}
</span>
{areaPx !== undefined && areaPct !== undefined ? (
<span className="font-medium text-primary">
{t("information.pixels", { ns: "common", area: areaPx })}{" "}
· {areaPct}%
</span>
) : (
<span>N/A</span>
)}
</div>
{attributeAreaPx !== undefined &&
attributeAreaPct !== undefined && (
<div className="flex items-center gap-1.5">
<span className="text-primary-variant">
{t("trackingDetails.lifecycleItemDesc.header.area")} (
{getTranslatedLabel(item.data.attribute)})
</span>
<span className="font-medium text-primary">
{t("information.pixels", {
ns: "common",
area: attributeAreaPx,
})}{" "}
· {attributeAreaPct}%
</span>
</div>
)}
</div>
)}
{item.data?.zones && item.data.zones.length > 0 && (
<div className="mt-1 flex flex-wrap items-center gap-2">
{item.data.zones.map((zone, zidx) => {
const color = getZoneColor(zone)?.join(",") ?? "0,0,0";
return (
<Badge
key={`${zone}-${zidx}`}
variant="outline"
className="inline-flex cursor-pointer items-center gap-2"
onClick={(e: React.MouseEvent) => {
e.stopPropagation();
setSelectedZone(zone);
}}
style={{
borderColor: `rgba(${color}, 0.6)`,
background: `rgba(${color}, 0.08)`,
}}
>
<span
className="size-1 rounded-full"
style={{
display: "inline-block",
width: 10,
height: 10,
backgroundColor: `rgb(${color})`,
}}
/>
<span
className={cn(
item.data?.zones_friendly_names?.[zidx] === zone &&
"smart-capitalize",
)}
>
{item.data?.zones_friendly_names?.[zidx]}
</span>
</Badge>
);
})}
</div>
)}
</div>
</div>
<div className="ml-3 flex-shrink-0 px-1 text-right text-xs text-primary-variant">
<div className="flex flex-row items-center gap-3">
<div className="whitespace-nowrap">{formattedEventTimestamp}</div>
{(config?.plus?.enabled || item.data.box) && (
{((isAdmin && config?.plus?.enabled) || item.data.box) && (
<DropdownMenu open={isOpen} onOpenChange={setIsOpen}>
<DropdownMenuTrigger>
<div className="rounded p-1 pr-2" role="button">
@ -1002,7 +1007,7 @@ function LifecycleIconRow({
</DropdownMenuTrigger>
<DropdownMenuPortal>
<DropdownMenuContent>
{config?.plus?.enabled && (
{isAdmin && config?.plus?.enabled && (
<DropdownMenuItem
className="cursor-pointer"
onSelect={async () => {

View File

@ -20,6 +20,7 @@ import ImageLoadingIndicator from "@/components/indicators/ImageLoadingIndicator
import { baseUrl } from "@/api/baseUrl";
import { getTranslatedLabel } from "@/utils/i18n";
import useImageLoaded from "@/hooks/use-image-loaded";
import { useIsAdmin } from "@/hooks/use-is-admin";
export type FrigatePlusDialogProps = {
upload?: Event;
@ -57,7 +58,9 @@ export function FrigatePlusDialog({
);
const [imgRef, imgLoaded, onImgLoad] = useImageLoaded();
const isAdmin = useIsAdmin();
const showCard =
isAdmin &&
!!upload &&
upload.data.type === "object" &&
upload.plus_id !== "not_enabled" &&

View File

@ -20,6 +20,7 @@ import { cn } from "@/lib/utils";
import { ASPECT_VERTICAL_LAYOUT, RecordingPlayerError } from "@/types/record";
import { useTranslation } from "react-i18next";
import ObjectTrackOverlay from "@/components/overlay/ObjectTrackOverlay";
import { useIsAdmin } from "@/hooks/use-is-admin";
// Android native hls does not seek correctly
const USE_NATIVE_HLS = false;
@ -83,6 +84,7 @@ export default function HlsVideoPlayer({
}: HlsVideoPlayerProps) {
const { t } = useTranslation("components/player");
const { data: config } = useSWR<FrigateConfig>("config");
const isAdmin = useIsAdmin();
// for detail stream context in History
const currentTime = currentTimeOverride;
@ -285,7 +287,7 @@ export default function HlsVideoPlayer({
volume: true,
seek: true,
playbackRate: true,
plusUpload: config?.plus?.enabled == true,
plusUpload: isAdmin && config?.plus?.enabled == true,
fullscreen: supportsFullscreen,
}}
setControlsOpen={setControlsOpen}

View File

@ -38,6 +38,7 @@ import { useCameraFriendlyName } from "@/hooks/use-camera-friendly-name";
type CameraStreamingDialogProps = {
camera: string;
groupStreamingSettings: GroupStreamingSettings;
streamMetadata?: { [key: string]: LiveStreamMetadata };
setGroupStreamingSettings: React.Dispatch<
React.SetStateAction<GroupStreamingSettings>
>;
@ -48,6 +49,7 @@ type CameraStreamingDialogProps = {
export function CameraStreamingDialog({
camera,
groupStreamingSettings,
streamMetadata,
setGroupStreamingSettings,
setIsDialogOpen,
onSave,
@ -76,12 +78,7 @@ export function CameraStreamingDialog({
[config, streamName],
);
const { data: cameraMetadata } = useSWR<LiveStreamMetadata>(
isRestreamed ? `go2rtc/streams/${streamName}` : null,
{
revalidateOnFocus: false,
},
);
const cameraMetadata = streamName ? streamMetadata?.[streamName] : undefined;
const supportsAudioOutput = useMemo(() => {
if (!cameraMetadata) {

View File

@ -13,6 +13,7 @@ import { useTranslation } from "react-i18next";
import { Event } from "@/types/event";
import { FrigateConfig } from "@/types/frigateConfig";
import { useState } from "react";
import { useIsAdmin } from "@/hooks/use-is-admin";
type EventMenuProps = {
event: Event;
@ -35,6 +36,7 @@ export default function EventMenu({
const navigate = useNavigate();
const { t } = useTranslation("views/explore");
const [isOpen, setIsOpen] = useState(false);
const isAdmin = useIsAdmin();
const handleObjectSelect = () => {
if (isSelected) {
@ -85,7 +87,8 @@ export default function EventMenu({
</a>
</DropdownMenuItem>
{event.has_snapshot &&
{isAdmin &&
event.has_snapshot &&
event.plus_id == undefined &&
event.data.type == "object" &&
config?.plus?.enabled && (

View File

@ -1,8 +1,8 @@
import { baseUrl } from "@/api/baseUrl";
import { CameraConfig, FrigateConfig } from "@/types/frigateConfig";
import { useCallback, useEffect, useState, useMemo } from "react";
import useSWR from "swr";
import { LivePlayerMode, LiveStreamMetadata } from "@/types/live";
import { LivePlayerMode } from "@/types/live";
import useDeferredStreamMetadata from "./use-deferred-stream-metadata";
export default function useCameraLiveMode(
cameras: CameraConfig[],
@ -11,9 +11,9 @@ export default function useCameraLiveMode(
) {
const { data: config } = useSWR<FrigateConfig>("config");
// Get comma-separated list of restreamed stream names for SWR key
const restreamedStreamsKey = useMemo(() => {
if (!cameras || !config) return null;
// Compute which streams need metadata (restreamed streams only)
const restreamedStreamNames = useMemo(() => {
if (!cameras || !config) return [];
const streamNames = new Set<string>();
cameras.forEach((camera) => {
@ -32,56 +32,13 @@ export default function useCameraLiveMode(
}
});
return streamNames.size > 0
? Array.from(streamNames).sort().join(",")
: null;
return Array.from(streamNames);
}, [cameras, config, activeStreams]);
const streamsFetcher = useCallback(async (key: string) => {
const streamNames = key.split(",");
const metadataPromises = streamNames.map(async (streamName) => {
try {
const response = await fetch(
`${baseUrl}api/go2rtc/streams/${streamName}`,
{
priority: "low",
},
);
if (response.ok) {
const data = await response.json();
return { streamName, data };
}
return { streamName, data: null };
} catch (error) {
// eslint-disable-next-line no-console
console.error(`Failed to fetch metadata for ${streamName}:`, error);
return { streamName, data: null };
}
});
const results = await Promise.allSettled(metadataPromises);
const metadata: { [key: string]: LiveStreamMetadata } = {};
results.forEach((result) => {
if (result.status === "fulfilled" && result.value.data) {
metadata[result.value.streamName] = result.value.data;
}
});
return metadata;
}, []);
const { data: allStreamMetadata = {} } = useSWR<{
[key: string]: LiveStreamMetadata;
}>(restreamedStreamsKey, streamsFetcher, {
revalidateOnFocus: false,
revalidateOnReconnect: false,
revalidateIfStale: false,
dedupingInterval: 60000,
});
// Fetch stream metadata with deferred loading (doesn't block initial render)
const streamMetadata = useDeferredStreamMetadata(restreamedStreamNames);
// Compute live mode states
const [preferredLiveModes, setPreferredLiveModes] = useState<{
[key: string]: LivePlayerMode;
}>({});
@ -122,10 +79,10 @@ export default function useCameraLiveMode(
newPreferredLiveModes[camera.name] = isRestreamed ? "mse" : "jsmpeg";
}
// check each stream for audio support
// Check each stream for audio support
if (isRestreamed) {
Object.values(camera.live.streams).forEach((streamName) => {
const metadata = allStreamMetadata?.[streamName];
const metadata = streamMetadata[streamName];
newSupportsAudioOutputStates[streamName] = {
supportsAudio: metadata
? metadata.producers.find(
@ -150,7 +107,7 @@ export default function useCameraLiveMode(
setPreferredLiveModes(newPreferredLiveModes);
setIsRestreamedStates(newIsRestreamedStates);
setSupportsAudioOutputStates(newSupportsAudioOutputStates);
}, [cameras, config, windowVisible, allStreamMetadata]);
}, [cameras, config, windowVisible, streamMetadata]);
const resetPreferredLiveMode = useCallback(
(cameraName: string) => {
@ -180,5 +137,6 @@ export default function useCameraLiveMode(
resetPreferredLiveMode,
isRestreamedStates,
supportsAudioOutputStates,
streamMetadata,
};
}

View File

@ -0,0 +1,90 @@
import { baseUrl } from "@/api/baseUrl";
import { useCallback, useEffect, useState, useMemo } from "react";
import useSWR from "swr";
import { LiveStreamMetadata } from "@/types/live";
const FETCH_TIMEOUT_MS = 10000;
const DEFER_DELAY_MS = 2000;
/**
* Hook that fetches go2rtc stream metadata with deferred loading.
*
* Metadata fetching is delayed to prevent blocking initial page load
* and camera image requests.
*
* @param streamNames - Array of stream names to fetch metadata for
* @returns Object containing stream metadata keyed by stream name
*/
export default function useDeferredStreamMetadata(streamNames: string[]) {
const [fetchEnabled, setFetchEnabled] = useState(false);
useEffect(() => {
const timeoutId = setTimeout(() => {
setFetchEnabled(true);
}, DEFER_DELAY_MS);
return () => clearTimeout(timeoutId);
}, []);
const swrKey = useMemo(() => {
if (!fetchEnabled || streamNames.length === 0) return null;
// Use spread to avoid mutating the original array
return `deferred-streams:${[...streamNames].sort().join(",")}`;
}, [fetchEnabled, streamNames]);
const fetcher = useCallback(async (key: string) => {
// Extract stream names from key (remove prefix)
const names = key.replace("deferred-streams:", "").split(",");
const promises = names.map(async (streamName) => {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), FETCH_TIMEOUT_MS);
try {
const response = await fetch(
`${baseUrl}api/go2rtc/streams/${streamName}`,
{
priority: "low",
signal: controller.signal,
},
);
clearTimeout(timeoutId);
if (response.ok) {
const data = await response.json();
return { streamName, data };
}
return { streamName, data: null };
} catch (error) {
clearTimeout(timeoutId);
if ((error as Error).name !== "AbortError") {
// eslint-disable-next-line no-console
console.error(`Failed to fetch metadata for ${streamName}:`, error);
}
return { streamName, data: null };
}
});
const results = await Promise.allSettled(promises);
const metadata: { [key: string]: LiveStreamMetadata } = {};
results.forEach((result) => {
if (result.status === "fulfilled" && result.value.data) {
metadata[result.value.streamName] = result.value.data;
}
});
return metadata;
}, []);
const { data: metadata = {} } = useSWR<{
[key: string]: LiveStreamMetadata;
}>(swrKey, fetcher, {
revalidateOnFocus: false,
revalidateOnReconnect: false,
revalidateIfStale: false,
dedupingInterval: 60000,
});
return metadata;
}

View File

@ -305,6 +305,7 @@ export type CustomClassificationModelConfig = {
enabled: boolean;
name: string;
threshold: number;
save_attempts?: number;
object_config?: {
objects: string[];
classification_type: string;

View File

@ -24,6 +24,7 @@ import "react-resizable/css/styles.css";
import {
AudioState,
LivePlayerMode,
LiveStreamMetadata,
StatsState,
VolumeState,
} from "@/types/live";
@ -47,7 +48,6 @@ import {
TooltipContent,
} from "@/components/ui/tooltip";
import { Toaster } from "@/components/ui/sonner";
import useCameraLiveMode from "@/hooks/use-camera-live-mode";
import LiveContextMenu from "@/components/menu/LiveContextMenu";
import { useStreamingSettings } from "@/context/streaming-settings-provider";
import { useTranslation } from "react-i18next";
@ -65,6 +65,16 @@ type DraggableGridLayoutProps = {
setIsEditMode: React.Dispatch<React.SetStateAction<boolean>>;
fullscreen: boolean;
toggleFullscreen: () => void;
preferredLiveModes: { [key: string]: LivePlayerMode };
setPreferredLiveModes: React.Dispatch<
React.SetStateAction<{ [key: string]: LivePlayerMode }>
>;
resetPreferredLiveMode: (cameraName: string) => void;
isRestreamedStates: { [key: string]: boolean };
supportsAudioOutputStates: {
[key: string]: { supportsAudio: boolean; cameraName: string };
};
streamMetadata: { [key: string]: LiveStreamMetadata };
};
export default function DraggableGridLayout({
cameras,
@ -79,6 +89,12 @@ export default function DraggableGridLayout({
setIsEditMode,
fullscreen,
toggleFullscreen,
preferredLiveModes,
setPreferredLiveModes,
resetPreferredLiveMode,
isRestreamedStates,
supportsAudioOutputStates,
streamMetadata,
}: DraggableGridLayoutProps) {
const { t } = useTranslation(["views/live"]);
const { data: config } = useSWR<FrigateConfig>("config");
@ -98,33 +114,6 @@ export default function DraggableGridLayout({
}
}, [allGroupsStreamingSettings, cameraGroup]);
const activeStreams = useMemo(() => {
const streams: { [cameraName: string]: string } = {};
cameras.forEach((camera) => {
const availableStreams = camera.live.streams || {};
const streamNameFromSettings =
currentGroupStreamingSettings?.[camera.name]?.streamName || "";
const streamExists =
streamNameFromSettings &&
Object.values(availableStreams).includes(streamNameFromSettings);
const streamName = streamExists
? streamNameFromSettings
: Object.values(availableStreams)[0] || "";
streams[camera.name] = streamName;
});
return streams;
}, [cameras, currentGroupStreamingSettings]);
const {
preferredLiveModes,
setPreferredLiveModes,
resetPreferredLiveMode,
isRestreamedStates,
supportsAudioOutputStates,
} = useCameraLiveMode(cameras, windowVisible, activeStreams);
// grid layout
const ResponsiveGridLayout = useMemo(() => WidthProvider(Responsive), []);
@ -624,6 +613,7 @@ export default function DraggableGridLayout({
resetPreferredLiveMode(camera.name)
}
config={config}
streamMetadata={streamMetadata}
>
<LivePlayer
key={camera.name}
@ -838,6 +828,7 @@ type GridLiveContextMenuProps = {
unmuteAll: () => void;
resetPreferredLiveMode: () => void;
config?: FrigateConfig;
streamMetadata?: { [key: string]: LiveStreamMetadata };
};
const GridLiveContextMenu = React.forwardRef<
@ -868,6 +859,7 @@ const GridLiveContextMenu = React.forwardRef<
unmuteAll,
resetPreferredLiveMode,
config,
streamMetadata,
...props
},
ref,
@ -899,6 +891,7 @@ const GridLiveContextMenu = React.forwardRef<
unmuteAll={unmuteAll}
resetPreferredLiveMode={resetPreferredLiveMode}
config={config}
streamMetadata={streamMetadata}
>
{children}
</LiveContextMenu>

View File

@ -265,6 +265,7 @@ export default function LiveDashboardView({
resetPreferredLiveMode,
isRestreamedStates,
supportsAudioOutputStates,
streamMetadata,
} = useCameraLiveMode(cameras, windowVisible, activeStreams);
const birdseyeConfig = useMemo(() => config?.birdseye, [config]);
@ -650,6 +651,12 @@ export default function LiveDashboardView({
setIsEditMode={setIsEditMode}
fullscreen={fullscreen}
toggleFullscreen={toggleFullscreen}
preferredLiveModes={preferredLiveModes}
setPreferredLiveModes={setPreferredLiveModes}
resetPreferredLiveMode={resetPreferredLiveMode}
isRestreamedStates={isRestreamedStates}
supportsAudioOutputStates={supportsAudioOutputStates}
streamMetadata={streamMetadata}
/>
)}
</>

View File

@ -478,33 +478,32 @@ export default function AuthenticationView({
<TableCell className="text-right">
<TooltipProvider>
<div className="flex items-center justify-end gap-2">
{user.username !== "admin" &&
user.username !== "viewer" && (
<Tooltip>
<TooltipTrigger asChild>
<Button
size="sm"
variant="outline"
className="h-8 px-2"
onClick={() => {
setSelectedUser(user.username);
setSelectedUserRole(
user.role || "viewer",
);
setShowRoleChange(true);
}}
>
<LuUserCog className="size-3.5" />
<span className="ml-1.5 hidden sm:inline-block">
{t("role.title", { ns: "common" })}
</span>
</Button>
</TooltipTrigger>
<TooltipContent>
<p>{t("users.table.changeRole")}</p>
</TooltipContent>
</Tooltip>
)}
{user.username !== "admin" && (
<Tooltip>
<TooltipTrigger asChild>
<Button
size="sm"
variant="outline"
className="h-8 px-2"
onClick={() => {
setSelectedUser(user.username);
setSelectedUserRole(
user.role || "viewer",
);
setShowRoleChange(true);
}}
>
<LuUserCog className="size-3.5" />
<span className="ml-1.5 hidden sm:inline-block">
{t("role.title", { ns: "common" })}
</span>
</Button>
</TooltipTrigger>
<TooltipContent>
<p>{t("users.table.changeRole")}</p>
</TooltipContent>
</Tooltip>
)}
<Tooltip>
<TooltipTrigger asChild>