Compare commits

...

11 Commits

Author SHA1 Message Date
Josh Hawkins
9ec65d7aa9
Review stream tweaks (#20656)
* add blue dot instead of blue outline

* fix layout for portrait cameras

* fix light mode
2025-10-24 16:30:12 -06:00
Nicolas Mowen
83fa651ada
Various fixes (#20655) 2025-10-24 16:38:35 -05:00
Josh Hawkins
eb51eb3c9d
UI tweaks (#20649)
* match face wizard with camera and classification wizards

* remove review detail dialog and link chip to detail stream in history

* remove footer on explore images and move to overlay

* use consistent overlay button styles

* spacing tweak

* ensure selected ring stays on top of gradients

* fix z-index

* match object lifecycle with details
2025-10-24 11:08:59 -06:00
Josh Hawkins
49f5d595ea
Review stream tweaks (#20648)
* add detail stream selector to mobile drawer

* tweak getDurationFromTimestamps for i18n and abbreviations

* improve lifecycle description labeling

* i18n

* match figma

* fix progress line and add area and ratio tooltip

* allow clicking on chevron without triggering playback

* tweaks

* add key

* change wording

* clean up

* clean up

* remove check

* clean up
2025-10-24 07:50:06 -05:00
Josh Hawkins
e2da8aa04c
Camera wizard tweaks (#20643)
* dialog size tweaks

* move icon and make links in popvers clickable

* colors and clickable links
2025-10-23 13:58:22 -06:00
Nicolas Mowen
f5a57edcc9
Implement Wizard for Creating Classification Models (#20622)
* Implement extraction of images for classification state models

* Add object classification dataset preparation

* Add first step wizard

* Update i18n

* Add state classification image selection step

* Improve box handling

* Add object selector

* Improve object cropping implementation

* Fix state classification selection

* Finalize training and image selection step

* Cleanup

* Design optimizations

* Cleanup mobile styling

* Update no models screen

* Cleanups and fixes

* Fix bugs

* Improve model training and creation process

* Cleanup

* Dynamically add metrics for new model

* Add loading when hitting continue

* Improve image selection mechanism

* Remove unused translation keys

* Adjust wording

* Add retry button for image generation

* Make no models view more specific

* Adjust plus icon

* Adjust form label

* Start with correct type selected

* Cleanup sizing and more font colors

* Small tweaks

* Add tips and more info

* Cleanup dialog sizing

* Add cursor rule for frontend

* Cleanup

* remove underline

* Lazy loading
2025-10-23 13:27:28 -06:00
Nicolas Mowen
4df7793587
Set correct nginx container value (#20641) 2025-10-23 12:44:12 -06:00
Josh Hawkins
ac5de290ab
fix missing i18n key (#20639) 2025-10-23 12:40:06 -05:00
Nicolas Mowen
8c3c596dee
Fix ffmpeg command (#20637) 2025-10-23 11:51:16 -05:00
Nicolas Mowen
c5def83e08
Always use fmp4 for HLS (#20638) 2025-10-23 11:50:37 -05:00
Josh Hawkins
81df534784
Camera wizard improvements (#20636)
* use avg_frame_rate

* probe metadata and snapshot separately

* improve ffprobe error reporting

* show error messages in toaster
2025-10-23 08:34:52 -05:00
46 changed files with 3419 additions and 627 deletions

View File

@ -0,0 +1,6 @@
---
globs: ["**/*.ts", "**/*.tsx"]
alwaysApply: false
---
Never write strings in the frontend directly, always write to and reference the relevant translations file.

View File

@ -73,6 +73,8 @@ http {
vod_manifest_segment_durations_mode accurate;
vod_ignore_edit_list on;
vod_segment_duration 10000;
# MPEG-TS settings (not used when fMP4 is enabled, kept for reference)
vod_hls_mpegts_align_frames off;
vod_hls_mpegts_interleave_frames on;
@ -105,6 +107,10 @@ http {
aio threads;
vod hls;
# Use fMP4 (fragmented MP4) instead of MPEG-TS for better performance
# Smaller segments, faster generation, better browser compatibility
vod_hls_container_format fmp4;
secure_token $args;
secure_token_types application/vnd.apple.mpegurl;

View File

@ -12,7 +12,18 @@ Object classification models are lightweight and run very fast on CPU. Inference
Training the model does briefly use a high amount of system resources for about 13 minutes per training run. On lower-power devices, training may take longer.
When running the `-tensorrt` image, Nvidia GPUs will automatically be used to accelerate training.
### Sub label vs Attribute
## Classes
Classes are the categories your model will learn to distinguish between. Each class represents a distinct visual category that the model will predict.
For object classification:
- Define classes that represent different types or attributes of the detected object
- Examples: For `person` objects, classes might be `delivery_person`, `resident`, `stranger`
- Include a `none` class for objects that don't fit any specific category
- Keep classes visually distinct to improve accuracy
### Classification Type
- **Sub label**:

View File

@ -12,6 +12,17 @@ State classification models are lightweight and run very fast on CPU. Inference
Training the model does briefly use a high amount of system resources for about 13 minutes per training run. On lower-power devices, training may take longer.
When running the `-tensorrt` image, Nvidia GPUs will automatically be used to accelerate training.
## Classes
Classes are the different states an area on your camera can be in. Each class represents a distinct visual state that the model will learn to recognize.
For state classification:
- Define classes that represent mutually exclusive states
- Examples: `open` and `closed` for a garage door, `on` and `off` for lights
- Use at least 2 classes (typically binary states work best)
- Keep class names clear and descriptive
## Example use cases
- **Door state**: Detect if a garage or front door is open vs closed.

View File

@ -387,20 +387,28 @@ def config_set(request: Request, body: AppConfigSetBody):
old_config: FrigateConfig = request.app.frigate_config
request.app.frigate_config = config
if body.update_topic and body.update_topic.startswith("config/cameras/"):
_, _, camera, field = body.update_topic.split("/")
if body.update_topic:
if body.update_topic.startswith("config/cameras/"):
_, _, camera, field = body.update_topic.split("/")
if field == "add":
settings = config.cameras[camera]
elif field == "remove":
settings = old_config.cameras[camera]
if field == "add":
settings = config.cameras[camera]
elif field == "remove":
settings = old_config.cameras[camera]
else:
settings = config.get_nested_object(body.update_topic)
request.app.config_publisher.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum[field], camera),
settings,
)
else:
# Handle nested config updates (e.g., config/classification/custom/{name})
settings = config.get_nested_object(body.update_topic)
request.app.config_publisher.publish_update(
CameraConfigUpdateTopic(CameraConfigUpdateEnum[field], camera),
settings,
)
if settings:
request.app.config_publisher.publisher.publish(
body.update_topic, settings
)
return JSONResponse(
content=(

View File

@ -199,19 +199,30 @@ def ffprobe(request: Request, paths: str = "", detailed: bool = False):
request.app.frigate_config.ffmpeg, path.strip(), detailed=detailed
)
result = {
"return_code": ffprobe.returncode,
"stderr": (
ffprobe.stderr.decode("unicode_escape").strip()
if ffprobe.returncode != 0
else ""
),
"stdout": (
json.loads(ffprobe.stdout.decode("unicode_escape").strip())
if ffprobe.returncode == 0
else ""
),
}
if ffprobe.returncode != 0:
try:
stderr_decoded = ffprobe.stderr.decode("utf-8")
except UnicodeDecodeError:
try:
stderr_decoded = ffprobe.stderr.decode("unicode_escape")
except Exception:
stderr_decoded = str(ffprobe.stderr)
stderr_lines = [
line.strip() for line in stderr_decoded.split("\n") if line.strip()
]
result = {
"return_code": ffprobe.returncode,
"stderr": stderr_lines,
"stdout": "",
}
else:
result = {
"return_code": ffprobe.returncode,
"stderr": [],
"stdout": json.loads(ffprobe.stdout.decode("unicode_escape").strip()),
}
# Add detailed metadata if requested and probe was successful
if detailed and ffprobe.returncode == 0 and result["stdout"]:

View File

@ -3,7 +3,9 @@
import datetime
import logging
import os
import random
import shutil
import string
from typing import Any
import cv2
@ -17,6 +19,8 @@ from frigate.api.auth import require_role
from frigate.api.defs.request.classification_body import (
AudioTranscriptionBody,
DeleteFaceImagesBody,
GenerateObjectExamplesBody,
GenerateStateExamplesBody,
RenameFaceBody,
)
from frigate.api.defs.response.classification_response import (
@ -30,6 +34,10 @@ from frigate.config.camera import DetectConfig
from frigate.const import CLIPS_DIR, FACE_DIR
from frigate.embeddings import EmbeddingsContext
from frigate.models import Event
from frigate.util.classification import (
collect_object_classification_examples,
collect_state_classification_examples,
)
from frigate.util.path import get_event_snapshot
logger = logging.getLogger(__name__)
@ -159,8 +167,7 @@ def train_face(request: Request, name: str, body: dict = None):
new_name = f"{sanitized_name}-{datetime.datetime.now().timestamp()}.webp"
new_file_folder = os.path.join(FACE_DIR, f"{sanitized_name}")
if not os.path.exists(new_file_folder):
os.mkdir(new_file_folder)
os.makedirs(new_file_folder, exist_ok=True)
if training_file_name:
shutil.move(training_file, os.path.join(new_file_folder, new_name))
@ -701,13 +708,14 @@ def categorize_classification_image(request: Request, name: str, body: dict = No
status_code=404,
)
new_name = f"{category}-{datetime.datetime.now().timestamp()}.png"
random_id = "".join(random.choices(string.ascii_lowercase + string.digits, k=6))
timestamp = datetime.datetime.now().timestamp()
new_name = f"{category}-{timestamp}-{random_id}.png"
new_file_folder = os.path.join(
CLIPS_DIR, sanitize_filename(name), "dataset", category
)
if not os.path.exists(new_file_folder):
os.mkdir(new_file_folder)
os.makedirs(new_file_folder, exist_ok=True)
# use opencv because webp images can not be used to train
img = cv2.imread(training_file)
@ -756,3 +764,43 @@ def delete_classification_train_images(request: Request, name: str, body: dict =
content=({"success": True, "message": "Successfully deleted faces."}),
status_code=200,
)
@router.post(
"/classification/generate_examples/state",
response_model=GenericResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Generate state classification examples",
)
async def generate_state_examples(request: Request, body: GenerateStateExamplesBody):
"""Generate examples for state classification."""
model_name = sanitize_filename(body.model_name)
cameras_normalized = {
camera_name: tuple(crop)
for camera_name, crop in body.cameras.items()
if camera_name in request.app.frigate_config.cameras
}
collect_state_classification_examples(model_name, cameras_normalized)
return JSONResponse(
content={"success": True, "message": "Example generation completed"},
status_code=200,
)
@router.post(
"/classification/generate_examples/object",
response_model=GenericResponse,
dependencies=[Depends(require_role(["admin"]))],
summary="Generate object classification examples",
)
async def generate_object_examples(request: Request, body: GenerateObjectExamplesBody):
"""Generate examples for object classification."""
model_name = sanitize_filename(body.model_name)
collect_object_classification_examples(model_name, body.label)
return JSONResponse(
content={"success": True, "message": "Example generation completed"},
status_code=200,
)

View File

@ -1,17 +1,31 @@
from typing import List
from typing import Dict, List, Tuple
from pydantic import BaseModel, Field
class RenameFaceBody(BaseModel):
new_name: str
new_name: str = Field(description="New name for the face")
class AudioTranscriptionBody(BaseModel):
event_id: str
event_id: str = Field(description="ID of the event to transcribe audio for")
class DeleteFaceImagesBody(BaseModel):
ids: List[str] = Field(
description="List of image filenames to delete from the face folder"
)
class GenerateStateExamplesBody(BaseModel):
model_name: str = Field(description="Name of the classification model")
cameras: Dict[str, Tuple[float, float, float, float]] = Field(
description="Dictionary mapping camera names to normalized crop coordinates in [x1, y1, x2, y2] format (values 0-1)"
)
class GenerateObjectExamplesBody(BaseModel):
model_name: str = Field(description="Name of the classification model")
label: str = Field(
description="Object label to collect examples for (e.g., 'person', 'car')"
)

View File

@ -53,9 +53,17 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
self.tensor_output_details: dict[str, Any] | None = None
self.labelmap: dict[int, str] = {}
self.classifications_per_second = EventsPerSecond()
self.inference_speed = InferenceSpeed(
self.metrics.classification_speeds[self.model_config.name]
)
if (
self.metrics
and self.model_config.name in self.metrics.classification_speeds
):
self.inference_speed = InferenceSpeed(
self.metrics.classification_speeds[self.model_config.name]
)
else:
self.inference_speed = None
self.last_run = datetime.datetime.now().timestamp()
self.__build_detector()
@ -83,12 +91,14 @@ class CustomStateClassificationProcessor(RealTimeProcessorApi):
def __update_metrics(self, duration: float) -> None:
self.classifications_per_second.update()
self.inference_speed.update(duration)
if self.inference_speed:
self.inference_speed.update(duration)
def process_frame(self, frame_data: dict[str, Any], frame: np.ndarray):
self.metrics.classification_cps[
self.model_config.name
].value = self.classifications_per_second.eps()
if self.metrics and self.model_config.name in self.metrics.classification_cps:
self.metrics.classification_cps[
self.model_config.name
].value = self.classifications_per_second.eps()
camera = frame_data.get("camera")
if camera not in self.model_config.state_config.cameras:
@ -223,9 +233,17 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
self.detected_objects: dict[str, float] = {}
self.labelmap: dict[int, str] = {}
self.classifications_per_second = EventsPerSecond()
self.inference_speed = InferenceSpeed(
self.metrics.classification_speeds[self.model_config.name]
)
if (
self.metrics
and self.model_config.name in self.metrics.classification_speeds
):
self.inference_speed = InferenceSpeed(
self.metrics.classification_speeds[self.model_config.name]
)
else:
self.inference_speed = None
self.__build_detector()
@redirect_output_to_logger(logger, logging.DEBUG)
@ -251,12 +269,14 @@ class CustomObjectClassificationProcessor(RealTimeProcessorApi):
def __update_metrics(self, duration: float) -> None:
self.classifications_per_second.update()
self.inference_speed.update(duration)
if self.inference_speed:
self.inference_speed.update(duration)
def process_frame(self, obj_data, frame):
self.metrics.classification_cps[
self.model_config.name
].value = self.classifications_per_second.eps()
if self.metrics and self.model_config.name in self.metrics.classification_cps:
self.metrics.classification_cps[
self.model_config.name
].value = self.classifications_per_second.eps()
if obj_data["false_positive"]:
return

View File

@ -9,6 +9,7 @@ from typing import Any
from peewee import DoesNotExist
from frigate.comms.config_updater import ConfigSubscriber
from frigate.comms.detections_updater import DetectionSubscriber, DetectionTypeEnum
from frigate.comms.embeddings_updater import (
EmbeddingsRequestEnum,
@ -95,6 +96,9 @@ class EmbeddingMaintainer(threading.Thread):
CameraConfigUpdateEnum.semantic_search,
],
)
self.classification_config_subscriber = ConfigSubscriber(
"config/classification/custom/"
)
# Configure Frigate DB
db = SqliteVecQueueDatabase(
@ -255,6 +259,7 @@ class EmbeddingMaintainer(threading.Thread):
"""Maintain a SQLite-vec database for semantic search."""
while not self.stop_event.is_set():
self.config_updater.check_for_updates()
self._check_classification_config_updates()
self._process_requests()
self._process_updates()
self._process_recordings_updates()
@ -265,6 +270,7 @@ class EmbeddingMaintainer(threading.Thread):
self._process_event_metadata()
self.config_updater.stop()
self.classification_config_subscriber.stop()
self.event_subscriber.stop()
self.event_end_subscriber.stop()
self.recordings_subscriber.stop()
@ -275,6 +281,46 @@ class EmbeddingMaintainer(threading.Thread):
self.requestor.stop()
logger.info("Exiting embeddings maintenance...")
def _check_classification_config_updates(self) -> None:
"""Check for classification config updates and add new processors."""
topic, model_config = self.classification_config_subscriber.check_for_update()
if topic and model_config:
model_name = topic.split("/")[-1]
self.config.classification.custom[model_name] = model_config
# Check if processor already exists
for processor in self.realtime_processors:
if isinstance(
processor,
(
CustomStateClassificationProcessor,
CustomObjectClassificationProcessor,
),
):
if processor.model_config.name == model_name:
logger.debug(
f"Classification processor for model {model_name} already exists, skipping"
)
return
if model_config.state_config is not None:
processor = CustomStateClassificationProcessor(
self.config, model_config, self.requestor, self.metrics
)
else:
processor = CustomObjectClassificationProcessor(
self.config,
model_config,
self.event_metadata_publisher,
self.metrics,
)
self.realtime_processors.append(processor)
logger.info(
f"Added classification processor for model: {model_name} (type: {type(processor).__name__})"
)
def _process_requests(self) -> None:
"""Process embeddings requests"""

View File

@ -150,10 +150,10 @@ PRESETS_HW_ACCEL_SCALE["preset-rk-h265"] = PRESETS_HW_ACCEL_SCALE[FFMPEG_HWACCEL
PRESETS_HW_ACCEL_ENCODE_BIRDSEYE = {
"preset-rpi-64-h264": "{0} -hide_banner {1} -c:v h264_v4l2m2m {2}",
"preset-rpi-64-h265": "{0} -hide_banner {1} -c:v hevc_v4l2m2m {2}",
FFMPEG_HWACCEL_VAAPI: "{0} -hide_banner -hwaccel vaapi -hwaccel_output_format vaapi {3} {1} -c:v h264_vaapi -g 50 -bf 0 -profile:v high -level:v 4.1 -sei:v 0 -an -vf format=vaapi|nv12,hwupload {2}",
FFMPEG_HWACCEL_VAAPI: "{0} -hide_banner -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device {3} {1} -c:v h264_vaapi -g 50 -bf 0 -profile:v high -level:v 4.1 -sei:v 0 -an -vf format=vaapi|nv12,hwupload {2}",
"preset-intel-qsv-h264": "{0} -hide_banner {1} -c:v h264_qsv -g 50 -bf 0 -profile:v high -level:v 4.1 -async_depth:v 1 {2}",
"preset-intel-qsv-h265": "{0} -hide_banner {1} -c:v h264_qsv -g 50 -bf 0 -profile:v main -level:v 4.1 -async_depth:v 1 {2}",
FFMPEG_HWACCEL_NVIDIA: "{0} -hide_banner {1} {3} -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll {2}",
FFMPEG_HWACCEL_NVIDIA: "{0} -hide_banner {1} -hwaccel device {3} -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll {2}",
"preset-jetson-h264": "{0} -hide_banner {1} -c:v h264_nvmpi -profile high {2}",
"preset-jetson-h265": "{0} -hide_banner {1} -c:v h264_nvmpi -profile main {2}",
FFMPEG_HWACCEL_RKMPP: "{0} -hide_banner {1} -c:v h264_rkmpp -profile:v high {2}",

View File

@ -20,8 +20,8 @@ class OllamaClient(GenAIClient):
LOCAL_OPTIMIZED_OPTIONS = {
"options": {
"temperature": 0.5,
"repeat_penalty": 1.15,
"presence_penalty": 0.1,
"repeat_penalty": 1.05,
"presence_penalty": 0.3,
},
}

View File

@ -2,12 +2,15 @@
import logging
import os
import random
from collections import defaultdict
import cv2
import numpy as np
from frigate.comms.embeddings_updater import EmbeddingsRequestEnum, EmbeddingsRequestor
from frigate.comms.inter_process import InterProcessRequestor
from frigate.config import FfmpegConfig
from frigate.const import (
CLIPS_DIR,
MODEL_CACHE_DIR,
@ -15,7 +18,10 @@ from frigate.const import (
UPDATE_MODEL_STATE,
)
from frigate.log import redirect_output_to_logger
from frigate.models import Event, Recordings, ReviewSegment
from frigate.types import ModelStatusTypesEnum
from frigate.util.image import get_image_from_recording
from frigate.util.path import get_event_thumbnail_bytes
from frigate.util.process import FrigateProcess
BATCH_SIZE = 16
@ -69,6 +75,7 @@ class ClassificationTrainingProcess(FrigateProcess):
logger.info(f"Kicking off classification training for {self.model_name}.")
dataset_dir = os.path.join(CLIPS_DIR, self.model_name, "dataset")
model_dir = os.path.join(MODEL_CACHE_DIR, self.model_name)
os.makedirs(model_dir, exist_ok=True)
num_classes = len(
[
d
@ -139,7 +146,6 @@ class ClassificationTrainingProcess(FrigateProcess):
f.write(tflite_model)
@staticmethod
def kickoff_model_training(
embeddingRequestor: EmbeddingsRequestor, model_name: str
) -> None:
@ -172,3 +178,520 @@ def kickoff_model_training(
},
)
requestor.stop()
@staticmethod
def collect_state_classification_examples(
model_name: str, cameras: dict[str, tuple[float, float, float, float]]
) -> None:
"""
Collect representative state classification examples from review items.
This function:
1. Queries review items from specified cameras
2. Selects 100 balanced timestamps across the data
3. Extracts keyframes from recordings (cropped to specified regions)
4. Selects 20 most visually distinct images
5. Saves them to the dataset directory
Args:
model_name: Name of the classification model
cameras: Dict mapping camera names to normalized crop coordinates [x1, y1, x2, y2] (0-1)
"""
dataset_dir = os.path.join(CLIPS_DIR, model_name, "dataset")
temp_dir = os.path.join(dataset_dir, "temp")
os.makedirs(temp_dir, exist_ok=True)
# Step 1: Get review items for the cameras
camera_names = list(cameras.keys())
review_items = list(
ReviewSegment.select()
.where(ReviewSegment.camera.in_(camera_names))
.where(ReviewSegment.end_time.is_null(False))
.order_by(ReviewSegment.start_time.asc())
)
if not review_items:
logger.warning(f"No review items found for cameras: {camera_names}")
return
# Step 2: Create balanced timestamp selection (100 samples)
timestamps = _select_balanced_timestamps(review_items, target_count=100)
# Step 3: Extract keyframes from recordings with crops applied
keyframes = _extract_keyframes(
"/usr/lib/ffmpeg/7.0/bin/ffmpeg", timestamps, temp_dir, cameras
)
# Step 4: Select 24 most visually distinct images (they're already cropped)
distinct_images = _select_distinct_images(keyframes, target_count=24)
# Step 5: Save to train directory for later classification
train_dir = os.path.join(CLIPS_DIR, model_name, "train")
os.makedirs(train_dir, exist_ok=True)
saved_count = 0
for idx, image_path in enumerate(distinct_images):
dest_path = os.path.join(train_dir, f"example_{idx:03d}.jpg")
try:
img = cv2.imread(image_path)
if img is not None:
cv2.imwrite(dest_path, img)
saved_count += 1
except Exception as e:
logger.error(f"Failed to save image {image_path}: {e}")
import shutil
try:
shutil.rmtree(temp_dir)
except Exception as e:
logger.warning(f"Failed to clean up temp directory: {e}")
def _select_balanced_timestamps(
review_items: list[ReviewSegment], target_count: int = 100
) -> list[dict]:
"""
Select balanced timestamps from review items.
Strategy:
- Group review items by camera and time of day
- Sample evenly across groups to ensure diversity
- For each selected review item, pick a random timestamp within its duration
Returns:
List of dicts with keys: camera, timestamp, review_item
"""
# Group by camera and hour of day for temporal diversity
grouped = defaultdict(list)
for item in review_items:
camera = item.camera
# Group by 6-hour blocks for temporal diversity
hour_block = int(item.start_time // (6 * 3600))
key = f"{camera}_{hour_block}"
grouped[key].append(item)
# Calculate how many samples per group
num_groups = len(grouped)
if num_groups == 0:
return []
samples_per_group = max(1, target_count // num_groups)
timestamps = []
# Sample from each group
for group_items in grouped.values():
# Take samples_per_group items from this group
sample_size = min(samples_per_group, len(group_items))
sampled_items = random.sample(group_items, sample_size)
for item in sampled_items:
# Pick a random timestamp within the review item's duration
duration = item.end_time - item.start_time
if duration <= 0:
continue
# Sample from middle 80% to avoid edge artifacts
offset = random.uniform(duration * 0.1, duration * 0.9)
timestamp = item.start_time + offset
timestamps.append(
{
"camera": item.camera,
"timestamp": timestamp,
"review_item": item,
}
)
# If we don't have enough, sample more from larger groups
while len(timestamps) < target_count and len(timestamps) < len(review_items):
for group_items in grouped.values():
if len(timestamps) >= target_count:
break
# Pick a random item not already sampled
item = random.choice(group_items)
duration = item.end_time - item.start_time
if duration <= 0:
continue
offset = random.uniform(duration * 0.1, duration * 0.9)
timestamp = item.start_time + offset
# Check if we already have a timestamp near this one
if not any(abs(t["timestamp"] - timestamp) < 1.0 for t in timestamps):
timestamps.append(
{
"camera": item.camera,
"timestamp": timestamp,
"review_item": item,
}
)
return timestamps[:target_count]
def _extract_keyframes(
ffmpeg_path: str,
timestamps: list[dict],
output_dir: str,
camera_crops: dict[str, tuple[float, float, float, float]],
) -> list[str]:
"""
Extract keyframes from recordings at specified timestamps and crop to specified regions.
Args:
ffmpeg_path: Path to ffmpeg binary
timestamps: List of timestamp dicts from _select_balanced_timestamps
output_dir: Directory to save extracted frames
camera_crops: Dict mapping camera names to normalized crop coordinates [x1, y1, x2, y2] (0-1)
Returns:
List of paths to successfully extracted and cropped keyframe images
"""
keyframe_paths = []
for idx, ts_info in enumerate(timestamps):
camera = ts_info["camera"]
timestamp = ts_info["timestamp"]
if camera not in camera_crops:
logger.warning(f"No crop coordinates for camera {camera}")
continue
norm_x1, norm_y1, norm_x2, norm_y2 = camera_crops[camera]
try:
recording = (
Recordings.select()
.where(
(timestamp >= Recordings.start_time)
& (timestamp <= Recordings.end_time)
& (Recordings.camera == camera)
)
.order_by(Recordings.start_time.desc())
.limit(1)
.get()
)
except Exception:
continue
relative_time = timestamp - recording.start_time
try:
config = FfmpegConfig(path="/usr/lib/ffmpeg/7.0")
image_data = get_image_from_recording(
config,
recording.path,
relative_time,
codec="mjpeg",
height=None,
)
if image_data:
nparr = np.frombuffer(image_data, np.uint8)
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
if img is not None:
height, width = img.shape[:2]
x1 = int(norm_x1 * width)
y1 = int(norm_y1 * height)
x2 = int(norm_x2 * width)
y2 = int(norm_y2 * height)
x1_clipped = max(0, min(x1, width))
y1_clipped = max(0, min(y1, height))
x2_clipped = max(0, min(x2, width))
y2_clipped = max(0, min(y2, height))
if x2_clipped > x1_clipped and y2_clipped > y1_clipped:
cropped = img[y1_clipped:y2_clipped, x1_clipped:x2_clipped]
resized = cv2.resize(cropped, (224, 224))
output_path = os.path.join(output_dir, f"frame_{idx:04d}.jpg")
cv2.imwrite(output_path, resized)
keyframe_paths.append(output_path)
except Exception as e:
logger.debug(
f"Failed to extract frame from {recording.path} at {relative_time}s: {e}"
)
continue
return keyframe_paths
def _select_distinct_images(
image_paths: list[str], target_count: int = 20
) -> list[str]:
"""
Select the most visually distinct images from a set of keyframes.
Uses a greedy algorithm based on image histograms:
1. Start with a random image
2. Iteratively add the image that is most different from already selected images
3. Difference is measured using histogram comparison
Args:
image_paths: List of paths to candidate images
target_count: Number of distinct images to select
Returns:
List of paths to selected images
"""
if len(image_paths) <= target_count:
return image_paths
histograms = {}
valid_paths = []
for path in image_paths:
try:
img = cv2.imread(path)
if img is None:
continue
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
hist = cv2.calcHist(
[hsv], [0, 1, 2], None, [8, 8, 8], [0, 180, 0, 256, 0, 256]
)
hist = cv2.normalize(hist, hist).flatten()
histograms[path] = hist
valid_paths.append(path)
except Exception as e:
logger.debug(f"Failed to process image {path}: {e}")
continue
if len(valid_paths) <= target_count:
return valid_paths
selected = []
first_image = random.choice(valid_paths)
selected.append(first_image)
remaining = [p for p in valid_paths if p != first_image]
while len(selected) < target_count and remaining:
max_min_distance = -1
best_candidate = None
for candidate in remaining:
min_distance = float("inf")
for selected_img in selected:
distance = cv2.compareHist(
histograms[candidate],
histograms[selected_img],
cv2.HISTCMP_BHATTACHARYYA,
)
min_distance = min(min_distance, distance)
if min_distance > max_min_distance:
max_min_distance = min_distance
best_candidate = candidate
if best_candidate:
selected.append(best_candidate)
remaining.remove(best_candidate)
else:
break
return selected
@staticmethod
def collect_object_classification_examples(
model_name: str,
label: str,
) -> None:
"""
Collect representative object classification examples from event thumbnails.
This function:
1. Queries events for the specified label
2. Selects 100 balanced events across different cameras and times
3. Retrieves thumbnails for selected events (with 33% center crop applied)
4. Selects 24 most visually distinct thumbnails
5. Saves to dataset directory
Args:
model_name: Name of the classification model
label: Object label to collect (e.g., "person", "car")
cameras: List of camera names to collect examples from
"""
dataset_dir = os.path.join(CLIPS_DIR, model_name, "dataset")
temp_dir = os.path.join(dataset_dir, "temp")
os.makedirs(temp_dir, exist_ok=True)
# Step 1: Query events for the specified label and cameras
events = list(
Event.select().where((Event.label == label)).order_by(Event.start_time.asc())
)
if not events:
logger.warning(f"No events found for label '{label}'")
return
logger.debug(f"Found {len(events)} events")
# Step 2: Select balanced events (100 samples)
selected_events = _select_balanced_events(events, target_count=100)
logger.debug(f"Selected {len(selected_events)} events")
# Step 3: Extract thumbnails from events
thumbnails = _extract_event_thumbnails(selected_events, temp_dir)
logger.debug(f"Successfully extracted {len(thumbnails)} thumbnails")
# Step 4: Select 24 most visually distinct thumbnails
distinct_images = _select_distinct_images(thumbnails, target_count=24)
logger.debug(f"Selected {len(distinct_images)} distinct images")
# Step 5: Save to train directory for later classification
train_dir = os.path.join(CLIPS_DIR, model_name, "train")
os.makedirs(train_dir, exist_ok=True)
saved_count = 0
for idx, image_path in enumerate(distinct_images):
dest_path = os.path.join(train_dir, f"example_{idx:03d}.jpg")
try:
img = cv2.imread(image_path)
if img is not None:
cv2.imwrite(dest_path, img)
saved_count += 1
except Exception as e:
logger.error(f"Failed to save image {image_path}: {e}")
import shutil
try:
shutil.rmtree(temp_dir)
except Exception as e:
logger.warning(f"Failed to clean up temp directory: {e}")
logger.debug(
f"Successfully collected {saved_count} classification examples in {train_dir}"
)
def _select_balanced_events(
events: list[Event], target_count: int = 100
) -> list[Event]:
"""
Select balanced events from the event list.
Strategy:
- Group events by camera and time of day
- Sample evenly across groups to ensure diversity
- Prioritize events with higher scores
Returns:
List of selected events
"""
grouped = defaultdict(list)
for event in events:
camera = event.camera
hour_block = int(event.start_time // (6 * 3600))
key = f"{camera}_{hour_block}"
grouped[key].append(event)
num_groups = len(grouped)
if num_groups == 0:
return []
samples_per_group = max(1, target_count // num_groups)
selected = []
for group_events in grouped.values():
sorted_events = sorted(
group_events,
key=lambda e: e.data.get("score", 0) if e.data else 0,
reverse=True,
)
sample_size = min(samples_per_group, len(sorted_events))
selected.extend(sorted_events[:sample_size])
if len(selected) < target_count:
remaining = [e for e in events if e not in selected]
remaining_sorted = sorted(
remaining,
key=lambda e: e.data.get("score", 0) if e.data else 0,
reverse=True,
)
needed = target_count - len(selected)
selected.extend(remaining_sorted[:needed])
return selected[:target_count]
def _extract_event_thumbnails(events: list[Event], output_dir: str) -> list[str]:
"""
Extract thumbnails from events and save to disk.
Args:
events: List of Event objects
output_dir: Directory to save thumbnails
Returns:
List of paths to successfully extracted thumbnail images
"""
thumbnail_paths = []
for idx, event in enumerate(events):
try:
thumbnail_bytes = get_event_thumbnail_bytes(event)
if thumbnail_bytes:
nparr = np.frombuffer(thumbnail_bytes, np.uint8)
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
if img is not None:
height, width = img.shape[:2]
crop_size = 1.0
if event.data and "box" in event.data and "region" in event.data:
box = event.data["box"]
region = event.data["region"]
if len(box) == 4 and len(region) == 4:
box_w, box_h = box[2], box[3]
region_w, region_h = region[2], region[3]
box_area = (box_w * box_h) / (region_w * region_h)
if box_area < 0.05:
crop_size = 0.4
elif box_area < 0.10:
crop_size = 0.5
elif box_area < 0.20:
crop_size = 0.65
elif box_area < 0.35:
crop_size = 0.80
else:
crop_size = 0.95
crop_width = int(width * crop_size)
crop_height = int(height * crop_size)
x1 = (width - crop_width) // 2
y1 = (height - crop_height) // 2
x2 = x1 + crop_width
y2 = y1 + crop_height
cropped = img[y1:y2, x1:x2]
resized = cv2.resize(cropped, (224, 224))
output_path = os.path.join(output_dir, f"thumbnail_{idx:04d}.jpg")
cv2.imwrite(output_path, resized)
thumbnail_paths.append(output_path)
except Exception as e:
logger.debug(f"Failed to extract thumbnail for event {event.id}: {e}")
continue
return thumbnail_paths

View File

@ -577,7 +577,7 @@ def ffprobe_stream(ffmpeg, path: str, detailed: bool = False) -> sp.CompletedPro
if detailed and format_entries:
ffprobe_cmd.extend(["-show_entries", f"format={format_entries}"])
ffprobe_cmd.extend(["-loglevel", "quiet", clean_path])
ffprobe_cmd.extend(["-loglevel", "error", clean_path])
return sp.run(ffprobe_cmd, capture_output=True)

View File

@ -1,4 +1,5 @@
{
"documentTitle": "Classification Models",
"button": {
"deleteClassificationAttempts": "Delete Classification Images",
"renameCategory": "Rename Class",
@ -50,8 +51,85 @@
},
"categorizeImageAs": "Classify Image As:",
"categorizeImage": "Classify Image",
"noModels": {
"object": {
"title": "No Object Classification Models",
"description": "Create a custom model to classify detected objects.",
"buttonText": "Create Object Model"
},
"state": {
"title": "No State Classification Models",
"description": "Create a custom model to monitor and classify state changes in specific camera areas.",
"buttonText": "Create State Model"
}
},
"wizard": {
"title": "Create New Classification",
"description": "Create a new state or object classification model."
"steps": {
"nameAndDefine": "Name & Define",
"stateArea": "State Area",
"chooseExamples": "Choose Examples"
},
"step1": {
"description": "State models monitor fixed camera areas for changes (e.g., door open/closed). Object models add classifications to detected objects (e.g., known animals, delivery persons, etc.).",
"name": "Name",
"namePlaceholder": "Enter model name...",
"type": "Type",
"typeState": "State",
"typeObject": "Object",
"objectLabel": "Object Label",
"objectLabelPlaceholder": "Select object type...",
"classificationType": "Classification Type",
"classificationTypeTip": "Learn about classification types",
"classificationTypeDesc": "Sub Labels add additional text to the object label (e.g., 'Person: UPS'). Attributes are searchable metadata stored separately in the object metadata.",
"classificationSubLabel": "Sub Label",
"classificationAttribute": "Attribute",
"classes": "Classes",
"classesTip": "Learn about classes",
"classesStateDesc": "Define the different states your camera area can be in. For example: 'open' and 'closed' for a garage door.",
"classesObjectDesc": "Define the different categories to classify detected objects into. For example: 'delivery_person', 'resident', 'stranger' for person classification.",
"classPlaceholder": "Enter class name...",
"errors": {
"nameRequired": "Model name is required",
"nameLength": "Model name must be 64 characters or less",
"nameOnlyNumbers": "Model name cannot contain only numbers",
"classRequired": "At least 1 class is required",
"classesUnique": "Class names must be unique",
"stateRequiresTwoClasses": "State models require at least 2 classes",
"objectLabelRequired": "Please select an object label",
"objectTypeRequired": "Please select a classification type"
}
},
"step2": {
"description": "Select cameras and define the area to monitor for each camera. The model will classify the state of these areas.",
"cameras": "Cameras",
"selectCamera": "Select Camera",
"noCameras": "Click + to add cameras",
"selectCameraPrompt": "Select a camera from the list to define its monitoring area"
},
"step3": {
"selectImagesPrompt": "Select all images with: {{className}}",
"selectImagesDescription": "Click on images to select them. Click Continue when you're done with this class.",
"generating": {
"title": "Generating Sample Images",
"description": "Frigate is pulling representative images from your recordings. This may take a moment..."
},
"training": {
"title": "Training Model",
"description": "Your model is being trained in the background. Close this dialog, and your model will start running as soon as training is complete."
},
"retryGenerate": "Retry Generation",
"noImages": "No sample images generated",
"classifying": "Classifying & Training...",
"trainingStarted": "Training started successfully",
"errors": {
"noCameras": "No cameras configured",
"noObjectLabel": "No object label selected",
"generateFailed": "Failed to generate examples: {{error}}",
"generationFailed": "Generation failed. Please try again.",
"classifyFailed": "Failed to classify images: {{error}}"
},
"generateSuccess": "Successfully generated sample images"
}
}
}

View File

@ -19,10 +19,11 @@
"noFoundForTimePeriod": "No events found for this time period."
},
"detail": {
"label": "Detail",
"noDataFound": "No detail data to review",
"aria": "Toggle detail view",
"trackedObject_one": "tracked object",
"trackedObject_other": "tracked objects",
"trackedObject_one": "object",
"trackedObject_other": "objects",
"noObjectDetailData": "No object detail data available."
},
"objectTrack": {

View File

@ -194,6 +194,12 @@
},
"deleteTrackedObject": {
"label": "Delete this tracked object"
},
"showObjectDetails": {
"label": "Show object path"
},
"hideObjectDetails": {
"label": "Hide object path"
}
},
"dialog": {

View File

@ -1,14 +1,10 @@
{
"description": {
"addFace": "Walk through adding a new collection to the Face Library.",
"addFace": "Add a new collection to the Face Library by uploading your first image.",
"placeholder": "Enter a name for this collection",
"invalidName": "Invalid name. Names can only include letters, numbers, spaces, apostrophes, underscores, and hyphens."
},
"details": {
"subLabelScore": "Sub Label Score",
"scoreInfo": "The sub label score is the weighted score for all of the recognized face confidences, so this may differ from the score shown on the snapshot.",
"face": "Face Details",
"faceDesc": "Details of the tracked object that generated this face",
"timestamp": "Timestamp",
"unknown": "Unknown"
},
@ -19,8 +15,6 @@
},
"collections": "Collections",
"createFaceLibrary": {
"title": "Create Collection",
"desc": "Create a new collection",
"new": "Create New Face",
"nextSteps": "To build a strong foundation:<li>Use the Recent Recognitions tab to select and train on images for each detected person.</li><li>Focus on straight-on images for best results; avoid training images that capture faces at an angle.</li></ul>"
},
@ -37,8 +31,6 @@
"aria": "Select recent recognitions",
"empty": "There are no recent face recognition attempts"
},
"selectItem": "Select {{item}}",
"selectFace": "Select Face",
"deleteFaceLibrary": {
"title": "Delete Name",
"desc": "Are you sure you want to delete the collection {{name}}? This will permanently delete all associated faces."
@ -69,7 +61,6 @@
"maxSize": "Max size: {{size}}MB"
},
"nofaces": "No faces available",
"pixels": "{{area}}px",
"trainFaceAs": "Train Face as:",
"trainFace": "Train Face",
"toast": {

View File

@ -188,6 +188,10 @@
"testSuccess": "Connection test successful!",
"testFailed": "Connection test failed. Please check your input and try again.",
"streamDetails": "Stream Details",
"testing": {
"probingMetadata": "Probing camera metadata...",
"fetchingSnapshot": "Fetching camera snapshot..."
},
"warnings": {
"noSnapshot": "Unable to fetch a snapshot from the configured stream."
},
@ -197,8 +201,9 @@
"nameLength": "Camera name must be 64 characters or less",
"invalidCharacters": "Camera name contains invalid characters",
"nameExists": "Camera name already exists",
"customUrlRtspRequired": "Custom URLs must begin with \"rtsp://\". Manual configuration is required for non-RTSP camera streams.",
"brands": {
"reolink-rtsp": "Reolink RTSP is not recommended. It is recommended to enable http in the camera settings and restart the camera wizard."
"reolink-rtsp": "Reolink RTSP is not recommended. Enable HTTP in the camera's firmware settings and restart the wizard."
}
},
"docs": {

View File

@ -126,6 +126,7 @@ export const ClassificationCard = forwardRef<
imgClassName,
isMobile && "w-full",
)}
loading="lazy"
onLoad={() => setImageLoaded(true)}
src={`${baseUrl}${data.filepath}`}
/>
@ -213,7 +214,9 @@ export function GroupedClassificationCard({
});
if (!best) {
return group.at(-1);
// select an item from the middle of the time series as this usually correlates
// to a more representative image than the first or last
return group.at(Math.floor(group.length / 2));
}
const bestTyped: ClassificationItemData = best;
@ -304,7 +307,7 @@ export function GroupedClassificationCard({
<div>
<ContentTitle
className={cn(
"flex items-center gap-1 font-normal capitalize",
"flex items-center gap-2 font-normal capitalize",
isMobile && "px-2",
)}
>

View File

@ -42,11 +42,11 @@ export default function SearchThumbnailFooter({
return (
<div
className={cn(
"flex w-full flex-row items-center justify-between gap-2",
"flex w-full flex-row items-center justify-between gap-2 text-white",
columns > 4 && "items-start sm:flex-col lg:flex-row lg:items-center",
)}
>
<div className="flex flex-col items-start text-xs text-primary-variant">
<div className="flex flex-col items-start text-xs text-white/90 drop-shadow-lg">
{searchResult.end_time ? (
<TimeAgo time={searchResult.start_time * 1000} dense />
) : (

View File

@ -7,58 +7,198 @@ import {
DialogHeader,
DialogTitle,
} from "../ui/dialog";
import { useState } from "react";
import { useReducer, useMemo } from "react";
import Step1NameAndDefine, { Step1FormData } from "./wizard/Step1NameAndDefine";
import Step2StateArea, { Step2FormData } from "./wizard/Step2StateArea";
import Step3ChooseExamples, {
Step3FormData,
} from "./wizard/Step3ChooseExamples";
import { cn } from "@/lib/utils";
import { isDesktop } from "react-device-detect";
const STEPS = [
"classificationWizard.steps.nameAndDefine",
"classificationWizard.steps.stateArea",
"classificationWizard.steps.chooseExamples",
"classificationWizard.steps.train",
const OBJECT_STEPS = [
"wizard.steps.nameAndDefine",
"wizard.steps.chooseExamples",
];
const STATE_STEPS = [
"wizard.steps.nameAndDefine",
"wizard.steps.stateArea",
"wizard.steps.chooseExamples",
];
type ClassificationModelWizardDialogProps = {
open: boolean;
onClose: () => void;
defaultModelType?: "state" | "object";
};
type WizardState = {
currentStep: number;
step1Data?: Step1FormData;
step2Data?: Step2FormData;
step3Data?: Step3FormData;
};
type WizardAction =
| { type: "NEXT_STEP"; payload?: Partial<WizardState> }
| { type: "PREVIOUS_STEP" }
| { type: "SET_STEP_1"; payload: Step1FormData }
| { type: "SET_STEP_2"; payload: Step2FormData }
| { type: "SET_STEP_3"; payload: Step3FormData }
| { type: "RESET" };
const initialState: WizardState = {
currentStep: 0,
};
function wizardReducer(state: WizardState, action: WizardAction): WizardState {
switch (action.type) {
case "SET_STEP_1":
return {
...state,
step1Data: action.payload,
currentStep: 1,
};
case "SET_STEP_2":
return {
...state,
step2Data: action.payload,
currentStep: 2,
};
case "SET_STEP_3":
return {
...state,
step3Data: action.payload,
currentStep: 3,
};
case "NEXT_STEP":
return {
...state,
...action.payload,
currentStep: state.currentStep + 1,
};
case "PREVIOUS_STEP":
return {
...state,
currentStep: Math.max(0, state.currentStep - 1),
};
case "RESET":
return initialState;
default:
return state;
}
}
export default function ClassificationModelWizardDialog({
open,
onClose,
defaultModelType,
}: ClassificationModelWizardDialogProps) {
const { t } = useTranslation(["views/classificationModel"]);
// step management
const [currentStep, _] = useState(0);
const [wizardState, dispatch] = useReducer(wizardReducer, initialState);
const steps = useMemo(() => {
if (!wizardState.step1Data) {
return OBJECT_STEPS;
}
return wizardState.step1Data.modelType === "state"
? STATE_STEPS
: OBJECT_STEPS;
}, [wizardState.step1Data]);
const handleStep1Next = (data: Step1FormData) => {
dispatch({ type: "SET_STEP_1", payload: data });
};
const handleStep2Next = (data: Step2FormData) => {
dispatch({ type: "SET_STEP_2", payload: data });
};
const handleBack = () => {
dispatch({ type: "PREVIOUS_STEP" });
};
const handleCancel = () => {
dispatch({ type: "RESET" });
onClose();
};
return (
<Dialog
open={open}
onOpenChange={(open) => {
if (!open) {
onClose;
handleCancel();
}
}}
>
<DialogContent
className="max-h-[90dvh] max-w-4xl overflow-y-auto"
className={cn(
"",
isDesktop &&
wizardState.currentStep == 0 &&
"max-h-[90%] overflow-y-auto xl:max-h-[80%]",
isDesktop &&
wizardState.currentStep > 0 &&
"max-h-[90%] max-w-[70%] overflow-y-auto xl:max-h-[80%]",
)}
onInteractOutside={(e) => {
e.preventDefault();
}}
>
<StepIndicator
steps={STEPS}
currentStep={currentStep}
steps={steps}
currentStep={wizardState.currentStep}
variant="dots"
className="mb-4 justify-start"
/>
<DialogHeader>
<DialogTitle>{t("wizard.title")}</DialogTitle>
{currentStep === 0 && (
<DialogDescription>{t("wizard.description")}</DialogDescription>
{wizardState.currentStep === 0 && (
<DialogDescription>
{t("wizard.step1.description")}
</DialogDescription>
)}
{wizardState.currentStep === 1 &&
wizardState.step1Data?.modelType === "state" && (
<DialogDescription>
{t("wizard.step2.description")}
</DialogDescription>
)}
</DialogHeader>
<div className="pb-4">
<div className="size-full"></div>
{wizardState.currentStep === 0 && (
<Step1NameAndDefine
initialData={wizardState.step1Data}
defaultModelType={defaultModelType}
onNext={handleStep1Next}
onCancel={handleCancel}
/>
)}
{wizardState.currentStep === 1 &&
wizardState.step1Data?.modelType === "state" && (
<Step2StateArea
initialData={wizardState.step2Data}
onNext={handleStep2Next}
onBack={handleBack}
/>
)}
{((wizardState.currentStep === 2 &&
wizardState.step1Data?.modelType === "state") ||
(wizardState.currentStep === 1 &&
wizardState.step1Data?.modelType === "object")) &&
wizardState.step1Data && (
<Step3ChooseExamples
step1Data={wizardState.step1Data}
step2Data={wizardState.step2Data}
initialData={wizardState.step3Data}
onClose={onClose}
onBack={handleBack}
/>
)}
</div>
</DialogContent>
</Dialog>

View File

@ -0,0 +1,498 @@
import { Button } from "@/components/ui/button";
import {
Form,
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from "@/components/ui/form";
import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import { RadioGroup, RadioGroupItem } from "@/components/ui/radio-group";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { useForm } from "react-hook-form";
import { zodResolver } from "@hookform/resolvers/zod";
import { z } from "zod";
import { useTranslation } from "react-i18next";
import { useMemo } from "react";
import { LuX, LuPlus, LuInfo, LuExternalLink } from "react-icons/lu";
import useSWR from "swr";
import { FrigateConfig } from "@/types/frigateConfig";
import { getTranslatedLabel } from "@/utils/i18n";
import { useDocDomain } from "@/hooks/use-doc-domain";
import {
Popover,
PopoverContent,
PopoverTrigger,
} from "@/components/ui/popover";
export type ModelType = "state" | "object";
export type ObjectClassificationType = "sub_label" | "attribute";
export type Step1FormData = {
modelName: string;
modelType: ModelType;
objectLabel?: string;
objectType?: ObjectClassificationType;
classes: string[];
};
type Step1NameAndDefineProps = {
initialData?: Partial<Step1FormData>;
defaultModelType?: "state" | "object";
onNext: (data: Step1FormData) => void;
onCancel: () => void;
};
export default function Step1NameAndDefine({
initialData,
defaultModelType,
onNext,
onCancel,
}: Step1NameAndDefineProps) {
const { t } = useTranslation(["views/classificationModel"]);
const { data: config } = useSWR<FrigateConfig>("config");
const { getLocaleDocUrl } = useDocDomain();
const objectLabels = useMemo(() => {
if (!config) return [];
const labels = new Set<string>();
Object.values(config.cameras).forEach((cameraConfig) => {
if (!cameraConfig.enabled || !cameraConfig.enabled_in_config) {
return;
}
cameraConfig.objects.track.forEach((label) => {
if (!config.model.all_attributes.includes(label)) {
labels.add(label);
}
});
});
return [...labels].sort();
}, [config]);
const step1FormData = z
.object({
modelName: z
.string()
.min(1, t("wizard.step1.errors.nameRequired"))
.max(64, t("wizard.step1.errors.nameLength"))
.refine((value) => !/^\d+$/.test(value), {
message: t("wizard.step1.errors.nameOnlyNumbers"),
}),
modelType: z.enum(["state", "object"]),
objectLabel: z.string().optional(),
objectType: z.enum(["sub_label", "attribute"]).optional(),
classes: z
.array(z.string())
.min(1, t("wizard.step1.errors.classRequired"))
.refine(
(classes) => {
const nonEmpty = classes.filter((c) => c.trim().length > 0);
return nonEmpty.length >= 1;
},
{ message: t("wizard.step1.errors.classRequired") },
)
.refine(
(classes) => {
const nonEmpty = classes.filter((c) => c.trim().length > 0);
const unique = new Set(nonEmpty.map((c) => c.toLowerCase()));
return unique.size === nonEmpty.length;
},
{ message: t("wizard.step1.errors.classesUnique") },
),
})
.refine(
(data) => {
// State models require at least 2 classes
if (data.modelType === "state") {
const nonEmpty = data.classes.filter((c) => c.trim().length > 0);
return nonEmpty.length >= 2;
}
return true;
},
{
message: t("wizard.step1.errors.stateRequiresTwoClasses"),
path: ["classes"],
},
)
.refine(
(data) => {
if (data.modelType === "object") {
return data.objectLabel !== undefined && data.objectLabel !== "";
}
return true;
},
{
message: t("wizard.step1.errors.objectLabelRequired"),
path: ["objectLabel"],
},
)
.refine(
(data) => {
if (data.modelType === "object") {
return data.objectType !== undefined;
}
return true;
},
{
message: t("wizard.step1.errors.objectTypeRequired"),
path: ["objectType"],
},
);
const form = useForm<z.infer<typeof step1FormData>>({
resolver: zodResolver(step1FormData),
defaultValues: {
modelName: initialData?.modelName || "",
modelType: initialData?.modelType || defaultModelType || "state",
objectLabel: initialData?.objectLabel,
objectType: initialData?.objectType || "sub_label",
classes: initialData?.classes?.length ? initialData.classes : [""],
},
mode: "onChange",
});
const watchedClasses = form.watch("classes");
const watchedModelType = form.watch("modelType");
const watchedObjectType = form.watch("objectType");
const handleAddClass = () => {
const currentClasses = form.getValues("classes");
form.setValue("classes", [...currentClasses, ""], { shouldValidate: true });
};
const handleRemoveClass = (index: number) => {
const currentClasses = form.getValues("classes");
const newClasses = currentClasses.filter((_, i) => i !== index);
// Ensure at least one field remains (even if empty)
if (newClasses.length === 0) {
form.setValue("classes", [""], { shouldValidate: true });
} else {
form.setValue("classes", newClasses, { shouldValidate: true });
}
};
const onSubmit = (data: z.infer<typeof step1FormData>) => {
// Filter out empty classes
const filteredClasses = data.classes.filter((c) => c.trim().length > 0);
onNext({
...data,
classes: filteredClasses,
});
};
return (
<div className="space-y-6">
<Form {...form}>
<form onSubmit={form.handleSubmit(onSubmit)} className="space-y-4">
<FormField
control={form.control}
name="modelName"
render={({ field }) => (
<FormItem>
<FormLabel className="text-primary-variant">
{t("wizard.step1.name")}
</FormLabel>
<FormControl>
<Input
className="h-8"
placeholder={t("wizard.step1.namePlaceholder")}
{...field}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="modelType"
render={({ field }) => (
<FormItem>
<FormLabel className="text-primary-variant">
{t("wizard.step1.type")}
</FormLabel>
<FormControl>
<RadioGroup
onValueChange={field.onChange}
defaultValue={field.value}
className="flex flex-col gap-4 pt-2"
>
<div className="flex items-center gap-2">
<RadioGroupItem
className={
watchedModelType === "state"
? "bg-selected from-selected/50 to-selected/90 text-selected"
: "bg-secondary from-secondary/50 to-secondary/90 text-secondary"
}
id="state"
value="state"
/>
<Label className="cursor-pointer" htmlFor="state">
{t("wizard.step1.typeState")}
</Label>
</div>
<div className="flex items-center gap-2">
<RadioGroupItem
className={
watchedModelType === "object"
? "bg-selected from-selected/50 to-selected/90 text-selected"
: "bg-secondary from-secondary/50 to-secondary/90 text-secondary"
}
id="object"
value="object"
/>
<Label className="cursor-pointer" htmlFor="object">
{t("wizard.step1.typeObject")}
</Label>
</div>
</RadioGroup>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
{watchedModelType === "object" && (
<>
<FormField
control={form.control}
name="objectLabel"
render={({ field }) => (
<FormItem>
<FormLabel className="text-primary-variant">
{t("wizard.step1.objectLabel")}
</FormLabel>
<Select
onValueChange={field.onChange}
defaultValue={field.value}
>
<FormControl>
<SelectTrigger className="h-8">
<SelectValue
placeholder={t(
"wizard.step1.objectLabelPlaceholder",
)}
/>
</SelectTrigger>
</FormControl>
<SelectContent>
{objectLabels.map((label) => (
<SelectItem
key={label}
value={label}
className="cursor-pointer hover:bg-secondary-highlight"
>
{getTranslatedLabel(label)}
</SelectItem>
))}
</SelectContent>
</Select>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="objectType"
render={({ field }) => (
<FormItem>
<div className="flex items-center gap-1">
<FormLabel className="text-primary-variant">
{t("wizard.step1.classificationType")}
</FormLabel>
<Popover>
<PopoverTrigger asChild>
<Button
variant="ghost"
size="sm"
className="h-4 w-4 p-0"
>
<LuInfo className="size-3" />
</Button>
</PopoverTrigger>
<PopoverContent className="pointer-events-auto w-80 text-xs">
<div className="flex flex-col gap-2">
<div className="text-sm">
{t("wizard.step1.classificationTypeDesc")}
</div>
<div className="mt-3 flex items-center text-primary">
<a
href={getLocaleDocUrl(
"configuration/custom_classification/object_classification#classification-type",
)}
target="_blank"
rel="noopener noreferrer"
className="inline cursor-pointer"
>
{t("readTheDocumentation", { ns: "common" })}
<LuExternalLink className="ml-2 inline-flex size-3" />
</a>
</div>
</div>
</PopoverContent>
</Popover>
</div>
<FormControl>
<RadioGroup
onValueChange={field.onChange}
defaultValue={field.value}
className="flex flex-col gap-4 pt-2"
>
<div className="flex items-center gap-2">
<RadioGroupItem
className={
watchedObjectType === "sub_label"
? "bg-selected from-selected/50 to-selected/90 text-selected"
: "bg-secondary from-secondary/50 to-secondary/90 text-secondary"
}
id="sub_label"
value="sub_label"
/>
<Label className="cursor-pointer" htmlFor="sub_label">
{t("wizard.step1.classificationSubLabel")}
</Label>
</div>
<div className="flex items-center gap-2">
<RadioGroupItem
className={
watchedObjectType === "attribute"
? "bg-selected from-selected/50 to-selected/90 text-selected"
: "bg-secondary from-secondary/50 to-secondary/90 text-secondary"
}
id="attribute"
value="attribute"
/>
<Label className="cursor-pointer" htmlFor="attribute">
{t("wizard.step1.classificationAttribute")}
</Label>
</div>
</RadioGroup>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</>
)}
<div className="space-y-2">
<div className="flex items-center justify-between">
<div className="flex items-center gap-1">
<FormLabel className="text-primary-variant">
{t("wizard.step1.classes")}
</FormLabel>
<Popover>
<PopoverTrigger asChild>
<Button variant="ghost" size="sm" className="h-4 w-4 p-0">
<LuInfo className="size-3" />
</Button>
</PopoverTrigger>
<PopoverContent className="pointer-events-auto w-80 text-xs">
<div className="flex flex-col gap-2">
<div className="text-sm">
{watchedModelType === "state"
? t("wizard.step1.classesStateDesc")
: t("wizard.step1.classesObjectDesc")}
</div>
<div className="mt-3 flex items-center text-primary">
<a
href={getLocaleDocUrl(
watchedModelType === "state"
? "configuration/custom_classification/state_classification"
: "configuration/custom_classification/object_classification",
)}
target="_blank"
rel="noopener noreferrer"
className="inline cursor-pointer"
>
{t("readTheDocumentation", { ns: "common" })}
<LuExternalLink className="ml-2 inline-flex size-3" />
</a>
</div>
</div>
</PopoverContent>
</Popover>
</div>
<Button
type="button"
variant="secondary"
className="size-6 rounded-md bg-secondary-foreground p-1 text-background"
onClick={handleAddClass}
>
<LuPlus />
</Button>
</div>
<div className="space-y-2">
{watchedClasses.map((_, index) => (
<FormField
key={index}
control={form.control}
name={`classes.${index}`}
render={({ field }) => (
<FormItem>
<FormControl>
<div className="flex items-center gap-2">
<Input
className="h-8"
placeholder={t("wizard.step1.classPlaceholder")}
{...field}
/>
{watchedClasses.length > 1 && (
<Button
type="button"
variant="ghost"
size="sm"
className="h-8 w-8 p-0"
onClick={() => handleRemoveClass(index)}
>
<LuX className="size-4" />
</Button>
)}
</div>
</FormControl>
</FormItem>
)}
/>
))}
</div>
{form.formState.errors.classes && (
<p className="text-sm font-medium text-destructive">
{form.formState.errors.classes.message}
</p>
)}
</div>
</form>
</Form>
<div className="flex flex-col gap-3 pt-3 sm:flex-row sm:justify-end sm:gap-4">
<Button type="button" onClick={onCancel} className="sm:flex-1">
{t("button.cancel", { ns: "common" })}
</Button>
<Button
type="button"
onClick={form.handleSubmit(onSubmit)}
variant="select"
className="flex items-center justify-center gap-2 sm:flex-1"
disabled={!form.formState.isValid}
>
{t("button.continue", { ns: "common" })}
</Button>
</div>
</div>
);
}

View File

@ -0,0 +1,479 @@
import { Button } from "@/components/ui/button";
import { useTranslation } from "react-i18next";
import { useState, useMemo, useRef, useCallback, useEffect } from "react";
import useSWR from "swr";
import { FrigateConfig } from "@/types/frigateConfig";
import {
Popover,
PopoverContent,
PopoverTrigger,
} from "@/components/ui/popover";
import { LuX, LuPlus } from "react-icons/lu";
import { Stage, Layer, Rect, Transformer } from "react-konva";
import Konva from "konva";
import { useResizeObserver } from "@/hooks/resize-observer";
import { useApiHost } from "@/api";
import { resolveCameraName } from "@/hooks/use-camera-friendly-name";
import Heading from "@/components/ui/heading";
import { isMobile } from "react-device-detect";
import { cn } from "@/lib/utils";
export type CameraAreaConfig = {
camera: string;
crop: [number, number, number, number];
};
export type Step2FormData = {
cameraAreas: CameraAreaConfig[];
};
type Step2StateAreaProps = {
initialData?: Partial<Step2FormData>;
onNext: (data: Step2FormData) => void;
onBack: () => void;
};
export default function Step2StateArea({
initialData,
onNext,
onBack,
}: Step2StateAreaProps) {
const { t } = useTranslation(["views/classificationModel"]);
const { data: config } = useSWR<FrigateConfig>("config");
const apiHost = useApiHost();
const [cameraAreas, setCameraAreas] = useState<CameraAreaConfig[]>(
initialData?.cameraAreas || [],
);
const [selectedCameraIndex, setSelectedCameraIndex] = useState<number>(0);
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
const [imageLoaded, setImageLoaded] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const imageRef = useRef<HTMLImageElement>(null);
const stageRef = useRef<Konva.Stage>(null);
const rectRef = useRef<Konva.Rect>(null);
const transformerRef = useRef<Konva.Transformer>(null);
const [{ width: containerWidth }] = useResizeObserver(containerRef);
const availableCameras = useMemo(() => {
if (!config) return [];
const selectedCameraNames = cameraAreas.map((ca) => ca.camera);
return Object.entries(config.cameras)
.sort()
.filter(
([name, cam]) =>
cam.enabled &&
cam.enabled_in_config &&
!selectedCameraNames.includes(name),
)
.map(([name]) => ({
name,
displayName: resolveCameraName(config, name),
}));
}, [config, cameraAreas]);
const selectedCamera = useMemo(() => {
if (cameraAreas.length === 0) return null;
return cameraAreas[selectedCameraIndex];
}, [cameraAreas, selectedCameraIndex]);
const selectedCameraConfig = useMemo(() => {
if (!config || !selectedCamera) return null;
return config.cameras[selectedCamera.camera];
}, [config, selectedCamera]);
const imageSize = useMemo(() => {
if (!containerWidth || !selectedCameraConfig) {
return { width: 0, height: 0 };
}
const containerAspectRatio = 16 / 9;
const containerHeight = containerWidth / containerAspectRatio;
const cameraAspectRatio =
selectedCameraConfig.detect.width / selectedCameraConfig.detect.height;
// Fit camera within 16:9 container
let imageWidth, imageHeight;
if (cameraAspectRatio > containerAspectRatio) {
imageWidth = containerWidth;
imageHeight = imageWidth / cameraAspectRatio;
} else {
imageHeight = containerHeight;
imageWidth = imageHeight * cameraAspectRatio;
}
return { width: imageWidth, height: imageHeight };
}, [containerWidth, selectedCameraConfig]);
const handleAddCamera = useCallback(
(cameraName: string) => {
// Calculate a square crop in pixel space
const camera = config?.cameras[cameraName];
if (!camera) return;
const cameraAspect = camera.detect.width / camera.detect.height;
const cropSize = 0.3;
let x1, y1, x2, y2;
if (cameraAspect >= 1) {
const pixelSize = cropSize * camera.detect.height;
const normalizedWidth = pixelSize / camera.detect.width;
x1 = (1 - normalizedWidth) / 2;
y1 = (1 - cropSize) / 2;
x2 = x1 + normalizedWidth;
y2 = y1 + cropSize;
} else {
const pixelSize = cropSize * camera.detect.width;
const normalizedHeight = pixelSize / camera.detect.height;
x1 = (1 - cropSize) / 2;
y1 = (1 - normalizedHeight) / 2;
x2 = x1 + cropSize;
y2 = y1 + normalizedHeight;
}
const newArea: CameraAreaConfig = {
camera: cameraName,
crop: [x1, y1, x2, y2],
};
setCameraAreas([...cameraAreas, newArea]);
setSelectedCameraIndex(cameraAreas.length);
setIsPopoverOpen(false);
},
[cameraAreas, config],
);
const handleRemoveCamera = useCallback(
(index: number) => {
const newAreas = cameraAreas.filter((_, i) => i !== index);
setCameraAreas(newAreas);
if (selectedCameraIndex >= newAreas.length) {
setSelectedCameraIndex(Math.max(0, newAreas.length - 1));
}
},
[cameraAreas, selectedCameraIndex],
);
const handleCropChange = useCallback(
(crop: [number, number, number, number]) => {
const newAreas = [...cameraAreas];
newAreas[selectedCameraIndex] = {
...newAreas[selectedCameraIndex],
crop,
};
setCameraAreas(newAreas);
},
[cameraAreas, selectedCameraIndex],
);
useEffect(() => {
setImageLoaded(false);
}, [selectedCamera]);
useEffect(() => {
const rect = rectRef.current;
const transformer = transformerRef.current;
if (
rect &&
transformer &&
selectedCamera &&
imageSize.width > 0 &&
imageLoaded
) {
rect.scaleX(1);
rect.scaleY(1);
transformer.nodes([rect]);
transformer.getLayer()?.batchDraw();
}
}, [selectedCamera, imageSize, imageLoaded]);
const handleRectChange = useCallback(() => {
const rect = rectRef.current;
if (rect && imageSize.width > 0) {
const actualWidth = rect.width() * rect.scaleX();
const actualHeight = rect.height() * rect.scaleY();
// Average dimensions to maintain perfect square
const size = (actualWidth + actualHeight) / 2;
rect.width(size);
rect.height(size);
rect.scaleX(1);
rect.scaleY(1);
const x1 = rect.x() / imageSize.width;
const y1 = rect.y() / imageSize.height;
const x2 = (rect.x() + size) / imageSize.width;
const y2 = (rect.y() + size) / imageSize.height;
handleCropChange([x1, y1, x2, y2]);
}
}, [imageSize, handleCropChange]);
const handleContinue = useCallback(() => {
onNext({ cameraAreas });
}, [cameraAreas, onNext]);
const canContinue = cameraAreas.length > 0;
return (
<div className="flex flex-col gap-4">
<div
className={cn(
"flex gap-4 overflow-hidden",
isMobile ? "flex-col" : "flex-row",
)}
>
<div
className={cn(
"flex flex-shrink-0 flex-col gap-2 overflow-y-auto rounded-lg bg-secondary p-4",
isMobile ? "w-full" : "w-64",
)}
>
<div className="flex items-center justify-between">
<h3 className="text-sm font-medium">{t("wizard.step2.cameras")}</h3>
{availableCameras.length > 0 ? (
<Popover
open={isPopoverOpen}
onOpenChange={setIsPopoverOpen}
modal={true}
>
<PopoverTrigger asChild>
<Button
type="button"
variant="secondary"
className="size-6 rounded-md bg-secondary-foreground p-1 text-background"
aria-label="Add camera"
>
<LuPlus />
</Button>
</PopoverTrigger>
<PopoverContent
className="scrollbar-container w-64 border bg-background p-3 shadow-lg"
align="start"
sideOffset={5}
onOpenAutoFocus={(e) => e.preventDefault()}
>
<div className="flex flex-col gap-2">
<Heading as="h4" className="text-sm text-primary-variant">
{t("wizard.step2.selectCamera")}
</Heading>
<div className="scrollbar-container flex max-h-[30vh] flex-col gap-1 overflow-y-auto">
{availableCameras.map((cam) => (
<Button
key={cam.name}
type="button"
variant="ghost"
size="sm"
className="h-auto justify-start p-2 capitalize text-primary"
onClick={() => {
handleAddCamera(cam.name);
}}
>
{cam.displayName}
</Button>
))}
</div>
</div>
</PopoverContent>
</Popover>
) : (
<Button
variant="secondary"
className="size-6 cursor-not-allowed rounded-md bg-muted p-1 text-muted-foreground"
disabled
>
<LuPlus />
</Button>
)}
</div>
<div className="flex flex-col gap-1">
{cameraAreas.map((area, index) => {
const isSelected = index === selectedCameraIndex;
const displayName = resolveCameraName(config, area.camera);
return (
<div
key={area.camera}
className={`flex items-center justify-between rounded-md p-2 ${
isSelected
? "bg-selected/20 ring-1 ring-selected"
: "hover:bg-secondary/50"
} cursor-pointer`}
onClick={() => setSelectedCameraIndex(index)}
>
<span className="text-sm capitalize">{displayName}</span>
<Button
type="button"
variant="ghost"
size="sm"
className="h-6 w-6 p-0"
onClick={(e) => {
e.stopPropagation();
handleRemoveCamera(index);
}}
>
<LuX className="size-4" />
</Button>
</div>
);
})}
</div>
{cameraAreas.length === 0 && (
<div className="flex flex-1 items-center justify-center text-center text-sm text-muted-foreground">
{t("wizard.step2.noCameras")}
</div>
)}
</div>
<div className="flex flex-1 items-center justify-center overflow-hidden rounded-lg p-4">
<div
ref={containerRef}
className="flex items-center justify-center"
style={{
width: "100%",
aspectRatio: "16 / 9",
maxHeight: "100%",
}}
>
{selectedCamera && selectedCameraConfig && imageSize.width > 0 ? (
<div
style={{
width: imageSize.width,
height: imageSize.height,
position: "relative",
}}
>
<img
ref={imageRef}
src={`${apiHost}api/${selectedCamera.camera}/latest.jpg?h=500`}
alt={resolveCameraName(config, selectedCamera.camera)}
className="h-full w-full object-contain"
onLoad={() => setImageLoaded(true)}
/>
<Stage
ref={stageRef}
width={imageSize.width}
height={imageSize.height}
className="absolute inset-0"
>
<Layer>
<Rect
ref={rectRef}
x={selectedCamera.crop[0] * imageSize.width}
y={selectedCamera.crop[1] * imageSize.height}
width={
(selectedCamera.crop[2] - selectedCamera.crop[0]) *
imageSize.width
}
height={
(selectedCamera.crop[3] - selectedCamera.crop[1]) *
imageSize.height
}
stroke="#3b82f6"
strokeWidth={2}
fill="rgba(59, 130, 246, 0.1)"
draggable
dragBoundFunc={(pos) => {
const rect = rectRef.current;
if (!rect) return pos;
const size = rect.width();
const x = Math.max(
0,
Math.min(pos.x, imageSize.width - size),
);
const y = Math.max(
0,
Math.min(pos.y, imageSize.height - size),
);
return { x, y };
}}
onDragEnd={handleRectChange}
onTransformEnd={handleRectChange}
/>
<Transformer
ref={transformerRef}
rotateEnabled={false}
enabledAnchors={[
"top-left",
"top-right",
"bottom-left",
"bottom-right",
]}
boundBoxFunc={(_oldBox, newBox) => {
const minSize = 50;
const maxSize = Math.min(
imageSize.width,
imageSize.height,
);
// Clamp dimensions to stage bounds first
const clampedWidth = Math.max(
minSize,
Math.min(newBox.width, maxSize),
);
const clampedHeight = Math.max(
minSize,
Math.min(newBox.height, maxSize),
);
// Enforce square using average
const size = (clampedWidth + clampedHeight) / 2;
// Clamp position to keep square within bounds
const x = Math.max(
0,
Math.min(newBox.x, imageSize.width - size),
);
const y = Math.max(
0,
Math.min(newBox.y, imageSize.height - size),
);
return {
...newBox,
x,
y,
width: size,
height: size,
};
}}
/>
</Layer>
</Stage>
</div>
) : (
<div className="flex items-center justify-center text-muted-foreground">
{t("wizard.step2.selectCameraPrompt")}
</div>
)}
</div>
</div>
</div>
<div className="flex flex-col gap-3 pt-3 sm:flex-row sm:justify-end sm:gap-4">
<Button type="button" onClick={onBack} className="sm:flex-1">
{t("button.back", { ns: "common" })}
</Button>
<Button
type="button"
onClick={handleContinue}
variant="select"
className="flex items-center justify-center gap-2 sm:flex-1"
disabled={!canContinue}
>
{t("button.continue", { ns: "common" })}
</Button>
</div>
</div>
);
}

View File

@ -0,0 +1,444 @@
import { Button } from "@/components/ui/button";
import { useTranslation } from "react-i18next";
import { useState, useEffect, useCallback, useMemo } from "react";
import ActivityIndicator from "@/components/indicators/activity-indicator";
import axios from "axios";
import { toast } from "sonner";
import { Step1FormData } from "./Step1NameAndDefine";
import { Step2FormData } from "./Step2StateArea";
import useSWR from "swr";
import { baseUrl } from "@/api/baseUrl";
import { isMobile } from "react-device-detect";
import { cn } from "@/lib/utils";
export type Step3FormData = {
examplesGenerated: boolean;
imageClassifications?: { [imageName: string]: string };
};
type Step3ChooseExamplesProps = {
step1Data: Step1FormData;
step2Data?: Step2FormData;
initialData?: Partial<Step3FormData>;
onClose: () => void;
onBack: () => void;
};
export default function Step3ChooseExamples({
step1Data,
step2Data,
initialData,
onClose,
onBack,
}: Step3ChooseExamplesProps) {
const { t } = useTranslation(["views/classificationModel"]);
const [isGenerating, setIsGenerating] = useState(false);
const [hasGenerated, setHasGenerated] = useState(
initialData?.examplesGenerated || false,
);
const [imageClassifications, setImageClassifications] = useState<{
[imageName: string]: string;
}>(initialData?.imageClassifications || {});
const [isTraining, setIsTraining] = useState(false);
const [isProcessing, setIsProcessing] = useState(false);
const [currentClassIndex, setCurrentClassIndex] = useState(0);
const [selectedImages, setSelectedImages] = useState<Set<string>>(new Set());
const { data: trainImages, mutate: refreshTrainImages } = useSWR<string[]>(
hasGenerated ? `classification/${step1Data.modelName}/train` : null,
);
const unknownImages = useMemo(() => {
if (!trainImages) return [];
return trainImages;
}, [trainImages]);
const toggleImageSelection = useCallback((imageName: string) => {
setSelectedImages((prev) => {
const newSet = new Set(prev);
if (newSet.has(imageName)) {
newSet.delete(imageName);
} else {
newSet.add(imageName);
}
return newSet;
});
}, []);
// Get all classes (excluding "none" - it will be auto-assigned)
const allClasses = useMemo(() => {
return [...step1Data.classes];
}, [step1Data.classes]);
const currentClass = allClasses[currentClassIndex];
const processClassificationsAndTrain = useCallback(
async (classifications: { [imageName: string]: string }) => {
// Step 1: Create config for the new model
const modelConfig: {
enabled: boolean;
name: string;
threshold: number;
state_config?: {
cameras: Record<string, { crop: number[] }>;
motion: boolean;
};
object_config?: { objects: string[]; classification_type: string };
} = {
enabled: true,
name: step1Data.modelName,
threshold: 0.8,
};
if (step1Data.modelType === "state") {
// State model config
const cameras: Record<string, { crop: number[] }> = {};
step2Data?.cameraAreas.forEach((area) => {
cameras[area.camera] = {
crop: area.crop,
};
});
modelConfig.state_config = {
cameras,
motion: true,
};
} else {
// Object model config
modelConfig.object_config = {
objects: step1Data.objectLabel ? [step1Data.objectLabel] : [],
classification_type: step1Data.objectType || "sub_label",
} as { objects: string[]; classification_type: string };
}
// Update config via config API
await axios.put("/config/set", {
requires_restart: 0,
update_topic: `config/classification/custom/${step1Data.modelName}`,
config_data: {
classification: {
custom: {
[step1Data.modelName]: modelConfig,
},
},
},
});
// Step 2: Classify each image by moving it to the correct category folder
const categorizePromises = Object.entries(classifications).map(
([imageName, className]) => {
if (!className) return Promise.resolve();
return axios.post(
`/classification/${step1Data.modelName}/dataset/categorize`,
{
training_file: imageName,
category: className === "none" ? "none" : className,
},
);
},
);
await Promise.all(categorizePromises);
// Step 3: Kick off training
await axios.post(`/classification/${step1Data.modelName}/train`);
toast.success(t("wizard.step3.trainingStarted"));
setIsTraining(true);
},
[step1Data, step2Data, t],
);
const handleContinueClassification = useCallback(async () => {
// Mark selected images with current class
const newClassifications = { ...imageClassifications };
selectedImages.forEach((imageName) => {
newClassifications[imageName] = currentClass;
});
// Check if we're on the last class to select
const isLastClass = currentClassIndex === allClasses.length - 1;
if (isLastClass) {
// Assign remaining unclassified images
unknownImages.slice(0, 24).forEach((imageName) => {
if (!newClassifications[imageName]) {
// For state models with 2 classes, assign to the last class
// For object models, assign to "none"
if (step1Data.modelType === "state" && allClasses.length === 2) {
newClassifications[imageName] = allClasses[allClasses.length - 1];
} else {
newClassifications[imageName] = "none";
}
}
});
// All done, trigger training immediately
setImageClassifications(newClassifications);
setIsProcessing(true);
try {
await processClassificationsAndTrain(newClassifications);
} catch (error) {
const axiosError = error as {
response?: { data?: { message?: string; detail?: string } };
message?: string;
};
const errorMessage =
axiosError.response?.data?.message ||
axiosError.response?.data?.detail ||
axiosError.message ||
"Failed to classify images";
toast.error(
t("wizard.step3.errors.classifyFailed", { error: errorMessage }),
);
setIsProcessing(false);
}
} else {
// Move to next class
setImageClassifications(newClassifications);
setCurrentClassIndex((prev) => prev + 1);
setSelectedImages(new Set());
}
}, [
selectedImages,
currentClass,
currentClassIndex,
allClasses,
imageClassifications,
unknownImages,
step1Data,
processClassificationsAndTrain,
t,
]);
const generateExamples = useCallback(async () => {
setIsGenerating(true);
try {
if (step1Data.modelType === "state") {
// For state models, use cameras and crop areas
if (!step2Data?.cameraAreas || step2Data.cameraAreas.length === 0) {
toast.error(t("wizard.step3.errors.noCameras"));
setIsGenerating(false);
return;
}
const cameras: { [key: string]: [number, number, number, number] } = {};
step2Data.cameraAreas.forEach((area) => {
cameras[area.camera] = area.crop;
});
await axios.post("/classification/generate_examples/state", {
model_name: step1Data.modelName,
cameras,
});
} else {
// For object models, use label
if (!step1Data.objectLabel) {
toast.error(t("wizard.step3.errors.noObjectLabel"));
setIsGenerating(false);
return;
}
// For now, use all enabled cameras
// TODO: In the future, we might want to let users select specific cameras
await axios.post("/classification/generate_examples/object", {
model_name: step1Data.modelName,
label: step1Data.objectLabel,
});
}
setHasGenerated(true);
toast.success(t("wizard.step3.generateSuccess"));
await refreshTrainImages();
} catch (error) {
const axiosError = error as {
response?: { data?: { message?: string; detail?: string } };
message?: string;
};
const errorMessage =
axiosError.response?.data?.message ||
axiosError.response?.data?.detail ||
axiosError.message ||
"Failed to generate examples";
toast.error(
t("wizard.step3.errors.generateFailed", { error: errorMessage }),
);
} finally {
setIsGenerating(false);
}
}, [step1Data, step2Data, t, refreshTrainImages]);
useEffect(() => {
if (!hasGenerated && !isGenerating) {
generateExamples();
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
const handleContinue = useCallback(async () => {
setIsProcessing(true);
try {
await processClassificationsAndTrain(imageClassifications);
} catch (error) {
const axiosError = error as {
response?: { data?: { message?: string; detail?: string } };
message?: string;
};
const errorMessage =
axiosError.response?.data?.message ||
axiosError.response?.data?.detail ||
axiosError.message ||
"Failed to classify images";
toast.error(
t("wizard.step3.errors.classifyFailed", { error: errorMessage }),
);
setIsProcessing(false);
}
}, [imageClassifications, processClassificationsAndTrain, t]);
const unclassifiedImages = useMemo(() => {
if (!unknownImages) return [];
const images = unknownImages.slice(0, 24);
// Only filter if we have any classifications
if (Object.keys(imageClassifications).length === 0) {
return images;
}
return images.filter((img) => !imageClassifications[img]);
}, [unknownImages, imageClassifications]);
const allImagesClassified = useMemo(() => {
return unclassifiedImages.length === 0;
}, [unclassifiedImages]);
return (
<div className="flex flex-col gap-6">
{isTraining ? (
<div className="flex flex-col items-center gap-6 py-12">
<ActivityIndicator className="size-12" />
<div className="text-center">
<h3 className="mb-2 text-lg font-medium">
{t("wizard.step3.training.title")}
</h3>
<p className="text-sm text-muted-foreground">
{t("wizard.step3.training.description")}
</p>
</div>
<Button onClick={onClose} variant="select" className="mt-4">
{t("button.close", { ns: "common" })}
</Button>
</div>
) : isGenerating ? (
<div className="flex h-[50vh] flex-col items-center justify-center gap-4">
<ActivityIndicator className="size-12" />
<div className="text-center">
<h3 className="mb-2 text-lg font-medium">
{t("wizard.step3.generating.title")}
</h3>
<p className="text-sm text-muted-foreground">
{t("wizard.step3.generating.description")}
</p>
</div>
</div>
) : hasGenerated ? (
<div className="flex flex-col gap-4">
{!allImagesClassified && (
<div className="text-center">
<h3 className="text-lg font-medium">
{t("wizard.step3.selectImagesPrompt", {
className: currentClass,
})}
</h3>
<p className="text-sm text-muted-foreground">
{t("wizard.step3.selectImagesDescription")}
</p>
</div>
)}
<div
className={cn(
"rounded-lg bg-secondary/30 p-4",
isMobile && "max-h-[60vh] overflow-y-auto",
)}
>
{!unknownImages || unknownImages.length === 0 ? (
<div className="flex h-[40vh] flex-col items-center justify-center gap-4">
<p className="text-muted-foreground">
{t("wizard.step3.noImages")}
</p>
<Button onClick={generateExamples} variant="select">
{t("wizard.step3.retryGenerate")}
</Button>
</div>
) : allImagesClassified && isProcessing ? (
<div className="flex h-[40vh] flex-col items-center justify-center gap-4">
<ActivityIndicator className="size-12" />
<p className="text-lg font-medium">
{t("wizard.step3.classifying")}
</p>
</div>
) : (
<div className="grid grid-cols-2 gap-4 sm:grid-cols-6">
{unclassifiedImages.map((imageName, index) => {
const isSelected = selectedImages.has(imageName);
return (
<div
key={imageName}
className={cn(
"aspect-square cursor-pointer overflow-hidden rounded-lg border-2 bg-background transition-all",
isSelected && "border-selected ring-2 ring-selected",
)}
onClick={() => toggleImageSelection(imageName)}
>
<img
src={`${baseUrl}clips/${step1Data.modelName}/train/${imageName}`}
alt={`Example ${index + 1}`}
className="h-full w-full object-cover"
/>
</div>
);
})}
</div>
)}
</div>
</div>
) : (
<div className="flex h-[50vh] flex-col items-center justify-center gap-4">
<p className="text-sm text-destructive">
{t("wizard.step3.errors.generationFailed")}
</p>
<Button onClick={generateExamples} variant="select">
{t("wizard.step3.retryGenerate")}
</Button>
</div>
)}
{!isTraining && (
<div className="flex flex-col gap-3 pt-3 sm:flex-row sm:justify-end sm:gap-4">
<Button type="button" onClick={onBack} className="sm:flex-1">
{t("button.back", { ns: "common" })}
</Button>
<Button
type="button"
onClick={
allImagesClassified
? handleContinue
: handleContinueClassification
}
variant="select"
className="flex items-center justify-center gap-2 sm:flex-1"
disabled={!hasGenerated || isGenerating || isProcessing}
>
{isProcessing && <ActivityIndicator className="size-4" />}
{t("button.continue", { ns: "common" })}
</Button>
</div>
)}
</div>
);
}

View File

@ -214,10 +214,14 @@ export default function SearchResultActions({
searchResult.data.type == "object" && (
<Tooltip>
<TooltipTrigger>
<MdImageSearch
className="size-5 cursor-pointer text-primary-variant hover:text-primary"
onClick={findSimilar}
/>
<div className="group relative inline-flex items-center justify-center">
{/* blurred circular hover background */}
<div className="pointer-events-none absolute inset-0 m-auto size-5 scale-95 rounded-full bg-black opacity-0 blur-sm transition-all duration-200 group-hover:scale-100 group-hover:opacity-100 group-hover:blur-xl" />
<MdImageSearch
className="relative z-10 size-5 cursor-pointer text-white/85 hover:text-white"
onClick={findSimilar}
/>
</div>
</TooltipTrigger>
<TooltipContent>
{t("itemMenu.findSimilar.label")}
@ -233,10 +237,13 @@ export default function SearchResultActions({
!searchResult.plus_id && (
<Tooltip>
<TooltipTrigger>
<FrigatePlusIcon
className="size-5 cursor-pointer text-primary-variant hover:text-primary"
onClick={showSnapshot}
/>
<div className="group relative inline-flex items-center justify-center">
<div className="pointer-events-none absolute inset-0 m-auto size-5 scale-95 rounded-full bg-black opacity-0 blur-sm transition-all duration-200 group-hover:scale-100 group-hover:opacity-100 group-hover:blur-xl" />
<FrigatePlusIcon
className="relative z-10 size-5 cursor-pointer text-white/85 hover:text-white"
onClick={showSnapshot}
/>
</div>
</TooltipTrigger>
<TooltipContent>
{t("itemMenu.submitToPlus.label")}
@ -246,7 +253,10 @@ export default function SearchResultActions({
<DropdownMenu>
<DropdownMenuTrigger>
<FiMoreVertical className="size-5 cursor-pointer text-primary-variant hover:text-primary" />
<div className="group relative inline-flex items-center justify-center">
<div className="pointer-events-none absolute inset-0 m-auto size-5 scale-95 rounded-full bg-black opacity-0 blur-sm transition-all duration-200 group-hover:scale-100 group-hover:opacity-100 group-hover:blur-xl" />
<FiMoreVertical className="relative z-10 size-5 cursor-pointer text-white/85 hover:text-white" />
</div>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">{menuItems}</DropdownMenuContent>
</DropdownMenu>

View File

@ -51,6 +51,15 @@ export default function MobileTimelineDrawer({
>
{t("events.label")}
</div>
<div
className={`mx-4 w-full py-2 text-center smart-capitalize ${selected == "detail" ? "rounded-lg bg-secondary" : ""}`}
onClick={() => {
onSelect("detail");
setDrawer(false);
}}
>
{t("detail.label")}
</div>
</DrawerContent>
</Drawer>
);

View File

@ -102,17 +102,23 @@ export default function CreateFaceWizardDialog({
}}
>
<Content
className={cn("flex flex-col gap-4", isDesktop ? "max-w-3xl" : "p-4")}
className={cn(
"flex flex-col gap-4",
isDesktop ? (step == 0 ? "max-w-xl" : "max-w-3xl") : "p-4",
)}
>
<Header>
<Title>{t("button.addFace")}</Title>
{isDesktop && <Description>{t("description.addFace")}</Description>}
</Header>
<StepIndicator
steps={STEPS}
currentStep={step}
translationNameSpace="views/faceLibrary"
className="mb-4 justify-start"
variant="dots"
/>
<Header>
<Title>{t("button.addFace")}</Title>
{isDesktop && <Description>{t("description.addFace")}</Description>}
</Header>
{step == 0 && (
<TextEntry
placeholder={t("description.placeholder")}

View File

@ -47,6 +47,7 @@ import { getLifecycleItemDescription } from "@/utils/lifecycleUtil";
import { IoPlayCircleOutline } from "react-icons/io5";
import { useTranslation } from "react-i18next";
import { getTranslatedLabel } from "@/utils/i18n";
import { Badge } from "@/components/ui/badge";
type ObjectLifecycleProps = {
className?: string;
@ -355,6 +356,52 @@ export default function ObjectLifecycle({
return idx === -1 ? 0 : idx;
}, [eventSequence, timeIndex]);
// Calculate how far down the blue line should extend based on timeIndex
const calculateLineHeight = () => {
if (!eventSequence || eventSequence.length === 0) return 0;
const currentTime = timeIndex ?? 0;
// Find which events have been passed
let lastPassedIndex = -1;
for (let i = 0; i < eventSequence.length; i++) {
if (currentTime >= (eventSequence[i].timestamp ?? 0)) {
lastPassedIndex = i;
} else {
break;
}
}
// No events passed yet
if (lastPassedIndex < 0) return 0;
// All events passed
if (lastPassedIndex >= eventSequence.length - 1) return 100;
// Calculate percentage based on item position, not time
// Each item occupies an equal visual space regardless of time gaps
const itemPercentage = 100 / (eventSequence.length - 1);
// Find progress between current and next event for smooth transition
const currentEvent = eventSequence[lastPassedIndex];
const nextEvent = eventSequence[lastPassedIndex + 1];
const currentTimestamp = currentEvent.timestamp ?? 0;
const nextTimestamp = nextEvent.timestamp ?? 0;
// Calculate interpolation between the two events
const timeBetween = nextTimestamp - currentTimestamp;
const timeElapsed = currentTime - currentTimestamp;
const interpolation = timeBetween > 0 ? timeElapsed / timeBetween : 0;
// Base position plus interpolated progress to next item
return Math.min(
100,
lastPassedIndex * itemPercentage + interpolation * itemPercentage,
);
};
const blueLineHeight = calculateLineHeight();
if (!config) {
return <ActivityIndicator />;
}
@ -569,7 +616,7 @@ export default function ObjectLifecycle({
<div className="mt-4">
<div
className={cn(
"rounded-md bg-secondary p-2 outline outline-[3px] -outline-offset-[2.8px] outline-transparent duration-500",
"rounded-md bg-secondary p-3 outline outline-[3px] -outline-offset-[2.8px] outline-transparent duration-500",
)}
>
<div className="flex w-full items-center justify-between">
@ -581,10 +628,12 @@ export default function ObjectLifecycle({
}}
role="button"
>
{getIconForLabel(
event.label,
"size-6 text-primary dark:text-white",
)}
<div className={cn("ml-1 rounded-full bg-muted-foreground p-2")}>
{getIconForLabel(
event.label,
"size-6 text-primary dark:text-white",
)}
</div>
<div className="flex items-end gap-2">
<span>{getTranslatedLabel(event.label)}</span>
<span className="text-secondary-foreground">
@ -602,147 +651,79 @@ export default function ObjectLifecycle({
{t("detail.noObjectDetailData", { ns: "views/events" })}
</div>
) : (
<div className="mx-2 mt-4 space-y-2">
{eventSequence.map((item, idx) => {
const isActive =
Math.abs((timeIndex ?? 0) - (item.timestamp ?? 0)) <= 0.5;
const formattedEventTimestamp = config
? formatUnixTimestampToDateTime(item.timestamp ?? 0, {
timezone: config.ui.timezone,
date_format:
config.ui.time_format == "24hour"
? t(
"time.formattedTimestampHourMinuteSecond.24hour",
{ ns: "common" },
)
: t(
"time.formattedTimestampHourMinuteSecond.12hour",
{ ns: "common" },
),
time_style: "medium",
date_style: "medium",
})
: "";
<div className="-pb-2 relative mx-2">
<div className="absolute -top-2 bottom-8 left-4 z-0 w-0.5 -translate-x-1/2 bg-secondary-foreground" />
<div
className="absolute left-4 top-2 z-[5] max-h-[calc(100%-3rem)] w-0.5 -translate-x-1/2 bg-selected transition-all duration-300"
style={{ height: `${blueLineHeight}%` }}
/>
<div className="space-y-2">
{eventSequence.map((item, idx) => {
const isActive =
Math.abs((timeIndex ?? 0) - (item.timestamp ?? 0)) <= 0.5;
const formattedEventTimestamp = config
? formatUnixTimestampToDateTime(item.timestamp ?? 0, {
timezone: config.ui.timezone,
date_format:
config.ui.time_format == "24hour"
? t(
"time.formattedTimestampHourMinuteSecond.24hour",
{ ns: "common" },
)
: t(
"time.formattedTimestampHourMinuteSecond.12hour",
{ ns: "common" },
),
time_style: "medium",
date_style: "medium",
})
: "";
const ratio =
Array.isArray(item.data.box) && item.data.box.length >= 4
? (
aspectRatio *
(item.data.box[2] / item.data.box[3])
).toFixed(2)
: "N/A";
const areaPx =
Array.isArray(item.data.box) && item.data.box.length >= 4
? Math.round(
(config.cameras[event.camera]?.detect?.width ?? 0) *
(config.cameras[event.camera]?.detect?.height ??
0) *
(item.data.box[2] * item.data.box[3]),
)
: undefined;
const areaPct =
Array.isArray(item.data.box) && item.data.box.length >= 4
? (item.data.box[2] * item.data.box[3]).toFixed(4)
: undefined;
const ratio =
Array.isArray(item.data.box) && item.data.box.length >= 4
? (
aspectRatio *
(item.data.box[2] / item.data.box[3])
).toFixed(2)
: "N/A";
const areaPx =
Array.isArray(item.data.box) && item.data.box.length >= 4
? Math.round(
(config.cameras[event.camera]?.detect?.width ?? 0) *
(config.cameras[event.camera]?.detect?.height ??
0) *
(item.data.box[2] * item.data.box[3]),
)
: undefined;
const areaPct =
Array.isArray(item.data.box) && item.data.box.length >= 4
? (item.data.box[2] * item.data.box[3]).toFixed(4)
: undefined;
return (
<div
key={`${item.timestamp}-${item.source_id ?? ""}-${idx}`}
role="button"
onClick={() => {
setTimeIndex(item.timestamp ?? 0);
handleSetBox(
item.data.box ?? [],
item.data.attribute_box,
);
setLifecycleZones(item.data.zones);
setSelectedZone("");
}}
className={cn(
"flex cursor-pointer flex-col gap-1 rounded-md p-2 text-sm text-primary-variant",
isActive
? "bg-secondary-highlight font-semibold text-primary outline-[1.5px] -outline-offset-[1.1px] outline-primary/40 dark:font-normal"
: "duration-500",
)}
>
<div className="flex items-center gap-2">
<div className="flex size-7 items-center justify-center">
<LifecycleIcon
lifecycleItem={item}
className="size-5"
/>
</div>
<div className="flex w-full flex-row justify-between">
<div>{getLifecycleItemDescription(item)}</div>
<div className={cn("p-1 text-sm")}>
{formattedEventTimestamp}
</div>
</div>
</div>
<div className="ml-8 mt-1 flex flex-wrap items-center gap-3 text-sm text-secondary-foreground">
<div className="flex flex-col gap-1">
<div className="flex items-center gap-1">
<span className="text-muted-foreground">
{t(
"objectLifecycle.lifecycleItemDesc.header.ratio",
)}
</span>
<span className="font-medium text-foreground">
{ratio}
</span>
</div>
<div className="flex items-center gap-1">
<span className="text-muted-foreground">
{t(
"objectLifecycle.lifecycleItemDesc.header.area",
)}
</span>
{areaPx !== undefined && areaPct !== undefined ? (
<span className="font-medium text-foreground">
px: {areaPx} · %: {areaPct}
</span>
) : (
<span>N/A</span>
)}
</div>
{item.class_type === "entered_zone" && (
<div className="flex items-center gap-2">
<span className="text-muted-foreground">
{t(
"objectLifecycle.lifecycleItemDesc.header.zones",
)}
</span>
<div className="flex flex-wrap items-center gap-2">
{item.data.zones.map((zone, zidx) => (
<div
key={`${zone}-${zidx}`}
className="flex cursor-pointer items-center gap-1"
onClick={(e) => {
e.stopPropagation();
setSelectedZone(zone);
}}
>
<div
className="size-3 rounded"
style={{
backgroundColor: `rgb(${getZoneColor(zone)})`,
}}
/>
<span className="smart-capitalize">
{zone.replaceAll("_", " ")}
</span>
</div>
))}
</div>
</div>
)}
</div>
</div>
</div>
);
})}
return (
<LifecycleIconRow
key={`${item.timestamp}-${item.source_id ?? ""}-${idx}`}
item={item}
isActive={isActive}
formattedEventTimestamp={formattedEventTimestamp}
ratio={ratio}
areaPx={areaPx}
areaPct={areaPct}
onClick={() => {
setTimeIndex(item.timestamp ?? 0);
handleSetBox(
item.data.box ?? [],
item.data.attribute_box,
);
setLifecycleZones(item.data.zones);
setSelectedZone("");
}}
setSelectedZone={setSelectedZone}
getZoneColor={getZoneColor}
/>
);
})}
</div>
</div>
)}
</div>
@ -789,3 +770,117 @@ export function LifecycleIcon({
return null;
}
}
type LifecycleIconRowProps = {
item: ObjectLifecycleSequence;
isActive?: boolean;
formattedEventTimestamp: string;
ratio: string;
areaPx?: number;
areaPct?: string;
onClick: () => void;
setSelectedZone: (z: string) => void;
getZoneColor: (zoneName: string) => number[] | undefined;
};
function LifecycleIconRow({
item,
isActive,
formattedEventTimestamp,
ratio,
areaPx,
areaPct,
onClick,
setSelectedZone,
getZoneColor,
}: LifecycleIconRowProps) {
const { t } = useTranslation(["views/explore"]);
return (
<div
role="button"
onClick={onClick}
className={cn(
"rounded-md p-2 text-sm text-primary-variant",
isActive && "bg-secondary-highlight font-semibold text-primary",
!isActive && "duration-500",
)}
>
<div className="flex items-center gap-2">
<div className="relative flex size-4 items-center justify-center">
<LuCircle
className={cn(
"relative z-10 ml-[1px] size-2.5 fill-secondary-foreground stroke-none",
isActive && "fill-selected duration-300",
)}
/>
</div>
<div className="flex w-full flex-row justify-between">
<div className="flex flex-col">
<div>{getLifecycleItemDescription(item)}</div>
<div className="mt-1 flex flex-wrap items-center gap-2 text-sm text-secondary-foreground md:gap-5">
<div className="flex items-center gap-1">
<span className="text-primary-variant">
{t("objectLifecycle.lifecycleItemDesc.header.ratio")}
</span>
<span className="font-medium text-primary">{ratio}</span>
</div>
<div className="flex items-center gap-1">
<span className="text-primary-variant">
{t("objectLifecycle.lifecycleItemDesc.header.area")}
</span>
{areaPx !== undefined && areaPct !== undefined ? (
<span className="font-medium text-primary">
{t("information.pixels", { ns: "common", area: areaPx })} ·{" "}
{areaPct}%
</span>
) : (
<span>N/A</span>
)}
</div>
{item.data?.zones && item.data.zones.length > 0 && (
<div className="flex flex-wrap items-center gap-2">
{item.data.zones.map((zone, zidx) => {
const color = getZoneColor(zone)?.join(",") ?? "0,0,0";
return (
<Badge
key={`${zone}-${zidx}`}
variant="outline"
className="inline-flex cursor-pointer items-center gap-2"
onClick={(e: React.MouseEvent) => {
e.stopPropagation();
setSelectedZone(zone);
}}
style={{
borderColor: `rgba(${color}, 0.6)`,
background: `rgba(${color}, 0.08)`,
}}
>
<span
className="size-1 rounded-full"
style={{
display: "inline-block",
width: 10,
height: 10,
backgroundColor: `rgb(${color})`,
}}
/>
<span className="smart-capitalize">
{zone.replaceAll("_", " ")}
</span>
</Badge>
);
})}
</div>
)}
</div>
</div>
<div className={cn("p-1 text-sm")}>{formattedEventTimestamp}</div>
</div>
</div>
</div>
);
}

View File

@ -20,6 +20,8 @@ import type {
ConfigSetBody,
} from "@/types/cameraWizard";
import { processCameraName } from "@/utils/cameraUtil";
import { isDesktop } from "react-device-detect";
import { cn } from "@/lib/utils";
type WizardState = {
wizardData: Partial<WizardFormData>;
@ -335,7 +337,15 @@ export default function CameraWizardDialog({
return (
<Dialog open={open} onOpenChange={handleClose}>
<DialogContent
className="max-h-[90dvh] max-w-4xl overflow-y-auto"
className={cn(
"max-h-[90dvh] max-w-xl overflow-y-auto",
isDesktop &&
currentStep == 0 &&
state.wizardData?.streams?.[0]?.testResult?.snapshot &&
"max-w-4xl",
isDesktop && currentStep == 1 && "max-w-2xl",
isDesktop && currentStep > 1 && "max-w-4xl",
)}
onInteractOutside={(e) => {
e.preventDefault();
}}

View File

@ -6,7 +6,6 @@ import {
FormItem,
FormLabel,
FormMessage,
FormDescription,
} from "@/components/ui/form";
import { Input } from "@/components/ui/input";
import {
@ -65,6 +64,7 @@ export default function Step1NameCamera({
const { data: config } = useSWR<FrigateConfig>("config");
const [showPassword, setShowPassword] = useState(false);
const [isTesting, setIsTesting] = useState(false);
const [testStatus, setTestStatus] = useState<string>("");
const [testResult, setTestResult] = useState<TestResult | null>(null);
const existingCameraNames = useMemo(() => {
@ -88,7 +88,13 @@ export default function Step1NameCamera({
username: z.string().optional(),
password: z.string().optional(),
brandTemplate: z.enum(CAMERA_BRAND_VALUES).optional(),
customUrl: z.string().optional(),
customUrl: z
.string()
.optional()
.refine(
(val) => !val || val.startsWith("rtsp://"),
t("cameraWizard.step1.errors.customUrlRtspRequired"),
),
})
.refine(
(data) => {
@ -204,24 +210,17 @@ export default function Step1NameCamera({
}
setIsTesting(true);
setTestStatus("");
setTestResult(null);
// First get probe data for metadata
const probePromise = axios.get("ffprobe", {
params: { paths: streamUrl, detailed: true },
timeout: 10000,
});
// Then get snapshot for preview
const snapshotPromise = axios.get("ffprobe/snapshot", {
params: { url: streamUrl },
responseType: "blob",
timeout: 10000,
});
try {
// First get probe data for metadata
const probeResponse = await probePromise;
setTestStatus(t("cameraWizard.step1.testing.probingMetadata"));
const probeResponse = await axios.get("ffprobe", {
params: { paths: streamUrl, detailed: true },
timeout: 10000,
});
let probeData = null;
if (
probeResponse.data &&
@ -234,8 +233,13 @@ export default function Step1NameCamera({
// Then get snapshot for preview (only if probe succeeded)
let snapshotBlob = null;
if (probeData) {
setTestStatus(t("cameraWizard.step1.testing.fetchingSnapshot"));
try {
const snapshotResponse = await snapshotPromise;
const snapshotResponse = await axios.get("ffprobe/snapshot", {
params: { url: streamUrl },
responseType: "blob",
timeout: 10000,
});
snapshotBlob = snapshotResponse.data;
} catch (snapshotError) {
// Snapshot is optional, don't fail if it doesn't work
@ -293,14 +297,21 @@ export default function Step1NameCamera({
};
setTestResult(testResult);
onUpdate({ streams: [{ id: "", url: "", roles: [], testResult }] });
toast.success(t("cameraWizard.step1.testSuccess"));
} else {
const error = probeData?.stderr || "Unknown error";
const error =
Array.isArray(probeResponse.data?.[0]?.stderr) &&
probeResponse.data[0].stderr.length > 0
? probeResponse.data[0].stderr.join("\n")
: "Unable to probe stream";
setTestResult({
success: false,
error: error,
});
toast.error(t("cameraWizard.commonErrors.testFailed", { error }));
toast.error(t("cameraWizard.commonErrors.testFailed", { error }), {
duration: 6000,
});
}
} catch (error) {
const axiosError = error as {
@ -318,11 +329,15 @@ export default function Step1NameCamera({
});
toast.error(
t("cameraWizard.commonErrors.testFailed", { error: errorMessage }),
{
duration: 10000,
},
);
} finally {
setIsTesting(false);
setTestStatus("");
}
}, [form, generateStreamUrl, t]);
}, [form, generateStreamUrl, t, onUpdate]);
const onSubmit = (data: z.infer<typeof step1FormData>) => {
onUpdate(data);
@ -365,7 +380,9 @@ export default function Step1NameCamera({
name="cameraName"
render={({ field }) => (
<FormItem>
<FormLabel>{t("cameraWizard.step1.cameraName")}</FormLabel>
<FormLabel className="text-primary-variant">
{t("cameraWizard.step1.cameraName")}
</FormLabel>
<FormControl>
<Input
className="h-8"
@ -385,7 +402,43 @@ export default function Step1NameCamera({
name="brandTemplate"
render={({ field }) => (
<FormItem>
<FormLabel>{t("cameraWizard.step1.cameraBrand")}</FormLabel>
<div className="flex items-center gap-1 pb-1">
<FormLabel className="text-primary-variant">
{t("cameraWizard.step1.cameraBrand")}
</FormLabel>
{field.value &&
(() => {
const selectedBrand = CAMERA_BRANDS.find(
(brand) => brand.value === field.value,
);
return selectedBrand &&
selectedBrand.value != "other" ? (
<Popover>
<PopoverTrigger asChild>
<Button
variant="ghost"
size="sm"
className="h-4 w-4 p-0"
>
<LuInfo className="size-3" />
</Button>
</PopoverTrigger>
<PopoverContent className="pointer-events-auto w-80 text-primary-variant">
<div className="space-y-2">
<h4 className="font-medium">
{selectedBrand.label}
</h4>
<p className="break-all text-sm text-muted-foreground">
{t("cameraWizard.step1.brandUrlFormat", {
exampleUrl: selectedBrand.exampleUrl,
})}
</p>
</div>
</PopoverContent>
</Popover>
) : null;
})()}
</div>
<Select
onValueChange={field.onChange}
defaultValue={field.value}
@ -406,37 +459,6 @@ export default function Step1NameCamera({
</SelectContent>
</Select>
<FormMessage />
{field.value &&
(() => {
const selectedBrand = CAMERA_BRANDS.find(
(brand) => brand.value === field.value,
);
return selectedBrand &&
selectedBrand.value != "other" ? (
<FormDescription className="mt-1 pt-0.5 text-xs text-muted-foreground">
<Popover>
<PopoverTrigger>
<div className="flex flex-row items-center gap-0.5 text-xs text-muted-foreground hover:text-primary">
<LuInfo className="mr-1 size-3" />
{t("cameraWizard.step1.brandInformation")}
</div>
</PopoverTrigger>
<PopoverContent className="w-80">
<div className="space-y-2">
<h4 className="font-medium">
{selectedBrand.label}
</h4>
<p className="break-all text-sm text-muted-foreground">
{t("cameraWizard.step1.brandUrlFormat", {
exampleUrl: selectedBrand.exampleUrl,
})}
</p>
</div>
</PopoverContent>
</Popover>
</FormDescription>
) : null;
})()}
</FormItem>
)}
/>
@ -448,7 +470,9 @@ export default function Step1NameCamera({
name="host"
render={({ field }) => (
<FormItem>
<FormLabel>{t("cameraWizard.step1.host")}</FormLabel>
<FormLabel className="text-primary-variant">
{t("cameraWizard.step1.host")}
</FormLabel>
<FormControl>
<Input
className="h-8"
@ -466,7 +490,7 @@ export default function Step1NameCamera({
name="username"
render={({ field }) => (
<FormItem>
<FormLabel>
<FormLabel className="text-primary-variant">
{t("cameraWizard.step1.username")}
</FormLabel>
<FormControl>
@ -488,7 +512,7 @@ export default function Step1NameCamera({
name="password"
render={({ field }) => (
<FormItem>
<FormLabel>
<FormLabel className="text-primary-variant">
{t("cameraWizard.step1.password")}
</FormLabel>
<FormControl>
@ -529,7 +553,9 @@ export default function Step1NameCamera({
name="customUrl"
render={({ field }) => (
<FormItem>
<FormLabel>{t("cameraWizard.step1.customUrl")}</FormLabel>
<FormLabel className="text-primary-variant">
{t("cameraWizard.step1.customUrl")}
</FormLabel>
<FormControl>
<Input
className="h-8"
@ -610,7 +636,9 @@ export default function Step1NameCamera({
className="flex items-center justify-center gap-2 sm:flex-1"
>
{isTesting && <ActivityIndicator className="size-4" />}
{t("cameraWizard.step1.testConnection")}
{isTesting && testStatus
? testStatus
: t("cameraWizard.step1.testConnection")}
</Button>
)}
</div>

View File

@ -151,9 +151,9 @@ export default function Step2StreamConfig({
? `${videoStream.width}x${videoStream.height}`
: undefined;
const fps = videoStream?.r_frame_rate
? parseFloat(videoStream.r_frame_rate.split("/")[0]) /
parseFloat(videoStream.r_frame_rate.split("/")[1])
const fps = videoStream?.avg_frame_rate
? parseFloat(videoStream.avg_frame_rate.split("/")[0]) /
parseFloat(videoStream.avg_frame_rate.split("/")[1])
: undefined;
const testResult: TestResult = {
@ -277,7 +277,7 @@ export default function Step2StreamConfig({
<div className="grid grid-cols-1 gap-4">
<div className="space-y-2">
<label className="text-sm font-medium">
<label className="text-sm font-medium text-primary-variant">
{t("cameraWizard.step2.url")}
</label>
<div className="flex flex-row items-center gap-2">
@ -325,7 +325,7 @@ export default function Step2StreamConfig({
<div className="space-y-2">
<div className="flex items-center gap-1">
<Label className="text-sm font-medium">
<Label className="text-sm font-medium text-primary-variant">
{t("cameraWizard.step2.roles")}
</Label>
<Popover>
@ -334,7 +334,7 @@ export default function Step2StreamConfig({
<LuInfo className="size-3" />
</Button>
</PopoverTrigger>
<PopoverContent className="w-80 text-xs">
<PopoverContent className="pointer-events-auto w-80 text-xs">
<div className="space-y-2">
<div className="font-medium">
{t("cameraWizard.step2.rolesPopover.title")}
@ -395,7 +395,7 @@ export default function Step2StreamConfig({
<div className="space-y-2">
<div className="flex items-center gap-1">
<Label className="text-sm font-medium">
<Label className="text-sm font-medium text-primary-variant">
{t("cameraWizard.step2.featuresTitle")}
</Label>
<Popover>
@ -404,7 +404,7 @@ export default function Step2StreamConfig({
<LuInfo className="size-3" />
</Button>
</PopoverTrigger>
<PopoverContent className="w-80 text-xs">
<PopoverContent className="pointer-events-auto w-80 text-xs">
<div className="space-y-2">
<div className="font-medium">
{t("cameraWizard.step2.featuresPopover.title")}

View File

@ -85,9 +85,9 @@ export default function Step3Validation({
? `${videoStream.width}x${videoStream.height}`
: undefined;
const fps = videoStream?.r_frame_rate
? parseFloat(videoStream.r_frame_rate.split("/")[0]) /
parseFloat(videoStream.r_frame_rate.split("/")[1])
const fps = videoStream?.avg_frame_rate
? parseFloat(videoStream.avg_frame_rate.split("/")[0]) /
parseFloat(videoStream.avg_frame_rate.split("/")[1])
: undefined;
return {
@ -323,7 +323,7 @@ export default function Step3Validation({
)}
<div className="mb-2 flex flex-col justify-between gap-1 md:flex-row md:items-center">
<span className="text-sm text-muted-foreground">
<span className="break-all text-sm text-muted-foreground">
{stream.url}
</span>
<Button

View File

@ -1,13 +1,12 @@
import { useEffect, useMemo, useRef, useState } from "react";
import { ObjectLifecycleSequence } from "@/types/timeline";
import { LifecycleIcon } from "@/components/overlay/detail/ObjectLifecycle";
import { getLifecycleItemDescription } from "@/utils/lifecycleUtil";
import { useDetailStream } from "@/context/detail-stream-context";
import scrollIntoView from "scroll-into-view-if-needed";
import useUserInteraction from "@/hooks/use-user-interaction";
import {
formatUnixTimestampToDateTime,
formatSecondsToDuration,
getDurationFromTimestamps,
} from "@/utils/dateUtil";
import { useTranslation } from "react-i18next";
import AnnotationOffsetSlider from "@/components/overlay/detail/AnnotationOffsetSlider";
@ -17,26 +16,24 @@ import ActivityIndicator from "../indicators/activity-indicator";
import { Event } from "@/types/event";
import { getIconForLabel } from "@/utils/iconUtil";
import { ReviewSegment } from "@/types/review";
import {
Collapsible,
CollapsibleTrigger,
CollapsibleContent,
} from "@/components/ui/collapsible";
import { LuChevronUp, LuChevronDown } from "react-icons/lu";
import { LuChevronDown, LuCircle, LuChevronRight } from "react-icons/lu";
import { getTranslatedLabel } from "@/utils/i18n";
import EventMenu from "@/components/timeline/EventMenu";
import { FrigatePlusDialog } from "@/components/overlay/dialog/FrigatePlusDialog";
import { cn } from "@/lib/utils";
import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip";
type DetailStreamProps = {
reviewItems?: ReviewSegment[];
currentTime: number;
isPlaying?: boolean;
onSeek: (timestamp: number, play?: boolean) => void;
};
export default function DetailStream({
reviewItems,
currentTime,
isPlaying = false,
onSeek,
}: DetailStreamProps) {
const { data: config } = useSWR<FrigateConfig>("config");
@ -54,6 +51,10 @@ export default function DetailStream({
const effectiveTime = currentTime + annotationOffset / 1000;
const [upload, setUpload] = useState<Event | undefined>(undefined);
const onSeekCheckPlaying = (timestamp: number) => {
onSeek(timestamp, isPlaying);
};
// Ensure we initialize the active review when reviewItems first arrive.
// This helps when the component mounts while the video is already
// playing — it guarantees the matching review is highlighted right
@ -89,7 +90,7 @@ export default function DetailStream({
// Auto-scroll to current time
useEffect(() => {
if (!scrollRef.current || userInteracting) return;
if (!scrollRef.current || userInteracting || !isPlaying) return;
// Prefer the review whose range contains the effectiveTime. If none
// contains it, pick the nearest review (by mid-point distance). This is
// robust to unordered reviewItems and avoids always picking the last
@ -121,11 +122,20 @@ export default function DetailStream({
`[data-review-id="${id}"]`,
) as HTMLElement;
if (element) {
setProgrammaticScroll();
scrollIntoView(element, {
scrollMode: "if-needed",
behavior: "smooth",
});
// Only scroll if element is completely out of view
const containerRect = scrollRef.current.getBoundingClientRect();
const elementRect = element.getBoundingClientRect();
const isFullyInvisible =
elementRect.bottom < containerRect.top ||
elementRect.top > containerRect.bottom;
if (isFullyInvisible) {
setProgrammaticScroll();
scrollIntoView(element, {
scrollMode: "if-needed",
behavior: "smooth",
});
}
}
}
}, [
@ -134,6 +144,7 @@ export default function DetailStream({
annotationOffset,
userInteracting,
setProgrammaticScroll,
isPlaying,
]);
// Auto-select active review based on effectiveTime (if inside a review range)
@ -165,9 +176,9 @@ export default function DetailStream({
<div
ref={scrollRef}
className="scrollbar-container h-[calc(100vh-70px)] overflow-y-auto bg-secondary"
className="scrollbar-container h-[calc(100vh-70px)] overflow-y-auto"
>
<div className="space-y-2 p-4">
<div className="space-y-4 py-2">
{reviewItems?.length === 0 ? (
<div className="py-8 text-center text-muted-foreground">
{t("detail.noDataFound")}
@ -181,7 +192,7 @@ export default function DetailStream({
id={id}
review={review}
config={config}
onSeek={onSeek}
onSeek={onSeekCheckPlaying}
effectiveTime={effectiveTime}
isActive={activeReviewId == id}
onActivate={() => setActiveReviewId(id)}
@ -220,6 +231,7 @@ function ReviewGroup({
effectiveTime,
}: ReviewGroupProps) {
const { t } = useTranslation("views/events");
const [open, setOpen] = useState(false);
const start = review.start_time ?? 0;
const displayTime = formatUnixTimestampToDateTime(start, {
@ -234,7 +246,7 @@ function ReviewGroup({
const shouldFetchEvents = review?.data?.detections?.length > 0;
const { data: fetchedEvents } = useSWR<Event[]>(
const { data: fetchedEvents, isValidating } = useSWR<Event[]>(
shouldFetchEvents
? ["event_ids", { ids: review.data.detections.join(",") }]
: null,
@ -259,74 +271,118 @@ function ReviewGroup({
}
const reviewInfo = useMemo(() => {
if (review.data.metadata?.title) {
return review.data.metadata.title;
} else {
const objectCount = fetchedEvents
? fetchedEvents.length
: (review.data.objects ?? []).length;
const objectCount = fetchedEvents
? fetchedEvents.length
: (review.data.objects ?? []).length;
return `${objectCount} ${t("detail.trackedObject", { count: objectCount })}`;
}
return `${objectCount} ${t("detail.trackedObject", { count: objectCount })}`;
}, [review, t, fetchedEvents]);
const reviewDuration =
review.end_time != null
? formatSecondsToDuration(
Math.max(0, Math.floor((review.end_time ?? 0) - start)),
)
: null;
const reviewDuration = useMemo(
() =>
getDurationFromTimestamps(
review.start_time,
review.end_time ?? null,
true,
),
[review.start_time, review.end_time],
);
return (
<div
data-review-id={id}
className={`cursor-pointer rounded-lg border bg-background p-3 outline outline-[3px] -outline-offset-[2.8px] ${
isActive
? "shadow-selected outline-selected"
: "outline-transparent duration-500"
}`}
className="cursor-pointer rounded-lg bg-secondary py-3"
>
<div
className="flex items-center justify-between"
className={cn(
"flex items-start",
open && "border-b border-secondary-highlight pb-4",
)}
onClick={() => {
onActivate?.();
onSeek(start);
}}
>
<div className="flex items-center gap-2">
<div className="flex flex-col">
<div className="text-sm font-medium">{displayTime}</div>
{reviewDuration && (
<div className="text-xs text-muted-foreground">
{reviewDuration}
</div>
<div className="ml-4 mr-2 mt-1.5 flex flex-row items-start">
<LuCircle
className={cn(
"size-3",
isActive
? "fill-selected text-selected"
: "fill-muted duration-500 dark:fill-secondary-highlight dark:text-secondary-highlight",
)}
<div className="text-xs text-muted-foreground">{reviewInfo}</div>
</div>
/>
</div>
<div className="flex items-center gap-2">
{iconLabels.slice(0, 5).map((lbl, idx) => (
<span key={`${lbl}-${idx}`}>
{getIconForLabel(lbl, "size-4 text-primary dark:text-white")}
</span>
))}
<div className="mr-3 flex w-full justify-between">
<div className="ml-1 flex flex-col items-start gap-1.5">
<div className="flex flex-row gap-3">
<div className="text-sm font-medium">{displayTime}</div>
<div className="flex items-center gap-2">
{iconLabels.slice(0, 5).map((lbl, idx) => (
<div
key={`${lbl}-${idx}`}
className="rounded-full bg-muted-foreground p-1"
>
{getIconForLabel(lbl, "size-3 text-white")}
</div>
))}
</div>
</div>
<div className="flex flex-col gap-0.5">
{review.data.metadata?.title && (
<div className="mb-1 text-sm text-primary-variant">
{review.data.metadata.title}
</div>
)}
<div className="flex flex-row items-center gap-1.5">
<div className="text-xs text-primary-variant">{reviewInfo}</div>
{reviewDuration && (
<>
<span className="text-[5px] text-primary-variant"></span>
<div className="text-xs text-primary-variant">
{reviewDuration}
</div>
</>
)}
</div>
</div>
</div>
<div
onClick={(e) => {
e.stopPropagation();
setOpen((v) => !v);
}}
className="ml-2 inline-flex items-center justify-center rounded p-1 hover:bg-secondary/10"
>
{open ? (
<LuChevronDown className="size-4 text-primary-variant" />
) : (
<LuChevronRight className="size-4 text-primary-variant" />
)}
</div>
</div>
</div>
{isActive && (
<div className="mt-2 space-y-2">
{shouldFetchEvents && !fetchedEvents ? (
{open && (
<div className="space-y-0.5">
{shouldFetchEvents && isValidating && !fetchedEvents ? (
<ActivityIndicator />
) : (
(fetchedEvents || []).map((event) => {
(fetchedEvents || []).map((event, index) => {
return (
<EventCollapsible
key={event.id}
event={event}
effectiveTime={effectiveTime}
onSeek={onSeek}
onOpenUpload={onOpenUpload}
/>
<div
key={`event-${event.id}-${index}`}
className="border-b border-secondary-highlight pb-0.5 last:border-0 last:pb-0"
>
<EventList
key={event.id}
event={event}
effectiveTime={effectiveTime}
onSeek={onSeek}
onOpenUpload={onOpenUpload}
/>
</div>
);
})
)}
@ -337,11 +393,10 @@ function ReviewGroup({
key={audioLabel}
className="rounded-md bg-secondary p-2 outline outline-[3px] -outline-offset-[2.8px] outline-transparent duration-500"
>
<div className="flex items-center gap-2 text-sm font-medium">
{getIconForLabel(
audioLabel,
"size-4 text-primary dark:text-white",
)}
<div className="ml-1.5 flex items-center gap-2 text-sm font-medium">
<div className="rounded-full bg-muted-foreground p-1">
{getIconForLabel(audioLabel, "size-3 text-white")}
</div>
<span>{getTranslatedLabel(audioLabel)}</span>
</div>
</div>
@ -354,55 +409,30 @@ function ReviewGroup({
);
}
type EventCollapsibleProps = {
type EventListProps = {
event: Event;
effectiveTime?: number;
onSeek: (ts: number, play?: boolean) => void;
onOpenUpload?: (e: Event) => void;
};
function EventCollapsible({
function EventList({
event,
effectiveTime,
onSeek,
onOpenUpload,
}: EventCollapsibleProps) {
const [open, setOpen] = useState(false);
const { t } = useTranslation("views/events");
}: EventListProps) {
const { data: config } = useSWR<FrigateConfig>("config");
const { selectedObjectId, setSelectedObjectId } = useDetailStream();
const formattedStart = config
? formatUnixTimestampToDateTime(event.start_time ?? 0, {
timezone: config.ui.timezone,
date_format:
config.ui.time_format == "24hour"
? t("time.formattedTimestampHourMinuteSecond.24hour", {
ns: "common",
})
: t("time.formattedTimestampHourMinuteSecond.12hour", {
ns: "common",
}),
time_style: "medium",
date_style: "medium",
})
: "";
const formattedEnd = config
? formatUnixTimestampToDateTime(event.end_time ?? 0, {
timezone: config.ui.timezone,
date_format:
config.ui.time_format == "24hour"
? t("time.formattedTimestampHourMinuteSecond.24hour", {
ns: "common",
})
: t("time.formattedTimestampHourMinuteSecond.12hour", {
ns: "common",
}),
time_style: "medium",
date_style: "medium",
})
: "";
const handleObjectSelect = (event: Event | undefined) => {
if (event) {
onSeek(event.start_time ?? 0);
setSelectedObjectId(event.id);
} else {
setSelectedObjectId(undefined);
}
};
// Clear selectedObjectId when effectiveTime has passed this event's end_time
useEffect(() => {
@ -420,91 +450,97 @@ function EventCollapsible({
]);
return (
<Collapsible open={open} onOpenChange={(o) => setOpen(o)}>
<>
<div
className={cn(
"rounded-md bg-secondary p-2 outline outline-[3px] -outline-offset-[2.8px]",
"rounded-md bg-secondary p-2",
event.id == selectedObjectId
? "shadow-selected outline-selected"
? "bg-secondary-highlight"
: "outline-transparent duration-500",
event.id != selectedObjectId &&
(effectiveTime ?? 0) >= (event.start_time ?? 0) - 0.5 &&
(effectiveTime ?? 0) <=
(event.end_time ?? event.start_time ?? 0) + 0.5 &&
"bg-secondary-highlight outline-[1.5px] -outline-offset-[1.1px] outline-primary/40",
"bg-secondary-highlight",
)}
>
<div className="flex w-full items-center justify-between">
<div className="ml-1.5 flex w-full items-center justify-between">
<div
className="flex items-center gap-2 text-sm font-medium"
onClick={(e) => {
e.stopPropagation();
onSeek(event.start_time ?? 0);
if (event.id) setSelectedObjectId(event.id);
handleObjectSelect(
event.id == selectedObjectId ? undefined : event,
);
}}
role="button"
>
{getIconForLabel(
event.label,
"size-4 text-primary dark:text-white",
)}
<div
className={cn(
"rounded-full p-1",
event.id == selectedObjectId
? "bg-selected"
: "bg-muted-foreground",
)}
>
{getIconForLabel(event.label, "size-3 text-white")}
</div>
<div className="flex items-end gap-2">
<span>{getTranslatedLabel(event.label)}</span>
<span className="text-xs text-secondary-foreground">
{formattedStart ?? ""} - {formattedEnd ?? ""}
</span>
</div>
</div>
<div className="flex flex-1 flex-row justify-end">
<div className="mr-2 flex flex-1 flex-row justify-end">
<EventMenu
event={event}
config={config}
onOpenUpload={(e) => onOpenUpload?.(e)}
selectedObjectId={selectedObjectId}
setSelectedObjectId={handleObjectSelect}
/>
</div>
<div className="flex items-center gap-2">
<CollapsibleTrigger asChild>
<button
onClick={(e) => e.stopPropagation()}
className="rounded bg-muted px-2 py-1 text-xs"
aria-label={t("detail.aria")}
>
{open ? (
<LuChevronUp className="size-3" />
) : (
<LuChevronDown className="size-3" />
)}
</button>
</CollapsibleTrigger>
</div>
</div>
<CollapsibleContent>
<div className="mt-2">
<ObjectTimeline
eventId={event.id}
onSeek={onSeek}
effectiveTime={effectiveTime}
/>
</div>
</CollapsibleContent>
<div className="mt-2">
<ObjectTimeline
eventId={event.id}
onSeek={onSeek}
effectiveTime={effectiveTime}
/>
</div>
</div>
</Collapsible>
</>
);
}
type LifecycleItemProps = {
event: ObjectLifecycleSequence;
item: ObjectLifecycleSequence;
isActive?: boolean;
onSeek?: (timestamp: number, play?: boolean) => void;
effectiveTime?: number;
};
function LifecycleItem({ event, isActive, onSeek }: LifecycleItemProps) {
function LifecycleItem({
item,
isActive,
onSeek,
effectiveTime,
}: LifecycleItemProps) {
const { t } = useTranslation("views/events");
const { data: config } = useSWR<FrigateConfig>("config");
const aspectRatio = useMemo(() => {
if (!config || !item?.camera) {
return 16 / 9;
}
return (
config.cameras[item.camera].detect.width /
config.cameras[item.camera].detect.height
);
}, [config, item]);
const formattedEventTimestamp = config
? formatUnixTimestampToDateTime(event.timestamp ?? 0, {
? formatUnixTimestampToDateTime(item?.timestamp ?? 0, {
timezone: config.ui.timezone,
date_format:
config.ui.time_format == "24hour"
@ -519,11 +555,28 @@ function LifecycleItem({ event, isActive, onSeek }: LifecycleItemProps) {
})
: "";
const ratio =
Array.isArray(item?.data.box) && item?.data.box.length >= 4
? (aspectRatio * (item?.data.box[2] / item?.data.box[3])).toFixed(2)
: "N/A";
const areaPx =
Array.isArray(item?.data.box) && item?.data.box.length >= 4
? Math.round(
(config?.cameras[item?.camera]?.detect?.width ?? 0) *
(config?.cameras[item?.camera]?.detect?.height ?? 0) *
(item?.data.box[2] * item?.data.box[3]),
)
: undefined;
const areaPct =
Array.isArray(item?.data.box) && item?.data.box.length >= 4
? (item?.data.box[2] * item?.data.box[3]).toFixed(4)
: undefined;
return (
<div
role="button"
onClick={() => {
onSeek?.(event.timestamp ?? 0, false);
onSeek?.(item.timestamp ?? 0, false);
}}
className={cn(
"flex cursor-pointer items-center gap-2 text-sm text-primary-variant",
@ -532,11 +585,48 @@ function LifecycleItem({ event, isActive, onSeek }: LifecycleItemProps) {
: "duration-500",
)}
>
<div className="flex size-4 items-center justify-center">
<LifecycleIcon lifecycleItem={event} className="size-3" />
<div className="relative flex size-4 items-center justify-center">
<LuCircle
className={cn(
"relative z-10 ml-[1px] size-2.5 fill-secondary-foreground stroke-none",
(isActive || (effectiveTime ?? 0) >= (item?.timestamp ?? 0)) &&
"fill-selected duration-300",
)}
/>
</div>
<div className="flex w-full flex-row justify-between">
<div>{getLifecycleItemDescription(event)}</div>
<Tooltip>
<TooltipTrigger>
<div className="flex items-start text-left">
{getLifecycleItemDescription(item)}
</div>
</TooltipTrigger>
<TooltipContent>
<div className="mt-1 flex flex-wrap items-start gap-3 text-sm text-secondary-foreground">
<div className="flex flex-col gap-1">
<div className="flex items-start gap-1">
<span className="text-muted-foreground">
{t("objectLifecycle.lifecycleItemDesc.header.ratio")}
</span>
<span className="font-medium text-foreground">{ratio}</span>
</div>
<div className="flex items-start gap-1">
<span className="text-muted-foreground">
{t("objectLifecycle.lifecycleItemDesc.header.area")}
</span>
{areaPx !== undefined && areaPct !== undefined ? (
<span className="font-medium text-foreground">
{areaPx} {t("pixels", { ns: "common" })} · {areaPct}%
</span>
) : (
<span>N/A</span>
)}
</div>
</div>
</div>
</TooltipContent>
</Tooltip>
<div className={cn("p-1 text-xs")}>{formattedEventTimestamp}</div>
</div>
</div>
@ -561,8 +651,8 @@ function ObjectTimeline({
},
]);
if ((!timeline || timeline.length === 0) && isValidating) {
return <ActivityIndicator className="h-2 w-2" size={2} />;
if (isValidating && (!timeline || timeline.length === 0)) {
return <ActivityIndicator className="ml-2 size-3" />;
}
if (!timeline || timeline.length === 0) {
@ -573,20 +663,75 @@ function ObjectTimeline({
);
}
// Calculate how far down the blue line should extend based on effectiveTime
const calculateLineHeight = () => {
if (!timeline || timeline.length === 0) return 0;
const currentTime = effectiveTime ?? 0;
// Find which events have been passed
let lastPassedIndex = -1;
for (let i = 0; i < timeline.length; i++) {
if (currentTime >= (timeline[i].timestamp ?? 0)) {
lastPassedIndex = i;
} else {
break;
}
}
// No events passed yet
if (lastPassedIndex < 0) return 0;
// All events passed
if (lastPassedIndex >= timeline.length - 1) return 100;
// Calculate percentage based on item position, not time
// Each item occupies an equal visual space regardless of time gaps
const itemPercentage = 100 / (timeline.length - 1);
// Find progress between current and next event for smooth transition
const currentEvent = timeline[lastPassedIndex];
const nextEvent = timeline[lastPassedIndex + 1];
const currentTimestamp = currentEvent.timestamp ?? 0;
const nextTimestamp = nextEvent.timestamp ?? 0;
// Calculate interpolation between the two events
const timeBetween = nextTimestamp - currentTimestamp;
const timeElapsed = currentTime - currentTimestamp;
const interpolation = timeBetween > 0 ? timeElapsed / timeBetween : 0;
// Base position plus interpolated progress to next item
return Math.min(
100,
lastPassedIndex * itemPercentage + interpolation * itemPercentage,
);
};
const blueLineHeight = calculateLineHeight();
return (
<div className="mx-2 mt-4 space-y-2">
{timeline.map((event, idx) => {
const isActive =
Math.abs((effectiveTime ?? 0) - (event.timestamp ?? 0)) <= 0.5;
return (
<LifecycleItem
key={`${event.timestamp}-${event.source_id ?? ""}-${idx}`}
event={event}
onSeek={onSeek}
isActive={isActive}
/>
);
})}
<div className="-pb-2 relative mx-2">
<div className="absolute -top-2 bottom-2 left-2 z-0 w-0.5 -translate-x-1/2 bg-secondary-foreground" />
<div
className="absolute left-2 top-2 z-[5] max-h-[calc(100%-1rem)] w-0.5 -translate-x-1/2 bg-selected transition-all duration-300"
style={{ height: `${blueLineHeight}%` }}
/>
<div className="space-y-2">
{timeline.map((event, idx) => {
const isActive =
Math.abs((effectiveTime ?? 0) - (event.timestamp ?? 0)) <= 0.5;
return (
<LifecycleItem
key={`${event.timestamp}-${event.source_id ?? ""}-${idx}`}
item={event}
onSeek={onSeek}
isActive={isActive}
effectiveTime={effectiveTime}
/>
);
})}
</div>
</div>
);
}

View File

@ -4,19 +4,22 @@ import {
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuPortal,
DropdownMenuSeparator,
} from "@/components/ui/dropdown-menu";
import { HiDotsHorizontal } from "react-icons/hi";
import { useApiHost } from "@/api";
import { useNavigate } from "react-router-dom";
import { useTranslation } from "react-i18next";
import type { Event } from "@/types/event";
import type { FrigateConfig } from "@/types/frigateConfig";
import { Event } from "@/types/event";
import { FrigateConfig } from "@/types/frigateConfig";
type EventMenuProps = {
event: Event;
config?: FrigateConfig;
onOpenUpload?: (e: Event) => void;
onOpenSimilarity?: (e: Event) => void;
selectedObjectId?: string;
setSelectedObjectId?: (event: Event | undefined) => void;
};
export default function EventMenu({
@ -24,71 +27,87 @@ export default function EventMenu({
config,
onOpenUpload,
onOpenSimilarity,
selectedObjectId,
setSelectedObjectId,
}: EventMenuProps) {
const apiHost = useApiHost();
const navigate = useNavigate();
const { t } = useTranslation("views/explore");
const handleObjectSelect = () => {
if (event.id === selectedObjectId) {
setSelectedObjectId?.(undefined);
} else {
setSelectedObjectId?.(event);
}
};
return (
<DropdownMenu>
<DropdownMenuTrigger>
<button
className="mr-2 rounded p-1"
aria-label={t("itemMenu.openMenu", { ns: "common" })}
>
<HiDotsHorizontal className="size-4 text-muted-foreground" />
</button>
</DropdownMenuTrigger>
<DropdownMenuPortal>
<DropdownMenuContent>
<DropdownMenuItem
onSelect={() => {
navigate(`/explore?event_id=${event.id}`);
}}
>
{t("details.item.button.viewInExplore")}
</DropdownMenuItem>
<DropdownMenuItem asChild>
<a
download
href={
event.has_snapshot
? `${apiHost}api/events/${event.id}/snapshot.jpg`
: `${apiHost}api/events/${event.id}/thumbnail.webp`
}
>
{t("itemMenu.downloadSnapshot.label")}
</a>
</DropdownMenuItem>
{event.has_snapshot &&
event.plus_id == undefined &&
event.data.type == "object" &&
config?.plus?.enabled && (
<DropdownMenuItem
onSelect={() => {
onOpenUpload?.(event);
}}
>
{t("itemMenu.submitToPlus.label")}
</DropdownMenuItem>
)}
{event.has_snapshot && config?.semantic_search?.enabled && (
<>
<span tabIndex={0} className="sr-only" />
<DropdownMenu>
<DropdownMenuTrigger>
<div className="rounded p-1 pr-2" role="button">
<HiDotsHorizontal className="size-4 text-muted-foreground" />
</div>
</DropdownMenuTrigger>
<DropdownMenuPortal>
<DropdownMenuContent>
<DropdownMenuItem onSelect={handleObjectSelect}>
{event.id === selectedObjectId
? t("itemMenu.hideObjectDetails.label")
: t("itemMenu.showObjectDetails.label")}
</DropdownMenuItem>
<DropdownMenuSeparator className="my-0.5" />
<DropdownMenuItem
onSelect={() => {
if (onOpenSimilarity) onOpenSimilarity(event);
else
navigate(
`/explore?search_type=similarity&event_id=${event.id}`,
);
navigate(`/explore?event_id=${event.id}`);
}}
>
{t("itemMenu.findSimilar.label")}
{t("details.item.button.viewInExplore")}
</DropdownMenuItem>
)}
</DropdownMenuContent>
</DropdownMenuPortal>
</DropdownMenu>
<DropdownMenuItem asChild>
<a
download
href={
event.has_snapshot
? `${apiHost}api/events/${event.id}/snapshot.jpg`
: `${apiHost}api/events/${event.id}/thumbnail.webp`
}
>
{t("itemMenu.downloadSnapshot.label")}
</a>
</DropdownMenuItem>
{event.has_snapshot &&
event.plus_id == undefined &&
event.data.type == "object" &&
config?.plus?.enabled && (
<DropdownMenuItem
onSelect={() => {
onOpenUpload?.(event);
}}
>
{t("itemMenu.submitToPlus.label")}
</DropdownMenuItem>
)}
{event.has_snapshot && config?.semantic_search?.enabled && (
<DropdownMenuItem
onSelect={() => {
if (onOpenSimilarity) onOpenSimilarity(event);
else
navigate(
`/explore?search_type=similarity&event_id=${event.id}`,
);
}}
>
{t("itemMenu.findSimilar.label")}
</DropdownMenuItem>
)}
</DropdownMenuContent>
</DropdownMenuPortal>
</DropdownMenu>
</>
);
}

View File

@ -66,6 +66,7 @@ export default function Events() {
camera: resp.data.camera,
startTime,
severity: resp.data.severity,
timelineType: "detail",
},
true,
);

View File

@ -858,14 +858,20 @@ function FaceAttemptGroup({
faceNames={faceNames}
onTrainAttempt={(name) => onTrainAttempt(data, name)}
>
<AddFaceIcon className="size-7 cursor-pointer p-1 text-gray-200 hover:rounded-full hover:bg-primary-foreground/40" />
<div className="group relative inline-flex items-center justify-center">
<div className="pointer-events-none absolute inset-0 m-auto size-5 scale-95 rounded-full bg-black opacity-0 blur-sm transition-all duration-200 group-hover:scale-100 group-hover:opacity-100 group-hover:blur-xl" />
<AddFaceIcon className="relative z-10 size-5 cursor-pointer text-white/85 hover:text-white" />
</div>
</FaceSelectionDialog>
<Tooltip>
<TooltipTrigger>
<LuRefreshCw
className="size-7 cursor-pointer p-1 text-gray-200 hover:rounded-full hover:bg-primary-foreground/40"
onClick={() => onReprocess(data)}
/>
<div className="group relative inline-flex items-center justify-center">
<div className="pointer-events-none absolute inset-0 m-auto size-5 scale-95 rounded-full bg-black opacity-0 blur-sm transition-all duration-200 group-hover:scale-100 group-hover:opacity-100 group-hover:blur-xl" />
<LuRefreshCw
className="relative z-10 size-5 cursor-pointer text-white/85 hover:text-white"
onClick={() => onReprocess(data)}
/>
</div>
</TooltipTrigger>
<TooltipContent>{t("button.reprocessFace")}</TooltipContent>
</Tooltip>

View File

@ -37,6 +37,7 @@ export type RecordingStartingPoint = {
camera: string;
startTime: number;
severity: ReviewSeverity;
timelineType?: "timeline" | "events" | "detail";
};
export type RecordingPlayerError = "stalled" | "startup";

View File

@ -1,6 +1,7 @@
import { fromUnixTime, intervalToDuration, formatDuration } from "date-fns";
import { Locale } from "date-fns/locale";
import { formatInTimeZone } from "date-fns-tz";
import i18n from "@/utils/i18n";
export const longToDate = (long: number): Date => new Date(long * 1000);
export const epochToLong = (date: number): number => date / 1000;
export const dateToLong = (date: Date): number => epochToLong(date.getTime());
@ -234,11 +235,13 @@ export const formatUnixTimestampToDateTime = (
* If end time is not provided, it returns 'In Progress'
* @param start_time: number - Unix timestamp for start time
* @param end_time: number|null - Unix timestamp for end time
* @param abbreviated: boolean - Whether to use abbreviated forms (h, m, s) instead of full words
* @returns string - duration or 'In Progress' if end time is not provided
*/
export const getDurationFromTimestamps = (
start_time: number,
end_time: number | null,
abbreviated: boolean = false,
): string => {
if (isNaN(start_time)) {
return "Invalid start time";
@ -250,12 +253,39 @@ export const getDurationFromTimestamps = (
}
const start = fromUnixTime(start_time);
const end = fromUnixTime(end_time);
duration = formatDuration(intervalToDuration({ start, end }), {
format: ["hours", "minutes", "seconds"],
})
.replace("hours", "h")
.replace("minutes", "m")
.replace("seconds", "s");
const durationObj = intervalToDuration({ start, end });
// Build duration string using i18n keys or abbreviations
const parts: string[] = [];
if (durationObj.hours) {
const count = durationObj.hours;
if (abbreviated) {
parts.push(`${count}h`);
} else {
const key = count === 1 ? "hour_one" : "hour_other";
parts.push(i18n.t(`time.${key}`, { time: count, ns: "common" }));
}
}
if (durationObj.minutes) {
const count = durationObj.minutes;
if (abbreviated) {
parts.push(`${count}m`);
} else {
const key = count === 1 ? "minute_one" : "minute_other";
parts.push(i18n.t(`time.${key}`, { time: count, ns: "common" }));
}
}
if (durationObj.seconds) {
const count = durationObj.seconds;
if (abbreviated) {
parts.push(`${count}s`);
} else {
const key = count === 1 ? "second_one" : "second_other";
parts.push(i18n.t(`time.${key}`, { time: count, ns: "common" }));
}
}
duration = parts.join(" ");
}
return duration;
};

View File

@ -9,7 +9,9 @@ export function getLifecycleItemDescription(
? lifecycleItem.data.sub_label[0]
: lifecycleItem.data.sub_label || lifecycleItem.data.label;
const label = getTranslatedLabel(rawLabel);
const label = lifecycleItem.data.sub_label
? rawLabel
: getTranslatedLabel(rawLabel);
switch (lifecycleItem.class_type) {
case "visible":
@ -44,14 +46,18 @@ export function getLifecycleItemDescription(
{
ns: "views/explore",
label,
attribute: lifecycleItem.data.attribute.replaceAll("_", " "),
attribute: getTranslatedLabel(
lifecycleItem.data.attribute.replaceAll("_", " "),
),
},
);
} else {
title = t("objectLifecycle.lifecycleItemDesc.attribute.other", {
ns: "views/explore",
label: lifecycleItem.data.label,
attribute: lifecycleItem.data.attribute.replaceAll("_", " "),
attribute: getTranslatedLabel(
lifecycleItem.data.attribute.replaceAll("_", " "),
),
});
}
return title;

View File

@ -10,11 +10,14 @@ import {
CustomClassificationModelConfig,
FrigateConfig,
} from "@/types/frigateConfig";
import { useMemo, useState } from "react";
import { useEffect, useMemo, useState } from "react";
import { isMobile } from "react-device-detect";
import { useTranslation } from "react-i18next";
import { FaFolderPlus } from "react-icons/fa";
import { MdModelTraining } from "react-icons/md";
import useSWR from "swr";
import Heading from "@/components/ui/heading";
import { useOverlayState } from "@/hooks/use-overlay-state";
const allModelTypes = ["objects", "states"] as const;
type ModelType = (typeof allModelTypes)[number];
@ -26,11 +29,24 @@ export default function ModelSelectionView({
onClick,
}: ModelSelectionViewProps) {
const { t } = useTranslation(["views/classificationModel"]);
const [page, setPage] = useState<ModelType>("objects");
const [pageToggle, setPageToggle] = useOptimisticState(page, setPage, 100);
const { data: config } = useSWR<FrigateConfig>("config", {
revalidateOnFocus: false,
});
const [page, setPage] = useOverlayState<ModelType>("objects", "objects");
const [pageToggle, setPageToggle] = useOptimisticState(
page || "objects",
setPage,
100,
);
const { data: config, mutate: refreshConfig } = useSWR<FrigateConfig>(
"config",
{
revalidateOnFocus: false,
},
);
// title
useEffect(() => {
document.title = t("documentTitle");
}, [t]);
// data
@ -64,15 +80,15 @@ export default function ModelSelectionView({
return <ActivityIndicator />;
}
if (classificationConfigs.length == 0) {
return <div>You need to setup a custom model configuration.</div>;
}
return (
<div className="flex size-full flex-col p-2">
<ClassificationModelWizardDialog
open={newModel}
onClose={() => setNewModel(false)}
defaultModelType={pageToggle === "objects" ? "object" : "state"}
onClose={() => {
setNewModel(false);
refreshConfig();
}}
/>
<div className="flex h-12 w-full items-center justify-between">
@ -84,7 +100,6 @@ export default function ModelSelectionView({
value={pageToggle}
onValueChange={(value: ModelType) => {
if (value) {
// Restrict viewer navigation
setPageToggle(value);
}
}}
@ -117,13 +132,46 @@ export default function ModelSelectionView({
</div>
</div>
<div className="flex size-full gap-2 p-2">
{selectedClassificationConfigs.map((config) => (
<ModelCard
key={config.name}
config={config}
onClick={() => onClick(config)}
{selectedClassificationConfigs.length === 0 ? (
<NoModelsView
onCreateModel={() => setNewModel(true)}
modelType={pageToggle}
/>
))}
) : (
selectedClassificationConfigs.map((config) => (
<ModelCard
key={config.name}
config={config}
onClick={() => onClick(config)}
/>
))
)}
</div>
</div>
);
}
function NoModelsView({
onCreateModel,
modelType,
}: {
onCreateModel: () => void;
modelType: ModelType;
}) {
const { t } = useTranslation(["views/classificationModel"]);
const typeKey = modelType === "objects" ? "object" : "state";
return (
<div className="flex size-full items-center justify-center">
<div className="flex flex-col items-center gap-2">
<MdModelTraining className="size-8" />
<Heading as="h4">{t(`noModels.${typeKey}.title`)}</Heading>
<div className="mb-3 text-center text-secondary-foreground">
{t(`noModels.${typeKey}.description`)}
</div>
<Button size="sm" variant="select" onClick={onCreateModel}>
{t(`noModels.${typeKey}.buttonText`)}
</Button>
</div>
</div>
);
@ -139,13 +187,17 @@ function ModelCard({ config, onClick }: ModelCardProps) {
}>(`classification/${config.name}/dataset`, { revalidateOnFocus: false });
const coverImage = useMemo(() => {
if (!dataset?.length) {
if (!dataset) {
return undefined;
}
const keys = Object.keys(dataset).filter((key) => key != "none");
const selectedKey = keys[0];
if (!dataset[selectedKey]) {
return undefined;
}
return {
name: selectedKey,
img: dataset[selectedKey][0],

View File

@ -642,6 +642,7 @@ function DatasetGrid({
filepath: `clips/${modelName}/dataset/${categoryName}/${image}`,
name: "",
}}
showArea={false}
selected={selectedImages.includes(image)}
i18nLibrary="views/classificationModel"
onClick={(data, _) => onClickImages([data.filename], true)}
@ -810,7 +811,10 @@ function StateTrainGrid({
image={data.filename}
onRefresh={onRefresh}
>
<TbCategoryPlus className="size-7 cursor-pointer p-1 text-gray-200 hover:rounded-full hover:bg-primary-foreground/40" />
<div className="group relative inline-flex items-center justify-center">
<div className="pointer-events-none absolute inset-0 m-auto size-5 scale-95 rounded-full bg-black opacity-0 blur-sm transition-all duration-200 group-hover:scale-100 group-hover:opacity-100 group-hover:blur-xl" />
<TbCategoryPlus className="relative z-10 size-5 cursor-pointer text-white/85 hover:text-white" />
</div>
</ClassificationSelectionDialog>
</ClassificationCard>
</div>
@ -957,7 +961,10 @@ function ObjectTrainGrid({
image={data.filename}
onRefresh={onRefresh}
>
<TbCategoryPlus className="size-7 cursor-pointer p-1 text-gray-200 hover:rounded-full hover:bg-primary-foreground/40" />
<div className="group relative inline-flex items-center justify-center">
<div className="pointer-events-none absolute inset-0 m-auto size-5 scale-95 rounded-full bg-black opacity-0 blur-sm transition-all duration-200 group-hover:scale-100 group-hover:opacity-100 group-hover:blur-xl" />
<TbCategoryPlus className="relative z-10 size-5 cursor-pointer text-white/85 hover:text-white" />
</div>
</ClassificationSelectionDialog>
</>
)}

View File

@ -53,8 +53,6 @@ import { cn } from "@/lib/utils";
import { FilterList, LAST_24_HOURS_KEY } from "@/types/filter";
import { GiSoundWaves } from "react-icons/gi";
import useKeyboardListener from "@/hooks/use-keyboard-listener";
import ReviewDetailDialog from "@/components/overlay/detail/ReviewDetailDialog";
import { useTimelineZoom } from "@/hooks/use-timeline-zoom";
import { useTranslation } from "react-i18next";
@ -398,6 +396,7 @@ export default function EventView({
onSelectAllReviews={onSelectAllReviews}
setSelectedReviews={setSelectedReviews}
pullLatestData={pullLatestData}
onOpenRecording={onOpenRecording}
/>
)}
{severity == "significant_motion" && (
@ -441,6 +440,7 @@ type DetectionReviewProps = {
onSelectAllReviews: () => void;
setSelectedReviews: (reviews: ReviewSegment[]) => void;
pullLatestData: () => void;
onOpenRecording: (recordingInfo: RecordingStartingPoint) => void;
};
function DetectionReview({
contentRef,
@ -460,15 +460,12 @@ function DetectionReview({
onSelectAllReviews,
setSelectedReviews,
pullLatestData,
onOpenRecording,
}: DetectionReviewProps) {
const { t } = useTranslation(["views/events"]);
const reviewTimelineRef = useRef<HTMLDivElement>(null);
// detail
const [reviewDetail, setReviewDetail] = useState<ReviewSegment>();
// preview
const [previewTime, setPreviewTime] = useState<number>();
@ -688,8 +685,6 @@ function DetectionReview({
return (
<>
<ReviewDetailDialog review={reviewDetail} setReview={setReviewDetail} />
<div
ref={contentRef}
className="no-scrollbar flex flex-1 flex-wrap content-start gap-2 overflow-y-auto md:gap-4"
@ -750,7 +745,12 @@ function DetectionReview({
detail: boolean,
) => {
if (detail) {
setReviewDetail(review);
onOpenRecording({
camera: review.camera,
startTime: review.start_time - REVIEW_PADDING,
severity: review.severity,
timelineType: "detail",
});
} else {
onSelectReview(review, ctrl);
}

View File

@ -53,6 +53,7 @@ import {
ASPECT_VERTICAL_LAYOUT,
ASPECT_WIDE_LAYOUT,
RecordingSegment,
RecordingStartingPoint,
} from "@/types/record";
import { useResizeObserver } from "@/hooks/resize-observer";
import { cn } from "@/lib/utils";
@ -141,9 +142,15 @@ export function RecordingView({
// timeline
const [recording] = useOverlayState<RecordingStartingPoint>(
"recording",
undefined,
false,
);
const [timelineType, setTimelineType] = useOverlayState<TimelineType>(
"timelineType",
"timeline",
recording?.timelineType ?? "timeline",
);
const chunkedTimeRange = useMemo(
@ -688,7 +695,7 @@ export function RecordingView({
"flex flex-1 flex-wrap",
isDesktop
? timelineType === "detail"
? "w-full"
? "md:w-[40%] lg:w-[70%] xl:w-full"
: "w-[80%]"
: "",
)}
@ -696,11 +703,9 @@ export function RecordingView({
<div
className={cn(
"flex size-full items-center",
timelineType === "detail"
? "flex-col"
: mainCameraAspect == "tall"
? "flex-row justify-evenly"
: "flex-col justify-center gap-2",
mainCameraAspect == "tall"
? "flex-row justify-evenly"
: "flex-col justify-center gap-2",
)}
>
<div
@ -782,7 +787,7 @@ export function RecordingView({
? "h-full w-72 flex-col"
: `h-28 w-full`,
previewRowOverflows ? "" : "items-center justify-center",
timelineType == "detail" && "mt-4",
timelineType == "detail" && isDesktop && "mt-4",
)}
>
<div className="w-2" />
@ -847,6 +852,7 @@ export function RecordingView({
setScrubbing={setScrubbing}
setExportRange={setExportRange}
onAnalysisOpen={onAnalysisOpen}
isPlaying={mainControllerRef?.current?.isPlaying() ?? false}
/>
</div>
</div>
@ -864,6 +870,7 @@ type TimelineProps = {
activeReviewItem?: ReviewSegment;
currentTime: number;
exportRange?: TimeRange;
isPlaying?: boolean;
setCurrentTime: React.Dispatch<React.SetStateAction<number>>;
manuallySetCurrentTime: (time: number, force: boolean) => void;
setScrubbing: React.Dispatch<React.SetStateAction<boolean>>;
@ -880,6 +887,7 @@ function Timeline({
activeReviewItem,
currentTime,
exportRange,
isPlaying,
setCurrentTime,
manuallySetCurrentTime,
setScrubbing,
@ -965,16 +973,20 @@ function Timeline({
className={cn(
"relative",
isDesktop
? `${timelineType == "timeline" ? "w-[100px]" : timelineType == "detail" ? "w-[30%]" : "w-60"} no-scrollbar overflow-y-auto`
: `overflow-hidden portrait:flex-grow ${timelineType == "timeline" ? "landscape:w-[100px]" : timelineType == "detail" ? "flex-1" : "landscape:w-[175px]"} `,
? `${timelineType == "timeline" ? "w-[100px]" : timelineType == "detail" ? "w-[30%] min-w-[350px]" : "w-60"} no-scrollbar overflow-y-auto`
: `overflow-hidden portrait:flex-grow ${timelineType == "timeline" ? "landscape:w-[100px]" : timelineType == "detail" && isDesktop ? "flex-1" : "landscape:w-[300px]"} `,
)}
>
{isMobile && (
<GenAISummaryDialog review={activeReviewItem} onOpen={onAnalysisOpen} />
)}
<div className="pointer-events-none absolute inset-x-0 top-0 z-20 h-[30px] w-full bg-gradient-to-b from-secondary to-transparent"></div>
<div className="pointer-events-none absolute inset-x-0 bottom-0 z-20 h-[30px] w-full bg-gradient-to-t from-secondary to-transparent"></div>
{timelineType != "detail" && (
<>
<div className="pointer-events-none absolute inset-x-0 top-0 z-20 h-[30px] w-full bg-gradient-to-b from-secondary to-transparent"></div>
<div className="pointer-events-none absolute inset-x-0 bottom-0 z-20 h-[30px] w-full bg-gradient-to-t from-secondary to-transparent"></div>
</>
)}
{timelineType == "timeline" ? (
!isLoading ? (
<MotionReviewTimeline
@ -1009,6 +1021,7 @@ function Timeline({
manuallySetCurrentTime(timestamp, play ?? true)
}
reviewItems={mainCameraReviewItems}
isPlaying={isPlaying}
/>
) : (
<div className="scrollbar-container h-full overflow-auto bg-secondary">

View File

@ -577,7 +577,7 @@ export default function SearchView({
>
<div
className={cn(
"aspect-square w-full overflow-hidden rounded-t-lg border",
"relative aspect-square w-full overflow-hidden rounded-lg",
)}
>
<SearchThumbnail
@ -634,38 +634,38 @@ export default function SearchView({
</Tooltip>
</div>
)}
<div className="absolute bottom-0 left-0 right-0 z-30 bg-gradient-to-t from-black/70 to-transparent p-2">
<SearchThumbnailFooter
searchResult={value}
columns={columns}
findSimilar={() => {
if (config?.semantic_search.enabled) {
setSimilaritySearch(value);
}
}}
refreshResults={refresh}
showObjectLifecycle={() =>
onSelectSearch(value, false, "object_lifecycle")
}
showSnapshot={() =>
onSelectSearch(value, false, "snapshot")
}
addTrigger={() => {
if (
config?.semantic_search.enabled &&
value.data.type == "object"
) {
navigate(
`/settings?page=triggers&camera=${value.camera}&event_id=${value.id}`,
);
}
}}
/>
</div>
</div>
<div
className={`review-item-ring pointer-events-none absolute inset-0 z-10 size-full rounded-lg outline outline-[3px] -outline-offset-[2.8px] ${selected ? `shadow-selected outline-selected` : "outline-transparent duration-500"}`}
className={`review-item-ring pointer-events-none absolute inset-0 z-30 size-full rounded-lg outline outline-[3px] -outline-offset-[2.8px] ${selected ? `shadow-selected outline-selected` : "outline-transparent duration-500"}`}
/>
<div className="flex w-full grow items-center justify-between rounded-b-lg border border-t-0 bg-card p-3 text-card-foreground">
<SearchThumbnailFooter
searchResult={value}
columns={columns}
findSimilar={() => {
if (config?.semantic_search.enabled) {
setSimilaritySearch(value);
}
}}
refreshResults={refresh}
showObjectLifecycle={() =>
onSelectSearch(value, false, "object_lifecycle")
}
showSnapshot={() =>
onSelectSearch(value, false, "snapshot")
}
addTrigger={() => {
if (
config?.semantic_search.enabled &&
value.data.type == "object"
) {
navigate(
`/settings?page=triggers&camera=${value.camera}&event_id=${value.id}`,
);
}
}}
/>
</div>
</div>
);
})}

View File

@ -66,8 +66,13 @@ export default function CameraManagementView({
return (
<>
<Toaster
richColors
className="z-[1000]"
position="top-center"
closeButton
/>
<div className="flex size-full flex-col md:flex-row">
<Toaster position="top-center" closeButton={true} />
<div className="scrollbar-container order-last mb-10 mt-2 flex h-full w-full flex-col overflow-y-auto pb-2 md:order-none">
{viewMode === "settings" ? (
<>