mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-06-02 07:00:32 +00:00
Refactor motion search (#23378)
* refactor motion search * cleanup dead code and tests * tweaks * fix multi-day seeking * start playback a few seconds before the change so the motion is in view
This commit is contained in:
Josh Hawkins
GitHub
parent
47a06c8b30
commit
8073174c20
@ -155,21 +155,22 @@ Motion Search lets you scan recorded footage for changes inside a region of inte
|
||||
|
||||
To start a search, open the Actions menu in History or click the kebab menu on a camera in the <NavPath path="Review > Motion" /> page and choose **Motion Search**. In the dialog:
|
||||
|
||||
1. Pick the camera and time range to scan.
|
||||
1. Pick the camera and time range to scan. In the date pickers, days that have recordings available are underlined.
|
||||
2. Draw a polygon on the camera frame to define the region of interest.
|
||||
3. Adjust the search parameters if needed:
|
||||
|
||||
| Field | Description |
|
||||
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Sensitivity Threshold** | Per-pixel luminance change required to count as motion inside the ROI. Behaves like Frigate's motion detection `threshold` setting. |
|
||||
| **Minimum Change Area** | Minimum percentage of the region of interest that must change for a frame to be considered significant. Raise it to ignore small movements (leaves, distant motion); lower it when the object you care about only covers a small slice of the ROI. |
|
||||
| **Frame Skip** | Number of frames to skip between samples — at a camera recording 20 fps, a skip value of 20 takes motion samples roughly once per second. Higher values scan much faster and are usually the right choice; lower it only when you need to catch the exact appearance or disappearance of a fast-moving object. |
|
||||
| **Maximum Results** | Maximum number of matching timestamps to return. |
|
||||
| **Parallel mode** | Process multiple recording segments in parallel. Speeds up large time ranges at the cost of higher CPU usage. |
|
||||
| Field | Description |
|
||||
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Sensitivity Threshold** | Per-pixel luminance change required to count as motion inside the ROI. Behaves like Frigate's motion detection `threshold` setting. |
|
||||
| **Minimum Change Area** | Minimum size of a single moving region, as a percentage of the ROI, for a frame to count as significant. Raise it to ignore small movements (leaves, distant motion); lower it when your subject covers only a small slice of the ROI. Every result shows the percentage it scored, so you can use those values to tune this. |
|
||||
| **Maximum Results** | Maximum number of matching timestamps to return. The search stops once it reaches this many results, so a lower value finishes sooner while a higher value scans further into the range. |
|
||||
| **Parallel mode** | Decode multiple recording ranges at the same time. Speeds up large time ranges at the cost of higher decoding and CPU usage. |
|
||||
|
||||
Motion Search samples each recording's keyframes automatically, so there is no frame-rate or sampling setting to tune.
|
||||
|
||||
Once running, Frigate scans the recording segments that overlap the time range and reports timestamps where changes were detected inside the polygon, along with the percentage of the ROI that changed. Clicking a result seeks the player to that moment so you can review what happened.
|
||||
|
||||
The status panel shows live progress and metrics such as how many segments were scanned, how many were skipped because no motion was recorded for that segment (using the stored motion heatmap), how many frames were decoded, and the total wall-clock time. Segments with no recorded motion in the selected ROI are skipped automatically, which is what makes searching long time ranges practical.
|
||||
The results panel shows the time range being scanned, a live progress bar with the timestamp currently being analyzed, and the running result count. A collapsible **Search Metrics** section reports how many segments were scanned and processed, how many were skipped because no motion was recorded in the ROI (using the stored motion heatmap), how many frames were decoded, and the total search time. Skipping segments with no recorded motion in the selected ROI is what makes searching long time ranges practical.
|
||||
|
||||
#### Common use cases
|
||||
|
||||
@ -179,6 +180,15 @@ Frigate's main use case is to record and surface tracked objects, so Motion Sear
|
||||
- **An object that was never detected.** Something Frigate doesn't have a model label for, an object too small or distant to be detected, or movement in a region where detection isn't running. The activity left no tracked object but did change the pixels, so a search can still find it.
|
||||
- **Activity while detection was effectively paused.** Changes that occurred while object detection was disabled, motion was suppressed by `skip_motion_threshold`, or inside an area covered by a motion mask, won't appear as review items or tracked objects but can be recovered by searching the recordings directly.
|
||||
|
||||
#### Examples
|
||||
|
||||
These show how to choose the ROI and **Minimum Change Area** for two common goals. Minimum Change Area is the size of a single moving region as a percentage of the ROI you draw, so the right value depends on how much of the ROI your subject — and its movement between samples — covers.
|
||||
|
||||
Because samples are a second or more apart, a moving subject usually appears in two places at once in the comparison, so even ordinary motion often scores tens of percent and a low threshold lets in almost everything. The most reliable approach is to **run a search, look at the percentage each result scored, and set Minimum Change Area just below the values for the events you care about.** The default is 20%; the suggestions below are starting points.
|
||||
|
||||
- **When did this item first appear (or disappear)?** A package was dropped off, a car parked, or a trash can was moved, and you want the exact moment. Draw a **tight ROI** around the spot the item occupies and **raise Minimum Change Area** (start around 40–60%). Because the item fills most of a tight ROI, its arrival or removal is a large change, while smaller nearby motion (shadows, a passing pedestrian) stays below the threshold. The **earliest result** is when it appeared; if you only care about that moment, a low Maximum Results finishes faster. If you get no hits, the ROI is probably looser than the item — lower the threshold or tighten the ROI.
|
||||
- **What's been getting into the garden?** Something has been trampling a flower bed overnight and no object was ever tracked. Draw a **looser ROI** covering the whole bed and use a **lower Minimum Change Area than the case above** — start near the 20% default and lower it (toward 5–10%) only if a small or distant subject is missed, since it covers just a slice of a large region. Expect more results to scan through — step through the timestamps and jump to each to see what triggered it. If wind-blown plants add noise, raise Minimum Change Area or the Sensitivity Threshold.
|
||||
|
||||
#### Expected performance
|
||||
|
||||
Motion Search analyzes the saved recordings on demand rather than reading a pre-built index, so a search over a long range takes longer than browsing Motion Previews. Cost scales mainly with how much footage has to be examined: segments with no recorded motion in your ROI are skipped using the stored motion heatmap (shown as "segments skipped" in the status panel), so a quiet range finishes quickly while a busy one takes longer.
|
||||
@ -186,5 +196,6 @@ Motion Search analyzes the saved recordings on demand rather than reading a pre-
|
||||
To increase the speed of searches:
|
||||
|
||||
- Draw a tight ROI. Because **Minimum Change Area** is measured as a percentage of the region you draw, a tight ROI around where you expect the change makes the object fill a larger share of the area, so it clears the threshold more easily. A loose ROI makes the same object a small fraction of the region, so it can fall below the threshold and be missed — forcing you to lower Minimum Change Area, which lets in more noise.
|
||||
- Keep Frame Skip high. A higher value samples fewer frames and speeds up the search considerably, while still landing within a few seconds of when the motion or object appeared — close enough to seek to in the recording. Only lower it when you need to pinpoint the exact frame something appears or disappears.
|
||||
- Use Parallel mode to shorten wall-clock time on multi-core systems, at the cost of higher CPU usage while it runs.
|
||||
- Narrow the time range to the window you care about, so there is less footage to examine.
|
||||
- Lower **Maximum Results** when you only need the first few hits. Because the search stops once it reaches that many results, a smaller value lets a busy range finish early instead of scanning the whole window.
|
||||
- Use Parallel mode to shorten wall-clock time on multi-core systems, at the cost of higher decoding and CPU usage while it runs.
|
||||
|
||||
17
docs/static/frigate-api.yaml
vendored
17
docs/static/frigate-api.yaml
vendored
@ -7288,13 +7288,6 @@ components:
|
||||
title: Min Area
|
||||
description: Minimum change area as a percentage of the ROI
|
||||
default: 5
|
||||
frame_skip:
|
||||
type: integer
|
||||
maximum: 30
|
||||
minimum: 1
|
||||
title: Frame Skip
|
||||
description: "Process every Nth frame (1=all frames, 5=every 5th frame)"
|
||||
default: 5
|
||||
parallel:
|
||||
type: boolean
|
||||
title: Parallel
|
||||
@ -7380,6 +7373,16 @@ components:
|
||||
anyOf:
|
||||
- $ref: "#/components/schemas/MotionSearchMetricsResponse"
|
||||
- type: "null"
|
||||
scanning_timestamp:
|
||||
anyOf:
|
||||
- type: number
|
||||
- type: "null"
|
||||
title: Scanning Timestamp
|
||||
progress:
|
||||
anyOf:
|
||||
- type: number
|
||||
- type: "null"
|
||||
title: Progress
|
||||
type: object
|
||||
required:
|
||||
- success
|
||||
|
||||
@ -41,12 +41,6 @@ class MotionSearchRequest(BaseModel):
|
||||
le=100.0,
|
||||
description="Minimum change area as a percentage of the ROI",
|
||||
)
|
||||
frame_skip: int = Field(
|
||||
default=30,
|
||||
ge=1,
|
||||
le=120,
|
||||
description="Process every Nth frame (1=all frames, 5=every 5th frame)",
|
||||
)
|
||||
parallel: bool = Field(
|
||||
default=False,
|
||||
description="Enable parallel scanning across segments",
|
||||
@ -97,6 +91,8 @@ class MotionSearchStatusResponse(BaseModel):
|
||||
total_frames_processed: Optional[int] = None
|
||||
error_message: Optional[str] = None
|
||||
metrics: Optional[MotionSearchMetricsResponse] = None
|
||||
scanning_timestamp: Optional[float] = None
|
||||
progress: Optional[float] = None
|
||||
|
||||
|
||||
@router.post(
|
||||
@ -151,7 +147,6 @@ async def start_motion_search(
|
||||
polygon_points=body.polygon_points,
|
||||
threshold=body.threshold,
|
||||
min_area=body.min_area,
|
||||
frame_skip=body.frame_skip,
|
||||
parallel=body.parallel,
|
||||
max_results=body.max_results,
|
||||
)
|
||||
@ -231,6 +226,9 @@ async def get_motion_search_status_endpoint(
|
||||
if job.metrics:
|
||||
response_content["metrics"] = job.metrics.to_dict()
|
||||
|
||||
response_content["scanning_timestamp"] = job.scanning_timestamp
|
||||
response_content["progress"] = job.progress
|
||||
|
||||
return JSONResponse(content=response_content)
|
||||
|
||||
|
||||
|
||||
@ -3,6 +3,8 @@
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Callable, Generator, Iterable
|
||||
from concurrent.futures import Future, ThreadPoolExecutor, as_completed
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from datetime import datetime
|
||||
@ -19,6 +21,18 @@ from frigate.jobs.manager import (
|
||||
get_job_by_id,
|
||||
set_current_job,
|
||||
)
|
||||
from frigate.jobs.motion_search_batch import (
|
||||
build_segment_time_map,
|
||||
coalesce_runs,
|
||||
stream_time_to_absolute,
|
||||
)
|
||||
from frigate.jobs.motion_search_decode import (
|
||||
iter_vod_frames,
|
||||
keyframe_sampling_eligible,
|
||||
probe_video_dimensions,
|
||||
probe_vod_keyframe_pts,
|
||||
resolve_motion_decode_args,
|
||||
)
|
||||
from frigate.models import Recordings
|
||||
from frigate.types import JobStatusTypesEnum
|
||||
|
||||
@ -26,6 +40,18 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
# Constants
|
||||
HEATMAP_GRID_SIZE = 16
|
||||
# Max wall-clock span of one VOD run request (seconds). Bounds per-request size
|
||||
# and gives streaming/cancel/early-exit granularity.
|
||||
MAX_RUN_SECONDS = 600.0
|
||||
# Treat segments within this many seconds end-to-start as time-contiguous.
|
||||
RUN_GAP_EPSILON = 1.0
|
||||
# Longest-side pixels for the ROI downscale before motion detection.
|
||||
SCALE_TARGET = 400
|
||||
# Minimum wall seconds between intra-run progress broadcasts.
|
||||
PROGRESS_BROADCAST_INTERVAL = 1.0
|
||||
# Output frame rate for the fixed-cadence fallback used on long-GOP cameras
|
||||
# (where keyframe sampling is too sparse). Keyframe cameras ignore this.
|
||||
FALLBACK_SAMPLE_FPS = 2.0
|
||||
|
||||
|
||||
@dataclass
|
||||
@ -69,13 +95,16 @@ class MotionSearchJob(Job):
|
||||
polygon_points: list[list[float]] = field(default_factory=list)
|
||||
threshold: int = 30
|
||||
min_area: float = 5.0
|
||||
frame_skip: int = 5
|
||||
parallel: bool = False
|
||||
max_results: int = 25
|
||||
|
||||
# Track progress
|
||||
total_frames_processed: int = 0
|
||||
|
||||
# Live progress (ride the existing to_dict() websocket broadcast)
|
||||
scanning_timestamp: Optional[float] = None
|
||||
progress: float = 0.0
|
||||
|
||||
# Metrics for observability
|
||||
metrics: Optional[MotionSearchMetrics] = None
|
||||
|
||||
@ -100,6 +129,113 @@ def create_polygon_mask(
|
||||
return mask
|
||||
|
||||
|
||||
def compute_roi_crop_and_scale(
|
||||
polygon_points: list[list[float]],
|
||||
frame_width: int,
|
||||
frame_height: int,
|
||||
scale_target: int,
|
||||
) -> tuple[tuple[int, int, int, int], tuple[int, int]]:
|
||||
"""Compute the ROI crop box and never-upscale scaled dimensions.
|
||||
|
||||
Returns ((crop_w, crop_h, crop_x, crop_y), (scaled_w, scaled_h)) in pixels.
|
||||
The crop is the polygon's bounding box in frame pixels; the scaled size fits
|
||||
the crop's longest side to ``scale_target`` without ever enlarging it.
|
||||
"""
|
||||
xs = [p[0] for p in polygon_points]
|
||||
ys = [p[1] for p in polygon_points]
|
||||
# nv12 (4:2:0) hwdownload requires even crop offsets and even crop/scale
|
||||
# dimensions; otherwise ffmpeg rounds the chroma planes and the raw byte
|
||||
# stream stops matching the expected frame size. Force even values, and the
|
||||
# mask is built from these same values so the two stay aligned.
|
||||
crop_x = int(min(xs) * frame_width)
|
||||
crop_y = int(min(ys) * frame_height)
|
||||
crop_x -= crop_x % 2
|
||||
crop_y -= crop_y % 2
|
||||
crop_w = max(2, int(max(xs) * frame_width) - crop_x)
|
||||
crop_h = max(2, int(max(ys) * frame_height) - crop_y)
|
||||
crop_w -= crop_w % 2
|
||||
crop_h -= crop_h % 2
|
||||
|
||||
longest = max(crop_w, crop_h)
|
||||
factor = min(1.0, scale_target / longest)
|
||||
scaled_w = max(2, round(crop_w * factor))
|
||||
scaled_h = max(2, round(crop_h * factor))
|
||||
scaled_w -= scaled_w % 2
|
||||
scaled_h -= scaled_h % 2
|
||||
return (crop_w, crop_h, crop_x, crop_y), (scaled_w, scaled_h)
|
||||
|
||||
|
||||
def build_scaled_roi_mask(
|
||||
polygon_points: list[list[float]],
|
||||
frame_width: int,
|
||||
frame_height: int,
|
||||
crop: tuple[int, int, int, int],
|
||||
scaled: tuple[int, int],
|
||||
) -> np.ndarray:
|
||||
"""Rasterize the polygon mask at the scaled ROI size.
|
||||
|
||||
Builds the full-resolution mask, crops it to the ROI box, and nearest-
|
||||
neighbor resizes it to the scaled dimensions so it lines up exactly with the
|
||||
frames ffmpeg crops and scales.
|
||||
"""
|
||||
crop_w, crop_h, crop_x, crop_y = crop
|
||||
scaled_w, scaled_h = scaled
|
||||
full_mask = create_polygon_mask(polygon_points, frame_width, frame_height)
|
||||
cropped = full_mask[crop_y : crop_y + crop_h, crop_x : crop_x + crop_w]
|
||||
return cv2.resize(cropped, (scaled_w, scaled_h), interpolation=cv2.INTER_NEAREST)
|
||||
|
||||
|
||||
def detect_motion_scaled(
|
||||
frames: Iterable[tuple[int, np.ndarray]],
|
||||
mask: np.ndarray,
|
||||
threshold: int,
|
||||
min_area: float,
|
||||
timestamp_fn: Callable[[int], float],
|
||||
) -> list[MotionSearchResult]:
|
||||
"""Detect motion across pre-cropped, pre-scaled gray frames.
|
||||
|
||||
``frames`` yields (absolute_frame_index, gray_roi_frame); ``mask`` is the
|
||||
scaled ROI mask. ``min_area`` is a percentage of the masked ROI. Mirrors the
|
||||
full-res detection math (absdiff -> blur -> threshold -> dilate -> contours)
|
||||
on the already-reduced frames.
|
||||
"""
|
||||
results: list[MotionSearchResult] = []
|
||||
mask_area = np.count_nonzero(mask)
|
||||
if mask_area == 0:
|
||||
return results
|
||||
min_area_pixels = int((min_area / 100.0) * mask_area)
|
||||
|
||||
prev: np.ndarray | None = None
|
||||
for frame_idx, gray in frames:
|
||||
masked = cv2.bitwise_and(gray, gray, mask=mask)
|
||||
if prev is not None:
|
||||
diff = cv2.absdiff(prev, masked)
|
||||
diff_blurred = cv2.GaussianBlur(diff, (3, 3), 0)
|
||||
_, thresh = cv2.threshold(diff_blurred, threshold, 255, cv2.THRESH_BINARY)
|
||||
thresh_dilated = cv2.dilate(thresh, None, iterations=1) # type: ignore[call-overload]
|
||||
thresh_masked = cv2.bitwise_and(thresh_dilated, thresh_dilated, mask=mask)
|
||||
change_pixels = cv2.countNonZero(thresh_masked)
|
||||
if change_pixels > min_area_pixels:
|
||||
contours, _ = cv2.findContours(
|
||||
thresh_masked, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
|
||||
)
|
||||
total_change_area = sum(
|
||||
cv2.contourArea(c)
|
||||
for c in contours
|
||||
if cv2.contourArea(c) >= min_area_pixels
|
||||
)
|
||||
if total_change_area > 0:
|
||||
change_percentage = (total_change_area / mask_area) * 100
|
||||
results.append(
|
||||
MotionSearchResult(
|
||||
timestamp=timestamp_fn(frame_idx),
|
||||
change_percentage=round(change_percentage, 2),
|
||||
)
|
||||
)
|
||||
prev = masked
|
||||
return results
|
||||
|
||||
|
||||
def compute_roi_bbox_normalized(
|
||||
polygon_points: list[list[float]],
|
||||
) -> tuple[float, float, float, float]:
|
||||
@ -184,6 +320,22 @@ def segment_passes_heatmap_gate(
|
||||
return heatmap_overlaps_roi(heatmap, roi_bbox)
|
||||
|
||||
|
||||
def resolve_internal_port(config: FrigateConfig) -> int:
|
||||
"""Return the unauthenticated internal nginx port for VOD requests."""
|
||||
listen = config.networking.listen.internal
|
||||
if isinstance(listen, str):
|
||||
return int(listen.split(":")[-1])
|
||||
return int(listen)
|
||||
|
||||
|
||||
def build_vod_url(internal_port: int, camera: str, start: float, end: float) -> str:
|
||||
"""Build the internal VOD HLS URL for a camera time range."""
|
||||
return (
|
||||
f"http://127.0.0.1:{internal_port}/vod/{camera}"
|
||||
f"/start/{start}/end/{end}/index.m3u8"
|
||||
)
|
||||
|
||||
|
||||
class MotionSearchRunner(threading.Thread):
|
||||
"""Thread-based runner for motion search jobs with parallel verification."""
|
||||
|
||||
@ -206,6 +358,23 @@ class MotionSearchRunner(threading.Thread):
|
||||
cpu_count = os.cpu_count() or 1
|
||||
self.max_workers = min(4, cpu_count)
|
||||
|
||||
# Resolved once per job in _execute_search
|
||||
self.ffmpeg_path: str = "ffmpeg"
|
||||
self.ffprobe_path: str = "ffprobe"
|
||||
self.decode_args: list[str] = []
|
||||
# Keyframe sampling decision, decided once per job from the first run's
|
||||
# GOP. The fallback cadence is a fixed rate (see FALLBACK_SAMPLE_FPS).
|
||||
self.use_keyframe: bool = True
|
||||
self.fps_rate: float = FALLBACK_SAMPLE_FPS
|
||||
# ROI crop/scale + scaled mask, computed once from the VOD-stream
|
||||
# dimensions (which can differ from the detect resolution).
|
||||
self.crop: tuple[int, int, int, int] = (0, 0, 0, 0)
|
||||
self.scaled: tuple[int, int] = (0, 0)
|
||||
self.scaled_mask: np.ndarray = np.zeros((0, 0), dtype=np.uint8)
|
||||
self.channels: int = 1
|
||||
self.internal_port: int = 5000
|
||||
self._last_progress_broadcast: float = 0.0
|
||||
|
||||
def run(self) -> None:
|
||||
"""Execute the motion search job."""
|
||||
try:
|
||||
@ -281,6 +450,9 @@ class MotionSearchRunner(threading.Thread):
|
||||
if frame_width is None or frame_height is None:
|
||||
raise ValueError(f"Camera {camera_name} detect dimensions not configured")
|
||||
|
||||
self.ffmpeg_path = camera_config.ffmpeg.ffmpeg_path
|
||||
self.ffprobe_path = camera_config.ffmpeg.ffprobe_path
|
||||
|
||||
# Create polygon mask
|
||||
polygon_mask = create_polygon_mask(
|
||||
self.job.polygon_points, frame_width, frame_height
|
||||
@ -384,205 +556,274 @@ class MotionSearchRunner(threading.Thread):
|
||||
self.metrics.heatmap_roi_skip_segments,
|
||||
)
|
||||
|
||||
if self.job.parallel:
|
||||
return self._search_motion_parallel(filtered_recordings, polygon_mask)
|
||||
# Resolve decode backend (allowlisted hwaccel or software), coalesce the
|
||||
# gate-passing segments into time-contiguous runs, and probe the first
|
||||
# run's VOD stream once for dimensions + keyframe layout. VOD output is
|
||||
# what we decode, so crop/scale/mask are computed against it.
|
||||
self.internal_port = resolve_internal_port(self.config)
|
||||
self.decode_args = resolve_motion_decode_args(camera_config)
|
||||
ffprobe_path = self.ffprobe_path
|
||||
|
||||
return self._search_motion_sequential(filtered_recordings, polygon_mask)
|
||||
runs = coalesce_runs(filtered_recordings, MAX_RUN_SECONDS, RUN_GAP_EPSILON)
|
||||
if not runs:
|
||||
return []
|
||||
|
||||
def _search_motion_parallel(
|
||||
self,
|
||||
recordings: list[Recordings],
|
||||
polygon_mask: np.ndarray,
|
||||
) -> list[MotionSearchResult]:
|
||||
"""Search for motion in parallel across segments, streaming results."""
|
||||
all_results: list[MotionSearchResult] = []
|
||||
total_frames = 0
|
||||
next_recording_idx_to_merge = 0
|
||||
first_run = runs[0]
|
||||
first_url = build_vod_url(
|
||||
self.internal_port,
|
||||
camera_name,
|
||||
float(first_run[0].start_time),
|
||||
float(first_run[-1].end_time),
|
||||
)
|
||||
dims = probe_video_dimensions(ffprobe_path, first_url)
|
||||
if dims is None:
|
||||
raise ValueError(f"Could not probe VOD dimensions for camera {camera_name}")
|
||||
rec_width, rec_height, _rec_fps = dims
|
||||
|
||||
self.crop, self.scaled = compute_roi_crop_and_scale(
|
||||
self.job.polygon_points, rec_width, rec_height, SCALE_TARGET
|
||||
)
|
||||
self.scaled_mask = build_scaled_roi_mask(
|
||||
self.job.polygon_points, rec_width, rec_height, self.crop, self.scaled
|
||||
)
|
||||
self.channels = 1 # always gray output
|
||||
|
||||
# Decide keyframe vs fixed-cadence sampling once from the first run's GOP
|
||||
# (keyframe structure is a per-camera constant).
|
||||
first_pts = probe_vod_keyframe_pts(ffprobe_path, first_url)
|
||||
self.use_keyframe = keyframe_sampling_eligible(first_pts)
|
||||
|
||||
logger.debug(
|
||||
"Motion search job %s: starting motion search with %d workers "
|
||||
"across %d segments",
|
||||
"Motion search job %s: %d runs, sampling=%s, hwaccel=%s, vod=%dx%d",
|
||||
self.job.id,
|
||||
self.max_workers,
|
||||
len(recordings),
|
||||
len(runs),
|
||||
"keyframe" if self.use_keyframe else "cadence",
|
||||
bool(self.decode_args),
|
||||
rec_width,
|
||||
rec_height,
|
||||
)
|
||||
|
||||
# Initialize partial results on the job so they stream to the frontend
|
||||
return self._search_runs(runs)
|
||||
|
||||
def _emit_progress(self, abs_ts: float) -> None:
|
||||
"""Throttled intra-run progress broadcast (scanning cursor)."""
|
||||
now = time.monotonic()
|
||||
if now - self._last_progress_broadcast < PROGRESS_BROADCAST_INTERVAL:
|
||||
return
|
||||
self._last_progress_broadcast = now
|
||||
self.job.scanning_timestamp = abs_ts
|
||||
self._broadcast_status()
|
||||
|
||||
def _detect_with_progress(
|
||||
self,
|
||||
indexed_frames: list[tuple[int, np.ndarray]],
|
||||
timestamp_fn: Callable[[int], float],
|
||||
) -> list[MotionSearchResult]:
|
||||
"""Run detection while firing throttled progress as frames are scanned."""
|
||||
|
||||
def _gen() -> Generator[tuple[int, np.ndarray], None, None]:
|
||||
for i, frame in indexed_frames:
|
||||
if not self._should_stop():
|
||||
self._emit_progress(timestamp_fn(i))
|
||||
yield i, frame
|
||||
|
||||
return detect_motion_scaled(
|
||||
_gen(),
|
||||
self.scaled_mask,
|
||||
self.job.threshold,
|
||||
self.job.min_area,
|
||||
timestamp_fn,
|
||||
)
|
||||
|
||||
def _process_run(
|
||||
self, run: list[Recordings]
|
||||
) -> tuple[list[MotionSearchResult], int]:
|
||||
"""Decode one run's VOD stream and detect motion.
|
||||
|
||||
Keyframe mode compares every decoded keyframe (free recall, since they
|
||||
are all decoded anyway) paired with its probed PTS; if the decoded and
|
||||
probed counts disagree (the decoder ignored ``-skip_frame nokey`` or the
|
||||
stream is corrupt) this run re-runs in the fixed-cadence fallback.
|
||||
Returns ``(results, frame_count)``.
|
||||
"""
|
||||
run_start: float = run[0].start_time # type: ignore[assignment]
|
||||
run_end: float = run[-1].end_time # type: ignore[assignment]
|
||||
vod_url = build_vod_url(self.internal_port, self.job.camera, run_start, run_end)
|
||||
time_map = build_segment_time_map(run)
|
||||
|
||||
if self.use_keyframe:
|
||||
kf_pts = probe_vod_keyframe_pts(self.ffprobe_path, vod_url)
|
||||
frames = list(
|
||||
iter_vod_frames(
|
||||
self.ffmpeg_path,
|
||||
vod_url,
|
||||
self.scaled[0],
|
||||
self.scaled[1],
|
||||
self.channels,
|
||||
self.decode_args,
|
||||
self.crop,
|
||||
self.scaled,
|
||||
True,
|
||||
self._should_stop,
|
||||
skip_nonkey=True,
|
||||
fps_rate=None,
|
||||
)
|
||||
)
|
||||
if kf_pts and len(frames) == len(kf_pts):
|
||||
abs_times = [stream_time_to_absolute(time_map, p) for p in kf_pts]
|
||||
indexed = list(enumerate(frames))
|
||||
|
||||
def _ts_kf(i: int) -> float:
|
||||
return abs_times[i]
|
||||
|
||||
results = self._detect_with_progress(indexed, _ts_kf)
|
||||
return results, len(frames)
|
||||
|
||||
logger.debug(
|
||||
"Keyframe count mismatch (%d decoded vs %d probed), using cadence",
|
||||
len(frames),
|
||||
len(kf_pts),
|
||||
)
|
||||
|
||||
return self._process_run_cadence(vod_url, time_map)
|
||||
|
||||
def _process_run_cadence(
|
||||
self, vod_url: str, time_map: list[tuple[float, float, float]]
|
||||
) -> tuple[list[MotionSearchResult], int]:
|
||||
"""Fixed-cadence fallback: fps-filtered VOD decode, evenly spaced times."""
|
||||
frames = list(
|
||||
iter_vod_frames(
|
||||
self.ffmpeg_path,
|
||||
vod_url,
|
||||
self.scaled[0],
|
||||
self.scaled[1],
|
||||
self.channels,
|
||||
self.decode_args,
|
||||
self.crop,
|
||||
self.scaled,
|
||||
True,
|
||||
self._should_stop,
|
||||
skip_nonkey=False,
|
||||
fps_rate=self.fps_rate,
|
||||
)
|
||||
)
|
||||
indexed = list(enumerate(frames))
|
||||
|
||||
def _ts_fps(i: int) -> float:
|
||||
return stream_time_to_absolute(time_map, i / self.fps_rate)
|
||||
|
||||
results = self._detect_with_progress(indexed, _ts_fps)
|
||||
return results, len(frames)
|
||||
|
||||
def _merge_run(
|
||||
self,
|
||||
run: list[Recordings],
|
||||
run_results: list[MotionSearchResult],
|
||||
frames: int,
|
||||
state: dict[str, Any],
|
||||
) -> bool:
|
||||
"""Fold one run's output into the running results; stream + dedup.
|
||||
|
||||
Returns True once ``max_results`` deduped hits have accumulated.
|
||||
"""
|
||||
state["completed_runs"] += 1
|
||||
state["all_results"].extend(run_results)
|
||||
state["total_frames"] += frames
|
||||
self.job.total_frames_processed = state["total_frames"]
|
||||
self.metrics.frames_decoded = state["total_frames"]
|
||||
self.metrics.segments_processed += len(run)
|
||||
self.job.progress = state["completed_runs"] / state["total_runs"]
|
||||
|
||||
state["all_results"].sort(key=lambda r: r.timestamp)
|
||||
deduped = self._deduplicate_results(state["all_results"])[
|
||||
: self.job.max_results
|
||||
]
|
||||
self.job.results = {
|
||||
"results": [r.to_dict() for r in deduped],
|
||||
"total_frames_processed": state["total_frames"],
|
||||
}
|
||||
self._broadcast_status()
|
||||
return len(deduped) >= self.job.max_results
|
||||
|
||||
def _search_runs(self, runs: list[list[Recordings]]) -> list[MotionSearchResult]:
|
||||
"""Decode runs (parallel pool when enabled), merge in order, stream."""
|
||||
state: dict[str, Any] = {
|
||||
"all_results": [],
|
||||
"total_frames": 0,
|
||||
"completed_runs": 0,
|
||||
"total_runs": len(runs),
|
||||
}
|
||||
self.job.results = {"results": [], "total_frames_processed": 0}
|
||||
|
||||
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
|
||||
futures: dict[Future, int] = {}
|
||||
completed_segments: dict[int, tuple[list[MotionSearchResult], int]] = {}
|
||||
logger.debug(
|
||||
"Motion search job %s: searching %d runs (parallel=%s, workers=%d)",
|
||||
self.job.id,
|
||||
len(runs),
|
||||
self.job.parallel,
|
||||
self.max_workers,
|
||||
)
|
||||
|
||||
for idx, recording in enumerate(recordings):
|
||||
if self._should_stop():
|
||||
break
|
||||
if self.job.parallel and len(runs) > 1:
|
||||
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
|
||||
futures: dict[Future, int] = {}
|
||||
for idx, run in enumerate(runs):
|
||||
if self._should_stop():
|
||||
break
|
||||
futures[executor.submit(self._process_run, run)] = idx
|
||||
|
||||
rec_start: float = recording.start_time # type: ignore[assignment]
|
||||
rec_end: float = recording.end_time # type: ignore[assignment]
|
||||
future = executor.submit(
|
||||
self._process_recording_for_motion,
|
||||
str(recording.path),
|
||||
rec_start,
|
||||
rec_end,
|
||||
self.job.start_time_range,
|
||||
self.job.end_time_range,
|
||||
polygon_mask,
|
||||
self.job.threshold,
|
||||
self.job.min_area,
|
||||
self.job.frame_skip,
|
||||
)
|
||||
futures[future] = idx
|
||||
completed: dict[int, tuple[list[MotionSearchResult], int]] = {}
|
||||
next_idx = 0
|
||||
for future in as_completed(futures):
|
||||
if self._should_stop():
|
||||
break
|
||||
run_idx = futures[future]
|
||||
try:
|
||||
completed[run_idx] = future.result()
|
||||
except Exception as e:
|
||||
self.metrics.segments_with_errors += 1
|
||||
logger.warning("Error processing run %d: %s", run_idx, e)
|
||||
completed[run_idx] = ([], 0)
|
||||
|
||||
for future in as_completed(futures):
|
||||
if self._should_stop():
|
||||
# Cancel remaining futures
|
||||
for f in futures:
|
||||
f.cancel()
|
||||
break
|
||||
|
||||
recording_idx = futures[future]
|
||||
recording = recordings[recording_idx]
|
||||
|
||||
try:
|
||||
results, frames = future.result()
|
||||
self.metrics.segments_processed += 1
|
||||
completed_segments[recording_idx] = (results, frames)
|
||||
|
||||
while next_recording_idx_to_merge in completed_segments:
|
||||
segment_results, segment_frames = completed_segments.pop(
|
||||
next_recording_idx_to_merge
|
||||
)
|
||||
|
||||
all_results.extend(segment_results)
|
||||
total_frames += segment_frames
|
||||
self.job.total_frames_processed = total_frames
|
||||
self.metrics.frames_decoded = total_frames
|
||||
|
||||
if segment_results:
|
||||
deduped = self._deduplicate_results(all_results)
|
||||
self.job.results = {
|
||||
"results": [
|
||||
r.to_dict() for r in deduped[: self.job.max_results]
|
||||
],
|
||||
"total_frames_processed": total_frames,
|
||||
}
|
||||
|
||||
self._broadcast_status()
|
||||
|
||||
if segment_results and len(deduped) >= self.job.max_results:
|
||||
while next_idx in completed:
|
||||
run_results, frames = completed.pop(next_idx)
|
||||
if self._merge_run(runs[next_idx], run_results, frames, state):
|
||||
self.internal_stop_event.set()
|
||||
for pending_future in futures:
|
||||
pending_future.cancel()
|
||||
for pending in futures:
|
||||
pending.cancel()
|
||||
break
|
||||
|
||||
next_recording_idx_to_merge += 1
|
||||
next_idx += 1
|
||||
|
||||
if self.internal_stop_event.is_set():
|
||||
break
|
||||
|
||||
else:
|
||||
for run in runs:
|
||||
if self._should_stop():
|
||||
break
|
||||
try:
|
||||
run_results, frames = self._process_run(run)
|
||||
except Exception as e:
|
||||
self.metrics.segments_processed += 1
|
||||
self.metrics.segments_with_errors += 1
|
||||
self.metrics.segments_processed += len(run)
|
||||
self._broadcast_status()
|
||||
logger.warning(
|
||||
"Error processing segment %s: %s",
|
||||
recording.path,
|
||||
e,
|
||||
)
|
||||
|
||||
self.job.total_frames_processed = total_frames
|
||||
self.metrics.frames_decoded = total_frames
|
||||
|
||||
logger.debug(
|
||||
"Motion search job %s: motion search complete, "
|
||||
"found %d raw results, decoded %d frames, %d segment errors",
|
||||
self.job.id,
|
||||
len(all_results),
|
||||
total_frames,
|
||||
self.metrics.segments_with_errors,
|
||||
)
|
||||
|
||||
# Sort and deduplicate results
|
||||
all_results.sort(key=lambda x: x.timestamp)
|
||||
return self._deduplicate_results(all_results)[: self.job.max_results]
|
||||
|
||||
def _search_motion_sequential(
|
||||
self,
|
||||
recordings: list[Recordings],
|
||||
polygon_mask: np.ndarray,
|
||||
) -> list[MotionSearchResult]:
|
||||
"""Search for motion sequentially across segments, streaming results."""
|
||||
all_results: list[MotionSearchResult] = []
|
||||
total_frames = 0
|
||||
|
||||
logger.debug(
|
||||
"Motion search job %s: starting sequential motion search across %d segments",
|
||||
self.job.id,
|
||||
len(recordings),
|
||||
)
|
||||
|
||||
self.job.results = {"results": [], "total_frames_processed": 0}
|
||||
|
||||
for recording in recordings:
|
||||
if self.cancel_event.is_set():
|
||||
break
|
||||
|
||||
try:
|
||||
rec_start: float = recording.start_time # type: ignore[assignment]
|
||||
rec_end: float = recording.end_time # type: ignore[assignment]
|
||||
results, frames = self._process_recording_for_motion(
|
||||
str(recording.path),
|
||||
rec_start,
|
||||
rec_end,
|
||||
self.job.start_time_range,
|
||||
self.job.end_time_range,
|
||||
polygon_mask,
|
||||
self.job.threshold,
|
||||
self.job.min_area,
|
||||
self.job.frame_skip,
|
||||
)
|
||||
all_results.extend(results)
|
||||
total_frames += frames
|
||||
|
||||
self.job.total_frames_processed = total_frames
|
||||
self.metrics.frames_decoded = total_frames
|
||||
self.metrics.segments_processed += 1
|
||||
|
||||
if results:
|
||||
all_results.sort(key=lambda x: x.timestamp)
|
||||
deduped = self._deduplicate_results(all_results)[
|
||||
: self.job.max_results
|
||||
]
|
||||
self.job.results = {
|
||||
"results": [r.to_dict() for r in deduped],
|
||||
"total_frames_processed": total_frames,
|
||||
}
|
||||
|
||||
self._broadcast_status()
|
||||
|
||||
if results and len(deduped) >= self.job.max_results:
|
||||
logger.warning("Error processing run: %s", e)
|
||||
continue
|
||||
if self._merge_run(run, run_results, frames, state):
|
||||
break
|
||||
|
||||
except Exception as e:
|
||||
self.metrics.segments_processed += 1
|
||||
self.metrics.segments_with_errors += 1
|
||||
self._broadcast_status()
|
||||
logger.warning("Error processing segment %s: %s", recording.path, e)
|
||||
|
||||
self.job.total_frames_processed = total_frames
|
||||
self.metrics.frames_decoded = total_frames
|
||||
all_results: list[MotionSearchResult] = state["all_results"]
|
||||
self.job.total_frames_processed = state["total_frames"]
|
||||
self.metrics.frames_decoded = state["total_frames"]
|
||||
self.job.progress = 1.0
|
||||
|
||||
logger.debug(
|
||||
"Motion search job %s: sequential motion search complete, "
|
||||
"found %d raw results, decoded %d frames, %d segment errors",
|
||||
"Motion search job %s: complete, %d raw results, %d frames, %d errors",
|
||||
self.job.id,
|
||||
len(all_results),
|
||||
total_frames,
|
||||
state["total_frames"],
|
||||
self.metrics.segments_with_errors,
|
||||
)
|
||||
|
||||
all_results.sort(key=lambda x: x.timestamp)
|
||||
all_results.sort(key=lambda r: r.timestamp)
|
||||
return self._deduplicate_results(all_results)[: self.job.max_results]
|
||||
|
||||
def _deduplicate_results(
|
||||
@ -602,160 +843,6 @@ class MotionSearchRunner(threading.Thread):
|
||||
|
||||
return deduplicated
|
||||
|
||||
def _process_recording_for_motion(
|
||||
self,
|
||||
recording_path: str,
|
||||
recording_start: float,
|
||||
recording_end: float,
|
||||
search_start: float,
|
||||
search_end: float,
|
||||
polygon_mask: np.ndarray,
|
||||
threshold: int,
|
||||
min_area: float,
|
||||
frame_skip: int,
|
||||
) -> tuple[list[MotionSearchResult], int]:
|
||||
"""Process a single recording file for motion detection.
|
||||
|
||||
This method is designed to be called from a thread pool.
|
||||
|
||||
Args:
|
||||
min_area: Minimum change area as a percentage of the ROI (0-100).
|
||||
"""
|
||||
results: list[MotionSearchResult] = []
|
||||
frames_processed = 0
|
||||
|
||||
if not os.path.exists(recording_path):
|
||||
logger.warning("Recording file not found: %s", recording_path)
|
||||
return results, frames_processed
|
||||
|
||||
cap = cv2.VideoCapture(recording_path)
|
||||
if not cap.isOpened():
|
||||
logger.error("Could not open recording: %s", recording_path)
|
||||
return results, frames_processed
|
||||
|
||||
try:
|
||||
fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
|
||||
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
|
||||
recording_duration = recording_end - recording_start
|
||||
|
||||
# Calculate frame range
|
||||
start_offset = max(0, search_start - recording_start)
|
||||
end_offset = min(recording_duration, search_end - recording_start)
|
||||
start_frame = int(start_offset * fps)
|
||||
end_frame = int(end_offset * fps)
|
||||
start_frame = max(0, min(start_frame, total_frames - 1))
|
||||
end_frame = max(0, min(end_frame, total_frames))
|
||||
|
||||
if start_frame >= end_frame:
|
||||
return results, frames_processed
|
||||
|
||||
cap.set(cv2.CAP_PROP_POS_FRAMES, start_frame)
|
||||
|
||||
# Get ROI bounding box
|
||||
roi_bbox = cv2.boundingRect(polygon_mask)
|
||||
roi_x, roi_y, roi_w, roi_h = roi_bbox
|
||||
|
||||
prev_frame_gray = None
|
||||
frame_step = max(frame_skip, 1)
|
||||
frame_idx = start_frame
|
||||
|
||||
while frame_idx < end_frame:
|
||||
if self._should_stop():
|
||||
break
|
||||
|
||||
ret, frame = cap.read()
|
||||
if not ret:
|
||||
frame_idx += 1
|
||||
continue
|
||||
|
||||
if (frame_idx - start_frame) % frame_step != 0:
|
||||
frame_idx += 1
|
||||
continue
|
||||
|
||||
frames_processed += 1
|
||||
|
||||
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
|
||||
|
||||
# Handle frame dimension changes
|
||||
if gray.shape != polygon_mask.shape:
|
||||
resized_mask = cv2.resize(
|
||||
polygon_mask,
|
||||
(gray.shape[1], gray.shape[0]),
|
||||
interpolation=cv2.INTER_NEAREST,
|
||||
)
|
||||
current_bbox = cv2.boundingRect(resized_mask)
|
||||
else:
|
||||
resized_mask = polygon_mask
|
||||
current_bbox = roi_bbox
|
||||
|
||||
roi_x, roi_y, roi_w, roi_h = current_bbox
|
||||
cropped_gray = gray[roi_y : roi_y + roi_h, roi_x : roi_x + roi_w]
|
||||
cropped_mask = resized_mask[
|
||||
roi_y : roi_y + roi_h, roi_x : roi_x + roi_w
|
||||
]
|
||||
|
||||
cropped_mask_area = np.count_nonzero(cropped_mask)
|
||||
if cropped_mask_area == 0:
|
||||
frame_idx += 1
|
||||
continue
|
||||
|
||||
# Convert percentage to pixel count for this ROI
|
||||
min_area_pixels = int((min_area / 100.0) * cropped_mask_area)
|
||||
|
||||
masked_gray = cv2.bitwise_and(
|
||||
cropped_gray, cropped_gray, mask=cropped_mask
|
||||
)
|
||||
|
||||
if prev_frame_gray is not None:
|
||||
diff = cv2.absdiff(prev_frame_gray, masked_gray) # type: ignore[unreachable]
|
||||
diff_blurred = cv2.GaussianBlur(diff, (3, 3), 0)
|
||||
_, thresh = cv2.threshold(
|
||||
diff_blurred, threshold, 255, cv2.THRESH_BINARY
|
||||
)
|
||||
thresh_dilated = cv2.dilate(thresh, None, iterations=1)
|
||||
thresh_masked = cv2.bitwise_and(
|
||||
thresh_dilated, thresh_dilated, mask=cropped_mask
|
||||
)
|
||||
|
||||
change_pixels = cv2.countNonZero(thresh_masked)
|
||||
if change_pixels > min_area_pixels:
|
||||
contours, _ = cv2.findContours(
|
||||
thresh_masked, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
|
||||
)
|
||||
total_change_area = sum(
|
||||
cv2.contourArea(c)
|
||||
for c in contours
|
||||
if cv2.contourArea(c) >= min_area_pixels
|
||||
)
|
||||
if total_change_area > 0:
|
||||
frame_time_offset = (frame_idx - start_frame) / fps
|
||||
timestamp = (
|
||||
recording_start + start_offset + frame_time_offset
|
||||
)
|
||||
change_percentage = (
|
||||
total_change_area / cropped_mask_area
|
||||
) * 100
|
||||
results.append(
|
||||
MotionSearchResult(
|
||||
timestamp=timestamp,
|
||||
change_percentage=round(change_percentage, 2),
|
||||
)
|
||||
)
|
||||
|
||||
prev_frame_gray = masked_gray
|
||||
frame_idx += 1
|
||||
|
||||
finally:
|
||||
cap.release()
|
||||
|
||||
logger.debug(
|
||||
"Motion search segment complete: %s, %d frames processed, %d results found",
|
||||
recording_path,
|
||||
frames_processed,
|
||||
len(results),
|
||||
)
|
||||
return results, frames_processed
|
||||
|
||||
|
||||
# Module-level state for managing per-camera jobs
|
||||
_motion_search_jobs: dict[str, tuple[MotionSearchJob, threading.Event]] = {}
|
||||
@ -779,7 +866,6 @@ def start_motion_search_job(
|
||||
polygon_points: list[list[float]],
|
||||
threshold: int = 30,
|
||||
min_area: float = 5.0,
|
||||
frame_skip: int = 5,
|
||||
parallel: bool = False,
|
||||
max_results: int = 25,
|
||||
) -> str:
|
||||
@ -794,7 +880,6 @@ def start_motion_search_job(
|
||||
polygon_points=polygon_points,
|
||||
threshold=threshold,
|
||||
min_area=min_area,
|
||||
frame_skip=frame_skip,
|
||||
parallel=parallel,
|
||||
max_results=max_results,
|
||||
)
|
||||
@ -812,14 +897,13 @@ def start_motion_search_job(
|
||||
logger.debug(
|
||||
"Started motion search job %s for camera %s: "
|
||||
"time_range=%.1f-%.1f, threshold=%d, min_area=%.1f%%, "
|
||||
"frame_skip=%d, parallel=%s, max_results=%d, polygon_points=%d vertices",
|
||||
"parallel=%s, max_results=%d, polygon_points=%d vertices",
|
||||
job.id,
|
||||
camera_name,
|
||||
start_time,
|
||||
end_time,
|
||||
threshold,
|
||||
min_area,
|
||||
frame_skip,
|
||||
parallel,
|
||||
max_results,
|
||||
len(polygon_points),
|
||||
|
||||
75
frigate/jobs/motion_search_batch.py
Normal file
75
frigate/jobs/motion_search_batch.py
Normal file
@ -0,0 +1,75 @@
|
||||
"""Pure helpers for VOD-batched motion search.
|
||||
|
||||
Coalescing gate-passing segments into time-contiguous runs, mapping a frame's
|
||||
VOD stream time back to an absolute timestamp, and thinning sample times to a
|
||||
target interval. No I/O or ffmpeg here so the tricky math stays unit-testable.
|
||||
"""
|
||||
|
||||
from bisect import bisect_right
|
||||
from typing import Any
|
||||
|
||||
|
||||
def coalesce_runs(
|
||||
segments: list[Any], max_seconds: float, epsilon: float
|
||||
) -> list[list[Any]]:
|
||||
"""Group gate-passing segments into time-contiguous runs.
|
||||
|
||||
A run extends while each segment's ``start_time`` is within ``epsilon`` of
|
||||
the previous segment's ``end_time`` (no recording gap) and the run's total
|
||||
span stays at or below ``max_seconds``. A gap or the cap starts a new run.
|
||||
Each segment must expose ``start_time`` / ``end_time``.
|
||||
"""
|
||||
runs: list[list[Any]] = []
|
||||
current: list[Any] = []
|
||||
for seg in segments:
|
||||
if not current:
|
||||
current = [seg]
|
||||
continue
|
||||
prev_end = float(current[-1].end_time)
|
||||
run_start = float(current[0].start_time)
|
||||
contiguous = abs(float(seg.start_time) - prev_end) <= epsilon
|
||||
within_cap = (float(seg.end_time) - run_start) <= max_seconds
|
||||
if contiguous and within_cap:
|
||||
current.append(seg)
|
||||
else:
|
||||
runs.append(current)
|
||||
current = [seg]
|
||||
if current:
|
||||
runs.append(current)
|
||||
return runs
|
||||
|
||||
|
||||
def build_segment_time_map(
|
||||
run: list[Any],
|
||||
) -> list[tuple[float, float, float]]:
|
||||
"""Build a (stream_offset, abs_start, duration) row per segment in a run.
|
||||
|
||||
``stream_offset`` is the segment's start in continuous VOD stream time (the
|
||||
cumulative sum of preceding segment durations); ``abs_start`` is its absolute
|
||||
``start_time``. Built from each segment's own duration; for a gap-free run
|
||||
this makes stream time equal ``run_start + offset``.
|
||||
"""
|
||||
rows: list[tuple[float, float, float]] = []
|
||||
offset = 0.0
|
||||
for seg in run:
|
||||
duration = float(seg.end_time) - float(seg.start_time)
|
||||
rows.append((offset, float(seg.start_time), duration))
|
||||
offset += duration
|
||||
return rows
|
||||
|
||||
|
||||
def stream_time_to_absolute(
|
||||
time_map: list[tuple[float, float, float]], stream_time: float
|
||||
) -> float:
|
||||
"""Map a VOD stream time to an absolute timestamp via the run's table.
|
||||
|
||||
Binary-searches the segment whose stream range contains ``stream_time`` and
|
||||
returns ``abs_start + (stream_time - stream_offset)``. Times past the last
|
||||
segment map into the last segment (clamped at the run edge).
|
||||
"""
|
||||
offsets = [row[0] for row in time_map]
|
||||
idx = bisect_right(offsets, stream_time) - 1
|
||||
if idx < 0:
|
||||
idx = 0
|
||||
stream_offset, abs_start, _duration = time_map[idx]
|
||||
return abs_start + (stream_time - stream_offset)
|
||||
382
frigate/jobs/motion_search_decode.py
Normal file
382
frigate/jobs/motion_search_decode.py
Normal file
@ -0,0 +1,382 @@
|
||||
"""Hardware-accelerated ffmpeg decode for motion search.
|
||||
|
||||
Decodes a recording run's VOD/HLS stream with an ffmpeg subprocess, optionally
|
||||
selecting only keyframes, and streams raw frames over a pipe for the motion
|
||||
math. Output is the requested ``pix_fmt`` (gray or ``bgr24``) with optional
|
||||
crop/scale applied in the filter graph so downstream pixels are unchanged.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import subprocess as sp
|
||||
import tempfile
|
||||
from collections.abc import Callable, Generator
|
||||
from typing import IO
|
||||
|
||||
import numpy as np
|
||||
|
||||
from frigate.config import CameraConfig
|
||||
from frigate.ffmpeg_presets import parse_preset_hardware_acceleration_decode
|
||||
from frigate.util.services import auto_detect_hwaccel
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Output-format surfaces that download cleanly to nv12 via the fixed
|
||||
# ``hwdownload,format=nv12`` step the decode path appends. Other surfaces
|
||||
# (drm_prime from rkmpp, vulkan, amf) need a different download step, so motion
|
||||
# search decodes them in software to keep results byte-identical rather than risk
|
||||
# a wrong-but-valid-sized frame the zero-frame fallback gate would not catch.
|
||||
_NV12_OUTPUT_FORMATS = frozenset({"vaapi", "cuda", "qsv"})
|
||||
|
||||
|
||||
def _hwaccel_output_format(decode_args: list[str]) -> str | None:
|
||||
"""Return the ``-hwaccel_output_format`` value in ffmpeg args, or None."""
|
||||
try:
|
||||
idx = decode_args.index("-hwaccel_output_format")
|
||||
except ValueError:
|
||||
return None
|
||||
return decode_args[idx + 1] if idx + 1 < len(decode_args) else None
|
||||
|
||||
|
||||
def resolve_motion_decode_args(camera_config: CameraConfig) -> list[str]:
|
||||
"""Resolve the ffmpeg hwaccel decode args for a camera's recordings.
|
||||
|
||||
``auto`` is resolved via ``auto_detect_hwaccel`` and the preset is expanded
|
||||
by ``parse_preset_hardware_acceleration_decode`` (the same table the live
|
||||
pipeline uses). Acceleration is kept only when the decoded surface downloads
|
||||
cleanly to nv12 -- decided by reading ``-hwaccel_output_format`` back from the
|
||||
resolved args rather than a separate preset allowlist that could drift from
|
||||
``PRESETS_HW_ACCEL_DECODE``. Anything else (custom args, a software-only
|
||||
preset, or an nv12-incompatible surface) returns an empty list, meaning
|
||||
software decode, so results stay byte-identical.
|
||||
"""
|
||||
raw = camera_config.ffmpeg.hwaccel_args
|
||||
preset = auto_detect_hwaccel() if raw == "auto" else raw
|
||||
|
||||
# Custom args (a list) decode in software so results stay byte-identical.
|
||||
if not isinstance(preset, str):
|
||||
return []
|
||||
|
||||
decode_args = parse_preset_hardware_acceleration_decode(
|
||||
preset,
|
||||
camera_config.detect.fps,
|
||||
camera_config.detect.width or 0,
|
||||
camera_config.detect.height or 0,
|
||||
camera_config.ffmpeg.gpu,
|
||||
)
|
||||
if not decode_args:
|
||||
return []
|
||||
|
||||
if _hwaccel_output_format(decode_args) not in _NV12_OUTPUT_FORMATS:
|
||||
return []
|
||||
|
||||
return decode_args
|
||||
|
||||
|
||||
def _read_exact(stream: IO[bytes], size: int) -> bytes | None:
|
||||
"""Read exactly ``size`` bytes from a pipe, or None at clean EOF.
|
||||
|
||||
Pipe reads can return fewer bytes than requested, so loop until the frame
|
||||
is complete. A short read at the start of a frame means end-of-stream.
|
||||
"""
|
||||
buf = bytearray()
|
||||
while len(buf) < size:
|
||||
chunk = stream.read(size - len(buf))
|
||||
if not chunk:
|
||||
return None
|
||||
buf.extend(chunk)
|
||||
return bytes(buf)
|
||||
|
||||
|
||||
def _terminate(proc: sp.Popen[bytes]) -> None:
|
||||
"""Stop an ffmpeg decode process promptly."""
|
||||
# Close the read end first so a blocked ffmpeg write unblocks (ffmpeg then
|
||||
# sees a broken pipe), then signal it. The resulting ffmpeg write error is
|
||||
# harmless and goes to the captured stderr.
|
||||
if proc.stdout is not None:
|
||||
try:
|
||||
proc.stdout.close()
|
||||
except OSError:
|
||||
pass
|
||||
if proc.poll() is None:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=5)
|
||||
except sp.TimeoutExpired:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
|
||||
|
||||
KEYFRAME_MAX_GAP_SECONDS = 2.0
|
||||
|
||||
|
||||
def keyframe_sampling_eligible(
|
||||
keyframe_pts: list[float], max_gap: float = KEYFRAME_MAX_GAP_SECONDS
|
||||
) -> bool:
|
||||
"""True if keyframes are dense and regular enough for keyframe-only sampling.
|
||||
|
||||
Requires at least two keyframes and no gap longer than ``max_gap`` seconds, so
|
||||
a multi-second motion event necessarily spans a sampled keyframe.
|
||||
"""
|
||||
if len(keyframe_pts) < 2:
|
||||
return False
|
||||
gaps = [b - a for a, b in zip(keyframe_pts, keyframe_pts[1:])]
|
||||
return max(gaps) <= max_gap
|
||||
|
||||
|
||||
VOD_PROTOCOL_ARGS = ["-protocol_whitelist", "pipe,file,http,tcp"]
|
||||
|
||||
|
||||
def build_vod_decode_command(
|
||||
ffmpeg_path: str,
|
||||
vod_url: str,
|
||||
decode_args: list[str],
|
||||
crop: tuple[int, int, int, int] | None,
|
||||
scale: tuple[int, int] | None,
|
||||
gray: bool,
|
||||
*,
|
||||
skip_nonkey: bool,
|
||||
fps_rate: float | None,
|
||||
) -> list[str]:
|
||||
"""Build the ffmpeg argv to decode a VOD HLS URL.
|
||||
|
||||
``skip_nonkey`` adds ``-skip_frame nokey`` (keyframe-only). ``fps_rate`` adds
|
||||
an ``fps`` filter for the fixed-cadence fallback. They are mutually
|
||||
exclusive: keyframe mode passes ``skip_nonkey=True``/``fps_rate=None``; the
|
||||
fallback passes ``skip_nonkey=False`` with a rate.
|
||||
"""
|
||||
filters: list[str] = []
|
||||
# With hwaccel the decoded frames are GPU surfaces; pull them back to system
|
||||
# memory before the CPU fps/crop/scale filters and the rawvideo encoder.
|
||||
if decode_args:
|
||||
filters.append("hwdownload")
|
||||
filters.append("format=nv12")
|
||||
if fps_rate is not None:
|
||||
filters.append(f"fps={fps_rate}")
|
||||
if crop is not None:
|
||||
cw, ch, cx, cy = crop
|
||||
filters.append(f"crop={cw}:{ch}:{cx}:{cy}")
|
||||
if scale is not None:
|
||||
sw, sh = scale
|
||||
filters.append(f"scale={sw}:{sh}")
|
||||
|
||||
pix_fmt = "gray" if gray else "bgr24"
|
||||
cmd = [ffmpeg_path, "-hide_banner", "-loglevel", "error"]
|
||||
if skip_nonkey:
|
||||
cmd += ["-skip_frame", "nokey"]
|
||||
cmd += [*decode_args, *VOD_PROTOCOL_ARGS, "-i", vod_url, "-an"]
|
||||
if filters:
|
||||
cmd += ["-vf", ",".join(filters)]
|
||||
cmd += ["-vsync", "0", "-f", "rawvideo", "-pix_fmt", pix_fmt, "pipe:"]
|
||||
return cmd
|
||||
|
||||
|
||||
def _run_vod_decode(
|
||||
ffmpeg_path: str,
|
||||
vod_url: str,
|
||||
out_width: int,
|
||||
out_height: int,
|
||||
channels: int,
|
||||
decode_args: list[str],
|
||||
crop: tuple[int, int, int, int] | None,
|
||||
scale: tuple[int, int] | None,
|
||||
gray: bool,
|
||||
should_stop: Callable[[], bool],
|
||||
*,
|
||||
skip_nonkey: bool,
|
||||
fps_rate: float | None,
|
||||
software_retry: bool,
|
||||
) -> Generator[np.ndarray, None, None]:
|
||||
"""Run one VOD decode, yielding raw frames; retry in software if empty."""
|
||||
cmd = build_vod_decode_command(
|
||||
ffmpeg_path,
|
||||
vod_url,
|
||||
decode_args,
|
||||
crop,
|
||||
scale,
|
||||
gray,
|
||||
skip_nonkey=skip_nonkey,
|
||||
fps_rate=fps_rate,
|
||||
)
|
||||
frame_size = out_width * out_height * channels
|
||||
stderr_file = tempfile.SpooledTemporaryFile(max_size=65536)
|
||||
proc = sp.Popen(cmd, stdout=sp.PIPE, stderr=stderr_file)
|
||||
assert proc.stdout is not None
|
||||
|
||||
count = 0
|
||||
try:
|
||||
while True:
|
||||
if should_stop():
|
||||
break
|
||||
buf = _read_exact(proc.stdout, frame_size)
|
||||
if buf is None:
|
||||
break
|
||||
if channels == 1:
|
||||
frame = np.frombuffer(buf, dtype=np.uint8).reshape(
|
||||
(out_height, out_width)
|
||||
)
|
||||
else:
|
||||
frame = np.frombuffer(buf, dtype=np.uint8).reshape(
|
||||
(out_height, out_width, channels)
|
||||
)
|
||||
count += 1
|
||||
yield frame
|
||||
finally:
|
||||
_terminate(proc)
|
||||
stderr_file.close()
|
||||
|
||||
if count == 0 and software_retry and not should_stop():
|
||||
logger.warning("Hardware VOD decode produced no frames, retrying in software")
|
||||
yield from _run_vod_decode(
|
||||
ffmpeg_path,
|
||||
vod_url,
|
||||
out_width,
|
||||
out_height,
|
||||
channels,
|
||||
[],
|
||||
crop,
|
||||
scale,
|
||||
gray,
|
||||
should_stop,
|
||||
skip_nonkey=skip_nonkey,
|
||||
fps_rate=fps_rate,
|
||||
software_retry=False,
|
||||
)
|
||||
|
||||
|
||||
def iter_vod_frames(
|
||||
ffmpeg_path: str,
|
||||
vod_url: str,
|
||||
out_width: int,
|
||||
out_height: int,
|
||||
channels: int,
|
||||
decode_args: list[str],
|
||||
crop: tuple[int, int, int, int] | None,
|
||||
scale: tuple[int, int] | None,
|
||||
gray: bool,
|
||||
should_stop: Callable[[], bool],
|
||||
*,
|
||||
skip_nonkey: bool,
|
||||
fps_rate: float | None,
|
||||
) -> Generator[np.ndarray, None, None]:
|
||||
"""Decode a VOD HLS URL and yield raw frames in order.
|
||||
|
||||
Pair keyframe-mode output with probed keyframe PTS; pair fallback output with
|
||||
a fixed cadence. Falls back once to software decode if a hwaccel decode yields
|
||||
no frames.
|
||||
"""
|
||||
yield from _run_vod_decode(
|
||||
ffmpeg_path,
|
||||
vod_url,
|
||||
out_width,
|
||||
out_height,
|
||||
channels,
|
||||
decode_args,
|
||||
crop,
|
||||
scale,
|
||||
gray,
|
||||
should_stop,
|
||||
skip_nonkey=skip_nonkey,
|
||||
fps_rate=fps_rate,
|
||||
software_retry=bool(decode_args),
|
||||
)
|
||||
|
||||
|
||||
def probe_vod_keyframe_pts(ffprobe_path: str, vod_url: str) -> list[float]:
|
||||
"""Return keyframe presentation timestamps (VOD stream time) in order.
|
||||
|
||||
Reads packet flags via ffprobe over the VOD URL (no decode). Returns [] on
|
||||
any failure so the caller can fall back.
|
||||
"""
|
||||
cmd = [
|
||||
ffprobe_path,
|
||||
"-v",
|
||||
"error",
|
||||
*VOD_PROTOCOL_ARGS,
|
||||
"-i",
|
||||
vod_url,
|
||||
"-select_streams",
|
||||
"v:0",
|
||||
"-show_packets",
|
||||
"-show_entries",
|
||||
"packet=pts_time,flags",
|
||||
"-of",
|
||||
"json",
|
||||
]
|
||||
try:
|
||||
completed = sp.run(cmd, capture_output=True, text=True, timeout=120)
|
||||
except (OSError, sp.SubprocessError):
|
||||
logger.warning("ffprobe failed for VOD keyframe probe")
|
||||
return []
|
||||
|
||||
if completed.returncode != 0 or not completed.stdout:
|
||||
return []
|
||||
|
||||
try:
|
||||
packets = json.loads(completed.stdout).get("packets", [])
|
||||
except json.JSONDecodeError:
|
||||
return []
|
||||
|
||||
pts: list[float] = []
|
||||
for pkt in packets:
|
||||
flags = pkt.get("flags", "")
|
||||
pts_time = pkt.get("pts_time")
|
||||
if flags.startswith("K") and pts_time is not None:
|
||||
try:
|
||||
pts.append(float(pts_time))
|
||||
except ValueError:
|
||||
continue
|
||||
return sorted(pts)
|
||||
|
||||
|
||||
def probe_video_dimensions(
|
||||
ffprobe_path: str, recording_path: str
|
||||
) -> tuple[int, int, float] | None:
|
||||
"""Return (width, height, fps) for a recording's video stream, or None.
|
||||
|
||||
Reads stream metadata via ffprobe (no decode). The record stream resolution
|
||||
can differ from the camera's detect resolution, so this is probed once per
|
||||
job against a real segment.
|
||||
"""
|
||||
cmd = [
|
||||
ffprobe_path,
|
||||
"-v",
|
||||
"error",
|
||||
"-select_streams",
|
||||
"v:0",
|
||||
"-show_entries",
|
||||
"stream=width,height,avg_frame_rate",
|
||||
"-of",
|
||||
"json",
|
||||
recording_path,
|
||||
]
|
||||
try:
|
||||
completed = sp.run(cmd, capture_output=True, text=True, timeout=30)
|
||||
except (OSError, sp.SubprocessError):
|
||||
return None
|
||||
|
||||
if completed.returncode != 0 or not completed.stdout:
|
||||
return None
|
||||
|
||||
try:
|
||||
streams = json.loads(completed.stdout).get("streams", [])
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
if not streams:
|
||||
return None
|
||||
|
||||
stream = streams[0]
|
||||
width = int(stream.get("width", 0) or 0)
|
||||
height = int(stream.get("height", 0) or 0)
|
||||
rate = stream.get("avg_frame_rate", "0/0") or "0/0"
|
||||
try:
|
||||
num, _, den = rate.partition("/")
|
||||
fps = float(num) / float(den) if float(den) != 0 else 0.0
|
||||
except (ValueError, ZeroDivisionError):
|
||||
fps = 0.0
|
||||
|
||||
if width <= 0 or height <= 0:
|
||||
return None
|
||||
|
||||
return width, height, fps
|
||||
58
frigate/test/test_motion_search_batch.py
Normal file
58
frigate/test/test_motion_search_batch.py
Normal file
@ -0,0 +1,58 @@
|
||||
"""Tests for motion search batch helpers (runs + timestamp mapping)."""
|
||||
|
||||
import unittest
|
||||
from dataclasses import dataclass
|
||||
|
||||
from frigate.jobs.motion_search_batch import (
|
||||
build_segment_time_map,
|
||||
coalesce_runs,
|
||||
stream_time_to_absolute,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Seg:
|
||||
path: str
|
||||
start_time: float
|
||||
end_time: float
|
||||
|
||||
|
||||
def _run_seconds(run):
|
||||
return float(run[-1].end_time) - float(run[0].start_time)
|
||||
|
||||
|
||||
class TestCoalesceRuns(unittest.TestCase):
|
||||
def test_contiguous_segments_form_one_run(self):
|
||||
segs = [_Seg("a", 0.0, 10.0), _Seg("b", 10.0, 20.0), _Seg("c", 20.0, 30.0)]
|
||||
runs = coalesce_runs(segs, max_seconds=600.0, epsilon=0.5)
|
||||
self.assertEqual(len(runs), 1)
|
||||
self.assertEqual(len(runs[0]), 3)
|
||||
|
||||
def test_time_gap_splits_runs(self):
|
||||
# b ends 20, c starts 25 -> 5s gap > epsilon -> two runs.
|
||||
segs = [_Seg("a", 0.0, 10.0), _Seg("b", 10.0, 20.0), _Seg("c", 25.0, 35.0)]
|
||||
runs = coalesce_runs(segs, max_seconds=600.0, epsilon=0.5)
|
||||
self.assertEqual([len(r) for r in runs], [2, 1])
|
||||
|
||||
def test_max_duration_caps_a_run(self):
|
||||
# Five contiguous 10s segments, cap 25s.
|
||||
segs = [_Seg(str(i), i * 10.0, i * 10.0 + 10.0) for i in range(5)]
|
||||
runs = coalesce_runs(segs, max_seconds=25.0, epsilon=0.5)
|
||||
self.assertTrue(all(_run_seconds(r) <= 30.0 for r in runs))
|
||||
self.assertEqual(sum(len(r) for r in runs), 5)
|
||||
|
||||
def test_empty(self):
|
||||
self.assertEqual(coalesce_runs([], max_seconds=600.0, epsilon=0.5), [])
|
||||
|
||||
|
||||
class TestTimestampMapping(unittest.TestCase):
|
||||
def test_gapfree_run_maps_to_start_plus_pts(self):
|
||||
run = [_Seg("a", 1000.0, 1010.0), _Seg("b", 1010.0, 1020.0)]
|
||||
time_map = build_segment_time_map(run)
|
||||
self.assertAlmostEqual(stream_time_to_absolute(time_map, 3.0), 1003.0)
|
||||
self.assertAlmostEqual(stream_time_to_absolute(time_map, 12.0), 1012.0)
|
||||
|
||||
def test_past_end_clamps(self):
|
||||
run = [_Seg("a", 1000.0, 1010.0)]
|
||||
time_map = build_segment_time_map(run)
|
||||
self.assertAlmostEqual(stream_time_to_absolute(time_map, 9.9), 1009.9)
|
||||
190
frigate/test/test_motion_search_decode.py
Normal file
190
frigate/test/test_motion_search_decode.py
Normal file
@ -0,0 +1,190 @@
|
||||
"""Tests for the motion search hardware-accelerated decode helpers."""
|
||||
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
from unittest import mock
|
||||
|
||||
from frigate.jobs.motion_search_decode import (
|
||||
KEYFRAME_MAX_GAP_SECONDS,
|
||||
build_vod_decode_command,
|
||||
keyframe_sampling_eligible,
|
||||
probe_video_dimensions,
|
||||
probe_vod_keyframe_pts,
|
||||
resolve_motion_decode_args,
|
||||
)
|
||||
|
||||
|
||||
def _fake_camera_config(
|
||||
hwaccel_args, gpu=0, fps=5, width=1280, height=720, ffmpeg_path="ffmpeg"
|
||||
):
|
||||
return SimpleNamespace(
|
||||
ffmpeg=SimpleNamespace(
|
||||
hwaccel_args=hwaccel_args, gpu=gpu, ffmpeg_path=ffmpeg_path
|
||||
),
|
||||
detect=SimpleNamespace(fps=fps, width=width, height=height),
|
||||
)
|
||||
|
||||
|
||||
class TestResolveMotionDecodeArgs(unittest.TestCase):
|
||||
def test_vaapi_preset_is_accelerated(self):
|
||||
args = resolve_motion_decode_args(_fake_camera_config("preset-vaapi"))
|
||||
self.assertIn("-hwaccel", args)
|
||||
self.assertIn("vaapi", args)
|
||||
|
||||
def test_non_nv12_preset_falls_back_to_software(self):
|
||||
# rkmpp produces drm_prime surfaces that do not download to nv12, so it
|
||||
# must resolve to software decode (empty args) rather than risk corrupt
|
||||
# frames.
|
||||
self.assertEqual(
|
||||
resolve_motion_decode_args(_fake_camera_config("preset-rkmpp")), []
|
||||
)
|
||||
|
||||
def test_custom_args_fall_back_to_software(self):
|
||||
# Arbitrary custom hwaccel args (a list, not a preset) decode in software
|
||||
# to preserve byte-identical results.
|
||||
self.assertEqual(
|
||||
resolve_motion_decode_args(_fake_camera_config(["-hwaccel", "vulkan"])),
|
||||
[],
|
||||
)
|
||||
|
||||
def test_nvidia_codec_preset_is_accelerated(self):
|
||||
# Codec-specific nvidia presets resolve to the same cuda decode args as
|
||||
# the bare preset, so eligibility is derived from -hwaccel_output_format
|
||||
# rather than a hardcoded list that omitted these aliases.
|
||||
args = resolve_motion_decode_args(_fake_camera_config("preset-nvidia-h264"))
|
||||
self.assertIn("-hwaccel_output_format", args)
|
||||
self.assertIn("cuda", args)
|
||||
|
||||
def test_software_only_preset_falls_back_to_software(self):
|
||||
# A preset with no -hwaccel_output_format (decoder-based, no GPU surface)
|
||||
# cannot use the nv12 download step, so it decodes in software.
|
||||
self.assertEqual(
|
||||
resolve_motion_decode_args(_fake_camera_config("preset-rpi-64-h264")), []
|
||||
)
|
||||
|
||||
|
||||
class TestKeyframeEligibility(unittest.TestCase):
|
||||
def test_regular_short_gop_is_eligible(self):
|
||||
pts = [0.0, 0.5, 1.0, 1.5, 2.0] # 0.5s gaps
|
||||
self.assertTrue(keyframe_sampling_eligible(pts))
|
||||
|
||||
def test_long_gop_is_ineligible(self):
|
||||
pts = [0.0, 5.0, 10.0] # 5s gaps
|
||||
self.assertFalse(keyframe_sampling_eligible(pts))
|
||||
|
||||
def test_irregular_gop_ineligible_when_a_gap_is_long(self):
|
||||
pts = [0.0, 0.5, 1.0, 8.0] # one 7s gap
|
||||
self.assertFalse(keyframe_sampling_eligible(pts))
|
||||
|
||||
def test_too_few_keyframes_ineligible(self):
|
||||
self.assertFalse(keyframe_sampling_eligible([1.0]))
|
||||
self.assertFalse(keyframe_sampling_eligible([]))
|
||||
|
||||
def test_default_max_gap_constant(self):
|
||||
self.assertEqual(KEYFRAME_MAX_GAP_SECONDS, 2.0)
|
||||
|
||||
|
||||
class TestVodDecodeCommand(unittest.TestCase):
|
||||
URL = "http://127.0.0.1:5000/vod/cam/start/1/end/2/index.m3u8"
|
||||
|
||||
def test_keyframe_command_shape(self):
|
||||
cmd = build_vod_decode_command(
|
||||
"ffmpeg",
|
||||
self.URL,
|
||||
decode_args=[],
|
||||
crop=(100, 80, 10, 20),
|
||||
scale=(50, 40),
|
||||
gray=True,
|
||||
skip_nonkey=True,
|
||||
fps_rate=None,
|
||||
)
|
||||
joined = " ".join(cmd)
|
||||
self.assertIn("-skip_frame nokey", joined)
|
||||
self.assertIn("-protocol_whitelist pipe,file,http,tcp", joined)
|
||||
self.assertIn(f"-i {self.URL}", joined)
|
||||
self.assertIn("crop=100:80:10:20", joined)
|
||||
self.assertIn("scale=50:40", joined)
|
||||
self.assertIn("-pix_fmt gray", joined)
|
||||
self.assertNotIn("fps=", joined)
|
||||
|
||||
def test_fps_command_uses_fps_filter_not_skip_frame(self):
|
||||
cmd = build_vod_decode_command(
|
||||
"ffmpeg",
|
||||
self.URL,
|
||||
decode_args=[],
|
||||
crop=None,
|
||||
scale=None,
|
||||
gray=False,
|
||||
skip_nonkey=False,
|
||||
fps_rate=2.0,
|
||||
)
|
||||
joined = " ".join(cmd)
|
||||
self.assertNotIn("skip_frame", joined)
|
||||
self.assertIn("fps=2.0", joined)
|
||||
self.assertIn("-pix_fmt bgr24", joined)
|
||||
|
||||
def test_hwaccel_inserts_hwdownload(self):
|
||||
cmd = build_vod_decode_command(
|
||||
"ffmpeg",
|
||||
self.URL,
|
||||
decode_args=["-hwaccel", "vaapi"],
|
||||
crop=None,
|
||||
scale=None,
|
||||
gray=True,
|
||||
skip_nonkey=True,
|
||||
fps_rate=None,
|
||||
)
|
||||
joined = " ".join(cmd)
|
||||
self.assertIn("hwdownload", joined)
|
||||
self.assertIn("format=nv12", joined)
|
||||
|
||||
|
||||
class TestProbeVodKeyframePts(unittest.TestCase):
|
||||
def test_parses_keyframe_packets(self):
|
||||
sample = (
|
||||
'{"packets":['
|
||||
'{"pts_time":"0.000000","flags":"K__"},'
|
||||
'{"pts_time":"1.000000","flags":"___"},'
|
||||
'{"pts_time":"2.000000","flags":"K__"}]}'
|
||||
)
|
||||
completed = mock.Mock(stdout=sample, returncode=0)
|
||||
with mock.patch(
|
||||
"frigate.jobs.motion_search_decode.sp.run", return_value=completed
|
||||
):
|
||||
pts = probe_vod_keyframe_pts("ffprobe", "http://x/index.m3u8")
|
||||
self.assertEqual(pts, [0.0, 2.0])
|
||||
|
||||
def test_returns_empty_on_failure(self):
|
||||
with mock.patch(
|
||||
"frigate.jobs.motion_search_decode.sp.run",
|
||||
side_effect=OSError("boom"),
|
||||
):
|
||||
self.assertEqual(probe_vod_keyframe_pts("ffprobe", "http://x"), [])
|
||||
|
||||
|
||||
class TestProbeVideoDimensions(unittest.TestCase):
|
||||
def test_parses_dimensions_and_fps(self):
|
||||
sample = (
|
||||
'{"streams":[{"width":1920,"height":1080,"avg_frame_rate":"30000/1001"}]}'
|
||||
)
|
||||
completed = mock.Mock(stdout=sample, returncode=0)
|
||||
with mock.patch(
|
||||
"frigate.jobs.motion_search_decode.sp.run", return_value=completed
|
||||
):
|
||||
dims = probe_video_dimensions("ffprobe", "/tmp/a.mp4")
|
||||
assert dims is not None
|
||||
width, height, fps = dims
|
||||
self.assertEqual((width, height), (1920, 1080))
|
||||
self.assertAlmostEqual(fps, 29.97, places=2)
|
||||
|
||||
def test_returns_none_on_zero_dimensions(self):
|
||||
sample = '{"streams":[{"width":0,"height":0,"avg_frame_rate":"0/0"}]}'
|
||||
completed = mock.Mock(stdout=sample, returncode=0)
|
||||
with mock.patch(
|
||||
"frigate.jobs.motion_search_decode.sp.run", return_value=completed
|
||||
):
|
||||
self.assertIsNone(probe_video_dimensions("ffprobe", "/tmp/a.mp4"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
87
frigate/test/test_motion_search_spatial.py
Normal file
87
frigate/test/test_motion_search_spatial.py
Normal file
@ -0,0 +1,87 @@
|
||||
"""Tests for motion search spatial (crop/scale/mask) helpers."""
|
||||
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
|
||||
from frigate.jobs.motion_search import (
|
||||
build_scaled_roi_mask,
|
||||
compute_roi_crop_and_scale,
|
||||
detect_motion_scaled,
|
||||
)
|
||||
|
||||
|
||||
class TestComputeRoiCropAndScale(unittest.TestCase):
|
||||
def test_crop_box_in_record_pixels(self):
|
||||
# ROI covering x [0.25, 0.75], y [0.5, 1.0] of a 1000x600 frame.
|
||||
polygon = [[0.25, 0.5], [0.75, 0.5], [0.75, 1.0], [0.25, 1.0]]
|
||||
crop, scaled = compute_roi_crop_and_scale(polygon, 1000, 600, scale_target=125)
|
||||
cw, ch, cx, cy = crop
|
||||
self.assertEqual((cx, cy), (250, 300))
|
||||
self.assertEqual((cw, ch), (500, 300))
|
||||
# longest side 500 -> factor 0.25 -> (125, 75), rounded down to even.
|
||||
self.assertEqual(scaled, (124, 74))
|
||||
|
||||
def test_never_upscales(self):
|
||||
polygon = [[0.0, 0.0], [0.1, 0.0], [0.1, 0.1], [0.0, 0.1]]
|
||||
crop, scaled = compute_roi_crop_and_scale(polygon, 200, 200, scale_target=400)
|
||||
cw, ch, _, _ = crop
|
||||
# crop is 20x20; target 400 would upscale, so scaled == crop size.
|
||||
self.assertEqual(scaled, (cw, ch))
|
||||
|
||||
def test_scaled_dims_are_at_least_one(self):
|
||||
polygon = [[0.0, 0.0], [0.02, 0.0], [0.02, 0.02], [0.0, 0.02]]
|
||||
crop, scaled = compute_roi_crop_and_scale(polygon, 50, 50, scale_target=1)
|
||||
self.assertGreaterEqual(scaled[0], 1)
|
||||
self.assertGreaterEqual(scaled[1], 1)
|
||||
|
||||
def test_all_dims_are_even_for_nv12(self):
|
||||
# Odd-aligned ROI on an odd-ish frame must still yield even crop/scale so
|
||||
# the nv12 hwdownload byte stream matches the expected frame size.
|
||||
polygon = [[0.123, 0.321], [0.777, 0.321], [0.777, 0.901], [0.123, 0.901]]
|
||||
crop, scaled = compute_roi_crop_and_scale(polygon, 1377, 911, scale_target=257)
|
||||
for value in (*crop, *scaled):
|
||||
self.assertEqual(value % 2, 0, f"{value} is not even")
|
||||
|
||||
|
||||
class TestBuildScaledRoiMask(unittest.TestCase):
|
||||
def test_mask_matches_scaled_dims_and_has_coverage(self):
|
||||
polygon = [[0.25, 0.5], [0.75, 0.5], [0.75, 1.0], [0.25, 1.0]]
|
||||
crop, scaled = compute_roi_crop_and_scale(polygon, 1000, 600, scale_target=125)
|
||||
mask = build_scaled_roi_mask(polygon, 1000, 600, crop, scaled)
|
||||
self.assertEqual(mask.shape, (scaled[1], scaled[0]))
|
||||
self.assertEqual(mask.dtype, np.uint8)
|
||||
# A full rectangle ROI fills its whole crop -> mask is all 255.
|
||||
self.assertGreater(np.count_nonzero(mask), 0)
|
||||
self.assertEqual(np.count_nonzero(mask), mask.size)
|
||||
|
||||
|
||||
class TestDetectMotionScaled(unittest.TestCase):
|
||||
def _ts(self, idx):
|
||||
return float(idx)
|
||||
|
||||
def test_finds_change_between_frames(self):
|
||||
mask = np.full((60, 80), 255, dtype=np.uint8)
|
||||
f0 = np.zeros((60, 80), dtype=np.uint8)
|
||||
f1 = np.zeros((60, 80), dtype=np.uint8)
|
||||
f1[10:50, 20:60] = 255 # big bright block appears
|
||||
frames = [(0, f0), (30, f1)]
|
||||
results = detect_motion_scaled(
|
||||
frames, mask, threshold=30, min_area=1.0, timestamp_fn=self._ts
|
||||
)
|
||||
self.assertEqual(len(results), 1)
|
||||
self.assertEqual(results[0].timestamp, 30.0)
|
||||
self.assertGreater(results[0].change_percentage, 0.0)
|
||||
|
||||
def test_no_change_yields_nothing(self):
|
||||
mask = np.full((60, 80), 255, dtype=np.uint8)
|
||||
f0 = np.zeros((60, 80), dtype=np.uint8)
|
||||
f1 = np.zeros((60, 80), dtype=np.uint8)
|
||||
results = detect_motion_scaled(
|
||||
[(0, f0), (30, f1)], mask, threshold=30, min_area=1.0, timestamp_fn=self._ts
|
||||
)
|
||||
self.assertEqual(results, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@ -8,6 +8,7 @@
|
||||
"searchCancelled": "Search cancelled",
|
||||
"cancelSearch": "Cancel",
|
||||
"searching": "Search in progress.",
|
||||
"scanning": "Scanning {{time}}",
|
||||
"searchComplete": "Search complete",
|
||||
"noResultsYet": "Run a search to find motion changes in the selected region",
|
||||
"noChangesFound": "No pixel changes detected in the selected region",
|
||||
@ -42,13 +43,11 @@
|
||||
"settings": {
|
||||
"title": "Search Settings",
|
||||
"parallelMode": "Parallel mode",
|
||||
"parallelModeDesc": "Scan multiple recording segments at the same time (faster, but significantly more CPU intensive)",
|
||||
"parallelModeDesc": "Scan multiple recording ranges at the same time (faster; uses more decoding resources)",
|
||||
"threshold": "Sensitivity Threshold",
|
||||
"thresholdDesc": "Lower values detect smaller changes (1-255)",
|
||||
"minArea": "Minimum Change Area",
|
||||
"minAreaDesc": "Minimum percentage of the region of interest that must change to be considered significant",
|
||||
"frameSkip": "Frame Skip",
|
||||
"frameSkipDesc": "Process every Nth frame. Set this to your camera's frame rate to process one frame per second (e.g. 5 for a 5 FPS camera, 30 for a 30 FPS camera). Higher values will be faster, but may miss short motion events.",
|
||||
"minAreaDesc": "Minimum size of a single moving region, as a percentage of the region of interest",
|
||||
"maxResults": "Maximum Results",
|
||||
"maxResultsDesc": "Stop after this many matching timestamps"
|
||||
},
|
||||
@ -72,6 +71,8 @@
|
||||
"framesDecoded": "Frames decoded",
|
||||
"wallTime": "Search time",
|
||||
"segmentErrors": "Segment errors",
|
||||
"seconds": "{{seconds}}s"
|
||||
"seconds": "{{seconds}}s",
|
||||
"minutesSeconds": "{{minutes}}m {{seconds}}s",
|
||||
"scanSummary": "{{segments}} segments · {{time}}"
|
||||
}
|
||||
}
|
||||
|
||||
@ -13,6 +13,13 @@ import { useTimezone } from "@/hooks/use-date-utils";
|
||||
|
||||
type WeekStartsOnType = 0 | 1 | 2 | 3 | 4 | 5 | 6;
|
||||
|
||||
function formatCalendarDay(day: Date): string {
|
||||
const y = day.getFullYear();
|
||||
const m = String(day.getMonth() + 1).padStart(2, "0");
|
||||
const d = String(day.getDate()).padStart(2, "0");
|
||||
return `${y}-${m}-${d}`;
|
||||
}
|
||||
|
||||
type ReviewActivityCalendarProps = {
|
||||
reviewSummary?: ReviewSummary;
|
||||
recordingsSummary?: RecordingsSummary;
|
||||
@ -62,17 +69,10 @@ export default function ReviewActivityCalendar({
|
||||
}
|
||||
}
|
||||
|
||||
const formatDay = (day: Date) => {
|
||||
const y = day.getFullYear();
|
||||
const m = String(day.getMonth() + 1).padStart(2, "0");
|
||||
const d = String(day.getDate()).padStart(2, "0");
|
||||
return `${y}-${m}-${d}`;
|
||||
};
|
||||
|
||||
return {
|
||||
recordings: (day: Date) => recordingsSet.has(formatDay(day)),
|
||||
alerts: (day: Date) => alertsSet.has(formatDay(day)),
|
||||
detections: (day: Date) => detectionsSet.has(formatDay(day)),
|
||||
recordings: (day: Date) => recordingsSet.has(formatCalendarDay(day)),
|
||||
alerts: (day: Date) => alertsSet.has(formatCalendarDay(day)),
|
||||
detections: (day: Date) => detectionsSet.has(formatCalendarDay(day)),
|
||||
};
|
||||
}, [reviewSummary, recordingsSummary]);
|
||||
|
||||
@ -156,14 +156,32 @@ type TimezoneAwareCalendarProps = {
|
||||
timezone?: string;
|
||||
selectedDay?: Date;
|
||||
onSelect: (day?: Date) => void;
|
||||
recordingsSummary?: RecordingsSummary;
|
||||
};
|
||||
export function TimezoneAwareCalendar({
|
||||
timezone,
|
||||
selectedDay,
|
||||
onSelect,
|
||||
recordingsSummary,
|
||||
}: TimezoneAwareCalendarProps) {
|
||||
const [weekStartsOn] = useUserPersistence("weekStartsOn", 0);
|
||||
|
||||
// When a recordings summary is supplied, underline days that have footage
|
||||
const recordingsModifier = useMemo(() => {
|
||||
if (!recordingsSummary) {
|
||||
return undefined;
|
||||
}
|
||||
const recordingsSet = new Set<string>();
|
||||
for (const date of Object.keys(recordingsSummary)) {
|
||||
if (date !== LAST_24_HOURS_KEY) {
|
||||
recordingsSet.add(date);
|
||||
}
|
||||
}
|
||||
return {
|
||||
recordings: (day: Date) => recordingsSet.has(formatCalendarDay(day)),
|
||||
};
|
||||
}, [recordingsSummary]);
|
||||
|
||||
const timezoneOffset = useMemo(
|
||||
() =>
|
||||
timezone ? Math.round(getUTCOffset(new Date(), timezone)) : undefined,
|
||||
@ -217,6 +235,10 @@ export function TimezoneAwareCalendar({
|
||||
onSelect={onSelect}
|
||||
defaultMonth={selectedDay ?? new Date()}
|
||||
weekStartsOn={(weekStartsOn ?? 0) as WeekStartsOnType}
|
||||
modifiers={recordingsModifier}
|
||||
components={
|
||||
recordingsModifier ? { DayButton: ReviewActivityDay } : undefined
|
||||
}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
@ -14,7 +14,6 @@ export interface MotionSearchRequest {
|
||||
parallel?: boolean;
|
||||
threshold?: number;
|
||||
min_area?: number;
|
||||
frame_skip?: number;
|
||||
max_results?: number;
|
||||
}
|
||||
|
||||
@ -43,4 +42,21 @@ export interface MotionSearchStatusResponse {
|
||||
total_frames_processed?: number;
|
||||
error_message?: string;
|
||||
metrics?: MotionSearchMetrics;
|
||||
scanning_timestamp?: number;
|
||||
progress?: number;
|
||||
}
|
||||
|
||||
export interface MotionSearchJobResults {
|
||||
results: MotionSearchResult[];
|
||||
total_frames_processed: number;
|
||||
}
|
||||
|
||||
export interface MotionSearchJobPayload {
|
||||
id?: string;
|
||||
status: "queued" | "running" | "success" | "failed" | "cancelled";
|
||||
results?: MotionSearchJobResults;
|
||||
metrics?: MotionSearchMetrics;
|
||||
scanning_timestamp?: number;
|
||||
progress?: number;
|
||||
error_message?: string;
|
||||
}
|
||||
|
||||
@ -7,6 +7,7 @@ import { LuHand, LuPencil } from "react-icons/lu";
|
||||
|
||||
import { FrigateConfig } from "@/types/frigateConfig";
|
||||
import { TimeRange } from "@/types/timeline";
|
||||
import { RecordingsSummary } from "@/types/review";
|
||||
import { ASPECT_PORTRAIT_LAYOUT, ASPECT_WIDE_LAYOUT } from "@/types/record";
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
@ -44,7 +45,11 @@ import { CameraNameLabel } from "@/components/camera/FriendlyNameLabel";
|
||||
import { TimezoneAwareCalendar } from "@/components/overlay/ReviewActivityCalendar";
|
||||
|
||||
import { useApiHost } from "@/api";
|
||||
import { useFormattedTimestamp, use24HourTime } from "@/hooks/use-date-utils";
|
||||
import {
|
||||
useFormattedTimestamp,
|
||||
use24HourTime,
|
||||
useTimezone,
|
||||
} from "@/hooks/use-date-utils";
|
||||
import { getUTCOffset } from "@/utils/dateUtil";
|
||||
import useSWR from "swr";
|
||||
import { cn } from "@/lib/utils";
|
||||
@ -69,8 +74,6 @@ type MotionSearchDialogProps = {
|
||||
setThreshold: React.Dispatch<React.SetStateAction<number>>;
|
||||
minArea: number;
|
||||
setMinArea: React.Dispatch<React.SetStateAction<number>>;
|
||||
frameSkip: number;
|
||||
setFrameSkip: React.Dispatch<React.SetStateAction<number>>;
|
||||
maxResults: number;
|
||||
setMaxResults: React.Dispatch<React.SetStateAction<number>>;
|
||||
searchRange?: TimeRange;
|
||||
@ -100,8 +103,6 @@ export default function MotionSearchDialog({
|
||||
setThreshold,
|
||||
minArea,
|
||||
setMinArea,
|
||||
frameSkip,
|
||||
setFrameSkip,
|
||||
maxResults,
|
||||
setMaxResults,
|
||||
searchRange,
|
||||
@ -117,6 +118,16 @@ export default function MotionSearchDialog({
|
||||
const [imageLoaded, setImageLoaded] = useState(false);
|
||||
const [panMode, setPanMode] = useState(false);
|
||||
|
||||
const recordingsTimezone = useTimezone(config);
|
||||
const { data: recordingsSummary } = useSWR<RecordingsSummary>(
|
||||
selectedCamera
|
||||
? [
|
||||
"recordings/summary",
|
||||
{ timezone: recordingsTimezone, cameras: selectedCamera },
|
||||
]
|
||||
: null,
|
||||
);
|
||||
|
||||
const cameraConfig = useMemo(() => {
|
||||
if (!selectedCamera) return undefined;
|
||||
return config.cameras[selectedCamera];
|
||||
@ -437,23 +448,6 @@ export default function MotionSearchDialog({
|
||||
{t("settings.minAreaDesc")}
|
||||
</p>
|
||||
</div>
|
||||
<div className="grid gap-2">
|
||||
<Label htmlFor="frameSkip">{t("settings.frameSkip")}</Label>
|
||||
<div className="flex items-center gap-2">
|
||||
<Slider
|
||||
id="frameSkip"
|
||||
min={1}
|
||||
max={120}
|
||||
step={1}
|
||||
value={[frameSkip]}
|
||||
onValueChange={([value]) => setFrameSkip(value)}
|
||||
/>
|
||||
<span className="w-12 text-sm">{frameSkip}</span>
|
||||
</div>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{t("settings.frameSkipDesc")}
|
||||
</p>
|
||||
</div>
|
||||
<div className="grid gap-2">
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<Label htmlFor="parallelMode">
|
||||
@ -496,6 +490,7 @@ export default function MotionSearchDialog({
|
||||
setRange={setSearchRange}
|
||||
defaultRange={defaultRange}
|
||||
timezone={timezone}
|
||||
recordingsSummary={recordingsSummary}
|
||||
/>
|
||||
|
||||
<Button
|
||||
@ -519,6 +514,7 @@ type SearchRangeSelectorProps = {
|
||||
setRange: React.Dispatch<React.SetStateAction<TimeRange | undefined>>;
|
||||
defaultRange: TimeRange;
|
||||
timezone?: string;
|
||||
recordingsSummary?: RecordingsSummary;
|
||||
};
|
||||
|
||||
function SearchRangeSelector({
|
||||
@ -526,6 +522,7 @@ function SearchRangeSelector({
|
||||
setRange,
|
||||
defaultRange,
|
||||
timezone,
|
||||
recordingsSummary,
|
||||
}: SearchRangeSelectorProps) {
|
||||
const { t } = useTranslation(["views/motionSearch", "common"]);
|
||||
const [startOpen, setStartOpen] = useState(false);
|
||||
@ -630,6 +627,7 @@ function SearchRangeSelector({
|
||||
<PopoverContent className="flex flex-col items-center">
|
||||
<TimezoneAwareCalendar
|
||||
timezone={timezone}
|
||||
recordingsSummary={recordingsSummary}
|
||||
selectedDay={new Date(startTime * 1000)}
|
||||
onSelect={(day) => {
|
||||
if (!day) {
|
||||
@ -696,6 +694,7 @@ function SearchRangeSelector({
|
||||
<PopoverContent className="flex flex-col items-center">
|
||||
<TimezoneAwareCalendar
|
||||
timezone={timezone}
|
||||
recordingsSummary={recordingsSummary}
|
||||
selectedDay={new Date(endTime * 1000)}
|
||||
onSelect={(day) => {
|
||||
if (!day) {
|
||||
|
||||
@ -12,10 +12,12 @@ import { ExportMode } from "@/types/filter";
|
||||
import {
|
||||
MotionSearchRequest,
|
||||
MotionSearchStartResponse,
|
||||
MotionSearchStatusResponse,
|
||||
MotionSearchResult,
|
||||
MotionSearchMetrics,
|
||||
MotionSearchJobResults,
|
||||
MotionSearchJobPayload,
|
||||
} from "@/types/motionSearch";
|
||||
import { useJobStatus } from "@/api/ws";
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Switch } from "@/components/ui/switch";
|
||||
@ -25,6 +27,11 @@ import { cn } from "@/lib/utils";
|
||||
import { Skeleton } from "@/components/ui/skeleton";
|
||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||
import { Progress } from "@/components/ui/progress";
|
||||
import {
|
||||
Collapsible,
|
||||
CollapsibleContent,
|
||||
CollapsibleTrigger,
|
||||
} from "@/components/ui/collapsible";
|
||||
|
||||
import DynamicVideoPlayer from "@/components/player/dynamic/DynamicVideoPlayer";
|
||||
import { DynamicVideoController } from "@/components/player/dynamic/DynamicVideoController";
|
||||
@ -44,7 +51,7 @@ import { useTimelineUtils } from "@/hooks/use-timeline-utils";
|
||||
import { useCameraPreviews } from "@/hooks/use-camera-previews";
|
||||
import { getChunkedTimeDay } from "@/utils/timelineUtil";
|
||||
|
||||
import { MotionData, ZoomLevel } from "@/types/review";
|
||||
import { MotionData, REVIEW_PADDING, ZoomLevel } from "@/types/review";
|
||||
import {
|
||||
ASPECT_VERTICAL_LAYOUT,
|
||||
ASPECT_WIDE_LAYOUT,
|
||||
@ -52,14 +59,17 @@ import {
|
||||
RecordingSegment,
|
||||
} from "@/types/record";
|
||||
import { VideoResolutionType } from "@/types/live";
|
||||
import { useFormattedTimestamp, use24HourTime } from "@/hooks/use-date-utils";
|
||||
import {
|
||||
useFormattedTimestamp,
|
||||
useFormattedRange,
|
||||
use24HourTime,
|
||||
} from "@/hooks/use-date-utils";
|
||||
import MotionSearchROICanvas from "./MotionSearchROICanvas";
|
||||
import MotionSearchDialog from "./MotionSearchDialog";
|
||||
import { IoMdArrowRoundBack } from "react-icons/io";
|
||||
import { FaArrowDown, FaCalendarAlt, FaCog, FaFire } from "react-icons/fa";
|
||||
import { useNavigate } from "react-router-dom";
|
||||
import { LuSearch } from "react-icons/lu";
|
||||
import ActivityIndicator from "@/components/indicators/activity-indicator";
|
||||
import { LuSearch, LuChevronRight, LuX } from "react-icons/lu";
|
||||
|
||||
type MotionSearchViewProps = {
|
||||
config: FrigateConfig;
|
||||
@ -118,10 +128,11 @@ export default function MotionSearchView({
|
||||
"actions" | "calendar"
|
||||
>("actions");
|
||||
|
||||
// Recordings summary for calendar – defer until dialog is closed
|
||||
// so the preview image in the dialog loads without competing requests
|
||||
// Recordings summary for the calendars (main view + search dialog). Fetched
|
||||
// whenever a camera is selected so it is cached and ready by the time the
|
||||
// dialog's date pickers render, regardless of open/closed state.
|
||||
const { data: recordingsSummary } = useSWR<RecordingsSummary>(
|
||||
selectedCamera && !isSearchDialogOpen
|
||||
selectedCamera
|
||||
? [
|
||||
"recordings/summary",
|
||||
{
|
||||
@ -146,17 +157,18 @@ export default function MotionSearchView({
|
||||
const [parallelMode, setParallelMode] = useState(false);
|
||||
const [threshold, setThreshold] = useState(30);
|
||||
const [minArea, setMinArea] = useState(20);
|
||||
const [frameSkip, setFrameSkip] = useState(30);
|
||||
const [maxResults, setMaxResults] = useState(25);
|
||||
|
||||
// Job state
|
||||
const [jobId, setJobId] = useState<string | null>(null);
|
||||
const [jobCamera, setJobCamera] = useState<string | null>(null);
|
||||
|
||||
// Job polling with SWR
|
||||
const { data: jobStatus } = useSWR<MotionSearchStatusResponse>(
|
||||
jobId && jobCamera ? [`${jobCamera}/search/motion/${jobId}`] : null,
|
||||
{ refreshInterval: 1000 },
|
||||
const { payload: jobPayload } =
|
||||
useJobStatus<MotionSearchJobResults>("motion_search");
|
||||
const jobStatus = jobPayload as MotionSearchJobPayload | null;
|
||||
const formattedScanningTimestamp = useFormattedTimestamp(
|
||||
jobStatus?.scanning_timestamp ?? 0,
|
||||
resultTimestampFormat,
|
||||
timezone,
|
||||
);
|
||||
|
||||
// Search state
|
||||
@ -170,6 +182,15 @@ export default function MotionSearchView({
|
||||
undefined,
|
||||
);
|
||||
const [pendingSeekTime, setPendingSeekTime] = useState<number | null>(null);
|
||||
const pendingSeekTimeRef = useRef<number | null>(null);
|
||||
|
||||
// Formatted search window shown above the results (same date+time convention).
|
||||
const formattedSearchRange = useFormattedRange(
|
||||
searchRange?.after ?? 0,
|
||||
searchRange?.before ?? 0,
|
||||
resultTimestampFormat,
|
||||
timezone,
|
||||
);
|
||||
|
||||
// Export state
|
||||
const [exportMode, setExportMode] = useState<ExportMode>("none");
|
||||
@ -622,7 +643,7 @@ export default function MotionSearchView({
|
||||
]);
|
||||
|
||||
useEffect(() => {
|
||||
if (pendingSeekTime != null) {
|
||||
if (pendingSeekTimeRef.current != null) {
|
||||
return;
|
||||
}
|
||||
|
||||
@ -636,7 +657,7 @@ export default function MotionSearchView({
|
||||
setPlaybackStart(nextTime);
|
||||
setSelectedRangeIdx(index === -1 ? chunkedTimeRange.length - 1 : index);
|
||||
mainControllerRef.current?.seekToTimestamp(nextTime, true);
|
||||
}, [pendingSeekTime, timeRange, chunkedTimeRange]);
|
||||
}, [timeRange, chunkedTimeRange]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!scrubbing) {
|
||||
@ -733,10 +754,9 @@ export default function MotionSearchView({
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
cancelMotionSearchJobViaBeacon(jobIdRef.current, jobCameraRef.current);
|
||||
void cancelMotionSearchJob(jobIdRef.current, jobCameraRef.current);
|
||||
};
|
||||
}, [cancelMotionSearchJob, cancelMotionSearchJobViaBeacon]);
|
||||
}, [cancelMotionSearchJob]);
|
||||
|
||||
useEffect(() => {
|
||||
const handleBeforeUnload = () => {
|
||||
@ -802,7 +822,6 @@ export default function MotionSearchView({
|
||||
parallel: parallelMode,
|
||||
threshold,
|
||||
min_area: minArea,
|
||||
frame_skip: frameSkip,
|
||||
max_results: maxResults,
|
||||
};
|
||||
|
||||
@ -877,7 +896,6 @@ export default function MotionSearchView({
|
||||
parallelMode,
|
||||
threshold,
|
||||
minArea,
|
||||
frameSkip,
|
||||
maxResults,
|
||||
t,
|
||||
]);
|
||||
@ -888,23 +906,27 @@ export default function MotionSearchView({
|
||||
return;
|
||||
}
|
||||
|
||||
if (!jobId || jobStatus.id !== jobId) {
|
||||
return;
|
||||
}
|
||||
|
||||
const resultList = jobStatus.results?.results;
|
||||
|
||||
if (jobStatus.status === "success") {
|
||||
setSearchResults(jobStatus.results ?? []);
|
||||
setSearchResults(resultList ?? []);
|
||||
setSearchMetrics(jobStatus.metrics ?? null);
|
||||
setIsSearching(false);
|
||||
setJobId(null);
|
||||
setJobCamera(null);
|
||||
toast.success(
|
||||
t("changesFound", { count: jobStatus.results?.length ?? 0 }),
|
||||
);
|
||||
toast.success(t("changesFound", { count: resultList?.length ?? 0 }));
|
||||
} else if (
|
||||
jobStatus.status === "queued" ||
|
||||
jobStatus.status === "running"
|
||||
) {
|
||||
setSearchMetrics(jobStatus.metrics ?? null);
|
||||
// Stream partial results as they arrive
|
||||
if (jobStatus.results && jobStatus.results.length > 0) {
|
||||
setSearchResults(jobStatus.results);
|
||||
if (resultList && resultList.length > 0) {
|
||||
setSearchResults(resultList);
|
||||
}
|
||||
} else if (jobStatus.status === "failed") {
|
||||
setIsSearching(false);
|
||||
@ -912,7 +934,7 @@ export default function MotionSearchView({
|
||||
setJobCamera(null);
|
||||
toast.error(
|
||||
t("errors.searchFailed", {
|
||||
message: jobStatus.error_message || jobStatus.message,
|
||||
message: jobStatus.error_message || t("errors.unknown"),
|
||||
}),
|
||||
);
|
||||
} else if (jobStatus.status === "cancelled") {
|
||||
@ -921,7 +943,7 @@ export default function MotionSearchView({
|
||||
setJobCamera(null);
|
||||
toast.message(t("searchCancelled"));
|
||||
}
|
||||
}, [jobStatus, t]);
|
||||
}, [jobStatus, jobId, t]);
|
||||
|
||||
// Handle result click
|
||||
const handleResultClick = useCallback(
|
||||
@ -930,12 +952,14 @@ export default function MotionSearchView({
|
||||
result.timestamp < timeRange.after ||
|
||||
result.timestamp > timeRange.before
|
||||
) {
|
||||
pendingSeekTimeRef.current = result.timestamp;
|
||||
setPendingSeekTime(result.timestamp);
|
||||
onDaySelect(new Date(result.timestamp * 1000));
|
||||
return;
|
||||
}
|
||||
|
||||
manuallySetCurrentTime(result.timestamp, true);
|
||||
// start playback a few seconds before the change so the motion is in view
|
||||
manuallySetCurrentTime(result.timestamp - REVIEW_PADDING, true);
|
||||
},
|
||||
[manuallySetCurrentTime, onDaySelect, timeRange],
|
||||
);
|
||||
@ -949,8 +973,9 @@ export default function MotionSearchView({
|
||||
pendingSeekTime >= timeRange.after &&
|
||||
pendingSeekTime <= timeRange.before
|
||||
) {
|
||||
manuallySetCurrentTime(pendingSeekTime, true);
|
||||
manuallySetCurrentTime(pendingSeekTime - REVIEW_PADDING, true);
|
||||
setPendingSeekTime(null);
|
||||
pendingSeekTimeRef.current = null;
|
||||
}
|
||||
}, [pendingSeekTime, timeRange, manuallySetCurrentTime]);
|
||||
|
||||
@ -1028,6 +1053,9 @@ export default function MotionSearchView({
|
||||
|
||||
const progressMetrics = jobStatus?.metrics ?? searchMetrics;
|
||||
const progressValue = (() => {
|
||||
if (jobStatus?.progress != null) {
|
||||
return Math.min(100, Math.max(0, jobStatus.progress * 100));
|
||||
}
|
||||
if (!progressMetrics || progressMetrics.segments_scanned <= 0) {
|
||||
return 0;
|
||||
}
|
||||
@ -1042,23 +1070,48 @@ export default function MotionSearchView({
|
||||
return Math.min(100, Math.max(0, (doneWork / totalWork) * 100));
|
||||
})();
|
||||
|
||||
const wallTimeLabel = searchMetrics
|
||||
? searchMetrics.wall_time_seconds >= 60
|
||||
? t("metrics.minutesSeconds", {
|
||||
minutes: Math.floor(searchMetrics.wall_time_seconds / 60),
|
||||
seconds: Math.round(searchMetrics.wall_time_seconds % 60),
|
||||
})
|
||||
: t("metrics.seconds", {
|
||||
seconds: searchMetrics.wall_time_seconds.toFixed(1),
|
||||
})
|
||||
: "";
|
||||
|
||||
const resultsPanel = (
|
||||
<>
|
||||
<div className="p-2">
|
||||
<h3 className="font-medium">{t("results")}</h3>
|
||||
</div>
|
||||
{(hasSearched || isSearching) && (
|
||||
<div className="flex flex-col gap-1 px-3 py-2.5">
|
||||
{searchRange && (
|
||||
<div className="text-sm font-medium text-foreground">
|
||||
{formattedSearchRange}
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics && (
|
||||
<div className="text-xs text-muted-foreground">
|
||||
{t("metrics.scanSummary", {
|
||||
segments: searchMetrics.segments_scanned,
|
||||
time: wallTimeLabel,
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<ScrollArea className="flex-1">
|
||||
<ScrollArea className="flex-1 [&>[data-radix-scroll-area-viewport]>div]:!block">
|
||||
{isSearching && (
|
||||
<div className="flex flex-col gap-2 border-b p-3 text-sm text-muted-foreground">
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<div className="flex flex-col gap-1 text-wrap">
|
||||
<ActivityIndicator className="mr-2 size-4" />
|
||||
<div>{t("searching")}</div>
|
||||
</div>
|
||||
<div className="flex flex-col gap-1.5 border-b p-3">
|
||||
<div className="flex items-center gap-2">
|
||||
<Progress className="h-1.5 flex-1" value={progressValue} />
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="sm"
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
className="size-6 shrink-0 text-muted-foreground"
|
||||
aria-label={t("cancelSearch")}
|
||||
title={t("cancelSearch")}
|
||||
onClick={() => {
|
||||
void cancelMotionSearchJob(jobId, jobCamera);
|
||||
setIsSearching(false);
|
||||
@ -1067,76 +1120,90 @@ export default function MotionSearchView({
|
||||
toast.success(t("searchCancelled"));
|
||||
}}
|
||||
>
|
||||
{t("cancelSearch")}
|
||||
<LuX className="size-4" />
|
||||
</Button>
|
||||
</div>
|
||||
<Progress className="h-1" value={progressValue} />
|
||||
<div className="text-xs text-muted-foreground">
|
||||
{jobStatus?.scanning_timestamp != null
|
||||
? t("scanning", { time: formattedScanningTimestamp })
|
||||
: t("searching")}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics && (isSearching || searchResults.length > 0) && (
|
||||
<div className="mx-2 my-3 rounded-lg border bg-secondary p-2">
|
||||
<div className="space-y-0.5 text-xs text-muted-foreground">
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.segmentsScanned")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.segments_scanned}
|
||||
</span>
|
||||
</div>
|
||||
{searchMetrics.segments_processed > 0 && (
|
||||
<div className="flex justify-between font-medium">
|
||||
<span>{t("metrics.segmentsProcessed")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.segments_processed}
|
||||
</span>
|
||||
{searchMetrics &&
|
||||
(isSearching || searchResults.length > 0 || hasSearched) && (
|
||||
<Collapsible>
|
||||
<CollapsibleTrigger className="group flex w-full items-center gap-1 px-3 py-2.5 text-left text-xs text-muted-foreground hover:bg-accent">
|
||||
<LuChevronRight className="size-3 shrink-0 transition-transform group-data-[state=open]:rotate-90" />
|
||||
{t("metrics.title")}
|
||||
</CollapsibleTrigger>
|
||||
<CollapsibleContent>
|
||||
<div className="space-y-0.5 px-3 pb-3 text-xs text-muted-foreground">
|
||||
{searchMetrics.segments_processed > 0 && (
|
||||
<div className="flex justify-between font-medium">
|
||||
<span>{t("metrics.segmentsProcessed")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.segments_processed}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics.metadata_inactive_segments > 0 && (
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.segmentsSkippedInactive")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.metadata_inactive_segments}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics.heatmap_roi_skip_segments > 0 && (
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.segmentsSkippedHeatmap")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.heatmap_roi_skip_segments}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics.fallback_full_range_segments > 0 && (
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.fallbackFullRange")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.fallback_full_range_segments}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.framesDecoded")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.frames_decoded}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.wallTime")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{wallTimeLabel}
|
||||
</span>
|
||||
</div>
|
||||
{searchMetrics.segments_with_errors > 0 && (
|
||||
<div className="flex justify-between text-destructive">
|
||||
<span>{t("metrics.segmentErrors")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.segments_with_errors}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics.metadata_inactive_segments > 0 && (
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.segmentsSkippedInactive")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.metadata_inactive_segments}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics.heatmap_roi_skip_segments > 0 && (
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.segmentsSkippedHeatmap")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.heatmap_roi_skip_segments}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{searchMetrics.fallback_full_range_segments > 0 && (
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.fallbackFullRange")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.fallback_full_range_segments}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.framesDecoded")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.frames_decoded}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span>{t("metrics.wallTime")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{t("metrics.seconds", {
|
||||
seconds: searchMetrics.wall_time_seconds.toFixed(1),
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
{searchMetrics.segments_with_errors > 0 && (
|
||||
<div className="flex justify-between text-destructive">
|
||||
<span>{t("metrics.segmentErrors")}</span>
|
||||
<span className="text-primary-variant">
|
||||
{searchMetrics.segments_with_errors}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</CollapsibleContent>
|
||||
</Collapsible>
|
||||
)}
|
||||
|
||||
{(searchResults.length > 0 || (hasSearched && !isSearching)) && (
|
||||
<div className="border-t px-1.5 pb-1.5 pt-3">
|
||||
<h3 className="text-sm font-medium tracking-wide text-muted-foreground">
|
||||
{searchResults.length > 0 && (
|
||||
<span className="ml-1.5">{searchResults.length}</span>
|
||||
)}{" "}
|
||||
{t("results")}
|
||||
</h3>
|
||||
</div>
|
||||
)}
|
||||
|
||||
@ -1145,7 +1212,7 @@ export default function MotionSearchView({
|
||||
{hasSearched ? t("noChangesFound") : t("noResultsYet")}
|
||||
</div>
|
||||
) : searchResults.length > 0 ? (
|
||||
<div className="flex flex-col gap-1 p-2">
|
||||
<div className="flex flex-col gap-1 px-1 pb-2">
|
||||
{searchResults.map((result, index) => (
|
||||
<SearchResultItem
|
||||
key={index}
|
||||
@ -1187,8 +1254,6 @@ export default function MotionSearchView({
|
||||
setThreshold={setThreshold}
|
||||
minArea={minArea}
|
||||
setMinArea={setMinArea}
|
||||
frameSkip={frameSkip}
|
||||
setFrameSkip={setFrameSkip}
|
||||
maxResults={maxResults}
|
||||
setMaxResults={setMaxResults}
|
||||
searchRange={searchRange}
|
||||
@ -1516,15 +1581,20 @@ function SearchResultItem({
|
||||
|
||||
return (
|
||||
<button
|
||||
className="flex w-full flex-col rounded-md p-2 text-left hover:bg-accent"
|
||||
className="flex w-full items-center justify-between gap-2 rounded-md p-2 text-left hover:bg-accent"
|
||||
onClick={onClick}
|
||||
title={t("jumpToTime")}
|
||||
>
|
||||
<span className="text-sm font-medium">{formattedTime}</span>
|
||||
<span className="text-xs text-muted-foreground">
|
||||
{t("changePercentage", {
|
||||
<span className="min-w-0 truncate text-sm font-medium">
|
||||
{formattedTime}
|
||||
</span>
|
||||
<span
|
||||
className="shrink-0 text-xs tabular-nums text-muted-foreground"
|
||||
title={t("changePercentage", {
|
||||
percentage: result.change_percentage.toFixed(1),
|
||||
})}
|
||||
>
|
||||
{result.change_percentage.toFixed(1)}%
|
||||
</span>
|
||||
</button>
|
||||
);
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user