mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-04-03 06:40:22 +00:00
Compare commits
7 Commits
6559c973bf
...
c9d4e3f33f
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c9d4e3f33f | ||
|
|
224cbdc2d6 | ||
|
|
3f9b153758 | ||
|
|
8e8346099e | ||
|
|
b0527df3c7 | ||
|
|
301e0a1a3a | ||
|
|
793906bb68 |
@ -320,6 +320,12 @@ http {
|
||||
add_header Cache-Control "public";
|
||||
}
|
||||
|
||||
location /fonts/ {
|
||||
access_log off;
|
||||
expires 1y;
|
||||
add_header Cache-Control "public";
|
||||
}
|
||||
|
||||
location /locales/ {
|
||||
access_log off;
|
||||
add_header Cache-Control "public";
|
||||
|
||||
@ -25,7 +25,7 @@ Examples of available modules are:
|
||||
|
||||
- `frigate.app`
|
||||
- `frigate.mqtt`
|
||||
- `frigate.object_detection`
|
||||
- `frigate.object_detection.base`
|
||||
- `detector.<detector_name>`
|
||||
- `watchdog.<camera_name>`
|
||||
- `ffmpeg.<camera_name>.<sorted_roles>` NOTE: All FFmpeg logs are sent as `error` level.
|
||||
|
||||
@ -35,6 +35,15 @@ For object classification:
|
||||
- Ideal when multiple attributes can coexist independently.
|
||||
- Example: Detecting if a `person` in a construction yard is wearing a helmet or not.
|
||||
|
||||
## Assignment Requirements
|
||||
|
||||
Sub labels and attributes are only assigned when both conditions are met:
|
||||
|
||||
1. **Threshold**: Each classification attempt must have a confidence score that meets or exceeds the configured `threshold` (default: `0.8`).
|
||||
2. **Class Consensus**: After at least 3 classification attempts, 60% of attempts must agree on the same class label. If the consensus class is `none`, no assignment is made.
|
||||
|
||||
This two-step verification prevents false positives by requiring consistent predictions across multiple frames before assigning a sub label or attribute.
|
||||
|
||||
## Example use cases
|
||||
|
||||
### Sub label
|
||||
@ -66,14 +75,18 @@ classification:
|
||||
|
||||
## Training the model
|
||||
|
||||
Creating and training the model is done within the Frigate UI using the `Classification` page.
|
||||
Creating and training the model is done within the Frigate UI using the `Classification` page. The process consists of two steps:
|
||||
|
||||
### Getting Started
|
||||
### Step 1: Name and Define
|
||||
|
||||
Enter a name for your model, select the object label to classify (e.g., `person`, `dog`, `car`), choose the classification type (sub label or attribute), and define your classes. Include a `none` class for objects that don't fit any specific category.
|
||||
|
||||
### Step 2: Assign Training Examples
|
||||
|
||||
The system will automatically generate example images from detected objects matching your selected label. You'll be guided through each class one at a time to select which images represent that class. Any images not assigned to a specific class will automatically be assigned to `none` when you complete the last class. Once all images are processed, training will begin automatically.
|
||||
|
||||
When choosing which objects to classify, start with a small number of visually distinct classes and ensure your training samples match camera viewpoints and distances typical for those objects.
|
||||
|
||||
// TODO add this section once UI is implemented. Explain process of selecting objects and curating training examples.
|
||||
|
||||
### Improving the Model
|
||||
|
||||
- **Problem framing**: Keep classes visually distinct and relevant to the chosen object types.
|
||||
|
||||
@ -48,13 +48,23 @@ classification:
|
||||
|
||||
## Training the model
|
||||
|
||||
Creating and training the model is done within the Frigate UI using the `Classification` page.
|
||||
Creating and training the model is done within the Frigate UI using the `Classification` page. The process consists of three steps:
|
||||
|
||||
### Getting Started
|
||||
### Step 1: Name and Define
|
||||
|
||||
When choosing a portion of the camera frame for state classification, it is important to make the crop tight around the area of interest to avoid extra signals unrelated to what is being classified.
|
||||
Enter a name for your model and define at least 2 classes (states) that represent mutually exclusive states. For example, `open` and `closed` for a door, or `on` and `off` for lights.
|
||||
|
||||
// TODO add this section once UI is implemented. Explain process of selecting a crop.
|
||||
### Step 2: Select the Crop Area
|
||||
|
||||
Choose one or more cameras and draw a rectangle over the area of interest for each camera. The crop should be tight around the region you want to classify to avoid extra signals unrelated to what is being classified. You can drag and resize the rectangle to adjust the crop area.
|
||||
|
||||
### Step 3: Assign Training Examples
|
||||
|
||||
The system will automatically generate example images from your camera feeds. You'll be guided through each class one at a time to select which images represent that state.
|
||||
|
||||
**Important**: All images must be assigned to a state before training can begin. This includes images that may not be optimal, such as when people temporarily block the view, sun glare is present, or other distractions occur. Assign these images to the state that is actually present (based on what you know the state to be), not based on the distraction. This training helps the model correctly identify the state even when such conditions occur during inference.
|
||||
|
||||
Once all images are assigned, training will begin automatically.
|
||||
|
||||
### Improving the Model
|
||||
|
||||
|
||||
@ -70,7 +70,7 @@ You should have at least 8 GB of RAM available (or VRAM if running on GPU) to ru
|
||||
genai:
|
||||
provider: ollama
|
||||
base_url: http://localhost:11434
|
||||
model: llava:7b
|
||||
model: qwen3-vl:4b
|
||||
```
|
||||
|
||||
## Google Gemini
|
||||
|
||||
@ -35,19 +35,18 @@ Each model is available in multiple parameter sizes (3b, 4b, 8b, etc.). Larger s
|
||||
|
||||
:::tip
|
||||
|
||||
If you are trying to use a single model for Frigate and HomeAssistant, it will need to support vision and tools calling. https://github.com/skye-harris/ollama-modelfiles contains optimized model configs for this task.
|
||||
If you are trying to use a single model for Frigate and HomeAssistant, it will need to support vision and tools calling. qwen3-VL supports vision and tools simultaneously in Ollama.
|
||||
|
||||
:::
|
||||
|
||||
The following models are recommended:
|
||||
|
||||
| Model | Notes |
|
||||
| ----------------- | ----------------------------------------------------------- |
|
||||
| `qwen3-vl` | Strong visual and situational understanding |
|
||||
| `Intern3.5VL` | Relatively fast with good vision comprehension |
|
||||
| `gemma3` | Strong frame-to-frame understanding, slower inference times |
|
||||
| `qwen2.5-vl` | Fast but capable model with good vision comprehension |
|
||||
| `llava-phi3` | Lightweight and fast model with vision comprehension |
|
||||
| Model | Notes |
|
||||
| ----------------- | -------------------------------------------------------------------- |
|
||||
| `qwen3-vl` | Strong visual and situational understanding, higher vram requirement |
|
||||
| `Intern3.5VL` | Relatively fast with good vision comprehension |
|
||||
| `gemma3` | Strong frame-to-frame understanding, slower inference times |
|
||||
| `qwen2.5-vl` | Fast but capable model with good vision comprehension |
|
||||
|
||||
:::note
|
||||
|
||||
|
||||
@ -962,7 +962,6 @@ model:
|
||||
# path: /config/yolov9.zip
|
||||
# The .zip file must contain:
|
||||
# ├── yolov9.dfp (a file ending with .dfp)
|
||||
# └── yolov9_post.onnx (optional; only if the model includes a cropped post-processing network)
|
||||
```
|
||||
|
||||
#### YOLOX
|
||||
@ -989,7 +988,7 @@ model:
|
||||
# Optional: The model is normally fetched through the runtime, so 'path' can be omitted unless you want to use a custom or local model.
|
||||
# path: /config/yolox.zip
|
||||
# The .zip file must contain:
|
||||
# ├── yolox.dfp (a file ending with .dfp)
|
||||
# ├── yolox.dfp (a file ending with .dfp)
|
||||
```
|
||||
|
||||
#### SSDLite MobileNet v2
|
||||
|
||||
@ -849,6 +849,7 @@ async def vod_ts(camera_name: str, start_ts: float, end_ts: float):
|
||||
|
||||
clips = []
|
||||
durations = []
|
||||
min_duration_ms = 100 # Minimum 100ms to ensure at least one video frame
|
||||
max_duration_ms = MAX_SEGMENT_DURATION * 1000
|
||||
|
||||
recording: Recordings
|
||||
@ -866,11 +867,11 @@ async def vod_ts(camera_name: str, start_ts: float, end_ts: float):
|
||||
if recording.end_time > end_ts:
|
||||
duration -= int((recording.end_time - end_ts) * 1000)
|
||||
|
||||
if duration <= 0:
|
||||
# skip if the clip has no valid duration
|
||||
if duration < min_duration_ms:
|
||||
# skip if the clip has no valid duration (too short to contain frames)
|
||||
continue
|
||||
|
||||
if 0 < duration < max_duration_ms:
|
||||
if min_duration_ms <= duration < max_duration_ms:
|
||||
clip["keyFrameDurations"] = [duration]
|
||||
clips.append(clip)
|
||||
durations.append(duration)
|
||||
|
||||
@ -792,6 +792,10 @@ class FrigateConfig(FrigateBaseModel):
|
||||
# copy over auth and proxy config in case auth needs to be enforced
|
||||
safe_config["auth"] = config.get("auth", {})
|
||||
safe_config["proxy"] = config.get("proxy", {})
|
||||
|
||||
# copy over database config for auth and so a new db is not created
|
||||
safe_config["database"] = config.get("database", {})
|
||||
|
||||
return cls.parse_object(safe_config, **context)
|
||||
|
||||
# Validate and return the config dict.
|
||||
|
||||
@ -18,7 +18,6 @@ from frigate.detectors.detector_config import (
|
||||
ModelTypeEnum,
|
||||
)
|
||||
from frigate.util.file import FileLock
|
||||
from frigate.util.model import post_process_yolo
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@ -178,13 +177,6 @@ class MemryXDetector(DetectionApi):
|
||||
logger.error(f"Failed to initialize MemryX model: {e}")
|
||||
raise
|
||||
|
||||
def load_yolo_constants(self):
|
||||
base = f"{self.cache_dir}/{self.model_folder}"
|
||||
# constants for yolov9 post-processing
|
||||
self.const_A = np.load(f"{base}/_model_22_Constant_9_output_0.npy")
|
||||
self.const_B = np.load(f"{base}/_model_22_Constant_10_output_0.npy")
|
||||
self.const_C = np.load(f"{base}/_model_22_Constant_12_output_0.npy")
|
||||
|
||||
def check_and_prepare_model(self):
|
||||
if not os.path.exists(self.cache_dir):
|
||||
os.makedirs(self.cache_dir, exist_ok=True)
|
||||
@ -236,7 +228,6 @@ class MemryXDetector(DetectionApi):
|
||||
|
||||
# Handle post model requirements by model type
|
||||
if self.memx_model_type in [
|
||||
ModelTypeEnum.yologeneric,
|
||||
ModelTypeEnum.yolonas,
|
||||
ModelTypeEnum.ssd,
|
||||
]:
|
||||
@ -245,7 +236,10 @@ class MemryXDetector(DetectionApi):
|
||||
f"No *_post.onnx file found in custom model zip for {self.memx_model_type.name}."
|
||||
)
|
||||
self.memx_post_model = post_candidates[0]
|
||||
elif self.memx_model_type == ModelTypeEnum.yolox:
|
||||
elif self.memx_model_type in [
|
||||
ModelTypeEnum.yolox,
|
||||
ModelTypeEnum.yologeneric,
|
||||
]:
|
||||
# Explicitly ignore any post model even if present
|
||||
self.memx_post_model = None
|
||||
else:
|
||||
@ -273,8 +267,6 @@ class MemryXDetector(DetectionApi):
|
||||
logger.info("Using cached models.")
|
||||
self.memx_model_path = dfp_path
|
||||
self.memx_post_model = post_path
|
||||
if self.memx_model_type == ModelTypeEnum.yologeneric:
|
||||
self.load_yolo_constants()
|
||||
return
|
||||
|
||||
# ---------- CASE 3: download MemryX model (no cache) ----------
|
||||
@ -303,9 +295,6 @@ class MemryXDetector(DetectionApi):
|
||||
else None
|
||||
)
|
||||
|
||||
if self.memx_model_type == ModelTypeEnum.yologeneric:
|
||||
self.load_yolo_constants()
|
||||
|
||||
finally:
|
||||
if os.path.exists(zip_path):
|
||||
try:
|
||||
@ -600,127 +589,232 @@ class MemryXDetector(DetectionApi):
|
||||
|
||||
self.output_queue.put(final_detections)
|
||||
|
||||
def onnx_reshape_with_allowzero(
|
||||
self, data: np.ndarray, shape: np.ndarray, allowzero: int = 0
|
||||
def _generate_anchors(self, sizes=[80, 40, 20]):
|
||||
"""Generate anchor points for YOLOv9 style processing"""
|
||||
yscales = []
|
||||
xscales = []
|
||||
for s in sizes:
|
||||
r = np.arange(s) + 0.5
|
||||
yscales.append(np.repeat(r, s))
|
||||
xscales.append(np.repeat(r[None, ...], s, axis=0).flatten())
|
||||
|
||||
yscales = np.concatenate(yscales)
|
||||
xscales = np.concatenate(xscales)
|
||||
anchors = np.stack([xscales, yscales], axis=1)
|
||||
return anchors
|
||||
|
||||
def _generate_scales(self, sizes=[80, 40, 20]):
|
||||
"""Generate scaling factors for each detection level"""
|
||||
factors = [8, 16, 32]
|
||||
s = np.concatenate([np.ones([int(s * s)]) * f for s, f in zip(sizes, factors)])
|
||||
return s[:, None]
|
||||
|
||||
@staticmethod
|
||||
def _softmax(x: np.ndarray, axis: int) -> np.ndarray:
|
||||
"""Efficient softmax implementation"""
|
||||
x = x - np.max(x, axis=axis, keepdims=True)
|
||||
np.exp(x, out=x)
|
||||
x /= np.sum(x, axis=axis, keepdims=True)
|
||||
return x
|
||||
|
||||
def dfl(self, x: np.ndarray) -> np.ndarray:
|
||||
"""Distribution Focal Loss decoding - YOLOv9 style"""
|
||||
x = x.reshape(-1, 4, 16)
|
||||
weights = np.arange(16, dtype=np.float32)
|
||||
p = self._softmax(x, axis=2)
|
||||
p = p * weights[None, None, :]
|
||||
out = np.sum(p, axis=2, keepdims=False)
|
||||
return out
|
||||
|
||||
def dist2bbox(
|
||||
self, x: np.ndarray, anchors: np.ndarray, scales: np.ndarray
|
||||
) -> np.ndarray:
|
||||
shape = shape.astype(int)
|
||||
input_shape = data.shape
|
||||
output_shape = []
|
||||
"""Convert distances to bounding boxes - YOLOv9 style"""
|
||||
lt = x[:, :2]
|
||||
rb = x[:, 2:]
|
||||
|
||||
for i, dim in enumerate(shape):
|
||||
if dim == 0 and allowzero == 0:
|
||||
output_shape.append(input_shape[i]) # Copy dimension from input
|
||||
else:
|
||||
output_shape.append(dim)
|
||||
x1y1 = anchors - lt
|
||||
x2y2 = anchors + rb
|
||||
|
||||
# Now let NumPy infer any -1 if needed
|
||||
reshaped = np.reshape(data, output_shape)
|
||||
wh = x2y2 - x1y1
|
||||
c_xy = (x1y1 + x2y2) / 2
|
||||
|
||||
return reshaped
|
||||
out = np.concatenate([c_xy, wh], axis=1)
|
||||
out = out * scales
|
||||
return out
|
||||
|
||||
def post_process_yolo_optimized(self, outputs):
|
||||
"""
|
||||
Custom YOLOv9 post-processing optimized for MemryX ONNX outputs.
|
||||
Implements DFL decoding, confidence filtering, and NMS in pure NumPy.
|
||||
"""
|
||||
# YOLOv9 outputs: 6 outputs (lbox, lcls, mbox, mcls, sbox, scls)
|
||||
conv_out1, conv_out2, conv_out3, conv_out4, conv_out5, conv_out6 = outputs
|
||||
|
||||
# Determine grid sizes based on input resolution
|
||||
# YOLOv9 uses 3 detection heads with strides [8, 16, 32]
|
||||
# Grid sizes = input_size / stride
|
||||
sizes = [
|
||||
self.memx_model_height
|
||||
// 8, # Large objects (e.g., 80 for 640x640, 40 for 320x320)
|
||||
self.memx_model_height
|
||||
// 16, # Medium objects (e.g., 40 for 640x640, 20 for 320x320)
|
||||
self.memx_model_height
|
||||
// 32, # Small objects (e.g., 20 for 640x640, 10 for 320x320)
|
||||
]
|
||||
|
||||
# Generate anchors and scales if not already done
|
||||
if not hasattr(self, "anchors"):
|
||||
self.anchors = self._generate_anchors(sizes)
|
||||
self.scales = self._generate_scales(sizes)
|
||||
|
||||
# Process outputs in YOLOv9 format: reshape and moveaxis for ONNX format
|
||||
lbox = np.moveaxis(conv_out1, 1, -1) # Large boxes
|
||||
lcls = np.moveaxis(conv_out2, 1, -1) # Large classes
|
||||
mbox = np.moveaxis(conv_out3, 1, -1) # Medium boxes
|
||||
mcls = np.moveaxis(conv_out4, 1, -1) # Medium classes
|
||||
sbox = np.moveaxis(conv_out5, 1, -1) # Small boxes
|
||||
scls = np.moveaxis(conv_out6, 1, -1) # Small classes
|
||||
|
||||
# Determine number of classes dynamically from the class output shape
|
||||
# lcls shape should be (batch, height, width, num_classes)
|
||||
num_classes = lcls.shape[-1]
|
||||
|
||||
# Validate that all class outputs have the same number of classes
|
||||
if not (mcls.shape[-1] == num_classes and scls.shape[-1] == num_classes):
|
||||
raise ValueError(
|
||||
f"Class output shapes mismatch: lcls={lcls.shape}, mcls={mcls.shape}, scls={scls.shape}"
|
||||
)
|
||||
|
||||
# Concatenate boxes and classes
|
||||
boxes = np.concatenate(
|
||||
[
|
||||
lbox.reshape(-1, 64), # 64 is for 4 bbox coords * 16 DFL bins
|
||||
mbox.reshape(-1, 64),
|
||||
sbox.reshape(-1, 64),
|
||||
],
|
||||
axis=0,
|
||||
)
|
||||
|
||||
classes = np.concatenate(
|
||||
[
|
||||
lcls.reshape(-1, num_classes),
|
||||
mcls.reshape(-1, num_classes),
|
||||
scls.reshape(-1, num_classes),
|
||||
],
|
||||
axis=0,
|
||||
)
|
||||
|
||||
# Apply sigmoid to classes
|
||||
classes = self.sigmoid(classes)
|
||||
|
||||
# Apply DFL to box predictions
|
||||
boxes = self.dfl(boxes)
|
||||
|
||||
# YOLOv9 postprocessing with confidence filtering and NMS
|
||||
confidence_thres = 0.4
|
||||
iou_thres = 0.6
|
||||
|
||||
# Find the class with the highest score for each detection
|
||||
max_scores = np.max(classes, axis=1) # Maximum class score for each detection
|
||||
class_ids = np.argmax(classes, axis=1) # Index of the best class
|
||||
|
||||
# Filter out detections with scores below the confidence threshold
|
||||
valid_indices = np.where(max_scores >= confidence_thres)[0]
|
||||
if len(valid_indices) == 0:
|
||||
# Return empty detections array
|
||||
final_detections = np.zeros((20, 6), np.float32)
|
||||
return final_detections
|
||||
|
||||
# Select only valid detections
|
||||
valid_boxes = boxes[valid_indices]
|
||||
valid_class_ids = class_ids[valid_indices]
|
||||
valid_scores = max_scores[valid_indices]
|
||||
|
||||
# Convert distances to actual bounding boxes using anchors and scales
|
||||
valid_boxes = self.dist2bbox(
|
||||
valid_boxes, self.anchors[valid_indices], self.scales[valid_indices]
|
||||
)
|
||||
|
||||
# Convert bounding box coordinates from (x_center, y_center, w, h) to (x_min, y_min, x_max, y_max)
|
||||
x_center, y_center, width, height = (
|
||||
valid_boxes[:, 0],
|
||||
valid_boxes[:, 1],
|
||||
valid_boxes[:, 2],
|
||||
valid_boxes[:, 3],
|
||||
)
|
||||
x_min = x_center - width / 2
|
||||
y_min = y_center - height / 2
|
||||
x_max = x_center + width / 2
|
||||
y_max = y_center + height / 2
|
||||
|
||||
# Convert to format expected by cv2.dnn.NMSBoxes: [x, y, width, height]
|
||||
boxes_for_nms = []
|
||||
scores_for_nms = []
|
||||
|
||||
for i in range(len(valid_indices)):
|
||||
# Ensure coordinates are within bounds and positive
|
||||
x_min_clipped = max(0, x_min[i])
|
||||
y_min_clipped = max(0, y_min[i])
|
||||
x_max_clipped = min(self.memx_model_width, x_max[i])
|
||||
y_max_clipped = min(self.memx_model_height, y_max[i])
|
||||
|
||||
width_clipped = x_max_clipped - x_min_clipped
|
||||
height_clipped = y_max_clipped - y_min_clipped
|
||||
|
||||
if width_clipped > 0 and height_clipped > 0:
|
||||
boxes_for_nms.append(
|
||||
[x_min_clipped, y_min_clipped, width_clipped, height_clipped]
|
||||
)
|
||||
scores_for_nms.append(float(valid_scores[i]))
|
||||
|
||||
final_detections = np.zeros((20, 6), np.float32)
|
||||
|
||||
if len(boxes_for_nms) == 0:
|
||||
return final_detections
|
||||
|
||||
# Apply NMS using OpenCV
|
||||
indices = cv2.dnn.NMSBoxes(
|
||||
boxes_for_nms, scores_for_nms, confidence_thres, iou_thres
|
||||
)
|
||||
|
||||
if len(indices) > 0:
|
||||
# Flatten indices if they are returned as a list of arrays
|
||||
if isinstance(indices[0], list) or isinstance(indices[0], np.ndarray):
|
||||
indices = [i[0] for i in indices]
|
||||
|
||||
# Limit to top 20 detections
|
||||
indices = indices[:20]
|
||||
|
||||
# Convert to Frigate format: [class_id, confidence, y_min, x_min, y_max, x_max] (normalized)
|
||||
for i, idx in enumerate(indices):
|
||||
class_id = valid_class_ids[idx]
|
||||
confidence = valid_scores[idx]
|
||||
|
||||
# Get the box coordinates
|
||||
box = boxes_for_nms[idx]
|
||||
x_min_norm = box[0] / self.memx_model_width
|
||||
y_min_norm = box[1] / self.memx_model_height
|
||||
x_max_norm = (box[0] + box[2]) / self.memx_model_width
|
||||
y_max_norm = (box[1] + box[3]) / self.memx_model_height
|
||||
|
||||
final_detections[i] = [
|
||||
class_id,
|
||||
confidence,
|
||||
y_min_norm, # Frigate expects y_min first
|
||||
x_min_norm,
|
||||
y_max_norm,
|
||||
x_max_norm,
|
||||
]
|
||||
|
||||
return final_detections
|
||||
|
||||
def process_output(self, *outputs):
|
||||
"""Output callback function -- receives frames from the MX3 and triggers post-processing"""
|
||||
if self.memx_model_type == ModelTypeEnum.yologeneric:
|
||||
if not self.memx_post_model:
|
||||
conv_out1 = outputs[0]
|
||||
conv_out2 = outputs[1]
|
||||
conv_out3 = outputs[2]
|
||||
conv_out4 = outputs[3]
|
||||
conv_out5 = outputs[4]
|
||||
conv_out6 = outputs[5]
|
||||
# Use complete YOLOv9-style postprocessing (includes NMS)
|
||||
final_detections = self.post_process_yolo_optimized(outputs)
|
||||
|
||||
concat_1 = self.onnx_concat([conv_out1, conv_out2], axis=1)
|
||||
concat_2 = self.onnx_concat([conv_out3, conv_out4], axis=1)
|
||||
concat_3 = self.onnx_concat([conv_out5, conv_out6], axis=1)
|
||||
|
||||
shape = np.array([1, 144, -1], dtype=np.int64)
|
||||
|
||||
reshaped_1 = self.onnx_reshape_with_allowzero(
|
||||
concat_1, shape, allowzero=0
|
||||
)
|
||||
reshaped_2 = self.onnx_reshape_with_allowzero(
|
||||
concat_2, shape, allowzero=0
|
||||
)
|
||||
reshaped_3 = self.onnx_reshape_with_allowzero(
|
||||
concat_3, shape, allowzero=0
|
||||
)
|
||||
|
||||
concat_4 = self.onnx_concat([reshaped_1, reshaped_2, reshaped_3], 2)
|
||||
|
||||
axis = 1
|
||||
split_sizes = [64, 80]
|
||||
|
||||
# Calculate indices at which to split
|
||||
indices = np.cumsum(split_sizes)[
|
||||
:-1
|
||||
] # [64] — split before the second chunk
|
||||
|
||||
# Perform split along axis 1
|
||||
split_0, split_1 = np.split(concat_4, indices, axis=axis)
|
||||
|
||||
num_boxes = 2100 if self.memx_model_height == 320 else 8400
|
||||
shape1 = np.array([1, 4, 16, num_boxes])
|
||||
reshape_4 = self.onnx_reshape_with_allowzero(
|
||||
split_0, shape1, allowzero=0
|
||||
)
|
||||
|
||||
transpose_1 = reshape_4.transpose(0, 2, 1, 3)
|
||||
|
||||
axis = 1 # As per ONNX softmax node
|
||||
|
||||
# Subtract max for numerical stability
|
||||
x_max = np.max(transpose_1, axis=axis, keepdims=True)
|
||||
x_exp = np.exp(transpose_1 - x_max)
|
||||
x_sum = np.sum(x_exp, axis=axis, keepdims=True)
|
||||
softmax_output = x_exp / x_sum
|
||||
|
||||
# Weight W from the ONNX initializer (1, 16, 1, 1) with values 0 to 15
|
||||
W = np.arange(16, dtype=np.float32).reshape(
|
||||
1, 16, 1, 1
|
||||
) # (1, 16, 1, 1)
|
||||
|
||||
# Apply 1x1 convolution: this is a weighted sum over channels
|
||||
conv_output = np.sum(
|
||||
softmax_output * W, axis=1, keepdims=True
|
||||
) # shape: (1, 1, 4, 8400)
|
||||
|
||||
shape2 = np.array([1, 4, num_boxes])
|
||||
reshape_5 = self.onnx_reshape_with_allowzero(
|
||||
conv_output, shape2, allowzero=0
|
||||
)
|
||||
|
||||
# ONNX Slice — get first 2 channels: [0:2] along axis 1
|
||||
slice_output1 = reshape_5[:, 0:2, :] # Result: (1, 2, 8400)
|
||||
|
||||
# Slice channels 2 to 4 → axis = 1
|
||||
slice_output2 = reshape_5[:, 2:4, :]
|
||||
|
||||
# Perform Subtraction
|
||||
sub_output = self.const_A - slice_output1 # Equivalent to ONNX Sub
|
||||
|
||||
# Perform the ONNX-style Add
|
||||
add_output = self.const_B + slice_output2
|
||||
|
||||
sub1 = add_output - sub_output
|
||||
|
||||
add1 = sub_output + add_output
|
||||
|
||||
div_output = add1 / 2.0
|
||||
|
||||
concat_5 = self.onnx_concat([div_output, sub1], axis=1)
|
||||
|
||||
# Expand B to (1, 1, 8400) so it can broadcast across axis=1 (4 channels)
|
||||
const_C_expanded = self.const_C[:, np.newaxis, :] # Shape: (1, 1, 8400)
|
||||
|
||||
# Perform ONNX-style element-wise multiplication
|
||||
mul_output = concat_5 * const_C_expanded # Result: (1, 4, 8400)
|
||||
|
||||
sigmoid_output = self.sigmoid(split_1)
|
||||
outputs = self.onnx_concat([mul_output, sigmoid_output], axis=1)
|
||||
|
||||
final_detections = post_process_yolo(
|
||||
outputs, self.memx_model_width, self.memx_model_height
|
||||
)
|
||||
self.output_queue.put(final_detections)
|
||||
|
||||
elif self.memx_model_type == ModelTypeEnum.yolonas:
|
||||
|
||||
@ -362,7 +362,7 @@ def stats_snapshot(
|
||||
stats["embeddings"]["review_description_speed"] = round(
|
||||
embeddings_metrics.review_desc_speed.value * 1000, 2
|
||||
)
|
||||
stats["embeddings"]["review_descriptions"] = round(
|
||||
stats["embeddings"]["review_description_events_per_second"] = round(
|
||||
embeddings_metrics.review_desc_dps.value, 2
|
||||
)
|
||||
|
||||
@ -370,7 +370,7 @@ def stats_snapshot(
|
||||
stats["embeddings"]["object_description_speed"] = round(
|
||||
embeddings_metrics.object_desc_speed.value * 1000, 2
|
||||
)
|
||||
stats["embeddings"]["object_descriptions"] = round(
|
||||
stats["embeddings"]["object_description_events_per_second"] = round(
|
||||
embeddings_metrics.object_desc_dps.value, 2
|
||||
)
|
||||
|
||||
@ -378,7 +378,7 @@ def stats_snapshot(
|
||||
stats["embeddings"][f"{key}_classification_speed"] = round(
|
||||
embeddings_metrics.classification_speeds[key].value * 1000, 2
|
||||
)
|
||||
stats["embeddings"][f"{key}_classification"] = round(
|
||||
stats["embeddings"][f"{key}_classification_events_per_second"] = round(
|
||||
embeddings_metrics.classification_cps[key].value, 2
|
||||
)
|
||||
|
||||
|
||||
@ -177,6 +177,10 @@
|
||||
"noCameras": {
|
||||
"title": "No Cameras Configured",
|
||||
"description": "Get started by connecting a camera to Frigate.",
|
||||
"buttonText": "Add Camera"
|
||||
"buttonText": "Add Camera",
|
||||
"restricted": {
|
||||
"title": "No Cameras Available",
|
||||
"description": "You don't have permission to view any cameras in this group."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -76,7 +76,12 @@
|
||||
}
|
||||
},
|
||||
"npuUsage": "NPU Usage",
|
||||
"npuMemory": "NPU Memory"
|
||||
"npuMemory": "NPU Memory",
|
||||
"intelGpuWarning": {
|
||||
"title": "Intel GPU Stats Warning",
|
||||
"message": "GPU stats unavailable",
|
||||
"description": "This is a known bug in Intel's GPU stats reporting tools (intel_gpu_top) where it will break and repeatedly return a GPU usage of 0% even in cases where hardware acceleration and object detection are correctly running on the (i)GPU. This is not a Frigate bug. You can restart the host to temporarily fix the issue and confirm that the GPU is working correctly. This does not affect performance."
|
||||
}
|
||||
},
|
||||
"otherProcesses": {
|
||||
"title": "Other Processes",
|
||||
@ -169,6 +174,7 @@
|
||||
"enrichments": {
|
||||
"title": "Enrichments",
|
||||
"infPerSecond": "Inferences Per Second",
|
||||
"averageInf": "Average Inference Time",
|
||||
"embeddings": {
|
||||
"image_embedding": "Image Embedding",
|
||||
"text_embedding": "Text Embedding",
|
||||
@ -180,7 +186,13 @@
|
||||
"plate_recognition_speed": "Plate Recognition Speed",
|
||||
"text_embedding_speed": "Text Embedding Speed",
|
||||
"yolov9_plate_detection_speed": "YOLOv9 Plate Detection Speed",
|
||||
"yolov9_plate_detection": "YOLOv9 Plate Detection"
|
||||
"yolov9_plate_detection": "YOLOv9 Plate Detection",
|
||||
"review_description": "Review Description",
|
||||
"review_description_speed": "Review Description Speed",
|
||||
"review_description_events_per_second": "Review Description",
|
||||
"object_description": "Object Description",
|
||||
"object_description_speed": "Object Description Speed",
|
||||
"object_description_events_per_second": "Object Description"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -9,7 +9,7 @@ import useSWR from "swr";
|
||||
import { MdHome } from "react-icons/md";
|
||||
import { usePersistedOverlayState } from "@/hooks/use-overlay-state";
|
||||
import { Button, buttonVariants } from "../ui/button";
|
||||
import { useCallback, useMemo, useState } from "react";
|
||||
import { useCallback, useEffect, useMemo, useState } from "react";
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip";
|
||||
import { LuPencil, LuPlus } from "react-icons/lu";
|
||||
import {
|
||||
@ -87,6 +87,8 @@ type CameraGroupSelectorProps = {
|
||||
export function CameraGroupSelector({ className }: CameraGroupSelectorProps) {
|
||||
const { t } = useTranslation(["components/camera"]);
|
||||
const { data: config } = useSWR<FrigateConfig>("config");
|
||||
const allowedCameras = useAllowedCameras();
|
||||
const isCustomRole = useIsCustomRole();
|
||||
|
||||
// tooltip
|
||||
|
||||
@ -119,10 +121,22 @@ export function CameraGroupSelector({ className }: CameraGroupSelectorProps) {
|
||||
return [];
|
||||
}
|
||||
|
||||
return Object.entries(config.camera_groups).sort(
|
||||
(a, b) => a[1].order - b[1].order,
|
||||
);
|
||||
}, [config]);
|
||||
const allGroups = Object.entries(config.camera_groups);
|
||||
|
||||
// If custom role, filter out groups where user has no accessible cameras
|
||||
if (isCustomRole) {
|
||||
return allGroups
|
||||
.filter(([, groupConfig]) => {
|
||||
// Check if user has access to at least one camera in this group
|
||||
return groupConfig.cameras.some((cameraName) =>
|
||||
allowedCameras.includes(cameraName),
|
||||
);
|
||||
})
|
||||
.sort((a, b) => a[1].order - b[1].order);
|
||||
}
|
||||
|
||||
return allGroups.sort((a, b) => a[1].order - b[1].order);
|
||||
}, [config, allowedCameras, isCustomRole]);
|
||||
|
||||
// add group
|
||||
|
||||
@ -139,6 +153,7 @@ export function CameraGroupSelector({ className }: CameraGroupSelectorProps) {
|
||||
activeGroup={group}
|
||||
setGroup={setGroup}
|
||||
deleteGroup={deleteGroup}
|
||||
isCustomRole={isCustomRole}
|
||||
/>
|
||||
<Scroller className={`${isMobile ? "whitespace-nowrap" : ""}`}>
|
||||
<div
|
||||
@ -206,14 +221,16 @@ export function CameraGroupSelector({ className }: CameraGroupSelectorProps) {
|
||||
);
|
||||
})}
|
||||
|
||||
<Button
|
||||
className="bg-secondary text-muted-foreground"
|
||||
aria-label={t("group.add")}
|
||||
size="xs"
|
||||
onClick={() => setAddGroup(true)}
|
||||
>
|
||||
<LuPlus className="size-4 text-primary" />
|
||||
</Button>
|
||||
{!isCustomRole && (
|
||||
<Button
|
||||
className="bg-secondary text-muted-foreground"
|
||||
aria-label={t("group.add")}
|
||||
size="xs"
|
||||
onClick={() => setAddGroup(true)}
|
||||
>
|
||||
<LuPlus className="size-4 text-primary" />
|
||||
</Button>
|
||||
)}
|
||||
{isMobile && <ScrollBar orientation="horizontal" className="h-0" />}
|
||||
</div>
|
||||
</Scroller>
|
||||
@ -228,6 +245,7 @@ type NewGroupDialogProps = {
|
||||
activeGroup?: string;
|
||||
setGroup: (value: string | undefined, replace?: boolean | undefined) => void;
|
||||
deleteGroup: () => void;
|
||||
isCustomRole?: boolean;
|
||||
};
|
||||
function NewGroupDialog({
|
||||
open,
|
||||
@ -236,6 +254,7 @@ function NewGroupDialog({
|
||||
activeGroup,
|
||||
setGroup,
|
||||
deleteGroup,
|
||||
isCustomRole,
|
||||
}: NewGroupDialogProps) {
|
||||
const { t } = useTranslation(["components/camera"]);
|
||||
const { mutate: updateConfig } = useSWR<FrigateConfig>("config");
|
||||
@ -261,6 +280,12 @@ function NewGroupDialog({
|
||||
`${activeGroup}-draggable-layout`,
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) {
|
||||
setEditState("none");
|
||||
}
|
||||
}, [open]);
|
||||
|
||||
// callbacks
|
||||
|
||||
const onDeleteGroup = useCallback(
|
||||
@ -349,13 +374,7 @@ function NewGroupDialog({
|
||||
position="top-center"
|
||||
closeButton={true}
|
||||
/>
|
||||
<Overlay
|
||||
open={open}
|
||||
onOpenChange={(open) => {
|
||||
setEditState("none");
|
||||
setOpen(open);
|
||||
}}
|
||||
>
|
||||
<Overlay open={open} onOpenChange={setOpen}>
|
||||
<Content
|
||||
className={cn(
|
||||
"scrollbar-container overflow-y-auto",
|
||||
@ -371,28 +390,30 @@ function NewGroupDialog({
|
||||
>
|
||||
<Title>{t("group.label")}</Title>
|
||||
<Description className="sr-only">{t("group.edit")}</Description>
|
||||
<div
|
||||
className={cn(
|
||||
"absolute",
|
||||
isDesktop && "right-6 top-10",
|
||||
isMobile && "absolute right-0 top-4",
|
||||
)}
|
||||
>
|
||||
<Button
|
||||
size="sm"
|
||||
{!isCustomRole && (
|
||||
<div
|
||||
className={cn(
|
||||
isDesktop &&
|
||||
"size-6 rounded-md bg-secondary-foreground p-1 text-background",
|
||||
isMobile && "text-secondary-foreground",
|
||||
"absolute",
|
||||
isDesktop && "right-6 top-10",
|
||||
isMobile && "absolute right-0 top-4",
|
||||
)}
|
||||
aria-label={t("group.add")}
|
||||
onClick={() => {
|
||||
setEditState("add");
|
||||
}}
|
||||
>
|
||||
<LuPlus />
|
||||
</Button>
|
||||
</div>
|
||||
<Button
|
||||
size="sm"
|
||||
className={cn(
|
||||
isDesktop &&
|
||||
"size-6 rounded-md bg-secondary-foreground p-1 text-background",
|
||||
isMobile && "text-secondary-foreground",
|
||||
)}
|
||||
aria-label={t("group.add")}
|
||||
onClick={() => {
|
||||
setEditState("add");
|
||||
}}
|
||||
>
|
||||
<LuPlus />
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
</Header>
|
||||
<div className="flex flex-col gap-4 md:gap-3">
|
||||
{currentGroups.map((group) => (
|
||||
@ -401,6 +422,7 @@ function NewGroupDialog({
|
||||
group={group}
|
||||
onDeleteGroup={() => onDeleteGroup(group[0])}
|
||||
onEditGroup={() => onEditGroup(group)}
|
||||
isReadOnly={isCustomRole}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
@ -512,12 +534,14 @@ type CameraGroupRowProps = {
|
||||
group: [string, CameraGroupConfig];
|
||||
onDeleteGroup: () => void;
|
||||
onEditGroup: () => void;
|
||||
isReadOnly?: boolean;
|
||||
};
|
||||
|
||||
export function CameraGroupRow({
|
||||
group,
|
||||
onDeleteGroup,
|
||||
onEditGroup,
|
||||
isReadOnly,
|
||||
}: CameraGroupRowProps) {
|
||||
const { t } = useTranslation(["components/camera"]);
|
||||
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false);
|
||||
@ -564,7 +588,7 @@ export function CameraGroupRow({
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
|
||||
{isMobile && (
|
||||
{isMobile && !isReadOnly && (
|
||||
<>
|
||||
<DropdownMenu modal={!isDesktop}>
|
||||
<DropdownMenuTrigger>
|
||||
@ -589,7 +613,7 @@ export function CameraGroupRow({
|
||||
</DropdownMenu>
|
||||
</>
|
||||
)}
|
||||
{!isMobile && (
|
||||
{!isMobile && !isReadOnly && (
|
||||
<div className="flex flex-row items-center gap-2">
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
|
||||
@ -807,6 +807,15 @@ function ObjectDetailsTab({
|
||||
}
|
||||
}, [search]);
|
||||
|
||||
const isEventsKey = useCallback((key: unknown): boolean => {
|
||||
const candidate = Array.isArray(key) ? key[0] : key;
|
||||
const EVENTS_KEY_PATTERNS = ["events", "events/search", "events/explore"];
|
||||
return (
|
||||
typeof candidate === "string" &&
|
||||
EVENTS_KEY_PATTERNS.some((p) => candidate.includes(p))
|
||||
);
|
||||
}, []);
|
||||
|
||||
const updateDescription = useCallback(() => {
|
||||
if (!search) {
|
||||
return;
|
||||
@ -821,11 +830,7 @@ function ObjectDetailsTab({
|
||||
});
|
||||
}
|
||||
mutate(
|
||||
(key) =>
|
||||
typeof key === "string" &&
|
||||
(key.includes("events") ||
|
||||
key.includes("events/search") ||
|
||||
key.includes("events/explore")),
|
||||
(key) => isEventsKey(key),
|
||||
(currentData: SearchResult[][] | SearchResult[] | undefined) =>
|
||||
mapSearchResults(currentData, (event) =>
|
||||
event.id === search.id
|
||||
@ -838,6 +843,7 @@ function ObjectDetailsTab({
|
||||
revalidate: false,
|
||||
},
|
||||
);
|
||||
setSearch({ ...search, data: { ...search.data, description: desc } });
|
||||
})
|
||||
.catch((error) => {
|
||||
const errorMessage =
|
||||
@ -854,7 +860,7 @@ function ObjectDetailsTab({
|
||||
);
|
||||
setDesc(search.data.description);
|
||||
});
|
||||
}, [desc, search, mutate, t, mapSearchResults]);
|
||||
}, [desc, search, mutate, t, mapSearchResults, isEventsKey, setSearch]);
|
||||
|
||||
const regenerateDescription = useCallback(
|
||||
(source: "snapshot" | "thumbnails") => {
|
||||
@ -921,11 +927,7 @@ function ObjectDetailsTab({
|
||||
});
|
||||
|
||||
mutate(
|
||||
(key) =>
|
||||
typeof key === "string" &&
|
||||
(key.includes("events") ||
|
||||
key.includes("events/search") ||
|
||||
key.includes("events/explore")),
|
||||
(key) => isEventsKey(key),
|
||||
(currentData: SearchResult[][] | SearchResult[] | undefined) =>
|
||||
mapSearchResults(currentData, (event) =>
|
||||
event.id === search.id
|
||||
@ -972,7 +974,7 @@ function ObjectDetailsTab({
|
||||
);
|
||||
});
|
||||
},
|
||||
[search, apiHost, mutate, setSearch, t, mapSearchResults],
|
||||
[search, apiHost, mutate, setSearch, t, mapSearchResults, isEventsKey],
|
||||
);
|
||||
|
||||
// recognized plate
|
||||
@ -996,11 +998,7 @@ function ObjectDetailsTab({
|
||||
});
|
||||
|
||||
mutate(
|
||||
(key) =>
|
||||
typeof key === "string" &&
|
||||
(key.includes("events") ||
|
||||
key.includes("events/search") ||
|
||||
key.includes("events/explore")),
|
||||
(key) => isEventsKey(key),
|
||||
(currentData: SearchResult[][] | SearchResult[] | undefined) =>
|
||||
mapSearchResults(currentData, (event) =>
|
||||
event.id === search.id
|
||||
@ -1047,7 +1045,7 @@ function ObjectDetailsTab({
|
||||
);
|
||||
});
|
||||
},
|
||||
[search, apiHost, mutate, setSearch, t, mapSearchResults],
|
||||
[search, apiHost, mutate, setSearch, t, mapSearchResults, isEventsKey],
|
||||
);
|
||||
|
||||
// speech transcription
|
||||
@ -1103,12 +1101,9 @@ function ObjectDetailsTab({
|
||||
});
|
||||
|
||||
setState("submitted");
|
||||
setSearch({ ...search, plus_id: "new_upload" });
|
||||
mutate(
|
||||
(key) =>
|
||||
typeof key === "string" &&
|
||||
(key.includes("events") ||
|
||||
key.includes("events/search") ||
|
||||
key.includes("events/explore")),
|
||||
(key) => isEventsKey(key),
|
||||
(currentData: SearchResult[][] | SearchResult[] | undefined) =>
|
||||
mapSearchResults(currentData, (event) =>
|
||||
event.id === search.id
|
||||
@ -1122,7 +1117,7 @@ function ObjectDetailsTab({
|
||||
},
|
||||
);
|
||||
},
|
||||
[search, mutate, mapSearchResults],
|
||||
[search, mutate, mapSearchResults, setSearch, isEventsKey],
|
||||
);
|
||||
|
||||
const popoverContainerRef = useRef<HTMLDivElement | null>(null);
|
||||
|
||||
@ -56,6 +56,7 @@ export function TrackingDetails({
|
||||
const apiHost = useApiHost();
|
||||
const imgRef = useRef<HTMLImageElement | null>(null);
|
||||
const [imgLoaded, setImgLoaded] = useState(false);
|
||||
const [isVideoLoading, setIsVideoLoading] = useState(true);
|
||||
const [displaySource, _setDisplaySource] = useState<"video" | "image">(
|
||||
"video",
|
||||
);
|
||||
@ -70,6 +71,10 @@ export function TrackingDetails({
|
||||
(event.start_time ?? 0) + annotationOffset / 1000 - REVIEW_PADDING,
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
setIsVideoLoading(true);
|
||||
}, [event.id]);
|
||||
|
||||
const { data: eventSequence } = useSWR<TrackingDetailsSequence[]>([
|
||||
"timeline",
|
||||
{
|
||||
@ -527,22 +532,28 @@ export function TrackingDetails({
|
||||
)}
|
||||
>
|
||||
{displaySource == "video" && (
|
||||
<HlsVideoPlayer
|
||||
videoRef={videoRef}
|
||||
containerRef={containerRef}
|
||||
visible={true}
|
||||
currentSource={videoSource}
|
||||
hotKeys={false}
|
||||
supportsFullscreen={false}
|
||||
fullscreen={false}
|
||||
frigateControls={true}
|
||||
onTimeUpdate={handleTimeUpdate}
|
||||
onSeekToTime={handleSeekToTime}
|
||||
onUploadFrame={onUploadFrameToPlus}
|
||||
isDetailMode={true}
|
||||
camera={event.camera}
|
||||
currentTimeOverride={currentTime}
|
||||
/>
|
||||
<>
|
||||
<HlsVideoPlayer
|
||||
videoRef={videoRef}
|
||||
containerRef={containerRef}
|
||||
visible={true}
|
||||
currentSource={videoSource}
|
||||
hotKeys={false}
|
||||
supportsFullscreen={false}
|
||||
fullscreen={false}
|
||||
frigateControls={true}
|
||||
onTimeUpdate={handleTimeUpdate}
|
||||
onSeekToTime={handleSeekToTime}
|
||||
onUploadFrame={onUploadFrameToPlus}
|
||||
onPlaying={() => setIsVideoLoading(false)}
|
||||
isDetailMode={true}
|
||||
camera={event.camera}
|
||||
currentTimeOverride={currentTime}
|
||||
/>
|
||||
{isVideoLoading && (
|
||||
<ActivityIndicator className="absolute left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2" />
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
{displaySource == "image" && (
|
||||
<>
|
||||
|
||||
@ -6,51 +6,199 @@ import {
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import { Event } from "@/types/event";
|
||||
import { isDesktop, isMobile } from "react-device-detect";
|
||||
import { ObjectSnapshotTab } from "../detail/SearchDetailDialog";
|
||||
import { isDesktop, isMobile, isSafari } from "react-device-detect";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import axios from "axios";
|
||||
import { useTranslation, Trans } from "react-i18next";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import ActivityIndicator from "@/components/indicators/activity-indicator";
|
||||
import { FaCheckCircle } from "react-icons/fa";
|
||||
import { Card, CardContent } from "@/components/ui/card";
|
||||
import { TransformComponent, TransformWrapper } from "react-zoom-pan-pinch";
|
||||
import ImageLoadingIndicator from "@/components/indicators/ImageLoadingIndicator";
|
||||
import { baseUrl } from "@/api/baseUrl";
|
||||
import { getTranslatedLabel } from "@/utils/i18n";
|
||||
import useImageLoaded from "@/hooks/use-image-loaded";
|
||||
|
||||
type FrigatePlusDialogProps = {
|
||||
export type FrigatePlusDialogProps = {
|
||||
upload?: Event;
|
||||
dialog?: boolean;
|
||||
onClose: () => void;
|
||||
onEventUploaded: () => void;
|
||||
};
|
||||
|
||||
export function FrigatePlusDialog({
|
||||
upload,
|
||||
dialog = true,
|
||||
onClose,
|
||||
onEventUploaded,
|
||||
}: FrigatePlusDialogProps) {
|
||||
if (!upload) {
|
||||
return;
|
||||
}
|
||||
if (dialog) {
|
||||
return (
|
||||
<Dialog
|
||||
open={upload != undefined}
|
||||
onOpenChange={(open) => (!open ? onClose() : null)}
|
||||
const { t, i18n } = useTranslation(["components/dialog"]);
|
||||
|
||||
type SubmissionState = "reviewing" | "uploading" | "submitted";
|
||||
const [state, setState] = useState<SubmissionState>(
|
||||
upload?.plus_id ? "submitted" : "reviewing",
|
||||
);
|
||||
useEffect(() => {
|
||||
setState(upload?.plus_id ? "submitted" : "reviewing");
|
||||
}, [upload?.plus_id]);
|
||||
|
||||
const onSubmitToPlus = useCallback(
|
||||
async (falsePositive: boolean) => {
|
||||
if (!upload) return;
|
||||
falsePositive
|
||||
? axios.put(`events/${upload.id}/false_positive`)
|
||||
: axios.post(`events/${upload.id}/plus`, { include_annotation: 1 });
|
||||
setState("submitted");
|
||||
onEventUploaded();
|
||||
},
|
||||
[upload, onEventUploaded],
|
||||
);
|
||||
|
||||
const [imgRef, imgLoaded, onImgLoad] = useImageLoaded();
|
||||
const showCard =
|
||||
!!upload &&
|
||||
upload.data.type === "object" &&
|
||||
upload.plus_id !== "not_enabled" &&
|
||||
upload.end_time &&
|
||||
upload.label !== "on_demand";
|
||||
|
||||
if (!dialog || !upload) return null;
|
||||
|
||||
return (
|
||||
<Dialog open={true} onOpenChange={(open) => (!open ? onClose() : null)}>
|
||||
<DialogContent
|
||||
className={cn(
|
||||
"scrollbar-container overflow-y-auto",
|
||||
isDesktop &&
|
||||
"max-h-[95dvh] sm:max-w-xl md:max-w-4xl lg:max-w-4xl xl:max-w-7xl",
|
||||
isMobile && "px-4",
|
||||
)}
|
||||
>
|
||||
<DialogContent
|
||||
className={cn(
|
||||
"scrollbar-container overflow-y-auto",
|
||||
isDesktop &&
|
||||
"max-h-[95dvh] sm:max-w-xl md:max-w-4xl lg:max-w-4xl xl:max-w-7xl",
|
||||
isMobile && "px-4",
|
||||
)}
|
||||
>
|
||||
<DialogHeader>
|
||||
<DialogTitle className="sr-only">Submit to Frigate+</DialogTitle>
|
||||
<DialogDescription className="sr-only">
|
||||
Submit this snapshot to Frigate+
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<ObjectSnapshotTab
|
||||
search={upload}
|
||||
onEventUploaded={onEventUploaded}
|
||||
<DialogHeader>
|
||||
<DialogTitle className="sr-only">Submit to Frigate+</DialogTitle>
|
||||
<DialogDescription className="sr-only">
|
||||
Submit this snapshot to Frigate+
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<div className="relative size-full">
|
||||
<ImageLoadingIndicator
|
||||
className="absolute inset-0 aspect-video min-h-[60dvh] w-full"
|
||||
imgLoaded={imgLoaded}
|
||||
/>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
<div className={imgLoaded ? "visible" : "invisible"}>
|
||||
<TransformWrapper minScale={1.0} wheel={{ smoothStep: 0.005 }}>
|
||||
<div className="flex flex-col space-y-3">
|
||||
<TransformComponent
|
||||
wrapperStyle={{ width: "100%", height: "100%" }}
|
||||
contentStyle={{
|
||||
position: "relative",
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
}}
|
||||
>
|
||||
{upload.id && (
|
||||
<div className="relative mx-auto">
|
||||
<img
|
||||
ref={imgRef}
|
||||
className="mx-auto max-h-[60dvh] rounded-lg bg-black object-contain"
|
||||
src={`${baseUrl}api/events/${upload.id}/snapshot.jpg`}
|
||||
alt={`${upload.label}`}
|
||||
loading={isSafari ? "eager" : "lazy"}
|
||||
onLoad={onImgLoad}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</TransformComponent>
|
||||
|
||||
{showCard && (
|
||||
<Card className="p-1 text-sm md:p-2">
|
||||
<CardContent className="flex flex-col items-center justify-between gap-3 p-2 md:flex-row">
|
||||
<div className="flex flex-col space-y-3">
|
||||
<div className="text-lg leading-none">
|
||||
{t("explore.plus.submitToPlus.label")}
|
||||
</div>
|
||||
<div className="text-sm text-muted-foreground">
|
||||
{t("explore.plus.submitToPlus.desc")}
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex w-full flex-1 flex-col justify-center gap-2 md:ml-8 md:w-auto md:justify-end">
|
||||
{state === "reviewing" && (
|
||||
<>
|
||||
<div>
|
||||
{i18n.language === "en" ? (
|
||||
/^[aeiou]/i.test(upload.label || "") ? (
|
||||
<Trans
|
||||
ns="components/dialog"
|
||||
values={{ label: upload.label }}
|
||||
>
|
||||
explore.plus.review.question.ask_an
|
||||
</Trans>
|
||||
) : (
|
||||
<Trans
|
||||
ns="components/dialog"
|
||||
values={{ label: upload.label }}
|
||||
>
|
||||
explore.plus.review.question.ask_a
|
||||
</Trans>
|
||||
)
|
||||
) : (
|
||||
<Trans
|
||||
ns="components/dialog"
|
||||
values={{
|
||||
untranslatedLabel: upload.label,
|
||||
translatedLabel: getTranslatedLabel(
|
||||
upload.label,
|
||||
),
|
||||
}}
|
||||
>
|
||||
explore.plus.review.question.ask_full
|
||||
</Trans>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex w-full flex-row gap-2">
|
||||
<Button
|
||||
className="flex-1 bg-success"
|
||||
aria-label={t("button.yes", { ns: "common" })}
|
||||
onClick={() => {
|
||||
setState("uploading");
|
||||
onSubmitToPlus(false);
|
||||
}}
|
||||
>
|
||||
{t("button.yes", { ns: "common" })}
|
||||
</Button>
|
||||
<Button
|
||||
className="flex-1 text-white"
|
||||
aria-label={t("button.no", { ns: "common" })}
|
||||
variant="destructive"
|
||||
onClick={() => {
|
||||
setState("uploading");
|
||||
onSubmitToPlus(true);
|
||||
}}
|
||||
>
|
||||
{t("button.no", { ns: "common" })}
|
||||
</Button>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
{state === "uploading" && <ActivityIndicator />}
|
||||
{state === "submitted" && (
|
||||
<div className="flex flex-row items-center justify-center gap-2">
|
||||
<FaCheckCircle className="size-4 text-success" />
|
||||
{t("explore.plus.review.state.submitted")}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)}
|
||||
</div>
|
||||
</TransformWrapper>
|
||||
</div>
|
||||
</div>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
|
||||
@ -6,7 +6,7 @@ import {
|
||||
useState,
|
||||
} from "react";
|
||||
import Hls from "hls.js";
|
||||
import { isAndroid, isDesktop, isMobile } from "react-device-detect";
|
||||
import { isDesktop, isMobile } from "react-device-detect";
|
||||
import { TransformComponent, TransformWrapper } from "react-zoom-pan-pinch";
|
||||
import VideoControls from "./VideoControls";
|
||||
import { VideoResolutionType } from "@/types/live";
|
||||
@ -22,7 +22,7 @@ import { useTranslation } from "react-i18next";
|
||||
import ObjectTrackOverlay from "@/components/overlay/ObjectTrackOverlay";
|
||||
|
||||
// Android native hls does not seek correctly
|
||||
const USE_NATIVE_HLS = !isAndroid;
|
||||
const USE_NATIVE_HLS = false;
|
||||
const HLS_MIME_TYPE = "application/vnd.apple.mpegurl" as const;
|
||||
const unsupportedErrorCodes = [
|
||||
MediaError.MEDIA_ERR_SRC_NOT_SUPPORTED,
|
||||
@ -130,6 +130,8 @@ export default function HlsVideoPlayer({
|
||||
return;
|
||||
}
|
||||
|
||||
setLoadedMetadata(false);
|
||||
|
||||
const currentPlaybackRate = videoRef.current.playbackRate;
|
||||
|
||||
if (!useHlsCompat) {
|
||||
|
||||
@ -309,6 +309,7 @@ function PreviewVideoPlayer({
|
||||
playsInline
|
||||
muted
|
||||
disableRemotePlayback
|
||||
disablePictureInPicture
|
||||
onSeeked={onPreviewSeeked}
|
||||
onLoadedData={() => {
|
||||
if (firstLoad) {
|
||||
|
||||
@ -2,7 +2,10 @@ import { Recording } from "@/types/record";
|
||||
import { DynamicPlayback } from "@/types/playback";
|
||||
import { PreviewController } from "../PreviewPlayer";
|
||||
import { TimeRange, TrackingDetailsSequence } from "@/types/timeline";
|
||||
import { calculateInpointOffset } from "@/utils/videoUtil";
|
||||
import {
|
||||
calculateInpointOffset,
|
||||
calculateSeekPosition,
|
||||
} from "@/utils/videoUtil";
|
||||
|
||||
type PlayerMode = "playback" | "scrubbing";
|
||||
|
||||
@ -72,38 +75,20 @@ export class DynamicVideoController {
|
||||
return;
|
||||
}
|
||||
|
||||
if (
|
||||
this.recordings.length == 0 ||
|
||||
time < this.recordings[0].start_time ||
|
||||
time > this.recordings[this.recordings.length - 1].end_time
|
||||
) {
|
||||
this.setNoRecording(true);
|
||||
return;
|
||||
}
|
||||
|
||||
if (this.playerMode != "playback") {
|
||||
this.playerMode = "playback";
|
||||
}
|
||||
|
||||
let seekSeconds = 0;
|
||||
(this.recordings || []).every((segment) => {
|
||||
// if the next segment is past the desired time, stop calculating
|
||||
if (segment.start_time > time) {
|
||||
return false;
|
||||
}
|
||||
const seekSeconds = calculateSeekPosition(
|
||||
time,
|
||||
this.recordings,
|
||||
this.inpointOffset,
|
||||
);
|
||||
|
||||
if (segment.end_time < time) {
|
||||
seekSeconds += segment.end_time - segment.start_time;
|
||||
return true;
|
||||
}
|
||||
|
||||
seekSeconds +=
|
||||
segment.end_time - segment.start_time - (segment.end_time - time);
|
||||
return true;
|
||||
});
|
||||
|
||||
// adjust for HLS inpoint offset
|
||||
seekSeconds -= this.inpointOffset;
|
||||
if (seekSeconds === undefined) {
|
||||
this.setNoRecording(true);
|
||||
return;
|
||||
}
|
||||
|
||||
if (seekSeconds != 0) {
|
||||
this.playerController.currentTime = seekSeconds;
|
||||
|
||||
@ -14,7 +14,10 @@ import { VideoResolutionType } from "@/types/live";
|
||||
import axios from "axios";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { calculateInpointOffset } from "@/utils/videoUtil";
|
||||
import {
|
||||
calculateInpointOffset,
|
||||
calculateSeekPosition,
|
||||
} from "@/utils/videoUtil";
|
||||
import { isFirefox } from "react-device-detect";
|
||||
|
||||
/**
|
||||
@ -109,10 +112,10 @@ export default function DynamicVideoPlayer({
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
const [isBuffering, setIsBuffering] = useState(false);
|
||||
const [loadingTimeout, setLoadingTimeout] = useState<NodeJS.Timeout>();
|
||||
const [source, setSource] = useState<HlsSource>({
|
||||
playlist: `${apiHost}vod/${camera}/start/${timeRange.after}/end/${timeRange.before}/master.m3u8`,
|
||||
startPosition: startTimestamp ? timeRange.after - startTimestamp : 0,
|
||||
});
|
||||
|
||||
// Don't set source until recordings load - we need accurate startPosition
|
||||
// to avoid hls.js clamping to video end when startPosition exceeds duration
|
||||
const [source, setSource] = useState<HlsSource | undefined>(undefined);
|
||||
|
||||
// start at correct time
|
||||
|
||||
@ -184,7 +187,7 @@ export default function DynamicVideoPlayer({
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!controller || !recordings?.length) {
|
||||
if (!recordings?.length) {
|
||||
if (recordings?.length == 0) {
|
||||
setNoRecording(true);
|
||||
}
|
||||
@ -192,10 +195,6 @@ export default function DynamicVideoPlayer({
|
||||
return;
|
||||
}
|
||||
|
||||
if (playerRef.current) {
|
||||
playerRef.current.autoplay = !isScrubbing;
|
||||
}
|
||||
|
||||
let startPosition = undefined;
|
||||
|
||||
if (startTimestamp) {
|
||||
@ -203,14 +202,12 @@ export default function DynamicVideoPlayer({
|
||||
recordingParams.after,
|
||||
(recordings || [])[0],
|
||||
);
|
||||
const idealStartPosition = Math.max(
|
||||
0,
|
||||
startTimestamp - timeRange.after - inpointOffset,
|
||||
);
|
||||
|
||||
if (idealStartPosition >= recordings[0].start_time - timeRange.after) {
|
||||
startPosition = idealStartPosition;
|
||||
}
|
||||
startPosition = calculateSeekPosition(
|
||||
startTimestamp,
|
||||
recordings,
|
||||
inpointOffset,
|
||||
);
|
||||
}
|
||||
|
||||
setSource({
|
||||
@ -218,6 +215,18 @@ export default function DynamicVideoPlayer({
|
||||
startPosition,
|
||||
});
|
||||
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [recordings]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!controller || !recordings?.length) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (playerRef.current) {
|
||||
playerRef.current.autoplay = !isScrubbing;
|
||||
}
|
||||
|
||||
setLoadingTimeout(setTimeout(() => setIsLoading(true), 1000));
|
||||
|
||||
controller.newPlayback({
|
||||
@ -225,7 +234,7 @@ export default function DynamicVideoPlayer({
|
||||
timeRange,
|
||||
});
|
||||
|
||||
// we only want this to change when recordings update
|
||||
// we only want this to change when controller or recordings update
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [controller, recordings]);
|
||||
|
||||
@ -263,46 +272,48 @@ export default function DynamicVideoPlayer({
|
||||
|
||||
return (
|
||||
<>
|
||||
<HlsVideoPlayer
|
||||
videoRef={playerRef}
|
||||
containerRef={containerRef}
|
||||
visible={!(isScrubbing || isLoading)}
|
||||
currentSource={source}
|
||||
hotKeys={hotKeys}
|
||||
supportsFullscreen={supportsFullscreen}
|
||||
fullscreen={fullscreen}
|
||||
inpointOffset={inpointOffset}
|
||||
onTimeUpdate={onTimeUpdate}
|
||||
onPlayerLoaded={onPlayerLoaded}
|
||||
onClipEnded={onValidateClipEnd}
|
||||
onSeekToTime={(timestamp, play) => {
|
||||
if (onSeekToTime) {
|
||||
onSeekToTime(timestamp, play);
|
||||
}
|
||||
}}
|
||||
onPlaying={() => {
|
||||
if (isScrubbing) {
|
||||
playerRef.current?.pause();
|
||||
}
|
||||
{source && (
|
||||
<HlsVideoPlayer
|
||||
videoRef={playerRef}
|
||||
containerRef={containerRef}
|
||||
visible={!(isScrubbing || isLoading)}
|
||||
currentSource={source}
|
||||
hotKeys={hotKeys}
|
||||
supportsFullscreen={supportsFullscreen}
|
||||
fullscreen={fullscreen}
|
||||
inpointOffset={inpointOffset}
|
||||
onTimeUpdate={onTimeUpdate}
|
||||
onPlayerLoaded={onPlayerLoaded}
|
||||
onClipEnded={onValidateClipEnd}
|
||||
onSeekToTime={(timestamp, play) => {
|
||||
if (onSeekToTime) {
|
||||
onSeekToTime(timestamp, play);
|
||||
}
|
||||
}}
|
||||
onPlaying={() => {
|
||||
if (isScrubbing) {
|
||||
playerRef.current?.pause();
|
||||
}
|
||||
|
||||
if (loadingTimeout) {
|
||||
clearTimeout(loadingTimeout);
|
||||
}
|
||||
if (loadingTimeout) {
|
||||
clearTimeout(loadingTimeout);
|
||||
}
|
||||
|
||||
setNoRecording(false);
|
||||
}}
|
||||
setFullResolution={setFullResolution}
|
||||
onUploadFrame={onUploadFrameToPlus}
|
||||
toggleFullscreen={toggleFullscreen}
|
||||
onError={(error) => {
|
||||
if (error == "stalled" && !isScrubbing) {
|
||||
setIsBuffering(true);
|
||||
}
|
||||
}}
|
||||
isDetailMode={isDetailMode}
|
||||
camera={contextCamera || camera}
|
||||
currentTimeOverride={currentTime}
|
||||
/>
|
||||
setNoRecording(false);
|
||||
}}
|
||||
setFullResolution={setFullResolution}
|
||||
onUploadFrame={onUploadFrameToPlus}
|
||||
toggleFullscreen={toggleFullscreen}
|
||||
onError={(error) => {
|
||||
if (error == "stalled" && !isScrubbing) {
|
||||
setIsBuffering(true);
|
||||
}
|
||||
}}
|
||||
isDetailMode={isDetailMode}
|
||||
camera={contextCamera || camera}
|
||||
currentTimeOverride={currentTime}
|
||||
/>
|
||||
)}
|
||||
<PreviewPlayer
|
||||
className={cn(
|
||||
className,
|
||||
|
||||
@ -377,7 +377,7 @@ export default function Step1NameCamera({
|
||||
);
|
||||
return selectedBrand &&
|
||||
selectedBrand.value != "other" ? (
|
||||
<Popover>
|
||||
<Popover modal={true}>
|
||||
<PopoverTrigger asChild>
|
||||
<Button
|
||||
variant="ghost"
|
||||
|
||||
@ -600,7 +600,7 @@ export default function Step3StreamConfig({
|
||||
<Label className="text-sm font-medium text-primary-variant">
|
||||
{t("cameraWizard.step3.roles")}
|
||||
</Label>
|
||||
<Popover>
|
||||
<Popover modal={true}>
|
||||
<PopoverTrigger asChild>
|
||||
<Button variant="ghost" size="sm" className="h-4 w-4 p-0">
|
||||
<LuInfo className="size-3" />
|
||||
@ -670,7 +670,7 @@ export default function Step3StreamConfig({
|
||||
<Label className="text-sm font-medium text-primary-variant">
|
||||
{t("cameraWizard.step3.featuresTitle")}
|
||||
</Label>
|
||||
<Popover>
|
||||
<Popover modal={true}>
|
||||
<PopoverTrigger asChild>
|
||||
<Button variant="ghost" size="sm" className="h-4 w-4 p-0">
|
||||
<LuInfo className="size-3" />
|
||||
|
||||
@ -93,19 +93,23 @@ function Live() {
|
||||
const allowedCameras = useAllowedCameras();
|
||||
|
||||
const includesBirdseye = useMemo(() => {
|
||||
// Restricted users should never have access to birdseye
|
||||
if (isCustomRole) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (
|
||||
config &&
|
||||
Object.keys(config.camera_groups).length &&
|
||||
cameraGroup &&
|
||||
config.camera_groups[cameraGroup] &&
|
||||
cameraGroup != "default" &&
|
||||
(!isCustomRole || "birdseye" in allowedCameras)
|
||||
cameraGroup != "default"
|
||||
) {
|
||||
return config.camera_groups[cameraGroup].cameras.includes("birdseye");
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}, [config, cameraGroup, allowedCameras, isCustomRole]);
|
||||
}, [config, cameraGroup, isCustomRole]);
|
||||
|
||||
const cameras = useMemo(() => {
|
||||
if (!config) {
|
||||
|
||||
@ -37,6 +37,7 @@ import EnrichmentsSettingsView from "@/views/settings/EnrichmentsSettingsView";
|
||||
import UiSettingsView from "@/views/settings/UiSettingsView";
|
||||
import FrigatePlusSettingsView from "@/views/settings/FrigatePlusSettingsView";
|
||||
import { useSearchEffect } from "@/hooks/use-overlay-state";
|
||||
import { usePersistence } from "@/hooks/use-persistence";
|
||||
import { useNavigate, useSearchParams } from "react-router-dom";
|
||||
import { useInitialCameraState } from "@/api/ws";
|
||||
import { useIsAdmin } from "@/hooks/use-is-admin";
|
||||
@ -207,7 +208,21 @@ export default function Settings() {
|
||||
.sort((aConf, bConf) => aConf.ui.order - bConf.ui.order);
|
||||
}, [config]);
|
||||
|
||||
const [selectedCamera, setSelectedCamera] = useState<string>("");
|
||||
const [persistedCamera, setPersistedCamera] = usePersistence(
|
||||
"selectedCamera",
|
||||
"",
|
||||
);
|
||||
const [selectedCamera, setSelectedCamera] = useState(persistedCamera);
|
||||
useEffect(() => {
|
||||
if (persistedCamera) {
|
||||
setSelectedCamera(persistedCamera);
|
||||
}
|
||||
}, [persistedCamera]);
|
||||
useEffect(() => {
|
||||
if (selectedCamera) {
|
||||
setPersistedCamera(selectedCamera);
|
||||
}
|
||||
}, [selectedCamera, setPersistedCamera]);
|
||||
|
||||
const { payload: allCameraStates } = useInitialCameraState(
|
||||
cameras.length > 0 ? cameras[0].name : "",
|
||||
|
||||
@ -24,3 +24,57 @@ export function calculateInpointOffset(
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates the video player time (in seconds) for a given timestamp
|
||||
* by iterating through recording segments and summing their durations.
|
||||
* This accounts for the fact that the video is a concatenation of segments,
|
||||
* not a single continuous stream.
|
||||
*
|
||||
* @param timestamp - The target timestamp to seek to
|
||||
* @param recordings - Array of recording segments
|
||||
* @param inpointOffset - HLS inpoint offset to subtract from the result
|
||||
* @returns The calculated seek position in seconds, or undefined if timestamp is out of range
|
||||
*/
|
||||
export function calculateSeekPosition(
|
||||
timestamp: number,
|
||||
recordings: Recording[],
|
||||
inpointOffset: number = 0,
|
||||
): number | undefined {
|
||||
if (!recordings || recordings.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
// Check if timestamp is within the recordings range
|
||||
if (
|
||||
timestamp < recordings[0].start_time ||
|
||||
timestamp > recordings[recordings.length - 1].end_time
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
let seekSeconds = 0;
|
||||
|
||||
(recordings || []).every((segment) => {
|
||||
// if the next segment is past the desired time, stop calculating
|
||||
if (segment.start_time > timestamp) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (segment.end_time < timestamp) {
|
||||
// Add the full duration of this segment
|
||||
seekSeconds += segment.end_time - segment.start_time;
|
||||
return true;
|
||||
}
|
||||
|
||||
// We're in this segment - calculate position within it
|
||||
seekSeconds +=
|
||||
segment.end_time - segment.start_time - (segment.end_time - timestamp);
|
||||
return true;
|
||||
});
|
||||
|
||||
// Adjust for HLS inpoint offset
|
||||
seekSeconds -= inpointOffset;
|
||||
|
||||
return seekSeconds >= 0 ? seekSeconds : undefined;
|
||||
}
|
||||
|
||||
@ -39,6 +39,7 @@ import {
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog";
|
||||
import BlurredIconButton from "@/components/button/BlurredIconButton";
|
||||
import { Skeleton } from "@/components/ui/skeleton";
|
||||
|
||||
const allModelTypes = ["objects", "states"] as const;
|
||||
type ModelType = (typeof allModelTypes)[number];
|
||||
@ -332,9 +333,7 @@ function ModelCard({ config, onClick, onUpdate, onDelete }: ModelCardProps) {
|
||||
<ImageShadowOverlay lowerClassName="h-[30%] z-0" />
|
||||
</>
|
||||
) : (
|
||||
<div className="flex size-full items-center justify-center bg-background_alt">
|
||||
<MdModelTraining className="size-16 text-muted-foreground" />
|
||||
</div>
|
||||
<Skeleton className="flex size-full items-center justify-center" />
|
||||
)}
|
||||
<div className="absolute bottom-2 left-3 text-lg text-white smart-capitalize">
|
||||
{config.name}
|
||||
|
||||
@ -20,7 +20,14 @@ import {
|
||||
FrigateConfig,
|
||||
} from "@/types/frigateConfig";
|
||||
import { ReviewSegment } from "@/types/review";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import {
|
||||
useCallback,
|
||||
useContext,
|
||||
useEffect,
|
||||
useMemo,
|
||||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
import {
|
||||
isDesktop,
|
||||
isMobile,
|
||||
@ -46,6 +53,8 @@ import { useStreamingSettings } from "@/context/streaming-settings-provider";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { EmptyCard } from "@/components/card/EmptyCard";
|
||||
import { BsFillCameraVideoOffFill } from "react-icons/bs";
|
||||
import { AuthContext } from "@/context/auth-context";
|
||||
import { useIsCustomRole } from "@/hooks/use-is-custom-role";
|
||||
|
||||
type LiveDashboardViewProps = {
|
||||
cameras: CameraConfig[];
|
||||
@ -374,10 +383,6 @@ export default function LiveDashboardView({
|
||||
onSaveMuting(true);
|
||||
};
|
||||
|
||||
if (cameras.length == 0 && !includeBirdseye) {
|
||||
return <NoCameraView />;
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
className="scrollbar-container size-full select-none overflow-y-auto px-1 pt-2 md:p-2"
|
||||
@ -439,198 +444,215 @@ export default function LiveDashboardView({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{!fullscreen && events && events.length > 0 && (
|
||||
<ScrollArea>
|
||||
<TooltipProvider>
|
||||
<div className="flex items-center gap-2 px-1">
|
||||
{events.map((event) => {
|
||||
return (
|
||||
<AnimatedEventCard
|
||||
key={event.id}
|
||||
event={event}
|
||||
selectedGroup={cameraGroup}
|
||||
updateEvents={updateEvents}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</TooltipProvider>
|
||||
<ScrollBar orientation="horizontal" />
|
||||
</ScrollArea>
|
||||
)}
|
||||
|
||||
{!cameraGroup || cameraGroup == "default" || isMobileOnly ? (
|
||||
{cameras.length == 0 && !includeBirdseye ? (
|
||||
<NoCameraView />
|
||||
) : (
|
||||
<>
|
||||
<div
|
||||
className={cn(
|
||||
"mt-2 grid grid-cols-1 gap-2 px-2 md:gap-4",
|
||||
mobileLayout == "grid" &&
|
||||
"grid-cols-2 xl:grid-cols-3 3xl:grid-cols-4",
|
||||
isMobile && "px-0",
|
||||
)}
|
||||
>
|
||||
{includeBirdseye && birdseyeConfig?.enabled && (
|
||||
{!fullscreen && events && events.length > 0 && (
|
||||
<ScrollArea>
|
||||
<TooltipProvider>
|
||||
<div className="flex items-center gap-2 px-1">
|
||||
{events.map((event) => {
|
||||
return (
|
||||
<AnimatedEventCard
|
||||
key={event.id}
|
||||
event={event}
|
||||
selectedGroup={cameraGroup}
|
||||
updateEvents={updateEvents}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</TooltipProvider>
|
||||
<ScrollBar orientation="horizontal" />
|
||||
</ScrollArea>
|
||||
)}
|
||||
|
||||
{!cameraGroup || cameraGroup == "default" || isMobileOnly ? (
|
||||
<>
|
||||
<div
|
||||
className={(() => {
|
||||
const aspectRatio =
|
||||
birdseyeConfig.width / birdseyeConfig.height;
|
||||
if (aspectRatio > 2) {
|
||||
return `${mobileLayout == "grid" && "col-span-2"} aspect-wide`;
|
||||
} else if (aspectRatio < 1) {
|
||||
return `${mobileLayout == "grid" && "row-span-2 h-full"} aspect-tall`;
|
||||
} else {
|
||||
return "aspect-video";
|
||||
}
|
||||
})()}
|
||||
ref={birdseyeContainerRef}
|
||||
className={cn(
|
||||
"mt-2 grid grid-cols-1 gap-2 px-2 md:gap-4",
|
||||
mobileLayout == "grid" &&
|
||||
"grid-cols-2 xl:grid-cols-3 3xl:grid-cols-4",
|
||||
isMobile && "px-0",
|
||||
)}
|
||||
>
|
||||
<BirdseyeLivePlayer
|
||||
birdseyeConfig={birdseyeConfig}
|
||||
liveMode={birdseyeConfig.restream ? "mse" : "jsmpeg"}
|
||||
onClick={() => onSelectCamera("birdseye")}
|
||||
containerRef={birdseyeContainerRef}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
{cameras.map((camera) => {
|
||||
let grow;
|
||||
const aspectRatio = camera.detect.width / camera.detect.height;
|
||||
if (aspectRatio > 2) {
|
||||
grow = `${mobileLayout == "grid" && "col-span-2"} aspect-wide`;
|
||||
} else if (aspectRatio < 1) {
|
||||
grow = `${mobileLayout == "grid" && "row-span-2 h-full"} aspect-tall`;
|
||||
} else {
|
||||
grow = "aspect-video";
|
||||
}
|
||||
const availableStreams = camera.live.streams || {};
|
||||
const firstStreamEntry = Object.values(availableStreams)[0] || "";
|
||||
|
||||
const streamNameFromSettings =
|
||||
currentGroupStreamingSettings?.[camera.name]?.streamName || "";
|
||||
const streamExists =
|
||||
streamNameFromSettings &&
|
||||
Object.values(availableStreams).includes(
|
||||
streamNameFromSettings,
|
||||
);
|
||||
|
||||
const streamName = streamExists
|
||||
? streamNameFromSettings
|
||||
: firstStreamEntry;
|
||||
const streamType =
|
||||
currentGroupStreamingSettings?.[camera.name]?.streamType;
|
||||
const autoLive =
|
||||
streamType !== undefined
|
||||
? streamType !== "no-streaming"
|
||||
: undefined;
|
||||
const showStillWithoutActivity =
|
||||
currentGroupStreamingSettings?.[camera.name]?.streamType !==
|
||||
"continuous";
|
||||
const useWebGL =
|
||||
currentGroupStreamingSettings?.[camera.name]
|
||||
?.compatibilityMode || false;
|
||||
return (
|
||||
<LiveContextMenu
|
||||
className={grow}
|
||||
key={camera.name}
|
||||
camera={camera.name}
|
||||
cameraGroup={cameraGroup}
|
||||
streamName={streamName}
|
||||
preferredLiveMode={preferredLiveModes[camera.name] ?? "mse"}
|
||||
isRestreamed={isRestreamedStates[camera.name]}
|
||||
supportsAudio={
|
||||
supportsAudioOutputStates[streamName]?.supportsAudio ??
|
||||
false
|
||||
}
|
||||
audioState={audioStates[camera.name]}
|
||||
toggleAudio={() => toggleAudio(camera.name)}
|
||||
statsState={statsStates[camera.name]}
|
||||
toggleStats={() => toggleStats(camera.name)}
|
||||
volumeState={volumeStates[camera.name] ?? 1}
|
||||
setVolumeState={(value) =>
|
||||
setVolumeStates({
|
||||
[camera.name]: value,
|
||||
})
|
||||
}
|
||||
muteAll={muteAll}
|
||||
unmuteAll={unmuteAll}
|
||||
resetPreferredLiveMode={() =>
|
||||
resetPreferredLiveMode(camera.name)
|
||||
}
|
||||
config={config}
|
||||
>
|
||||
<LivePlayer
|
||||
cameraRef={cameraRef}
|
||||
key={camera.name}
|
||||
className={`${grow} rounded-lg bg-black md:rounded-2xl`}
|
||||
windowVisible={
|
||||
windowVisible && visibleCameras.includes(camera.name)
|
||||
}
|
||||
cameraConfig={camera}
|
||||
preferredLiveMode={preferredLiveModes[camera.name] ?? "mse"}
|
||||
autoLive={autoLive ?? globalAutoLive}
|
||||
showStillWithoutActivity={showStillWithoutActivity ?? true}
|
||||
alwaysShowCameraName={displayCameraNames}
|
||||
useWebGL={useWebGL}
|
||||
playInBackground={false}
|
||||
showStats={statsStates[camera.name]}
|
||||
streamName={streamName}
|
||||
onClick={() => onSelectCamera(camera.name)}
|
||||
onError={(e) => handleError(camera.name, e)}
|
||||
onResetLiveMode={() => resetPreferredLiveMode(camera.name)}
|
||||
playAudio={audioStates[camera.name] ?? false}
|
||||
volume={volumeStates[camera.name]}
|
||||
/>
|
||||
</LiveContextMenu>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
{isDesktop && (
|
||||
<div
|
||||
className={cn(
|
||||
"fixed",
|
||||
isDesktop && "bottom-12 lg:bottom-9",
|
||||
isMobile && "bottom-12 lg:bottom-16",
|
||||
hasScrollbar && isDesktop ? "right-6" : "right-3",
|
||||
"z-50 flex flex-row gap-2",
|
||||
)}
|
||||
>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
{includeBirdseye && birdseyeConfig?.enabled && (
|
||||
<div
|
||||
className="cursor-pointer rounded-lg bg-secondary text-secondary-foreground opacity-60 transition-all duration-300 hover:bg-muted hover:opacity-100"
|
||||
onClick={toggleFullscreen}
|
||||
className={(() => {
|
||||
const aspectRatio =
|
||||
birdseyeConfig.width / birdseyeConfig.height;
|
||||
if (aspectRatio > 2) {
|
||||
return `${mobileLayout == "grid" && "col-span-2"} aspect-wide`;
|
||||
} else if (aspectRatio < 1) {
|
||||
return `${mobileLayout == "grid" && "row-span-2 h-full"} aspect-tall`;
|
||||
} else {
|
||||
return "aspect-video";
|
||||
}
|
||||
})()}
|
||||
ref={birdseyeContainerRef}
|
||||
>
|
||||
{fullscreen ? (
|
||||
<FaCompress className="size-5 md:m-[6px]" />
|
||||
) : (
|
||||
<FaExpand className="size-5 md:m-[6px]" />
|
||||
)}
|
||||
<BirdseyeLivePlayer
|
||||
birdseyeConfig={birdseyeConfig}
|
||||
liveMode={birdseyeConfig.restream ? "mse" : "jsmpeg"}
|
||||
onClick={() => onSelectCamera("birdseye")}
|
||||
containerRef={birdseyeContainerRef}
|
||||
/>
|
||||
</div>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
{fullscreen
|
||||
? t("button.exitFullscreen", { ns: "common" })
|
||||
: t("button.fullscreen", { ns: "common" })}
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</div>
|
||||
)}
|
||||
{cameras.map((camera) => {
|
||||
let grow;
|
||||
const aspectRatio =
|
||||
camera.detect.width / camera.detect.height;
|
||||
if (aspectRatio > 2) {
|
||||
grow = `${mobileLayout == "grid" && "col-span-2"} aspect-wide`;
|
||||
} else if (aspectRatio < 1) {
|
||||
grow = `${mobileLayout == "grid" && "row-span-2 h-full"} aspect-tall`;
|
||||
} else {
|
||||
grow = "aspect-video";
|
||||
}
|
||||
const availableStreams = camera.live.streams || {};
|
||||
const firstStreamEntry =
|
||||
Object.values(availableStreams)[0] || "";
|
||||
|
||||
const streamNameFromSettings =
|
||||
currentGroupStreamingSettings?.[camera.name]?.streamName ||
|
||||
"";
|
||||
const streamExists =
|
||||
streamNameFromSettings &&
|
||||
Object.values(availableStreams).includes(
|
||||
streamNameFromSettings,
|
||||
);
|
||||
|
||||
const streamName = streamExists
|
||||
? streamNameFromSettings
|
||||
: firstStreamEntry;
|
||||
const streamType =
|
||||
currentGroupStreamingSettings?.[camera.name]?.streamType;
|
||||
const autoLive =
|
||||
streamType !== undefined
|
||||
? streamType !== "no-streaming"
|
||||
: undefined;
|
||||
const showStillWithoutActivity =
|
||||
currentGroupStreamingSettings?.[camera.name]?.streamType !==
|
||||
"continuous";
|
||||
const useWebGL =
|
||||
currentGroupStreamingSettings?.[camera.name]
|
||||
?.compatibilityMode || false;
|
||||
return (
|
||||
<LiveContextMenu
|
||||
className={grow}
|
||||
key={camera.name}
|
||||
camera={camera.name}
|
||||
cameraGroup={cameraGroup}
|
||||
streamName={streamName}
|
||||
preferredLiveMode={
|
||||
preferredLiveModes[camera.name] ?? "mse"
|
||||
}
|
||||
isRestreamed={isRestreamedStates[camera.name]}
|
||||
supportsAudio={
|
||||
supportsAudioOutputStates[streamName]?.supportsAudio ??
|
||||
false
|
||||
}
|
||||
audioState={audioStates[camera.name]}
|
||||
toggleAudio={() => toggleAudio(camera.name)}
|
||||
statsState={statsStates[camera.name]}
|
||||
toggleStats={() => toggleStats(camera.name)}
|
||||
volumeState={volumeStates[camera.name] ?? 1}
|
||||
setVolumeState={(value) =>
|
||||
setVolumeStates({
|
||||
[camera.name]: value,
|
||||
})
|
||||
}
|
||||
muteAll={muteAll}
|
||||
unmuteAll={unmuteAll}
|
||||
resetPreferredLiveMode={() =>
|
||||
resetPreferredLiveMode(camera.name)
|
||||
}
|
||||
config={config}
|
||||
>
|
||||
<LivePlayer
|
||||
cameraRef={cameraRef}
|
||||
key={camera.name}
|
||||
className={`${grow} rounded-lg bg-black md:rounded-2xl`}
|
||||
windowVisible={
|
||||
windowVisible && visibleCameras.includes(camera.name)
|
||||
}
|
||||
cameraConfig={camera}
|
||||
preferredLiveMode={
|
||||
preferredLiveModes[camera.name] ?? "mse"
|
||||
}
|
||||
autoLive={autoLive ?? globalAutoLive}
|
||||
showStillWithoutActivity={
|
||||
showStillWithoutActivity ?? true
|
||||
}
|
||||
alwaysShowCameraName={displayCameraNames}
|
||||
useWebGL={useWebGL}
|
||||
playInBackground={false}
|
||||
showStats={statsStates[camera.name]}
|
||||
streamName={streamName}
|
||||
onClick={() => onSelectCamera(camera.name)}
|
||||
onError={(e) => handleError(camera.name, e)}
|
||||
onResetLiveMode={() =>
|
||||
resetPreferredLiveMode(camera.name)
|
||||
}
|
||||
playAudio={audioStates[camera.name] ?? false}
|
||||
volume={volumeStates[camera.name]}
|
||||
/>
|
||||
</LiveContextMenu>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
{isDesktop && (
|
||||
<div
|
||||
className={cn(
|
||||
"fixed",
|
||||
isDesktop && "bottom-12 lg:bottom-9",
|
||||
isMobile && "bottom-12 lg:bottom-16",
|
||||
hasScrollbar && isDesktop ? "right-6" : "right-3",
|
||||
"z-50 flex flex-row gap-2",
|
||||
)}
|
||||
>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<div
|
||||
className="cursor-pointer rounded-lg bg-secondary text-secondary-foreground opacity-60 transition-all duration-300 hover:bg-muted hover:opacity-100"
|
||||
onClick={toggleFullscreen}
|
||||
>
|
||||
{fullscreen ? (
|
||||
<FaCompress className="size-5 md:m-[6px]" />
|
||||
) : (
|
||||
<FaExpand className="size-5 md:m-[6px]" />
|
||||
)}
|
||||
</div>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
{fullscreen
|
||||
? t("button.exitFullscreen", { ns: "common" })
|
||||
: t("button.fullscreen", { ns: "common" })}
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
) : (
|
||||
<DraggableGridLayout
|
||||
cameras={cameras}
|
||||
cameraGroup={cameraGroup}
|
||||
containerRef={containerRef}
|
||||
cameraRef={cameraRef}
|
||||
includeBirdseye={includeBirdseye}
|
||||
onSelectCamera={onSelectCamera}
|
||||
windowVisible={windowVisible}
|
||||
visibleCameras={visibleCameras}
|
||||
isEditMode={isEditMode}
|
||||
setIsEditMode={setIsEditMode}
|
||||
fullscreen={fullscreen}
|
||||
toggleFullscreen={toggleFullscreen}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
) : (
|
||||
<DraggableGridLayout
|
||||
cameras={cameras}
|
||||
cameraGroup={cameraGroup}
|
||||
containerRef={containerRef}
|
||||
cameraRef={cameraRef}
|
||||
includeBirdseye={includeBirdseye}
|
||||
onSelectCamera={onSelectCamera}
|
||||
windowVisible={windowVisible}
|
||||
visibleCameras={visibleCameras}
|
||||
isEditMode={isEditMode}
|
||||
setIsEditMode={setIsEditMode}
|
||||
fullscreen={fullscreen}
|
||||
toggleFullscreen={toggleFullscreen}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
@ -638,15 +660,26 @@ export default function LiveDashboardView({
|
||||
|
||||
function NoCameraView() {
|
||||
const { t } = useTranslation(["views/live"]);
|
||||
const { auth } = useContext(AuthContext);
|
||||
const isCustomRole = useIsCustomRole();
|
||||
|
||||
// Check if this is a restricted user with no cameras in this group
|
||||
const isRestricted = isCustomRole && auth.isAuthenticated;
|
||||
|
||||
return (
|
||||
<div className="flex size-full items-center justify-center">
|
||||
<EmptyCard
|
||||
icon={<BsFillCameraVideoOffFill className="size-8" />}
|
||||
title={t("noCameras.title")}
|
||||
description={t("noCameras.description")}
|
||||
buttonText={t("noCameras.buttonText")}
|
||||
link="/settings?page=cameraManagement"
|
||||
title={
|
||||
isRestricted ? t("noCameras.restricted.title") : t("noCameras.title")
|
||||
}
|
||||
description={
|
||||
isRestricted
|
||||
? t("noCameras.restricted.description")
|
||||
: t("noCameras.description")
|
||||
}
|
||||
buttonText={!isRestricted ? t("noCameras.buttonText") : undefined}
|
||||
link={!isRestricted ? "/settings?page=cameraManagement" : undefined}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
|
||||
@ -198,9 +198,9 @@ export default function TriggerView({
|
||||
|
||||
return axios
|
||||
.put("config/set", configBody)
|
||||
.then((configResponse) => {
|
||||
.then(async (configResponse) => {
|
||||
if (configResponse.status === 200) {
|
||||
updateConfig();
|
||||
await updateConfig();
|
||||
const displayName =
|
||||
friendly_name && friendly_name !== ""
|
||||
? `${friendly_name} (${name})`
|
||||
@ -353,9 +353,9 @@ export default function TriggerView({
|
||||
|
||||
return axios
|
||||
.put("config/set", configBody)
|
||||
.then((configResponse) => {
|
||||
.then(async (configResponse) => {
|
||||
if (configResponse.status === 200) {
|
||||
updateConfig();
|
||||
await updateConfig();
|
||||
const friendly =
|
||||
config?.cameras?.[selectedCamera]?.semantic_search
|
||||
?.triggers?.[name]?.friendly_name;
|
||||
|
||||
@ -67,13 +67,14 @@ export default function EnrichmentMetrics({
|
||||
|
||||
// features stats
|
||||
|
||||
const embeddingInferenceTimeSeries = useMemo(() => {
|
||||
const groupedEnrichmentMetrics = useMemo(() => {
|
||||
if (!statsHistory) {
|
||||
return [];
|
||||
}
|
||||
|
||||
const series: {
|
||||
[key: string]: {
|
||||
rawKey: string;
|
||||
name: string;
|
||||
metrics: Threshold;
|
||||
data: { x: number; y: number }[];
|
||||
@ -90,6 +91,7 @@ export default function EnrichmentMetrics({
|
||||
|
||||
if (!(key in series)) {
|
||||
series[key] = {
|
||||
rawKey,
|
||||
name: t("enrichments.embeddings." + rawKey),
|
||||
metrics: getThreshold(rawKey),
|
||||
data: [],
|
||||
@ -99,7 +101,57 @@ export default function EnrichmentMetrics({
|
||||
series[key].data.push({ x: statsIdx + 1, y: stat });
|
||||
});
|
||||
});
|
||||
return Object.values(series);
|
||||
|
||||
// Group series by category (extract base name from raw key)
|
||||
const grouped: {
|
||||
[category: string]: {
|
||||
categoryName: string;
|
||||
speedSeries?: {
|
||||
name: string;
|
||||
metrics: Threshold;
|
||||
data: { x: number; y: number }[];
|
||||
};
|
||||
eventsSeries?: {
|
||||
name: string;
|
||||
metrics: Threshold;
|
||||
data: { x: number; y: number }[];
|
||||
};
|
||||
};
|
||||
} = {};
|
||||
|
||||
Object.values(series).forEach((s) => {
|
||||
// Extract base category name from raw key
|
||||
// All metrics follow the pattern: {base}_speed and {base}_events_per_second
|
||||
let categoryKey = s.rawKey;
|
||||
let isSpeed = false;
|
||||
|
||||
if (s.rawKey.endsWith("_speed")) {
|
||||
categoryKey = s.rawKey.replace("_speed", "");
|
||||
isSpeed = true;
|
||||
} else if (s.rawKey.endsWith("_events_per_second")) {
|
||||
categoryKey = s.rawKey.replace("_events_per_second", "");
|
||||
isSpeed = false;
|
||||
}
|
||||
|
||||
// Get translated category name
|
||||
const categoryName = t("enrichments.embeddings." + categoryKey);
|
||||
|
||||
if (!(categoryKey in grouped)) {
|
||||
grouped[categoryKey] = {
|
||||
categoryName,
|
||||
speedSeries: undefined,
|
||||
eventsSeries: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
if (isSpeed) {
|
||||
grouped[categoryKey].speedSeries = s;
|
||||
} else {
|
||||
grouped[categoryKey].eventsSeries = s;
|
||||
}
|
||||
});
|
||||
|
||||
return Object.values(grouped);
|
||||
}, [statsHistory, t, getThreshold]);
|
||||
|
||||
return (
|
||||
@ -110,35 +162,42 @@ export default function EnrichmentMetrics({
|
||||
</div>
|
||||
<div
|
||||
className={cn(
|
||||
"mt-4 grid w-full grid-cols-1 gap-2 sm:grid-cols-3",
|
||||
embeddingInferenceTimeSeries && "sm:grid-cols-4",
|
||||
"mt-4 grid w-full grid-cols-1 gap-2 sm:grid-cols-2 md:grid-cols-4",
|
||||
)}
|
||||
>
|
||||
{statsHistory.length != 0 ? (
|
||||
<>
|
||||
{embeddingInferenceTimeSeries.map((series) => (
|
||||
<div className="rounded-lg bg-background_alt p-2.5 md:rounded-2xl">
|
||||
<div className="mb-5 smart-capitalize">{series.name}</div>
|
||||
{series.name.endsWith("Speed") ? (
|
||||
<ThresholdBarGraph
|
||||
key={series.name}
|
||||
graphId={`${series.name}-inference`}
|
||||
name={series.name}
|
||||
unit="ms"
|
||||
threshold={series.metrics}
|
||||
updateTimes={updateTimes}
|
||||
data={[series]}
|
||||
/>
|
||||
) : (
|
||||
<EventsPerSecondsLineGraph
|
||||
key={series.name}
|
||||
graphId={`${series.name}-fps`}
|
||||
unit=""
|
||||
name={t("enrichments.infPerSecond")}
|
||||
updateTimes={updateTimes}
|
||||
data={[series]}
|
||||
/>
|
||||
)}
|
||||
{groupedEnrichmentMetrics.map((group) => (
|
||||
<div
|
||||
key={group.categoryName}
|
||||
className="rounded-lg bg-background_alt p-2.5 md:rounded-2xl"
|
||||
>
|
||||
<div className="mb-5 smart-capitalize">
|
||||
{group.categoryName}
|
||||
</div>
|
||||
<div className="space-y-4">
|
||||
{group.speedSeries && (
|
||||
<ThresholdBarGraph
|
||||
key={`${group.categoryName}-speed`}
|
||||
graphId={`${group.categoryName}-inference`}
|
||||
name={t("enrichments.averageInf")}
|
||||
unit="ms"
|
||||
threshold={group.speedSeries.metrics}
|
||||
updateTimes={updateTimes}
|
||||
data={[group.speedSeries]}
|
||||
/>
|
||||
)}
|
||||
{group.eventsSeries && (
|
||||
<EventsPerSecondsLineGraph
|
||||
key={`${group.categoryName}-events`}
|
||||
graphId={`${group.categoryName}-fps`}
|
||||
unit=""
|
||||
name={t("enrichments.infPerSecond")}
|
||||
updateTimes={updateTimes}
|
||||
data={[group.eventsSeries]}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</>
|
||||
|
||||
@ -375,6 +375,50 @@ export default function GeneralMetrics({
|
||||
return Object.keys(series).length > 0 ? Object.values(series) : undefined;
|
||||
}, [statsHistory]);
|
||||
|
||||
// Check if Intel GPU has all 0% usage values (known bug)
|
||||
const showIntelGpuWarning = useMemo(() => {
|
||||
if (!statsHistory || statsHistory.length < 3) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const gpuKeys = Object.keys(statsHistory[0]?.gpu_usages ?? {});
|
||||
const hasIntelGpu = gpuKeys.some(
|
||||
(key) => key === "intel-vaapi" || key === "intel-qsv",
|
||||
);
|
||||
|
||||
if (!hasIntelGpu) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Check if all GPU usage values are 0% across all stats
|
||||
let allZero = true;
|
||||
let hasDataPoints = false;
|
||||
|
||||
for (const stats of statsHistory) {
|
||||
if (!stats) {
|
||||
continue;
|
||||
}
|
||||
|
||||
Object.entries(stats.gpu_usages || {}).forEach(([key, gpuStats]) => {
|
||||
if (key === "intel-vaapi" || key === "intel-qsv") {
|
||||
if (gpuStats.gpu) {
|
||||
hasDataPoints = true;
|
||||
const gpuValue = parseFloat(gpuStats.gpu.slice(0, -1));
|
||||
if (!isNaN(gpuValue) && gpuValue > 0) {
|
||||
allZero = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (!allZero) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return hasDataPoints && allZero;
|
||||
}, [statsHistory]);
|
||||
|
||||
// npu stats
|
||||
|
||||
const npuSeries = useMemo(() => {
|
||||
@ -639,8 +683,46 @@ export default function GeneralMetrics({
|
||||
<>
|
||||
{statsHistory.length != 0 ? (
|
||||
<div className="rounded-lg bg-background_alt p-2.5 md:rounded-2xl">
|
||||
<div className="mb-5">
|
||||
<div className="mb-5 flex flex-row items-center justify-between">
|
||||
{t("general.hardwareInfo.gpuUsage")}
|
||||
{showIntelGpuWarning && (
|
||||
<Popover>
|
||||
<PopoverTrigger asChild>
|
||||
<button
|
||||
className="flex flex-row items-center gap-1.5 text-yellow-600 focus:outline-none dark:text-yellow-500"
|
||||
aria-label={t(
|
||||
"general.hardwareInfo.intelGpuWarning.title",
|
||||
)}
|
||||
>
|
||||
<CiCircleAlert
|
||||
className="size-5"
|
||||
aria-label={t(
|
||||
"general.hardwareInfo.intelGpuWarning.title",
|
||||
)}
|
||||
/>
|
||||
<span className="text-sm">
|
||||
{t(
|
||||
"general.hardwareInfo.intelGpuWarning.message",
|
||||
)}
|
||||
</span>
|
||||
</button>
|
||||
</PopoverTrigger>
|
||||
<PopoverContent className="w-80">
|
||||
<div className="space-y-2">
|
||||
<div className="font-semibold">
|
||||
{t(
|
||||
"general.hardwareInfo.intelGpuWarning.title",
|
||||
)}
|
||||
</div>
|
||||
<div>
|
||||
{t(
|
||||
"general.hardwareInfo.intelGpuWarning.description",
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</PopoverContent>
|
||||
</Popover>
|
||||
)}
|
||||
</div>
|
||||
{gpuSeries.map((series) => (
|
||||
<ThresholdBarGraph
|
||||
@ -729,33 +811,32 @@ export default function GeneralMetrics({
|
||||
) : (
|
||||
<Skeleton className="aspect-video w-full" />
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
{statsHistory[0]?.npu_usages && (
|
||||
<div
|
||||
className={cn("mt-4 grid grid-cols-1 gap-2 sm:grid-cols-2")}
|
||||
>
|
||||
{statsHistory.length != 0 ? (
|
||||
<div className="rounded-lg bg-background_alt p-2.5 md:rounded-2xl">
|
||||
<div className="mb-5">
|
||||
{t("general.hardwareInfo.npuUsage")}
|
||||
</div>
|
||||
{npuSeries.map((series) => (
|
||||
<ThresholdBarGraph
|
||||
key={series.name}
|
||||
graphId={`${series.name}-npu`}
|
||||
name={series.name}
|
||||
unit="%"
|
||||
threshold={GPUUsageThreshold}
|
||||
updateTimes={updateTimes}
|
||||
data={[series]}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
) : (
|
||||
<Skeleton className="aspect-video w-full" />
|
||||
|
||||
{statsHistory[0]?.npu_usages && (
|
||||
<>
|
||||
{statsHistory.length != 0 ? (
|
||||
<div className="rounded-lg bg-background_alt p-2.5 md:rounded-2xl">
|
||||
<div className="mb-5">
|
||||
{t("general.hardwareInfo.npuUsage")}
|
||||
</div>
|
||||
{npuSeries.map((series) => (
|
||||
<ThresholdBarGraph
|
||||
key={series.name}
|
||||
graphId={`${series.name}-npu`}
|
||||
name={series.name}
|
||||
unit="%"
|
||||
threshold={GPUUsageThreshold}
|
||||
updateTimes={updateTimes}
|
||||
data={[series]}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
) : (
|
||||
<Skeleton className="aspect-video w-full" />
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</>
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user