nik/frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2026-10-18 07:29:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improve PPU decoding and introduce strides handling

Browse Source
This commit is contained in:
aahmetdurmaz 2026-09-11 14:09:14 +03:00
parent 238073f943
commit 66c1c06d62
5 changed files with 194 additions and 528 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/data/object_detectors_models.yaml
View File

@ -991,7 +991,7 @@ deepx:
- key: yolo
label: YOLO
recommended: true
download: No model is bundled with Frigate. Download a pre-compiled YOLO `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo) or compile your own with DX-COM, then bind-mount it into the container and point the model's `path` at it. The recommended model is `yolox-s_640x640_ppu.dxnn`. Its Post-Processing Unit (PPU) compile moves candidate selection onto the NPU, which makes it the fastest ModelZoo model measured through Frigate (about 13 ms on a DX-M1). The output layout is read from the compiled model, so anchor-based, anchor-free, NMS-in-head and PPU models (anchor-based or anchor-free) all need no extra configuration; prefer a `PPU` variant whenever the ModelZoo offers one.
download: No model is bundled with Frigate. Download a pre-compiled YOLO `.dxnn` model from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo) or compile your own with DX-COM, then bind-mount it into the container and point the model's `path` at it. The recommended model is `yolox-s_640x640_ppu.dxnn`. Its Post-Processing Unit (PPU) compile moves candidate selection onto the NPU, which makes it the fastest ModelZoo model measured through Frigate (about 13 ms on a DX-M1). The output layout is read from the compiled model, so anchor-based, anchor-free, NMS-in-head and PPU models (anchor-based or anchor-free) all need no extra configuration; prefer a `PPU` variant whenever the ModelZoo offers one. PPU models must be compiled with DX-COM 2.4.0 or later, which writes the head layout Frigate reads into the file.
ui: |-
Navigate to **Settings > System > Detection models** and select **DEEPX NPU** from the **Hardware** dropdown. Then, on the same model, open the **Custom Model** tab and configure:

2
docs/docs/configuration/object_detectors.md
View File

@ -674,6 +674,8 @@ models:
height: 640
```
For PPU models, use a `.dxnn` compiled with DX-COM 2.4.0 or later. Frigate reads the PPU head layout the compiler writes into the file and refuses to load a PPU model without it.
`model_type` must be set to `yolo-generic`, `yolox` or `damo-yolo` to match the model; `yolo-generic` is the recommended default unless the model is a raw YOLOX or DAMO-YOLO export. Frigate defaults it to `ssd`, which this detector does not support, so a model that leaves it unset is rejected at startup.
`width` and `height` must match the resolution the model was compiled for. Quantization parameters are baked into the `.dxnn` file at compile time, so no normalization is applied on the host and Frigate's default `input_tensor`, `input_pixel_format`, and `input_dtype` values do not need to be overridden.

2
docs/docs/frigate/hardware.md
View File

@ -267,7 +267,7 @@ Frigate supports the DEEPX NPU in both of its form factors: the **DX-M1** M.2 mo
DEEPX NPU support in Frigate is developed and maintained by [Sixfab](https://sixfab.com).
The NPU runs models compiled to the `.dxnn` format with DEEPX's DX-COM compiler. Pre-compiled YOLO and DAMO-YOLO models can be downloaded from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo). Models compiled with Post-Processing Unit (PPU) support move candidate selection onto the NPU, which reduces host CPU usage; both anchor-based and anchor-free PPU heads are supported. Frigate reads the PPU head's kind and number of detection scales from the `.dxnn` file when the model loads, so no configuration is needed for either.
The NPU runs models compiled to the `.dxnn` format with DEEPX's DX-COM compiler. Pre-compiled YOLO and DAMO-YOLO models can be downloaded from the [DEEPX ModelZoo](https://developer.deepx.ai/modelzoo). Models compiled with Post-Processing Unit (PPU) support move candidate selection onto the NPU, which reduces host CPU usage; both anchor-based and anchor-free PPU heads are supported. Frigate reads the PPU head's kind and number of detection scales from the `.dxnn` file when the model loads, so no configuration is needed for either. PPU models must be compiled with DX-COM 2.4.0 or later; a `.dxnn` without that layout is refused at load.
Exception: **YOLOv7 and YOLOv7-x PPU models are not supported**. A PPU record has no anchor sizes, only a grid position, and the `.dxnn` does not carry them either, so Frigate decodes anchor-based PPU heads against a fixed anchor table per scale count: the usual COCO anchor set for three-scale heads (the one DEEPX's own reference decoder uses for every other three-scale model it ships) and the usual tiny set for two-scale heads. YOLOv7's anchors differ and nothing in the model tells them apart, so it decodes to wrong box sizes. Non-PPU YOLOv7 is unaffected (its raw head already outputs pixel-space boxes). An end-to-end PPU head that emits corner boxes rather than a centre and size is told apart from the records themselves. **Face and pose PPU models (SCRFD and the pose export) are not supported**: their records carry landmarks or keypoints in a different layout, and Frigate only decodes object detection records.

286
frigate/detectors/plugins/deepx.py
View File

@ -76,9 +76,6 @@ PPU_ANCHORS_BY_SCALES = {
),
}
PPU_MAX_STRIDE = 32
PPU_MAX_KNOWN_SCALES = max(PPU_ANCHORS_BY_SCALES)
# .dxnn: "DXNN", a 4-byte LE version, then a JSON index padded to
# DXNN_HEADER_SIZE; every offset in the index counts from there.
DXNN_MAGIC = b"DXNN"
@ -441,22 +438,6 @@ def ppu_records(outputs: list[np.ndarray]) -> np.ndarray | None:
return records
def ppu_boxes_are_anchor_based(boxes: np.ndarray) -> bool | None:
"""A value above 1 settles anchor-free (raw sigmoids stay inside 0..1),
a frame strictly inside 0..1 settles anchor-based, anything else is
None."""
if len(boxes) == 0:
return None
if np.any(boxes > 1.0):
return False
if np.all((boxes > 0.0) & (boxes <= 1.0)):
return True
return None
def ppu_boxes_are_centres(boxes: np.ndarray) -> bool:
"""Whether these pixel boxes hold a centre and size rather than two
corners: a corner box always runs x1<=x2, y1<=y2, so only a centre box
@ -464,14 +445,6 @@ def ppu_boxes_are_centres(boxes: np.ndarray) -> bool:
return bool(np.any(boxes[:, 2] < boxes[:, 0]) or np.any(boxes[:, 3] < boxes[:, 1]))
def ppu_needs_grid_decode(records: np.ndarray) -> bool:
"""Whether the layer field is real pyramid-level info (YOLOX-style,
still grid-relative) rather than a constant, unused value."""
layer_idx = reinterpret(records, PPU_GRID_BYTES, np.uint8)[:, 3]
return len(np.unique(layer_idx)) > 1
@dataclass(frozen=True)
class PpuLayout:
"""The PPU head's kind (None if unnamed) and each scale's grid, finest
@ -484,6 +457,10 @@ class PpuLayout:
def scale_count(self) -> int:
return len(self.grids)
def strides(self, width: int) -> tuple[int, ...]:
"""One stride per scale, finest first, from the input width."""
return tuple(round(width / grid_w) for grid_w, _ in self.grids)
def read_ppu_layout(path: str) -> PpuLayout | None:
"""Read the PPU head layout the compiler wrote into a .dxnn, or None
@ -600,58 +577,20 @@ def _ppu_outputs_are_anchor_based(outputs: list[dict], ppu: dict | None) -> bool
return None
def ppu_scale_count_lower_bound(records: np.ndarray, width: int, height: int) -> int:
"""The fewest detection scales these records can come from. Each proves
one past its layer index, and its grid cell can prove more: cell g of
layer L only exists if that layer's stride keeps stride * g < input,
and the stride is PPU_MAX_STRIDE >> (scales - 1 - L), so a cell too
far right or down for a coarse stride forces finer strides below it.
Only a lower bound: nothing in a record rules out scales above it."""
grid = reinterpret(records, PPU_GRID_BYTES, np.uint8)
grid_y = grid[:, 0].astype(np.int64)
grid_x = grid[:, 1].astype(np.int64)
layer_idx = grid[:, 3].astype(np.int64)
# strides from the coarsest down; each one a cell overflows is one more
# halving the layer needs, i.e. one more scale below it
strides = PPU_MAX_STRIDE >> np.arange(PPU_MAX_STRIDE.bit_length())
overflow_x = (strides[None, :] * grid_x[:, None] >= width).sum(axis=1)
overflow_y = (strides[None, :] * grid_y[:, None] >= height).sum(axis=1)
return int((layer_idx + 1 + np.maximum(overflow_x, overflow_y)).max())
def resolve_scale_count(
records: np.ndarray, scale_count: int | None, width: int, height: int
) -> int:
"""`scale_count` if given, else PPU_MAX_KNOWN_SCALES unless this frame
proves more; a frame showing only the finer layers near the origin is
what a bigger head looks like too, so it never argues for fewer."""
if scale_count is not None:
return scale_count
if len(records) == 0:
return PPU_MAX_KNOWN_SCALES
return max(
PPU_MAX_KNOWN_SCALES, ppu_scale_count_lower_bound(records, width, height)
)
def ppu_grid_regression_geometry(
records: np.ndarray, boxes: np.ndarray, scale_count: int
records: np.ndarray, boxes: np.ndarray, strides: np.ndarray
) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
"""Pixel centre and size for a YOLOX-style grid-relative box: centre
offset from the grid cell, size a log-scale multiple of the stride.
`scale_count` picks the stride each layer_idx was measured in."""
`strides` is one per scale, finest first."""
grid = reinterpret(records, PPU_GRID_BYTES, np.uint8)
grid_y = grid[:, 0].astype(np.float32)
grid_x = grid[:, 1].astype(np.float32)
layer_idx = grid[:, 3].astype(np.int32)
known = layer_idx < scale_count
known = layer_idx < len(strides)
layer = np.where(known, layer_idx, 0)
stride = (PPU_MAX_STRIDE >> (scale_count - 1 - layer)).astype(np.float32)
stride = strides[layer]
centre_x = (boxes[:, 0] + grid_x) * stride
centre_y = (boxes[:, 1] + grid_y) * stride
@ -662,27 +601,21 @@ def ppu_grid_regression_geometry(
def ppu_anchor_geometry(
records: np.ndarray, boxes: np.ndarray, scale_count: int
records: np.ndarray, boxes: np.ndarray, strides: np.ndarray
) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
"""Pixel centre and size for anchor-based PPU records, with the mask of
records the anchor table covers. `scale_count` picks the table and
stride; below the largest table it is under-evidenced, not disproven,
so it still gets that table as a best guess, but above it every record
is left unplaced instead of guessing at the wrong table."""
records the anchor table covers. `strides` is one per scale, finest
first; a scale count with no table leaves every record unplaced."""
grid = reinterpret(records, PPU_GRID_BYTES, np.uint8)
grid_y = grid[:, 0].astype(np.float32)
grid_x = grid[:, 1].astype(np.float32)
box_idx = grid[:, 2].astype(np.int32)
layer_idx = grid[:, 3].astype(np.int32)
anchor_table = PPU_ANCHORS_BY_SCALES.get(scale_count)
anchor_table = PPU_ANCHORS_BY_SCALES.get(len(strides))
if anchor_table is None:
if scale_count > PPU_MAX_KNOWN_SCALES:
# a real head proven to have more scales than any table covers
zeros = np.zeros(len(layer_idx), np.float32)
return zeros, zeros, zeros, zeros, np.zeros(len(layer_idx), dtype=bool)
anchor_table = PPU_ANCHORS_BY_SCALES[PPU_MAX_KNOWN_SCALES]
zeros = np.zeros(len(layer_idx), np.float32)
return zeros, zeros, zeros, zeros, np.zeros(len(layer_idx), dtype=bool)
levels, per_level = anchor_table.shape[:2]
known = (layer_idx < levels) & (box_idx < per_level)
@ -691,7 +624,7 @@ def ppu_anchor_geometry(
# then let the mask drop them rather than raising here
layer = np.where(known, layer_idx, 0)
anchors = anchor_table[layer, np.where(known, box_idx, 0)]
stride = (PPU_MAX_STRIDE >> (levels - 1 - layer)).astype(np.float32)
stride = strides[layer]
centre_x = (boxes[:, 0] * 2.0 - 0.5 + grid_x) * stride
centre_y = (boxes[:, 1] * 2.0 - 0.5 + grid_y) * stride
@ -707,17 +640,17 @@ def decode_ppu(
height: int,
score_threshold: float,
nms_threshold: float,
anchor_based: bool | None = None,
scale_count: int | None = None,
needs_grid_decode: bool | None = None,
*,
strides: tuple[int, ...] | np.ndarray,
anchor_based: bool,
records: np.ndarray | None = None,
centre_boxes: bool | None = None,
boxes: np.ndarray | None = None,
) -> np.ndarray:
"""Decode PPU records, leaving only NMS. `anchor_based`,
`needs_grid_decode` and `centre_boxes` skip their heuristics once a
caller settles them; `records` and `boxes` reuse an already-parsed
buffer instead of re-parsing `outputs`."""
"""Decode PPU records, leaving only NMS. `strides` is one per scale,
finest first, as the model file's grids give them; `centre_boxes`
skips its heuristic once a caller settles it; `records` and `boxes`
reuse an already-parsed buffer instead of re-parsing `outputs`."""
if records is None:
records = ppu_records(outputs)
@ -727,45 +660,34 @@ def decode_ppu(
if boxes is None:
boxes = reinterpret(records, PPU_BOX_BYTES, np.float32).reshape(-1, 4)
if anchor_based is None:
anchor_based = ppu_boxes_are_anchor_based(boxes)
if anchor_based is None:
# nothing conclusive in this frame; no pixel box to draw either
return np.zeros((20, 6), np.float32)
scores = reinterpret(records, PPU_SCORE_BYTES, np.float32).flatten()
labels = reinterpret(records, PPU_LABEL_BYTES, np.uint32).flatten()
strides = np.asarray(strides, np.float32)
if anchor_based:
scale_count = resolve_scale_count(records, scale_count, width, height)
centre_x, centre_y, box_w, box_h, known = ppu_anchor_geometry(
records, boxes, scale_count
records, boxes, strides
)
# a record the table cannot place is dropped by the score filter
scores = np.where(known, scores, 0.0)
elif len(strides) > 1:
# a multi-scale anchor-free head is YOLOX-style, grid-relative
centre_x, centre_y, box_w, box_h, known = ppu_grid_regression_geometry(
records, boxes, strides
)
scores = np.where(known, scores, 0.0)
else:
if needs_grid_decode is None:
needs_grid_decode = ppu_needs_grid_decode(records)
if centre_boxes is None:
centre_boxes = ppu_boxes_are_centres(boxes)
if needs_grid_decode:
scale_count = resolve_scale_count(records, scale_count, width, height)
centre_x, centre_y, box_w, box_h, known = ppu_grid_regression_geometry(
records, boxes, scale_count
)
scores = np.where(known, scores, 0.0)
if centre_boxes:
centre_x, centre_y = boxes[:, 0], boxes[:, 1]
box_w, box_h = boxes[:, 2], boxes[:, 3]
else:
if centre_boxes is None:
centre_boxes = ppu_boxes_are_centres(boxes)
if centre_boxes:
centre_x, centre_y = boxes[:, 0], boxes[:, 1]
box_w, box_h = boxes[:, 2], boxes[:, 3]
else:
box_w = boxes[:, 2] - boxes[:, 0]
box_h = boxes[:, 3] - boxes[:, 1]
centre_x = boxes[:, 0] + box_w * 0.5
centre_y = boxes[:, 1] + box_h * 0.5
box_w = boxes[:, 2] - boxes[:, 0]
box_h = boxes[:, 3] - boxes[:, 1]
centre_x = boxes[:, 0] + box_w * 0.5
centre_y = boxes[:, 1] + box_h * 0.5
x_min = centre_x - box_w * 0.5
y_min = centre_y - box_h * 0.5
@ -1025,71 +947,40 @@ class DeepxDetector(DetectionApi):
# the shared YOLOX decoder's cell grids and strides for this input
self.calculate_grids_strides()
self.logged_layout = False
# set once a frame proves the PPU head anchor-free, then kept
self.ppu_anchor_free = False
# set once proven needed, then kept
self.ppu_needs_grid_decode = False
self.ppu_scale_count = 0
# set once proven; a corner-format head never produces such a record
self.ppu_centre_boxes = False
self.ppu_unsupported_scale_reported = False
self.damoyolo_outputs_reported = False
# the model's own head layout; None falls back to the fields above
self.ppu_layout = None
self.ppu_layout: PpuLayout | None = None
if self.output is not None and self.output.layout is YoloLayout.ppu:
self.ppu_layout = self.inspect_ppu_head(model_path)
def inspect_ppu_head(self, model_path: str) -> PpuLayout | None:
"""Settle the PPU head from the model file, so the anchor table and
strides are right from the first frame."""
def inspect_ppu_head(self, model_path: str) -> PpuLayout:
"""Read the PPU head layout DX-COM writes into the model file; the
head kind, anchor table and strides all come from it."""
layout = read_ppu_layout(model_path)
if layout is None:
logger.warning(
"Could not read the PPU head layout from %s; assuming a "
"%d-scale head, so a head with fewer scales will decode to "
"misplaced boxes",
model_path,
PPU_MAX_KNOWN_SCALES,
)
return None
self.ppu_scale_count = layout.scale_count
if layout.anchor_based is not None:
self.ppu_anchor_free = not layout.anchor_based
# only a grid-relative head keeps one tensor per scale
self.ppu_needs_grid_decode = layout.anchor_based is False and (
layout.scale_count > 1
raise ValueError(
f"Could not read the PPU head layout from {model_path}; the "
"file is damaged or was compiled before DX-COM 2.4.0. Compile "
"PPU models with DX-COM 2.4.0 or later."
)
# a head laid out differently than the decoders assume would
# decode to wrong positions, so say so once here
if layout.anchor_based or layout.scale_count > 1:
strides = [
PPU_MAX_STRIDE >> (layout.scale_count - 1 - i)
for i in range(layout.scale_count)
]
expected = tuple(
(-(-self.width // s), -(-self.height // s)) for s in strides
if layout.anchor_based is None:
raise ValueError(
f"The PPU head in {model_path} names no kind Frigate can decode "
"(anchor-based or anchor-free YOLO); face and pose PPU models are "
"not supported, and PPU models must be compiled with DX-COM 2.4.0 "
"or later."
)
if layout.grids != expected:
logger.warning(
"PPU head grids %s do not match the %d-scale layout Frigate "
"decodes (%s for %dx%d input); boxes may be misplaced",
layout.grids,
layout.scale_count,
expected,
self.width,
self.height,
)
logger.info(
"DEEPX PPU head from the model: %s, %d scale(s)",
{True: "anchor-based", False: "anchor-free"}.get(
layout.anchor_based, "unknown kind"
),
"DEEPX PPU head from the model: %s, %d scale(s), grids %s",
"anchor-based" if layout.anchor_based else "anchor-free",
layout.scale_count,
layout.grids,
)
return layout
@ -1181,51 +1072,26 @@ class DeepxDetector(DetectionApi):
)
def decode_ppu(self, outputs: list[np.ndarray]) -> np.ndarray:
"""Decode PPU records. The head layout read from the model fixes
the kind, scale count and grid decode from the first frame; without
it, the kind and grid decode are settled from the records and kept
once proven, and the scale count is PPU_MAX_KNOWN_SCALES unless a
record proves more (a partial frame never argues for fewer)."""
"""Decode PPU records against the head layout read from the model."""
layout = self.ppu_layout
assert layout is not None
records = ppu_records(outputs)
boxes = None
if records is not None and len(records):
boxes = reinterpret(records, PPU_BOX_BYTES, np.float32).reshape(-1, 4)
if self.ppu_layout is not None and self.ppu_layout.anchor_based is not None:
anchor_based = self.ppu_layout.anchor_based
elif self.ppu_anchor_free:
anchor_based = False
else:
anchor_based = (
ppu_boxes_are_anchor_based(boxes) if boxes is not None else None
)
self.ppu_anchor_free = anchor_based is False
if boxes is not None:
if self.ppu_layout is None:
self.ppu_scale_count = max(
PPU_MAX_KNOWN_SCALES,
self.ppu_scale_count,
ppu_scale_count_lower_bound(records, self.width, self.height),
)
if not anchor_based and not self.ppu_needs_grid_decode:
if self.ppu_layout is None:
self.ppu_needs_grid_decode = ppu_needs_grid_decode(records)
else:
self.ppu_needs_grid_decode = self.ppu_layout.scale_count > 1
if (
anchor_based is False
and not self.ppu_needs_grid_decode
and not self.ppu_centre_boxes
):
self.ppu_centre_boxes = ppu_boxes_are_centres(boxes)
if (
boxes is not None
and not layout.anchor_based
and layout.scale_count == 1
and not self.ppu_centre_boxes
):
self.ppu_centre_boxes = ppu_boxes_are_centres(boxes)
if (
anchor_based
and self.ppu_scale_count
and self.ppu_scale_count not in PPU_ANCHORS_BY_SCALES
layout.anchor_based
and layout.scale_count not in PPU_ANCHORS_BY_SCALES
and not self.ppu_unsupported_scale_reported
):
self.ppu_unsupported_scale_reported = True
@ -1233,26 +1099,18 @@ class DeepxDetector(DetectionApi):
"This PPU model's anchor-based head reports %d detection "
"scales, which Frigate has no anchor table for (supported: "
"%s); its detections cannot be decoded",
self.ppu_scale_count,
layout.scale_count,
", ".join(str(n) for n in sorted(PPU_ANCHORS_BY_SCALES)),
)
if self.ppu_layout is not None:
needs_grid_decode = self.ppu_needs_grid_decode
else:
# None leaves it to the frame; only a proven True is kept
needs_grid_decode = True if self.ppu_needs_grid_decode else None
# anchor_based None here means nothing conclusive; no box to draw
return decode_ppu(
outputs,
self.width,
self.height,
SCORE_THRESHOLD,
NMS_THRESHOLD,
anchor_based=anchor_based,
scale_count=self.ppu_scale_count or None,
needs_grid_decode=needs_grid_decode,
strides=layout.strides(self.width),
anchor_based=bool(layout.anchor_based),
records=records,
# None leaves it to the frame; only a proven True is kept
centre_boxes=True if self.ppu_centre_boxes else None,

430
frigate/test/test_deepx.py
View File

@ -28,7 +28,6 @@ from frigate.detectors.plugins.deepx import (
decode_raw_anchor,
decode_raw_nms_in_head,
infer_yolo_layout,
ppu_scale_count_lower_bound,
read_ppu_layout,
resolve_device,
rows_with_columns,
@ -62,6 +61,11 @@ def build_ppu_records(*records: np.ndarray) -> np.ndarray:
return np.concatenate(records, axis=1)
def strides_for(scales: int) -> tuple[int, ...]:
"""The 32 >> k stride ladder of a `scales`-scale head, finest first."""
return tuple(32 >> (scales - 1 - i) for i in range(scales))
# compile_config.ppu as DX-COM writes it for the two head kinds; the layer
# names inside are irrelevant to the reader, only the type is
ANCHOR_BASED_PPU = {"type": 0, "num_classes": 80, "activation": "Sigmoid"}
@ -230,14 +234,16 @@ class TestDeepxPpuLayoutFile(unittest.TestCase):
class TestDeepxPpuDecode(unittest.TestCase):
def test_reads_anchor_free_box_bytes_as_pixel_geometry(self):
"""The grid columns carry no meaning for an anchor-free head and must
not influence the result."""
"""The grid columns carry no meaning for a single-scale anchor-free
head and must not influence the result."""
detections = decode_ppu(
[build_ppu_record((320.0, 160.0, 64.0, 32.0), label=3)],
640,
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
)
self.assertEqual(detections[0][0], 3)
@ -252,7 +258,13 @@ class TestDeepxPpuDecode(unittest.TestCase):
"""An anchor-based head leaves ratios in the box fields, which the
grid columns and the anchor table turn back into pixels."""
detections = decode_ppu(
[build_ppu_record((0.6, 0.4, 0.3, 0.7), label=5)], 640, 640, 0.25, 0.45
[build_ppu_record((0.6, 0.4, 0.3, 0.7), label=5)],
640,
640,
0.25,
0.45,
strides=strides_for(3),
anchor_based=True,
)
# layer 2 is stride 32 and box 2 is the 373x326 anchor, so the centre
@ -267,23 +279,17 @@ class TestDeepxPpuDecode(unittest.TestCase):
self,
):
"""YOLOX's classic anchor-free head leaves its box regression
relative to the grid cell it was predicted at, unlike anchor-based
heads, which the PPU decodes to pixels outright; more than
one layer present in the frame is what tells the two apart. A
second, low-scoring record on another layer is what proves that."""
relative to the grid cell it was predicted at, unlike a single-scale
head, which the PPU decodes to pixels outright; the scale count
tells the two apart."""
detections = decode_ppu(
[
build_ppu_records(
build_ppu_record((1.2, 0.5, 1.0, 0.5), grid=(9, 10, 0, 2), label=7),
build_ppu_record(
(0.2, 0.3, 0.1, 0.4), grid=(3, 4, 0, 0), score=0.1
),
)
],
[build_ppu_record((1.2, 0.5, 1.0, 0.5), grid=(9, 10, 0, 2), label=7)],
640,
640,
0.25,
0.45,
strides=strides_for(3),
anchor_based=False,
)
# layer 2 of 3 is stride 32: centre (10+1.2, 9+0.5)*32 = (358.4, 304),
@ -295,16 +301,17 @@ class TestDeepxPpuDecode(unittest.TestCase):
self.assertAlmostEqual(detections[0][4], 0.516218, places=5)
self.assertAlmostEqual(detections[0][5], 0.627957, places=5)
def test_a_single_layer_anchor_free_frame_still_reads_as_pixel_geometry(self):
"""A frame that only ever shows one layer cannot prove the grid and
layer fields are meaningful, so it falls back to the pixel-geometry
reading, same as an actual pixel-decoded head would need."""
def test_a_single_scale_anchor_free_head_reads_pixel_geometry(self):
"""Whatever a single-scale head leaves in the grid and layer fields,
its box fields are pixels."""
detections = decode_ppu(
[build_ppu_record((320.0, 160.0, 64.0, 32.0), grid=(7, 9, 0, 1))],
640,
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
)
self.assertAlmostEqual(detections[0][2], 144 / 640, places=5)
@ -319,6 +326,8 @@ class TestDeepxPpuDecode(unittest.TestCase):
640,
0.25,
0.45,
strides=strides_for(3),
anchor_based=True,
)
self.assertTrue(np.all(detections == 0))
@ -333,7 +342,8 @@ class TestDeepxPpuDecode(unittest.TestCase):
640,
0.25,
0.45,
scale_count=2,
strides=strides_for(2),
anchor_based=True,
)
# layer 0 of 2 is stride 16 and box 1 is the 23x27 anchor, so the
@ -343,95 +353,6 @@ class TestDeepxPpuDecode(unittest.TestCase):
self.assertAlmostEqual(detections[0][4], 0.223844, places=5)
self.assertAlmostEqual(detections[0][5], 0.248969, places=5)
def test_scale_count_defaults_to_the_three_scale_table(self):
"""With no scale_count given, a frame that only ever shows layer 0
cannot tell a two-scale head from the first layer of a three-scale
one, so it falls back to the three-scale table."""
detections = decode_ppu(
[build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 9, 1, 0))],
640,
640,
0.25,
0.45,
)
# layer 0 of 3 is stride 8 and box 1 is the 16x30 anchor, so the
# centre is (9.7, 7.3) cells out and the box is 5.76 x 58.8
self.assertAlmostEqual(detections[0][2], 0.045313, places=5)
self.assertAlmostEqual(detections[0][3], 0.116750, places=5)
self.assertAlmostEqual(detections[0][4], 0.137188, places=5)
self.assertAlmostEqual(detections[0][5], 0.125750, places=5)
def test_a_frame_of_only_the_finer_layers_still_reads_as_three_scales(self):
"""Layers 0 and 1 near the origin look the same from a two-scale
head and from the finer two thirds of a three-scale one, so with no
scale_count given the frame reads against the three-scale table,
never the two-scale one its highest layer index alone suggests."""
detections = decode_ppu(
[
build_ppu_records(
build_ppu_record((0.5, 0.5, 0.4, 0.4), grid=(3, 4, 0, 1)),
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 9, 1, 0)),
)
],
640,
640,
0.25,
0.45,
)
# layer 1 of 3 is stride 16 and box 0 is the 30x61 anchor, so the
# centre is (4.5, 3.5) cells out and the box is 19.2 x 39.04
layer_one = detections[np.isclose(detections[:, 3], 0.0975)]
self.assertEqual(len(layer_one), 1)
self.assertAlmostEqual(layer_one[0][2], 0.057, places=5)
self.assertAlmostEqual(layer_one[0][4], 0.118, places=5)
self.assertAlmostEqual(layer_one[0][5], 0.1275, places=5)
# layer 0 of 3 is stride 8 and box 1 is the 16x30 anchor
layer_zero = detections[np.isclose(detections[:, 3], 0.116750)]
self.assertEqual(len(layer_zero), 1)
self.assertAlmostEqual(layer_zero[0][2], 0.045313, places=5)
def test_scale_count_lower_bound_reads_layer_and_grid_position(self):
"""Every record proves at least one scale past its layer, and a
grid cell that does not fit a coarser stride proves more: a cell
index g at layer L needs stride * g < input, and the stride of
layer L in an S-scale head is 32 >> (S - 1 - L)."""
cases = (
# layer 0 cell 19 fits stride 32 (20 cells), so one scale suffices
((7, 19, 0, 0), 640, 640, 1),
# cell 20 needs stride 16, which is layer 0 of a two-scale head
((7, 20, 0, 0), 640, 640, 2),
# cell 39 still fits stride 16 (40 cells)
((7, 39, 0, 0), 640, 640, 2),
# cell 40 needs stride 8, which is layer 0 of a three-scale head
((7, 40, 0, 0), 640, 640, 3),
# layer 1 near the origin proves only two scales
((3, 4, 0, 1), 640, 640, 2),
# layer 1 at cell 20 needs stride 16, so three scales
((3, 20, 0, 1), 640, 640, 3),
# rows count against the height, here 416: cell 26 needs stride 8
((26, 0, 0, 0), 640, 416, 3),
# layer 2 alone proves three scales without any grid evidence
((0, 0, 0, 2), 640, 640, 3),
)
for grid, width, height, expected in cases:
with self.subTest(grid=grid, width=width, height=height):
records = build_ppu_record((0.5, 0.5, 0.4, 0.4), grid=grid)[0]
self.assertEqual(
ppu_scale_count_lower_bound(records, width, height), expected
)
def test_scale_count_lower_bound_takes_the_strongest_record(self):
"""Records that prove different counts settle on the largest."""
records = build_ppu_records(
build_ppu_record((0.5, 0.5, 0.4, 0.4), grid=(3, 4, 0, 1)),
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 40, 1, 0)),
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(1, 1, 0, 0)),
)[0]
self.assertEqual(ppu_scale_count_lower_bound(records, 640, 640), 3)
def test_anchor_based_records_with_an_unsupported_scale_count_are_dropped(self):
"""A scale_count with no anchor table must drop the records rather
than guess against the wrong table."""
@ -441,7 +362,8 @@ class TestDeepxPpuDecode(unittest.TestCase):
640,
0.25,
0.45,
scale_count=4,
strides=strides_for(4),
anchor_based=True,
)
self.assertTrue(np.all(detections == 0))
@ -458,6 +380,8 @@ class TestDeepxPpuDecode(unittest.TestCase):
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
)
self.assertEqual(detections[0][0], 2)
self.assertAlmostEqual(detections[0][2], 50 / 640, places=5)
@ -477,6 +401,8 @@ class TestDeepxPpuDecode(unittest.TestCase):
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
)
self.assertEqual(detections[0][0], 2)
# centre (100, 50), size 300 x 250, clipped at the frame edge
@ -492,26 +418,12 @@ class TestDeepxPpuDecode(unittest.TestCase):
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
centre_boxes=True,
)
self.assertAlmostEqual(detections[0][5], 250 / 640, places=5)
def test_needs_grid_decode_true_is_kept_for_a_single_layer_frame(self):
"""A caller that already proved the head needs grid decoding must
not have that overridden by a frame that, alone, looks ambiguous."""
detections = decode_ppu(
[build_ppu_record((1.2, 0.5, 1.0, 0.5), grid=(9, 10, 0, 2))],
640,
640,
0.25,
0.45,
needs_grid_decode=True,
)
# layer 2 of 3 is stride 32: centre (10+1.2, 9+0.5)*32 = (358.4, 304),
# size exp(1.0)*32 x exp(0.5)*32 = 86.985 x 52.759
self.assertAlmostEqual(detections[0][3], 0.492043, places=5)
def test_drops_records_below_the_score_threshold(self):
detections = decode_ppu(
[build_ppu_record((320.0, 160.0, 64.0, 32.0), score=0.1)],
@ -519,6 +431,8 @@ class TestDeepxPpuDecode(unittest.TestCase):
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
)
self.assertTrue(np.all(detections == 0))
@ -530,12 +444,38 @@ class TestDeepxPpuDecode(unittest.TestCase):
with self.subTest(shape=shape):
out = np.zeros(shape, dtype=np.uint8)
self.assertTrue(np.all(decode_ppu([out], 640, 640, 0.25, 0.45) == 0))
self.assertTrue(
np.all(
decode_ppu(
[out],
640,
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
)
== 0
)
)
def test_returns_empty_detections_for_an_unexpected_record_width(self):
out = np.zeros((1, 3, 16), dtype=np.uint8)
self.assertTrue(np.all(decode_ppu([out], 640, 640, 0.25, 0.45) == 0))
self.assertTrue(
np.all(
decode_ppu(
[out],
640,
640,
0.25,
0.45,
strides=strides_for(1),
anchor_based=False,
)
== 0
)
)
def layout_of(shapes, num_classes, ppu=False, dynamic_output=False) -> YoloLayout:
@ -1159,21 +1099,17 @@ class TestDeepxDetectorLayout(unittest.TestCase):
self.assertEqual(detector.output.columns, 84)
def test_a_ppu_model_is_recognized_by_the_runtime_flag(self):
# the shape DX-RT 3.4 reports for the compiled PPU model
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
)
detector = self._ppu_detector_for(ANCHOR_FREE_PPU, [(100, 84, 1)])
self.assertIs(detector.output.layout, YoloLayout.ppu)
def test_an_anchor_based_ppu_model_decodes(self):
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
detector = self._ppu_detector_for(
ANCHOR_BASED_PPU, [(80, 80, 3), (40, 40, 3), (20, 20, 3)]
)
detector.session.run.return_value = [
build_ppu_record((0.6, 0.4, 0.3, 0.7), label=5)
]
detector.width, detector.height = 640, 640
detections = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
@ -1181,129 +1117,25 @@ class TestDeepxDetectorLayout(unittest.TestCase):
self.assertAlmostEqual(detections[0][3], 0.380094, places=5)
self.assertAlmostEqual(detections[0][5], 0.589906, places=5)
def test_anchor_free_is_kept_once_proven(self):
"""A later frame whose boxes all happen to fall within 1.0 must not
re-open the question and start dropping detections."""
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
)
detector.width, detector.height = 640, 640
detector.session.run.return_value = [
build_ppu_record((320.0, 160.0, 64.0, 32.0), label=3)
]
first = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertTrue(detector.ppu_anchor_free)
detector.session.run.return_value = [build_ppu_record((0.6, 0.4, 0.3, 0.7))]
second = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertEqual(first[0][0], 3)
# still read as pixels, which is a sub-pixel box in the corner
self.assertAlmostEqual(second[0][1], 0.9, places=5)
def test_an_anchor_based_verdict_is_not_kept(self):
"""An anchor-free model whose first record is a sub-pixel box in the
corner looks anchor-based for that frame only; the next normal box
decodes and settles the head."""
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
)
detector.width, detector.height = 640, 640
detector.session.run.return_value = [build_ppu_record((0.6, 0.4, 0.3, 0.7))]
first = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertAlmostEqual(first[0][3], 0.380094, places=5)
self.assertFalse(detector.ppu_anchor_free)
detector.session.run.return_value = [
build_ppu_record((320.0, 160.0, 64.0, 32.0), label=3)
]
second = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertEqual(second[0][0], 3)
self.assertTrue(detector.ppu_anchor_free)
def test_ppu_scale_count_without_a_layout_never_drops_below_three(self):
"""With no layout to read, a three-scale head whose first frames
only hold layers 0 and 1 near the origin must not be read as a
two-scale head: those records are exactly what a three-scale head
emits too, so every frame keeps the three-scale table."""
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
)
detector.width, detector.height = 640, 640
detector.session.run.return_value = [
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 9, 1, 0))
]
first = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
# layer 0 of 3: stride 8, anchor 16x30
self.assertAlmostEqual(first[0][3], 0.116750, places=5)
self.assertEqual(detector.ppu_scale_count, 3)
detector.session.run.return_value = [
build_ppu_record((0.5, 0.5, 0.4, 0.4), grid=(3, 4, 0, 1))
]
second = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
# layer 1 of 3: stride 16, anchor 30x61, not layer 1 of 2
self.assertAlmostEqual(second[0][3], 0.0975, places=5)
self.assertEqual(detector.ppu_scale_count, 3)
detector.session.run.return_value = [
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 9, 1, 0))
]
third = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertAlmostEqual(third[0][3], 0.116750, places=5)
def test_ppu_scale_count_grows_when_a_grid_cell_proves_a_fourth_scale(self):
"""A layer-0 record at column 80 of 640 needs stride 4, which proves
a fourth scale with no layer-3 record ever appearing; there is no
table for that, so it is reported once and every frame after that
stays unplaced rather than reading against the three-scale table."""
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
)
detector.width, detector.height = 640, 640
layer_one = build_ppu_record((0.5, 0.5, 0.4, 0.4), grid=(3, 4, 0, 1))
detector.session.run.return_value = [layer_one]
first = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertAlmostEqual(first[0][3], 0.0975, places=5)
detector.session.run.return_value = [
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 80, 1, 0))
]
with self.assertLogs("frigate.detectors.plugins.deepx", level="ERROR") as logs:
second = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
detector.session.run.return_value = [layer_one]
third = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertEqual(detector.ppu_scale_count, 4)
self.assertTrue(np.all(second == 0) and np.all(third == 0))
self.assertEqual(len([r for r in logs.records if r.levelname == "ERROR"]), 1)
def _ppu_detector_for(self, ppu, layers):
def _ppu_detector_for(self, ppu, layers, model_type=ModelTypeEnum.yologeneric):
"""A PPU detector loaded from a model file that describes its head."""
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
path = write_dxnn(tmp.name, ppu, layers)
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True, model_path=path
model_type, [{"shape": [8400]}], ppu=True, model_path=path
)
detector.width, detector.height = 640, 640
return detector
def test_the_ppu_scale_count_is_read_from_the_model_at_load(self):
"""A three-scale head reads against the three-scale table from its
first frame. A layer-0 record near the origin followed by a layer-1
one is exactly what record evidence alone would call a two-scale
head; the model file says otherwise and the count never moves."""
first frame, whatever layers that frame holds."""
detector = self._ppu_detector_for(
ANCHOR_BASED_PPU, [(80, 80, 3), (40, 40, 3), (20, 20, 3)]
)
self.assertEqual(detector.ppu_scale_count, 3)
self.assertEqual(detector.ppu_layout.scale_count, 3)
detector.session.run.return_value = [
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 9, 1, 0))
@ -1318,16 +1150,12 @@ class TestDeepxDetectorLayout(unittest.TestCase):
second = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
# layer 1 of 3: stride 16, anchor 30x61, not layer 1 of 2
self.assertAlmostEqual(second[0][3], 0.0975, places=5)
self.assertEqual(detector.ppu_scale_count, 3)
def test_an_anchor_free_model_settles_the_head_at_load(self):
"""With the head kind in the file, a sub-pixel box in the corner,
which the per-frame heuristic would take for an anchor-based
record, is read as the pixels it is; and a YOLOX-style head is
grid-decoded from a first frame holding one record at one layer."""
"""An anchor-free head reads a sub-pixel box in the corner as the
pixels it is, and a YOLOX-style head is grid-decoded from a first
frame holding one record at one layer."""
detector = self._ppu_detector_for(ANCHOR_FREE_PPU, [(100, 84, 1)])
self.assertTrue(detector.ppu_anchor_free)
self.assertFalse(detector.ppu_needs_grid_decode)
detector.session.run.return_value = [build_ppu_record((0.6, 0.4, 0.3, 0.7))]
detections = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
@ -1336,25 +1164,45 @@ class TestDeepxDetectorLayout(unittest.TestCase):
detector = self._ppu_detector_for(
ANCHOR_FREE_PPU, [(80, 80, 1), (40, 40, 1), (20, 20, 1)]
)
self.assertTrue(detector.ppu_needs_grid_decode)
detector.session.run.return_value = [
build_ppu_record((1.2, 0.5, 1.0, 0.5), grid=(9, 10, 0, 2), label=7)
]
detections = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertAlmostEqual(detections[0][3], 0.492043, places=5)
def test_an_unreadable_model_file_falls_back_to_record_evidence(self):
with self.assertLogs(
"frigate.detectors.plugins.deepx", level="WARNING"
) as logs:
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
def test_a_model_without_a_ppu_layout_is_refused(self):
with self.assertRaisesRegex(ValueError, "DX-COM 2.4.0"):
self._detector(ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True)
def test_a_ppu_head_frigate_cannot_decode_is_refused(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
path = write_dxnn(tmp.name, {"type": 2}, [(80, 80, 1)])
with self.assertRaisesRegex(ValueError, "face and pose"):
self._detector(
ModelTypeEnum.yologeneric,
[{"shape": [8400]}],
ppu=True,
model_path=path,
)
self.assertTrue(any("PPU" in r.getMessage() for r in logs.records))
self.assertIsNone(detector.ppu_layout)
self.assertEqual(detector.ppu_scale_count, 0)
def test_strides_come_from_the_grids_in_the_model(self):
"""A head whose grids are not the 32 >> k ladder, here P4 to P6
at 16, 32 and 64, decodes at the strides its grids imply."""
detector = self._ppu_detector_for(
ANCHOR_FREE_PPU, [(40, 40, 1), (20, 20, 1), (10, 10, 1)]
)
detector.session.run.return_value = [
build_ppu_record((1.2, 0.5, 1.0, 0.5), grid=(9, 10, 0, 0), label=7)
]
detections = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
# layer 0 is stride 16: centre (11.2, 9.5) * 16, size e * 16 x sqrt(e) * 16
self.assertEqual(detections[0][0], 7)
self.assertAlmostEqual(detections[0][2], (152 - np.exp(0.5) * 8) / 640, 5)
self.assertAlmostEqual(detections[0][3], (179.2 - np.exp(1.0) * 8) / 640, 5)
def test_centre_boxes_are_kept_once_a_record_proves_them(self):
"""A centre-format head is settled by its first record with a width
@ -1384,40 +1232,13 @@ class TestDeepxDetectorLayout(unittest.TestCase):
self.assertFalse(detector.ppu_centre_boxes)
self.assertAlmostEqual(first[0][5], 300 / 640, places=5)
def test_needs_grid_decode_is_kept_once_proven(self):
"""A frame proving a YOLOX-style grid-relative head must not fall
back to reading raw pixels just because a later frame holds only
one detection at one layer."""
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
)
detector.width, detector.height = 640, 640
detector.session.run.return_value = [
build_ppu_records(
build_ppu_record((1.2, 0.5, 1.0, 0.5), grid=(9, 10, 0, 2), label=7),
build_ppu_record((0.2, 0.3, 0.1, 0.4), grid=(3, 4, 0, 0), score=0.1),
)
]
first = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertAlmostEqual(first[0][3], 0.492043, places=5)
self.assertTrue(detector.ppu_needs_grid_decode)
detector.session.run.return_value = [
build_ppu_record((1.2, 0.5, 1.0, 0.5), grid=(9, 10, 0, 2), label=7)
]
second = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
# still decoded through the grid and stride, not as raw pixels
self.assertAlmostEqual(second[0][3], 0.492043, places=5)
def test_an_unsupported_ppu_scale_count_is_reported_once(self):
"""An anchor-based head with a scale count Frigate has no anchor
table for logs once and drops the frame instead of guessing."""
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
detector = self._ppu_detector_for(
ANCHOR_BASED_PPU,
[(160, 160, 3), (80, 80, 3), (40, 40, 3), (20, 20, 3)],
)
detector.width, detector.height = 640, 640
detector.session.run.return_value = [
build_ppu_record((0.6, 0.4, 0.3, 0.7), grid=(7, 9, 2, 3))
]
@ -1429,25 +1250,6 @@ class TestDeepxDetectorLayout(unittest.TestCase):
self.assertTrue(np.all(first == 0) and np.all(second == 0))
self.assertEqual(len([r for r in logs.records if r.levelname == "ERROR"]), 1)
def test_an_empty_or_all_zero_ppu_frame_proves_nothing(self):
"""Only a box value above 1 (anchor-free) or a frame of values
strictly inside 0..1 (anchor-based) is evidence; an empty frame or a
degenerate all-zero record is neither."""
detector, _ = self._detector(
ModelTypeEnum.yologeneric, [{"shape": [8400]}], ppu=True
)
detector.width, detector.height = 640, 640
for frame in (
np.zeros((1, 0, PPU_RECORD_SIZE), np.uint8),
build_ppu_record((0.0, 0.0, 0.0, 0.0)),
):
detector.session.run.return_value = [frame]
detections = detector.detect_raw(np.zeros((1, 640, 640, 3), np.uint8))
self.assertTrue(np.all(detections == 0))
self.assertFalse(detector.ppu_anchor_free)
def test_a_yolox_raw_head_is_decoded_through_the_grid(self):
"""model_type yolox reads the raw head: a cell's x, y offsets are
added to its grid position and scaled by the stride, its w, h are
@ -1481,7 +1283,11 @@ class TestDeepxDetectorLayout(unittest.TestCase):
self.assertAlmostEqual(detections[0][5], 100 / 640, places=5)
def test_a_yolox_compiled_with_ppu_support_is_read_as_ppu(self):
detector, _ = self._detector(ModelTypeEnum.yolox, [{"shape": [8400]}], ppu=True)
detector = self._ppu_detector_for(
ANCHOR_FREE_PPU,
[(80, 80, 1), (40, 40, 1), (20, 20, 1)],
model_type=ModelTypeEnum.yolox,
)
self.assertIs(detector.output.layout, YoloLayout.ppu)