mirror of
https://github.com/navidrome/navidrome.git
synced 2026-08-31 07:30:32 +00:00
* feat(cli): add `artwork cancel` to call off queued artwork work A bulk backfill had no off switch. Changing an artwork setting bumps the config fingerprint, which enqueues every entity in the library, and the only way to stop it was to turn agents off -- which changes the fingerprint again and enqueues a second full backfill. The escape hatch was the trap. `artwork cancel` deletes pending queue rows selected by --kind and/or --priority, with the --dry-run/confirm/-y flow `reprocess` already uses. Cancelling by priority is the point: it drops a runaway backfill while leaving the bump-priority rows an operator queued by hand. It only touches the queue. Resolved artwork and the item_artwork state behind `artwork explain` are left alone, and the trace of why a cancelled item last failed goes with its row. Preserving that trace would mean writing it to last_failure, which `explain` prints under "Gave up after" -- reporting a cancellation as an exhausted retry budget. The help text says the trace is discarded instead. Two limits the help text states, because neither is guessable: work already dequeued is not interrupted, and an item with no artwork state yet can be queued again by the hourly missing-artwork recheck. Cancel calls off queued work; it does not stop the worker. --kind validates against RefreshableKinds, not the RecheckKinds `reprocess` uses: the queue holds media file rows, so --all has to reach them. PurgeQueued follows the repository's naming rule -- it finds its own rows and reports how many went -- and ignores retry_at, since a row still backing off is pending work. The preview reuses CountByKindAndPriority rather than adding a counter. reprocessConfirm became confirmUnlessYes(yes, in, verb) now that two commands prompt. * refactor(cli): share the artwork queue filter between the preview and the delete Follow-up cleanup on the previous commit; no change to what the command does, apart from --all, noted below. The "which rows does cancel touch" predicate was written three times: once as SQL in PurgeQueued, once in Go in cmd's matchingQueueStats, and once more in the mock. The preview and the delete could therefore drift, and the mock would keep the tests green while they did. persistence now has one artworkQueueFilter, shared by PurgeQueued and a new CountQueued, and cmd does no filtering at all. That also makes the preview cheaper. It counted the whole queue and filtered in Go, so `artwork cancel --kind al` scanned every row of every kind to print a handful. CountQueued pushes the filter into SQL, which the drain index serves as a range seek. CountByKindAndPriority is gone: it is CountQueued(nil, nil). --all now selects with an empty filter instead of enumerating RefreshableKinds. It is what the flag help already claimed, and the enumeration was narrower than its own documentation -- a queue row whose item_kind this build does not know survived `--all` with no flag combination able to remove it. It also restores SQLite's truncate path: measured with EXPLAIN QUERY PLAN, a bare DELETE plans to nothing, while `WHERE (1=1)` -- which an empty squirrel And renders -- plans to a full index scan. A test pins the filter's emptiness so that cannot regress silently. Also folded together three copies of the parse-and-dedup loop (parseAll), two copies of the queue-stats table (printQueueStats, now shared with `artwork status`), two copies of the stat sum (queueTotal), and four copies of the kind-to-prefix mapping (model.KindPrefixes). The PurgeQueued specs became one DescribeTable that asserts count and delete agree on every selection. * docs(cli): say when `artwork cancel` evaluates its selection The help text covered the two limits that surprise an operator after the fact, but not the one that bites during the prompt: the count is a preview, and the filters run again on confirm. A scan or a manual refresh landing in between is cancelled without ever appearing in the table the operator agreed to. Deleting only the previewed rows was considered and rejected. The exposure is one item re-resolving on next view instead of immediately: clearing an item's artwork state is what every recovery path selects on, so a lost Bump row from artwork.Refresh comes back at the same priority via provisional() on the next request, and otherwise within the hour via EnqueueAllMissing. Buying a guarantee against that costs the truncate path on --all, the flag that exists for a 29k-item backfill. * refactor(cli): share one set of flag targets across the artwork subcommands reprocess and cancel each declared their own kinds/all/dry-run/yes variables, but cobra only ever parses the one subcommand being run, so the two sets could never hold values at the same time. backup.go already binds one backupDir across two subcommands and one force across two more; this follows that. Ten package-level variables become six. Each command keeps its own help string and its own valid-kind list, so --kind still reports RecheckKinds for reprocess and RefreshableKinds for cancel, and --source and --priority stay registered only on the command that has them. The priority lookup table is now knownPriorities, freeing the artworkPriorities name for the flag. The new name also reads better against priorityName's fallback for a value it does not know.
182 lines
8.1 KiB
Go
182 lines
8.1 KiB
Go
package model
|
|
|
|
import "time"
|
|
|
|
// Artwork is one unique image, identified by the XXH3-64 hash of its bytes.
|
|
type Artwork struct {
|
|
Hash string `structs:"hash"`
|
|
Mime string `structs:"mime"`
|
|
Width int `structs:"width"`
|
|
Height int `structs:"height"`
|
|
SizeBytes int64 `structs:"size_bytes"`
|
|
BlurHash string `structs:"blur_hash"`
|
|
ThumbHash string `structs:"thumb_hash"`
|
|
// DominantColor is "#rrggbb": a flat placeholder clients can paint before any decode.
|
|
DominantColor string `structs:"dominant_color"`
|
|
CreatedAt time.Time `structs:"created_at"`
|
|
}
|
|
|
|
const ImageTypePrimary = "primary"
|
|
|
|
// ItemImage is per-entity artwork state hydrated at query time; never persisted.
|
|
type ItemImage struct {
|
|
ImageHash string `structs:"-" json:"imageHash,omitempty"`
|
|
ImageAbsent bool `structs:"-" json:"imageAbsent,omitempty"`
|
|
// BlurHash is Jellyfin's; its mappers read this field directly, so it stays off native JSON.
|
|
BlurHash string `structs:"-" json:"-"`
|
|
ThumbHash string `structs:"-" json:"thumbHash,omitempty"`
|
|
// DominantColor is the only placeholder needing no decode, so it can paint on the first frame.
|
|
DominantColor string `structs:"-" json:"dominantColor,omitempty"`
|
|
// A thumbhash's own aspect is quantised, so clients need these to shape the placeholder exactly.
|
|
ImageWidth int `structs:"-" json:"imageWidth,omitempty"`
|
|
ImageHeight int `structs:"-" json:"imageHeight,omitempty"`
|
|
}
|
|
|
|
// AspectRatio is the image's width/height, or nil when the image or its dimensions are unknown.
|
|
func (i ItemImage) AspectRatio() *float64 {
|
|
if i.ImageAbsent || i.ImageWidth <= 0 || i.ImageHeight <= 0 {
|
|
return nil
|
|
}
|
|
return new(float64(i.ImageWidth) / float64(i.ImageHeight))
|
|
}
|
|
|
|
// ItemArtwork is an entity's resolved artwork state. Hash=="" means known absent.
|
|
type ItemArtwork struct {
|
|
ItemKind string `structs:"item_kind"`
|
|
ItemID string `structs:"item_id"`
|
|
ImageType string `structs:"image_type"`
|
|
Hash string `structs:"hash"`
|
|
Source string `structs:"source"`
|
|
// SourcePath is the backing file (folder/upload: the image; embedded: the audio file); "" otherwise.
|
|
SourcePath string `structs:"source_path"`
|
|
// RefMtime is SourcePath's mtime (unix-nanoseconds) at resolution; 0 when there is no SourcePath.
|
|
RefMtime int64 `structs:"ref_mtime"`
|
|
// Trace is the encoded walk that produced this state; LastFailure is the walk of the attempt
|
|
// that exhausted the retry budget. Both are JSON, read back with artwork.DecodeTrace.
|
|
Trace string `structs:"trace"`
|
|
LastFailure string `structs:"last_failure"`
|
|
// Nullable in the schema, but every insert must set them: these non-pointer fields cannot scan NULL.
|
|
AttemptedAt time.Time `structs:"attempted_at"`
|
|
UpdatedAt time.Time `structs:"updated_at"`
|
|
}
|
|
|
|
// ItemArtworkInfo is the list-hydration projection (item_artwork joined with artwork).
|
|
type ItemArtworkInfo struct {
|
|
ItemID string
|
|
Hash string
|
|
BlurHash string
|
|
ThumbHash string
|
|
DominantColor string
|
|
Width int
|
|
Height int
|
|
}
|
|
|
|
// Absent reports a known-absent artwork state (resolved, no image).
|
|
func (i ItemArtworkInfo) Absent() bool { return i.Hash == "" }
|
|
|
|
// Image projects the hydration entry onto the entity-facing struct.
|
|
func (i ItemArtworkInfo) Image() ItemImage {
|
|
return ItemImage{
|
|
ImageHash: i.Hash,
|
|
ImageAbsent: i.Absent(),
|
|
BlurHash: i.BlurHash,
|
|
ThumbHash: i.ThumbHash,
|
|
DominantColor: i.DominantColor,
|
|
ImageWidth: i.Width,
|
|
ImageHeight: i.Height,
|
|
}
|
|
}
|
|
|
|
type ArtworkQueueItem struct {
|
|
ItemKind string `structs:"item_kind"`
|
|
ItemID string `structs:"item_id"`
|
|
ImageType string `structs:"image_type"`
|
|
Priority int `structs:"priority"`
|
|
Attempts int `structs:"attempts"`
|
|
RetryAt time.Time `structs:"retry_at"`
|
|
EnqueuedAt time.Time `structs:"enqueued_at"`
|
|
// Trace is why the last attempt failed. Only Get reads it; the drain projects it away.
|
|
Trace string `structs:"trace"`
|
|
}
|
|
|
|
// Queue priorities: higher drains first.
|
|
const (
|
|
ArtworkPriorityRecheck = 0
|
|
ArtworkPriorityBackfill = 10
|
|
ArtworkPriorityScan = 50
|
|
ArtworkPriorityBump = 100
|
|
)
|
|
|
|
// Delete* takes the rows to remove; Purge* finds them itself and reports how many went.
|
|
type ArtworkRepository interface {
|
|
GetImage(hash string) (*Artwork, error)
|
|
PutImage(a *Artwork) error
|
|
// PurgeOrphans deletes rows referenced by no item_artwork row and older than cutoff.
|
|
PurgeOrphans(createdBefore time.Time) (int64, error)
|
|
GetItemArtwork(kind Kind, id, imageType string) (*ItemArtwork, error)
|
|
PutItemArtwork(ia *ItemArtwork) error
|
|
// PutLastFailure records the trace of the attempt that exhausted the retry budget.
|
|
PutLastFailure(kind Kind, id, imageType, trace string) error
|
|
DeleteForItems(kind Kind, ids []string) error
|
|
// GetInfoForItems hydrates a page in one batched query.
|
|
GetInfoForItems(kind Kind, ids []string) (map[string]ItemArtworkInfo, error)
|
|
// GetMimeByHash returns hash -> current mime for every stored artwork.
|
|
GetMimeByHash() (map[string]string, error)
|
|
// PurgeDanglingItems removes state rows whose entity no longer exists.
|
|
PurgeDanglingItems() (int64, error)
|
|
}
|
|
|
|
type ArtworkQueueRepository interface {
|
|
// Get returns the pending row for an item, or ErrNotFound when it is not queued.
|
|
Get(kind Kind, id, imageType string) (*ArtworkQueueItem, error)
|
|
// Enqueue upserts; an existing row keeps the higher priority and has its retry_at reset.
|
|
Enqueue(items ...ArtworkQueueItem) error
|
|
// EnqueuePreservingBackoff upserts like Enqueue but preserves an existing row's retry_at, so a
|
|
// request-triggered read-through never resets a failed resolution's backoff.
|
|
EnqueuePreservingBackoff(items ...ArtworkQueueItem) error
|
|
// EnqueueStaleAbsent inserts queue rows (priority Recheck) for absent states older than cutoff.
|
|
EnqueueStaleAbsent(kind Kind, attemptedBefore time.Time) (int64, error)
|
|
// EnqueueAllMissing inserts queue rows for all entities with no item_artwork row, at the given priority.
|
|
EnqueueAllMissing(kind Kind, priority int) (int64, error)
|
|
// EnqueueIfMissing inserts only for items with no item_artwork row yet.
|
|
EnqueueIfMissing(items ...ArtworkQueueItem) error
|
|
// CountBySource reports how many items of a kind currently resolve from the given sources.
|
|
// An empty sources slice means every source; "" matches absent state.
|
|
CountBySource(kind Kind, sources []string) (int64, error)
|
|
// SourcesInUse lists the distinct sources items of a kind currently resolve from, "" included.
|
|
SourcesInUse(kind Kind) ([]string, error)
|
|
// EnqueueBySource inserts queue rows for items of a kind whose current source matches.
|
|
// It does not clear existing artwork state: the current image stays until it is replaced.
|
|
EnqueueBySource(kind Kind, sources []string, priority int) (int64, error)
|
|
// DequeueBatch returns up to n items with retry_at <= now, priority desc, enqueued_at asc.
|
|
// Restricted to the given kinds when any are passed, so one kind cannot block another's drain.
|
|
DequeueBatch(n int, kinds ...string) ([]ArtworkQueueItem, error)
|
|
// MarkFailedIfUnchanged applies the failure backoff only while retry_at still matches
|
|
// seenRetryAt, so a concurrent re-enqueue keeps its fresh eligibility.
|
|
MarkFailedIfUnchanged(kind, id, imageType string, seenRetryAt, retryAt time.Time, trace string) error
|
|
// DeleteIfUnchanged deletes only while retry_at still matches, sparing a concurrent re-enqueue.
|
|
DeleteIfUnchanged(kind, id, imageType string, retryAt time.Time) error
|
|
Count() (int64, error)
|
|
// CountQueued reports the pending rows matching the kinds and priorities, grouped by both;
|
|
// an empty filter means every one.
|
|
CountQueued(kinds []Kind, priorities []int) ([]ArtworkQueueStat, error)
|
|
// CountAbsent reports the absent states of a kind, and how many of those EnqueueStaleAbsent
|
|
// would pick up at the given cutoff.
|
|
CountAbsent(kind Kind, attemptedBefore time.Time) (ArtworkAbsentStat, error)
|
|
// PurgeDangling removes queue rows whose entity no longer exists.
|
|
PurgeDangling() (int64, error)
|
|
// PurgeQueued removes pending rows matching the kinds and priorities; an empty filter means every one.
|
|
PurgeQueued(kinds []Kind, priorities []int) (int64, error)
|
|
}
|
|
|
|
type ArtworkQueueStat struct {
|
|
ItemKind string
|
|
Priority int
|
|
Count int64
|
|
}
|
|
|
|
type ArtworkAbsentStat struct {
|
|
Total int64
|
|
Stale int64
|
|
}
|