mirror of
https://github.com/navidrome/navidrome.git
synced 2026-08-31 07:30:32 +00:00
feat(cli): add an artwork command group for diagnosing and re-driving artwork (#5957)
* feat(artwork): add a resolution chain trace collector * feat(artwork): trace the local priority chain * fix(artwork): record priority candidates the chain never evaluated * refactor(artwork): report never-evaluated candidates as skipped * feat(artwork): trace external agents at the gate seam * feat(artwork): add repository queries to enqueue by current source * feat(artwork): expose a tracing resolver for the CLI * feat(artwork): read a single queue row by item The explain CLI must report whether an item is queued, at what priority and when it retries; the queue repository could only be drained in eligibility batches, which cannot see a row that is still backing off. * feat(cli): add artwork explain Prints why an item has the artwork it has: the stored state, its queue row, the governing config, the resolver's priority-chain walk and the verdict. Offline by default so a diagnostic run cannot add load to an external provider; --live asks the agents for real. Playlists and radios do not walk a priority chain, so they report that instead of an empty chain table. * fix(artwork): trace an external tier that never reaches an agent A configured 'external' token vanished from the chain when no enabled agent provided images for that entity type, and for synthetic artists, leaving the trace unable to say whether the tier was even considered. * fix(cli): never state an artwork outcome the walk did not observe A transient external failure traced as 'error' fell through to 'not resolved', which is the most common state behind a missing-artwork report. It is now indeterminate, and an offline win that a skipped higher-priority external candidate could have taken says so instead of naming a winner the live chain might not pick. * feat(cli): add artwork refresh * feat(cli): add artwork reprocess Bulk re-enqueues artwork by kind and/or by the source an item currently resolves from, previewing the matched count and confirming before queueing. The preview counts with CountBySource (rows matched) and reports separately what EnqueueBySource inserted: its DO NOTHING conflict policy leaves an already-queued row untouched, so the two numbers differ and the output must not claim the skipped rows were re-queued. An unknown --source is rejected against the sources present in item_artwork, rather than silently matching nothing and printing a reassuring 0. * fix(cli): cover the reprocess selection rule and validate sources table-wide The reconciliation that makes --source alone target every kind was only exercised through runReprocess, which no test calls: mutating it to `all := reprocessAll` left the suite green. It is now reprocessSelectsAll, covered for all three selectors. Scoping source validation to the selected kinds made the same well-formed filter valid or invalid depending on which other kinds were selected, and its error read the same for a typo as for a source that simply does not apply to the chosen kind. Validation is now table-wide: a typo still aborts, while a valid-but-inapplicable source falls through to "Nothing matches". Also: the prompt now counts only the kinds that reach an external agent as external cost, and --dry-run on an empty selection reports a dry run. * fix(cli): cover the reprocess --yes guard and preview the external cost Mutating the --yes check to `if true` left the suite green, so the one bypass of the confirmation was unverified. The choice is now reprocessConfirm(yes, in), covered in both directions. The external estimate only reached the operator through the prompt, which --dry-run skips — hiding the number in the one mode that exists to show it before committing. The preview now carries it, and the prompt drops the clause when no lookup will be made. An empty selection says so again under --dry-run. * feat(artwork): add read-only queue and absent counters Both are needed by the artwork status CLI: a queue breakdown by kind and priority, and the absent totals split against the recheck cutoff. * feat(cli): add artwork status Reports the queue, where artwork currently resolves from, absent counts against the 24h recheck window, and the stored config fingerprint versus the current one — the line that turns 'why is my server re-resolving everything?' into one command. fingerprint() and staleAbsentAge are exported so the CLI reports the values backfill itself compares, instead of a second copy of the formula that can silently drift. * fix(cli): lead the artwork status backfill line with the queued backlog By the time anyone runs a diagnostic, backfill has usually already stored the new fingerprint, so 'up to date' was printed while thousands of items churned through external providers. The backlog is the finding; the fingerprint is context. Also echoes the config inputs the fingerprint covers, so a change can be traced to the setting that caused it, and pins the rendered rows: the Absent values, the queue TOTAL and a queue-scoped kind/priority pair were all unasserted, so kindName and priorityName were effectively untested. FingerprintInputs is now the single listing ConfigFingerprint hashes; a pinned hash proves the value did not change. * refactor(artwork): export the trace outcome vocabulary The CLI hardcoded the outcome literals and the "external:" prefix, so renaming a constant's value in core/artwork left cmd compiling and the suite green while `artwork explain` silently degraded its verdict. Renaming a value now fails the golden vocabulary test in core/artwork and the explainResult tests in cmd. * fix(cli): keep the re-enqueue warning when a backfill is already running A stale stored fingerprint with items already queued is the worst state the system can be in: a second full re-enqueue is pending on top of the one running. The line carried the weakest wording of the three, and was untested. * refactor(artwork): drop the unreachable breaker branch from the tracing gate --live wires the tracing gate straight to passthroughGate, so errBreakerOpen can never reach it; the test only passed by injecting a fake gate. * refactor(artwork): delete the never-emitted not-reached outcome Candidates after the winner are lower priority and say nothing about why a source won; the ones that matter sit above it and are already recorded. * refactor(artwork): make the trace nil-safe in one place only add already handles a nil trace, so record's own guard was dead; Steps was the odd one out and would panic where every other method tolerates nil. * refactor(artwork): export the trace types directly ChainTrace and TraceStep were unexported types re-exported through aliases, which existed only so the CLI had a name to refer to them by. The types are public API — Resolver.Steps returns []TraceStep and the CLI constructs a ChainTrace — so name them that way and drop the indirection. Encapsulation is unchanged: add, mu and steps stay unexported, so only this package can write a step. * refactor(cli): simplify parseArtworkKind with slices.Contains Replaces a nested loop and a manual append with slices.Contains and the repo's slice.Map helper. Same behaviour, same error message. * fix(cli): print the absent artwork source under the name --source accepts `artwork explain` rendered the stored empty source as "(absent)", while `artwork reprocess --source` only accepts "absent", so pasting what explain printed straight back into reprocess was rejected as an unknown source. * refactor(artwork): own the kind list and the chain predicate in the package Export RecheckKinds and add WalksPriorityChain so the CLI stops keeping its own copies of both, and unexport externalCandidate, which nothing outside the package consumes. * refactor(cli): drop the artwork command's duplicated state and formatting Reuse artwork.RecheckKinds and artwork.WalksPriorityChain, extract newTabWriter and externalEstimate, fold reprocessSelectsAll into selectedKinds, and derive the queue total and the walks-chain flag instead of carrying them in the report structs. * test(persistence): drop two artwork-queue specs that cannot fail One seeded hash and source together and then asserted the two counts agree, so its setup guaranteed the result; the other repeated the count-does-not- enqueue property already covered by the CountBySource spec. * refactor(artwork): rename Resolver to TracingResolver for clarity * fix(cli): count playlists in the artwork reprocess external estimate The estimate used WalksPriorityChain, which is true only for artist and album, so a playlist-only reprocess reported "External lookups: none" and the confirmation prompt dropped the external-cost warning. Playlists do reach the network: through the m3u ExternalImageURL fetch when EnableM3UExternalAlbumArt is on, and — verified by test — through the generated grid, whose tiles resolve album art via the full album priority chain. Adds artwork.MayFetchExternal, a config-aware predicate for "can this kind's resolver reach the network", and uses it for the estimate. WalksPriorityChain keeps its separate job of deciding whether explain prints a chain block. * fix(cli): estimate artwork reprocess external lookups per agent, not per item The reprocess prompt billed one external lookup per externally-capable item. fetchArtistImage/fetchAlbumImage try every enabled image agent and stop early only on a hit, and resolvePlaylist can fetch the m3u image and then resolve up to four sampled albums for the grid, each walking the album agents again. The number the operator confirmed could understate real provider traffic several fold, in the prompt whose whole job is to stop a provider flood. ExternalLookupsPerItem now multiplies by the visible image-agent count and adds the playlist grid factor. It stays a floor: the CLI never calls Manager.Start(), so the plugin registry is empty and plugin-provided agents are dropped by getEnabledAgentNames. On an install with 5 agents of which 3 are plugins the count is well under the truth, so the wording is now "at least N" rather than "up to N" — a zero visible count still bills one lookup for the same reason. Fixing the plugin visibility is out of scope: Manager.Start() needs a Subsonic router and writes to the DB via syncPlugins, breaking this command group's read-only guarantee. * fix(cli): state the artwork reprocess estimate as an estimate, not a bound Neither bound is true. A ceiling is false because plugin agents are invisible to a CLI that never starts the plugin manager, and a floor is false because a local hit ends the walk before any agent is asked and a hit on the first agent skips the rest. "at least N" traded one wrong claim for another. The line now names its blind spots instead: External lookups: ~340 estimated (plugin agents not counted; local hits may need fewer). The same line is reused in the confirmation prompt, and the zero case still reads "External lookups: none." with the prompt dropping the clause entirely. The count itself is unchanged. * fix(cli): account for every configured agent in artwork explain The Agents: line printed the raw config while the Chain only showed the agents the CLI could construct, with nothing explaining the gap: plugin agents are never registered in a CLI that does not start the plugin manager, and a built-in without credentials returns nil. Three of five agents could vanish, including ones ranked above the one shown. Also treat a live external error before the winning hit like the already-handled would-try case: the resolver serves such a hit provisionally and retries later, so the verdict is indeterminate. The Result line is still not qualified when an unavailable agent might have won; that needs agent ranking, and is left to the follow-up that makes the CLI load plugin agents for real. * fix(cli): do not call an external artwork win indeterminate explainResult qualified the verdict whenever an external OutcomeError appeared before the winning hit. When a later external agent returns an image, fetchArtistImage/fetchAlbumImage discard the earlier error, so extError is false: the worker settles the item and schedules no retry. Telling the operator it may resolve differently on a retry was wrong. The warning is only correct when a lower-priority local source won while an external error was recorded, which is the case that carries extError. * fix(cli): accept --source absent when nothing is currently absent validateSources checks the requested sources against the ones item_artwork actually uses, to catch a typo. The reserved empty source (spelled 'absent' on the CLI) is a valid filter even when it matches nothing, so a scheduled 'artwork reprocess --source absent --yes' stopped working the moment the library finished resolving. Treat it as intrinsically valid and let the existing zero-match path report it. * feat(artwork): explain disc and media file artwork from the CLI `artwork explain` rejected `dc` and `mf` because it validated against RecheckKinds, the list of kinds the backfill revisits. Those are different questions: a kind with no recheck path still has artwork someone can report as wrong. Disc artwork now walks DiscArtPriority under a trace, so explain reports which entry won and why the others lost, including entries that map to no source at all (external is unsupported, a disc with no subtitle, an album folder with no images). Media file artwork traces its single embedded candidate, separating "EnableMediaFileCoverArt is off" from "the track has no embedded art" — stored state cannot tell those apart. Each command now validates against the kinds it can actually serve: explain takes all six, refresh takes artwork.RefreshableKinds (which nativeapi now shares instead of keeping its own copy), reprocess still takes RecheckKinds. Disc artwork stays out of refresh: the worker cannot resolve it, so the queue row would be rejected on every drain. WalksPriorityChain becomes Explainable, and ResolveArtist/ResolveAlbum collapse into Resolve(kind, id). * refactor(artwork): one disc-artwork walk for serving and explain resolveDisc duplicated the loop selectImageReader already ran: try each source in priority order, take the first that yields an image. The serving path and the CLI diverged on two details as a result — only selectImageReader checked ctx between candidates and logged each attempt. Both now call discArtworkReader.selectImage, which takes the chainState the CLI already uses for the other kinds. The serving path passes an untraced one, whose nil trace makes recording a no-op. selectImageReader had no other caller and is gone. The disc tests move from fromDiscArtPriority to discCandidates, so they assert the skip reason for an entry that maps to no source rather than that it silently vanished, and cancellation mid-walk is now covered. * fix(artwork): reject a nil reader in the resize cache instead of panicking resizedItem.Reader closes what open() hands back, so an open() that reports "no image" as (nil, nil) rather than an error takes the request down with a nil-pointer panic. Every caller returns an error today, and no test covered it: the resolution e2e harness stubs the resize reader out entirely, so no e2e path reaches this code at all. Guard it and cover Reader directly. * refactor(artwork): move the keeps-state fact into core, drop a redundant guard keepsArtworkState lived in package cmd and re-derived by hand what RefreshableKinds already encodes: the same five-of-six kinds. It is now artwork.KeepsState, beside the list, with a test pinning the two together — nothing else stopped them drifting, and a drift would have explain report stored state for a kind that keeps none. serveDisc's closure also hand-rolled a nil-reader error that both consumers of open() now produce themselves: serveSource for a full-size request, resizedItem.Reader for a resized one. * fix(artwork): route disc candidates through the shared resolvers openCandidate ran its own source loop and threw the error away, so a disc track that exists but cannot be parsed traced as "miss" — indistinguishable from a track with no embedded art. fromTag and fromFFmpegTag already report that case as errSourceUnreadable; only this loop was discarding it. Telling those two apart is what the trace is for. Candidates now carry a resolve func instead of raw sources: embedded goes to resolveEmbedded, and the folder-backed entries to resolveFolderSource, extracted from resolveFolderFile so both callers classify an unopenable file the same way. openCandidate and its absolute-path special case go away with it. Disc's own fromExternalFile and fromDiscSubtitle still swallow open errors, so folder candidates cannot report unreadable yet; that is a change to their error contracts. * fix(artwork): report an unreadable local candidate as indeterminate processor.acquire treats resolution.localError exactly as it treats extError: a fault is not a definitive "no image", so it retries instead of settling absent. explainResult qualified only the external case, so a chain that ended on an unreadable local candidate printed "not resolved" — the one verdict that says the walk was conclusive. The qualification belongs only to the unresolved branch. chainState.try stamps extErr onto a hit and deliberately drops localErr, so an unreadable step followed by a hit is settled as found and must not carry a warning; a test pins that. Found by Codex on 5f65d7cfa.
This commit is contained in:
Deluan Quintão
GitHub
parent
b617a878b9
commit
dc40bcaf80
778
cmd/artwork.go
Normal file
778
cmd/artwork.go
Normal file
@ -0,0 +1,778 @@
|
||||
package cmd
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
"github.com/navidrome/navidrome/consts"
|
||||
"github.com/navidrome/navidrome/core/agents"
|
||||
"github.com/navidrome/navidrome/core/artwork"
|
||||
"github.com/navidrome/navidrome/db"
|
||||
"github.com/navidrome/navidrome/log"
|
||||
"github.com/navidrome/navidrome/model"
|
||||
"github.com/navidrome/navidrome/utils/slice"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var explainLive bool
|
||||
|
||||
var (
|
||||
reprocessKinds []string
|
||||
reprocessSources []string
|
||||
reprocessAll bool
|
||||
reprocessDryRun bool
|
||||
reprocessYes bool
|
||||
)
|
||||
|
||||
func init() {
|
||||
artworkExplainCmd.Flags().BoolVar(&explainLive, "live", false,
|
||||
"perform real external lookups instead of reporting what would be tried")
|
||||
artworkReprocessCmd.Flags().StringSliceVar(&reprocessKinds, "kind", nil,
|
||||
"kinds to reprocess ("+kindPrefixes(artwork.RecheckKinds)+"); repeatable")
|
||||
artworkReprocessCmd.Flags().StringSliceVar(&reprocessSources, "source", nil,
|
||||
"only items currently resolved from these sources (e.g. folder, external:deezer, absent)")
|
||||
artworkReprocessCmd.Flags().BoolVar(&reprocessAll, "all", false, "reprocess every kind")
|
||||
artworkReprocessCmd.Flags().BoolVar(&reprocessDryRun, "dry-run", false,
|
||||
"report what would be queued and exit without queueing")
|
||||
artworkReprocessCmd.Flags().BoolVarP(&reprocessYes, "yes", "y", false, "skip the confirmation prompt")
|
||||
artworkCmd.AddCommand(artworkExplainCmd)
|
||||
artworkCmd.AddCommand(artworkRefreshCmd)
|
||||
artworkCmd.AddCommand(artworkReprocessCmd)
|
||||
artworkCmd.AddCommand(artworkStatusCmd)
|
||||
rootCmd.AddCommand(artworkCmd)
|
||||
}
|
||||
|
||||
var artworkCmd = &cobra.Command{
|
||||
Use: "artwork",
|
||||
Short: "Inspect and re-resolve artwork",
|
||||
}
|
||||
|
||||
var artworkExplainCmd = &cobra.Command{
|
||||
Use: "explain <kind> <id>",
|
||||
Short: "Explain why an item's artwork resolved the way it did",
|
||||
Long: "Explain why an item's artwork resolved the way it did.\n\n" +
|
||||
"<kind> is one of: " + kindPrefixes(explainKinds) + ".\n" +
|
||||
"A disc artwork id is the album id and the disc number, joined by a colon: <albumID>:2",
|
||||
Args: cobra.ExactArgs(2),
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
kind, err := parseArtworkKind(args[0], explainKinds)
|
||||
if err != nil {
|
||||
log.Fatal(cmd.Context(), err)
|
||||
}
|
||||
runExplain(cmd.Context(), kind, args[1])
|
||||
},
|
||||
}
|
||||
|
||||
var artworkRefreshCmd = &cobra.Command{
|
||||
Use: "refresh <kind> <id>...",
|
||||
Short: "Clear an item's artwork state and re-resolve it",
|
||||
Args: cobra.MinimumNArgs(2),
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
kind, err := parseArtworkKind(args[0], artwork.RefreshableKinds)
|
||||
if err != nil {
|
||||
log.Fatal(cmd.Context(), err)
|
||||
}
|
||||
runRefresh(cmd.Context(), kind, args[1:])
|
||||
},
|
||||
}
|
||||
|
||||
var artworkReprocessCmd = &cobra.Command{
|
||||
Use: "reprocess",
|
||||
Short: "Re-enqueue artwork in bulk, by kind and/or by the source it currently resolves from",
|
||||
Args: cobra.NoArgs,
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
runReprocess(cmd.Context())
|
||||
},
|
||||
}
|
||||
|
||||
var artworkStatusCmd = &cobra.Command{
|
||||
Use: "status",
|
||||
Short: "Report the artwork queue, where artwork resolves from, and the backfill state",
|
||||
Args: cobra.NoArgs,
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
runStatus(cmd.Context())
|
||||
},
|
||||
}
|
||||
|
||||
func runStatus(ctx context.Context) {
|
||||
defer db.Init(ctx)()
|
||||
ds, ctx := getAdminContext(ctx)
|
||||
|
||||
rep, err := collectStatus(ctx, ds)
|
||||
if err != nil {
|
||||
log.Fatal(ctx, err)
|
||||
}
|
||||
fmt.Print(formatStatus(rep))
|
||||
}
|
||||
|
||||
type sourceCount struct {
|
||||
kind model.Kind
|
||||
source string
|
||||
count int64
|
||||
}
|
||||
|
||||
type absentCount struct {
|
||||
kind model.Kind
|
||||
model.ArtworkAbsentStat
|
||||
}
|
||||
|
||||
type statusReport struct {
|
||||
queue []model.ArtworkQueueStat
|
||||
sources []sourceCount
|
||||
absent []absentCount
|
||||
inputs []artwork.FingerprintInput
|
||||
stored string
|
||||
current string
|
||||
}
|
||||
|
||||
func (r statusReport) queueTotal() int64 {
|
||||
var n int64
|
||||
for _, s := range r.queue {
|
||||
n += s.Count
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func (r statusReport) backfillQueued() int64 {
|
||||
var n int64
|
||||
for _, s := range r.queue {
|
||||
if s.Priority == model.ArtworkPriorityBackfill {
|
||||
n += s.Count
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func collectStatus(ctx context.Context, ds model.DataStore) (statusReport, error) {
|
||||
q := ds.ArtworkQueue(ctx)
|
||||
var rep statusReport
|
||||
var err error
|
||||
if rep.queue, err = q.CountByKindAndPriority(); err != nil {
|
||||
return rep, fmt.Errorf("breaking the artwork queue down by kind: %w", err)
|
||||
}
|
||||
|
||||
cutoff := time.Now().Add(-artwork.StaleAbsentAge)
|
||||
for _, k := range artwork.RecheckKinds {
|
||||
sources, err := q.SourcesInUse(k)
|
||||
if err != nil {
|
||||
return rep, fmt.Errorf("listing the sources in use by %s artwork: %w", k, err)
|
||||
}
|
||||
slices.Sort(sources)
|
||||
for _, s := range sources {
|
||||
n, err := q.CountBySource(k, []string{s})
|
||||
if err != nil {
|
||||
return rep, fmt.Errorf("counting %s artwork resolved from %s: %w", k, displaySource(s), err)
|
||||
}
|
||||
rep.sources = append(rep.sources, sourceCount{kind: k, source: s, count: n})
|
||||
}
|
||||
stat, err := q.CountAbsent(k, cutoff)
|
||||
if err != nil {
|
||||
return rep, fmt.Errorf("counting absent %s artwork: %w", k, err)
|
||||
}
|
||||
rep.absent = append(rep.absent, absentCount{kind: k, ArtworkAbsentStat: stat})
|
||||
}
|
||||
|
||||
rep.current, rep.inputs = artwork.ConfigFingerprint(), artwork.FingerprintInputs()
|
||||
if rep.stored, err = ds.Property(ctx).DefaultGet(consts.ArtConfFingerprintPropertyKey, ""); err != nil {
|
||||
return rep, fmt.Errorf("reading the stored artwork fingerprint: %w", err)
|
||||
}
|
||||
return rep, nil
|
||||
}
|
||||
|
||||
func formatStatus(rep statusReport) string {
|
||||
var sb strings.Builder
|
||||
w := newTabWriter(&sb)
|
||||
|
||||
fmt.Fprintln(w, "Queue")
|
||||
if len(rep.queue) == 0 {
|
||||
fmt.Fprintln(w, " (empty)")
|
||||
} else {
|
||||
fmt.Fprintln(w, " KIND\tPRIORITY\tITEMS")
|
||||
for _, s := range rep.queue {
|
||||
fmt.Fprintf(w, " %s\t%s\t%d\n", kindName(s.ItemKind), priorityName(s.Priority), s.Count)
|
||||
}
|
||||
fmt.Fprintf(w, " TOTAL\t\t%d\n", rep.queueTotal())
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "\nSources")
|
||||
fmt.Fprintln(w, " KIND\tSOURCE\tITEMS")
|
||||
for _, s := range rep.sources {
|
||||
fmt.Fprintf(w, " %s\t%s\t%d\n", s.kind, displaySource(s.source), s.count)
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "\nAbsent (resolved, no image found)")
|
||||
fmt.Fprintln(w, " KIND\tABSENT\tDUE FOR RECHECK")
|
||||
for _, a := range rep.absent {
|
||||
fmt.Fprintf(w, " %s\t%d\t%d\n", a.kind, a.Total, a.Stale)
|
||||
}
|
||||
fmt.Fprintf(w, " (rechecked once the last attempt is older than %gh)\n", artwork.StaleAbsentAge.Hours())
|
||||
|
||||
fmt.Fprintln(w, "\nBackfill")
|
||||
fmt.Fprintf(w, " State:\t%s\n", backfillState(rep))
|
||||
fmt.Fprintf(w, " Stored fingerprint:\t%s\n", cmp.Or(rep.stored, "(none)"))
|
||||
fmt.Fprintf(w, " Current fingerprint:\t%s\n", rep.current)
|
||||
if len(rep.inputs) > 0 {
|
||||
fmt.Fprintln(w, " Fingerprint inputs (changing any of these re-resolves the whole library):")
|
||||
for _, in := range rep.inputs {
|
||||
fmt.Fprintf(w, " %s:\t%s\n", in.Name, in.Value)
|
||||
}
|
||||
}
|
||||
|
||||
w.Flush()
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// backfillState leads with the queued backlog: by the time anyone runs this, backfill has usually
|
||||
// already stored the new fingerprint, and "up to date" would bury the flood it is still working through.
|
||||
func backfillState(rep statusReport) string {
|
||||
pending := "fingerprint changed — every artist, album, playlist and radio will be re-enqueued on the next startup"
|
||||
if n := rep.backfillQueued(); n > 0 {
|
||||
if rep.stored != rep.current {
|
||||
return fmt.Sprintf("backfill running: %d items queued, and %s", n, pending)
|
||||
}
|
||||
return fmt.Sprintf("backfill running: %d items queued (fingerprint up to date)", n)
|
||||
}
|
||||
if rep.stored != rep.current {
|
||||
return pending
|
||||
}
|
||||
return "up to date"
|
||||
}
|
||||
|
||||
func kindName(prefix string) string {
|
||||
if k, ok := model.ParseKind(prefix); ok {
|
||||
return k.String()
|
||||
}
|
||||
return prefix
|
||||
}
|
||||
|
||||
func priorityName(p int) string {
|
||||
switch p {
|
||||
case model.ArtworkPriorityRecheck:
|
||||
return "recheck"
|
||||
case model.ArtworkPriorityBackfill:
|
||||
return "backfill"
|
||||
case model.ArtworkPriorityScan:
|
||||
return "scan"
|
||||
case model.ArtworkPriorityBump:
|
||||
return "bump"
|
||||
}
|
||||
return strconv.Itoa(p)
|
||||
}
|
||||
|
||||
func runReprocess(ctx context.Context) {
|
||||
kinds, err := selectedKinds(reprocessKinds, reprocessSources, reprocessAll)
|
||||
if err != nil {
|
||||
log.Fatal(ctx, err)
|
||||
}
|
||||
|
||||
defer db.Init(ctx)()
|
||||
ds, ctx := getAdminContext(ctx)
|
||||
|
||||
if err := reprocessArtwork(ctx, ds, kinds, repositorySources(reprocessSources), imageAgentCount(ds),
|
||||
reprocessDryRun, reprocessConfirm(reprocessYes, os.Stdin), os.Stdout); err != nil {
|
||||
log.Fatal(ctx, err)
|
||||
}
|
||||
}
|
||||
|
||||
func selectedKinds(kinds, sources []string, all bool) ([]model.Kind, error) {
|
||||
// A source filter on its own is already a complete selection, so it does not also need a kind.
|
||||
if all || (len(kinds) == 0 && len(sources) > 0) {
|
||||
return artwork.RecheckKinds, nil
|
||||
}
|
||||
if len(kinds) == 0 {
|
||||
return nil, fmt.Errorf("no selector given: pass --kind, --source or --all")
|
||||
}
|
||||
out := make([]model.Kind, 0, len(kinds))
|
||||
for _, k := range kinds {
|
||||
kind, err := parseArtworkKind(k, artwork.RecheckKinds)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, kind)
|
||||
}
|
||||
// A repeated kind would be counted twice, overstating the cost the operator confirms.
|
||||
return slice.Unique(out), nil
|
||||
}
|
||||
|
||||
// absentSource is how the stored empty source — resolved, no image — is spelled on the CLI.
|
||||
const absentSource = "absent"
|
||||
|
||||
func repositorySources(sources []string) []string {
|
||||
return slice.Map(sources, func(s string) string {
|
||||
if s == absentSource {
|
||||
return ""
|
||||
}
|
||||
return s
|
||||
})
|
||||
}
|
||||
|
||||
func displaySource(s string) string { return cmp.Or(s, absentSource) }
|
||||
|
||||
type confirmFunc func(out io.Writer, total, external int64) bool
|
||||
|
||||
func reprocessConfirm(yes bool, in io.Reader) confirmFunc {
|
||||
if yes {
|
||||
return func(io.Writer, int64, int64) bool { return true }
|
||||
}
|
||||
return promptConfirm(in)
|
||||
}
|
||||
|
||||
// externalEstimate claims no bound: a local hit ends the walk before any agent is asked, and plugin
|
||||
// agents are unregistered in a CLI that never starts the plugin manager.
|
||||
func externalEstimate(n int64) string {
|
||||
if n == 0 {
|
||||
return "none"
|
||||
}
|
||||
return fmt.Sprintf("~%d estimated (plugin agents not counted; local hits may need fewer)", n)
|
||||
}
|
||||
|
||||
func externalLookupLine(n int64) string {
|
||||
return fmt.Sprintf("External lookups: %s.", externalEstimate(n))
|
||||
}
|
||||
|
||||
// imageAgentCount counts only the built-in image agents, for the same reason.
|
||||
func imageAgentCount(ds model.DataStore) artwork.ImageAgentCount {
|
||||
ag := agents.GetAgents(ds, getPluginManager())
|
||||
return artwork.ImageAgentCount{Artist: len(ag.ArtistImageAgents()), Album: len(ag.AlbumImageAgents())}
|
||||
}
|
||||
|
||||
func promptConfirm(in io.Reader) confirmFunc {
|
||||
return func(out io.Writer, total, external int64) bool {
|
||||
var cost string
|
||||
if external > 0 {
|
||||
cost = fmt.Sprintf(" %s", externalLookupLine(external))
|
||||
}
|
||||
fmt.Fprintf(out, "\nThis will re-resolve %d items.%s Continue? [y/N] ", total, cost)
|
||||
var answer string
|
||||
if _, err := fmt.Fscanln(in, &answer); err != nil {
|
||||
return false
|
||||
}
|
||||
answer = strings.ToLower(strings.TrimSpace(answer))
|
||||
return answer == "y" || answer == "yes"
|
||||
}
|
||||
}
|
||||
|
||||
// validateSources rejects a typo'd source: matching nothing silently reads as "nothing to do" when
|
||||
// it means the filter was wrong. Checked table-wide, so a filter is never a typo for one --kind only.
|
||||
func validateSources(q model.ArtworkQueueRepository, sources []string) error {
|
||||
if len(sources) == 0 {
|
||||
return nil
|
||||
}
|
||||
var inUse []string
|
||||
for _, k := range artwork.RecheckKinds {
|
||||
found, err := q.SourcesInUse(k)
|
||||
if err != nil {
|
||||
return fmt.Errorf("listing the sources in use by %s artwork: %w", k, err)
|
||||
}
|
||||
inUse = slice.Unique(append(inUse, found...))
|
||||
}
|
||||
var unknown []string
|
||||
for _, s := range sources {
|
||||
if s != "" && !slices.Contains(inUse, s) { // the reserved absent source is valid even when nothing is absent
|
||||
unknown = append(unknown, displaySource(s))
|
||||
}
|
||||
}
|
||||
if len(unknown) == 0 {
|
||||
return nil
|
||||
}
|
||||
valid := slice.Map(inUse, displaySource)
|
||||
slices.Sort(valid)
|
||||
return fmt.Errorf("no artwork resolves from %s; sources in use: %s",
|
||||
strings.Join(unknown, ", "), cmp.Or(strings.Join(valid, ", "), "(none)"))
|
||||
}
|
||||
|
||||
// reprocessArtwork previews from CountBySource — rows matched — then reports what EnqueueBySource
|
||||
// actually inserted; the two differ because an already-queued row is left untouched.
|
||||
func reprocessArtwork(ctx context.Context, ds model.DataStore, kinds []model.Kind, sources []string,
|
||||
imageAgents artwork.ImageAgentCount, dryRun bool, confirm confirmFunc, out io.Writer) error {
|
||||
q := ds.ArtworkQueue(ctx)
|
||||
if err := validateSources(q, sources); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
matched := make([]int64, len(kinds))
|
||||
var total, external int64
|
||||
for i, k := range kinds {
|
||||
n, err := q.CountBySource(k, sources)
|
||||
if err != nil {
|
||||
return fmt.Errorf("counting %s artwork: %w", k, err)
|
||||
}
|
||||
matched[i] = n
|
||||
total += n
|
||||
external += n * artwork.ExternalLookupsPerItem(k, imageAgents)
|
||||
}
|
||||
printReprocessPreview(out, kinds, matched, total, external, sources)
|
||||
|
||||
switch {
|
||||
case dryRun:
|
||||
fmt.Fprintln(out, "\nDry run: nothing was queued.")
|
||||
return nil
|
||||
case total == 0:
|
||||
fmt.Fprintln(out, "Nothing was queued.")
|
||||
return nil
|
||||
case !confirm(out, total, external):
|
||||
fmt.Fprintln(out, "Aborted: nothing was queued.")
|
||||
return nil
|
||||
}
|
||||
|
||||
var queued int64
|
||||
for i, k := range kinds {
|
||||
if matched[i] == 0 {
|
||||
continue
|
||||
}
|
||||
n, err := q.EnqueueBySource(k, sources, model.ArtworkPriorityRecheck)
|
||||
if err != nil {
|
||||
return fmt.Errorf("queueing %s artwork: %w", k, err)
|
||||
}
|
||||
queued += n
|
||||
fmt.Fprintf(out, "%s: %d queued\n", k, n)
|
||||
}
|
||||
fmt.Fprintf(out, "Queued %d of %d matched items.\n", queued, total)
|
||||
if skipped := total - queued; skipped > 0 {
|
||||
fmt.Fprintf(out, "Already queued, left unchanged: %d (priority and retry backoff untouched).\n", skipped)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// printReprocessPreview also states the external estimate, which --dry-run must show because it
|
||||
// skips the prompt that would otherwise carry it.
|
||||
func printReprocessPreview(out io.Writer, kinds []model.Kind, matched []int64, total, external int64, sources []string) {
|
||||
w := newTabWriter(out)
|
||||
shown := slice.Map(sources, displaySource)
|
||||
fmt.Fprintf(w, "Sources:\t%s\n\n", cmp.Or(strings.Join(shown, ", "), "(any)"))
|
||||
fmt.Fprintln(w, "KIND\tMATCHED")
|
||||
for i, k := range kinds {
|
||||
fmt.Fprintf(w, "%s\t%d\n", k, matched[i])
|
||||
}
|
||||
fmt.Fprintf(w, "TOTAL\t%d\n", total)
|
||||
w.Flush()
|
||||
|
||||
fmt.Fprintf(out, "\n%s\n", externalLookupLine(external))
|
||||
if total == 0 {
|
||||
fmt.Fprintln(out, "\nNothing matches this selection.")
|
||||
}
|
||||
}
|
||||
|
||||
func runRefresh(ctx context.Context, kind model.Kind, ids []string) {
|
||||
defer db.Init(ctx)()
|
||||
ds, ctx := getAdminContext(ctx)
|
||||
|
||||
if failed := refreshItems(ctx, ds, kind, ids, os.Stdout); failed > 0 {
|
||||
log.Fatal(ctx, "Failed to refresh artwork", "kind", kind, "failed", failed, "total", len(ids))
|
||||
}
|
||||
}
|
||||
|
||||
// refreshItems keeps going after a failure — the ids are independent — and returns how many failed.
|
||||
func refreshItems(ctx context.Context, ds model.DataStore, kind model.Kind, ids []string, out io.Writer) int {
|
||||
var failed int
|
||||
for _, id := range ids {
|
||||
// artwork.Refresh would happily queue an id that does not exist, orphaning a queue row.
|
||||
if _, err := artworkItemName(ctx, ds, kind, id); err != nil {
|
||||
log.Error(ctx, "Item not found", "kind", kind, "id", id, err)
|
||||
failed++
|
||||
continue
|
||||
}
|
||||
if err := artwork.Refresh(ctx, ds, kind, id); err != nil {
|
||||
log.Error(ctx, "Error refreshing artwork", "kind", kind, "id", id, err)
|
||||
failed++
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(out, "%s/%s: queued\n", kind.Prefix(), id)
|
||||
}
|
||||
return failed
|
||||
}
|
||||
|
||||
// explainKinds is every kind explain accepts: it reports stored state and config too, so a kind
|
||||
// with no chain to walk still has something to answer with.
|
||||
var explainKinds = []model.Kind{
|
||||
model.KindArtistArtwork, model.KindAlbumArtwork, model.KindDiscArtwork,
|
||||
model.KindMediaFileArtwork, model.KindPlaylistArtwork, model.KindRadioArtwork,
|
||||
}
|
||||
|
||||
func kindPrefixes(kinds []model.Kind) string {
|
||||
return strings.Join(slice.Map(kinds, func(k model.Kind) string { return k.Prefix() }), ", ")
|
||||
}
|
||||
|
||||
func parseArtworkKind(s string, valid []model.Kind) (model.Kind, error) {
|
||||
kind, ok := model.ParseKind(s)
|
||||
if ok && slices.Contains(valid, kind) {
|
||||
return kind, nil
|
||||
}
|
||||
return kind, fmt.Errorf("invalid kind %q, expected one of: %s", s, kindPrefixes(valid))
|
||||
}
|
||||
|
||||
// explainAgents accounts for every configured agent: one the CLI cannot construct (a plugin, or a
|
||||
// built-in missing its credentials) never reaches the Chain, so the raw list alone overstates it.
|
||||
func explainAgents(configured string, available []string) string {
|
||||
if strings.TrimSpace(configured) == "" {
|
||||
return "(none)"
|
||||
}
|
||||
var unavailable bool
|
||||
names := slice.Map(strings.Split(configured, ","), func(name string) string {
|
||||
name = strings.TrimSpace(name)
|
||||
if slices.Contains(available, name) {
|
||||
return name
|
||||
}
|
||||
unavailable = true
|
||||
return name + "*"
|
||||
})
|
||||
line := strings.Join(names, ", ")
|
||||
if unavailable {
|
||||
line += " (* not available to the CLI)"
|
||||
}
|
||||
return line
|
||||
}
|
||||
|
||||
// availableImageAgents names the agents that can actually supply an image for kind.
|
||||
func availableImageAgents(ds model.DataStore, kind model.Kind) []string {
|
||||
ag := agents.GetAgents(ds, getPluginManager())
|
||||
if kind == model.KindArtistArtwork {
|
||||
return slice.Map(ag.ArtistImageAgents(), func(a agents.ArtistImageAgent) string { return a.Name })
|
||||
}
|
||||
return slice.Map(ag.AlbumImageAgents(), func(a agents.AlbumImageAgent) string { return a.Name })
|
||||
}
|
||||
|
||||
// explainResult states the verdict of the walk. A skipped or failed external tier, or a local
|
||||
// candidate that would not open, leaves the outcome unknown: nothing observed that there is no artwork.
|
||||
func explainResult(source string, steps []artwork.TraceStep) string {
|
||||
if source != "" {
|
||||
for _, s := range steps {
|
||||
if s.Outcome == artwork.OutcomeHit {
|
||||
break
|
||||
}
|
||||
if s.Outcome == artwork.OutcomeWouldTry {
|
||||
return "resolved from " + source +
|
||||
" (offline: a higher-priority external candidate was not tried; re-run with --live)"
|
||||
}
|
||||
// An external winner discards the earlier error, so the resolver settles it with no retry.
|
||||
if s.Outcome == artwork.OutcomeError && strings.HasPrefix(s.Candidate, artwork.ExternalPrefix) &&
|
||||
!strings.HasPrefix(source, artwork.ExternalPrefix) {
|
||||
return "resolved from " + source +
|
||||
" (indeterminate: a higher-priority external lookup failed; this may resolve differently on a retry)"
|
||||
}
|
||||
}
|
||||
return "resolved from " + source
|
||||
}
|
||||
for _, s := range steps {
|
||||
switch {
|
||||
case s.Outcome == artwork.OutcomeWouldTry:
|
||||
return "indeterminate (external agents not called; re-run with --live)"
|
||||
case s.Outcome == artwork.OutcomeError && strings.HasPrefix(s.Candidate, artwork.ExternalPrefix):
|
||||
return "indeterminate (an external lookup failed; the item may resolve on a later attempt)"
|
||||
// The worker treats an unreadable local candidate exactly as it treats a failed external one:
|
||||
// it retries instead of settling absent, so the verdict must not read as a clean miss.
|
||||
case s.Outcome == artwork.OutcomeUnreadable:
|
||||
return "indeterminate (a candidate exists but could not be read; the worker retries rather than settling absent)"
|
||||
}
|
||||
}
|
||||
return "not resolved"
|
||||
}
|
||||
|
||||
// explainConfig names the setting that decides where a kind's artwork comes from, and its value.
|
||||
func explainConfig(kind model.Kind) (name, value string) {
|
||||
switch kind {
|
||||
case model.KindArtistArtwork:
|
||||
return "ArtistArtPriority", conf.Server.ArtistArtPriority
|
||||
case model.KindAlbumArtwork:
|
||||
return "CoverArtPriority", conf.Server.CoverArtPriority
|
||||
case model.KindDiscArtwork:
|
||||
return "DiscArtPriority", conf.Server.DiscArtPriority
|
||||
case model.KindMediaFileArtwork:
|
||||
return "EnableMediaFileCoverArt", strconv.FormatBool(conf.Server.EnableMediaFileCoverArt)
|
||||
}
|
||||
return "", ""
|
||||
}
|
||||
|
||||
type explainReport struct {
|
||||
kind model.Kind
|
||||
id string
|
||||
name string
|
||||
stored *model.ItemArtwork
|
||||
queued *model.ArtworkQueueItem
|
||||
agents string
|
||||
steps []artwork.TraceStep
|
||||
source string
|
||||
resolveErr error
|
||||
}
|
||||
|
||||
func formatExplain(rep explainReport) string {
|
||||
var sb strings.Builder
|
||||
w := newTabWriter(&sb)
|
||||
explainable := artwork.Explainable(rep.kind)
|
||||
stateful := artwork.KeepsState(rep.kind)
|
||||
|
||||
fmt.Fprintln(w, "Item")
|
||||
fmt.Fprintf(w, " Kind:\t%s (%s)\n", rep.kind, rep.kind.Prefix())
|
||||
fmt.Fprintf(w, " ID:\t%s\n", rep.id)
|
||||
fmt.Fprintf(w, " Name:\t%s\n", rep.name)
|
||||
|
||||
fmt.Fprintln(w, "\nStored")
|
||||
switch {
|
||||
case !stateful:
|
||||
fmt.Fprintf(w, " (%s artwork is resolved on every request and never recorded)\n", rep.kind)
|
||||
case rep.stored == nil:
|
||||
fmt.Fprintln(w, " (no artwork state recorded)")
|
||||
default:
|
||||
fmt.Fprintf(w, " Source:\t%s\n", displaySource(rep.stored.Source))
|
||||
fmt.Fprintf(w, " Hash:\t%s\n", cmp.Or(rep.stored.Hash, "(absent)"))
|
||||
if rep.stored.SourcePath != "" {
|
||||
fmt.Fprintf(w, " Source path:\t%s\n", rep.stored.SourcePath)
|
||||
}
|
||||
fmt.Fprintf(w, " Attempted at:\t%s\n", formatTime(rep.stored.AttemptedAt))
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "\nQueue")
|
||||
switch {
|
||||
case !stateful:
|
||||
fmt.Fprintln(w, " (never queued)")
|
||||
case rep.queued == nil:
|
||||
fmt.Fprintln(w, " (not queued)")
|
||||
default:
|
||||
fmt.Fprintf(w, " Priority:\t%s (%d)\n", priorityName(rep.queued.Priority), rep.queued.Priority)
|
||||
fmt.Fprintf(w, " Attempts:\t%d\n", rep.queued.Attempts)
|
||||
fmt.Fprintf(w, " Retry at:\t%s\n", formatTime(rep.queued.RetryAt))
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "\nConfig")
|
||||
if setting, value := explainConfig(rep.kind); setting == "" {
|
||||
fmt.Fprintln(w, " (no artwork source configuration applies)")
|
||||
} else {
|
||||
fmt.Fprintf(w, " %s:\t%s\n", setting, value)
|
||||
if rep.agents != "" {
|
||||
fmt.Fprintf(w, " Agents:\t%s\n", rep.agents)
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "\nChain")
|
||||
if !explainable {
|
||||
fmt.Fprintf(w, " (%s artwork does not walk a priority chain)\n", rep.kind)
|
||||
} else {
|
||||
fmt.Fprintln(w, " CANDIDATE\tOUTCOME\tDETAIL")
|
||||
for _, s := range rep.steps {
|
||||
// A row with an empty last cell would end tabwriter's column block, breaking alignment.
|
||||
fmt.Fprintf(w, " %s\t%s\t%s\n", s.Candidate, s.Outcome, cmp.Or(s.Detail, "-"))
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "\nResult")
|
||||
switch {
|
||||
case rep.resolveErr != nil:
|
||||
fmt.Fprintf(w, " resolution failed: %s\n", rep.resolveErr)
|
||||
case !explainable:
|
||||
fmt.Fprintln(w, " not evaluated (no chain was walked; see Stored above)")
|
||||
default:
|
||||
fmt.Fprintf(w, " %s\n", explainResult(rep.source, rep.steps))
|
||||
}
|
||||
|
||||
w.Flush()
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
func formatTime(t time.Time) string {
|
||||
if t.IsZero() {
|
||||
return "-"
|
||||
}
|
||||
return t.Format(time.RFC3339)
|
||||
}
|
||||
|
||||
func runExplain(ctx context.Context, kind model.Kind, id string) {
|
||||
defer db.Init(ctx)()
|
||||
ds, ctx := getAdminContext(ctx)
|
||||
|
||||
name, err := artworkItemName(ctx, ds, kind, id)
|
||||
if err != nil {
|
||||
log.Fatal(ctx, "Item not found", "kind", kind, "id", id, err)
|
||||
}
|
||||
rep := explainReport{kind: kind, id: id, name: name}
|
||||
if artwork.KeepsState(kind) {
|
||||
rep.stored, err = ds.Artwork(ctx).GetItemArtwork(kind, id, model.ImageTypePrimary)
|
||||
if err != nil && !errors.Is(err, model.ErrNotFound) {
|
||||
log.Fatal(ctx, "Failed to read artwork state", "kind", kind, "id", id, err)
|
||||
}
|
||||
rep.queued, err = ds.ArtworkQueue(ctx).Get(kind, id, model.ImageTypePrimary)
|
||||
if err != nil && !errors.Is(err, model.ErrNotFound) {
|
||||
log.Fatal(ctx, "Failed to read the artwork queue", "kind", kind, "id", id, err)
|
||||
}
|
||||
}
|
||||
|
||||
if artwork.Explainable(kind) {
|
||||
if kind == model.KindArtistArtwork || kind == model.KindAlbumArtwork {
|
||||
rep.agents = explainAgents(conf.Server.Agents, availableImageAgents(ds, kind))
|
||||
}
|
||||
trace := &artwork.ChainTrace{}
|
||||
rep.source, rep.resolveErr = CreateArtworkResolver(trace, explainLive).Resolve(ctx, kind, id)
|
||||
rep.steps = trace.Steps()
|
||||
}
|
||||
|
||||
fmt.Print(formatExplain(rep))
|
||||
// The steps taken before a failed walk are the diagnosis, so report them before exiting.
|
||||
if rep.resolveErr != nil {
|
||||
log.Fatal(ctx, "Failed to resolve artwork", "kind", kind, "id", id, rep.resolveErr)
|
||||
}
|
||||
}
|
||||
|
||||
// artworkItemName looks the entity up under its own kind, so a mismatched kind/id pair is
|
||||
// reported as not found instead of silently explaining another entity's artwork.
|
||||
func artworkItemName(ctx context.Context, ds model.DataStore, kind model.Kind, id string) (string, error) {
|
||||
switch kind {
|
||||
case model.KindArtistArtwork:
|
||||
ar, err := ds.Artist(ctx).Get(id)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return ar.Name, nil
|
||||
case model.KindAlbumArtwork:
|
||||
al, err := ds.Album(ctx).Get(id)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return al.Name, nil
|
||||
case model.KindPlaylistArtwork:
|
||||
pls, err := ds.Playlist(ctx).Get(id)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return pls.Name, nil
|
||||
case model.KindRadioArtwork:
|
||||
rd, err := ds.Radio(ctx).Get(id)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return rd.Name, nil
|
||||
case model.KindMediaFileArtwork:
|
||||
mf, err := ds.MediaFile(ctx).Get(id)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return mf.Title, nil
|
||||
case model.KindDiscArtwork:
|
||||
return discArtworkName(ctx, ds, id)
|
||||
}
|
||||
return "", fmt.Errorf("unsupported kind %q", kind.Prefix())
|
||||
}
|
||||
|
||||
func discArtworkName(ctx context.Context, ds model.DataStore, id string) (string, error) {
|
||||
albumID, discNumber, err := model.ParseDiscArtworkID(id)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
al, err := ds.Album(ctx).Get(albumID)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
name := fmt.Sprintf("%s (disc %d)", al.Name, discNumber)
|
||||
// The subtitle is itself a DiscArtPriority candidate, so name it where the chain can be read against it.
|
||||
if subtitle := strings.TrimSpace(al.Discs[discNumber]); subtitle != "" {
|
||||
name += ": " + subtitle
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
899
cmd/artwork_test.go
Normal file
899
cmd/artwork_test.go
Normal file
@ -0,0 +1,899 @@
|
||||
package cmd
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
"github.com/navidrome/navidrome/conf/configtest"
|
||||
"github.com/navidrome/navidrome/consts"
|
||||
"github.com/navidrome/navidrome/core/artwork"
|
||||
"github.com/navidrome/navidrome/model"
|
||||
"github.com/navidrome/navidrome/tests"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("parseArtworkKind", func() {
|
||||
It("accepts a supported kind", func() {
|
||||
k, err := parseArtworkKind("ar", artwork.RecheckKinds)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(k).To(Equal(model.KindArtistArtwork))
|
||||
})
|
||||
|
||||
It("rejects an unknown kind and lists the valid ones", func() {
|
||||
_, err := parseArtworkKind("zz", artwork.RecheckKinds)
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("ar"))
|
||||
Expect(err.Error()).To(ContainSubstring("al"))
|
||||
})
|
||||
|
||||
It("rejects a known kind the command does not accept", func() {
|
||||
_, err := parseArtworkKind("mf", artwork.RecheckKinds)
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
|
||||
DescribeTable("accepts the kinds each command supports",
|
||||
func(prefix string, valid []model.Kind) {
|
||||
_, err := parseArtworkKind(prefix, valid)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
},
|
||||
Entry("explain reads disc artwork", "dc", explainKinds),
|
||||
Entry("explain reads media file artwork", "mf", explainKinds),
|
||||
// Disc artwork has no state to clear and the worker cannot resolve it, so refresh must not
|
||||
// accept it: the queue row would be rejected on every drain.
|
||||
Entry("refresh re-queues media files", "mf", artwork.RefreshableKinds),
|
||||
)
|
||||
|
||||
It("rejects disc artwork for refresh", func() {
|
||||
_, err := parseArtworkKind("dc", artwork.RefreshableKinds)
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("explainResult", func() {
|
||||
It("reports the winning source", func() {
|
||||
steps := []artwork.TraceStep{{Candidate: "folder", Outcome: "hit", Detail: "/music/a.jpg"}}
|
||||
Expect(explainResult("folder", steps)).To(ContainSubstring("resolved from folder"))
|
||||
})
|
||||
|
||||
It("reports not resolved when every candidate was tried and missed", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "artist.*", Outcome: "miss"},
|
||||
{Candidate: "external:deezer", Outcome: "miss"},
|
||||
}
|
||||
Expect(explainResult("", steps)).To(Equal("not resolved"))
|
||||
})
|
||||
|
||||
It("reports indeterminate when a local candidate exists but could not be read", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "cover.*", Outcome: "miss"},
|
||||
{Candidate: "embedded", Outcome: "unreadable"},
|
||||
}
|
||||
Expect(explainResult("", steps)).To(ContainSubstring("indeterminate"),
|
||||
"the worker retries an unreadable candidate instead of settling absent, so this is not a clean miss")
|
||||
})
|
||||
|
||||
It("does not qualify a hit that an earlier unreadable candidate preceded", func() {
|
||||
// chainState.try stamps only the external error onto a hit and drops the local one, so the
|
||||
// worker settles this as found; warning about it would be a false alarm.
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "embedded", Outcome: "unreadable"},
|
||||
{Candidate: "cover.*", Outcome: "hit", Detail: "/music/cover.jpg"},
|
||||
}
|
||||
Expect(explainResult("folder", steps)).To(Equal("resolved from folder"))
|
||||
})
|
||||
|
||||
It("reports indeterminate when an external lookup failed transiently", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "artist.*", Outcome: "miss"},
|
||||
{Candidate: "external:deezer", Outcome: "error", Detail: "context deadline exceeded"},
|
||||
}
|
||||
Expect(explainResult("", steps)).To(ContainSubstring("indeterminate"),
|
||||
"a failed network call is not evidence that the item has no artwork")
|
||||
})
|
||||
|
||||
It("qualifies a win a skipped higher-priority external candidate could have taken", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "external:deezer", Outcome: "would-try"},
|
||||
{Candidate: "artist.*", Outcome: "hit", Detail: "/music/artist.jpg"},
|
||||
}
|
||||
res := explainResult("artist.*", steps)
|
||||
Expect(res).To(ContainSubstring("resolved from artist.*"))
|
||||
Expect(res).To(ContainSubstring("--live"),
|
||||
"offline, the winner is only the winner because the external tier was skipped")
|
||||
})
|
||||
|
||||
It("does not qualify a win that no skipped candidate outranked", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "artist.*", Outcome: "hit"},
|
||||
{Candidate: "external:deezer", Outcome: "would-try"},
|
||||
}
|
||||
Expect(explainResult("artist.*", steps)).To(Equal("resolved from artist.*"))
|
||||
})
|
||||
|
||||
It("reports indeterminate when external agents were never called", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "artist.*", Outcome: "miss"},
|
||||
{Candidate: "external:deezer", Outcome: "would-try"},
|
||||
}
|
||||
Expect(explainResult("", steps)).To(ContainSubstring("indeterminate"),
|
||||
"an offline run must not claim an item is unresolvable when external agents were skipped")
|
||||
})
|
||||
|
||||
It("qualifies a win a failed higher-priority external lookup could have taken", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "external:deezer", Outcome: "error", Detail: "context deadline exceeded"},
|
||||
{Candidate: "artist.*", Outcome: "hit", Detail: "/music/artist.jpg"},
|
||||
}
|
||||
res := explainResult("artist.*", steps)
|
||||
Expect(res).To(ContainSubstring("resolved from artist.*"))
|
||||
Expect(res).To(ContainSubstring("indeterminate"),
|
||||
"the resolver serves this hit but retries later, so the winner is provisional")
|
||||
})
|
||||
|
||||
It("does not qualify an external win that followed a failed external lookup", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "external:deezer", Outcome: "error", Detail: "context deadline exceeded"},
|
||||
{Candidate: "external:lastfm", Outcome: "hit", Detail: "http://img"},
|
||||
}
|
||||
Expect(explainResult("external:lastfm", steps)).To(Equal("resolved from external:lastfm"),
|
||||
"a later agent supplying the image discards the earlier error, so there is no retry to warn about")
|
||||
})
|
||||
|
||||
It("does not qualify a win that outranked the failed external lookup", func() {
|
||||
steps := []artwork.TraceStep{
|
||||
{Candidate: "artist.*", Outcome: "hit"},
|
||||
{Candidate: "external:deezer", Outcome: "error", Detail: "context deadline exceeded"},
|
||||
}
|
||||
Expect(explainResult("artist.*", steps)).To(Equal("resolved from artist.*"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("explainAgents", func() {
|
||||
It("accounts for every configured agent, marking the ones the CLI could not use", func() {
|
||||
out := explainAgents("artist-nfo-metadata,apple-music,deezer,lastfm", []string{"deezer"})
|
||||
for _, name := range []string{"artist-nfo-metadata", "apple-music", "deezer", "lastfm"} {
|
||||
Expect(out).To(ContainSubstring(name),
|
||||
"a configured agent missing from this line reads as if it had never been configured")
|
||||
}
|
||||
Expect(out).To(ContainSubstring("not available to the CLI"))
|
||||
})
|
||||
|
||||
It("does not mark anything when every configured agent is available", func() {
|
||||
out := explainAgents("deezer, lastfm", []string{"lastfm", "deezer"})
|
||||
Expect(out).To(Equal("deezer, lastfm"))
|
||||
})
|
||||
|
||||
It("reports an empty configuration as none, not as an unavailable agent", func() {
|
||||
Expect(explainAgents("", nil)).To(Equal("(none)"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("formatExplain", func() {
|
||||
var rep explainReport
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.ArtistArtPriority = "external, artist.*"
|
||||
rep = explainReport{
|
||||
kind: model.KindArtistArtwork,
|
||||
id: "ar-1",
|
||||
name: "Radiohead",
|
||||
agents: "lastfm,spotify",
|
||||
steps: []artwork.TraceStep{
|
||||
{Candidate: "upload", Outcome: "skipped", Detail: "no uploaded image"},
|
||||
{Candidate: "external:deezer", Outcome: "would-try"},
|
||||
},
|
||||
source: "",
|
||||
}
|
||||
})
|
||||
|
||||
It("reports the item, its config and the chain it walked", func() {
|
||||
out := formatExplain(rep)
|
||||
Expect(out).To(ContainSubstring("Radiohead"))
|
||||
Expect(out).To(ContainSubstring("ar-1"))
|
||||
Expect(out).To(ContainSubstring("ArtistArtPriority"))
|
||||
Expect(out).To(ContainSubstring("lastfm,spotify"))
|
||||
Expect(out).To(ContainSubstring("external:deezer"))
|
||||
Expect(out).To(ContainSubstring("would-try"))
|
||||
Expect(out).To(ContainSubstring("indeterminate"))
|
||||
})
|
||||
|
||||
It("reports the absence of stored state and of a queue row", func() {
|
||||
out := formatExplain(rep)
|
||||
Expect(out).To(ContainSubstring("no artwork state recorded"))
|
||||
Expect(out).To(ContainSubstring("not queued"))
|
||||
})
|
||||
|
||||
It("prints the stored state and the queue row when they exist", func() {
|
||||
attempted := time.Date(2026, 8, 13, 10, 0, 0, 0, time.UTC)
|
||||
rep.stored = &model.ItemArtwork{Source: "folder", Hash: "abc123",
|
||||
SourcePath: "/music/cover.jpg", AttemptedAt: attempted}
|
||||
rep.queued = &model.ArtworkQueueItem{Priority: model.ArtworkPriorityScan, Attempts: 2,
|
||||
RetryAt: attempted.Add(time.Hour)}
|
||||
rep.source = "folder"
|
||||
|
||||
out := formatExplain(rep)
|
||||
Expect(out).To(ContainSubstring("abc123"))
|
||||
Expect(out).To(ContainSubstring("/music/cover.jpg"))
|
||||
Expect(out).To(ContainSubstring("2026-08-13T10:00:00Z"))
|
||||
Expect(out).To(ContainSubstring("scan (50)"), "a bare 50 makes the operator look the priority up")
|
||||
Expect(out).To(ContainSubstring("resolved from folder"))
|
||||
})
|
||||
|
||||
It("marks a known-absent stored state instead of printing an empty hash", func() {
|
||||
rep.stored = &model.ItemArtwork{AttemptedAt: time.Now()}
|
||||
Expect(formatExplain(rep)).To(ContainSubstring("absent"))
|
||||
})
|
||||
|
||||
It("reports a failed walk as failed, not as unresolved", func() {
|
||||
rep.resolveErr = errors.New("no such directory")
|
||||
|
||||
out := formatExplain(rep)
|
||||
Expect(out).To(ContainSubstring("resolution failed: no such directory"))
|
||||
Expect(out).ToNot(ContainSubstring("indeterminate"))
|
||||
Expect(out).To(ContainSubstring("would-try"), "the steps taken before the failure still print")
|
||||
})
|
||||
|
||||
It("says a kind that does not walk a chain has no chain, without an empty table", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*, embedded"
|
||||
rep.kind = model.KindPlaylistArtwork
|
||||
rep.steps = nil
|
||||
rep.agents = ""
|
||||
|
||||
out := formatExplain(rep)
|
||||
Expect(out).To(ContainSubstring("does not walk a priority chain"))
|
||||
Expect(out).ToNot(ContainSubstring("CANDIDATE"),
|
||||
"an empty chain table reads as 'nothing was tried', which is false")
|
||||
Expect(out).ToNot(ContainSubstring("not resolved"),
|
||||
"nothing was resolved because nothing was attempted")
|
||||
Expect(out).ToNot(ContainSubstring("CoverArtPriority"),
|
||||
"the priority chain config does not govern this kind")
|
||||
})
|
||||
|
||||
It("says disc artwork keeps no state instead of reporting it as unresolved state", func() {
|
||||
conf.Server.DiscArtPriority = "cover.jpg, embedded"
|
||||
rep = explainReport{
|
||||
kind: model.KindDiscArtwork, id: "al-1:2", name: "OK Computer (disc 2)",
|
||||
steps: []artwork.TraceStep{{Candidate: "cover.jpg", Outcome: "hit", Detail: "/music/cover.jpg"}},
|
||||
source: "folder",
|
||||
}
|
||||
|
||||
out := formatExplain(rep)
|
||||
Expect(out).To(ContainSubstring("never recorded"))
|
||||
Expect(out).To(ContainSubstring("never queued"))
|
||||
Expect(out).ToNot(ContainSubstring("no artwork state recorded"),
|
||||
"a missing row would read as a lookup that failed, when disc artwork has no row by design")
|
||||
Expect(out).To(ContainSubstring("DiscArtPriority"))
|
||||
Expect(out).ToNot(ContainSubstring("Agents:"), "disc artwork never asks an agent")
|
||||
Expect(out).To(ContainSubstring("resolved from folder"))
|
||||
})
|
||||
|
||||
It("reports the setting that governs media file artwork", func() {
|
||||
conf.Server.EnableMediaFileCoverArt = false
|
||||
rep = explainReport{
|
||||
kind: model.KindMediaFileArtwork, id: "mf-1", name: "Airbag",
|
||||
steps: []artwork.TraceStep{
|
||||
{Candidate: "embedded", Outcome: "skipped", Detail: "EnableMediaFileCoverArt is off"},
|
||||
},
|
||||
}
|
||||
|
||||
out := formatExplain(rep)
|
||||
Expect(out).To(ContainSubstring("EnableMediaFileCoverArt"))
|
||||
Expect(out).To(ContainSubstring("false"))
|
||||
Expect(out).To(ContainSubstring("not resolved"))
|
||||
Expect(out).To(ContainSubstring("no artwork state recorded"), "media files do keep state")
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("explainConfig", func() {
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.DiscArtPriority = "cover.jpg"
|
||||
conf.Server.EnableMediaFileCoverArt = true
|
||||
})
|
||||
|
||||
DescribeTable("names the setting that decides where a kind's artwork comes from",
|
||||
func(kind model.Kind, setting, value string) {
|
||||
gotSetting, gotValue := explainConfig(kind)
|
||||
Expect(gotSetting).To(Equal(setting))
|
||||
Expect(gotValue).To(Equal(value))
|
||||
},
|
||||
Entry("disc", model.KindDiscArtwork, "DiscArtPriority", "cover.jpg"),
|
||||
Entry("media file", model.KindMediaFileArtwork, "EnableMediaFileCoverArt", "true"),
|
||||
Entry("playlist has none", model.KindPlaylistArtwork, "", ""),
|
||||
)
|
||||
})
|
||||
|
||||
var _ = Describe("discArtworkName", func() {
|
||||
var ds *tests.MockDataStore
|
||||
|
||||
BeforeEach(func() {
|
||||
albumRepo := tests.CreateMockAlbumRepo()
|
||||
albumRepo.SetData(model.Albums{{ID: "al-1", Name: "Sandinista!", Discs: model.Discs{2: "Side Three"}}})
|
||||
ds = &tests.MockDataStore{MockedAlbum: albumRepo}
|
||||
})
|
||||
|
||||
It("names the album, the disc and its subtitle", func() {
|
||||
name, err := artworkItemName(context.Background(), ds, model.KindDiscArtwork, "al-1:2")
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(name).To(Equal("Sandinista! (disc 2): Side Three"))
|
||||
})
|
||||
|
||||
It("omits the subtitle when the disc has none", func() {
|
||||
name, err := artworkItemName(context.Background(), ds, model.KindDiscArtwork, "al-1:1")
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(name).To(Equal("Sandinista! (disc 1)"))
|
||||
})
|
||||
|
||||
It("rejects an id that is not <albumID>:<disc>", func() {
|
||||
_, err := artworkItemName(context.Background(), ds, model.KindDiscArtwork, "al-1")
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("artwork refresh command", func() {
|
||||
It("requires at least a kind and one id", func() {
|
||||
Expect(artworkRefreshCmd.Args(artworkRefreshCmd, []string{"ar"})).To(HaveOccurred())
|
||||
Expect(artworkRefreshCmd.Args(artworkRefreshCmd, []string{"ar", "id1"})).ToNot(HaveOccurred())
|
||||
Expect(artworkRefreshCmd.Args(artworkRefreshCmd, []string{"ar", "id1", "id2"})).ToNot(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("artwork reprocess selection", func() {
|
||||
It("errors when no selector is given", func() {
|
||||
_, err := selectedKinds(nil, nil, false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("--all"))
|
||||
})
|
||||
|
||||
It("returns every kind for --all", func() {
|
||||
ks, err := selectedKinds(nil, nil, true)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(ks).To(ConsistOf(artwork.RecheckKinds))
|
||||
})
|
||||
|
||||
It("returns every kind for a source filter given without a kind", func() {
|
||||
ks, err := selectedKinds(nil, []string{"folder"}, false)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(ks).To(ConsistOf(artwork.RecheckKinds), "--source alone is already a complete selection")
|
||||
})
|
||||
|
||||
It("returns only the named kinds", func() {
|
||||
ks, err := selectedKinds([]string{"ar"}, nil, false)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(ks).To(Equal([]model.Kind{model.KindArtistArtwork}))
|
||||
})
|
||||
|
||||
It("keeps a named kind selection alongside a source filter", func() {
|
||||
ks, err := selectedKinds([]string{"ar"}, []string{"folder"}, false)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(ks).To(Equal([]model.Kind{model.KindArtistArtwork}))
|
||||
})
|
||||
|
||||
It("rejects an unknown kind", func() {
|
||||
_, err := selectedKinds([]string{"zz"}, nil, false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
|
||||
It("counts a repeated kind once", func() {
|
||||
ks, err := selectedKinds([]string{"ar", "ar"}, nil, false)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(ks).To(Equal([]model.Kind{model.KindArtistArtwork}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("explain/reprocess source round trip", func() {
|
||||
ctx := context.Background()
|
||||
|
||||
// storedSource reads back the Source line explain printed, as an operator would copy it.
|
||||
storedSource := func(out string) string {
|
||||
GinkgoHelper()
|
||||
for line := range strings.SplitSeq(out, "\n") {
|
||||
if after, ok := strings.CutPrefix(strings.TrimSpace(line), "Source:"); ok {
|
||||
return strings.TrimSpace(after)
|
||||
}
|
||||
}
|
||||
Fail("explain printed no Source line")
|
||||
return ""
|
||||
}
|
||||
|
||||
It("names the absent state as reprocess --source accepts it", func() {
|
||||
ds := &tests.MockDataStore{}
|
||||
art := ds.Artwork(ctx).(*tests.MockArtworkRepo)
|
||||
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: model.KindArtistArtwork.Prefix(),
|
||||
ItemID: "ar-1", ImageType: model.ImageTypePrimary})).To(Succeed())
|
||||
|
||||
shown := storedSource(formatExplain(explainReport{kind: model.KindArtistArtwork, id: "ar-1",
|
||||
stored: &model.ItemArtwork{AttemptedAt: time.Now()}}))
|
||||
|
||||
q := ds.ArtworkQueue(ctx)
|
||||
Expect(validateSources(q, repositorySources([]string{shown}))).To(Succeed(),
|
||||
"explain's spelling of a source must be pasteable into --source")
|
||||
Expect(validateSources(q, repositorySources([]string{"(" + shown + ")"}))).ToNot(Succeed(),
|
||||
"a parenthesised name would be rejected, so explain must not print one")
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("repositorySources", func() {
|
||||
It("maps the user-facing absent name onto the stored empty source", func() {
|
||||
Expect(repositorySources([]string{"absent", "folder"})).To(Equal([]string{"", "folder"}))
|
||||
})
|
||||
|
||||
It("keeps an empty selection empty, meaning every source", func() {
|
||||
Expect(repositorySources(nil)).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("promptConfirm", func() {
|
||||
var out strings.Builder
|
||||
|
||||
BeforeEach(func() { out.Reset() })
|
||||
|
||||
It("states the external cost and accepts an explicit yes", func() {
|
||||
Expect(promptConfirm(strings.NewReader("y\n"))(&out, 42, 7)).To(BeTrue())
|
||||
Expect(out.String()).To(ContainSubstring("re-resolve 42 items"))
|
||||
Expect(out.String()).To(ContainSubstring("External lookups: ~7 estimated"))
|
||||
})
|
||||
|
||||
It("defaults to no on anything else", func() {
|
||||
Expect(promptConfirm(strings.NewReader("\n"))(&out, 1, 1)).To(BeFalse())
|
||||
Expect(promptConfirm(strings.NewReader("nope\n"))(&out, 1, 1)).To(BeFalse())
|
||||
Expect(promptConfirm(strings.NewReader(""))(&out, 1, 1)).To(BeFalse())
|
||||
})
|
||||
|
||||
It("drops the external clause when no lookup will be made", func() {
|
||||
Expect(promptConfirm(strings.NewReader("y\n"))(&out, 3, 0)).To(BeTrue())
|
||||
Expect(out.String()).To(ContainSubstring("re-resolve 3 items."))
|
||||
Expect(out.String()).ToNot(ContainSubstring("External lookups"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("reprocessConfirm", func() {
|
||||
var out strings.Builder
|
||||
|
||||
BeforeEach(func() { out.Reset() })
|
||||
|
||||
It("prompts when --yes was not given", func() {
|
||||
Expect(reprocessConfirm(false, strings.NewReader("n\n"))(&out, 5, 5)).To(BeFalse())
|
||||
Expect(out.String()).To(ContainSubstring("Continue?"))
|
||||
})
|
||||
|
||||
It("bypasses the prompt only for --yes", func() {
|
||||
Expect(reprocessConfirm(true, strings.NewReader(""))(&out, 5, 5)).To(BeTrue())
|
||||
Expect(out.String()).To(BeEmpty(), "--yes must not print a prompt it never reads")
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("reprocessArtwork", func() {
|
||||
var ds *tests.MockDataStore
|
||||
var art *tests.MockArtworkRepo
|
||||
var queue *tests.MockArtworkQueueRepo
|
||||
var out strings.Builder
|
||||
var imageAgents artwork.ImageAgentCount
|
||||
ctx := context.Background()
|
||||
kinds := []model.Kind{model.KindArtistArtwork, model.KindAlbumArtwork}
|
||||
accept := func(io.Writer, int64, int64) bool { return true }
|
||||
decline := func(io.Writer, int64, int64) bool { return false }
|
||||
|
||||
put := func(kind model.Kind, id, source string) {
|
||||
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: kind.Prefix(), ItemID: id,
|
||||
ImageType: model.ImageTypePrimary, Hash: "h" + id, Source: source})).To(Succeed())
|
||||
}
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.CoverArtPriority = "cover.*, external"
|
||||
conf.Server.ArtistArtPriority = "artist.*, external"
|
||||
conf.Server.EnableM3UExternalAlbumArt = false
|
||||
imageAgents = artwork.ImageAgentCount{Artist: 1, Album: 1}
|
||||
ds = &tests.MockDataStore{}
|
||||
art = ds.Artwork(ctx).(*tests.MockArtworkRepo)
|
||||
queue = ds.ArtworkQueue(ctx).(*tests.MockArtworkQueueRepo)
|
||||
out.Reset()
|
||||
put(model.KindArtistArtwork, "ar-1", "external:deezer")
|
||||
put(model.KindArtistArtwork, "ar-2", "")
|
||||
put(model.KindAlbumArtwork, "al-1", "external:deezer")
|
||||
put(model.KindAlbumArtwork, "al-2", "folder")
|
||||
})
|
||||
|
||||
It("previews the per-kind breakdown and queues nothing on a dry run", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, []string{"external:deezer"}, imageAgents, true, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("external:deezer"))
|
||||
Expect(out.String()).To(ContainSubstring("artist"))
|
||||
Expect(out.String()).To(ContainSubstring("album"))
|
||||
Expect(out.String()).To(ContainSubstring("TOTAL"))
|
||||
Expect(out.String()).To(ContainSubstring("Dry run"))
|
||||
Expect(queue.Count()).To(BeZero())
|
||||
})
|
||||
|
||||
It("queues nothing when the operator declines", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, nil, imageAgents, false, decline, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("Aborted"))
|
||||
Expect(queue.Count()).To(BeZero())
|
||||
})
|
||||
|
||||
It("queues the matching items at recheck priority, leaving their artwork state alone", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, []string{"external:deezer"}, imageAgents, false, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(queue.Count()).To(Equal(int64(2)))
|
||||
queued, err := queue.Get(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(queued.Priority).To(Equal(model.ArtworkPriorityRecheck))
|
||||
_, err = queue.Get(model.KindAlbumArtwork, "al-2", model.ImageTypePrimary)
|
||||
Expect(err).To(MatchError(model.ErrNotFound), "a non-matching source must not be queued")
|
||||
|
||||
stored, err := art.GetItemArtwork(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(stored.Hash).To(Equal("hal-1"), "bulk reprocessing must not blank the current artwork")
|
||||
})
|
||||
|
||||
It("targets the absent state", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, []string{""}, imageAgents, false, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(queue.Count()).To(Equal(int64(1)))
|
||||
_, err := queue.Get(model.KindArtistArtwork, "ar-2", model.ImageTypePrimary)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
})
|
||||
|
||||
It("reports matched and queued separately when part of the set is already queued", func() {
|
||||
Expect(queue.Enqueue(model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-1",
|
||||
ImageType: model.ImageTypePrimary, Priority: model.ArtworkPriorityBump})).To(Succeed())
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, []string{"external:deezer"}, imageAgents, false, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("Queued 1 of 2 matched items"))
|
||||
Expect(out.String()).To(ContainSubstring("Already queued, left unchanged: 1"))
|
||||
queued, err := queue.Get(model.KindArtistArtwork, "ar-1", model.ImageTypePrimary)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(queued.Priority).To(Equal(model.ArtworkPriorityBump),
|
||||
"an already-queued row keeps its priority and backoff")
|
||||
})
|
||||
|
||||
It("stops at a selection that matches nothing instead of prompting", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindRadioArtwork}, nil, imageAgents, false,
|
||||
func(io.Writer, int64, int64) bool {
|
||||
Fail("must not prompt when there is nothing to queue")
|
||||
return true
|
||||
}, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("Nothing"))
|
||||
Expect(queue.Count()).To(BeZero())
|
||||
})
|
||||
|
||||
It("reports an empty selection as a dry run when one was asked for", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindRadioArtwork}, nil, imageAgents, true, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("Nothing matches"))
|
||||
Expect(out.String()).To(ContainSubstring("Dry run"))
|
||||
})
|
||||
|
||||
It("shows the external estimate on a dry run, which never reaches the prompt", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindAlbumArtwork}, nil, imageAgents, true, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("External lookups: ~2 estimated"))
|
||||
})
|
||||
|
||||
It("bills every agent per item, not one lookup per item", func() {
|
||||
imageAgents = artwork.ImageAgentCount{Artist: 2, Album: 3}
|
||||
var external int64
|
||||
capture := func(_ io.Writer, _, e int64) bool { external = e; return false }
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, nil, imageAgents, false, capture, &out)).To(Succeed())
|
||||
|
||||
Expect(external).To(Equal(int64(2*2+2*3)), "2 artists at 2 agents plus 2 albums at 3 agents")
|
||||
Expect(out.String()).To(ContainSubstring("External lookups: ~10 estimated"))
|
||||
})
|
||||
|
||||
It("names the estimate's blind spots instead of claiming a bound it cannot hold", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, nil, imageAgents, true, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("plugin agents not counted"))
|
||||
Expect(out.String()).To(ContainSubstring("local hits may need fewer"))
|
||||
Expect(out.String()).ToNot(ContainSubstring("up to"), "plugin agents make any ceiling false")
|
||||
Expect(out.String()).ToNot(ContainSubstring("at least"), "a local hit makes any floor false")
|
||||
})
|
||||
|
||||
It("says so when the selection needs no external lookup", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*"
|
||||
put(model.KindPlaylistArtwork, "pl-1", "playlist")
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindPlaylistArtwork}, nil, imageAgents, true, accept, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("External lookups: none"))
|
||||
})
|
||||
|
||||
It("counts playlists as external cost when the m3u image fetch is enabled", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*"
|
||||
conf.Server.EnableM3UExternalAlbumArt = true
|
||||
put(model.KindPlaylistArtwork, "pl-1", "playlist")
|
||||
var external int64
|
||||
capture := func(_ io.Writer, _, e int64) bool { external = e; return false }
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindPlaylistArtwork}, nil, imageAgents, false, capture, &out)).To(Succeed())
|
||||
|
||||
Expect(external).To(Equal(int64(1)))
|
||||
Expect(out.String()).To(ContainSubstring("External lookups: ~1 estimated"))
|
||||
})
|
||||
|
||||
It("bills a playlist for every album its grid samples, at every agent", func() {
|
||||
imageAgents = artwork.ImageAgentCount{Album: 3}
|
||||
put(model.KindPlaylistArtwork, "pl-1", "playlist")
|
||||
var external int64
|
||||
capture := func(_ io.Writer, _, e int64) bool { external = e; return false }
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindPlaylistArtwork}, nil, imageAgents, false, capture, &out)).To(Succeed())
|
||||
|
||||
Expect(external).To(Equal(int64(artwork.PlaylistGridSamples*3)),
|
||||
"one playlist samples 4 albums, each walking all 3 album agents")
|
||||
})
|
||||
|
||||
It("counts only the kinds that call an external agent as external cost", func() {
|
||||
put(model.KindRadioArtwork, "ra-1", "upload")
|
||||
var total, external int64
|
||||
capture := func(_ io.Writer, t, e int64) bool { total, external = t, e; return false }
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindAlbumArtwork, model.KindRadioArtwork},
|
||||
nil, imageAgents, false, capture, &out)).To(Succeed())
|
||||
|
||||
Expect(total).To(Equal(int64(3)))
|
||||
Expect(external).To(Equal(int64(2)), "radio artwork never reaches an external agent")
|
||||
})
|
||||
|
||||
It("rejects an unknown source and names the ones in use", func() {
|
||||
err := reprocessArtwork(ctx, ds, kinds, []string{"externa:deezer"}, imageAgents, true, accept, &out)
|
||||
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("externa:deezer"))
|
||||
Expect(err.Error()).To(ContainSubstring("external:deezer"))
|
||||
Expect(err.Error()).To(ContainSubstring("folder"))
|
||||
Expect(err.Error()).To(ContainSubstring("absent"), "the empty source prints under its user-facing name")
|
||||
Expect(queue.Count()).To(BeZero())
|
||||
})
|
||||
|
||||
It("accepts the absent filter with nothing absent, still rejecting a typo", func() {
|
||||
put(model.KindArtistArtwork, "ar-2", "folder")
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, repositorySources([]string{absentSource}),
|
||||
imageAgents, false, accept, &out)).To(Succeed(),
|
||||
"a reserved source must stay valid once the library has none of it")
|
||||
Expect(out.String()).To(ContainSubstring("Nothing matches"))
|
||||
Expect(queue.Count()).To(BeZero())
|
||||
|
||||
Expect(reprocessArtwork(ctx, ds, kinds, repositorySources([]string{"absnt"}),
|
||||
imageAgents, true, accept, &out)).ToNot(Succeed(), "a typo must still be rejected")
|
||||
})
|
||||
|
||||
It("accepts a source another kind uses, letting the empty selection report itself", func() {
|
||||
Expect(reprocessArtwork(ctx, ds, []model.Kind{model.KindArtistArtwork}, []string{"folder"},
|
||||
imageAgents, false, decline, &out)).To(Succeed())
|
||||
|
||||
Expect(out.String()).To(ContainSubstring("Nothing matches"),
|
||||
"a well-formed filter must not be reported as a typo because of the kinds selected")
|
||||
Expect(queue.Count()).To(BeZero())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("artwork status command", func() {
|
||||
It("takes no arguments", func() {
|
||||
Expect(artworkStatusCmd.Args(artworkStatusCmd, []string{})).ToNot(HaveOccurred())
|
||||
Expect(artworkStatusCmd.Args(artworkStatusCmd, []string{"x"})).To(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("collectStatus", func() {
|
||||
var ds *tests.MockDataStore
|
||||
var art *tests.MockArtworkRepo
|
||||
var queue *tests.MockArtworkQueueRepo
|
||||
ctx := context.Background()
|
||||
|
||||
BeforeEach(func() {
|
||||
ds = &tests.MockDataStore{}
|
||||
art = ds.Artwork(ctx).(*tests.MockArtworkRepo)
|
||||
queue = ds.ArtworkQueue(ctx).(*tests.MockArtworkQueueRepo)
|
||||
put := func(kind model.Kind, id, source, hash string, attempted time.Time) {
|
||||
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: kind.Prefix(), ItemID: id,
|
||||
ImageType: model.ImageTypePrimary, Source: source, Hash: hash, AttemptedAt: attempted})).To(Succeed())
|
||||
}
|
||||
put(model.KindArtistArtwork, "ar-1", "external:deezer", "h1", time.Now())
|
||||
put(model.KindArtistArtwork, "ar-2", "", "", time.Now().Add(-48*time.Hour))
|
||||
put(model.KindArtistArtwork, "ar-3", "", "", time.Now())
|
||||
put(model.KindAlbumArtwork, "al-1", "folder", "h2", time.Now())
|
||||
Expect(queue.Enqueue(model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-9",
|
||||
ImageType: model.ImageTypePrimary, Priority: model.ArtworkPriorityBackfill})).To(Succeed())
|
||||
})
|
||||
|
||||
It("reports the queue, the source distribution and the absent ages", func() {
|
||||
rep, err := collectStatus(ctx, ds)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
|
||||
Expect(rep.queue).To(ConsistOf(model.ArtworkQueueStat{ItemKind: "ar",
|
||||
Priority: model.ArtworkPriorityBackfill, Count: 1}))
|
||||
Expect(rep.sources).To(ContainElements(
|
||||
sourceCount{kind: model.KindArtistArtwork, source: "external:deezer", count: 1},
|
||||
sourceCount{kind: model.KindArtistArtwork, source: "", count: 2},
|
||||
sourceCount{kind: model.KindAlbumArtwork, source: "folder", count: 1},
|
||||
))
|
||||
Expect(rep.absent).To(ContainElement(absentCount{kind: model.KindArtistArtwork,
|
||||
ArtworkAbsentStat: model.ArtworkAbsentStat{Total: 2, Stale: 1}}))
|
||||
})
|
||||
|
||||
It("compares the stored fingerprint against the current one", func() {
|
||||
Expect(ds.Property(ctx).Put(consts.ArtConfFingerprintPropertyKey, "old-fingerprint")).To(Succeed())
|
||||
|
||||
rep, err := collectStatus(ctx, ds)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(rep.stored).To(Equal("old-fingerprint"))
|
||||
Expect(rep.current).To(Equal(artwork.ConfigFingerprint()),
|
||||
"the CLI must report the value backfill itself compares")
|
||||
})
|
||||
|
||||
It("queues nothing", func() {
|
||||
_, err := collectStatus(ctx, ds)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(queue.Count()).To(Equal(int64(1)), "status must not enqueue anything")
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("formatStatus", func() {
|
||||
var rep statusReport
|
||||
|
||||
BeforeEach(func() {
|
||||
rep = statusReport{
|
||||
queue: []model.ArtworkQueueStat{
|
||||
{ItemKind: "ar", Priority: model.ArtworkPriorityBackfill, Count: 2},
|
||||
{ItemKind: "al", Priority: model.ArtworkPriorityScan, Count: 1},
|
||||
},
|
||||
sources: []sourceCount{
|
||||
{kind: model.KindArtistArtwork, source: "external:deezer", count: 5},
|
||||
{kind: model.KindArtistArtwork, source: "", count: 2},
|
||||
},
|
||||
absent: []absentCount{
|
||||
{kind: model.KindArtistArtwork, ArtworkAbsentStat: model.ArtworkAbsentStat{Total: 2, Stale: 1}},
|
||||
},
|
||||
inputs: []artwork.FingerprintInput{{Name: "Agents", Value: "deezer,lastfm"}},
|
||||
stored: "abc123",
|
||||
current: "abc123",
|
||||
}
|
||||
})
|
||||
|
||||
// block isolates one section, so an assertion cannot be satisfied by a coincidence elsewhere.
|
||||
block := func(out, header string) string {
|
||||
GinkgoHelper()
|
||||
_, after, found := strings.Cut(out, header+"\n")
|
||||
Expect(found).To(BeTrue(), "the %q block must be printed", header)
|
||||
body, _, _ := strings.Cut(after, "\n\n")
|
||||
return body
|
||||
}
|
||||
|
||||
It("names the kind and the priority of every queued row", func() {
|
||||
queue := block(formatStatus(rep), "Queue")
|
||||
Expect(queue).To(MatchRegexp(`artist\s+backfill\s+2`))
|
||||
Expect(queue).To(MatchRegexp(`album\s+scan\s+1`))
|
||||
})
|
||||
|
||||
It("totals the queue", func() {
|
||||
Expect(block(formatStatus(rep), "Queue")).To(MatchRegexp(`TOTAL\s+3`))
|
||||
})
|
||||
|
||||
It("counts each source, naming the empty one absent", func() {
|
||||
sources := block(formatStatus(rep), "Sources")
|
||||
Expect(sources).To(MatchRegexp(`artist\s+external:deezer\s+5`))
|
||||
Expect(sources).To(MatchRegexp(`artist\s+absent\s+2`))
|
||||
})
|
||||
|
||||
It("prints the absent total and how many are due for recheck", func() {
|
||||
absent := block(formatStatus(rep), "Absent (resolved, no image found)")
|
||||
Expect(absent).To(MatchRegexp(`artist\s+2\s+1`))
|
||||
})
|
||||
|
||||
It("states the recheck window the absent counts are bucketed against", func() {
|
||||
Expect(formatStatus(rep)).To(ContainSubstring("24h"))
|
||||
})
|
||||
|
||||
It("leads with the queued backlog, which is the finding, not with the fingerprint verdict", func() {
|
||||
out := block(formatStatus(rep), "Backfill")
|
||||
Expect(out).To(MatchRegexp(`State:\s+backfill running: 2 items queued`),
|
||||
"an operator scanning for trouble must not read 'up to date' while 2 items churn")
|
||||
Expect(out).To(ContainSubstring("fingerprint up to date"))
|
||||
})
|
||||
|
||||
It("keeps the re-enqueue warning while a backfill is already running", func() {
|
||||
rep.stored = "older"
|
||||
|
||||
out := block(formatStatus(rep), "Backfill")
|
||||
Expect(out).To(MatchRegexp(`State:\s+backfill running: 2 items queued`))
|
||||
Expect(out).To(ContainSubstring("re-enqueued"),
|
||||
"the stored fingerprint is still stale, so a second full re-enqueue is pending on top of this one")
|
||||
})
|
||||
|
||||
It("reports up to date only once the backfill has drained", func() {
|
||||
rep.queue = []model.ArtworkQueueStat{{ItemKind: "al", Priority: model.ArtworkPriorityScan, Count: 1}}
|
||||
|
||||
Expect(block(formatStatus(rep), "Backfill")).To(MatchRegexp(`State:\s+up to date`))
|
||||
})
|
||||
|
||||
It("echoes the config inputs a fingerprint change would have come from", func() {
|
||||
out := block(formatStatus(rep), "Backfill")
|
||||
Expect(out).To(MatchRegexp(`Agents:\s+deezer,lastfm`))
|
||||
Expect(out).To(ContainSubstring("abc123"), "the fingerprint values themselves must be printed")
|
||||
})
|
||||
|
||||
It("reports a changed fingerprint as a pending re-resolve of everything", func() {
|
||||
rep.stored = "older"
|
||||
rep.queue = nil
|
||||
|
||||
out := formatStatus(rep)
|
||||
Expect(out).To(ContainSubstring("fingerprint changed"))
|
||||
Expect(out).ToNot(ContainSubstring("up to date"))
|
||||
})
|
||||
|
||||
It("reports a never-recorded fingerprint without printing an empty value", func() {
|
||||
rep.stored = ""
|
||||
|
||||
out := formatStatus(rep)
|
||||
Expect(out).To(ContainSubstring("(none)"))
|
||||
Expect(out).To(ContainSubstring("fingerprint changed"))
|
||||
})
|
||||
|
||||
It("says the queue is empty instead of printing a headless table", func() {
|
||||
rep.queue = nil
|
||||
|
||||
out := formatStatus(rep)
|
||||
Expect(out).To(ContainSubstring("empty"))
|
||||
Expect(out).ToNot(ContainSubstring("PRIORITY"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("refreshItems", func() {
|
||||
var ds *tests.MockDataStore
|
||||
var queue *tests.MockArtworkQueueRepo
|
||||
var art *tests.MockArtworkRepo
|
||||
var out strings.Builder
|
||||
ctx := context.Background()
|
||||
|
||||
BeforeEach(func() {
|
||||
albums := tests.CreateMockAlbumRepo()
|
||||
albums.SetData(model.Albums{{ID: "al-1"}, {ID: "al-3"}})
|
||||
ds = &tests.MockDataStore{MockedAlbum: albums}
|
||||
art = ds.Artwork(ctx).(*tests.MockArtworkRepo)
|
||||
queue = ds.ArtworkQueue(ctx).(*tests.MockArtworkQueueRepo)
|
||||
out.Reset()
|
||||
})
|
||||
|
||||
It("clears the stored state and queues each id at Bump priority", func() {
|
||||
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: model.KindAlbumArtwork.Prefix(),
|
||||
ItemID: "al-1", ImageType: model.ImageTypePrimary, Hash: "abc123"})).To(Succeed())
|
||||
|
||||
Expect(refreshItems(ctx, ds, model.KindAlbumArtwork, []string{"al-1", "al-3"}, &out)).To(BeZero())
|
||||
|
||||
_, err := art.GetItemArtwork(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
|
||||
Expect(err).To(MatchError(model.ErrNotFound))
|
||||
queued, err := queue.Get(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(queued.Priority).To(Equal(model.ArtworkPriorityBump))
|
||||
Expect(out.String()).To(Equal("al/al-1: queued\nal/al-3: queued\n"))
|
||||
})
|
||||
|
||||
It("skips an id that does not exist instead of queuing it", func() {
|
||||
Expect(refreshItems(ctx, ds, model.KindAlbumArtwork, []string{"al-2"}, &out)).To(Equal(1))
|
||||
|
||||
_, err := queue.Get(model.KindAlbumArtwork, "al-2", model.ImageTypePrimary)
|
||||
Expect(err).To(MatchError(model.ErrNotFound), "a typo must not leave an orphan queue row")
|
||||
Expect(out.String()).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("continues past a failing id and counts the failures", func() {
|
||||
Expect(refreshItems(ctx, ds, model.KindAlbumArtwork,
|
||||
[]string{"al-1", "al-2", "al-3"}, &out)).To(Equal(1))
|
||||
|
||||
Expect(out.String()).To(Equal("al/al-1: queued\nal/al-3: queued\n"),
|
||||
"the ids after a failure are still refreshed")
|
||||
})
|
||||
})
|
||||
@ -9,7 +9,6 @@ import (
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"text/tabwriter"
|
||||
"time"
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
@ -314,7 +313,7 @@ func formatPluginList(list model.Plugins, format string) (string, error) {
|
||||
return sb.String(), w.Error()
|
||||
case "table":
|
||||
var sb strings.Builder
|
||||
w := tabwriter.NewWriter(&sb, 0, 4, 2, ' ', 0)
|
||||
w := newTabWriter(&sb)
|
||||
fmt.Fprintln(w, "ID\tNAME\tVERSION\tENABLED\tLAST ERROR")
|
||||
for _, p := range list {
|
||||
name, version := manifestSummary(p)
|
||||
|
||||
@ -4,6 +4,8 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"text/tabwriter"
|
||||
|
||||
"github.com/navidrome/navidrome/core/auth"
|
||||
"github.com/navidrome/navidrome/db"
|
||||
@ -13,6 +15,11 @@ import (
|
||||
"github.com/navidrome/navidrome/persistence"
|
||||
)
|
||||
|
||||
// newTabWriter keeps every CLI table on the same column settings.
|
||||
func newTabWriter(out io.Writer) *tabwriter.Writer {
|
||||
return tabwriter.NewWriter(out, 0, 4, 2, ' ', 0)
|
||||
}
|
||||
|
||||
func getAdminContext(ctx context.Context) (model.DataStore, context.Context) {
|
||||
sqlDB := db.Db()
|
||||
ds := persistence.New(sqlDB)
|
||||
|
||||
@ -223,6 +223,18 @@ func CreateArtworkWorker() *artwork.Worker {
|
||||
return worker
|
||||
}
|
||||
|
||||
func CreateArtworkResolver(trace *artwork.ChainTrace, live bool) *artwork.TracingResolver {
|
||||
sqlDB := db.Db()
|
||||
dataStore := persistence.New(sqlDB)
|
||||
broker := events.GetBroker()
|
||||
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
|
||||
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
|
||||
agentsAgents := agents.GetAgents(dataStore, manager)
|
||||
fFmpeg := ffmpeg.New()
|
||||
tracingResolver := artwork.NewTracingResolver(dataStore, agentsAgents, fFmpeg, trace, live)
|
||||
return tracingResolver
|
||||
}
|
||||
|
||||
func getPluginManager() *plugins.Manager {
|
||||
sqlDB := db.Db()
|
||||
dataStore := persistence.New(sqlDB)
|
||||
|
||||
@ -144,6 +144,13 @@ func CreateArtworkWorker() *artwork.Worker {
|
||||
))
|
||||
}
|
||||
|
||||
func CreateArtworkResolver(trace *artwork.ChainTrace, live bool) *artwork.TracingResolver {
|
||||
panic(wire.Build(
|
||||
allProviders,
|
||||
artwork.NewTracingResolver,
|
||||
))
|
||||
}
|
||||
|
||||
func getPluginManager() *plugins.Manager {
|
||||
panic(wire.Build(
|
||||
allProviders,
|
||||
|
||||
@ -47,10 +47,17 @@ func fetchArtistImage(ctx context.Context, ag *agents.Agents, gate gateFunc, ar
|
||||
// Synthetic artists would otherwise get an unrelated agent result assigned to them.
|
||||
switch ar.ID {
|
||||
case consts.UnknownArtistID, consts.VariousArtistsID:
|
||||
traceFrom(ctx).add(TraceStep{Candidate: externalCandidate, Outcome: OutcomeSkipped, Detail: "synthetic artist"})
|
||||
return nil, "", false
|
||||
}
|
||||
name := externalName(ar.Name)
|
||||
for _, a := range ag.ArtistImageAgents() {
|
||||
imageAgents := ag.ArtistImageAgents()
|
||||
if len(imageAgents) == 0 {
|
||||
traceFrom(ctx).add(TraceStep{Candidate: externalCandidate, Outcome: OutcomeSkipped,
|
||||
Detail: "no enabled agent provides artist images"})
|
||||
return nil, "", false
|
||||
}
|
||||
for _, a := range imageAgents {
|
||||
reader, _, err := gate(a.Name, func() (io.ReadCloser, string, error) {
|
||||
imgs, err := a.Retriever.GetArtistImages(ctx, ar.ID, name, ar.MbzArtistID)
|
||||
if err != nil {
|
||||
@ -76,7 +83,13 @@ func fetchArtistImage(ctx context.Context, ag *agents.Agents, gate gateFunc, ar
|
||||
// fetchAlbumImage is the album counterpart of fetchArtistImage.
|
||||
func fetchAlbumImage(ctx context.Context, ag *agents.Agents, gate gateFunc, al model.Album) (r io.ReadCloser, agentName string, extErr bool) {
|
||||
name, artist := externalName(al.Name), externalName(al.AlbumArtist)
|
||||
for _, a := range ag.AlbumImageAgents() {
|
||||
imageAgents := ag.AlbumImageAgents()
|
||||
if len(imageAgents) == 0 {
|
||||
traceFrom(ctx).add(TraceStep{Candidate: externalCandidate, Outcome: OutcomeSkipped,
|
||||
Detail: "no enabled agent provides album images"})
|
||||
return nil, "", false
|
||||
}
|
||||
for _, a := range imageAgents {
|
||||
reader, _, err := gate(a.Name, func() (io.ReadCloser, string, error) {
|
||||
imgs, err := a.Retriever.GetAlbumImages(ctx, name, artist, al.MbzAlbumID)
|
||||
if err != nil {
|
||||
|
||||
@ -173,6 +173,30 @@ var _ = Describe("agent images", func() {
|
||||
Expect(a.artistCalls).To(Equal(0), "synthetic artists never reach the agents")
|
||||
})
|
||||
|
||||
It("records a skipped external candidate when no agent provides artist images", func() {
|
||||
ag := imageAgents()
|
||||
t := &ChainTrace{}
|
||||
|
||||
r, _, extErr := fetchArtistImage(withTrace(ctx, t), ag, passthroughGate, model.Artist{ID: "ar1"})
|
||||
Expect(r).To(BeNil())
|
||||
Expect(extErr).To(BeFalse())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{{Candidate: "external", Outcome: OutcomeSkipped,
|
||||
Detail: "no enabled agent provides artist images"}}),
|
||||
"a configured external token must never be silently absent from the chain")
|
||||
})
|
||||
|
||||
It("records a skipped external candidate for synthetic artists", func() {
|
||||
a := &fakeImageAgent{name: "agentA", imgs: []agents.ExternalImage{img("/a", 100)}}
|
||||
ag := imageAgents(a)
|
||||
t := &ChainTrace{}
|
||||
|
||||
_, _, _ = fetchArtistImage(withTrace(ctx, t), ag, passthroughGate,
|
||||
model.Artist{ID: consts.VariousArtistsID, Name: "Various Artists"})
|
||||
Expect(t.Steps()).To(HaveLen(1))
|
||||
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeSkipped))
|
||||
Expect(t.Steps()[0].Detail).To(ContainSubstring("synthetic"))
|
||||
})
|
||||
|
||||
It("clears typographic characters from the query name unless preserving unicode", func() {
|
||||
conf.Server.DevPreserveUnicodeInExternalCalls = false
|
||||
a := &fakeImageAgent{name: "agentA"}
|
||||
@ -231,6 +255,18 @@ var _ = Describe("agent images", func() {
|
||||
Expect(a.albumCalls).To(Equal(1))
|
||||
})
|
||||
|
||||
It("records a skipped external candidate when no agent provides album images", func() {
|
||||
ag := imageAgents()
|
||||
t := &ChainTrace{}
|
||||
|
||||
r, _, extErr := fetchAlbumImage(withTrace(ctx, t), ag, passthroughGate, model.Album{Name: "Album"})
|
||||
Expect(r).To(BeNil())
|
||||
Expect(extErr).To(BeFalse())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{{Candidate: "external", Outcome: OutcomeSkipped,
|
||||
Detail: "no enabled agent provides album images"}}),
|
||||
"a configured external token must never be silently absent from the chain")
|
||||
})
|
||||
|
||||
It("reports extErr when the only agent fails transiently", func() {
|
||||
a := &fakeImageAgent{name: "agentA", err: context.DeadlineExceeded}
|
||||
ag := imageAgents(a)
|
||||
|
||||
@ -11,6 +11,7 @@ import (
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
"github.com/navidrome/navidrome/consts"
|
||||
"github.com/navidrome/navidrome/core/agents"
|
||||
"github.com/navidrome/navidrome/core/ffmpeg"
|
||||
"github.com/navidrome/navidrome/log"
|
||||
"github.com/navidrome/navidrome/model"
|
||||
@ -118,7 +119,7 @@ func (s *service) Get(ctx context.Context, artID model.ArtworkID, size int, squa
|
||||
}
|
||||
|
||||
// requestRecheckAge throttles view-triggered rechecks so reopening a genuinely-absent page can't
|
||||
// hammer external services; below staleAbsentAge to catch younger absences.
|
||||
// hammer external services; below StaleAbsentAge to catch younger absences.
|
||||
const requestRecheckAge = time.Hour
|
||||
|
||||
func (s *service) serveEntity(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error) {
|
||||
@ -317,16 +318,15 @@ func (s *service) serveDisc(ctx context.Context, artID model.ArtworkID, size int
|
||||
return nil, err
|
||||
}
|
||||
// Single-disc albums run the chain too: a disc can carry art distinct from the album cover.
|
||||
selectImage := func() (io.ReadCloser, string, error) {
|
||||
funcs := dr.fromDiscArtPriority(ctx, s.ffmpeg, conf.Server.DiscArtPriority)
|
||||
return selectImageReader(ctx, artID, funcs...)
|
||||
selectImage := func() (io.ReadCloser, error) {
|
||||
res, err := dr.selectImage(ctx, s.ffmpeg, conf.Server.DiscArtPriority, &chainState{})
|
||||
return res.reader, err
|
||||
}
|
||||
albumArtID := model.ArtworkID{Kind: model.KindAlbumArtwork, ID: dr.album.ID}
|
||||
// Disc art has no state row, hence no content hash: keying on id, album mtime and
|
||||
// DiscArtPriority lets a warm cache answer without running the chain or touching the disk.
|
||||
key := fmt.Sprintf("%s|%d|%s", artID.ID, dr.cacheTime().UnixNano(), conf.Server.DiscArtPriority)
|
||||
img, err := s.serveSource(ctx, key, "", dr.cacheTime(), size, square,
|
||||
func() (io.ReadCloser, error) { rc, _, err := selectImage(); return rc, err })
|
||||
img, err := s.serveSource(ctx, key, "", dr.cacheTime(), size, square, selectImage)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.Canceled) {
|
||||
return nil, err
|
||||
@ -386,6 +386,54 @@ func (s *service) parseArtworkID(ctx context.Context, id string) (model.ArtworkI
|
||||
return model.ArtworkID{}, model.ErrNotFound
|
||||
}
|
||||
|
||||
// TracingResolver is the CLI's read-only view of resolution: it walks the priority chain, records
|
||||
// the walk and reports the winning source, without ever writing artwork state.
|
||||
type TracingResolver struct {
|
||||
inner *resolver
|
||||
trace *ChainTrace
|
||||
}
|
||||
|
||||
// NewTracingResolver builds a TracingResolver that records its priority-chain walk. With live
|
||||
// false the external tier is reported but never called.
|
||||
func NewTracingResolver(ds model.DataStore, ag *agents.Agents, ffm ffmpeg.FFmpeg, t *ChainTrace, live bool) *TracingResolver {
|
||||
gate := offlineGate(t)
|
||||
if live {
|
||||
// A diagnostic must show the provider's real answer, and one item is at most one call
|
||||
// per agent, so --live deliberately bypasses the rate limiter and circuit breaker.
|
||||
gate = tracingGate(t, passthroughGate)
|
||||
}
|
||||
return &TracingResolver{inner: newResolver(ds, ag, ffm, gate), trace: t}
|
||||
}
|
||||
|
||||
// Resolve walks kind's sources for id, recording the walk, and reports the winning source
|
||||
// ("" when none produced an image).
|
||||
func (r *TracingResolver) Resolve(ctx context.Context, kind model.Kind, id string) (string, error) {
|
||||
switch kind {
|
||||
case model.KindArtistArtwork:
|
||||
return r.explain(ctx, r.inner.resolveArtist, id)
|
||||
case model.KindAlbumArtwork:
|
||||
return r.explain(ctx, r.inner.resolveAlbum, id)
|
||||
case model.KindDiscArtwork:
|
||||
return r.explain(ctx, r.inner.resolveDisc, id)
|
||||
case model.KindMediaFileArtwork:
|
||||
return r.explain(ctx, r.inner.resolveMediaFile, id)
|
||||
}
|
||||
return "", fmt.Errorf("artwork: %s artwork has no chain to explain", kind)
|
||||
}
|
||||
|
||||
// explain discards the bytes: nothing downstream persists this resolution, so nothing else
|
||||
// would close the reader either.
|
||||
func (r *TracingResolver) explain(ctx context.Context, resolve func(context.Context, string) (resolution, error), id string) (string, error) {
|
||||
res, err := resolve(withTrace(ctx, r.trace), id)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if res.reader != nil {
|
||||
_ = res.reader.Close()
|
||||
}
|
||||
return res.source, nil
|
||||
}
|
||||
|
||||
func unixMtime(mtime int64) time.Time {
|
||||
if mtime <= 0 {
|
||||
return time.Time{}
|
||||
|
||||
@ -113,27 +113,71 @@ func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.A
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *discArtworkReader) fromDiscArtPriority(ctx context.Context, ffmpeg ffmpeg.FFmpeg, priority string) []sourceFunc {
|
||||
var ff []sourceFunc
|
||||
// discCandidate is one DiscArtPriority entry. skip is set when the entry maps to no source at
|
||||
// all, so a chain walk can say why instead of leaving a configured entry unaccounted for.
|
||||
type discCandidate struct {
|
||||
pattern string
|
||||
resolve func() (resolution, bool)
|
||||
skip string
|
||||
}
|
||||
|
||||
func (d *discArtworkReader) discCandidates(ctx context.Context, ffmpeg ffmpeg.FFmpeg, priority string) []discCandidate {
|
||||
folder := func(sf sourceFunc) func() (resolution, bool) {
|
||||
return func() (resolution, bool) { return resolveFolderSource(d.lib, sf) }
|
||||
}
|
||||
var cc []discCandidate
|
||||
for pattern := range strings.SplitSeq(strings.ToLower(priority), ",") {
|
||||
pattern = strings.TrimSpace(pattern)
|
||||
if pattern == "" {
|
||||
continue
|
||||
}
|
||||
c := discCandidate{pattern: pattern}
|
||||
switch {
|
||||
case pattern == "embedded":
|
||||
ff = append(ff,
|
||||
fromTag(ctx, d.lib.FS, d.firstTrackRel),
|
||||
fromFFmpegTag(ctx, ffmpeg, d.lib.Abs(d.firstTrackRel)),
|
||||
)
|
||||
case pattern == "external":
|
||||
// Not supported for disc art, silently ignore
|
||||
case pattern == "discsubtitle":
|
||||
if subtitle := strings.TrimSpace(d.album.Discs[d.discNumber]); subtitle != "" {
|
||||
ff = append(ff, d.fromDiscSubtitle(ctx, subtitle))
|
||||
c.resolve = func() (resolution, bool) {
|
||||
return resolveEmbedded(ctx, d.lib, ffmpeg, d.firstTrackRel)
|
||||
}
|
||||
case len(d.imgFiles) > 0:
|
||||
ff = append(ff, d.fromExternalFile(ctx, pattern))
|
||||
case pattern == externalCandidate:
|
||||
c.skip = "external sources are not supported for disc artwork"
|
||||
case pattern == "discsubtitle":
|
||||
subtitle := strings.TrimSpace(d.album.Discs[d.discNumber])
|
||||
if subtitle == "" {
|
||||
c.skip = "disc has no subtitle"
|
||||
} else {
|
||||
c.resolve = folder(d.fromDiscSubtitle(ctx, subtitle))
|
||||
}
|
||||
case len(d.imgFiles) == 0:
|
||||
c.skip = "no images in album folder"
|
||||
default:
|
||||
c.resolve = folder(d.fromExternalFile(ctx, pattern))
|
||||
}
|
||||
cc = append(cc, c)
|
||||
}
|
||||
return cc
|
||||
}
|
||||
|
||||
// selectImage walks the DiscArtPriority entries and returns the first that yields an image.
|
||||
// chain records the walk; the serving path passes an untraced one and pays nothing for it.
|
||||
func (d *discArtworkReader) selectImage(ctx context.Context, ffmpeg ffmpeg.FFmpeg, priority string,
|
||||
chain *chainState) (resolution, error) {
|
||||
for _, c := range d.discCandidates(ctx, ffmpeg, priority) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return resolution{}, err
|
||||
}
|
||||
if c.skip != "" {
|
||||
chain.record(c.pattern, OutcomeSkipped, c.skip)
|
||||
continue
|
||||
}
|
||||
start := time.Now()
|
||||
res, ok := c.resolve()
|
||||
log.Trace(ctx, "Artwork: Tried a disc artwork candidate", "albumID", d.album.ID,
|
||||
"disc", d.discNumber, "pattern", c.pattern, "hit", ok, "path", res.sourcePath,
|
||||
"elapsed", time.Since(start))
|
||||
if res, ok = chain.try(c.pattern, res, ok); ok {
|
||||
return res, nil
|
||||
}
|
||||
}
|
||||
return ff
|
||||
return chain.exhausted(), nil
|
||||
}
|
||||
|
||||
// fromDiscSubtitle returns a sourceFunc that matches image files whose stem
|
||||
|
||||
@ -6,6 +6,8 @@ import (
|
||||
"path/filepath"
|
||||
|
||||
"github.com/navidrome/navidrome/model"
|
||||
"github.com/navidrome/navidrome/tests"
|
||||
"github.com/navidrome/navidrome/utils/slice"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
@ -179,19 +181,19 @@ var _ = Describe("Disc Artwork Reader", func() {
|
||||
lib: libraryView{FS: osDirFS{os.DirFS(tmpDir)}, absRoot: tmpDir},
|
||||
}
|
||||
|
||||
ff := reader.fromDiscArtPriority(ctx, nil, "disc*.*, cover.*")
|
||||
Expect(ff).To(HaveLen(2))
|
||||
r, path, err := ff[0]()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(path).To(Equal(f2))
|
||||
r.Close()
|
||||
cc := reader.discCandidates(ctx, nil, "disc*.*, cover.*")
|
||||
Expect(cc).To(HaveLen(2))
|
||||
res, ok := cc[0].resolve()
|
||||
Expect(ok).To(BeTrue())
|
||||
Expect(res.sourcePath).To(Equal(reader.lib.Abs(f2)))
|
||||
res.reader.Close()
|
||||
|
||||
ff = reader.fromDiscArtPriority(ctx, nil, "cover.*, disc*.*")
|
||||
Expect(ff).To(HaveLen(2))
|
||||
r, path, err = ff[0]()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(path).To(Equal(f1))
|
||||
r.Close()
|
||||
cc = reader.discCandidates(ctx, nil, "cover.*, disc*.*")
|
||||
Expect(cc).To(HaveLen(2))
|
||||
res, ok = cc[0].resolve()
|
||||
Expect(ok).To(BeTrue())
|
||||
Expect(res.sourcePath).To(Equal(reader.lib.Abs(f1)))
|
||||
res.reader.Close()
|
||||
})
|
||||
|
||||
DescribeTable("numbered match wins over shared fallback within a pattern",
|
||||
@ -428,64 +430,109 @@ var _ = Describe("Disc Artwork Reader", func() {
|
||||
})
|
||||
|
||||
Describe("discArtworkReader", func() {
|
||||
Describe("fromDiscArtPriority", func() {
|
||||
var (
|
||||
reader *discArtworkReader
|
||||
tmpDir string
|
||||
var (
|
||||
reader *discArtworkReader
|
||||
tmpDir string
|
||||
)
|
||||
|
||||
BeforeEach(func() {
|
||||
tmpDir = GinkgoT().TempDir()
|
||||
reader = &discArtworkReader{
|
||||
discNumber: 2,
|
||||
isMultiFolder: true,
|
||||
discFoldersRel: map[string]bool{"music/album/cd2": true},
|
||||
imgFiles: []string{
|
||||
"music/album/cd1/disc.jpg",
|
||||
"music/album/cd2/disc.jpg",
|
||||
"music/album/cd2/disc2.jpg",
|
||||
},
|
||||
firstTrackRel: "music/album/cd2/track1.flac",
|
||||
lib: libraryView{FS: osDirFS{os.DirFS(tmpDir)}, absRoot: tmpDir},
|
||||
}
|
||||
})
|
||||
|
||||
Describe("selectImage", func() {
|
||||
It("abandons the walk when the context is cancelled", func() {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
|
||||
res, err := reader.selectImage(ctx, nil, "disc*.*, cover.*", &chainState{})
|
||||
|
||||
Expect(err).To(MatchError(context.Canceled))
|
||||
Expect(res.reader).To(BeNil())
|
||||
})
|
||||
|
||||
// "the track has no embedded art" and "the track is there but unreadable" are the two
|
||||
// answers a wrong-artwork report needs told apart; only the second is worth retrying.
|
||||
It("reports a track it cannot parse as unreadable, not as a miss", func() {
|
||||
trace := &ChainTrace{}
|
||||
track := filepath.Join(tmpDir, filepath.FromSlash(reader.firstTrackRel))
|
||||
Expect(os.MkdirAll(filepath.Dir(track), 0755)).To(Succeed())
|
||||
Expect(os.WriteFile(track, []byte("not audio"), 0600)).To(Succeed())
|
||||
|
||||
res, err := reader.selectImage(context.Background(), tests.NewMockFFmpeg(""), "embedded",
|
||||
&chainState{trace: trace})
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.localError).To(BeTrue())
|
||||
Expect(trace.Steps()).To(Equal([]TraceStep{{Candidate: "embedded", Outcome: OutcomeUnreadable}}))
|
||||
})
|
||||
|
||||
It("reports a disc with no tracks to read as a miss", func() {
|
||||
trace := &ChainTrace{}
|
||||
reader.firstTrackRel = ""
|
||||
|
||||
res, err := reader.selectImage(context.Background(), tests.NewMockFFmpeg(""), "embedded",
|
||||
&chainState{trace: trace})
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.localError).To(BeFalse(), "there was nothing to read, so nothing failed to read")
|
||||
Expect(trace.Steps()).To(Equal([]TraceStep{{Candidate: "embedded", Outcome: OutcomeMiss}}))
|
||||
})
|
||||
})
|
||||
|
||||
Describe("discCandidates", func() {
|
||||
It("returns a resolvable candidate for glob patterns", func() {
|
||||
cc := reader.discCandidates(context.Background(), nil, "disc*.*")
|
||||
Expect(cc).To(HaveLen(1))
|
||||
Expect(cc[0].resolve).ToNot(BeNil())
|
||||
})
|
||||
|
||||
It("returns one candidate per entry, in order", func() {
|
||||
cc := reader.discCandidates(context.Background(), nil, "disc*.*, cd*.*, embedded")
|
||||
Expect(slice.Map(cc, func(c discCandidate) string { return c.pattern })).
|
||||
To(Equal([]string{"disc*.*", "cd*.*", "embedded"}))
|
||||
})
|
||||
|
||||
It("skips an empty entry rather than building a glob that matches nothing", func() {
|
||||
cc := reader.discCandidates(context.Background(), nil, "disc*.*,")
|
||||
Expect(cc).To(HaveLen(1))
|
||||
})
|
||||
|
||||
// The skip reasons below are what `artwork explain` prints, so an entry that maps to no
|
||||
// source must say why instead of vanishing from the walk.
|
||||
DescribeTable("keeps an entry that maps to no source, with its reason",
|
||||
func(setup func(), priority, reason string) {
|
||||
setup()
|
||||
cc := reader.discCandidates(context.Background(), nil, priority)
|
||||
Expect(cc).To(HaveLen(1))
|
||||
Expect(cc[0].resolve).To(BeNil())
|
||||
Expect(cc[0].skip).To(Equal(reason))
|
||||
},
|
||||
Entry("external is unsupported", func() {}, "external",
|
||||
"external sources are not supported for disc artwork"),
|
||||
Entry("no images in the album folder", func() { reader.imgFiles = nil }, "disc*.*",
|
||||
"no images in album folder"),
|
||||
Entry("the disc has no subtitle",
|
||||
func() { reader.album = model.Album{Discs: model.Discs{2: ""}} }, "discsubtitle",
|
||||
"disc has no subtitle"),
|
||||
)
|
||||
|
||||
BeforeEach(func() {
|
||||
tmpDir = GinkgoT().TempDir()
|
||||
reader = &discArtworkReader{
|
||||
discNumber: 2,
|
||||
isMultiFolder: true,
|
||||
discFoldersRel: map[string]bool{"music/album/cd2": true},
|
||||
imgFiles: []string{
|
||||
"music/album/cd1/disc.jpg",
|
||||
"music/album/cd2/disc.jpg",
|
||||
"music/album/cd2/disc2.jpg",
|
||||
},
|
||||
firstTrackRel: "music/album/cd2/track1.flac",
|
||||
lib: libraryView{FS: osDirFS{os.DirFS(tmpDir)}, absRoot: tmpDir},
|
||||
}
|
||||
})
|
||||
|
||||
It("returns source funcs for glob patterns", func() {
|
||||
ff := reader.fromDiscArtPriority(context.Background(), nil, "disc*.*")
|
||||
Expect(ff).To(HaveLen(1))
|
||||
})
|
||||
|
||||
It("returns source funcs for embedded pattern", func() {
|
||||
ff := reader.fromDiscArtPriority(context.Background(), nil, "embedded")
|
||||
Expect(ff).To(HaveLen(2)) // fromTag + fromFFmpegTag
|
||||
})
|
||||
|
||||
It("handles multiple comma-separated patterns", func() {
|
||||
ff := reader.fromDiscArtPriority(context.Background(), nil, "disc*.*, cd*.*, embedded")
|
||||
Expect(ff).To(HaveLen(4)) // disc*.* + cd*.* + fromTag + fromFFmpegTag
|
||||
})
|
||||
|
||||
It("ignores 'external' pattern silently", func() {
|
||||
ff := reader.fromDiscArtPriority(context.Background(), nil, "external")
|
||||
Expect(ff).To(HaveLen(0))
|
||||
})
|
||||
|
||||
It("returns no source funcs when imgFiles is empty and pattern is not embedded", func() {
|
||||
reader.imgFiles = nil
|
||||
ff := reader.fromDiscArtPriority(context.Background(), nil, "disc*.*")
|
||||
Expect(ff).To(HaveLen(0))
|
||||
})
|
||||
|
||||
It("returns source func for discsubtitle pattern", func() {
|
||||
reader.album = model.Album{Discs: model.Discs{2: "Bonus Tracks"}}
|
||||
ff := reader.fromDiscArtPriority(context.Background(), nil, "discsubtitle")
|
||||
Expect(ff).To(HaveLen(1))
|
||||
})
|
||||
|
||||
It("returns no source func for discsubtitle when disc has no subtitle", func() {
|
||||
reader.album = model.Album{Discs: model.Discs{2: ""}}
|
||||
ff := reader.fromDiscArtPriority(context.Background(), nil, "discsubtitle")
|
||||
Expect(ff).To(HaveLen(0))
|
||||
cc := reader.discCandidates(context.Background(), nil, "discsubtitle")
|
||||
Expect(cc).To(HaveLen(1))
|
||||
Expect(cc[0].resolve).ToNot(BeNil())
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@ -6,6 +6,8 @@ import (
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
@ -16,27 +18,54 @@ import (
|
||||
"github.com/navidrome/navidrome/utils/slice"
|
||||
)
|
||||
|
||||
const staleAbsentAge = 24 * time.Hour
|
||||
// StaleAbsentAge is how long an absent state is trusted before a recheck retries it.
|
||||
const StaleAbsentAge = 24 * time.Hour
|
||||
|
||||
// recheckKinds omits media files: they resolve embedded-only, at scan or on view.
|
||||
var recheckKinds = []model.Kind{
|
||||
// RecheckKinds omits media files: they resolve embedded-only, at scan or on view.
|
||||
var RecheckKinds = []model.Kind{
|
||||
model.KindArtistArtwork, model.KindAlbumArtwork, model.KindPlaylistArtwork, model.KindRadioArtwork,
|
||||
}
|
||||
|
||||
// KeepsState reports whether a kind is recorded in item_artwork and the artwork queue. Disc
|
||||
// artwork is read through on every request and cached by content key, so it has neither.
|
||||
func KeepsState(kind model.Kind) bool { return kind != model.KindDiscArtwork }
|
||||
|
||||
// RefreshableKinds is every kind Refresh can clear and re-queue, so it holds exactly the kinds
|
||||
// KeepsState admits. Media files are absent from RecheckKinds but belong here: the worker
|
||||
// resolves them, it just never revisits them on its own.
|
||||
var RefreshableKinds = append(slices.Clone(RecheckKinds), model.KindMediaFileArtwork)
|
||||
|
||||
// hasRecheckPath reports whether a periodic job will revisit this kind, making an absent settle recoverable.
|
||||
func hasRecheckPath(prefix string) bool {
|
||||
kind, ok := model.ParseKind(prefix)
|
||||
return ok && slices.Contains(recheckKinds, kind)
|
||||
return ok && slices.Contains(RecheckKinds, kind)
|
||||
}
|
||||
|
||||
// artworkEpoch invalidates all resolution state when bumped; bump it whenever resolution semantics change.
|
||||
const artworkEpoch = 1
|
||||
|
||||
// fingerprint covers the inputs that affect resolution outcomes; a change invalidates stored state.
|
||||
func fingerprint() string {
|
||||
raw := fmt.Sprintf("%s|%s|%s|%s|%t|%t|%d",
|
||||
conf.Server.CoverArtPriority, conf.Server.ArtistArtPriority, conf.Server.ArtistImageFolder,
|
||||
conf.Server.Agents, conf.Server.EnableExternalServices, conf.Server.EnableM3UExternalAlbumArt, artworkEpoch)
|
||||
// FingerprintInput is one config value the fingerprint covers, named after the setting it came from.
|
||||
type FingerprintInput struct {
|
||||
Name string
|
||||
Value string
|
||||
}
|
||||
|
||||
// FingerprintInputs is the single listing of what ConfigFingerprint hashes.
|
||||
func FingerprintInputs() []FingerprintInput {
|
||||
return []FingerprintInput{
|
||||
{"CoverArtPriority", conf.Server.CoverArtPriority},
|
||||
{"ArtistArtPriority", conf.Server.ArtistArtPriority},
|
||||
{"ArtistImageFolder", conf.Server.ArtistImageFolder},
|
||||
{"Agents", conf.Server.Agents},
|
||||
{"EnableExternalServices", strconv.FormatBool(conf.Server.EnableExternalServices)},
|
||||
{"EnableM3UExternalAlbumArt", strconv.FormatBool(conf.Server.EnableM3UExternalAlbumArt)},
|
||||
}
|
||||
}
|
||||
|
||||
// ConfigFingerprint covers the inputs that affect resolution outcomes; a change invalidates stored state.
|
||||
func ConfigFingerprint() string {
|
||||
values := slice.Map(FingerprintInputs(), func(i FingerprintInput) string { return i.Value })
|
||||
raw := fmt.Sprintf("%s|%d", strings.Join(values, "|"), artworkEpoch)
|
||||
sum := md5.Sum([]byte(raw)) //nolint:gosec // fingerprint, not security-sensitive
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
@ -45,7 +74,7 @@ func fingerprint() string {
|
||||
func backfill(ctx context.Context, ds model.DataStore) (bool, error) {
|
||||
start := time.Now()
|
||||
ctx = auth.WithAdminUser(ctx, ds)
|
||||
current := fingerprint()
|
||||
current := ConfigFingerprint()
|
||||
props := ds.Property(ctx)
|
||||
stored, err := props.DefaultGet(consts.ArtConfFingerprintPropertyKey, "")
|
||||
if err != nil {
|
||||
@ -95,9 +124,9 @@ func enqueueBackfillKind(ctx context.Context, ds model.DataStore, kind model.Kin
|
||||
}
|
||||
|
||||
func enqueueStaleAbsentAll(ctx context.Context, ds model.DataStore) error {
|
||||
cutoff := time.Now().Add(-staleAbsentAge)
|
||||
cutoff := time.Now().Add(-StaleAbsentAge)
|
||||
queue := ds.ArtworkQueue(ctx)
|
||||
for _, kind := range recheckKinds {
|
||||
for _, kind := range RecheckKinds {
|
||||
if _, err := queue.EnqueueStaleAbsent(kind, cutoff); err != nil {
|
||||
return err
|
||||
}
|
||||
@ -108,7 +137,7 @@ func enqueueStaleAbsentAll(ctx context.Context, ds model.DataStore) error {
|
||||
// enqueueMissingAll is the safety net for entities a scan never enqueued (added between scans, or scanner off).
|
||||
func enqueueMissingAll(ctx context.Context, ds model.DataStore) error {
|
||||
queue := ds.ArtworkQueue(ctx)
|
||||
for _, kind := range recheckKinds {
|
||||
for _, kind := range RecheckKinds {
|
||||
if _, err := queue.EnqueueAllMissing(kind, model.ArtworkPriorityRecheck); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@ -52,6 +52,19 @@ func (o *orderTrackingQueueRepo) Enqueue(items ...model.ArtworkQueueItem) error
|
||||
return o.MockArtworkQueueRepo.Enqueue(items...)
|
||||
}
|
||||
|
||||
var _ = Describe("RefreshableKinds", func() {
|
||||
// The two are meant to describe the same fact. Nothing but this test stops them from drifting,
|
||||
// and a drift would have `artwork explain` report state for a kind that keeps none.
|
||||
It("holds exactly the kinds that keep state", func() {
|
||||
for _, k := range []model.Kind{
|
||||
model.KindArtistArtwork, model.KindAlbumArtwork, model.KindPlaylistArtwork,
|
||||
model.KindRadioArtwork, model.KindMediaFileArtwork, model.KindDiscArtwork,
|
||||
} {
|
||||
Expect(slices.Contains(RefreshableKinds, k)).To(Equal(KeepsState(k)), k.String())
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("Housekeeping", func() {
|
||||
var (
|
||||
ctx context.Context
|
||||
@ -93,32 +106,54 @@ var _ = Describe("Housekeeping", func() {
|
||||
|
||||
Describe("Fingerprint", func() {
|
||||
It("changes when a fingerprint-affecting config value changes", func() {
|
||||
f1 := fingerprint()
|
||||
f1 := ConfigFingerprint()
|
||||
conf.Server.CoverArtPriority = "folder, embedded"
|
||||
f2 := fingerprint()
|
||||
f2 := ConfigFingerprint()
|
||||
Expect(f1).NotTo(Equal(f2))
|
||||
})
|
||||
|
||||
It("changes when ArtistImageFolder changes", func() {
|
||||
conf.Server.ArtistImageFolder = "/before"
|
||||
f1 := fingerprint()
|
||||
f1 := ConfigFingerprint()
|
||||
conf.Server.ArtistImageFolder = "/after"
|
||||
Expect(fingerprint()).NotTo(Equal(f1))
|
||||
Expect(ConfigFingerprint()).NotTo(Equal(f1))
|
||||
})
|
||||
|
||||
It("changes when EnableM3UExternalAlbumArt is toggled", func() {
|
||||
conf.Server.EnableM3UExternalAlbumArt = false
|
||||
f1 := fingerprint()
|
||||
f1 := ConfigFingerprint()
|
||||
conf.Server.EnableM3UExternalAlbumArt = true
|
||||
Expect(fingerprint()).NotTo(Equal(f1))
|
||||
Expect(ConfigFingerprint()).NotTo(Equal(f1))
|
||||
})
|
||||
|
||||
// Pinned: a changed formula re-resolves every library on upgrade, flooding external providers.
|
||||
It("hashes a given config to a stable value", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*, embedded"
|
||||
conf.Server.ArtistArtPriority = "artist.*, external"
|
||||
conf.Server.ArtistImageFolder = ""
|
||||
conf.Server.Agents = "lastfm,spotify"
|
||||
conf.Server.EnableExternalServices = true
|
||||
conf.Server.EnableM3UExternalAlbumArt = false
|
||||
|
||||
Expect(ConfigFingerprint()).To(Equal("7e537a22febc07d3d5ca40546e88da54"))
|
||||
})
|
||||
|
||||
It("reports the config inputs it hashes, so a change can be traced to a setting", func() {
|
||||
conf.Server.Agents = "lastfm,spotify"
|
||||
conf.Server.CoverArtPriority = "cover.*, embedded"
|
||||
|
||||
Expect(FingerprintInputs()).To(ContainElements(
|
||||
FingerprintInput{Name: "Agents", Value: "lastfm,spotify"},
|
||||
FingerprintInput{Name: "CoverArtPriority", Value: "cover.*, embedded"},
|
||||
))
|
||||
})
|
||||
|
||||
It("does not change when the server version changes", func() {
|
||||
original := consts.Version
|
||||
DeferCleanup(func() { consts.Version = original })
|
||||
f1 := fingerprint()
|
||||
f1 := ConfigFingerprint()
|
||||
consts.Version = original + "-next"
|
||||
Expect(fingerprint()).To(Equal(f1),
|
||||
Expect(ConfigFingerprint()).To(Equal(f1),
|
||||
"the version must not invalidate artwork state: it would re-resolve every entity on every build")
|
||||
})
|
||||
})
|
||||
@ -126,7 +161,7 @@ var _ = Describe("Housekeeping", func() {
|
||||
Describe("Backfill", func() {
|
||||
It("enqueues nothing and returns false when the stored fingerprint matches", func() {
|
||||
seedEntities()
|
||||
Expect(propRepo.Put(consts.ArtConfFingerprintPropertyKey, fingerprint())).To(Succeed())
|
||||
Expect(propRepo.Put(consts.ArtConfFingerprintPropertyKey, ConfigFingerprint())).To(Succeed())
|
||||
|
||||
did, err := backfill(ctx, ds)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
@ -150,7 +185,7 @@ var _ = Describe("Housekeeping", func() {
|
||||
|
||||
stored, err := propRepo.Get(consts.ArtConfFingerprintPropertyKey)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(stored).To(Equal(fingerprint()))
|
||||
Expect(stored).To(Equal(ConfigFingerprint()))
|
||||
})
|
||||
|
||||
It("enqueues a private playlist by resolving it under an admin context", func() {
|
||||
|
||||
@ -55,6 +55,10 @@ func (r *resizedItem) Reader(ctx context.Context) (io.ReadCloser, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// An open() that reports "no image" as a nil reader would otherwise panic on the Close below.
|
||||
if orig == nil {
|
||||
return nil, ErrUnavailable
|
||||
}
|
||||
defer orig.Close()
|
||||
data, err := readCapped(orig)
|
||||
if err != nil {
|
||||
|
||||
41
core/artwork/image_cache_test.go
Normal file
41
core/artwork/image_cache_test.go
Normal file
@ -0,0 +1,41 @@
|
||||
package artwork
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("resizedItem", func() {
|
||||
Describe("Reader", func() {
|
||||
newItem := func(open func() (io.ReadCloser, error)) *resizedItem {
|
||||
return &resizedItem{hash: "abc123", size: 300, open: open}
|
||||
}
|
||||
|
||||
It("reports a nil reader as unavailable instead of panicking on it", func() {
|
||||
// Every caller is expected to report "no image" as an error, but a nil reader reaches
|
||||
// the deferred Close as a nil interface, which takes the whole request down.
|
||||
_, err := newItem(func() (io.ReadCloser, error) { return nil, nil }).Reader(context.Background())
|
||||
Expect(err).To(MatchError(ErrUnavailable))
|
||||
})
|
||||
|
||||
It("propagates the open error", func() {
|
||||
boom := errors.New("boom")
|
||||
_, err := newItem(func() (io.ReadCloser, error) { return nil, boom }).Reader(context.Background())
|
||||
Expect(err).To(MatchError(boom))
|
||||
})
|
||||
|
||||
It("serves the original bytes when they cannot be resized", func() {
|
||||
rc, err := newItem(func() (io.ReadCloser, error) {
|
||||
return io.NopCloser(strings.NewReader("not an image")), nil
|
||||
}).Reader(context.Background())
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
defer rc.Close()
|
||||
Expect(io.ReadAll(rc)).To(Equal([]byte("not an image")))
|
||||
})
|
||||
})
|
||||
})
|
||||
@ -34,18 +34,31 @@ type resolution struct {
|
||||
|
||||
// chainState carries what a priority walk has seen so far. A hit takes extErr with it so a
|
||||
// transient external failure still retries; localErr is dropped, as the scanner re-lists changes.
|
||||
type chainState struct{ extErr, localErr bool }
|
||||
type chainState struct {
|
||||
extErr, localErr bool
|
||||
trace *ChainTrace // nil unless the CLI asked for a trace
|
||||
}
|
||||
|
||||
// try stamps the accumulated external failure onto a hit, and records the miss otherwise.
|
||||
func (c *chainState) try(res resolution, ok bool) (resolution, bool) {
|
||||
func (c *chainState) try(candidate string, res resolution, ok bool) (resolution, bool) {
|
||||
if ok {
|
||||
res.extError = c.extErr
|
||||
c.record(candidate, OutcomeHit, res.sourcePath)
|
||||
return res, true
|
||||
}
|
||||
c.localErr = c.localErr || res.localError
|
||||
if res.localError {
|
||||
c.record(candidate, OutcomeUnreadable, "")
|
||||
} else {
|
||||
c.record(candidate, OutcomeMiss, "")
|
||||
}
|
||||
return resolution{}, false
|
||||
}
|
||||
|
||||
func (c *chainState) record(candidate string, out Outcome, detail string) {
|
||||
c.trace.add(TraceStep{Candidate: candidate, Outcome: out, Detail: detail})
|
||||
}
|
||||
|
||||
// exhausted is the outcome when no source in the chain yielded an image.
|
||||
func (c *chainState) exhausted() resolution {
|
||||
return resolution{extError: c.extErr, localError: c.localErr}
|
||||
@ -95,8 +108,69 @@ func (r *resolver) resolve(ctx context.Context, item model.ArtworkQueueItem) (re
|
||||
}
|
||||
}
|
||||
|
||||
// fetchExternalAlbum and fetchExternalArtist are the only places resolution touches the network,
|
||||
// so a local-only resolver is stopped here rather than at each point in the chain walk.
|
||||
// Explainable reports whether TracingResolver can walk this kind's sources and report which one
|
||||
// won; playlists and radios resolve from a fixed internal order, with nothing configured to explain.
|
||||
func Explainable(kind model.Kind) bool {
|
||||
switch kind {
|
||||
case model.KindArtistArtwork, model.KindAlbumArtwork, model.KindDiscArtwork, model.KindMediaFileArtwork:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// MayFetchExternal reports whether resolving this kind can issue an external request under the
|
||||
// current config. Playlists inherit the album chain: the generated grid resolves album art.
|
||||
func MayFetchExternal(kind model.Kind) bool {
|
||||
switch kind {
|
||||
case model.KindArtistArtwork:
|
||||
return chainFetchesExternal(conf.Server.ArtistArtPriority)
|
||||
case model.KindAlbumArtwork:
|
||||
return chainFetchesExternal(conf.Server.CoverArtPriority)
|
||||
case model.KindPlaylistArtwork:
|
||||
return conf.Server.EnableM3UExternalAlbumArt || chainFetchesExternal(conf.Server.CoverArtPriority)
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// ImageAgentCount is how many enabled agents provide artist and album images.
|
||||
type ImageAgentCount struct{ Artist, Album int }
|
||||
|
||||
// ExternalLookupsPerItem reports what resolving one item of this kind can cost: every image agent is
|
||||
// tried, and a zero count still bills one, so agents the caller cannot see never read as free.
|
||||
func ExternalLookupsPerItem(kind model.Kind, agents ImageAgentCount) int64 {
|
||||
if !MayFetchExternal(kind) {
|
||||
return 0
|
||||
}
|
||||
switch kind {
|
||||
case model.KindArtistArtwork:
|
||||
return int64(max(agents.Artist, 1))
|
||||
case model.KindAlbumArtwork:
|
||||
return int64(max(agents.Album, 1))
|
||||
case model.KindPlaylistArtwork:
|
||||
var n int64
|
||||
if conf.Server.EnableM3UExternalAlbumArt {
|
||||
n++
|
||||
}
|
||||
if chainFetchesExternal(conf.Server.CoverArtPriority) {
|
||||
n += PlaylistGridSamples * int64(max(agents.Album, 1))
|
||||
}
|
||||
return n
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func chainFetchesExternal(priority string) bool {
|
||||
for pattern := range strings.SplitSeq(strings.ToLower(priority), ",") {
|
||||
if strings.TrimSpace(pattern) == externalCandidate {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Album and artist fetches stop here when the resolver is local-only, rather than at each point in
|
||||
// the chain walk; resolvePlaylist gates the third network path, the m3u image URL, itself.
|
||||
func (r *resolver) fetchExternalAlbum(ctx context.Context, al model.Album) (io.ReadCloser, string, bool) {
|
||||
if r.ext == nil {
|
||||
return nil, "", false
|
||||
@ -126,24 +200,31 @@ func (r *resolver) resolveAlbum(ctx context.Context, albumID string) (resolution
|
||||
return resolution{}, err
|
||||
}
|
||||
|
||||
var chain chainState
|
||||
chain := chainState{trace: traceFrom(ctx)}
|
||||
for pattern := range strings.SplitSeq(strings.ToLower(conf.Server.CoverArtPriority), ",") {
|
||||
pattern = strings.TrimSpace(pattern)
|
||||
if pattern == "" {
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case pattern == "embedded":
|
||||
if res, ok := chain.try(resolveEmbedded(ctx, lib, r.ffmpeg, al.EmbedArtPath)); ok {
|
||||
res, ok := resolveEmbedded(ctx, lib, r.ffmpeg, al.EmbedArtPath)
|
||||
if res, ok = chain.try(pattern, res, ok); ok {
|
||||
return res, nil
|
||||
}
|
||||
case pattern == "external":
|
||||
case pattern == externalCandidate:
|
||||
if rd, name, isErr := r.fetchExternalAlbum(ctx, *al); rd != nil {
|
||||
return resolution{reader: rd, source: "external:" + name}, nil
|
||||
return resolution{reader: rd, source: ExternalPrefix + name}, nil
|
||||
} else if isErr {
|
||||
chain.extErr = true
|
||||
}
|
||||
case len(imgFiles) > 0:
|
||||
if res, ok := chain.try(resolveFolderFile(ctx, lib, imgFiles, pattern)); ok {
|
||||
res, ok := resolveFolderFile(ctx, lib, imgFiles, pattern)
|
||||
if res, ok = chain.try(pattern, res, ok); ok {
|
||||
return res, nil
|
||||
}
|
||||
default:
|
||||
chain.record(pattern, OutcomeSkipped, "no images in album folder")
|
||||
}
|
||||
}
|
||||
return chain.exhausted(), nil
|
||||
@ -155,9 +236,10 @@ func (r *resolver) resolveArtist(ctx context.Context, artistID string) (resoluti
|
||||
if err != nil {
|
||||
return resolution{}, err
|
||||
}
|
||||
upload, ok := resolveLocalFile(ar.UploadedImagePath(), "upload")
|
||||
if ok {
|
||||
return upload, nil
|
||||
chain := chainState{trace: traceFrom(ctx)}
|
||||
upload, uploadOK := resolveLocalFile(ar.UploadedImagePath(), "upload")
|
||||
if res, ok := chain.try("upload", upload, uploadOK); ok {
|
||||
return res, nil
|
||||
}
|
||||
if upload.localError {
|
||||
// The upload outranks every other source; falling through would persist a lower-priority
|
||||
@ -191,32 +273,43 @@ func (r *resolver) resolveArtist(ctx context.Context, artistID string) (resoluti
|
||||
}
|
||||
}
|
||||
|
||||
var chain chainState
|
||||
for pattern := range strings.SplitSeq(strings.ToLower(conf.Server.ArtistArtPriority), ",") {
|
||||
pattern = strings.TrimSpace(pattern)
|
||||
if pattern == "" {
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case pattern == "external":
|
||||
case pattern == externalCandidate:
|
||||
if rd, name, isErr := r.fetchExternalArtist(ctx, *ar); rd != nil {
|
||||
return resolution{reader: rd, source: "external:" + name}, nil
|
||||
return resolution{reader: rd, source: ExternalPrefix + name}, nil
|
||||
} else if isErr {
|
||||
chain.extErr = true
|
||||
}
|
||||
case pattern == "image-folder":
|
||||
if res, ok := chain.try(resolveArtistImageFolder(ar)); ok {
|
||||
res, ok := resolveArtistImageFolder(ar)
|
||||
if res, ok = chain.try(pattern, res, ok); ok {
|
||||
return res, nil
|
||||
}
|
||||
case strings.HasPrefix(pattern, "album/"):
|
||||
if lib.FS == nil {
|
||||
chain.record(pattern, OutcomeSkipped, "artist has no albums")
|
||||
continue
|
||||
}
|
||||
if res, ok := chain.try(resolveFolderFile(ctx, lib, imgFiles, strings.TrimPrefix(pattern, "album/"))); ok {
|
||||
res, ok := resolveFolderFile(ctx, lib, imgFiles, strings.TrimPrefix(pattern, "album/"))
|
||||
if res, ok = chain.try(pattern, res, ok); ok {
|
||||
return res, nil
|
||||
}
|
||||
default:
|
||||
if lib.FS == nil || artistFolder == "" {
|
||||
if lib.FS == nil {
|
||||
chain.record(pattern, OutcomeSkipped, "artist has no albums")
|
||||
continue
|
||||
}
|
||||
if res, ok := chain.try(resolveArtistFolderPattern(ctx, lib, artistFolder, pattern)); ok {
|
||||
if artistFolder == "" {
|
||||
chain.record(pattern, OutcomeSkipped, "no artist folder")
|
||||
continue
|
||||
}
|
||||
res, ok := resolveArtistFolderPattern(ctx, lib, artistFolder, pattern)
|
||||
if res, ok = chain.try(pattern, res, ok); ok {
|
||||
return res, nil
|
||||
}
|
||||
}
|
||||
@ -224,6 +317,9 @@ func (r *resolver) resolveArtist(ctx context.Context, artistID string) (resoluti
|
||||
return chain.exhausted(), nil
|
||||
}
|
||||
|
||||
// PlaylistGridSamples is how many albums resolvePlaylist samples to build the generated grid.
|
||||
const PlaylistGridSamples = 4
|
||||
|
||||
// resolvePlaylist tries the uploaded image, the sidecar and ExternalImageURL, then a generated grid.
|
||||
func (r *resolver) resolvePlaylist(ctx context.Context, playlistID string) (resolution, error) {
|
||||
pl, err := r.ds.Playlist(ctx).Get(playlistID)
|
||||
@ -269,7 +365,8 @@ func (r *resolver) resolvePlaylist(ctx context.Context, playlistID string) (reso
|
||||
}
|
||||
}
|
||||
|
||||
albumIDs, err := r.ds.Playlist(ctx).Tracks(pl.ID, false).GetAlbumIDs(model.QueryOptions{Max: 4, Sort: "random()"})
|
||||
albumIDs, err := r.ds.Playlist(ctx).Tracks(pl.ID, false).
|
||||
GetAlbumIDs(model.QueryOptions{Max: PlaylistGridSamples, Sort: "random()"})
|
||||
if err != nil {
|
||||
return resolution{}, err
|
||||
}
|
||||
@ -295,7 +392,7 @@ func (r *resolver) resolvePlaylist(ctx context.Context, playlistID string) (reso
|
||||
if decErr == nil {
|
||||
tiles = append(tiles, tile)
|
||||
}
|
||||
if len(tiles) == 4 {
|
||||
if len(tiles) == PlaylistGridSamples {
|
||||
break
|
||||
}
|
||||
}
|
||||
@ -337,15 +434,35 @@ func (r *resolver) resolveMediaFile(ctx context.Context, id string) (resolution,
|
||||
if err != nil {
|
||||
return resolution{}, err
|
||||
}
|
||||
if !conf.Server.EnableMediaFileCoverArt || !mf.HasCoverArt {
|
||||
chain := chainState{trace: traceFrom(ctx)}
|
||||
switch {
|
||||
case !conf.Server.EnableMediaFileCoverArt:
|
||||
chain.record("embedded", OutcomeSkipped, "EnableMediaFileCoverArt is off")
|
||||
return resolution{}, nil
|
||||
case !mf.HasCoverArt:
|
||||
chain.record("embedded", OutcomeMiss, "the track has no embedded cover art")
|
||||
return resolution{}, nil
|
||||
}
|
||||
lib, err := loadLibraryView(ctx, r.ds, mf.LibraryID)
|
||||
if err != nil {
|
||||
return resolution{}, err
|
||||
}
|
||||
res, _ := resolveEmbedded(ctx, lib, r.ffmpeg, mf.Path)
|
||||
return res, nil
|
||||
res, ok := resolveEmbedded(ctx, lib, r.ffmpeg, mf.Path)
|
||||
if res, ok = chain.try("embedded", res, ok); ok {
|
||||
return res, nil
|
||||
}
|
||||
return chain.exhausted(), nil
|
||||
}
|
||||
|
||||
// resolveDisc walks conf.Server.DiscArtPriority. Disc artwork keeps no state row and is never
|
||||
// queued: the serving path reads it through on every request, so this only ever explains.
|
||||
func (r *resolver) resolveDisc(ctx context.Context, id string) (resolution, error) {
|
||||
dr, err := newDiscArtworkReader(ctx, r.ds, model.ArtworkID{Kind: model.KindDiscArtwork, ID: id})
|
||||
if err != nil {
|
||||
return resolution{}, err
|
||||
}
|
||||
chain := chainState{trace: traceFrom(ctx)}
|
||||
return dr.selectImage(ctx, r.ffmpeg, conf.Server.DiscArtPriority, &chain)
|
||||
}
|
||||
|
||||
// resolveExternalStep runs a single external sourceFunc through the named gate. extErr excludes
|
||||
@ -353,7 +470,7 @@ func (r *resolver) resolveMediaFile(ctx context.Context, id string) (resolution,
|
||||
func resolveExternalStep(gate gateFunc, name string, sf sourceFunc) (res resolution, ok bool, extErr bool) {
|
||||
r, path, err := gate(name, sf)
|
||||
if r != nil {
|
||||
return resolution{reader: r, source: "external", sourcePath: path}, true, false
|
||||
return resolution{reader: r, source: externalCandidate, sourcePath: path}, true, false
|
||||
}
|
||||
return resolution{}, false, err != nil && !errors.Is(err, model.ErrNotFound)
|
||||
}
|
||||
@ -394,14 +511,20 @@ func resolveEmbedded(ctx context.Context, lib libraryView, ffm ffmpeg.FFmpeg, em
|
||||
return resolution{localError: unreadable}, false
|
||||
}
|
||||
|
||||
func resolveFolderFile(ctx context.Context, lib libraryView, imgFiles []string, pattern string) (resolution, bool) {
|
||||
r, path, err := fromExternalFile(ctx, lib.FS, imgFiles, pattern)()
|
||||
// resolveFolderSource turns a source that yields a library-relative image path into a folder
|
||||
// resolution, keeping an existing-but-unopenable file distinct from an absent one.
|
||||
func resolveFolderSource(lib libraryView, sf sourceFunc) (resolution, bool) {
|
||||
r, path, err := sf()
|
||||
if r == nil {
|
||||
return resolution{localError: errors.Is(err, errSourceUnreadable)}, false
|
||||
}
|
||||
return resolution{reader: r, source: "folder", sourcePath: lib.Abs(path), refMtime: mtimeViaFS(lib.FS, path)}, true
|
||||
}
|
||||
|
||||
func resolveFolderFile(ctx context.Context, lib libraryView, imgFiles []string, pattern string) (resolution, bool) {
|
||||
return resolveFolderSource(lib, fromExternalFile(ctx, lib.FS, imgFiles, pattern))
|
||||
}
|
||||
|
||||
func resolveArtistImageFolder(ar *model.Artist) (resolution, bool) {
|
||||
folder := conf.Server.ArtistImageFolder
|
||||
if folder == "" {
|
||||
|
||||
@ -394,6 +394,28 @@ var _ = Describe("resolveItem", func() {
|
||||
Entry("4 albums -> full grid", []string{"t1", "t2", "t3", "t4"}, tileSize-1),
|
||||
)
|
||||
|
||||
// The grid samples album art through the full album chain, so a playlist reaches the
|
||||
// network even with the m3u fetch off.
|
||||
It("calls the album image agents for its grid tiles when m3u art is disabled", func() {
|
||||
conf.Server.EnableM3UExternalAlbumArt = false
|
||||
conf.Server.CoverArtPriority = "external"
|
||||
folderRepo.result = nil
|
||||
plRepo := tests.CreateMockPlaylistRepo()
|
||||
plRepo.SetData(model.Playlists{{ID: "plgrid", Name: "Playlist"}})
|
||||
plRepo.TracksRepo = &tests.MockPlaylistTrackRepo{AlbumIDs: []string{"t1", "t2"}}
|
||||
ds.MockedPlaylist = plRepo
|
||||
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("boom")})
|
||||
var gatedNames []string
|
||||
gate := func(name string, f func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error) {
|
||||
gatedNames = append(gatedNames, name)
|
||||
return f()
|
||||
}
|
||||
|
||||
_, err := newResolver(ds, ag, ffm, gate).resolve(ctx, model.ArtworkQueueItem{ItemKind: "pl", ItemID: "plgrid"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(gatedNames).To(Equal([]string{"failAgent", "failAgent"}), "one lookup per sampled album")
|
||||
})
|
||||
|
||||
It("resolves the uploaded image before the generated grid", func() {
|
||||
tmpDir := GinkgoT().TempDir()
|
||||
conf.Server.DataFolder = conf.NewDir(tmpDir)
|
||||
@ -630,3 +652,110 @@ var _ = Describe("decodeTile", func() {
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("Explainable", func() {
|
||||
It("is true for the kinds the resolver walks", func() {
|
||||
Expect(Explainable(model.KindArtistArtwork)).To(BeTrue())
|
||||
Expect(Explainable(model.KindAlbumArtwork)).To(BeTrue())
|
||||
Expect(Explainable(model.KindDiscArtwork)).To(BeTrue())
|
||||
Expect(Explainable(model.KindMediaFileArtwork)).To(BeTrue())
|
||||
})
|
||||
|
||||
It("is false for the kinds resolved from a fixed internal order", func() {
|
||||
Expect(Explainable(model.KindPlaylistArtwork)).To(BeFalse())
|
||||
Expect(Explainable(model.KindRadioArtwork)).To(BeFalse())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("MayFetchExternal", func() {
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.CoverArtPriority = "cover.*, embedded"
|
||||
conf.Server.ArtistArtPriority = "artist.*"
|
||||
conf.Server.EnableM3UExternalAlbumArt = false
|
||||
})
|
||||
|
||||
It("is true for the kinds whose chain includes the external candidate", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*, external"
|
||||
conf.Server.ArtistArtPriority = "artist.*, external"
|
||||
Expect(MayFetchExternal(model.KindAlbumArtwork)).To(BeTrue())
|
||||
Expect(MayFetchExternal(model.KindArtistArtwork)).To(BeTrue())
|
||||
})
|
||||
|
||||
It("is false for a chain with no external candidate", func() {
|
||||
Expect(MayFetchExternal(model.KindAlbumArtwork)).To(BeFalse())
|
||||
Expect(MayFetchExternal(model.KindArtistArtwork)).To(BeFalse())
|
||||
})
|
||||
|
||||
It("is true for playlists when the m3u image fetch is enabled", func() {
|
||||
conf.Server.EnableM3UExternalAlbumArt = true
|
||||
Expect(MayFetchExternal(model.KindPlaylistArtwork)).To(BeTrue())
|
||||
})
|
||||
|
||||
It("is true for playlists whose grid tiles resolve through an external album chain", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*, external"
|
||||
Expect(MayFetchExternal(model.KindPlaylistArtwork)).To(BeTrue())
|
||||
})
|
||||
|
||||
It("is false for playlists with both paths off", func() {
|
||||
Expect(MayFetchExternal(model.KindPlaylistArtwork)).To(BeFalse())
|
||||
})
|
||||
|
||||
It("is false for the kinds that only read local files", func() {
|
||||
conf.Server.CoverArtPriority = "external"
|
||||
conf.Server.ArtistArtPriority = "external"
|
||||
conf.Server.EnableM3UExternalAlbumArt = true
|
||||
Expect(MayFetchExternal(model.KindRadioArtwork)).To(BeFalse())
|
||||
Expect(MayFetchExternal(model.KindMediaFileArtwork)).To(BeFalse())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("ExternalLookupsPerItem", func() {
|
||||
count := ImageAgentCount{Artist: 3, Album: 2}
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.CoverArtPriority = "cover.*, external"
|
||||
conf.Server.ArtistArtPriority = "artist.*, external"
|
||||
conf.Server.EnableM3UExternalAlbumArt = false
|
||||
})
|
||||
|
||||
It("bills one call per agent, since the walk only stops early on a hit", func() {
|
||||
Expect(ExternalLookupsPerItem(model.KindArtistArtwork, count)).To(Equal(int64(3)))
|
||||
Expect(ExternalLookupsPerItem(model.KindAlbumArtwork, count)).To(Equal(int64(2)))
|
||||
})
|
||||
|
||||
It("bills a playlist for every album its grid samples", func() {
|
||||
Expect(ExternalLookupsPerItem(model.KindPlaylistArtwork, count)).
|
||||
To(Equal(int64(PlaylistGridSamples) * 2))
|
||||
})
|
||||
|
||||
It("adds the m3u image fetch on top of the grid", func() {
|
||||
conf.Server.EnableM3UExternalAlbumArt = true
|
||||
Expect(ExternalLookupsPerItem(model.KindPlaylistArtwork, count)).
|
||||
To(Equal(int64(PlaylistGridSamples)*2 + 1))
|
||||
})
|
||||
|
||||
It("bills only the m3u fetch when the album chain stays local", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*"
|
||||
conf.Server.EnableM3UExternalAlbumArt = true
|
||||
Expect(ExternalLookupsPerItem(model.KindPlaylistArtwork, count)).To(Equal(int64(1)))
|
||||
})
|
||||
|
||||
It("still bills a call when no agent is visible, which plugins never are offline", func() {
|
||||
none := ImageAgentCount{}
|
||||
Expect(ExternalLookupsPerItem(model.KindArtistArtwork, none)).To(Equal(int64(1)))
|
||||
Expect(ExternalLookupsPerItem(model.KindAlbumArtwork, none)).To(Equal(int64(1)))
|
||||
Expect(ExternalLookupsPerItem(model.KindPlaylistArtwork, none)).
|
||||
To(Equal(int64(PlaylistGridSamples)))
|
||||
})
|
||||
|
||||
It("is zero whenever the kind reaches no agent at all", func() {
|
||||
conf.Server.CoverArtPriority = "cover.*"
|
||||
conf.Server.ArtistArtPriority = "artist.*"
|
||||
Expect(ExternalLookupsPerItem(model.KindArtistArtwork, count)).To(BeZero())
|
||||
Expect(ExternalLookupsPerItem(model.KindAlbumArtwork, count)).To(BeZero())
|
||||
Expect(ExternalLookupsPerItem(model.KindPlaylistArtwork, count)).To(BeZero())
|
||||
Expect(ExternalLookupsPerItem(model.KindRadioArtwork, count)).To(BeZero())
|
||||
})
|
||||
})
|
||||
|
||||
@ -27,23 +27,6 @@ import (
|
||||
// to open it is not evidence the entity has no artwork, so callers must not settle on absent.
|
||||
var errSourceUnreadable = errors.New("artwork source unreadable")
|
||||
|
||||
func selectImageReader(ctx context.Context, artID model.ArtworkID, extractFuncs ...sourceFunc) (io.ReadCloser, string, error) {
|
||||
for _, f := range extractFuncs {
|
||||
if ctx.Err() != nil {
|
||||
return nil, "", ctx.Err()
|
||||
}
|
||||
start := time.Now()
|
||||
r, path, err := f()
|
||||
if r != nil {
|
||||
msg := fmt.Sprintf("Artwork: Found %s artwork", artID.Kind)
|
||||
log.Debug(ctx, msg, "artID", artID, "path", path, "source", f, "elapsed", time.Since(start))
|
||||
return r, path, nil
|
||||
}
|
||||
log.Trace(ctx, "Artwork: Failed trying to extract artwork", "artID", artID, "source", f, "elapsed", time.Since(start), err)
|
||||
}
|
||||
return nil, "", fmt.Errorf("could not get `%s` cover art for %s: %w", artID.Kind, artID, ErrUnavailable)
|
||||
}
|
||||
|
||||
type sourceFunc func() (r io.ReadCloser, path string, err error)
|
||||
|
||||
func (f sourceFunc) String() string {
|
||||
|
||||
99
core/artwork/trace.go
Normal file
99
core/artwork/trace.go
Normal file
@ -0,0 +1,99 @@
|
||||
package artwork
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"slices"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Outcome is what the priority chain observed for one candidate; the CLI renders and branches on these.
|
||||
type Outcome string
|
||||
|
||||
const (
|
||||
OutcomeHit Outcome = "hit"
|
||||
OutcomeMiss Outcome = "miss"
|
||||
OutcomeUnreadable Outcome = "unreadable"
|
||||
OutcomeSkipped Outcome = "skipped"
|
||||
OutcomeWouldTry Outcome = "would-try"
|
||||
OutcomeError Outcome = "error"
|
||||
)
|
||||
|
||||
const (
|
||||
// externalCandidate labels the external tier itself, for the cases that never reach an agent.
|
||||
externalCandidate = "external"
|
||||
// ExternalPrefix qualifies a candidate or a stored source with the agent that produced it.
|
||||
ExternalPrefix = externalCandidate + ":"
|
||||
)
|
||||
|
||||
// TraceStep is one candidate the priority chain considered.
|
||||
type TraceStep struct {
|
||||
Candidate string
|
||||
Outcome Outcome
|
||||
Detail string
|
||||
}
|
||||
|
||||
// ChainTrace collects the walk of a single resolution. The artwork worker never attaches
|
||||
// one; only the CLI does, so resolution stays allocation-free in the hot path.
|
||||
type ChainTrace struct {
|
||||
mu sync.Mutex
|
||||
steps []TraceStep
|
||||
}
|
||||
|
||||
func (t *ChainTrace) add(step TraceStep) {
|
||||
if t == nil {
|
||||
return
|
||||
}
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
t.steps = append(t.steps, step)
|
||||
}
|
||||
|
||||
func (t *ChainTrace) Steps() []TraceStep {
|
||||
if t == nil {
|
||||
return nil
|
||||
}
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
return slices.Clone(t.steps)
|
||||
}
|
||||
|
||||
type traceCtxKey struct{}
|
||||
|
||||
func withTrace(ctx context.Context, t *ChainTrace) context.Context {
|
||||
return context.WithValue(ctx, traceCtxKey{}, t)
|
||||
}
|
||||
|
||||
func traceFrom(ctx context.Context) *ChainTrace {
|
||||
t, _ := ctx.Value(traceCtxKey{}).(*ChainTrace)
|
||||
return t
|
||||
}
|
||||
|
||||
var errOfflineSkipped = errors.New("artwork: external lookup skipped (offline)")
|
||||
|
||||
// tracingGate records each external agent's outcome without changing what the gate returns.
|
||||
func tracingGate(t *ChainTrace, inner gateFunc) gateFunc {
|
||||
return func(name string, f func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error) {
|
||||
r, path, err := inner(name, f)
|
||||
candidate := ExternalPrefix + name
|
||||
switch {
|
||||
case r != nil:
|
||||
t.add(TraceStep{Candidate: candidate, Outcome: OutcomeHit, Detail: path})
|
||||
case isTransientExternal(err):
|
||||
t.add(TraceStep{Candidate: candidate, Outcome: OutcomeError, Detail: err.Error()})
|
||||
default:
|
||||
t.add(TraceStep{Candidate: candidate, Outcome: OutcomeMiss})
|
||||
}
|
||||
return r, path, err
|
||||
}
|
||||
}
|
||||
|
||||
// offlineGate reports which agents would be asked without asking them, so a diagnostic
|
||||
// command cannot add load to a provider that is already rate-limiting us.
|
||||
func offlineGate(t *ChainTrace) gateFunc {
|
||||
return func(name string, _ func() (io.ReadCloser, string, error)) (io.ReadCloser, string, error) {
|
||||
t.add(TraceStep{Candidate: ExternalPrefix + name, Outcome: OutcomeWouldTry})
|
||||
return nil, "", errOfflineSkipped
|
||||
}
|
||||
}
|
||||
628
core/artwork/trace_test.go
Normal file
628
core/artwork/trace_test.go
Normal file
@ -0,0 +1,628 @@
|
||||
package artwork
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
"github.com/navidrome/navidrome/conf/configtest"
|
||||
"github.com/navidrome/navidrome/consts"
|
||||
"github.com/navidrome/navidrome/core/agents"
|
||||
"github.com/navidrome/navidrome/model"
|
||||
"github.com/navidrome/navidrome/tests"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("trace vocabulary", func() {
|
||||
// The CLI renders these verbatim and branches on them; a value change is a change to
|
||||
// what `artwork explain` tells an operator, so it must be made deliberately.
|
||||
It("pins the wire values the CLI reads", func() {
|
||||
Expect([]Outcome{
|
||||
OutcomeHit, OutcomeMiss, OutcomeUnreadable, OutcomeSkipped, OutcomeWouldTry, OutcomeError,
|
||||
}).To(Equal([]Outcome{"hit", "miss", "unreadable", "skipped", "would-try", "error"}))
|
||||
Expect(externalCandidate).To(Equal("external"))
|
||||
Expect(ExternalPrefix).To(Equal("external:"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("chainTrace", func() {
|
||||
It("returns nil when no trace is attached", func() {
|
||||
Expect(traceFrom(context.Background())).To(BeNil())
|
||||
})
|
||||
|
||||
It("collects steps in order", func() {
|
||||
t := &ChainTrace{}
|
||||
ctx := withTrace(context.Background(), t)
|
||||
|
||||
traceFrom(ctx).add(TraceStep{Candidate: "cover.*", Outcome: OutcomeMiss})
|
||||
traceFrom(ctx).add(TraceStep{Candidate: "embedded", Outcome: OutcomeHit, Detail: "/music/a.flac"})
|
||||
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "cover.*", Outcome: OutcomeMiss},
|
||||
{Candidate: "embedded", Outcome: OutcomeHit, Detail: "/music/a.flac"},
|
||||
}))
|
||||
|
||||
s := t.Steps()
|
||||
s[0].Candidate = "mutated"
|
||||
Expect(t.Steps()[0].Candidate).To(Equal("cover.*"))
|
||||
})
|
||||
|
||||
It("does not panic when the trace is nil", func() {
|
||||
var t *ChainTrace
|
||||
Expect(func() { t.add(TraceStep{Candidate: "cover.*", Outcome: OutcomeMiss}) }).ToNot(Panic())
|
||||
Expect(t.Steps()).To(BeEmpty(), "a nil trace collects nothing, so reading it must be as safe as writing it")
|
||||
})
|
||||
|
||||
It("is safe to use concurrently", func() {
|
||||
t := &ChainTrace{}
|
||||
done := make(chan struct{})
|
||||
for range 10 {
|
||||
go func() {
|
||||
defer GinkgoRecover()
|
||||
t.add(TraceStep{Candidate: "x", Outcome: OutcomeMiss})
|
||||
done <- struct{}{}
|
||||
}()
|
||||
}
|
||||
for range 10 {
|
||||
<-done
|
||||
}
|
||||
Expect(t.Steps()).To(HaveLen(10))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("chainState tracing", func() {
|
||||
It("records a miss when the candidate was absent", func() {
|
||||
t := &ChainTrace{}
|
||||
c := chainState{trace: t}
|
||||
|
||||
_, ok := c.try("cover.*", resolution{}, false)
|
||||
|
||||
Expect(ok).To(BeFalse())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{{Candidate: "cover.*", Outcome: OutcomeMiss}}))
|
||||
})
|
||||
|
||||
It("records unreadable when the candidate existed but could not be read", func() {
|
||||
t := &ChainTrace{}
|
||||
c := chainState{trace: t}
|
||||
|
||||
_, ok := c.try("cover.*", resolution{localError: true}, false)
|
||||
|
||||
Expect(ok).To(BeFalse())
|
||||
Expect(t.Steps()).To(HaveLen(1))
|
||||
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeUnreadable),
|
||||
"a candidate that existed and failed to decode must be distinguishable from one that was absent")
|
||||
})
|
||||
|
||||
It("records a hit with the backing path", func() {
|
||||
t := &ChainTrace{}
|
||||
c := chainState{trace: t}
|
||||
|
||||
res, ok := c.try("embedded", resolution{reader: nil, source: "embedded", sourcePath: "/music/a.flac"}, true)
|
||||
|
||||
Expect(ok).To(BeTrue())
|
||||
Expect(res.source).To(Equal("embedded"))
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "embedded", Outcome: OutcomeHit, Detail: "/music/a.flac"},
|
||||
}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("external gate tracing", func() {
|
||||
hit := func() (io.ReadCloser, string, error) {
|
||||
return io.NopCloser(strings.NewReader("x")), "http://img", nil
|
||||
}
|
||||
miss := func() (io.ReadCloser, string, error) { return nil, "", agents.ErrNotFound }
|
||||
boom := func() (io.ReadCloser, string, error) { return nil, "", errors.New("returned status 429") }
|
||||
|
||||
It("records a hit with the image path", func() {
|
||||
t := &ChainTrace{}
|
||||
g := tracingGate(t, passthroughGate)
|
||||
|
||||
r, _, err := g("deezer", hit)
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(r).ToNot(BeNil())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "external:deezer", Outcome: OutcomeHit, Detail: "http://img"},
|
||||
}))
|
||||
})
|
||||
|
||||
It("records a miss for a not-found", func() {
|
||||
t := &ChainTrace{}
|
||||
_, _, _ = tracingGate(t, passthroughGate)("deezer", miss)
|
||||
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeMiss))
|
||||
})
|
||||
|
||||
It("records a miss for a model not-found", func() {
|
||||
t := &ChainTrace{}
|
||||
notFound := func() (io.ReadCloser, string, error) { return nil, "", model.ErrNotFound }
|
||||
_, _, _ = tracingGate(t, passthroughGate)("deezer", notFound)
|
||||
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeMiss),
|
||||
"both not-found flavours are definitive answers, not faults")
|
||||
})
|
||||
|
||||
It("records an error with its reason", func() {
|
||||
t := &ChainTrace{}
|
||||
_, _, _ = tracingGate(t, passthroughGate)("apple-music", boom)
|
||||
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeError))
|
||||
Expect(t.Steps()[0].Detail).To(ContainSubstring("429"))
|
||||
})
|
||||
|
||||
It("never calls the agent in offline mode", func() {
|
||||
t := &ChainTrace{}
|
||||
called := false
|
||||
counting := func() (io.ReadCloser, string, error) {
|
||||
called = true
|
||||
return hit()
|
||||
}
|
||||
|
||||
_, _, err := offlineGate(t)("deezer", counting)
|
||||
|
||||
Expect(called).To(BeFalse(), "offline mode must not perform external requests")
|
||||
Expect(err).To(MatchError(errOfflineSkipped))
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "external:deezer", Outcome: OutcomeWouldTry},
|
||||
}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("resolveAlbum tracing", func() {
|
||||
var (
|
||||
ctx context.Context
|
||||
ds *tests.MockDataStore
|
||||
albumRepo *tests.MockAlbumRepo
|
||||
folderRepo *fakeFolderRepo
|
||||
ffm *tests.MockFFmpeg
|
||||
ag *agents.Agents
|
||||
t *ChainTrace
|
||||
)
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.CoverArtPriority = "cover.jpg, embedded"
|
||||
repoRoot, err := os.Getwd()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
libRepo := &tests.MockLibraryRepo{}
|
||||
libRepo.SetData(model.Libraries{{ID: 0, Path: testFileLibPath(repoRoot)}})
|
||||
albumRepo = tests.CreateMockAlbumRepo()
|
||||
folderRepo = &fakeFolderRepo{}
|
||||
ds = &tests.MockDataStore{
|
||||
MockedAlbum: albumRepo,
|
||||
MockedFolder: folderRepo,
|
||||
MockedLibrary: libRepo,
|
||||
}
|
||||
ffm = tests.NewMockFFmpeg("")
|
||||
ag = agents.GetAgents(&tests.MockDataStore{}, nil)
|
||||
t = &ChainTrace{}
|
||||
ctx = withTrace(context.Background(), t)
|
||||
})
|
||||
|
||||
It("records a pattern skipped because the album folder holds no images", func() {
|
||||
albumRepo.SetData(model.Albums{
|
||||
{ID: "al1", Name: "Album", EmbedArtPath: "tests/fixtures/artist/an-album/test.mp3", FolderIDs: []string{"f1"}},
|
||||
})
|
||||
|
||||
res, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al1"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.reader).ToNot(BeNil())
|
||||
defer res.reader.Close()
|
||||
Expect(t.Steps()).To(HaveLen(2), "a configured pattern must appear even when the chain never evaluated it")
|
||||
Expect(t.Steps()[0]).To(Equal(TraceStep{
|
||||
Candidate: "cover.jpg", Outcome: OutcomeSkipped, Detail: "no images in album folder",
|
||||
}))
|
||||
Expect(t.Steps()[1].Candidate).To(Equal("embedded"))
|
||||
Expect(t.Steps()[1].Outcome).To(Equal(OutcomeHit))
|
||||
})
|
||||
|
||||
It("records an evaluated pattern that matched nothing as a miss, not a skip", func() {
|
||||
folderRepo.result = []model.Folder{{
|
||||
Path: "tests/fixtures/artist/an-album",
|
||||
ImageFiles: []string{"artist.png"},
|
||||
}}
|
||||
albumRepo.SetData(model.Albums{
|
||||
{ID: "al3", Name: "Album", EmbedArtPath: "tests/fixtures/artist/an-album/test.mp3", FolderIDs: []string{"f1"}},
|
||||
})
|
||||
|
||||
res, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al3"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.reader).ToNot(BeNil())
|
||||
defer res.reader.Close()
|
||||
Expect(t.Steps()[0]).To(Equal(TraceStep{Candidate: "cover.jpg", Outcome: OutcomeMiss}),
|
||||
"the folder was searched and held no cover.jpg, which is not the same as never looking")
|
||||
})
|
||||
|
||||
It("ignores an empty priority token", func() {
|
||||
conf.Server.CoverArtPriority = "cover.jpg,"
|
||||
albumRepo.SetData(model.Albums{{ID: "al2", Name: "Album", FolderIDs: []string{"f1"}}})
|
||||
|
||||
_, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al2"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "cover.jpg", Outcome: OutcomeSkipped, Detail: "no images in album folder"},
|
||||
}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("resolveArtist tracing", func() {
|
||||
var (
|
||||
ctx context.Context
|
||||
ds *tests.MockDataStore
|
||||
artistRepo *tests.MockArtistRepo
|
||||
albumRepo *tests.MockAlbumRepo
|
||||
folderRepo *fakeFolderRepo
|
||||
ffm *tests.MockFFmpeg
|
||||
ag *agents.Agents
|
||||
t *ChainTrace
|
||||
repoRoot string
|
||||
)
|
||||
|
||||
uploadPath := func(file string) string {
|
||||
path := model.UploadedImagePath(consts.EntityArtist, file)
|
||||
Expect(os.MkdirAll(filepath.Dir(path), 0o755)).To(Succeed())
|
||||
Expect(os.WriteFile(path, []byte("uploaded artist image"), 0o600)).To(Succeed())
|
||||
return path
|
||||
}
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.DataFolder = conf.NewDir(GinkgoT().TempDir())
|
||||
conf.Server.ArtistArtPriority = "album/artist.*"
|
||||
var err error
|
||||
repoRoot, err = os.Getwd()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
libRepo := &tests.MockLibraryRepo{}
|
||||
libRepo.SetData(model.Libraries{{ID: 0, Path: testFileLibPath(repoRoot)}})
|
||||
artistRepo = tests.CreateMockArtistRepo()
|
||||
albumRepo = tests.CreateMockAlbumRepo()
|
||||
folderRepo = &fakeFolderRepo{}
|
||||
ds = &tests.MockDataStore{
|
||||
MockedArtist: artistRepo,
|
||||
MockedAlbum: albumRepo,
|
||||
MockedFolder: folderRepo,
|
||||
MockedLibrary: libRepo,
|
||||
}
|
||||
ffm = tests.NewMockFFmpeg("")
|
||||
ag = agents.GetAgents(&tests.MockDataStore{}, nil)
|
||||
t = &ChainTrace{}
|
||||
ctx = withTrace(context.Background(), t)
|
||||
})
|
||||
|
||||
It("records the upload short-circuit as a hit", func() {
|
||||
path := uploadPath("ar1_test.jpg")
|
||||
artistRepo.SetData(model.Artists{{ID: "ar1", Name: "Artist", UploadedImage: "ar1_test.jpg"}})
|
||||
|
||||
res, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar1"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.reader).ToNot(BeNil())
|
||||
defer res.reader.Close()
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{{Candidate: "upload", Outcome: OutcomeHit, Detail: path}}))
|
||||
})
|
||||
|
||||
It("records an upload miss before walking the chain", func() {
|
||||
artistRepo.SetData(model.Artists{{ID: "ar2", Name: "Artist"}})
|
||||
|
||||
_, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar2"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(t.Steps()[0]).To(Equal(TraceStep{Candidate: "upload", Outcome: OutcomeMiss}))
|
||||
})
|
||||
|
||||
It("labels each step with the configured priority token", func() {
|
||||
folderRepo.result = []model.Folder{{
|
||||
LibraryPath: testFileLibPath(repoRoot),
|
||||
Path: "tests/fixtures/artist/an-album",
|
||||
ImageFiles: []string{"artist.png"},
|
||||
}}
|
||||
artistRepo.SetData(model.Artists{{ID: "ar4", Name: "Artist"}})
|
||||
albumRepo.All = model.Albums{{ID: "al9", Name: "Album", LibraryID: 0, FolderIDs: []string{"f1"}}}
|
||||
|
||||
res, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar4"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.reader).ToNot(BeNil())
|
||||
defer res.reader.Close()
|
||||
Expect(t.Steps()).To(HaveLen(2))
|
||||
Expect(t.Steps()[1].Candidate).To(Equal("album/artist.*"),
|
||||
"the step must be labelled with the priority token, not the pattern it was rewritten into")
|
||||
Expect(t.Steps()[1].Outcome).To(Equal(OutcomeHit))
|
||||
Expect(filepath.ToSlash(t.Steps()[1].Detail)).To(HaveSuffix("tests/fixtures/artist/an-album/artist.png"))
|
||||
})
|
||||
|
||||
It("records a configured pattern that could not be evaluated", func() {
|
||||
artistRepo.SetData(model.Artists{{ID: "ar5", Name: "Artist"}})
|
||||
|
||||
_, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar5"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "upload", Outcome: OutcomeMiss},
|
||||
{Candidate: "album/artist.*", Outcome: OutcomeSkipped, Detail: "artist has no albums"},
|
||||
}), "a configured pattern that was never evaluated must still appear, and say why")
|
||||
})
|
||||
|
||||
It("records why an artist folder pattern was skipped", func() {
|
||||
conf.Server.ArtistArtPriority = "artist.*"
|
||||
artistRepo.SetData(model.Artists{{ID: "ar6", Name: "Artist"}})
|
||||
albumRepo.All = model.Albums{{ID: "al10", Name: "Album", LibraryID: 0, FolderIDs: []string{"f1"}}}
|
||||
|
||||
_, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar6"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "upload", Outcome: OutcomeMiss},
|
||||
{Candidate: "artist.*", Outcome: OutcomeSkipped, Detail: "no artist folder"},
|
||||
}))
|
||||
})
|
||||
|
||||
It("records an upload that exists but cannot be read as unreadable", func() {
|
||||
if runtime.GOOS == "windows" {
|
||||
Skip("chmod does not restrict read access on Windows")
|
||||
}
|
||||
path := uploadPath("ar3_test.jpg")
|
||||
Expect(os.Chmod(path, 0o000)).To(Succeed())
|
||||
DeferCleanup(func() { _ = os.Chmod(path, 0o600) })
|
||||
artistRepo.SetData(model.Artists{{ID: "ar3", Name: "Artist", UploadedImage: "ar3_test.jpg"}})
|
||||
|
||||
_, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar3"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(t.Steps()).To(HaveLen(1))
|
||||
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeUnreadable),
|
||||
"an upload that exists and will not open must not look like an absent upload")
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("NewTracingResolver", func() {
|
||||
var (
|
||||
ds *tests.MockDataStore
|
||||
albumRepo *tests.MockAlbumRepo
|
||||
artistRepo *tests.MockArtistRepo
|
||||
artworkRepo *tests.MockArtworkRepo
|
||||
queueRepo *tests.MockArtworkQueueRepo
|
||||
ffm *tests.MockFFmpeg
|
||||
t *ChainTrace
|
||||
)
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.DataFolder = conf.NewDir(GinkgoT().TempDir())
|
||||
conf.Server.CoverArtPriority = "external, embedded"
|
||||
conf.Server.ArtistArtPriority = "external"
|
||||
repoRoot, err := os.Getwd()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
libRepo := &tests.MockLibraryRepo{}
|
||||
libRepo.SetData(model.Libraries{{ID: 0, Path: testFileLibPath(repoRoot)}})
|
||||
albumRepo = tests.CreateMockAlbumRepo()
|
||||
artistRepo = tests.CreateMockArtistRepo()
|
||||
artworkRepo = tests.CreateMockArtworkRepo()
|
||||
queueRepo = tests.CreateMockArtworkQueueRepo()
|
||||
ds = &tests.MockDataStore{
|
||||
MockedAlbum: albumRepo,
|
||||
MockedArtist: artistRepo,
|
||||
MockedFolder: &fakeFolderRepo{},
|
||||
MockedLibrary: libRepo,
|
||||
MockedArtwork: artworkRepo,
|
||||
MockedArtworkQueue: queueRepo,
|
||||
}
|
||||
ffm = tests.NewMockFFmpeg("")
|
||||
t = &ChainTrace{}
|
||||
})
|
||||
|
||||
Context("offline", func() {
|
||||
var fake *fakeImageAgent
|
||||
|
||||
BeforeEach(func() {
|
||||
fake = &fakeImageAgent{name: "offline-probe"}
|
||||
albumRepo.SetData(model.Albums{{
|
||||
ID: "al1", Name: "Album", EmbedArtPath: "tests/fixtures/artist/an-album/test.mp3", FolderIDs: []string{"f1"},
|
||||
}})
|
||||
artistRepo.SetData(model.Artists{{ID: "ar1", Name: "Artist"}})
|
||||
})
|
||||
|
||||
It("reports the external tier without asking any agent", func() {
|
||||
source, err := NewTracingResolver(ds, imageAgents(fake), ffm, t, false).Resolve(context.Background(), model.KindAlbumArtwork, "al1")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(source).To(Equal("embedded"))
|
||||
Expect(fake.albumCalls).To(BeZero(), "offline mode must not add load to an external provider")
|
||||
Expect(t.Steps()).To(ContainElement(TraceStep{Candidate: "external:offline-probe", Outcome: OutcomeWouldTry}))
|
||||
})
|
||||
|
||||
It("records the local chain steps too", func() {
|
||||
_, err := NewTracingResolver(ds, imageAgents(fake), ffm, t, false).Resolve(context.Background(), model.KindAlbumArtwork, "al1")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
last := t.Steps()[len(t.Steps())-1]
|
||||
Expect(last.Candidate).To(Equal("embedded"), "the local chain must be traced, not just the external gate")
|
||||
Expect(last.Outcome).To(Equal(OutcomeHit))
|
||||
})
|
||||
|
||||
It("never persists artwork state", func() {
|
||||
_, err := NewTracingResolver(ds, imageAgents(fake), ffm, t, false).Resolve(context.Background(), model.KindAlbumArtwork, "al1")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(artworkRepo.ItemData).To(BeEmpty(),
|
||||
"an offline resolution carries extError, which must never be recorded as a real provider failure")
|
||||
Expect(queueRepo.Data).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("resolves an artist without persisting anything", func() {
|
||||
source, err := NewTracingResolver(ds, imageAgents(fake), ffm, t, false).Resolve(context.Background(), model.KindArtistArtwork, "ar1")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(source).To(BeEmpty())
|
||||
Expect(fake.artistCalls).To(BeZero())
|
||||
Expect(t.Steps()).To(ContainElement(TraceStep{Candidate: "external:offline-probe", Outcome: OutcomeWouldTry}))
|
||||
Expect(artworkRepo.ItemData).To(BeEmpty())
|
||||
Expect(queueRepo.Data).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("closes the reader it does not hand back", func() {
|
||||
conf.Server.CoverArtPriority = "embedded"
|
||||
ffm = tests.NewMockFFmpeg("fake image bytes")
|
||||
albumRepo.SetData(model.Albums{{
|
||||
ID: "al2", Name: "Album", EmbedArtPath: "tests/fixtures/artist/an-album/no-such-file.mp3", FolderIDs: []string{"f1"},
|
||||
}})
|
||||
|
||||
source, err := NewTracingResolver(ds, imageAgents(fake), ffm, t, false).Resolve(context.Background(), model.KindAlbumArtwork, "al2")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(source).To(Equal("embedded"))
|
||||
Expect(ffm.IsClosed()).To(BeTrue(), "nothing downstream closes it, so a leak is one file handle per invocation")
|
||||
})
|
||||
|
||||
It("propagates a lookup error", func() {
|
||||
_, err := NewTracingResolver(ds, imageAgents(fake), ffm, t, false).Resolve(context.Background(), model.KindAlbumArtwork, "nope")
|
||||
Expect(err).To(MatchError(model.ErrNotFound))
|
||||
})
|
||||
})
|
||||
|
||||
It("asks the agents when live is true", func() {
|
||||
fake := &fakeImageAgent{name: "live-probe", err: agents.ErrNotFound}
|
||||
albumRepo.SetData(model.Albums{{
|
||||
ID: "al1", Name: "Album", EmbedArtPath: "tests/fixtures/artist/an-album/test.mp3", FolderIDs: []string{"f1"},
|
||||
}})
|
||||
|
||||
_, err := NewTracingResolver(ds, imageAgents(fake), ffm, t, true).Resolve(context.Background(), model.KindAlbumArtwork, "al1")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(fake.albumCalls).To(Equal(1))
|
||||
Expect(t.Steps()).To(ContainElement(TraceStep{Candidate: "external:live-probe", Outcome: OutcomeMiss}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("resolveDisc tracing", func() {
|
||||
var (
|
||||
ctx context.Context
|
||||
ds *tests.MockDataStore
|
||||
albumRepo *tests.MockAlbumRepo
|
||||
folderRepo *fakeFolderRepo
|
||||
ffm *tests.MockFFmpeg
|
||||
t *ChainTrace
|
||||
)
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
repoRoot, err := os.Getwd()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
libRepo := &tests.MockLibraryRepo{}
|
||||
libRepo.SetData(model.Libraries{{ID: 0, Path: testFileLibPath(repoRoot)}})
|
||||
albumRepo = tests.CreateMockAlbumRepo()
|
||||
albumRepo.SetData(model.Albums{{ID: "al1", Name: "Album", FolderIDs: []string{"f1"}}})
|
||||
folderRepo = &fakeFolderRepo{}
|
||||
mfRepo := tests.CreateMockMediaFileRepo()
|
||||
mfRepo.SetData(model.MediaFiles{{ID: "mf1", AlbumID: "al1", DiscNumber: 2, Path: "tests/fixtures/artist/an-album/test.mp3"}})
|
||||
ds = &tests.MockDataStore{
|
||||
MockedAlbum: albumRepo,
|
||||
MockedMediaFile: mfRepo,
|
||||
MockedFolder: folderRepo,
|
||||
MockedLibrary: libRepo,
|
||||
}
|
||||
ffm = tests.NewMockFFmpeg("")
|
||||
t = &ChainTrace{}
|
||||
ctx = withTrace(context.Background(), t)
|
||||
})
|
||||
|
||||
It("accounts for every configured entry, including the ones that map to no source", func() {
|
||||
conf.Server.DiscArtPriority = "external, discsubtitle, cover.jpg"
|
||||
|
||||
res, err := newResolver(ds, nil, ffm, nil).resolveDisc(ctx, model.DiscArtworkID("al1", 2))
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.reader).To(BeNil())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "external", Outcome: OutcomeSkipped, Detail: "external sources are not supported for disc artwork"},
|
||||
{Candidate: "discsubtitle", Outcome: OutcomeSkipped, Detail: "disc has no subtitle"},
|
||||
{Candidate: "cover.jpg", Outcome: OutcomeSkipped, Detail: "no images in album folder"},
|
||||
}))
|
||||
})
|
||||
|
||||
It("records the entry that won and stops there", func() {
|
||||
conf.Server.DiscArtPriority = "disc*.*, cover.jpg, embedded"
|
||||
folderRepo.result = []model.Folder{{
|
||||
Path: "tests/fixtures/artist/an-album",
|
||||
ImageFiles: []string{"cover.jpg"},
|
||||
}}
|
||||
|
||||
res, err := newResolver(ds, nil, ffm, nil).resolveDisc(ctx, model.DiscArtworkID("al1", 2))
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.reader).ToNot(BeNil())
|
||||
defer res.reader.Close()
|
||||
Expect(res.source).To(Equal("folder"))
|
||||
Expect(t.Steps()).To(HaveLen(2), "the walk must stop at the winner, and record nothing below it")
|
||||
Expect(t.Steps()[0]).To(Equal(TraceStep{Candidate: "disc*.*", Outcome: OutcomeMiss}))
|
||||
Expect(t.Steps()[1].Candidate).To(Equal("cover.jpg"))
|
||||
Expect(t.Steps()[1].Outcome).To(Equal(OutcomeHit))
|
||||
Expect(t.Steps()[1].Detail).To(HaveSuffix(filepath.FromSlash("tests/fixtures/artist/an-album/cover.jpg")))
|
||||
Expect(t.Steps()[1].Detail).ToNot(Equal("tests/fixtures/artist/an-album/cover.jpg"),
|
||||
"a library-relative path sends the operator looking in the wrong place")
|
||||
})
|
||||
|
||||
It("reports an unparseable disc id rather than explaining another disc", func() {
|
||||
conf.Server.DiscArtPriority = "cover.jpg"
|
||||
_, err := newResolver(ds, nil, ffm, nil).resolveDisc(ctx, "al1")
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("resolveMediaFile tracing", func() {
|
||||
var (
|
||||
ctx context.Context
|
||||
ds *tests.MockDataStore
|
||||
mfRepo *tests.MockMediaFileRepo
|
||||
ffm *tests.MockFFmpeg
|
||||
t *ChainTrace
|
||||
)
|
||||
|
||||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
conf.Server.EnableMediaFileCoverArt = true
|
||||
repoRoot, err := os.Getwd()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
libRepo := &tests.MockLibraryRepo{}
|
||||
libRepo.SetData(model.Libraries{{ID: 0, Path: testFileLibPath(repoRoot)}})
|
||||
mfRepo = tests.CreateMockMediaFileRepo()
|
||||
mfRepo.SetData(model.MediaFiles{{
|
||||
ID: "mf1", Title: "Song", HasCoverArt: true, Path: "tests/fixtures/artist/an-album/test.mp3",
|
||||
}})
|
||||
ds = &tests.MockDataStore{MockedMediaFile: mfRepo, MockedLibrary: libRepo}
|
||||
ffm = tests.NewMockFFmpeg("")
|
||||
t = &ChainTrace{}
|
||||
ctx = withTrace(context.Background(), t)
|
||||
})
|
||||
|
||||
It("separates a disabled setting from a track with nothing embedded", func() {
|
||||
conf.Server.EnableMediaFileCoverArt = false
|
||||
|
||||
_, err := newResolver(ds, nil, ffm, nil).resolveMediaFile(ctx, "mf1")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "embedded", Outcome: OutcomeSkipped, Detail: "EnableMediaFileCoverArt is off"},
|
||||
}))
|
||||
})
|
||||
|
||||
It("records a track with no embedded art as a miss", func() {
|
||||
mfRepo.SetData(model.MediaFiles{{ID: "mf2", Title: "Song", HasCoverArt: false}})
|
||||
|
||||
_, err := newResolver(ds, nil, ffm, nil).resolveMediaFile(ctx, "mf2")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(t.Steps()).To(Equal([]TraceStep{
|
||||
{Candidate: "embedded", Outcome: OutcomeMiss, Detail: "the track has no embedded cover art"},
|
||||
}))
|
||||
})
|
||||
|
||||
It("records the embedded hit", func() {
|
||||
res, err := newResolver(ds, nil, ffm, nil).resolveMediaFile(ctx, "mf1")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(res.reader).ToNot(BeNil())
|
||||
defer res.reader.Close()
|
||||
Expect(res.source).To(Equal("embedded"))
|
||||
Expect(t.Steps()).To(HaveLen(1))
|
||||
Expect(t.Steps()[0].Outcome).To(Equal(OutcomeHit))
|
||||
})
|
||||
})
|
||||
@ -137,7 +137,7 @@ func (w *Worker) Backfill(ctx context.Context) (bool, error) {
|
||||
return backfill(ctx, w.proc.ds)
|
||||
}
|
||||
|
||||
// EnqueueStaleAbsentAll requeues known-absent entries older than staleAbsentAge.
|
||||
// EnqueueStaleAbsentAll requeues known-absent entries older than StaleAbsentAge.
|
||||
func (w *Worker) EnqueueStaleAbsentAll(ctx context.Context) error {
|
||||
return enqueueStaleAbsentAll(ctx, w.proc.ds)
|
||||
}
|
||||
|
||||
@ -362,7 +362,7 @@ var _ = Describe("Worker", func() {
|
||||
Expect(ia.Hash).To(Equal("cafebabe"), "a persistent outage must not discard served art")
|
||||
})
|
||||
|
||||
// Media files are excluded from recheckKinds, so an absent row here would never be
|
||||
// Media files are excluded from RecheckKinds, so an absent row here would never be
|
||||
// revisited: a transient read error would look permanent.
|
||||
It("does not settle absent on exhaustion for a kind with no recheck path", func() {
|
||||
conf.Server.EnableMediaFileCoverArt = true
|
||||
|
||||
@ -119,6 +119,8 @@ type ArtworkRepository interface {
|
||||
}
|
||||
|
||||
type ArtworkQueueRepository interface {
|
||||
// Get returns the pending row for an item, or ErrNotFound when it is not queued.
|
||||
Get(kind Kind, id, imageType string) (*ArtworkQueueItem, error)
|
||||
// Enqueue upserts; an existing row keeps the higher priority and has its retry_at reset.
|
||||
Enqueue(items ...ArtworkQueueItem) error
|
||||
// EnqueuePreservingBackoff upserts like Enqueue but preserves an existing row's retry_at, so a
|
||||
@ -130,6 +132,14 @@ type ArtworkQueueRepository interface {
|
||||
EnqueueAllMissing(kind Kind, priority int) (int64, error)
|
||||
// EnqueueIfMissing inserts only for items with no item_artwork row yet.
|
||||
EnqueueIfMissing(items ...ArtworkQueueItem) error
|
||||
// CountBySource reports how many items of a kind currently resolve from the given sources.
|
||||
// An empty sources slice means every source; "" matches absent state.
|
||||
CountBySource(kind Kind, sources []string) (int64, error)
|
||||
// SourcesInUse lists the distinct sources items of a kind currently resolve from, "" included.
|
||||
SourcesInUse(kind Kind) ([]string, error)
|
||||
// EnqueueBySource inserts queue rows for items of a kind whose current source matches.
|
||||
// It does not clear existing artwork state: the current image stays until it is replaced.
|
||||
EnqueueBySource(kind Kind, sources []string, priority int) (int64, error)
|
||||
// DequeueBatch returns up to n items with retry_at <= now, priority desc, enqueued_at asc.
|
||||
// Restricted to the given kinds when any are passed, so one kind cannot block another's drain.
|
||||
DequeueBatch(n int, kinds ...string) ([]ArtworkQueueItem, error)
|
||||
@ -139,6 +149,22 @@ type ArtworkQueueRepository interface {
|
||||
// DeleteIfUnchanged deletes only while retry_at still matches, sparing a concurrent re-enqueue.
|
||||
DeleteIfUnchanged(kind, id, imageType string, retryAt time.Time) error
|
||||
Count() (int64, error)
|
||||
// CountByKindAndPriority reports the pending queue rows grouped by kind and priority.
|
||||
CountByKindAndPriority() ([]ArtworkQueueStat, error)
|
||||
// CountAbsent reports the absent states of a kind, and how many of those EnqueueStaleAbsent
|
||||
// would pick up at the given cutoff.
|
||||
CountAbsent(kind Kind, attemptedBefore time.Time) (ArtworkAbsentStat, error)
|
||||
// PurgeDangling removes queue rows whose entity no longer exists.
|
||||
PurgeDangling() (int64, error)
|
||||
}
|
||||
|
||||
type ArtworkQueueStat struct {
|
||||
ItemKind string
|
||||
Priority int
|
||||
Count int64
|
||||
}
|
||||
|
||||
type ArtworkAbsentStat struct {
|
||||
Total int64
|
||||
Stale int64
|
||||
}
|
||||
|
||||
@ -10,6 +10,7 @@ import (
|
||||
|
||||
. "github.com/Masterminds/squirrel"
|
||||
"github.com/navidrome/navidrome/model"
|
||||
"github.com/navidrome/navidrome/utils/slice"
|
||||
"github.com/pocketbase/dbx"
|
||||
)
|
||||
|
||||
@ -31,6 +32,16 @@ func NewArtworkQueueRepository(ctx context.Context, db dbx.Builder) model.Artwor
|
||||
return r
|
||||
}
|
||||
|
||||
func (r *artworkQueueRepository) Get(kind model.Kind, id, imageType string) (*model.ArtworkQueueItem, error) {
|
||||
var res model.ArtworkQueueItem
|
||||
err := r.queryOne(Select("*").From(r.tableName).
|
||||
Where(Eq{"item_kind": kind.Prefix(), "item_id": id, "image_type": imageType}), &res)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &res, nil
|
||||
}
|
||||
|
||||
// Enqueue also resets enqueued_at, so a fresh request does not inherit an old row's spent retry budget.
|
||||
func (r *artworkQueueRepository) Enqueue(items ...model.ArtworkQueueItem) error {
|
||||
return r.enqueue(`ON CONFLICT (item_kind, item_id, image_type) DO UPDATE SET
|
||||
@ -87,12 +98,49 @@ func (r *artworkQueueRepository) EnqueueIfMissing(items ...model.ArtworkQueueIte
|
||||
return nil
|
||||
}
|
||||
|
||||
// insertIfNotQueued inserts the rows selected by the given SQL, optionally prefixed by a CTE. DO NOTHING is
|
||||
// deliberate: a recheck must not bump the priority or retry_at of an already-queued item.
|
||||
// DO NOTHING is deliberate: a recheck must not bump the priority or retry_at of an already-queued item.
|
||||
const skipIfQueued = ` ON CONFLICT (item_kind, item_id, image_type) DO NOTHING`
|
||||
|
||||
// insertIfNotQueued inserts the rows selected by the given SQL, optionally prefixed by a CTE.
|
||||
func (r *artworkQueueRepository) insertIfNotQueued(with, sql string, args ...any) (int64, error) {
|
||||
return r.executeSQL(Expr(with+`INSERT INTO `+r.tableName+
|
||||
` (`+strings.Join(enqueueColumns, ", ")+`) `+sql+
|
||||
` ON CONFLICT (item_kind, item_id, image_type) DO NOTHING`, args...))
|
||||
` (`+strings.Join(enqueueColumns, ", ")+`) `+sql+skipIfQueued, args...))
|
||||
}
|
||||
|
||||
// artworkSourceFilter selects item_artwork rows of a kind; no sources means every source, "" the absent state.
|
||||
func artworkSourceFilter(kind model.Kind, sources []string) Sqlizer {
|
||||
f := And{Eq{"item_kind": kind.Prefix()}}
|
||||
if len(sources) > 0 {
|
||||
f = append(f, Eq{"source": sources})
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
func (r *artworkQueueRepository) CountBySource(kind model.Kind, sources []string) (int64, error) {
|
||||
var res struct{ Count int64 }
|
||||
err := r.queryOne(Select("count(*) as count").From(itemArtworkTable).
|
||||
Where(artworkSourceFilter(kind, sources)), &res)
|
||||
return res.Count, err
|
||||
}
|
||||
|
||||
func (r *artworkQueueRepository) SourcesInUse(kind model.Kind) ([]string, error) {
|
||||
var res []struct{ Source string }
|
||||
err := r.queryAll(Select("distinct source").From(itemArtworkTable).
|
||||
Where(Eq{"item_kind": kind.Prefix()}), &res)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return slice.Map(res, func(s struct{ Source string }) string { return s.Source }), nil
|
||||
}
|
||||
|
||||
// EnqueueBySource deliberately leaves item_artwork alone: clearing state in bulk would blank the
|
||||
// library's artwork until every item is resolved again.
|
||||
func (r *artworkQueueRepository) EnqueueBySource(kind model.Kind, sources []string, priority int) (int64, error) {
|
||||
now := time.Now()
|
||||
sel := Select("item_kind", "item_id", "image_type").
|
||||
Column(Expr("?", priority)).Column("0").Column(Expr("?", now)).Column(Expr("?", now)).
|
||||
From(itemArtworkTable).Where(artworkSourceFilter(kind, sources))
|
||||
return r.executeSQL(Insert(r.tableName).Columns(enqueueColumns...).Select(sel).Suffix(skipIfQueued))
|
||||
}
|
||||
|
||||
func (r *artworkQueueRepository) enqueue(conflict string, items []model.ArtworkQueueItem) error {
|
||||
@ -146,4 +194,20 @@ func (r *artworkQueueRepository) Count() (int64, error) {
|
||||
return res.Count, err
|
||||
}
|
||||
|
||||
func (r *artworkQueueRepository) CountByKindAndPriority() ([]model.ArtworkQueueStat, error) {
|
||||
var res []model.ArtworkQueueStat
|
||||
err := r.queryAll(Select("item_kind", "priority", "count(*) as count").From(r.tableName).
|
||||
GroupBy("item_kind", "priority").OrderBy("item_kind", "priority desc"), &res)
|
||||
return res, err
|
||||
}
|
||||
|
||||
// CountAbsent matches EnqueueStaleAbsent on hash, so the stale count is what a recheck would queue.
|
||||
func (r *artworkQueueRepository) CountAbsent(kind model.Kind, attemptedBefore time.Time) (model.ArtworkAbsentStat, error) {
|
||||
var res model.ArtworkAbsentStat
|
||||
err := r.queryOne(Select("count(*) as total").
|
||||
Column(Expr("coalesce(sum(attempted_at < ?), 0) as stale", attemptedBefore)).
|
||||
From(itemArtworkTable).Where(Eq{"item_kind": kind.Prefix(), "hash": ""}), &res)
|
||||
return res, err
|
||||
}
|
||||
|
||||
var _ model.ArtworkQueueRepository = (*artworkQueueRepository)(nil)
|
||||
|
||||
@ -54,6 +54,22 @@ var _ = Describe("ArtworkQueueRepository", func() {
|
||||
Expect(got[0].ItemID).To(Equal("high"))
|
||||
})
|
||||
|
||||
It("Get returns a queued row, including one still backing off", func() {
|
||||
Expect(repo.Enqueue(item("ar", "g1", model.ArtworkPriorityScan))).To(Succeed())
|
||||
backOff("ar", "g1", time.Now().Add(time.Hour))
|
||||
|
||||
got, err := repo.Get(model.KindArtistArtwork, "g1", model.ImageTypePrimary)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(got.Priority).To(Equal(model.ArtworkPriorityScan))
|
||||
Expect(got.Attempts).To(Equal(1))
|
||||
Expect(got.RetryAt).To(BeTemporally(">", time.Now()))
|
||||
})
|
||||
|
||||
It("Get reports ErrNotFound when the item is not queued", func() {
|
||||
_, err := repo.Get(model.KindArtistArtwork, "nope", model.ImageTypePrimary)
|
||||
Expect(err).To(MatchError(model.ErrNotFound))
|
||||
})
|
||||
|
||||
It("keeps the higher priority on duplicate enqueue", func() {
|
||||
Expect(repo.Enqueue(item("al", "a1", model.ArtworkPriorityBump))).To(Succeed())
|
||||
Expect(repo.Enqueue(item("al", "a1", model.ArtworkPriorityBackfill))).To(Succeed())
|
||||
@ -265,6 +281,89 @@ var _ = Describe("ArtworkQueueRepository", func() {
|
||||
Expect(got[0].Priority).To(Equal(model.ArtworkPriorityBump), "the existing priority must survive")
|
||||
})
|
||||
|
||||
Describe("EnqueueBySource", func() {
|
||||
BeforeEach(func() {
|
||||
artRepo := NewArtworkRepository(context.Background(), GetDBXBuilder())
|
||||
for _, ia := range []model.ItemArtwork{
|
||||
{ItemKind: "ar", ItemID: "ar1", ImageType: model.ImageTypePrimary, Hash: "h1", Source: "external:deezer"},
|
||||
{ItemKind: "ar", ItemID: "ar2", ImageType: model.ImageTypePrimary, Hash: "h2", Source: "external:lastfm"},
|
||||
{ItemKind: "ar", ItemID: "ar3", ImageType: model.ImageTypePrimary, Hash: "", Source: ""},
|
||||
{ItemKind: "al", ItemID: "al1", ImageType: model.ImageTypePrimary, Hash: "h4", Source: "external:deezer"},
|
||||
} {
|
||||
Expect(artRepo.PutItemArtwork(&ia)).To(Succeed())
|
||||
}
|
||||
})
|
||||
|
||||
It("enqueues only the matching source within the kind", func() {
|
||||
n, err := repo.EnqueueBySource(model.KindArtistArtwork, []string{"external:deezer"}, model.ArtworkPriorityRecheck)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(n).To(Equal(int64(1)), "al1 is a different kind and must not be touched")
|
||||
|
||||
got, err := repo.DequeueBatch(10)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(slice.Map(got, func(it model.ArtworkQueueItem) string { return it.ItemID })).To(ConsistOf("ar1"))
|
||||
})
|
||||
|
||||
It("treats the empty source as absent", func() {
|
||||
n, err := repo.EnqueueBySource(model.KindArtistArtwork, []string{""}, model.ArtworkPriorityRecheck)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(n).To(Equal(int64(1)))
|
||||
|
||||
got, _ := repo.DequeueBatch(10)
|
||||
Expect(slice.Map(got, func(it model.ArtworkQueueItem) string { return it.ItemID })).To(ConsistOf("ar3"))
|
||||
})
|
||||
|
||||
It("enqueues every source when none is given", func() {
|
||||
n, err := repo.EnqueueBySource(model.KindArtistArtwork, nil, model.ArtworkPriorityRecheck)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(n).To(Equal(int64(3)))
|
||||
})
|
||||
|
||||
It("leaves the current artwork state in place", func() {
|
||||
_, err := repo.EnqueueBySource(model.KindArtistArtwork, []string{"external:deezer"}, model.ArtworkPriorityRecheck)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
|
||||
artRepo := NewArtworkRepository(context.Background(), GetDBXBuilder())
|
||||
ia, err := artRepo.GetItemArtwork(model.KindArtistArtwork, "ar1", model.ImageTypePrimary)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(ia.Hash).To(Equal("h1"), "the current image must survive until it is replaced")
|
||||
Expect(ia.Source).To(Equal("external:deezer"))
|
||||
})
|
||||
|
||||
It("does not disturb an already-queued row", func() {
|
||||
Expect(repo.Enqueue(item("ar", "ar1", model.ArtworkPriorityBump))).To(Succeed())
|
||||
|
||||
n, err := repo.EnqueueBySource(model.KindArtistArtwork, []string{"external:deezer"}, model.ArtworkPriorityRecheck)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(n).To(BeZero())
|
||||
|
||||
got, _ := repo.DequeueBatch(10)
|
||||
Expect(got).To(HaveLen(1))
|
||||
Expect(got[0].Priority).To(Equal(model.ArtworkPriorityBump))
|
||||
})
|
||||
|
||||
It("counts without enqueueing", func() {
|
||||
n, err := repo.CountBySource(model.KindArtistArtwork, []string{"external:deezer"})
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(n).To(Equal(int64(1)))
|
||||
|
||||
queued, err := repo.Count()
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(queued).To(BeZero(), "CountBySource must not enqueue")
|
||||
})
|
||||
|
||||
It("counts the absent source and every source", func() {
|
||||
Expect(repo.CountBySource(model.KindArtistArtwork, []string{""})).To(Equal(int64(1)))
|
||||
Expect(repo.CountBySource(model.KindArtistArtwork, nil)).To(Equal(int64(3)))
|
||||
})
|
||||
|
||||
It("lists the distinct sources in use by a kind", func() {
|
||||
Expect(repo.SourcesInUse(model.KindArtistArtwork)).To(ConsistOf("", "external:deezer", "external:lastfm"))
|
||||
Expect(repo.SourcesInUse(model.KindAlbumArtwork)).To(ConsistOf("external:deezer"))
|
||||
Expect(repo.SourcesInUse(model.KindRadioArtwork)).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
It("does not disturb an already-queued entity when enqueueing missing rows", func() {
|
||||
Expect(repo.Enqueue(item("al", albumRadioactivity.ID, model.ArtworkPriorityBump))).To(Succeed())
|
||||
|
||||
@ -281,4 +380,43 @@ var _ = Describe("ArtworkQueueRepository", func() {
|
||||
}
|
||||
Expect(count).To(Equal(1), "the already-queued row must not be duplicated")
|
||||
})
|
||||
|
||||
Describe("status counters", func() {
|
||||
It("groups queue rows by kind and priority", func() {
|
||||
Expect(repo.Enqueue(item("ar", "a1", model.ArtworkPriorityBackfill))).To(Succeed())
|
||||
Expect(repo.Enqueue(item("ar", "a2", model.ArtworkPriorityBackfill))).To(Succeed())
|
||||
Expect(repo.Enqueue(item("ar", "a3", model.ArtworkPriorityBump))).To(Succeed())
|
||||
Expect(repo.Enqueue(item("al", "b1", model.ArtworkPriorityScan))).To(Succeed())
|
||||
|
||||
Expect(repo.CountByKindAndPriority()).To(ConsistOf(
|
||||
model.ArtworkQueueStat{ItemKind: "ar", Priority: model.ArtworkPriorityBackfill, Count: 2},
|
||||
model.ArtworkQueueStat{ItemKind: "ar", Priority: model.ArtworkPriorityBump, Count: 1},
|
||||
model.ArtworkQueueStat{ItemKind: "al", Priority: model.ArtworkPriorityScan, Count: 1},
|
||||
))
|
||||
})
|
||||
|
||||
It("reports an empty queue as no rows", func() {
|
||||
Expect(repo.CountByKindAndPriority()).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("counts absent states and how many are due for recheck", func() {
|
||||
awRepo := NewArtworkRepository(context.Background(), GetDBXBuilder())
|
||||
old := time.Now().Add(-48 * time.Hour)
|
||||
for _, ia := range []model.ItemArtwork{
|
||||
{ItemKind: "ar", ItemID: "stale1", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: old},
|
||||
{ItemKind: "ar", ItemID: "fresh1", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: time.Now()},
|
||||
{ItemKind: "ar", ItemID: "found1", ImageType: model.ImageTypePrimary, Hash: "hX", AttemptedAt: old},
|
||||
{ItemKind: "al", ItemID: "stale2", ImageType: model.ImageTypePrimary, Hash: "", AttemptedAt: old},
|
||||
} {
|
||||
Expect(awRepo.PutItemArtwork(&ia)).To(Succeed())
|
||||
}
|
||||
|
||||
Expect(repo.CountAbsent(model.KindArtistArtwork, time.Now().Add(-24*time.Hour))).
|
||||
To(Equal(model.ArtworkAbsentStat{Total: 2, Stale: 1}))
|
||||
})
|
||||
|
||||
It("reports a kind with no absent state as zero, not as an error", func() {
|
||||
Expect(repo.CountAbsent(model.KindRadioArtwork, time.Now())).To(Equal(model.ArtworkAbsentStat{}))
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@ -10,14 +10,6 @@ import (
|
||||
"github.com/navidrome/navidrome/model"
|
||||
)
|
||||
|
||||
var refreshableArtworkKinds = []model.Kind{
|
||||
model.KindAlbumArtwork,
|
||||
model.KindArtistArtwork,
|
||||
model.KindPlaylistArtwork,
|
||||
model.KindRadioArtwork,
|
||||
model.KindMediaFileArtwork,
|
||||
}
|
||||
|
||||
func (api *Router) addArtworkRoute(r chi.Router) {
|
||||
r.Post("/artwork/{kind}/{id}/refresh", api.refreshArtwork())
|
||||
}
|
||||
@ -28,7 +20,7 @@ func (api *Router) refreshArtwork() http.HandlerFunc {
|
||||
ctx := r.Context()
|
||||
kind, _ := model.ParseKind(chi.URLParam(r, "kind"))
|
||||
id := chi.URLParam(r, "id")
|
||||
if !slices.Contains(refreshableArtworkKinds, kind) {
|
||||
if !slices.Contains(artwork.RefreshableKinds, kind) {
|
||||
http.Error(w, "invalid artwork kind", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
@ -7,6 +7,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/navidrome/navidrome/model"
|
||||
"github.com/navidrome/navidrome/utils/slice"
|
||||
)
|
||||
|
||||
type MockArtworkQueueRepo struct {
|
||||
@ -25,6 +26,19 @@ func CreateMockArtworkQueueRepo() *MockArtworkQueueRepo {
|
||||
return &MockArtworkQueueRepo{Data: map[string]model.ArtworkQueueItem{}}
|
||||
}
|
||||
|
||||
func (m *MockArtworkQueueRepo) Get(kind model.Kind, id, imageType string) (*model.ArtworkQueueItem, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.Err != nil {
|
||||
return nil, m.Err
|
||||
}
|
||||
it, ok := m.Data[iaKey(kind.Prefix(), id, imageType)]
|
||||
if !ok {
|
||||
return nil, model.ErrNotFound
|
||||
}
|
||||
return &it, nil
|
||||
}
|
||||
|
||||
func (m *MockArtworkQueueRepo) Enqueue(items ...model.ArtworkQueueItem) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
@ -161,6 +175,49 @@ func (m *MockArtworkQueueRepo) Count() (int64, error) {
|
||||
return int64(len(m.Data)), nil
|
||||
}
|
||||
|
||||
func (m *MockArtworkQueueRepo) CountByKindAndPriority() ([]model.ArtworkQueueStat, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.Err != nil {
|
||||
return nil, m.Err
|
||||
}
|
||||
var res []model.ArtworkQueueStat
|
||||
for _, it := range m.Data {
|
||||
i := slices.IndexFunc(res, func(s model.ArtworkQueueStat) bool {
|
||||
return s.ItemKind == it.ItemKind && s.Priority == it.Priority
|
||||
})
|
||||
if i < 0 {
|
||||
res = append(res, model.ArtworkQueueStat{ItemKind: it.ItemKind, Priority: it.Priority, Count: 1})
|
||||
continue
|
||||
}
|
||||
res[i].Count++
|
||||
}
|
||||
slices.SortFunc(res, func(a, b model.ArtworkQueueStat) int {
|
||||
return cmp.Or(cmp.Compare(a.ItemKind, b.ItemKind), cmp.Compare(b.Priority, a.Priority))
|
||||
})
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// CountAbsent mirrors the SQL predicate: an absent state is one with no hash.
|
||||
func (m *MockArtworkQueueRepo) CountAbsent(kind model.Kind, attemptedBefore time.Time) (model.ArtworkAbsentStat, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
var res model.ArtworkAbsentStat
|
||||
if m.Err != nil || m.ItemArtworkSource == nil {
|
||||
return res, m.Err
|
||||
}
|
||||
for _, ia := range m.ItemArtworkSource.ItemData {
|
||||
if ia.ItemKind != kind.Prefix() || ia.Hash != "" {
|
||||
continue
|
||||
}
|
||||
res.Total++
|
||||
if ia.AttemptedAt.Before(attemptedBefore) {
|
||||
res.Stale++
|
||||
}
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
func (m *MockArtworkQueueRepo) EnqueuePreservingBackoff(items ...model.ArtworkQueueItem) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
@ -216,6 +273,65 @@ func (m *MockArtworkQueueRepo) EnqueueStaleAbsent(kind model.Kind, attemptedBefo
|
||||
return inserted, nil
|
||||
}
|
||||
|
||||
// matchingSource mirrors the SQL filter: no sources means every source, "" the absent state.
|
||||
func (m *MockArtworkQueueRepo) matchingSource(kind model.Kind, sources []string) []model.ItemArtwork {
|
||||
if m.ItemArtworkSource == nil {
|
||||
return nil
|
||||
}
|
||||
var res []model.ItemArtwork
|
||||
for _, ia := range m.ItemArtworkSource.ItemData {
|
||||
if ia.ItemKind == kind.Prefix() && (len(sources) == 0 || slices.Contains(sources, ia.Source)) {
|
||||
res = append(res, ia)
|
||||
}
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
func (m *MockArtworkQueueRepo) CountBySource(kind model.Kind, sources []string) (int64, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.Err != nil {
|
||||
return 0, m.Err
|
||||
}
|
||||
return int64(len(m.matchingSource(kind, sources))), nil
|
||||
}
|
||||
|
||||
func (m *MockArtworkQueueRepo) SourcesInUse(kind model.Kind) ([]string, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.Err != nil {
|
||||
return nil, m.Err
|
||||
}
|
||||
sources := slice.Map(m.matchingSource(kind, nil), func(ia model.ItemArtwork) string { return ia.Source })
|
||||
return slice.Unique(sources), nil
|
||||
}
|
||||
|
||||
func (m *MockArtworkQueueRepo) EnqueueBySource(kind model.Kind, sources []string, priority int) (int64, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.Err != nil {
|
||||
return 0, m.Err
|
||||
}
|
||||
now := time.Now()
|
||||
var inserted int64
|
||||
for _, ia := range m.matchingSource(kind, sources) {
|
||||
k := iaKey(ia.ItemKind, ia.ItemID, ia.ImageType)
|
||||
if _, ok := m.Data[k]; ok { // DO NOTHING: never touch existing queue rows
|
||||
continue
|
||||
}
|
||||
m.Data[k] = model.ArtworkQueueItem{
|
||||
ItemKind: ia.ItemKind,
|
||||
ItemID: ia.ItemID,
|
||||
ImageType: ia.ImageType,
|
||||
Priority: priority,
|
||||
RetryAt: now,
|
||||
EnqueuedAt: now,
|
||||
}
|
||||
inserted++
|
||||
}
|
||||
return inserted, nil
|
||||
}
|
||||
|
||||
// EnqueueMissing mirrors the SQL set-difference insert: ExistingIDs[kind] minus ItemArtworkSource.
|
||||
func (m *MockArtworkQueueRepo) EnqueueAllMissing(kind model.Kind, priority int) (int64, error) {
|
||||
m.mu.Lock()
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user