navidrome/plugins/capabilities/metadata_agent.go
Deluan Quintão 59448e9283
fix(scrobbler): back off when a provider asks us to, instead of retrying per play (#6028)
* feat(agents): retry-later error type with optional server delay

Add agents.ErrRetryLater and agents.RetryLaterError, which carries the
delay requested by an external service (e.g. ListenBrainz's
X-RateLimit-Reset-In). scrobbler.ErrRetryLater becomes an alias of the new
sentinel, so existing errors.Is checks and the plugin error-string protocol
keep working unchanged. Groundwork for honoring server-requested retry
delays across scrobbling, metadata agents and artwork.

Song.Equals tests moved to song_test.go to enable external test package.

* fix(scrobbler): honor backoff window and server-requested retry delay

ListenBrainz 429s were decoded into a typed error that classified as
unrecoverable, silently discarding the scrobble (a JSON-bodied 429 was
measured live). The client now maps any 429 to agents.RetryLaterError,
carrying X-RateLimit-Reset-In when present (capped at 1h). Last.fm error 29
(rate limit) is now retryable like 11/16. The buffer's drain loop no longer
lets wake signals bypass an active backoff window - new plays enqueue but
drain only when the window closes - and the wait honors the server delay
via max(backoff, retryIn).

* feat(agents): skip cooling-down agents in aggregate calls

When an agent reports retry-later, remember a per-agent cooldown deadline
(the server-requested delay, or 1 minute when unspecified) and skip that
agent in all aggregate metadata calls until it passes. A round that found
no data but skipped or saw a throttled agent returns ErrRetryLater instead
of ErrNotFound, so callers cannot mistake rate limiting for a definitive
'no data' answer.

* feat(artwork): honor server-requested retry delay when rescheduling

When an external image lookup fails with a retry-later error carrying a
delay (e.g. a 429 with X-RateLimit-Reset-In), the chain trace carries the
largest such hint back to the worker, which reschedules the item at
max(exponential backoff, server delay) instead of backoff alone.

* feat(plugins): retry-later with optional delay for scrobbler and agent plugins

Scrobbler plugins can now return scrobbler(retry_later:N) to request a
retry in N seconds (capped at 1h); the bare token keeps its old meaning.
Metadata-agent plugins, which had no error vocabulary at all, gain the
parallel agent(retry_later[:N]) token, mapped to agents.RetryLaterError so
the aggregate's cooldown and the artwork worker honor plugin throttling
the same way as built-in agents.

* fix: address whole-branch review findings for retry-later handling

Narrow the aggregate's throttled rule to the spec sentence: core.Agents returns
ErrRetryLater only when no agent answered at all (all skipped-cooling or
retry-later). An agent that does not implement the called method now returns an
internal errUnsupported instead of ErrNotFound, so it counts as "did not run" —
without that, the always-appended local agent would answer for biography, URL
and images and make ErrRetryLater unreachable.

Wire the consequence in core/external: a throttled round no longer stamps
ExternalInfoUpdatedAt (artist and album), so the empty result is not cached for
the TTL, and TopSongs maps ErrRetryLater to the same empty-200 the not-found
path already produced instead of a new client-facing error.

Move the Last.fm code-29 mapping into the client's central error construction so
every metadata path produces RetryLaterError, and map ListenBrainz's body-level
code 429 (sent with a non-429 HTTP status) the same way.

Clamp server- and plugin-requested delays in seconds before scaling to a
Duration, in all three parse sites: a header of 18446744074 wrapped past 2^64 and
came out as a 0.29s delay.

Also: extract the artwork worker's reschedule computation into retryDelay() and
cover both it and the trace RetryIn wiring with tests; collapse the double regex
call in mapScrobblerError; drop capabilities.ScrobblerErrorRetryLaterIn (ndpgen
never emits funcs, so plugin authors could not reach it); regenerate the PDKs so
MetadataAgentError reaches the Go and Rust SDKs; de-flake the cooldown tests
(long RetryIn for the skip case, separate expiry spec); and cover the max()
retry-delay aggregation across users in the scrobble buffer.

* refactor: dedupe retry-later parsing and simplify error collection

- Add agents.NewRetryLater and agents.RetryLaterFromSeconds, with a single
  1h cap, replacing the parse+clamp+multiply logic and the maxRetryInSeconds
  constant duplicated across listenbrainz, plugins and the agent adapter.
- Move HTTP header parsing to httpclient.RetryAfter, so the transport layer
  owns it and stays domain-agnostic; drop retryInFromHeaders from the
  ListenBrainz client. Covered by a new Ginkgo table in that package.
- Collapse the two near-identical plugin retry_later regexes into one
  parseRetryLater(prefix, msg) shared by the agent and scrobbler adapters.
- Fold the duplicated noteRetryIn snippet from fetchArtistImage and
  fetchAlbumImage into recordAgent, which already branched on the same
  isTransientExternal condition.
- Replace the atomic.Bool + note() closure in populateArtistInfo with
  errgroup's own error collection; the group carries no context, so a
  returned error does not cancel its siblings.
- Reuse recoveringScrobbler for the per-user delay test instead of a third
  double, and switch fakeScrobbler's mutex-guarded error to the
  atomic.Pointer idiom already used in the same package.

* refactor(listenbrainz): keep rate-limit header parsing in the adapter

The X-RateLimit-Reset-In header is ListenBrainz's own convention, not a
shared one: Last.fm sends no rate-limit headers at all and reports its
limit as a body code, and no other integration in tree sends Retry-After.
A parser in utils/httpclient implied a uniformity across services that
does not exist, so it moves back next to the only client that can know
which header its service sends.

* refactor(agents): collapse the retry-later sentinel and error into one type

ErrRetryLater is now the zero-delay RetryLaterError rather than a separate
errors.New value, so errors.Is and errors.AsType both match the sentinel and
every delay-carrying variant. That removes the trap where a bare sentinel
silently skipped the AsType path, and lets every consumer read the delay off
the error directly: the RetryIn accessor and the two constructors are gone,
with the policy cap applied where untrusted input is parsed.

* refactor(agents): split the cooldown store from the per-dispatch tally

The cooldown map and mutex become a cooldowns value with active/park, holding
no knowledge of errors; agentAttempts records one dispatch's outcomes and owns
the classification that noteAgentError used to hide behind a bool. The three
dispatch loops now touch a single object: skip folds the cooldown check and the
throttled flag into one call, so the store never appears in the loops.

* refactor(agents): share one dispatch loop between the agent call helpers

callAgentMethod and callAgentSliceMethod ran identical loops, differing only in
how they test a result for emptiness: a slice cannot be compared against its
zero value, so the two could not share a constraint. Both now delegate to
callAgent, which takes that test as a parameter. Keeping the loop in one place
matters more than the lines saved: it holds the cooldown skip, the attempt
recording and the empty-dispatch verdict, and a fix applied to one copy but not
the other would be silent.

* test: cover the two retry-later paths a mutation could break silently

Both gaps were proven, not guessed: making the artwork worker pass 0 instead
of the collected hint left all 386 specs green, and replacing the default
agent cooldown with 0 left the agents suite green. The worker test drives a
throttled image agent through drain and asserts the persisted retry_at, and
the cooldown test parks an agent that asked to be retried without naming a
delay, which is what Last.fm does on every rate limit.

* refactor(artwork): carry the external failure as an error, not a flag plus a trace field

The retry delay was riding on ChainTrace, a diagnostic that gets persisted, while
the very same signal — an external source faulted — already travelled by value as
resolution.extError. That was two mechanisms for one idea, and it put control-flow
state inside a serializable trace.

resolution.extError and chainState.extErr become the error itself, so a caller
checks err != nil for the fault and errors.AsType for the delay the provider asked
for. The agent loops return that error last, per convention, and longerRetry keeps
whichever failure wants the longer wait. ChainTrace goes back to holding only steps
and no longer imports core/agents.

* fix(artwork): check the resolve error before reading its resolution

Reading res.extError before the err check was safe only because every error path
in resolve returns a bare resolution{}; a future path returning a partly-filled
one would have been read silently. The failure path now returns no delay
explicitly.

* test(artwork): assert the delay acquire reports, not just its downstream effect

acquire's retry delay was only covered through the worker's persisted retry_at,
one layer away from where the value is computed. Both outcomes are now pinned at
the processor: a plain failure asks for nothing, a throttled provider's delay is
passed through.

* refactor: share the retry-seconds parse and drop the backoff deadline arithmetic

The clamp-before-scaling invariant lived in two parsers and was independently
re-tested in three files with the same magic number; a fix applied to one copy
would have left the others wrapping a huge value down to a fraction of a second.
It moves to agents.ParseRetryIn.

The buffer tracked an absolute retryDeadline only to re-arm a timer that was
already armed for the same instant; a backingOff flag says the same thing without
the arithmetic. The plugin token regex now carries its capability in the pattern
instead of capturing and comparing, so another capability's token in the same
message cannot mask it. resolution.extError becomes extErr, matching its
chainState counterpart.

* fix(agents): keep the longer cooldown when parks overlap

Calls to one agent overlap, so a short cooldown could land after a long one
started and cut it short. park now keeps whichever deadline is later, matching
the rule longerRetry already applies on the artwork side. No in-tree provider
can currently produce two different delays for the same agent, so this is
hardening rather than a fix for observed behaviour.

* fix(agents): parse the retry delay at a fixed width

strconv.Atoi parses into the native int, so on the 32-bit targets we ship
(linux/386, windows/386, three ARM variants) a delay above MaxInt32 seconds
overflowed and became unspecified instead of being capped. No provider sends a
68-year delay, so this is not user-visible, but the overflow tests asserted the
cap and would have failed on those architectures, where tests never run.

* fix(plugins): anchor the retry_later regex to a word boundary

Prevents a superstring like useragent(retry_later) from matching the
agent capability token.
2026-08-29 17:28:29 -04:00

239 lines
8.1 KiB
Go

package capabilities
import "github.com/navidrome/navidrome/plugins/types"
// MetadataAgent provides artist and album metadata retrieval.
// This capability allows plugins to provide external metadata for artists and albums,
// such as biographies, images, similar artists, and top songs.
//
// Plugins implementing this capability can choose which methods to implement.
// Each method is optional - plugins only need to provide the functionality they support.
//
// To say "no data for this item", return a nil response and a nil error. Return an error only when
// the plugin itself failed, because Navidrome retries failed calls with backoff.
//
//nd:capability name=metadata
type MetadataAgent interface {
// GetArtistMBID retrieves the MusicBrainz ID for an artist.
//nd:export name=nd_get_artist_mbid
GetArtistMBID(ArtistMBIDRequest) (*ArtistMBIDResponse, error)
// GetArtistURL retrieves the external URL for an artist.
//nd:export name=nd_get_artist_url
GetArtistURL(ArtistRequest) (*ArtistURLResponse, error)
// GetArtistBiography retrieves the biography for an artist.
//nd:export name=nd_get_artist_biography
GetArtistBiography(ArtistRequest) (*ArtistBiographyResponse, error)
// GetSimilarArtists retrieves similar artists for a given artist.
//nd:export name=nd_get_similar_artists
GetSimilarArtists(SimilarArtistsRequest) (*SimilarArtistsResponse, error)
// GetArtistImages retrieves images for an artist.
//nd:export name=nd_get_artist_images
GetArtistImages(ArtistRequest) (*ArtistImagesResponse, error)
// GetArtistTopSongs retrieves top songs for an artist.
//nd:export name=nd_get_artist_top_songs
GetArtistTopSongs(TopSongsRequest) (*TopSongsResponse, error)
// GetAlbumInfo retrieves album information.
//nd:export name=nd_get_album_info
GetAlbumInfo(AlbumRequest) (*AlbumInfoResponse, error)
// GetAlbumImages retrieves images for an album.
//nd:export name=nd_get_album_images
GetAlbumImages(AlbumRequest) (*AlbumImagesResponse, error)
// GetSimilarSongsByTrack retrieves songs similar to a specific track.
//nd:export name=nd_get_similar_songs_by_track
GetSimilarSongsByTrack(SimilarSongsByTrackRequest) (*SimilarSongsResponse, error)
// GetSimilarSongsByAlbum retrieves songs similar to tracks on an album.
//nd:export name=nd_get_similar_songs_by_album
GetSimilarSongsByAlbum(SimilarSongsByAlbumRequest) (*SimilarSongsResponse, error)
// GetSimilarSongsByArtist retrieves songs similar to an artist's catalog.
//nd:export name=nd_get_similar_songs_by_artist
GetSimilarSongsByArtist(SimilarSongsByArtistRequest) (*SimilarSongsResponse, error)
}
// ArtistMBIDRequest is the request for GetArtistMBID.
type ArtistMBIDRequest struct {
// ID is the internal Navidrome artist ID.
ID string `json:"id"`
// Name is the artist name.
Name string `json:"name"`
}
// ArtistMBIDResponse is the response for GetArtistMBID.
type ArtistMBIDResponse struct {
// MBID is the MusicBrainz ID for the artist.
MBID string `json:"mbid"`
}
// ArtistRequest is the common request for artist-related functions.
type ArtistRequest struct {
// ID is the internal Navidrome artist ID.
ID string `json:"id"`
// Name is the artist name.
Name string `json:"name"`
// MBID is the MusicBrainz ID for the artist (if known).
MBID string `json:"mbid,omitempty"`
}
// ArtistURLResponse is the response for GetArtistURL.
type ArtistURLResponse struct {
// URL is the external URL for the artist.
URL string `json:"url"`
}
// ArtistBiographyResponse is the response for GetArtistBiography.
type ArtistBiographyResponse struct {
// Biography is the artist biography text.
Biography string `json:"biography"`
}
// SimilarArtistsRequest is the request for GetSimilarArtists.
type SimilarArtistsRequest struct {
// ID is the internal Navidrome artist ID.
ID string `json:"id"`
// Name is the artist name.
Name string `json:"name"`
// MBID is the MusicBrainz ID for the artist (if known).
MBID string `json:"mbid,omitempty"`
// Limit is the maximum number of similar artists to return.
Limit int32 `json:"limit"`
}
// Deprecated: use types.ArtistRef.
type ArtistRef = types.ArtistRef
// SimilarArtistsResponse is the response for GetSimilarArtists.
type SimilarArtistsResponse struct {
// Artists is the list of similar artists.
Artists []types.ArtistRef `json:"artists"`
}
// ImageInfo represents an image with URL and size.
type ImageInfo struct {
// URL is the URL of the image.
URL string `json:"url"`
// Size is the size of the image in pixels (width or height).
Size int32 `json:"size"`
}
// ArtistImagesResponse is the response for GetArtistImages.
type ArtistImagesResponse struct {
// Images is the list of artist images.
Images []ImageInfo `json:"images"`
}
// TopSongsRequest is the request for GetArtistTopSongs.
type TopSongsRequest struct {
// ID is the internal Navidrome artist ID.
ID string `json:"id"`
// Name is the artist name.
Name string `json:"name"`
// MBID is the MusicBrainz ID for the artist (if known).
MBID string `json:"mbid,omitempty"`
// Count is the maximum number of top songs to return.
Count int32 `json:"count"`
}
// Deprecated: use types.SongRef.
type SongRef = types.SongRef
// TopSongsResponse is the response for GetArtistTopSongs.
type TopSongsResponse struct {
// Songs is the list of top songs.
Songs []types.SongRef `json:"songs"`
}
// AlbumRequest is the common request for album-related functions.
type AlbumRequest struct {
// Name is the album name.
Name string `json:"name"`
// Artist is the album artist name.
Artist string `json:"artist"`
// MBID is the MusicBrainz ID for the album (if known).
MBID string `json:"mbid,omitempty"`
}
// AlbumInfoResponse is the response for GetAlbumInfo.
type AlbumInfoResponse struct {
// Name is the album name.
Name string `json:"name"`
// MBID is the MusicBrainz ID for the album.
MBID string `json:"mbid"`
// Description is the album description/notes.
Description string `json:"description"`
// URL is the external URL for the album.
URL string `json:"url"`
}
// AlbumImagesResponse is the response for GetAlbumImages.
type AlbumImagesResponse struct {
// Images is the list of album images.
Images []ImageInfo `json:"images"`
}
// SimilarSongsByTrackRequest is the request for GetSimilarSongsByTrack.
type SimilarSongsByTrackRequest struct {
// ID is the internal Navidrome mediafile ID.
ID string `json:"id"`
// Name is the track title.
Name string `json:"name"`
// Artist is the artist name.
Artist string `json:"artist"`
// MBID is the MusicBrainz recording ID (if known).
MBID string `json:"mbid,omitempty"`
// Count is the maximum number of similar songs to return.
Count int32 `json:"count"`
}
// SimilarSongsByAlbumRequest is the request for GetSimilarSongsByAlbum.
type SimilarSongsByAlbumRequest struct {
// ID is the internal Navidrome album ID.
ID string `json:"id"`
// Name is the album name.
Name string `json:"name"`
// Artist is the album artist name.
Artist string `json:"artist"`
// MBID is the MusicBrainz release ID (if known).
MBID string `json:"mbid,omitempty"`
// Count is the maximum number of similar songs to return.
Count int32 `json:"count"`
}
// SimilarSongsByArtistRequest is the request for GetSimilarSongsByArtist.
type SimilarSongsByArtistRequest struct {
// ID is the internal Navidrome artist ID.
ID string `json:"id"`
// Name is the artist name.
Name string `json:"name"`
// MBID is the MusicBrainz artist ID (if known).
MBID string `json:"mbid,omitempty"`
// Count is the maximum number of similar songs to return.
Count int32 `json:"count"`
}
// SimilarSongsResponse is the response for GetSimilarSongsBy* functions.
type SimilarSongsResponse struct {
// Songs is the list of similar songs.
Songs []types.SongRef `json:"songs"`
}
// MetadataAgentError represents an error type for metadata agent operations.
type MetadataAgentError string
const (
// MetadataAgentErrorRetryLater indicates the provider is throttling; retry later.
// Append ":<seconds>" inside the parentheses to request a specific delay.
MetadataAgentErrorRetryLater MetadataAgentError = "agent(retry_later)"
)
// Error implements the error interface for MetadataAgentError.
func (e MetadataAgentError) Error() string { return string(e) }