Fix timer lifecycle bugs in the playlist syncer: always store
RetryInterval (including 0 to disable retries), cancel discovery timers
when RefreshInterval becomes 0, and cancel stale refresh timers when
ValidUntil becomes 0. Extract cancelRefreshTimer helper to deduplicate
the timer cleanup pattern. Improve plugin playlist update restrictions
in both the Subsonic and REST API paths to compare actual values instead
of just checking pointer presence or field inclusion, so passing
unchanged name/comment no longer triggers a false rejection.
Signed-off-by: Deluan <deluan@navidrome.org>
Add a ValidUntil field to the Playlist model and persist it from the
plugin's GetPlaylistResponse during sync. This allows clients to know
when a plugin playlist's data will be refreshed. The value is exposed
in the OpenSubsonic playlist response alongside the existing
smart playlist ValidUntil calculation. The migration is consolidated
into a single multi-statement ExecContext call.
Eliminate redundant work and minor issues found during code review:
- Replace manual PlaylistTrack construction in syncPlaylist with the
existing Playlist.AddMediaFiles helper, removing duplicated logic
- Pre-sanitize track fields once per artist batch in the matcher's
fuzzy matching loop, avoiding redundant sanitization in both
findBestMatch and computeSpecificityLevel on every iteration
- Cache resolved usernames in discoverAndSync to avoid N+1 DB lookups
when multiple playlists share the same owner
- Use the local loadedPlugin variable instead of reading m.plugins[p.ID]
after releasing the lock in loadPluginWithConfig
- Fix misleading uint32 comparison (<=0 to ==0) in durationProximity
- Update stale comment on checkTracksEditable to mention plugin playlists
PlaylistProvider capability now requires 'users' permission in the
manifest (matching existing Scrobbler behavior) and validates that the
resolved owner user ID is in the plugin's allowed users list before
creating playlists.
Rename the capability from PlaylistGenerator to PlaylistProvider and the
internal orchestrator struct from playlistGeneratorOrchestrator to
playlistSyncer. The new names better describe what the capability does
(provides playlists) rather than how it works internally. All source
files, test plugin, PDK packages (Go/Rust), YAML schemas, and exported
WASM function names are updated accordingly.
Replaced inline matcher.New(ds) calls in external.provider and
PlaylistGenerator orchestrator with proper dependency injection via
Google Wire. Added matcher.New to the Wire provider set, updated
NewProvider and GetManager signatures to accept *matcher.Matcher, and
deleted the trivial provider_matching.go wrapper. This eliminates tight
coupling where each caller knew how to construct a Matcher, following
the same DI pattern used by other core services.
Signed-off-by: Deluan <deluan@navidrome.org>
Move the PlaylistGenerator orchestrator from a separate map on the Manager
into each plugin's closers list, giving it the same lifecycle as other host
services (Scheduler, TaskQueue, WebSocket). The orchestrator now implements
io.Closer and is created during plugin load, so it starts/stops automatically
with the plugin. This removes the playlistGenerators map, the
startPlaylistGenerators() method, and special-case cleanup code from both
Stop() and unloadPlugin(), simplifying reload and unload paths.
Replace the timer-per-playlist + WaitGroup.Go model with a single worker
goroutine processing work items from a buffered channel. This eliminates
race conditions from parallel plugin calls (rate limiting risk),
unsynchronized map/field access across goroutines, and overlapping
discovery and refresh timers. The worker owns all mutable state
(refreshTimers, discoveryTimer) exclusively, while retryInterval and
refreshTimerCount use atomics for safe test observability. Also adds
retry-on-failure for GetAvailablePlaylists (5 min delay) and makes
MockPlaylistRepo thread-safe with sync.RWMutex.
Rename GetPlaylists → GetAvailablePlaylists for clarity. Add
PlaylistGeneratorErrorNotFound so plugins can signal a playlist is
temporarily unavailable (orchestrator skips it but retries on next
discovery). Add RetryInterval on GetAvailablePlaylistsResponse for
automatic retry of transient GetPlaylist failures.
Rename PlaylistInfo.OwnerUserID to OwnerUsername so plugins provide
usernames instead of internal user IDs, with server-side resolution via
FindByUsername. Fix race condition on playlistGenerators map by using
write lock in startPlaylistGenerators and moving map access inside the
lock in unloadPlugin/Stop. Add context and WaitGroup to the orchestrator
for proper cancellation and goroutine tracking. Include owner_id in the
plugin playlist unique index. Use SetTracks instead of direct assignment
to refresh playlist duration/size/song count stats.
Add a test playlist generator plugin and integration tests for the
PlaylistGenerator capability. The test plugin exercises GetPlaylists and
GetPlaylist WASM functions, and the tests verify orchestration flow
including capability detection, playlist discovery/sync, deterministic
ID generation, error handling, and graceful stop.
Also initialize playlistGenerators map in the test helper to prevent nil
map panics when loading plugins with PlaylistGenerator capability.
Orchestrates plugin playlist lifecycle: discovery via GetPlaylists,
data fetch via GetPlaylist, track matching via core/matcher, DB upsert,
and timer-based refresh (ValidUntil per playlist, RefreshInterval for
re-discovery).
The cleanupLoop goroutine could execute cleanupExpired against a closed
database because Close() did not wait for the goroutine to exit before
calling db.Close(). This caused 'sql: database is closed' errors during
plugin unload or shutdown.
Close() now cancels the cleanup goroutine's context and waits for it to
finish via a sync.WaitGroup before running the final cleanup and closing
the database.
Signed-off-by: Deluan <deluan@navidrome.org>
Allow plugins to opt out of automatic redirect following on a per-request
basis. When set to true, the response returns the redirect status code and
Location header directly instead of following to the final destination.
* refactor: remove built-in Spotify integration
Remove the Spotify adapter and all related configuration, replacing
the built-in integration with the plugin system. This deletes the
adapters/spotify package, removes Spotify config options (ID/Secret),
updates the default agents list from "deezer,lastfm,spotify" to
"deezer,lastfm", and cleans up all references across configuration,
metrics, logging, artwork caching, and documentation. Users with
Spotify config options will now see a warning that the options are
no longer available.
* feat: add ListenBrainz to list of default agents
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): add lyrics provider plugin capability
Refactor the lyrics system from a static function to an interface-based
service that supports WASM plugin providers. Plugins listed in the
LyricsPriority config (alongside "embedded" and file extensions) are
now resolved through the plugin system.
Includes capability definition, Go/Rust PDK, adapter, Wire integration,
and tests for plugin fallback behavior.
* test(plugins): add lyrics capability integration test with test plugin
* fix(plugins): default lyrics language to 'xxx' when plugin omits it
Per the OpenSubsonic spec, the server must return 'und' or 'xxx' when
the lyrics language is unknown. The lyrics plugin adapter was passing
an empty string through when a plugin didn't provide a language value.
This defaults the language to 'xxx', consistent with all other callers
of model.ToLyrics() in the codebase.
* refactor(plugins): rename lyrics import to improve clarity
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor(lyrics): update TrackInfo description for clarity
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(lyrics): enhance lyrics plugin handling and case sensitivity
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): update payload type to string with byte format for task data
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): define TaskQueue host service interface
Add the TaskQueueService interface with CreateQueue, Enqueue,
GetTaskStatus, and CancelTask methods plus QueueConfig struct.
* feat(plugins): define TaskWorker capability for task execution callbacks
* feat(plugins): add taskqueue permission to manifest schema
Add TaskQueuePermission with maxConcurrency option.
* feat(plugins): implement TaskQueue service with SQLite persistence and workers
Per-plugin SQLite database with queues and tasks tables. Worker goroutines
dequeue tasks and invoke nd_task_execute callback. Exponential backoff
retries, rate limiting via delayMs, automatic cleanup of terminal tasks.
* feat(plugins): require TaskWorker capability for taskqueue permission
* feat(plugins): register TaskQueue host service in manager
* feat(plugins): add test-taskqueue plugin for integration testing
* feat(plugins): add integration tests for TaskQueue host service
* docs: document TaskQueue module for persistent task queues
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): harden TaskQueue host service with validation and safety improvements
Add input validation (queue name length, payload size limits), extract
status string constants to eliminate raw SQL literals, make CreateQueue
idempotent via upsert for crash recovery, fix RetentionMs default check
for negative values, cap exponential backoff at 1 hour to prevent
overflow, and replace manual mutex-based delay enforcement with
rate.Limiter from golang.org/x/time/rate for correct concurrent worker
serialization.
* refactor(plugins): remove capability check for TaskWorker in TaskQueue host service
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): use context-aware database execution in TaskQueue host service
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor(plugins): streamline task queue configuration and error handling
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): increase maxConcurrency for task queue and handle budget exhaustion
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor(plugins): simplify goroutine management in task queue service
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): update TaskWorker interface to return status messages and refactor task queue service
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): add ClearQueue function to remove pending tasks from a specified queue
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor(plugins): use migrateDB for task queue schema and fix constant name collision
Replaced the raw db.Exec call in createTaskQueueSchema with migrateDB,
matching the pattern used by createKVStoreSchema. This enables version-tracked
schema migrations via SQLite's PRAGMA user_version, allowing future schema
changes to be appended incrementally. Also renamed cleanupInterval to
taskCleanupInterval to resolve a redeclaration conflict with host_kvstore.go.
* regenerate PDKs
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
* test(plugins): speed up integration tests with shared wazero cache
Reduce plugin test suite runtime from ~22s to ~12s by:
- Creating a shared wazero compilation cache directory in TestPlugins()
and setting conf.Server.CacheFolder globally so all test Manager
instances reuse compiled WASM binaries from disk cache
- Moving 6 createTestManager* calls from inside It blocks to BeforeAll
blocks in scrobbler_adapter_test.go and manager_call_test.go
- Replacing time.Sleep(2s) in KVStore TTL test with Eventually polling
- Reducing WebSocket callback sleeps from 100ms to 10ms
Signed-off-by: Deluan <deluan@navidrome.org>
* test(plugins): enhance websocket tests by storing server messages for verification
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
Introduced a typed Claims struct in core/auth to replace the raw
map[string]any approach used for JWT claims throughout the codebase.
This provides compile-time safety and better readability when creating,
validating, and extracting JWT tokens. Also upgraded lestrrat-go/jwx
from v2 to v3 and go-chi/jwtauth to v5.4.0, adapting all callers to
the new API where token accessor methods now return tuples instead of
bare values. Updated all affected handlers, middleware, and tests.
Signed-off-by: Deluan <deluan@navidrome.org>
Changed the TTL expiration check from strict greater-than to greater-or-equal
in the notExpiredFilter SQL condition. SQLite's datetime has second-level
precision, so a 1-second TTL set late in a second could appear expired
immediately when read at the next second boundary (e.g. expires_at of T+1
fails the check 'T+1 > T+1'). Updated the cleanup query consistently to use
strict less-than, so rows are only deleted after their expiration second has
fully passed.
Plugins that entered an error state (e.g., incompatible with the
Navidrome version) would remain in that state across restarts, blocking
the user from retrying. This adds a ClearErrors method to
PluginRepository that resets the last_error field on all plugins, and
calls it during plugin manager startup before syncing and loading.
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): add expires_at column to kvstore schema
* feat(plugins): filter expired keys in kvstore Get, Has, List
* feat(plugins): add periodic cleanup of expired kvstore keys
* feat(plugins): add SetWithTTL, DeleteByPrefix, and GetMany to kvstore
Add three new methods to the KVStore host service:
- SetWithTTL: store key-value pairs with automatic expiration
- DeleteByPrefix: remove all keys matching a prefix in one operation
- GetMany: retrieve multiple values in a single call
All methods include comprehensive unit tests covering edge cases,
expiration behavior, size tracking, and LIKE-special characters.
* feat(plugins): regenerate code and update test plugin for new kvstore methods
Regenerate host function wrappers and PDK bindings for Go, Python,
and Rust. Update the test-kvstore plugin to exercise SetWithTTL,
DeleteByPrefix, and GetMany.
* feat(plugins): add integration tests for new kvstore methods
Add WASM integration tests for SetWithTTL, DeleteByPrefix, and GetMany
operations through the plugin boundary, verifying end-to-end behavior
including TTL expiration, prefix deletion, and batch retrieval.
* fix(plugins): address lint issues in kvstore implementation
Handle tx.Rollback error return and suppress gosec false positive
for parameterized SQL query construction in GetMany.
* fix(plugins): Set clears expires_at when overwriting a TTL'd key
Previously, calling Set() on a key that was stored with SetWithTTL()
would leave the expires_at value intact, causing the key to silently
expire even though Set implies permanent storage.
Also excludes expired keys from currentSize calculation at startup.
* refactor(plugins): simplify kvstore by removing in-memory size cache
Replaced the in-memory currentSize cache (atomic.Int64), periodic cleanup
timer, and mutex with direct database queries for storage accounting.
This eliminates race conditions and cache drift issues at negligible
performance cost for plugin-sized datasets. Also unified Set and
SetWithTTL into a shared setValue method, simplified DeleteByPrefix to
use RowsAffected instead of a transaction, and added an index on
expires_at for efficient expiration filtering.
* feat(plugins): add generic SQLite migration helper and refactor kvstore schema
Add a reusable migrateDB helper that tracks schema versions via SQLite's
PRAGMA user_version and applies pending migrations transactionally. Replace
the ad-hoc createKVStoreSchema function in kvstore with a declarative
migrations slice, making it easy to add future schema changes. Remove the
now-redundant schema migration test since migrateDB has its own test suite
and every kvstore test exercises the migrations implicitly.
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): harden kvstore with explicit NULL handling, prefix validation, and cleanup timeout
- Use sql.NullString for expires_at to explicitly send NULL instead of
relying on datetime('now', '') returning NULL by accident
- Reject empty prefix in DeleteByPrefix to prevent accidental data wipe
- Add 5s timeout context to cleanupExpired on Close
- Replace time.Sleep in unit tests with pre-expired timestamps
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor(plugins): use batch processing in GetMany
Process keys in chunks of 200 using slice.CollectChunks to avoid
hitting SQLite's SQLITE_MAX_VARIABLE_NUMBER limit with large key sets.
* feat(plugins): add periodic cleanup goroutine for expired kvstore keys
Use the manager's context to control a background goroutine that purges
expired keys every hour, stopping naturally on shutdown when the context
is cancelled.
---------
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): mount library directories as read-only by default
Add an AllowWriteAccess boolean to the plugin model, defaulting to
false. When off, library directories are mounted with the extism "ro:"
prefix (read-only). Admins can explicitly grant write access via a new
toggle in the Library Permission card.
* test: add tests to buildAllowedPaths
Signed-off-by: Deluan <deluan@navidrome.org>
* chore: improve allowed paths logging for library access
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): add base64 handling for []byte and remove raw=true
Go's json.Marshal automatically base64-encodes []byte fields, but Rust's
serde_json serializes Vec<u8> as a JSON array and Python's json.dumps
raises TypeError on bytes. This fixes both directions of plugin
communication by adding proper base64 encoding/decoding in generated
client code.
For Rust templates (client and capability): adds a base64_bytes serde
helper module with #[serde(with = "base64_bytes")] on all Vec<u8> fields,
and adds base64 as a dependency. For Python templates: wraps bytes params
with base64.b64encode() and responses with base64.b64decode().
Also removes the raw=true binary framing protocol from all templates,
the parser, and the Method type. The raw mechanism added complexity that
is no longer needed once []byte works properly over JSON.
* fix(plugins): update production code and tests for base64 migration
Remove raw=true annotation from SubsonicAPI.CallRaw, delete all raw
test fixtures, remove raw-related test cases from parser, generator, and
integration tests, and add new test cases validating base64 handling
for Rust and Python templates.
* fix(plugins): update golden files and regenerate production code
Update golden test fixtures for codec and comprehensive services to
include base64 handling for []byte fields. Regenerate all production
PDK code (Go, Rust, Python) and host wrappers to use standard JSON
with base64-encoded byte fields instead of binary framing protocol.
* refactor: remove base64 helper duplication from rust template
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): add base64 dependency to capabilities' Cargo.toml
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
Move scheduler capability check from runtime (when callback fires) to
load-time validation in ValidateWithCapabilities. This ensures plugins
declaring the scheduler permission must export the nd_scheduler_callback
function, failing fast with a clear error instead of silently skipping
callbacks at runtime.
* feat(httpclient): implement HttpClient service for outbound HTTP requests in plugins
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(httpclient): enhance SSRF protection by validating host requests against private IPs
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(httpclient): support DELETE requests with body in HttpClient service
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(httpclient): refactor HTTP client initialization and enhance redirect handling
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor(http): standardize naming conventions for HTTP types and methods
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor example plugin to use host.HTTPSend for improved error management
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): fix IPv6 SSRF bypass and wildcard host matching
Fix two bugs in the plugin HTTP/WebSocket host validation:
1. extractHostname now strips IPv6 brackets when no port is present
(e.g. "[::1]" → "::1"). Previously, net.SplitHostPort failed for
bracketed IPv6 without a port, leaving brackets intact. This caused
net.ParseIP to return nil, bypassing the private/loopback SSRF guard.
2. matchHostPattern now treats "*" as an allow-all pattern. Previously,
a bare "*" only matched via exact equality, so plugins declaring
requiredHosts: ["*"] (like webhook-rs) had all requests rejected.
---------
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: add ISRC support to similar songs matching and plugin interface
Add ISRC (International Standard Recording Code) as a high-priority
identifier in the provider matching algorithm, alongside MBID. The
matching pipeline now uses four strategies in priority order:
ID > MBID > ISRC > Title+Artist fuzzy match.
- Add ISRC field to agents.Song struct
- Add ISRC field to plugin capability SongRef (Go, Rust PDKs)
- Add loadTracksByISRC using json_tree query on tags column
- Integrate ISRC into matchSongsToLibrary, selectBestMatchingSongs,
and buildTitleQueries
https://claude.ai/code/session_01Dd4mTq1VQZag4RNjCVusiF
* chore: regenerate plugin schema after ISRC addition
Run `make gen` to update the generated YAML schema for the
metadata agent capability with the new ISRC field on SongRef.
https://claude.ai/code/session_01Dd4mTq1VQZag4RNjCVusiF
* feat(mediafile): add GetAllByTags method to MediaFileRepository for tag-based retrieval
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(provider): speed up track matching by incorporating prior matches in ISRC and MBID lookups
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Claude <noreply@anthropic.com>
* feat: add duration filtering for similar songs matching
Signed-off-by: Deluan <deluan@navidrome.org>
* test: refactor expectations for similar songs in provider matching tests
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(plugins): add functions to retrieve similar songs by track, album, and artist
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): support uint32 in ndpgen
Signed-off-by: Deluan <deluan@navidrome.org>
* fix(plugins): update duration field to use seconds as float instead of milliseconds as uint32
Signed-off-by: Deluan <deluan@navidrome.org>
* fix: add helper functions for Rust's skip_serializing_if with numeric types
Signed-off-by: Deluan <deluan@navidrome.org>
* feat(provider): enhance track matching logic to fallback to title match when duration-filtered tracks fail
---------
Signed-off-by: Deluan <deluan@navidrome.org>
