From 54de0dbc52a25a8a5fbf1b44d5ca6703a61d87ec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Sat, 21 Feb 2026 17:52:42 -0500 Subject: [PATCH 01/60] feat(server): implement FTS5-based full-text search (#5079) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * build: add sqlite_fts5 build tag to enable FTS5 support * feat: add SearchBackend config option (default: fts) * feat: add buildFTS5Query for safe FTS5 query preprocessing * feat: add FTS5 search backend with config toggle, refactor legacy search - Add searchExprFunc type and getSearchExpr() for backend selection - Rename fullTextExpr to legacySearchExpr - Add ftsSearchExpr using FTS5 MATCH subquery - Update fullTextFilter in sql_restful.go to use configured backend * feat: add FTS5 migration with virtual tables, triggers, and search_participants Creates FTS5 virtual tables for media_file, album, and artist with unicode61 tokenizer and diacritic folding. Adds search_participants column, populates from JSON, and sets up INSERT/UPDATE/DELETE triggers. * feat: populate search_participants in PostMapArgs for FTS5 indexing * test: add FTS5 search integration tests * fix: exclude FTS5 virtual tables from e2e DB restore The restoreDB function iterates all tables in sqlite_master and runs DELETE + INSERT to reset state. FTS5 contentless virtual tables cannot be directly deleted from. Since triggers handle FTS5 sync automatically, simply skip tables matching *_fts and *_fts_* patterns. * build: add compile-time guard for sqlite_fts5 build tag Same pattern as netgo: compilation fails with a clear error if the sqlite_fts5 build tag is missing. * build: add sqlite_fts5 tag to reflex dev server config * build: extract GO_BUILD_TAGS variable in Makefile to avoid duplication * fix: strip leading * from FTS5 queries to prevent "unknown special query" error * feat: auto-append prefix wildcard to FTS5 search tokens for broader matching Every plain search token now gets a trailing * appended (e.g., "love" becomes "love*"), so searching for "love" also matches "lovelace", "lovely", etc. Quoted phrases are preserved as exact matches without wildcards. Results are ordered alphabetically by name/title, so shorter exact matches naturally appear first. * fix: clarify comments about FTS5 operator neutralization The comments said "strip" but the code lowercases operators to neutralize them (FTS5 operators are case-sensitive). Updated comments to accurately describe the behavior. * fix: use fmt.Sprintf for FTS5 phrase placeholders The previous encoding used rune('0'+index) which silently breaks with 10+ quoted phrases. Use fmt.Sprintf for arbitrary index support. * fix: validate and normalize SearchBackend config option Normalize the value to lowercase and fall back to "fts" with a log warning for unrecognized values. This prevents silent misconfiguration from typos like "FTS", "Legacy", or "fts5". * refactor: improve documentation for build tags and FTS5 requirements Signed-off-by: Deluan * refactor: convert FTS5 query and search backend normalization tests to DescribeTable format Signed-off-by: Deluan * fix: add sqlite_fts5 build tag to golangci configuration Signed-off-by: Deluan * feat: add UISearchDebounceMs configuration option and update related components Signed-off-by: Deluan * fix: fall back to legacy search when SearchFullString is enabled FTS5 is token-based and cannot match substrings within words, so getSearchExpr now returns legacySearchExpr when SearchFullString is true, regardless of SearchBackend setting. * fix: add sqlite_fts5 build tag to CI pipeline and Dockerfile * fix: add WHEN clauses to FTS5 AFTER UPDATE triggers Added WHEN clauses to the media_file_fts_au, album_fts_au, and artist_fts_au triggers so they only fire when FTS-indexed columns actually change. Previously, every row update (e.g., play count, rating, starred status) triggered an unnecessary delete+insert cycle in the FTS shadow tables. The WHEN clauses use IS NOT for NULL-safe comparison of each indexed column, avoiding FTS index churn for non-indexed updates. * feat: add SearchBackend configuration option to data and insights components Signed-off-by: Deluan * fix: enhance input sanitization for FTS5 by stripping additional punctuation and special characters Signed-off-by: Deluan * feat: add search_normalized column for punctuated name search (R.E.M., AC/DC) Add index-time normalization and query-time single-letter collapsing to fix FTS5 search for punctuated names. A new search_normalized column stores concatenated forms of punctuated words (e.g., "R.E.M." → "REM", "AC/DC" → "ACDC") and is indexed in FTS5 tables. At query time, runs of consecutive single letters (from dot-stripping) are collapsed into OR expressions like ("R E M" OR REM*) to match both the original tokens and the normalized form. This enables searching by "R.E.M.", "REM", "AC/DC", "ACDC", "A-ha", or "Aha" and finding the correct results. * refactor: simplify isSingleUnicodeLetter to avoid []rune allocation Use utf8.DecodeRuneInString to check for a single Unicode letter instead of converting the entire string to a []rune slice. * feat: define ftsSearchColumns for flexible FTS5 search column inclusion Signed-off-by: Deluan * feat: update collapseSingleLetterRuns to return quoted phrases for abbreviations Signed-off-by: Deluan * feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation Signed-off-by: Deluan * feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation Signed-off-by: Deluan * refactor: punctuated word handling to improve processing of artist/album names Signed-off-by: Deluan * feat: add CJK support for search queries with LIKE filters Signed-off-by: Deluan * feat: enhance FTS5 search by adding album version support and CJK handling Signed-off-by: Deluan * refactor: search configuration to use structured options Signed-off-by: Deluan * feat: enhance search functionality to support punctuation-only queries and update related tests Signed-off-by: Deluan --------- Signed-off-by: Deluan --- .github/workflows/pipeline.yml | 2 +- .golangci.yml | 1 + Dockerfile | 2 +- Makefile | 13 +- conf/buildtags/buildtags.go | 4 - conf/buildtags/doc.go | 6 + conf/buildtags/netgo.go | 6 +- conf/buildtags/sqlite_fts5.go | 8 + conf/configuration.go | 26 +- conf/configuration_test.go | 14 + conf/export_test.go | 2 + consts/consts.go | 11 +- core/metrics/insights.go | 3 +- core/metrics/insights/data.go | 1 + .../20260220173400_add_fts5_search.go | 391 ++++++++++++++++++ main.go | 5 +- persistence/album_repository.go | 5 +- persistence/album_repository_test.go | 6 + persistence/artist_repository.go | 1 + persistence/artist_repository_test.go | 50 ++- persistence/mediafile_repository.go | 5 +- persistence/mediafile_repository_test.go | 2 +- persistence/persistence_suite_test.go | 31 +- persistence/sql_restful.go | 3 +- persistence/sql_restful_test.go | 9 +- persistence/sql_search.go | 32 +- persistence/sql_search_fts.go | 261 ++++++++++++ persistence/sql_search_fts_test.go | 333 +++++++++++++++ persistence/sql_search_test.go | 97 +++++ reflex.conf | 2 +- server/e2e/e2e_suite_test.go | 2 +- server/serve_index.go | 1 + server/serve_index_test.go | 1 + ui/src/common/List.jsx | 2 + ui/src/config.js | 1 + 35 files changed, 1283 insertions(+), 56 deletions(-) delete mode 100644 conf/buildtags/buildtags.go create mode 100644 conf/buildtags/doc.go create mode 100644 conf/buildtags/sqlite_fts5.go create mode 100644 db/migrations/20260220173400_add_fts5_search.go create mode 100644 persistence/sql_search_fts.go create mode 100644 persistence/sql_search_fts_test.go diff --git a/.github/workflows/pipeline.yml b/.github/workflows/pipeline.yml index 336205550..fd8edcd1c 100644 --- a/.github/workflows/pipeline.yml +++ b/.github/workflows/pipeline.yml @@ -117,7 +117,7 @@ jobs: - name: Test run: | pkg-config --define-prefix --cflags --libs taglib # for debugging - go test -shuffle=on -tags netgo -race ./... -v + go test -shuffle=on -tags netgo,sqlite_fts5 -race ./... -v - name: Test ndpgen run: | diff --git a/.golangci.yml b/.golangci.yml index 996dafccb..1937c2f77 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -2,6 +2,7 @@ version: "2" run: build-tags: - netgo + - sqlite_fts5 linters: enable: - asasalint diff --git a/Dockerfile b/Dockerfile index ad43e247b..b32c1df56 100644 --- a/Dockerfile +++ b/Dockerfile @@ -109,7 +109,7 @@ RUN --mount=type=bind,source=. \ export EXT=".exe" fi - go build -tags=netgo -ldflags="${LD_EXTRA} -w -s \ + go build -tags=netgo,sqlite_fts5 -ldflags="${LD_EXTRA} -w -s \ -X github.com/navidrome/navidrome/consts.gitSha=${GIT_SHA} \ -X github.com/navidrome/navidrome/consts.gitTag=${GIT_TAG}" \ -o /out/navidrome${EXT} . diff --git a/Makefile b/Makefile index d16cb2821..f7b7b1b05 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,6 @@ GO_VERSION=$(shell grep "^go " go.mod | cut -f 2 -d ' ') NODE_VERSION=$(shell cat .nvmrc) +GO_BUILD_TAGS=netgo,sqlite_fts5 # Set global environment variables, required for most targets export CGO_CFLAGS_ALLOW=--define-prefix @@ -46,12 +47,12 @@ stop: ##@Development Stop development servers (UI and backend) .PHONY: stop watch: ##@Development Start Go tests in watch mode (re-run when code changes) - go tool ginkgo watch -tags=netgo -notify ./... + go tool ginkgo watch -tags=$(GO_BUILD_TAGS) -notify ./... .PHONY: watch PKG ?= ./... test: ##@Development Run Go tests. Use PKG variable to specify packages to test, e.g. make test PKG=./server - go test -tags netgo $(PKG) + go test -tags $(GO_BUILD_TAGS) $(PKG) .PHONY: test test-ndpgen: ##@Development Run tests for ndpgen plugin @@ -62,7 +63,7 @@ testall: test test-ndpgen test-i18n test-js ##@Development Run Go and JS tests .PHONY: testall test-race: ##@Development Run Go tests with race detector - go test -tags netgo -race -shuffle=on $(PKG) + go test -tags $(GO_BUILD_TAGS) -race -shuffle=on $(PKG) .PHONY: test-race test-js: ##@Development Run JS tests @@ -108,7 +109,7 @@ format: ##@Development Format code .PHONY: format wire: check_go_env ##@Development Update Dependency Injection - go tool wire gen -tags=netgo ./... + go tool wire gen -tags=$(GO_BUILD_TAGS) ./... .PHONY: wire gen: check_go_env ##@Development Run go generate for code generation @@ -144,14 +145,14 @@ setup-git: ##@Development Setup Git hooks (pre-commit and pre-push) .PHONY: setup-git build: check_go_env buildjs ##@Build Build the project - go build -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=netgo + go build -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=$(GO_BUILD_TAGS) .PHONY: build buildall: deprecated build .PHONY: buildall debug-build: check_go_env buildjs ##@Build Build the project (with remote debug on) - go build -gcflags="all=-N -l" -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=netgo + go build -gcflags="all=-N -l" -ldflags="-X github.com/navidrome/navidrome/consts.gitSha=$(GIT_SHA) -X github.com/navidrome/navidrome/consts.gitTag=$(GIT_TAG)" -tags=$(GO_BUILD_TAGS) .PHONY: debug-build buildjs: check_node_env ui/build/index.html ##@Build Build only frontend diff --git a/conf/buildtags/buildtags.go b/conf/buildtags/buildtags.go deleted file mode 100644 index 5fc125087..000000000 --- a/conf/buildtags/buildtags.go +++ /dev/null @@ -1,4 +0,0 @@ -package buildtags - -// This file is left intentionally empty. It is used to make sure the package is not empty, in the case all -// required build tags are disabled. diff --git a/conf/buildtags/doc.go b/conf/buildtags/doc.go new file mode 100644 index 000000000..f637b6355 --- /dev/null +++ b/conf/buildtags/doc.go @@ -0,0 +1,6 @@ +// Package buildtags provides compile-time enforcement of required build tags. +// +// Each file in this package is guarded by a build constraint and exports a variable +// that main.go references. If a required tag is missing during compilation, the build +// fails with an "undefined" error, directing the developer to use `make build`. +package buildtags diff --git a/conf/buildtags/netgo.go b/conf/buildtags/netgo.go index 0062ad2bc..407004703 100644 --- a/conf/buildtags/netgo.go +++ b/conf/buildtags/netgo.go @@ -2,10 +2,6 @@ package buildtags -// NOTICE: This file was created to force the inclusion of the `netgo` tag when compiling the project. -// If the tag is not included, the compilation will fail because this variable won't be defined, and the `main.go` -// file requires it. - -// Why this tag is required? See https://github.com/navidrome/navidrome/issues/700 +// The `netgo` tag is required when compiling the project. See https://github.com/navidrome/navidrome/issues/700 var NETGO = true diff --git a/conf/buildtags/sqlite_fts5.go b/conf/buildtags/sqlite_fts5.go new file mode 100644 index 000000000..1476e04cd --- /dev/null +++ b/conf/buildtags/sqlite_fts5.go @@ -0,0 +1,8 @@ +//go:build sqlite_fts5 + +package buildtags + +// FTS5 is required for full-text search. Without this tag, the SQLite driver +// won't include FTS5 support, causing runtime failures on migrations and search queries. + +var SQLITE_FTS5 = true diff --git a/conf/configuration.go b/conf/configuration.go index 8994c1c88..000bffb58 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -58,7 +58,7 @@ type configOptions struct { SmartPlaylistRefreshDelay time.Duration AutoTranscodeDownload bool DefaultDownsamplingFormat string - SearchFullString bool + Search searchOptions `json:",omitzero"` SimilarSongsMatchThreshold int RecentlyAddedByModTime bool PreferSortTags bool @@ -82,6 +82,7 @@ type configOptions struct { DefaultTheme string DefaultLanguage string DefaultUIVolume int + UISearchDebounceMs int EnableReplayGain bool EnableCoverAnimation bool EnableNowPlaying bool @@ -251,6 +252,11 @@ type extAuthOptions struct { UserHeader string } +type searchOptions struct { + Backend string + FullString bool +} + var ( Server = &configOptions{} hooks []func() @@ -344,6 +350,8 @@ func Load(noConfigDump bool) { os.Exit(1) } + Server.Search.Backend = normalizeSearchBackend(Server.Search.Backend) + if Server.BaseURL != "" { u, err := url.Parse(Server.BaseURL) if err != nil { @@ -392,6 +400,7 @@ func Load(noConfigDump bool) { logDeprecatedOptions("Scanner.GenreSeparators", "") logDeprecatedOptions("Scanner.GroupAlbumReleases", "") logDeprecatedOptions("DevEnableBufferedScrobble", "") // Deprecated: Buffered scrobbling is now always enabled and this option is ignored + logDeprecatedOptions("SearchFullString", "Search.FullString") logDeprecatedOptions("ReverseProxyWhitelist", "ExtAuth.TrustedSources") logDeprecatedOptions("ReverseProxyUserHeader", "ExtAuth.UserHeader") logDeprecatedOptions("HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions") @@ -539,6 +548,17 @@ func validateSchedule(schedule, field string) (string, error) { return schedule, err } +func normalizeSearchBackend(value string) string { + v := strings.ToLower(strings.TrimSpace(value)) + switch v { + case "fts", "legacy": + return v + default: + log.Error("Invalid Search.Backend value, falling back to 'fts'", "value", value) + return "fts" + } +} + // AddHook is used to register initialization code that should run as soon as the config is loaded func AddHook(hook func()) { hooks = append(hooks, hook) @@ -585,7 +605,8 @@ func setViperDefaults() { viper.SetDefault("enablemediafilecoverart", true) viper.SetDefault("autotranscodedownload", false) viper.SetDefault("defaultdownsamplingformat", consts.DefaultDownsamplingFormat) - viper.SetDefault("searchfullstring", false) + viper.SetDefault("search.fullstring", false) + viper.SetDefault("search.backend", "fts") viper.SetDefault("similarsongsmatchthreshold", 85) viper.SetDefault("recentlyaddedbymodtime", false) viper.SetDefault("prefersorttags", false) @@ -604,6 +625,7 @@ func setViperDefaults() { viper.SetDefault("defaulttheme", "Dark") viper.SetDefault("defaultlanguage", "") viper.SetDefault("defaultuivolume", consts.DefaultUIVolume) + viper.SetDefault("uisearchdebouncems", consts.DefaultUISearchDebounceMs) viper.SetDefault("enablereplaygain", true) viper.SetDefault("enablecoveranimation", true) viper.SetDefault("enablenowplaying", true) diff --git a/conf/configuration_test.go b/conf/configuration_test.go index 90604f54f..b4ed6ca2d 100644 --- a/conf/configuration_test.go +++ b/conf/configuration_test.go @@ -52,6 +52,20 @@ var _ = Describe("Configuration", func() { }) }) + DescribeTable("NormalizeSearchBackend", + func(input, expected string) { + Expect(conf.NormalizeSearchBackend(input)).To(Equal(expected)) + }, + Entry("accepts 'fts'", "fts", "fts"), + Entry("accepts 'legacy'", "legacy", "legacy"), + Entry("normalizes 'FTS' to lowercase", "FTS", "fts"), + Entry("normalizes 'Legacy' to lowercase", "Legacy", "legacy"), + Entry("trims whitespace", " fts ", "fts"), + Entry("falls back to 'fts' for 'fts5'", "fts5", "fts"), + Entry("falls back to 'fts' for unrecognized values", "invalid", "fts"), + Entry("falls back to 'fts' for empty string", "", "fts"), + ) + DescribeTable("should load configuration from", func(format string) { filename := filepath.Join("testdata", "cfg."+format) diff --git a/conf/export_test.go b/conf/export_test.go index abd2fa5a4..7344dc4ca 100644 --- a/conf/export_test.go +++ b/conf/export_test.go @@ -7,3 +7,5 @@ func ResetConf() { var SetViperDefaults = setViperDefaults var ParseLanguages = parseLanguages + +var NormalizeSearchBackend = normalizeSearchBackend diff --git a/consts/consts.go b/consts/consts.go index eb10fdc03..ebde9d1d9 100644 --- a/consts/consts.go +++ b/consts/consts.go @@ -66,11 +66,12 @@ const ( I18nFolder = "i18n" ScanIgnoreFile = ".ndignore" - PlaceholderArtistArt = "artist-placeholder.webp" - PlaceholderAlbumArt = "album-placeholder.webp" - PlaceholderAvatar = "logo-192x192.png" - UICoverArtSize = 300 - DefaultUIVolume = 100 + PlaceholderArtistArt = "artist-placeholder.webp" + PlaceholderAlbumArt = "album-placeholder.webp" + PlaceholderAvatar = "logo-192x192.png" + UICoverArtSize = 300 + DefaultUIVolume = 100 + DefaultUISearchDebounceMs = 200 DefaultHttpClientTimeOut = 10 * time.Second diff --git a/core/metrics/insights.go b/core/metrics/insights.go index 849ddd6fc..f059d739a 100644 --- a/core/metrics/insights.go +++ b/core/metrics/insights.go @@ -208,7 +208,8 @@ var staticData = sync.OnceValue(func() insights.Data { data.Config.TranscodingCacheSize = conf.Server.TranscodingCacheSize data.Config.ImageCacheSize = conf.Server.ImageCacheSize data.Config.SessionTimeout = uint64(math.Trunc(conf.Server.SessionTimeout.Seconds())) - data.Config.SearchFullString = conf.Server.SearchFullString + data.Config.SearchFullString = conf.Server.Search.FullString + data.Config.SearchBackend = conf.Server.Search.Backend data.Config.RecentlyAddedByModTime = conf.Server.RecentlyAddedByModTime data.Config.PreferSortTags = conf.Server.PreferSortTags data.Config.BackupSchedule = conf.Server.Backup.Schedule diff --git a/core/metrics/insights/data.go b/core/metrics/insights/data.go index 99eb542a7..5580d895d 100644 --- a/core/metrics/insights/data.go +++ b/core/metrics/insights/data.go @@ -68,6 +68,7 @@ type Data struct { EnableNowPlaying bool `json:"enableNowPlaying,omitempty"` SessionTimeout uint64 `json:"sessionTimeout,omitempty"` SearchFullString bool `json:"searchFullString,omitempty"` + SearchBackend string `json:"searchBackend,omitempty"` RecentlyAddedByModTime bool `json:"recentlyAddedByModTime,omitempty"` PreferSortTags bool `json:"preferSortTags,omitempty"` BackupSchedule string `json:"backupSchedule,omitempty"` diff --git a/db/migrations/20260220173400_add_fts5_search.go b/db/migrations/20260220173400_add_fts5_search.go new file mode 100644 index 000000000..dc4cd647b --- /dev/null +++ b/db/migrations/20260220173400_add_fts5_search.go @@ -0,0 +1,391 @@ +package migrations + +import ( + "context" + "database/sql" + "fmt" + + "github.com/pressly/goose/v3" +) + +func init() { + goose.AddMigrationContext(upAddFts5Search, downAddFts5Search) +} + +// stripPunct generates a SQL expression that strips common punctuation from a column or expression. +// Used during migration to approximate the Go normalizeForFTS function for bulk-populating search_normalized. +func stripPunct(col string) string { + return fmt.Sprintf( + `REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(%s, '.', ''), '/', ''), '-', ''), '''', ''), '&', ''), ',', '')`, + col, + ) +} + +func upAddFts5Search(ctx context.Context, tx *sql.Tx) error { + notice(tx, "Adding FTS5 full-text search indexes. This may take a moment on large libraries.") + + // Step 1: Add search_participants and search_normalized columns to media_file, album, and artist + _, err := tx.ExecContext(ctx, `ALTER TABLE media_file ADD COLUMN search_participants TEXT NOT NULL DEFAULT ''`) + if err != nil { + return fmt.Errorf("adding search_participants to media_file: %w", err) + } + _, err = tx.ExecContext(ctx, `ALTER TABLE media_file ADD COLUMN search_normalized TEXT NOT NULL DEFAULT ''`) + if err != nil { + return fmt.Errorf("adding search_normalized to media_file: %w", err) + } + _, err = tx.ExecContext(ctx, `ALTER TABLE album ADD COLUMN search_participants TEXT NOT NULL DEFAULT ''`) + if err != nil { + return fmt.Errorf("adding search_participants to album: %w", err) + } + _, err = tx.ExecContext(ctx, `ALTER TABLE album ADD COLUMN search_normalized TEXT NOT NULL DEFAULT ''`) + if err != nil { + return fmt.Errorf("adding search_normalized to album: %w", err) + } + _, err = tx.ExecContext(ctx, `ALTER TABLE artist ADD COLUMN search_normalized TEXT NOT NULL DEFAULT ''`) + if err != nil { + return fmt.Errorf("adding search_normalized to artist: %w", err) + } + + // Step 2: Populate search_participants from participants JSON. + // Extract all "name" values from the participants JSON structure. + // participants is a JSON object like: {"artist":[{"name":"...","id":"..."}],"albumartist":[...]} + // We use json_each + json_extract to flatten all names into a space-separated string. + _, err = tx.ExecContext(ctx, ` + UPDATE media_file SET search_participants = COALESCE( + (SELECT group_concat(json_extract(je2.value, '$.name'), ' ') + FROM json_each(media_file.participants) AS je1, + json_each(je1.value) AS je2 + WHERE json_extract(je2.value, '$.name') IS NOT NULL), + '' + ) + WHERE participants IS NOT NULL AND participants != '' AND participants != '{}' + `) + if err != nil { + return fmt.Errorf("populating media_file search_participants: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + UPDATE album SET search_participants = COALESCE( + (SELECT group_concat(json_extract(je2.value, '$.name'), ' ') + FROM json_each(album.participants) AS je1, + json_each(je1.value) AS je2 + WHERE json_extract(je2.value, '$.name') IS NOT NULL), + '' + ) + WHERE participants IS NOT NULL AND participants != '' AND participants != '{}' + `) + if err != nil { + return fmt.Errorf("populating album search_participants: %w", err) + } + + // Step 2b: Populate search_normalized using SQL REPLACE chains for common punctuation. + // The Go code will compute the precise value on next scan; this is a best-effort approximation. + _, err = tx.ExecContext(ctx, fmt.Sprintf(` + UPDATE artist SET search_normalized = %s + WHERE name != %s`, + stripPunct("name"), stripPunct("name"))) + if err != nil { + return fmt.Errorf("populating artist search_normalized: %w", err) + } + + _, err = tx.ExecContext(ctx, fmt.Sprintf(` + UPDATE album SET search_normalized = TRIM(%s || ' ' || %s) + WHERE name != %s OR COALESCE(album_artist, '') != %s`, + stripPunct("name"), stripPunct("COALESCE(album_artist, '')"), + stripPunct("name"), stripPunct("COALESCE(album_artist, '')"))) + if err != nil { + return fmt.Errorf("populating album search_normalized: %w", err) + } + + _, err = tx.ExecContext(ctx, fmt.Sprintf(` + UPDATE media_file SET search_normalized = + TRIM(%s || ' ' || %s || ' ' || %s || ' ' || %s) + WHERE title != %s + OR COALESCE(album, '') != %s + OR COALESCE(artist, '') != %s + OR COALESCE(album_artist, '') != %s`, + stripPunct("title"), stripPunct("COALESCE(album, '')"), + stripPunct("COALESCE(artist, '')"), stripPunct("COALESCE(album_artist, '')"), + stripPunct("title"), stripPunct("COALESCE(album, '')"), + stripPunct("COALESCE(artist, '')"), stripPunct("COALESCE(album_artist, '')"))) + if err != nil { + return fmt.Errorf("populating media_file search_normalized: %w", err) + } + + // Step 3: Create FTS5 virtual tables + _, err = tx.ExecContext(ctx, ` + CREATE VIRTUAL TABLE IF NOT EXISTS media_file_fts USING fts5( + title, album, artist, album_artist, + sort_title, sort_album_name, sort_artist_name, sort_album_artist_name, + disc_subtitle, search_participants, search_normalized, + content='', content_rowid='rowid', + tokenize='unicode61 remove_diacritics 2' + ) + `) + if err != nil { + return fmt.Errorf("creating media_file_fts: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE VIRTUAL TABLE IF NOT EXISTS album_fts USING fts5( + name, sort_album_name, album_artist, + search_participants, discs, catalog_num, album_version, search_normalized, + content='', content_rowid='rowid', + tokenize='unicode61 remove_diacritics 2' + ) + `) + if err != nil { + return fmt.Errorf("creating album_fts: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE VIRTUAL TABLE IF NOT EXISTS artist_fts USING fts5( + name, sort_artist_name, search_normalized, + content='', content_rowid='rowid', + tokenize='unicode61 remove_diacritics 2' + ) + `) + if err != nil { + return fmt.Errorf("creating artist_fts: %w", err) + } + + // Step 4: Bulk-populate FTS5 indexes from existing data + _, err = tx.ExecContext(ctx, ` + INSERT INTO media_file_fts(rowid, title, album, artist, album_artist, + sort_title, sort_album_name, sort_artist_name, sort_album_artist_name, + disc_subtitle, search_participants, search_normalized) + SELECT rowid, title, album, artist, album_artist, + sort_title, sort_album_name, sort_artist_name, sort_album_artist_name, + COALESCE(disc_subtitle, ''), COALESCE(search_participants, ''), + COALESCE(search_normalized, '') + FROM media_file + `) + if err != nil { + return fmt.Errorf("populating media_file_fts: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + INSERT INTO album_fts(rowid, name, sort_album_name, album_artist, + search_participants, discs, catalog_num, album_version, search_normalized) + SELECT rowid, name, COALESCE(sort_album_name, ''), COALESCE(album_artist, ''), + COALESCE(search_participants, ''), COALESCE(discs, ''), + COALESCE(catalog_num, ''), + COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ') + FROM json_each(album.tags, '$.albumversion') AS je), ''), + COALESCE(search_normalized, '') + FROM album + `) + if err != nil { + return fmt.Errorf("populating album_fts: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + INSERT INTO artist_fts(rowid, name, sort_artist_name, search_normalized) + SELECT rowid, name, COALESCE(sort_artist_name, ''), COALESCE(search_normalized, '') + FROM artist + `) + if err != nil { + return fmt.Errorf("populating artist_fts: %w", err) + } + + // Step 5: Create triggers for media_file + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER media_file_fts_ai AFTER INSERT ON media_file BEGIN + INSERT INTO media_file_fts(rowid, title, album, artist, album_artist, + sort_title, sort_album_name, sort_artist_name, sort_album_artist_name, + disc_subtitle, search_participants, search_normalized) + VALUES (NEW.rowid, NEW.title, NEW.album, NEW.artist, NEW.album_artist, + NEW.sort_title, NEW.sort_album_name, NEW.sort_artist_name, NEW.sort_album_artist_name, + COALESCE(NEW.disc_subtitle, ''), COALESCE(NEW.search_participants, ''), + COALESCE(NEW.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating media_file_fts insert trigger: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER media_file_fts_ad AFTER DELETE ON media_file BEGIN + INSERT INTO media_file_fts(media_file_fts, rowid, title, album, artist, album_artist, + sort_title, sort_album_name, sort_artist_name, sort_album_artist_name, + disc_subtitle, search_participants, search_normalized) + VALUES ('delete', OLD.rowid, OLD.title, OLD.album, OLD.artist, OLD.album_artist, + OLD.sort_title, OLD.sort_album_name, OLD.sort_artist_name, OLD.sort_album_artist_name, + COALESCE(OLD.disc_subtitle, ''), COALESCE(OLD.search_participants, ''), + COALESCE(OLD.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating media_file_fts delete trigger: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER media_file_fts_au AFTER UPDATE ON media_file + WHEN + OLD.title IS NOT NEW.title OR + OLD.album IS NOT NEW.album OR + OLD.artist IS NOT NEW.artist OR + OLD.album_artist IS NOT NEW.album_artist OR + OLD.sort_title IS NOT NEW.sort_title OR + OLD.sort_album_name IS NOT NEW.sort_album_name OR + OLD.sort_artist_name IS NOT NEW.sort_artist_name OR + OLD.sort_album_artist_name IS NOT NEW.sort_album_artist_name OR + OLD.disc_subtitle IS NOT NEW.disc_subtitle OR + OLD.search_participants IS NOT NEW.search_participants OR + OLD.search_normalized IS NOT NEW.search_normalized + BEGIN + INSERT INTO media_file_fts(media_file_fts, rowid, title, album, artist, album_artist, + sort_title, sort_album_name, sort_artist_name, sort_album_artist_name, + disc_subtitle, search_participants, search_normalized) + VALUES ('delete', OLD.rowid, OLD.title, OLD.album, OLD.artist, OLD.album_artist, + OLD.sort_title, OLD.sort_album_name, OLD.sort_artist_name, OLD.sort_album_artist_name, + COALESCE(OLD.disc_subtitle, ''), COALESCE(OLD.search_participants, ''), + COALESCE(OLD.search_normalized, '')); + INSERT INTO media_file_fts(rowid, title, album, artist, album_artist, + sort_title, sort_album_name, sort_artist_name, sort_album_artist_name, + disc_subtitle, search_participants, search_normalized) + VALUES (NEW.rowid, NEW.title, NEW.album, NEW.artist, NEW.album_artist, + NEW.sort_title, NEW.sort_album_name, NEW.sort_artist_name, NEW.sort_album_artist_name, + COALESCE(NEW.disc_subtitle, ''), COALESCE(NEW.search_participants, ''), + COALESCE(NEW.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating media_file_fts update trigger: %w", err) + } + + // Step 6: Create triggers for album + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER album_fts_ai AFTER INSERT ON album BEGIN + INSERT INTO album_fts(rowid, name, sort_album_name, album_artist, + search_participants, discs, catalog_num, album_version, search_normalized) + VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_album_name, ''), COALESCE(NEW.album_artist, ''), + COALESCE(NEW.search_participants, ''), COALESCE(NEW.discs, ''), + COALESCE(NEW.catalog_num, ''), + COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ') + FROM json_each(NEW.tags, '$.albumversion') AS je), ''), + COALESCE(NEW.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating album_fts insert trigger: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER album_fts_ad AFTER DELETE ON album BEGIN + INSERT INTO album_fts(album_fts, rowid, name, sort_album_name, album_artist, + search_participants, discs, catalog_num, album_version, search_normalized) + VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_album_name, ''), COALESCE(OLD.album_artist, ''), + COALESCE(OLD.search_participants, ''), COALESCE(OLD.discs, ''), + COALESCE(OLD.catalog_num, ''), + COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ') + FROM json_each(OLD.tags, '$.albumversion') AS je), ''), + COALESCE(OLD.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating album_fts delete trigger: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER album_fts_au AFTER UPDATE ON album + WHEN + OLD.name IS NOT NEW.name OR + OLD.sort_album_name IS NOT NEW.sort_album_name OR + OLD.album_artist IS NOT NEW.album_artist OR + OLD.search_participants IS NOT NEW.search_participants OR + OLD.discs IS NOT NEW.discs OR + OLD.catalog_num IS NOT NEW.catalog_num OR + OLD.tags IS NOT NEW.tags OR + OLD.search_normalized IS NOT NEW.search_normalized + BEGIN + INSERT INTO album_fts(album_fts, rowid, name, sort_album_name, album_artist, + search_participants, discs, catalog_num, album_version, search_normalized) + VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_album_name, ''), COALESCE(OLD.album_artist, ''), + COALESCE(OLD.search_participants, ''), COALESCE(OLD.discs, ''), + COALESCE(OLD.catalog_num, ''), + COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ') + FROM json_each(OLD.tags, '$.albumversion') AS je), ''), + COALESCE(OLD.search_normalized, '')); + INSERT INTO album_fts(rowid, name, sort_album_name, album_artist, + search_participants, discs, catalog_num, album_version, search_normalized) + VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_album_name, ''), COALESCE(NEW.album_artist, ''), + COALESCE(NEW.search_participants, ''), COALESCE(NEW.discs, ''), + COALESCE(NEW.catalog_num, ''), + COALESCE((SELECT group_concat(json_extract(je.value, '$.value'), ' ') + FROM json_each(NEW.tags, '$.albumversion') AS je), ''), + COALESCE(NEW.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating album_fts update trigger: %w", err) + } + + // Step 7: Create triggers for artist + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER artist_fts_ai AFTER INSERT ON artist BEGIN + INSERT INTO artist_fts(rowid, name, sort_artist_name, search_normalized) + VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_artist_name, ''), + COALESCE(NEW.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating artist_fts insert trigger: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER artist_fts_ad AFTER DELETE ON artist BEGIN + INSERT INTO artist_fts(artist_fts, rowid, name, sort_artist_name, search_normalized) + VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_artist_name, ''), + COALESCE(OLD.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating artist_fts delete trigger: %w", err) + } + + _, err = tx.ExecContext(ctx, ` + CREATE TRIGGER artist_fts_au AFTER UPDATE ON artist + WHEN + OLD.name IS NOT NEW.name OR + OLD.sort_artist_name IS NOT NEW.sort_artist_name OR + OLD.search_normalized IS NOT NEW.search_normalized + BEGIN + INSERT INTO artist_fts(artist_fts, rowid, name, sort_artist_name, search_normalized) + VALUES ('delete', OLD.rowid, OLD.name, COALESCE(OLD.sort_artist_name, ''), + COALESCE(OLD.search_normalized, '')); + INSERT INTO artist_fts(rowid, name, sort_artist_name, search_normalized) + VALUES (NEW.rowid, NEW.name, COALESCE(NEW.sort_artist_name, ''), + COALESCE(NEW.search_normalized, '')); + END + `) + if err != nil { + return fmt.Errorf("creating artist_fts update trigger: %w", err) + } + + return nil +} + +func downAddFts5Search(ctx context.Context, tx *sql.Tx) error { + for _, trigger := range []string{ + "media_file_fts_ai", "media_file_fts_ad", "media_file_fts_au", + "album_fts_ai", "album_fts_ad", "album_fts_au", + "artist_fts_ai", "artist_fts_ad", "artist_fts_au", + } { + _, err := tx.ExecContext(ctx, "DROP TRIGGER IF EXISTS "+trigger) + if err != nil { + return fmt.Errorf("dropping trigger %s: %w", trigger, err) + } + } + + for _, table := range []string{"media_file_fts", "album_fts", "artist_fts"} { + _, err := tx.ExecContext(ctx, "DROP TABLE IF EXISTS "+table) + if err != nil { + return fmt.Errorf("dropping table %s: %w", table, err) + } + } + + // Note: We don't drop search_participants columns because SQLite doesn't support DROP COLUMN + // on older versions, and the column is harmless if left in place. + return nil +} diff --git a/main.go b/main.go index 65db162ac..b5fb508b4 100644 --- a/main.go +++ b/main.go @@ -9,11 +9,12 @@ import ( //goland:noinspection GoBoolExpressions func main() { - // This import is used to force the inclusion of the `netgo` tag when compiling the project. + // These references force the inclusion of build tags when compiling the project. // If you get compilation errors like "undefined: buildtags.NETGO", this means you forgot to specify - // the `netgo` build tag when compiling the project. + // the required build tags when compiling the project. // To avoid these kind of errors, you should use `make build` to compile the project. _ = buildtags.NETGO + _ = buildtags.SQLITE_FTS5 cmd.Execute() } diff --git a/persistence/album_repository.go b/persistence/album_repository.go index 651953a12..58bfcca51 100644 --- a/persistence/album_repository.go +++ b/persistence/album_repository.go @@ -62,11 +62,14 @@ func (a *dbAlbum) PostScan() error { func (a *dbAlbum) PostMapArgs(args map[string]any) error { fullText := []string{a.Name, a.SortAlbumName, a.AlbumArtist} - fullText = append(fullText, a.Album.Participants.AllNames()...) + participantNames := a.Album.Participants.AllNames() + fullText = append(fullText, participantNames...) fullText = append(fullText, slices.Collect(maps.Values(a.Album.Discs))...) fullText = append(fullText, a.Album.Tags[model.TagAlbumVersion]...) fullText = append(fullText, a.Album.Tags[model.TagCatalogNumber]...) args["full_text"] = formatFullText(fullText...) + args["search_participants"] = strings.Join(participantNames, " ") + args["search_normalized"] = normalizeForFTS(a.Name, a.AlbumArtist) args["tags"] = marshalTags(a.Album.Tags) args["participants"] = marshalParticipants(a.Album.Participants) diff --git a/persistence/album_repository_test.go b/persistence/album_repository_test.go index 8c82dc47c..9fbc6b974 100644 --- a/persistence/album_repository_test.go +++ b/persistence/album_repository_test.go @@ -56,17 +56,23 @@ var _ = Describe("AlbumRepository", func() { It("returns all records sorted", func() { Expect(GetAll(model.QueryOptions{Sort: "name"})).To(Equal(model.Albums{ albumAbbeyRoad, + albumWithVersion, + albumCJK, albumMultiDisc, albumRadioactivity, albumSgtPeppers, + albumPunctuation, })) }) It("returns all records sorted desc", func() { Expect(GetAll(model.QueryOptions{Sort: "name", Order: "desc"})).To(Equal(model.Albums{ + albumPunctuation, albumSgtPeppers, albumRadioactivity, albumMultiDisc, + albumCJK, + albumWithVersion, albumAbbeyRoad, })) }) diff --git a/persistence/artist_repository.go b/persistence/artist_repository.go index 5623bd7f8..f801787d8 100644 --- a/persistence/artist_repository.go +++ b/persistence/artist_repository.go @@ -102,6 +102,7 @@ func (a *dbArtist) PostMapArgs(m map[string]any) error { similarArtists, _ := json.Marshal(sa) m["similar_artists"] = string(similarArtists) m["full_text"] = formatFullText(a.Name, a.SortArtistName) + m["search_normalized"] = normalizeForFTS(a.Name) // Do not override the sort_artist_name and mbz_artist_id fields if they are empty // TODO: Better way to handle this? diff --git a/persistence/artist_repository_test.go b/persistence/artist_repository_test.go index 18883378d..15340eeb4 100644 --- a/persistence/artist_repository_test.go +++ b/persistence/artist_repository_test.go @@ -193,7 +193,7 @@ var _ = Describe("ArtistRepository", func() { Describe("Basic Operations", func() { Describe("Count", func() { It("returns the number of artists in the DB", func() { - Expect(repo.CountAll()).To(Equal(int64(2))) + Expect(repo.CountAll()).To(Equal(int64(4))) }) }) @@ -228,13 +228,19 @@ var _ = Describe("ArtistRepository", func() { idx, err := repo.GetIndex(false, []int{1}) Expect(err).ToNot(HaveOccurred()) - Expect(idx).To(HaveLen(2)) + Expect(idx).To(HaveLen(4)) Expect(idx[0].ID).To(Equal("F")) Expect(idx[0].Artists).To(HaveLen(1)) Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name)) Expect(idx[1].ID).To(Equal("K")) Expect(idx[1].Artists).To(HaveLen(1)) Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name)) + Expect(idx[2].ID).To(Equal("R")) + Expect(idx[2].Artists).To(HaveLen(1)) + Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name)) + Expect(idx[3].ID).To(Equal("S")) + Expect(idx[3].Artists).To(HaveLen(1)) + Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name)) // Restore the original value artistBeatles.SortArtistName = "" @@ -246,13 +252,19 @@ var _ = Describe("ArtistRepository", func() { XIt("returns the index when PreferSortTags is true and SortArtistName is empty", func() { idx, err := repo.GetIndex(false, []int{1}) Expect(err).ToNot(HaveOccurred()) - Expect(idx).To(HaveLen(2)) + Expect(idx).To(HaveLen(4)) Expect(idx[0].ID).To(Equal("B")) Expect(idx[0].Artists).To(HaveLen(1)) Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name)) Expect(idx[1].ID).To(Equal("K")) Expect(idx[1].Artists).To(HaveLen(1)) Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name)) + Expect(idx[2].ID).To(Equal("R")) + Expect(idx[2].Artists).To(HaveLen(1)) + Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name)) + Expect(idx[3].ID).To(Equal("S")) + Expect(idx[3].Artists).To(HaveLen(1)) + Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name)) }) }) @@ -268,13 +280,19 @@ var _ = Describe("ArtistRepository", func() { idx, err := repo.GetIndex(false, []int{1}) Expect(err).ToNot(HaveOccurred()) - Expect(idx).To(HaveLen(2)) + Expect(idx).To(HaveLen(4)) Expect(idx[0].ID).To(Equal("B")) Expect(idx[0].Artists).To(HaveLen(1)) Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name)) Expect(idx[1].ID).To(Equal("K")) Expect(idx[1].Artists).To(HaveLen(1)) Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name)) + Expect(idx[2].ID).To(Equal("R")) + Expect(idx[2].Artists).To(HaveLen(1)) + Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name)) + Expect(idx[3].ID).To(Equal("S")) + Expect(idx[3].Artists).To(HaveLen(1)) + Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name)) // Restore the original value artistBeatles.SortArtistName = "" @@ -285,13 +303,19 @@ var _ = Describe("ArtistRepository", func() { It("returns the index when SortArtistName is empty", func() { idx, err := repo.GetIndex(false, []int{1}) Expect(err).ToNot(HaveOccurred()) - Expect(idx).To(HaveLen(2)) + Expect(idx).To(HaveLen(4)) Expect(idx[0].ID).To(Equal("B")) Expect(idx[0].Artists).To(HaveLen(1)) Expect(idx[0].Artists[0].Name).To(Equal(artistBeatles.Name)) Expect(idx[1].ID).To(Equal("K")) Expect(idx[1].Artists).To(HaveLen(1)) Expect(idx[1].Artists[0].Name).To(Equal(artistKraftwerk.Name)) + Expect(idx[2].ID).To(Equal("R")) + Expect(idx[2].Artists).To(HaveLen(1)) + Expect(idx[2].Artists[0].Name).To(Equal(artistPunctuation.Name)) + Expect(idx[3].ID).To(Equal("S")) + Expect(idx[3].Artists).To(HaveLen(1)) + Expect(idx[3].Artists[0].Name).To(Equal(artistCJK.Name)) }) }) @@ -377,7 +401,7 @@ var _ = Describe("ArtistRepository", func() { // Admin users can see all content when valid library IDs are provided idx, err := repo.GetIndex(false, []int{1}) Expect(err).ToNot(HaveOccurred()) - Expect(idx).To(HaveLen(2)) + Expect(idx).To(HaveLen(4)) // With non-existent library ID, admin users see no content because no artists are associated with that library idx, err = repo.GetIndex(false, []int{999}) @@ -625,11 +649,11 @@ var _ = Describe("ArtistRepository", func() { It("sees all artists regardless of library permissions", func() { count, err := repo.CountAll() Expect(err).ToNot(HaveOccurred()) - Expect(count).To(Equal(int64(2))) + Expect(count).To(Equal(int64(4))) artists, err := repo.GetAll() Expect(err).ToNot(HaveOccurred()) - Expect(artists).To(HaveLen(2)) + Expect(artists).To(HaveLen(4)) exists, err := repo.Exists(artistBeatles.ID) Expect(err).ToNot(HaveOccurred()) @@ -661,7 +685,7 @@ var _ = Describe("ArtistRepository", func() { // Should see missing artist in GetAll by default for admin users artists, err := repo.GetAll() Expect(err).ToNot(HaveOccurred()) - Expect(artists).To(HaveLen(3)) // Including the missing artist + Expect(artists).To(HaveLen(5)) // Including the missing artist // Search never returns missing artists (hardcoded behavior) results, err := repo.Search("Missing Artist", 0, 10) @@ -767,19 +791,19 @@ var _ = Describe("ArtistRepository", func() { It("CountAll returns correct count after gaining access", func() { count, err := restrictedRepo.CountAll() Expect(err).ToNot(HaveOccurred()) - Expect(count).To(Equal(int64(2))) // Beatles and Kraftwerk + Expect(count).To(Equal(int64(4))) // Beatles, Kraftwerk, Seatbelts, and The Roots }) It("GetAll returns artists after gaining access", func() { artists, err := restrictedRepo.GetAll() Expect(err).ToNot(HaveOccurred()) - Expect(artists).To(HaveLen(2)) + Expect(artists).To(HaveLen(4)) var names []string for _, artist := range artists { names = append(names, artist.Name) } - Expect(names).To(ContainElements("The Beatles", "Kraftwerk")) + Expect(names).To(ContainElements("The Beatles", "Kraftwerk", "シートベルツ", "The Roots")) }) It("Exists returns true for accessible artists", func() { @@ -796,7 +820,7 @@ var _ = Describe("ArtistRepository", func() { // With valid library access, should see artists idx, err := restrictedRepo.GetIndex(false, []int{1}) Expect(err).ToNot(HaveOccurred()) - Expect(idx).To(HaveLen(2)) + Expect(idx).To(HaveLen(4)) // With non-existent library ID, should see nothing (non-admin user) idx, err = restrictedRepo.GetIndex(false, []int{999}) diff --git a/persistence/mediafile_repository.go b/persistence/mediafile_repository.go index 617cce4c8..264be6f3f 100644 --- a/persistence/mediafile_repository.go +++ b/persistence/mediafile_repository.go @@ -58,8 +58,11 @@ func (m *dbMediaFile) PostScan() error { func (m *dbMediaFile) PostMapArgs(args map[string]any) error { fullText := []string{m.FullTitle(), m.Album, m.Artist, m.AlbumArtist, m.SortTitle, m.SortAlbumName, m.SortArtistName, m.SortAlbumArtistName, m.DiscSubtitle} - fullText = append(fullText, m.MediaFile.Participants.AllNames()...) + participantNames := m.MediaFile.Participants.AllNames() + fullText = append(fullText, participantNames...) args["full_text"] = formatFullText(fullText...) + args["search_participants"] = strings.Join(participantNames, " ") + args["search_normalized"] = normalizeForFTS(m.FullTitle(), m.Album, m.Artist, m.AlbumArtist) args["tags"] = marshalTags(m.MediaFile.Tags) args["participants"] = marshalParticipants(m.MediaFile.Participants) return nil diff --git a/persistence/mediafile_repository_test.go b/persistence/mediafile_repository_test.go index 84cfd464b..853480b32 100644 --- a/persistence/mediafile_repository_test.go +++ b/persistence/mediafile_repository_test.go @@ -39,7 +39,7 @@ var _ = Describe("MediaRepository", func() { }) It("counts the number of mediafiles in the DB", func() { - Expect(mr.CountAll()).To(Equal(int64(10))) + Expect(mr.CountAll()).To(Equal(int64(13))) }) Describe("CountBySuffix", func() { diff --git a/persistence/persistence_suite_test.go b/persistence/persistence_suite_test.go index 559ca3d4c..0ee1570a1 100644 --- a/persistence/persistence_suite_test.go +++ b/persistence/persistence_suite_test.go @@ -56,12 +56,22 @@ func al(al model.Album) model.Album { return al } +func alWithTags(a model.Album, tags model.Tags) model.Album { + a = al(a) + a.Tags = tags + return a +} + var ( - artistKraftwerk = model.Artist{ID: "2", Name: "Kraftwerk", OrderArtistName: "kraftwerk"} - artistBeatles = model.Artist{ID: "3", Name: "The Beatles", OrderArtistName: "beatles"} - testArtists = model.Artists{ + artistKraftwerk = model.Artist{ID: "2", Name: "Kraftwerk", OrderArtistName: "kraftwerk"} + artistBeatles = model.Artist{ID: "3", Name: "The Beatles", OrderArtistName: "beatles"} + artistCJK = model.Artist{ID: "4", Name: "シートベルツ", SortArtistName: "Seatbelts", OrderArtistName: "seatbelts"} + artistPunctuation = model.Artist{ID: "5", Name: "The Roots", OrderArtistName: "roots"} + testArtists = model.Artists{ artistKraftwerk, artistBeatles, + artistCJK, + artistPunctuation, } ) @@ -70,11 +80,18 @@ var ( albumAbbeyRoad = al(model.Album{ID: "102", Name: "Abbey Road", AlbumArtist: "The Beatles", OrderAlbumName: "abbey road", AlbumArtistID: "3", EmbedArtPath: p("/beatles/1/come together.mp3"), SongCount: 1, MaxYear: 1969}) albumRadioactivity = al(model.Album{ID: "103", Name: "Radioactivity", AlbumArtist: "Kraftwerk", OrderAlbumName: "radioactivity", AlbumArtistID: "2", EmbedArtPath: p("/kraft/radio/radio.mp3"), SongCount: 2}) albumMultiDisc = al(model.Album{ID: "104", Name: "Multi Disc Album", AlbumArtist: "Test Artist", OrderAlbumName: "multi disc album", AlbumArtistID: "1", EmbedArtPath: p("/test/multi/disc1/track1.mp3"), SongCount: 4}) - testAlbums = model.Albums{ + albumCJK = al(model.Album{ID: "105", Name: "COWBOY BEBOP", AlbumArtist: "シートベルツ", OrderAlbumName: "cowboy bebop", AlbumArtistID: "4", EmbedArtPath: p("/seatbelts/cowboy-bebop/track1.mp3"), SongCount: 1}) + albumWithVersion = alWithTags(model.Album{ID: "106", Name: "Abbey Road", AlbumArtist: "The Beatles", OrderAlbumName: "abbey road", AlbumArtistID: "3", EmbedArtPath: p("/beatles/2/come together.mp3"), SongCount: 1, MaxYear: 2019}, + model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}}) + albumPunctuation = al(model.Album{ID: "107", Name: "Things Fall Apart", AlbumArtist: "The Roots", OrderAlbumName: "things fall apart", AlbumArtistID: "5", EmbedArtPath: p("/roots/things/track1.mp3"), SongCount: 1}) + testAlbums = model.Albums{ albumSgtPeppers, albumAbbeyRoad, albumRadioactivity, albumMultiDisc, + albumCJK, + albumWithVersion, + albumPunctuation, } ) @@ -101,6 +118,9 @@ var ( songDisc1Track01 = mf(model.MediaFile{ID: "2002", Title: "Disc 1 Track 1", ArtistID: "1", Artist: "Test Artist", AlbumID: "104", Album: "Multi Disc Album", DiscNumber: 1, TrackNumber: 1, Path: p("/test/multi/disc1/track1.mp3"), OrderAlbumName: "multi disc album", OrderArtistName: "test artist"}) songDisc2Track01 = mf(model.MediaFile{ID: "2003", Title: "Disc 2 Track 1", ArtistID: "1", Artist: "Test Artist", AlbumID: "104", Album: "Multi Disc Album", DiscNumber: 2, TrackNumber: 1, Path: p("/test/multi/disc2/track1.mp3"), OrderAlbumName: "multi disc album", OrderArtistName: "test artist"}) songDisc1Track02 = mf(model.MediaFile{ID: "2004", Title: "Disc 1 Track 2", ArtistID: "1", Artist: "Test Artist", AlbumID: "104", Album: "Multi Disc Album", DiscNumber: 1, TrackNumber: 2, Path: p("/test/multi/disc1/track2.mp3"), OrderAlbumName: "multi disc album", OrderArtistName: "test artist"}) + songCJK = mf(model.MediaFile{ID: "3001", Title: "プラチナ・ジェット", ArtistID: "4", Artist: "シートベルツ", AlbumID: "105", Album: "COWBOY BEBOP", Path: p("/seatbelts/cowboy-bebop/track1.mp3")}) + songVersioned = mf(model.MediaFile{ID: "3002", Title: "Come Together", ArtistID: "3", Artist: "The Beatles", AlbumID: "106", Album: "Abbey Road", Path: p("/beatles/2/come together.mp3")}) + songPunctuation = mf(model.MediaFile{ID: "3003", Title: "!!!!!!!", ArtistID: "5", Artist: "The Roots", AlbumID: "107", Album: "Things Fall Apart", Path: p("/roots/things/track1.mp3")}) testSongs = model.MediaFiles{ songDayInALife, songComeTogether, @@ -112,6 +132,9 @@ var ( songDisc1Track01, songDisc2Track01, songDisc1Track02, + songCJK, + songVersioned, + songPunctuation, } ) diff --git a/persistence/sql_restful.go b/persistence/sql_restful.go index ff0d06a8b..27207c45d 100644 --- a/persistence/sql_restful.go +++ b/persistence/sql_restful.go @@ -109,9 +109,10 @@ func booleanFilter(field string, value any) Sqlizer { func fullTextFilter(tableName string, mbidFields ...string) func(string, any) Sqlizer { return func(field string, value any) Sqlizer { v := strings.ToLower(value.(string)) + searchExpr := getSearchExpr() cond := cmp.Or( mbidExpr(tableName, v, mbidFields...), - fullTextExpr(tableName, v), + searchExpr(tableName, v), ) return cond } diff --git a/persistence/sql_restful_test.go b/persistence/sql_restful_test.go index e3399df48..ea0f802af 100644 --- a/persistence/sql_restful_test.go +++ b/persistence/sql_restful_test.go @@ -26,7 +26,9 @@ var _ = Describe("sqlRestful", func() { Expect(r.parseRestFilters(context.Background(), options)).To(BeNil()) }) - It(`returns nil if tries a filter with fullTextExpr("'")`, func() { + It(`returns nil if tries a filter with legacySearchExpr("'")`, func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" r.filterMappings = map[string]filterFunc{ "name": fullTextFilter("table"), } @@ -77,6 +79,7 @@ var _ = Describe("sqlRestful", func() { BeforeEach(func() { DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" tableName = "test_table" mbidFields = []string{"mbid", "artist_mbid"} filter = fullTextFilter(tableName, mbidFields...) @@ -136,7 +139,7 @@ var _ = Describe("sqlRestful", func() { Context("when SearchFullString config changes behavior", func() { It("uses different separator with SearchFullString=false", func() { - conf.Server.SearchFullString = false + conf.Server.Search.FullString = false result := filter("search", "test query") andCondition, ok := result.(squirrel.And) @@ -149,7 +152,7 @@ var _ = Describe("sqlRestful", func() { }) It("uses no separator with SearchFullString=true", func() { - conf.Server.SearchFullString = true + conf.Server.Search.FullString = true result := filter("search", "test query") andCondition, ok := result.(squirrel.And) diff --git a/persistence/sql_search.go b/persistence/sql_search.go index 0d3bfb743..e5c245bdf 100644 --- a/persistence/sql_search.go +++ b/persistence/sql_search.go @@ -6,6 +6,7 @@ import ( . "github.com/Masterminds/squirrel" "github.com/google/uuid" "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/str" ) @@ -15,6 +16,26 @@ func formatFullText(text ...string) string { return " " + fullText } +// searchExprFunc is the function signature for search expression builders. +type searchExprFunc func(tableName string, query string) Sqlizer + +// getSearchExpr returns the active search expression function based on config. +// It falls back to legacySearchExpr when Search.FullString is enabled, because +// FTS5 is token-based and cannot match substrings within words. +// CJK queries are routed to likeSearchExpr, since FTS5's unicode61 tokenizer +// cannot segment CJK text. +func getSearchExpr() searchExprFunc { + if conf.Server.Search.Backend == "legacy" || conf.Server.Search.FullString { + return legacySearchExpr + } + return func(tableName, query string) Sqlizer { + if containsCJK(query) { + return likeSearchExpr(tableName, query) + } + return ftsSearchExpr(tableName, query) + } +} + // doSearch performs a full-text search with the specified parameters. // The naturalOrder is used to sort results when no full-text filter is applied. It is useful for cases like // OpenSubsonic, where an empty search query should return all results in a natural order. Normally the parameter @@ -26,7 +47,8 @@ func (r sqlRepository) doSearch(sq SelectBuilder, q string, offset, size int, re return nil } - filter := fullTextExpr(r.tableName, q) + searchExpr := getSearchExpr() + filter := searchExpr(r.tableName, q) if filter != nil { sq = sq.Where(filter) sq = sq.OrderBy(orderBys...) @@ -59,13 +81,16 @@ func mbidExpr(tableName, mbid string, mbidFields ...string) Sqlizer { return Or(cond) } -func fullTextExpr(tableName string, s string) Sqlizer { +// legacySearchExpr generates LIKE-based search filters against the full_text column. +// This is the original search implementation, used when Search.Backend="legacy". +func legacySearchExpr(tableName string, s string) Sqlizer { q := str.SanitizeStrings(s) if q == "" { + log.Trace("Search using legacy backend, query is empty", "table", tableName) return nil } var sep string - if !conf.Server.SearchFullString { + if !conf.Server.Search.FullString { sep = " " } parts := strings.Split(q, " ") @@ -73,5 +98,6 @@ func fullTextExpr(tableName string, s string) Sqlizer { for _, part := range parts { filters = append(filters, Like{tableName + ".full_text": "%" + sep + part + "%"}) } + log.Trace("Search using legacy backend", "query", filters, "table", tableName) return filters } diff --git a/persistence/sql_search_fts.go b/persistence/sql_search_fts.go new file mode 100644 index 000000000..25f16cb8d --- /dev/null +++ b/persistence/sql_search_fts.go @@ -0,0 +1,261 @@ +package persistence + +import ( + "fmt" + "regexp" + "strings" + "unicode" + "unicode/utf8" + + . "github.com/Masterminds/squirrel" + "github.com/navidrome/navidrome/log" +) + +// containsCJK returns true if the string contains any CJK (Chinese/Japanese/Korean) characters. +// CJK text doesn't use spaces between words, so FTS5's unicode61 tokenizer treats entire +// CJK phrases as single tokens, making token-based search ineffective for CJK content. +func containsCJK(s string) bool { + for _, r := range s { + if unicode.Is(unicode.Han, r) || + unicode.Is(unicode.Hiragana, r) || + unicode.Is(unicode.Katakana, r) || + unicode.Is(unicode.Hangul, r) { + return true + } + } + return false +} + +// fts5SpecialChars matches characters that should be stripped from user input. +// We keep only Unicode letters, numbers, whitespace, * (prefix wildcard), " (phrase quotes), +// and \x00 (internal placeholder marker). All punctuation is removed because the unicode61 +// tokenizer treats it as token separators, and characters like ' can cause FTS5 parse errors +// as unbalanced string delimiters. +var fts5SpecialChars = regexp.MustCompile(`[^\p{L}\p{N}\s*"\x00]`) + +// fts5PunctStrip strips everything except letters and numbers (no whitespace, wildcards, or quotes). +// Used for normalizing words at index time to create concatenated forms (e.g., "R.E.M." → "REM"). +var fts5PunctStrip = regexp.MustCompile(`[^\p{L}\p{N}]`) + +// fts5Operators matches FTS5 boolean operators as whole words (case-insensitive). +var fts5Operators = regexp.MustCompile(`(?i)\b(AND|OR|NOT|NEAR)\b`) + +// fts5LeadingStar matches a * at the start of a token. FTS5 only supports * at the end (prefix queries). +var fts5LeadingStar = regexp.MustCompile(`(^|[\s])\*+`) + +// normalizeForFTS takes multiple strings, strips non-letter/non-number characters from each word, +// and returns a space-separated string of words that changed after stripping (deduplicated). +// This is used at index time to create concatenated forms: "R.E.M." → "REM", "AC/DC" → "ACDC". +func normalizeForFTS(values ...string) string { + seen := make(map[string]struct{}) + var result []string + for _, v := range values { + for _, word := range strings.Fields(v) { + stripped := fts5PunctStrip.ReplaceAllString(word, "") + if stripped == "" || stripped == word { + continue + } + lower := strings.ToLower(stripped) + if _, ok := seen[lower]; ok { + continue + } + seen[lower] = struct{}{} + result = append(result, stripped) + } + } + return strings.Join(result, " ") +} + +// isSingleUnicodeLetter returns true if token is exactly one Unicode letter. +func isSingleUnicodeLetter(token string) bool { + r, size := utf8.DecodeRuneInString(token) + return size == len(token) && size > 0 && unicode.IsLetter(r) +} + +// namePunctuation is the set of characters commonly used as separators in artist/album +// names (hyphens, slashes, dots, apostrophes). Only words containing these are candidates +// for punctuated-word processing; other special characters (^, :, &) are just stripped. +const namePunctuation = `-/.''` + +// processPunctuatedWords handles words with embedded name punctuation before the general +// special-character stripping. For each punctuated word it produces either: +// - A quoted phrase for dotted abbreviations: R.E.M. → "R E M" +// - A phrase+concat OR for other patterns: a-ha → ("a ha" OR aha*) +func processPunctuatedWords(input string, phrases []string) (string, []string) { + words := strings.Fields(input) + var result []string + for _, w := range words { + if strings.HasPrefix(w, "\x00") || strings.ContainsAny(w, `*"`) || !strings.ContainsAny(w, namePunctuation) { + result = append(result, w) + continue + } + concat := fts5PunctStrip.ReplaceAllString(w, "") + if concat == "" || concat == w { + result = append(result, w) + continue + } + subTokens := strings.Fields(fts5SpecialChars.ReplaceAllString(w, " ")) + if len(subTokens) < 2 { + // Single sub-token after splitting (e.g., N' → N): just use the stripped form + result = append(result, concat) + continue + } + // Dotted abbreviations (R.E.M., U.K.) — all single letters separated by dots only + if isDottedAbbreviation(w, subTokens) { + phrases = append(phrases, fmt.Sprintf(`"%s"`, strings.Join(subTokens, " "))) + } else { + // Punctuated names (a-ha, AC/DC, Jay-Z) — phrase for adjacency + concat for search_normalized + phrases = append(phrases, fmt.Sprintf(`("%s" OR %s*)`, strings.Join(subTokens, " "), concat)) + } + result = append(result, fmt.Sprintf("\x00PHRASE%d\x00", len(phrases)-1)) + } + return strings.Join(result, " "), phrases +} + +// isDottedAbbreviation returns true if w uses only dots as punctuation and all sub-tokens +// are single letters (e.g., "R.E.M.", "U.K." but not "a-ha" or "AC/DC"). +func isDottedAbbreviation(w string, subTokens []string) bool { + for _, r := range w { + if !unicode.IsLetter(r) && !unicode.IsNumber(r) && r != '.' { + return false + } + } + for _, st := range subTokens { + if !isSingleUnicodeLetter(st) { + return false + } + } + return true +} + +// buildFTS5Query preprocesses user input into a safe FTS5 MATCH expression. +// It preserves quoted phrases and * prefix wildcards, neutralizes FTS5 operators +// (by lowercasing them, since FTS5 operators are case-sensitive) and strips +// special characters to prevent query injection. +func buildFTS5Query(userInput string) string { + q := strings.TrimSpace(userInput) + if q == "" { + return "" + } + + var phrases []string + result := q + for { + start := strings.Index(result, `"`) + if start == -1 { + break + } + end := strings.Index(result[start+1:], `"`) + if end == -1 { + // Unmatched quote — remove it + result = result[:start] + result[start+1:] + break + } + end += start + 1 + phrase := result[start : end+1] // includes quotes + phrases = append(phrases, phrase) + result = result[:start] + fmt.Sprintf("\x00PHRASE%d\x00", len(phrases)-1) + result[end+1:] + } + + // Neutralize FTS5 operators by lowercasing them (FTS5 operators are case-sensitive: + // AND, OR, NOT, NEAR are operators, but and, or, not, near are plain tokens) + result = fts5Operators.ReplaceAllStringFunc(result, strings.ToLower) + + // Handle words with embedded punctuation (a-ha, AC/DC, R.E.M.) before stripping + result, phrases = processPunctuatedWords(result, phrases) + + result = fts5SpecialChars.ReplaceAllString(result, " ") + result = fts5LeadingStar.ReplaceAllString(result, "$1") + tokens := strings.Fields(result) + + // Append * to plain tokens for prefix matching (e.g., "love" → "love*"). + // Skip tokens that are already wildcarded or are quoted phrase placeholders. + for i, t := range tokens { + if strings.HasPrefix(t, "\x00") || strings.HasSuffix(t, "*") { + continue + } + tokens[i] = t + "*" + } + + result = strings.Join(tokens, " ") + + for i, phrase := range phrases { + placeholder := fmt.Sprintf("\x00PHRASE%d\x00", i) + result = strings.ReplaceAll(result, placeholder, phrase) + } + + return result +} + +// likeSearchColumns defines the core columns to search with LIKE queries. +// These are the primary user-visible fields for each entity type. +// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input). +var likeSearchColumns = map[string][]string{ + "media_file": {"title", "album", "artist", "album_artist"}, + "album": {"name", "album_artist"}, + "artist": {"name"}, +} + +// likeSearchExpr generates LIKE-based search filters against core columns. +// Each word in the query must match at least one column (AND between words), +// and each word can match any column (OR within a word). +// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input). +func likeSearchExpr(tableName string, s string) Sqlizer { + s = strings.TrimSpace(s) + if s == "" { + log.Trace("Search using LIKE backend, query is empty", "table", tableName) + return nil + } + columns, ok := likeSearchColumns[tableName] + if !ok { + log.Trace("Search using LIKE backend, couldn't find columns for this table", "table", tableName) + return nil + } + words := strings.Fields(s) + wordFilters := And{} + for _, word := range words { + colFilters := Or{} + for _, col := range columns { + colFilters = append(colFilters, Like{tableName + "." + col: "%" + word + "%"}) + } + wordFilters = append(wordFilters, colFilters) + } + log.Trace("Search using LIKE backend", "query", wordFilters, "table", tableName) + return wordFilters +} + +// ftsSearchColumns defines which FTS5 columns are included in general search. +// Columns not listed here are indexed but not searched by default, +// enabling future additions (comments, lyrics, bios) without affecting general search. +var ftsSearchColumns = map[string]string{ + "media_file": "{title album artist album_artist sort_title sort_album_name sort_artist_name sort_album_artist_name disc_subtitle search_participants search_normalized}", + "album": "{name sort_album_name album_artist search_participants discs catalog_num album_version search_normalized}", + "artist": "{name sort_artist_name search_normalized}", +} + +// ftsSearchExpr generates an FTS5 MATCH-based search filter. +// If the query produces no FTS tokens (e.g., punctuation-only like "!!!!!!!"), +// it falls back to LIKE-based search. +func ftsSearchExpr(tableName string, s string) Sqlizer { + q := buildFTS5Query(s) + if q == "" { + s = strings.TrimSpace(s) + if s != "" { + log.Trace("Search using LIKE fallback for non-tokenizable query", "table", tableName, "query", s) + return likeSearchExpr(tableName, s) + } + return nil + } + ftsTable := tableName + "_fts" + matchExpr := q + if cols, ok := ftsSearchColumns[tableName]; ok { + matchExpr = cols + " : (" + q + ")" + } + + filter := Expr( + tableName+".rowid IN (SELECT rowid FROM "+ftsTable+" WHERE "+ftsTable+" MATCH ?)", + matchExpr, + ) + log.Trace("Search using FTS5 backend", "table", tableName, "query", q, "filter", filter) + return filter +} diff --git a/persistence/sql_search_fts_test.go b/persistence/sql_search_fts_test.go new file mode 100644 index 000000000..317252950 --- /dev/null +++ b/persistence/sql_search_fts_test.go @@ -0,0 +1,333 @@ +package persistence + +import ( + "context" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = DescribeTable("buildFTS5Query", + func(input, expected string) { + Expect(buildFTS5Query(input)).To(Equal(expected)) + }, + Entry("returns empty string for empty input", "", ""), + Entry("returns empty string for whitespace-only input", " ", ""), + Entry("appends * to a single word for prefix matching", "beatles", "beatles*"), + Entry("appends * to each word for prefix matching", "abbey road", "abbey* road*"), + Entry("preserves quoted phrases without appending *", `"the beatles"`, `"the beatles"`), + Entry("does not double-append * to existing prefix wildcard", "beat*", "beat*"), + Entry("strips FTS5 operators and appends * to lowercased words", "AND OR NOT NEAR", "and* or* not* near*"), + Entry("strips special FTS5 syntax characters and appends *", "test^col:val", "test* col* val*"), + Entry("handles mixed phrases and words", `"the beatles" abbey`, `"the beatles" abbey*`), + Entry("handles prefix with multiple words", "beat* abbey", "beat* abbey*"), + Entry("collapses multiple spaces", "abbey road", "abbey* road*"), + Entry("strips leading * from tokens and appends trailing *", "*livia", "livia*"), + Entry("strips leading * and preserves existing trailing *", "*livia oliv*", "livia* oliv*"), + Entry("strips standalone *", "*", ""), + Entry("strips apostrophe from input", "Guns N' Roses", "Guns* N* Roses*"), + Entry("converts slashed word to phrase+concat OR", "AC/DC", `("AC DC" OR ACDC*)`), + Entry("converts hyphenated word to phrase+concat OR", "a-ha", `("a ha" OR aha*)`), + Entry("converts partial hyphenated word to phrase+concat OR", "a-h", `("a h" OR ah*)`), + Entry("converts hyphenated name to phrase+concat OR", "Jay-Z", `("Jay Z" OR JayZ*)`), + Entry("converts contraction to phrase+concat OR", "it's", `("it s" OR its*)`), + Entry("handles punctuated word mixed with plain words", "best of a-ha", `best* of* ("a ha" OR aha*)`), + Entry("strips miscellaneous punctuation", "rock & roll, vol. 2", "rock* roll* vol* 2*"), + Entry("preserves unicode characters with diacritics", "Björk début", "Björk* début*"), + Entry("collapses dotted abbreviation into phrase", "R.E.M.", `"R E M"`), + Entry("collapses abbreviation without trailing dot", "R.E.M", `"R E M"`), + Entry("collapses abbreviation mixed with words", "best of R.E.M.", `best* of* "R E M"`), + Entry("collapses two-letter abbreviation", "U.K.", `"U K"`), + Entry("does not collapse single letter surrounded by words", "I am fine", "I* am* fine*"), + Entry("does not collapse single standalone letter", "A test", "A* test*"), + Entry("preserves quoted phrase with punctuation verbatim", `"ac/dc"`, `"ac/dc"`), + Entry("preserves quoted abbreviation verbatim", `"R.E.M."`, `"R.E.M."`), + Entry("returns empty string for punctuation-only input", "!!!!!!!", ""), + Entry("returns empty string for mixed punctuation", "!@#$%^&", ""), +) + +var _ = DescribeTable("normalizeForFTS", + func(expected string, values ...string) { + Expect(normalizeForFTS(values...)).To(Equal(expected)) + }, + Entry("strips dots and concatenates", "REM", "R.E.M."), + Entry("strips slash", "ACDC", "AC/DC"), + Entry("strips hyphen", "Aha", "A-ha"), + Entry("skips unchanged words", "", "The Beatles"), + Entry("handles mixed input", "REM", "R.E.M.", "Automatic for the People"), + Entry("deduplicates", "REM", "R.E.M.", "R.E.M."), + Entry("strips apostrophe from word", "N", "Guns N' Roses"), + Entry("handles multiple values with punctuation", "REM ACDC", "R.E.M.", "AC/DC"), +) + +var _ = DescribeTable("containsCJK", + func(input string, expected bool) { + Expect(containsCJK(input)).To(Equal(expected)) + }, + Entry("returns false for empty string", "", false), + Entry("returns false for ASCII text", "hello world", false), + Entry("returns false for Latin with diacritics", "Björk début", false), + Entry("detects Chinese characters (Han)", "周杰伦", true), + Entry("detects Japanese Hiragana", "こんにちは", true), + Entry("detects Japanese Katakana", "カタカナ", true), + Entry("detects Korean Hangul", "한국어", true), + Entry("detects CJK mixed with Latin", "best of 周杰伦", true), + Entry("detects single CJK character", "a曲b", true), +) + +var _ = Describe("likeSearchExpr", func() { + It("returns nil for empty query", func() { + Expect(likeSearchExpr("media_file", "")).To(BeNil()) + }) + + It("returns nil for whitespace-only query", func() { + Expect(likeSearchExpr("media_file", " ")).To(BeNil()) + }) + + It("generates LIKE filters against core columns for single CJK word", func() { + expr := likeSearchExpr("media_file", "周杰伦") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + // Should have OR between columns for the single word + Expect(sql).To(ContainSubstring("OR")) + Expect(sql).To(ContainSubstring("media_file.title LIKE")) + Expect(sql).To(ContainSubstring("media_file.album LIKE")) + Expect(sql).To(ContainSubstring("media_file.artist LIKE")) + Expect(sql).To(ContainSubstring("media_file.album_artist LIKE")) + Expect(args).To(HaveLen(4)) + for _, arg := range args { + Expect(arg).To(Equal("%周杰伦%")) + } + }) + + It("generates AND of OR groups for multi-word query", func() { + expr := likeSearchExpr("media_file", "周杰伦 greatest") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + // Two groups AND'd together, each with 4 columns OR'd + Expect(sql).To(ContainSubstring("AND")) + Expect(args).To(HaveLen(8)) + }) + + It("uses correct columns for album table", func() { + expr := likeSearchExpr("album", "周杰伦") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("album.name LIKE")) + Expect(sql).To(ContainSubstring("album.album_artist LIKE")) + Expect(args).To(HaveLen(2)) + }) + + It("uses correct columns for artist table", func() { + expr := likeSearchExpr("artist", "周杰伦") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("artist.name LIKE")) + Expect(args).To(HaveLen(1)) + }) + + It("returns nil for unknown table", func() { + Expect(likeSearchExpr("unknown_table", "周杰伦")).To(BeNil()) + }) +}) + +var _ = Describe("ftsSearchExpr", func() { + It("returns nil for empty query", func() { + Expect(ftsSearchExpr("media_file", "")).To(BeNil()) + }) + + It("generates rowid IN subquery with MATCH and column filter", func() { + expr := ftsSearchExpr("media_file", "beatles") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("media_file.rowid IN")) + Expect(sql).To(ContainSubstring("media_file_fts")) + Expect(sql).To(ContainSubstring("MATCH")) + Expect(args).To(HaveLen(1)) + Expect(args[0]).To(HavePrefix("{title album artist album_artist")) + Expect(args[0]).To(ContainSubstring("beatles*")) + }) + + It("generates correct FTS table name per entity", func() { + for _, table := range []string{"media_file", "album", "artist"} { + expr := ftsSearchExpr(table, "test") + sql, _, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring(table + ".rowid IN")) + Expect(sql).To(ContainSubstring(table + "_fts")) + } + }) + + It("wraps query with column filter for known tables", func() { + expr := ftsSearchExpr("artist", "Beatles") + _, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(args[0]).To(Equal("{name sort_artist_name search_normalized} : (Beatles*)")) + }) + + It("passes query without column filter for unknown tables", func() { + expr := ftsSearchExpr("unknown_table", "test") + _, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(args[0]).To(Equal("test*")) + }) + + It("preserves phrase queries inside column filter", func() { + expr := ftsSearchExpr("media_file", `"the beatles"`) + _, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(args[0]).To(ContainSubstring(`"the beatles"`)) + }) + + It("preserves prefix queries inside column filter", func() { + expr := ftsSearchExpr("media_file", "beat*") + _, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(args[0]).To(ContainSubstring("beat*")) + }) + + It("falls back to LIKE search for punctuation-only query", func() { + expr := ftsSearchExpr("media_file", "!!!!!!!") + Expect(expr).ToNot(BeNil()) + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("LIKE")) + Expect(args).To(ContainElement("%!!!!!!!%")) + }) + + It("returns nil for empty string even with LIKE fallback", func() { + Expect(ftsSearchExpr("media_file", "")).To(BeNil()) + Expect(ftsSearchExpr("media_file", " ")).To(BeNil()) + }) +}) + +var _ = Describe("FTS5 Integration Search", func() { + var ( + mr model.MediaFileRepository + alr model.AlbumRepository + arr model.ArtistRepository + ) + + BeforeEach(func() { + ctx := log.NewContext(context.TODO()) + ctx = request.WithUser(ctx, adminUser) + conn := GetDBXBuilder() + mr = NewMediaFileRepository(ctx, conn) + alr = NewAlbumRepository(ctx, conn) + arr = NewArtistRepository(ctx, conn) + }) + + Describe("MediaFile search", func() { + It("finds media files by title", func() { + results, err := mr.Search("Radioactivity", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Title).To(Equal("Radioactivity")) + Expect(results[0].ID).To(Equal(songRadioactivity.ID)) + }) + + It("finds media files by artist name", func() { + results, err := mr.Search("Beatles", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(3)) + for _, r := range results { + Expect(r.Artist).To(Equal("The Beatles")) + } + }) + }) + + Describe("Album search", func() { + It("finds albums by name", func() { + results, err := alr.Search("Sgt Peppers", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Name).To(Equal("Sgt Peppers")) + Expect(results[0].ID).To(Equal(albumSgtPeppers.ID)) + }) + + It("finds albums with multi-word search", func() { + results, err := alr.Search("Abbey Road", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(2)) + }) + }) + + Describe("Artist search", func() { + It("finds artists by name", func() { + results, err := arr.Search("Kraftwerk", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Name).To(Equal("Kraftwerk")) + Expect(results[0].ID).To(Equal(artistKraftwerk.ID)) + }) + }) + + Describe("CJK search", func() { + It("finds media files by CJK title", func() { + results, err := mr.Search("プラチナ", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Title).To(Equal("プラチナ・ジェット")) + Expect(results[0].ID).To(Equal(songCJK.ID)) + }) + + It("finds media files by CJK artist name", func() { + results, err := mr.Search("シートベルツ", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Artist).To(Equal("シートベルツ")) + }) + + It("finds albums by CJK artist name", func() { + results, err := alr.Search("シートベルツ", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Name).To(Equal("COWBOY BEBOP")) + Expect(results[0].ID).To(Equal(albumCJK.ID)) + }) + + It("finds artists by CJK name", func() { + results, err := arr.Search("シートベルツ", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Name).To(Equal("シートベルツ")) + Expect(results[0].ID).To(Equal(artistCJK.ID)) + }) + }) + + Describe("Album version search", func() { + It("finds albums by version tag via FTS", func() { + results, err := alr.Search("Deluxe", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].ID).To(Equal(albumWithVersion.ID)) + }) + }) + + Describe("Punctuation-only search", func() { + It("finds media files with punctuation-only title", func() { + results, err := mr.Search("!!!!!!!", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Title).To(Equal("!!!!!!!")) + Expect(results[0].ID).To(Equal(songPunctuation.ID)) + }) + }) + + Describe("Legacy backend fallback", func() { + It("returns results using legacy LIKE-based search when configured", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" + + results, err := mr.Search("Radioactivity", 0, 10) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Title).To(Equal("Radioactivity")) + }) + }) +}) diff --git a/persistence/sql_search_test.go b/persistence/sql_search_test.go index 6bfd88d9f..b59570af3 100644 --- a/persistence/sql_search_test.go +++ b/persistence/sql_search_test.go @@ -1,6 +1,8 @@ package persistence import ( + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ) @@ -11,4 +13,99 @@ var _ = Describe("sqlRepository", func() { Expect(formatFullText("legiao urbana")).To(Equal(" legiao urbana")) }) }) + + Describe("legacySearchExpr", func() { + It("returns nil for empty query", func() { + Expect(legacySearchExpr("media_file", "")).To(BeNil()) + }) + + It("generates LIKE filter for single word", func() { + expr := legacySearchExpr("media_file", "beatles") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("media_file.full_text LIKE")) + Expect(args).To(ContainElement("% beatles%")) + }) + + It("generates AND of LIKE filters for multiple words", func() { + expr := legacySearchExpr("media_file", "abbey road") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("AND")) + Expect(args).To(HaveLen(2)) + }) + }) + + Describe("getSearchExpr", func() { + It("returns ftsSearchExpr by default", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "fts" + conf.Server.Search.FullString = false + + expr := getSearchExpr()("media_file", "test") + sql, _, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("MATCH")) + }) + + It("returns legacySearchExpr when SearchBackend is legacy", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" + conf.Server.Search.FullString = false + + expr := getSearchExpr()("media_file", "test") + sql, _, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("LIKE")) + }) + + It("falls back to legacySearchExpr when SearchFullString is enabled", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "fts" + conf.Server.Search.FullString = true + + expr := getSearchExpr()("media_file", "test") + sql, _, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("LIKE")) + }) + + It("routes CJK queries to likeSearchExpr instead of ftsSearchExpr", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "fts" + conf.Server.Search.FullString = false + + expr := getSearchExpr()("media_file", "周杰伦") + sql, _, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + // CJK should use LIKE, not MATCH + Expect(sql).To(ContainSubstring("LIKE")) + Expect(sql).NotTo(ContainSubstring("MATCH")) + }) + + It("routes non-CJK queries to ftsSearchExpr", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "fts" + conf.Server.Search.FullString = false + + expr := getSearchExpr()("media_file", "beatles") + sql, _, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("MATCH")) + }) + + It("uses legacy for CJK when SearchBackend is legacy", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" + conf.Server.Search.FullString = false + + expr := getSearchExpr()("media_file", "周杰伦") + sql, _, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + // Legacy should still use full_text column LIKE + Expect(sql).To(ContainSubstring("LIKE")) + Expect(sql).To(ContainSubstring("full_text")) + }) + }) + }) diff --git a/reflex.conf b/reflex.conf index 9633ab910..47dd775ab 100644 --- a/reflex.conf +++ b/reflex.conf @@ -1 +1 @@ --s -r "(\.go$$|\.cpp$$|\.h$$|navidrome.toml|resources|token_received.html)" -R "(^ui|^data|^db/migrations)" -R "_test\.go$$" -- go run -race -tags netgo . +-s -r "(\.go$$|\.cpp$$|\.h$$|navidrome.toml|resources|token_received.html)" -R "(^ui|^data|^db/migrations)" -R "_test\.go$$" -- go run -race -tags netgo,sqlite_fts5 . diff --git a/server/e2e/e2e_suite_test.go b/server/e2e/e2e_suite_test.go index 479f9c680..92214950a 100644 --- a/server/e2e/e2e_suite_test.go +++ b/server/e2e/e2e_suite_test.go @@ -363,7 +363,7 @@ func restoreDB() { _, err = sqlDB.Exec("ATTACH DATABASE ? AS snapshot", snapshotPath) Expect(err).ToNot(HaveOccurred()) - rows, err := sqlDB.Query("SELECT name FROM main.sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'") + rows, err := sqlDB.Query("SELECT name FROM main.sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' AND name NOT LIKE '%_fts' AND name NOT LIKE '%_fts_%'") Expect(err).ToNot(HaveOccurred()) var tables []string for rows.Next() { diff --git a/server/serve_index.go b/server/serve_index.go index b9a3b3a2b..b5b364267 100644 --- a/server/serve_index.go +++ b/server/serve_index.go @@ -54,6 +54,7 @@ func serveIndex(ds model.DataStore, fs fs.FS, shareInfo *model.Share) http.Handl "defaultTheme": conf.Server.DefaultTheme, "defaultLanguage": conf.Server.DefaultLanguage, "defaultUIVolume": conf.Server.DefaultUIVolume, + "uiSearchDebounceMs": conf.Server.UISearchDebounceMs, "enableCoverAnimation": conf.Server.EnableCoverAnimation, "enableNowPlaying": conf.Server.EnableNowPlaying, "gaTrackingId": conf.Server.GATrackingID, diff --git a/server/serve_index_test.go b/server/serve_index_test.go index 4f179f22a..9d6f480ff 100644 --- a/server/serve_index_test.go +++ b/server/serve_index_test.go @@ -85,6 +85,7 @@ var _ = Describe("serveIndex", func() { Entry("defaultTheme", func() { conf.Server.DefaultTheme = "Light" }, "defaultTheme", "Light"), Entry("defaultLanguage", func() { conf.Server.DefaultLanguage = "pt" }, "defaultLanguage", "pt"), Entry("defaultUIVolume", func() { conf.Server.DefaultUIVolume = 45 }, "defaultUIVolume", float64(45)), + Entry("uiSearchDebounceMs", func() { conf.Server.UISearchDebounceMs = 500 }, "uiSearchDebounceMs", float64(500)), Entry("enableCoverAnimation", func() { conf.Server.EnableCoverAnimation = true }, "enableCoverAnimation", true), Entry("enableNowPlaying", func() { conf.Server.EnableNowPlaying = true }, "enableNowPlaying", true), Entry("gaTrackingId", func() { conf.Server.GATrackingID = "UA-12345" }, "gaTrackingId", "UA-12345"), diff --git a/ui/src/common/List.jsx b/ui/src/common/List.jsx index f74ab027e..72c2d9482 100644 --- a/ui/src/common/List.jsx +++ b/ui/src/common/List.jsx @@ -1,5 +1,6 @@ import React from 'react' import { List as RAList } from 'react-admin' +import config from '../config' import { Pagination } from './Pagination' import { Title } from './index' @@ -13,6 +14,7 @@ export const List = (props) => { args={{ smart_count: 2 }} /> } + debounce={config.uiSearchDebounceMs} perPage={15} pagination={} {...props} diff --git a/ui/src/config.js b/ui/src/config.js index 9582e95ee..5acf10b69 100644 --- a/ui/src/config.js +++ b/ui/src/config.js @@ -20,6 +20,7 @@ const defaultConfig = { defaultTheme: 'Dark', defaultLanguage: '', defaultUIVolume: 100, + uiSearchDebounceMs: 200, enableUserEditing: true, enableSharing: true, shareURL: '', From 1cf3fd91618cfd9eb3f1a4dde7f53b6881da9458 Mon Sep 17 00:00:00 2001 From: Deluan Date: Sat, 21 Feb 2026 18:51:16 -0500 Subject: [PATCH 02/60] fix(scanner): prevent ScanOnStartup when scanner is disabled Gate the ScanOnStartup config on Scanner.Enabled so that setting Scanner.Enabled=false prevents automatic startup scans. Other automatic scan triggers (interrupted scan resume, PID change, post-migration) are preserved regardless of the Enabled flag to maintain data integrity. --- cmd/root.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cmd/root.go b/cmd/root.go index 74a15abc1..ff9a574ee 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -196,7 +196,8 @@ func runInitialScan(ctx context.Context) func() error { if err != nil { return err } - scanNeeded := conf.Server.Scanner.ScanOnStartup || inProgress || fullScanRequired == "1" || pidHasChanged + scanOnStartup := conf.Server.Scanner.Enabled && conf.Server.Scanner.ScanOnStartup + scanNeeded := scanOnStartup || inProgress || fullScanRequired == "1" || pidHasChanged time.Sleep(2 * time.Second) // Wait 2 seconds before the initial scan if scanNeeded { s := CreateScanner(ctx) From 76c01566a91f7dda869f38214e79f79da70e8fe6 Mon Sep 17 00:00:00 2001 From: Deluan Date: Sat, 21 Feb 2026 18:56:26 -0500 Subject: [PATCH 03/60] test(ui): change datagrid from table to div to fix warning Signed-off-by: Deluan --- ui/src/plugin/PluginList.test.jsx | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/ui/src/plugin/PluginList.test.jsx b/ui/src/plugin/PluginList.test.jsx index 2fab2a3e0..0ed41b98c 100644 --- a/ui/src/plugin/PluginList.test.jsx +++ b/ui/src/plugin/PluginList.test.jsx @@ -34,9 +34,7 @@ vi.mock('react-admin', async () => { TopToolbar: ({ children }) => (
{children}
), - Datagrid: ({ children }) => ( - {children}
- ), + Datagrid: ({ children }) =>
{children}
, TextField: ({ source }) => , } }) From 7ad290771907e8ce4c1cb0455a7e9a5ef2016c97 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Sat, 21 Feb 2026 19:57:13 -0500 Subject: [PATCH 04/60] refactor: move playlist business logic from repositories to service layer (#5027) * refactor: move playlist business logic from repositories to core.Playlists service Move authorization, permission checks, and orchestration logic from playlist repositories to the core.Playlists service, following the existing pattern used by core.Share and core.Library. Changes: - Expand core.Playlists interface with read, mutation, track management, and REST adapter methods - Add playlistRepositoryWrapper for REST Save/Update/Delete with permission checks (follows Share/Library pattern) - Simplify persistence/playlist_repository.go: remove isWritable(), auth checks from Delete()/Put()/updatePlaylist() - Simplify persistence/playlist_track_repository.go: remove isTracksEditable() and permission checks from Add/Delete/Reorder - Update Subsonic API handlers to route through service - Update Native API handlers to accept core.Playlists instead of model.DataStore * test: add coverage for playlist service methods and REST wrapper Add 30 new tests covering the service methods added during the playlist refactoring: - Delete: owner, admin, denied, not found - Create: new playlist, replace tracks, admin bypass, denied, not found - AddTracks: owner, admin, denied, smart playlist, not found - RemoveTracks: owner, smart playlist denied, non-owner denied - ReorderTrack: owner, smart playlist denied - NewRepository wrapper: Save (owner assignment, ID clearing), Update (owner, admin, denied, ownership change, not found), Delete (delegation with permission checks) Expand mockedPlaylistRepo with Get, Delete, Tracks, GetWithTracks, and rest.Persistable methods. Add mockedPlaylistTrackRepo for track operation verification. * fix: add authorization check to playlist Update method Added ownership verification to the Subsonic Update endpoint in the playlist service layer. The authorization check was present in the old repository code but was not carried over during the refactoring to the service layer, allowing any authenticated user to modify playlists they don't own via the Subsonic API. Also added corresponding tests for the Update method's permission logic. * refactor: improve playlist permission checks and error handling, add e2e tests Signed-off-by: Deluan * refactor: rename core.Playlists to playlists package and update references Signed-off-by: Deluan * refactor: rename playlists_internal_test.go to parse_m3u_test.go and update tests; add new parse_nsp.go and rest_adapter.go files Signed-off-by: Deluan * fix: block track mutations on smart playlists in Create and Update Create now rejects replacing tracks on smart playlists (pre-existing gap). Update now uses checkTracksEditable instead of checkWritable when track changes are requested, restoring the protection that was removed from the repository layer during the refactoring. Metadata-only updates on smart playlists remain allowed. * test: add smart playlist protection tests to ensure readonly behavior and mutation restrictions * refactor: optimize track removal and renumbering in playlists Signed-off-by: Deluan * refactor: implement track reordering in playlists with SQL updates Signed-off-by: Deluan * refactor: wrap track deletion and reordering in transactions for consistency Signed-off-by: Deluan * refactor: remove unused getTracks method from playlistTrackRepository Signed-off-by: Deluan * refactor: optimize playlist track renumbering with CTE-based UPDATE Replace the DELETE + re-INSERT renumbering strategy with a two-step UPDATE approach using a materialized CTE and ROW_NUMBER() window function. The previous approach (SELECT all IDs, DELETE all tracks, re-INSERT in chunks of 200) required 13 SQL operations for a 2000-track playlist. The new approach uses just 2 UPDATEs: first negating all IDs to clear the positive space, then assigning sequential positions via UPDATE...FROM with a CTE. This avoids the UNIQUE constraint violations that affected the original correlated subquery while reducing per-delete request time from ~110ms to ~12ms on a 2000-track playlist. Signed-off-by: Deluan * refactor: rename New function to NewPlaylists for clarity Signed-off-by: Deluan * refactor: update mock playlist repository and tests for consistency Signed-off-by: Deluan --------- Signed-off-by: Deluan --- cmd/scan.go | 4 +- cmd/wire_gen.go | 21 +- core/playlists/import.go | 119 +++++ .../import_test.go} | 68 +-- core/{playlists.go => playlists/parse_m3u.go} | 300 +----------- .../parse_m3u_test.go} | 60 +-- core/playlists/parse_nsp.go | 103 ++++ core/playlists/parse_nsp_test.go | 213 +++++++++ core/playlists/playlists.go | 265 +++++++++++ core/playlists/playlists_suite_test.go | 17 + core/playlists/playlists_test.go | 297 ++++++++++++ core/playlists/rest_adapter.go | 95 ++++ core/playlists/rest_adapter_test.go | 120 +++++ core/wire_providers.go | 3 +- persistence/playlist_repository.go | 75 +-- persistence/playlist_repository_test.go | 73 +++ persistence/playlist_track_repository.go | 65 +-- scanner/controller.go | 8 +- scanner/controller_test.go | 4 +- scanner/folder_entry.go | 4 +- scanner/phase_4_playlists.go | 6 +- scanner/phase_4_playlists_test.go | 4 +- scanner/scanner.go | 4 +- scanner/scanner_benchmark_test.go | 4 +- scanner/scanner_multilibrary_test.go | 4 +- scanner/scanner_selective_test.go | 4 +- scanner/scanner_test.go | 4 +- server/e2e/e2e_suite_test.go | 25 +- server/e2e/subsonic_multilibrary_test.go | 4 +- server/e2e/subsonic_playlists_test.go | 446 +++++++++++++++++- server/e2e/subsonic_scan_test.go | 2 - server/nativeapi/native_api.go | 21 +- server/nativeapi/playlists.go | 70 ++- server/nativeapi/playlists_test.go | 27 +- server/subsonic/api.go | 7 +- server/subsonic/playlists.go | 36 +- server/subsonic/playlists_test.go | 6 +- tests/mock_data_store.go | 2 +- tests/mock_playlist_repo.go | 107 ++++- tests/mock_playlist_track_repo.go | 53 +++ 40 files changed, 2131 insertions(+), 619 deletions(-) create mode 100644 core/playlists/import.go rename core/{playlists_test.go => playlists/import_test.go} (94%) rename core/{playlists.go => playlists/parse_m3u.go} (53%) rename core/{playlists_internal_test.go => playlists/parse_m3u_test.go} (91%) create mode 100644 core/playlists/parse_nsp.go create mode 100644 core/playlists/parse_nsp_test.go create mode 100644 core/playlists/playlists.go create mode 100644 core/playlists/playlists_suite_test.go create mode 100644 core/playlists/playlists_test.go create mode 100644 core/playlists/rest_adapter.go create mode 100644 core/playlists/rest_adapter_test.go create mode 100644 tests/mock_playlist_track_repo.go diff --git a/cmd/scan.go b/cmd/scan.go index daf58b29c..ffb77b108 100644 --- a/cmd/scan.go +++ b/cmd/scan.go @@ -8,7 +8,7 @@ import ( "os" "strings" - "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -74,7 +74,7 @@ func runScanner(ctx context.Context) { sqlDB := db.Db() defer db.Db().Close() ds := persistence.New(sqlDB) - pls := core.NewPlaylists(ds) + pls := playlists.NewPlaylists(ds) // Parse targets from command line or file var scanTargets []model.ScanTarget diff --git a/cmd/wire_gen.go b/cmd/wire_gen.go index 7a9d38d92..204d90ba8 100644 --- a/cmd/wire_gen.go +++ b/cmd/wire_gen.go @@ -18,6 +18,7 @@ import ( "github.com/navidrome/navidrome/core/ffmpeg" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/model" @@ -61,7 +62,7 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router { sqlDB := db.Db() dataStore := persistence.New(sqlDB) share := core.NewShare(dataStore) - playlists := core.NewPlaylists(dataStore) + playlistsPlaylists := playlists.NewPlaylists(dataStore) insights := metrics.GetInstance(dataStore) fileCache := artwork.GetImageCache() fFmpeg := ffmpeg.New() @@ -72,12 +73,12 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router { provider := external.NewProvider(dataStore, agentsAgents) artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider) cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache) - modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics) + modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics) watcher := scanner.GetWatcher(dataStore, modelScanner) library := core.NewLibrary(dataStore, modelScanner, watcher, broker, manager) user := core.NewUser(dataStore, manager) maintenance := core.NewMaintenance(dataStore) - router := nativeapi.New(dataStore, share, playlists, insights, library, user, maintenance, manager) + router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager) return router } @@ -98,11 +99,11 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router { archiver := core.NewArchiver(mediaStreamer, dataStore, share) players := core.NewPlayers(dataStore) cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache) - playlists := core.NewPlaylists(dataStore) - modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics) + playlistsPlaylists := playlists.NewPlaylists(dataStore) + modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics) playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager) playbackServer := playback.GetInstance(dataStore) - router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlists, playTracker, share, playbackServer, metricsMetrics) + router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics) return router } @@ -165,8 +166,8 @@ func CreateScanner(ctx context.Context) model.Scanner { provider := external.NewProvider(dataStore, agentsAgents) artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider) cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache) - playlists := core.NewPlaylists(dataStore) - modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics) + playlistsPlaylists := playlists.NewPlaylists(dataStore) + modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics) return modelScanner } @@ -182,8 +183,8 @@ func CreateScanWatcher(ctx context.Context) scanner.Watcher { provider := external.NewProvider(dataStore, agentsAgents) artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider) cacheWarmer := artwork.NewCacheWarmer(artworkArtwork, fileCache) - playlists := core.NewPlaylists(dataStore) - modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlists, metricsMetrics) + playlistsPlaylists := playlists.NewPlaylists(dataStore) + modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics) watcher := scanner.GetWatcher(dataStore, modelScanner) return watcher } diff --git a/core/playlists/import.go b/core/playlists/import.go new file mode 100644 index 000000000..40e230527 --- /dev/null +++ b/core/playlists/import.go @@ -0,0 +1,119 @@ +package playlists + +import ( + "context" + "errors" + "io" + "os" + "path/filepath" + "strings" + "time" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" + "github.com/navidrome/navidrome/utils/ioutils" + "golang.org/x/text/unicode/norm" +) + +func (s *playlists) ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) { + pls, err := s.parsePlaylist(ctx, filename, folder) + if err != nil { + log.Error(ctx, "Error parsing playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err) + return nil, err + } + log.Debug(ctx, "Found playlist", "name", pls.Name, "lastUpdated", pls.UpdatedAt, "path", pls.Path, "numTracks", len(pls.Tracks)) + err = s.updatePlaylist(ctx, pls) + if err != nil { + log.Error(ctx, "Error updating playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err) + } + return pls, err +} + +func (s *playlists) ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error) { + owner, _ := request.UserFrom(ctx) + pls := &model.Playlist{ + OwnerID: owner.ID, + Public: false, + Sync: false, + } + err := s.parseM3U(ctx, pls, nil, reader) + if err != nil { + log.Error(ctx, "Error parsing playlist", err) + return nil, err + } + err = s.ds.Playlist(ctx).Put(pls) + if err != nil { + log.Error(ctx, "Error saving playlist", err) + return nil, err + } + return pls, nil +} + +func (s *playlists) parsePlaylist(ctx context.Context, playlistFile string, folder *model.Folder) (*model.Playlist, error) { + pls, err := s.newSyncedPlaylist(folder.AbsolutePath(), playlistFile) + if err != nil { + return nil, err + } + + file, err := os.Open(pls.Path) + if err != nil { + return nil, err + } + defer file.Close() + + reader := ioutils.UTF8Reader(file) + extension := strings.ToLower(filepath.Ext(playlistFile)) + switch extension { + case ".nsp": + err = s.parseNSP(ctx, pls, reader) + default: + err = s.parseM3U(ctx, pls, folder, reader) + } + return pls, err +} + +func (s *playlists) updatePlaylist(ctx context.Context, newPls *model.Playlist) error { + owner, _ := request.UserFrom(ctx) + + // Try to find existing playlist by path. Since filesystem normalization differs across + // platforms (macOS uses NFD, Linux/Windows use NFC), we try both forms to match + // playlists that may have been imported on a different platform. + pls, err := s.ds.Playlist(ctx).FindByPath(newPls.Path) + if errors.Is(err, model.ErrNotFound) { + // Try alternate normalization form + altPath := norm.NFD.String(newPls.Path) + if altPath == newPls.Path { + altPath = norm.NFC.String(newPls.Path) + } + if altPath != newPls.Path { + pls, err = s.ds.Playlist(ctx).FindByPath(altPath) + } + } + if err != nil && !errors.Is(err, model.ErrNotFound) { + return err + } + if err == nil && !pls.Sync { + log.Debug(ctx, "Playlist already imported and not synced", "playlist", pls.Name, "path", pls.Path) + return nil + } + + if err == nil { + log.Info(ctx, "Updating synced playlist", "playlist", pls.Name, "path", newPls.Path) + newPls.ID = pls.ID + newPls.Name = pls.Name + newPls.Comment = pls.Comment + newPls.OwnerID = pls.OwnerID + newPls.Public = pls.Public + newPls.EvaluatedAt = &time.Time{} + } else { + log.Info(ctx, "Adding synced playlist", "playlist", newPls.Name, "path", newPls.Path, "owner", owner.UserName) + newPls.OwnerID = owner.ID + // For NSP files, Public may already be set from the file; for M3U, use server default + if !newPls.IsSmartPlaylist() { + newPls.Public = conf.Server.DefaultPlaylistPublicVisibility + } + } + return s.ds.Playlist(ctx).Put(newPls) +} diff --git a/core/playlists_test.go b/core/playlists/import_test.go similarity index 94% rename from core/playlists_test.go rename to core/playlists/import_test.go index 7712a268e..a42c3f3eb 100644 --- a/core/playlists_test.go +++ b/core/playlists/import_test.go @@ -1,4 +1,4 @@ -package core_test +package playlists_test import ( "context" @@ -9,7 +9,7 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" - "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/criteria" "github.com/navidrome/navidrome/model/request" @@ -19,18 +19,18 @@ import ( "golang.org/x/text/unicode/norm" ) -var _ = Describe("Playlists", func() { +var _ = Describe("Playlists - Import", func() { var ds *tests.MockDataStore - var ps core.Playlists - var mockPlsRepo mockedPlaylistRepo + var ps playlists.Playlists + var mockPlsRepo *tests.MockPlaylistRepo var mockLibRepo *tests.MockLibraryRepo ctx := context.Background() BeforeEach(func() { - mockPlsRepo = mockedPlaylistRepo{} + mockPlsRepo = tests.CreateMockPlaylistRepo() mockLibRepo = &tests.MockLibraryRepo{} ds = &tests.MockDataStore{ - MockedPlaylist: &mockPlsRepo, + MockedPlaylist: mockPlsRepo, MockedLibrary: mockLibRepo, } ctx = request.WithUser(ctx, model.User{ID: "123"}) @@ -39,7 +39,7 @@ var _ = Describe("Playlists", func() { Describe("ImportFile", func() { var folder *model.Folder BeforeEach(func() { - ps = core.NewPlaylists(ds) + ps = playlists.NewPlaylists(ds) ds.MockedMediaFile = &mockedMediaFileRepo{} libPath, _ := os.Getwd() // Set up library with the actual library path that matches the folder @@ -61,7 +61,7 @@ var _ = Describe("Playlists", func() { Expect(pls.Tracks).To(HaveLen(2)) Expect(pls.Tracks[0].Path).To(Equal("tests/fixtures/playlists/test.mp3")) Expect(pls.Tracks[1].Path).To(Equal("tests/fixtures/playlists/test.ogg")) - Expect(mockPlsRepo.last).To(Equal(pls)) + Expect(mockPlsRepo.Last).To(Equal(pls)) }) It("parses playlists using LF ending", func() { @@ -99,7 +99,7 @@ var _ = Describe("Playlists", func() { It("parses well-formed playlists", func() { pls, err := ps.ImportFile(ctx, folder, "recently_played.nsp") Expect(err).ToNot(HaveOccurred()) - Expect(mockPlsRepo.last).To(Equal(pls)) + Expect(mockPlsRepo.Last).To(Equal(pls)) Expect(pls.OwnerID).To(Equal("123")) Expect(pls.Name).To(Equal("Recently Played")) Expect(pls.Comment).To(Equal("Recently played tracks")) @@ -149,7 +149,7 @@ var _ = Describe("Playlists", func() { tmpDir := GinkgoT().TempDir() mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{}} - ps = core.NewPlaylists(ds) + ps = playlists.NewPlaylists(ds) // Create the playlist file on disk with the filesystem's normalization form plsFile := tmpDir + "/" + filesystemName + ".m3u" @@ -163,7 +163,7 @@ var _ = Describe("Playlists", func() { Path: storedPath, Sync: true, } - mockPlsRepo.data = map[string]*model.Playlist{storedPath: existingPls} + mockPlsRepo.PathMap = map[string]*model.Playlist{storedPath: existingPls} // Import using the filesystem's normalization form plsFolder := &model.Folder{ @@ -209,7 +209,7 @@ var _ = Describe("Playlists", func() { "def.mp3", // This is playlists/def.mp3 relative to plsDir }, } - ps = core.NewPlaylists(ds) + ps = playlists.NewPlaylists(ds) }) It("handles relative paths that reference files in other libraries", func() { @@ -365,7 +365,7 @@ var _ = Describe("Playlists", func() { }, } // Recreate playlists service to pick up new mock - ps = core.NewPlaylists(ds) + ps = playlists.NewPlaylists(ds) // Create playlist in music library that references both tracks plsContent := "#PLAYLIST:Same Path Test\nalbum/track.mp3\n../classical/album/track.mp3" @@ -408,7 +408,7 @@ var _ = Describe("Playlists", func() { BeforeEach(func() { repo = &mockedMediaFileFromListRepo{} ds.MockedMediaFile = repo - ps = core.NewPlaylists(ds) + ps = playlists.NewPlaylists(ds) mockLibRepo.SetData([]model.Library{{ID: 1, Path: "/music"}, {ID: 2, Path: "/new"}}) ctx = request.WithUser(ctx, model.User{ID: "123"}) }) @@ -439,7 +439,7 @@ var _ = Describe("Playlists", func() { Expect(pls.Tracks[1].Path).To(Equal("tests/test.ogg")) Expect(pls.Tracks[2].Path).To(Equal("downloads/newfile.flac")) Expect(pls.Tracks[3].Path).To(Equal("tests/01 Invisible (RED) Edit Version.mp3")) - Expect(mockPlsRepo.last).To(Equal(pls)) + Expect(mockPlsRepo.Last).To(Equal(pls)) }) It("sets the playlist name as a timestamp if the #PLAYLIST directive is not present", func() { @@ -460,7 +460,7 @@ var _ = Describe("Playlists", func() { Expect(pls.Tracks).To(HaveLen(2)) }) - It("returns only tracks that exist in the database and in the same other as the m3u", func() { + It("returns only tracks that exist in the database and in the same order as the m3u", func() { repo.data = []string{ "album1/test1.mp3", "album2/test2.mp3", @@ -570,7 +570,7 @@ var _ = Describe("Playlists", func() { }) - Describe("InPlaylistsPath", func() { + Describe("InPath", func() { var folder model.Folder BeforeEach(func() { @@ -584,27 +584,27 @@ var _ = Describe("Playlists", func() { It("returns true if PlaylistsPath is empty", func() { conf.Server.PlaylistsPath = "" - Expect(core.InPlaylistsPath(folder)).To(BeTrue()) + Expect(playlists.InPath(folder)).To(BeTrue()) }) It("returns true if PlaylistsPath is any (**/**)", func() { conf.Server.PlaylistsPath = "**/**" - Expect(core.InPlaylistsPath(folder)).To(BeTrue()) + Expect(playlists.InPath(folder)).To(BeTrue()) }) It("returns true if folder is in PlaylistsPath", func() { conf.Server.PlaylistsPath = "other/**:playlists/**" - Expect(core.InPlaylistsPath(folder)).To(BeTrue()) + Expect(playlists.InPath(folder)).To(BeTrue()) }) It("returns false if folder is not in PlaylistsPath", func() { conf.Server.PlaylistsPath = "other" - Expect(core.InPlaylistsPath(folder)).To(BeFalse()) + Expect(playlists.InPath(folder)).To(BeFalse()) }) It("returns true if for a playlist in root of MusicFolder if PlaylistsPath is '.'", func() { conf.Server.PlaylistsPath = "." - Expect(core.InPlaylistsPath(folder)).To(BeFalse()) + Expect(playlists.InPath(folder)).To(BeFalse()) folder2 := model.Folder{ LibraryPath: "/music", @@ -612,7 +612,7 @@ var _ = Describe("Playlists", func() { Name: ".", } - Expect(core.InPlaylistsPath(folder2)).To(BeTrue()) + Expect(playlists.InPath(folder2)).To(BeTrue()) }) }) }) @@ -693,23 +693,3 @@ func (r *mockedMediaFileFromListRepo) FindByPaths(paths []string) (model.MediaFi } return mfs, nil } - -type mockedPlaylistRepo struct { - last *model.Playlist - data map[string]*model.Playlist // keyed by path - model.PlaylistRepository -} - -func (r *mockedPlaylistRepo) FindByPath(path string) (*model.Playlist, error) { - if r.data != nil { - if pls, ok := r.data[path]; ok { - return pls, nil - } - } - return nil, model.ErrNotFound -} - -func (r *mockedPlaylistRepo) Put(pls *model.Playlist) error { - r.last = pls - return nil -} diff --git a/core/playlists.go b/core/playlists/parse_m3u.go similarity index 53% rename from core/playlists.go rename to core/playlists/parse_m3u.go index 1c070a110..4e79d15c6 100644 --- a/core/playlists.go +++ b/core/playlists/parse_m3u.go @@ -1,183 +1,28 @@ -package core +package playlists import ( "cmp" "context" - "encoding/json" - "errors" "fmt" "io" "net/url" - "os" "path/filepath" "slices" "strings" "time" - "github.com/RaveNoX/go-jsoncommentstrip" - "github.com/bmatcuk/doublestar/v4" - "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" - "github.com/navidrome/navidrome/model/criteria" - "github.com/navidrome/navidrome/model/request" - "github.com/navidrome/navidrome/utils/ioutils" "github.com/navidrome/navidrome/utils/slice" "golang.org/x/text/unicode/norm" ) -type Playlists interface { - ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) - Update(ctx context.Context, playlistID string, name *string, comment *string, public *bool, idsToAdd []string, idxToRemove []int) error - ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error) -} - -type playlists struct { - ds model.DataStore -} - -func NewPlaylists(ds model.DataStore) Playlists { - return &playlists{ds: ds} -} - -func InPlaylistsPath(folder model.Folder) bool { - if conf.Server.PlaylistsPath == "" { - return true - } - rel, _ := filepath.Rel(folder.LibraryPath, folder.AbsolutePath()) - for path := range strings.SplitSeq(conf.Server.PlaylistsPath, string(filepath.ListSeparator)) { - if match, _ := doublestar.Match(path, rel); match { - return true - } - } - return false -} - -func (s *playlists) ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) { - pls, err := s.parsePlaylist(ctx, filename, folder) - if err != nil { - log.Error(ctx, "Error parsing playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err) - return nil, err - } - log.Debug("Found playlist", "name", pls.Name, "lastUpdated", pls.UpdatedAt, "path", pls.Path, "numTracks", len(pls.Tracks)) - err = s.updatePlaylist(ctx, pls) - if err != nil { - log.Error(ctx, "Error updating playlist", "path", filepath.Join(folder.AbsolutePath(), filename), err) - } - return pls, err -} - -func (s *playlists) ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error) { - owner, _ := request.UserFrom(ctx) - pls := &model.Playlist{ - OwnerID: owner.ID, - Public: false, - Sync: false, - } - err := s.parseM3U(ctx, pls, nil, reader) - if err != nil { - log.Error(ctx, "Error parsing playlist", err) - return nil, err - } - err = s.ds.Playlist(ctx).Put(pls) - if err != nil { - log.Error(ctx, "Error saving playlist", err) - return nil, err - } - return pls, nil -} - -func (s *playlists) parsePlaylist(ctx context.Context, playlistFile string, folder *model.Folder) (*model.Playlist, error) { - pls, err := s.newSyncedPlaylist(folder.AbsolutePath(), playlistFile) - if err != nil { - return nil, err - } - - file, err := os.Open(pls.Path) - if err != nil { - return nil, err - } - defer file.Close() - - reader := ioutils.UTF8Reader(file) - extension := strings.ToLower(filepath.Ext(playlistFile)) - switch extension { - case ".nsp": - err = s.parseNSP(ctx, pls, reader) - default: - err = s.parseM3U(ctx, pls, folder, reader) - } - return pls, err -} - -func (s *playlists) newSyncedPlaylist(baseDir string, playlistFile string) (*model.Playlist, error) { - playlistPath := filepath.Join(baseDir, playlistFile) - info, err := os.Stat(playlistPath) - if err != nil { - return nil, err - } - - var extension = filepath.Ext(playlistFile) - var name = playlistFile[0 : len(playlistFile)-len(extension)] - - pls := &model.Playlist{ - Name: name, - Comment: fmt.Sprintf("Auto-imported from '%s'", playlistFile), - Public: false, - Path: playlistPath, - Sync: true, - UpdatedAt: info.ModTime(), - } - return pls, nil -} - -func getPositionFromOffset(data []byte, offset int64) (line, column int) { - line = 1 - for _, b := range data[:offset] { - if b == '\n' { - line++ - column = 1 - } else { - column++ - } - } - return -} - -func (s *playlists) parseNSP(_ context.Context, pls *model.Playlist, reader io.Reader) error { - nsp := &nspFile{} - reader = io.LimitReader(reader, 100*1024) // Limit to 100KB - reader = jsoncommentstrip.NewReader(reader) - input, err := io.ReadAll(reader) - if err != nil { - return fmt.Errorf("reading SmartPlaylist: %w", err) - } - err = json.Unmarshal(input, nsp) - if err != nil { - var syntaxErr *json.SyntaxError - if errors.As(err, &syntaxErr) { - line, col := getPositionFromOffset(input, syntaxErr.Offset) - return fmt.Errorf("JSON syntax error in SmartPlaylist at line %d, column %d: %w", line, col, err) - } - return fmt.Errorf("JSON parsing error in SmartPlaylist: %w", err) - } - pls.Rules = &nsp.Criteria - if nsp.Name != "" { - pls.Name = nsp.Name - } - if nsp.Comment != "" { - pls.Comment = nsp.Comment - } - if nsp.Public != nil { - pls.Public = *nsp.Public - } else { - pls.Public = conf.Server.DefaultPlaylistPublicVisibility - } - return nil -} - func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *model.Folder, reader io.Reader) error { mediaFileRepository := s.ds.MediaFile(ctx) + resolver, err := newPathResolver(ctx, s.ds) + if err != nil { + return err + } var mfs model.MediaFiles // Chunk size of 100 lines, as each line can generate up to 4 lookup candidates // (NFC/NFD × raw/lowercase), and SQLite has a max expression tree depth of 1000. @@ -202,7 +47,7 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *m } filteredLines = append(filteredLines, line) } - resolvedPaths, err := s.resolvePaths(ctx, folder, filteredLines) + resolvedPaths, err := resolver.resolvePaths(ctx, folder, filteredLines) if err != nil { log.Warn(ctx, "Error resolving paths in playlist", "playlist", pls.Name, err) continue @@ -258,7 +103,9 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *m existing[key] = idx } - // Find media files in the order of the resolved paths, to keep playlist order + // Find media files in the order of the resolved paths, to keep playlist order. + // Both `existing` keys and `resolvedPaths` use the library-qualified format "libraryID:relativePath", + // so normalizing the full string produces matching keys (digits and ':' are ASCII-invariant). for _, path := range resolvedPaths { key := strings.ToLower(norm.NFC.String(path)) idx, ok := existing[key] @@ -398,15 +245,10 @@ func (r *pathResolver) findInLibraries(absolutePath string) pathResolution { // resolvePaths converts playlist file paths to library-qualified paths (format: "libraryID:relativePath"). // For relative paths, it resolves them to absolute paths first, then determines which // library they belong to. This allows playlists to reference files across library boundaries. -func (s *playlists) resolvePaths(ctx context.Context, folder *model.Folder, lines []string) ([]string, error) { - resolver, err := newPathResolver(ctx, s.ds) - if err != nil { - return nil, err - } - +func (r *pathResolver) resolvePaths(ctx context.Context, folder *model.Folder, lines []string) ([]string, error) { results := make([]string, 0, len(lines)) for idx, line := range lines { - resolution := resolver.resolvePath(line, folder) + resolution := r.resolvePath(line, folder) if !resolution.valid { log.Warn(ctx, "Path in playlist not found in any library", "path", line, "line", idx) @@ -425,123 +267,3 @@ func (s *playlists) resolvePaths(ctx context.Context, folder *model.Folder, line return results, nil } - -func (s *playlists) updatePlaylist(ctx context.Context, newPls *model.Playlist) error { - owner, _ := request.UserFrom(ctx) - - // Try to find existing playlist by path. Since filesystem normalization differs across - // platforms (macOS uses NFD, Linux/Windows use NFC), we try both forms to match - // playlists that may have been imported on a different platform. - pls, err := s.ds.Playlist(ctx).FindByPath(newPls.Path) - if errors.Is(err, model.ErrNotFound) { - // Try alternate normalization form - altPath := norm.NFD.String(newPls.Path) - if altPath == newPls.Path { - altPath = norm.NFC.String(newPls.Path) - } - if altPath != newPls.Path { - pls, err = s.ds.Playlist(ctx).FindByPath(altPath) - } - } - if err != nil && !errors.Is(err, model.ErrNotFound) { - return err - } - if err == nil && !pls.Sync { - log.Debug(ctx, "Playlist already imported and not synced", "playlist", pls.Name, "path", pls.Path) - return nil - } - - if err == nil { - log.Info(ctx, "Updating synced playlist", "playlist", pls.Name, "path", newPls.Path) - newPls.ID = pls.ID - newPls.Name = pls.Name - newPls.Comment = pls.Comment - newPls.OwnerID = pls.OwnerID - newPls.Public = pls.Public - newPls.EvaluatedAt = &time.Time{} - } else { - log.Info(ctx, "Adding synced playlist", "playlist", newPls.Name, "path", newPls.Path, "owner", owner.UserName) - newPls.OwnerID = owner.ID - // For NSP files, Public may already be set from the file; for M3U, use server default - if !newPls.IsSmartPlaylist() { - newPls.Public = conf.Server.DefaultPlaylistPublicVisibility - } - } - return s.ds.Playlist(ctx).Put(newPls) -} - -func (s *playlists) Update(ctx context.Context, playlistID string, - name *string, comment *string, public *bool, - idsToAdd []string, idxToRemove []int) error { - needsInfoUpdate := name != nil || comment != nil || public != nil - needsTrackRefresh := len(idxToRemove) > 0 - - return s.ds.WithTxImmediate(func(tx model.DataStore) error { - var pls *model.Playlist - var err error - repo := tx.Playlist(ctx) - tracks := repo.Tracks(playlistID, true) - if tracks == nil { - return fmt.Errorf("%w: playlist '%s'", model.ErrNotFound, playlistID) - } - if needsTrackRefresh { - pls, err = repo.GetWithTracks(playlistID, true, false) - pls.RemoveTracks(idxToRemove) - pls.AddMediaFilesByID(idsToAdd) - } else { - if len(idsToAdd) > 0 { - _, err = tracks.Add(idsToAdd) - if err != nil { - return err - } - } - if needsInfoUpdate { - pls, err = repo.Get(playlistID) - } - } - if err != nil { - return err - } - if !needsTrackRefresh && !needsInfoUpdate { - return nil - } - - if name != nil { - pls.Name = *name - } - if comment != nil { - pls.Comment = *comment - } - if public != nil { - pls.Public = *public - } - // Special case: The playlist is now empty - if len(idxToRemove) > 0 && len(pls.Tracks) == 0 { - if err = tracks.DeleteAll(); err != nil { - return err - } - } - return repo.Put(pls) - }) -} - -type nspFile struct { - criteria.Criteria - Name string `json:"name"` - Comment string `json:"comment"` - Public *bool `json:"public"` -} - -func (i *nspFile) UnmarshalJSON(data []byte) error { - m := map[string]any{} - err := json.Unmarshal(data, &m) - if err != nil { - return err - } - i.Name, _ = m["name"].(string) - i.Comment, _ = m["comment"].(string) - if public, ok := m["public"].(bool); ok { - i.Public = &public - } - return json.Unmarshal(data, &i.Criteria) -} diff --git a/core/playlists_internal_test.go b/core/playlists/parse_m3u_test.go similarity index 91% rename from core/playlists_internal_test.go rename to core/playlists/parse_m3u_test.go index 88e36cc3a..05e1c30e1 100644 --- a/core/playlists_internal_test.go +++ b/core/playlists/parse_m3u_test.go @@ -1,4 +1,4 @@ -package core +package playlists import ( "context" @@ -214,38 +214,38 @@ var _ = Describe("pathResolver", func() { }) Describe("resolvePath", func() { - It("resolves absolute paths", func() { - resolution := resolver.resolvePath("/music/artist/album/track.mp3", nil) + Context("basic", func() { + It("resolves absolute paths", func() { + resolution := resolver.resolvePath("/music/artist/album/track.mp3", nil) - Expect(resolution.valid).To(BeTrue()) - Expect(resolution.libraryID).To(Equal(1)) - Expect(resolution.libraryPath).To(Equal("/music")) - Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3")) + Expect(resolution.valid).To(BeTrue()) + Expect(resolution.libraryID).To(Equal(1)) + Expect(resolution.libraryPath).To(Equal("/music")) + Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3")) + }) + + It("resolves relative paths when folder is provided", func() { + folder := &model.Folder{ + Path: "playlists", + LibraryPath: "/music", + LibraryID: 1, + } + + resolution := resolver.resolvePath("../artist/album/track.mp3", folder) + + Expect(resolution.valid).To(BeTrue()) + Expect(resolution.libraryID).To(Equal(1)) + Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3")) + }) + + It("returns invalid resolution for paths outside any library", func() { + resolution := resolver.resolvePath("/outside/library/track.mp3", nil) + + Expect(resolution.valid).To(BeFalse()) + }) }) - It("resolves relative paths when folder is provided", func() { - folder := &model.Folder{ - Path: "playlists", - LibraryPath: "/music", - LibraryID: 1, - } - - resolution := resolver.resolvePath("../artist/album/track.mp3", folder) - - Expect(resolution.valid).To(BeTrue()) - Expect(resolution.libraryID).To(Equal(1)) - Expect(resolution.absolutePath).To(Equal("/music/artist/album/track.mp3")) - }) - - It("returns invalid resolution for paths outside any library", func() { - resolution := resolver.resolvePath("/outside/library/track.mp3", nil) - - Expect(resolution.valid).To(BeFalse()) - }) - }) - - Describe("resolvePath", func() { - Context("With absolute paths", func() { + Context("cross-library", func() { It("resolves path within a library", func() { resolution := resolver.resolvePath("/music/track.mp3", nil) diff --git a/core/playlists/parse_nsp.go b/core/playlists/parse_nsp.go new file mode 100644 index 000000000..c7f7dd946 --- /dev/null +++ b/core/playlists/parse_nsp.go @@ -0,0 +1,103 @@ +package playlists + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + + "github.com/RaveNoX/go-jsoncommentstrip" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/criteria" +) + +func (s *playlists) newSyncedPlaylist(baseDir string, playlistFile string) (*model.Playlist, error) { + playlistPath := filepath.Join(baseDir, playlistFile) + info, err := os.Stat(playlistPath) + if err != nil { + return nil, err + } + + var extension = filepath.Ext(playlistFile) + var name = playlistFile[0 : len(playlistFile)-len(extension)] + + pls := &model.Playlist{ + Name: name, + Comment: fmt.Sprintf("Auto-imported from '%s'", playlistFile), + Public: false, + Path: playlistPath, + Sync: true, + UpdatedAt: info.ModTime(), + } + return pls, nil +} + +func getPositionFromOffset(data []byte, offset int64) (line, column int) { + line = 1 + for _, b := range data[:offset] { + if b == '\n' { + line++ + column = 1 + } else { + column++ + } + } + return +} + +func (s *playlists) parseNSP(_ context.Context, pls *model.Playlist, reader io.Reader) error { + nsp := &nspFile{} + reader = io.LimitReader(reader, 100*1024) // Limit to 100KB + reader = jsoncommentstrip.NewReader(reader) + input, err := io.ReadAll(reader) + if err != nil { + return fmt.Errorf("reading SmartPlaylist: %w", err) + } + err = json.Unmarshal(input, nsp) + if err != nil { + var syntaxErr *json.SyntaxError + if errors.As(err, &syntaxErr) { + line, col := getPositionFromOffset(input, syntaxErr.Offset) + return fmt.Errorf("JSON syntax error in SmartPlaylist at line %d, column %d: %w", line, col, err) + } + return fmt.Errorf("JSON parsing error in SmartPlaylist: %w", err) + } + pls.Rules = &nsp.Criteria + if nsp.Name != "" { + pls.Name = nsp.Name + } + if nsp.Comment != "" { + pls.Comment = nsp.Comment + } + if nsp.Public != nil { + pls.Public = *nsp.Public + } else { + pls.Public = conf.Server.DefaultPlaylistPublicVisibility + } + return nil +} + +type nspFile struct { + criteria.Criteria + Name string `json:"name"` + Comment string `json:"comment"` + Public *bool `json:"public"` +} + +func (i *nspFile) UnmarshalJSON(data []byte) error { + m := map[string]any{} + err := json.Unmarshal(data, &m) + if err != nil { + return err + } + i.Name, _ = m["name"].(string) + i.Comment, _ = m["comment"].(string) + if public, ok := m["public"].(bool); ok { + i.Public = &public + } + return json.Unmarshal(data, &i.Criteria) +} diff --git a/core/playlists/parse_nsp_test.go b/core/playlists/parse_nsp_test.go new file mode 100644 index 000000000..0a7b2727e --- /dev/null +++ b/core/playlists/parse_nsp_test.go @@ -0,0 +1,213 @@ +package playlists + +import ( + "context" + "os" + "path/filepath" + "strings" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/criteria" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("parseNSP", func() { + var s *playlists + ctx := context.Background() + + BeforeEach(func() { + s = &playlists{} + }) + + It("parses a well-formed NSP with all fields", func() { + nsp := `{ + "name": "My Smart Playlist", + "comment": "A test playlist", + "public": true, + "all": [{"is": {"loved": true}}], + "sort": "title", + "order": "asc", + "limit": 50 + }` + pls := &model.Playlist{Name: "default-name"} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Name).To(Equal("My Smart Playlist")) + Expect(pls.Comment).To(Equal("A test playlist")) + Expect(pls.Public).To(BeTrue()) + Expect(pls.Rules).ToNot(BeNil()) + Expect(pls.Rules.Sort).To(Equal("title")) + Expect(pls.Rules.Order).To(Equal("asc")) + Expect(pls.Rules.Limit).To(Equal(50)) + Expect(pls.Rules.Expression).To(BeAssignableToTypeOf(criteria.All{})) + }) + + It("keeps existing name when NSP has no name field", func() { + nsp := `{"all": [{"is": {"loved": true}}]}` + pls := &model.Playlist{Name: "Original Name"} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Name).To(Equal("Original Name")) + }) + + It("keeps existing comment when NSP has no comment field", func() { + nsp := `{"all": [{"is": {"loved": true}}]}` + pls := &model.Playlist{Comment: "Original Comment"} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Comment).To(Equal("Original Comment")) + }) + + It("strips JSON comments before parsing", func() { + nsp := `{ + // Line comment + "name": "Commented Playlist", + /* Block comment */ + "all": [{"is": {"loved": true}}] + }` + pls := &model.Playlist{} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Name).To(Equal("Commented Playlist")) + }) + + It("uses server default when public field is absent", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.DefaultPlaylistPublicVisibility = true + + nsp := `{"all": [{"is": {"loved": true}}]}` + pls := &model.Playlist{} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Public).To(BeTrue()) + }) + + It("honors explicit public: false over server default", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.DefaultPlaylistPublicVisibility = true + + nsp := `{"public": false, "all": [{"is": {"loved": true}}]}` + pls := &model.Playlist{} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Public).To(BeFalse()) + }) + + It("returns a syntax error with line and column info", func() { + nsp := "{\n \"name\": \"Bad\",\n \"all\": [INVALID]\n}" + pls := &model.Playlist{} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("JSON syntax error in SmartPlaylist")) + Expect(err.Error()).To(MatchRegexp(`line \d+, column \d+`)) + }) + + It("returns a parsing error for completely invalid JSON", func() { + nsp := `not json at all` + pls := &model.Playlist{} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("SmartPlaylist")) + }) + + It("gracefully handles non-string name field", func() { + nsp := `{"name": 123, "all": [{"is": {"loved": true}}]}` + pls := &model.Playlist{Name: "Original"} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + // Type assertion in UnmarshalJSON fails silently; name stays as original + Expect(pls.Name).To(Equal("Original")) + }) + + It("parses criteria with multiple rules", func() { + nsp := `{ + "all": [ + {"is": {"loved": true}}, + {"contains": {"title": "rock"}} + ], + "sort": "lastPlayed", + "order": "desc", + "limit": 100 + }` + pls := &model.Playlist{} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Rules).ToNot(BeNil()) + Expect(pls.Rules.Sort).To(Equal("lastPlayed")) + Expect(pls.Rules.Order).To(Equal("desc")) + Expect(pls.Rules.Limit).To(Equal(100)) + }) +}) + +var _ = Describe("getPositionFromOffset", func() { + It("returns correct position on first line", func() { + data := []byte("hello world") + line, col := getPositionFromOffset(data, 5) + Expect(line).To(Equal(1)) + Expect(col).To(Equal(5)) + }) + + It("returns correct position after newlines", func() { + data := []byte("line1\nline2\nline3") + // Offsets: l(0) i(1) n(2) e(3) 1(4) \n(5) l(6) i(7) n(8) + line, col := getPositionFromOffset(data, 8) + Expect(line).To(Equal(2)) + Expect(col).To(Equal(3)) + }) + + It("returns correct position at start of new line", func() { + data := []byte("line1\nline2") + // After \n at offset 5, col resets to 1; offset 6 is 'l' -> col=1 + line, col := getPositionFromOffset(data, 6) + Expect(line).To(Equal(2)) + Expect(col).To(Equal(1)) + }) + + It("handles multiple newlines", func() { + data := []byte("a\nb\nc\nd") + // a(0) \n(1) b(2) \n(3) c(4) \n(5) d(6) + line, col := getPositionFromOffset(data, 6) + Expect(line).To(Equal(4)) + Expect(col).To(Equal(1)) + }) +}) + +var _ = Describe("newSyncedPlaylist", func() { + var s *playlists + + BeforeEach(func() { + s = &playlists{} + }) + + It("creates a synced playlist with correct attributes", func() { + tmpDir := GinkgoT().TempDir() + Expect(os.WriteFile(filepath.Join(tmpDir, "test.m3u"), []byte("content"), 0600)).To(Succeed()) + + pls, err := s.newSyncedPlaylist(tmpDir, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Name).To(Equal("test")) + Expect(pls.Comment).To(Equal("Auto-imported from 'test.m3u'")) + Expect(pls.Public).To(BeFalse()) + Expect(pls.Path).To(Equal(filepath.Join(tmpDir, "test.m3u"))) + Expect(pls.Sync).To(BeTrue()) + Expect(pls.UpdatedAt).ToNot(BeZero()) + }) + + It("strips extension from filename to derive name", func() { + tmpDir := GinkgoT().TempDir() + Expect(os.WriteFile(filepath.Join(tmpDir, "My Favorites.nsp"), []byte("{}"), 0600)).To(Succeed()) + + pls, err := s.newSyncedPlaylist(tmpDir, "My Favorites.nsp") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Name).To(Equal("My Favorites")) + }) + + It("returns error for non-existent file", func() { + tmpDir := GinkgoT().TempDir() + _, err := s.newSyncedPlaylist(tmpDir, "nonexistent.m3u") + Expect(err).To(HaveOccurred()) + }) +}) diff --git a/core/playlists/playlists.go b/core/playlists/playlists.go new file mode 100644 index 000000000..5a9a908ee --- /dev/null +++ b/core/playlists/playlists.go @@ -0,0 +1,265 @@ +package playlists + +import ( + "context" + "io" + "path/filepath" + "strconv" + "strings" + + "github.com/bmatcuk/doublestar/v4" + "github.com/deluan/rest" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" +) + +type Playlists interface { + // Reads + GetAll(ctx context.Context, options ...model.QueryOptions) (model.Playlists, error) + Get(ctx context.Context, id string) (*model.Playlist, error) + GetWithTracks(ctx context.Context, id string) (*model.Playlist, error) + GetPlaylists(ctx context.Context, mediaFileId string) (model.Playlists, error) + + // Mutations + Create(ctx context.Context, playlistId string, name string, ids []string) (string, error) + Delete(ctx context.Context, id string) error + Update(ctx context.Context, playlistID string, name *string, comment *string, public *bool, idsToAdd []string, idxToRemove []int) error + + // Track management + AddTracks(ctx context.Context, playlistID string, ids []string) (int, error) + AddAlbums(ctx context.Context, playlistID string, albumIds []string) (int, error) + AddArtists(ctx context.Context, playlistID string, artistIds []string) (int, error) + AddDiscs(ctx context.Context, playlistID string, discs []model.DiscID) (int, error) + RemoveTracks(ctx context.Context, playlistID string, trackIds []string) error + ReorderTrack(ctx context.Context, playlistID string, pos int, newPos int) error + + // Import + ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) + ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error) + + // REST adapters (follows Share/Library pattern) + NewRepository(ctx context.Context) rest.Repository + TracksRepository(ctx context.Context, playlistId string, refreshSmartPlaylist bool) rest.Repository +} + +type playlists struct { + ds model.DataStore +} + +func NewPlaylists(ds model.DataStore) Playlists { + return &playlists{ds: ds} +} + +func InPath(folder model.Folder) bool { + if conf.Server.PlaylistsPath == "" { + return true + } + rel, _ := filepath.Rel(folder.LibraryPath, folder.AbsolutePath()) + for path := range strings.SplitSeq(conf.Server.PlaylistsPath, string(filepath.ListSeparator)) { + if match, _ := doublestar.Match(path, rel); match { + return true + } + } + return false +} + +// --- Read operations --- + +func (s *playlists) GetAll(ctx context.Context, options ...model.QueryOptions) (model.Playlists, error) { + return s.ds.Playlist(ctx).GetAll(options...) +} + +func (s *playlists) Get(ctx context.Context, id string) (*model.Playlist, error) { + return s.ds.Playlist(ctx).Get(id) +} + +func (s *playlists) GetWithTracks(ctx context.Context, id string) (*model.Playlist, error) { + return s.ds.Playlist(ctx).GetWithTracks(id, true, false) +} + +func (s *playlists) GetPlaylists(ctx context.Context, mediaFileId string) (model.Playlists, error) { + return s.ds.Playlist(ctx).GetPlaylists(mediaFileId) +} + +// --- Mutation operations --- + +// Create creates a new playlist (when name is provided) or replaces tracks on an existing +// playlist (when playlistId is provided). This matches the Subsonic createPlaylist semantics. +func (s *playlists) Create(ctx context.Context, playlistId string, name string, ids []string) (string, error) { + usr, _ := request.UserFrom(ctx) + err := s.ds.WithTxImmediate(func(tx model.DataStore) error { + var pls *model.Playlist + var err error + + if playlistId != "" { + pls, err = tx.Playlist(ctx).Get(playlistId) + if err != nil { + return err + } + if pls.IsSmartPlaylist() { + return model.ErrNotAuthorized + } + if !usr.IsAdmin && pls.OwnerID != usr.ID { + return model.ErrNotAuthorized + } + } else { + pls = &model.Playlist{Name: name} + pls.OwnerID = usr.ID + } + pls.Tracks = nil + pls.AddMediaFilesByID(ids) + + err = tx.Playlist(ctx).Put(pls) + playlistId = pls.ID + return err + }) + return playlistId, err +} + +func (s *playlists) Delete(ctx context.Context, id string) error { + if _, err := s.checkWritable(ctx, id); err != nil { + return err + } + return s.ds.Playlist(ctx).Delete(id) +} + +func (s *playlists) Update(ctx context.Context, playlistID string, + name *string, comment *string, public *bool, + idsToAdd []string, idxToRemove []int) error { + var pls *model.Playlist + var err error + hasTrackChanges := len(idsToAdd) > 0 || len(idxToRemove) > 0 + if hasTrackChanges { + pls, err = s.checkTracksEditable(ctx, playlistID) + } else { + pls, err = s.checkWritable(ctx, playlistID) + } + if err != nil { + return err + } + return s.ds.WithTxImmediate(func(tx model.DataStore) error { + repo := tx.Playlist(ctx) + + if len(idxToRemove) > 0 { + tracksRepo := repo.Tracks(playlistID, false) + // Convert 0-based indices to 1-based position IDs and delete them directly, + // avoiding the need to load all tracks into memory. + positions := make([]string, len(idxToRemove)) + for i, idx := range idxToRemove { + positions[i] = strconv.Itoa(idx + 1) + } + if err := tracksRepo.Delete(positions...); err != nil { + return err + } + if len(idsToAdd) > 0 { + if _, err := tracksRepo.Add(idsToAdd); err != nil { + return err + } + } + return s.updateMetadata(ctx, tx, pls, name, comment, public) + } + + if len(idsToAdd) > 0 { + if _, err := repo.Tracks(playlistID, false).Add(idsToAdd); err != nil { + return err + } + } + if name == nil && comment == nil && public == nil { + return nil + } + // Reuse the playlist from checkWritable (no tracks loaded, so Put only refreshes counters) + return s.updateMetadata(ctx, tx, pls, name, comment, public) + }) +} + +// --- Permission helpers --- + +// checkWritable fetches the playlist and verifies the current user can modify it. +func (s *playlists) checkWritable(ctx context.Context, id string) (*model.Playlist, error) { + pls, err := s.ds.Playlist(ctx).Get(id) + if err != nil { + return nil, err + } + usr, _ := request.UserFrom(ctx) + if !usr.IsAdmin && pls.OwnerID != usr.ID { + return nil, model.ErrNotAuthorized + } + return pls, nil +} + +// checkTracksEditable verifies the user can modify tracks (ownership + not smart playlist). +func (s *playlists) checkTracksEditable(ctx context.Context, playlistID string) (*model.Playlist, error) { + pls, err := s.checkWritable(ctx, playlistID) + if err != nil { + return nil, err + } + if pls.IsSmartPlaylist() { + return nil, model.ErrNotAuthorized + } + return pls, nil +} + +// updateMetadata applies optional metadata changes to a playlist and persists it. +// Accepts a DataStore parameter so it can be used inside transactions. +// The caller is responsible for permission checks. +func (s *playlists) updateMetadata(ctx context.Context, ds model.DataStore, pls *model.Playlist, name *string, comment *string, public *bool) error { + if name != nil { + pls.Name = *name + } + if comment != nil { + pls.Comment = *comment + } + if public != nil { + pls.Public = *public + } + return ds.Playlist(ctx).Put(pls) +} + +// --- Track management operations --- + +func (s *playlists) AddTracks(ctx context.Context, playlistID string, ids []string) (int, error) { + if _, err := s.checkTracksEditable(ctx, playlistID); err != nil { + return 0, err + } + return s.ds.Playlist(ctx).Tracks(playlistID, false).Add(ids) +} + +func (s *playlists) AddAlbums(ctx context.Context, playlistID string, albumIds []string) (int, error) { + if _, err := s.checkTracksEditable(ctx, playlistID); err != nil { + return 0, err + } + return s.ds.Playlist(ctx).Tracks(playlistID, false).AddAlbums(albumIds) +} + +func (s *playlists) AddArtists(ctx context.Context, playlistID string, artistIds []string) (int, error) { + if _, err := s.checkTracksEditable(ctx, playlistID); err != nil { + return 0, err + } + return s.ds.Playlist(ctx).Tracks(playlistID, false).AddArtists(artistIds) +} + +func (s *playlists) AddDiscs(ctx context.Context, playlistID string, discs []model.DiscID) (int, error) { + if _, err := s.checkTracksEditable(ctx, playlistID); err != nil { + return 0, err + } + return s.ds.Playlist(ctx).Tracks(playlistID, false).AddDiscs(discs) +} + +func (s *playlists) RemoveTracks(ctx context.Context, playlistID string, trackIds []string) error { + if _, err := s.checkTracksEditable(ctx, playlistID); err != nil { + return err + } + return s.ds.WithTx(func(tx model.DataStore) error { + return tx.Playlist(ctx).Tracks(playlistID, false).Delete(trackIds...) + }) +} + +func (s *playlists) ReorderTrack(ctx context.Context, playlistID string, pos int, newPos int) error { + if _, err := s.checkTracksEditable(ctx, playlistID); err != nil { + return err + } + return s.ds.WithTx(func(tx model.DataStore) error { + return tx.Playlist(ctx).Tracks(playlistID, false).Reorder(pos, newPos) + }) +} diff --git a/core/playlists/playlists_suite_test.go b/core/playlists/playlists_suite_test.go new file mode 100644 index 000000000..b57248490 --- /dev/null +++ b/core/playlists/playlists_suite_test.go @@ -0,0 +1,17 @@ +package playlists_test + +import ( + "testing" + + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +func TestPlaylists(t *testing.T) { + tests.Init(t, false) + log.SetLevel(log.LevelFatal) + RegisterFailHandler(Fail) + RunSpecs(t, "Playlists Suite") +} diff --git a/core/playlists/playlists_test.go b/core/playlists/playlists_test.go new file mode 100644 index 000000000..a4c309d77 --- /dev/null +++ b/core/playlists/playlists_test.go @@ -0,0 +1,297 @@ +package playlists_test + +import ( + "context" + + "github.com/navidrome/navidrome/core/playlists" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/criteria" + "github.com/navidrome/navidrome/model/request" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("Playlists", func() { + var ds *tests.MockDataStore + var ps playlists.Playlists + var mockPlsRepo *tests.MockPlaylistRepo + ctx := context.Background() + + BeforeEach(func() { + mockPlsRepo = tests.CreateMockPlaylistRepo() + ds = &tests.MockDataStore{ + MockedPlaylist: mockPlsRepo, + MockedLibrary: &tests.MockLibraryRepo{}, + } + ctx = request.WithUser(ctx, model.User{ID: "123"}) + }) + + Describe("Delete", func() { + var mockTracks *tests.MockPlaylistTrackRepo + + BeforeEach(func() { + mockTracks = &tests.MockPlaylistTrackRepo{AddCount: 3} + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"}, + } + mockPlsRepo.TracksRepo = mockTracks + ps = playlists.NewPlaylists(ds) + }) + + It("allows owner to delete their playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.Delete(ctx, "pls-1") + Expect(err).ToNot(HaveOccurred()) + Expect(mockPlsRepo.Deleted).To(ContainElement("pls-1")) + }) + + It("allows admin to delete any playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true}) + err := ps.Delete(ctx, "pls-1") + Expect(err).ToNot(HaveOccurred()) + Expect(mockPlsRepo.Deleted).To(ContainElement("pls-1")) + }) + + It("denies non-owner, non-admin from deleting", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + err := ps.Delete(ctx, "pls-1") + Expect(err).To(MatchError(model.ErrNotAuthorized)) + Expect(mockPlsRepo.Deleted).To(BeEmpty()) + }) + + It("returns error when playlist not found", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.Delete(ctx, "nonexistent") + Expect(err).To(Equal(model.ErrNotFound)) + }) + }) + + Describe("Create", func() { + BeforeEach(func() { + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "Existing", OwnerID: "user-1"}, + "pls-2": {ID: "pls-2", Name: "Other's", OwnerID: "other-user"}, + "pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1", + Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}}, + } + ps = playlists.NewPlaylists(ds) + }) + + It("creates a new playlist with owner set from context", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + id, err := ps.Create(ctx, "", "New Playlist", []string{"song-1", "song-2"}) + Expect(err).ToNot(HaveOccurred()) + Expect(id).ToNot(BeEmpty()) + Expect(mockPlsRepo.Last.Name).To(Equal("New Playlist")) + Expect(mockPlsRepo.Last.OwnerID).To(Equal("user-1")) + }) + + It("replaces tracks on existing playlist when owner matches", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + id, err := ps.Create(ctx, "pls-1", "", []string{"song-3"}) + Expect(err).ToNot(HaveOccurred()) + Expect(id).To(Equal("pls-1")) + Expect(mockPlsRepo.Last.Tracks).To(HaveLen(1)) + }) + + It("allows admin to replace tracks on any playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true}) + id, err := ps.Create(ctx, "pls-2", "", []string{"song-3"}) + Expect(err).ToNot(HaveOccurred()) + Expect(id).To(Equal("pls-2")) + }) + + It("denies non-owner, non-admin from replacing tracks on existing playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + _, err := ps.Create(ctx, "pls-2", "", []string{"song-3"}) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("returns error when existing playlistId not found", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + _, err := ps.Create(ctx, "nonexistent", "", []string{"song-1"}) + Expect(err).To(Equal(model.ErrNotFound)) + }) + + It("denies replacing tracks on a smart playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + _, err := ps.Create(ctx, "pls-smart", "", []string{"song-1"}) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + }) + + Describe("Update", func() { + var mockTracks *tests.MockPlaylistTrackRepo + + BeforeEach(func() { + mockTracks = &tests.MockPlaylistTrackRepo{AddCount: 2} + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"}, + "pls-other": {ID: "pls-other", Name: "Other's", OwnerID: "other-user"}, + "pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1", + Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}}, + } + mockPlsRepo.TracksRepo = mockTracks + ps = playlists.NewPlaylists(ds) + }) + + It("allows owner to update their playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + newName := "Updated Name" + err := ps.Update(ctx, "pls-1", &newName, nil, nil, nil, nil) + Expect(err).ToNot(HaveOccurred()) + }) + + It("allows admin to update any playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true}) + newName := "Updated Name" + err := ps.Update(ctx, "pls-other", &newName, nil, nil, nil, nil) + Expect(err).ToNot(HaveOccurred()) + }) + + It("denies non-owner, non-admin from updating", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + newName := "Updated Name" + err := ps.Update(ctx, "pls-1", &newName, nil, nil, nil, nil) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("returns error when playlist not found", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + newName := "Updated Name" + err := ps.Update(ctx, "nonexistent", &newName, nil, nil, nil, nil) + Expect(err).To(Equal(model.ErrNotFound)) + }) + + It("denies adding tracks to a smart playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.Update(ctx, "pls-smart", nil, nil, nil, []string{"song-1"}, nil) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("denies removing tracks from a smart playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.Update(ctx, "pls-smart", nil, nil, nil, nil, []int{0}) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("allows metadata updates on a smart playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + newName := "Updated Smart" + err := ps.Update(ctx, "pls-smart", &newName, nil, nil, nil, nil) + Expect(err).ToNot(HaveOccurred()) + }) + }) + + Describe("AddTracks", func() { + var mockTracks *tests.MockPlaylistTrackRepo + + BeforeEach(func() { + mockTracks = &tests.MockPlaylistTrackRepo{AddCount: 2} + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"}, + "pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1", + Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}}, + "pls-other": {ID: "pls-other", Name: "Other's", OwnerID: "other-user"}, + } + mockPlsRepo.TracksRepo = mockTracks + ps = playlists.NewPlaylists(ds) + }) + + It("allows owner to add tracks", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + count, err := ps.AddTracks(ctx, "pls-1", []string{"song-1", "song-2"}) + Expect(err).ToNot(HaveOccurred()) + Expect(count).To(Equal(2)) + Expect(mockTracks.AddedIds).To(ConsistOf("song-1", "song-2")) + }) + + It("allows admin to add tracks to any playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true}) + count, err := ps.AddTracks(ctx, "pls-other", []string{"song-1"}) + Expect(err).ToNot(HaveOccurred()) + Expect(count).To(Equal(2)) + }) + + It("denies non-owner, non-admin", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + _, err := ps.AddTracks(ctx, "pls-1", []string{"song-1"}) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("denies editing smart playlists", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + _, err := ps.AddTracks(ctx, "pls-smart", []string{"song-1"}) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("returns error when playlist not found", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + _, err := ps.AddTracks(ctx, "nonexistent", []string{"song-1"}) + Expect(err).To(Equal(model.ErrNotFound)) + }) + }) + + Describe("RemoveTracks", func() { + var mockTracks *tests.MockPlaylistTrackRepo + + BeforeEach(func() { + mockTracks = &tests.MockPlaylistTrackRepo{} + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"}, + "pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1", + Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}}, + } + mockPlsRepo.TracksRepo = mockTracks + ps = playlists.NewPlaylists(ds) + }) + + It("allows owner to remove tracks", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.RemoveTracks(ctx, "pls-1", []string{"track-1", "track-2"}) + Expect(err).ToNot(HaveOccurred()) + Expect(mockTracks.DeletedIds).To(ConsistOf("track-1", "track-2")) + }) + + It("denies on smart playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.RemoveTracks(ctx, "pls-smart", []string{"track-1"}) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("denies non-owner", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + err := ps.RemoveTracks(ctx, "pls-1", []string{"track-1"}) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + }) + + Describe("ReorderTrack", func() { + var mockTracks *tests.MockPlaylistTrackRepo + + BeforeEach(func() { + mockTracks = &tests.MockPlaylistTrackRepo{} + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"}, + "pls-smart": {ID: "pls-smart", Name: "Smart", OwnerID: "user-1", + Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}}, + } + mockPlsRepo.TracksRepo = mockTracks + ps = playlists.NewPlaylists(ds) + }) + + It("allows owner to reorder", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.ReorderTrack(ctx, "pls-1", 1, 3) + Expect(err).ToNot(HaveOccurred()) + Expect(mockTracks.Reordered).To(BeTrue()) + }) + + It("denies on smart playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.ReorderTrack(ctx, "pls-smart", 1, 3) + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + }) +}) diff --git a/core/playlists/rest_adapter.go b/core/playlists/rest_adapter.go new file mode 100644 index 000000000..3865d97e9 --- /dev/null +++ b/core/playlists/rest_adapter.go @@ -0,0 +1,95 @@ +package playlists + +import ( + "context" + "errors" + + "github.com/deluan/rest" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" +) + +// --- REST adapter (follows Share/Library pattern) --- + +func (s *playlists) NewRepository(ctx context.Context) rest.Repository { + return &playlistRepositoryWrapper{ + ctx: ctx, + PlaylistRepository: s.ds.Playlist(ctx), + service: s, + } +} + +// playlistRepositoryWrapper wraps the playlist repository as a thin REST-to-service adapter. +// It satisfies rest.Repository through the embedded PlaylistRepository (via ResourceRepository), +// and rest.Persistable by delegating to service methods for all mutations. +type playlistRepositoryWrapper struct { + model.PlaylistRepository + ctx context.Context + service *playlists +} + +func (r *playlistRepositoryWrapper) Save(entity any) (string, error) { + return r.service.savePlaylist(r.ctx, entity.(*model.Playlist)) +} + +func (r *playlistRepositoryWrapper) Update(id string, entity any, cols ...string) error { + return r.service.updatePlaylistEntity(r.ctx, id, entity.(*model.Playlist), cols...) +} + +func (r *playlistRepositoryWrapper) Delete(id string) error { + err := r.service.Delete(r.ctx, id) + switch { + case errors.Is(err, model.ErrNotFound): + return rest.ErrNotFound + case errors.Is(err, model.ErrNotAuthorized): + return rest.ErrPermissionDenied + default: + return err + } +} + +func (s *playlists) TracksRepository(ctx context.Context, playlistId string, refreshSmartPlaylist bool) rest.Repository { + repo := s.ds.Playlist(ctx) + tracks := repo.Tracks(playlistId, refreshSmartPlaylist) + if tracks == nil { + return nil + } + return tracks.(rest.Repository) +} + +// savePlaylist creates a new playlist, assigning the owner from context. +func (s *playlists) savePlaylist(ctx context.Context, pls *model.Playlist) (string, error) { + usr, _ := request.UserFrom(ctx) + pls.OwnerID = usr.ID + pls.ID = "" // Force new creation + err := s.ds.Playlist(ctx).Put(pls) + if err != nil { + return "", err + } + return pls.ID, nil +} + +// updatePlaylistEntity updates playlist metadata with permission checks. +// Used by the REST API wrapper. +func (s *playlists) updatePlaylistEntity(ctx context.Context, id string, entity *model.Playlist, cols ...string) error { + current, err := s.checkWritable(ctx, id) + if err != nil { + switch { + case errors.Is(err, model.ErrNotFound): + return rest.ErrNotFound + case errors.Is(err, model.ErrNotAuthorized): + return rest.ErrPermissionDenied + default: + return err + } + } + usr, _ := request.UserFrom(ctx) + if !usr.IsAdmin && entity.OwnerID != "" && entity.OwnerID != current.OwnerID { + return rest.ErrPermissionDenied + } + // Apply ownership change (admin only) + if entity.OwnerID != "" { + current.OwnerID = entity.OwnerID + } + return s.updateMetadata(ctx, s.ds, current, &entity.Name, &entity.Comment, &entity.Public) +} diff --git a/core/playlists/rest_adapter_test.go b/core/playlists/rest_adapter_test.go new file mode 100644 index 000000000..b65095957 --- /dev/null +++ b/core/playlists/rest_adapter_test.go @@ -0,0 +1,120 @@ +package playlists_test + +import ( + "context" + + "github.com/deluan/rest" + "github.com/navidrome/navidrome/core/playlists" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("REST Adapter", func() { + var ds *tests.MockDataStore + var ps playlists.Playlists + var mockPlsRepo *tests.MockPlaylistRepo + ctx := context.Background() + + BeforeEach(func() { + mockPlsRepo = tests.CreateMockPlaylistRepo() + ds = &tests.MockDataStore{ + MockedPlaylist: mockPlsRepo, + MockedLibrary: &tests.MockLibraryRepo{}, + } + ctx = request.WithUser(ctx, model.User{ID: "123"}) + }) + + Describe("NewRepository", func() { + var repo rest.Persistable + + BeforeEach(func() { + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"}, + } + ps = playlists.NewPlaylists(ds) + }) + + Describe("Save", func() { + It("sets the owner from the context user", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + pls := &model.Playlist{Name: "New Playlist"} + id, err := repo.Save(pls) + Expect(err).ToNot(HaveOccurred()) + Expect(id).ToNot(BeEmpty()) + Expect(pls.OwnerID).To(Equal("user-1")) + }) + + It("forces a new creation by clearing ID", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + pls := &model.Playlist{ID: "should-be-cleared", Name: "New"} + _, err := repo.Save(pls) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ID).ToNot(Equal("should-be-cleared")) + }) + }) + + Describe("Update", func() { + It("allows owner to update their playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + pls := &model.Playlist{Name: "Updated"} + err := repo.Update("pls-1", pls) + Expect(err).ToNot(HaveOccurred()) + }) + + It("allows admin to update any playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true}) + repo = ps.NewRepository(ctx).(rest.Persistable) + pls := &model.Playlist{Name: "Updated"} + err := repo.Update("pls-1", pls) + Expect(err).ToNot(HaveOccurred()) + }) + + It("denies non-owner, non-admin", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + pls := &model.Playlist{Name: "Updated"} + err := repo.Update("pls-1", pls) + Expect(err).To(Equal(rest.ErrPermissionDenied)) + }) + + It("denies regular user from changing ownership", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + pls := &model.Playlist{Name: "Updated", OwnerID: "other-user"} + err := repo.Update("pls-1", pls) + Expect(err).To(Equal(rest.ErrPermissionDenied)) + }) + + It("returns rest.ErrNotFound when playlist doesn't exist", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + pls := &model.Playlist{Name: "Updated"} + err := repo.Update("nonexistent", pls) + Expect(err).To(Equal(rest.ErrNotFound)) + }) + }) + + Describe("Delete", func() { + It("delegates to service Delete with permission checks", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + err := repo.Delete("pls-1") + Expect(err).ToNot(HaveOccurred()) + Expect(mockPlsRepo.Deleted).To(ContainElement("pls-1")) + }) + + It("denies non-owner", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + err := repo.Delete("pls-1") + Expect(err).To(Equal(rest.ErrPermissionDenied)) + }) + }) + }) +}) diff --git a/core/wire_providers.go b/core/wire_providers.go index a8b1fde03..503feb789 100644 --- a/core/wire_providers.go +++ b/core/wire_providers.go @@ -7,6 +7,7 @@ import ( "github.com/navidrome/navidrome/core/ffmpeg" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" ) @@ -16,7 +17,7 @@ var Set = wire.NewSet( NewArchiver, NewPlayers, NewShare, - NewPlaylists, + playlists.NewPlaylists, NewLibrary, NewUser, NewMaintenance, diff --git a/persistence/playlist_repository.go b/persistence/playlist_repository.go index 220c1210b..955a13bc9 100644 --- a/persistence/playlist_repository.go +++ b/persistence/playlist_repository.go @@ -96,16 +96,6 @@ func (r *playlistRepository) Exists(id string) (bool, error) { } func (r *playlistRepository) Delete(id string) error { - usr := loggedUser(r.ctx) - if !usr.IsAdmin { - pls, err := r.Get(id) - if err != nil { - return err - } - if pls.OwnerID != usr.ID { - return rest.ErrPermissionDenied - } - } return r.delete(And{Eq{"id": id}, r.userFilter()}) } @@ -113,14 +103,6 @@ func (r *playlistRepository) Put(p *model.Playlist) error { pls := dbPlaylist{Playlist: *p} if pls.ID == "" { pls.CreatedAt = time.Now() - } else { - ok, err := r.Exists(pls.ID) - if err != nil { - return err - } - if !ok { - return model.ErrNotAuthorized - } } pls.UpdatedAt = time.Now() @@ -132,7 +114,6 @@ func (r *playlistRepository) Put(p *model.Playlist) error { if p.IsSmartPlaylist() { // Do not update tracks at this point, as it may take a long time and lock the DB, breaking the scan process - //r.refreshSmartPlaylist(p) return nil } // Only update tracks if they were specified @@ -320,10 +301,6 @@ func (r *playlistRepository) updateTracks(id string, tracks model.MediaFiles) er } func (r *playlistRepository) updatePlaylist(playlistId string, mediaFileIds []string) error { - if !r.isWritable(playlistId) { - return rest.ErrPermissionDenied - } - // Remove old tracks del := Delete("playlist_tracks").Where(Eq{"playlist_id": playlistId}) _, err := r.executeSQL(del) @@ -439,8 +416,7 @@ func (r *playlistRepository) NewInstance() any { func (r *playlistRepository) Save(entity any) (string, error) { pls := entity.(*model.Playlist) - pls.OwnerID = loggedUser(r.ctx).ID - pls.ID = "" // Make sure we don't override an existing playlist + pls.ID = "" // Force new creation err := r.Put(pls) if err != nil { return "", err @@ -450,24 +426,9 @@ func (r *playlistRepository) Save(entity any) (string, error) { func (r *playlistRepository) Update(id string, entity any, cols ...string) error { pls := dbPlaylist{Playlist: *entity.(*model.Playlist)} - current, err := r.Get(id) - if err != nil { - return err - } - usr := loggedUser(r.ctx) - if !usr.IsAdmin { - // Only the owner can update the playlist - if current.OwnerID != usr.ID { - return rest.ErrPermissionDenied - } - // Regular users can't change the ownership of a playlist - if pls.OwnerID != "" && pls.OwnerID != usr.ID { - return rest.ErrPermissionDenied - } - } pls.ID = id pls.UpdatedAt = time.Now() - _, err = r.put(id, pls, append(cols, "updatedAt")...) + _, err := r.put(id, pls, append(cols, "updatedAt")...) if errors.Is(err, model.ErrNotFound) { return rest.ErrNotFound } @@ -507,23 +468,31 @@ func (r *playlistRepository) removeOrphans() error { return nil } +// renumber updates the position of all tracks in the playlist to be sequential starting from 1, ordered by their +// current position. This is needed after removing orphan tracks, to ensure there are no gaps in the track numbering. +// The two-step approach (negate then reassign via CTE) avoids UNIQUE constraint violations on (playlist_id, id). func (r *playlistRepository) renumber(id string) error { - var ids []string - sq := Select("media_file_id").From("playlist_tracks").Where(Eq{"playlist_id": id}).OrderBy("id") - err := r.queryAllSlice(sq, &ids) + // Step 1: Negate all IDs to clear the positive ID space + _, err := r.executeSQL(Expr( + `UPDATE playlist_tracks SET id = -id WHERE playlist_id = ? AND id > 0`, id)) if err != nil { return err } - return r.updatePlaylist(id, ids) -} - -func (r *playlistRepository) isWritable(playlistId string) bool { - usr := loggedUser(r.ctx) - if usr.IsAdmin { - return true + // Step 2: Assign new sequential positive IDs using UPDATE...FROM with a CTE. + // The CTE is fully materialized before the UPDATE begins, avoiding self-referencing issues. + // ORDER BY id DESC restores original order since IDs are now negative. + _, err = r.executeSQL(Expr( + `WITH new_ids AS ( + SELECT rowid as rid, ROW_NUMBER() OVER (ORDER BY id DESC) as new_id + FROM playlist_tracks WHERE playlist_id = ? + ) + UPDATE playlist_tracks SET id = new_ids.new_id + FROM new_ids + WHERE playlist_tracks.rowid = new_ids.rid AND playlist_tracks.playlist_id = ?`, id, id)) + if err != nil { + return err } - pls, err := r.Get(playlistId) - return err == nil && pls.OwnerID == usr.ID + return r.refreshCounters(&model.Playlist{ID: id}) } var _ model.PlaylistRepository = (*playlistRepository)(nil) diff --git a/persistence/playlist_repository_test.go b/persistence/playlist_repository_test.go index 232eb14b4..5230390fb 100644 --- a/persistence/playlist_repository_test.go +++ b/persistence/playlist_repository_test.go @@ -401,6 +401,79 @@ var _ = Describe("PlaylistRepository", func() { }) }) + Describe("Track Deletion and Renumbering", func() { + var testPlaylistID string + + AfterEach(func() { + if testPlaylistID != "" { + Expect(repo.Delete(testPlaylistID)).To(BeNil()) + testPlaylistID = "" + } + }) + + // helper to get track positions and media file IDs + getTrackInfo := func(playlistID string) (ids []string, mediaFileIDs []string) { + pls, err := repo.GetWithTracks(playlistID, false, false) + Expect(err).ToNot(HaveOccurred()) + for _, t := range pls.Tracks { + ids = append(ids, t.ID) + mediaFileIDs = append(mediaFileIDs, t.MediaFileID) + } + return + } + + It("renumbers correctly after deleting a track from the middle", func() { + By("creating a playlist with 4 tracks") + newPls := model.Playlist{Name: "Renumber Test Middle", OwnerID: "userid"} + newPls.AddMediaFilesByID([]string{"1001", "1002", "1003", "1004"}) + Expect(repo.Put(&newPls)).To(Succeed()) + testPlaylistID = newPls.ID + + By("deleting the second track (position 2)") + tracksRepo := repo.Tracks(newPls.ID, false) + Expect(tracksRepo.Delete("2")).To(Succeed()) + + By("verifying remaining tracks are renumbered sequentially") + ids, mediaFileIDs := getTrackInfo(newPls.ID) + Expect(ids).To(Equal([]string{"1", "2", "3"})) + Expect(mediaFileIDs).To(Equal([]string{"1001", "1003", "1004"})) + }) + + It("renumbers correctly after deleting the first track", func() { + By("creating a playlist with 3 tracks") + newPls := model.Playlist{Name: "Renumber Test First", OwnerID: "userid"} + newPls.AddMediaFilesByID([]string{"1001", "1002", "1003"}) + Expect(repo.Put(&newPls)).To(Succeed()) + testPlaylistID = newPls.ID + + By("deleting the first track (position 1)") + tracksRepo := repo.Tracks(newPls.ID, false) + Expect(tracksRepo.Delete("1")).To(Succeed()) + + By("verifying remaining tracks are renumbered sequentially") + ids, mediaFileIDs := getTrackInfo(newPls.ID) + Expect(ids).To(Equal([]string{"1", "2"})) + Expect(mediaFileIDs).To(Equal([]string{"1002", "1003"})) + }) + + It("renumbers correctly after deleting the last track", func() { + By("creating a playlist with 3 tracks") + newPls := model.Playlist{Name: "Renumber Test Last", OwnerID: "userid"} + newPls.AddMediaFilesByID([]string{"1001", "1002", "1003"}) + Expect(repo.Put(&newPls)).To(Succeed()) + testPlaylistID = newPls.ID + + By("deleting the last track (position 3)") + tracksRepo := repo.Tracks(newPls.ID, false) + Expect(tracksRepo.Delete("3")).To(Succeed()) + + By("verifying remaining tracks are renumbered sequentially") + ids, mediaFileIDs := getTrackInfo(newPls.ID) + Expect(ids).To(Equal([]string{"1", "2"})) + Expect(mediaFileIDs).To(Equal([]string{"1001", "1002"})) + }) + }) + Describe("Smart Playlists Library Filtering", func() { var mfRepo model.MediaFileRepository var testPlaylistID string diff --git a/persistence/playlist_track_repository.go b/persistence/playlist_track_repository.go index c72abb180..1a7062cc2 100644 --- a/persistence/playlist_track_repository.go +++ b/persistence/playlist_track_repository.go @@ -140,15 +140,7 @@ func (r *playlistTrackRepository) NewInstance() any { return &model.PlaylistTrack{} } -func (r *playlistTrackRepository) isTracksEditable() bool { - return r.playlistRepo.isWritable(r.playlistId) && !r.playlist.IsSmartPlaylist() -} - func (r *playlistTrackRepository) Add(mediaFileIds []string) (int, error) { - if !r.isTracksEditable() { - return 0, rest.ErrPermissionDenied - } - if len(mediaFileIds) > 0 { log.Debug(r.ctx, "Adding songs to playlist", "playlistId", r.playlistId, "mediaFileIds", mediaFileIds) } else { @@ -196,22 +188,7 @@ func (r *playlistTrackRepository) AddDiscs(discs []model.DiscID) (int, error) { return r.addMediaFileIds(clauses) } -// Get ids from all current tracks -func (r *playlistTrackRepository) getTracks() ([]string, error) { - all := r.newSelect().Columns("media_file_id").Where(Eq{"playlist_id": r.playlistId}).OrderBy("id") - var ids []string - err := r.queryAllSlice(all, &ids) - if err != nil { - log.Error(r.ctx, "Error querying current tracks from playlist", "playlistId", r.playlistId, err) - return nil, err - } - return ids, nil -} - func (r *playlistTrackRepository) Delete(ids ...string) error { - if !r.isTracksEditable() { - return rest.ErrPermissionDenied - } err := r.delete(And{Eq{"playlist_id": r.playlistId}, Eq{"id": ids}}) if err != nil { return err @@ -221,9 +198,6 @@ func (r *playlistTrackRepository) Delete(ids ...string) error { } func (r *playlistTrackRepository) DeleteAll() error { - if !r.isTracksEditable() { - return rest.ErrPermissionDenied - } err := r.delete(Eq{"playlist_id": r.playlistId}) if err != nil { return err @@ -232,16 +206,45 @@ func (r *playlistTrackRepository) DeleteAll() error { return r.playlistRepo.renumber(r.playlistId) } +// Reorder moves a track from pos to newPos, shifting other tracks accordingly. func (r *playlistTrackRepository) Reorder(pos int, newPos int) error { - if !r.isTracksEditable() { - return rest.ErrPermissionDenied + if pos == newPos { + return nil } - ids, err := r.getTracks() + pid := r.playlistId + + // Step 1: Move the source track out of the way (temporary sentinel value) + _, err := r.executeSQL(Expr( + `UPDATE playlist_tracks SET id = -999999 WHERE playlist_id = ? AND id = ?`, pid, pos)) if err != nil { return err } - newOrder := slice.Move(ids, pos-1, newPos-1) - return r.playlistRepo.updatePlaylist(r.playlistId, newOrder) + + // Step 2: Shift the affected range using negative values to avoid unique constraint violations + if pos < newPos { + _, err = r.executeSQL(Expr( + `UPDATE playlist_tracks SET id = -(id - 1) WHERE playlist_id = ? AND id > ? AND id <= ?`, + pid, pos, newPos)) + } else { + _, err = r.executeSQL(Expr( + `UPDATE playlist_tracks SET id = -(id + 1) WHERE playlist_id = ? AND id >= ? AND id < ?`, + pid, newPos, pos)) + } + if err != nil { + return err + } + + // Step 3: Flip the shifted range back to positive + _, err = r.executeSQL(Expr( + `UPDATE playlist_tracks SET id = -id WHERE playlist_id = ? AND id < 0 AND id != -999999`, pid)) + if err != nil { + return err + } + + // Step 4: Place the source track at its new position + _, err = r.executeSQL(Expr( + `UPDATE playlist_tracks SET id = ? WHERE playlist_id = ? AND id = -999999`, newPos, pid)) + return err } var _ model.PlaylistTrackRepository = (*playlistTrackRepository)(nil) diff --git a/scanner/controller.go b/scanner/controller.go index db6444fa9..94248ffd0 100644 --- a/scanner/controller.go +++ b/scanner/controller.go @@ -9,10 +9,10 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/consts" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/auth" "github.com/navidrome/navidrome/core/metrics" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/request" @@ -27,7 +27,7 @@ var ( ) func New(rootCtx context.Context, ds model.DataStore, cw artwork.CacheWarmer, broker events.Broker, - pls core.Playlists, m metrics.Metrics) model.Scanner { + pls playlists.Playlists, m metrics.Metrics) model.Scanner { c := &controller{ rootCtx: rootCtx, ds: ds, @@ -53,7 +53,7 @@ func (s *controller) getScanner() scanner { // CallScan starts an in-process scan of specific library/folder pairs. // If targets is empty, it scans all libraries. // This is meant to be called from the command line (see cmd/scan.go). -func CallScan(ctx context.Context, ds model.DataStore, pls core.Playlists, fullScan bool, targets []model.ScanTarget) (<-chan *ProgressInfo, error) { +func CallScan(ctx context.Context, ds model.DataStore, pls playlists.Playlists, fullScan bool, targets []model.ScanTarget) (<-chan *ProgressInfo, error) { release, err := lockScan(ctx) if err != nil { return nil, err @@ -98,7 +98,7 @@ type controller struct { cw artwork.CacheWarmer broker events.Broker metrics metrics.Metrics - pls core.Playlists + pls playlists.Playlists limiter *rate.Sometimes devExternalScanner bool count atomic.Uint32 diff --git a/scanner/controller_test.go b/scanner/controller_test.go index f5ccabc86..2af52066b 100644 --- a/scanner/controller_test.go +++ b/scanner/controller_test.go @@ -5,9 +5,9 @@ import ( "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/consts" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/metrics" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/persistence" @@ -31,7 +31,7 @@ var _ = Describe("Controller", func() { DeferCleanup(configtest.SetupConfig()) ds = &tests.MockDataStore{RealDS: persistence.New(db.Db())} ds.MockedProperty = &tests.MockedPropertyRepo{} - ctrl = scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), core.NewPlaylists(ds), metrics.NewNoopInstance()) + ctrl = scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), playlists.NewPlaylists(ds), metrics.NewNoopInstance()) }) It("includes last scan error", func() { diff --git a/scanner/folder_entry.go b/scanner/folder_entry.go index 9d8d0c571..c7cc88ee1 100644 --- a/scanner/folder_entry.go +++ b/scanner/folder_entry.go @@ -10,7 +10,7 @@ import ( "slices" "time" - "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/chrono" ) @@ -72,7 +72,7 @@ func (f *folderEntry) isOutdated() bool { func (f *folderEntry) toFolder() *model.Folder { folder := model.NewFolder(f.job.lib, f.path) folder.NumAudioFiles = len(f.audioFiles) - if core.InPlaylistsPath(*folder) { + if playlists.InPath(*folder) { folder.NumPlaylists = f.numPlaylists } folder.ImageFiles = slices.Collect(maps.Keys(f.imageFiles)) diff --git a/scanner/phase_4_playlists.go b/scanner/phase_4_playlists.go index c98b51ee6..ab5f77ae0 100644 --- a/scanner/phase_4_playlists.go +++ b/scanner/phase_4_playlists.go @@ -10,8 +10,8 @@ import ( ppl "github.com/google/go-pipeline/pkg/pipeline" "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/request" @@ -21,12 +21,12 @@ type phasePlaylists struct { ctx context.Context scanState *scanState ds model.DataStore - pls core.Playlists + pls playlists.Playlists cw artwork.CacheWarmer refreshed atomic.Uint32 } -func createPhasePlaylists(ctx context.Context, scanState *scanState, ds model.DataStore, pls core.Playlists, cw artwork.CacheWarmer) *phasePlaylists { +func createPhasePlaylists(ctx context.Context, scanState *scanState, ds model.DataStore, pls playlists.Playlists, cw artwork.CacheWarmer) *phasePlaylists { return &phasePlaylists{ ctx: ctx, scanState: scanState, diff --git a/scanner/phase_4_playlists_test.go b/scanner/phase_4_playlists_test.go index 218aa3c7b..0b50d39cb 100644 --- a/scanner/phase_4_playlists_test.go +++ b/scanner/phase_4_playlists_test.go @@ -9,8 +9,8 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/request" "github.com/navidrome/navidrome/tests" @@ -130,7 +130,7 @@ var _ = Describe("phasePlaylists", func() { type mockPlaylists struct { mock.Mock - core.Playlists + playlists.Playlists } func (p *mockPlaylists) ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) { diff --git a/scanner/scanner.go b/scanner/scanner.go index ba1e76ff2..871b0c696 100644 --- a/scanner/scanner.go +++ b/scanner/scanner.go @@ -11,8 +11,8 @@ import ( ppl "github.com/google/go-pipeline/pkg/pipeline" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/consts" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -23,7 +23,7 @@ import ( type scannerImpl struct { ds model.DataStore cw artwork.CacheWarmer - pls core.Playlists + pls playlists.Playlists } // scanState holds the state of an in-progress scan, to be passed to the various phases diff --git a/scanner/scanner_benchmark_test.go b/scanner/scanner_benchmark_test.go index 2b1c0a140..1ac7b50a4 100644 --- a/scanner/scanner_benchmark_test.go +++ b/scanner/scanner_benchmark_test.go @@ -12,9 +12,9 @@ import ( "github.com/dustin/go-humanize" "github.com/google/uuid" "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/metrics" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/storage/storagetest" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/model" @@ -40,7 +40,7 @@ func BenchmarkScan(b *testing.B) { ds := persistence.New(db.Db()) conf.Server.DevExternalScanner = false s := scanner.New(context.Background(), ds, artwork.NoopCacheWarmer(), events.NoopBroker(), - core.NewPlaylists(ds), metrics.NewNoopInstance()) + playlists.NewPlaylists(ds), metrics.NewNoopInstance()) fs := storagetest.FakeFS{} storagetest.Register("fake", &fs) diff --git a/scanner/scanner_multilibrary_test.go b/scanner/scanner_multilibrary_test.go index 107e66a99..6990f1984 100644 --- a/scanner/scanner_multilibrary_test.go +++ b/scanner/scanner_multilibrary_test.go @@ -11,9 +11,9 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/consts" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/metrics" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/storage/storagetest" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/log" @@ -77,7 +77,7 @@ var _ = Describe("Scanner - Multi-Library", Ordered, func() { Expect(ds.User(ctx).Put(&adminUser)).To(Succeed()) s = scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), - core.NewPlaylists(ds), metrics.NewNoopInstance()) + playlists.NewPlaylists(ds), metrics.NewNoopInstance()) // Create two test libraries (let DB auto-assign IDs) lib1 = model.Library{Name: "Rock Collection", Path: "rock:///music"} diff --git a/scanner/scanner_selective_test.go b/scanner/scanner_selective_test.go index 629826db4..6e4511179 100644 --- a/scanner/scanner_selective_test.go +++ b/scanner/scanner_selective_test.go @@ -8,9 +8,9 @@ import ( "github.com/Masterminds/squirrel" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/metrics" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/storage/storagetest" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/log" @@ -63,7 +63,7 @@ var _ = Describe("ScanFolders", Ordered, func() { Expect(ds.User(ctx).Put(&adminUser)).To(Succeed()) s = scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), - core.NewPlaylists(ds), metrics.NewNoopInstance()) + playlists.NewPlaylists(ds), metrics.NewNoopInstance()) lib = model.Library{ID: 1, Name: "Fake Library", Path: "fake:///music"} Expect(ds.Library(ctx).Put(&lib)).To(Succeed()) diff --git a/scanner/scanner_test.go b/scanner/scanner_test.go index 351255ae8..d5688a1dc 100644 --- a/scanner/scanner_test.go +++ b/scanner/scanner_test.go @@ -11,9 +11,9 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/consts" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/metrics" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/storage/storagetest" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/log" @@ -84,7 +84,7 @@ var _ = Describe("Scanner", Ordered, func() { Expect(ds.User(ctx).Put(&adminUser)).To(Succeed()) s = scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), - core.NewPlaylists(ds), metrics.NewNoopInstance()) + playlists.NewPlaylists(ds), metrics.NewNoopInstance()) lib = model.Library{ID: 1, Name: "Fake Library", Path: "fake:///music"} Expect(ds.Library(ctx).Put(&lib)).To(Succeed()) diff --git a/server/e2e/e2e_suite_test.go b/server/e2e/e2e_suite_test.go index 92214950a..7bda1527c 100644 --- a/server/e2e/e2e_suite_test.go +++ b/server/e2e/e2e_suite_test.go @@ -21,6 +21,7 @@ import ( "github.com/navidrome/navidrome/core/external" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" "github.com/navidrome/navidrome/core/storage/storagetest" "github.com/navidrome/navidrome/db" @@ -69,6 +70,14 @@ var ( Name: "Admin User", IsAdmin: true, } + + // Regular (non-admin) user for permission tests + regularUser = model.User{ + ID: "regular-1", + UserName: "regular", + Name: "Regular User", + IsAdmin: false, + } ) func createFS(files fstest.MapFS) storagetest.FakeFS { @@ -288,19 +297,29 @@ var _ = BeforeSuite(func() { adminUserWithPass.NewPassword = "password" Expect(initDS.User(ctx).Put(&adminUserWithPass)).To(Succeed()) + regularUserWithPass := regularUser + regularUserWithPass.NewPassword = "password" + Expect(initDS.User(ctx).Put(®ularUserWithPass)).To(Succeed()) + lib = model.Library{ID: 1, Name: "Music Library", Path: "fake:///music"} Expect(initDS.Library(ctx).Put(&lib)).To(Succeed()) Expect(initDS.User(ctx).SetUserLibraries(adminUser.ID, []int{lib.ID})).To(Succeed()) + Expect(initDS.User(ctx).SetUserLibraries(regularUser.ID, []int{lib.ID})).To(Succeed()) loadedUser, err := initDS.User(ctx).FindByUsername(adminUser.UserName) Expect(err).ToNot(HaveOccurred()) adminUser.Libraries = loadedUser.Libraries + + loadedRegular, err := initDS.User(ctx).FindByUsername(regularUser.UserName) + Expect(err).ToNot(HaveOccurred()) + regularUser.Libraries = loadedRegular.Libraries + ctx = request.WithUser(GinkgoT().Context(), adminUser) buildTestFS() s := scanner.New(ctx, initDS, artwork.NoopCacheWarmer(), events.NoopBroker(), - core.NewPlaylists(initDS), metrics.NewNoopInstance()) + playlists.NewPlaylists(initDS), metrics.NewNoopInstance()) _, err = s.ScanAll(ctx, true) Expect(err).ToNot(HaveOccurred()) @@ -334,7 +353,7 @@ func setupTestDB() { // Create the Subsonic Router with real DS + noop stubs s := scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), - core.NewPlaylists(ds), metrics.NewNoopInstance()) + playlists.NewPlaylists(ds), metrics.NewNoopInstance()) router = subsonic.New( ds, noopArtwork{}, @@ -344,7 +363,7 @@ func setupTestDB() { noopProvider{}, s, events.NoopBroker(), - core.NewPlaylists(ds), + playlists.NewPlaylists(ds), noopPlayTracker{}, core.NewShare(ds), playback.PlaybackServer(nil), diff --git a/server/e2e/subsonic_multilibrary_test.go b/server/e2e/subsonic_multilibrary_test.go index 2292bfab1..1eb04d41c 100644 --- a/server/e2e/subsonic_multilibrary_test.go +++ b/server/e2e/subsonic_multilibrary_test.go @@ -6,9 +6,9 @@ import ( "github.com/Masterminds/squirrel" "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/metrics" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/storage/storagetest" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/scanner" @@ -53,7 +53,7 @@ var _ = Describe("Multi-Library Support", Ordered, func() { // Run incremental scan to import lib2 content (lib1 files unchanged → skipped) s := scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), - core.NewPlaylists(ds), metrics.NewNoopInstance()) + playlists.NewPlaylists(ds), metrics.NewNoopInstance()) _, err = s.ScanAll(ctx, false) Expect(err).ToNot(HaveOccurred()) diff --git a/server/e2e/subsonic_playlists_test.go b/server/e2e/subsonic_playlists_test.go index 6e9c23765..3468979f4 100644 --- a/server/e2e/subsonic_playlists_test.go +++ b/server/e2e/subsonic_playlists_test.go @@ -1,7 +1,11 @@ package e2e import ( + "time" + + "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/criteria" "github.com/navidrome/navidrome/server/subsonic/responses" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" @@ -15,9 +19,9 @@ var _ = Describe("Playlist Endpoints", Ordered, func() { setupTestDB() // Look up song IDs from scanned data for playlist operations - songs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{Sort: "title", Max: 3}) + songs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{Sort: "title", Max: 6}) Expect(err).ToNot(HaveOccurred()) - Expect(len(songs)).To(BeNumerically(">=", 3)) + Expect(len(songs)).To(BeNumerically(">=", 5)) for _, s := range songs { songIDs = append(songIDs, s.ID) } @@ -32,24 +36,30 @@ var _ = Describe("Playlist Endpoints", Ordered, func() { }) It("createPlaylist creates a new playlist with songs", func() { - resp := doReq("createPlaylist", "name", "Test Playlist", "songId", songIDs[0], "songId", songIDs[1]) + resp := doReq("createPlaylist", "name", "Test Playlist", + "songId", songIDs[0], "songId", songIDs[1], "songId", songIDs[2]) Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.Playlist).ToNot(BeNil()) Expect(resp.Playlist.Name).To(Equal("Test Playlist")) - Expect(resp.Playlist.SongCount).To(Equal(int32(2))) + Expect(resp.Playlist.SongCount).To(Equal(int32(3))) + Expect(resp.Playlist.Entry).To(HaveLen(3)) + Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[0])) + Expect(resp.Playlist.Entry[1].Id).To(Equal(songIDs[1])) + Expect(resp.Playlist.Entry[2].Id).To(Equal(songIDs[2])) playlistID = resp.Playlist.Id }) - It("getPlaylist returns playlist with tracks", func() { + It("getPlaylist returns playlist with tracks in order", func() { resp := doReq("getPlaylist", "id", playlistID) Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.Playlist).ToNot(BeNil()) Expect(resp.Playlist.Name).To(Equal("Test Playlist")) - Expect(resp.Playlist.Entry).To(HaveLen(2)) + Expect(resp.Playlist.Entry).To(HaveLen(3)) Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[0])) Expect(resp.Playlist.Entry[1].Id).To(Equal(songIDs[1])) + Expect(resp.Playlist.Entry[2].Id).To(Equal(songIDs[2])) }) It("createPlaylist without name or playlistId returns error", func() { @@ -59,40 +69,150 @@ var _ = Describe("Playlist Endpoints", Ordered, func() { Expect(resp.Error).ToNot(BeNil()) }) + It("createPlaylist with playlistId replaces tracks on existing playlist", func() { + // Replace tracks: the playlist had [song0, song1, song2], replace with [song3, song4] + resp := doReq("createPlaylist", "playlistId", playlistID, + "songId", songIDs[3], "songId", songIDs[4]) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.Playlist).ToNot(BeNil()) + Expect(resp.Playlist.Id).To(Equal(playlistID)) + Expect(resp.Playlist.SongCount).To(Equal(int32(2))) + Expect(resp.Playlist.Entry).To(HaveLen(2)) + Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[3])) + Expect(resp.Playlist.Entry[1].Id).To(Equal(songIDs[4])) + }) + It("updatePlaylist can rename the playlist", func() { resp := doReq("updatePlaylist", "playlistId", playlistID, "name", "Renamed Playlist") - Expect(resp.Status).To(Equal(responses.StatusOK)) // Verify the rename resp = doReq("getPlaylist", "id", playlistID) - Expect(resp.Playlist.Name).To(Equal("Renamed Playlist")) + // Tracks should be unchanged + Expect(resp.Playlist.SongCount).To(Equal(int32(2))) + }) + + It("updatePlaylist can set comment", func() { + resp := doReq("updatePlaylist", "playlistId", playlistID, "comment", "My favorite songs") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.Comment).To(Equal("My favorite songs")) + }) + + It("updatePlaylist can set public visibility", func() { + resp := doReq("updatePlaylist", "playlistId", playlistID, "public", "true") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.Public).To(BeTrue()) }) It("updatePlaylist can add songs", func() { - resp := doReq("updatePlaylist", "playlistId", playlistID, "songIdToAdd", songIDs[2]) - + // Playlist currently has [song3, song4], add song0 + resp := doReq("updatePlaylist", "playlistId", playlistID, "songIdToAdd", songIDs[0]) Expect(resp.Status).To(Equal(responses.StatusOK)) - // Verify the song was added resp = doReq("getPlaylist", "id", playlistID) - Expect(resp.Playlist.SongCount).To(Equal(int32(3))) Expect(resp.Playlist.Entry).To(HaveLen(3)) + Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[3])) + Expect(resp.Playlist.Entry[1].Id).To(Equal(songIDs[4])) + Expect(resp.Playlist.Entry[2].Id).To(Equal(songIDs[0])) }) - It("updatePlaylist can remove songs by index", func() { - // Remove the first song (index 0) - resp := doReq("updatePlaylist", "playlistId", playlistID, "songIndexToRemove", "0") - + It("updatePlaylist can add multiple songs at once", func() { + // Playlist currently has [song3, song4, song0], add song1 and song2 + resp := doReq("updatePlaylist", "playlistId", playlistID, + "songIdToAdd", songIDs[1], "songIdToAdd", songIDs[2]) Expect(resp.Status).To(Equal(responses.StatusOK)) - // Verify the song was removed resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.SongCount).To(Equal(int32(5))) + Expect(resp.Playlist.Entry).To(HaveLen(5)) + }) + It("updatePlaylist can remove songs by index and verifies correct songs remain", func() { + // Playlist has [song3, song4, song0, song1, song2] + // Remove index 0 (song3) and index 2 (song0) + resp := doReq("updatePlaylist", "playlistId", playlistID, + "songIndexToRemove", "0", "songIndexToRemove", "2") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.SongCount).To(Equal(int32(3))) + Expect(resp.Playlist.Entry).To(HaveLen(3)) + Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[4])) + Expect(resp.Playlist.Entry[1].Id).To(Equal(songIDs[1])) + Expect(resp.Playlist.Entry[2].Id).To(Equal(songIDs[2])) + }) + + It("updatePlaylist can remove and add songs in a single call", func() { + // Playlist has [song4, song1, song2] + // Remove index 1 (song1) and add song3 + resp := doReq("updatePlaylist", "playlistId", playlistID, + "songIndexToRemove", "1", "songIdToAdd", songIDs[3]) + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.SongCount).To(Equal(int32(3))) + Expect(resp.Playlist.Entry).To(HaveLen(3)) + Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[4])) + Expect(resp.Playlist.Entry[1].Id).To(Equal(songIDs[2])) + Expect(resp.Playlist.Entry[2].Id).To(Equal(songIDs[3])) + }) + + It("updatePlaylist can combine metadata change with track removal", func() { + // Playlist has [song4, song2, song3] + // Rename + remove index 0 (song4) + resp := doReq("updatePlaylist", "playlistId", playlistID, + "name", "Final Playlist", "songIndexToRemove", "0") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.Name).To(Equal("Final Playlist")) Expect(resp.Playlist.SongCount).To(Equal(int32(2))) - Expect(resp.Playlist.Entry).To(HaveLen(2)) + Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[2])) + Expect(resp.Playlist.Entry[1].Id).To(Equal(songIDs[3])) + }) + + It("updatePlaylist can remove all songs from playlist", func() { + // Playlist has [song2, song3] — remove both + resp := doReq("updatePlaylist", "playlistId", playlistID, + "songIndexToRemove", "0", "songIndexToRemove", "1") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.SongCount).To(Equal(int32(0))) + Expect(resp.Playlist.Entry).To(BeEmpty()) + }) + + It("updatePlaylist can add songs to an empty playlist", func() { + resp := doReq("updatePlaylist", "playlistId", playlistID, + "songIdToAdd", songIDs[0]) + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", playlistID) + Expect(resp.Playlist.SongCount).To(Equal(int32(1))) + Expect(resp.Playlist.Entry).To(HaveLen(1)) + Expect(resp.Playlist.Entry[0].Id).To(Equal(songIDs[0])) + }) + + It("updatePlaylist without playlistId returns error", func() { + resp := doReq("updatePlaylist", "name", "No ID") + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + + It("getPlaylists shows the playlist", func() { + resp := doReq("getPlaylists") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.Playlists.Playlist).To(HaveLen(1)) + Expect(resp.Playlists.Playlist[0].Id).To(Equal(playlistID)) }) It("deletePlaylist removes the playlist", func() { @@ -107,4 +227,294 @@ var _ = Describe("Playlist Endpoints", Ordered, func() { Expect(resp.Status).To(Equal(responses.StatusFailed)) Expect(resp.Error).ToNot(BeNil()) }) + + It("getPlaylists returns empty after deletion", func() { + resp := doReq("getPlaylists") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.Playlists.Playlist).To(BeEmpty()) + }) + + Describe("Playlist Permissions", Ordered, func() { + var songIDs []string + var adminPrivateID string + var adminPublicID string + var regularPlaylistID string + + BeforeAll(func() { + setupTestDB() + + songs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{Sort: "title", Max: 6}) + Expect(err).ToNot(HaveOccurred()) + Expect(len(songs)).To(BeNumerically(">=", 3)) + for _, s := range songs { + songIDs = append(songIDs, s.ID) + } + }) + + It("admin creates a private playlist", func() { + resp := doReqWithUser(adminUser, "createPlaylist", "name", "Admin Private", + "songId", songIDs[0], "songId", songIDs[1]) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + adminPrivateID = resp.Playlist.Id + }) + + It("admin creates a public playlist", func() { + resp := doReqWithUser(adminUser, "createPlaylist", "name", "Admin Public", + "songId", songIDs[0], "songId", songIDs[1]) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + adminPublicID = resp.Playlist.Id + + // Make it public + resp = doReqWithUser(adminUser, "updatePlaylist", + "playlistId", adminPublicID, "public", "true") + Expect(resp.Status).To(Equal(responses.StatusOK)) + }) + + It("regular user creates a playlist", func() { + resp := doReqWithUser(regularUser, "createPlaylist", "name", "Regular Playlist", + "songId", songIDs[0]) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + regularPlaylistID = resp.Playlist.Id + }) + + // --- Private playlist: regular user gets "not found" (repo hides it entirely) --- + + It("regular user cannot see admin's private playlist", func() { + resp := doReqWithUser(regularUser, "getPlaylist", "id", adminPrivateID) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + + It("regular user cannot update admin's private playlist (not found)", func() { + resp := doReqWithUser(regularUser, "updatePlaylist", + "playlistId", adminPrivateID, "name", "Hacked") + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + + It("regular user cannot delete admin's private playlist (not found)", func() { + resp := doReqWithUser(regularUser, "deletePlaylist", "id", adminPrivateID) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + + // --- Public playlist: regular user can see but cannot modify (authorization fail, code 50) --- + + It("regular user can see admin's public playlist", func() { + resp := doReqWithUser(regularUser, "getPlaylist", "id", adminPublicID) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.Playlist.Name).To(Equal("Admin Public")) + }) + + It("regular user cannot update admin's public playlist", func() { + resp := doReqWithUser(regularUser, "updatePlaylist", + "playlistId", adminPublicID, "name", "Hacked") + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(int32(50))) + }) + + It("regular user cannot add songs to admin's public playlist", func() { + resp := doReqWithUser(regularUser, "updatePlaylist", + "playlistId", adminPublicID, "songIdToAdd", songIDs[2]) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error.Code).To(Equal(int32(50))) + }) + + It("regular user cannot remove songs from admin's public playlist", func() { + resp := doReqWithUser(regularUser, "updatePlaylist", + "playlistId", adminPublicID, "songIndexToRemove", "0") + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error.Code).To(Equal(int32(50))) + }) + + It("regular user cannot delete admin's public playlist", func() { + resp := doReqWithUser(regularUser, "deletePlaylist", "id", adminPublicID) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error.Code).To(Equal(int32(50))) + }) + + It("regular user cannot replace tracks on admin's public playlist via createPlaylist", func() { + resp := doReqWithUser(regularUser, "createPlaylist", + "playlistId", adminPublicID, "songId", songIDs[2]) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + + // --- Regular user can manage their own playlists --- + + It("regular user can update their own playlist", func() { + resp := doReqWithUser(regularUser, "updatePlaylist", + "playlistId", regularPlaylistID, "name", "My Updated Playlist") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReqWithUser(regularUser, "getPlaylist", "id", regularPlaylistID) + Expect(resp.Playlist.Name).To(Equal("My Updated Playlist")) + }) + + It("regular user can add songs to their own playlist", func() { + resp := doReqWithUser(regularUser, "updatePlaylist", + "playlistId", regularPlaylistID, "songIdToAdd", songIDs[1]) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReqWithUser(regularUser, "getPlaylist", "id", regularPlaylistID) + Expect(resp.Playlist.SongCount).To(Equal(int32(2))) + }) + + It("regular user can delete their own playlist", func() { + resp := doReqWithUser(regularUser, "deletePlaylist", "id", regularPlaylistID) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + }) + + // --- Admin can manage any user's playlists --- + + It("admin can update any user's playlist", func() { + resp := doReqWithUser(regularUser, "createPlaylist", "name", "To Be Admin-Edited", + "songId", songIDs[0]) + Expect(resp.Status).To(Equal(responses.StatusOK)) + plsID := resp.Playlist.Id + + resp = doReqWithUser(adminUser, "updatePlaylist", + "playlistId", plsID, "name", "Admin Edited") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReqWithUser(adminUser, "getPlaylist", "id", plsID) + Expect(resp.Playlist.Name).To(Equal("Admin Edited")) + }) + + It("admin can delete any user's playlist", func() { + resp := doReqWithUser(regularUser, "createPlaylist", "name", "To Be Admin-Deleted", + "songId", songIDs[0]) + Expect(resp.Status).To(Equal(responses.StatusOK)) + plsID := resp.Playlist.Id + + resp = doReqWithUser(adminUser, "deletePlaylist", "id", plsID) + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReqWithUser(adminUser, "getPlaylist", "id", plsID) + Expect(resp.Status).To(Equal(responses.StatusFailed)) + }) + + // --- Verify admin's playlists are unchanged --- + + It("admin's private playlist is unchanged after failed regular user operations", func() { + resp := doReqWithUser(adminUser, "getPlaylist", "id", adminPrivateID) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.Playlist.Name).To(Equal("Admin Private")) + Expect(resp.Playlist.SongCount).To(Equal(int32(2))) + }) + + It("admin's public playlist is unchanged after failed regular user operations", func() { + resp := doReqWithUser(adminUser, "getPlaylist", "id", adminPublicID) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.Playlist.Name).To(Equal("Admin Public")) + Expect(resp.Playlist.SongCount).To(Equal(int32(2))) + }) + }) + + Describe("Smart Playlist Protection", Ordered, func() { + var smartPlaylistID string + var songID string + + BeforeAll(func() { + setupTestDB() + + // Look up a song ID for mutation tests + songs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{Sort: "title", Max: 1}) + Expect(err).ToNot(HaveOccurred()) + Expect(songs).ToNot(BeEmpty()) + songID = songs[0].ID + + // Insert a smart playlist directly into the DB + smartPls := &model.Playlist{ + Name: "Smart Playlist", + OwnerID: adminUser.ID, + Public: false, + Rules: &criteria.Criteria{Expression: criteria.Contains{"title": ""}}, + } + Expect(ds.Playlist(ctx).Put(smartPls)).To(Succeed()) + smartPlaylistID = smartPls.ID + }) + + It("getPlaylist returns smart playlist with readonly flag and validUntil", func() { + resp := doReq("getPlaylist", "id", smartPlaylistID) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.Playlist.Name).To(Equal("Smart Playlist")) + Expect(resp.Playlist.OpenSubsonicPlaylist).ToNot(BeNil()) + Expect(resp.Playlist.OpenSubsonicPlaylist.Readonly).To(BeTrue()) + expectedValidUntil := time.Now().Add(conf.Server.SmartPlaylistRefreshDelay) + Expect(*resp.Playlist.OpenSubsonicPlaylist.ValidUntil).To(BeTemporally("~", expectedValidUntil, time.Second)) + }) + + It("createPlaylist rejects replacing tracks on smart playlist", func() { + resp := doReq("createPlaylist", "playlistId", smartPlaylistID, "songId", songID) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(int32(50))) + }) + + It("updatePlaylist rejects adding songs to smart playlist", func() { + resp := doReq("updatePlaylist", "playlistId", smartPlaylistID, + "songIdToAdd", songID) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(int32(50))) + }) + + It("updatePlaylist rejects removing songs from smart playlist", func() { + resp := doReq("updatePlaylist", "playlistId", smartPlaylistID, + "songIndexToRemove", "0") + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(int32(50))) + }) + + It("updatePlaylist allows renaming smart playlist", func() { + resp := doReq("updatePlaylist", "playlistId", smartPlaylistID, + "name", "Renamed Smart") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", smartPlaylistID) + Expect(resp.Playlist.Name).To(Equal("Renamed Smart")) + }) + + It("updatePlaylist allows setting comment on smart playlist", func() { + resp := doReq("updatePlaylist", "playlistId", smartPlaylistID, + "comment", "Auto-generated playlist") + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", smartPlaylistID) + Expect(resp.Playlist.Comment).To(Equal("Auto-generated playlist")) + }) + + It("deletePlaylist can delete smart playlist", func() { + resp := doReq("deletePlaylist", "id", smartPlaylistID) + Expect(resp.Status).To(Equal(responses.StatusOK)) + + resp = doReq("getPlaylist", "id", smartPlaylistID) + Expect(resp.Status).To(Equal(responses.StatusFailed)) + }) + }) }) diff --git a/server/e2e/subsonic_scan_test.go b/server/e2e/subsonic_scan_test.go index a6fb28bc4..8bac6fe96 100644 --- a/server/e2e/subsonic_scan_test.go +++ b/server/e2e/subsonic_scan_test.go @@ -22,8 +22,6 @@ var _ = Describe("Scan Endpoints", func() { }) It("startScan requires admin user", func() { - regularUser := createUser("user-2", "regular", "Regular User", false) - resp := doReqWithUser(regularUser, "startScan") Expect(resp.Status).To(Equal(responses.StatusFailed)) diff --git a/server/nativeapi/native_api.go b/server/nativeapi/native_api.go index 52e633bee..27b85a605 100644 --- a/server/nativeapi/native_api.go +++ b/server/nativeapi/native_api.go @@ -14,6 +14,7 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/metrics" + playlistsvc "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/request" @@ -37,7 +38,7 @@ type Router struct { http.Handler ds model.DataStore share core.Share - playlists core.Playlists + playlists playlistsvc.Playlists insights metrics.Insights libs core.Library users core.User @@ -45,7 +46,7 @@ type Router struct { pluginManager PluginManager } -func New(ds model.DataStore, share core.Share, playlists core.Playlists, insights metrics.Insights, libraryService core.Library, userService core.User, maintenance core.Maintenance, pluginManager PluginManager) *Router { +func New(ds model.DataStore, share core.Share, playlists playlistsvc.Playlists, insights metrics.Insights, libraryService core.Library, userService core.User, maintenance core.Maintenance, pluginManager PluginManager) *Router { r := &Router{ds: ds, share: share, playlists: playlists, insights: insights, libs: libraryService, users: userService, maintenance: maintenance, pluginManager: pluginManager} r.Handler = r.routes() return r @@ -121,7 +122,7 @@ func (api *Router) RX(r chi.Router, pathPrefix string, constructor rest.Reposito func (api *Router) addPlaylistRoute(r chi.Router) { constructor := func(ctx context.Context) rest.Repository { - return api.ds.Resource(ctx, model.Playlist{}) + return api.playlists.NewRepository(ctx) } r.Route("/playlist", func(r chi.Router) { @@ -146,26 +147,26 @@ func (api *Router) addPlaylistRoute(r chi.Router) { func (api *Router) addPlaylistTrackRoute(r chi.Router) { r.Route("/playlist/{playlistId}/tracks", func(r chi.Router) { r.Get("/", func(w http.ResponseWriter, r *http.Request) { - getPlaylist(api.ds)(w, r) + getPlaylist(api.playlists)(w, r) }) r.With(server.URLParamsMiddleware).Route("/", func(r chi.Router) { r.Delete("/", func(w http.ResponseWriter, r *http.Request) { - deleteFromPlaylist(api.ds)(w, r) + deleteFromPlaylist(api.playlists)(w, r) }) r.Post("/", func(w http.ResponseWriter, r *http.Request) { - addToPlaylist(api.ds)(w, r) + addToPlaylist(api.playlists)(w, r) }) }) r.Route("/{id}", func(r chi.Router) { r.Use(server.URLParamsMiddleware) r.Get("/", func(w http.ResponseWriter, r *http.Request) { - getPlaylistTrack(api.ds)(w, r) + getPlaylistTrack(api.playlists)(w, r) }) r.Put("/", func(w http.ResponseWriter, r *http.Request) { - reorderItem(api.ds)(w, r) + reorderItem(api.playlists)(w, r) }) r.Delete("/", func(w http.ResponseWriter, r *http.Request) { - deleteFromPlaylist(api.ds)(w, r) + deleteFromPlaylist(api.playlists)(w, r) }) }) }) @@ -173,7 +174,7 @@ func (api *Router) addPlaylistTrackRoute(r chi.Router) { func (api *Router) addSongPlaylistsRoute(r chi.Router) { r.With(server.URLParamsMiddleware).Get("/song/{id}/playlists", func(w http.ResponseWriter, r *http.Request) { - getSongPlaylists(api.ds)(w, r) + getSongPlaylists(api.playlists)(w, r) }) } diff --git a/server/nativeapi/playlists.go b/server/nativeapi/playlists.go index 1e2c5e07e..60e8024bd 100644 --- a/server/nativeapi/playlists.go +++ b/server/nativeapi/playlists.go @@ -11,7 +11,7 @@ import ( "github.com/deluan/rest" "github.com/go-chi/chi/v5" - "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/req" @@ -19,10 +19,10 @@ import ( type restHandler = func(rest.RepositoryConstructor, ...rest.Logger) http.HandlerFunc -func playlistTracksHandler(ds model.DataStore, handler restHandler, refreshSmartPlaylist func(*http.Request) bool) http.HandlerFunc { +func playlistTracksHandler(pls playlists.Playlists, handler restHandler, refreshSmartPlaylist func(*http.Request) bool) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { plsId := chi.URLParam(r, "playlistId") - tracks := ds.Playlist(r.Context()).Tracks(plsId, refreshSmartPlaylist(r)) + tracks := pls.TracksRepository(r.Context(), plsId, refreshSmartPlaylist(r)) if tracks == nil { http.Error(w, "not found", http.StatusNotFound) return @@ -31,27 +31,27 @@ func playlistTracksHandler(ds model.DataStore, handler restHandler, refreshSmart } } -func getPlaylist(ds model.DataStore) http.HandlerFunc { - handler := playlistTracksHandler(ds, rest.GetAll, func(r *http.Request) bool { +func getPlaylist(pls playlists.Playlists) http.HandlerFunc { + handler := playlistTracksHandler(pls, rest.GetAll, func(r *http.Request) bool { return req.Params(r).Int64Or("_start", 0) == 0 }) return func(w http.ResponseWriter, r *http.Request) { if strings.ToLower(r.Header.Get("accept")) == "audio/x-mpegurl" { - handleExportPlaylist(ds)(w, r) + handleExportPlaylist(pls)(w, r) return } handler(w, r) } } -func getPlaylistTrack(ds model.DataStore) http.HandlerFunc { - return playlistTracksHandler(ds, rest.Get, func(*http.Request) bool { return true }) +func getPlaylistTrack(pls playlists.Playlists) http.HandlerFunc { + return playlistTracksHandler(pls, rest.Get, func(*http.Request) bool { return true }) } -func createPlaylistFromM3U(playlists core.Playlists) http.HandlerFunc { +func createPlaylistFromM3U(pls playlists.Playlists) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { ctx := r.Context() - pls, err := playlists.ImportM3U(ctx, r.Body) + pl, err := pls.ImportM3U(ctx, r.Body) if err != nil { log.Error(r.Context(), "Error parsing playlist", err) // TODO: consider returning StatusBadRequest for playlists that are malformed @@ -59,7 +59,7 @@ func createPlaylistFromM3U(playlists core.Playlists) http.HandlerFunc { return } w.WriteHeader(http.StatusCreated) - _, err = w.Write([]byte(pls.ToM3U8())) //nolint:gosec + _, err = w.Write([]byte(pl.ToM3U8())) //nolint:gosec if err != nil { log.Error(ctx, "Error sending m3u contents", err) http.Error(w, err.Error(), http.StatusInternalServerError) @@ -68,45 +68,41 @@ func createPlaylistFromM3U(playlists core.Playlists) http.HandlerFunc { } } -func handleExportPlaylist(ds model.DataStore) http.HandlerFunc { +func handleExportPlaylist(pls playlists.Playlists) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { ctx := r.Context() - plsRepo := ds.Playlist(ctx) plsId := chi.URLParam(r, "playlistId") - pls, err := plsRepo.GetWithTracks(plsId, true, false) + playlist, err := pls.GetWithTracks(ctx, plsId) if errors.Is(err, model.ErrNotFound) { - log.Warn(r.Context(), "Playlist not found", "playlistId", plsId) + log.Warn(ctx, "Playlist not found", "playlistId", plsId) http.Error(w, "not found", http.StatusNotFound) return } if err != nil { - log.Error(r.Context(), "Error retrieving the playlist", "playlistId", plsId, err) + log.Error(ctx, "Error retrieving the playlist", "playlistId", plsId, err) http.Error(w, err.Error(), http.StatusInternalServerError) return } - log.Debug(ctx, "Exporting playlist as M3U", "playlistId", plsId, "name", pls.Name) + log.Debug(ctx, "Exporting playlist as M3U", "playlistId", plsId, "name", playlist.Name) w.Header().Set("Content-Type", "audio/x-mpegurl") - disposition := fmt.Sprintf("attachment; filename=\"%s.m3u\"", pls.Name) + disposition := fmt.Sprintf("attachment; filename=\"%s.m3u\"", playlist.Name) w.Header().Set("Content-Disposition", disposition) - _, err = w.Write([]byte(pls.ToM3U8())) //nolint:gosec + _, err = w.Write([]byte(playlist.ToM3U8())) //nolint:gosec if err != nil { - log.Error(ctx, "Error sending playlist", "name", pls.Name) + log.Error(ctx, "Error sending playlist", "name", playlist.Name) return } } } -func deleteFromPlaylist(ds model.DataStore) http.HandlerFunc { +func deleteFromPlaylist(pls playlists.Playlists) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { p := req.Params(r) playlistId, _ := p.String(":playlistId") ids, _ := p.Strings("id") - err := ds.WithTxImmediate(func(tx model.DataStore) error { - tracksRepo := tx.Playlist(r.Context()).Tracks(playlistId, true) - return tracksRepo.Delete(ids...) - }) + err := pls.RemoveTracks(r.Context(), playlistId, ids) if len(ids) == 1 && errors.Is(err, model.ErrNotFound) { log.Warn(r.Context(), "Track not found in playlist", "playlistId", playlistId, "id", ids[0]) http.Error(w, "not found", http.StatusNotFound) @@ -121,7 +117,7 @@ func deleteFromPlaylist(ds model.DataStore) http.HandlerFunc { } } -func addToPlaylist(ds model.DataStore) http.HandlerFunc { +func addToPlaylist(pls playlists.Playlists) http.HandlerFunc { type addTracksPayload struct { Ids []string `json:"ids"` AlbumIds []string `json:"albumIds"` @@ -130,6 +126,7 @@ func addToPlaylist(ds model.DataStore) http.HandlerFunc { } return func(w http.ResponseWriter, r *http.Request) { + ctx := r.Context() p := req.Params(r) playlistId, _ := p.String(":playlistId") var payload addTracksPayload @@ -138,24 +135,23 @@ func addToPlaylist(ds model.DataStore) http.HandlerFunc { http.Error(w, err.Error(), http.StatusBadRequest) return } - tracksRepo := ds.Playlist(r.Context()).Tracks(playlistId, true) count, c := 0, 0 - if c, err = tracksRepo.Add(payload.Ids); err != nil { + if c, err = pls.AddTracks(ctx, playlistId, payload.Ids); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } count += c - if c, err = tracksRepo.AddAlbums(payload.AlbumIds); err != nil { + if c, err = pls.AddAlbums(ctx, playlistId, payload.AlbumIds); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } count += c - if c, err = tracksRepo.AddArtists(payload.ArtistIds); err != nil { + if c, err = pls.AddArtists(ctx, playlistId, payload.ArtistIds); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } count += c - if c, err = tracksRepo.AddDiscs(payload.Discs); err != nil { + if c, err = pls.AddDiscs(ctx, playlistId, payload.Discs); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } @@ -169,12 +165,13 @@ func addToPlaylist(ds model.DataStore) http.HandlerFunc { } } -func reorderItem(ds model.DataStore) http.HandlerFunc { +func reorderItem(pls playlists.Playlists) http.HandlerFunc { type reorderPayload struct { InsertBefore string `json:"insert_before"` } return func(w http.ResponseWriter, r *http.Request) { + ctx := r.Context() p := req.Params(r) playlistId, _ := p.String(":playlistId") id := p.IntOr(":id", 0) @@ -193,9 +190,8 @@ func reorderItem(ds model.DataStore) http.HandlerFunc { http.Error(w, err.Error(), http.StatusBadRequest) return } - tracksRepo := ds.Playlist(r.Context()).Tracks(playlistId, true) - err = tracksRepo.Reorder(id, newPos) - if errors.Is(err, rest.ErrPermissionDenied) { + err = pls.ReorderTrack(ctx, playlistId, id, newPos) + if errors.Is(err, model.ErrNotAuthorized) { http.Error(w, err.Error(), http.StatusForbidden) return } @@ -211,11 +207,11 @@ func reorderItem(ds model.DataStore) http.HandlerFunc { } } -func getSongPlaylists(ds model.DataStore) http.HandlerFunc { +func getSongPlaylists(svc playlists.Playlists) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { p := req.Params(r) trackId, _ := p.String(":id") - playlists, err := ds.Playlist(r.Context()).GetPlaylists(trackId) + playlists, err := svc.GetPlaylists(r.Context(), trackId) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return diff --git a/server/nativeapi/playlists_test.go b/server/nativeapi/playlists_test.go index 319e41cd5..961d10b68 100644 --- a/server/nativeapi/playlists_test.go +++ b/server/nativeapi/playlists_test.go @@ -1,6 +1,7 @@ package nativeapi import ( + "context" "encoding/json" "net/http" "net/http/httptest" @@ -11,6 +12,7 @@ import ( "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/core/auth" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/server" "github.com/navidrome/navidrome/tests" @@ -48,11 +50,19 @@ func (m *mockPlaylistTrackRepo) Read(id string) (any, error) { return nil, rest.ErrNotFound } +type mockPlaylistsService struct { + playlists.Playlists + tracksRepo rest.Repository +} + +func (m *mockPlaylistsService) TracksRepository(_ context.Context, _ string, _ bool) rest.Repository { + return m.tracksRepo +} + var _ = Describe("Playlist Tracks Endpoint", func() { var ( router http.Handler - ds *tests.MockDataStore - plsRepo *tests.MockPlaylistRepo + plsSvc *mockPlaylistsService userRepo *tests.MockedUserRepo w *httptest.ResponseRecorder ) @@ -61,11 +71,10 @@ var _ = Describe("Playlist Tracks Endpoint", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.SessionTimeout = time.Minute - plsRepo = &tests.MockPlaylistRepo{} + plsSvc = &mockPlaylistsService{} userRepo = tests.CreateMockUserRepo() - ds = &tests.MockDataStore{ - MockedPlaylist: plsRepo, + ds := &tests.MockDataStore{ MockedUser: userRepo, MockedProperty: &tests.MockedPropertyRepo{}, } @@ -82,7 +91,7 @@ var _ = Describe("Playlist Tracks Endpoint", func() { err := userRepo.Put(&testUser) Expect(err).ToNot(HaveOccurred()) - nativeRouter := New(ds, nil, nil, nil, tests.NewMockLibraryService(), tests.NewMockUserService(), nil, nil) + nativeRouter := New(ds, nil, plsSvc, nil, tests.NewMockLibraryService(), tests.NewMockUserService(), nil, nil) router = server.JWTVerifier(nativeRouter) w = httptest.NewRecorder() }) @@ -105,7 +114,7 @@ var _ = Describe("Playlist Tracks Endpoint", func() { }) It("returns tracks when playlist exists", func() { - plsRepo.TracksReturn = &mockPlaylistTrackRepo{ + plsSvc.tracksRepo = &mockPlaylistTrackRepo{ tracks: model.PlaylistTracks{ {ID: "1", MediaFileID: "mf-1", PlaylistID: "pls-1"}, {ID: "2", MediaFileID: "mf-2", PlaylistID: "pls-1"}, @@ -135,7 +144,7 @@ var _ = Describe("Playlist Tracks Endpoint", func() { }) It("returns the track when playlist exists", func() { - plsRepo.TracksReturn = &mockPlaylistTrackRepo{ + plsSvc.tracksRepo = &mockPlaylistTrackRepo{ tracks: model.PlaylistTracks{ {ID: "1", MediaFileID: "mf-1", PlaylistID: "pls-1"}, }, @@ -154,7 +163,7 @@ var _ = Describe("Playlist Tracks Endpoint", func() { }) It("returns 404 when track does not exist in playlist", func() { - plsRepo.TracksReturn = &mockPlaylistTrackRepo{ + plsSvc.tracksRepo = &mockPlaylistTrackRepo{ tracks: model.PlaylistTracks{}, } diff --git a/server/subsonic/api.go b/server/subsonic/api.go index c3108ea5b..d0d9bb169 100644 --- a/server/subsonic/api.go +++ b/server/subsonic/api.go @@ -16,6 +16,7 @@ import ( "github.com/navidrome/navidrome/core/external" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" + playlistsvc "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -40,7 +41,7 @@ type Router struct { archiver core.Archiver players core.Players provider external.Provider - playlists core.Playlists + playlists playlistsvc.Playlists scanner model.Scanner broker events.Broker scrobbler scrobbler.PlayTracker @@ -51,7 +52,7 @@ type Router struct { func New(ds model.DataStore, artwork artwork.Artwork, streamer core.MediaStreamer, archiver core.Archiver, players core.Players, provider external.Provider, scanner model.Scanner, broker events.Broker, - playlists core.Playlists, scrobbler scrobbler.PlayTracker, share core.Share, playback playback.PlaybackServer, + playlists playlistsvc.Playlists, scrobbler scrobbler.PlayTracker, share core.Share, playback playback.PlaybackServer, metrics metrics.Metrics, ) *Router { r := &Router{ @@ -290,6 +291,8 @@ func mapToSubsonicError(err error) subError { err = newError(responses.ErrorGeneric, err.Error()) case errors.Is(err, model.ErrNotFound): err = newError(responses.ErrorDataNotFound, "data not found") + case errors.Is(err, model.ErrNotAuthorized): + err = newError(responses.ErrorAuthorizationFail) default: err = newError(responses.ErrorGeneric, fmt.Sprintf("Internal Server Error: %s", err)) } diff --git a/server/subsonic/playlists.go b/server/subsonic/playlists.go index b8807563e..baae7514b 100644 --- a/server/subsonic/playlists.go +++ b/server/subsonic/playlists.go @@ -19,7 +19,7 @@ import ( func (api *Router) GetPlaylists(r *http.Request) (*responses.Subsonic, error) { ctx := r.Context() - allPls, err := api.ds.Playlist(ctx).GetAll(model.QueryOptions{Sort: "name"}) + allPls, err := api.playlists.GetAll(ctx, model.QueryOptions{Sort: "name"}) if err != nil { log.Error(r, err) return nil, err @@ -42,7 +42,7 @@ func (api *Router) GetPlaylist(r *http.Request) (*responses.Subsonic, error) { } func (api *Router) getPlaylist(ctx context.Context, id string) (*responses.Subsonic, error) { - pls, err := api.ds.Playlist(ctx).GetWithTracks(id, true, false) + pls, err := api.playlists.GetWithTracks(ctx, id) if errors.Is(err, model.ErrNotFound) { log.Error(ctx, err.Error(), "id", id) return nil, newError(responses.ErrorDataNotFound, "playlist not found") @@ -60,34 +60,6 @@ func (api *Router) getPlaylist(ctx context.Context, id string) (*responses.Subso return response, nil } -func (api *Router) create(ctx context.Context, playlistId, name string, ids []string) (string, error) { - err := api.ds.WithTxImmediate(func(tx model.DataStore) error { - owner := getUser(ctx) - var pls *model.Playlist - var err error - - if playlistId != "" { - pls, err = tx.Playlist(ctx).Get(playlistId) - if err != nil { - return err - } - if owner.ID != pls.OwnerID { - return model.ErrNotAuthorized - } - } else { - pls = &model.Playlist{Name: name} - pls.OwnerID = owner.ID - } - pls.Tracks = nil - pls.AddMediaFilesByID(ids) - - err = tx.Playlist(ctx).Put(pls) - playlistId = pls.ID - return err - }) - return playlistId, err -} - func (api *Router) CreatePlaylist(r *http.Request) (*responses.Subsonic, error) { ctx := r.Context() p := req.Params(r) @@ -97,7 +69,7 @@ func (api *Router) CreatePlaylist(r *http.Request) (*responses.Subsonic, error) if playlistId == "" && name == "" { return nil, errors.New("required parameter name is missing") } - id, err := api.create(ctx, playlistId, name, songIds) + id, err := api.playlists.Create(ctx, playlistId, name, songIds) if err != nil { log.Error(r, err) return nil, err @@ -111,7 +83,7 @@ func (api *Router) DeletePlaylist(r *http.Request) (*responses.Subsonic, error) if err != nil { return nil, err } - err = api.ds.Playlist(r.Context()).Delete(id) + err = api.playlists.Delete(r.Context(), id) if errors.Is(err, model.ErrNotAuthorized) { return nil, newError(responses.ErrorAuthorizationFail) } diff --git a/server/subsonic/playlists_test.go b/server/subsonic/playlists_test.go index 05701fc1f..d99e244b0 100644 --- a/server/subsonic/playlists_test.go +++ b/server/subsonic/playlists_test.go @@ -5,7 +5,7 @@ import ( "time" "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/criteria" "github.com/navidrome/navidrome/model/request" @@ -14,7 +14,7 @@ import ( . "github.com/onsi/gomega" ) -var _ core.Playlists = (*fakePlaylists)(nil) +var _ playlists.Playlists = (*fakePlaylists)(nil) var _ = Describe("buildPlaylist", func() { var router *Router @@ -272,7 +272,7 @@ var _ = Describe("UpdatePlaylist", func() { }) type fakePlaylists struct { - core.Playlists + playlists.Playlists lastPlaylistID string lastName *string lastComment *string diff --git a/tests/mock_data_store.go b/tests/mock_data_store.go index c4b0113fc..754f0c084 100644 --- a/tests/mock_data_store.go +++ b/tests/mock_data_store.go @@ -121,7 +121,7 @@ func (db *MockDataStore) Playlist(ctx context.Context) model.PlaylistRepository if db.RealDS != nil { return db.RealDS.Playlist(ctx) } - db.MockedPlaylist = &MockPlaylistRepo{} + db.MockedPlaylist = CreateMockPlaylistRepo() return db.MockedPlaylist } diff --git a/tests/mock_playlist_repo.go b/tests/mock_playlist_repo.go index 1c37107e2..9bdc52152 100644 --- a/tests/mock_playlist_repo.go +++ b/tests/mock_playlist_repo.go @@ -1,38 +1,111 @@ package tests import ( + "errors" + "github.com/deluan/rest" "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/id" ) +func CreateMockPlaylistRepo() *MockPlaylistRepo { + return &MockPlaylistRepo{ + Data: make(map[string]*model.Playlist), + PathMap: make(map[string]*model.Playlist), + } +} + type MockPlaylistRepo struct { model.PlaylistRepository - - Entity *model.Playlist - Error error - TracksReturn model.PlaylistTrackRepository + Data map[string]*model.Playlist // keyed by ID + PathMap map[string]*model.Playlist // keyed by path + Last *model.Playlist + Deleted []string + Err bool + TracksRepo model.PlaylistTrackRepository } -func (m *MockPlaylistRepo) Get(_ string) (*model.Playlist, error) { - if m.Error != nil { - return nil, m.Error +func (m *MockPlaylistRepo) SetError(err bool) { + m.Err = err +} + +func (m *MockPlaylistRepo) Get(id string) (*model.Playlist, error) { + if m.Err { + return nil, errors.New("error") } - if m.Entity == nil { - return nil, model.ErrNotFound + if m.Data != nil { + if pls, ok := m.Data[id]; ok { + return pls, nil + } } - return m.Entity, nil + return nil, model.ErrNotFound +} + +func (m *MockPlaylistRepo) GetWithTracks(id string, _, _ bool) (*model.Playlist, error) { + return m.Get(id) +} + +func (m *MockPlaylistRepo) Put(pls *model.Playlist) error { + if m.Err { + return errors.New("error") + } + if pls.ID == "" { + pls.ID = id.NewRandom() + } + m.Last = pls + if m.Data != nil { + m.Data[pls.ID] = pls + } + return nil +} + +func (m *MockPlaylistRepo) FindByPath(path string) (*model.Playlist, error) { + if m.Err { + return nil, errors.New("error") + } + if m.PathMap != nil { + if pls, ok := m.PathMap[path]; ok { + return pls, nil + } + } + return nil, model.ErrNotFound +} + +func (m *MockPlaylistRepo) Delete(id string) error { + if m.Err { + return errors.New("error") + } + m.Deleted = append(m.Deleted, id) + return nil } func (m *MockPlaylistRepo) Tracks(_ string, _ bool) model.PlaylistTrackRepository { - return m.TracksReturn + return m.TracksRepo +} + +func (m *MockPlaylistRepo) Exists(id string) (bool, error) { + if m.Err { + return false, errors.New("error") + } + if m.Data != nil { + _, found := m.Data[id] + return found, nil + } + return false, nil } func (m *MockPlaylistRepo) Count(_ ...rest.QueryOptions) (int64, error) { - if m.Error != nil { - return 0, m.Error + if m.Err { + return 0, errors.New("error") } - if m.Entity == nil { - return 0, nil - } - return 1, nil + return int64(len(m.Data)), nil } + +func (m *MockPlaylistRepo) CountAll(_ ...model.QueryOptions) (int64, error) { + if m.Err { + return 0, errors.New("error") + } + return int64(len(m.Data)), nil +} + +var _ model.PlaylistRepository = (*MockPlaylistRepo)(nil) diff --git a/tests/mock_playlist_track_repo.go b/tests/mock_playlist_track_repo.go new file mode 100644 index 000000000..c11b077d2 --- /dev/null +++ b/tests/mock_playlist_track_repo.go @@ -0,0 +1,53 @@ +package tests + +import "github.com/navidrome/navidrome/model" + +type MockPlaylistTrackRepo struct { + model.PlaylistTrackRepository + AddedIds []string + DeletedIds []string + Reordered bool + AddCount int + Err error +} + +func (m *MockPlaylistTrackRepo) Add(ids []string) (int, error) { + m.AddedIds = append(m.AddedIds, ids...) + if m.Err != nil { + return 0, m.Err + } + return m.AddCount, nil +} + +func (m *MockPlaylistTrackRepo) AddAlbums(_ []string) (int, error) { + if m.Err != nil { + return 0, m.Err + } + return m.AddCount, nil +} + +func (m *MockPlaylistTrackRepo) AddArtists(_ []string) (int, error) { + if m.Err != nil { + return 0, m.Err + } + return m.AddCount, nil +} + +func (m *MockPlaylistTrackRepo) AddDiscs(_ []model.DiscID) (int, error) { + if m.Err != nil { + return 0, m.Err + } + return m.AddCount, nil +} + +func (m *MockPlaylistTrackRepo) Delete(ids ...string) error { + m.DeletedIds = append(m.DeletedIds, ids...) + return m.Err +} + +func (m *MockPlaylistTrackRepo) Reorder(_, _ int) error { + m.Reordered = true + return m.Err +} + +var _ model.PlaylistTrackRepository = (*MockPlaylistTrackRepo)(nil) From ec75808153e9478fb74b4df983df8f5ff7319b27 Mon Sep 17 00:00:00 2001 From: Deluan Date: Sat, 21 Feb 2026 22:00:00 -0500 Subject: [PATCH 05/60] fix(subsonic): handle empty quoted phrases in FTS5 query and search expression Signed-off-by: Deluan --- persistence/sql_search_fts.go | 4 ++-- persistence/sql_search_fts_test.go | 5 +++++ server/e2e/subsonic_searching_test.go | 10 ++++++++++ 3 files changed, 17 insertions(+), 2 deletions(-) diff --git a/persistence/sql_search_fts.go b/persistence/sql_search_fts.go index 25f16cb8d..ea70518b6 100644 --- a/persistence/sql_search_fts.go +++ b/persistence/sql_search_fts.go @@ -134,7 +134,7 @@ func isDottedAbbreviation(w string, subTokens []string) bool { // special characters to prevent query injection. func buildFTS5Query(userInput string) string { q := strings.TrimSpace(userInput) - if q == "" { + if q == "" || q == `""` { return "" } @@ -239,7 +239,7 @@ var ftsSearchColumns = map[string]string{ func ftsSearchExpr(tableName string, s string) Sqlizer { q := buildFTS5Query(s) if q == "" { - s = strings.TrimSpace(s) + s = strings.TrimSpace(strings.ReplaceAll(s, `"`, "")) if s != "" { log.Trace("Search using LIKE fallback for non-tokenizable query", "table", tableName, "query", s) return likeSearchExpr(tableName, s) diff --git a/persistence/sql_search_fts_test.go b/persistence/sql_search_fts_test.go index 317252950..e0fead8ae 100644 --- a/persistence/sql_search_fts_test.go +++ b/persistence/sql_search_fts_test.go @@ -49,6 +49,7 @@ var _ = DescribeTable("buildFTS5Query", Entry("preserves quoted abbreviation verbatim", `"R.E.M."`, `"R.E.M."`), Entry("returns empty string for punctuation-only input", "!!!!!!!", ""), Entry("returns empty string for mixed punctuation", "!@#$%^&", ""), + Entry("returns empty string for empty quoted phrase", `""`, ""), ) var _ = DescribeTable("normalizeForFTS", @@ -204,6 +205,10 @@ var _ = Describe("ftsSearchExpr", func() { Expect(ftsSearchExpr("media_file", "")).To(BeNil()) Expect(ftsSearchExpr("media_file", " ")).To(BeNil()) }) + + It("returns nil for empty quoted phrase", func() { + Expect(ftsSearchExpr("media_file", `""`)).To(BeNil()) + }) }) var _ = Describe("FTS5 Integration Search", func() { diff --git a/server/e2e/subsonic_searching_test.go b/server/e2e/subsonic_searching_test.go index 3a7512fd2..6ea6559c2 100644 --- a/server/e2e/subsonic_searching_test.go +++ b/server/e2e/subsonic_searching_test.go @@ -107,6 +107,16 @@ var _ = Describe("Search Endpoints", func() { Expect(resp.SearchResult3.Artist[0].Id).ToNot(BeEmpty()) }) + It("returns all results when query is empty (OpenSubsonic)", func() { + resp := doReq("search3", "query", "") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).To(HaveLen(4)) + Expect(resp.SearchResult3.Album).To(HaveLen(5)) + Expect(resp.SearchResult3.Song).To(HaveLen(6)) + }) + It("finds across all entity types simultaneously", func() { // "Beatles" should match artist, albums, and songs by The Beatles resp := doReq("search3", "query", "Beatles") From d02bf9a53dbed0c1376d93cdb76f3f0a82749f41 Mon Sep 17 00:00:00 2001 From: Deluan Date: Sun, 22 Feb 2026 00:32:14 -0500 Subject: [PATCH 06/60] test(e2e): add MusicBrainz ID tests for song and album searches Signed-off-by: Deluan --- server/e2e/e2e_suite_test.go | 41 +++++++++++++--- server/e2e/subsonic_searching_test.go | 71 +++++++++++++++++++++++++++ 2 files changed, 104 insertions(+), 8 deletions(-) diff --git a/server/e2e/e2e_suite_test.go b/server/e2e/e2e_suite_test.go index 7bda1527c..5981f4a86 100644 --- a/server/e2e/e2e_suite_test.go +++ b/server/e2e/e2e_suite_test.go @@ -52,6 +52,17 @@ type _t = map[string]any var template = storagetest.Template var track = storagetest.Track +// MusicBrainz ID constants for test data (valid UUID v4 values) +const ( + mbidBeatlesArtist = "b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d" + mbidAbbeyRoadAlbum = "a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d" + mbidAbbeyRoadRelGroup = "d4c3b2a1-f6e5-4b7a-9d8c-1f0e3a2b5c4d" + mbidComeTogether = "11111111-1111-4111-a111-111111111111" // mbz_release_track_id + mbidComeTogetherRec = "22222222-2222-4222-a222-222222222222" // mbz_recording_id + mbidSomething = "33333333-3333-4333-a333-333333333333" // mbz_release_track_id + mbidSomethingRec = "44444444-4444-4444-a444-444444444444" // mbz_recording_id +) + // Shared test state var ( ctx context.Context @@ -89,23 +100,37 @@ func createFS(files fstest.MapFS) storagetest.FakeFS { // buildTestFS creates the full test filesystem matching the plan func buildTestFS() storagetest.FakeFS { - abbeyRoad := template(_t{"albumartist": "The Beatles", "artist": "The Beatles", "album": "Abbey Road", "year": 1969, "genre": "Rock"}) + abbeyRoad := template(_t{ + "albumartist": "The Beatles", + "artist": "The Beatles", + "album": "Abbey Road", + "year": 1969, + "genre": "Rock", + "musicbrainz_artistid": mbidBeatlesArtist, + "musicbrainz_albumartistid": mbidBeatlesArtist, + "musicbrainz_albumid": mbidAbbeyRoadAlbum, + "musicbrainz_releasegroupid": mbidAbbeyRoadRelGroup, + }) help := template(_t{"albumartist": "The Beatles", "artist": "The Beatles", "album": "Help!", "year": 1965, "genre": "Rock"}) ledZepIV := template(_t{"albumartist": "Led Zeppelin", "artist": "Led Zeppelin", "album": "IV", "year": 1971, "genre": "Rock"}) kindOfBlue := template(_t{"albumartist": "Miles Davis", "artist": "Miles Davis", "album": "Kind of Blue", "year": 1959, "genre": "Jazz"}) popTrack := template(_t{"albumartist": "Various", "artist": "Various", "album": "Pop", "year": 2020, "genre": "Pop"}) return createFS(fstest.MapFS{ - // Rock / The Beatles / Abbey Road - "Rock/The Beatles/Abbey Road/01 - Come Together.mp3": abbeyRoad(track(1, "Come Together")), - "Rock/The Beatles/Abbey Road/02 - Something.mp3": abbeyRoad(track(2, "Something")), - // Rock / The Beatles / Help! + // Rock / The Beatles / Abbey Road (with MBIDs) + // Note: "musicbrainz_trackid" is an alias for the musicbrainz_recordingid tag (populates MbzRecordingID), + // "musicbrainz_releasetrackid" is an alias for the musicbrainz_trackid tag (populates MbzReleaseTrackID). + "Rock/The Beatles/Abbey Road/01 - Come Together.mp3": abbeyRoad(track(1, "Come Together", + _t{"musicbrainz_releasetrackid": mbidComeTogether, "musicbrainz_trackid": mbidComeTogetherRec})), + "Rock/The Beatles/Abbey Road/02 - Something.mp3": abbeyRoad(track(2, "Something", + _t{"musicbrainz_releasetrackid": mbidSomething, "musicbrainz_trackid": mbidSomethingRec})), + // Rock / The Beatles / Help! (no MBIDs) "Rock/The Beatles/Help!/01 - Help.mp3": help(track(1, "Help!")), - // Rock / Led Zeppelin / IV + // Rock / Led Zeppelin / IV (no MBIDs) "Rock/Led Zeppelin/IV/01 - Stairway To Heaven.mp3": ledZepIV(track(1, "Stairway To Heaven")), - // Jazz / Miles Davis / Kind of Blue + // Jazz / Miles Davis / Kind of Blue (no MBIDs) "Jazz/Miles Davis/Kind of Blue/01 - So What.mp3": kindOfBlue(track(1, "So What")), - // Pop (standalone track) + // Pop (standalone track, no MBIDs) "Pop/01 - Standalone Track.mp3": popTrack(track(1, "Standalone Track")), // _empty folder (directory with no audio) "_empty/.keep": &fstest.MapFile{Data: []byte{}, ModTime: time.Now()}, diff --git a/server/e2e/subsonic_searching_test.go b/server/e2e/subsonic_searching_test.go index 6ea6559c2..d0447ecd7 100644 --- a/server/e2e/subsonic_searching_test.go +++ b/server/e2e/subsonic_searching_test.go @@ -1,6 +1,7 @@ package e2e import ( + "github.com/google/uuid" "github.com/navidrome/navidrome/server/subsonic/responses" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" @@ -146,5 +147,75 @@ var _ = Describe("Search Endpoints", func() { Expect(s.Title).ToNot(BeEmpty()) } }) + + Describe("MBID search", func() { + It("finds songs by mbz_recording_id", func() { + resp := doReq("search3", "query", mbidComeTogetherRec) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Song).To(HaveLen(1)) + Expect(resp.SearchResult3.Song[0].Title).To(Equal("Come Together")) + }) + + It("finds songs by mbz_release_track_id", func() { + resp := doReq("search3", "query", mbidSomething) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Song).To(HaveLen(1)) + Expect(resp.SearchResult3.Song[0].Title).To(Equal("Something")) + }) + + It("finds albums by mbz_album_id", func() { + resp := doReq("search3", "query", mbidAbbeyRoadAlbum) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Album).To(HaveLen(1)) + Expect(resp.SearchResult3.Album[0].Name).To(Equal("Abbey Road")) + }) + + It("finds albums by mbz_release_group_id", func() { + resp := doReq("search3", "query", mbidAbbeyRoadRelGroup) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Album).To(HaveLen(1)) + Expect(resp.SearchResult3.Album[0].Name).To(Equal("Abbey Road")) + }) + + It("finds artists by mbz_artist_id", func() { + resp := doReq("search3", "query", mbidBeatlesArtist) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).To(HaveLen(1)) + Expect(resp.SearchResult3.Artist[0].Name).To(Equal("The Beatles")) + }) + + It("returns empty results for non-matching UUID", func() { + nonMatchingUUID := uuid.NewString() + resp := doReq("search3", "query", nonMatchingUUID) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).To(BeEmpty()) + Expect(resp.SearchResult3.Album).To(BeEmpty()) + Expect(resp.SearchResult3.Song).To(BeEmpty()) + }) + + It("does not return songs for artist MBID", func() { + // media_file MBID search only checks mbz_recording_id and mbz_release_track_id, + // so an artist MBID should return only the artist, not songs + resp := doReq("search3", "query", mbidBeatlesArtist) + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).To(HaveLen(1)) + Expect(resp.SearchResult3.Artist[0].Name).To(Equal("The Beatles")) + Expect(resp.SearchResult3.Song).To(BeEmpty()) + }) + }) }) }) From 23bf256a66f47713d225269546bc542bad1c9e40 Mon Sep 17 00:00:00 2001 From: Valeri Sokolov Date: Mon, 23 Feb 2026 04:05:59 +0100 Subject: [PATCH 07/60] feat: make album and artist annotations available to smart playlists (#4927) * feat(criteria): make album ratings available to smart playlist queries Expose an "albumrating" field mapping to album annotations. Signed-off-by: Valeri Sokolov * fix(criteria): use query parameters Signed-off-by: Valeri Sokolov * feat: add album and artist annotation fields to smart playlists Extend smart playlists to filter songs by album or artist annotations (rating, loved, play count, last played, date loved, date rated). This adds 12 new fields (6 album, 6 artist) with conditional JOINs that are only added when the criteria or sort references them, avoiding unnecessary query overhead. The album table JOIN is also removed since media_file.album_id can be used directly. --------- Signed-off-by: Valeri Sokolov Co-authored-by: Deluan --- model/criteria/criteria.go | 19 +++ model/criteria/criteria_test.go | 96 ++++++++++++++- model/criteria/fields.go | 151 ++++++++++++++++-------- model/criteria/operators_test.go | 62 ++++++---- persistence/playlist_repository.go | 23 +++- persistence/playlist_repository_test.go | 100 ++++++++++++++++ 6 files changed, 374 insertions(+), 77 deletions(-) diff --git a/model/criteria/criteria.go b/model/criteria/criteria.go index 54ac59697..bc3fe801c 100644 --- a/model/criteria/criteria.go +++ b/model/criteria/criteria.go @@ -95,6 +95,25 @@ func (c Criteria) ToSql() (sql string, args []any, err error) { return c.Expression.ToSql() } +// RequiredJoins inspects the expression tree and Sort field to determine which +// additional JOINs are needed when evaluating this criteria. +func (c Criteria) RequiredJoins() JoinType { + result := JoinNone + if c.Expression != nil { + result |= extractJoinTypes(c.Expression) + } + // Also check Sort fields + if c.Sort != "" { + for _, p := range strings.Split(c.Sort, ",") { + p = strings.TrimSpace(p) + p = strings.TrimLeft(p, "+-") + p = strings.TrimSpace(p) + result |= fieldJoinType(p) + } + } + return result +} + func (c Criteria) ChildPlaylistIds() []string { if c.Expression == nil { return nil diff --git a/model/criteria/criteria_test.go b/model/criteria/criteria_test.go index 032ead5c8..9a4da360c 100644 --- a/model/criteria/criteria_test.go +++ b/model/criteria/criteria_test.go @@ -27,6 +27,7 @@ var _ = Describe("Criteria", func() { StartsWith{"comment": "this"}, InTheRange{"year": []int{1980, 1990}}, IsNot{"genre": "Rock"}, + Gt{"albumrating": 3}, }, }, Sort: "title", @@ -48,7 +49,8 @@ var _ = Describe("Criteria", func() { { "all": [ { "startsWith": {"comment": "this"} }, { "inTheRange": {"year":[1980,1990]} }, - { "isNot": { "genre": "Rock" }} + { "isNot": { "genre": "Rock" }}, + { "gt": { "albumrating": 3 } } ] } ], @@ -68,10 +70,10 @@ var _ = Describe("Criteria", func() { gomega.Expect(err).ToNot(gomega.HaveOccurred()) gomega.Expect(sql).To(gomega.Equal( `(media_file.title LIKE ? AND media_file.title NOT LIKE ? ` + - `AND (not exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?) ` + + `AND (not exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?) ` + `OR media_file.album = ?) AND (media_file.comment LIKE ? AND (media_file.year >= ? AND media_file.year <= ?) ` + - `AND not exists (select 1 from json_tree(tags, '$.genre') where key='value' and value = ?)))`)) - gomega.Expect(args).To(gomega.HaveExactElements("%love%", "%hate%", "u2", "best of", "this%", 1980, 1990, "Rock")) + `AND not exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value = ?) AND COALESCE(album_annotation.rating, 0) > ?))`)) + gomega.Expect(args).To(gomega.HaveExactElements("%love%", "%hate%", "u2", "best of", "this%", 1980, 1990, "Rock", 3)) }) It("marshals to JSON", func() { j, err := json.Marshal(goObj) @@ -172,13 +174,95 @@ var _ = Describe("Criteria", func() { sql, args, err := goObj.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) gomega.Expect(sql).To(gomega.Equal( - `(exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?) AND ` + - `exists (select 1 from json_tree(participants, '$.composer') where key='name' and value LIKE ?))`, + `(exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?) AND ` + + `exists (select 1 from json_tree(media_file.participants, '$.composer') where key='name' and value LIKE ?))`, )) gomega.Expect(args).To(gomega.HaveExactElements("The Beatles", "%Lennon%")) }) }) + Describe("RequiredJoins", func() { + It("returns JoinNone when no annotation fields are used", func() { + c := Criteria{ + Expression: All{ + Contains{"title": "love"}, + }, + } + gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinNone)) + }) + It("returns JoinNone for media_file annotation fields", func() { + c := Criteria{ + Expression: All{ + Is{"loved": true}, + Gt{"playCount": 5}, + }, + } + gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinNone)) + }) + It("returns JoinAlbumAnnotation for album annotation fields", func() { + c := Criteria{ + Expression: All{ + Gt{"albumRating": 3}, + }, + } + gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinAlbumAnnotation)) + }) + It("returns JoinArtistAnnotation for artist annotation fields", func() { + c := Criteria{ + Expression: All{ + Is{"artistLoved": true}, + }, + } + gomega.Expect(c.RequiredJoins()).To(gomega.Equal(JoinArtistAnnotation)) + }) + It("returns both join types when both are used", func() { + c := Criteria{ + Expression: All{ + Gt{"albumRating": 3}, + Is{"artistLoved": true}, + }, + } + j := c.RequiredJoins() + gomega.Expect(j.Has(JoinAlbumAnnotation)).To(gomega.BeTrue()) + gomega.Expect(j.Has(JoinArtistAnnotation)).To(gomega.BeTrue()) + }) + It("detects join types in nested expressions", func() { + c := Criteria{ + Expression: All{ + Any{ + All{ + Is{"albumLoved": true}, + }, + }, + Any{ + Gt{"artistPlayCount": 10}, + }, + }, + } + j := c.RequiredJoins() + gomega.Expect(j.Has(JoinAlbumAnnotation)).To(gomega.BeTrue()) + gomega.Expect(j.Has(JoinArtistAnnotation)).To(gomega.BeTrue()) + }) + It("detects join types from Sort field", func() { + c := Criteria{ + Expression: All{ + Contains{"title": "love"}, + }, + Sort: "albumRating", + } + gomega.Expect(c.RequiredJoins().Has(JoinAlbumAnnotation)).To(gomega.BeTrue()) + }) + It("detects join types from Sort field with direction prefix", func() { + c := Criteria{ + Expression: All{ + Contains{"title": "love"}, + }, + Sort: "-artistRating", + } + gomega.Expect(c.RequiredJoins().Has(JoinArtistAnnotation)).To(gomega.BeTrue()) + }) + }) + Context("with child playlists", func() { var ( topLevelInPlaylistID string diff --git a/model/criteria/fields.go b/model/criteria/fields.go index b5ac92718..4dfc50f11 100644 --- a/model/criteria/fields.go +++ b/model/criteria/fields.go @@ -9,45 +9,71 @@ import ( "github.com/navidrome/navidrome/log" ) +// JoinType is a bitmask indicating which additional JOINs are needed by a smart playlist expression. +type JoinType int + +const ( + JoinNone JoinType = 0 + JoinAlbumAnnotation JoinType = 1 << iota + JoinArtistAnnotation +) + +// Has returns true if j contains all bits in other. +func (j JoinType) Has(other JoinType) bool { return j&other != 0 } + var fieldMap = map[string]*mappedField{ - "title": {field: "media_file.title"}, - "album": {field: "media_file.album"}, - "hascoverart": {field: "media_file.has_cover_art"}, - "tracknumber": {field: "media_file.track_number"}, - "discnumber": {field: "media_file.disc_number"}, - "year": {field: "media_file.year"}, - "date": {field: "media_file.date", alias: "recordingdate"}, - "originalyear": {field: "media_file.original_year"}, - "originaldate": {field: "media_file.original_date"}, - "releaseyear": {field: "media_file.release_year"}, - "releasedate": {field: "media_file.release_date"}, - "size": {field: "media_file.size"}, - "compilation": {field: "media_file.compilation"}, - "explicitstatus": {field: "media_file.explicit_status"}, - "dateadded": {field: "media_file.created_at"}, - "datemodified": {field: "media_file.updated_at"}, - "discsubtitle": {field: "media_file.disc_subtitle"}, - "comment": {field: "media_file.comment"}, - "lyrics": {field: "media_file.lyrics"}, - "sorttitle": {field: "media_file.sort_title"}, - "sortalbum": {field: "media_file.sort_album_name"}, - "sortartist": {field: "media_file.sort_artist_name"}, - "sortalbumartist": {field: "media_file.sort_album_artist_name"}, - "albumcomment": {field: "media_file.mbz_album_comment"}, - "catalognumber": {field: "media_file.catalog_num"}, - "filepath": {field: "media_file.path"}, - "filetype": {field: "media_file.suffix"}, - "duration": {field: "media_file.duration"}, - "bitrate": {field: "media_file.bit_rate"}, - "bitdepth": {field: "media_file.bit_depth"}, - "bpm": {field: "media_file.bpm"}, - "channels": {field: "media_file.channels"}, - "loved": {field: "COALESCE(annotation.starred, false)"}, - "dateloved": {field: "annotation.starred_at"}, - "lastplayed": {field: "annotation.play_date"}, - "daterated": {field: "annotation.rated_at"}, - "playcount": {field: "COALESCE(annotation.play_count, 0)"}, - "rating": {field: "COALESCE(annotation.rating, 0)"}, + "title": {field: "media_file.title"}, + "album": {field: "media_file.album"}, + "hascoverart": {field: "media_file.has_cover_art"}, + "tracknumber": {field: "media_file.track_number"}, + "discnumber": {field: "media_file.disc_number"}, + "year": {field: "media_file.year"}, + "date": {field: "media_file.date", alias: "recordingdate"}, + "originalyear": {field: "media_file.original_year"}, + "originaldate": {field: "media_file.original_date"}, + "releaseyear": {field: "media_file.release_year"}, + "releasedate": {field: "media_file.release_date"}, + "size": {field: "media_file.size"}, + "compilation": {field: "media_file.compilation"}, + "explicitstatus": {field: "media_file.explicit_status"}, + "dateadded": {field: "media_file.created_at"}, + "datemodified": {field: "media_file.updated_at"}, + "discsubtitle": {field: "media_file.disc_subtitle"}, + "comment": {field: "media_file.comment"}, + "lyrics": {field: "media_file.lyrics"}, + "sorttitle": {field: "media_file.sort_title"}, + "sortalbum": {field: "media_file.sort_album_name"}, + "sortartist": {field: "media_file.sort_artist_name"}, + "sortalbumartist": {field: "media_file.sort_album_artist_name"}, + "albumcomment": {field: "media_file.mbz_album_comment"}, + "catalognumber": {field: "media_file.catalog_num"}, + "filepath": {field: "media_file.path"}, + "filetype": {field: "media_file.suffix"}, + "duration": {field: "media_file.duration"}, + "bitrate": {field: "media_file.bit_rate"}, + "bitdepth": {field: "media_file.bit_depth"}, + "bpm": {field: "media_file.bpm"}, + "channels": {field: "media_file.channels"}, + "loved": {field: "COALESCE(annotation.starred, false)"}, + "dateloved": {field: "annotation.starred_at"}, + "lastplayed": {field: "annotation.play_date"}, + "daterated": {field: "annotation.rated_at"}, + "playcount": {field: "COALESCE(annotation.play_count, 0)"}, + "rating": {field: "COALESCE(annotation.rating, 0)"}, + "albumrating": {field: "COALESCE(album_annotation.rating, 0)", joinType: JoinAlbumAnnotation}, + "albumloved": {field: "COALESCE(album_annotation.starred, false)", joinType: JoinAlbumAnnotation}, + "albumplaycount": {field: "COALESCE(album_annotation.play_count, 0)", joinType: JoinAlbumAnnotation}, + "albumlastplayed": {field: "album_annotation.play_date", joinType: JoinAlbumAnnotation}, + "albumdateloved": {field: "album_annotation.starred_at", joinType: JoinAlbumAnnotation}, + "albumdaterated": {field: "album_annotation.rated_at", joinType: JoinAlbumAnnotation}, + + "artistrating": {field: "COALESCE(artist_annotation.rating, 0)", joinType: JoinArtistAnnotation}, + "artistloved": {field: "COALESCE(artist_annotation.starred, false)", joinType: JoinArtistAnnotation}, + "artistplaycount": {field: "COALESCE(artist_annotation.play_count, 0)", joinType: JoinArtistAnnotation}, + "artistlastplayed": {field: "artist_annotation.play_date", joinType: JoinArtistAnnotation}, + "artistdateloved": {field: "artist_annotation.starred_at", joinType: JoinArtistAnnotation}, + "artistdaterated": {field: "artist_annotation.rated_at", joinType: JoinArtistAnnotation}, + "mbz_album_id": {field: "media_file.mbz_album_id"}, "mbz_album_artist_id": {field: "media_file.mbz_album_artist_id"}, "mbz_artist_id": {field: "media_file.mbz_artist_id"}, @@ -65,12 +91,13 @@ var fieldMap = map[string]*mappedField{ } type mappedField struct { - field string - order string - isRole bool // true if the field is a role (e.g. "artist", "composer", "conductor", etc.) - isTag bool // true if the field is a tag imported from the file metadata - alias string // name from `mappings.yml` that may differ from the name used in the smart playlist - numeric bool // true if the field/tag should be treated as numeric + field string + order string + isRole bool // true if the field is a role (e.g. "artist", "composer", "conductor", etc.) + isTag bool // true if the field is a tag imported from the file metadata + alias string // name from `mappings.yml` that may differ from the name used in the smart playlist + numeric bool // true if the field/tag should be treated as numeric + joinType JoinType // which additional JOINs this field requires } func mapFields(expr map[string]any) map[string]any { @@ -169,7 +196,7 @@ func (e tagCond) ToSql() (string, []any, error) { } } - cond = fmt.Sprintf("exists (select 1 from json_tree(tags, '$.%s') where key='value' and %s)", + cond = fmt.Sprintf("exists (select 1 from json_tree(media_file.tags, '$.%s') where key='value' and %s)", tagName, cond) if e.not { cond = "not " + cond @@ -189,7 +216,7 @@ type roleCond struct { func (e roleCond) ToSql() (string, []any, error) { cond, args, err := e.cond.ToSql() - cond = fmt.Sprintf(`exists (select 1 from json_tree(participants, '$.%s') where key='name' and %s)`, + cond = fmt.Sprintf(`exists (select 1 from json_tree(media_file.participants, '$.%s') where key='name' and %s)`, e.role, cond) if e.not { cond = "not " + cond @@ -197,6 +224,38 @@ func (e roleCond) ToSql() (string, []any, error) { return cond, args, err } +// fieldJoinType returns the JoinType for a given field name (case-insensitive). +func fieldJoinType(name string) JoinType { + if f, ok := fieldMap[strings.ToLower(name)]; ok { + return f.joinType + } + return JoinNone +} + +// extractJoinTypes walks an expression tree and collects all required JoinType flags. +func extractJoinTypes(expr any) JoinType { + result := JoinNone + switch e := expr.(type) { + case All: + for _, sub := range e { + result |= extractJoinTypes(sub) + } + case Any: + for _, sub := range e { + result |= extractJoinTypes(sub) + } + default: + // Leaf expression: use reflection to check if it's a map with field names + rv := reflect.ValueOf(expr) + if rv.Kind() == reflect.Map && rv.Type().Key().Kind() == reflect.String { + for _, key := range rv.MapKeys() { + result |= fieldJoinType(key.String()) + } + } + } + return result +} + // AddRoles adds roles to the field map. This is used to add all artist roles to the field map, so they can be used in // smart playlists. If a role already exists in the field map, it is ignored, so calls to this function are idempotent. func AddRoles(roles []string) { diff --git a/model/criteria/operators_test.go b/model/criteria/operators_test.go index 4c1db1303..f0681af6b 100644 --- a/model/criteria/operators_test.go +++ b/model/criteria/operators_test.go @@ -54,23 +54,43 @@ var _ = Describe("Operators", func() { Entry("inTheLast", InTheLast{"lastPlayed": 30}, "annotation.play_date > ?", StartOfPeriod(30, time.Now())), Entry("notInTheLast", NotInTheLast{"lastPlayed": 30}, "(annotation.play_date < ? OR annotation.play_date IS NULL)", StartOfPeriod(30, time.Now())), + // Album annotation fields + Entry("albumRating", Gt{"albumRating": 3}, "COALESCE(album_annotation.rating, 0) > ?", 3), + Entry("albumLoved", Is{"albumLoved": true}, "COALESCE(album_annotation.starred, false) = ?", true), + Entry("albumPlayCount", Gt{"albumPlayCount": 5}, "COALESCE(album_annotation.play_count, 0) > ?", 5), + Entry("albumLastPlayed", After{"albumLastPlayed": rangeStart}, "album_annotation.play_date > ?", rangeStart), + Entry("albumDateLoved", Before{"albumDateLoved": rangeStart}, "album_annotation.starred_at < ?", rangeStart), + Entry("albumDateRated", After{"albumDateRated": rangeStart}, "album_annotation.rated_at > ?", rangeStart), + Entry("albumLastPlayed inTheLast", InTheLast{"albumLastPlayed": 30}, "album_annotation.play_date > ?", StartOfPeriod(30, time.Now())), + Entry("albumLastPlayed notInTheLast", NotInTheLast{"albumLastPlayed": 30}, "(album_annotation.play_date < ? OR album_annotation.play_date IS NULL)", StartOfPeriod(30, time.Now())), + + // Artist annotation fields + Entry("artistRating", Gt{"artistRating": 3}, "COALESCE(artist_annotation.rating, 0) > ?", 3), + Entry("artistLoved", Is{"artistLoved": true}, "COALESCE(artist_annotation.starred, false) = ?", true), + Entry("artistPlayCount", Gt{"artistPlayCount": 5}, "COALESCE(artist_annotation.play_count, 0) > ?", 5), + Entry("artistLastPlayed", After{"artistLastPlayed": rangeStart}, "artist_annotation.play_date > ?", rangeStart), + Entry("artistDateLoved", Before{"artistDateLoved": rangeStart}, "artist_annotation.starred_at < ?", rangeStart), + Entry("artistDateRated", After{"artistDateRated": rangeStart}, "artist_annotation.rated_at > ?", rangeStart), + Entry("artistLastPlayed inTheLast", InTheLast{"artistLastPlayed": 30}, "artist_annotation.play_date > ?", StartOfPeriod(30, time.Now())), + Entry("artistLastPlayed notInTheLast", NotInTheLast{"artistLastPlayed": 30}, "(artist_annotation.play_date < ? OR artist_annotation.play_date IS NULL)", StartOfPeriod(30, time.Now())), + // Tag tests - Entry("tag is [string]", Is{"genre": "Rock"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value = ?)", "Rock"), - Entry("tag isNot [string]", IsNot{"genre": "Rock"}, "not exists (select 1 from json_tree(tags, '$.genre') where key='value' and value = ?)", "Rock"), - Entry("tag gt", Gt{"genre": "A"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value > ?)", "A"), - Entry("tag lt", Lt{"genre": "Z"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value < ?)", "Z"), - Entry("tag contains", Contains{"genre": "Rock"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"), - Entry("tag not contains", NotContains{"genre": "Rock"}, "not exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"), - Entry("tag startsWith", StartsWith{"genre": "Soft"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "Soft%"), - Entry("tag endsWith", EndsWith{"genre": "Rock"}, "exists (select 1 from json_tree(tags, '$.genre') where key='value' and value LIKE ?)", "%Rock"), + Entry("tag is [string]", Is{"genre": "Rock"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value = ?)", "Rock"), + Entry("tag isNot [string]", IsNot{"genre": "Rock"}, "not exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value = ?)", "Rock"), + Entry("tag gt", Gt{"genre": "A"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value > ?)", "A"), + Entry("tag lt", Lt{"genre": "Z"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value < ?)", "Z"), + Entry("tag contains", Contains{"genre": "Rock"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"), + Entry("tag not contains", NotContains{"genre": "Rock"}, "not exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "%Rock%"), + Entry("tag startsWith", StartsWith{"genre": "Soft"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "Soft%"), + Entry("tag endsWith", EndsWith{"genre": "Rock"}, "exists (select 1 from json_tree(media_file.tags, '$.genre') where key='value' and value LIKE ?)", "%Rock"), // Artist roles tests - Entry("role is [string]", Is{"artist": "u2"}, "exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?)", "u2"), - Entry("role isNot [string]", IsNot{"artist": "u2"}, "not exists (select 1 from json_tree(participants, '$.artist') where key='name' and value = ?)", "u2"), - Entry("role contains [string]", Contains{"artist": "u2"}, "exists (select 1 from json_tree(participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"), - Entry("role not contains [string]", NotContains{"artist": "u2"}, "not exists (select 1 from json_tree(participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"), - Entry("role startsWith [string]", StartsWith{"composer": "John"}, "exists (select 1 from json_tree(participants, '$.composer') where key='name' and value LIKE ?)", "John%"), - Entry("role endsWith [string]", EndsWith{"composer": "Lennon"}, "exists (select 1 from json_tree(participants, '$.composer') where key='name' and value LIKE ?)", "%Lennon"), + Entry("role is [string]", Is{"artist": "u2"}, "exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?)", "u2"), + Entry("role isNot [string]", IsNot{"artist": "u2"}, "not exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value = ?)", "u2"), + Entry("role contains [string]", Contains{"artist": "u2"}, "exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"), + Entry("role not contains [string]", NotContains{"artist": "u2"}, "not exists (select 1 from json_tree(media_file.participants, '$.artist') where key='name' and value LIKE ?)", "%u2%"), + Entry("role startsWith [string]", StartsWith{"composer": "John"}, "exists (select 1 from json_tree(media_file.participants, '$.composer') where key='name' and value LIKE ?)", "John%"), + Entry("role endsWith [string]", EndsWith{"composer": "Lennon"}, "exists (select 1 from json_tree(media_file.participants, '$.composer') where key='name' and value LIKE ?)", "%Lennon"), ) // TODO Validate operators that are not valid for each field type. @@ -88,7 +108,7 @@ var _ = Describe("Operators", func() { op := EndsWith{"mood": "Soft"} sql, args, err := op.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) - gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.mood') where key='value' and value LIKE ?)")) + gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.mood') where key='value' and value LIKE ?)")) gomega.Expect(args).To(gomega.HaveExactElements("%Soft")) }) It("casts numeric comparisons", func() { @@ -96,7 +116,7 @@ var _ = Describe("Operators", func() { op := Lt{"rate": 6} sql, args, err := op.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) - gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.rate') where key='value' and CAST(value AS REAL) < ?)")) + gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.rate') where key='value' and CAST(value AS REAL) < ?)")) gomega.Expect(args).To(gomega.HaveExactElements(6)) }) It("skips unknown tag names", func() { @@ -110,7 +130,7 @@ var _ = Describe("Operators", func() { op := Contains{"releasetype": "soundtrack"} sql, args, err := op.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) - gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value LIKE ?)")) + gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value LIKE ?)")) gomega.Expect(args).To(gomega.HaveExactElements("%soundtrack%")) }) It("supports albumtype as alias for releasetype", func() { @@ -118,7 +138,7 @@ var _ = Describe("Operators", func() { op := Contains{"albumtype": "live"} sql, args, err := op.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) - gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value LIKE ?)")) + gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value LIKE ?)")) gomega.Expect(args).To(gomega.HaveExactElements("%live%")) }) It("supports albumtype alias with Is operator", func() { @@ -127,7 +147,7 @@ var _ = Describe("Operators", func() { sql, args, err := op.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) // Should query $.releasetype, not $.albumtype - gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value = ?)")) + gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value = ?)")) gomega.Expect(args).To(gomega.HaveExactElements("album")) }) It("supports albumtype alias with IsNot operator", func() { @@ -136,7 +156,7 @@ var _ = Describe("Operators", func() { sql, args, err := op.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) // Should query $.releasetype, not $.albumtype - gomega.Expect(sql).To(gomega.Equal("not exists (select 1 from json_tree(tags, '$.releasetype') where key='value' and value = ?)")) + gomega.Expect(sql).To(gomega.Equal("not exists (select 1 from json_tree(media_file.tags, '$.releasetype') where key='value' and value = ?)")) gomega.Expect(args).To(gomega.HaveExactElements("compilation")) }) }) @@ -147,7 +167,7 @@ var _ = Describe("Operators", func() { op := EndsWith{"producer": "Eno"} sql, args, err := op.ToSql() gomega.Expect(err).ToNot(gomega.HaveOccurred()) - gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(participants, '$.producer') where key='name' and value LIKE ?)")) + gomega.Expect(sql).To(gomega.Equal("exists (select 1 from json_tree(media_file.participants, '$.producer') where key='name' and value LIKE ?)")) gomega.Expect(args).To(gomega.HaveExactElements("%Eno")) }) It("skips unknown roles", func() { diff --git a/persistence/playlist_repository.go b/persistence/playlist_repository.go index 955a13bc9..11b9cd8b4 100644 --- a/persistence/playlist_repository.go +++ b/persistence/playlist_repository.go @@ -241,10 +241,25 @@ func (r *playlistRepository) refreshSmartPlaylist(pls *model.Playlist) bool { } sq := Select("row_number() over (order by "+rules.OrderBy()+") as id", "'"+pls.ID+"' as playlist_id", "media_file.id as media_file_id"). - From("media_file").LeftJoin("annotation on (" + - "annotation.item_id = media_file.id" + - " AND annotation.item_type = 'media_file'" + - " AND annotation.user_id = '" + usr.ID + "')") + From("media_file").LeftJoin("annotation on ("+ + "annotation.item_id = media_file.id"+ + " AND annotation.item_type = 'media_file'"+ + " AND annotation.user_id = ?)", usr.ID) + + // Conditionally join album/artist annotation tables only when referenced by criteria or sort + requiredJoins := rules.RequiredJoins() + if requiredJoins.Has(criteria.JoinAlbumAnnotation) { + sq = sq.LeftJoin("annotation AS album_annotation ON ("+ + "album_annotation.item_id = media_file.album_id"+ + " AND album_annotation.item_type = 'album'"+ + " AND album_annotation.user_id = ?)", usr.ID) + } + if requiredJoins.Has(criteria.JoinArtistAnnotation) { + sq = sq.LeftJoin("annotation AS artist_annotation ON ("+ + "artist_annotation.item_id = media_file.artist_id"+ + " AND artist_annotation.item_type = 'artist'"+ + " AND artist_annotation.user_id = ?)", usr.ID) + } // Only include media files from libraries the user has access to sq = r.applyLibraryFilter(sq, "media_file") diff --git a/persistence/playlist_repository_test.go b/persistence/playlist_repository_test.go index 5230390fb..c091cb32b 100644 --- a/persistence/playlist_repository_test.go +++ b/persistence/playlist_repository_test.go @@ -287,6 +287,106 @@ var _ = Describe("PlaylistRepository", func() { }) }) + Describe("Smart Playlists with Album/Artist Annotation Criteria", func() { + var testPlaylistID string + + AfterEach(func() { + if testPlaylistID != "" { + _ = repo.Delete(testPlaylistID) + testPlaylistID = "" + } + }) + + It("matches tracks from starred albums using albumLoved", func() { + // albumRadioactivity (ID "103") is starred in test fixtures + // Songs in album 103: 1003, 1004, 1005, 1006 + rules := &criteria.Criteria{ + Expression: criteria.All{ + criteria.Is{"albumLoved": true}, + }, + } + newPls := model.Playlist{Name: "Starred Album Songs", OwnerID: "userid", Rules: rules} + Expect(repo.Put(&newPls)).To(Succeed()) + testPlaylistID = newPls.ID + + conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second + pls, err := repo.GetWithTracks(newPls.ID, true, false) + Expect(err).ToNot(HaveOccurred()) + + trackIDs := make([]string, len(pls.Tracks)) + for i, t := range pls.Tracks { + trackIDs[i] = t.MediaFileID + } + Expect(trackIDs).To(ConsistOf("1003", "1004", "1005", "1006")) + }) + + It("matches tracks from starred artists using artistLoved", func() { + // artistBeatles (ID "3") is starred in test fixtures + // Songs with ArtistID "3": 1001, 1002, 3002 + rules := &criteria.Criteria{ + Expression: criteria.All{ + criteria.Is{"artistLoved": true}, + }, + } + newPls := model.Playlist{Name: "Starred Artist Songs", OwnerID: "userid", Rules: rules} + Expect(repo.Put(&newPls)).To(Succeed()) + testPlaylistID = newPls.ID + + conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second + pls, err := repo.GetWithTracks(newPls.ID, true, false) + Expect(err).ToNot(HaveOccurred()) + + trackIDs := make([]string, len(pls.Tracks)) + for i, t := range pls.Tracks { + trackIDs[i] = t.MediaFileID + } + Expect(trackIDs).To(ConsistOf("1001", "1002", "3002")) + }) + + It("matches tracks with combined album and artist criteria", func() { + // albumLoved=true → songs from album 103 (1003, 1004, 1005, 1006) + // artistLoved=true → songs with artist 3 (1001, 1002) + // Using Any: union of both sets + rules := &criteria.Criteria{ + Expression: criteria.Any{ + criteria.Is{"albumLoved": true}, + criteria.Is{"artistLoved": true}, + }, + } + newPls := model.Playlist{Name: "Combined Album+Artist", OwnerID: "userid", Rules: rules} + Expect(repo.Put(&newPls)).To(Succeed()) + testPlaylistID = newPls.ID + + conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second + pls, err := repo.GetWithTracks(newPls.ID, true, false) + Expect(err).ToNot(HaveOccurred()) + + trackIDs := make([]string, len(pls.Tracks)) + for i, t := range pls.Tracks { + trackIDs[i] = t.MediaFileID + } + Expect(trackIDs).To(ConsistOf("1001", "1002", "1003", "1004", "1005", "1006", "3002")) + }) + + It("returns no tracks when no albums/artists match", func() { + // No album has rating 5 in fixtures + rules := &criteria.Criteria{ + Expression: criteria.All{ + criteria.Is{"albumRating": 5}, + }, + } + newPls := model.Playlist{Name: "No Match", OwnerID: "userid", Rules: rules} + Expect(repo.Put(&newPls)).To(Succeed()) + testPlaylistID = newPls.ID + + conf.Server.SmartPlaylistRefreshDelay = -1 * time.Second + pls, err := repo.GetWithTracks(newPls.ID, true, false) + Expect(err).ToNot(HaveOccurred()) + + Expect(pls.Tracks).To(BeEmpty()) + }) + }) + Describe("Smart Playlists with Tag Criteria", func() { var mfRepo model.MediaFileRepository var testPlaylistID string From b59eb329610964f226f66fecb7190b06d28d052c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 23 Feb 2026 08:51:54 -0500 Subject: [PATCH 08/60] feat(subsonic): sort search3 results by relevance (#5086) * fix(subsonic): optimize search3 for high-cardinality FTS queries Use a two-phase query strategy for FTS5 searches to avoid the performance penalty of expensive LEFT JOINs (annotation, bookmark, library) on high-cardinality results like "the". Phase 1 runs a lightweight query (main table + FTS index only) to get sorted, paginated rowids. Phase 2 hydrates only those few rowids with the full JOINs, making them nearly free. For queries with complex ORDER BY expressions that reference joined tables (e.g. artist search sorted by play count), the optimization is skipped and the original single-query approach is used. * fix(search): update order by clauses to include 'rank' for FTS queries Signed-off-by: Deluan * fix(search): reintroduce 'rank' in Phase 2 ORDER BY for FTS queries Signed-off-by: Deluan * fix(search): remove 'rank' from ORDER BY in non-FTS queries and adjust two-phase query handling Signed-off-by: Deluan * fix(search): update FTS ranking to use bm25 weights and simplify ORDER BY qualification Signed-off-by: Deluan * fix(search): refine FTS query handling and improve comments for clarity Signed-off-by: Deluan * fix(search): refactor full-text search handling to streamline query strategy selection and improve LIKE fallback logic. Increase e2e coverage for search3 Signed-off-by: Deluan * refactor: enhance FTS column definitions and relevance weights Signed-off-by: Deluan * fix(search): refactor Search method signatures to remove offset and size parameters, streamline query handling Signed-off-by: Deluan * fix(search): allow single-character queries in search strategies and update related tests Signed-off-by: Deluan * fix(search): make FTS Phase 1 treat Max=0 as no limit, reorganize tests FTS Phase 1 unconditionally called Limit(uint64(options.Max)), which produced LIMIT 0 when Max was zero. This diverged from applyOptions where Max=0 means no limit. Now Phase 1 mirrors applyOptions: only add LIMIT/OFFSET when the value is positive. Also moved legacy backend integration tests from sql_search_fts_test.go to sql_search_like_test.go and added regression tests for the Max=0 behavior on both backends. * refactor: simplify callSearch function by removing variadic options and directly using QueryOptions Signed-off-by: Deluan * fix(search): implement ftsQueryDegraded function to detect significant content loss in FTS queries Signed-off-by: Deluan --------- Signed-off-by: Deluan --- model/searchable.go | 2 +- persistence/album_repository.go | 26 +- persistence/artist_repository.go | 32 +-- persistence/artist_repository_test.go | 16 +- persistence/mediafile_repository.go | 26 +- persistence/mediafile_repository_test.go | 14 +- persistence/sql_restful.go | 6 +- persistence/sql_restful_test.go | 122 +++++---- persistence/sql_search.go | 118 +++++---- persistence/sql_search_fts.go | 271 +++++++++++++++----- persistence/sql_search_fts_test.go | 299 +++++++++++++++-------- persistence/sql_search_like.go | 106 ++++++++ persistence/sql_search_like_test.go | 134 ++++++++++ persistence/sql_search_test.go | 83 +++---- server/e2e/e2e_suite_test.go | 3 + server/e2e/subsonic_album_lists_test.go | 40 +-- server/e2e/subsonic_browsing_test.go | 4 +- server/e2e/subsonic_multilibrary_test.go | 18 +- server/e2e/subsonic_searching_test.go | 59 ++++- server/subsonic/searching.go | 35 ++- tests/mock_album_repo.go | 2 +- tests/mock_artist_repo.go | 2 +- tests/mock_mediafile_repo.go | 2 +- 23 files changed, 1005 insertions(+), 415 deletions(-) create mode 100644 persistence/sql_search_like.go create mode 100644 persistence/sql_search_like_test.go diff --git a/model/searchable.go b/model/searchable.go index 631a11726..a64a0171c 100644 --- a/model/searchable.go +++ b/model/searchable.go @@ -1,5 +1,5 @@ package model type SearchableRepository[T any] interface { - Search(q string, offset, size int, options ...QueryOptions) (T, error) + Search(q string, options ...QueryOptions) (T, error) } diff --git a/persistence/album_repository.go b/persistence/album_repository.go index 58bfcca51..077e33d17 100644 --- a/persistence/album_repository.go +++ b/persistence/album_repository.go @@ -12,7 +12,6 @@ import ( . "github.com/Masterminds/squirrel" "github.com/deluan/rest" - "github.com/google/uuid" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -353,18 +352,21 @@ func (r *albumRepository) purgeEmpty(libraryIDs ...int) error { return nil } -func (r *albumRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Albums, error) { +var albumSearchConfig = searchConfig{ + NaturalOrder: "album.rowid", + OrderBy: []string{"name"}, + MBIDFields: []string{"mbz_album_id", "mbz_release_group_id"}, +} + +func (r *albumRepository) Search(q string, options ...model.QueryOptions) (model.Albums, error) { + var opts model.QueryOptions + if len(options) > 0 { + opts = options[0] + } var res dbAlbums - if uuid.Validate(q) == nil { - err := r.searchByMBID(r.selectAlbum(options...), q, []string{"mbz_album_id", "mbz_release_group_id"}, &res) - if err != nil { - return nil, fmt.Errorf("searching album by MBID %q: %w", q, err) - } - } else { - err := r.doSearch(r.selectAlbum(options...), q, offset, size, &res, "album.rowid", "name") - if err != nil { - return nil, fmt.Errorf("searching album by query %q: %w", q, err) - } + err := r.doSearch(r.selectAlbum(options...), q, &res, albumSearchConfig, opts) + if err != nil { + return nil, fmt.Errorf("searching album %q: %w", q, err) } return res.toModels(), nil } diff --git a/persistence/artist_repository.go b/persistence/artist_repository.go index f801787d8..07824e21f 100644 --- a/persistence/artist_repository.go +++ b/persistence/artist_repository.go @@ -11,7 +11,6 @@ import ( . "github.com/Masterminds/squirrel" "github.com/deluan/rest" - "github.com/google/uuid" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -513,20 +512,25 @@ func (r *artistRepository) RefreshStats(allArtists bool) (int64, error) { return totalRowsAffected, nil } -func (r *artistRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Artists, error) { - var res dbArtists - if uuid.Validate(q) == nil { - err := r.searchByMBID(r.selectArtist(options...), q, []string{"mbz_artist_id"}, &res) - if err != nil { - return nil, fmt.Errorf("searching artist by MBID %q: %w", q, err) - } - } else { +func (r *artistRepository) searchCfg() searchConfig { + return searchConfig{ // Natural order for artists is more performant by ID, due to GROUP BY clause in selectArtist - err := r.doSearch(r.selectArtist(options...), q, offset, size, &res, "artist.id", - "sum(json_extract(stats, '$.total.m')) desc", "name") - if err != nil { - return nil, fmt.Errorf("searching artist by query %q: %w", q, err) - } + NaturalOrder: "artist.id", + OrderBy: []string{"sum(json_extract(stats, '$.total.m')) desc", "name"}, + MBIDFields: []string{"mbz_artist_id"}, + LibraryFilter: r.applyLibraryFilterToArtistQuery, + } +} + +func (r *artistRepository) Search(q string, options ...model.QueryOptions) (model.Artists, error) { + var opts model.QueryOptions + if len(options) > 0 { + opts = options[0] + } + var res dbArtists + err := r.doSearch(r.selectArtist(options...), q, &res, r.searchCfg(), opts) + if err != nil { + return nil, fmt.Errorf("searching artist %q: %w", q, err) } return res.toModels(), nil } diff --git a/persistence/artist_repository_test.go b/persistence/artist_repository_test.go index 15340eeb4..90e449e8d 100644 --- a/persistence/artist_repository_test.go +++ b/persistence/artist_repository_test.go @@ -512,7 +512,7 @@ var _ = Describe("ArtistRepository", func() { Expect(err).ToNot(HaveOccurred()) // Test the search - results, err := (*testRepo).Search("550e8400-e29b-41d4-a716-446655440010", 0, 10) + results, err := (*testRepo).Search("550e8400-e29b-41d4-a716-446655440010", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) if shouldFind { @@ -543,12 +543,12 @@ var _ = Describe("ArtistRepository", func() { Expect(err).ToNot(HaveOccurred()) // Restricted user should not find this artist - results, err := restrictedRepo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", 0, 10) + results, err := restrictedRepo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty()) // But admin should find it - results, err = repo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", 0, 10) + results, err = repo.Search("a74b1b7f-71a5-4011-9441-d0b5e4122711", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) @@ -560,7 +560,7 @@ var _ = Describe("ArtistRepository", func() { Context("Text Search", func() { It("allows admin to find artists by name regardless of library", func() { - results, err := repo.Search("Beatles", 0, 10) + results, err := repo.Search("Beatles", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Name).To(Equal("The Beatles")) @@ -580,7 +580,7 @@ var _ = Describe("ArtistRepository", func() { Expect(err).ToNot(HaveOccurred()) // Restricted user should not find this artist - results, err := restrictedRepo.Search("Unique Search Name", 0, 10) + results, err := restrictedRepo.Search("Unique Search Name", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty(), "Text search should respect library filtering") @@ -688,7 +688,7 @@ var _ = Describe("ArtistRepository", func() { Expect(artists).To(HaveLen(5)) // Including the missing artist // Search never returns missing artists (hardcoded behavior) - results, err := repo.Search("Missing Artist", 0, 10) + results, err := repo.Search("Missing Artist", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty()) }) @@ -742,11 +742,11 @@ var _ = Describe("ArtistRepository", func() { }) It("Search returns empty results for users without library access", func() { - results, err := restrictedRepo.Search("Beatles", 0, 10) + results, err := restrictedRepo.Search("Beatles", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty()) - results, err = restrictedRepo.Search("Kraftwerk", 0, 10) + results, err = restrictedRepo.Search("Kraftwerk", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty()) }) diff --git a/persistence/mediafile_repository.go b/persistence/mediafile_repository.go index 264be6f3f..394ca5b70 100644 --- a/persistence/mediafile_repository.go +++ b/persistence/mediafile_repository.go @@ -11,7 +11,6 @@ import ( . "github.com/Masterminds/squirrel" "github.com/deluan/rest" - "github.com/google/uuid" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -428,18 +427,21 @@ func (r *mediaFileRepository) FindRecentFilesByProperties(missing model.MediaFil return res.toModels(), nil } -func (r *mediaFileRepository) Search(q string, offset int, size int, options ...model.QueryOptions) (model.MediaFiles, error) { +var mediaFileSearchConfig = searchConfig{ + NaturalOrder: "media_file.rowid", + OrderBy: []string{"title"}, + MBIDFields: []string{"mbz_recording_id", "mbz_release_track_id"}, +} + +func (r *mediaFileRepository) Search(q string, options ...model.QueryOptions) (model.MediaFiles, error) { + var opts model.QueryOptions + if len(options) > 0 { + opts = options[0] + } var res dbMediaFiles - if uuid.Validate(q) == nil { - err := r.searchByMBID(r.selectMediaFile(options...), q, []string{"mbz_recording_id", "mbz_release_track_id"}, &res) - if err != nil { - return nil, fmt.Errorf("searching media_file by MBID %q: %w", q, err) - } - } else { - err := r.doSearch(r.selectMediaFile(options...), q, offset, size, &res, "media_file.rowid", "title") - if err != nil { - return nil, fmt.Errorf("searching media_file by query %q: %w", q, err) - } + err := r.doSearch(r.selectMediaFile(options...), q, &res, mediaFileSearchConfig, opts) + if err != nil { + return nil, fmt.Errorf("searching media_file %q: %w", q, err) } return res.toModels(), nil } diff --git a/persistence/mediafile_repository_test.go b/persistence/mediafile_repository_test.go index 853480b32..7a04d79e4 100644 --- a/persistence/mediafile_repository_test.go +++ b/persistence/mediafile_repository_test.go @@ -527,7 +527,7 @@ var _ = Describe("MediaRepository", func() { Describe("Search", func() { Context("text search", func() { It("finds media files by title", func() { - results, err := mr.Search("Antenna", 0, 10) + results, err := mr.Search("Antenna", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(3)) // songAntenna, songAntennaWithLyrics, songAntenna2 for _, result := range results { @@ -536,7 +536,7 @@ var _ = Describe("MediaRepository", func() { }) It("finds media files case insensitively", func() { - results, err := mr.Search("antenna", 0, 10) + results, err := mr.Search("antenna", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(3)) for _, result := range results { @@ -545,7 +545,7 @@ var _ = Describe("MediaRepository", func() { }) It("returns empty result when no matches found", func() { - results, err := mr.Search("nonexistent", 0, 10) + results, err := mr.Search("nonexistent", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty()) }) @@ -578,7 +578,7 @@ var _ = Describe("MediaRepository", func() { }) It("finds media file by mbz_recording_id", func() { - results, err := mr.Search("550e8400-e29b-41d4-a716-446655440020", 0, 10) + results, err := mr.Search("550e8400-e29b-41d4-a716-446655440020", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].ID).To(Equal("test-mbid-mediafile")) @@ -586,7 +586,7 @@ var _ = Describe("MediaRepository", func() { }) It("finds media file by mbz_release_track_id", func() { - results, err := mr.Search("550e8400-e29b-41d4-a716-446655440021", 0, 10) + results, err := mr.Search("550e8400-e29b-41d4-a716-446655440021", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].ID).To(Equal("test-mbid-mediafile")) @@ -594,7 +594,7 @@ var _ = Describe("MediaRepository", func() { }) It("returns empty result when MBID is not found", func() { - results, err := mr.Search("550e8400-e29b-41d4-a716-446655440099", 0, 10) + results, err := mr.Search("550e8400-e29b-41d4-a716-446655440099", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty()) }) @@ -614,7 +614,7 @@ var _ = Describe("MediaRepository", func() { Expect(err).ToNot(HaveOccurred()) // Search never returns missing media files (hardcoded behavior) - results, err := mr.Search("550e8400-e29b-41d4-a716-446655440022", 0, 10) + results, err := mr.Search("550e8400-e29b-41d4-a716-446655440022", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(BeEmpty()) diff --git a/persistence/sql_restful.go b/persistence/sql_restful.go index 27207c45d..02162387c 100644 --- a/persistence/sql_restful.go +++ b/persistence/sql_restful.go @@ -109,12 +109,10 @@ func booleanFilter(field string, value any) Sqlizer { func fullTextFilter(tableName string, mbidFields ...string) func(string, any) Sqlizer { return func(field string, value any) Sqlizer { v := strings.ToLower(value.(string)) - searchExpr := getSearchExpr() - cond := cmp.Or( + return cmp.Or[Sqlizer]( mbidExpr(tableName, v, mbidFields...), - searchExpr(tableName, v), + getSearchStrategy(tableName, v), ) - return cond } } diff --git a/persistence/sql_restful_test.go b/persistence/sql_restful_test.go index ea0f802af..32f418b8e 100644 --- a/persistence/sql_restful_test.go +++ b/persistence/sql_restful_test.go @@ -102,11 +102,11 @@ var _ = Describe("sqlRestful", func() { uuid := "550e8400-e29b-41d4-a716-446655440000" result := noMbidFilter("search", uuid) - // mbidExpr with no fields returns nil, so cmp.Or falls back to fullTextExpr - expected := squirrel.And{ - squirrel.Like{"test_table.full_text": "% 550e8400-e29b-41d4-a716-446655440000%"}, - } - Expect(result).To(Equal(expected)) + // mbidExpr with no fields returns nil, so cmp.Or falls back to search strategy + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(ContainElement("% 550e8400-e29b-41d4-a716-446655440000%")) }) }) @@ -114,26 +114,25 @@ var _ = Describe("sqlRestful", func() { It("returns full text search condition only", func() { result := filter("search", "beatles") - // mbidExpr returns nil for non-UUIDs, so fullTextExpr result is returned directly - expected := squirrel.And{ - squirrel.Like{"test_table.full_text": "% beatles%"}, - } - Expect(result).To(Equal(expected)) + // mbidExpr returns nil for non-UUIDs, so search strategy result is returned directly + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(ContainElement("% beatles%")) }) It("handles multi-word search terms", func() { result := filter("search", "the beatles abbey road") - // Should return And condition directly - andCondition, ok := result.(squirrel.And) - Expect(ok).To(BeTrue()) - Expect(andCondition).To(HaveLen(4)) - - // Check that all words are present (order may vary) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% the%"})) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% beatles%"})) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% abbey%"})) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% road%"})) + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + // All words should be present as LIKE conditions + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(HaveLen(4)) + Expect(args).To(ContainElement("% the%")) + Expect(args).To(ContainElement("% beatles%")) + Expect(args).To(ContainElement("% abbey%")) + Expect(args).To(ContainElement("% road%")) }) }) @@ -142,26 +141,48 @@ var _ = Describe("sqlRestful", func() { conf.Server.Search.FullString = false result := filter("search", "test query") - andCondition, ok := result.(squirrel.And) - Expect(ok).To(BeTrue()) - Expect(andCondition).To(HaveLen(2)) - - // Check that all words are present with leading space (order may vary) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% test%"})) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "% query%"})) + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(HaveLen(2)) + Expect(args).To(ContainElement("% test%")) + Expect(args).To(ContainElement("% query%")) }) It("uses no separator with SearchFullString=true", func() { conf.Server.Search.FullString = true result := filter("search", "test query") - andCondition, ok := result.(squirrel.And) - Expect(ok).To(BeTrue()) - Expect(andCondition).To(HaveLen(2)) + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(HaveLen(2)) + Expect(args).To(ContainElement("%test%")) + Expect(args).To(ContainElement("%query%")) + }) + }) - // Check that all words are present without leading space (order may vary) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "%test%"})) - Expect(andCondition).To(ContainElement(squirrel.Like{"test_table.full_text": "%query%"})) + Context("single-character queries (regression: must not be rejected)", func() { + It("returns valid filter for single-char query with legacy backend", func() { + conf.Server.Search.Backend = "legacy" + result := filter("search", "a") + Expect(result).ToNot(BeNil(), "single-char REST filter must not be dropped") + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("LIKE")) + Expect(args).ToNot(BeEmpty()) + }) + + It("returns valid filter for single-char query with FTS backend", func() { + conf.Server.Search.Backend = "fts" + conf.Server.Search.FullString = false + ftsFilter := fullTextFilter(tableName, mbidFields...) + result := ftsFilter("search", "a") + Expect(result).ToNot(BeNil(), "single-char REST filter must not be dropped") + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("MATCH")) + Expect(args).ToNot(BeEmpty()) }) }) @@ -179,10 +200,10 @@ var _ = Describe("sqlRestful", func() { It("handles special characters that are sanitized", func() { result := filter("search", "don't") - expected := squirrel.And{ - squirrel.Like{"test_table.full_text": "% dont%"}, // str.SanitizeStrings removes quotes - } - Expect(result).To(Equal(expected)) + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(ContainElement("% dont%")) }) It("returns nil for single quote (SQL injection protection)", func() { @@ -206,31 +227,30 @@ var _ = Describe("sqlRestful", func() { result := filter("search", "550e8400-invalid-uuid") // Should return full text filter since UUID is invalid - expected := squirrel.And{ - squirrel.Like{"test_table.full_text": "% 550e8400-invalid-uuid%"}, - } - Expect(result).To(Equal(expected)) + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(ContainElement("% 550e8400-invalid-uuid%")) }) It("handles empty mbid fields array", func() { emptyMbidFilter := fullTextFilter(tableName, []string{}...) result := emptyMbidFilter("search", "test") - // mbidExpr with empty fields returns nil, so cmp.Or falls back to fullTextExpr - expected := squirrel.And{ - squirrel.Like{"test_table.full_text": "% test%"}, - } - Expect(result).To(Equal(expected)) + // mbidExpr with empty fields returns nil, so search strategy result is returned directly + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(ContainElement("% test%")) }) It("converts value to lowercase before processing", func() { result := filter("search", "TEST") - // The function converts to lowercase internally - expected := squirrel.And{ - squirrel.Like{"test_table.full_text": "% test%"}, - } - Expect(result).To(Equal(expected)) + sql, args, err := result.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("test_table.full_text LIKE")) + Expect(args).To(ContainElement("% test%")) }) }) }) diff --git a/persistence/sql_search.go b/persistence/sql_search.go index e5c245bdf..43965ebb7 100644 --- a/persistence/sql_search.go +++ b/persistence/sql_search.go @@ -6,7 +6,6 @@ import ( . "github.com/Masterminds/squirrel" "github.com/google/uuid" "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/str" ) @@ -16,57 +15,71 @@ func formatFullText(text ...string) string { return " " + fullText } -// searchExprFunc is the function signature for search expression builders. -type searchExprFunc func(tableName string, query string) Sqlizer - -// getSearchExpr returns the active search expression function based on config. -// It falls back to legacySearchExpr when Search.FullString is enabled, because -// FTS5 is token-based and cannot match substrings within words. -// CJK queries are routed to likeSearchExpr, since FTS5's unicode61 tokenizer -// cannot segment CJK text. -func getSearchExpr() searchExprFunc { - if conf.Server.Search.Backend == "legacy" || conf.Server.Search.FullString { - return legacySearchExpr - } - return func(tableName, query string) Sqlizer { - if containsCJK(query) { - return likeSearchExpr(tableName, query) - } - return ftsSearchExpr(tableName, query) - } +// searchConfig holds per-repository constants for doSearch. +type searchConfig struct { + NaturalOrder string // ORDER BY for empty-query results (e.g. "album.rowid") + OrderBy []string // ORDER BY for text search results (e.g. ["name"]) + MBIDFields []string // columns to match when query is a UUID + // LibraryFilter overrides the default applyLibraryFilter for FTS Phase 1. + // Needed when library access requires a junction table (e.g. artist → library_artist). + LibraryFilter func(sq SelectBuilder) SelectBuilder } -// doSearch performs a full-text search with the specified parameters. -// The naturalOrder is used to sort results when no full-text filter is applied. It is useful for cases like -// OpenSubsonic, where an empty search query should return all results in a natural order. Normally the parameter -// should be `tableName + ".rowid"`, but some repositories (ex: artist) may use a different natural order. -func (r sqlRepository) doSearch(sq SelectBuilder, q string, offset, size int, results any, naturalOrder string, orderBys ...string) error { +// searchStrategy defines how to execute a text search against a repository table. +// options carries filters and pagination that must reach all query phases, +// including FTS Phase 1 which builds its own query outside sq. +type searchStrategy interface { + Sqlizer + execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error +} + +// getSearchStrategy returns the appropriate search strategy based on config and query content. +// Returns nil when the query produces no searchable tokens. +func getSearchStrategy(tableName, query string) searchStrategy { + if conf.Server.Search.Backend == "legacy" || conf.Server.Search.FullString { + return newLegacySearch(tableName, query) + } + if containsCJK(query) { + return newLikeSearch(tableName, query) + } + return newFTSSearch(tableName, query) +} + +// doSearch dispatches a search query: empty → natural order, UUID → MBID match, +// otherwise delegates to getSearchStrategy. sq must already have LIMIT/OFFSET set +// via newSelect(options...). options is forwarded so FTS Phase 1 can apply the same +// filters and pagination independently. +func (r sqlRepository) doSearch(sq SelectBuilder, q string, results any, cfg searchConfig, options model.QueryOptions) error { q = strings.TrimSpace(q) q = strings.TrimSuffix(q, "*") + + sq = sq.Where(Eq{r.tableName + ".missing": false}) + + // Empty query (OpenSubsonic `search3?query=""`) — return all in natural order. + if q == "" || q == `""` { + sq = sq.OrderBy(cfg.NaturalOrder) + return r.queryAll(sq, results, options) + } + + // MBID search: if query is a valid UUID, search by MBID fields instead + if uuid.Validate(q) == nil && len(cfg.MBIDFields) > 0 { + sq = sq.Where(mbidExpr(r.tableName, q, cfg.MBIDFields...)) + return r.queryAll(sq, results) + } + + // Min-length guard: single-character queries are too broad for search3. + // This check lives here (not in the strategies) so that fullTextFilter + // (REST filter path) can still use single-character queries. if len(q) < 2 { return nil } - searchExpr := getSearchExpr() - filter := searchExpr(r.tableName, q) - if filter != nil { - sq = sq.Where(filter) - sq = sq.OrderBy(orderBys...) - } else { - // This is to speed up the results of `search3?query=""`, for OpenSubsonic - // If the filter is empty, we sort by the specified natural order. - sq = sq.OrderBy(naturalOrder) + strategy := getSearchStrategy(r.tableName, q) + if strategy == nil { + return nil } - sq = sq.Where(Eq{r.tableName + ".missing": false}) - sq = sq.Limit(uint64(size)).Offset(uint64(offset)) - return r.queryAll(sq, results, model.QueryOptions{Offset: offset}) -} -func (r sqlRepository) searchByMBID(sq SelectBuilder, mbid string, mbidFields []string, results any) error { - sq = sq.Where(mbidExpr(r.tableName, mbid, mbidFields...)) - sq = sq.Where(Eq{r.tableName + ".missing": false}) - - return r.queryAll(sq, results) + return strategy.execute(r, sq, results, cfg, options) } func mbidExpr(tableName, mbid string, mbidFields ...string) Sqlizer { @@ -80,24 +93,3 @@ func mbidExpr(tableName, mbid string, mbidFields ...string) Sqlizer { } return Or(cond) } - -// legacySearchExpr generates LIKE-based search filters against the full_text column. -// This is the original search implementation, used when Search.Backend="legacy". -func legacySearchExpr(tableName string, s string) Sqlizer { - q := str.SanitizeStrings(s) - if q == "" { - log.Trace("Search using legacy backend, query is empty", "table", tableName) - return nil - } - var sep string - if !conf.Server.Search.FullString { - sep = " " - } - parts := strings.Split(q, " ") - filters := And{} - for _, part := range parts { - filters = append(filters, Like{tableName + ".full_text": "%" + sep + part + "%"}) - } - log.Trace("Search using legacy backend", "query", filters, "table", tableName) - return filters -} diff --git a/persistence/sql_search_fts.go b/persistence/sql_search_fts.go index ea70518b6..9eb01f0cf 100644 --- a/persistence/sql_search_fts.go +++ b/persistence/sql_search_fts.go @@ -9,6 +9,7 @@ import ( . "github.com/Masterminds/squirrel" "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" ) // containsCJK returns true if the string contains any CJK (Chinese/Japanese/Korean) characters. @@ -187,75 +188,235 @@ func buildFTS5Query(userInput string) string { return result } -// likeSearchColumns defines the core columns to search with LIKE queries. -// These are the primary user-visible fields for each entity type. -// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input). -var likeSearchColumns = map[string][]string{ - "media_file": {"title", "album", "artist", "album_artist"}, - "album": {"name", "album_artist"}, - "artist": {"name"}, +// ftsColumn pairs an FTS5 column name with its BM25 relevance weight. +type ftsColumn struct { + Name string + Weight float64 } -// likeSearchExpr generates LIKE-based search filters against core columns. -// Each word in the query must match at least one column (AND between words), -// and each word can match any column (OR within a word). -// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input). -func likeSearchExpr(tableName string, s string) Sqlizer { - s = strings.TrimSpace(s) - if s == "" { - log.Trace("Search using LIKE backend, query is empty", "table", tableName) - return nil - } - columns, ok := likeSearchColumns[tableName] - if !ok { - log.Trace("Search using LIKE backend, couldn't find columns for this table", "table", tableName) - return nil - } - words := strings.Fields(s) - wordFilters := And{} - for _, word := range words { - colFilters := Or{} - for _, col := range columns { - colFilters = append(colFilters, Like{tableName + "." + col: "%" + word + "%"}) +// ftsColumnDefs defines FTS5 columns and their BM25 relevance weights. +// The order MUST match the column order in the FTS5 table definition (see migrations). +// All columns are both searched and ranked. When adding indexed-but-not-searched +// columns in the future, use Weight: 0 to exclude from the search column filter. +var ftsColumnDefs = map[string][]ftsColumn{ + "media_file": { + {"title", 10.0}, + {"album", 5.0}, + {"artist", 3.0}, + {"album_artist", 3.0}, + {"sort_title", 1.0}, + {"sort_album_name", 1.0}, + {"sort_artist_name", 1.0}, + {"sort_album_artist_name", 1.0}, + {"disc_subtitle", 1.0}, + {"search_participants", 2.0}, + {"search_normalized", 1.0}, + }, + "album": { + {"name", 10.0}, + {"sort_album_name", 1.0}, + {"album_artist", 3.0}, + {"search_participants", 2.0}, + {"discs", 1.0}, + {"catalog_num", 1.0}, + {"album_version", 1.0}, + {"search_normalized", 1.0}, + }, + "artist": { + {"name", 10.0}, + {"sort_artist_name", 1.0}, + {"search_normalized", 1.0}, + }, +} + +// ftsColumnFilters and ftsBM25Weights are precomputed from ftsColumnDefs at init time +// to avoid per-query allocations. +var ( + ftsColumnFilters = map[string]string{} + ftsBM25Weights = map[string]string{} +) + +func init() { + for table, cols := range ftsColumnDefs { + var names []string + weights := make([]string, len(cols)) + for i, c := range cols { + if c.Weight > 0 { + names = append(names, c.Name) + } + weights[i] = fmt.Sprintf("%.1f", c.Weight) } - wordFilters = append(wordFilters, colFilters) + ftsColumnFilters[table] = "{" + strings.Join(names, " ") + "}" + ftsBM25Weights[table] = strings.Join(weights, ", ") } - log.Trace("Search using LIKE backend", "query", wordFilters, "table", tableName) - return wordFilters } -// ftsSearchColumns defines which FTS5 columns are included in general search. -// Columns not listed here are indexed but not searched by default, -// enabling future additions (comments, lyrics, bios) without affecting general search. -var ftsSearchColumns = map[string]string{ - "media_file": "{title album artist album_artist sort_title sort_album_name sort_artist_name sort_album_artist_name disc_subtitle search_participants search_normalized}", - "album": "{name sort_album_name album_artist search_participants discs catalog_num album_version search_normalized}", - "artist": "{name sort_artist_name search_normalized}", +// ftsSearch implements searchStrategy using FTS5 full-text search with BM25 ranking. +type ftsSearch struct { + tableName string + ftsTable string + matchExpr string + rankExpr string } -// ftsSearchExpr generates an FTS5 MATCH-based search filter. -// If the query produces no FTS tokens (e.g., punctuation-only like "!!!!!!!"), -// it falls back to LIKE-based search. -func ftsSearchExpr(tableName string, s string) Sqlizer { - q := buildFTS5Query(s) - if q == "" { - s = strings.TrimSpace(strings.ReplaceAll(s, `"`, "")) - if s != "" { - log.Trace("Search using LIKE fallback for non-tokenizable query", "table", tableName, "query", s) - return likeSearchExpr(tableName, s) +// ToSql returns a single-query fallback for the REST filter path (no two-phase split). +func (s *ftsSearch) ToSql() (string, []interface{}, error) { + sql := s.tableName + ".rowid IN (SELECT rowid FROM " + s.ftsTable + " WHERE " + s.ftsTable + " MATCH ?)" + return sql, []interface{}{s.matchExpr}, nil +} + +// execute runs a two-phase FTS5 search: +// - Phase 1: lightweight rowid query (main table + FTS + library filter) for ranking and pagination. +// - Phase 2: full SELECT with all JOINs, scoped to Phase 1's rowid set. +// +// Complex ORDER BY (function calls, aggregations) are dropped from Phase 1. +func (s *ftsSearch) execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error { + qualifiedOrderBys := []string{s.rankExpr} + for _, ob := range cfg.OrderBy { + if qualified := qualifyOrderBy(s.tableName, ob); qualified != "" { + qualifiedOrderBys = append(qualifiedOrderBys, qualified) + } + } + + // Phase 1: fresh query — must set LIMIT/OFFSET from options explicitly. + // Mirror applyOptions behavior: Max=0 means no limit, not LIMIT 0. + rowidQuery := Select(s.tableName+".rowid"). + From(s.tableName). + Join(s.ftsTable+" ON "+s.ftsTable+".rowid = "+s.tableName+".rowid AND "+s.ftsTable+" MATCH ?", s.matchExpr). + Where(Eq{s.tableName + ".missing": false}). + OrderBy(qualifiedOrderBys...) + if options.Max > 0 { + rowidQuery = rowidQuery.Limit(uint64(options.Max)) + } + if options.Offset > 0 { + rowidQuery = rowidQuery.Offset(uint64(options.Offset)) + } + + // Library filter + musicFolderId must be applied here, before pagination. + if cfg.LibraryFilter != nil { + rowidQuery = cfg.LibraryFilter(rowidQuery) + } else { + rowidQuery = r.applyLibraryFilter(rowidQuery) + } + if options.Filters != nil { + rowidQuery = rowidQuery.Where(options.Filters) + } + + rowidSQL, rowidArgs, err := rowidQuery.ToSql() + if err != nil { + return fmt.Errorf("building FTS rowid query: %w", err) + } + + // Phase 2: strip LIMIT/OFFSET from sq (Phase 1 handled pagination), + // join on the ranked rowid set to hydrate with full columns. + sq = sq.RemoveLimit().RemoveOffset() + rankedSubquery := fmt.Sprintf( + "(SELECT rowid as _rid, row_number() OVER () AS _rn FROM (%s)) AS _ranked", + rowidSQL, + ) + sq = sq.Join(rankedSubquery+" ON "+s.tableName+".rowid = _ranked._rid", rowidArgs...) + sq = sq.OrderBy("_ranked._rn") + return r.queryAll(sq, dest) +} + +// qualifyOrderBy prepends tableName to a simple column name. Returns empty string for +// complex expressions (function calls, aggregations) that can't be used in Phase 1. +func qualifyOrderBy(tableName, orderBy string) string { + orderBy = strings.TrimSpace(orderBy) + if orderBy == "" || strings.ContainsAny(orderBy, "(,") { + return "" + } + parts := strings.Fields(orderBy) + if !strings.Contains(parts[0], ".") { + parts[0] = tableName + "." + parts[0] + } + return strings.Join(parts, " ") +} + +// ftsQueryDegraded returns true when the FTS query lost significant discriminating +// content compared to the original input. This happens when special characters that +// are part of the entity name (e.g., "1+", "C++", "!!!", "C#") get stripped by FTS +// tokenization, leaving only very short/broad tokens. Also detects quoted phrases +// that would be degraded by FTS5's unicode61 tokenizer (e.g., "1+" → token "1"). +func ftsQueryDegraded(original, ftsQuery string) bool { + original = strings.TrimSpace(original) + if original == "" || ftsQuery == "" { + return false + } + // Strip quotes from original for comparison — we want the raw content + stripped := strings.ReplaceAll(original, `"`, "") + // Extract the alphanumeric content from the original query + alphaNum := fts5PunctStrip.ReplaceAllString(stripped, "") + // If the original is entirely alphanumeric, nothing was stripped — not degraded + if len(alphaNum) == len(stripped) { + return false + } + // Check if all effective FTS tokens are very short (≤2 chars). + // Short tokens with prefix matching are too broad when special chars were stripped. + // For quoted phrases, extract the content and check the tokens inside. + tokens := strings.Fields(ftsQuery) + for _, t := range tokens { + t = strings.TrimSuffix(t, "*") + // Skip internal phrase placeholders + if strings.HasPrefix(t, "\x00") { + return false + } + // For OR groups from processPunctuatedWords (e.g., ("a ha" OR aha*)), + // the punctuated word was already handled meaningfully — not degraded. + if strings.HasPrefix(t, "(") { + return false + } + // For quoted phrases, check the tokens inside as FTS5 will tokenize them + if strings.HasPrefix(t, `"`) { + // Extract content between quotes + inner := strings.Trim(t, `"`) + innerAlpha := fts5PunctStrip.ReplaceAllString(inner, " ") + for _, it := range strings.Fields(innerAlpha) { + if len(it) > 2 { + return false + } + } + continue + } + if len(t) > 2 { + return false + } + } + return true +} + +// newFTSSearch creates an FTS5 search strategy. Falls back to LIKE search if the +// query produces no FTS tokens (e.g., punctuation-only like "!!!!!!!") or if FTS +// tokenization stripped significant content from the query (e.g., "1+" → "1*"). +// Returns nil when the query produces no searchable tokens at all. +func newFTSSearch(tableName, query string) searchStrategy { + q := buildFTS5Query(query) + if q == "" || ftsQueryDegraded(query, q) { + // Fallback: try LIKE search with the raw query + cleaned := strings.TrimSpace(strings.ReplaceAll(query, `"`, "")) + if cleaned != "" { + log.Trace("Search using LIKE fallback for non-tokenizable query", "table", tableName, "query", cleaned) + return newLikeSearch(tableName, cleaned) } return nil } ftsTable := tableName + "_fts" matchExpr := q - if cols, ok := ftsSearchColumns[tableName]; ok { + if cols, ok := ftsColumnFilters[tableName]; ok { matchExpr = cols + " : (" + q + ")" } - filter := Expr( - tableName+".rowid IN (SELECT rowid FROM "+ftsTable+" WHERE "+ftsTable+" MATCH ?)", - matchExpr, - ) - log.Trace("Search using FTS5 backend", "table", tableName, "query", q, "filter", filter) - return filter + rankExpr := ftsTable + ".rank" + if weights, ok := ftsBM25Weights[tableName]; ok { + rankExpr = "bm25(" + ftsTable + ", " + weights + ")" + } + + s := &ftsSearch{ + tableName: tableName, + ftsTable: ftsTable, + matchExpr: matchExpr, + rankExpr: rankExpr, + } + log.Trace("Search using FTS5 backend", "table", tableName, "query", q, "filter", s) + return s } diff --git a/persistence/sql_search_fts_test.go b/persistence/sql_search_fts_test.go index e0fead8ae..337d54201 100644 --- a/persistence/sql_search_fts_test.go +++ b/persistence/sql_search_fts_test.go @@ -3,8 +3,6 @@ package persistence import ( "context" - "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/request" @@ -52,6 +50,23 @@ var _ = DescribeTable("buildFTS5Query", Entry("returns empty string for empty quoted phrase", `""`, ""), ) +var _ = DescribeTable("ftsQueryDegraded", + func(original, ftsQuery string, expected bool) { + Expect(ftsQueryDegraded(original, ftsQuery)).To(Equal(expected)) + }, + Entry("not degraded for empty original", "", "1*", false), + Entry("not degraded for empty ftsQuery", "1+", "", false), + Entry("not degraded for purely alphanumeric query", "beatles", "beatles*", false), + Entry("not degraded when long tokens remain", "test^val", "test* val*", false), + Entry("not degraded for quoted phrase with long tokens", `"the beatles"`, `"the beatles"`, false), + Entry("degraded for quoted phrase with only short tokens after tokenizer strips special chars", `"1+"`, `"1+"`, true), + Entry("not degraded for quoted phrase with meaningful content", `"C++ programming"`, `"C++ programming"`, false), + Entry("degraded when special chars stripped leaving short token", "1+", "1*", true), + Entry("degraded when special chars stripped leaving two short tokens", "C# 1", "C* 1*", true), + Entry("not degraded when at least one long token remains", "1+ beatles", "1* beatles*", false), + Entry("not degraded for OR groups from processPunctuatedWords", "AC/DC", `("AC DC" OR ACDC*)`, false), +) + var _ = DescribeTable("normalizeForFTS", func(expected string, values ...string) { Expect(normalizeForFTS(values...)).To(Equal(expected)) @@ -81,133 +96,211 @@ var _ = DescribeTable("containsCJK", Entry("detects single CJK character", "a曲b", true), ) -var _ = Describe("likeSearchExpr", func() { - It("returns nil for empty query", func() { - Expect(likeSearchExpr("media_file", "")).To(BeNil()) +var _ = DescribeTable("qualifyOrderBy", + func(tableName, orderBy, expected string) { + Expect(qualifyOrderBy(tableName, orderBy)).To(Equal(expected)) + }, + Entry("returns empty string for empty input", "artist", "", ""), + Entry("qualifies simple column with table name", "artist", "name", "artist.name"), + Entry("qualifies column with direction", "artist", "name desc", "artist.name desc"), + Entry("preserves already-qualified column", "artist", "artist.name", "artist.name"), + Entry("preserves already-qualified column with direction", "artist", "artist.name desc", "artist.name desc"), + Entry("returns empty for function call expression", "artist", "sum(json_extract(stats, '$.total.m')) desc", ""), + Entry("returns empty for expression with comma", "artist", "a, b", ""), + Entry("qualifies media_file column", "media_file", "title", "media_file.title"), +) + +var _ = Describe("ftsColumnDefs helpers", func() { + Describe("ftsColumnFilters", func() { + It("returns column filter for media_file", func() { + Expect(ftsColumnFilters).To(HaveKeyWithValue("media_file", + "{title album artist album_artist sort_title sort_album_name sort_artist_name sort_album_artist_name disc_subtitle search_participants search_normalized}", + )) + }) + + It("returns column filter for album", func() { + Expect(ftsColumnFilters).To(HaveKeyWithValue("album", + "{name sort_album_name album_artist search_participants discs catalog_num album_version search_normalized}", + )) + }) + + It("returns column filter for artist", func() { + Expect(ftsColumnFilters).To(HaveKeyWithValue("artist", + "{name sort_artist_name search_normalized}", + )) + }) + + It("has no entry for unknown table", func() { + Expect(ftsColumnFilters).ToNot(HaveKey("unknown")) + }) }) - It("returns nil for whitespace-only query", func() { - Expect(likeSearchExpr("media_file", " ")).To(BeNil()) + Describe("ftsBM25Weights", func() { + It("returns weight CSV for media_file", func() { + Expect(ftsBM25Weights).To(HaveKeyWithValue("media_file", + "10.0, 5.0, 3.0, 3.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0", + )) + }) + + It("returns weight CSV for album", func() { + Expect(ftsBM25Weights).To(HaveKeyWithValue("album", + "10.0, 1.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0", + )) + }) + + It("returns weight CSV for artist", func() { + Expect(ftsBM25Weights).To(HaveKeyWithValue("artist", + "10.0, 1.0, 1.0", + )) + }) + + It("has no entry for unknown table", func() { + Expect(ftsBM25Weights).ToNot(HaveKey("unknown")) + }) }) - It("generates LIKE filters against core columns for single CJK word", func() { - expr := likeSearchExpr("media_file", "周杰伦") - sql, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - // Should have OR between columns for the single word - Expect(sql).To(ContainSubstring("OR")) - Expect(sql).To(ContainSubstring("media_file.title LIKE")) - Expect(sql).To(ContainSubstring("media_file.album LIKE")) - Expect(sql).To(ContainSubstring("media_file.artist LIKE")) - Expect(sql).To(ContainSubstring("media_file.album_artist LIKE")) - Expect(args).To(HaveLen(4)) - for _, arg := range args { - Expect(arg).To(Equal("%周杰伦%")) + It("has definitions for all known tables", func() { + for _, table := range []string{"media_file", "album", "artist"} { + Expect(ftsColumnDefs).To(HaveKey(table)) + Expect(ftsColumnDefs[table]).ToNot(BeEmpty()) } }) - It("generates AND of OR groups for multi-word query", func() { - expr := likeSearchExpr("media_file", "周杰伦 greatest") - sql, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - // Two groups AND'd together, each with 4 columns OR'd - Expect(sql).To(ContainSubstring("AND")) - Expect(args).To(HaveLen(8)) - }) - - It("uses correct columns for album table", func() { - expr := likeSearchExpr("album", "周杰伦") - sql, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(sql).To(ContainSubstring("album.name LIKE")) - Expect(sql).To(ContainSubstring("album.album_artist LIKE")) - Expect(args).To(HaveLen(2)) - }) - - It("uses correct columns for artist table", func() { - expr := likeSearchExpr("artist", "周杰伦") - sql, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(sql).To(ContainSubstring("artist.name LIKE")) - Expect(args).To(HaveLen(1)) - }) - - It("returns nil for unknown table", func() { - Expect(likeSearchExpr("unknown_table", "周杰伦")).To(BeNil()) + It("has matching column count between filter and weights", func() { + for table, cols := range ftsColumnDefs { + // Column filter only includes Weight > 0 columns + filterCount := 0 + for _, c := range cols { + if c.Weight > 0 { + filterCount++ + } + } + // For now, all columns have Weight > 0, so filter count == total count + Expect(filterCount).To(Equal(len(cols)), "table %s: all columns should have positive weights", table) + } }) }) -var _ = Describe("ftsSearchExpr", func() { +var _ = Describe("newFTSSearch", func() { It("returns nil for empty query", func() { - Expect(ftsSearchExpr("media_file", "")).To(BeNil()) + Expect(newFTSSearch("media_file", "")).To(BeNil()) }) - It("generates rowid IN subquery with MATCH and column filter", func() { - expr := ftsSearchExpr("media_file", "beatles") - sql, args, err := expr.ToSql() + It("returns non-nil for single-character query", func() { + strategy := newFTSSearch("media_file", "a") + Expect(strategy).ToNot(BeNil(), "single-char queries must not be rejected; min-length is enforced in doSearch, not here") + sql, _, err := strategy.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("MATCH")) + }) + + It("returns ftsSearch with correct table names and MATCH expression", func() { + strategy := newFTSSearch("media_file", "beatles") + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.tableName).To(Equal("media_file")) + Expect(fts.ftsTable).To(Equal("media_file_fts")) + Expect(fts.matchExpr).To(HavePrefix("{title album artist album_artist")) + Expect(fts.matchExpr).To(ContainSubstring("beatles*")) + }) + + It("ToSql generates rowid IN subquery with MATCH (fallback path)", func() { + strategy := newFTSSearch("media_file", "beatles") + sql, args, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) Expect(sql).To(ContainSubstring("media_file.rowid IN")) Expect(sql).To(ContainSubstring("media_file_fts")) Expect(sql).To(ContainSubstring("MATCH")) Expect(args).To(HaveLen(1)) - Expect(args[0]).To(HavePrefix("{title album artist album_artist")) - Expect(args[0]).To(ContainSubstring("beatles*")) }) It("generates correct FTS table name per entity", func() { for _, table := range []string{"media_file", "album", "artist"} { - expr := ftsSearchExpr(table, "test") - sql, _, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(sql).To(ContainSubstring(table + ".rowid IN")) - Expect(sql).To(ContainSubstring(table + "_fts")) + strategy := newFTSSearch(table, "test") + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.tableName).To(Equal(table)) + Expect(fts.ftsTable).To(Equal(table + "_fts")) } }) + It("builds bm25() rank expression with column weights", func() { + strategy := newFTSSearch("media_file", "beatles") + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.rankExpr).To(HavePrefix("bm25(media_file_fts,")) + Expect(fts.rankExpr).To(ContainSubstring("10.0")) + + strategy = newFTSSearch("artist", "beatles") + fts, ok = strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.rankExpr).To(HavePrefix("bm25(artist_fts,")) + }) + + It("falls back to ftsTable.rank for unknown tables", func() { + strategy := newFTSSearch("unknown_table", "test") + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.rankExpr).To(Equal("unknown_table_fts.rank")) + }) + It("wraps query with column filter for known tables", func() { - expr := ftsSearchExpr("artist", "Beatles") - _, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(args[0]).To(Equal("{name sort_artist_name search_normalized} : (Beatles*)")) + strategy := newFTSSearch("artist", "Beatles") + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.matchExpr).To(Equal("{name sort_artist_name search_normalized} : (Beatles*)")) }) It("passes query without column filter for unknown tables", func() { - expr := ftsSearchExpr("unknown_table", "test") - _, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(args[0]).To(Equal("test*")) + strategy := newFTSSearch("unknown_table", "test") + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.matchExpr).To(Equal("test*")) }) It("preserves phrase queries inside column filter", func() { - expr := ftsSearchExpr("media_file", `"the beatles"`) - _, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(args[0]).To(ContainSubstring(`"the beatles"`)) + strategy := newFTSSearch("media_file", `"the beatles"`) + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.matchExpr).To(ContainSubstring(`"the beatles"`)) }) It("preserves prefix queries inside column filter", func() { - expr := ftsSearchExpr("media_file", "beat*") - _, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(args[0]).To(ContainSubstring("beat*")) + strategy := newFTSSearch("media_file", "beat*") + fts, ok := strategy.(*ftsSearch) + Expect(ok).To(BeTrue()) + Expect(fts.matchExpr).To(ContainSubstring("beat*")) }) It("falls back to LIKE search for punctuation-only query", func() { - expr := ftsSearchExpr("media_file", "!!!!!!!") - Expect(expr).ToNot(BeNil()) - sql, args, err := expr.ToSql() + strategy := newFTSSearch("media_file", "!!!!!!!") + Expect(strategy).ToNot(BeNil()) + _, ok := strategy.(*ftsSearch) + Expect(ok).To(BeFalse(), "punctuation-only should fall back to LIKE, not FTS") + sql, args, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) Expect(sql).To(ContainSubstring("LIKE")) Expect(args).To(ContainElement("%!!!!!!!%")) }) + It("falls back to LIKE search for degraded query (special chars stripped leaving short tokens)", func() { + strategy := newFTSSearch("album", "1+") + Expect(strategy).ToNot(BeNil()) + _, ok := strategy.(*ftsSearch) + Expect(ok).To(BeFalse(), "degraded query should fall back to LIKE, not FTS") + sql, args, err := strategy.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("LIKE")) + Expect(args).To(ContainElement("%1+%")) + }) + It("returns nil for empty string even with LIKE fallback", func() { - Expect(ftsSearchExpr("media_file", "")).To(BeNil()) - Expect(ftsSearchExpr("media_file", " ")).To(BeNil()) + Expect(newFTSSearch("media_file", "")).To(BeNil()) + Expect(newFTSSearch("media_file", " ")).To(BeNil()) }) It("returns nil for empty quoted phrase", func() { - Expect(ftsSearchExpr("media_file", `""`)).To(BeNil()) + Expect(newFTSSearch("media_file", `""`)).To(BeNil()) }) }) @@ -229,7 +322,7 @@ var _ = Describe("FTS5 Integration Search", func() { Describe("MediaFile search", func() { It("finds media files by title", func() { - results, err := mr.Search("Radioactivity", 0, 10) + results, err := mr.Search("Radioactivity", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Title).To(Equal("Radioactivity")) @@ -237,7 +330,7 @@ var _ = Describe("FTS5 Integration Search", func() { }) It("finds media files by artist name", func() { - results, err := mr.Search("Beatles", 0, 10) + results, err := mr.Search("Beatles", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(3)) for _, r := range results { @@ -248,7 +341,7 @@ var _ = Describe("FTS5 Integration Search", func() { Describe("Album search", func() { It("finds albums by name", func() { - results, err := alr.Search("Sgt Peppers", 0, 10) + results, err := alr.Search("Sgt Peppers", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Name).To(Equal("Sgt Peppers")) @@ -256,7 +349,7 @@ var _ = Describe("FTS5 Integration Search", func() { }) It("finds albums with multi-word search", func() { - results, err := alr.Search("Abbey Road", 0, 10) + results, err := alr.Search("Abbey Road", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(2)) }) @@ -264,7 +357,7 @@ var _ = Describe("FTS5 Integration Search", func() { Describe("Artist search", func() { It("finds artists by name", func() { - results, err := arr.Search("Kraftwerk", 0, 10) + results, err := arr.Search("Kraftwerk", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Name).To(Equal("Kraftwerk")) @@ -274,7 +367,7 @@ var _ = Describe("FTS5 Integration Search", func() { Describe("CJK search", func() { It("finds media files by CJK title", func() { - results, err := mr.Search("プラチナ", 0, 10) + results, err := mr.Search("プラチナ", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Title).To(Equal("プラチナ・ジェット")) @@ -282,14 +375,14 @@ var _ = Describe("FTS5 Integration Search", func() { }) It("finds media files by CJK artist name", func() { - results, err := mr.Search("シートベルツ", 0, 10) + results, err := mr.Search("シートベルツ", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Artist).To(Equal("シートベルツ")) }) It("finds albums by CJK artist name", func() { - results, err := alr.Search("シートベルツ", 0, 10) + results, err := alr.Search("シートベルツ", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Name).To(Equal("COWBOY BEBOP")) @@ -297,7 +390,7 @@ var _ = Describe("FTS5 Integration Search", func() { }) It("finds artists by CJK name", func() { - results, err := arr.Search("シートベルツ", 0, 10) + results, err := arr.Search("シートベルツ", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Name).To(Equal("シートベルツ")) @@ -307,7 +400,7 @@ var _ = Describe("FTS5 Integration Search", func() { Describe("Album version search", func() { It("finds albums by version tag via FTS", func() { - results, err := alr.Search("Deluxe", 0, 10) + results, err := alr.Search("Deluxe", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].ID).To(Equal(albumWithVersion.ID)) @@ -316,7 +409,7 @@ var _ = Describe("FTS5 Integration Search", func() { Describe("Punctuation-only search", func() { It("finds media files with punctuation-only title", func() { - results, err := mr.Search("!!!!!!!", 0, 10) + results, err := mr.Search("!!!!!!!", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) Expect(results).To(HaveLen(1)) Expect(results[0].Title).To(Equal("!!!!!!!")) @@ -324,15 +417,19 @@ var _ = Describe("FTS5 Integration Search", func() { }) }) - Describe("Legacy backend fallback", func() { - It("returns results using legacy LIKE-based search when configured", func() { - DeferCleanup(configtest.SetupConfig()) - conf.Server.Search.Backend = "legacy" - - results, err := mr.Search("Radioactivity", 0, 10) + Describe("Single-character search (doSearch min-length guard)", func() { + It("returns empty results for single-char query via Search", func() { + results, err := mr.Search("a", model.QueryOptions{Max: 10}) Expect(err).ToNot(HaveOccurred()) - Expect(results).To(HaveLen(1)) - Expect(results[0].Title).To(Equal("Radioactivity")) + Expect(results).To(BeEmpty(), "doSearch should reject single-char queries") + }) + }) + + Describe("Max=0 means no limit (regression: must not produce LIMIT 0)", func() { + It("returns results with Max=0", func() { + results, err := mr.Search("Beatles", model.QueryOptions{Max: 0}) + Expect(err).ToNot(HaveOccurred()) + Expect(results).ToNot(BeEmpty(), "Max=0 should mean no limit, not LIMIT 0") }) }) }) diff --git a/persistence/sql_search_like.go b/persistence/sql_search_like.go new file mode 100644 index 000000000..769a911d5 --- /dev/null +++ b/persistence/sql_search_like.go @@ -0,0 +1,106 @@ +package persistence + +import ( + "strings" + + . "github.com/Masterminds/squirrel" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/utils/str" +) + +// likeSearch implements searchStrategy using LIKE-based SQL filters. +// Used for legacy full_text searches, CJK fallback, and punctuation-only fallback. +type likeSearch struct { + filter Sqlizer +} + +func (s *likeSearch) ToSql() (string, []interface{}, error) { + return s.filter.ToSql() +} + +func (s *likeSearch) execute(r sqlRepository, sq SelectBuilder, dest any, cfg searchConfig, options model.QueryOptions) error { + sq = sq.Where(s.filter) + sq = sq.OrderBy(cfg.OrderBy...) + return r.queryAll(sq, dest, options) +} + +// newLegacySearch creates a LIKE search against the full_text column. +// Returns nil when the query produces no searchable tokens. +func newLegacySearch(tableName, query string) searchStrategy { + filter := legacySearchExpr(tableName, query) + if filter == nil { + return nil + } + return &likeSearch{filter: filter} +} + +// newLikeSearch creates a LIKE search against core entity columns (CJK, punctuation fallback). +// No minimum length is enforced, since single CJK characters are meaningful words. +// Returns nil when the query produces no searchable tokens. +func newLikeSearch(tableName, query string) searchStrategy { + filter := likeSearchExpr(tableName, query) + if filter == nil { + return nil + } + return &likeSearch{filter: filter} +} + +// legacySearchExpr generates LIKE-based search filters against the full_text column. +// This is the original search implementation, used when Search.Backend="legacy". +func legacySearchExpr(tableName string, s string) Sqlizer { + q := str.SanitizeStrings(s) + if q == "" { + log.Trace("Search using legacy backend, query is empty", "table", tableName) + return nil + } + var sep string + if !conf.Server.Search.FullString { + sep = " " + } + parts := strings.Split(q, " ") + filters := And{} + for _, part := range parts { + filters = append(filters, Like{tableName + ".full_text": "%" + sep + part + "%"}) + } + log.Trace("Search using legacy backend", "query", filters, "table", tableName) + return filters +} + +// likeSearchColumns defines the core columns to search with LIKE queries. +// These are the primary user-visible fields for each entity type. +// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input). +var likeSearchColumns = map[string][]string{ + "media_file": {"title", "album", "artist", "album_artist"}, + "album": {"name", "album_artist"}, + "artist": {"name"}, +} + +// likeSearchExpr generates LIKE-based search filters against core columns. +// Each word in the query must match at least one column (AND between words), +// and each word can match any column (OR within a word). +// Used as a fallback when FTS5 cannot handle the query (e.g., CJK text, punctuation-only input). +func likeSearchExpr(tableName string, s string) Sqlizer { + s = strings.TrimSpace(s) + if s == "" { + log.Trace("Search using LIKE backend, query is empty", "table", tableName) + return nil + } + columns, ok := likeSearchColumns[tableName] + if !ok { + log.Trace("Search using LIKE backend, couldn't find columns for this table", "table", tableName) + return nil + } + words := strings.Fields(s) + wordFilters := And{} + for _, word := range words { + colFilters := Or{} + for _, col := range columns { + colFilters = append(colFilters, Like{tableName + "." + col: "%" + word + "%"}) + } + wordFilters = append(wordFilters, colFilters) + } + log.Trace("Search using LIKE backend", "query", wordFilters, "table", tableName) + return wordFilters +} diff --git a/persistence/sql_search_like_test.go b/persistence/sql_search_like_test.go new file mode 100644 index 000000000..8ee4ef93c --- /dev/null +++ b/persistence/sql_search_like_test.go @@ -0,0 +1,134 @@ +package persistence + +import ( + "context" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("newLegacySearch", func() { + It("returns non-nil for single-character query", func() { + strategy := newLegacySearch("media_file", "a") + Expect(strategy).ToNot(BeNil(), "single-char queries must not be rejected; min-length is enforced in doSearch, not here") + sql, _, err := strategy.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("LIKE")) + }) +}) + +var _ = Describe("legacySearchExpr", func() { + It("returns nil for empty query", func() { + Expect(legacySearchExpr("media_file", "")).To(BeNil()) + }) + + It("generates LIKE filter for single word", func() { + expr := legacySearchExpr("media_file", "beatles") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("media_file.full_text LIKE")) + Expect(args).To(ContainElement("% beatles%")) + }) + + It("generates AND of LIKE filters for multiple words", func() { + expr := legacySearchExpr("media_file", "abbey road") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("AND")) + Expect(args).To(HaveLen(2)) + }) +}) + +var _ = Describe("likeSearchExpr", func() { + It("returns nil for empty query", func() { + Expect(likeSearchExpr("media_file", "")).To(BeNil()) + }) + + It("returns nil for whitespace-only query", func() { + Expect(likeSearchExpr("media_file", " ")).To(BeNil()) + }) + + It("generates LIKE filters against core columns for single CJK word", func() { + expr := likeSearchExpr("media_file", "周杰伦") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + // Should have OR between columns for the single word + Expect(sql).To(ContainSubstring("OR")) + Expect(sql).To(ContainSubstring("media_file.title LIKE")) + Expect(sql).To(ContainSubstring("media_file.album LIKE")) + Expect(sql).To(ContainSubstring("media_file.artist LIKE")) + Expect(sql).To(ContainSubstring("media_file.album_artist LIKE")) + Expect(args).To(HaveLen(4)) + for _, arg := range args { + Expect(arg).To(Equal("%周杰伦%")) + } + }) + + It("generates AND of OR groups for multi-word query", func() { + expr := likeSearchExpr("media_file", "周杰伦 greatest") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + // Two groups AND'd together, each with 4 columns OR'd + Expect(sql).To(ContainSubstring("AND")) + Expect(args).To(HaveLen(8)) + }) + + It("uses correct columns for album table", func() { + expr := likeSearchExpr("album", "周杰伦") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("album.name LIKE")) + Expect(sql).To(ContainSubstring("album.album_artist LIKE")) + Expect(args).To(HaveLen(2)) + }) + + It("uses correct columns for artist table", func() { + expr := likeSearchExpr("artist", "周杰伦") + sql, args, err := expr.ToSql() + Expect(err).ToNot(HaveOccurred()) + Expect(sql).To(ContainSubstring("artist.name LIKE")) + Expect(args).To(HaveLen(1)) + }) + + It("returns nil for unknown table", func() { + Expect(likeSearchExpr("unknown_table", "周杰伦")).To(BeNil()) + }) +}) + +var _ = Describe("Legacy Integration Search", func() { + var mr model.MediaFileRepository + + BeforeEach(func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" + + ctx := log.NewContext(context.TODO()) + ctx = request.WithUser(ctx, adminUser) + conn := GetDBXBuilder() + mr = NewMediaFileRepository(ctx, conn) + }) + + It("returns results using legacy LIKE-based search", func() { + results, err := mr.Search("Radioactivity", model.QueryOptions{Max: 10}) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(HaveLen(1)) + Expect(results[0].Title).To(Equal("Radioactivity")) + }) + + It("returns empty results for single-char query (doSearch min-length guard)", func() { + results, err := mr.Search("a", model.QueryOptions{Max: 10}) + Expect(err).ToNot(HaveOccurred()) + Expect(results).To(BeEmpty(), "doSearch should reject single-char queries") + }) + + It("returns results with Max=0 (regression: must not produce LIMIT 0)", func() { + results, err := mr.Search("Beatles", model.QueryOptions{Max: 0}) + Expect(err).ToNot(HaveOccurred()) + Expect(results).ToNot(BeEmpty(), "Max=0 should mean no limit, not LIMIT 0") + }) +}) diff --git a/persistence/sql_search_test.go b/persistence/sql_search_test.go index b59570af3..68ee205b4 100644 --- a/persistence/sql_search_test.go +++ b/persistence/sql_search_test.go @@ -14,98 +14,99 @@ var _ = Describe("sqlRepository", func() { }) }) - Describe("legacySearchExpr", func() { - It("returns nil for empty query", func() { - Expect(legacySearchExpr("media_file", "")).To(BeNil()) - }) - - It("generates LIKE filter for single word", func() { - expr := legacySearchExpr("media_file", "beatles") - sql, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(sql).To(ContainSubstring("media_file.full_text LIKE")) - Expect(args).To(ContainElement("% beatles%")) - }) - - It("generates AND of LIKE filters for multiple words", func() { - expr := legacySearchExpr("media_file", "abbey road") - sql, args, err := expr.ToSql() - Expect(err).ToNot(HaveOccurred()) - Expect(sql).To(ContainSubstring("AND")) - Expect(args).To(HaveLen(2)) - }) - }) - - Describe("getSearchExpr", func() { - It("returns ftsSearchExpr by default", func() { + Describe("getSearchStrategy", func() { + It("returns FTS strategy by default", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.Search.Backend = "fts" conf.Server.Search.FullString = false - expr := getSearchExpr()("media_file", "test") - sql, _, err := expr.ToSql() + strategy := getSearchStrategy("media_file", "test") + Expect(strategy).ToNot(BeNil()) + sql, _, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) Expect(sql).To(ContainSubstring("MATCH")) }) - It("returns legacySearchExpr when SearchBackend is legacy", func() { + It("returns legacy LIKE strategy when SearchBackend is legacy", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.Search.Backend = "legacy" conf.Server.Search.FullString = false - expr := getSearchExpr()("media_file", "test") - sql, _, err := expr.ToSql() + strategy := getSearchStrategy("media_file", "test") + Expect(strategy).ToNot(BeNil()) + sql, _, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) Expect(sql).To(ContainSubstring("LIKE")) }) - It("falls back to legacySearchExpr when SearchFullString is enabled", func() { + It("falls back to legacy LIKE strategy when SearchFullString is enabled", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.Search.Backend = "fts" conf.Server.Search.FullString = true - expr := getSearchExpr()("media_file", "test") - sql, _, err := expr.ToSql() + strategy := getSearchStrategy("media_file", "test") + Expect(strategy).ToNot(BeNil()) + sql, _, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) Expect(sql).To(ContainSubstring("LIKE")) }) - It("routes CJK queries to likeSearchExpr instead of ftsSearchExpr", func() { + It("routes CJK queries to LIKE strategy instead of FTS", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.Search.Backend = "fts" conf.Server.Search.FullString = false - expr := getSearchExpr()("media_file", "周杰伦") - sql, _, err := expr.ToSql() + strategy := getSearchStrategy("media_file", "周杰伦") + Expect(strategy).ToNot(BeNil()) + sql, _, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) // CJK should use LIKE, not MATCH Expect(sql).To(ContainSubstring("LIKE")) Expect(sql).NotTo(ContainSubstring("MATCH")) }) - It("routes non-CJK queries to ftsSearchExpr", func() { + It("routes non-CJK queries to FTS strategy", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.Search.Backend = "fts" conf.Server.Search.FullString = false - expr := getSearchExpr()("media_file", "beatles") - sql, _, err := expr.ToSql() + strategy := getSearchStrategy("media_file", "beatles") + Expect(strategy).ToNot(BeNil()) + sql, _, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) Expect(sql).To(ContainSubstring("MATCH")) }) + It("returns non-nil for single-character query (no min-length in strategy)", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "fts" + conf.Server.Search.FullString = false + + strategy := getSearchStrategy("media_file", "a") + Expect(strategy).ToNot(BeNil(), "single-char queries must be accepted by strategies (min-length is enforced in doSearch)") + }) + + It("returns non-nil for single-character query with legacy backend", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" + conf.Server.Search.FullString = false + + strategy := getSearchStrategy("media_file", "a") + Expect(strategy).ToNot(BeNil(), "single-char queries must be accepted by legacy strategy (min-length is enforced in doSearch)") + }) + It("uses legacy for CJK when SearchBackend is legacy", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.Search.Backend = "legacy" conf.Server.Search.FullString = false - expr := getSearchExpr()("media_file", "周杰伦") - sql, _, err := expr.ToSql() + strategy := getSearchStrategy("media_file", "周杰伦") + Expect(strategy).ToNot(BeNil()) + sql, _, err := strategy.ToSql() Expect(err).ToNot(HaveOccurred()) // Legacy should still use full_text column LIKE Expect(sql).To(ContainSubstring("LIKE")) Expect(sql).To(ContainSubstring("full_text")) }) }) - }) diff --git a/server/e2e/e2e_suite_test.go b/server/e2e/e2e_suite_test.go index 5981f4a86..0e0ca606a 100644 --- a/server/e2e/e2e_suite_test.go +++ b/server/e2e/e2e_suite_test.go @@ -115,6 +115,7 @@ func buildTestFS() storagetest.FakeFS { ledZepIV := template(_t{"albumartist": "Led Zeppelin", "artist": "Led Zeppelin", "album": "IV", "year": 1971, "genre": "Rock"}) kindOfBlue := template(_t{"albumartist": "Miles Davis", "artist": "Miles Davis", "album": "Kind of Blue", "year": 1959, "genre": "Jazz"}) popTrack := template(_t{"albumartist": "Various", "artist": "Various", "album": "Pop", "year": 2020, "genre": "Pop"}) + cowboyBebop := template(_t{"albumartist": "シートベルツ", "artist": "シートベルツ", "album": "COWBOY BEBOP", "year": 1998, "genre": "Jazz"}) return createFS(fstest.MapFS{ // Rock / The Beatles / Abbey Road (with MBIDs) @@ -132,6 +133,8 @@ func buildTestFS() storagetest.FakeFS { "Jazz/Miles Davis/Kind of Blue/01 - So What.mp3": kindOfBlue(track(1, "So What")), // Pop (standalone track, no MBIDs) "Pop/01 - Standalone Track.mp3": popTrack(track(1, "Standalone Track")), + // CJK / シートベルツ / COWBOY BEBOP (Japanese artist, for CJK search tests) + "CJK/シートベルツ/COWBOY BEBOP/01 - プラチナ・ジェット.mp3": cowboyBebop(track(1, "プラチナ・ジェット")), // _empty folder (directory with no audio) "_empty/.keep": &fstest.MapFile{Data: []byte{}, ModTime: time.Now()}, }) diff --git a/server/e2e/subsonic_album_lists_test.go b/server/e2e/subsonic_album_lists_test.go index f7a5af173..d3d24a7c5 100644 --- a/server/e2e/subsonic_album_lists_test.go +++ b/server/e2e/subsonic_album_lists_test.go @@ -19,7 +19,7 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.AlbumList).ToNot(BeNil()) - Expect(resp.AlbumList.Album).To(HaveLen(5)) + Expect(resp.AlbumList.Album).To(HaveLen(6)) }) It("type=alphabeticalByName sorts albums by name", func() { @@ -27,13 +27,14 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.AlbumList).ToNot(BeNil()) albums := resp.AlbumList.Album - Expect(albums).To(HaveLen(5)) - // Verify alphabetical order: Abbey Road, Help!, IV, Kind of Blue, Pop + Expect(albums).To(HaveLen(6)) + // Verify alphabetical order: Abbey Road, COWBOY BEBOP, Help!, IV, Kind of Blue, Pop Expect(albums[0].Title).To(Equal("Abbey Road")) - Expect(albums[1].Title).To(Equal("Help!")) - Expect(albums[2].Title).To(Equal("IV")) - Expect(albums[3].Title).To(Equal("Kind of Blue")) - Expect(albums[4].Title).To(Equal("Pop")) + Expect(albums[1].Title).To(Equal("COWBOY BEBOP")) + Expect(albums[2].Title).To(Equal("Help!")) + Expect(albums[3].Title).To(Equal("IV")) + Expect(albums[4].Title).To(Equal("Kind of Blue")) + Expect(albums[5].Title).To(Equal("Pop")) }) It("type=alphabeticalByArtist sorts albums by artist name", func() { @@ -41,29 +42,32 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.AlbumList).ToNot(BeNil()) albums := resp.AlbumList.Album - Expect(albums).To(HaveLen(5)) + Expect(albums).To(HaveLen(6)) // Articles like "The" are stripped for sorting, so "The Beatles" sorts as "Beatles" - // Non-compilations first: Beatles (x2), Led Zeppelin, Miles Davis, then compilations: Various + // Non-compilations first: Beatles (x2), Led Zeppelin, Miles Davis, then compilations: Various, then CJK: シートベルツ Expect(albums[0].Artist).To(Equal("The Beatles")) Expect(albums[1].Artist).To(Equal("The Beatles")) Expect(albums[2].Artist).To(Equal("Led Zeppelin")) Expect(albums[3].Artist).To(Equal("Miles Davis")) Expect(albums[4].Artist).To(Equal("Various")) + Expect(albums[5].Artist).To(Equal("シートベルツ")) }) It("type=random returns albums", func() { resp := doReq("getAlbumList", "type", "random") Expect(resp.AlbumList).ToNot(BeNil()) - Expect(resp.AlbumList.Album).To(HaveLen(5)) + Expect(resp.AlbumList.Album).To(HaveLen(6)) }) It("type=byGenre filters by genre parameter", func() { resp := doReq("getAlbumList", "type", "byGenre", "genre", "Jazz") Expect(resp.AlbumList).ToNot(BeNil()) - Expect(resp.AlbumList.Album).To(HaveLen(1)) - Expect(resp.AlbumList.Album[0].Title).To(Equal("Kind of Blue")) + Expect(resp.AlbumList.Album).To(HaveLen(2)) + for _, a := range resp.AlbumList.Album { + Expect(a.Genre).To(Equal("Jazz")) + } }) It("type=byYear filters by fromYear/toYear range", func() { @@ -184,7 +188,7 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.AlbumList2).ToNot(BeNil()) albums := resp.AlbumList2.Album - Expect(albums).To(HaveLen(5)) + Expect(albums).To(HaveLen(6)) // Verify AlbumID3 format fields Expect(albums[0].Name).To(Equal("Abbey Road")) Expect(albums[0].Id).ToNot(BeEmpty()) @@ -195,7 +199,7 @@ var _ = Describe("Album List Endpoints", func() { resp := doReq("getAlbumList2", "type", "newest") Expect(resp.AlbumList2).ToNot(BeNil()) - Expect(resp.AlbumList2.Album).To(HaveLen(5)) + Expect(resp.AlbumList2.Album).To(HaveLen(6)) }) }) @@ -240,7 +244,7 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.RandomSongs).ToNot(BeNil()) Expect(resp.RandomSongs.Songs).ToNot(BeEmpty()) - Expect(len(resp.RandomSongs.Songs)).To(BeNumerically("<=", 6)) + Expect(resp.RandomSongs.Songs).To(HaveLen(7)) }) It("respects size parameter", func() { @@ -254,8 +258,10 @@ var _ = Describe("Album List Endpoints", func() { resp := doReq("getRandomSongs", "size", "500", "genre", "Jazz") Expect(resp.RandomSongs).ToNot(BeNil()) - Expect(resp.RandomSongs.Songs).To(HaveLen(1)) - Expect(resp.RandomSongs.Songs[0].Genre).To(Equal("Jazz")) + Expect(resp.RandomSongs.Songs).To(HaveLen(2)) + for _, s := range resp.RandomSongs.Songs { + Expect(s.Genre).To(Equal("Jazz")) + } }) }) diff --git a/server/e2e/subsonic_browsing_test.go b/server/e2e/subsonic_browsing_test.go index 403e1ba6f..5a2da8737 100644 --- a/server/e2e/subsonic_browsing_test.go +++ b/server/e2e/subsonic_browsing_test.go @@ -327,8 +327,8 @@ var _ = Describe("Browsing Endpoints", func() { } } Expect(jazzGenre).ToNot(BeNil()) - Expect(jazzGenre.SongCount).To(Equal(int32(1))) - Expect(jazzGenre.AlbumCount).To(Equal(int32(1))) + Expect(jazzGenre.SongCount).To(Equal(int32(2))) + Expect(jazzGenre.AlbumCount).To(Equal(int32(2))) }) It("reports correct song and album counts for Pop", func() { diff --git a/server/e2e/subsonic_multilibrary_test.go b/server/e2e/subsonic_multilibrary_test.go index 1eb04d41c..aa4a0c626 100644 --- a/server/e2e/subsonic_multilibrary_test.go +++ b/server/e2e/subsonic_multilibrary_test.go @@ -141,7 +141,7 @@ var _ = Describe("Multi-Library Support", Ordered, func() { resp := doReqWithUser(adminWithLibs, "getAlbumList", "type", "alphabeticalByName", "musicFolderId", fmt.Sprintf("%d", lib.ID)) Expect(resp.AlbumList).ToNot(BeNil()) - Expect(resp.AlbumList.Album).To(HaveLen(5)) + Expect(resp.AlbumList.Album).To(HaveLen(6)) for _, a := range resp.AlbumList.Album { Expect(a.Title).ToNot(Equal("Symphony No. 9")) } @@ -275,5 +275,21 @@ var _ = Describe("Multi-Library Support", Ordered, func() { Expect(artistNames).To(ContainElements("The Beatles", "Led Zeppelin", "Miles Davis")) Expect(artistNames).ToNot(ContainElement("Ludwig van Beethoven")) }) + + It("non-admin user search returns only their library's content", func() { + resp := doReqWithUser(userLib1Only, "search3", "query", "Beethoven") + + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).To(BeEmpty(), "userLib1Only should not see Beethoven (lib2)") + Expect(resp.SearchResult3.Album).To(BeEmpty()) + Expect(resp.SearchResult3.Song).To(BeEmpty()) + }) + + It("non-admin user search finds content from their library", func() { + resp := doReqWithUser(userLib1Only, "search3", "query", "Beatles") + + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).ToNot(BeEmpty(), "userLib1Only should find Beatles (lib1)") + }) }) }) diff --git a/server/e2e/subsonic_searching_test.go b/server/e2e/subsonic_searching_test.go index d0447ecd7..bfcbbc8ee 100644 --- a/server/e2e/subsonic_searching_test.go +++ b/server/e2e/subsonic_searching_test.go @@ -2,6 +2,8 @@ package e2e import ( "github.com/google/uuid" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/server/subsonic/responses" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" @@ -113,9 +115,9 @@ var _ = Describe("Search Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.SearchResult3).ToNot(BeNil()) - Expect(resp.SearchResult3.Artist).To(HaveLen(4)) - Expect(resp.SearchResult3.Album).To(HaveLen(5)) - Expect(resp.SearchResult3.Song).To(HaveLen(6)) + Expect(resp.SearchResult3.Artist).To(HaveLen(5)) + Expect(resp.SearchResult3.Album).To(HaveLen(6)) + Expect(resp.SearchResult3.Song).To(HaveLen(7)) }) It("finds across all entity types simultaneously", func() { @@ -217,5 +219,56 @@ var _ = Describe("Search Endpoints", func() { Expect(resp.SearchResult3.Song).To(BeEmpty()) }) }) + + Describe("CJK search", func() { + It("finds songs by CJK title", func() { + resp := doReq("search3", "query", "プラチナ") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Song).To(HaveLen(1)) + Expect(resp.SearchResult3.Song[0].Title).To(Equal("プラチナ・ジェット")) + }) + + It("finds artists by CJK name", func() { + resp := doReq("search3", "query", "シートベルツ") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).To(HaveLen(1)) + Expect(resp.SearchResult3.Artist[0].Name).To(Equal("シートベルツ")) + }) + + It("finds albums by CJK artist name", func() { + resp := doReq("search3", "query", "シートベルツ") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Album).To(HaveLen(1)) + Expect(resp.SearchResult3.Album[0].Name).To(Equal("COWBOY BEBOP")) + }) + }) + + Describe("Legacy backend", func() { + It("returns results using legacy LIKE-based search when configured", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.Search.Backend = "legacy" + + resp := doReq("search3", "query", "Beatles") + + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.SearchResult3).ToNot(BeNil()) + Expect(resp.SearchResult3.Artist).ToNot(BeEmpty()) + + found := false + for _, a := range resp.SearchResult3.Artist { + if a.Name == "The Beatles" { + found = true + break + } + } + Expect(found).To(BeTrue(), "expected to find artist 'The Beatles' with legacy backend") + }) + }) }) }) diff --git a/server/subsonic/searching.go b/server/subsonic/searching.go index 5a19e0a3b..fd7e29587 100644 --- a/server/subsonic/searching.go +++ b/server/subsonic/searching.go @@ -42,17 +42,17 @@ func (api *Router) getSearchParams(r *http.Request) (*searchParams, error) { return sp, nil } -type searchFunc[T any] func(q string, offset int, size int, options ...model.QueryOptions) (T, error) +type searchFunc[T any] func(q string, options ...model.QueryOptions) (T, error) -func callSearch[T any](ctx context.Context, s searchFunc[T], q string, offset, size int, result *T, options ...model.QueryOptions) func() error { +func callSearch[T any](ctx context.Context, s searchFunc[T], q string, options model.QueryOptions, result *T) func() error { return func() error { - if size == 0 { + if options.Max == 0 { return nil } typ := strings.TrimPrefix(reflect.TypeOf(*result).String(), "model.") var err error start := time.Now() - *result, err = s(q, offset, size, options...) + *result, err = s(q, options) if err != nil { log.Error(ctx, "Error searching "+typ, "query", q, "elapsed", time.Since(start), err) } else { @@ -66,27 +66,22 @@ func (api *Router) searchAll(ctx context.Context, sp *searchParams, musicFolderI start := time.Now() q := sanitize.Accents(strings.ToLower(strings.TrimSuffix(sp.query, "*"))) - // Create query options for library filtering - var options []model.QueryOptions - var artistOptions []model.QueryOptions + // Build options with offset/size/filters packed in + songOpts := model.QueryOptions{Max: sp.songCount, Offset: sp.songOffset} + albumOpts := model.QueryOptions{Max: sp.albumCount, Offset: sp.albumOffset} + artistOpts := model.QueryOptions{Max: sp.artistCount, Offset: sp.artistOffset} + if len(musicFolderIds) > 0 { - // For MediaFiles and Albums, use direct library_id filter - options = append(options, model.QueryOptions{ - Filters: Eq{"library_id": musicFolderIds}, - }) - // For Artists, use the repository's built-in library filtering mechanism - // which properly handles the library_artist table joins - // TODO Revisit library filtering in sql_base_repository.go - artistOptions = append(artistOptions, model.QueryOptions{ - Filters: Eq{"library_artist.library_id": musicFolderIds}, - }) + songOpts.Filters = Eq{"library_id": musicFolderIds} + albumOpts.Filters = Eq{"library_id": musicFolderIds} + artistOpts.Filters = Eq{"library_artist.library_id": musicFolderIds} } // Run searches in parallel g, ctx := errgroup.WithContext(ctx) - g.Go(callSearch(ctx, api.ds.MediaFile(ctx).Search, q, sp.songOffset, sp.songCount, &mediaFiles, options...)) - g.Go(callSearch(ctx, api.ds.Album(ctx).Search, q, sp.albumOffset, sp.albumCount, &albums, options...)) - g.Go(callSearch(ctx, api.ds.Artist(ctx).Search, q, sp.artistOffset, sp.artistCount, &artists, artistOptions...)) + g.Go(callSearch(ctx, api.ds.MediaFile(ctx).Search, q, songOpts, &mediaFiles)) + g.Go(callSearch(ctx, api.ds.Album(ctx).Search, q, albumOpts, &albums)) + g.Go(callSearch(ctx, api.ds.Artist(ctx).Search, q, artistOpts, &artists)) err := g.Wait() if err == nil { log.Debug(ctx, fmt.Sprintf("Search resulted in %d songs, %d albums and %d artists", diff --git a/tests/mock_album_repo.go b/tests/mock_album_repo.go index 8b5f5d9c1..3428813f6 100644 --- a/tests/mock_album_repo.go +++ b/tests/mock_album_repo.go @@ -119,7 +119,7 @@ func (m *MockAlbumRepo) UpdateExternalInfo(album *model.Album) error { return nil } -func (m *MockAlbumRepo) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Albums, error) { +func (m *MockAlbumRepo) Search(q string, options ...model.QueryOptions) (model.Albums, error) { if len(options) > 0 { m.Options = options[0] } diff --git a/tests/mock_artist_repo.go b/tests/mock_artist_repo.go index 6d4792f83..b7a6fb811 100644 --- a/tests/mock_artist_repo.go +++ b/tests/mock_artist_repo.go @@ -145,7 +145,7 @@ func (m *MockArtistRepo) GetIndex(includeMissing bool, libraryIds []int, roles . return result, nil } -func (m *MockArtistRepo) Search(q string, offset int, size int, options ...model.QueryOptions) (model.Artists, error) { +func (m *MockArtistRepo) Search(q string, options ...model.QueryOptions) (model.Artists, error) { if len(options) > 0 { m.Options = options[0] } diff --git a/tests/mock_mediafile_repo.go b/tests/mock_mediafile_repo.go index 2812fd507..1d4527c88 100644 --- a/tests/mock_mediafile_repo.go +++ b/tests/mock_mediafile_repo.go @@ -238,7 +238,7 @@ func (m *MockMediaFileRepo) NewInstance() any { return &model.MediaFile{} } -func (m *MockMediaFileRepo) Search(q string, offset int, size int, options ...model.QueryOptions) (model.MediaFiles, error) { +func (m *MockMediaFileRepo) Search(q string, options ...model.QueryOptions) (model.MediaFiles, error) { if len(options) > 0 { m.Options = options[0] } From 0c3cc8653591c677f91ba284fa7650481151976d Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 23 Feb 2026 18:16:50 -0500 Subject: [PATCH 09/60] fix(subsonic): restore `public` attribute for playlists in XML responses This was causing issues with DSub and DSub2000 Signed-off-by: Deluan --- .../Responses Playlists with data should match .XML | 4 ++-- server/subsonic/responses/responses.go | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/server/subsonic/responses/.snapshots/Responses Playlists with data should match .XML b/server/subsonic/responses/.snapshots/Responses Playlists with data should match .XML index 759f7b52d..38e0944cf 100644 --- a/server/subsonic/responses/.snapshots/Responses Playlists with data should match .XML +++ b/server/subsonic/responses/.snapshots/Responses Playlists with data should match .XML @@ -1,7 +1,7 @@ - - + + diff --git a/server/subsonic/responses/responses.go b/server/subsonic/responses/responses.go index 8c4e330b0..0fdbf1be6 100644 --- a/server/subsonic/responses/responses.go +++ b/server/subsonic/responses/responses.go @@ -303,7 +303,7 @@ type Playlist struct { Comment string `xml:"comment,attr,omitempty" json:"comment,omitempty"` SongCount int32 `xml:"songCount,attr" json:"songCount"` Duration int32 `xml:"duration,attr" json:"duration"` - Public bool `xml:"public,attr,omitempty" json:"public,omitempty"` + Public bool `xml:"public,attr" json:"public,omitempty"` Owner string `xml:"owner,attr,omitempty" json:"owner,omitempty"` Created time.Time `xml:"created,attr" json:"created"` Changed time.Time `xml:"changed,attr" json:"changed"` From d1c5e6a2f2929e445fd1b527e9c3686079f62140 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 23 Feb 2026 19:06:45 -0500 Subject: [PATCH 10/60] chore(deps): bump goreleaser/goreleaser-action in /.github/workflows (#5089) Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 6 to 7. - [Release notes](https://github.com/goreleaser/goreleaser-action/releases) - [Commits](https://github.com/goreleaser/goreleaser-action/compare/v6...v7) --- updated-dependencies: - dependency-name: goreleaser/goreleaser-action dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pipeline.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/pipeline.yml b/.github/workflows/pipeline.yml index fd8edcd1c..5fcc9526b 100644 --- a/.github/workflows/pipeline.yml +++ b/.github/workflows/pipeline.yml @@ -424,7 +424,7 @@ jobs: run: echo 'RELEASE_FLAGS=--skip=publish --snapshot' >> $GITHUB_ENV - name: Run GoReleaser - uses: goreleaser/goreleaser-action@v6 + uses: goreleaser/goreleaser-action@v7 with: version: '~> v2' args: "release --clean -f release/goreleaser.yml ${{ env.RELEASE_FLAGS }}" From 2bb13e5ff1b693c74a1586327d8a012735db7224 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 23 Feb 2026 20:28:38 -0500 Subject: [PATCH 11/60] feat(server): add ExtAuth logout URL configuration (#5074) * feat(server): add ExtAuth logout URL configuration (#4467) When external authentication (reverse proxy auth) is active, the Logout button is hidden because authentication is managed externally. Many external auth services (Authelia, Authentik, Keycloak) provide a logout URL that can terminate the session. Add `ExtAuth.LogoutURL` config option that, when set, shows the Logout button in the UI and redirects the user to the external auth provider's logout endpoint instead of the Navidrome login page. * feat(server): add validation for ExtAuth logout URL configuration * feat(server): refactor ExtAuth logout URL validation to a reusable function * fix(configuration): rename URL validation functions for consistency Signed-off-by: Deluan * fix(configuration): rename URL validation functions for consistency Signed-off-by: Deluan --------- Signed-off-by: Deluan --- conf/configuration.go | 30 +++++++++++++++++++++++++++ conf/configuration_test.go | 42 ++++++++++++++++++++++++++++++++++++++ conf/export_test.go | 2 ++ server/serve_index.go | 1 + server/serve_index_test.go | 1 + ui/src/authProvider.js | 4 ++++ ui/src/layout/UserMenu.jsx | 2 +- 7 files changed, 81 insertions(+), 1 deletion(-) diff --git a/conf/configuration.go b/conf/configuration.go index 000bffb58..555d8f587 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -250,6 +250,7 @@ type pluginsOptions struct { type extAuthOptions struct { TrustedSources string UserHeader string + LogoutURL string } type searchOptions struct { @@ -345,6 +346,7 @@ func Load(noConfigDump bool) { validateBackupSchedule, validatePlaylistsPath, validatePurgeMissingOption, + validateURL("ExtAuth.LogoutURL", Server.ExtAuth.LogoutURL), ) if err != nil { os.Exit(1) @@ -548,6 +550,33 @@ func validateSchedule(schedule, field string) (string, error) { return schedule, err } +// validateURL checks if the provided URL is valid and has either http or https scheme. +// It returns a function that can be used as a hook to validate URLs in the config. +func validateURL(optionName, optionURL string) func() error { + return func() error { + if optionURL == "" { + return nil + } + u, err := url.Parse(optionURL) + if err != nil { + log.Error(fmt.Sprintf("Invalid %s: it could not be parsed", optionName), "url", optionURL, "err", err) + return err + } + if u.Scheme != "http" && u.Scheme != "https" { + err := fmt.Errorf("invalid scheme for %s: '%s'. Only 'http' and 'https' are allowed", optionName, u.Scheme) + log.Error(err.Error()) + return err + } + // Require an absolute URL with a non-empty host and no opaque component. + if u.Host == "" || u.Opaque != "" { + err := fmt.Errorf("invalid %s: '%s'. A full http(s) URL with a non-empty host is required", optionName, optionURL) + log.Error(err.Error()) + return err + } + return nil + } +} + func normalizeSearchBackend(value string) string { v := strings.ToLower(strings.TrimSpace(value)) switch v { @@ -641,6 +670,7 @@ func setViperDefaults() { viper.SetDefault("passwordencryptionkey", "") viper.SetDefault("extauth.userheader", "Remote-User") viper.SetDefault("extauth.trustedsources", "") + viper.SetDefault("extauth.logouturl", "") viper.SetDefault("prometheus.enabled", false) viper.SetDefault("prometheus.metricspath", consts.PrometheusDefaultPath) viper.SetDefault("prometheus.password", "") diff --git a/conf/configuration_test.go b/conf/configuration_test.go index b4ed6ca2d..73fec4196 100644 --- a/conf/configuration_test.go +++ b/conf/configuration_test.go @@ -52,6 +52,48 @@ var _ = Describe("Configuration", func() { }) }) + Describe("ValidateURL", func() { + It("accepts a valid http URL", func() { + fn := conf.ValidateURL("TestOption", "http://example.com/path") + Expect(fn()).To(Succeed()) + }) + + It("accepts a valid https URL", func() { + fn := conf.ValidateURL("TestOption", "https://example.com/path") + Expect(fn()).To(Succeed()) + }) + + It("rejects a URL with no scheme", func() { + fn := conf.ValidateURL("TestOption", "example.com/path") + Expect(fn()).To(MatchError(ContainSubstring("invalid scheme"))) + }) + + It("rejects a URL with an unsupported scheme", func() { + fn := conf.ValidateURL("TestOption", "javascript://example.com/path") + Expect(fn()).To(MatchError(ContainSubstring("invalid scheme"))) + }) + + It("accepts an empty URL (optional config)", func() { + fn := conf.ValidateURL("TestOption", "") + Expect(fn()).To(Succeed()) + }) + + It("includes the option name in the error message", func() { + fn := conf.ValidateURL("MyOption", "ftp://example.com") + Expect(fn()).To(MatchError(ContainSubstring("MyOption"))) + }) + + It("rejects a URL that cannot be parsed", func() { + fn := conf.ValidateURL("TestOption", "://invalid") + Expect(fn()).To(HaveOccurred()) + }) + + It("rejects a URL without a host", func() { + fn := conf.ValidateURL("TestOption", "http:///path") + Expect(fn()).To(MatchError(ContainSubstring("non-empty host is required"))) + }) + }) + DescribeTable("NormalizeSearchBackend", func(input, expected string) { Expect(conf.NormalizeSearchBackend(input)).To(Equal(expected)) diff --git a/conf/export_test.go b/conf/export_test.go index 7344dc4ca..d1d1bb3a9 100644 --- a/conf/export_test.go +++ b/conf/export_test.go @@ -8,4 +8,6 @@ var SetViperDefaults = setViperDefaults var ParseLanguages = parseLanguages +var ValidateURL = validateURL + var NormalizeSearchBackend = normalizeSearchBackend diff --git a/server/serve_index.go b/server/serve_index.go index b5b364267..92ef47e23 100644 --- a/server/serve_index.go +++ b/server/serve_index.go @@ -76,6 +76,7 @@ func serveIndex(ds model.DataStore, fs fs.FS, shareInfo *model.Share) http.Handl "separator": string(os.PathSeparator), "enableInspect": conf.Server.Inspect.Enabled, "pluginsEnabled": conf.Server.Plugins.Enabled, + "extAuthLogoutURL": conf.Server.ExtAuth.LogoutURL, } if strings.HasPrefix(conf.Server.UILoginBackgroundURL, "/") { appConfig["loginBackgroundURL"] = path.Join(conf.Server.BasePath, conf.Server.UILoginBackgroundURL) diff --git a/server/serve_index_test.go b/server/serve_index_test.go index 9d6f480ff..e08a42643 100644 --- a/server/serve_index_test.go +++ b/server/serve_index_test.go @@ -104,6 +104,7 @@ var _ = Describe("serveIndex", func() { Entry("enableUserEditing", func() { conf.Server.EnableUserEditing = false }, "enableUserEditing", false), Entry("enableSharing", func() { conf.Server.EnableSharing = true }, "enableSharing", true), Entry("devNewEventStream", func() { conf.Server.DevNewEventStream = true }, "devNewEventStream", true), + Entry("extAuthLogoutURL", func() { conf.Server.ExtAuth.LogoutURL = "https://auth.example.com/logout" }, "extAuthLogoutURL", "https://auth.example.com/logout"), ) DescribeTable("sets other UI configuration values", diff --git a/ui/src/authProvider.js b/ui/src/authProvider.js index 4ae238eec..813a4f5b4 100644 --- a/ui/src/authProvider.js +++ b/ui/src/authProvider.js @@ -66,6 +66,10 @@ const authProvider = { logout: () => { removeItems() + if (config.extAuthLogoutURL) { + window.location.href = config.extAuthLogoutURL + return Promise.resolve(false) + } return Promise.resolve() }, diff --git a/ui/src/layout/UserMenu.jsx b/ui/src/layout/UserMenu.jsx index a5757a73c..e33185578 100644 --- a/ui/src/layout/UserMenu.jsx +++ b/ui/src/layout/UserMenu.jsx @@ -122,7 +122,7 @@ const UserMenu = (props) => { }) : null, )} - {!config.auth && logout} + {(!config.auth || !!config.extAuthLogoutURL) && logout} From 652c27690be6abcce58839465f2f34bd00b041ff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Tue, 24 Feb 2026 14:28:36 -0500 Subject: [PATCH 12/60] feat(plugins): add HTTP host service (#5095) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(httpclient): implement HttpClient service for outbound HTTP requests in plugins Signed-off-by: Deluan * feat(httpclient): enhance SSRF protection by validating host requests against private IPs Signed-off-by: Deluan * feat(httpclient): support DELETE requests with body in HttpClient service Signed-off-by: Deluan * feat(httpclient): refactor HTTP client initialization and enhance redirect handling Signed-off-by: Deluan * refactor(http): standardize naming conventions for HTTP types and methods Signed-off-by: Deluan * refactor example plugin to use host.HTTPSend for improved error management Signed-off-by: Deluan * fix(plugins): fix IPv6 SSRF bypass and wildcard host matching Fix two bugs in the plugin HTTP/WebSocket host validation: 1. extractHostname now strips IPv6 brackets when no port is present (e.g. "[::1]" → "::1"). Previously, net.SplitHostPort failed for bracketed IPv6 without a port, leaving brackets intact. This caused net.ParseIP to return nil, bypassing the private/loopback SSRF guard. 2. matchHostPattern now treats "*" as an allow-all pattern. Previously, a bare "*" only matched via exact equality, so plugins declaring requiredHosts: ["*"] (like webhook-rs) had all requests rejected. --------- Signed-off-by: Deluan --- plugins/examples/wikimedia/main.go | 51 +- plugins/host/httpclient.go | 40 ++ plugins/host/httpclient_gen.go | 88 +++ plugins/host_httpclient.go | 190 ++++++ plugins/host_httpclient_test.go | 565 ++++++++++++++++++ plugins/host_websocket.go | 5 +- plugins/host_websocket_test.go | 6 + plugins/manager_loader.go | 9 + plugins/pdk/go/host/doc.go | 1 + plugins/pdk/go/host/nd_host_httpclient.go | 87 +++ .../pdk/go/host/nd_host_httpclient_stub.go | 55 ++ plugins/pdk/python/host/nd_host_httpclient.py | 59 ++ plugins/pdk/rust/nd-pdk-host/src/lib.rs | 8 + .../pdk/rust/nd-pdk-host/src/nd_host_http.rs | 83 +++ 14 files changed, 1228 insertions(+), 19 deletions(-) create mode 100644 plugins/host/httpclient.go create mode 100644 plugins/host/httpclient_gen.go create mode 100644 plugins/host_httpclient.go create mode 100644 plugins/host_httpclient_test.go create mode 100644 plugins/pdk/go/host/nd_host_httpclient.go create mode 100644 plugins/pdk/go/host/nd_host_httpclient_stub.go create mode 100644 plugins/pdk/python/host/nd_host_httpclient.py create mode 100644 plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs diff --git a/plugins/examples/wikimedia/main.go b/plugins/examples/wikimedia/main.go index 6f56d4221..8508354cf 100644 --- a/plugins/examples/wikimedia/main.go +++ b/plugins/examples/wikimedia/main.go @@ -14,6 +14,7 @@ import ( "net/url" "strings" + "github.com/navidrome/navidrome/plugins/pdk/go/host" "github.com/navidrome/navidrome/plugins/pdk/go/metadata" "github.com/navidrome/navidrome/plugins/pdk/go/pdk" ) @@ -77,21 +78,28 @@ func sparqlQuery(endpoint, query string) (*SPARQLResult, error) { form := url.Values{} form.Set("query", query) - req := pdk.NewHTTPRequest(pdk.MethodPost, endpoint) - req.SetHeader("Accept", "application/sparql-results+json") - req.SetHeader("Content-Type", "application/x-www-form-urlencoded") - req.SetHeader("User-Agent", "NavidromeWikimediaPlugin/1.0") - req.SetBody([]byte(form.Encode())) - pdk.Log(pdk.LogDebug, fmt.Sprintf("SPARQL query to %s: %s", endpoint, query)) - resp := req.Send() - if resp.Status() != 200 { - return nil, fmt.Errorf("SPARQL HTTP error: status %d", resp.Status()) + resp, err := host.HTTPSend(host.HTTPRequest{ + Method: "POST", + URL: endpoint, + Headers: map[string]string{ + "Accept": "application/sparql-results+json", + "Content-Type": "application/x-www-form-urlencoded", + "User-Agent": "NavidromeWikimediaPlugin/1.0", + }, + Body: []byte(form.Encode()), + TimeoutMs: 10000, + }) + if err != nil { + return nil, fmt.Errorf("SPARQL HTTP error: %w", err) + } + if resp.StatusCode != 200 { + return nil, fmt.Errorf("SPARQL HTTP error: status %d", resp.StatusCode) } var result SPARQLResult - if err := json.Unmarshal(resp.Body(), &result); err != nil { + if err := json.Unmarshal(resp.Body, &result); err != nil { return nil, fmt.Errorf("failed to parse SPARQL response: %w", err) } if len(result.Results.Bindings) == 0 { @@ -104,15 +112,22 @@ func sparqlQuery(endpoint, query string) (*SPARQLResult, error) { func mediawikiQuery(params url.Values) ([]byte, error) { apiURL := fmt.Sprintf("%s?%s", mediawikiAPIEndpoint, params.Encode()) - req := pdk.NewHTTPRequest(pdk.MethodGet, apiURL) - req.SetHeader("Accept", "application/json") - req.SetHeader("User-Agent", "NavidromeWikimediaPlugin/1.0") - - resp := req.Send() - if resp.Status() != 200 { - return nil, fmt.Errorf("MediaWiki HTTP error: status %d", resp.Status()) + resp, err := host.HTTPSend(host.HTTPRequest{ + Method: "GET", + URL: apiURL, + Headers: map[string]string{ + "Accept": "application/json", + "User-Agent": "NavidromeWikimediaPlugin/1.0", + }, + TimeoutMs: 10000, + }) + if err != nil { + return nil, fmt.Errorf("MediaWiki HTTP error: %w", err) } - return resp.Body(), nil + if resp.StatusCode != 200 { + return nil, fmt.Errorf("MediaWiki HTTP error: status %d", resp.StatusCode) + } + return resp.Body, nil } // getWikidataWikipediaURL fetches the Wikipedia URL from Wikidata using MBID or name diff --git a/plugins/host/httpclient.go b/plugins/host/httpclient.go new file mode 100644 index 000000000..b61361c57 --- /dev/null +++ b/plugins/host/httpclient.go @@ -0,0 +1,40 @@ +package host + +import "context" + +// HTTPRequest represents an outbound HTTP request from a plugin. +type HTTPRequest struct { + Method string `json:"method"` + URL string `json:"url"` + Headers map[string]string `json:"headers,omitempty"` + Body []byte `json:"body,omitempty"` + TimeoutMs int32 `json:"timeoutMs,omitempty"` +} + +// HTTPResponse represents the response from an outbound HTTP request. +type HTTPResponse struct { + StatusCode int32 `json:"statusCode"` + Headers map[string]string `json:"headers,omitempty"` + Body []byte `json:"body,omitempty"` +} + +// HTTPService provides outbound HTTP request capabilities for plugins. +// +// This service allows plugins to make HTTP requests to external services. +// Requests are validated against the plugin's declared requiredHosts patterns +// from the http permission in the manifest. Redirects are followed but each +// redirect destination is also validated against the allowed hosts. +// +//nd:hostservice name=HTTP permission=http +type HTTPService interface { + // Send executes an HTTP request and returns the response. + // + // Parameters: + // - request: The HTTP request to execute, including method, URL, headers, body, and timeout + // + // Returns the HTTP response with status code, headers, and body. + // Network errors, timeouts, and permission failures are returned as Go errors. + // Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. + //nd:hostfunc + Send(ctx context.Context, request HTTPRequest) (*HTTPResponse, error) +} diff --git a/plugins/host/httpclient_gen.go b/plugins/host/httpclient_gen.go new file mode 100644 index 000000000..c14a533d0 --- /dev/null +++ b/plugins/host/httpclient_gen.go @@ -0,0 +1,88 @@ +// Code generated by ndpgen. DO NOT EDIT. + +package host + +import ( + "context" + "encoding/json" + + extism "github.com/extism/go-sdk" +) + +// HTTPSendRequest is the request type for HTTP.Send. +type HTTPSendRequest struct { + Request HTTPRequest `json:"request"` +} + +// HTTPSendResponse is the response type for HTTP.Send. +type HTTPSendResponse struct { + Result *HTTPResponse `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +// RegisterHTTPHostFunctions registers HTTP service host functions. +// The returned host functions should be added to the plugin's configuration. +func RegisterHTTPHostFunctions(service HTTPService) []extism.HostFunction { + return []extism.HostFunction{ + newHTTPSendHostFunction(service), + } +} + +func newHTTPSendHostFunction(service HTTPService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "http_send", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + httpWriteError(p, stack, err) + return + } + var req HTTPSendRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + httpWriteError(p, stack, err) + return + } + + // Call the service method + result, svcErr := service.Send(ctx, req.Request) + if svcErr != nil { + httpWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := HTTPSendResponse{ + Result: result, + } + httpWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + +// httpWriteResponse writes a JSON response to plugin memory. +func httpWriteResponse(p *extism.CurrentPlugin, stack []uint64, resp any) { + respBytes, err := json.Marshal(resp) + if err != nil { + httpWriteError(p, stack, err) + return + } + respPtr, err := p.WriteBytes(respBytes) + if err != nil { + stack[0] = 0 + return + } + stack[0] = respPtr +} + +// httpWriteError writes an error response to plugin memory. +func httpWriteError(p *extism.CurrentPlugin, stack []uint64, err error) { + errResp := struct { + Error string `json:"error"` + }{Error: err.Error()} + respBytes, _ := json.Marshal(errResp) + respPtr, _ := p.WriteBytes(respBytes) + stack[0] = respPtr +} diff --git a/plugins/host_httpclient.go b/plugins/host_httpclient.go new file mode 100644 index 000000000..4dc8a2b9e --- /dev/null +++ b/plugins/host_httpclient.go @@ -0,0 +1,190 @@ +package plugins + +import ( + "bytes" + "cmp" + "context" + "fmt" + "io" + "net" + "net/http" + "net/url" + "strings" + "time" + + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/plugins/host" +) + +const ( + httpClientDefaultTimeout = 10 * time.Second + httpClientMaxRedirects = 5 + httpClientMaxResponseBodyLen = 10 * 1024 * 1024 // 10 MB +) + +// httpServiceImpl implements host.HTTPService. +type httpServiceImpl struct { + pluginName string + requiredHosts []string + client *http.Client +} + +// newHTTPService creates a new HTTPService for a plugin. +func newHTTPService(pluginName string, permission *HTTPPermission) *httpServiceImpl { + var requiredHosts []string + if permission != nil { + requiredHosts = permission.RequiredHosts + } + svc := &httpServiceImpl{ + pluginName: pluginName, + requiredHosts: requiredHosts, + } + svc.client = &http.Client{ + Transport: http.DefaultTransport, + // Timeout is set per-request via context deadline, not here. + // CheckRedirect validates hosts and enforces redirect limits. + CheckRedirect: func(req *http.Request, via []*http.Request) error { + if len(via) >= httpClientMaxRedirects { + log.Warn(req.Context(), "HTTP redirect limit exceeded", "plugin", svc.pluginName, "url", req.URL.String(), "redirectCount", len(via)) + return http.ErrUseLastResponse + } + if err := svc.validateHost(req.Context(), req.URL.Host); err != nil { + log.Warn(req.Context(), "HTTP redirect blocked", "plugin", svc.pluginName, "url", req.URL.String(), "err", err) + return err + } + return nil + }, + } + return svc +} + +func (s *httpServiceImpl) Send(ctx context.Context, request host.HTTPRequest) (*host.HTTPResponse, error) { + // Parse and validate URL + parsedURL, err := url.Parse(request.URL) + if err != nil { + return nil, fmt.Errorf("invalid URL: %w", err) + } + + // Validate URL scheme + if parsedURL.Scheme != "http" && parsedURL.Scheme != "https" { + return nil, fmt.Errorf("invalid URL scheme %q: must be http or https", parsedURL.Scheme) + } + + // Validate host against allowed hosts and private IP restrictions + if err := s.validateHost(ctx, parsedURL.Host); err != nil { + return nil, err + } + + // Apply per-request timeout via context deadline + timeout := cmp.Or(time.Duration(request.TimeoutMs)*time.Millisecond, httpClientDefaultTimeout) + ctx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + + // Build request body + method := strings.ToUpper(request.Method) + var body io.Reader + if len(request.Body) > 0 { + body = bytes.NewReader(request.Body) + } + + // Create HTTP request + httpReq, err := http.NewRequestWithContext(ctx, method, request.URL, body) + if err != nil { + return nil, fmt.Errorf("creating request: %w", err) + } + for k, v := range request.Headers { + httpReq.Header.Set(k, v) + } + + // Execute request + resp, err := s.client.Do(httpReq) //nolint:gosec // URL is validated against requiredHosts + if err != nil { + return nil, err + } + defer resp.Body.Close() + + log.Trace(ctx, "HTTP request", "plugin", s.pluginName, "method", method, "url", request.URL, "status", resp.StatusCode) + + // Read response body (with size limit to prevent memory exhaustion) + respBody, err := io.ReadAll(io.LimitReader(resp.Body, httpClientMaxResponseBodyLen)) + if err != nil { + return nil, fmt.Errorf("reading response body: %w", err) + } + + // Flatten response headers (first value only) + headers := make(map[string]string, len(resp.Header)) + for k, v := range resp.Header { + if len(v) > 0 { + headers[k] = v[0] + } + } + + return &host.HTTPResponse{ + StatusCode: int32(resp.StatusCode), + Headers: headers, + Body: respBody, + }, nil +} + +// validateHost checks whether a request to the given host is permitted. +// When requiredHosts is set, it checks against the allowlist. +// When requiredHosts is empty, it blocks private/loopback IPs to prevent SSRF. +func (s *httpServiceImpl) validateHost(ctx context.Context, hostStr string) error { + hostname := extractHostname(hostStr) + + if len(s.requiredHosts) > 0 { + if !s.isHostAllowed(hostname) { + return fmt.Errorf("host %q is not allowed", hostStr) + } + return nil + } + + // No explicit allowlist: block private/loopback IPs + if isPrivateOrLoopback(hostname) { + log.Warn(ctx, "HTTP request to private/loopback address blocked", "plugin", s.pluginName, "host", hostStr) + return fmt.Errorf("host %q is not allowed: private/loopback addresses require explicit requiredHosts in manifest", hostStr) + } + return nil +} + +func (s *httpServiceImpl) isHostAllowed(hostname string) bool { + for _, pattern := range s.requiredHosts { + if matchHostPattern(pattern, hostname) { + return true + } + } + return false +} + +// extractHostname returns the hostname portion of a host string, stripping +// any port number and IPv6 brackets. It handles IPv6 addresses correctly +// (e.g. "[::1]:8080" → "::1", "[::1]" → "::1"). +func extractHostname(hostStr string) string { + if h, _, err := net.SplitHostPort(hostStr); err == nil { + return h + } + // Strip IPv6 brackets when no port is present (e.g. "[::1]" → "::1") + if strings.HasPrefix(hostStr, "[") && strings.HasSuffix(hostStr, "]") { + return hostStr[1 : len(hostStr)-1] + } + return hostStr +} + +// isPrivateOrLoopback returns true if the given hostname resolves to or is +// a private, loopback, or link-local IP address. This includes: +// IPv4: 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16 +// IPv6: ::1, fc00::/7, fe80::/10 +// It also blocks "localhost" by name. +func isPrivateOrLoopback(hostname string) bool { + if strings.EqualFold(hostname, "localhost") { + return true + } + ip := net.ParseIP(hostname) + if ip == nil { + return false + } + return ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() || ip.IsLinkLocalMulticast() +} + +// Verify interface implementation +var _ host.HTTPService = (*httpServiceImpl)(nil) diff --git a/plugins/host_httpclient_test.go b/plugins/host_httpclient_test.go new file mode 100644 index 000000000..29796b052 --- /dev/null +++ b/plugins/host_httpclient_test.go @@ -0,0 +1,565 @@ +//go:build !windows + +package plugins + +import ( + "context" + "io" + "net/http" + "net/http/httptest" + "strings" + "time" + + "github.com/navidrome/navidrome/plugins/host" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("httpServiceImpl", func() { + var ( + svc *httpServiceImpl + ts *httptest.Server + ) + + AfterEach(func() { + if ts != nil { + ts.Close() + } + }) + + Context("without host restrictions (default SSRF protection)", func() { + BeforeEach(func() { + svc = newHTTPService("test-plugin", nil) + }) + + It("should block requests to loopback IPs", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(200) + })) + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should block requests to localhost by name", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://localhost:12345/test", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should block requests to private IPs (10.x)", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://10.0.0.1/test", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should block requests to private IPs (192.168.x)", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://192.168.1.1/test", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should block requests to private IPs (172.16.x)", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://172.16.0.1/test", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should block requests to link-local IPs (169.254.x)", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://169.254.169.254/latest/meta-data/", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should block requests to IPv6 loopback with port", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://[::1]:8080/test", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should block requests to IPv6 loopback without port", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://[::1]/test", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("private/loopback")) + }) + + It("should allow requests to public hostnames", func() { + // This will fail at the network level (connection refused or DNS), + // but it should NOT fail with a "private/loopback" error + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://203.0.113.1:1/test", // TEST-NET-3, non-routable but not private + TimeoutMs: 100, + }) + // Should get a network error, not a permission error + if err != nil { + Expect(err.Error()).ToNot(ContainSubstring("private/loopback")) + } + }) + + It("should return error for invalid URL", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "://bad-url", + }) + Expect(err).To(HaveOccurred()) + }) + + It("should reject non-http/https URL schemes", func() { + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "ftp://example.com/file", + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("must be http or https")) + }) + }) + + Context("with explicit requiredHosts allowing loopback", func() { + BeforeEach(func() { + svc = newHTTPService("test-plugin", &HTTPPermission{ + RequiredHosts: []string{"127.0.0.1"}, + }) + }) + + It("should handle GET requests", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + Expect(r.Method).To(Equal("GET")) + w.Header().Set("X-Test", "ok") + w.WriteHeader(201) + _, _ = w.Write([]byte("hello")) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + Headers: map[string]string{"Accept": "text/plain"}, + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(resp.StatusCode).To(Equal(int32(201))) + Expect(string(resp.Body)).To(Equal("hello")) + Expect(resp.Headers["X-Test"]).To(Equal("ok")) + }) + + It("should handle POST requests with body", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + Expect(r.Method).To(Equal("POST")) + b, _ := io.ReadAll(r.Body) + _, _ = w.Write([]byte("got:" + string(b))) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "POST", + URL: ts.URL, + Body: []byte("abc"), + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(string(resp.Body)).To(Equal("got:abc")) + }) + + It("should handle PUT requests with body", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + Expect(r.Method).To(Equal("PUT")) + b, _ := io.ReadAll(r.Body) + _, _ = w.Write([]byte("put:" + string(b))) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "PUT", + URL: ts.URL, + Body: []byte("xyz"), + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(string(resp.Body)).To(Equal("put:xyz")) + }) + + It("should handle DELETE requests", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + Expect(r.Method).To(Equal("DELETE")) + w.WriteHeader(204) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "DELETE", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(resp.StatusCode).To(Equal(int32(204))) + }) + + It("should handle DELETE requests with body", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + Expect(r.Method).To(Equal("DELETE")) + b, _ := io.ReadAll(r.Body) + _, _ = w.Write([]byte("del:" + string(b))) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "DELETE", + URL: ts.URL, + Body: []byte(`{"id":"123"}`), + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(string(resp.Body)).To(Equal(`del:{"id":"123"}`)) + }) + + It("should handle PATCH requests with body", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + Expect(r.Method).To(Equal("PATCH")) + b, _ := io.ReadAll(r.Body) + _, _ = w.Write([]byte("patch:" + string(b))) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "PATCH", + URL: ts.URL, + Body: []byte("data"), + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(string(resp.Body)).To(Equal("patch:data")) + }) + + It("should handle HEAD requests", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + Expect(r.Method).To(Equal("HEAD")) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(200) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "HEAD", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(resp.StatusCode).To(Equal(int32(200))) + Expect(resp.Headers["Content-Type"]).To(Equal("application/json")) + Expect(resp.Body).To(BeEmpty()) + }) + + It("should use default timeout when TimeoutMs is 0", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(200) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(resp.StatusCode).To(Equal(int32(200))) + }) + + It("should return error on timeout", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + time.Sleep(50 * time.Millisecond) + })) + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 1, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("deadline exceeded")) + }) + + It("should return error on context cancellation", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + time.Sleep(50 * time.Millisecond) + })) + ctx, cancel := context.WithCancel(context.Background()) + go func() { + time.Sleep(1 * time.Millisecond) + cancel() + }() + _, err := svc.Send(ctx, host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 5000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("context canceled")) + }) + + It("should send request headers", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(r.Header.Get("X-Custom"))) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + Headers: map[string]string{"X-Custom": "myvalue"}, + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(string(resp.Body)).To(Equal("myvalue")) + }) + }) + + Context("with host restrictions", func() { + BeforeEach(func() { + svc = newHTTPService("test-plugin", &HTTPPermission{ + RequiredHosts: []string{"allowed.example.com", "*.allowed.org"}, + }) + }) + + It("should block requests to non-allowed hosts", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(200) + })) + // httptest server is on 127.0.0.1 which is not in requiredHosts + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("not allowed")) + }) + + It("should follow redirects to allowed hosts", func() { + // Create a destination server + dest := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("final")) + })) + defer dest.Close() + // Create a redirect server + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, dest.URL, http.StatusFound) + })) + // Allow both servers (both on 127.0.0.1) + svc.requiredHosts = []string{"127.0.0.1"} + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(resp.StatusCode).To(Equal(int32(200))) + Expect(string(resp.Body)).To(Equal("final")) + }) + + It("should block redirects to non-allowed hosts", func() { + // Server that redirects to a disallowed host + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "http://evil.example.com/steal", http.StatusFound) + })) + // Override requiredHosts to allow the test server + svc.requiredHosts = []string{"127.0.0.1"} + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("not allowed")) + }) + + It("should block redirects to private IPs when allowlist is set", func() { + // Server that redirects to a private IP + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "http://10.0.0.1/internal", http.StatusFound) + })) + // Allow the test server; redirect to 10.0.0.1 is blocked by allowlist + svc.requiredHosts = []string{"127.0.0.1"} + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(resp).To(BeNil()) + }) + + It("should allow wildcard host patterns", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("wildcard")) + })) + // *.allowed.org is in the requiredHosts from BeforeEach, but test server is 127.0.0.1 + // Override with a wildcard that matches the test server + svc.requiredHosts = []string{"*.0.0.1"} + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(string(resp.Body)).To(Equal("wildcard")) + }) + + It("should reject hosts not matching wildcard patterns", func() { + svc.requiredHosts = []string{"*.example.com"} + _, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: "http://evil.other.com/test", + TimeoutMs: 1000, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("not allowed")) + }) + }) + + Context("response body size limit", func() { + BeforeEach(func() { + svc = newHTTPService("test-plugin", &HTTPPermission{ + RequiredHosts: []string{"127.0.0.1"}, + }) + }) + + It("should truncate response body at the size limit", func() { + // Serve a body larger than the limit + oversizedBody := strings.Repeat("x", httpClientMaxResponseBodyLen+1024) + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(oversizedBody)) + })) + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "GET", + URL: ts.URL, + TimeoutMs: 5000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(len(resp.Body)).To(Equal(httpClientMaxResponseBodyLen)) + }) + }) + + Context("edge cases", func() { + BeforeEach(func() { + svc = newHTTPService("test-plugin", &HTTPPermission{ + RequiredHosts: []string{"127.0.0.1"}, + }) + }) + + It("should default empty method to GET", func() { + ts = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("method:" + r.Method)) + })) + // Empty method — Go's http.NewRequestWithContext normalizes "" to "GET" + resp, err := svc.Send(context.Background(), host.HTTPRequest{ + Method: "", + URL: ts.URL, + TimeoutMs: 1000, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(string(resp.Body)).To(Equal("method:GET")) + }) + }) +}) + +var _ = Describe("extractHostname", func() { + It("should extract hostname from host:port", func() { + Expect(extractHostname("example.com:8080")).To(Equal("example.com")) + }) + + It("should return hostname when no port", func() { + Expect(extractHostname("example.com")).To(Equal("example.com")) + }) + + It("should handle IPv6 with port", func() { + Expect(extractHostname("[::1]:8080")).To(Equal("::1")) + }) + + It("should handle IPv6 without port", func() { + Expect(extractHostname("::1")).To(Equal("::1")) + }) + + It("should strip brackets from IPv6 without port", func() { + Expect(extractHostname("[::1]")).To(Equal("::1")) + }) + + It("should handle IPv4 with port", func() { + Expect(extractHostname("127.0.0.1:9090")).To(Equal("127.0.0.1")) + }) + + It("should handle IPv4 without port", func() { + Expect(extractHostname("127.0.0.1")).To(Equal("127.0.0.1")) + }) +}) + +var _ = Describe("isPrivateOrLoopback", func() { + It("should detect IPv4 loopback", func() { + Expect(isPrivateOrLoopback("127.0.0.1")).To(BeTrue()) + Expect(isPrivateOrLoopback("127.0.0.2")).To(BeTrue()) + }) + + It("should detect IPv6 loopback", func() { + Expect(isPrivateOrLoopback("::1")).To(BeTrue()) + }) + + It("should detect localhost by name", func() { + Expect(isPrivateOrLoopback("localhost")).To(BeTrue()) + Expect(isPrivateOrLoopback("LOCALHOST")).To(BeTrue()) + }) + + It("should detect 10.x.x.x private range", func() { + Expect(isPrivateOrLoopback("10.0.0.1")).To(BeTrue()) + Expect(isPrivateOrLoopback("10.255.255.255")).To(BeTrue()) + }) + + It("should detect 172.16.x.x private range", func() { + Expect(isPrivateOrLoopback("172.16.0.1")).To(BeTrue()) + Expect(isPrivateOrLoopback("172.31.255.255")).To(BeTrue()) + }) + + It("should detect 192.168.x.x private range", func() { + Expect(isPrivateOrLoopback("192.168.0.1")).To(BeTrue()) + Expect(isPrivateOrLoopback("192.168.255.255")).To(BeTrue()) + }) + + It("should detect link-local addresses", func() { + Expect(isPrivateOrLoopback("169.254.169.254")).To(BeTrue()) + Expect(isPrivateOrLoopback("169.254.0.1")).To(BeTrue()) + }) + + It("should detect IPv6 private (fc00::/7)", func() { + Expect(isPrivateOrLoopback("fd00::1")).To(BeTrue()) + }) + + It("should detect IPv6 link-local (fe80::/10)", func() { + Expect(isPrivateOrLoopback("fe80::1")).To(BeTrue()) + }) + + It("should allow public IPs", func() { + Expect(isPrivateOrLoopback("8.8.8.8")).To(BeFalse()) + Expect(isPrivateOrLoopback("203.0.113.1")).To(BeFalse()) + Expect(isPrivateOrLoopback("2001:db8::1")).To(BeFalse()) + }) + + It("should allow non-IP hostnames (DNS names)", func() { + Expect(isPrivateOrLoopback("example.com")).To(BeFalse()) + Expect(isPrivateOrLoopback("api.example.com")).To(BeFalse()) + }) + + It("should not treat 172.32.x.x as private", func() { + Expect(isPrivateOrLoopback("172.32.0.1")).To(BeFalse()) + }) +}) diff --git a/plugins/host_websocket.go b/plugins/host_websocket.go index 06d905b49..84b28dd35 100644 --- a/plugins/host_websocket.go +++ b/plugins/host_websocket.go @@ -256,8 +256,11 @@ func (s *webSocketServiceImpl) isHostAllowed(host string) bool { } // matchHostPattern matches a host against a pattern. -// Supports wildcards like *.example.com +// Supports "*" (allow all) and wildcards like "*.example.com". func matchHostPattern(pattern, host string) bool { + if pattern == "*" { + return true + } if pattern == host { return true } diff --git a/plugins/host_websocket_test.go b/plugins/host_websocket_test.go index a3d8ee74a..7a0439129 100644 --- a/plugins/host_websocket_test.go +++ b/plugins/host_websocket_test.go @@ -575,6 +575,12 @@ var _ = Describe("WebSocketService", Ordered, func() { Expect(matchHostPattern("*.example.com", "deep.api.example.com")).To(BeTrue()) }) + It("should match bare '*' as allow-all", func() { + Expect(matchHostPattern("*", "anything.example.com")).To(BeTrue()) + Expect(matchHostPattern("*", "127.0.0.1")).To(BeTrue()) + Expect(matchHostPattern("*", "::1")).To(BeTrue()) + }) + It("should not match partial patterns", func() { Expect(matchHostPattern("*.example.com", "example.com.evil.org")).To(BeFalse()) }) diff --git a/plugins/manager_loader.go b/plugins/manager_loader.go index b558da1be..c6355911f 100644 --- a/plugins/manager_loader.go +++ b/plugins/manager_loader.go @@ -119,6 +119,15 @@ var hostServices = []hostServiceEntry{ return host.RegisterUsersHostFunctions(service), nil }, }, + { + name: "HTTP", + hasPermission: func(p *Permissions) bool { return p != nil && p.Http != nil }, + create: func(ctx *serviceContext) ([]extism.HostFunction, io.Closer) { + perm := ctx.permissions.Http + service := newHTTPService(ctx.pluginName, perm) + return host.RegisterHTTPHostFunctions(service), nil + }, + }, } // extractManifest reads manifest from an .ndp package and computes its SHA-256 hash. diff --git a/plugins/pdk/go/host/doc.go b/plugins/pdk/go/host/doc.go index 82dc2c4aa..b801db44b 100644 --- a/plugins/pdk/go/host/doc.go +++ b/plugins/pdk/go/host/doc.go @@ -38,6 +38,7 @@ The following host services are available: - Artwork: provides artwork public URL generation capabilities for plugins. - Cache: provides in-memory TTL-based caching capabilities for plugins. - Config: provides access to plugin configuration values. + - HTTP: provides outbound HTTP request capabilities for plugins. - KVStore: provides persistent key-value storage for plugins. - Library: provides access to music library metadata for plugins. - Scheduler: provides task scheduling capabilities for plugins. diff --git a/plugins/pdk/go/host/nd_host_httpclient.go b/plugins/pdk/go/host/nd_host_httpclient.go new file mode 100644 index 000000000..8bd960351 --- /dev/null +++ b/plugins/pdk/go/host/nd_host_httpclient.go @@ -0,0 +1,87 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains client wrappers for the HTTP host service. +// It is intended for use in Navidrome plugins built with TinyGo. +// +//go:build wasip1 + +package host + +import ( + "encoding/json" + "errors" + + "github.com/navidrome/navidrome/plugins/pdk/go/pdk" +) + +// HTTPRequest represents an outbound HTTP request from a plugin. +type HTTPRequest struct { + Method string `json:"method"` + URL string `json:"url"` + Headers map[string]string `json:"headers"` + Body []byte `json:"body"` + TimeoutMs int32 `json:"timeoutMs"` +} + +// HTTPResponse represents the response from an outbound HTTP request. +type HTTPResponse struct { + StatusCode int32 `json:"statusCode"` + Headers map[string]string `json:"headers"` + Body []byte `json:"body"` +} + +// http_send is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user http_send +func http_send(uint64) uint64 + +type httpSendRequest struct { + Request HTTPRequest `json:"request"` +} + +type httpSendResponse struct { + Result *HTTPResponse `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +// HTTPSend calls the http_send host function. +// Send executes an HTTP request and returns the response. +// +// Parameters: +// - request: The HTTP request to execute, including method, URL, headers, body, and timeout +// +// Returns the HTTP response with status code, headers, and body. +// Network errors, timeouts, and permission failures are returned as Go errors. +// Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. +func HTTPSend(request HTTPRequest) (*HTTPResponse, error) { + // Marshal request to JSON + req := httpSendRequest{ + Request: request, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return nil, err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := http_send(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse the response + var response httpSendResponse + if err := json.Unmarshal(responseBytes, &response); err != nil { + return nil, err + } + + // Convert Error field to Go error + if response.Error != "" { + return nil, errors.New(response.Error) + } + + return response.Result, nil +} diff --git a/plugins/pdk/go/host/nd_host_httpclient_stub.go b/plugins/pdk/go/host/nd_host_httpclient_stub.go new file mode 100644 index 000000000..053069391 --- /dev/null +++ b/plugins/pdk/go/host/nd_host_httpclient_stub.go @@ -0,0 +1,55 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains mock implementations for non-WASM builds. +// These mocks allow IDE support, compilation, and unit testing on non-WASM platforms. +// Plugin authors can use the exported mock instances to set expectations in tests. +// +//go:build !wasip1 + +package host + +import "github.com/stretchr/testify/mock" + +// HTTPRequest represents an outbound HTTP request from a plugin. +type HTTPRequest struct { + Method string `json:"method"` + URL string `json:"url"` + Headers map[string]string `json:"headers"` + Body []byte `json:"body"` + TimeoutMs int32 `json:"timeoutMs"` +} + +// HTTPResponse represents the response from an outbound HTTP request. +type HTTPResponse struct { + StatusCode int32 `json:"statusCode"` + Headers map[string]string `json:"headers"` + Body []byte `json:"body"` +} + +// mockHTTPService is the mock implementation for testing. +type mockHTTPService struct { + mock.Mock +} + +// HTTPMock is the auto-instantiated mock instance for testing. +// Use this to set expectations: host.HTTPMock.On("MethodName", args...).Return(values...) +var HTTPMock = &mockHTTPService{} + +// Send is the mock method for HTTPSend. +func (m *mockHTTPService) Send(request HTTPRequest) (*HTTPResponse, error) { + args := m.Called(request) + return args.Get(0).(*HTTPResponse), args.Error(1) +} + +// HTTPSend delegates to the mock instance. +// Send executes an HTTP request and returns the response. +// +// Parameters: +// - request: The HTTP request to execute, including method, URL, headers, body, and timeout +// +// Returns the HTTP response with status code, headers, and body. +// Network errors, timeouts, and permission failures are returned as Go errors. +// Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. +func HTTPSend(request HTTPRequest) (*HTTPResponse, error) { + return HTTPMock.Send(request) +} diff --git a/plugins/pdk/python/host/nd_host_httpclient.py b/plugins/pdk/python/host/nd_host_httpclient.py new file mode 100644 index 000000000..c6bfb77c0 --- /dev/null +++ b/plugins/pdk/python/host/nd_host_httpclient.py @@ -0,0 +1,59 @@ +# Code generated by ndpgen. DO NOT EDIT. +# +# This file contains client wrappers for the HTTP host service. +# It is intended for use in Navidrome plugins built with extism-py. +# +# IMPORTANT: Due to a limitation in extism-py, you cannot import this file directly. +# The @extism.import_fn decorators are only detected when defined in the plugin's +# main __init__.py file. Copy the needed functions from this file into your plugin. + +from dataclasses import dataclass +from typing import Any + +import extism +import json + + +class HostFunctionError(Exception): + """Raised when a host function returns an error.""" + pass + + +@extism.import_fn("extism:host/user", "http_send") +def _http_send(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +def http_send(request: Any) -> Any: + """Send executes an HTTP request and returns the response. + +Parameters: + - request: The HTTP request to execute, including method, URL, headers, body, and timeout + +Returns the HTTP response with status code, headers, and body. +Network errors, timeouts, and permission failures are returned as errors. +Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. + + Args: + request: Any parameter. + + Returns: + Any: The result value. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "request": request, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _http_send(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + return response.get("result", None) diff --git a/plugins/pdk/rust/nd-pdk-host/src/lib.rs b/plugins/pdk/rust/nd-pdk-host/src/lib.rs index 3dff68269..52a3a86cd 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/lib.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/lib.rs @@ -35,6 +35,7 @@ //! - [`artwork`] - provides artwork public URL generation capabilities for plugins. //! - [`cache`] - provides in-memory TTL-based caching capabilities for plugins. //! - [`config`] - provides access to plugin configuration values. +//! - [`http`] - provides outbound HTTP request capabilities for plugins. //! - [`kvstore`] - provides persistent key-value storage for plugins. //! - [`library`] - provides access to music library metadata for plugins. //! - [`scheduler`] - provides task scheduling capabilities for plugins. @@ -63,6 +64,13 @@ pub mod config { pub use super::nd_host_config::*; } +#[doc(hidden)] +mod nd_host_http; +/// provides outbound HTTP request capabilities for plugins. +pub mod http { + pub use super::nd_host_http::*; +} + #[doc(hidden)] mod nd_host_kvstore; /// provides persistent key-value storage for plugins. diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs new file mode 100644 index 000000000..c73241c80 --- /dev/null +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs @@ -0,0 +1,83 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains client wrappers for the HTTP host service. +// It is intended for use in Navidrome plugins built with extism-pdk. + +use extism_pdk::*; +use serde::{Deserialize, Serialize}; + +/// HTTPRequest represents an outbound HTTP request from a plugin. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct HttpRequest { + pub method: String, + pub url: String, + #[serde(default)] + pub headers: std::collections::HashMap, + #[serde(default)] + pub body: Vec, + #[serde(default)] + pub timeout_ms: i32, +} + +/// HTTPResponse represents the response from an outbound HTTP request. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct HttpResponse { + pub status_code: i32, + #[serde(default)] + pub headers: std::collections::HashMap, + #[serde(default)] + pub body: Vec, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct HTTPSendRequest { + request: HttpRequest, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct HTTPSendResponse { + #[serde(default)] + result: Option, + #[serde(default)] + error: Option, +} + +#[host_fn] +extern "ExtismHost" { + fn http_send(input: Json) -> Json; +} + +/// Send executes an HTTP request and returns the response. +/// +/// Parameters: +/// - request: The HTTP request to execute, including method, URL, headers, body, and timeout +/// +/// Returns the HTTP response with status code, headers, and body. +/// Network errors, timeouts, and permission failures are returned as errors. +/// Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. +/// +/// # Arguments +/// * `request` - HttpRequest parameter. +/// +/// # Returns +/// The result value. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn send(request: HttpRequest) -> Result, Error> { + let response = unsafe { + http_send(Json(HTTPSendRequest { + request: request, + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(response.0.result) +} From fc36f1daa69ddb46dc90cbfca929715158924e65 Mon Sep 17 00:00:00 2001 From: Deluan Date: Tue, 24 Feb 2026 20:45:58 -0500 Subject: [PATCH 13/60] chore(deps): update go-taglib dependency to latest version (mka fix) Signed-off-by: Deluan --- go.mod | 2 +- go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index f84ea65d0..8caddd5eb 100644 --- a/go.mod +++ b/go.mod @@ -7,7 +7,7 @@ replace ( github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8 => github.com/deluan/tag v0.0.0-20241002021117-dfe5e6ea396d // Fork to implement raw tags support - go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260221220301-2fab4903f48e + go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 ) require ( diff --git a/go.sum b/go.sum index 4e05f46bc..79f29c70a 100644 --- a/go.sum +++ b/go.sum @@ -36,8 +36,8 @@ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvwDRwnI3hwNaAHRnc= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40= -github.com/deluan/go-taglib v0.0.0-20260221220301-2fab4903f48e h1:yQF3eOcI2dMMtxqdKXm3cgfYZlDcq9SUDDv90bsMj2I= -github.com/deluan/go-taglib v0.0.0-20260221220301-2fab4903f48e/go.mod h1:sKDN0U4qXDlq6LFK+aOAkDH4Me5nDV1V/A4B+B69xBA= +github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 h1:seWJmkPAb+M1ysRNGzTGS7FfdrUe9wQTHhB9p2fxDWg= +github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1/go.mod h1:sKDN0U4qXDlq6LFK+aOAkDH4Me5nDV1V/A4B+B69xBA= github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf h1:tb246l2Zmpt/GpF9EcHCKTtwzrd0HGfEmoODFA/qnk4= github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf/go.mod h1:tSgDythFsl0QgS/PFWfIZqcJKnkADWneY80jaVRlqK8= github.com/deluan/sanitize v0.0.0-20241120162836-fdfd8fdfaa55 h1:wSCnggTs2f2ji6nFwQmfwgINcmSMj0xF0oHnoyRSPe4= From 14343d91b0165b49036f5022a22a39f783b37539 Mon Sep 17 00:00:00 2001 From: Deluan Date: Tue, 24 Feb 2026 21:44:04 -0500 Subject: [PATCH 14/60] chore(deps): update goose to 3.27.0 Signed-off-by: Deluan --- go.mod | 7 +++---- go.sum | 32 ++++++++++++++++---------------- 2 files changed, 19 insertions(+), 20 deletions(-) diff --git a/go.mod b/go.mod index 8caddd5eb..c52bb211d 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/navidrome/navidrome -go 1.25 +go 1.25.0 replace ( // Fork to fix https://github.com/navidrome/navidrome/issues/3254 @@ -53,7 +53,7 @@ require ( github.com/onsi/gomega v1.39.1 github.com/pelletier/go-toml/v2 v2.2.4 github.com/pocketbase/dbx v1.12.0 - github.com/pressly/goose/v3 v3.26.0 + github.com/pressly/goose/v3 v3.27.0 github.com/prometheus/client_golang v1.23.2 github.com/rjeczalik/notify v0.9.3 github.com/robfig/cron/v3 v3.0.1 @@ -88,7 +88,7 @@ require ( github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/creack/pty v1.1.24 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect - github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect + github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect github.com/dylibso/observe-sdk/go v0.0.0-20240828172851-9145d8ad07e1 // indirect github.com/fsnotify/fsnotify v1.9.0 // indirect github.com/go-logr/logr v1.4.3 // indirect @@ -140,7 +140,6 @@ require ( go.yaml.in/yaml/v2 v2.4.3 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect golang.org/x/crypto v0.48.0 // indirect - golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect golang.org/x/mod v0.33.0 // indirect golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4 // indirect golang.org/x/tools v0.42.0 // indirect diff --git a/go.sum b/go.sum index 79f29c70a..bbfd51e11 100644 --- a/go.sum +++ b/go.sum @@ -1,7 +1,7 @@ dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8= dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA= -filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA= -filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4= +filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo= +filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc= github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0= github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM= github.com/Masterminds/squirrel v1.5.4 h1:uUcX/aBc8O7Fg9kaISIUsHXdKuqehiXAMQTYX8afzqM= @@ -34,8 +34,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvwDRwnI3hwNaAHRnc= -github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 h1:5RVFMOWjMyRy8cARdy79nAmgYw3hK/4HUq48LQ6Wwqo= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40= github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 h1:seWJmkPAb+M1ysRNGzTGS7FfdrUe9wQTHhB9p2fxDWg= github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1/go.mod h1:sKDN0U4qXDlq6LFK+aOAkDH4Me5nDV1V/A4B+B69xBA= github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf h1:tb246l2Zmpt/GpF9EcHCKTtwzrd0HGfEmoODFA/qnk4= @@ -143,8 +143,8 @@ github.com/kardianos/service v1.2.4 h1:XNlGtZOYNx2u91urOdg/Kfmc+gfmuIo1Dd3rEi2Og github.com/kardianos/service v1.2.4/go.mod h1:E4V9ufUuY82F7Ztlu1eN9VXWIQxg8NoLQlmFe0MtrXc= github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs= github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= -github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= -github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c= +github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= @@ -193,8 +193,8 @@ github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQ github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= -github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= -github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= +github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= github.com/ogier/pflag v0.0.1 h1:RW6JSWSu/RkSatfcLtogGfFgpim5p7ARQ10ECk5O750= github.com/ogier/pflag v0.0.1/go.mod h1:zkFki7tvTa0tafRvTBIZTvzYyAu6kQhPZFnshFFPE+g= github.com/onsi/ginkgo/v2 v2.28.1 h1:S4hj+HbZp40fNKuLUQOYLDgZLwNUVn19N3Atb98NCyI= @@ -212,8 +212,8 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pocketbase/dbx v1.12.0 h1:/oLErM+A0b4xI0PWTGPqSDVjzix48PqI/bng2l0PzoA= github.com/pocketbase/dbx v1.12.0/go.mod h1:xXRCIAKTHMgUCyCKZm55pUOdvFziJjQfXaWKhu2vhMs= -github.com/pressly/goose/v3 v3.26.0 h1:KJakav68jdH0WDvoAcj8+n61WqOIaPGgH0bJWS6jpmM= -github.com/pressly/goose/v3 v3.26.0/go.mod h1:4hC1KrritdCxtuFsqgs1R4AU5bWtTAf+cnWvfhf2DNY= +github.com/pressly/goose/v3 v3.27.0 h1:/D30gVTuQhu0WsNZYbJi4DMOsx1lNq+6SkLe+Wp59BM= +github.com/pressly/goose/v3 v3.27.0/go.mod h1:3ZBeCXqzkgIRvrEMDkYh1guvtoJTU5oMMuDdkutoM78= github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= @@ -321,8 +321,8 @@ golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts= golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos= -golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU= -golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU= +golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa h1:Zt3DZoOFFYkKhDT3v7Lm9FDMEV06GpzjG2jrqW+QTE0= +golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa/go.mod h1:K79w1Vqn7PoiZn+TkNpx3BUWUQksGO3JcVX6qIjytmA= golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= golang.org/x/image v0.36.0 h1:Iknbfm1afbgtwPTmHnS2gTM/6PPZfH+z2EFuOkSbqwc= golang.org/x/image v0.36.0/go.mod h1:YsWD2TyyGKiIX1kZlu9QfKIsQ4nAAK9bdgdrIsE7xy4= @@ -423,11 +423,11 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -modernc.org/libc v1.66.3 h1:cfCbjTUcdsKyyZZfEUKfoHcP3S0Wkvz3jgSzByEWVCQ= -modernc.org/libc v1.66.3/go.mod h1:XD9zO8kt59cANKvHPXpx7yS2ELPheAey0vjIuZOhOU8= +modernc.org/libc v1.68.0 h1:PJ5ikFOV5pwpW+VqCK1hKJuEWsonkIJhhIXyuF/91pQ= +modernc.org/libc v1.68.0/go.mod h1:NnKCYeoYgsEqnY3PgvNgAeaJnso968ygU8Z0DxjoEc0= modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU= modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg= modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI= modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw= -modernc.org/sqlite v1.38.2 h1:Aclu7+tgjgcQVShZqim41Bbw9Cho0y/7WzYptXqkEek= -modernc.org/sqlite v1.38.2/go.mod h1:cPTJYSlgg3Sfg046yBShXENNtPrWrDX8bsbAQBzgQ5E= +modernc.org/sqlite v1.46.1 h1:eFJ2ShBLIEnUWlLy12raN0Z1plqmFX9Qe3rjQTKt6sU= +modernc.org/sqlite v1.46.1/go.mod h1:CzbrU2lSB1DKUusvwGz7rqEKIq+NUd8GWuBBZDs9/nA= From 5bc2bbb70e6236d7361b1e0e68d657a74847fd1d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Thu, 26 Feb 2026 10:50:12 -0500 Subject: [PATCH 15/60] feat(subsonic): append album version to names in Subsonic API (#5111) * feat(subsonic): append album version to album names in Subsonic API responses Add AppendAlbumVersion config option (default: true) that appends the album version tag to album names in Subsonic API responses, similar to how AppendSubtitle works for track titles. This affects album names in childFromAlbum and buildAlbumID3 responses. Signed-off-by: Deluan * feat(subsonic): append album version to media file album names in Subsonic API Add FullAlbumName() to MediaFile that appends the album version tag, mirroring the Album.FullName() behavior. Use it in childFromMediaFile and fakePath to ensure media file responses also show the album version. Signed-off-by: Deluan * fix(subsonic): use len() check for album version tag to prevent panic on empty slice Use len(tags) > 0 instead of != nil to safely guard against empty slices when accessing the first element of the album version tag. Signed-off-by: Deluan * fix(subsonic): use FullName in buildAlbumDirectory and deduplicate FullName calls Apply album.FullName() in buildAlbumDirectory (getMusicDirectory) so album names are consistent across all Subsonic endpoints. Also compute al.FullName() once in childFromAlbum to avoid redundant calls. Signed-off-by: Deluan * fix: use len() check in MediaFile.FullTitle() to prevent panic on empty slice Apply the same safety improvement as FullAlbumName() and Album.FullName() for consistency. Signed-off-by: Deluan * test: add tests for Album.FullName, MediaFile.FullTitle, and MediaFile.FullAlbumName Cover all cases: config enabled/disabled, tag present, tag absent, and empty tag slice. Signed-off-by: Deluan --------- Signed-off-by: Deluan --- conf/configuration.go | 2 ++ model/album.go | 10 ++++++++++ model/album_test.go | 19 +++++++++++++++++++ model/mediafile.go | 9 ++++++++- model/mediafile_test.go | 24 +++++++++++++++++++++++- server/subsonic/browsing.go | 2 +- server/subsonic/helpers.go | 13 +++++++------ 7 files changed, 70 insertions(+), 9 deletions(-) diff --git a/conf/configuration.go b/conf/configuration.go index 555d8f587..61448d315 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -155,6 +155,7 @@ type scannerOptions struct { type subsonicOptions struct { AppendSubtitle bool + AppendAlbumVersion bool ArtistParticipations bool DefaultReportRealPath bool EnableAverageRating bool @@ -689,6 +690,7 @@ func setViperDefaults() { viper.SetDefault("scanner.followsymlinks", true) viper.SetDefault("scanner.purgemissing", consts.PurgeMissingNever) viper.SetDefault("subsonic.appendsubtitle", true) + viper.SetDefault("subsonic.appendalbumversion", true) viper.SetDefault("subsonic.artistparticipations", false) viper.SetDefault("subsonic.defaultreportrealpath", false) viper.SetDefault("subsonic.enableaveragerating", true) diff --git a/model/album.go b/model/album.go index a8dcfe682..667f4695b 100644 --- a/model/album.go +++ b/model/album.go @@ -1,11 +1,14 @@ package model import ( + "fmt" "iter" "math" "sync" "time" + "github.com/navidrome/navidrome/conf" + "github.com/gohugoio/hashstructure" ) @@ -70,6 +73,13 @@ func (a Album) CoverArtID() ArtworkID { return artworkIDFromAlbum(a) } +func (a Album) FullName() string { + if conf.Server.Subsonic.AppendAlbumVersion && len(a.Tags[TagAlbumVersion]) > 0 { + return fmt.Sprintf("%s (%s)", a.Name, a.Tags[TagAlbumVersion][0]) + } + return a.Name +} + // Equals compares two Album structs, ignoring calculated fields func (a Album) Equals(other Album) bool { // Normalize float32 values to avoid false negatives diff --git a/model/album_test.go b/model/album_test.go index a45d16dd5..0f4c912cd 100644 --- a/model/album_test.go +++ b/model/album_test.go @@ -3,11 +3,30 @@ package model_test import ( "encoding/json" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" . "github.com/navidrome/navidrome/model" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ) +var _ = Describe("Album", func() { + BeforeEach(func() { + DeferCleanup(configtest.SetupConfig()) + }) + DescribeTable("FullName", + func(enabled bool, tags Tags, expected string) { + conf.Server.Subsonic.AppendAlbumVersion = enabled + a := Album{Name: "Album", Tags: tags} + Expect(a.FullName()).To(Equal(expected)) + }, + Entry("appends version when enabled and tag is present", true, Tags{TagAlbumVersion: []string{"Remastered"}}, "Album (Remastered)"), + Entry("returns just name when disabled", false, Tags{TagAlbumVersion: []string{"Remastered"}}, "Album"), + Entry("returns just name when tag is absent", true, Tags{}, "Album"), + Entry("returns just name when tag is an empty slice", true, Tags{TagAlbumVersion: []string{}}, "Album"), + ) +}) + var _ = Describe("Albums", func() { var albums Albums diff --git a/model/mediafile.go b/model/mediafile.go index 831f006bf..103b02639 100644 --- a/model/mediafile.go +++ b/model/mediafile.go @@ -95,12 +95,19 @@ type MediaFile struct { } func (mf MediaFile) FullTitle() string { - if conf.Server.Subsonic.AppendSubtitle && mf.Tags[TagSubtitle] != nil { + if conf.Server.Subsonic.AppendSubtitle && len(mf.Tags[TagSubtitle]) > 0 { return fmt.Sprintf("%s (%s)", mf.Title, mf.Tags[TagSubtitle][0]) } return mf.Title } +func (mf MediaFile) FullAlbumName() string { + if conf.Server.Subsonic.AppendAlbumVersion && len(mf.Tags[TagAlbumVersion]) > 0 { + return fmt.Sprintf("%s (%s)", mf.Album, mf.Tags[TagAlbumVersion][0]) + } + return mf.Album +} + func (mf MediaFile) ContentType() string { return mime.TypeByExtension("." + mf.Suffix) } diff --git a/model/mediafile_test.go b/model/mediafile_test.go index 635a61d30..0b9191fe5 100644 --- a/model/mediafile_test.go +++ b/model/mediafile_test.go @@ -475,7 +475,29 @@ var _ = Describe("MediaFile", func() { DeferCleanup(configtest.SetupConfig()) conf.Server.EnableMediaFileCoverArt = true }) - Describe(".CoverArtId()", func() { + DescribeTable("FullTitle", + func(enabled bool, tags Tags, expected string) { + conf.Server.Subsonic.AppendSubtitle = enabled + mf := MediaFile{Title: "Song", Tags: tags} + Expect(mf.FullTitle()).To(Equal(expected)) + }, + Entry("appends subtitle when enabled and tag is present", true, Tags{TagSubtitle: []string{"Live"}}, "Song (Live)"), + Entry("returns just title when disabled", false, Tags{TagSubtitle: []string{"Live"}}, "Song"), + Entry("returns just title when tag is absent", true, Tags{}, "Song"), + Entry("returns just title when tag is an empty slice", true, Tags{TagSubtitle: []string{}}, "Song"), + ) + DescribeTable("FullAlbumName", + func(enabled bool, tags Tags, expected string) { + conf.Server.Subsonic.AppendAlbumVersion = enabled + mf := MediaFile{Album: "Album", Tags: tags} + Expect(mf.FullAlbumName()).To(Equal(expected)) + }, + Entry("appends version when enabled and tag is present", true, Tags{TagAlbumVersion: []string{"Deluxe Edition"}}, "Album (Deluxe Edition)"), + Entry("returns just album name when disabled", false, Tags{TagAlbumVersion: []string{"Deluxe Edition"}}, "Album"), + Entry("returns just album name when tag is absent", true, Tags{}, "Album"), + Entry("returns just album name when tag is an empty slice", true, Tags{TagAlbumVersion: []string{}}, "Album"), + ) + Describe("CoverArtId()", func() { It("returns its own id if it HasCoverArt", func() { mf := MediaFile{ID: "111", AlbumID: "1", HasCoverArt: true} id := mf.CoverArtID() diff --git a/server/subsonic/browsing.go b/server/subsonic/browsing.go index 63939f6f4..5b9c4f3c9 100644 --- a/server/subsonic/browsing.go +++ b/server/subsonic/browsing.go @@ -443,7 +443,7 @@ func (api *Router) buildArtist(r *http.Request, artist *model.Artist) (*response func (api *Router) buildAlbumDirectory(ctx context.Context, album *model.Album) (*responses.Directory, error) { dir := &responses.Directory{} dir.Id = album.ID - dir.Name = album.Name + dir.Name = album.FullName() dir.Parent = album.AlbumArtistID dir.PlayCount = album.PlayCount if album.PlayCount > 0 { diff --git a/server/subsonic/helpers.go b/server/subsonic/helpers.go index 107e23133..598346901 100644 --- a/server/subsonic/helpers.go +++ b/server/subsonic/helpers.go @@ -197,7 +197,7 @@ func childFromMediaFile(ctx context.Context, mf model.MediaFile) responses.Child } child.Parent = mf.AlbumID - child.Album = mf.Album + child.Album = mf.FullAlbumName() child.Year = int32(mf.Year) child.Artist = mf.Artist child.Genre = mf.Genre @@ -302,7 +302,7 @@ func artistRefs(participants model.ParticipantList) []responses.ArtistID3Ref { func fakePath(mf model.MediaFile) string { builder := strings.Builder{} - builder.WriteString(fmt.Sprintf("%s/%s/", sanitizeSlashes(mf.AlbumArtist), sanitizeSlashes(mf.Album))) + builder.WriteString(fmt.Sprintf("%s/%s/", sanitizeSlashes(mf.AlbumArtist), sanitizeSlashes(mf.FullAlbumName()))) if mf.DiscNumber != 0 { builder.WriteString(fmt.Sprintf("%02d-", mf.DiscNumber)) } @@ -321,9 +321,10 @@ func childFromAlbum(ctx context.Context, al model.Album) responses.Child { child := responses.Child{} child.Id = al.ID child.IsDir = true - child.Title = al.Name - child.Name = al.Name - child.Album = al.Name + fullName := al.FullName() + child.Title = fullName + child.Name = fullName + child.Album = fullName child.Artist = al.AlbumArtist child.Year = int32(cmp.Or(al.MaxOriginalYear, al.MaxYear)) child.Genre = al.Genre @@ -405,7 +406,7 @@ func buildDiscSubtitles(a model.Album) []responses.DiscTitle { func buildAlbumID3(ctx context.Context, album model.Album) responses.AlbumID3 { dir := responses.AlbumID3{} dir.Id = album.ID - dir.Name = album.Name + dir.Name = album.FullName() dir.Artist = album.AlbumArtist dir.ArtistId = album.AlbumArtistID dir.CoverArt = album.CoverArtID().String() From cdd3432788f04114a8ec6f6342e354a456b4a9bc Mon Sep 17 00:00:00 2001 From: Deluan Date: Thu, 26 Feb 2026 16:19:37 -0500 Subject: [PATCH 16/60] refactor(http): rename HTTP client files and update struct names for consistency Signed-off-by: Deluan --- plugins/host/{httpclient.go => http.go} | 0 .../host/{httpclient_gen.go => http_gen.go} | 0 plugins/pdk/go/go.mod | 7 +++ ...{nd_host_httpclient.go => nd_host_http.go} | 10 ++-- ...ttpclient_stub.go => nd_host_http_stub.go} | 2 + plugins/pdk/python/host/nd_host_http.py | 59 +++++++++++++++++++ .../pdk/rust/nd-pdk-host/src/nd_host_http.rs | 18 +++--- 7 files changed, 83 insertions(+), 13 deletions(-) rename plugins/host/{httpclient.go => http.go} (100%) rename plugins/host/{httpclient_gen.go => http_gen.go} (100%) rename plugins/pdk/go/host/{nd_host_httpclient.go => nd_host_http.go} (90%) rename plugins/pdk/go/host/{nd_host_httpclient_stub.go => nd_host_http_stub.go} (94%) create mode 100644 plugins/pdk/python/host/nd_host_http.py diff --git a/plugins/host/httpclient.go b/plugins/host/http.go similarity index 100% rename from plugins/host/httpclient.go rename to plugins/host/http.go diff --git a/plugins/host/httpclient_gen.go b/plugins/host/http_gen.go similarity index 100% rename from plugins/host/httpclient_gen.go rename to plugins/host/http_gen.go diff --git a/plugins/pdk/go/go.mod b/plugins/pdk/go/go.mod index 3916cd749..4d5fcddfc 100644 --- a/plugins/pdk/go/go.mod +++ b/plugins/pdk/go/go.mod @@ -6,3 +6,10 @@ require ( github.com/extism/go-pdk v1.1.3 github.com/stretchr/testify v1.11.1 ) + +require ( + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/stretchr/objx v0.5.2 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) diff --git a/plugins/pdk/go/host/nd_host_httpclient.go b/plugins/pdk/go/host/nd_host_http.go similarity index 90% rename from plugins/pdk/go/host/nd_host_httpclient.go rename to plugins/pdk/go/host/nd_host_http.go index 8bd960351..d77db4762 100644 --- a/plugins/pdk/go/host/nd_host_httpclient.go +++ b/plugins/pdk/go/host/nd_host_http.go @@ -14,6 +14,7 @@ import ( "github.com/navidrome/navidrome/plugins/pdk/go/pdk" ) +// HTTPRequest represents the HTTPRequest data structure. // HTTPRequest represents an outbound HTTP request from a plugin. type HTTPRequest struct { Method string `json:"method"` @@ -23,6 +24,7 @@ type HTTPRequest struct { TimeoutMs int32 `json:"timeoutMs"` } +// HTTPResponse represents the HTTPResponse data structure. // HTTPResponse represents the response from an outbound HTTP request. type HTTPResponse struct { StatusCode int32 `json:"statusCode"` @@ -35,11 +37,11 @@ type HTTPResponse struct { //go:wasmimport extism:host/user http_send func http_send(uint64) uint64 -type httpSendRequest struct { +type hTTPSendRequest struct { Request HTTPRequest `json:"request"` } -type httpSendResponse struct { +type hTTPSendResponse struct { Result *HTTPResponse `json:"result,omitempty"` Error string `json:"error,omitempty"` } @@ -55,7 +57,7 @@ type httpSendResponse struct { // Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. func HTTPSend(request HTTPRequest) (*HTTPResponse, error) { // Marshal request to JSON - req := httpSendRequest{ + req := hTTPSendRequest{ Request: request, } reqBytes, err := json.Marshal(req) @@ -73,7 +75,7 @@ func HTTPSend(request HTTPRequest) (*HTTPResponse, error) { responseBytes := responseMem.ReadBytes() // Parse the response - var response httpSendResponse + var response hTTPSendResponse if err := json.Unmarshal(responseBytes, &response); err != nil { return nil, err } diff --git a/plugins/pdk/go/host/nd_host_httpclient_stub.go b/plugins/pdk/go/host/nd_host_http_stub.go similarity index 94% rename from plugins/pdk/go/host/nd_host_httpclient_stub.go rename to plugins/pdk/go/host/nd_host_http_stub.go index 053069391..b5d1eee75 100644 --- a/plugins/pdk/go/host/nd_host_httpclient_stub.go +++ b/plugins/pdk/go/host/nd_host_http_stub.go @@ -10,6 +10,7 @@ package host import "github.com/stretchr/testify/mock" +// HTTPRequest represents the HTTPRequest data structure. // HTTPRequest represents an outbound HTTP request from a plugin. type HTTPRequest struct { Method string `json:"method"` @@ -19,6 +20,7 @@ type HTTPRequest struct { TimeoutMs int32 `json:"timeoutMs"` } +// HTTPResponse represents the HTTPResponse data structure. // HTTPResponse represents the response from an outbound HTTP request. type HTTPResponse struct { StatusCode int32 `json:"statusCode"` diff --git a/plugins/pdk/python/host/nd_host_http.py b/plugins/pdk/python/host/nd_host_http.py new file mode 100644 index 000000000..8f196f985 --- /dev/null +++ b/plugins/pdk/python/host/nd_host_http.py @@ -0,0 +1,59 @@ +# Code generated by ndpgen. DO NOT EDIT. +# +# This file contains client wrappers for the HTTP host service. +# It is intended for use in Navidrome plugins built with extism-py. +# +# IMPORTANT: Due to a limitation in extism-py, you cannot import this file directly. +# The @extism.import_fn decorators are only detected when defined in the plugin's +# main __init__.py file. Copy the needed functions from this file into your plugin. + +from dataclasses import dataclass +from typing import Any + +import extism +import json + + +class HostFunctionError(Exception): + """Raised when a host function returns an error.""" + pass + + +@extism.import_fn("extism:host/user", "http_send") +def _http_send(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +def http_send(request: Any) -> Any: + """Send executes an HTTP request and returns the response. + +Parameters: + - request: The HTTP request to execute, including method, URL, headers, body, and timeout + +Returns the HTTP response with status code, headers, and body. +Network errors, timeouts, and permission failures are returned as Go errors. +Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. + + Args: + request: Any parameter. + + Returns: + Any: The result value. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "request": request, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _http_send(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + return response.get("result", None) diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs index c73241c80..b8aaad1e0 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs @@ -9,7 +9,7 @@ use serde::{Deserialize, Serialize}; /// HTTPRequest represents an outbound HTTP request from a plugin. #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] -pub struct HttpRequest { +pub struct HTTPRequest { pub method: String, pub url: String, #[serde(default)] @@ -23,7 +23,7 @@ pub struct HttpRequest { /// HTTPResponse represents the response from an outbound HTTP request. #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] -pub struct HttpResponse { +pub struct HTTPResponse { pub status_code: i32, #[serde(default)] pub headers: std::collections::HashMap, @@ -34,14 +34,14 @@ pub struct HttpResponse { #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] struct HTTPSendRequest { - request: HttpRequest, + request: HTTPRequest, } #[derive(Debug, Clone, Deserialize)] #[serde(rename_all = "camelCase")] struct HTTPSendResponse { #[serde(default)] - result: Option, + result: Option, #[serde(default)] error: Option, } @@ -52,23 +52,23 @@ extern "ExtismHost" { } /// Send executes an HTTP request and returns the response. -/// +/// /// Parameters: /// - request: The HTTP request to execute, including method, URL, headers, body, and timeout -/// +/// /// Returns the HTTP response with status code, headers, and body. -/// Network errors, timeouts, and permission failures are returned as errors. +/// Network errors, timeouts, and permission failures are returned as Go errors. /// Successful HTTP calls (including 4xx/5xx status codes) return a non-nil response with nil error. /// /// # Arguments -/// * `request` - HttpRequest parameter. +/// * `request` - HTTPRequest parameter. /// /// # Returns /// The result value. /// /// # Errors /// Returns an error if the host function call fails. -pub fn send(request: HttpRequest) -> Result, Error> { +pub fn send(request: HTTPRequest) -> Result, Error> { let response = unsafe { http_send(Json(HTTPSendRequest { request: request, From 582d1b3cd90a0f913976a706bc9ea5e64d411b76 Mon Sep 17 00:00:00 2001 From: Deluan Date: Thu, 26 Feb 2026 16:30:50 -0500 Subject: [PATCH 17/60] refactor(plugins): validate scheduler capability at load time Move scheduler capability check from runtime (when callback fires) to load-time validation in ValidateWithCapabilities. This ensures plugins declaring the scheduler permission must export the nd_scheduler_callback function, failing fast with a clear error instead of silently skipping callbacks at runtime. --- plugins/host_scheduler.go | 6 ------ plugins/manifest.go | 8 ++++++++ 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/plugins/host_scheduler.go b/plugins/host_scheduler.go index e7c97c271..131f56521 100644 --- a/plugins/host_scheduler.go +++ b/plugins/host_scheduler.go @@ -188,12 +188,6 @@ func (s *schedulerServiceImpl) invokeCallback(ctx context.Context, scheduleID st return } - // Check if plugin has the scheduler capability - if !hasCapability(instance.capabilities, CapabilityScheduler) { - log.Warn(ctx, "Plugin does not have scheduler capability", "plugin", s.pluginName, "scheduleID", scheduleID) - return - } - // Prepare callback input input := capabilities.SchedulerCallbackRequest{ ScheduleID: scheduleID, diff --git a/plugins/manifest.go b/plugins/manifest.go index f401a7f69..3ca2657cd 100644 --- a/plugins/manifest.go +++ b/plugins/manifest.go @@ -64,6 +64,14 @@ func ValidateWithCapabilities(m *Manifest, capabilities []Capability) error { return fmt.Errorf("scrobbler capability requires 'users' permission to be declared in manifest") } } + + // Scheduler permission requires SchedulerCallback capability + if m.Permissions != nil && m.Permissions.Scheduler != nil { + if !hasCapability(capabilities, CapabilityScheduler) { + return fmt.Errorf("'scheduler' permission requires plugin to export '%s' function", FuncSchedulerCallback) + } + } + return nil } From bd8032b3274a02fef7dbafb51c8ea6b5970b3931 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Fri, 27 Feb 2026 19:00:19 -0500 Subject: [PATCH 18/60] fix(plugins): add base64 handling for []byte and remove raw=true (#5121) * fix(plugins): add base64 handling for []byte and remove raw=true Go's json.Marshal automatically base64-encodes []byte fields, but Rust's serde_json serializes Vec as a JSON array and Python's json.dumps raises TypeError on bytes. This fixes both directions of plugin communication by adding proper base64 encoding/decoding in generated client code. For Rust templates (client and capability): adds a base64_bytes serde helper module with #[serde(with = "base64_bytes")] on all Vec fields, and adds base64 as a dependency. For Python templates: wraps bytes params with base64.b64encode() and responses with base64.b64decode(). Also removes the raw=true binary framing protocol from all templates, the parser, and the Method type. The raw mechanism added complexity that is no longer needed once []byte works properly over JSON. * fix(plugins): update production code and tests for base64 migration Remove raw=true annotation from SubsonicAPI.CallRaw, delete all raw test fixtures, remove raw-related test cases from parser, generator, and integration tests, and add new test cases validating base64 handling for Rust and Python templates. * fix(plugins): update golden files and regenerate production code Update golden test fixtures for codec and comprehensive services to include base64 handling for []byte fields. Regenerate all production PDK code (Go, Rust, Python) and host wrappers to use standard JSON with base64-encoded byte fields instead of binary framing protocol. * refactor: remove base64 helper duplication from rust template Signed-off-by: Deluan * fix(plugins): add base64 dependency to capabilities' Cargo.toml Signed-off-by: Deluan --------- Signed-off-by: Deluan --- plugins/cmd/ndpgen/integration_test.go | 3 - plugins/cmd/ndpgen/internal/generator.go | 18 ++ plugins/cmd/ndpgen/internal/generator_test.go | 266 +++++------------- plugins/cmd/ndpgen/internal/parser.go | 8 - plugins/cmd/ndpgen/internal/parser_test.go | 113 -------- .../internal/templates/base64_bytes.rs.tmpl | 25 ++ .../internal/templates/capability.rs.tmpl | 4 + .../ndpgen/internal/templates/client.go.tmpl | 27 +- .../ndpgen/internal/templates/client.py.tmpl | 45 ++- .../ndpgen/internal/templates/client.rs.tmpl | 88 +----- .../ndpgen/internal/templates/host.go.tmpl | 50 +--- plugins/cmd/ndpgen/internal/types.go | 57 +++- .../ndpgen/testdata/codec_client_expected.py | 5 +- .../ndpgen/testdata/codec_client_expected.rs | 25 ++ .../testdata/comprehensive_client_expected.py | 5 +- .../testdata/comprehensive_client_expected.rs | 25 ++ .../testdata/raw_client_expected.go.txt | 66 ----- .../ndpgen/testdata/raw_client_expected.py | 63 ----- .../ndpgen/testdata/raw_client_expected.rs | 73 ----- .../cmd/ndpgen/testdata/raw_service.go.txt | 10 - plugins/host/subsonicapi.go | 6 +- plugins/host/subsonicapi_gen.go | 44 +-- plugins/pdk/go/host/nd_host_subsonicapi.go | 37 ++- .../pdk/go/host/nd_host_subsonicapi_stub.go | 4 +- plugins/pdk/python/host/nd_host_cache.py | 5 +- plugins/pdk/python/host/nd_host_http.py | 1 + plugins/pdk/python/host/nd_host_kvstore.py | 5 +- .../pdk/python/host/nd_host_subsonicapi.py | 42 +-- plugins/pdk/python/host/nd_host_websocket.py | 3 +- .../pdk/rust/nd-pdk-capabilities/Cargo.toml | 1 + plugins/pdk/rust/nd-pdk-host/Cargo.toml | 1 + .../pdk/rust/nd-pdk-host/src/nd_host_cache.rs | 25 ++ .../pdk/rust/nd-pdk-host/src/nd_host_http.rs | 25 ++ .../rust/nd-pdk-host/src/nd_host_kvstore.rs | 25 ++ .../nd-pdk-host/src/nd_host_subsonicapi.rs | 90 +++--- .../rust/nd-pdk-host/src/nd_host_websocket.rs | 24 ++ 36 files changed, 460 insertions(+), 854 deletions(-) create mode 100644 plugins/cmd/ndpgen/internal/templates/base64_bytes.rs.tmpl delete mode 100644 plugins/cmd/ndpgen/testdata/raw_client_expected.go.txt delete mode 100644 plugins/cmd/ndpgen/testdata/raw_client_expected.py delete mode 100644 plugins/cmd/ndpgen/testdata/raw_client_expected.rs delete mode 100644 plugins/cmd/ndpgen/testdata/raw_service.go.txt diff --git a/plugins/cmd/ndpgen/integration_test.go b/plugins/cmd/ndpgen/integration_test.go index 13ebe14a4..db500c1fc 100644 --- a/plugins/cmd/ndpgen/integration_test.go +++ b/plugins/cmd/ndpgen/integration_test.go @@ -282,9 +282,6 @@ type ServiceB interface { Entry("option pattern (value, exists bool)", "config_service.go.txt", "config_client_expected.go.txt", "config_client_expected.py", "config_client_expected.rs"), - - Entry("raw=true binary response", - "raw_service.go.txt", "raw_client_expected.go.txt", "raw_client_expected.py", "raw_client_expected.rs"), ) It("generates compilable client code for comprehensive service", func() { diff --git a/plugins/cmd/ndpgen/internal/generator.go b/plugins/cmd/ndpgen/internal/generator.go index 69e232565..705cd4d36 100644 --- a/plugins/cmd/ndpgen/internal/generator.go +++ b/plugins/cmd/ndpgen/internal/generator.go @@ -256,6 +256,15 @@ func GenerateClientRust(svc Service) ([]byte, error) { return nil, fmt.Errorf("parsing template: %w", err) } + partialContent, err := templatesFS.ReadFile("templates/base64_bytes.rs.tmpl") + if err != nil { + return nil, fmt.Errorf("reading base64_bytes partial: %w", err) + } + tmpl, err = tmpl.Parse(string(partialContent)) + if err != nil { + return nil, fmt.Errorf("parsing base64_bytes partial: %w", err) + } + data := templateData{ Service: svc, } @@ -622,6 +631,15 @@ func GenerateCapabilityRust(cap Capability) ([]byte, error) { return nil, fmt.Errorf("parsing template: %w", err) } + partialContent, err := templatesFS.ReadFile("templates/base64_bytes.rs.tmpl") + if err != nil { + return nil, fmt.Errorf("reading base64_bytes partial: %w", err) + } + tmpl, err = tmpl.Parse(string(partialContent)) + if err != nil { + return nil, fmt.Errorf("parsing base64_bytes partial: %w", err) + } + data := capabilityTemplateData{ Package: cap.Name, Capability: cap, diff --git a/plugins/cmd/ndpgen/internal/generator_test.go b/plugins/cmd/ndpgen/internal/generator_test.go index ed15f174b..34c2c2886 100644 --- a/plugins/cmd/ndpgen/internal/generator_test.go +++ b/plugins/cmd/ndpgen/internal/generator_test.go @@ -264,96 +264,6 @@ var _ = Describe("Generator", func() { Expect(codeStr).To(ContainSubstring(`extism "github.com/extism/go-sdk"`)) }) - It("should generate binary framing for raw=true methods", func() { - svc := Service{ - Name: "Stream", - Permission: "stream", - Interface: "StreamService", - Methods: []Method{ - { - Name: "GetStream", - HasError: true, - Raw: true, - Params: []Param{NewParam("uri", "string")}, - Returns: []Param{ - NewParam("contentType", "string"), - NewParam("data", "[]byte"), - }, - }, - }, - } - - code, err := GenerateHost(svc, "host") - Expect(err).NotTo(HaveOccurred()) - - _, err = format.Source(code) - Expect(err).NotTo(HaveOccurred()) - - codeStr := string(code) - - // Should include encoding/binary import for raw methods - Expect(codeStr).To(ContainSubstring(`"encoding/binary"`)) - - // Should NOT generate a response type for raw methods - Expect(codeStr).NotTo(ContainSubstring("type StreamGetStreamResponse struct")) - - // Should generate request type (request is still JSON) - Expect(codeStr).To(ContainSubstring("type StreamGetStreamRequest struct")) - - // Should build binary frame [0x00][4-byte CT len][CT][data] - Expect(codeStr).To(ContainSubstring("frame[0] = 0x00")) - Expect(codeStr).To(ContainSubstring("binary.BigEndian.PutUint32")) - - // Should have writeRawError helper - Expect(codeStr).To(ContainSubstring("streamWriteRawError")) - - // Should use writeRawError instead of writeError for raw methods - Expect(codeStr).To(ContainSubstring("streamWriteRawError(p, stack")) - }) - - It("should generate both writeError and writeRawError for mixed services", func() { - svc := Service{ - Name: "API", - Permission: "api", - Interface: "APIService", - Methods: []Method{ - { - Name: "Call", - HasError: true, - Params: []Param{NewParam("uri", "string")}, - Returns: []Param{NewParam("response", "string")}, - }, - { - Name: "CallRaw", - HasError: true, - Raw: true, - Params: []Param{NewParam("uri", "string")}, - Returns: []Param{ - NewParam("contentType", "string"), - NewParam("data", "[]byte"), - }, - }, - }, - } - - code, err := GenerateHost(svc, "host") - Expect(err).NotTo(HaveOccurred()) - - _, err = format.Source(code) - Expect(err).NotTo(HaveOccurred()) - - codeStr := string(code) - - // Should have both helpers - Expect(codeStr).To(ContainSubstring("apiWriteResponse")) - Expect(codeStr).To(ContainSubstring("apiWriteError")) - Expect(codeStr).To(ContainSubstring("apiWriteRawError")) - - // Should generate response type for non-raw method only - Expect(codeStr).To(ContainSubstring("type APICallResponse struct")) - Expect(codeStr).NotTo(ContainSubstring("type APICallRawResponse struct")) - }) - It("should always include json import for JSON protocol", func() { // All services use JSON protocol, so json import is always needed svc := Service{ @@ -717,49 +627,7 @@ var _ = Describe("Generator", func() { Expect(codeStr).To(ContainSubstring(`response.get("boolVal", False)`)) }) - It("should generate binary frame parsing for raw methods", func() { - svc := Service{ - Name: "Stream", - Permission: "stream", - Interface: "StreamService", - Methods: []Method{ - { - Name: "GetStream", - HasError: true, - Raw: true, - Params: []Param{NewParam("uri", "string")}, - Returns: []Param{ - NewParam("contentType", "string"), - NewParam("data", "[]byte"), - }, - Doc: "GetStream returns raw binary stream data.", - }, - }, - } - - code, err := GenerateClientPython(svc) - Expect(err).NotTo(HaveOccurred()) - - codeStr := string(code) - - // Should import Tuple and struct for raw methods - Expect(codeStr).To(ContainSubstring("from typing import Any, Tuple")) - Expect(codeStr).To(ContainSubstring("import struct")) - - // Should return Tuple[str, bytes] - Expect(codeStr).To(ContainSubstring("-> Tuple[str, bytes]:")) - - // Should parse binary frame instead of JSON - Expect(codeStr).To(ContainSubstring("response_bytes = response_mem.bytes()")) - Expect(codeStr).To(ContainSubstring("response_bytes[0] == 0x01")) - Expect(codeStr).To(ContainSubstring("struct.unpack")) - Expect(codeStr).To(ContainSubstring("return content_type, data")) - - // Should NOT use json.loads for response - Expect(codeStr).NotTo(ContainSubstring("json.loads(extism.memory.string(response_mem))")) - }) - - It("should not import Tuple or struct for non-raw services", func() { + It("should not import base64 for non-byte services", func() { svc := Service{ Name: "Test", Permission: "test", @@ -779,8 +647,37 @@ var _ = Describe("Generator", func() { codeStr := string(code) - Expect(codeStr).NotTo(ContainSubstring("Tuple")) - Expect(codeStr).NotTo(ContainSubstring("import struct")) + Expect(codeStr).NotTo(ContainSubstring("import base64")) + }) + + It("should generate base64 encoding/decoding for byte fields", func() { + svc := Service{ + Name: "Codec", + Permission: "codec", + Interface: "CodecService", + Methods: []Method{ + { + Name: "Encode", + HasError: true, + Params: []Param{NewParam("data", "[]byte")}, + Returns: []Param{NewParam("result", "[]byte")}, + }, + }, + } + + code, err := GenerateClientPython(svc) + Expect(err).NotTo(HaveOccurred()) + + codeStr := string(code) + + // Should import base64 + Expect(codeStr).To(ContainSubstring("import base64")) + + // Should base64-encode byte params in request + Expect(codeStr).To(ContainSubstring(`base64.b64encode(data).decode("ascii")`)) + + // Should base64-decode byte returns in response + Expect(codeStr).To(ContainSubstring(`base64.b64decode(response.get("result", ""))`)) }) }) @@ -939,46 +836,6 @@ var _ = Describe("Generator", func() { Expect(codeStr).To(ContainSubstring("github.com/navidrome/navidrome/plugins/pdk/go/pdk")) }) - It("should include encoding/binary import for raw methods", func() { - svc := Service{ - Name: "Stream", - Permission: "stream", - Interface: "StreamService", - Methods: []Method{ - { - Name: "GetStream", - HasError: true, - Raw: true, - Params: []Param{NewParam("uri", "string")}, - Returns: []Param{ - NewParam("contentType", "string"), - NewParam("data", "[]byte"), - }, - }, - }, - } - - code, err := GenerateClientGo(svc, "host") - Expect(err).NotTo(HaveOccurred()) - - codeStr := string(code) - - // Should include encoding/binary for raw binary frame parsing - Expect(codeStr).To(ContainSubstring(`"encoding/binary"`)) - - // Should NOT generate response type struct for raw methods - Expect(codeStr).NotTo(ContainSubstring("streamGetStreamResponse struct")) - - // Should still generate request type - Expect(codeStr).To(ContainSubstring("streamGetStreamRequest struct")) - - // Should parse binary frame - Expect(codeStr).To(ContainSubstring("responseBytes[0] == 0x01")) - Expect(codeStr).To(ContainSubstring("binary.BigEndian.Uint32")) - - // Should return (string, []byte, error) - Expect(codeStr).To(ContainSubstring("func StreamGetStream(uri string) (string, []byte, error)")) - }) }) Describe("GenerateClientGoStub", func() { @@ -1748,22 +1605,17 @@ var _ = Describe("Rust Generation", func() { Expect(codeStr).NotTo(ContainSubstring("Option")) }) - It("should generate raw extern C import and binary frame parsing for raw methods", func() { + It("should generate base64 serde for Vec fields", func() { svc := Service{ - Name: "Stream", - Permission: "stream", - Interface: "StreamService", + Name: "Codec", + Permission: "codec", + Interface: "CodecService", Methods: []Method{ { - Name: "GetStream", + Name: "Encode", HasError: true, - Raw: true, - Params: []Param{NewParam("uri", "string")}, - Returns: []Param{ - NewParam("contentType", "string"), - NewParam("data", "[]byte"), - }, - Doc: "GetStream returns raw binary stream data.", + Params: []Param{NewParam("data", "[]byte")}, + Returns: []Param{NewParam("result", "[]byte")}, }, }, } @@ -1773,24 +1625,36 @@ var _ = Describe("Rust Generation", func() { codeStr := string(code) - // Should use extern "C" with wasm_import_module for raw methods, not #[host_fn] extern "ExtismHost" - Expect(codeStr).To(ContainSubstring(`#[link(wasm_import_module = "extism:host/user")]`)) - Expect(codeStr).To(ContainSubstring(`extern "C"`)) - Expect(codeStr).To(ContainSubstring("fn stream_getstream(offset: u64) -> u64")) + // Should generate base64_bytes serde module + Expect(codeStr).To(ContainSubstring("mod base64_bytes")) + Expect(codeStr).To(ContainSubstring("use base64::Engine as _")) - // Should NOT generate response type for raw methods - Expect(codeStr).NotTo(ContainSubstring("StreamGetStreamResponse")) + // Should add serde(with = "base64_bytes") on Vec fields + Expect(codeStr).To(ContainSubstring(`#[serde(with = "base64_bytes")]`)) + }) - // Should generate request type (request is still JSON) - Expect(codeStr).To(ContainSubstring("struct StreamGetStreamRequest")) + It("should not generate base64 module when no byte fields", func() { + svc := Service{ + Name: "Test", + Permission: "test", + Interface: "TestService", + Methods: []Method{ + { + Name: "Call", + HasError: true, + Params: []Param{NewParam("uri", "string")}, + Returns: []Param{NewParam("response", "string")}, + }, + }, + } - // Should return Result<(String, Vec), Error> - Expect(codeStr).To(ContainSubstring("Result<(String, Vec), Error>")) + code, err := GenerateClientRust(svc) + Expect(err).NotTo(HaveOccurred()) - // Should parse binary frame - Expect(codeStr).To(ContainSubstring("response_bytes[0] == 0x01")) - Expect(codeStr).To(ContainSubstring("u32::from_be_bytes")) - Expect(codeStr).To(ContainSubstring("String::from_utf8_lossy")) + codeStr := string(code) + + Expect(codeStr).NotTo(ContainSubstring("mod base64_bytes")) + Expect(codeStr).NotTo(ContainSubstring("use base64")) }) }) }) diff --git a/plugins/cmd/ndpgen/internal/parser.go b/plugins/cmd/ndpgen/internal/parser.go index c2d571779..4cb28f8d4 100644 --- a/plugins/cmd/ndpgen/internal/parser.go +++ b/plugins/cmd/ndpgen/internal/parser.go @@ -761,7 +761,6 @@ func parseMethod(name string, funcType *ast.FuncType, annotation map[string]stri m := Method{ Name: name, ExportName: annotation["name"], - Raw: annotation["raw"] == "true", Doc: doc, } @@ -800,13 +799,6 @@ func parseMethod(name string, funcType *ast.FuncType, annotation map[string]stri } } - // Validate raw=true methods: must return exactly (string, []byte, error) - if m.Raw { - if !m.HasError || len(m.Returns) != 2 || m.Returns[0].Type != "string" || m.Returns[1].Type != "[]byte" { - return m, fmt.Errorf("raw=true method %s must return (string, []byte, error) — content-type, data, error", name) - } - } - return m, nil } diff --git a/plugins/cmd/ndpgen/internal/parser_test.go b/plugins/cmd/ndpgen/internal/parser_test.go index f2bdbeded..f43578397 100644 --- a/plugins/cmd/ndpgen/internal/parser_test.go +++ b/plugins/cmd/ndpgen/internal/parser_test.go @@ -122,119 +122,6 @@ type TestService interface { Expect(services[0].Methods[0].Name).To(Equal("Exported")) }) - It("should parse raw=true annotation", func() { - src := `package host - -import "context" - -//nd:hostservice name=Stream permission=stream -type StreamService interface { - //nd:hostfunc raw=true - GetStream(ctx context.Context, uri string) (contentType string, data []byte, err error) -} -` - err := os.WriteFile(filepath.Join(tmpDir, "stream.go"), []byte(src), 0600) - Expect(err).NotTo(HaveOccurred()) - - services, err := ParseDirectory(tmpDir) - Expect(err).NotTo(HaveOccurred()) - Expect(services).To(HaveLen(1)) - - m := services[0].Methods[0] - Expect(m.Name).To(Equal("GetStream")) - Expect(m.Raw).To(BeTrue()) - Expect(m.HasError).To(BeTrue()) - Expect(m.Returns).To(HaveLen(2)) - Expect(m.Returns[0].Name).To(Equal("contentType")) - Expect(m.Returns[0].Type).To(Equal("string")) - Expect(m.Returns[1].Name).To(Equal("data")) - Expect(m.Returns[1].Type).To(Equal("[]byte")) - }) - - It("should set Raw=false when raw annotation is absent", func() { - src := `package host - -import "context" - -//nd:hostservice name=Test permission=test -type TestService interface { - //nd:hostfunc - Call(ctx context.Context, uri string) (response string, err error) -} -` - err := os.WriteFile(filepath.Join(tmpDir, "test.go"), []byte(src), 0600) - Expect(err).NotTo(HaveOccurred()) - - services, err := ParseDirectory(tmpDir) - Expect(err).NotTo(HaveOccurred()) - Expect(services[0].Methods[0].Raw).To(BeFalse()) - }) - - It("should reject raw=true with invalid return signature", func() { - src := `package host - -import "context" - -//nd:hostservice name=Test permission=test -type TestService interface { - //nd:hostfunc raw=true - BadRaw(ctx context.Context, uri string) (result string, err error) -} -` - err := os.WriteFile(filepath.Join(tmpDir, "test.go"), []byte(src), 0600) - Expect(err).NotTo(HaveOccurred()) - - _, err = ParseDirectory(tmpDir) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("raw=true")) - Expect(err.Error()).To(ContainSubstring("must return (string, []byte, error)")) - }) - - It("should reject raw=true without error return", func() { - src := `package host - -import "context" - -//nd:hostservice name=Test permission=test -type TestService interface { - //nd:hostfunc raw=true - BadRaw(ctx context.Context, uri string) (contentType string, data []byte) -} -` - err := os.WriteFile(filepath.Join(tmpDir, "test.go"), []byte(src), 0600) - Expect(err).NotTo(HaveOccurred()) - - _, err = ParseDirectory(tmpDir) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("raw=true")) - }) - - It("should parse mixed raw and non-raw methods", func() { - src := `package host - -import "context" - -//nd:hostservice name=API permission=api -type APIService interface { - //nd:hostfunc - Call(ctx context.Context, uri string) (responseJSON string, err error) - - //nd:hostfunc raw=true - CallRaw(ctx context.Context, uri string) (contentType string, data []byte, err error) -} -` - err := os.WriteFile(filepath.Join(tmpDir, "api.go"), []byte(src), 0600) - Expect(err).NotTo(HaveOccurred()) - - services, err := ParseDirectory(tmpDir) - Expect(err).NotTo(HaveOccurred()) - Expect(services).To(HaveLen(1)) - Expect(services[0].Methods).To(HaveLen(2)) - Expect(services[0].Methods[0].Raw).To(BeFalse()) - Expect(services[0].Methods[1].Raw).To(BeTrue()) - Expect(services[0].HasRawMethods()).To(BeTrue()) - }) - It("should handle custom export name", func() { src := `package host diff --git a/plugins/cmd/ndpgen/internal/templates/base64_bytes.rs.tmpl b/plugins/cmd/ndpgen/internal/templates/base64_bytes.rs.tmpl new file mode 100644 index 000000000..929aa8e3e --- /dev/null +++ b/plugins/cmd/ndpgen/internal/templates/base64_bytes.rs.tmpl @@ -0,0 +1,25 @@ +{{define "base64_bytes_module"}} +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} +{{- end}} \ No newline at end of file diff --git a/plugins/cmd/ndpgen/internal/templates/capability.rs.tmpl b/plugins/cmd/ndpgen/internal/templates/capability.rs.tmpl index 597a17338..790ed93e4 100644 --- a/plugins/cmd/ndpgen/internal/templates/capability.rs.tmpl +++ b/plugins/cmd/ndpgen/internal/templates/capability.rs.tmpl @@ -7,6 +7,7 @@ use serde::{Deserialize, Serialize}; {{- if hasHashMap .Capability}} use std::collections::HashMap; {{- end}} +{{- if .Capability.HasByteFields}}{{template "base64_bytes_module" .}}{{- end}} // Helper functions for skip_serializing_if with numeric types #[allow(dead_code)] @@ -70,6 +71,9 @@ pub struct {{.Name}} { #[serde(default, skip_serializing_if = "{{skipSerializingFunc .Type}}")] {{- else}} #[serde(default)] +{{- end}} +{{- if .IsByteSlice}} + #[serde(with = "base64_bytes")] {{- end}} pub {{rustFieldName .Name}}: {{fieldRustType .}}, {{- end}} diff --git a/plugins/cmd/ndpgen/internal/templates/client.go.tmpl b/plugins/cmd/ndpgen/internal/templates/client.go.tmpl index 971ae394a..a6ee04446 100644 --- a/plugins/cmd/ndpgen/internal/templates/client.go.tmpl +++ b/plugins/cmd/ndpgen/internal/templates/client.go.tmpl @@ -8,9 +8,6 @@ package {{.Package}} import ( -{{- if .Service.HasRawMethods}} - "encoding/binary" -{{- end}} "encoding/json" {{- if .Service.HasErrors}} "errors" @@ -52,7 +49,7 @@ type {{requestType .}} struct { {{- end}} } {{- end}} -{{- if and (not .IsErrorOnly) (not .Raw)}} +{{- if not .IsErrorOnly}} type {{responseType .}} struct { {{- range .Returns}} @@ -98,27 +95,7 @@ func {{$.Service.Name}}{{.Name}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{ // Read the response from memory responseMem := pdk.FindMemory(responsePtr) responseBytes := responseMem.ReadBytes() -{{- if .Raw}} - - // Parse binary-framed response - if len(responseBytes) == 0 { - return "", nil, errors.New("empty response from host") - } - if responseBytes[0] == 0x01 { // error - return "", nil, errors.New(string(responseBytes[1:])) - } - if responseBytes[0] != 0x00 { - return "", nil, errors.New("unknown response status") - } - if len(responseBytes) < 5 { - return "", nil, errors.New("malformed raw response: incomplete header") - } - ctLen := binary.BigEndian.Uint32(responseBytes[1:5]) - if uint32(len(responseBytes)) < 5+ctLen { - return "", nil, errors.New("malformed raw response: content-type overflow") - } - return string(responseBytes[5 : 5+ctLen]), responseBytes[5+ctLen:], nil -{{- else if .IsErrorOnly}} +{{- if .IsErrorOnly}} // Parse error-only response var response struct { diff --git a/plugins/cmd/ndpgen/internal/templates/client.py.tmpl b/plugins/cmd/ndpgen/internal/templates/client.py.tmpl index 84bff4abe..7ccaa6106 100644 --- a/plugins/cmd/ndpgen/internal/templates/client.py.tmpl +++ b/plugins/cmd/ndpgen/internal/templates/client.py.tmpl @@ -8,12 +8,12 @@ # main __init__.py file. Copy the needed functions from this file into your plugin. from dataclasses import dataclass -from typing import Any{{- if .Service.HasRawMethods}}, Tuple{{end}} +from typing import Any import extism import json -{{- if .Service.HasRawMethods}} -import struct +{{- if .Service.HasByteFields}} +import base64 {{- end}} @@ -32,7 +32,7 @@ def _{{exportName .}}(offset: int) -> int: {{- end}} {{- /* Generate dataclasses for multi-value returns */ -}} {{range .Service.Methods}} -{{- if and .NeedsResultClass (not .Raw)}} +{{- if .NeedsResultClass}} @dataclass @@ -47,7 +47,7 @@ class {{pythonResultType .}}: {{range .Service.Methods}} -def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonName}}: {{$p.PythonType}}{{end}}){{if .Raw}} -> Tuple[str, bytes]{{else if .NeedsResultClass}} -> {{pythonResultType .}}{{else if .HasReturns}} -> {{(index .Returns 0).PythonType}}{{else}} -> None{{end}}: +def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonName}}: {{$p.PythonType}}{{end}}){{if .NeedsResultClass}} -> {{pythonResultType .}}{{else if .HasReturns}} -> {{(index .Returns 0).PythonType}}{{else}} -> None{{end}}: """{{if .Doc}}{{.Doc}}{{else}}Call the {{exportName .}} host function.{{end}} {{- if .HasParams}} @@ -56,11 +56,7 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam {{.PythonName}}: {{.PythonType}} parameter. {{- end}} {{- end}} -{{- if .Raw}} - - Returns: - Tuple of (content_type, data) with the raw binary response. -{{- else if .HasReturns}} +{{- if .HasReturns}} Returns: {{- if .NeedsResultClass}} @@ -76,7 +72,11 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam {{- if .HasParams}} request = { {{- range .Params}} +{{- if .IsByteSlice}} + "{{.JSONName}}": base64.b64encode({{.PythonName}}).decode("ascii"), +{{- else}} "{{.JSONName}}": {{.PythonName}}, +{{- end}} {{- end}} } request_bytes = json.dumps(request).encode("utf-8") @@ -86,24 +86,6 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam request_mem = extism.memory.alloc(request_bytes) response_offset = _{{exportName .}}(request_mem.offset) response_mem = extism.memory.find(response_offset) -{{- if .Raw}} - response_bytes = response_mem.bytes() - - if len(response_bytes) == 0: - raise HostFunctionError("empty response from host") - if response_bytes[0] == 0x01: - raise HostFunctionError(response_bytes[1:].decode("utf-8")) - if response_bytes[0] != 0x00: - raise HostFunctionError("unknown response status") - if len(response_bytes) < 5: - raise HostFunctionError("malformed raw response: incomplete header") - ct_len = struct.unpack(">I", response_bytes[1:5])[0] - if len(response_bytes) < 5 + ct_len: - raise HostFunctionError("malformed raw response: content-type overflow") - content_type = response_bytes[5:5 + ct_len].decode("utf-8") - data = response_bytes[5 + ct_len:] - return content_type, data -{{- else}} response = json.loads(extism.memory.string(response_mem)) {{if .HasError}} if response.get("error"): @@ -112,10 +94,17 @@ def {{pythonFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.PythonNam {{- if .NeedsResultClass}} return {{pythonResultType .}}( {{- range .Returns}} +{{- if .IsByteSlice}} + {{.PythonName}}=base64.b64decode(response.get("{{.JSONName}}", "")), +{{- else}} {{.PythonName}}=response.get("{{.JSONName}}"{{pythonDefault .}}), +{{- end}} {{- end}} ) {{- else if .HasReturns}} +{{- if (index .Returns 0).IsByteSlice}} + return base64.b64decode(response.get("{{(index .Returns 0).JSONName}}", "")) +{{- else}} return response.get("{{(index .Returns 0).JSONName}}"{{pythonDefault (index .Returns 0)}}) {{- end}} {{- end}} diff --git a/plugins/cmd/ndpgen/internal/templates/client.rs.tmpl b/plugins/cmd/ndpgen/internal/templates/client.rs.tmpl index 2fb368bf1..f8b786849 100644 --- a/plugins/cmd/ndpgen/internal/templates/client.rs.tmpl +++ b/plugins/cmd/ndpgen/internal/templates/client.rs.tmpl @@ -5,6 +5,7 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +{{- if .Service.HasByteFields}}{{template "base64_bytes_module" .}}{{- end}} {{- /* Generate struct definitions */ -}} {{- range .Service.Structs}} {{if .Doc}} @@ -16,6 +17,9 @@ pub struct {{.Name}} { {{- range .Fields}} {{- if .NeedsDefault}} #[serde(default)] +{{- end}} +{{- if .IsByteSlice}} + #[serde(with = "base64_bytes")] {{- end}} pub {{.RustName}}: {{fieldRustType .}}, {{- end}} @@ -29,17 +33,22 @@ pub struct {{.Name}} { #[serde(rename_all = "camelCase")] struct {{requestType .}} { {{- range .Params}} +{{- if .IsByteSlice}} + #[serde(with = "base64_bytes")] +{{- end}} {{.RustName}}: {{rustType .}}, {{- end}} } {{- end}} -{{- if not .Raw}} #[derive(Debug, Clone, Deserialize)] #[serde(rename_all = "camelCase")] struct {{responseType .}} { {{- range .Returns}} #[serde(default)] +{{- if .IsByteSlice}} + #[serde(with = "base64_bytes")] +{{- end}} {{.RustName}}: {{rustType .}}, {{- end}} {{- if .HasError}} @@ -48,92 +57,16 @@ struct {{responseType .}} { {{- end}} } {{- end}} -{{- end}} #[host_fn] extern "ExtismHost" { {{- range .Service.Methods}} -{{- if not .Raw}} fn {{exportName .}}(input: Json<{{if .HasParams}}{{requestType .}}{{else}}serde_json::Value{{end}}>) -> Json<{{responseType .}}>; {{- end}} -{{- end}} } -{{- /* Declare raw extern "C" imports for raw methods */ -}} -{{- range .Service.Methods}} -{{- if .Raw}} - -#[link(wasm_import_module = "extism:host/user")] -extern "C" { - fn {{exportName .}}(offset: u64) -> u64; -} -{{- end}} -{{- end}} {{- /* Generate wrapper functions */ -}} {{range .Service.Methods}} -{{- if .Raw}} - -{{if .Doc}}{{rustDocComment .Doc}}{{else}}/// Calls the {{exportName .}} host function.{{end}} -{{- if .HasParams}} -/// -/// # Arguments -{{- range .Params}} -/// * `{{.RustName}}` - {{rustType .}} parameter. -{{- end}} -{{- end}} -/// -/// # Returns -/// A tuple of (content_type, data) with the raw binary response. -/// -/// # Errors -/// Returns an error if the host function call fails. -pub fn {{rustFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.RustName}}: {{rustParamType $p}}{{end}}) -> Result<(String, Vec), Error> { -{{- if .HasParams}} - let req = {{requestType .}} { -{{- range .Params}} - {{.RustName}}: {{.RustName}}{{if .NeedsToOwned}}.to_owned(){{end}}, -{{- end}} - }; - let input_bytes = serde_json::to_vec(&req).map_err(|e| Error::msg(e.to_string()))?; -{{- else}} - let input_bytes = b"{}".to_vec(); -{{- end}} - let input_mem = Memory::from_bytes(&input_bytes).map_err(|e| Error::msg(e.to_string()))?; - - let response_offset = unsafe { {{exportName .}}(input_mem.offset()) }; - - let response_mem = Memory::find(response_offset) - .ok_or_else(|| Error::msg("empty response from host"))?; - let response_bytes = response_mem.to_vec(); - - if response_bytes.is_empty() { - return Err(Error::msg("empty response from host")); - } - if response_bytes[0] == 0x01 { - let msg = String::from_utf8_lossy(&response_bytes[1..]).to_string(); - return Err(Error::msg(msg)); - } - if response_bytes[0] != 0x00 { - return Err(Error::msg("unknown response status")); - } - if response_bytes.len() < 5 { - return Err(Error::msg("malformed raw response: incomplete header")); - } - let ct_len = u32::from_be_bytes([ - response_bytes[1], - response_bytes[2], - response_bytes[3], - response_bytes[4], - ]) as usize; - if ct_len > response_bytes.len() - 5 { - return Err(Error::msg("malformed raw response: content-type overflow")); - } - let ct_end = 5 + ct_len; - let content_type = String::from_utf8_lossy(&response_bytes[5..ct_end]).to_string(); - let data = response_bytes[ct_end..].to_vec(); - Ok((content_type, data)) -} -{{- else}} {{if .Doc}}{{rustDocComment .Doc}}{{else}}/// Calls the {{exportName .}} host function.{{end}} {{- if .HasParams}} @@ -209,4 +142,3 @@ pub fn {{rustFunc .}}({{range $i, $p := .Params}}{{if $i}}, {{end}}{{$p.RustName } {{- end}} {{- end}} -{{- end}} diff --git a/plugins/cmd/ndpgen/internal/templates/host.go.tmpl b/plugins/cmd/ndpgen/internal/templates/host.go.tmpl index 12dd20475..083f7577e 100644 --- a/plugins/cmd/ndpgen/internal/templates/host.go.tmpl +++ b/plugins/cmd/ndpgen/internal/templates/host.go.tmpl @@ -4,9 +4,6 @@ package {{.Package}} import ( "context" -{{- if .Service.HasRawMethods}} - "encoding/binary" -{{- end}} "encoding/json" extism "github.com/extism/go-sdk" @@ -23,7 +20,6 @@ type {{requestType .}} struct { {{- end}} } {{- end}} -{{- if not .Raw}} // {{responseType .}} is the response type for {{$.Service.Name}}.{{.Name}}. type {{responseType .}} struct { @@ -34,7 +30,6 @@ type {{responseType .}} struct { Error string `json:"error,omitempty"` {{- end}} } -{{- end}} {{end}} // Register{{.Service.Name}}HostFunctions registers {{.Service.Name}} service host functions. @@ -56,48 +51,18 @@ func new{{$.Service.Name}}{{.Name}}HostFunction(service {{$.Service.Interface}}) // Read JSON request from plugin memory reqBytes, err := p.ReadBytes(stack[0]) if err != nil { -{{- if .Raw}} - {{$.Service.Name | lower}}WriteRawError(p, stack, err) -{{- else}} {{$.Service.Name | lower}}WriteError(p, stack, err) -{{- end}} return } var req {{requestType .}} if err := json.Unmarshal(reqBytes, &req); err != nil { -{{- if .Raw}} - {{$.Service.Name | lower}}WriteRawError(p, stack, err) -{{- else}} {{$.Service.Name | lower}}WriteError(p, stack, err) -{{- end}} return } {{- end}} // Call the service method -{{- if .Raw}} - {{range $i, $r := .Returns}}{{if $i}}, {{end}}{{lower $r.Name}}{{end}}, svcErr := service.{{.Name}}(ctx{{range .Params}}, req.{{title .Name}}{{end}}) - if svcErr != nil { - {{$.Service.Name | lower}}WriteRawError(p, stack, svcErr) - return - } - - // Write binary-framed response to plugin memory: - // [0x00][4-byte content-type length (big-endian)][content-type string][raw data] - ctBytes := []byte({{lower (index .Returns 0).Name}}) - frame := make([]byte, 1+4+len(ctBytes)+len({{lower (index .Returns 1).Name}})) - frame[0] = 0x00 // success - binary.BigEndian.PutUint32(frame[1:5], uint32(len(ctBytes))) - copy(frame[5:5+len(ctBytes)], ctBytes) - copy(frame[5+len(ctBytes):], {{lower (index .Returns 1).Name}}) - - respPtr, err := p.WriteBytes(frame) - if err != nil { - stack[0] = 0 - return - } - stack[0] = respPtr -{{- else if .HasReturns}} +{{- if .HasReturns}} {{- if .HasError}} {{range $i, $r := .Returns}}{{if $i}}, {{end}}{{lower $r.Name}}{{end}}, svcErr := service.{{.Name}}(ctx{{range .Params}}, req.{{title .Name}}{{end}}) if svcErr != nil { @@ -162,16 +127,3 @@ func {{.Service.Name | lower}}WriteError(p *extism.CurrentPlugin, stack []uint64 respPtr, _ := p.WriteBytes(respBytes) stack[0] = respPtr } -{{- if .Service.HasRawMethods}} - -// {{.Service.Name | lower}}WriteRawError writes a binary-framed error response to plugin memory. -// Format: [0x01][UTF-8 error message] -func {{.Service.Name | lower}}WriteRawError(p *extism.CurrentPlugin, stack []uint64, err error) { - errMsg := []byte(err.Error()) - frame := make([]byte, 1+len(errMsg)) - frame[0] = 0x01 // error - copy(frame[1:], errMsg) - respPtr, _ := p.WriteBytes(frame) - stack[0] = respPtr -} -{{- end}} diff --git a/plugins/cmd/ndpgen/internal/types.go b/plugins/cmd/ndpgen/internal/types.go index 29cf2316a..6132dfbc4 100644 --- a/plugins/cmd/ndpgen/internal/types.go +++ b/plugins/cmd/ndpgen/internal/types.go @@ -173,16 +173,6 @@ func (s Service) HasErrors() bool { return false } -// HasRawMethods returns true if any method in the service uses raw binary framing. -func (s Service) HasRawMethods() bool { - for _, m := range s.Methods { - if m.Raw { - return true - } - } - return false -} - // Method represents a host function method within a service. type Method struct { Name string // Go method name (e.g., "Call") @@ -191,7 +181,6 @@ type Method struct { Returns []Param // Return values (excluding error) HasError bool // Whether the method returns an error Doc string // Documentation comment for the method - Raw bool // If true, response uses binary framing instead of JSON } // FunctionName returns the Extism host function export name. @@ -343,6 +332,52 @@ type Param struct { JSONName string // JSON field name (camelCase) } +// IsByteSlice returns true if the parameter type is []byte. +func (p Param) IsByteSlice() bool { + return p.Type == "[]byte" +} + +// IsByteSlice returns true if the field type is []byte. +func (f FieldDef) IsByteSlice() bool { + return f.Type == "[]byte" +} + +// HasByteFields returns true if any method params, returns, or struct fields use []byte. +func (s Service) HasByteFields() bool { + for _, m := range s.Methods { + for _, p := range m.Params { + if p.IsByteSlice() { + return true + } + } + for _, r := range m.Returns { + if r.IsByteSlice() { + return true + } + } + } + for _, st := range s.Structs { + for _, f := range st.Fields { + if f.IsByteSlice() { + return true + } + } + } + return false +} + +// HasByteFields returns true if any capability struct fields use []byte. +func (c Capability) HasByteFields() bool { + for _, st := range c.Structs { + for _, f := range st.Fields { + if f.IsByteSlice() { + return true + } + } + } + return false +} + // NewParam creates a Param with auto-generated JSON name. func NewParam(name, typ string) Param { return Param{ diff --git a/plugins/cmd/ndpgen/testdata/codec_client_expected.py b/plugins/cmd/ndpgen/testdata/codec_client_expected.py index e1eb92501..5142ffd0e 100644 --- a/plugins/cmd/ndpgen/testdata/codec_client_expected.py +++ b/plugins/cmd/ndpgen/testdata/codec_client_expected.py @@ -12,6 +12,7 @@ from typing import Any import extism import json +import base64 class HostFunctionError(Exception): @@ -38,7 +39,7 @@ def codec_encode(data: bytes) -> bytes: HostFunctionError: If the host function returns an error. """ request = { - "data": data, + "data": base64.b64encode(data).decode("ascii"), } request_bytes = json.dumps(request).encode("utf-8") request_mem = extism.memory.alloc(request_bytes) @@ -49,4 +50,4 @@ def codec_encode(data: bytes) -> bytes: if response.get("error"): raise HostFunctionError(response["error"]) - return response.get("result", b"") + return base64.b64decode(response.get("result", "")) diff --git a/plugins/cmd/ndpgen/testdata/codec_client_expected.rs b/plugins/cmd/ndpgen/testdata/codec_client_expected.rs index ff61294d0..3e229ea8a 100644 --- a/plugins/cmd/ndpgen/testdata/codec_client_expected.rs +++ b/plugins/cmd/ndpgen/testdata/codec_client_expected.rs @@ -5,10 +5,34 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] struct CodecEncodeRequest { + #[serde(with = "base64_bytes")] data: Vec, } @@ -16,6 +40,7 @@ struct CodecEncodeRequest { #[serde(rename_all = "camelCase")] struct CodecEncodeResponse { #[serde(default)] + #[serde(with = "base64_bytes")] result: Vec, #[serde(default)] error: Option, diff --git a/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.py b/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.py index 0fdbfb0f2..93370ddcf 100644 --- a/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.py +++ b/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.py @@ -12,6 +12,7 @@ from typing import Any import extism import json +import base64 class HostFunctionError(Exception): @@ -327,7 +328,7 @@ def comprehensive_byte_slice(data: bytes) -> bytes: HostFunctionError: If the host function returns an error. """ request = { - "data": data, + "data": base64.b64encode(data).decode("ascii"), } request_bytes = json.dumps(request).encode("utf-8") request_mem = extism.memory.alloc(request_bytes) @@ -338,4 +339,4 @@ def comprehensive_byte_slice(data: bytes) -> bytes: if response.get("error"): raise HostFunctionError(response["error"]) - return response.get("result", b"") + return base64.b64decode(response.get("result", "")) diff --git a/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.rs b/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.rs index efcc1b8ef..08dae2901 100644 --- a/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.rs +++ b/plugins/cmd/ndpgen/testdata/comprehensive_client_expected.rs @@ -5,6 +5,29 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] @@ -144,6 +167,7 @@ struct ComprehensiveMultipleReturnsResponse { #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] struct ComprehensiveByteSliceRequest { + #[serde(with = "base64_bytes")] data: Vec, } @@ -151,6 +175,7 @@ struct ComprehensiveByteSliceRequest { #[serde(rename_all = "camelCase")] struct ComprehensiveByteSliceResponse { #[serde(default)] + #[serde(with = "base64_bytes")] result: Vec, #[serde(default)] error: Option, diff --git a/plugins/cmd/ndpgen/testdata/raw_client_expected.go.txt b/plugins/cmd/ndpgen/testdata/raw_client_expected.go.txt deleted file mode 100644 index 22d387041..000000000 --- a/plugins/cmd/ndpgen/testdata/raw_client_expected.go.txt +++ /dev/null @@ -1,66 +0,0 @@ -// Code generated by ndpgen. DO NOT EDIT. -// -// This file contains client wrappers for the Stream host service. -// It is intended for use in Navidrome plugins built with TinyGo. -// -//go:build wasip1 - -package ndpdk - -import ( - "encoding/binary" - "encoding/json" - "errors" - - "github.com/navidrome/navidrome/plugins/pdk/go/pdk" -) - -// stream_getstream is the host function provided by Navidrome. -// -//go:wasmimport extism:host/user stream_getstream -func stream_getstream(uint64) uint64 - -type streamGetStreamRequest struct { - Uri string `json:"uri"` -} - -// StreamGetStream calls the stream_getstream host function. -// GetStream returns raw binary stream data with content type. -func StreamGetStream(uri string) (string, []byte, error) { - // Marshal request to JSON - req := streamGetStreamRequest{ - Uri: uri, - } - reqBytes, err := json.Marshal(req) - if err != nil { - return "", nil, err - } - reqMem := pdk.AllocateBytes(reqBytes) - defer reqMem.Free() - - // Call the host function - responsePtr := stream_getstream(reqMem.Offset()) - - // Read the response from memory - responseMem := pdk.FindMemory(responsePtr) - responseBytes := responseMem.ReadBytes() - - // Parse binary-framed response - if len(responseBytes) == 0 { - return "", nil, errors.New("empty response from host") - } - if responseBytes[0] == 0x01 { // error - return "", nil, errors.New(string(responseBytes[1:])) - } - if responseBytes[0] != 0x00 { - return "", nil, errors.New("unknown response status") - } - if len(responseBytes) < 5 { - return "", nil, errors.New("malformed raw response: incomplete header") - } - ctLen := binary.BigEndian.Uint32(responseBytes[1:5]) - if uint32(len(responseBytes)) < 5+ctLen { - return "", nil, errors.New("malformed raw response: content-type overflow") - } - return string(responseBytes[5 : 5+ctLen]), responseBytes[5+ctLen:], nil -} diff --git a/plugins/cmd/ndpgen/testdata/raw_client_expected.py b/plugins/cmd/ndpgen/testdata/raw_client_expected.py deleted file mode 100644 index 45af2b6c6..000000000 --- a/plugins/cmd/ndpgen/testdata/raw_client_expected.py +++ /dev/null @@ -1,63 +0,0 @@ -# Code generated by ndpgen. DO NOT EDIT. -# -# This file contains client wrappers for the Stream host service. -# It is intended for use in Navidrome plugins built with extism-py. -# -# IMPORTANT: Due to a limitation in extism-py, you cannot import this file directly. -# The @extism.import_fn decorators are only detected when defined in the plugin's -# main __init__.py file. Copy the needed functions from this file into your plugin. - -from dataclasses import dataclass -from typing import Any, Tuple - -import extism -import json -import struct - - -class HostFunctionError(Exception): - """Raised when a host function returns an error.""" - pass - - -@extism.import_fn("extism:host/user", "stream_getstream") -def _stream_getstream(offset: int) -> int: - """Raw host function - do not call directly.""" - ... - - -def stream_get_stream(uri: str) -> Tuple[str, bytes]: - """GetStream returns raw binary stream data with content type. - - Args: - uri: str parameter. - - Returns: - Tuple of (content_type, data) with the raw binary response. - - Raises: - HostFunctionError: If the host function returns an error. - """ - request = { - "uri": uri, - } - request_bytes = json.dumps(request).encode("utf-8") - request_mem = extism.memory.alloc(request_bytes) - response_offset = _stream_getstream(request_mem.offset) - response_mem = extism.memory.find(response_offset) - response_bytes = response_mem.bytes() - - if len(response_bytes) == 0: - raise HostFunctionError("empty response from host") - if response_bytes[0] == 0x01: - raise HostFunctionError(response_bytes[1:].decode("utf-8")) - if response_bytes[0] != 0x00: - raise HostFunctionError("unknown response status") - if len(response_bytes) < 5: - raise HostFunctionError("malformed raw response: incomplete header") - ct_len = struct.unpack(">I", response_bytes[1:5])[0] - if len(response_bytes) < 5 + ct_len: - raise HostFunctionError("malformed raw response: content-type overflow") - content_type = response_bytes[5:5 + ct_len].decode("utf-8") - data = response_bytes[5 + ct_len:] - return content_type, data diff --git a/plugins/cmd/ndpgen/testdata/raw_client_expected.rs b/plugins/cmd/ndpgen/testdata/raw_client_expected.rs deleted file mode 100644 index 6de18be8e..000000000 --- a/plugins/cmd/ndpgen/testdata/raw_client_expected.rs +++ /dev/null @@ -1,73 +0,0 @@ -// Code generated by ndpgen. DO NOT EDIT. -// -// This file contains client wrappers for the Stream host service. -// It is intended for use in Navidrome plugins built with extism-pdk. - -use extism_pdk::*; -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, Serialize)] -#[serde(rename_all = "camelCase")] -struct StreamGetStreamRequest { - uri: String, -} - -#[host_fn] -extern "ExtismHost" { -} - -#[link(wasm_import_module = "extism:host/user")] -extern "C" { - fn stream_getstream(offset: u64) -> u64; -} - -/// GetStream returns raw binary stream data with content type. -/// -/// # Arguments -/// * `uri` - String parameter. -/// -/// # Returns -/// A tuple of (content_type, data) with the raw binary response. -/// -/// # Errors -/// Returns an error if the host function call fails. -pub fn get_stream(uri: &str) -> Result<(String, Vec), Error> { - let req = StreamGetStreamRequest { - uri: uri.to_owned(), - }; - let input_bytes = serde_json::to_vec(&req).map_err(|e| Error::msg(e.to_string()))?; - let input_mem = Memory::from_bytes(&input_bytes).map_err(|e| Error::msg(e.to_string()))?; - - let response_offset = unsafe { stream_getstream(input_mem.offset()) }; - - let response_mem = Memory::find(response_offset) - .ok_or_else(|| Error::msg("empty response from host"))?; - let response_bytes = response_mem.to_vec(); - - if response_bytes.is_empty() { - return Err(Error::msg("empty response from host")); - } - if response_bytes[0] == 0x01 { - let msg = String::from_utf8_lossy(&response_bytes[1..]).to_string(); - return Err(Error::msg(msg)); - } - if response_bytes[0] != 0x00 { - return Err(Error::msg("unknown response status")); - } - if response_bytes.len() < 5 { - return Err(Error::msg("malformed raw response: incomplete header")); - } - let ct_len = u32::from_be_bytes([ - response_bytes[1], - response_bytes[2], - response_bytes[3], - response_bytes[4], - ]) as usize; - if ct_len > response_bytes.len() - 5 { - return Err(Error::msg("malformed raw response: content-type overflow")); - } - let ct_end = 5 + ct_len; - let content_type = String::from_utf8_lossy(&response_bytes[5..ct_end]).to_string(); - let data = response_bytes[ct_end..].to_vec(); - Ok((content_type, data)) -} diff --git a/plugins/cmd/ndpgen/testdata/raw_service.go.txt b/plugins/cmd/ndpgen/testdata/raw_service.go.txt deleted file mode 100644 index c08332f5d..000000000 --- a/plugins/cmd/ndpgen/testdata/raw_service.go.txt +++ /dev/null @@ -1,10 +0,0 @@ -package testpkg - -import "context" - -//nd:hostservice name=Stream permission=stream -type StreamService interface { - // GetStream returns raw binary stream data with content type. - //nd:hostfunc raw=true - GetStream(ctx context.Context, uri string) (contentType string, data []byte, err error) -} diff --git a/plugins/host/subsonicapi.go b/plugins/host/subsonicapi.go index 117f8abff..32de75e77 100644 --- a/plugins/host/subsonicapi.go +++ b/plugins/host/subsonicapi.go @@ -17,8 +17,8 @@ type SubsonicAPIService interface { Call(ctx context.Context, uri string) (responseJSON string, err error) // CallRaw executes a Subsonic API request and returns the raw binary response. - // Optimized for binary endpoints like getCoverArt and stream that return - // non-JSON data. The response is returned as raw bytes without JSON encoding overhead. - //nd:hostfunc raw=true + // Designed for binary endpoints like getCoverArt and stream that return + // non-JSON data. The data is base64-encoded over JSON on the wire. + //nd:hostfunc CallRaw(ctx context.Context, uri string) (contentType string, data []byte, err error) } diff --git a/plugins/host/subsonicapi_gen.go b/plugins/host/subsonicapi_gen.go index 438c51c95..52474030e 100644 --- a/plugins/host/subsonicapi_gen.go +++ b/plugins/host/subsonicapi_gen.go @@ -4,7 +4,6 @@ package host import ( "context" - "encoding/binary" "encoding/json" extism "github.com/extism/go-sdk" @@ -26,6 +25,13 @@ type SubsonicAPICallRawRequest struct { Uri string `json:"uri"` } +// SubsonicAPICallRawResponse is the response type for SubsonicAPI.CallRaw. +type SubsonicAPICallRawResponse struct { + ContentType string `json:"contentType,omitempty"` + Data []byte `json:"data,omitempty"` + Error string `json:"error,omitempty"` +} + // RegisterSubsonicAPIHostFunctions registers SubsonicAPI service host functions. // The returned host functions should be added to the plugin's configuration. func RegisterSubsonicAPIHostFunctions(service SubsonicAPIService) []extism.HostFunction { @@ -76,37 +82,28 @@ func newSubsonicAPICallRawHostFunction(service SubsonicAPIService) extism.HostFu // Read JSON request from plugin memory reqBytes, err := p.ReadBytes(stack[0]) if err != nil { - subsonicapiWriteRawError(p, stack, err) + subsonicapiWriteError(p, stack, err) return } var req SubsonicAPICallRawRequest if err := json.Unmarshal(reqBytes, &req); err != nil { - subsonicapiWriteRawError(p, stack, err) + subsonicapiWriteError(p, stack, err) return } // Call the service method contenttype, data, svcErr := service.CallRaw(ctx, req.Uri) if svcErr != nil { - subsonicapiWriteRawError(p, stack, svcErr) + subsonicapiWriteError(p, stack, svcErr) return } - // Write binary-framed response to plugin memory: - // [0x00][4-byte content-type length (big-endian)][content-type string][raw data] - ctBytes := []byte(contenttype) - frame := make([]byte, 1+4+len(ctBytes)+len(data)) - frame[0] = 0x00 // success - binary.BigEndian.PutUint32(frame[1:5], uint32(len(ctBytes))) - copy(frame[5:5+len(ctBytes)], ctBytes) - copy(frame[5+len(ctBytes):], data) - - respPtr, err := p.WriteBytes(frame) - if err != nil { - stack[0] = 0 - return + // Write JSON response to plugin memory + resp := SubsonicAPICallRawResponse{ + ContentType: contenttype, + Data: data, } - stack[0] = respPtr + subsonicapiWriteResponse(p, stack, resp) }, []extism.ValueType{extism.ValueTypePTR}, []extism.ValueType{extism.ValueTypePTR}, @@ -137,14 +134,3 @@ func subsonicapiWriteError(p *extism.CurrentPlugin, stack []uint64, err error) { respPtr, _ := p.WriteBytes(respBytes) stack[0] = respPtr } - -// subsonicapiWriteRawError writes a binary-framed error response to plugin memory. -// Format: [0x01][UTF-8 error message] -func subsonicapiWriteRawError(p *extism.CurrentPlugin, stack []uint64, err error) { - errMsg := []byte(err.Error()) - frame := make([]byte, 1+len(errMsg)) - frame[0] = 0x01 // error - copy(frame[1:], errMsg) - respPtr, _ := p.WriteBytes(frame) - stack[0] = respPtr -} diff --git a/plugins/pdk/go/host/nd_host_subsonicapi.go b/plugins/pdk/go/host/nd_host_subsonicapi.go index 9bb4f4b15..e6e56ce6b 100644 --- a/plugins/pdk/go/host/nd_host_subsonicapi.go +++ b/plugins/pdk/go/host/nd_host_subsonicapi.go @@ -8,7 +8,6 @@ package host import ( - "encoding/binary" "encoding/json" "errors" @@ -38,6 +37,12 @@ type subsonicAPICallRawRequest struct { Uri string `json:"uri"` } +type subsonicAPICallRawResponse struct { + ContentType string `json:"contentType,omitempty"` + Data []byte `json:"data,omitempty"` + Error string `json:"error,omitempty"` +} + // SubsonicAPICall calls the subsonicapi_call host function. // Call executes a Subsonic API request and returns the JSON response. // @@ -78,8 +83,8 @@ func SubsonicAPICall(uri string) (string, error) { // SubsonicAPICallRaw calls the subsonicapi_callraw host function. // CallRaw executes a Subsonic API request and returns the raw binary response. -// Optimized for binary endpoints like getCoverArt and stream that return -// non-JSON data. The response is returned as raw bytes without JSON encoding overhead. +// Designed for binary endpoints like getCoverArt and stream that return +// non-JSON data. The data is base64-encoded over JSON on the wire. func SubsonicAPICallRaw(uri string) (string, []byte, error) { // Marshal request to JSON req := subsonicAPICallRawRequest{ @@ -99,22 +104,16 @@ func SubsonicAPICallRaw(uri string) (string, []byte, error) { responseMem := pdk.FindMemory(responsePtr) responseBytes := responseMem.ReadBytes() - // Parse binary-framed response - if len(responseBytes) == 0 { - return "", nil, errors.New("empty response from host") + // Parse the response + var response subsonicAPICallRawResponse + if err := json.Unmarshal(responseBytes, &response); err != nil { + return "", nil, err } - if responseBytes[0] == 0x01 { // error - return "", nil, errors.New(string(responseBytes[1:])) + + // Convert Error field to Go error + if response.Error != "" { + return "", nil, errors.New(response.Error) } - if responseBytes[0] != 0x00 { - return "", nil, errors.New("unknown response status") - } - if len(responseBytes) < 5 { - return "", nil, errors.New("malformed raw response: incomplete header") - } - ctLen := binary.BigEndian.Uint32(responseBytes[1:5]) - if uint32(len(responseBytes)) < 5+ctLen { - return "", nil, errors.New("malformed raw response: content-type overflow") - } - return string(responseBytes[5 : 5+ctLen]), responseBytes[5+ctLen:], nil + + return response.ContentType, response.Data, nil } diff --git a/plugins/pdk/go/host/nd_host_subsonicapi_stub.go b/plugins/pdk/go/host/nd_host_subsonicapi_stub.go index 95dd41558..2fdaf2403 100644 --- a/plugins/pdk/go/host/nd_host_subsonicapi_stub.go +++ b/plugins/pdk/go/host/nd_host_subsonicapi_stub.go @@ -42,8 +42,8 @@ func (m *mockSubsonicAPIService) CallRaw(uri string) (string, []byte, error) { // SubsonicAPICallRaw delegates to the mock instance. // CallRaw executes a Subsonic API request and returns the raw binary response. -// Optimized for binary endpoints like getCoverArt and stream that return -// non-JSON data. The response is returned as raw bytes without JSON encoding overhead. +// Designed for binary endpoints like getCoverArt and stream that return +// non-JSON data. The data is base64-encoded over JSON on the wire. func SubsonicAPICallRaw(uri string) (string, []byte, error) { return SubsonicAPIMock.CallRaw(uri) } diff --git a/plugins/pdk/python/host/nd_host_cache.py b/plugins/pdk/python/host/nd_host_cache.py index c22f95f6f..b24e983cc 100644 --- a/plugins/pdk/python/host/nd_host_cache.py +++ b/plugins/pdk/python/host/nd_host_cache.py @@ -12,6 +12,7 @@ from typing import Any import extism import json +import base64 class HostFunctionError(Exception): @@ -337,7 +338,7 @@ Returns an error if the operation fails. """ request = { "key": key, - "value": value, + "value": base64.b64encode(value).decode("ascii"), "ttlSeconds": ttl_seconds, } request_bytes = json.dumps(request).encode("utf-8") @@ -382,7 +383,7 @@ or the stored value is not a byte slice, exists will be false. raise HostFunctionError(response["error"]) return CacheGetBytesResult( - value=response.get("value", b""), + value=base64.b64decode(response.get("value", "")), exists=response.get("exists", False), ) diff --git a/plugins/pdk/python/host/nd_host_http.py b/plugins/pdk/python/host/nd_host_http.py index 8f196f985..a806c8456 100644 --- a/plugins/pdk/python/host/nd_host_http.py +++ b/plugins/pdk/python/host/nd_host_http.py @@ -12,6 +12,7 @@ from typing import Any import extism import json +import base64 class HostFunctionError(Exception): diff --git a/plugins/pdk/python/host/nd_host_kvstore.py b/plugins/pdk/python/host/nd_host_kvstore.py index 5485d2fb5..3c3e61f53 100644 --- a/plugins/pdk/python/host/nd_host_kvstore.py +++ b/plugins/pdk/python/host/nd_host_kvstore.py @@ -12,6 +12,7 @@ from typing import Any import extism import json +import base64 class HostFunctionError(Exception): @@ -80,7 +81,7 @@ Returns an error if the storage limit would be exceeded or the operation fails. """ request = { "key": key, - "value": value, + "value": base64.b64encode(value).decode("ascii"), } request_bytes = json.dumps(request).encode("utf-8") request_mem = extism.memory.alloc(request_bytes) @@ -123,7 +124,7 @@ Returns the value and whether the key exists. raise HostFunctionError(response["error"]) return KVStoreGetResult( - value=response.get("value", b""), + value=base64.b64decode(response.get("value", "")), exists=response.get("exists", False), ) diff --git a/plugins/pdk/python/host/nd_host_subsonicapi.py b/plugins/pdk/python/host/nd_host_subsonicapi.py index 4da8da77f..cf35bc043 100644 --- a/plugins/pdk/python/host/nd_host_subsonicapi.py +++ b/plugins/pdk/python/host/nd_host_subsonicapi.py @@ -8,11 +8,11 @@ # main __init__.py file. Copy the needed functions from this file into your plugin. from dataclasses import dataclass -from typing import Any, Tuple +from typing import Any import extism import json -import struct +import base64 class HostFunctionError(Exception): @@ -32,6 +32,13 @@ def _subsonicapi_callraw(offset: int) -> int: ... +@dataclass +class SubsonicAPICallRawResult: + """Result type for subsonicapi_call_raw.""" + content_type: str + data: bytes + + def subsonicapi_call(uri: str) -> str: """Call executes a Subsonic API request and returns the JSON response. @@ -62,16 +69,16 @@ e.g., "getAlbumList2?type=random&size=10". The response is returned as raw JSON. return response.get("responseJson", "") -def subsonicapi_call_raw(uri: str) -> Tuple[str, bytes]: +def subsonicapi_call_raw(uri: str) -> SubsonicAPICallRawResult: """CallRaw executes a Subsonic API request and returns the raw binary response. -Optimized for binary endpoints like getCoverArt and stream that return -non-JSON data. The response is returned as raw bytes without JSON encoding overhead. +Designed for binary endpoints like getCoverArt and stream that return +non-JSON data. The data is base64-encoded over JSON on the wire. Args: uri: str parameter. Returns: - Tuple of (content_type, data) with the raw binary response. + SubsonicAPICallRawResult containing content_type, data,. Raises: HostFunctionError: If the host function returns an error. @@ -83,19 +90,12 @@ non-JSON data. The response is returned as raw bytes without JSON encoding overh request_mem = extism.memory.alloc(request_bytes) response_offset = _subsonicapi_callraw(request_mem.offset) response_mem = extism.memory.find(response_offset) - response_bytes = response_mem.bytes() + response = json.loads(extism.memory.string(response_mem)) - if len(response_bytes) == 0: - raise HostFunctionError("empty response from host") - if response_bytes[0] == 0x01: - raise HostFunctionError(response_bytes[1:].decode("utf-8")) - if response_bytes[0] != 0x00: - raise HostFunctionError("unknown response status") - if len(response_bytes) < 5: - raise HostFunctionError("malformed raw response: incomplete header") - ct_len = struct.unpack(">I", response_bytes[1:5])[0] - if len(response_bytes) < 5 + ct_len: - raise HostFunctionError("malformed raw response: content-type overflow") - content_type = response_bytes[5:5 + ct_len].decode("utf-8") - data = response_bytes[5 + ct_len:] - return content_type, data + if response.get("error"): + raise HostFunctionError(response["error"]) + + return SubsonicAPICallRawResult( + content_type=response.get("contentType", ""), + data=base64.b64decode(response.get("data", "")), + ) diff --git a/plugins/pdk/python/host/nd_host_websocket.py b/plugins/pdk/python/host/nd_host_websocket.py index b62ee792c..4e882914c 100644 --- a/plugins/pdk/python/host/nd_host_websocket.py +++ b/plugins/pdk/python/host/nd_host_websocket.py @@ -12,6 +12,7 @@ from typing import Any import extism import json +import base64 class HostFunctionError(Exception): @@ -134,7 +135,7 @@ Returns an error if the connection is not found or if sending fails. """ request = { "connectionId": connection_id, - "data": data, + "data": base64.b64encode(data).decode("ascii"), } request_bytes = json.dumps(request).encode("utf-8") request_mem = extism.memory.alloc(request_bytes) diff --git a/plugins/pdk/rust/nd-pdk-capabilities/Cargo.toml b/plugins/pdk/rust/nd-pdk-capabilities/Cargo.toml index 98a91da1f..443f19da5 100644 --- a/plugins/pdk/rust/nd-pdk-capabilities/Cargo.toml +++ b/plugins/pdk/rust/nd-pdk-capabilities/Cargo.toml @@ -11,6 +11,7 @@ path = "src/lib.rs" crate-type = ["rlib"] [dependencies] +base64 = "0.22" extism-pdk = "1.2" serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" diff --git a/plugins/pdk/rust/nd-pdk-host/Cargo.toml b/plugins/pdk/rust/nd-pdk-host/Cargo.toml index 4cb828697..519096110 100644 --- a/plugins/pdk/rust/nd-pdk-host/Cargo.toml +++ b/plugins/pdk/rust/nd-pdk-host/Cargo.toml @@ -11,6 +11,7 @@ readme = "README.md" crate-type = ["rlib"] [dependencies] +base64 = "0.22" extism-pdk = "1.2" serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_cache.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_cache.rs index 1f3d69295..267654136 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/nd_host_cache.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_cache.rs @@ -5,6 +5,29 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] @@ -106,6 +129,7 @@ struct CacheGetFloatResponse { #[serde(rename_all = "camelCase")] struct CacheSetBytesRequest { key: String, + #[serde(with = "base64_bytes")] value: Vec, ttl_seconds: i64, } @@ -127,6 +151,7 @@ struct CacheGetBytesRequest { #[serde(rename_all = "camelCase")] struct CacheGetBytesResponse { #[serde(default)] + #[serde(with = "base64_bytes")] value: Vec, #[serde(default)] exists: bool, diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs index b8aaad1e0..d3bb2d326 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_http.rs @@ -5,6 +5,29 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} /// HTTPRequest represents an outbound HTTP request from a plugin. #[derive(Debug, Clone, Serialize, Deserialize)] @@ -15,6 +38,7 @@ pub struct HTTPRequest { #[serde(default)] pub headers: std::collections::HashMap, #[serde(default)] + #[serde(with = "base64_bytes")] pub body: Vec, #[serde(default)] pub timeout_ms: i32, @@ -28,6 +52,7 @@ pub struct HTTPResponse { #[serde(default)] pub headers: std::collections::HashMap, #[serde(default)] + #[serde(with = "base64_bytes")] pub body: Vec, } diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs index 5048f369c..20fe18c6f 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs @@ -5,11 +5,35 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] struct KVStoreSetRequest { key: String, + #[serde(with = "base64_bytes")] value: Vec, } @@ -30,6 +54,7 @@ struct KVStoreGetRequest { #[serde(rename_all = "camelCase")] struct KVStoreGetResponse { #[serde(default)] + #[serde(with = "base64_bytes")] value: Vec, #[serde(default)] exists: bool, diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_subsonicapi.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_subsonicapi.rs index 2c9e6545f..56ba1066e 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/nd_host_subsonicapi.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_subsonicapi.rs @@ -5,6 +5,29 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] @@ -27,14 +50,22 @@ struct SubsonicAPICallRawRequest { uri: String, } +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct SubsonicAPICallRawResponse { + #[serde(default)] + content_type: String, + #[serde(default)] + #[serde(with = "base64_bytes")] + data: Vec, + #[serde(default)] + error: Option, +} + #[host_fn] extern "ExtismHost" { fn subsonicapi_call(input: Json) -> Json; -} - -#[link(wasm_import_module = "extism:host/user")] -extern "C" { - fn subsonicapi_callraw(offset: u64) -> u64; + fn subsonicapi_callraw(input: Json) -> Json; } /// Call executes a Subsonic API request and returns the JSON response. @@ -65,54 +96,27 @@ pub fn call(uri: &str) -> Result { } /// CallRaw executes a Subsonic API request and returns the raw binary response. -/// Optimized for binary endpoints like getCoverArt and stream that return -/// non-JSON data. The response is returned as raw bytes without JSON encoding overhead. +/// Designed for binary endpoints like getCoverArt and stream that return +/// non-JSON data. The data is base64-encoded over JSON on the wire. /// /// # Arguments /// * `uri` - String parameter. /// /// # Returns -/// A tuple of (content_type, data) with the raw binary response. +/// A tuple of (content_type, data). /// /// # Errors /// Returns an error if the host function call fails. pub fn call_raw(uri: &str) -> Result<(String, Vec), Error> { - let req = SubsonicAPICallRawRequest { - uri: uri.to_owned(), + let response = unsafe { + subsonicapi_callraw(Json(SubsonicAPICallRawRequest { + uri: uri.to_owned(), + }))? }; - let input_bytes = serde_json::to_vec(&req).map_err(|e| Error::msg(e.to_string()))?; - let input_mem = Memory::from_bytes(&input_bytes).map_err(|e| Error::msg(e.to_string()))?; - let response_offset = unsafe { subsonicapi_callraw(input_mem.offset()) }; + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } - let response_mem = Memory::find(response_offset) - .ok_or_else(|| Error::msg("empty response from host"))?; - let response_bytes = response_mem.to_vec(); - - if response_bytes.is_empty() { - return Err(Error::msg("empty response from host")); - } - if response_bytes[0] == 0x01 { - let msg = String::from_utf8_lossy(&response_bytes[1..]).to_string(); - return Err(Error::msg(msg)); - } - if response_bytes[0] != 0x00 { - return Err(Error::msg("unknown response status")); - } - if response_bytes.len() < 5 { - return Err(Error::msg("malformed raw response: incomplete header")); - } - let ct_len = u32::from_be_bytes([ - response_bytes[1], - response_bytes[2], - response_bytes[3], - response_bytes[4], - ]) as usize; - if ct_len > response_bytes.len() - 5 { - return Err(Error::msg("malformed raw response: content-type overflow")); - } - let ct_end = 5 + ct_len; - let content_type = String::from_utf8_lossy(&response_bytes[5..ct_end]).to_string(); - let data = response_bytes[ct_end..].to_vec(); - Ok((content_type, data)) + Ok((response.0.content_type, response.0.data)) } diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_websocket.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_websocket.rs index 58a399028..05ceb5407 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/nd_host_websocket.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_websocket.rs @@ -5,6 +5,29 @@ use extism_pdk::*; use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] @@ -41,6 +64,7 @@ struct WebSocketSendTextResponse { #[serde(rename_all = "camelCase")] struct WebSocketSendBinaryRequest { connection_id: String, + #[serde(with = "base64_bytes")] data: Vec, } From d134de106189b888f7d57f63e311f0b50f5be12b Mon Sep 17 00:00:00 2001 From: Deluan Date: Sat, 28 Feb 2026 10:55:19 -0500 Subject: [PATCH 19/60] feat(server): add 'has_rating' filter to artist and mediafile repositories Signed-off-by: Deluan --- persistence/artist_repository.go | 1 + persistence/mediafile_repository.go | 1 + 2 files changed, 2 insertions(+) diff --git a/persistence/artist_repository.go b/persistence/artist_repository.go index 07824e21f..7f3d61540 100644 --- a/persistence/artist_repository.go +++ b/persistence/artist_repository.go @@ -134,6 +134,7 @@ func NewArtistRepository(ctx context.Context, db dbx.Builder) model.ArtistReposi "id": idFilter(r.tableName), "name": fullTextFilter(r.tableName, "mbz_artist_id"), "starred": annotationBoolFilter("starred"), + "has_rating": annotationBoolFilter("rating"), "role": roleFilter, "missing": booleanFilter, "library_id": artistLibraryIdFilter, diff --git a/persistence/mediafile_repository.go b/persistence/mediafile_repository.go index 394ca5b70..9034fa8f8 100644 --- a/persistence/mediafile_repository.go +++ b/persistence/mediafile_repository.go @@ -98,6 +98,7 @@ var mediaFileFilter = sync.OnceValue(func() map[string]filterFunc { "id": idFilter("media_file"), "title": fullTextFilter("media_file", "mbz_recording_id", "mbz_release_track_id"), "starred": annotationBoolFilter("starred"), + "has_rating": annotationBoolFilter("rating"), "genre_id": tagIDFilter, "missing": booleanFilter, "artists_id": artistFilter, From d9a215e1e3184ffd0ad85b2d4ddec250d661ab4b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Sat, 28 Feb 2026 10:59:13 -0500 Subject: [PATCH 20/60] feat(plugins): allow mounting library directories as read-write (#5122) * feat(plugins): mount library directories as read-only by default Add an AllowWriteAccess boolean to the plugin model, defaulting to false. When off, library directories are mounted with the extism "ro:" prefix (read-only). Admins can explicitly grant write access via a new toggle in the Library Permission card. * test: add tests to buildAllowedPaths Signed-off-by: Deluan * chore: improve allowed paths logging for library access Signed-off-by: Deluan --------- Signed-off-by: Deluan --- ...28020813_add_plugin_allow_write_access.sql | 5 ++ model/plugin.go | 27 ++++---- persistence/plugin_repository.go | 32 +++++----- plugins/manager.go | 3 +- plugins/manager_loader.go | 63 +++++++++++------- plugins/manager_loader_test.go | 64 +++++++++++++++++++ resources/i18n/pt-br.json | 4 +- server/nativeapi/native_api.go | 2 +- server/nativeapi/plugin.go | 21 +++--- tests/mock_plugin_manager.go | 22 ++++--- ui/src/i18n/en.json | 4 +- ui/src/plugin/LibraryPermissionCard.jsx | 30 +++++++++ ui/src/plugin/PluginShow.jsx | 27 +++++++- 13 files changed, 229 insertions(+), 75 deletions(-) create mode 100644 db/migrations/20260228020813_add_plugin_allow_write_access.sql diff --git a/db/migrations/20260228020813_add_plugin_allow_write_access.sql b/db/migrations/20260228020813_add_plugin_allow_write_access.sql new file mode 100644 index 000000000..e17d874a5 --- /dev/null +++ b/db/migrations/20260228020813_add_plugin_allow_write_access.sql @@ -0,0 +1,5 @@ +-- +goose Up +ALTER TABLE plugin ADD COLUMN allow_write_access BOOL NOT NULL DEFAULT false; + +-- +goose Down +ALTER TABLE plugin DROP COLUMN allow_write_access; diff --git a/model/plugin.go b/model/plugin.go index d23103995..f4bad6783 100644 --- a/model/plugin.go +++ b/model/plugin.go @@ -3,19 +3,20 @@ package model import "time" type Plugin struct { - ID string `structs:"id" json:"id"` - Path string `structs:"path" json:"path"` - Manifest string `structs:"manifest" json:"manifest"` - Config string `structs:"config" json:"config,omitempty"` - Users string `structs:"users" json:"users,omitempty"` - AllUsers bool `structs:"all_users" json:"allUsers,omitempty"` - Libraries string `structs:"libraries" json:"libraries,omitempty"` - AllLibraries bool `structs:"all_libraries" json:"allLibraries,omitempty"` - Enabled bool `structs:"enabled" json:"enabled"` - LastError string `structs:"last_error" json:"lastError,omitempty"` - SHA256 string `structs:"sha256" json:"sha256"` - CreatedAt time.Time `structs:"created_at" json:"createdAt"` - UpdatedAt time.Time `structs:"updated_at" json:"updatedAt"` + ID string `structs:"id" json:"id"` + Path string `structs:"path" json:"path"` + Manifest string `structs:"manifest" json:"manifest"` + Config string `structs:"config" json:"config,omitempty"` + Users string `structs:"users" json:"users,omitempty"` + AllUsers bool `structs:"all_users" json:"allUsers,omitempty"` + Libraries string `structs:"libraries" json:"libraries,omitempty"` + AllLibraries bool `structs:"all_libraries" json:"allLibraries,omitempty"` + AllowWriteAccess bool `structs:"allow_write_access" json:"allowWriteAccess,omitempty"` + Enabled bool `structs:"enabled" json:"enabled"` + LastError string `structs:"last_error" json:"lastError,omitempty"` + SHA256 string `structs:"sha256" json:"sha256"` + CreatedAt time.Time `structs:"created_at" json:"createdAt"` + UpdatedAt time.Time `structs:"updated_at" json:"updatedAt"` } type Plugins []Plugin diff --git a/persistence/plugin_repository.go b/persistence/plugin_repository.go index 4a98f148b..466abb40b 100644 --- a/persistence/plugin_repository.go +++ b/persistence/plugin_repository.go @@ -79,8 +79,8 @@ func (r *pluginRepository) Put(plugin *model.Plugin) error { // Upsert using INSERT ... ON CONFLICT for atomic operation _, err := r.db.NewQuery(` - INSERT INTO plugin (id, path, manifest, config, users, all_users, libraries, all_libraries, enabled, last_error, sha256, created_at, updated_at) - VALUES ({:id}, {:path}, {:manifest}, {:config}, {:users}, {:all_users}, {:libraries}, {:all_libraries}, {:enabled}, {:last_error}, {:sha256}, {:created_at}, {:updated_at}) + INSERT INTO plugin (id, path, manifest, config, users, all_users, libraries, all_libraries, allow_write_access, enabled, last_error, sha256, created_at, updated_at) + VALUES ({:id}, {:path}, {:manifest}, {:config}, {:users}, {:all_users}, {:libraries}, {:all_libraries}, {:allow_write_access}, {:enabled}, {:last_error}, {:sha256}, {:created_at}, {:updated_at}) ON CONFLICT(id) DO UPDATE SET path = excluded.path, manifest = excluded.manifest, @@ -89,24 +89,26 @@ func (r *pluginRepository) Put(plugin *model.Plugin) error { all_users = excluded.all_users, libraries = excluded.libraries, all_libraries = excluded.all_libraries, + allow_write_access = excluded.allow_write_access, enabled = excluded.enabled, last_error = excluded.last_error, sha256 = excluded.sha256, updated_at = excluded.updated_at `).Bind(dbx.Params{ - "id": plugin.ID, - "path": plugin.Path, - "manifest": plugin.Manifest, - "config": plugin.Config, - "users": plugin.Users, - "all_users": plugin.AllUsers, - "libraries": plugin.Libraries, - "all_libraries": plugin.AllLibraries, - "enabled": plugin.Enabled, - "last_error": plugin.LastError, - "sha256": plugin.SHA256, - "created_at": time.Now(), - "updated_at": plugin.UpdatedAt, + "id": plugin.ID, + "path": plugin.Path, + "manifest": plugin.Manifest, + "config": plugin.Config, + "users": plugin.Users, + "all_users": plugin.AllUsers, + "libraries": plugin.Libraries, + "all_libraries": plugin.AllLibraries, + "allow_write_access": plugin.AllowWriteAccess, + "enabled": plugin.Enabled, + "last_error": plugin.LastError, + "sha256": plugin.SHA256, + "created_at": time.Now(), + "updated_at": plugin.UpdatedAt, }).Execute() return err } diff --git a/plugins/manager.go b/plugins/manager.go index d8d4f28ef..f148a706c 100644 --- a/plugins/manager.go +++ b/plugins/manager.go @@ -428,10 +428,11 @@ func (m *Manager) UpdatePluginUsers(ctx context.Context, id, usersJSON string, a // If the plugin is enabled, it will be reloaded with the new settings. // If the plugin requires library permission and no libraries are configured (and allLibraries is false), // the plugin will be automatically disabled. -func (m *Manager) UpdatePluginLibraries(ctx context.Context, id, librariesJSON string, allLibraries bool) error { +func (m *Manager) UpdatePluginLibraries(ctx context.Context, id, librariesJSON string, allLibraries, allowWriteAccess bool) error { return m.updatePluginSettings(ctx, id, func(p *model.Plugin) { p.Libraries = librariesJSON p.AllLibraries = allLibraries + p.AllowWriteAccess = allowWriteAccess }) } diff --git a/plugins/manager_loader.go b/plugins/manager_loader.go index c6355911f..688c4519c 100644 --- a/plugins/manager_loader.go +++ b/plugins/manager_loader.go @@ -226,6 +226,8 @@ func (m *Manager) loadEnabledPlugins(ctx context.Context) error { // loadPluginWithConfig loads a plugin with configuration from DB. // The p.Path should point to an .ndp package file. func (m *Manager) loadPluginWithConfig(p *model.Plugin) error { + ctx := log.NewContext(m.ctx, "plugin", p.ID) + if m.stopped.Load() { return fmt.Errorf("manager is stopped") } @@ -283,27 +285,13 @@ func (m *Manager) loadPluginWithConfig(p *model.Plugin) error { // Configure filesystem access for library permission if pkg.Manifest.Permissions != nil && pkg.Manifest.Permissions.Library != nil && pkg.Manifest.Permissions.Library.Filesystem { - adminCtx := adminContext(m.ctx) + adminCtx := adminContext(ctx) libraries, err := m.ds.Library(adminCtx).GetAll() if err != nil { return fmt.Errorf("failed to get libraries for filesystem access: %w", err) } - // Build a set of allowed library IDs for fast lookup - allowedLibrarySet := make(map[int]struct{}, len(allowedLibraries)) - for _, id := range allowedLibraries { - allowedLibrarySet[id] = struct{}{} - } - - allowedPaths := make(map[string]string) - for _, lib := range libraries { - // Only mount if allLibraries is true or library is in the allowed list - if p.AllLibraries { - allowedPaths[lib.Path] = toPluginMountPoint(int32(lib.ID)) - } else if _, ok := allowedLibrarySet[lib.ID]; ok { - allowedPaths[lib.Path] = toPluginMountPoint(int32(lib.ID)) - } - } + allowedPaths := buildAllowedPaths(ctx, libraries, allowedLibraries, p.AllLibraries, p.AllowWriteAccess) pluginManifest.AllowedPaths = allowedPaths } @@ -339,7 +327,7 @@ func (m *Manager) loadPluginWithConfig(p *model.Plugin) error { // Enable experimental threads if requested in manifest if pkg.Manifest.HasExperimentalThreads() { runtimeConfig = runtimeConfig.WithCoreFeatures(api.CoreFeaturesV2 | experimental.CoreFeaturesThreads) - log.Debug(m.ctx, "Enabling experimental threads support", "plugin", p.ID) + log.Debug(ctx, "Enabling experimental threads support") } extismConfig := extism.PluginConfig{ @@ -347,24 +335,24 @@ func (m *Manager) loadPluginWithConfig(p *model.Plugin) error { RuntimeConfig: runtimeConfig, EnableHttpResponseHeaders: true, } - compiled, err := extism.NewCompiledPlugin(m.ctx, pluginManifest, extismConfig, hostFunctions) + compiled, err := extism.NewCompiledPlugin(ctx, pluginManifest, extismConfig, hostFunctions) if err != nil { return fmt.Errorf("compiling plugin: %w", err) } // Create instance to detect capabilities - instance, err := compiled.Instance(m.ctx, extism.PluginInstanceConfig{}) + instance, err := compiled.Instance(ctx, extism.PluginInstanceConfig{}) if err != nil { - compiled.Close(m.ctx) + compiled.Close(ctx) return fmt.Errorf("creating instance: %w", err) } instance.SetLogger(extismLogger(p.ID)) capabilities := detectCapabilities(instance) - instance.Close(m.ctx) + instance.Close(ctx) // Validate manifest against detected capabilities if err := ValidateWithCapabilities(pkg.Manifest, capabilities); err != nil { - compiled.Close(m.ctx) + compiled.Close(ctx) return fmt.Errorf("manifest validation: %w", err) } @@ -383,7 +371,7 @@ func (m *Manager) loadPluginWithConfig(p *model.Plugin) error { m.mu.Unlock() // Call plugin init function - callPluginInit(m.ctx, m.plugins[p.ID]) + callPluginInit(ctx, m.plugins[p.ID]) return nil } @@ -414,3 +402,32 @@ func parsePluginConfig(configJSON string) (map[string]string, error) { } return pluginConfig, nil } + +// buildAllowedPaths constructs the extism AllowedPaths map for filesystem access. +// When allowWriteAccess is false (default), paths are prefixed with "ro:" for read-only. +// Only libraries that match the allowed set (or all libraries if allLibraries is true) are included. +func buildAllowedPaths(ctx context.Context, libraries model.Libraries, allowedLibraryIDs []int, allLibraries, allowWriteAccess bool) map[string]string { + allowedLibrarySet := make(map[int]struct{}, len(allowedLibraryIDs)) + for _, id := range allowedLibraryIDs { + allowedLibrarySet[id] = struct{}{} + } + allowedPaths := make(map[string]string) + for _, lib := range libraries { + _, allowed := allowedLibrarySet[lib.ID] + if allLibraries || allowed { + mountPoint := toPluginMountPoint(int32(lib.ID)) + hostPath := lib.Path + if !allowWriteAccess { + hostPath = "ro:" + hostPath + } + allowedPaths[hostPath] = mountPoint + log.Trace(ctx, "Added library to allowed paths", "libraryID", lib.ID, "mountPoint", mountPoint, "writeAccess", allowWriteAccess, "hostPath", hostPath) + } + } + if allowWriteAccess { + log.Info(ctx, "Granting read-write filesystem access to libraries", "libraryCount", len(allowedPaths), "allLibraries", allLibraries) + } else { + log.Debug(ctx, "Granting read-only filesystem access to libraries", "libraryCount", len(allowedPaths), "allLibraries", allLibraries) + } + return allowedPaths +} diff --git a/plugins/manager_loader_test.go b/plugins/manager_loader_test.go index 64bc5e810..3a00b07b7 100644 --- a/plugins/manager_loader_test.go +++ b/plugins/manager_loader_test.go @@ -3,6 +3,7 @@ package plugins import ( + "github.com/navidrome/navidrome/model" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ) @@ -58,3 +59,66 @@ var _ = Describe("parsePluginConfig", func() { Expect(result).ToNot(BeNil()) }) }) + +var _ = Describe("buildAllowedPaths", func() { + var libraries model.Libraries + + BeforeEach(func() { + libraries = model.Libraries{ + {ID: 1, Path: "/music/library1"}, + {ID: 2, Path: "/music/library2"}, + {ID: 3, Path: "/music/library3"}, + } + }) + + Context("read-only (default)", func() { + It("mounts all libraries with ro: prefix when allLibraries is true", func() { + result := buildAllowedPaths(nil, libraries, nil, true, false) + Expect(result).To(HaveLen(3)) + Expect(result).To(HaveKeyWithValue("ro:/music/library1", "/libraries/1")) + Expect(result).To(HaveKeyWithValue("ro:/music/library2", "/libraries/2")) + Expect(result).To(HaveKeyWithValue("ro:/music/library3", "/libraries/3")) + }) + + It("mounts only selected libraries with ro: prefix", func() { + result := buildAllowedPaths(nil, libraries, []int{1, 3}, false, false) + Expect(result).To(HaveLen(2)) + Expect(result).To(HaveKeyWithValue("ro:/music/library1", "/libraries/1")) + Expect(result).To(HaveKeyWithValue("ro:/music/library3", "/libraries/3")) + Expect(result).ToNot(HaveKey("ro:/music/library2")) + }) + }) + + Context("read-write (allowWriteAccess=true)", func() { + It("mounts all libraries without ro: prefix when allLibraries is true", func() { + result := buildAllowedPaths(nil, libraries, nil, true, true) + Expect(result).To(HaveLen(3)) + Expect(result).To(HaveKeyWithValue("/music/library1", "/libraries/1")) + Expect(result).To(HaveKeyWithValue("/music/library2", "/libraries/2")) + Expect(result).To(HaveKeyWithValue("/music/library3", "/libraries/3")) + }) + + It("mounts only selected libraries without ro: prefix", func() { + result := buildAllowedPaths(nil, libraries, []int{2}, false, true) + Expect(result).To(HaveLen(1)) + Expect(result).To(HaveKeyWithValue("/music/library2", "/libraries/2")) + }) + }) + + Context("edge cases", func() { + It("returns empty map when no libraries match", func() { + result := buildAllowedPaths(nil, libraries, []int{99}, false, false) + Expect(result).To(BeEmpty()) + }) + + It("returns empty map when libraries list is empty", func() { + result := buildAllowedPaths(nil, nil, []int{1}, false, false) + Expect(result).To(BeEmpty()) + }) + + It("returns empty map when allLibraries is false and no IDs provided", func() { + result := buildAllowedPaths(nil, libraries, nil, false, false) + Expect(result).To(BeEmpty()) + }) + }) +}) diff --git a/resources/i18n/pt-br.json b/resources/i18n/pt-br.json index a844dccdb..1fc72bf30 100644 --- a/resources/i18n/pt-br.json +++ b/resources/i18n/pt-br.json @@ -353,7 +353,8 @@ "allUsers": "Permitir todos os usuários", "selectedUsers": "Usuários selecionados", "allLibraries": "Permitir todas as bibliotecas", - "selectedLibraries": "Bibliotecas selecionadas" + "selectedLibraries": "Bibliotecas selecionadas", + "allowWriteAccess": "Permitir acesso de escrita" }, "sections": { "status": "Status", @@ -396,6 +397,7 @@ "allLibrariesHelp": "Quando habilitado, o plugin terá acesso a todas as bibliotecas, incluindo as criadas no futuro.", "noLibraries": "Nenhuma biblioteca selecionada", "librariesRequired": "Este plugin requer acesso a informações de bibliotecas. Selecione quais bibliotecas o plugin pode acessar, ou habilite 'Permitir todas as bibliotecas'.", + "allowWriteAccessHelp": "Quando habilitado, o plugin pode modificar arquivos nos diretórios das bibliotecas. Por padrão, plugins têm acesso somente leitura.", "requiredHosts": "Hosts necessários", "configValidationError": "Falha na validação da configuração:", "schemaRenderError": "Não foi possível renderizar o formulário de configuração. O schema do plugin pode estar inválido." diff --git a/server/nativeapi/native_api.go b/server/nativeapi/native_api.go index 27b85a605..062cbf706 100644 --- a/server/nativeapi/native_api.go +++ b/server/nativeapi/native_api.go @@ -29,7 +29,7 @@ type PluginManager interface { ValidatePluginConfig(ctx context.Context, id, configJSON string) error UpdatePluginConfig(ctx context.Context, id, configJSON string) error UpdatePluginUsers(ctx context.Context, id, usersJSON string, allUsers bool) error - UpdatePluginLibraries(ctx context.Context, id, librariesJSON string, allLibraries bool) error + UpdatePluginLibraries(ctx context.Context, id, librariesJSON string, allLibraries, allowWriteAccess bool) error RescanPlugins(ctx context.Context) error UnloadDisabledPlugins(ctx context.Context) } diff --git a/server/nativeapi/plugin.go b/server/nativeapi/plugin.go index e733edc4b..a7d261681 100644 --- a/server/nativeapi/plugin.go +++ b/server/nativeapi/plugin.go @@ -56,12 +56,13 @@ func pluginsEnabledMiddleware(next http.Handler) http.Handler { // PluginUpdateRequest represents the fields that can be updated via the API type PluginUpdateRequest struct { - Enabled *bool `json:"enabled,omitempty"` - Config *string `json:"config,omitempty"` - Users *string `json:"users,omitempty"` - AllUsers *bool `json:"allUsers,omitempty"` - Libraries *string `json:"libraries,omitempty"` - AllLibraries *bool `json:"allLibraries,omitempty"` + Enabled *bool `json:"enabled,omitempty"` + Config *string `json:"config,omitempty"` + Users *string `json:"users,omitempty"` + AllUsers *bool `json:"allUsers,omitempty"` + Libraries *string `json:"libraries,omitempty"` + AllLibraries *bool `json:"allLibraries,omitempty"` + AllowWriteAccess *bool `json:"allowWriteAccess,omitempty"` } func (api *Router) updatePlugin(w http.ResponseWriter, r *http.Request) { @@ -109,7 +110,7 @@ func (api *Router) updatePlugin(w http.ResponseWriter, r *http.Request) { } // Handle libraries permission update (if provided) - if req.Libraries != nil || req.AllLibraries != nil { + if req.Libraries != nil || req.AllLibraries != nil || req.AllowWriteAccess != nil { if err := validateAndUpdateLibraries(ctx, api.pluginManager, repo, id, req, w); err != nil { log.Error(ctx, "Error updating plugin libraries", err) return @@ -245,6 +246,7 @@ func validateAndUpdateLibraries(ctx context.Context, pm PluginManager, repo mode librariesJSON := plugin.Libraries allLibraries := plugin.AllLibraries + allowWriteAccess := plugin.AllowWriteAccess if req.Libraries != nil { if *req.Libraries != "" && !isValidJSON(*req.Libraries) { @@ -256,8 +258,11 @@ func validateAndUpdateLibraries(ctx context.Context, pm PluginManager, repo mode if req.AllLibraries != nil { allLibraries = *req.AllLibraries } + if req.AllowWriteAccess != nil { + allowWriteAccess = *req.AllowWriteAccess + } - if err := pm.UpdatePluginLibraries(ctx, id, librariesJSON, allLibraries); err != nil { + if err := pm.UpdatePluginLibraries(ctx, id, librariesJSON, allLibraries, allowWriteAccess); err != nil { log.Error(ctx, "Error updating plugin libraries", "id", id, err) http.Error(w, "Error updating plugin libraries: "+err.Error(), http.StatusInternalServerError) return err diff --git a/tests/mock_plugin_manager.go b/tests/mock_plugin_manager.go index 9691f7a38..05375f31c 100644 --- a/tests/mock_plugin_manager.go +++ b/tests/mock_plugin_manager.go @@ -18,7 +18,7 @@ type MockPluginManager struct { // UpdatePluginUsersFn is called when UpdatePluginUsers is invoked. If nil, returns UsersError. UpdatePluginUsersFn func(ctx context.Context, id, usersJSON string, allUsers bool) error // UpdatePluginLibrariesFn is called when UpdatePluginLibraries is invoked. If nil, returns LibrariesError. - UpdatePluginLibrariesFn func(ctx context.Context, id, librariesJSON string, allLibraries bool) error + UpdatePluginLibrariesFn func(ctx context.Context, id, librariesJSON string, allLibraries, allowWriteAccess bool) error // RescanPluginsFn is called when RescanPlugins is invoked. If nil, returns RescanError. RescanPluginsFn func(ctx context.Context) error @@ -48,9 +48,10 @@ type MockPluginManager struct { AllUsers bool } UpdatePluginLibrariesCalls []struct { - ID string - LibrariesJSON string - AllLibraries bool + ID string + LibrariesJSON string + AllLibraries bool + AllowWriteAccess bool } RescanPluginsCalls int } @@ -105,14 +106,15 @@ func (m *MockPluginManager) UpdatePluginUsers(ctx context.Context, id, usersJSON return m.UsersError } -func (m *MockPluginManager) UpdatePluginLibraries(ctx context.Context, id, librariesJSON string, allLibraries bool) error { +func (m *MockPluginManager) UpdatePluginLibraries(ctx context.Context, id, librariesJSON string, allLibraries, allowWriteAccess bool) error { m.UpdatePluginLibrariesCalls = append(m.UpdatePluginLibrariesCalls, struct { - ID string - LibrariesJSON string - AllLibraries bool - }{ID: id, LibrariesJSON: librariesJSON, AllLibraries: allLibraries}) + ID string + LibrariesJSON string + AllLibraries bool + AllowWriteAccess bool + }{ID: id, LibrariesJSON: librariesJSON, AllLibraries: allLibraries, AllowWriteAccess: allowWriteAccess}) if m.UpdatePluginLibrariesFn != nil { - return m.UpdatePluginLibrariesFn(ctx, id, librariesJSON, allLibraries) + return m.UpdatePluginLibrariesFn(ctx, id, librariesJSON, allLibraries, allowWriteAccess) } return m.LibrariesError } diff --git a/ui/src/i18n/en.json b/ui/src/i18n/en.json index 678abaabd..224b1c437 100644 --- a/ui/src/i18n/en.json +++ b/ui/src/i18n/en.json @@ -355,7 +355,8 @@ "allUsers": "Allow all users", "selectedUsers": "Selected users", "allLibraries": "Allow all libraries", - "selectedLibraries": "Selected libraries" + "selectedLibraries": "Selected libraries", + "allowWriteAccess": "Allow write access" }, "sections": { "status": "Status", @@ -400,6 +401,7 @@ "allLibrariesHelp": "When enabled, the plugin will have access to all libraries, including those created in the future.", "noLibraries": "No libraries selected", "librariesRequired": "This plugin requires access to library information. Select which libraries the plugin can access, or enable 'Allow all libraries'.", + "allowWriteAccessHelp": "When enabled, the plugin can modify files in the library directories. By default, plugins have read-only access.", "requiredHosts": "Required hosts" }, "placeholders": { diff --git a/ui/src/plugin/LibraryPermissionCard.jsx b/ui/src/plugin/LibraryPermissionCard.jsx index 885ac010b..d3c237279 100644 --- a/ui/src/plugin/LibraryPermissionCard.jsx +++ b/ui/src/plugin/LibraryPermissionCard.jsx @@ -23,8 +23,10 @@ export const LibraryPermissionCard = ({ classes, selectedLibraries, allLibraries, + allowWriteAccess, onSelectedLibrariesChange, onAllLibrariesChange, + onAllowWriteAccessChange, }) => { const translate = useTranslate() @@ -58,9 +60,17 @@ export const LibraryPermissionCard = ({ [onAllLibrariesChange], ) + const handleAllowWriteAccessToggle = React.useCallback( + (event) => { + onAllowWriteAccessChange(event.target.checked) + }, + [onAllowWriteAccessChange], + ) + // Get permission reason from manifest const libraryPermission = manifest?.permissions?.library const reason = libraryPermission?.reason + const hasFilesystem = libraryPermission?.filesystem === true // Check if permission is required but not configured const isConfigurationRequired = @@ -107,6 +117,24 @@ export const LibraryPermissionCard = ({ + {hasFilesystem && ( + + + } + label={translate('resources.plugin.fields.allowWriteAccess')} + /> + + {translate('resources.plugin.messages.allowWriteAccessHelp')} + + + )} + {!allLibraries && ( @@ -166,6 +194,8 @@ LibraryPermissionCard.propTypes = { classes: PropTypes.object.isRequired, selectedLibraries: PropTypes.array.isRequired, allLibraries: PropTypes.bool.isRequired, + allowWriteAccess: PropTypes.bool.isRequired, onSelectedLibrariesChange: PropTypes.func.isRequired, onAllLibrariesChange: PropTypes.func.isRequired, + onAllowWriteAccessChange: PropTypes.func.isRequired, } diff --git a/ui/src/plugin/PluginShow.jsx b/ui/src/plugin/PluginShow.jsx index 38e858af4..caea44a75 100644 --- a/ui/src/plugin/PluginShow.jsx +++ b/ui/src/plugin/PluginShow.jsx @@ -48,8 +48,11 @@ const PluginShowLayout = () => { // Libraries permission state const [selectedLibraries, setSelectedLibraries] = useState([]) const [allLibraries, setAllLibraries] = useState(false) + const [allowWriteAccess, setAllowWriteAccess] = useState(false) const [lastRecordLibraries, setLastRecordLibraries] = useState(null) const [lastRecordAllLibraries, setLastRecordAllLibraries] = useState(null) + const [lastRecordAllowWriteAccess, setLastRecordAllowWriteAccess] = + useState(null) // Parse JSON config to object const jsonToObject = useCallback((jsonString) => { @@ -99,10 +102,12 @@ const PluginShowLayout = () => { if (record && !isDirty) { const recordLibraries = record.libraries || '' const recordAllLibraries = record.allLibraries || false + const recordAllowWriteAccess = record.allowWriteAccess || false if ( recordLibraries !== lastRecordLibraries || - recordAllLibraries !== lastRecordAllLibraries + recordAllLibraries !== lastRecordAllLibraries || + recordAllowWriteAccess !== lastRecordAllowWriteAccess ) { try { setSelectedLibraries( @@ -112,11 +117,19 @@ const PluginShowLayout = () => { setSelectedLibraries([]) } setAllLibraries(recordAllLibraries) + setAllowWriteAccess(recordAllowWriteAccess) setLastRecordLibraries(recordLibraries) setLastRecordAllLibraries(recordAllLibraries) + setLastRecordAllowWriteAccess(recordAllowWriteAccess) } } - }, [record, lastRecordLibraries, lastRecordAllLibraries, isDirty]) + }, [ + record, + lastRecordLibraries, + lastRecordAllLibraries, + lastRecordAllowWriteAccess, + isDirty, + ]) const handleConfigDataChange = useCallback( (newData, errors) => { @@ -152,6 +165,11 @@ const PluginShowLayout = () => { setIsDirty(true) }, []) + const handleAllowWriteAccessChange = useCallback((newAllowWriteAccess) => { + setAllowWriteAccess(newAllowWriteAccess) + setIsDirty(true) + }, []) + const [updatePlugin, { loading }] = useUpdate( 'plugin', record?.id, @@ -167,6 +185,7 @@ const PluginShowLayout = () => { setLastRecordAllUsers(null) setLastRecordLibraries(null) setLastRecordAllLibraries(null) + setLastRecordAllowWriteAccess(null) notify('resources.plugin.notifications.updated', 'info') }, onFailure: (err) => { @@ -199,6 +218,7 @@ const PluginShowLayout = () => { if (parsedManifest?.permissions?.library) { data.libraries = JSON.stringify(selectedLibraries) data.allLibraries = allLibraries + data.allowWriteAccess = allowWriteAccess } updatePlugin('plugin', record.id, data, record) @@ -210,6 +230,7 @@ const PluginShowLayout = () => { allUsers, selectedLibraries, allLibraries, + allowWriteAccess, ]) // Parse manifest @@ -294,8 +315,10 @@ const PluginShowLayout = () => { classes={classes} selectedLibraries={selectedLibraries} allLibraries={allLibraries} + allowWriteAccess={allowWriteAccess} onSelectedLibrariesChange={handleSelectedLibrariesChange} onAllLibrariesChange={handleAllLibrariesChange} + onAllowWriteAccessChange={handleAllowWriteAccessChange} /> From 2471bb9cf60227704e37d13c3b78c66fadf2f99a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Sat, 28 Feb 2026 23:12:17 -0500 Subject: [PATCH 21/60] feat(plugins): add TTL support, batch operations, and hardening to kvstore (#5127) * feat(plugins): add expires_at column to kvstore schema * feat(plugins): filter expired keys in kvstore Get, Has, List * feat(plugins): add periodic cleanup of expired kvstore keys * feat(plugins): add SetWithTTL, DeleteByPrefix, and GetMany to kvstore Add three new methods to the KVStore host service: - SetWithTTL: store key-value pairs with automatic expiration - DeleteByPrefix: remove all keys matching a prefix in one operation - GetMany: retrieve multiple values in a single call All methods include comprehensive unit tests covering edge cases, expiration behavior, size tracking, and LIKE-special characters. * feat(plugins): regenerate code and update test plugin for new kvstore methods Regenerate host function wrappers and PDK bindings for Go, Python, and Rust. Update the test-kvstore plugin to exercise SetWithTTL, DeleteByPrefix, and GetMany. * feat(plugins): add integration tests for new kvstore methods Add WASM integration tests for SetWithTTL, DeleteByPrefix, and GetMany operations through the plugin boundary, verifying end-to-end behavior including TTL expiration, prefix deletion, and batch retrieval. * fix(plugins): address lint issues in kvstore implementation Handle tx.Rollback error return and suppress gosec false positive for parameterized SQL query construction in GetMany. * fix(plugins): Set clears expires_at when overwriting a TTL'd key Previously, calling Set() on a key that was stored with SetWithTTL() would leave the expires_at value intact, causing the key to silently expire even though Set implies permanent storage. Also excludes expired keys from currentSize calculation at startup. * refactor(plugins): simplify kvstore by removing in-memory size cache Replaced the in-memory currentSize cache (atomic.Int64), periodic cleanup timer, and mutex with direct database queries for storage accounting. This eliminates race conditions and cache drift issues at negligible performance cost for plugin-sized datasets. Also unified Set and SetWithTTL into a shared setValue method, simplified DeleteByPrefix to use RowsAffected instead of a transaction, and added an index on expires_at for efficient expiration filtering. * feat(plugins): add generic SQLite migration helper and refactor kvstore schema Add a reusable migrateDB helper that tracks schema versions via SQLite's PRAGMA user_version and applies pending migrations transactionally. Replace the ad-hoc createKVStoreSchema function in kvstore with a declarative migrations slice, making it easy to add future schema changes. Remove the now-redundant schema migration test since migrateDB has its own test suite and every kvstore test exercises the migrations implicitly. Signed-off-by: Deluan * fix(plugins): harden kvstore with explicit NULL handling, prefix validation, and cleanup timeout - Use sql.NullString for expires_at to explicitly send NULL instead of relying on datetime('now', '') returning NULL by accident - Reject empty prefix in DeleteByPrefix to prevent accidental data wipe - Add 5s timeout context to cleanupExpired on Close - Replace time.Sleep in unit tests with pre-expired timestamps Signed-off-by: Deluan * refactor(plugins): use batch processing in GetMany Process keys in chunks of 200 using slice.CollectChunks to avoid hitting SQLite's SQLITE_MAX_VARIABLE_NUMBER limit with large key sets. * feat(plugins): add periodic cleanup goroutine for expired kvstore keys Use the manager's context to control a background goroutine that purges expired keys every hour, stopping naturally on shutdown when the context is cancelled. --------- Signed-off-by: Deluan --- plugins/host/kvstore.go | 41 +- plugins/host/kvstore_gen.go | 160 ++++++- plugins/host_kvstore.go | 260 ++++++++--- plugins/host_kvstore_test.go | 435 +++++++++++++++++- plugins/manager_loader.go | 2 +- plugins/migrate.go | 47 ++ plugins/migrate_test.go | 99 ++++ plugins/pdk/go/host/nd_host_kvstore.go | 208 ++++++++- plugins/pdk/go/host/nd_host_kvstore_stub.go | 77 +++- plugins/pdk/python/host/nd_host_kvstore.py | 138 +++++- .../rust/nd-pdk-host/src/nd_host_kvstore.rs | 167 ++++++- plugins/testdata/test-kvstore/main.go | 52 ++- 12 files changed, 1528 insertions(+), 158 deletions(-) create mode 100644 plugins/migrate.go create mode 100644 plugins/migrate_test.go diff --git a/plugins/host/kvstore.go b/plugins/host/kvstore.go index 4d9dafd20..aa2597f27 100644 --- a/plugins/host/kvstore.go +++ b/plugins/host/kvstore.go @@ -23,6 +23,20 @@ type KVStoreService interface { //nd:hostfunc Set(ctx context.Context, key string, value []byte) error + // SetWithTTL stores a byte value with the given key and a time-to-live. + // + // After ttlSeconds, the key is treated as non-existent and will be + // cleaned up lazily. ttlSeconds must be greater than 0. + // + // Parameters: + // - key: The storage key (max 256 bytes, UTF-8) + // - value: The byte slice to store + // - ttlSeconds: Time-to-live in seconds (must be > 0) + // + // Returns an error if the storage limit would be exceeded or the operation fails. + //nd:hostfunc + SetWithTTL(ctx context.Context, key string, value []byte, ttlSeconds int64) error + // Get retrieves a byte value from storage. // // Parameters: @@ -32,14 +46,15 @@ type KVStoreService interface { //nd:hostfunc Get(ctx context.Context, key string) (value []byte, exists bool, err error) - // Delete removes a value from storage. + // GetMany retrieves multiple values in a single call. // // Parameters: - // - key: The storage key + // - keys: The storage keys to retrieve // - // Returns an error if the operation fails. Does not return an error if the key doesn't exist. + // Returns a map of key to value for keys that exist and have not expired. + // Missing or expired keys are omitted from the result. //nd:hostfunc - Delete(ctx context.Context, key string) error + GetMany(ctx context.Context, keys []string) (values map[string][]byte, err error) // Has checks if a key exists in storage. // @@ -59,6 +74,24 @@ type KVStoreService interface { //nd:hostfunc List(ctx context.Context, prefix string) (keys []string, err error) + // Delete removes a value from storage. + // + // Parameters: + // - key: The storage key + // + // Returns an error if the operation fails. Does not return an error if the key doesn't exist. + //nd:hostfunc + Delete(ctx context.Context, key string) error + + // DeleteByPrefix removes all keys matching the given prefix. + // + // Parameters: + // - prefix: Key prefix to match (must not be empty) + // + // Returns the number of keys deleted. Includes expired keys. + //nd:hostfunc + DeleteByPrefix(ctx context.Context, prefix string) (deletedCount int64, err error) + // GetStorageUsed returns the total storage used by this plugin in bytes. //nd:hostfunc GetStorageUsed(ctx context.Context) (bytes int64, err error) diff --git a/plugins/host/kvstore_gen.go b/plugins/host/kvstore_gen.go index 2ad24959d..44ee3b131 100644 --- a/plugins/host/kvstore_gen.go +++ b/plugins/host/kvstore_gen.go @@ -20,6 +20,18 @@ type KVStoreSetResponse struct { Error string `json:"error,omitempty"` } +// KVStoreSetWithTTLRequest is the request type for KVStore.SetWithTTL. +type KVStoreSetWithTTLRequest struct { + Key string `json:"key"` + Value []byte `json:"value"` + TtlSeconds int64 `json:"ttlSeconds"` +} + +// KVStoreSetWithTTLResponse is the response type for KVStore.SetWithTTL. +type KVStoreSetWithTTLResponse struct { + Error string `json:"error,omitempty"` +} + // KVStoreGetRequest is the request type for KVStore.Get. type KVStoreGetRequest struct { Key string `json:"key"` @@ -32,14 +44,15 @@ type KVStoreGetResponse struct { Error string `json:"error,omitempty"` } -// KVStoreDeleteRequest is the request type for KVStore.Delete. -type KVStoreDeleteRequest struct { - Key string `json:"key"` +// KVStoreGetManyRequest is the request type for KVStore.GetMany. +type KVStoreGetManyRequest struct { + Keys []string `json:"keys"` } -// KVStoreDeleteResponse is the response type for KVStore.Delete. -type KVStoreDeleteResponse struct { - Error string `json:"error,omitempty"` +// KVStoreGetManyResponse is the response type for KVStore.GetMany. +type KVStoreGetManyResponse struct { + Values map[string][]byte `json:"values,omitempty"` + Error string `json:"error,omitempty"` } // KVStoreHasRequest is the request type for KVStore.Has. @@ -64,6 +77,27 @@ type KVStoreListResponse struct { Error string `json:"error,omitempty"` } +// KVStoreDeleteRequest is the request type for KVStore.Delete. +type KVStoreDeleteRequest struct { + Key string `json:"key"` +} + +// KVStoreDeleteResponse is the response type for KVStore.Delete. +type KVStoreDeleteResponse struct { + Error string `json:"error,omitempty"` +} + +// KVStoreDeleteByPrefixRequest is the request type for KVStore.DeleteByPrefix. +type KVStoreDeleteByPrefixRequest struct { + Prefix string `json:"prefix"` +} + +// KVStoreDeleteByPrefixResponse is the response type for KVStore.DeleteByPrefix. +type KVStoreDeleteByPrefixResponse struct { + DeletedCount int64 `json:"deletedCount,omitempty"` + Error string `json:"error,omitempty"` +} + // KVStoreGetStorageUsedResponse is the response type for KVStore.GetStorageUsed. type KVStoreGetStorageUsedResponse struct { Bytes int64 `json:"bytes,omitempty"` @@ -75,10 +109,13 @@ type KVStoreGetStorageUsedResponse struct { func RegisterKVStoreHostFunctions(service KVStoreService) []extism.HostFunction { return []extism.HostFunction{ newKVStoreSetHostFunction(service), + newKVStoreSetWithTTLHostFunction(service), newKVStoreGetHostFunction(service), - newKVStoreDeleteHostFunction(service), + newKVStoreGetManyHostFunction(service), newKVStoreHasHostFunction(service), newKVStoreListHostFunction(service), + newKVStoreDeleteHostFunction(service), + newKVStoreDeleteByPrefixHostFunction(service), newKVStoreGetStorageUsedHostFunction(service), } } @@ -114,6 +151,37 @@ func newKVStoreSetHostFunction(service KVStoreService) extism.HostFunction { ) } +func newKVStoreSetWithTTLHostFunction(service KVStoreService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "kvstore_setwithttl", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + kvstoreWriteError(p, stack, err) + return + } + var req KVStoreSetWithTTLRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + kvstoreWriteError(p, stack, err) + return + } + + // Call the service method + if svcErr := service.SetWithTTL(ctx, req.Key, req.Value, req.TtlSeconds); svcErr != nil { + kvstoreWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := KVStoreSetWithTTLResponse{} + kvstoreWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + func newKVStoreGetHostFunction(service KVStoreService) extism.HostFunction { return extism.NewHostFunctionWithStack( "kvstore_get", @@ -149,9 +217,9 @@ func newKVStoreGetHostFunction(service KVStoreService) extism.HostFunction { ) } -func newKVStoreDeleteHostFunction(service KVStoreService) extism.HostFunction { +func newKVStoreGetManyHostFunction(service KVStoreService) extism.HostFunction { return extism.NewHostFunctionWithStack( - "kvstore_delete", + "kvstore_getmany", func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { // Read JSON request from plugin memory reqBytes, err := p.ReadBytes(stack[0]) @@ -159,20 +227,23 @@ func newKVStoreDeleteHostFunction(service KVStoreService) extism.HostFunction { kvstoreWriteError(p, stack, err) return } - var req KVStoreDeleteRequest + var req KVStoreGetManyRequest if err := json.Unmarshal(reqBytes, &req); err != nil { kvstoreWriteError(p, stack, err) return } // Call the service method - if svcErr := service.Delete(ctx, req.Key); svcErr != nil { + values, svcErr := service.GetMany(ctx, req.Keys) + if svcErr != nil { kvstoreWriteError(p, stack, svcErr) return } // Write JSON response to plugin memory - resp := KVStoreDeleteResponse{} + resp := KVStoreGetManyResponse{ + Values: values, + } kvstoreWriteResponse(p, stack, resp) }, []extism.ValueType{extism.ValueTypePTR}, @@ -248,6 +319,71 @@ func newKVStoreListHostFunction(service KVStoreService) extism.HostFunction { ) } +func newKVStoreDeleteHostFunction(service KVStoreService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "kvstore_delete", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + kvstoreWriteError(p, stack, err) + return + } + var req KVStoreDeleteRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + kvstoreWriteError(p, stack, err) + return + } + + // Call the service method + if svcErr := service.Delete(ctx, req.Key); svcErr != nil { + kvstoreWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := KVStoreDeleteResponse{} + kvstoreWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + +func newKVStoreDeleteByPrefixHostFunction(service KVStoreService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "kvstore_deletebyprefix", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + kvstoreWriteError(p, stack, err) + return + } + var req KVStoreDeleteByPrefixRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + kvstoreWriteError(p, stack, err) + return + } + + // Call the service method + deletedcount, svcErr := service.DeleteByPrefix(ctx, req.Prefix) + if svcErr != nil { + kvstoreWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := KVStoreDeleteByPrefixResponse{ + DeletedCount: deletedcount, + } + kvstoreWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + func newKVStoreGetStorageUsedHostFunction(service KVStoreService) extism.HostFunction { return extism.NewHostFunctionWithStack( "kvstore_getstorageused", diff --git a/plugins/host_kvstore.go b/plugins/host_kvstore.go index 53d4da922..9aa37f7e5 100644 --- a/plugins/host_kvstore.go +++ b/plugins/host_kvstore.go @@ -7,14 +7,16 @@ import ( "fmt" "os" "path/filepath" + "slices" "strings" - "sync/atomic" + "time" "github.com/dustin/go-humanize" _ "github.com/mattn/go-sqlite3" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/plugins/host" + "github.com/navidrome/navidrome/utils/slice" ) const ( @@ -22,17 +24,22 @@ const ( maxKeyLength = 256 // Max key length in bytes ) +// notExpiredFilter is the SQL condition to exclude expired keys. +const notExpiredFilter = "(expires_at IS NULL OR expires_at > datetime('now'))" + +const cleanupInterval = 1 * time.Hour + // kvstoreServiceImpl implements the host.KVStoreService interface. // Each plugin gets its own SQLite database for isolation. type kvstoreServiceImpl struct { - pluginName string - db *sql.DB - maxSize int64 - currentSize atomic.Int64 // cached total size, updated on Set/Delete + pluginName string + db *sql.DB + maxSize int64 } // newKVStoreService creates a new kvstoreServiceImpl instance with its own SQLite database. -func newKVStoreService(pluginName string, perm *KVStorePermission) (*kvstoreServiceImpl, error) { +// The provided context controls the lifetime of the background cleanup goroutine. +func newKVStoreService(ctx context.Context, pluginName string, perm *KVStorePermission) (*kvstoreServiceImpl, error) { // Parse max size from permission, default to 1MB maxSize := int64(defaultMaxKVStoreSize) if perm != nil && perm.MaxSize != nil && *perm.MaxSize != "" { @@ -59,46 +66,69 @@ func newKVStoreService(pluginName string, perm *KVStorePermission) (*kvstoreServ db.SetMaxOpenConns(3) db.SetMaxIdleConns(1) - // Create schema + // Apply schema migrations if err := createKVStoreSchema(db); err != nil { db.Close() - return nil, fmt.Errorf("creating kvstore schema: %w", err) + return nil, fmt.Errorf("migrating kvstore schema: %w", err) } - // Load current storage size from database - var currentSize int64 - if err := db.QueryRow(`SELECT COALESCE(SUM(size), 0) FROM kvstore`).Scan(¤tSize); err != nil { - db.Close() - return nil, fmt.Errorf("loading storage size: %w", err) - } - - log.Debug("Initialized plugin kvstore", "plugin", pluginName, "path", dbPath, "maxSize", humanize.Bytes(uint64(maxSize)), "currentSize", humanize.Bytes(uint64(currentSize))) + log.Debug("Initialized plugin kvstore", "plugin", pluginName, "path", dbPath, "maxSize", humanize.Bytes(uint64(maxSize))) svc := &kvstoreServiceImpl{ pluginName: pluginName, db: db, maxSize: maxSize, } - svc.currentSize.Store(currentSize) + go svc.cleanupLoop(ctx) return svc, nil } +// createKVStoreSchema applies schema migrations to the kvstore database. +// New migrations must be appended at the end of the slice. func createKVStoreSchema(db *sql.DB) error { - _, err := db.Exec(` - CREATE TABLE IF NOT EXISTS kvstore ( + return migrateDB(db, []string{ + `CREATE TABLE IF NOT EXISTS kvstore ( key TEXT PRIMARY KEY NOT NULL, value BLOB NOT NULL, size INTEGER NOT NULL, created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP - ) - `) - return err + )`, + `ALTER TABLE kvstore ADD COLUMN expires_at DATETIME DEFAULT NULL`, + `CREATE INDEX idx_kvstore_expires_at ON kvstore(expires_at)`, + }) } -// Set stores a byte value with the given key. -func (s *kvstoreServiceImpl) Set(ctx context.Context, key string, value []byte) error { - // Validate key +// storageUsed returns the current total storage used by non-expired keys. +func (s *kvstoreServiceImpl) storageUsed(ctx context.Context) (int64, error) { + var used int64 + err := s.db.QueryRowContext(ctx, `SELECT COALESCE(SUM(size), 0) FROM kvstore WHERE `+notExpiredFilter).Scan(&used) + if err != nil { + return 0, fmt.Errorf("calculating storage used: %w", err) + } + return used, nil +} + +// checkStorageLimit verifies that adding delta bytes would not exceed the storage limit. +func (s *kvstoreServiceImpl) checkStorageLimit(ctx context.Context, delta int64) error { + if delta <= 0 { + return nil + } + used, err := s.storageUsed(ctx) + if err != nil { + return err + } + newTotal := used + delta + if newTotal > s.maxSize { + return fmt.Errorf("storage limit exceeded: would use %s of %s allowed", + humanize.Bytes(uint64(newTotal)), humanize.Bytes(uint64(s.maxSize))) + } + return nil +} + +// setValue is the shared implementation for Set and SetWithTTL. +// A ttlSeconds of 0 means no expiration. +func (s *kvstoreServiceImpl) setValue(ctx context.Context, key string, value []byte, ttlSeconds int64) error { if len(key) == 0 { return fmt.Errorf("key cannot be empty") } @@ -108,46 +138,59 @@ func (s *kvstoreServiceImpl) Set(ctx context.Context, key string, value []byte) newValueSize := int64(len(value)) - // Get current size of this key (if it exists) to calculate delta + // Get current size of this key (if it exists and not expired) to calculate delta var oldSize int64 - err := s.db.QueryRowContext(ctx, `SELECT COALESCE(size, 0) FROM kvstore WHERE key = ?`, key).Scan(&oldSize) + err := s.db.QueryRowContext(ctx, `SELECT COALESCE(size, 0) FROM kvstore WHERE key = ? AND `+notExpiredFilter, key).Scan(&oldSize) if err != nil && !errors.Is(err, sql.ErrNoRows) { return fmt.Errorf("checking existing key: %w", err) } - // Check size limits using cached total - delta := newValueSize - oldSize - newTotal := s.currentSize.Load() + delta - if newTotal > s.maxSize { - return fmt.Errorf("storage limit exceeded: would use %s of %s allowed", - humanize.Bytes(uint64(newTotal)), humanize.Bytes(uint64(s.maxSize))) + if err := s.checkStorageLimit(ctx, newValueSize-oldSize); err != nil { + return err + } + + // Compute expires_at: sql.NullString{Valid:false} sends NULL (no expiration), + // otherwise we send a concrete timestamp. + var expiresAt sql.NullString + if ttlSeconds > 0 { + expiresAt = sql.NullString{String: fmt.Sprintf("+%d seconds", ttlSeconds), Valid: true} } - // Upsert the value _, err = s.db.ExecContext(ctx, ` - INSERT INTO kvstore (key, value, size, created_at, updated_at) - VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) - ON CONFLICT(key) DO UPDATE SET - value = excluded.value, + INSERT INTO kvstore (key, value, size, created_at, updated_at, expires_at) + VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, datetime('now', ?)) + ON CONFLICT(key) DO UPDATE SET + value = excluded.value, size = excluded.size, - updated_at = CURRENT_TIMESTAMP - `, key, value, newValueSize) + updated_at = CURRENT_TIMESTAMP, + expires_at = excluded.expires_at + `, key, value, newValueSize, expiresAt) if err != nil { return fmt.Errorf("storing value: %w", err) } - // Update cached size - s.currentSize.Add(delta) - - log.Trace(ctx, "KVStore.Set", "plugin", s.pluginName, "key", key, "size", newValueSize) + log.Trace(ctx, "KVStore.Set", "plugin", s.pluginName, "key", key, "size", newValueSize, "ttlSeconds", ttlSeconds) return nil } +// Set stores a byte value with the given key. +func (s *kvstoreServiceImpl) Set(ctx context.Context, key string, value []byte) error { + return s.setValue(ctx, key, value, 0) +} + +// SetWithTTL stores a byte value with the given key and a time-to-live. +func (s *kvstoreServiceImpl) SetWithTTL(ctx context.Context, key string, value []byte, ttlSeconds int64) error { + if ttlSeconds <= 0 { + return fmt.Errorf("ttlSeconds must be greater than 0") + } + return s.setValue(ctx, key, value, ttlSeconds) +} + // Get retrieves a byte value from storage. func (s *kvstoreServiceImpl) Get(ctx context.Context, key string) ([]byte, bool, error) { var value []byte - err := s.db.QueryRowContext(ctx, `SELECT value FROM kvstore WHERE key = ?`, key).Scan(&value) - if err == sql.ErrNoRows { + err := s.db.QueryRowContext(ctx, `SELECT value FROM kvstore WHERE key = ? AND `+notExpiredFilter, key).Scan(&value) + if errors.Is(err, sql.ErrNoRows) { return nil, false, nil } if err != nil { @@ -160,25 +203,11 @@ func (s *kvstoreServiceImpl) Get(ctx context.Context, key string) ([]byte, bool, // Delete removes a value from storage. func (s *kvstoreServiceImpl) Delete(ctx context.Context, key string) error { - // Get size of the key being deleted to update cache - var oldSize int64 - err := s.db.QueryRowContext(ctx, `SELECT size FROM kvstore WHERE key = ?`, key).Scan(&oldSize) - if errors.Is(err, sql.ErrNoRows) { - // Key doesn't exist, nothing to delete - return nil - } - if err != nil { - return fmt.Errorf("checking key size: %w", err) - } - - _, err = s.db.ExecContext(ctx, `DELETE FROM kvstore WHERE key = ?`, key) + _, err := s.db.ExecContext(ctx, `DELETE FROM kvstore WHERE key = ?`, key) if err != nil { return fmt.Errorf("deleting value: %w", err) } - // Update cached size - s.currentSize.Add(-oldSize) - log.Trace(ctx, "KVStore.Delete", "plugin", s.pluginName, "key", key) return nil } @@ -186,7 +215,7 @@ func (s *kvstoreServiceImpl) Delete(ctx context.Context, key string) error { // Has checks if a key exists in storage. func (s *kvstoreServiceImpl) Has(ctx context.Context, key string) (bool, error) { var count int - err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM kvstore WHERE key = ?`, key).Scan(&count) + err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM kvstore WHERE key = ? AND `+notExpiredFilter, key).Scan(&count) if err != nil { return false, fmt.Errorf("checking key: %w", err) } @@ -200,12 +229,12 @@ func (s *kvstoreServiceImpl) List(ctx context.Context, prefix string) ([]string, var err error if prefix == "" { - rows, err = s.db.QueryContext(ctx, `SELECT key FROM kvstore ORDER BY key`) + rows, err = s.db.QueryContext(ctx, `SELECT key FROM kvstore WHERE `+notExpiredFilter+` ORDER BY key`) } else { // Escape special LIKE characters in prefix escapedPrefix := strings.ReplaceAll(prefix, "%", "\\%") escapedPrefix = strings.ReplaceAll(escapedPrefix, "_", "\\_") - rows, err = s.db.QueryContext(ctx, `SELECT key FROM kvstore WHERE key LIKE ? ESCAPE '\' ORDER BY key`, escapedPrefix+"%") + rows, err = s.db.QueryContext(ctx, `SELECT key FROM kvstore WHERE key LIKE ? ESCAPE '\' AND `+notExpiredFilter+` ORDER BY key`, escapedPrefix+"%") } if err != nil { return nil, fmt.Errorf("listing keys: %w", err) @@ -231,16 +260,113 @@ func (s *kvstoreServiceImpl) List(ctx context.Context, prefix string) ([]string, // GetStorageUsed returns the total storage used by this plugin in bytes. func (s *kvstoreServiceImpl) GetStorageUsed(ctx context.Context) (int64, error) { - used := s.currentSize.Load() + used, err := s.storageUsed(ctx) + if err != nil { + return 0, err + } log.Trace(ctx, "KVStore.GetStorageUsed", "plugin", s.pluginName, "bytes", used) return used, nil } -// Close closes the SQLite database connection. -// This is called when the plugin is unloaded. +// DeleteByPrefix removes all keys matching the given prefix. +func (s *kvstoreServiceImpl) DeleteByPrefix(ctx context.Context, prefix string) (int64, error) { + if prefix == "" { + return 0, fmt.Errorf("prefix cannot be empty") + } + + escapedPrefix := strings.ReplaceAll(prefix, "%", "\\%") + escapedPrefix = strings.ReplaceAll(escapedPrefix, "_", "\\_") + result, err := s.db.ExecContext(ctx, `DELETE FROM kvstore WHERE key LIKE ? ESCAPE '\'`, escapedPrefix+"%") + if err != nil { + return 0, fmt.Errorf("deleting keys: %w", err) + } + + count, err := result.RowsAffected() + if err != nil { + return 0, fmt.Errorf("getting deleted count: %w", err) + } + + log.Trace(ctx, "KVStore.DeleteByPrefix", "plugin", s.pluginName, "prefix", prefix, "deletedCount", count) + return count, nil +} + +// GetMany retrieves multiple values in a single call, processing keys in batches. +func (s *kvstoreServiceImpl) GetMany(ctx context.Context, keys []string) (map[string][]byte, error) { + if len(keys) == 0 { + return map[string][]byte{}, nil + } + + const batchSize = 200 + result := make(map[string][]byte) + for chunk := range slice.CollectChunks(slices.Values(keys), batchSize) { + placeholders := make([]string, len(chunk)) + args := make([]any, len(chunk)) + for i, key := range chunk { + placeholders[i] = "?" + args[i] = key + } + + query := `SELECT key, value FROM kvstore WHERE key IN (` + strings.Join(placeholders, ",") + `) AND ` + notExpiredFilter //nolint:gosec // placeholders are always "?" + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, fmt.Errorf("querying values: %w", err) + } + + for rows.Next() { + var key string + var value []byte + if err := rows.Scan(&key, &value); err != nil { + rows.Close() + return nil, fmt.Errorf("scanning value: %w", err) + } + result[key] = value + } + if err := rows.Err(); err != nil { + rows.Close() + return nil, fmt.Errorf("iterating values: %w", err) + } + rows.Close() + } + + log.Trace(ctx, "KVStore.GetMany", "plugin", s.pluginName, "requested", len(keys), "found", len(result)) + return result, nil +} + +// cleanupLoop periodically removes expired keys from the database. +// It stops when the provided context is cancelled. +func (s *kvstoreServiceImpl) cleanupLoop(ctx context.Context) { + ticker := time.NewTicker(cleanupInterval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + s.cleanupExpired(ctx) + } + } +} + +// cleanupExpired removes all expired keys from the database to reclaim disk space. +func (s *kvstoreServiceImpl) cleanupExpired(ctx context.Context) { + result, err := s.db.ExecContext(ctx, `DELETE FROM kvstore WHERE expires_at IS NOT NULL AND expires_at <= datetime('now')`) + if err != nil { + log.Error(ctx, "KVStore cleanup: failed to delete expired keys", "plugin", s.pluginName, err) + return + } + if count, err := result.RowsAffected(); err == nil && count > 0 { + log.Debug("KVStore cleanup completed", "plugin", s.pluginName, "deletedKeys", count) + } +} + +// Close runs a final cleanup and closes the SQLite database connection. +// The cleanup goroutine is stopped by the context passed to newKVStoreService. func (s *kvstoreServiceImpl) Close() error { if s.db != nil { log.Debug("Closing plugin kvstore", "plugin", s.pluginName) + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + s.cleanupExpired(ctx) return s.db.Close() } return nil diff --git a/plugins/host_kvstore_test.go b/plugins/host_kvstore_test.go index 3e2cbd01a..b900a659a 100644 --- a/plugins/host_kvstore_test.go +++ b/plugins/host_kvstore_test.go @@ -12,6 +12,7 @@ import ( "os" "path/filepath" "strings" + "time" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" @@ -37,7 +38,7 @@ var _ = Describe("KVStoreService", func() { // Create service with 1KB limit for testing maxSize := "1KB" - service, err = newKVStoreService("test_plugin", &KVStorePermission{MaxSize: &maxSize}) + service, err = newKVStoreService(ctx, "test_plugin", &KVStorePermission{MaxSize: &maxSize}) Expect(err).ToNot(HaveOccurred()) }) @@ -253,7 +254,7 @@ var _ = Describe("KVStoreService", func() { Expect(service.Close()).To(Succeed()) maxSize := "1KB" - service2, err := newKVStoreService("test_plugin", &KVStorePermission{MaxSize: &maxSize}) + service2, err := newKVStoreService(ctx, "test_plugin", &KVStorePermission{MaxSize: &maxSize}) Expect(err).ToNot(HaveOccurred()) defer service2.Close() @@ -302,7 +303,7 @@ var _ = Describe("KVStoreService", func() { Describe("Plugin Isolation", func() { It("isolates data between plugins", func() { - service2, err := newKVStoreService("other_plugin", &KVStorePermission{}) + service2, err := newKVStoreService(ctx, "other_plugin", &KVStorePermission{}) Expect(err).ToNot(HaveOccurred()) defer service2.Close() @@ -321,7 +322,7 @@ var _ = Describe("KVStoreService", func() { }) It("creates separate database files per plugin", func() { - service2, err := newKVStoreService("other_plugin", &KVStorePermission{}) + service2, err := newKVStoreService(ctx, "other_plugin", &KVStorePermission{}) Expect(err).ToNot(HaveOccurred()) defer service2.Close() @@ -343,6 +344,309 @@ var _ = Describe("KVStoreService", func() { Expect(err).To(HaveOccurred()) }) }) + + Describe("TTL Expiration", func() { + It("Get returns not-exists for expired keys", func() { + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('expired_key', 'old', 3, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + value, exists, err := service.Get(ctx, "expired_key") + Expect(err).ToNot(HaveOccurred()) + Expect(exists).To(BeFalse()) + Expect(value).To(BeNil()) + }) + It("Has returns false for expired keys", func() { + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('expired_has', 'old', 3, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + exists, err := service.Has(ctx, "expired_has") + Expect(err).ToNot(HaveOccurred()) + Expect(exists).To(BeFalse()) + }) + It("List excludes expired keys", func() { + Expect(service.Set(ctx, "live:1", []byte("alive"))).To(Succeed()) + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('live:expired', 'dead', 4, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + keys, err := service.List(ctx, "live:") + Expect(err).ToNot(HaveOccurred()) + Expect(keys).To(HaveLen(1)) + Expect(keys).To(ContainElement("live:1")) + }) + It("Get returns value for non-expired keys with TTL", func() { + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('future_key', 'still alive', 11, datetime('now', '+3600 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + value, exists, err := service.Get(ctx, "future_key") + Expect(err).ToNot(HaveOccurred()) + Expect(exists).To(BeTrue()) + Expect(value).To(Equal([]byte("still alive"))) + }) + It("Set clears expires_at from a key previously set with TTL", func() { + // Insert a key with a TTL that has already expired + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('ttl_then_set', 'temp', 4, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + + // Overwrite with Set (no TTL) — should become permanent + err = service.Set(ctx, "ttl_then_set", []byte("permanent")) + Expect(err).ToNot(HaveOccurred()) + + // Should exist because Set cleared expires_at + value, exists, err := service.Get(ctx, "ttl_then_set") + Expect(err).ToNot(HaveOccurred()) + Expect(exists).To(BeTrue()) + Expect(value).To(Equal([]byte("permanent"))) + + // Verify expires_at is actually NULL + var expiresAt *string + Expect(service.db.QueryRow(`SELECT expires_at FROM kvstore WHERE key = 'ttl_then_set'`).Scan(&expiresAt)).To(Succeed()) + Expect(expiresAt).To(BeNil()) + }) + It("expired keys are not counted in storage used", func() { + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('expired_key', '12345', 5, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + + // Expired keys should not be counted + used, err := service.GetStorageUsed(ctx) + Expect(err).ToNot(HaveOccurred()) + Expect(used).To(Equal(int64(0))) + }) + It("cleanup removes expired rows from disk", func() { + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('cleanup_me', '12345', 5, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + + // Row exists in DB but is logically expired + var count int + Expect(service.db.QueryRow(`SELECT COUNT(*) FROM kvstore`).Scan(&count)).To(Succeed()) + Expect(count).To(Equal(1)) + + service.cleanupExpired(ctx) + + // Row should be physically deleted + Expect(service.db.QueryRow(`SELECT COUNT(*) FROM kvstore`).Scan(&count)).To(Succeed()) + Expect(count).To(Equal(0)) + }) + }) + + Describe("SetWithTTL", func() { + It("stores value that is retrievable before expiry", func() { + err := service.SetWithTTL(ctx, "ttl_key", []byte("ttl_value"), 3600) + Expect(err).ToNot(HaveOccurred()) + + value, exists, err := service.Get(ctx, "ttl_key") + Expect(err).ToNot(HaveOccurred()) + Expect(exists).To(BeTrue()) + Expect(value).To(Equal([]byte("ttl_value"))) + }) + + It("value is not retrievable after expiry", func() { + // Insert a key with an already-expired TTL + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('short_ttl', 'gone_soon', 9, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + + _, exists, err := service.Get(ctx, "short_ttl") + Expect(err).ToNot(HaveOccurred()) + Expect(exists).To(BeFalse()) + }) + + It("rejects ttlSeconds <= 0", func() { + err := service.SetWithTTL(ctx, "bad_ttl", []byte("value"), 0) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("ttlSeconds must be greater than 0")) + + err = service.SetWithTTL(ctx, "bad_ttl", []byte("value"), -5) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("ttlSeconds must be greater than 0")) + }) + + It("validates key same as Set", func() { + err := service.SetWithTTL(ctx, "", []byte("value"), 60) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("key cannot be empty")) + }) + + It("enforces size limits same as Set", func() { + bigValue := make([]byte, 2048) + err := service.SetWithTTL(ctx, "big_ttl", bigValue, 60) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("storage limit exceeded")) + }) + + It("overwrites existing key and updates TTL", func() { + // Insert a key with an already-expired TTL + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('overwrite_ttl', 'first', 5, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + + // Overwrite with a long TTL — should be retrievable + err = service.SetWithTTL(ctx, "overwrite_ttl", []byte("second"), 3600) + Expect(err).ToNot(HaveOccurred()) + + value, exists, err := service.Get(ctx, "overwrite_ttl") + Expect(err).ToNot(HaveOccurred()) + Expect(exists).To(BeTrue()) + Expect(value).To(Equal([]byte("second"))) + }) + + It("tracks storage correctly", func() { + err := service.SetWithTTL(ctx, "sized_ttl", []byte("12345"), 3600) + Expect(err).ToNot(HaveOccurred()) + + used, err := service.GetStorageUsed(ctx) + Expect(err).ToNot(HaveOccurred()) + Expect(used).To(Equal(int64(5))) + }) + }) + + Describe("DeleteByPrefix", func() { + BeforeEach(func() { + Expect(service.Set(ctx, "cache:user:1", []byte("Alice"))).To(Succeed()) + Expect(service.Set(ctx, "cache:user:2", []byte("Bob"))).To(Succeed()) + Expect(service.Set(ctx, "cache:item:1", []byte("Widget"))).To(Succeed()) + Expect(service.Set(ctx, "data:important", []byte("keep"))).To(Succeed()) + }) + + It("deletes all keys with the given prefix", func() { + deleted, err := service.DeleteByPrefix(ctx, "cache:user:") + Expect(err).ToNot(HaveOccurred()) + Expect(deleted).To(Equal(int64(2))) + + keys, err := service.List(ctx, "") + Expect(err).ToNot(HaveOccurred()) + Expect(keys).To(HaveLen(2)) + Expect(keys).To(ContainElements("cache:item:1", "data:important")) + }) + + It("rejects empty prefix", func() { + _, err := service.DeleteByPrefix(ctx, "") + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("prefix cannot be empty")) + }) + + It("returns 0 when no keys match", func() { + deleted, err := service.DeleteByPrefix(ctx, "nonexistent:") + Expect(err).ToNot(HaveOccurred()) + Expect(deleted).To(Equal(int64(0))) + }) + + It("updates storage size correctly", func() { + usedBefore, _ := service.GetStorageUsed(ctx) + Expect(usedBefore).To(BeNumerically(">", 0)) + + _, err := service.DeleteByPrefix(ctx, "cache:") + Expect(err).ToNot(HaveOccurred()) + + usedAfter, _ := service.GetStorageUsed(ctx) + Expect(usedAfter).To(Equal(int64(4))) + }) + + It("handles special LIKE characters in prefix", func() { + Expect(service.Set(ctx, "test%special", []byte("v1"))).To(Succeed()) + Expect(service.Set(ctx, "test_special", []byte("v2"))).To(Succeed()) + Expect(service.Set(ctx, "testXspecial", []byte("v3"))).To(Succeed()) + + deleted, err := service.DeleteByPrefix(ctx, "test%") + Expect(err).ToNot(HaveOccurred()) + Expect(deleted).To(Equal(int64(1))) + + exists, _ := service.Has(ctx, "test_special") + Expect(exists).To(BeTrue()) + exists, _ = service.Has(ctx, "testXspecial") + Expect(exists).To(BeTrue()) + }) + + It("also deletes expired keys matching prefix", func() { + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('cache:expired', 'old', 3, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + + deleted, err := service.DeleteByPrefix(ctx, "cache:") + Expect(err).ToNot(HaveOccurred()) + Expect(deleted).To(Equal(int64(4))) + }) + }) + + Describe("GetMany", func() { + BeforeEach(func() { + Expect(service.Set(ctx, "key1", []byte("value1"))).To(Succeed()) + Expect(service.Set(ctx, "key2", []byte("value2"))).To(Succeed()) + Expect(service.Set(ctx, "key3", []byte("value3"))).To(Succeed()) + }) + + It("retrieves multiple values at once", func() { + values, err := service.GetMany(ctx, []string{"key1", "key2", "key3"}) + Expect(err).ToNot(HaveOccurred()) + Expect(values).To(HaveLen(3)) + Expect(values["key1"]).To(Equal([]byte("value1"))) + Expect(values["key2"]).To(Equal([]byte("value2"))) + Expect(values["key3"]).To(Equal([]byte("value3"))) + }) + + It("omits missing keys from result", func() { + values, err := service.GetMany(ctx, []string{"key1", "missing", "key3"}) + Expect(err).ToNot(HaveOccurred()) + Expect(values).To(HaveLen(2)) + Expect(values["key1"]).To(Equal([]byte("value1"))) + Expect(values["key3"]).To(Equal([]byte("value3"))) + _, hasMissing := values["missing"] + Expect(hasMissing).To(BeFalse()) + }) + + It("returns empty map for empty keys slice", func() { + values, err := service.GetMany(ctx, []string{}) + Expect(err).ToNot(HaveOccurred()) + Expect(values).To(BeEmpty()) + }) + + It("returns empty map for nil keys slice", func() { + values, err := service.GetMany(ctx, nil) + Expect(err).ToNot(HaveOccurred()) + Expect(values).To(BeEmpty()) + }) + + It("excludes expired keys", func() { + _, err := service.db.Exec(` + INSERT INTO kvstore (key, value, size, expires_at) + VALUES ('expired_many', 'old', 3, datetime('now', '-1 seconds')) + `) + Expect(err).ToNot(HaveOccurred()) + + values, err := service.GetMany(ctx, []string{"key1", "expired_many"}) + Expect(err).ToNot(HaveOccurred()) + Expect(values).To(HaveLen(1)) + Expect(values["key1"]).To(Equal([]byte("value1"))) + }) + + It("handles all keys missing", func() { + values, err := service.GetMany(ctx, []string{"nope1", "nope2"}) + Expect(err).ToNot(HaveOccurred()) + Expect(values).To(BeEmpty()) + }) + }) }) var _ = Describe("KVStoreService Integration", Ordered, func() { @@ -416,17 +720,21 @@ var _ = Describe("KVStoreService Integration", Ordered, func() { Describe("KVStore Operations via Plugin", func() { type testKVStoreInput struct { - Operation string `json:"operation"` - Key string `json:"key"` - Value []byte `json:"value,omitempty"` - Prefix string `json:"prefix,omitempty"` + Operation string `json:"operation"` + Key string `json:"key"` + Value []byte `json:"value,omitempty"` + Prefix string `json:"prefix,omitempty"` + TTLSeconds int64 `json:"ttl_seconds,omitempty"` + Keys []string `json:"keys,omitempty"` } type testKVStoreOutput struct { - Value []byte `json:"value,omitempty"` - Exists bool `json:"exists,omitempty"` - Keys []string `json:"keys,omitempty"` - StorageUsed int64 `json:"storage_used,omitempty"` - Error *string `json:"error,omitempty"` + Value []byte `json:"value,omitempty"` + Values map[string][]byte `json:"values,omitempty"` + Exists bool `json:"exists,omitempty"` + Keys []string `json:"keys,omitempty"` + StorageUsed int64 `json:"storage_used,omitempty"` + DeletedCount int64 `json:"deleted_count,omitempty"` + Error *string `json:"error,omitempty"` } callTestKVStore := func(ctx context.Context, input testKVStoreInput) (*testKVStoreOutput, error) { @@ -594,6 +902,107 @@ var _ = Describe("KVStoreService Integration", Ordered, func() { Expect(output.Exists).To(BeTrue()) Expect(output.Value).To(Equal(binaryData)) }) + + It("should set value with TTL and expire it", func() { + ctx := GinkgoT().Context() + + // Set value with 1 second TTL + _, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "set_with_ttl", + Key: "ttl_key", + Value: []byte("temporary"), + TTLSeconds: 1, + }) + Expect(err).ToNot(HaveOccurred()) + + // Immediately should exist + output, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "get", + Key: "ttl_key", + }) + Expect(err).ToNot(HaveOccurred()) + Expect(output.Exists).To(BeTrue()) + Expect(output.Value).To(Equal([]byte("temporary"))) + + // Wait for expiration + time.Sleep(2 * time.Second) + + // Should no longer exist + output, err = callTestKVStore(ctx, testKVStoreInput{ + Operation: "get", + Key: "ttl_key", + }) + Expect(err).ToNot(HaveOccurred()) + Expect(output.Exists).To(BeFalse()) + }) + + It("should delete keys by prefix", func() { + ctx := GinkgoT().Context() + + // Set multiple keys with shared prefix + for _, key := range []string{"del_prefix:a", "del_prefix:b", "keep:c"} { + _, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "set", + Key: key, + Value: []byte("value"), + }) + Expect(err).ToNot(HaveOccurred()) + } + + // Delete by prefix + output, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "delete_by_prefix", + Prefix: "del_prefix:", + }) + Expect(err).ToNot(HaveOccurred()) + Expect(output.DeletedCount).To(Equal(int64(2))) + + // Verify remaining key + getOutput, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "has", + Key: "keep:c", + }) + Expect(err).ToNot(HaveOccurred()) + Expect(getOutput.Exists).To(BeTrue()) + + // Verify deleted keys are gone + getOutput, err = callTestKVStore(ctx, testKVStoreInput{ + Operation: "has", + Key: "del_prefix:a", + }) + Expect(err).ToNot(HaveOccurred()) + Expect(getOutput.Exists).To(BeFalse()) + }) + + It("should get many values at once", func() { + ctx := GinkgoT().Context() + + // Set multiple keys + for _, kv := range []struct{ k, v string }{ + {"many:1", "val1"}, + {"many:2", "val2"}, + {"many:3", "val3"}, + } { + _, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "set", + Key: kv.k, + Value: []byte(kv.v), + }) + Expect(err).ToNot(HaveOccurred()) + } + + // Get many, including a missing key + output, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "get_many", + Keys: []string{"many:1", "many:3", "many:missing"}, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(output.Values).To(HaveLen(2)) + Expect(output.Values["many:1"]).To(Equal([]byte("val1"))) + Expect(output.Values["many:3"]).To(Equal([]byte("val3"))) + _, hasMissing := output.Values["many:missing"] + Expect(hasMissing).To(BeFalse()) + }) }) Describe("Database Isolation", func() { diff --git a/plugins/manager_loader.go b/plugins/manager_loader.go index 688c4519c..610dbd028 100644 --- a/plugins/manager_loader.go +++ b/plugins/manager_loader.go @@ -103,7 +103,7 @@ var hostServices = []hostServiceEntry{ hasPermission: func(p *Permissions) bool { return p != nil && p.Kvstore != nil }, create: func(ctx *serviceContext) ([]extism.HostFunction, io.Closer) { perm := ctx.permissions.Kvstore - service, err := newKVStoreService(ctx.pluginName, perm) + service, err := newKVStoreService(ctx.manager.ctx, ctx.pluginName, perm) if err != nil { log.Error("Failed to create KVStore service", "plugin", ctx.pluginName, err) return nil, nil diff --git a/plugins/migrate.go b/plugins/migrate.go new file mode 100644 index 000000000..332e34838 --- /dev/null +++ b/plugins/migrate.go @@ -0,0 +1,47 @@ +package plugins + +import ( + "database/sql" + "fmt" +) + +// migrateDB applies schema migrations to a SQLite database. +// +// Each entry in migrations is a single SQL statement. The current schema version +// is tracked using SQLite's built-in PRAGMA user_version. Only statements after +// the current version are executed, within a single transaction. +func migrateDB(db *sql.DB, migrations []string) error { + var version int + if err := db.QueryRow(`PRAGMA user_version`).Scan(&version); err != nil { + return fmt.Errorf("reading schema version: %w", err) + } + + if version >= len(migrations) { + return nil + } + + tx, err := db.Begin() + if err != nil { + return fmt.Errorf("starting migration transaction: %w", err) + } + defer func() { _ = tx.Rollback() }() + + for i := version; i < len(migrations); i++ { + if _, err := tx.Exec(migrations[i]); err != nil { + return fmt.Errorf("migration %d failed: %w", i+1, err) + } + } + + // PRAGMA statements cannot be executed inside a transaction in some SQLite + // drivers, but with mattn/go-sqlite3 this works. We set it inside the tx + // so that a failed commit leaves the version unchanged. + if _, err := tx.Exec(fmt.Sprintf(`PRAGMA user_version = %d`, len(migrations))); err != nil { + return fmt.Errorf("updating schema version: %w", err) + } + + if err := tx.Commit(); err != nil { + return fmt.Errorf("committing migrations: %w", err) + } + + return nil +} diff --git a/plugins/migrate_test.go b/plugins/migrate_test.go new file mode 100644 index 000000000..17ed43c5c --- /dev/null +++ b/plugins/migrate_test.go @@ -0,0 +1,99 @@ +//go:build !windows + +package plugins + +import ( + "database/sql" + + _ "github.com/mattn/go-sqlite3" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("migrateDB", func() { + var db *sql.DB + + BeforeEach(func() { + var err error + db, err = sql.Open("sqlite3", ":memory:") + Expect(err).ToNot(HaveOccurred()) + }) + + AfterEach(func() { + if db != nil { + db.Close() + } + }) + + getUserVersion := func() int { + var version int + Expect(db.QueryRow(`PRAGMA user_version`).Scan(&version)).To(Succeed()) + return version + } + + It("applies all migrations on a fresh database", func() { + migrations := []string{ + `CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT)`, + `ALTER TABLE test ADD COLUMN email TEXT`, + } + + Expect(migrateDB(db, migrations)).To(Succeed()) + Expect(getUserVersion()).To(Equal(2)) + + // Verify schema + _, err := db.Exec(`INSERT INTO test (id, name, email) VALUES (1, 'Alice', 'alice@test.com')`) + Expect(err).ToNot(HaveOccurred()) + }) + + It("skips already applied migrations", func() { + migrations1 := []string{ + `CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT)`, + } + Expect(migrateDB(db, migrations1)).To(Succeed()) + Expect(getUserVersion()).To(Equal(1)) + + // Add a new migration + migrations2 := []string{ + `CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT)`, + `ALTER TABLE test ADD COLUMN email TEXT`, + } + Expect(migrateDB(db, migrations2)).To(Succeed()) + Expect(getUserVersion()).To(Equal(2)) + + // Verify the new column exists + _, err := db.Exec(`INSERT INTO test (id, name, email) VALUES (1, 'Alice', 'alice@test.com')`) + Expect(err).ToNot(HaveOccurred()) + }) + + It("is a no-op when all migrations are applied", func() { + migrations := []string{ + `CREATE TABLE test (id INTEGER PRIMARY KEY)`, + } + Expect(migrateDB(db, migrations)).To(Succeed()) + Expect(migrateDB(db, migrations)).To(Succeed()) + Expect(getUserVersion()).To(Equal(1)) + }) + + It("is a no-op with empty migrations slice", func() { + Expect(migrateDB(db, nil)).To(Succeed()) + Expect(getUserVersion()).To(Equal(0)) + }) + + It("rolls back on failure", func() { + migrations := []string{ + `CREATE TABLE test (id INTEGER PRIMARY KEY)`, + `INVALID SQL STATEMENT`, + } + + err := migrateDB(db, migrations) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("migration 2 failed")) + + // Version should remain 0 (rolled back) + Expect(getUserVersion()).To(Equal(0)) + + // Table should not exist (rolled back) + _, err = db.Exec(`INSERT INTO test (id) VALUES (1)`) + Expect(err).To(HaveOccurred()) + }) +}) diff --git a/plugins/pdk/go/host/nd_host_kvstore.go b/plugins/pdk/go/host/nd_host_kvstore.go index 92ac9d772..15e1e366a 100644 --- a/plugins/pdk/go/host/nd_host_kvstore.go +++ b/plugins/pdk/go/host/nd_host_kvstore.go @@ -19,15 +19,20 @@ import ( //go:wasmimport extism:host/user kvstore_set func kvstore_set(uint64) uint64 +// kvstore_setwithttl is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user kvstore_setwithttl +func kvstore_setwithttl(uint64) uint64 + // kvstore_get is the host function provided by Navidrome. // //go:wasmimport extism:host/user kvstore_get func kvstore_get(uint64) uint64 -// kvstore_delete is the host function provided by Navidrome. +// kvstore_getmany is the host function provided by Navidrome. // -//go:wasmimport extism:host/user kvstore_delete -func kvstore_delete(uint64) uint64 +//go:wasmimport extism:host/user kvstore_getmany +func kvstore_getmany(uint64) uint64 // kvstore_has is the host function provided by Navidrome. // @@ -39,6 +44,16 @@ func kvstore_has(uint64) uint64 //go:wasmimport extism:host/user kvstore_list func kvstore_list(uint64) uint64 +// kvstore_delete is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user kvstore_delete +func kvstore_delete(uint64) uint64 + +// kvstore_deletebyprefix is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user kvstore_deletebyprefix +func kvstore_deletebyprefix(uint64) uint64 + // kvstore_getstorageused is the host function provided by Navidrome. // //go:wasmimport extism:host/user kvstore_getstorageused @@ -49,6 +64,12 @@ type kVStoreSetRequest struct { Value []byte `json:"value"` } +type kVStoreSetWithTTLRequest struct { + Key string `json:"key"` + Value []byte `json:"value"` + TtlSeconds int64 `json:"ttlSeconds"` +} + type kVStoreGetRequest struct { Key string `json:"key"` } @@ -59,8 +80,13 @@ type kVStoreGetResponse struct { Error string `json:"error,omitempty"` } -type kVStoreDeleteRequest struct { - Key string `json:"key"` +type kVStoreGetManyRequest struct { + Keys []string `json:"keys"` +} + +type kVStoreGetManyResponse struct { + Values map[string][]byte `json:"values,omitempty"` + Error string `json:"error,omitempty"` } type kVStoreHasRequest struct { @@ -81,6 +107,19 @@ type kVStoreListResponse struct { Error string `json:"error,omitempty"` } +type kVStoreDeleteRequest struct { + Key string `json:"key"` +} + +type kVStoreDeleteByPrefixRequest struct { + Prefix string `json:"prefix"` +} + +type kVStoreDeleteByPrefixResponse struct { + DeletedCount int64 `json:"deletedCount,omitempty"` + Error string `json:"error,omitempty"` +} + type kVStoreGetStorageUsedResponse struct { Bytes int64 `json:"bytes,omitempty"` Error string `json:"error,omitempty"` @@ -127,6 +166,52 @@ func KVStoreSet(key string, value []byte) error { return nil } +// KVStoreSetWithTTL calls the kvstore_setwithttl host function. +// SetWithTTL stores a byte value with the given key and a time-to-live. +// +// After ttlSeconds, the key is treated as non-existent and will be +// cleaned up lazily. ttlSeconds must be greater than 0. +// +// Parameters: +// - key: The storage key (max 256 bytes, UTF-8) +// - value: The byte slice to store +// - ttlSeconds: Time-to-live in seconds (must be > 0) +// +// Returns an error if the storage limit would be exceeded or the operation fails. +func KVStoreSetWithTTL(key string, value []byte, ttlSeconds int64) error { + // Marshal request to JSON + req := kVStoreSetWithTTLRequest{ + Key: key, + Value: value, + TtlSeconds: ttlSeconds, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := kvstore_setwithttl(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse error-only response + var response struct { + Error string `json:"error,omitempty"` + } + if err := json.Unmarshal(responseBytes, &response); err != nil { + return err + } + if response.Error != "" { + return errors.New(response.Error) + } + return nil +} + // KVStoreGet calls the kvstore_get host function. // Get retrieves a byte value from storage. // @@ -167,43 +252,45 @@ func KVStoreGet(key string) ([]byte, bool, error) { return response.Value, response.Exists, nil } -// KVStoreDelete calls the kvstore_delete host function. -// Delete removes a value from storage. +// KVStoreGetMany calls the kvstore_getmany host function. +// GetMany retrieves multiple values in a single call. // // Parameters: -// - key: The storage key +// - keys: The storage keys to retrieve // -// Returns an error if the operation fails. Does not return an error if the key doesn't exist. -func KVStoreDelete(key string) error { +// Returns a map of key to value for keys that exist and have not expired. +// Missing or expired keys are omitted from the result. +func KVStoreGetMany(keys []string) (map[string][]byte, error) { // Marshal request to JSON - req := kVStoreDeleteRequest{ - Key: key, + req := kVStoreGetManyRequest{ + Keys: keys, } reqBytes, err := json.Marshal(req) if err != nil { - return err + return nil, err } reqMem := pdk.AllocateBytes(reqBytes) defer reqMem.Free() // Call the host function - responsePtr := kvstore_delete(reqMem.Offset()) + responsePtr := kvstore_getmany(reqMem.Offset()) // Read the response from memory responseMem := pdk.FindMemory(responsePtr) responseBytes := responseMem.ReadBytes() - // Parse error-only response - var response struct { - Error string `json:"error,omitempty"` - } + // Parse the response + var response kVStoreGetManyResponse if err := json.Unmarshal(responseBytes, &response); err != nil { - return err + return nil, err } + + // Convert Error field to Go error if response.Error != "" { - return errors.New(response.Error) + return nil, errors.New(response.Error) } - return nil + + return response.Values, nil } // KVStoreHas calls the kvstore_has host function. @@ -286,6 +373,85 @@ func KVStoreList(prefix string) ([]string, error) { return response.Keys, nil } +// KVStoreDelete calls the kvstore_delete host function. +// Delete removes a value from storage. +// +// Parameters: +// - key: The storage key +// +// Returns an error if the operation fails. Does not return an error if the key doesn't exist. +func KVStoreDelete(key string) error { + // Marshal request to JSON + req := kVStoreDeleteRequest{ + Key: key, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := kvstore_delete(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse error-only response + var response struct { + Error string `json:"error,omitempty"` + } + if err := json.Unmarshal(responseBytes, &response); err != nil { + return err + } + if response.Error != "" { + return errors.New(response.Error) + } + return nil +} + +// KVStoreDeleteByPrefix calls the kvstore_deletebyprefix host function. +// DeleteByPrefix removes all keys matching the given prefix. +// +// Parameters: +// - prefix: Key prefix to match (must not be empty) +// +// Returns the number of keys deleted. Includes expired keys. +func KVStoreDeleteByPrefix(prefix string) (int64, error) { + // Marshal request to JSON + req := kVStoreDeleteByPrefixRequest{ + Prefix: prefix, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return 0, err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := kvstore_deletebyprefix(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse the response + var response kVStoreDeleteByPrefixResponse + if err := json.Unmarshal(responseBytes, &response); err != nil { + return 0, err + } + + // Convert Error field to Go error + if response.Error != "" { + return 0, errors.New(response.Error) + } + + return response.DeletedCount, nil +} + // KVStoreGetStorageUsed calls the kvstore_getstorageused host function. // GetStorageUsed returns the total storage used by this plugin in bytes. func KVStoreGetStorageUsed() (int64, error) { diff --git a/plugins/pdk/go/host/nd_host_kvstore_stub.go b/plugins/pdk/go/host/nd_host_kvstore_stub.go index 1b3ff1e8d..83b55d3a8 100644 --- a/plugins/pdk/go/host/nd_host_kvstore_stub.go +++ b/plugins/pdk/go/host/nd_host_kvstore_stub.go @@ -37,6 +37,28 @@ func KVStoreSet(key string, value []byte) error { return KVStoreMock.Set(key, value) } +// SetWithTTL is the mock method for KVStoreSetWithTTL. +func (m *mockKVStoreService) SetWithTTL(key string, value []byte, ttlSeconds int64) error { + args := m.Called(key, value, ttlSeconds) + return args.Error(0) +} + +// KVStoreSetWithTTL delegates to the mock instance. +// SetWithTTL stores a byte value with the given key and a time-to-live. +// +// After ttlSeconds, the key is treated as non-existent and will be +// cleaned up lazily. ttlSeconds must be greater than 0. +// +// Parameters: +// - key: The storage key (max 256 bytes, UTF-8) +// - value: The byte slice to store +// - ttlSeconds: Time-to-live in seconds (must be > 0) +// +// Returns an error if the storage limit would be exceeded or the operation fails. +func KVStoreSetWithTTL(key string, value []byte, ttlSeconds int64) error { + return KVStoreMock.SetWithTTL(key, value, ttlSeconds) +} + // Get is the mock method for KVStoreGet. func (m *mockKVStoreService) Get(key string) ([]byte, bool, error) { args := m.Called(key) @@ -54,21 +76,22 @@ func KVStoreGet(key string) ([]byte, bool, error) { return KVStoreMock.Get(key) } -// Delete is the mock method for KVStoreDelete. -func (m *mockKVStoreService) Delete(key string) error { - args := m.Called(key) - return args.Error(0) +// GetMany is the mock method for KVStoreGetMany. +func (m *mockKVStoreService) GetMany(keys []string) (map[string][]byte, error) { + args := m.Called(keys) + return args.Get(0).(map[string][]byte), args.Error(1) } -// KVStoreDelete delegates to the mock instance. -// Delete removes a value from storage. +// KVStoreGetMany delegates to the mock instance. +// GetMany retrieves multiple values in a single call. // // Parameters: -// - key: The storage key +// - keys: The storage keys to retrieve // -// Returns an error if the operation fails. Does not return an error if the key doesn't exist. -func KVStoreDelete(key string) error { - return KVStoreMock.Delete(key) +// Returns a map of key to value for keys that exist and have not expired. +// Missing or expired keys are omitted from the result. +func KVStoreGetMany(keys []string) (map[string][]byte, error) { + return KVStoreMock.GetMany(keys) } // Has is the mock method for KVStoreHas. @@ -105,6 +128,40 @@ func KVStoreList(prefix string) ([]string, error) { return KVStoreMock.List(prefix) } +// Delete is the mock method for KVStoreDelete. +func (m *mockKVStoreService) Delete(key string) error { + args := m.Called(key) + return args.Error(0) +} + +// KVStoreDelete delegates to the mock instance. +// Delete removes a value from storage. +// +// Parameters: +// - key: The storage key +// +// Returns an error if the operation fails. Does not return an error if the key doesn't exist. +func KVStoreDelete(key string) error { + return KVStoreMock.Delete(key) +} + +// DeleteByPrefix is the mock method for KVStoreDeleteByPrefix. +func (m *mockKVStoreService) DeleteByPrefix(prefix string) (int64, error) { + args := m.Called(prefix) + return args.Get(0).(int64), args.Error(1) +} + +// KVStoreDeleteByPrefix delegates to the mock instance. +// DeleteByPrefix removes all keys matching the given prefix. +// +// Parameters: +// - prefix: Key prefix to match (must not be empty) +// +// Returns the number of keys deleted. Includes expired keys. +func KVStoreDeleteByPrefix(prefix string) (int64, error) { + return KVStoreMock.DeleteByPrefix(prefix) +} + // GetStorageUsed is the mock method for KVStoreGetStorageUsed. func (m *mockKVStoreService) GetStorageUsed() (int64, error) { args := m.Called() diff --git a/plugins/pdk/python/host/nd_host_kvstore.py b/plugins/pdk/python/host/nd_host_kvstore.py index 3c3e61f53..33eaffc52 100644 --- a/plugins/pdk/python/host/nd_host_kvstore.py +++ b/plugins/pdk/python/host/nd_host_kvstore.py @@ -26,14 +26,20 @@ def _kvstore_set(offset: int) -> int: ... +@extism.import_fn("extism:host/user", "kvstore_setwithttl") +def _kvstore_setwithttl(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + @extism.import_fn("extism:host/user", "kvstore_get") def _kvstore_get(offset: int) -> int: """Raw host function - do not call directly.""" ... -@extism.import_fn("extism:host/user", "kvstore_delete") -def _kvstore_delete(offset: int) -> int: +@extism.import_fn("extism:host/user", "kvstore_getmany") +def _kvstore_getmany(offset: int) -> int: """Raw host function - do not call directly.""" ... @@ -50,6 +56,18 @@ def _kvstore_list(offset: int) -> int: ... +@extism.import_fn("extism:host/user", "kvstore_delete") +def _kvstore_delete(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +@extism.import_fn("extism:host/user", "kvstore_deletebyprefix") +def _kvstore_deletebyprefix(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + @extism.import_fn("extism:host/user", "kvstore_getstorageused") def _kvstore_getstorageused(offset: int) -> int: """Raw host function - do not call directly.""" @@ -94,6 +112,43 @@ Returns an error if the storage limit would be exceeded or the operation fails. +def kvstore_set_with_ttl(key: str, value: bytes, ttl_seconds: int) -> None: + """SetWithTTL stores a byte value with the given key and a time-to-live. + +After ttlSeconds, the key is treated as non-existent and will be +cleaned up lazily. ttlSeconds must be greater than 0. + +Parameters: + - key: The storage key (max 256 bytes, UTF-8) + - value: The byte slice to store + - ttlSeconds: Time-to-live in seconds (must be > 0) + +Returns an error if the storage limit would be exceeded or the operation fails. + + Args: + key: str parameter. + value: bytes parameter. + ttl_seconds: int parameter. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "key": key, + "value": base64.b64encode(value).decode("ascii"), + "ttlSeconds": ttl_seconds, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _kvstore_setwithttl(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + + def kvstore_get(key: str) -> KVStoreGetResult: """Get retrieves a byte value from storage. @@ -129,32 +184,37 @@ Returns the value and whether the key exists. ) -def kvstore_delete(key: str) -> None: - """Delete removes a value from storage. +def kvstore_get_many(keys: Any) -> Any: + """GetMany retrieves multiple values in a single call. Parameters: - - key: The storage key + - keys: The storage keys to retrieve -Returns an error if the operation fails. Does not return an error if the key doesn't exist. +Returns a map of key to value for keys that exist and have not expired. +Missing or expired keys are omitted from the result. Args: - key: str parameter. + keys: Any parameter. + + Returns: + Any: The result value. Raises: HostFunctionError: If the host function returns an error. """ request = { - "key": key, + "keys": keys, } request_bytes = json.dumps(request).encode("utf-8") request_mem = extism.memory.alloc(request_bytes) - response_offset = _kvstore_delete(request_mem.offset) + response_offset = _kvstore_getmany(request_mem.offset) response_mem = extism.memory.find(response_offset) response = json.loads(extism.memory.string(response_mem)) if response.get("error"): raise HostFunctionError(response["error"]) + return response.get("values", None) def kvstore_has(key: str) -> bool: @@ -221,6 +281,66 @@ Returns a slice of matching keys. return response.get("keys", None) +def kvstore_delete(key: str) -> None: + """Delete removes a value from storage. + +Parameters: + - key: The storage key + +Returns an error if the operation fails. Does not return an error if the key doesn't exist. + + Args: + key: str parameter. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "key": key, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _kvstore_delete(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + + +def kvstore_delete_by_prefix(prefix: str) -> int: + """DeleteByPrefix removes all keys matching the given prefix. + +Parameters: + - prefix: Key prefix to match (must not be empty) + +Returns the number of keys deleted. Includes expired keys. + + Args: + prefix: str parameter. + + Returns: + int: The result value. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "prefix": prefix, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _kvstore_deletebyprefix(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + return response.get("deletedCount", 0) + + def kvstore_get_storage_used() -> int: """GetStorageUsed returns the total storage used by this plugin in bytes. diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs index 20fe18c6f..a85e72895 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_kvstore.rs @@ -44,6 +44,22 @@ struct KVStoreSetResponse { error: Option, } +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct KVStoreSetWithTTLRequest { + key: String, + #[serde(with = "base64_bytes")] + value: Vec, + ttl_seconds: i64, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct KVStoreSetWithTTLResponse { + #[serde(default)] + error: Option, +} + #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] struct KVStoreGetRequest { @@ -64,13 +80,15 @@ struct KVStoreGetResponse { #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] -struct KVStoreDeleteRequest { - key: String, +struct KVStoreGetManyRequest { + keys: Vec, } #[derive(Debug, Clone, Deserialize)] #[serde(rename_all = "camelCase")] -struct KVStoreDeleteResponse { +struct KVStoreGetManyResponse { + #[serde(default)] + values: std::collections::HashMap>, #[serde(default)] error: Option, } @@ -105,6 +123,34 @@ struct KVStoreListResponse { error: Option, } +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct KVStoreDeleteRequest { + key: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct KVStoreDeleteResponse { + #[serde(default)] + error: Option, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct KVStoreDeleteByPrefixRequest { + prefix: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct KVStoreDeleteByPrefixResponse { + #[serde(default)] + deleted_count: i64, + #[serde(default)] + error: Option, +} + #[derive(Debug, Clone, Deserialize)] #[serde(rename_all = "camelCase")] struct KVStoreGetStorageUsedResponse { @@ -117,10 +163,13 @@ struct KVStoreGetStorageUsedResponse { #[host_fn] extern "ExtismHost" { fn kvstore_set(input: Json) -> Json; + fn kvstore_setwithttl(input: Json) -> Json; fn kvstore_get(input: Json) -> Json; - fn kvstore_delete(input: Json) -> Json; + fn kvstore_getmany(input: Json) -> Json; fn kvstore_has(input: Json) -> Json; fn kvstore_list(input: Json) -> Json; + fn kvstore_delete(input: Json) -> Json; + fn kvstore_deletebyprefix(input: Json) -> Json; fn kvstore_getstorageused(input: Json) -> Json; } @@ -153,6 +202,41 @@ pub fn set(key: &str, value: Vec) -> Result<(), Error> { Ok(()) } +/// SetWithTTL stores a byte value with the given key and a time-to-live. +/// +/// After ttlSeconds, the key is treated as non-existent and will be +/// cleaned up lazily. ttlSeconds must be greater than 0. +/// +/// Parameters: +/// - key: The storage key (max 256 bytes, UTF-8) +/// - value: The byte slice to store +/// - ttlSeconds: Time-to-live in seconds (must be > 0) +/// +/// Returns an error if the storage limit would be exceeded or the operation fails. +/// +/// # Arguments +/// * `key` - String parameter. +/// * `value` - Vec parameter. +/// * `ttl_seconds` - i64 parameter. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn set_with_ttl(key: &str, value: Vec, ttl_seconds: i64) -> Result<(), Error> { + let response = unsafe { + kvstore_setwithttl(Json(KVStoreSetWithTTLRequest { + key: key.to_owned(), + value: value, + ttl_seconds: ttl_seconds, + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(()) +} + /// Get retrieves a byte value from storage. /// /// Parameters: @@ -186,22 +270,26 @@ pub fn get(key: &str) -> Result>, Error> { } } -/// Delete removes a value from storage. +/// GetMany retrieves multiple values in a single call. /// /// Parameters: -/// - key: The storage key +/// - keys: The storage keys to retrieve /// -/// Returns an error if the operation fails. Does not return an error if the key doesn't exist. +/// Returns a map of key to value for keys that exist and have not expired. +/// Missing or expired keys are omitted from the result. /// /// # Arguments -/// * `key` - String parameter. +/// * `keys` - Vec parameter. +/// +/// # Returns +/// The values value. /// /// # Errors /// Returns an error if the host function call fails. -pub fn delete(key: &str) -> Result<(), Error> { +pub fn get_many(keys: Vec) -> Result>, Error> { let response = unsafe { - kvstore_delete(Json(KVStoreDeleteRequest { - key: key.to_owned(), + kvstore_getmany(Json(KVStoreGetManyRequest { + keys: keys, }))? }; @@ -209,7 +297,7 @@ pub fn delete(key: &str) -> Result<(), Error> { return Err(Error::msg(err)); } - Ok(()) + Ok(response.0.values) } /// Has checks if a key exists in storage. @@ -270,6 +358,61 @@ pub fn list(prefix: &str) -> Result, Error> { Ok(response.0.keys) } +/// Delete removes a value from storage. +/// +/// Parameters: +/// - key: The storage key +/// +/// Returns an error if the operation fails. Does not return an error if the key doesn't exist. +/// +/// # Arguments +/// * `key` - String parameter. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn delete(key: &str) -> Result<(), Error> { + let response = unsafe { + kvstore_delete(Json(KVStoreDeleteRequest { + key: key.to_owned(), + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(()) +} + +/// DeleteByPrefix removes all keys matching the given prefix. +/// +/// Parameters: +/// - prefix: Key prefix to match (must not be empty) +/// +/// Returns the number of keys deleted. Includes expired keys. +/// +/// # Arguments +/// * `prefix` - String parameter. +/// +/// # Returns +/// The deleted_count value. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn delete_by_prefix(prefix: &str) -> Result { + let response = unsafe { + kvstore_deletebyprefix(Json(KVStoreDeleteByPrefixRequest { + prefix: prefix.to_owned(), + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(response.0.deleted_count) +} + /// GetStorageUsed returns the total storage used by this plugin in bytes. /// /// # Returns diff --git a/plugins/testdata/test-kvstore/main.go b/plugins/testdata/test-kvstore/main.go index 9c289df01..7fba14abe 100644 --- a/plugins/testdata/test-kvstore/main.go +++ b/plugins/testdata/test-kvstore/main.go @@ -9,19 +9,23 @@ import ( // TestKVStoreInput is the input for nd_test_kvstore callback. type TestKVStoreInput struct { - Operation string `json:"operation"` // "set", "get", "delete", "has", "list", "get_storage_used" - Key string `json:"key"` // Storage key - Value []byte `json:"value"` // For set operations - Prefix string `json:"prefix"` // For list operation + Operation string `json:"operation"` // "set", "get", "delete", "has", "list", "get_storage_used", "set_with_ttl", "delete_by_prefix", "get_many" + Key string `json:"key"` // Storage key + Value []byte `json:"value"` // For set operations + Prefix string `json:"prefix"` // For list/delete_by_prefix operations + TTLSeconds int64 `json:"ttl_seconds,omitempty"` // For set_with_ttl + Keys []string `json:"keys,omitempty"` // For get_many } // TestKVStoreOutput is the output from nd_test_kvstore callback. type TestKVStoreOutput struct { - Value []byte `json:"value,omitempty"` - Exists bool `json:"exists,omitempty"` - Keys []string `json:"keys,omitempty"` - StorageUsed int64 `json:"storage_used,omitempty"` - Error *string `json:"error,omitempty"` + Value []byte `json:"value,omitempty"` + Values map[string][]byte `json:"values,omitempty"` + Exists bool `json:"exists,omitempty"` + Keys []string `json:"keys,omitempty"` + StorageUsed int64 `json:"storage_used,omitempty"` + DeletedCount int64 `json:"deleted_count,omitempty"` + Error *string `json:"error,omitempty"` } // nd_test_kvstore is the test callback that tests the kvstore host functions. @@ -96,6 +100,36 @@ func ndTestKVStore() int32 { pdk.OutputJSON(TestKVStoreOutput{StorageUsed: bytesUsed}) return 0 + case "set_with_ttl": + err := host.KVStoreSetWithTTL(input.Key, input.Value, input.TTLSeconds) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestKVStoreOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestKVStoreOutput{}) + return 0 + + case "delete_by_prefix": + deletedCount, err := host.KVStoreDeleteByPrefix(input.Prefix) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestKVStoreOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestKVStoreOutput{DeletedCount: deletedCount}) + return 0 + + case "get_many": + values, err := host.KVStoreGetMany(input.Keys) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestKVStoreOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestKVStoreOutput{Values: values}) + return 0 + default: errStr := "unknown operation: " + input.Operation pdk.OutputJSON(TestKVStoreOutput{Error: &errStr}) From 3476be01f71b36a17d122964a0275647a87e30ee Mon Sep 17 00:00:00 2001 From: Deluan Date: Sun, 1 Mar 2026 10:50:24 -0500 Subject: [PATCH 22/60] fix(scanner): handle nil mainCtx in Watcher to prevent panic Signed-off-by: Deluan --- scanner/watcher.go | 6 ++++++ scanner/watcher_test.go | 9 +++++++++ 2 files changed, 15 insertions(+) diff --git a/scanner/watcher.go b/scanner/watcher.go index 3efebaacc..101e3793a 100644 --- a/scanner/watcher.go +++ b/scanner/watcher.go @@ -145,6 +145,12 @@ func (w *watcher) Watch(ctx context.Context, lib *model.Library) error { w.mu.Lock() defer w.mu.Unlock() + // If Run() hasn't been called yet, mainCtx will be nil - skip watching + if w.mainCtx == nil { + log.Debug(ctx, "Watcher not started yet, skipping watch for library", "libraryID", lib.ID, "name", lib.Name) + return nil + } + // Stop existing watcher if any if existingInstance, exists := w.libraryWatchers[lib.ID]; exists { log.Debug(ctx, "Stopping existing watcher before starting new one", "libraryID", lib.ID, "name", lib.Name) diff --git a/scanner/watcher_test.go b/scanner/watcher_test.go index 01bfb2491..7a431d5a0 100644 --- a/scanner/watcher_test.go +++ b/scanner/watcher_test.go @@ -54,6 +54,15 @@ var _ = Describe("Watcher", func() { } }) + Describe("Watch before Run", func() { + It("returns nil and does not panic when mainCtx is nil", func() { + w.mainCtx = nil + err := w.Watch(ctx, lib) + Expect(err).ToNot(HaveOccurred()) + Expect(w.libraryWatchers).To(BeEmpty()) + }) + }) + Describe("Target Collection and Deduplication", func() { BeforeEach(func() { // Start watcher in background From 4e34d3ac1fa69f0875aabd4e7780974882ef1ed6 Mon Sep 17 00:00:00 2001 From: Deluan Date: Sun, 1 Mar 2026 10:50:37 -0500 Subject: [PATCH 23/60] feat(ui): conditionally display 'path' field in LibraryList for desktop view Signed-off-by: Deluan --- ui/src/library/LibraryList.jsx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/ui/src/library/LibraryList.jsx b/ui/src/library/LibraryList.jsx index aa1294882..35d627cbb 100644 --- a/ui/src/library/LibraryList.jsx +++ b/ui/src/library/LibraryList.jsx @@ -21,6 +21,7 @@ const LibraryFilter = (props) => ( const LibraryList = (props) => { const isXsmall = useMediaQuery((theme) => theme.breakpoints.down('xs')) + const isDesktop = useMediaQuery((theme) => theme.breakpoints.up('lg')) useResourceRefresh('library') return ( @@ -40,7 +41,7 @@ const LibraryList = (props) => { ) : ( - + {isDesktop && } From d004f99f8f2ab3f753e3847bdf28548f3a5a3753 Mon Sep 17 00:00:00 2001 From: adrbn <128328324+adrbn@users.noreply.github.com> Date: Sun, 1 Mar 2026 20:07:18 +0100 Subject: [PATCH 24/60] feat(playlist): add custom playlist cover art upload (#5110) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(playlist): add custom playlist cover art upload - #406 Allow users to upload, view, and remove custom cover images for playlists. Custom images take priority over the auto-generated tiled artwork. Backend: - Add `image_path` column to playlist table (migration with proper rollback) - Add `SetImage`/`RemoveImage` methods to playlist service - Add `POST/DELETE /api/playlist/{id}/image` endpoints - Prioritize custom image in artwork reader pipeline - Clean up image files on playlist deletion - Use glob-based cleanup to prevent orphaned files across format changes - Reject uploads with undetermined image type (400) Frontend: - Hover overlay on playlist cover with upload (camera) and remove (trash) buttons - Lightbox for full-size cover art viewing - Cover art thumbnails in the playlist list view - Loading/error states and i18n strings Closes #406 Co-Authored-By: Claude Opus 4.6 Signed-off-by: adrbn <128328324+adrbn@users.noreply.github.com> * refactor: rename playlist image path migration file Signed-off-by: Deluan * fix(playlist): address review feedback for cover art upload - #406 - Use httpClient instead of raw fetch for image upload/remove - Revert glob cleanup to simple imagePath check - Add log.Error before all error HTTP responses - Add backend tests for SetImage and RemoveImage Co-Authored-By: Claude Opus 4.6 Signed-off-by: adrbn <128328324+adrbn@users.noreply.github.com> * refactor(playlist): use Playlist.ArtworkPath() for image storage Migrate all playlist image path handling to use the new Playlist.ArtworkPath() method as the single source of truth. The DB now stores only the filename (e.g. "pls-1.jpg") instead of a relative path, and images are stored under {DataFolder}/artwork/playlist/ instead of {DataFolder}/playlist_images/. The artwork root directory is created at startup alongside DataFolder and CacheFolder. This also removes the conf dependency from reader_playlist.go since path resolution is now fully encapsulated in the model. Signed-off-by: Deluan * refactor(playlist): streamline artwork image selection logic Signed-off-by: Deluan * refactor: move translation keys, add pt-BR translations Signed-off-by: Deluan * refactor(playlist): rename image_path to image_file Rename the playlist cover art column and field from image_path/ImagePath to image_file/ImageFile across the migration, model, service, tests, and UI. The new name more accurately describes what the field stores (a filename, not a path) and aligns with the existing ImageFiles/IsImageFile naming conventions in the codebase. --------- Signed-off-by: adrbn <128328324+adrbn@users.noreply.github.com> Signed-off-by: Deluan Co-authored-by: Claude Opus 4.6 Co-authored-by: Deluan Quintão --- conf/configuration.go | 6 + consts/consts.go | 1 + core/artwork/reader_playlist.go | 19 ++- core/playlists/playlists.go | 72 +++++++++- core/playlists/playlists_test.go | 120 ++++++++++++++++ .../20260228172956_add_playlist_image_file.go | 22 +++ model/playlist.go | 11 ++ resources/i18n/pt-br.json | 8 +- server/nativeapi/native_api.go | 2 + server/nativeapi/playlists.go | 105 ++++++++++++++ ui/src/i18n/en.json | 8 +- ui/src/playlist/PlaylistDetails.jsx | 136 +++++++++++++++++- ui/src/playlist/PlaylistList.jsx | 27 ++++ 13 files changed, 529 insertions(+), 8 deletions(-) create mode 100644 db/migrations/20260228172956_add_playlist_image_file.go diff --git a/conf/configuration.go b/conf/configuration.go index 61448d315..3e3b42355 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -303,6 +303,12 @@ func Load(noConfigDump bool) { os.Exit(1) } + err = os.MkdirAll(filepath.Join(Server.DataFolder, consts.ArtworkFolder), os.ModePerm) + if err != nil { + _, _ = fmt.Fprintln(os.Stderr, "FATAL: Error creating artwork path:", err) + os.Exit(1) + } + if Server.Plugins.Enabled { if Server.Plugins.Folder == "" { Server.Plugins.Folder = filepath.Join(Server.DataFolder, "plugins") diff --git a/consts/consts.go b/consts/consts.go index ebde9d1d9..295abe8a9 100644 --- a/consts/consts.go +++ b/consts/consts.go @@ -65,6 +65,7 @@ const ( I18nFolder = "i18n" ScanIgnoreFile = ".ndignore" + ArtworkFolder = "artwork" PlaceholderArtistArt = "artist-placeholder.webp" PlaceholderAlbumArt = "album-placeholder.webp" diff --git a/core/artwork/reader_playlist.go b/core/artwork/reader_playlist.go index a9f289ad8..09bfe221b 100644 --- a/core/artwork/reader_playlist.go +++ b/core/artwork/reader_playlist.go @@ -8,6 +8,7 @@ import ( "image/draw" "image/png" "io" + "os" "time" "github.com/disintegration/imaging" @@ -43,11 +44,25 @@ func (a *playlistArtworkReader) LastUpdated() time.Time { } func (a *playlistArtworkReader) Reader(ctx context.Context) (io.ReadCloser, string, error) { - ff := []sourceFunc{ + return selectImageReader(ctx, a.artID, + a.fromPlaylistImage(), a.fromGeneratedTiledCover(ctx), fromAlbumPlaceholder(), + ) +} + +func (a *playlistArtworkReader) fromPlaylistImage() sourceFunc { + return func() (io.ReadCloser, string, error) { + absPath := a.pl.ArtworkPath() + if absPath == "" { + return nil, "", nil + } + f, err := os.Open(absPath) + if err != nil { + return nil, "", err + } + return f, absPath, nil } - return selectImageReader(ctx, a.artID, ff...) } func (a *playlistArtworkReader) fromGeneratedTiledCover(ctx context.Context) sourceFunc { diff --git a/core/playlists/playlists.go b/core/playlists/playlists.go index 5a9a908ee..7e881b730 100644 --- a/core/playlists/playlists.go +++ b/core/playlists/playlists.go @@ -2,7 +2,9 @@ package playlists import ( "context" + "fmt" "io" + "os" "path/filepath" "strconv" "strings" @@ -10,6 +12,7 @@ import ( "github.com/bmatcuk/doublestar/v4" "github.com/deluan/rest" "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/request" ) @@ -34,6 +37,10 @@ type Playlists interface { RemoveTracks(ctx context.Context, playlistID string, trackIds []string) error ReorderTrack(ctx context.Context, playlistID string, pos int, newPos int) error + // Cover art + SetImage(ctx context.Context, playlistID string, reader io.Reader, ext string) error + RemoveImage(ctx context.Context, playlistID string) error + // Import ImportFile(ctx context.Context, folder *model.Folder, filename string) (*model.Playlist, error) ImportM3U(ctx context.Context, reader io.Reader) (*model.Playlist, error) @@ -118,9 +125,18 @@ func (s *playlists) Create(ctx context.Context, playlistId string, name string, } func (s *playlists) Delete(ctx context.Context, id string) error { - if _, err := s.checkWritable(ctx, id); err != nil { + pls, err := s.checkWritable(ctx, id) + if err != nil { return err } + + // Clean up custom cover image file if one exists + if path := pls.ArtworkPath(); path != "" { + if err := os.Remove(path); err != nil && !os.IsNotExist(err) { + log.Warn(ctx, "Failed to remove playlist image on delete", "path", path, err) + } + } + return s.ds.Playlist(ctx).Delete(id) } @@ -263,3 +279,57 @@ func (s *playlists) ReorderTrack(ctx context.Context, playlistID string, pos int return tx.Playlist(ctx).Tracks(playlistID, false).Reorder(pos, newPos) }) } + +// --- Cover art operations --- + +func (s *playlists) SetImage(ctx context.Context, playlistID string, reader io.Reader, ext string) error { + pls, err := s.checkWritable(ctx, playlistID) + if err != nil { + return err + } + + filename := playlistID + ext + oldPath := pls.ArtworkPath() + pls.ImageFile = filename + absPath := pls.ArtworkPath() + + if err := os.MkdirAll(filepath.Dir(absPath), 0755); err != nil { + return fmt.Errorf("creating playlist images directory: %w", err) + } + + // Remove old image if it exists + if oldPath != "" { + if err := os.Remove(oldPath); err != nil && !os.IsNotExist(err) { + log.Warn(ctx, "Failed to remove old playlist image", "path", oldPath, err) + } + } + + // Save new image + f, err := os.Create(absPath) + if err != nil { + return fmt.Errorf("creating playlist image file: %w", err) + } + defer f.Close() + + if _, err := io.Copy(f, reader); err != nil { + return fmt.Errorf("writing playlist image file: %w", err) + } + + return s.ds.Playlist(ctx).Put(pls) +} + +func (s *playlists) RemoveImage(ctx context.Context, playlistID string) error { + pls, err := s.checkWritable(ctx, playlistID) + if err != nil { + return err + } + + if path := pls.ArtworkPath(); path != "" { + if err := os.Remove(path); err != nil && !os.IsNotExist(err) { + log.Warn(ctx, "Failed to remove playlist image", "path", path, err) + } + } + + pls.ImageFile = "" + return s.ds.Playlist(ctx).Put(pls) +} diff --git a/core/playlists/playlists_test.go b/core/playlists/playlists_test.go index a4c309d77..4d42bbeb9 100644 --- a/core/playlists/playlists_test.go +++ b/core/playlists/playlists_test.go @@ -2,7 +2,12 @@ package playlists_test import ( "context" + "os" + "path/filepath" + "strings" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/criteria" @@ -294,4 +299,119 @@ var _ = Describe("Playlists", func() { Expect(err).To(MatchError(model.ErrNotAuthorized)) }) }) + + Describe("SetImage", func() { + var tmpDir string + + BeforeEach(func() { + DeferCleanup(configtest.SetupConfig()) + tmpDir = GinkgoT().TempDir() + conf.Server.DataFolder = tmpDir + + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1"}, + "pls-other": {ID: "pls-other", Name: "Other's", OwnerID: "other-user"}, + } + ps = playlists.NewPlaylists(ds) + }) + + It("saves image file and updates ImageFile", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + reader := strings.NewReader("fake image data") + err := ps.SetImage(ctx, "pls-1", reader, ".jpg") + Expect(err).ToNot(HaveOccurred()) + + Expect(mockPlsRepo.Last.ImageFile).To(Equal("pls-1.jpg")) + absPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.jpg") + data, err := os.ReadFile(absPath) + Expect(err).ToNot(HaveOccurred()) + Expect(string(data)).To(Equal("fake image data")) + }) + + It("removes old image when replacing", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + + // Upload first image + err := ps.SetImage(ctx, "pls-1", strings.NewReader("first"), ".png") + Expect(err).ToNot(HaveOccurred()) + oldPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.png") + Expect(oldPath).To(BeAnExistingFile()) + + // Upload replacement image + err = ps.SetImage(ctx, "pls-1", strings.NewReader("second"), ".jpg") + Expect(err).ToNot(HaveOccurred()) + Expect(oldPath).ToNot(BeAnExistingFile()) + newPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.jpg") + Expect(newPath).To(BeAnExistingFile()) + }) + + It("allows admin to set image on any playlist", func() { + ctx = request.WithUser(ctx, model.User{ID: "admin-1", IsAdmin: true}) + err := ps.SetImage(ctx, "pls-other", strings.NewReader("data"), ".jpg") + Expect(err).ToNot(HaveOccurred()) + }) + + It("denies non-owner", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + err := ps.SetImage(ctx, "pls-1", strings.NewReader("data"), ".jpg") + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("returns error when playlist not found", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.SetImage(ctx, "nonexistent", strings.NewReader("data"), ".jpg") + Expect(err).To(Equal(model.ErrNotFound)) + }) + }) + + Describe("RemoveImage", func() { + var tmpDir string + + BeforeEach(func() { + DeferCleanup(configtest.SetupConfig()) + tmpDir = GinkgoT().TempDir() + conf.Server.DataFolder = tmpDir + + // Create a real image file on disk + imgDir := filepath.Join(tmpDir, "artwork", "playlist") + Expect(os.MkdirAll(imgDir, 0755)).To(Succeed()) + Expect(os.WriteFile(filepath.Join(imgDir, "pls-1.jpg"), []byte("img data"), 0600)).To(Succeed()) + + mockPlsRepo.Data = map[string]*model.Playlist{ + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1", ImageFile: "pls-1.jpg"}, + "pls-empty": {ID: "pls-empty", Name: "No Cover", OwnerID: "user-1"}, + "pls-other": {ID: "pls-other", Name: "Other's", OwnerID: "other-user"}, + } + ps = playlists.NewPlaylists(ds) + }) + + It("removes file and clears ImageFile", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.RemoveImage(ctx, "pls-1") + Expect(err).ToNot(HaveOccurred()) + + Expect(mockPlsRepo.Last.ImageFile).To(BeEmpty()) + absPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.jpg") + Expect(absPath).ToNot(BeAnExistingFile()) + }) + + It("succeeds even if playlist has no image", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.RemoveImage(ctx, "pls-empty") + Expect(err).ToNot(HaveOccurred()) + Expect(mockPlsRepo.Last.ImageFile).To(BeEmpty()) + }) + + It("denies non-owner", func() { + ctx = request.WithUser(ctx, model.User{ID: "other-user", IsAdmin: false}) + err := ps.RemoveImage(ctx, "pls-1") + Expect(err).To(MatchError(model.ErrNotAuthorized)) + }) + + It("returns error when playlist not found", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + err := ps.RemoveImage(ctx, "nonexistent") + Expect(err).To(Equal(model.ErrNotFound)) + }) + }) }) diff --git a/db/migrations/20260228172956_add_playlist_image_file.go b/db/migrations/20260228172956_add_playlist_image_file.go new file mode 100644 index 000000000..da2177aba --- /dev/null +++ b/db/migrations/20260228172956_add_playlist_image_file.go @@ -0,0 +1,22 @@ +package migrations + +import ( + "context" + "database/sql" + + "github.com/pressly/goose/v3" +) + +func init() { + goose.AddMigrationContext(upAddPlaylistImageFile, downAddPlaylistImageFile) +} + +func upAddPlaylistImageFile(ctx context.Context, tx *sql.Tx) error { + _, err := tx.ExecContext(ctx, `ALTER TABLE playlist ADD COLUMN image_file VARCHAR(255) DEFAULT '';`) + return err +} + +func downAddPlaylistImageFile(ctx context.Context, tx *sql.Tx) error { + _, err := tx.ExecContext(ctx, `ALTER TABLE playlist DROP COLUMN image_file;`) + return err +} diff --git a/model/playlist.go b/model/playlist.go index a87019ed5..b6a52dcf2 100644 --- a/model/playlist.go +++ b/model/playlist.go @@ -1,10 +1,13 @@ package model import ( + "path/filepath" "slices" "strconv" "time" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/model/criteria" ) @@ -21,6 +24,7 @@ type Playlist struct { Tracks PlaylistTracks `structs:"-" json:"tracks,omitempty"` Path string `structs:"path" json:"path"` Sync bool `structs:"sync" json:"sync"` + ImageFile string `structs:"image_file" json:"imageFile"` CreatedAt time.Time `structs:"created_at" json:"createdAt"` UpdatedAt time.Time `structs:"updated_at" json:"updatedAt"` @@ -106,6 +110,13 @@ func (pls Playlist) CoverArtID() ArtworkID { return artworkIDFromPlaylist(pls) } +func (pls Playlist) ArtworkPath() string { + if pls.ImageFile == "" { + return "" + } + return filepath.Join(conf.Server.DataFolder, consts.ArtworkFolder, "playlist", pls.ImageFile) +} + type Playlists []Playlist type PlaylistRepository interface { diff --git a/resources/i18n/pt-br.json b/resources/i18n/pt-br.json index 1fc72bf30..6c9e154d1 100644 --- a/resources/i18n/pt-br.json +++ b/resources/i18n/pt-br.json @@ -218,9 +218,15 @@ "saveQueue": "Salvar fila em nova Playlist", "searchOrCreate": "Buscar playlists ou criar nova...", "pressEnterToCreate": "Pressione Enter para criar nova playlist", - "removeFromSelection": "Remover da seleção" + "removeFromSelection": "Remover da seleção", + "uploadCover": "Enviar Capa", + "removeCover": "Remover Capa" }, "message": { + "coverUploaded": "Capa atualizada", + "coverRemoved": "Capa removida", + "coverUploadError": "Erro ao enviar capa", + "coverRemoveError": "Erro ao remover capa", "duplicate_song": "Adicionar músicas duplicadas", "song_exist": "Algumas destas músicas já existem na playlist. Você quer adicionar as duplicadas ou ignorá-las?", "noPlaylistsFound": "Nenhuma playlist encontrada", diff --git a/server/nativeapi/native_api.go b/server/nativeapi/native_api.go index 062cbf706..3191991eb 100644 --- a/server/nativeapi/native_api.go +++ b/server/nativeapi/native_api.go @@ -140,6 +140,8 @@ func (api *Router) addPlaylistRoute(r chi.Router) { r.Get("/", rest.Get(constructor)) r.Put("/", rest.Put(constructor)) r.Delete("/", rest.Delete(constructor)) + r.Post("/image", uploadPlaylistImage(api.playlists)) + r.Delete("/image", deletePlaylistImage(api.playlists)) }) }) } diff --git a/server/nativeapi/playlists.go b/server/nativeapi/playlists.go index 60e8024bd..c7230a209 100644 --- a/server/nativeapi/playlists.go +++ b/server/nativeapi/playlists.go @@ -5,7 +5,13 @@ import ( "encoding/json" "errors" "fmt" + "image" + _ "image/gif" + _ "image/jpeg" + _ "image/png" + "io" "net/http" + "path/filepath" "strconv" "strings" @@ -15,6 +21,7 @@ import ( "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/req" + _ "golang.org/x/image/webp" ) type restHandler = func(rest.RepositoryConstructor, ...rest.Logger) http.HandlerFunc @@ -224,3 +231,101 @@ func getSongPlaylists(svc playlists.Playlists) http.HandlerFunc { _, _ = w.Write(data) //nolint:gosec } } + +const maxImageSize = 10 << 20 // 10MB + +func uploadPlaylistImage(pls playlists.Playlists) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + ctx := r.Context() + p := req.Params(r) + playlistId, _ := p.String(":id") + + if err := r.ParseMultipartForm(maxImageSize); err != nil { + log.Error(ctx, "Error parsing multipart form", err) + http.Error(w, "file too large or invalid form", http.StatusBadRequest) + return + } + + file, header, err := r.FormFile("image") + if err != nil { + log.Error(ctx, "Error reading uploaded file", err) + http.Error(w, "missing image file", http.StatusBadRequest) + return + } + defer file.Close() + + // Validate the uploaded file is a valid image + _, format, err := image.DecodeConfig(file) + if err != nil { + log.Error(ctx, "Uploaded file is not a valid image", err) + http.Error(w, "invalid image file", http.StatusBadRequest) + return + } + + // Reset reader after DecodeConfig consumed some bytes + if seeker, ok := file.(io.Seeker); ok { + if _, err := seeker.Seek(0, io.SeekStart); err != nil { + log.Error(ctx, "Error seeking file", err) + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + } + + // Determine file extension from decoded format or original filename + ext := "." + format + if ext == "." { + ext = strings.ToLower(filepath.Ext(header.Filename)) + } + if ext == "" || ext == "." { + log.Error(ctx, "Could not determine image type", "playlistId", playlistId, "filename", header.Filename) + http.Error(w, "could not determine image type", http.StatusBadRequest) + return + } + + err = pls.SetImage(ctx, playlistId, file, ext) + if errors.Is(err, model.ErrNotAuthorized) { + log.Error(ctx, "Not authorized to upload playlist image", "playlistId", playlistId, err) + http.Error(w, "not authorized", http.StatusForbidden) + return + } + if errors.Is(err, model.ErrNotFound) { + log.Error(ctx, "Playlist not found for image upload", "playlistId", playlistId, err) + http.Error(w, "not found", http.StatusNotFound) + return + } + if err != nil { + log.Error(ctx, "Error saving playlist image", "playlistId", playlistId, err) + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + + _, _ = fmt.Fprintf(w, `{"status":"ok"}`) //nolint:gosec + } +} + +func deletePlaylistImage(pls playlists.Playlists) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + ctx := r.Context() + p := req.Params(r) + playlistId, _ := p.String(":id") + + err := pls.RemoveImage(ctx, playlistId) + if errors.Is(err, model.ErrNotAuthorized) { + log.Error(ctx, "Not authorized to remove playlist image", "playlistId", playlistId, err) + http.Error(w, "not authorized", http.StatusForbidden) + return + } + if errors.Is(err, model.ErrNotFound) { + log.Error(ctx, "Playlist not found for image removal", "playlistId", playlistId, err) + http.Error(w, "not found", http.StatusNotFound) + return + } + if err != nil { + log.Error(ctx, "Error removing playlist image", "playlistId", playlistId, err) + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + + _, _ = fmt.Fprintf(w, `{"status":"ok"}`) //nolint:gosec + } +} diff --git a/ui/src/i18n/en.json b/ui/src/i18n/en.json index 224b1c437..7c4b8ff09 100644 --- a/ui/src/i18n/en.json +++ b/ui/src/i18n/en.json @@ -218,9 +218,15 @@ "makePrivate": "Make Private", "searchOrCreate": "Search playlists or type to create new...", "pressEnterToCreate": "Press Enter to create new playlist", - "removeFromSelection": "Remove from selection" + "removeFromSelection": "Remove from selection", + "uploadCover": "Upload Cover", + "removeCover": "Remove Cover" }, "message": { + "coverUploaded": "Cover art updated", + "coverRemoved": "Cover art removed", + "coverUploadError": "Error uploading cover art", + "coverRemoveError": "Error removing cover art", "duplicate_song": "Add duplicated songs", "song_exist": "There are duplicates being added to the playlist. Would you like to add the duplicates or skip them?", "noPlaylistsFound": "No playlists found", diff --git a/ui/src/playlist/PlaylistDetails.jsx b/ui/src/playlist/PlaylistDetails.jsx index acccb15f7..4c242dd2a 100644 --- a/ui/src/playlist/PlaylistDetails.jsx +++ b/ui/src/playlist/PlaylistDetails.jsx @@ -2,16 +2,27 @@ import { Card, CardContent, CardMedia, + IconButton, + Tooltip, Typography, useMediaQuery, } from '@material-ui/core' import { makeStyles } from '@material-ui/core/styles' -import { useTranslate } from 'react-admin' -import { useCallback, useState, useEffect } from 'react' +import PhotoCameraIcon from '@material-ui/icons/PhotoCamera' +import DeleteIcon from '@material-ui/icons/Delete' +import { useTranslate, useNotify, useRefresh } from 'react-admin' +import { useCallback, useRef, useState, useEffect } from 'react' import Lightbox from 'react-image-lightbox' import 'react-image-lightbox/style.css' -import { CollapsibleComment, DurationField, SizeField } from '../common' +import { + CollapsibleComment, + DurationField, + SizeField, + isWritable, +} from '../common' import subsonic from '../subsonic' +import { REST_URL } from '../consts' +import { httpClient } from '../dataProvider' const useStyles = makeStyles( (theme) => ({ @@ -55,6 +66,7 @@ const useStyles = makeStyles( display: 'flex', alignItems: 'center', justifyContent: 'center', + position: 'relative', }, cover: { objectFit: 'contain', @@ -68,6 +80,31 @@ const useStyles = makeStyles( coverLoading: { opacity: 0.5, }, + coverOverlay: { + position: 'absolute', + bottom: 0, + right: 0, + display: 'flex', + gap: '2px', + padding: '2px', + backgroundColor: 'rgba(0,0,0,0.5)', + borderRadius: '4px 0 0 0', + opacity: 0, + transition: 'opacity 0.2s ease-in-out', + '$coverParent:hover &': { + opacity: 1, + }, + }, + overlayButton: { + color: '#fff', + padding: '4px', + '&:hover': { + backgroundColor: 'rgba(255,255,255,0.2)', + }, + }, + overlayIcon: { + fontSize: '1.2rem', + }, title: { overflow: 'hidden', textOverflow: 'ellipsis', @@ -86,14 +123,18 @@ const useStyles = makeStyles( const PlaylistDetails = (props) => { const { record = {} } = props const translate = useTranslate() + const notify = useNotify() + const refresh = useRefresh() const classes = useStyles() const isDesktop = useMediaQuery((theme) => theme.breakpoints.up('lg')) const [isLightboxOpen, setLightboxOpen] = useState(false) const [imageLoading, setImageLoading] = useState(false) const [imageError, setImageError] = useState(false) + const fileInputRef = useRef(null) const imageUrl = subsonic.getCoverArtUrl(record, 300, true) const fullImageUrl = subsonic.getCoverArtUrl(record) + const canEdit = isWritable(record.ownerId) // Reset image state when playlist changes useEffect(() => { @@ -119,6 +160,60 @@ const PlaylistDetails = (props) => { const handleCloseLightbox = useCallback(() => setLightboxOpen(false), []) + const handleUploadClick = useCallback( + (e) => { + e.stopPropagation() + if (fileInputRef.current) { + fileInputRef.current.click() + } + }, + [fileInputRef], + ) + + const handleFileChange = useCallback( + async (e) => { + const file = e.target.files[0] + if (!file || !record.id) return + + const formData = new FormData() + formData.append('image', file) + + try { + await httpClient(`${REST_URL}/playlist/${record.id}/image`, { + method: 'POST', + headers: new Headers({}), + body: formData, + }) + notify('resources.playlist.message.coverUploaded', 'success') + refresh() + } catch (err) { + notify('resources.playlist.message.coverUploadError', 'warning') + } + + // Reset file input so the same file can be re-selected + e.target.value = '' + }, + [record.id, notify, refresh], + ) + + const handleRemoveCover = useCallback( + async (e) => { + e.stopPropagation() + if (!record.id) return + + try { + await httpClient(`${REST_URL}/playlist/${record.id}/image`, { + method: 'DELETE', + }) + notify('resources.playlist.message.coverRemoved', 'success') + refresh() + } catch (err) { + notify('resources.playlist.message.coverRemoveError', 'warning') + } + }, + [record.id, notify, refresh], + ) + return (
@@ -138,6 +233,41 @@ const PlaylistDetails = (props) => { cursor: imageError ? 'default' : 'pointer', }} /> + {canEdit && ( +
+ + + + + + {record.imageFile && ( + + + + + + )} + +
+ )}
diff --git a/ui/src/playlist/PlaylistList.jsx b/ui/src/playlist/PlaylistList.jsx index eae9d863f..4ec2d5ca1 100644 --- a/ui/src/playlist/PlaylistList.jsx +++ b/ui/src/playlist/PlaylistList.jsx @@ -16,6 +16,7 @@ import { usePermissions, } from 'react-admin' import Switch from '@material-ui/core/Switch' +import { Avatar } from '@material-ui/core' import { makeStyles } from '@material-ui/core/styles' import { useMediaQuery } from '@material-ui/core' import { @@ -28,11 +29,17 @@ import { } from '../common' import PlaylistListActions from './PlaylistListActions' import ChangePublicStatusButton from './ChangePublicStatusButton' +import subsonic from '../subsonic' const useStyles = makeStyles((theme) => ({ button: { color: theme.palette.type === 'dark' ? 'white' : undefined, }, + coverArt: { + width: '40px', + height: '40px', + borderRadius: '4px', + }, })) const PlaylistFilter = (props) => { @@ -119,6 +126,25 @@ const ToggleAutoImport = ({ resource, source }) => { ) : null } +const CoverArtField = () => { + const classes = useStyles() + const record = useRecordContext() + if (!record) return null + return ( + + ) +} + +CoverArtField.defaultProps = { + label: '', + sortable: false, +} + const PlaylistListBulkActions = (props) => { const classes = useStyles() return ( @@ -176,6 +202,7 @@ const PlaylistList = (props) => { bulkActionButtons={!isXsmall && } > isWritable(r?.ownerId)}> + {columns} From 27a83547f71f643e1c5bd087028ac0b68e522610 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 2 Mar 2026 08:56:56 -0500 Subject: [PATCH 25/60] fix(plugins): clear plugin errors on startup to allow retrying Plugins that entered an error state (e.g., incompatible with the Navidrome version) would remain in that state across restarts, blocking the user from retrying. This adds a ClearErrors method to PluginRepository that resets the last_error field on all plugins, and calls it during plugin manager startup before syncing and loading. Signed-off-by: Deluan --- model/plugin.go | 1 + persistence/plugin_repository.go | 8 ++++++++ persistence/plugin_repository_test.go | 24 ++++++++++++++++++++++++ plugins/manager.go | 6 ++++++ tests/mock_plugin_repo.go | 14 ++++++++++++++ 5 files changed, 53 insertions(+) diff --git a/model/plugin.go b/model/plugin.go index f4bad6783..18d66e305 100644 --- a/model/plugin.go +++ b/model/plugin.go @@ -23,6 +23,7 @@ type Plugins []Plugin type PluginRepository interface { ResourceRepository + ClearErrors() error CountAll(options ...QueryOptions) (int64, error) Delete(id string) error Get(id string) (*Plugin, error) diff --git a/persistence/plugin_repository.go b/persistence/plugin_repository.go index 466abb40b..35c32de91 100644 --- a/persistence/plugin_repository.go +++ b/persistence/plugin_repository.go @@ -31,6 +31,14 @@ func (r *pluginRepository) isPermitted() bool { return user.IsAdmin } +func (r *pluginRepository) ClearErrors() error { + if !r.isPermitted() { + return rest.ErrPermissionDenied + } + _, err := r.db.NewQuery("UPDATE plugin SET last_error = '' WHERE last_error != ''").Execute() + return err +} + func (r *pluginRepository) CountAll(options ...model.QueryOptions) (int64, error) { if !r.isPermitted() { return 0, rest.ErrPermissionDenied diff --git a/persistence/plugin_repository_test.go b/persistence/plugin_repository_test.go index ee158a31c..dc68b0892 100644 --- a/persistence/plugin_repository_test.go +++ b/persistence/plugin_repository_test.go @@ -175,6 +175,30 @@ var _ = Describe("PluginRepository", func() { Expect(err.Error()).To(ContainSubstring("ID cannot be empty")) }) }) + + Describe("ClearErrors", func() { + It("clears last_error on all plugins with errors", func() { + _ = repo.Put(&model.Plugin{ID: "ok-plugin", Path: "/plugins/ok.wasm", Manifest: "{}", SHA256: "h1"}) + _ = repo.Put(&model.Plugin{ID: "err-plugin-1", Path: "/plugins/e1.wasm", Manifest: "{}", SHA256: "h2", LastError: "incompatible version"}) + _ = repo.Put(&model.Plugin{ID: "err-plugin-2", Path: "/plugins/e2.wasm", Manifest: "{}", SHA256: "h3", LastError: "missing export"}) + + err := repo.ClearErrors() + Expect(err).To(BeNil()) + + all, err := repo.GetAll() + Expect(err).To(BeNil()) + for _, p := range all { + Expect(p.LastError).To(BeEmpty(), "plugin %s should have no error", p.ID) + } + }) + + It("succeeds when no plugins have errors", func() { + _ = repo.Put(&model.Plugin{ID: "clean-plugin", Path: "/plugins/c.wasm", Manifest: "{}", SHA256: "h1"}) + + err := repo.ClearErrors() + Expect(err).To(BeNil()) + }) + }) }) Describe("Regular User", func() { diff --git a/plugins/manager.go b/plugins/manager.go index f148a706c..bf6bab6e8 100644 --- a/plugins/manager.go +++ b/plugins/manager.go @@ -146,6 +146,12 @@ func (m *Manager) Start(ctx context.Context) error { log.Info(ctx, "Starting plugin manager", "folder", folder) + // Clear previous error states so plugins can be retried on restart + adminCtx := adminContext(ctx) + if err := m.ds.Plugin(adminCtx).ClearErrors(); err != nil { + log.Error(ctx, "Error clearing plugin errors", err) + } + // Sync plugins folder with DB if err := m.syncPlugins(ctx, folder); err != nil { log.Error(ctx, "Error syncing plugins with DB", err) diff --git a/tests/mock_plugin_repo.go b/tests/mock_plugin_repo.go index dd08f6dec..5d22c26aa 100644 --- a/tests/mock_plugin_repo.go +++ b/tests/mock_plugin_repo.go @@ -29,6 +29,20 @@ func (m *MockPluginRepo) SetError(err bool) { m.Err = err } +func (m *MockPluginRepo) ClearErrors() error { + if m.Err { + return errors.New("unexpected error") + } + for i := range m.All { + m.All[i].LastError = "" + } + for k, p := range m.Data { + p.LastError = "" + m.Data[k] = p + } + return nil +} + func (m *MockPluginRepo) SetData(plugins model.Plugins) { m.Data = make(map[string]*model.Plugin, len(plugins)) m.All = plugins From c4fd8e31251a458450efad2a1e157c160da9bbc3 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 2 Mar 2026 11:20:25 -0500 Subject: [PATCH 26/60] fix(plugins): resolve kvstore TTL flaky test due to second-boundary race Changed the TTL expiration check from strict greater-than to greater-or-equal in the notExpiredFilter SQL condition. SQLite's datetime has second-level precision, so a 1-second TTL set late in a second could appear expired immediately when read at the next second boundary (e.g. expires_at of T+1 fails the check 'T+1 > T+1'). Updated the cleanup query consistently to use strict less-than, so rows are only deleted after their expiration second has fully passed. --- plugins/host_kvstore.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/host_kvstore.go b/plugins/host_kvstore.go index 9aa37f7e5..248e43c4d 100644 --- a/plugins/host_kvstore.go +++ b/plugins/host_kvstore.go @@ -25,7 +25,7 @@ const ( ) // notExpiredFilter is the SQL condition to exclude expired keys. -const notExpiredFilter = "(expires_at IS NULL OR expires_at > datetime('now'))" +const notExpiredFilter = "(expires_at IS NULL OR expires_at >= datetime('now'))" const cleanupInterval = 1 * time.Hour @@ -349,7 +349,7 @@ func (s *kvstoreServiceImpl) cleanupLoop(ctx context.Context) { // cleanupExpired removes all expired keys from the database to reclaim disk space. func (s *kvstoreServiceImpl) cleanupExpired(ctx context.Context) { - result, err := s.db.ExecContext(ctx, `DELETE FROM kvstore WHERE expires_at IS NOT NULL AND expires_at <= datetime('now')`) + result, err := s.db.ExecContext(ctx, `DELETE FROM kvstore WHERE expires_at IS NOT NULL AND expires_at < datetime('now')`) if err != nil { log.Error(ctx, "KVStore cleanup: failed to delete expired keys", "plugin", s.pluginName, err) return From acd69f6a4fbeb68a0bf86ae65a393531bb4d7d24 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 2 Mar 2026 11:39:59 -0500 Subject: [PATCH 27/60] feat(playlist): support #EXTALBUMARTURL directive and sidecar images (#5131) * feat(playlist): add migration for playlist image field rename and external URL * refactor(playlist): rename ImageFile to UploadedImage and ArtworkPath to UploadedImagePath Rename playlist model fields and methods for clarity in preparation for adding external image URL and sidecar image support. Add the new ExternalImageURL field to the Playlist model. * feat(playlist): parse #EXTALBUMARTURL directive in M3U imports * feat(playlist): always sync ExternalImageURL on re-scan, preserve UploadedImage * feat(artwork): add sidecar image discovery and cache invalidation for playlists Add playlist sidecar image support to the artwork reader fallback chain. A sidecar image (e.g., MyPlaylist.jpg next to MyPlaylist.m3u) is discovered via case-insensitive base name matching using model.IsImageFile(). Cache invalidation uses max(playlist.UpdatedAt, imageFile.ModTime()) to bust stale artwork when sidecar or ExternalImageURL local files change. * feat(artwork): add external image URL source to playlist artwork reader Add fromPlaylistExternalImage source function that resolves playlist cover art from ExternalImageURL, supporting both HTTP(S) URLs (via the existing fromURL helper) and local file paths (via os.Open). Insert it in the Reader() fallback chain between sidecar and tiled cover. * refactor(artwork): simplify playlist artwork source functions Extract shared fromLocalFile helper, use url.Parse for scheme check, and collapse sidecar directory scan conditions. * test(artwork): remove redundant fromPlaylistSidecar tests These tests duplicated scenarios already covered by findPlaylistSidecarPath tests combined with fromLocalFile (tested via fromPlaylistExternalImage). After refactoring fromPlaylistSidecar to a one-liner composing those two functions, the wrapper tests add no value. * fix(playlist): address security review comments from PR #5131: - Use url.PathUnescape instead of url.QueryUnescape for file:// URLs so that '+' in filenames is preserved (not decoded as space). - Validate all local image paths (file://, absolute, relative) against known library boundaries via libraryMatcher, rejecting paths outside any configured library. - Harden #EXTALBUMARTURL against path traversal and SSRF by adding EnableM3UExternalAlbumArt config flag (default false, also disabled by EnableExternalServices=false) to gate HTTP(S) URL storage at parse time and fetching at read time (defense in depth). - Log a warning when os.ReadDir fails in findPlaylistSidecarPath for diagnosability. - Extract resolveLocalPath helper to simplify resolveImageURL. Signed-off-by: Deluan * feat(playlist): implement human-friendly filename generation for uploaded playlist cover images Signed-off-by: Deluan --------- Signed-off-by: Deluan --- conf/configuration.go | 3 + core/artwork/artwork_internal_test.go | 107 ++++++++ core/artwork/reader_playlist.go | 90 ++++++- core/playlists/import.go | 1 + core/playlists/import_test.go | 229 +++++++++++++++++- core/playlists/parse_m3u.go | 57 ++++- core/playlists/playlists.go | 14 +- core/playlists/playlists_test.go | 18 +- ...0302021413_rename_playlist_image_fields.go | 30 +++ model/playlist.go | 52 ++-- model/playlist_test.go | 22 ++ tests/fixtures/playlists/pls-with-art-url.m3u | 5 + ui/src/playlist/PlaylistDetails.jsx | 2 +- utils/files.go | 17 ++ utils/files_test.go | 43 ++++ 15 files changed, 647 insertions(+), 43 deletions(-) create mode 100644 db/migrations/20260302021413_rename_playlist_image_fields.go create mode 100644 tests/fixtures/playlists/pls-with-art-url.m3u diff --git a/conf/configuration.go b/conf/configuration.go index 3e3b42355..0d32815ee 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -46,6 +46,7 @@ type configOptions struct { EnableTranscodingCancellation bool EnableDownloads bool EnableExternalServices bool + EnableM3UExternalAlbumArt bool EnableInsightsCollector bool EnableMediaFileCoverArt bool TranscodingCacheSize string @@ -474,6 +475,7 @@ func parseIniFileConfiguration() { func disableExternalServices() { log.Info("All external integrations are DISABLED!") Server.EnableInsightsCollector = false + Server.EnableM3UExternalAlbumArt = false Server.LastFM.Enabled = false Server.Spotify.ID = "" Server.Deezer.Enabled = false @@ -638,6 +640,7 @@ func setViperDefaults() { viper.SetDefault("smartPlaylistRefreshDelay", 5*time.Second) viper.SetDefault("enabledownloads", true) viper.SetDefault("enableexternalservices", true) + viper.SetDefault("enablem3uexternalalbumart", false) viper.SetDefault("enablemediafilecoverart", true) viper.SetDefault("autotranscodedownload", false) viper.SetDefault("defaultdownsamplingformat", consts.DefaultDownsamplingFormat) diff --git a/core/artwork/artwork_internal_test.go b/core/artwork/artwork_internal_test.go index c18caf737..e2ea7adb0 100644 --- a/core/artwork/artwork_internal_test.go +++ b/core/artwork/artwork_internal_test.go @@ -235,6 +235,113 @@ var _ = Describe("Artwork", func() { }) }) }) + Describe("playlistArtworkReader", func() { + Describe("findPlaylistSidecarPath", func() { + It("discovers sidecar image next to playlist file", func() { + tmpDir := GinkgoT().TempDir() + plsPath := filepath.Join(tmpDir, "MyPlaylist.m3u") + imgPath := filepath.Join(tmpDir, "MyPlaylist.jpg") + Expect(os.WriteFile(plsPath, []byte("#EXTM3U\n"), 0600)).To(Succeed()) + Expect(os.WriteFile(imgPath, []byte("fake image"), 0600)).To(Succeed()) + + result := findPlaylistSidecarPath(GinkgoT().Context(), plsPath) + Expect(result).To(Equal(imgPath)) + }) + + It("returns empty string when no sidecar image exists", func() { + tmpDir := GinkgoT().TempDir() + plsPath := filepath.Join(tmpDir, "MyPlaylist.m3u") + Expect(os.WriteFile(plsPath, []byte("#EXTM3U\n"), 0600)).To(Succeed()) + + result := findPlaylistSidecarPath(GinkgoT().Context(), plsPath) + Expect(result).To(BeEmpty()) + }) + + It("returns empty string when playlist has no path", func() { + result := findPlaylistSidecarPath(GinkgoT().Context(), "") + Expect(result).To(BeEmpty()) + }) + + It("finds sidecar with different case base name", func() { + tmpDir := GinkgoT().TempDir() + plsPath := filepath.Join(tmpDir, "myplaylist.m3u") + imgPath := filepath.Join(tmpDir, "MyPlaylist.jpg") + Expect(os.WriteFile(plsPath, []byte("#EXTM3U\n"), 0600)).To(Succeed()) + Expect(os.WriteFile(imgPath, []byte("fake image"), 0600)).To(Succeed()) + + result := findPlaylistSidecarPath(GinkgoT().Context(), plsPath) + Expect(result).To(Equal(imgPath)) + }) + }) + + Describe("fromPlaylistExternalImage", func() { + It("opens local path from ExternalImageURL", func() { + tmpDir := GinkgoT().TempDir() + imgPath := filepath.Join(tmpDir, "cover.jpg") + Expect(os.WriteFile(imgPath, []byte("external image data"), 0600)).To(Succeed()) + + reader := &playlistArtworkReader{ + pl: model.Playlist{ExternalImageURL: imgPath}, + } + r, path, err := reader.fromPlaylistExternalImage(ctx)() + Expect(err).ToNot(HaveOccurred()) + Expect(r).ToNot(BeNil()) + Expect(path).To(Equal(imgPath)) + data, _ := io.ReadAll(r) + Expect(string(data)).To(Equal("external image data")) + r.Close() + }) + + It("returns nil when ExternalImageURL is empty", func() { + reader := &playlistArtworkReader{ + pl: model.Playlist{ExternalImageURL: ""}, + } + r, path, err := reader.fromPlaylistExternalImage(ctx)() + Expect(err).ToNot(HaveOccurred()) + Expect(r).To(BeNil()) + Expect(path).To(BeEmpty()) + }) + + It("returns error when local file does not exist", func() { + reader := &playlistArtworkReader{ + pl: model.Playlist{ExternalImageURL: "/non/existent/path/cover.jpg"}, + } + r, _, err := reader.fromPlaylistExternalImage(ctx)() + Expect(err).To(HaveOccurred()) + Expect(r).To(BeNil()) + }) + + It("skips HTTP URL when EnableM3UExternalAlbumArt is false", func() { + conf.Server.EnableM3UExternalAlbumArt = false + + reader := &playlistArtworkReader{ + pl: model.Playlist{ExternalImageURL: "https://example.com/cover.jpg"}, + } + r, path, err := reader.fromPlaylistExternalImage(ctx)() + Expect(err).ToNot(HaveOccurred()) + Expect(r).To(BeNil()) + Expect(path).To(BeEmpty()) + }) + + It("still opens local path when EnableM3UExternalAlbumArt is false", func() { + conf.Server.EnableM3UExternalAlbumArt = false + + tmpDir := GinkgoT().TempDir() + imgPath := filepath.Join(tmpDir, "cover.jpg") + Expect(os.WriteFile(imgPath, []byte("local image"), 0600)).To(Succeed()) + + reader := &playlistArtworkReader{ + pl: model.Playlist{ExternalImageURL: imgPath}, + } + r, path, err := reader.fromPlaylistExternalImage(ctx)() + Expect(err).ToNot(HaveOccurred()) + Expect(r).ToNot(BeNil()) + Expect(path).To(Equal(imgPath)) + r.Close() + }) + }) + }) + Describe("resizedArtworkReader", func() { BeforeEach(func() { folderRepo.result = []model.Folder{{ diff --git a/core/artwork/reader_playlist.go b/core/artwork/reader_playlist.go index 09bfe221b..91d47b0b0 100644 --- a/core/artwork/reader_playlist.go +++ b/core/artwork/reader_playlist.go @@ -8,10 +8,14 @@ import ( "image/draw" "image/png" "io" + "net/url" "os" + "path/filepath" + "strings" "time" "github.com/disintegration/imaging" + "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/slice" @@ -36,6 +40,24 @@ func newPlaylistArtworkReader(ctx context.Context, artwork *artwork, artID model } a.cacheKey.artID = artID a.cacheKey.lastUpdate = pl.UpdatedAt + + // Check sidecar and ExternalImageURL local file ModTimes for cache invalidation. + // If either is newer than the playlist's UpdatedAt, use that instead so the + // cache is busted when a user replaces a sidecar image or local file reference. + for _, path := range []string{ + findPlaylistSidecarPath(ctx, pl.Path), + pl.ExternalImageURL, + } { + if path == "" || strings.HasPrefix(path, "http://") || strings.HasPrefix(path, "https://") { + continue + } + if info, err := os.Stat(path); err == nil { + if info.ModTime().After(a.cacheKey.lastUpdate) { + a.cacheKey.lastUpdate = info.ModTime() + } + } + } + return a, nil } @@ -45,26 +67,82 @@ func (a *playlistArtworkReader) LastUpdated() time.Time { func (a *playlistArtworkReader) Reader(ctx context.Context) (io.ReadCloser, string, error) { return selectImageReader(ctx, a.artID, - a.fromPlaylistImage(), + a.fromPlaylistUploadedImage(), + a.fromPlaylistSidecar(ctx), + a.fromPlaylistExternalImage(ctx), a.fromGeneratedTiledCover(ctx), fromAlbumPlaceholder(), ) } -func (a *playlistArtworkReader) fromPlaylistImage() sourceFunc { +func (a *playlistArtworkReader) fromPlaylistUploadedImage() sourceFunc { + return fromLocalFile(a.pl.UploadedImagePath()) +} + +func (a *playlistArtworkReader) fromPlaylistSidecar(ctx context.Context) sourceFunc { + return fromLocalFile(findPlaylistSidecarPath(ctx, a.pl.Path)) +} + +func (a *playlistArtworkReader) fromPlaylistExternalImage(ctx context.Context) sourceFunc { return func() (io.ReadCloser, string, error) { - absPath := a.pl.ArtworkPath() - if absPath == "" { + imgURL := a.pl.ExternalImageURL + if imgURL == "" { return nil, "", nil } - f, err := os.Open(absPath) + parsed, err := url.Parse(imgURL) if err != nil { return nil, "", err } - return f, absPath, nil + if parsed.Scheme == "http" || parsed.Scheme == "https" { + if !conf.Server.EnableM3UExternalAlbumArt { + return nil, "", nil + } + return fromURL(ctx, parsed) + } + return fromLocalFile(imgURL)() } } +// fromLocalFile returns a sourceFunc that opens the given local path. +// Returns (nil, "", nil) if path is empty — signalling "not found, try next source". +func fromLocalFile(path string) sourceFunc { + return func() (io.ReadCloser, string, error) { + if path == "" { + return nil, "", nil + } + f, err := os.Open(path) + if err != nil { + return nil, "", err + } + return f, path, nil + } +} + +// findPlaylistSidecarPath scans the directory of the playlist file for a sidecar +// image file with the same base name (case-insensitive). Returns empty string if +// no matching image is found or if plsPath is empty. +func findPlaylistSidecarPath(ctx context.Context, plsPath string) string { + if plsPath == "" { + return "" + } + dir := filepath.Dir(plsPath) + base := strings.TrimSuffix(filepath.Base(plsPath), filepath.Ext(plsPath)) + + entries, err := os.ReadDir(dir) + if err != nil { + log.Warn(ctx, "Could not read directory for playlist sidecar", "dir", dir, err) + return "" + } + for _, entry := range entries { + name := entry.Name() + nameBase := strings.TrimSuffix(name, filepath.Ext(name)) + if !entry.IsDir() && strings.EqualFold(nameBase, base) && model.IsImageFile(name) { + return filepath.Join(dir, name) + } + } + return "" +} + func (a *playlistArtworkReader) fromGeneratedTiledCover(ctx context.Context) sourceFunc { return func() (io.ReadCloser, string, error) { tiles, err := a.loadTiles(ctx) diff --git a/core/playlists/import.go b/core/playlists/import.go index 40e230527..4462554c7 100644 --- a/core/playlists/import.go +++ b/core/playlists/import.go @@ -106,6 +106,7 @@ func (s *playlists) updatePlaylist(ctx context.Context, newPls *model.Playlist) newPls.Comment = pls.Comment newPls.OwnerID = pls.OwnerID newPls.Public = pls.Public + newPls.UploadedImage = pls.UploadedImage // Preserve manual upload newPls.EvaluatedAt = &time.Time{} } else { log.Info(ctx, "Adding synced playlist", "playlist", newPls.Name, "path", newPls.Path, "owner", owner.UserName) diff --git a/core/playlists/import_test.go b/core/playlists/import_test.go index a42c3f3eb..5312df95d 100644 --- a/core/playlists/import_test.go +++ b/core/playlists/import_test.go @@ -2,7 +2,9 @@ package playlists_test import ( "context" + "fmt" "os" + "path/filepath" "strconv" "strings" "time" @@ -39,6 +41,7 @@ var _ = Describe("Playlists - Import", func() { Describe("ImportFile", func() { var folder *model.Folder BeforeEach(func() { + DeferCleanup(configtest.SetupConfig()) ps = playlists.NewPlaylists(ds) ds.MockedMediaFile = &mockedMediaFileRepo{} libPath, _ := os.Getwd() @@ -93,6 +96,213 @@ var _ = Describe("Playlists - Import", func() { Expect(pls.Tracks).To(HaveLen(1)) Expect(pls.Tracks[0].Path).To(Equal("tests/fixtures/playlists/test.mp3")) }) + + It("parses #EXTALBUMARTURL with HTTP URL", func() { + conf.Server.EnableM3UExternalAlbumArt = true + + pls, err := ps.ImportFile(ctx, folder, "pls-with-art-url.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(Equal("https://example.com/cover.jpg")) + Expect(pls.Tracks).To(HaveLen(2)) + }) + + It("parses #EXTALBUMARTURL with absolute local path", func() { + tmpDir := GinkgoT().TempDir() + imgPath := filepath.Join(tmpDir, "cover.jpg") + Expect(os.WriteFile(imgPath, []byte("fake image"), 0600)).To(Succeed()) + + m3u := fmt.Sprintf("#EXTALBUMARTURL:%s\ntest.mp3\ntest.ogg\n", imgPath) + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3", "test.ogg"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(Equal(imgPath)) + }) + + It("parses #EXTALBUMARTURL with relative local path", func() { + tmpDir := GinkgoT().TempDir() + Expect(os.WriteFile(filepath.Join(tmpDir, "cover.jpg"), []byte("fake image"), 0600)).To(Succeed()) + + m3u := "#EXTALBUMARTURL:cover.jpg\ntest.mp3\n" + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(Equal(filepath.Join(tmpDir, "cover.jpg"))) + }) + + It("parses #EXTALBUMARTURL with file:// URL", func() { + tmpDir := GinkgoT().TempDir() + imgPath := filepath.Join(tmpDir, "my cover.jpg") + Expect(os.WriteFile(imgPath, []byte("fake image"), 0600)).To(Succeed()) + + m3u := fmt.Sprintf("#EXTALBUMARTURL:file://%s\ntest.mp3\n", strings.ReplaceAll(imgPath, " ", "%20")) + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(Equal(imgPath)) + }) + + It("preserves + in file:// URLs (PathUnescape, not QueryUnescape)", func() { + tmpDir := GinkgoT().TempDir() + imgPath := filepath.Join(tmpDir, "A+B.jpg") + Expect(os.WriteFile(imgPath, []byte("fake image"), 0600)).To(Succeed()) + + m3u := fmt.Sprintf("#EXTALBUMARTURL:file://%s\ntest.mp3\n", imgPath) + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(Equal(imgPath)) + }) + + It("rejects #EXTALBUMARTURL with absolute path outside library boundaries", func() { + tmpDir := GinkgoT().TempDir() + + m3u := "#EXTALBUMARTURL:/etc/passwd\ntest.mp3\n" + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(BeEmpty()) + }) + + It("rejects #EXTALBUMARTURL with file:// URL outside library boundaries", func() { + tmpDir := GinkgoT().TempDir() + + m3u := "#EXTALBUMARTURL:file:///etc/passwd\ntest.mp3\n" + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(BeEmpty()) + }) + + It("rejects #EXTALBUMARTURL with relative path escaping library", func() { + tmpDir := GinkgoT().TempDir() + + m3u := "#EXTALBUMARTURL:../../etc/passwd\ntest.mp3\n" + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(BeEmpty()) + }) + + It("ignores HTTP #EXTALBUMARTURL when EnableM3UExternalAlbumArt is false", func() { + conf.Server.EnableM3UExternalAlbumArt = false + + tmpDir := GinkgoT().TempDir() + m3u := "#EXTALBUMARTURL:https://example.com/cover.jpg\ntest.mp3\n" + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(BeEmpty()) + }) + + It("updates ExternalImageURL on re-scan even when UploadedImage is set", func() { + conf.Server.EnableM3UExternalAlbumArt = true + + tmpDir := GinkgoT().TempDir() + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + m3u := "#EXTALBUMARTURL:https://example.com/new-cover.jpg\ntest.mp3\n" + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + existingPls := &model.Playlist{ + ID: "existing-id", + Name: "Existing Playlist", + Path: plsFile, + Sync: true, + UploadedImage: "existing-id.jpg", + ExternalImageURL: "https://example.com/old-cover.jpg", + } + mockPlsRepo.PathMap = map[string]*model.Playlist{plsFile: existingPls} + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.UploadedImage).To(Equal("existing-id.jpg")) + Expect(pls.ExternalImageURL).To(Equal("https://example.com/new-cover.jpg")) + }) + + It("clears ExternalImageURL on re-scan when directive is removed", func() { + tmpDir := GinkgoT().TempDir() + mockLibRepo.SetData([]model.Library{{ID: 1, Path: tmpDir}}) + ds.MockedMediaFile = &mockedMediaFileFromListRepo{data: []string{"test.mp3"}} + ps = playlists.NewPlaylists(ds) + + m3u := "test.mp3\n" + plsFile := filepath.Join(tmpDir, "test.m3u") + Expect(os.WriteFile(plsFile, []byte(m3u), 0600)).To(Succeed()) + + existingPls := &model.Playlist{ + ID: "existing-id", + Name: "Existing Playlist", + Path: plsFile, + Sync: true, + ExternalImageURL: "https://example.com/old-cover.jpg", + } + mockPlsRepo.PathMap = map[string]*model.Playlist{plsFile: existingPls} + + plsFolder := &model.Folder{ID: "1", LibraryID: 1, LibraryPath: tmpDir, Path: "", Name: ""} + pls, err := ps.ImportFile(ctx, plsFolder, "test.m3u") + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(BeEmpty()) + }) }) Describe("NSP", func() { @@ -125,7 +335,6 @@ var _ = Describe("Playlists - Import", func() { Expect(pls.Public).To(BeFalse()) }) It("uses server default when public field is absent", func() { - DeferCleanup(configtest.SetupConfig()) conf.Server.DefaultPlaylistPublicVisibility = true pls, err := ps.ImportFile(ctx, folder, "recently_played.nsp") @@ -495,6 +704,24 @@ var _ = Describe("Playlists - Import", func() { Expect(pls.Tracks[0].Path).To(Equal("abc/tEsT1.Mp3")) }) + It("parses #EXTALBUMARTURL with HTTP URL via ImportM3U", func() { + conf.Server.EnableM3UExternalAlbumArt = true + + repo.data = []string{"tests/test.mp3"} + m3u := "#EXTALBUMARTURL:https://example.com/cover.jpg\n/music/tests/test.mp3\n" + pls, err := ps.ImportM3U(ctx, strings.NewReader(m3u)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(Equal("https://example.com/cover.jpg")) + }) + + It("ignores relative #EXTALBUMARTURL when imported via API (no folder context)", func() { + repo.data = []string{"tests/test.mp3"} + m3u := "#EXTALBUMARTURL:cover.jpg\n/music/tests/test.mp3\n" + pls, err := ps.ImportM3U(ctx, strings.NewReader(m3u)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.ExternalImageURL).To(BeEmpty()) + }) + // Fullwidth characters (e.g., ＡＢＣＤ) are not handled by SQLite's NOCASE collation, // so we need exact matching for non-ASCII characters. It("matches fullwidth characters exactly (SQLite NOCASE limitation)", func() { diff --git a/core/playlists/parse_m3u.go b/core/playlists/parse_m3u.go index 4e79d15c6..97ed7df6f 100644 --- a/core/playlists/parse_m3u.go +++ b/core/playlists/parse_m3u.go @@ -11,6 +11,7 @@ import ( "strings" "time" + "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/slice" @@ -34,13 +35,17 @@ func (s *playlists) parseM3U(ctx context.Context, pls *model.Playlist, folder *m pls.Name = line[len("#PLAYLIST:"):] continue } + if after, ok := strings.CutPrefix(line, "#EXTALBUMARTURL:"); ok { + pls.ExternalImageURL = resolveImageURL(after, folder, resolver.matcher) + continue + } // Skip empty lines and extended info if line == "" || strings.HasPrefix(line, "#") { continue } if after, ok := strings.CutPrefix(line, "file://"); ok { line = after - line, _ = url.QueryUnescape(line) + line, _ = url.PathUnescape(line) } if !model.IsAudioFile(line) { continue @@ -267,3 +272,53 @@ func (r *pathResolver) resolvePaths(ctx context.Context, folder *model.Folder, l return results, nil } + +// resolveImageURL resolves an #EXTALBUMARTURL value to a storable string. +// HTTP(S) URLs are stored as-is (gated by EnableM3UExternalAlbumArt). +// Local paths (file://, absolute, or relative) are resolved to an absolute path +// and validated against known library boundaries via matcher. +func resolveImageURL(value string, folder *model.Folder, matcher *libraryMatcher) string { + value = strings.TrimSpace(value) + if value == "" { + return "" + } + + // HTTP(S) URLs — store as-is, but only if external album art is enabled + if strings.HasPrefix(value, "http://") || strings.HasPrefix(value, "https://") { + if !conf.Server.EnableM3UExternalAlbumArt { + return "" + } + return value + } + + // Resolve to local absolute path + localPath, ok := resolveLocalPath(value, folder) + if !ok { + return "" + } + + // Validate path is within a known library + if libID, _ := matcher.findLibraryForPath(localPath); libID == 0 { + return "" + } + return localPath +} + +// resolveLocalPath converts a file://, absolute, or relative path to a clean absolute path. +// Returns ("", false) if the path cannot be resolved. +func resolveLocalPath(value string, folder *model.Folder) (string, bool) { + if after, ok := strings.CutPrefix(value, "file://"); ok { + decoded, err := url.PathUnescape(after) + if err != nil { + return "", false + } + return filepath.Clean(decoded), true + } + if filepath.IsAbs(value) { + return filepath.Clean(value), true + } + if folder == nil { + return "", false + } + return filepath.Clean(filepath.Join(folder.AbsolutePath(), value)), true +} diff --git a/core/playlists/playlists.go b/core/playlists/playlists.go index 7e881b730..0649b16a9 100644 --- a/core/playlists/playlists.go +++ b/core/playlists/playlists.go @@ -131,7 +131,7 @@ func (s *playlists) Delete(ctx context.Context, id string) error { } // Clean up custom cover image file if one exists - if path := pls.ArtworkPath(); path != "" { + if path := pls.UploadedImagePath(); path != "" { if err := os.Remove(path); err != nil && !os.IsNotExist(err) { log.Warn(ctx, "Failed to remove playlist image on delete", "path", path, err) } @@ -288,10 +288,10 @@ func (s *playlists) SetImage(ctx context.Context, playlistID string, reader io.R return err } - filename := playlistID + ext - oldPath := pls.ArtworkPath() - pls.ImageFile = filename - absPath := pls.ArtworkPath() + filename := pls.ImageFilename(ext) + oldPath := pls.UploadedImagePath() + pls.UploadedImage = filename + absPath := pls.UploadedImagePath() if err := os.MkdirAll(filepath.Dir(absPath), 0755); err != nil { return fmt.Errorf("creating playlist images directory: %w", err) @@ -324,12 +324,12 @@ func (s *playlists) RemoveImage(ctx context.Context, playlistID string) error { return err } - if path := pls.ArtworkPath(); path != "" { + if path := pls.UploadedImagePath(); path != "" { if err := os.Remove(path); err != nil && !os.IsNotExist(err) { log.Warn(ctx, "Failed to remove playlist image", "path", path, err) } } - pls.ImageFile = "" + pls.UploadedImage = "" return s.ds.Playlist(ctx).Put(pls) } diff --git a/core/playlists/playlists_test.go b/core/playlists/playlists_test.go index 4d42bbeb9..ec73c329a 100644 --- a/core/playlists/playlists_test.go +++ b/core/playlists/playlists_test.go @@ -315,14 +315,14 @@ var _ = Describe("Playlists", func() { ps = playlists.NewPlaylists(ds) }) - It("saves image file and updates ImageFile", func() { + It("saves image file and updates UploadedImage", func() { ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) reader := strings.NewReader("fake image data") err := ps.SetImage(ctx, "pls-1", reader, ".jpg") Expect(err).ToNot(HaveOccurred()) - Expect(mockPlsRepo.Last.ImageFile).To(Equal("pls-1.jpg")) - absPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.jpg") + Expect(mockPlsRepo.Last.UploadedImage).To(Equal("pls-1_my_playlist.jpg")) + absPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1_my_playlist.jpg") data, err := os.ReadFile(absPath) Expect(err).ToNot(HaveOccurred()) Expect(string(data)).To(Equal("fake image data")) @@ -334,14 +334,14 @@ var _ = Describe("Playlists", func() { // Upload first image err := ps.SetImage(ctx, "pls-1", strings.NewReader("first"), ".png") Expect(err).ToNot(HaveOccurred()) - oldPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.png") + oldPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1_my_playlist.png") Expect(oldPath).To(BeAnExistingFile()) // Upload replacement image err = ps.SetImage(ctx, "pls-1", strings.NewReader("second"), ".jpg") Expect(err).ToNot(HaveOccurred()) Expect(oldPath).ToNot(BeAnExistingFile()) - newPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.jpg") + newPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1_my_playlist.jpg") Expect(newPath).To(BeAnExistingFile()) }) @@ -378,19 +378,19 @@ var _ = Describe("Playlists", func() { Expect(os.WriteFile(filepath.Join(imgDir, "pls-1.jpg"), []byte("img data"), 0600)).To(Succeed()) mockPlsRepo.Data = map[string]*model.Playlist{ - "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1", ImageFile: "pls-1.jpg"}, + "pls-1": {ID: "pls-1", Name: "My Playlist", OwnerID: "user-1", UploadedImage: "pls-1.jpg"}, "pls-empty": {ID: "pls-empty", Name: "No Cover", OwnerID: "user-1"}, "pls-other": {ID: "pls-other", Name: "Other's", OwnerID: "other-user"}, } ps = playlists.NewPlaylists(ds) }) - It("removes file and clears ImageFile", func() { + It("removes file and clears UploadedImage", func() { ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) err := ps.RemoveImage(ctx, "pls-1") Expect(err).ToNot(HaveOccurred()) - Expect(mockPlsRepo.Last.ImageFile).To(BeEmpty()) + Expect(mockPlsRepo.Last.UploadedImage).To(BeEmpty()) absPath := filepath.Join(tmpDir, "artwork", "playlist", "pls-1.jpg") Expect(absPath).ToNot(BeAnExistingFile()) }) @@ -399,7 +399,7 @@ var _ = Describe("Playlists", func() { ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) err := ps.RemoveImage(ctx, "pls-empty") Expect(err).ToNot(HaveOccurred()) - Expect(mockPlsRepo.Last.ImageFile).To(BeEmpty()) + Expect(mockPlsRepo.Last.UploadedImage).To(BeEmpty()) }) It("denies non-owner", func() { diff --git a/db/migrations/20260302021413_rename_playlist_image_fields.go b/db/migrations/20260302021413_rename_playlist_image_fields.go new file mode 100644 index 000000000..1e9754637 --- /dev/null +++ b/db/migrations/20260302021413_rename_playlist_image_fields.go @@ -0,0 +1,30 @@ +package migrations + +import ( + "context" + "database/sql" + + "github.com/pressly/goose/v3" +) + +func init() { + goose.AddMigrationContext(upRenamePlaylistImageFields, downRenamePlaylistImageFields) +} + +func upRenamePlaylistImageFields(ctx context.Context, tx *sql.Tx) error { + _, err := tx.ExecContext(ctx, `ALTER TABLE playlist RENAME COLUMN image_file TO uploaded_image;`) + if err != nil { + return err + } + _, err = tx.ExecContext(ctx, `ALTER TABLE playlist ADD COLUMN external_image_url VARCHAR(255) DEFAULT '';`) + return err +} + +func downRenamePlaylistImageFields(ctx context.Context, tx *sql.Tx) error { + _, err := tx.ExecContext(ctx, `ALTER TABLE playlist DROP COLUMN external_image_url;`) + if err != nil { + return err + } + _, err = tx.ExecContext(ctx, `ALTER TABLE playlist RENAME COLUMN uploaded_image TO image_file;`) + return err +} diff --git a/model/playlist.go b/model/playlist.go index b6a52dcf2..5c9052eb7 100644 --- a/model/playlist.go +++ b/model/playlist.go @@ -9,24 +9,26 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/model/criteria" + "github.com/navidrome/navidrome/utils" ) type Playlist struct { - ID string `structs:"id" json:"id"` - Name string `structs:"name" json:"name"` - Comment string `structs:"comment" json:"comment"` - Duration float32 `structs:"duration" json:"duration"` - Size int64 `structs:"size" json:"size"` - SongCount int `structs:"song_count" json:"songCount"` - OwnerName string `structs:"-" json:"ownerName"` - OwnerID string `structs:"owner_id" json:"ownerId"` - Public bool `structs:"public" json:"public"` - Tracks PlaylistTracks `structs:"-" json:"tracks,omitempty"` - Path string `structs:"path" json:"path"` - Sync bool `structs:"sync" json:"sync"` - ImageFile string `structs:"image_file" json:"imageFile"` - CreatedAt time.Time `structs:"created_at" json:"createdAt"` - UpdatedAt time.Time `structs:"updated_at" json:"updatedAt"` + ID string `structs:"id" json:"id"` + Name string `structs:"name" json:"name"` + Comment string `structs:"comment" json:"comment"` + Duration float32 `structs:"duration" json:"duration"` + Size int64 `structs:"size" json:"size"` + SongCount int `structs:"song_count" json:"songCount"` + OwnerName string `structs:"-" json:"ownerName"` + OwnerID string `structs:"owner_id" json:"ownerId"` + Public bool `structs:"public" json:"public"` + Tracks PlaylistTracks `structs:"-" json:"tracks,omitempty"` + Path string `structs:"path" json:"path"` + Sync bool `structs:"sync" json:"sync"` + UploadedImage string `structs:"uploaded_image" json:"uploadedImage"` + ExternalImageURL string `structs:"external_image_url" json:"externalImageUrl,omitempty"` + CreatedAt time.Time `structs:"created_at" json:"createdAt"` + UpdatedAt time.Time `structs:"updated_at" json:"updatedAt"` // SmartPlaylist attributes Rules *criteria.Criteria `structs:"rules" json:"rules"` @@ -106,15 +108,29 @@ func (pls *Playlist) AddMediaFiles(mfs MediaFiles) { pls.refreshStats() } +// ImageFilename returns a human-friendly filename for an uploaded playlist cover image. +// Format: _, falling back to if the name cleans to empty. +func (pls Playlist) ImageFilename(ext string) string { + clean := utils.CleanFileName(pls.Name) + if clean == "" { + return pls.ID + ext + } + return pls.ID + "_" + clean + ext +} + func (pls Playlist) CoverArtID() ArtworkID { return artworkIDFromPlaylist(pls) } -func (pls Playlist) ArtworkPath() string { - if pls.ImageFile == "" { +// UploadedImagePath returns the absolute filesystem path for a manually uploaded +// playlist cover image. Returns empty string if no image has been uploaded. +// This does NOT cover sidecar images or external URLs — those are resolved +// by the artwork reader's fallback chain. +func (pls Playlist) UploadedImagePath() string { + if pls.UploadedImage == "" { return "" } - return filepath.Join(conf.Server.DataFolder, consts.ArtworkFolder, "playlist", pls.ImageFile) + return filepath.Join(conf.Server.DataFolder, consts.ArtworkFolder, "playlist", pls.UploadedImage) } type Playlists []Playlist diff --git a/model/playlist_test.go b/model/playlist_test.go index a54cecd53..98dd4e978 100644 --- a/model/playlist_test.go +++ b/model/playlist_test.go @@ -7,6 +7,28 @@ import ( ) var _ = Describe("Playlist", func() { + Describe("ImageFilename", func() { + It("returns ID_cleanname.ext for a normal name", func() { + pls := model.Playlist{ID: "abc123", Name: "My Cool Playlist"} + Expect(pls.ImageFilename(".jpg")).To(Equal("abc123_my_cool_playlist.jpg")) + }) + + It("falls back to ID.ext when name cleans to empty", func() { + pls := model.Playlist{ID: "abc123", Name: "!!!"} + Expect(pls.ImageFilename(".png")).To(Equal("abc123.png")) + }) + + It("falls back to ID.ext for empty name", func() { + pls := model.Playlist{ID: "abc123", Name: ""} + Expect(pls.ImageFilename(".jpg")).To(Equal("abc123.jpg")) + }) + + It("handles names with special characters", func() { + pls := model.Playlist{ID: "x1", Name: "Rock & Roll! (2024)"} + Expect(pls.ImageFilename(".webp")).To(Equal("x1_rock__roll_2024.webp")) + }) + }) + Describe("ToM3U8()", func() { var pls model.Playlist BeforeEach(func() { diff --git a/tests/fixtures/playlists/pls-with-art-url.m3u b/tests/fixtures/playlists/pls-with-art-url.m3u new file mode 100644 index 000000000..9dbf180f8 --- /dev/null +++ b/tests/fixtures/playlists/pls-with-art-url.m3u @@ -0,0 +1,5 @@ +#EXTM3U +#PLAYLIST:Playlist With Art +#EXTALBUMARTURL:https://example.com/cover.jpg +test.mp3 +test.ogg diff --git a/ui/src/playlist/PlaylistDetails.jsx b/ui/src/playlist/PlaylistDetails.jsx index 4c242dd2a..b24446cb9 100644 --- a/ui/src/playlist/PlaylistDetails.jsx +++ b/ui/src/playlist/PlaylistDetails.jsx @@ -246,7 +246,7 @@ const PlaylistDetails = (props) => { - {record.imageFile && ( + {record.uploadedImage && ( diff --git a/utils/files.go b/utils/files.go index 9bdc262c5..2fce307ea 100644 --- a/utils/files.go +++ b/utils/files.go @@ -4,11 +4,14 @@ import ( "os" "path" "path/filepath" + "regexp" "strings" "github.com/navidrome/navidrome/model/id" ) +var cleanFileNameRe = regexp.MustCompile(`[^a-z0-9_-]`) + func TempFileName(prefix, suffix string) string { return filepath.Join(os.TempDir(), prefix+id.NewRandom()+suffix) } @@ -18,6 +21,20 @@ func BaseName(filePath string) string { return strings.TrimSuffix(p, path.Ext(p)) } +// CleanFileName produces a filesystem-safe, human-readable version of a name. +// It lowercases, replaces spaces with underscores, strips non-alphanumeric +// characters (except underscore and hyphen), and truncates to 50 characters. +func CleanFileName(name string) string { + s := strings.ToLower(strings.TrimSpace(name)) + s = strings.ReplaceAll(s, " ", "_") + s = cleanFileNameRe.ReplaceAllString(s, "") + if len(s) > 50 { + s = s[:50] + } + s = strings.TrimRight(s, "_-") + return s +} + // FileExists checks if a file or directory exists func FileExists(path string) bool { _, err := os.Stat(path) diff --git a/utils/files_test.go b/utils/files_test.go index dcb28aafb..72fc4f96f 100644 --- a/utils/files_test.go +++ b/utils/files_test.go @@ -99,6 +99,49 @@ var _ = Describe("BaseName", func() { }) }) +var _ = Describe("CleanFileName", func() { + It("lowercases and replaces spaces with underscores", func() { + Expect(utils.CleanFileName("My Cool Playlist")).To(Equal("my_cool_playlist")) + }) + + It("strips special characters", func() { + Expect(utils.CleanFileName("Rock & Roll! (2024)")).To(Equal("rock__roll_2024")) + }) + + It("handles unicode characters", func() { + Expect(utils.CleanFileName("Música Favorita")).To(Equal("msica_favorita")) + }) + + It("preserves hyphens", func() { + Expect(utils.CleanFileName("lo-fi beats")).To(Equal("lo-fi_beats")) + }) + + It("returns empty string for empty input", func() { + Expect(utils.CleanFileName("")).To(BeEmpty()) + }) + + It("returns empty string for whitespace-only input", func() { + Expect(utils.CleanFileName(" ")).To(BeEmpty()) + }) + + It("returns empty string when all characters are stripped", func() { + Expect(utils.CleanFileName("!!!@@@###")).To(BeEmpty()) + }) + + It("truncates to 50 characters", func() { + long := strings.Repeat("abcdefghij", 10) // 100 chars + result := utils.CleanFileName(long) + Expect(len(result)).To(Equal(50)) + }) + + It("trims trailing underscores and hyphens after truncation", func() { + // 49 a's + space + "b" = after clean: 49 a's + "_b" = 51 chars, truncated to 50 = 49 a's + "_" + name := strings.Repeat("a", 49) + " b" + result := utils.CleanFileName(name) + Expect(result).To(Equal(strings.Repeat("a", 49))) + }) +}) + var _ = Describe("FileExists", func() { var tempFile *os.File var tempDir string From 3d86d44fd9fd504e0ec6ee257fddd532accb390a Mon Sep 17 00:00:00 2001 From: Lokke Date: Mon, 2 Mar 2026 17:51:32 +0100 Subject: [PATCH 28/60] feat(server): add averageRating to smart playlists (#5092) --- model/criteria/fields.go | 1 + 1 file changed, 1 insertion(+) diff --git a/model/criteria/fields.go b/model/criteria/fields.go index 4dfc50f11..ed73de9f9 100644 --- a/model/criteria/fields.go +++ b/model/criteria/fields.go @@ -60,6 +60,7 @@ var fieldMap = map[string]*mappedField{ "daterated": {field: "annotation.rated_at"}, "playcount": {field: "COALESCE(annotation.play_count, 0)"}, "rating": {field: "COALESCE(annotation.rating, 0)"}, + "averagerating": {field: "media_file.average_rating", numeric: true}, "albumrating": {field: "COALESCE(album_annotation.rating, 0)", joinType: JoinAlbumAnnotation}, "albumloved": {field: "COALESCE(album_annotation.starred, false)", joinType: JoinAlbumAnnotation}, "albumplaycount": {field: "COALESCE(album_annotation.play_count, 0)", joinType: JoinAlbumAnnotation}, From 82f9f88c0f31762ac1a7e68f6356495db69f6c46 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 2 Mar 2026 13:15:31 -0500 Subject: [PATCH 29/60] refactor(auth): replace untyped JWT claims with typed Claims struct Introduced a typed Claims struct in core/auth to replace the raw map[string]any approach used for JWT claims throughout the codebase. This provides compile-time safety and better readability when creating, validating, and extracting JWT tokens. Also upgraded lestrrat-go/jwx from v2 to v3 and go-chi/jwtauth to v5.4.0, adapting all callers to the new API where token accessor methods now return tuples instead of bare values. Updated all affected handlers, middleware, and tests. Signed-off-by: Deluan --- adapters/deezer/client_auth.go | 6 +- adapters/deezer/client_auth_test.go | 5 +- core/auth/auth.go | 58 +++++++---------- core/auth/auth_test.go | 15 ++--- core/auth/claims.go | 94 +++++++++++++++++++++++++++ core/auth/claims_test.go | 99 +++++++++++++++++++++++++++++ core/publicurl/publicurl.go | 2 +- go.mod | 14 ++-- go.sum | 28 ++++---- plugins/host_artwork_test.go | 7 +- server/auth.go | 12 ++-- server/public/handle_images.go | 19 ++---- server/public/handle_images_test.go | 12 +--- server/public/handle_shares.go | 10 ++- server/public/handle_streams.go | 26 +++----- server/subsonic/middlewares.go | 2 +- 16 files changed, 284 insertions(+), 125 deletions(-) create mode 100644 core/auth/claims.go create mode 100644 core/auth/claims_test.go diff --git a/adapters/deezer/client_auth.go b/adapters/deezer/client_auth.go index d0924b768..eb664c00b 100644 --- a/adapters/deezer/client_auth.go +++ b/adapters/deezer/client_auth.go @@ -10,7 +10,7 @@ import ( "sync" "time" - "github.com/lestrrat-go/jwx/v2/jwt" + "github.com/lestrrat-go/jwx/v3/jwt" "github.com/navidrome/navidrome/log" ) @@ -84,8 +84,8 @@ func (c *client) getJWT(ctx context.Context) (string, error) { } // Calculate TTL with a 1-minute buffer for clock skew and network delays - expiresAt := token.Expiration() - if expiresAt.IsZero() { + expiresAt, ok := token.Expiration() + if !ok || expiresAt.IsZero() { return "", errors.New("deezer: JWT token has no expiration time") } diff --git a/adapters/deezer/client_auth_test.go b/adapters/deezer/client_auth_test.go index 59add7097..005a84e1a 100644 --- a/adapters/deezer/client_auth_test.go +++ b/adapters/deezer/client_auth_test.go @@ -9,7 +9,7 @@ import ( "sync" "time" - "github.com/lestrrat-go/jwx/v2/jwt" + "github.com/lestrrat-go/jwx/v3/jwt" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ) @@ -179,7 +179,8 @@ var _ = Describe("JWT Authentication", func() { Expect(err).To(BeNil()) // Verify token has no expiration - Expect(testToken.Expiration().IsZero()).To(BeTrue()) + _, hasExp := testToken.Expiration() + Expect(hasExp).To(BeFalse()) testJWT, err := jwt.Sign(testToken, jwt.WithInsecureNoSignature()) Expect(err).To(BeNil()) diff --git a/core/auth/auth.go b/core/auth/auth.go index e03820a17..f7ab3ac1b 100644 --- a/core/auth/auth.go +++ b/core/auth/auth.go @@ -4,12 +4,11 @@ import ( "cmp" "context" "crypto/sha256" - "maps" "sync" "time" "github.com/go-chi/jwtauth/v5" - "github.com/lestrrat-go/jwx/v2/jwt" + "github.com/lestrrat-go/jwx/v3/jwt" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/log" @@ -46,38 +45,30 @@ func Init(ds model.DataStore) { }) } -func createBaseClaims() map[string]any { - tokenClaims := map[string]any{} - tokenClaims[jwt.IssuerKey] = consts.JWTIssuer - return tokenClaims -} - -func CreatePublicToken(claims map[string]any) (string, error) { - tokenClaims := createBaseClaims() - maps.Copy(tokenClaims, claims) - _, token, err := TokenAuth.Encode(tokenClaims) - +func CreatePublicToken(claims Claims) (string, error) { + claims.Issuer = consts.JWTIssuer + _, token, err := TokenAuth.Encode(claims.ToMap()) return token, err } -func CreateExpiringPublicToken(exp time.Time, claims map[string]any) (string, error) { - tokenClaims := createBaseClaims() +func CreateExpiringPublicToken(exp time.Time, claims Claims) (string, error) { + claims.Issuer = consts.JWTIssuer if !exp.IsZero() { - tokenClaims[jwt.ExpirationKey] = exp.UTC().Unix() + claims.ExpiresAt = exp } - maps.Copy(tokenClaims, claims) - _, token, err := TokenAuth.Encode(tokenClaims) - + _, token, err := TokenAuth.Encode(claims.ToMap()) return token, err } func CreateToken(u *model.User) (string, error) { - claims := createBaseClaims() - claims[jwt.SubjectKey] = u.UserName - claims[jwt.IssuedAtKey] = time.Now().UTC().Unix() - claims["uid"] = u.ID - claims["adm"] = u.IsAdmin - token, _, err := TokenAuth.Encode(claims) + claims := Claims{ + Issuer: consts.JWTIssuer, + Subject: u.UserName, + IssuedAt: time.Now(), + UserID: u.ID, + IsAdmin: u.IsAdmin, + } + token, _, err := TokenAuth.Encode(claims.ToMap()) if err != nil { return "", err } @@ -86,23 +77,18 @@ func CreateToken(u *model.User) (string, error) { } func TouchToken(token jwt.Token) (string, error) { - claims, err := token.AsMap(context.Background()) - if err != nil { - return "", err - } - - claims[jwt.ExpirationKey] = time.Now().UTC().Add(conf.Server.SessionTimeout).Unix() - _, newToken, err := TokenAuth.Encode(claims) - + claims := ClaimsFromToken(token). + WithExpiresAt(time.Now().UTC().Add(conf.Server.SessionTimeout)) + _, newToken, err := TokenAuth.Encode(claims.ToMap()) return newToken, err } -func Validate(tokenStr string) (map[string]any, error) { +func Validate(tokenStr string) (Claims, error) { token, err := jwtauth.VerifyToken(TokenAuth, tokenStr) if err != nil { - return nil, err + return Claims{}, err } - return token.AsMap(context.Background()) + return ClaimsFromToken(token), nil } func WithAdminUser(ctx context.Context, ds model.DataStore) context.Context { diff --git a/core/auth/auth_test.go b/core/auth/auth_test.go index 38f6820f5..761dd205c 100644 --- a/core/auth/auth_test.go +++ b/core/auth/auth_test.go @@ -54,7 +54,7 @@ var _ = Describe("Auth", func() { decodedClaims, err := auth.Validate(tokenStr) Expect(err).NotTo(HaveOccurred()) - Expect(decodedClaims["iss"]).To(Equal("issuer")) + Expect(decodedClaims.Issuer).To(Equal("issuer")) }) It("returns ErrExpired if the `exp` field is in the past", func() { @@ -82,11 +82,11 @@ var _ = Describe("Auth", func() { claims, err := auth.Validate(tokenStr) Expect(err).NotTo(HaveOccurred()) - Expect(claims["iss"]).To(Equal(consts.JWTIssuer)) - Expect(claims["sub"]).To(Equal("johndoe")) - Expect(claims["uid"]).To(Equal("123")) - Expect(claims["adm"]).To(Equal(true)) - Expect(claims["exp"]).To(BeTemporally(">", time.Now())) + Expect(claims.Issuer).To(Equal(consts.JWTIssuer)) + Expect(claims.Subject).To(Equal("johndoe")) + Expect(claims.UserID).To(Equal("123")) + Expect(claims.IsAdmin).To(Equal(true)) + Expect(claims.ExpiresAt).To(BeTemporally(">", time.Now())) }) }) @@ -104,8 +104,7 @@ var _ = Describe("Auth", func() { decodedClaims, err := auth.Validate(touched) Expect(err).NotTo(HaveOccurred()) - exp := decodedClaims["exp"].(time.Time) - Expect(exp.Sub(yesterday)).To(BeNumerically(">=", oneDay)) + Expect(decodedClaims.ExpiresAt.Sub(yesterday)).To(BeNumerically(">=", oneDay)) }) }) }) diff --git a/core/auth/claims.go b/core/auth/claims.go new file mode 100644 index 000000000..ca496ae9a --- /dev/null +++ b/core/auth/claims.go @@ -0,0 +1,94 @@ +package auth + +import ( + "time" + + "github.com/lestrrat-go/jwx/v3/jwt" +) + +// Claims represents the typed JWT claims used throughout Navidrome, +// replacing the untyped map[string]any approach. +type Claims struct { + // Standard JWT claims + Issuer string + Subject string // username for session tokens + IssuedAt time.Time + ExpiresAt time.Time + + // Custom claims + UserID string // "uid" + IsAdmin bool // "adm" + ID string // "id" - artwork/mediafile ID + Format string // "f" - audio format + BitRate int // "b" - audio bitrate +} + +// ToMap converts Claims to a map[string]any for use with TokenAuth.Encode(). +// Only non-zero fields are included. +func (c Claims) ToMap() map[string]any { + m := make(map[string]any) + if c.Issuer != "" { + m[jwt.IssuerKey] = c.Issuer + } + if c.Subject != "" { + m[jwt.SubjectKey] = c.Subject + } + if !c.IssuedAt.IsZero() { + m[jwt.IssuedAtKey] = c.IssuedAt.UTC().Unix() + } + if !c.ExpiresAt.IsZero() { + m[jwt.ExpirationKey] = c.ExpiresAt.UTC().Unix() + } + if c.UserID != "" { + m["uid"] = c.UserID + } + if c.IsAdmin { + m["adm"] = c.IsAdmin + } + if c.ID != "" { + m["id"] = c.ID + } + if c.Format != "" { + m["f"] = c.Format + } + if c.BitRate != 0 { + m["b"] = c.BitRate + } + return m +} + +func (c Claims) WithExpiresAt(t time.Time) Claims { + c.ExpiresAt = t + return c +} + +// ClaimsFromToken extracts Claims directly from a jwt.Token using token.Get(). +func ClaimsFromToken(token jwt.Token) Claims { + var c Claims + c.Issuer, _ = token.Issuer() + c.Subject, _ = token.Subject() + c.IssuedAt, _ = token.IssuedAt() + c.ExpiresAt, _ = token.Expiration() + + var uid string + if err := token.Get("uid", &uid); err == nil { + c.UserID = uid + } + var adm bool + if err := token.Get("adm", &adm); err == nil { + c.IsAdmin = adm + } + var id string + if err := token.Get("id", &id); err == nil { + c.ID = id + } + var f string + if err := token.Get("f", &f); err == nil { + c.Format = f + } + var b int + if err := token.Get("b", &b); err == nil { + c.BitRate = b + } + return c +} diff --git a/core/auth/claims_test.go b/core/auth/claims_test.go new file mode 100644 index 000000000..cf6b07263 --- /dev/null +++ b/core/auth/claims_test.go @@ -0,0 +1,99 @@ +package auth_test + +import ( + "time" + + "github.com/go-chi/jwtauth/v5" + "github.com/navidrome/navidrome/core/auth" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("Claims", func() { + Describe("ToMap", func() { + It("includes only non-zero fields", func() { + c := auth.Claims{ + Issuer: "ND", + Subject: "johndoe", + UserID: "123", + IsAdmin: true, + } + m := c.ToMap() + Expect(m).To(HaveKeyWithValue("iss", "ND")) + Expect(m).To(HaveKeyWithValue("sub", "johndoe")) + Expect(m).To(HaveKeyWithValue("uid", "123")) + Expect(m).To(HaveKeyWithValue("adm", true)) + Expect(m).NotTo(HaveKey("exp")) + Expect(m).NotTo(HaveKey("iat")) + Expect(m).NotTo(HaveKey("id")) + Expect(m).NotTo(HaveKey("f")) + Expect(m).NotTo(HaveKey("b")) + }) + + It("includes expiration and issued-at when set", func() { + now := time.Now() + c := auth.Claims{ + IssuedAt: now, + ExpiresAt: now.Add(time.Hour), + } + m := c.ToMap() + Expect(m).To(HaveKey("iat")) + Expect(m).To(HaveKey("exp")) + }) + + It("includes custom claims for public tokens", func() { + c := auth.Claims{ + ID: "al-123", + Format: "mp3", + BitRate: 192, + } + m := c.ToMap() + Expect(m).To(HaveKeyWithValue("id", "al-123")) + Expect(m).To(HaveKeyWithValue("f", "mp3")) + Expect(m).To(HaveKeyWithValue("b", 192)) + }) + }) + + Describe("ClaimsFromToken", func() { + It("round-trips session claims through encode/decode", func() { + tokenAuth := jwtauth.New("HS256", []byte("test-secret"), nil) + now := time.Now().Truncate(time.Second) + original := auth.Claims{ + Issuer: "ND", + Subject: "johndoe", + UserID: "123", + IsAdmin: true, + } + m := original.ToMap() + m["iat"] = now.UTC().Unix() + token, _, err := tokenAuth.Encode(m) + Expect(err).NotTo(HaveOccurred()) + + c := auth.ClaimsFromToken(token) + Expect(c.Issuer).To(Equal("ND")) + Expect(c.Subject).To(Equal("johndoe")) + Expect(c.UserID).To(Equal("123")) + Expect(c.IsAdmin).To(BeTrue()) + Expect(c.IssuedAt.UTC()).To(Equal(now.UTC())) + }) + + It("round-trips public token claims through encode/decode", func() { + tokenAuth := jwtauth.New("HS256", []byte("test-secret"), nil) + original := auth.Claims{ + Issuer: "ND", + ID: "al-456", + Format: "opus", + BitRate: 128, + } + token, _, err := tokenAuth.Encode(original.ToMap()) + Expect(err).NotTo(HaveOccurred()) + + c := auth.ClaimsFromToken(token) + Expect(c.Issuer).To(Equal("ND")) + Expect(c.ID).To(Equal("al-456")) + Expect(c.Format).To(Equal("opus")) + Expect(c.BitRate).To(Equal(128)) + }) + }) + +}) diff --git a/core/publicurl/publicurl.go b/core/publicurl/publicurl.go index ff6f4221e..c1b8e01c4 100644 --- a/core/publicurl/publicurl.go +++ b/core/publicurl/publicurl.go @@ -18,7 +18,7 @@ import ( // ImageURL generates a public URL for artwork images. // It creates a signed token for the artwork ID and builds a complete public URL. func ImageURL(req *http.Request, artID model.ArtworkID, size int) string { - token, _ := auth.CreatePublicToken(map[string]any{"id": artID.String()}) + token, _ := auth.CreatePublicToken(auth.Claims{ID: artID.String()}) uri := path.Join(consts.URLPathPublicImages, token) params := url.Values{} if size > 0 { diff --git a/go.mod b/go.mod index c52bb211d..1f6fa30a6 100644 --- a/go.mod +++ b/go.mod @@ -31,7 +31,7 @@ require ( github.com/go-chi/chi/v5 v5.2.5 github.com/go-chi/cors v1.2.2 github.com/go-chi/httprate v0.15.0 - github.com/go-chi/jwtauth/v5 v5.3.3 + github.com/go-chi/jwtauth/v5 v5.4.0 github.com/go-viper/encoding/ini v0.1.1 github.com/gohugoio/hashstructure v0.6.0 github.com/google/go-pipeline v0.0.0-20230411140531-6cbedfc1d3fc @@ -43,7 +43,7 @@ require ( github.com/kardianos/service v1.2.4 github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 github.com/kr/pretty v0.3.1 - github.com/lestrrat-go/jwx/v2 v2.1.6 + github.com/lestrrat-go/jwx/v3 v3.0.13 github.com/maruel/natural v1.3.0 github.com/matoous/go-nanoid/v2 v2.1.0 github.com/mattn/go-sqlite3 v1.14.34 @@ -98,7 +98,7 @@ require ( github.com/goccy/go-json v0.10.5 // indirect github.com/goccy/go-yaml v1.19.2 // indirect github.com/google/go-cmp v0.7.0 // indirect - github.com/google/pprof v0.0.0-20260202012954-cb029daf43ef // indirect + github.com/google/pprof v0.0.0-20260302011040-a15ffb7f9dcc // indirect github.com/google/subcommands v1.2.0 // indirect github.com/gorilla/css v1.0.1 // indirect github.com/hashicorp/errwrap v1.1.0 // indirect @@ -109,10 +109,11 @@ require ( github.com/lann/builder v0.0.0-20180802200727-47ae307949d0 // indirect github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 // indirect github.com/lestrrat-go/blackmagic v1.0.4 // indirect + github.com/lestrrat-go/dsig v1.0.0 // indirect + github.com/lestrrat-go/dsig-secp256k1 v1.0.0 // indirect github.com/lestrrat-go/httpcc v1.0.1 // indirect - github.com/lestrrat-go/httprc v1.0.6 // indirect - github.com/lestrrat-go/iter v1.0.2 // indirect - github.com/lestrrat-go/option v1.0.1 // indirect + github.com/lestrrat-go/httprc/v3 v3.0.4 // indirect + github.com/lestrrat-go/option/v2 v2.0.0 // indirect github.com/mfridman/interpolate v0.0.2 // indirect github.com/mitchellh/go-wordwrap v1.0.1 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect @@ -134,6 +135,7 @@ require ( github.com/stretchr/objx v0.5.3 // indirect github.com/subosito/gotenv v1.6.0 // indirect github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834 // indirect + github.com/valyala/fastjson v1.6.10 // indirect github.com/zeebo/xxh3 v1.1.0 // indirect go.opentelemetry.io/proto/otlp v1.9.0 // indirect go.uber.org/multierr v1.11.0 // indirect diff --git a/go.sum b/go.sum index bbfd51e11..26e4d3925 100644 --- a/go.sum +++ b/go.sum @@ -83,8 +83,8 @@ github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE= github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58= github.com/go-chi/httprate v0.15.0 h1:j54xcWV9KGmPf/X4H32/aTH+wBlrvxL7P+SdnRqxh5g= github.com/go-chi/httprate v0.15.0/go.mod h1:rzGHhVrsBn3IMLYDOZQsSU4fJNWcjui4fWKJcCId1R4= -github.com/go-chi/jwtauth/v5 v5.3.3 h1:50Uzmacu35/ZP9ER2Ht6SazwPsnLQ9LRJy6zTZJpHEo= -github.com/go-chi/jwtauth/v5 v5.3.3/go.mod h1:O4QvPRuZLZghl9WvfVaON+ARfGzpD2PBX/QY5vUz7aQ= +github.com/go-chi/jwtauth/v5 v5.4.0 h1:Ieh0xMJsFvqylqJ02/mQHKzbbKO9DYNBh4DPKCwTwYI= +github.com/go-chi/jwtauth/v5 v5.4.0/go.mod h1:w6yjqUUXz1b8+oiJel64Sz1KJwduQM6qUA5QNzO5+bQ= github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= @@ -110,8 +110,8 @@ github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/go-pipeline v0.0.0-20230411140531-6cbedfc1d3fc h1:hd+uUVsB1vdxohPneMrhGH2YfQuH5hRIK9u4/XCeUtw= github.com/google/go-pipeline v0.0.0-20230411140531-6cbedfc1d3fc/go.mod h1:SL66SJVysrh7YbDCP9tH30b8a9o/N2HeiQNUm85EKhc= -github.com/google/pprof v0.0.0-20260202012954-cb029daf43ef h1:xpF9fUHpoIrrjX24DURVKiwHcFpw19ndIs+FwTSMbno= -github.com/google/pprof v0.0.0-20260202012954-cb029daf43ef/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI= +github.com/google/pprof v0.0.0-20260302011040-a15ffb7f9dcc h1:VBbFa1lDYWEeV5FZKUiYKYT0VxCp9twUmmaq9eb8sXw= +github.com/google/pprof v0.0.0-20260302011040-a15ffb7f9dcc/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI= github.com/google/subcommands v1.2.0 h1:vWQspBTo2nEqTUFita5/KeEWlUL8kQObDFbub/EN9oE= github.com/google/subcommands v1.2.0/go.mod h1:ZjhPrFU+Olkh9WazFPsl27BQ4UPiG37m3yTrtFlrHVk= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= @@ -163,16 +163,18 @@ github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 h1:P6pPBnrTSX3DEVR4fDembhR github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0/go.mod h1:vmVJ0l/dxyfGW6FmdpVm2joNMFikkuWg0EoCKLGUMNw= github.com/lestrrat-go/blackmagic v1.0.4 h1:IwQibdnf8l2KoO+qC3uT4OaTWsW7tuRQXy9TRN9QanA= github.com/lestrrat-go/blackmagic v1.0.4/go.mod h1:6AWFyKNNj0zEXQYfTMPfZrAXUWUfTIZ5ECEUEJaijtw= +github.com/lestrrat-go/dsig v1.0.0 h1:OE09s2r9Z81kxzJYRn07TFM9XA4akrUdoMwr0L8xj38= +github.com/lestrrat-go/dsig v1.0.0/go.mod h1:dEgoOYYEJvW6XGbLasr8TFcAxoWrKlbQvmJgCR0qkDo= +github.com/lestrrat-go/dsig-secp256k1 v1.0.0 h1:JpDe4Aybfl0soBvoVwjqDbp+9S1Y2OM7gcrVVMFPOzY= +github.com/lestrrat-go/dsig-secp256k1 v1.0.0/go.mod h1:CxUgAhssb8FToqbL8NjSPoGQlnO4w3LG1P0qPWQm/NU= github.com/lestrrat-go/httpcc v1.0.1 h1:ydWCStUeJLkpYyjLDHihupbn2tYmZ7m22BGkcvZZrIE= github.com/lestrrat-go/httpcc v1.0.1/go.mod h1:qiltp3Mt56+55GPVCbTdM9MlqhvzyuL6W/NMDA8vA5E= -github.com/lestrrat-go/httprc v1.0.6 h1:qgmgIRhpvBqexMJjA/PmwSvhNk679oqD1RbovdCGW8k= -github.com/lestrrat-go/httprc v1.0.6/go.mod h1:mwwz3JMTPBjHUkkDv/IGJ39aALInZLrhBp0X7KGUZlo= -github.com/lestrrat-go/iter v1.0.2 h1:gMXo1q4c2pHmC3dn8LzRhJfP1ceCbgSiT9lUydIzltI= -github.com/lestrrat-go/iter v1.0.2/go.mod h1:Momfcq3AnRlRjI5b5O8/G5/BvpzrhoFTZcn06fEOPt4= -github.com/lestrrat-go/jwx/v2 v2.1.6 h1:hxM1gfDILk/l5ylers6BX/Eq1m/pnxe9NBwW6lVfecA= -github.com/lestrrat-go/jwx/v2 v2.1.6/go.mod h1:Y722kU5r/8mV7fYDifjug0r8FK8mZdw0K0GpJw/l8pU= -github.com/lestrrat-go/option v1.0.1 h1:oAzP2fvZGQKWkvHa1/SAcFolBEca1oN+mQ7eooNBEYU= -github.com/lestrrat-go/option v1.0.1/go.mod h1:5ZHFbivi4xwXxhxY9XHDe2FHo6/Z7WWmtT7T5nBBp3I= +github.com/lestrrat-go/httprc/v3 v3.0.4 h1:pXyH2ppK8GYYggygxJ3TvxpCZnbEUWc9qSwRTTApaLA= +github.com/lestrrat-go/httprc/v3 v3.0.4/go.mod h1:mSMtkZW92Z98M5YoNNztbRGxbXHql7tSitCvaxvo9l0= +github.com/lestrrat-go/jwx/v3 v3.0.13 h1:AdHKiPIYeCSnOJtvdpipPg/0SuFh9rdkN+HF3O0VdSk= +github.com/lestrrat-go/jwx/v3 v3.0.13/go.mod h1:2m0PV1A9tM4b/jVLMx8rh6rBl7F6WGb3EG2hufN9OQU= +github.com/lestrrat-go/option/v2 v2.0.0 h1:XxrcaJESE1fokHy3FpaQ/cXW8ZsIdWcdFzzLOcID3Ss= +github.com/lestrrat-go/option/v2 v2.0.0/go.mod h1:oSySsmzMoR0iRzCDCaUfsCzxQHUEuhOViQObyy7S6Vg= github.com/maruel/natural v1.3.0 h1:VsmCsBmEyrR46RomtgHs5hbKADGRVtliHTyCOLFBpsg= github.com/maruel/natural v1.3.0/go.mod h1:v+Rfd79xlw1AgVBjbO0BEQmptqb5HvL/k9GRHB7ZKEg= github.com/matoous/go-nanoid/v2 v2.1.0 h1:P64+dmq21hhWdtvZfEAofnvJULaRR1Yib0+PnU669bE= @@ -296,6 +298,8 @@ github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY= github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28= github.com/unrolled/secure v1.17.0 h1:Io7ifFgo99Bnh0J7+Q+qcMzWM6kaDPCA5FroFZEdbWU= github.com/unrolled/secure v1.17.0/go.mod h1:BmF5hyM6tXczk3MpQkFf1hpKSRqCyhqcbiQtiAF7+40= +github.com/valyala/fastjson v1.6.10 h1:/yjJg8jaVQdYR3arGxPE2X5z89xrlhS0eGXdv+ADTh4= +github.com/valyala/fastjson v1.6.10/go.mod h1:e6FubmQouUNP73jtMLmcbxS6ydWIpOfhz34TSfO3JaE= github.com/xrash/smetrics v0.0.0-20250705151800-55b8f293f342 h1:FnBeRrxr7OU4VvAzt5X7s6266i6cSVkkFPS0TuXWbIg= github.com/xrash/smetrics v0.0.0-20250705151800-55b8f293f342/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM= github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= diff --git a/plugins/host_artwork_test.go b/plugins/host_artwork_test.go index 5e3c54a80..b97e2684d 100644 --- a/plugins/host_artwork_test.go +++ b/plugins/host_artwork_test.go @@ -229,11 +229,10 @@ func decodeArtworkURL(artworkURL string) model.ArtworkID { token, err := auth.TokenAuth.Decode(tokenPart) Expect(err).ToNot(HaveOccurred(), "Failed to decode JWT token") - claims, err := token.AsMap(context.Background()) - Expect(err).ToNot(HaveOccurred(), "Failed to get claims from token") + c := auth.ClaimsFromToken(token) - id, ok := claims["id"].(string) - Expect(ok).To(BeTrue(), "Token should contain 'id' claim") + id := c.ID + Expect(id).ToNot(BeEmpty(), "Token should contain 'id' claim") artID, err := model.ParseArtworkID(id) Expect(err).ToNot(HaveOccurred(), "Failed to parse artwork ID from token") diff --git a/server/auth.go b/server/auth.go index 86e63722b..a7edaab0a 100644 --- a/server/auth.go +++ b/server/auth.go @@ -185,12 +185,16 @@ func tokenFromHeader(r *http.Request) string { } func UsernameFromToken(r *http.Request) string { - token, claims, err := jwtauth.FromContext(r.Context()) - if err != nil || claims["sub"] == nil || token == nil { + token, _, err := jwtauth.FromContext(r.Context()) + if err != nil || token == nil { return "" } - log.Trace(r, "Found username in JWT token", "username", token.Subject()) - return token.Subject() + sub, _ := token.Subject() + if sub == "" { + return "" + } + log.Trace(r, "Found username in JWT token", "username", sub) + return sub } func UsernameFromExtAuthHeader(r *http.Request) string { diff --git a/server/public/handle_images.go b/server/public/handle_images.go index 5b1194cc9..f4985dea5 100644 --- a/server/public/handle_images.go +++ b/server/public/handle_images.go @@ -7,7 +7,6 @@ import ( "net/http" "time" - "github.com/lestrrat-go/jwx/v2/jwt" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/auth" "github.com/navidrome/navidrome/log" @@ -76,22 +75,14 @@ func decodeArtworkID(tokenString string) (model.ArtworkID, error) { if token == nil { return model.ArtworkID{}, errors.New("unauthorized") } - err = jwt.Validate(token, jwt.WithRequiredClaim("id")) - if err != nil { - return model.ArtworkID{}, err + c := auth.ClaimsFromToken(token) + if c.ID == "" { + return model.ArtworkID{}, errors.New("required claim \"id\" not found") } - claims, err := token.AsMap(context.Background()) - if err != nil { - return model.ArtworkID{}, err - } - id, ok := claims["id"].(string) - if !ok { - return model.ArtworkID{}, errors.New("invalid id type") - } - artID, err := model.ParseArtworkID(id) + artID, err := model.ParseArtworkID(c.ID) if err == nil { return artID, nil } // Try to default to mediafile artworkId (if used with a mediafileShare token) - return model.ParseArtworkID("mf-" + id) + return model.ParseArtworkID("mf-" + c.ID) } diff --git a/server/public/handle_images_test.go b/server/public/handle_images_test.go index 0995f4f61..6895241f6 100644 --- a/server/public/handle_images_test.go +++ b/server/public/handle_images_test.go @@ -3,7 +3,6 @@ package public import ( "github.com/go-chi/jwtauth/v5" "github.com/navidrome/navidrome/core/auth" - "github.com/navidrome/navidrome/model" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ) @@ -15,18 +14,11 @@ var _ = Describe("decodeArtworkID", func() { It("fails to decode an invalid token", func() { _, err := decodeArtworkID("xx-123") - Expect(err).To(MatchError("invalid JWT")) - }) - - It("defaults to kind mediafile for empty artwork ID", func() { - token, _ := auth.CreatePublicToken(map[string]any{"id": ""}) - id, err := decodeArtworkID(token) - Expect(err).ToNot(HaveOccurred()) - Expect(id.Kind).To(Equal(model.KindMediaFileArtwork)) + Expect(err).To(HaveOccurred()) }) It("fails to decode a token without an id", func() { - token, _ := auth.CreatePublicToken(map[string]any{}) + token, _ := auth.CreatePublicToken(auth.Claims{}) _, err := decodeArtworkID(token) Expect(err).To(HaveOccurred()) }) diff --git a/server/public/handle_shares.go b/server/public/handle_shares.go index 36764dece..15e63d4db 100644 --- a/server/public/handle_shares.go +++ b/server/public/handle_shares.go @@ -97,12 +97,10 @@ func (pub *Router) mapShareToM3U(r *http.Request, s model.Share) *model.Share { } func encodeMediafileShare(s model.Share, id string) string { - claims := map[string]any{"id": id} - if s.Format != "" { - claims["f"] = s.Format - } - if s.MaxBitRate != 0 { - claims["b"] = s.MaxBitRate + claims := auth.Claims{ + ID: id, + Format: s.Format, + BitRate: s.MaxBitRate, } token, _ := auth.CreateExpiringPublicToken(V(s.ExpiresAt), claims) return token diff --git a/server/public/handle_streams.go b/server/public/handle_streams.go index cf120f0b5..d6819974b 100644 --- a/server/public/handle_streams.go +++ b/server/public/handle_streams.go @@ -1,13 +1,11 @@ package public import ( - "context" "errors" "io" "net/http" "strconv" - "github.com/lestrrat-go/jwx/v2/jwt" "github.com/navidrome/navidrome/core/auth" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/utils/req" @@ -85,21 +83,13 @@ func decodeStreamInfo(tokenString string) (shareTrackInfo, error) { if token == nil { return shareTrackInfo{}, errors.New("unauthorized") } - err = jwt.Validate(token, jwt.WithRequiredClaim("id")) - if err != nil { - return shareTrackInfo{}, err + c := auth.ClaimsFromToken(token) + if c.ID == "" { + return shareTrackInfo{}, errors.New("required claim \"id\" not found") } - claims, err := token.AsMap(context.Background()) - if err != nil { - return shareTrackInfo{}, err - } - id, ok := claims["id"].(string) - if !ok { - return shareTrackInfo{}, errors.New("invalid id type") - } - resp := shareTrackInfo{} - resp.id = id - resp.format, _ = claims["f"].(string) - resp.bitrate, _ = claims["b"].(int) - return resp, nil + return shareTrackInfo{ + id: c.ID, + format: c.Format, + bitrate: c.BitRate, + }, nil } diff --git a/server/subsonic/middlewares.go b/server/subsonic/middlewares.go index d984bac42..7698a3c7b 100644 --- a/server/subsonic/middlewares.go +++ b/server/subsonic/middlewares.go @@ -159,7 +159,7 @@ func validateCredentials(user *model.User, pass, token, salt, jwt string) error switch { case jwt != "": claims, err := auth.Validate(jwt) - valid = err == nil && claims["sub"] == user.UserName + valid = err == nil && claims.Subject == user.UserName case pass != "": if strings.HasPrefix(pass, "enc:") { if dec, err := hex.DecodeString(pass[4:]); err == nil { From 30df004d4d8916a3d11c14a188c03585d0f28233 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 2 Mar 2026 16:18:30 -0500 Subject: [PATCH 30/60] test(plugins): speed up integration tests (~45% improvement) (#5137) * test(plugins): speed up integration tests with shared wazero cache Reduce plugin test suite runtime from ~22s to ~12s by: - Creating a shared wazero compilation cache directory in TestPlugins() and setting conf.Server.CacheFolder globally so all test Manager instances reuse compiled WASM binaries from disk cache - Moving 6 createTestManager* calls from inside It blocks to BeforeAll blocks in scrobbler_adapter_test.go and manager_call_test.go - Replacing time.Sleep(2s) in KVStore TTL test with Eventually polling - Reducing WebSocket callback sleeps from 100ms to 10ms Signed-off-by: Deluan * test(plugins): enhance websocket tests by storing server messages for verification Signed-off-by: Deluan --------- Signed-off-by: Deluan --- plugins/host_artwork_test.go | 1 - plugins/host_cache_test.go | 1 - plugins/host_config_test.go | 1 - plugins/host_kvstore_test.go | 20 ++-- plugins/host_library_test.go | 2 - plugins/host_scheduler_test.go | 1 - plugins/host_subsonicapi_test.go | 1 - plugins/host_users_test.go | 1 - plugins/host_websocket_test.go | 52 ++++------- plugins/manager_call_test.go | 84 ++++++++++------- plugins/plugins_suite_test.go | 15 ++- plugins/scrobbler_adapter_test.go | 116 +++++++++++++++--------- plugins/testdata/test-websocket/main.go | 5 +- 13 files changed, 165 insertions(+), 135 deletions(-) diff --git a/plugins/host_artwork_test.go b/plugins/host_artwork_test.go index b97e2684d..151a0d03c 100644 --- a/plugins/host_artwork_test.go +++ b/plugins/host_artwork_test.go @@ -49,7 +49,6 @@ var _ = Describe("ArtworkService", Ordered, func() { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Initialize auth (required for token generation) ds := &tests.MockDataStore{MockedProperty: &tests.MockedPropertyRepo{}} diff --git a/plugins/host_cache_test.go b/plugins/host_cache_test.go index ec225c1c2..0f55bcfda 100644 --- a/plugins/host_cache_test.go +++ b/plugins/host_cache_test.go @@ -345,7 +345,6 @@ var _ = Describe("CacheService Integration", Ordered, func() { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Setup mock DataStore with pre-enabled plugin mockPluginRepo := tests.CreateMockPluginRepo() diff --git a/plugins/host_config_test.go b/plugins/host_config_test.go index 5ad5af198..bd3368a67 100644 --- a/plugins/host_config_test.go +++ b/plugins/host_config_test.go @@ -59,7 +59,6 @@ func setupTestConfigPlugin(configJSON string) (*Manager, func(context.Context, t conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Setup mock DataStore mockPluginRepo := tests.CreateMockPluginRepo() diff --git a/plugins/host_kvstore_test.go b/plugins/host_kvstore_test.go index b900a659a..4928825ef 100644 --- a/plugins/host_kvstore_test.go +++ b/plugins/host_kvstore_test.go @@ -677,7 +677,6 @@ var _ = Describe("KVStoreService Integration", Ordered, func() { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") conf.Server.DataFolder = tmpDir // Setup mock DataStore with pre-enabled plugin @@ -924,16 +923,15 @@ var _ = Describe("KVStoreService Integration", Ordered, func() { Expect(output.Exists).To(BeTrue()) Expect(output.Value).To(Equal([]byte("temporary"))) - // Wait for expiration - time.Sleep(2 * time.Second) - - // Should no longer exist - output, err = callTestKVStore(ctx, testKVStoreInput{ - Operation: "get", - Key: "ttl_key", - }) - Expect(err).ToNot(HaveOccurred()) - Expect(output.Exists).To(BeFalse()) + // Poll until the key expires (1s TTL) + Eventually(func(g Gomega) { + output, err := callTestKVStore(ctx, testKVStoreInput{ + Operation: "get", + Key: "ttl_key", + }) + g.Expect(err).ToNot(HaveOccurred()) + g.Expect(output.Exists).To(BeFalse()) + }).WithTimeout(3 * time.Second).WithPolling(200 * time.Millisecond).Should(Succeed()) }) It("should delete keys by prefix", func() { diff --git a/plugins/host_library_test.go b/plugins/host_library_test.go index 413fc81c8..d92abfe30 100644 --- a/plugins/host_library_test.go +++ b/plugins/host_library_test.go @@ -264,7 +264,6 @@ var _ = Describe("LibraryService", Ordered, func() { DeferCleanup(configtest.SetupConfig()) conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Create mock &tests.MockLibraryRepo{} mockLibRepo := &tests.MockLibraryRepo{} @@ -360,7 +359,6 @@ var _ = Describe("LibraryService Integration", Ordered, func() { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Setup mock DataStore with pre-enabled plugin and library mockPluginRepo := tests.CreateMockPluginRepo() diff --git a/plugins/host_scheduler_test.go b/plugins/host_scheduler_test.go index 51311f1c4..334d9b738 100644 --- a/plugins/host_scheduler_test.go +++ b/plugins/host_scheduler_test.go @@ -53,7 +53,6 @@ var _ = Describe("SchedulerService", Ordered, func() { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Create mock scheduler and timer registry mockSched = newMockScheduler() diff --git a/plugins/host_subsonicapi_test.go b/plugins/host_subsonicapi_test.go index b0589fa12..607f3a64b 100644 --- a/plugins/host_subsonicapi_test.go +++ b/plugins/host_subsonicapi_test.go @@ -46,7 +46,6 @@ var _ = Describe("SubsonicAPI Host Function", Ordered, func() { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Setup mock router and data store router = &fakeSubsonicRouter{} diff --git a/plugins/host_users_test.go b/plugins/host_users_test.go index 2071a9320..1c0de7d03 100644 --- a/plugins/host_users_test.go +++ b/plugins/host_users_test.go @@ -486,7 +486,6 @@ func setupTestUsersConfig(tmpDir string) { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") } // testUsersInput represents input for test-users plugin calls diff --git a/plugins/host_websocket_test.go b/plugins/host_websocket_test.go index 7a0439129..e8cb9f8fd 100644 --- a/plugins/host_websocket_test.go +++ b/plugins/host_websocket_test.go @@ -14,7 +14,6 @@ import ( "path/filepath" "strings" "sync" - "time" "github.com/gorilla/websocket" "github.com/navidrome/navidrome/conf" @@ -54,7 +53,6 @@ var _ = Describe("WebSocketService", Ordered, func() { conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Setup mock DataStore with pre-enabled plugin mockPluginRepo := tests.CreateMockPluginRepo() @@ -295,10 +293,12 @@ var _ = Describe("WebSocketService", Ordered, func() { Describe("Plugin Callbacks", func() { var wsServer *httptest.Server var serverConn *websocket.Conn + var serverMessages []string var serverMu sync.Mutex BeforeEach(func() { serverConn = nil + serverMessages = nil upgrader := websocket.Upgrader{ CheckOrigin: func(r *http.Request) bool { return true }, @@ -312,12 +312,15 @@ var _ = Describe("WebSocketService", Ordered, func() { serverConn = conn serverMu.Unlock() - // Keep connection open + // Read and store messages for { - _, _, err := conn.ReadMessage() + _, msg, err := conn.ReadMessage() if err != nil { break } + serverMu.Lock() + serverMessages = append(serverMessages, string(msg)) + serverMu.Unlock() } })) @@ -336,36 +339,10 @@ var _ = Describe("WebSocketService", Ordered, func() { } }) - It("should invoke OnTextMessage callback when receiving text", func() { - ctx := GinkgoT().Context() - wsURL := "ws://" + strings.TrimPrefix(wsServer.URL, "http://") - connID, err := testService.Connect(ctx, wsURL, nil, "text-cb-conn") - Expect(err).ToNot(HaveOccurred()) - - // Wait for server to have the connection - Eventually(func() *websocket.Conn { - serverMu.Lock() - defer serverMu.Unlock() - return serverConn - }).ShouldNot(BeNil()) - - // Send message from server to plugin - serverMu.Lock() - err = serverConn.WriteMessage(websocket.TextMessage, []byte("test message")) - serverMu.Unlock() - Expect(err).ToNot(HaveOccurred()) - - // The plugin should have received the callback - // We can verify by checking the plugin's stored messages via vars - // For now we just verify no errors occurred - time.Sleep(100 * time.Millisecond) - _ = connID - }) - It("should invoke OnBinaryMessage callback when receiving binary", func() { ctx := GinkgoT().Context() wsURL := "ws://" + strings.TrimPrefix(wsServer.URL, "http://") - connID, err := testService.Connect(ctx, wsURL, nil, "binary-cb-conn") + _, err := testService.Connect(ctx, wsURL, nil, "binary-cb-conn") Expect(err).ToNot(HaveOccurred()) // Wait for server to have the connection @@ -382,9 +359,13 @@ var _ = Describe("WebSocketService", Ordered, func() { serverMu.Unlock() Expect(err).ToNot(HaveOccurred()) - // Give time for callback to execute - time.Sleep(100 * time.Millisecond) - _ = connID + // Plugin echoes binary data back as text prefixed with "binary_echo:" + expectedEcho := "binary_echo:" + base64.StdEncoding.EncodeToString(binaryData) + Eventually(func() []string { + serverMu.Lock() + defer serverMu.Unlock() + return serverMessages + }).Should(ContainElement(expectedEcho)) }) It("should invoke OnClose callback when server closes connection", func() { @@ -466,7 +447,7 @@ var _ = Describe("WebSocketService", Ordered, func() { It("should allow plugin to send messages via host function", func() { ctx := GinkgoT().Context() wsURL := "ws://" + strings.TrimPrefix(wsServer.URL, "http://") - connID, err := testService.Connect(ctx, wsURL, nil, "host-send-conn") + _, err := testService.Connect(ctx, wsURL, nil, "host-send-conn") Expect(err).ToNot(HaveOccurred()) // Wait for server to have the connection @@ -488,7 +469,6 @@ var _ = Describe("WebSocketService", Ordered, func() { defer serverMu.Unlock() return serverMessages }).Should(ContainElement("echo:echo")) - _ = connID }) It("should allow plugin to close connection via host function", func() { diff --git a/plugins/manager_call_test.go b/plugins/manager_call_test.go index 8865e1266..3e64f1cee 100644 --- a/plugins/manager_call_test.go +++ b/plugins/manager_call_test.go @@ -81,48 +81,66 @@ var _ = Describe("callPluginFunction metrics", Ordered, func() { Expect(calls[0].elapsed).To(BeNumerically(">=", 0)) }) - It("records metrics for failed plugin calls (error returned)", func() { - // Create a manager with error config to force plugin errors - errorRecorder := &mockMetricsRecorder{} - errorManager, _ := createTestManagerWithPluginsAndMetrics( - map[string]map[string]string{ - "test-metadata-agent": {"error": "simulated error"}, - }, - errorRecorder, - "test-metadata-agent"+PackageExtension, + Context("with error config", Ordered, func() { + var ( + errorRecorder *mockMetricsRecorder + errorAgent agents.Interface ) - errorAgent, ok := errorManager.LoadMediaAgent("test-metadata-agent") - Expect(ok).To(BeTrue()) + BeforeAll(func() { + errorRecorder = &mockMetricsRecorder{} + errorManager, _ := createTestManagerWithPluginsAndMetrics( + map[string]map[string]string{ + "test-metadata-agent": {"error": "simulated error"}, + }, + errorRecorder, + "test-metadata-agent"+PackageExtension, + ) - retriever := errorAgent.(agents.ArtistBiographyRetriever) - _, err := retriever.GetArtistBiography(GinkgoT().Context(), "artist-1", "Test Artist", "mbid") - Expect(err).To(HaveOccurred()) + var ok bool + errorAgent, ok = errorManager.LoadMediaAgent("test-metadata-agent") + Expect(ok).To(BeTrue()) + }) - calls := errorRecorder.getCalls() - Expect(calls).To(HaveLen(1)) - Expect(calls[0].plugin).To(Equal("test-metadata-agent")) - Expect(calls[0].method).To(Equal(FuncGetArtistBiography)) - Expect(calls[0].ok).To(BeFalse()) + It("records metrics for failed plugin calls (error returned)", func() { + retriever := errorAgent.(agents.ArtistBiographyRetriever) + _, err := retriever.GetArtistBiography(GinkgoT().Context(), "artist-1", "Test Artist", "mbid") + Expect(err).To(HaveOccurred()) + + calls := errorRecorder.getCalls() + Expect(calls).To(HaveLen(1)) + Expect(calls[0].plugin).To(Equal("test-metadata-agent")) + Expect(calls[0].method).To(Equal(FuncGetArtistBiography)) + Expect(calls[0].ok).To(BeFalse()) + }) }) - It("does not record metrics for not-implemented functions", func() { - // Use partial metadata agent that doesn't implement GetArtistMBID - partialRecorder := &mockMetricsRecorder{} - partialManager, _ := createTestManagerWithPluginsAndMetrics( - nil, - partialRecorder, - "partial-metadata-agent"+PackageExtension, + Context("with partial metadata agent", Ordered, func() { + var ( + partialRecorder *mockMetricsRecorder + partialAgent agents.Interface ) - partialAgent, ok := partialManager.LoadMediaAgent("partial-metadata-agent") - Expect(ok).To(BeTrue()) + BeforeAll(func() { + partialRecorder = &mockMetricsRecorder{} + partialManager, _ := createTestManagerWithPluginsAndMetrics( + nil, + partialRecorder, + "partial-metadata-agent"+PackageExtension, + ) - retriever := partialAgent.(agents.ArtistMBIDRetriever) - _, err := retriever.GetArtistMBID(GinkgoT().Context(), "artist-1", "Test Artist") - Expect(err).To(MatchError(errNotImplemented)) + var ok bool + partialAgent, ok = partialManager.LoadMediaAgent("partial-metadata-agent") + Expect(ok).To(BeTrue()) + }) - calls := partialRecorder.getCalls() - Expect(calls).To(HaveLen(0)) + It("does not record metrics for not-implemented functions", func() { + retriever := partialAgent.(agents.ArtistMBIDRetriever) + _, err := retriever.GetArtistMBID(GinkgoT().Context(), "artist-1", "Test Artist") + Expect(err).To(MatchError(errNotImplemented)) + + calls := partialRecorder.getCalls() + Expect(calls).To(HaveLen(0)) + }) }) }) diff --git a/plugins/plugins_suite_test.go b/plugins/plugins_suite_test.go index 3b5e610d6..1799ba3ce 100644 --- a/plugins/plugins_suite_test.go +++ b/plugins/plugins_suite_test.go @@ -36,6 +36,20 @@ var ( func TestPlugins(t *testing.T) { tests.Init(t, false) buildTestPlugins(t, testDataDir) + + // Create a shared wazero compilation cache directory. + // All test managers will point CacheFolder here so that WASM compilation + // is done once per binary and then reused from disk cache. + sharedCacheDir, err := os.MkdirTemp("", "plugins-shared-cache-*") + if err != nil { + t.Fatalf("Failed to create shared cache dir: %v", err) + } + t.Cleanup(func() { os.RemoveAll(sharedCacheDir) }) + + // Set CacheFolder globally so all tests (including those using + // configtest.SetupConfig) inherit it without needing to set it manually. + conf.Server.CacheFolder = sharedCacheDir + log.SetLevel(log.LevelFatal) RegisterFailHandler(Fail) RunSpecs(t, "Plugins Suite") @@ -114,7 +128,6 @@ func createTestManagerWithPluginsAndMetrics(pluginConfig map[string]map[string]s conf.Server.Plugins.Enabled = true conf.Server.Plugins.Folder = tmpDir conf.Server.Plugins.AutoReload = false - conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") // Setup mock DataStore with pre-enabled plugins mockPluginRepo := tests.CreateMockPluginRepo() diff --git a/plugins/scrobbler_adapter_test.go b/plugins/scrobbler_adapter_test.go index 05fc11757..ab8dc6f88 100644 --- a/plugins/scrobbler_adapter_test.go +++ b/plugins/scrobbler_adapter_test.go @@ -58,16 +58,23 @@ var _ = Describe("ScrobblerPlugin", Ordered, func() { Expect(result).To(BeTrue()) }) - It("returns false when plugin is configured to not authorize", func() { - manager, _ := createTestManagerWithPlugins(map[string]map[string]string{ - "test-scrobbler": {"authorized": "false"}, - }, "test-scrobbler"+PackageExtension) + Context("when plugin is configured to not authorize", Ordered, func() { + var notAuthScrobbler scrobbler.Scrobbler - sc, ok := manager.LoadScrobbler("test-scrobbler") - Expect(ok).To(BeTrue()) + BeforeAll(func() { + mgr, _ := createTestManagerWithPlugins(map[string]map[string]string{ + "test-scrobbler": {"authorized": "false"}, + }, "test-scrobbler"+PackageExtension) - result := sc.IsAuthorized(ctxWithUser(), "user-1") - Expect(result).To(BeFalse()) + var ok bool + notAuthScrobbler, ok = mgr.LoadScrobbler("test-scrobbler") + Expect(ok).To(BeTrue()) + }) + + It("returns false", func() { + result := notAuthScrobbler.IsAuthorized(ctxWithUser(), "user-1") + Expect(result).To(BeFalse()) + }) }) }) @@ -127,18 +134,25 @@ var _ = Describe("ScrobblerPlugin", Ordered, func() { Expect(err).ToNot(HaveOccurred()) }) - It("returns error when plugin returns error", func() { - manager, _ := createTestManagerWithPlugins(map[string]map[string]string{ - "test-scrobbler": {"error": "service unavailable", "error_type": "scrobbler(retry_later)"}, - }, "test-scrobbler"+PackageExtension) + Context("when plugin returns error", Ordered, func() { + var retryScrobbler scrobbler.Scrobbler - sc, ok := manager.LoadScrobbler("test-scrobbler") - Expect(ok).To(BeTrue()) + BeforeAll(func() { + mgr, _ := createTestManagerWithPlugins(map[string]map[string]string{ + "test-scrobbler": {"error": "service unavailable", "error_type": "scrobbler(retry_later)"}, + }, "test-scrobbler"+PackageExtension) - track := &model.MediaFile{ID: "track-1", Title: "Test Song"} - err := sc.NowPlaying(ctxWithUser(), "user-1", track, 30) - Expect(err).To(HaveOccurred()) - Expect(err).To(MatchError(scrobbler.ErrRetryLater)) + var ok bool + retryScrobbler, ok = mgr.LoadScrobbler("test-scrobbler") + Expect(ok).To(BeTrue()) + }) + + It("returns ErrRetryLater", func() { + track := &model.MediaFile{ID: "track-1", Title: "Test Song"} + err := retryScrobbler.NowPlaying(ctxWithUser(), "user-1", track, 30) + Expect(err).To(HaveOccurred()) + Expect(err).To(MatchError(scrobbler.ErrRetryLater)) + }) }) }) @@ -166,38 +180,52 @@ var _ = Describe("ScrobblerPlugin", Ordered, func() { Expect(err).ToNot(HaveOccurred()) }) - It("returns error when plugin returns not_authorized error", func() { - manager, _ := createTestManagerWithPlugins(map[string]map[string]string{ - "test-scrobbler": {"error": "user not linked", "error_type": "scrobbler(not_authorized)"}, - }, "test-scrobbler"+PackageExtension) + Context("when plugin returns not_authorized error", Ordered, func() { + var notAuthScrobbler scrobbler.Scrobbler - sc, ok := manager.LoadScrobbler("test-scrobbler") - Expect(ok).To(BeTrue()) + BeforeAll(func() { + mgr, _ := createTestManagerWithPlugins(map[string]map[string]string{ + "test-scrobbler": {"error": "user not linked", "error_type": "scrobbler(not_authorized)"}, + }, "test-scrobbler"+PackageExtension) - scrobble := scrobbler.Scrobble{ - MediaFile: model.MediaFile{ID: "track-1", Title: "Test Song"}, - TimeStamp: time.Now(), - } - err := sc.Scrobble(ctxWithUser(), "user-1", scrobble) - Expect(err).To(HaveOccurred()) - Expect(err).To(MatchError(scrobbler.ErrNotAuthorized)) + var ok bool + notAuthScrobbler, ok = mgr.LoadScrobbler("test-scrobbler") + Expect(ok).To(BeTrue()) + }) + + It("returns ErrNotAuthorized", func() { + scrobble := scrobbler.Scrobble{ + MediaFile: model.MediaFile{ID: "track-1", Title: "Test Song"}, + TimeStamp: time.Now(), + } + err := notAuthScrobbler.Scrobble(ctxWithUser(), "user-1", scrobble) + Expect(err).To(HaveOccurred()) + Expect(err).To(MatchError(scrobbler.ErrNotAuthorized)) + }) }) - It("returns error when plugin returns unrecoverable error", func() { - manager, _ := createTestManagerWithPlugins(map[string]map[string]string{ - "test-scrobbler": {"error": "track rejected", "error_type": "scrobbler(unrecoverable)"}, - }, "test-scrobbler"+PackageExtension) + Context("when plugin returns unrecoverable error", Ordered, func() { + var unrecoverableScrobbler scrobbler.Scrobbler - sc, ok := manager.LoadScrobbler("test-scrobbler") - Expect(ok).To(BeTrue()) + BeforeAll(func() { + mgr, _ := createTestManagerWithPlugins(map[string]map[string]string{ + "test-scrobbler": {"error": "track rejected", "error_type": "scrobbler(unrecoverable)"}, + }, "test-scrobbler"+PackageExtension) - scrobble := scrobbler.Scrobble{ - MediaFile: model.MediaFile{ID: "track-1", Title: "Test Song"}, - TimeStamp: time.Now(), - } - err := sc.Scrobble(ctxWithUser(), "user-1", scrobble) - Expect(err).To(HaveOccurred()) - Expect(err).To(MatchError(scrobbler.ErrUnrecoverable)) + var ok bool + unrecoverableScrobbler, ok = mgr.LoadScrobbler("test-scrobbler") + Expect(ok).To(BeTrue()) + }) + + It("returns ErrUnrecoverable", func() { + scrobble := scrobbler.Scrobble{ + MediaFile: model.MediaFile{ID: "track-1", Title: "Test Song"}, + TimeStamp: time.Now(), + } + err := unrecoverableScrobbler.Scrobble(ctxWithUser(), "user-1", scrobble) + Expect(err).To(HaveOccurred()) + Expect(err).To(MatchError(scrobbler.ErrUnrecoverable)) + }) }) }) diff --git a/plugins/testdata/test-websocket/main.go b/plugins/testdata/test-websocket/main.go index 7e2683e4d..b85eaa358 100644 --- a/plugins/testdata/test-websocket/main.go +++ b/plugins/testdata/test-websocket/main.go @@ -45,10 +45,11 @@ func (t *testWebSocket) OnTextMessage(input websocket.OnTextMessageRequest) erro } // OnBinaryMessage is called when a binary message is received. +// Echoes the data back as a text message prefixed with "binary_echo:" so tests +// can observe the callback fired. func (t *testWebSocket) OnBinaryMessage(input websocket.OnBinaryMessageRequest) error { - // Store received binary data for test verification storeReceivedMessage("binary:" + input.Data) - return nil + return host.WebSocketSendText(input.ConnectionID, "binary_echo:"+input.Data) } // OnError is called when an error occurs on a WebSocket connection. From 6fd044fb09f0ba9e8148ac0d519b78f477a9b653 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 2 Mar 2026 16:38:00 -0500 Subject: [PATCH 31/60] feat(plugins): change websockets Data field type to []byte for binary support Signed-off-by: Deluan --- plugins/capabilities/websocket_callback.go | 2 +- plugins/capabilities/websocket_callback.yaml | 1 + plugins/cmd/ndpgen/internal/xtp_schema.go | 8 +++- .../cmd/ndpgen/internal/xtp_schema_test.go | 39 +++++++++++++++++++ plugins/host_websocket.go | 3 +- plugins/host_websocket_test.go | 24 ++++++------ plugins/pdk/go/websocket/websocket.go | 2 +- plugins/pdk/go/websocket/websocket_stub.go | 2 +- .../rust/nd-pdk-capabilities/src/websocket.rs | 26 ++++++++++++- plugins/testdata/test-websocket/main.go | 9 +++-- 10 files changed, 94 insertions(+), 22 deletions(-) diff --git a/plugins/capabilities/websocket_callback.go b/plugins/capabilities/websocket_callback.go index 07db029f0..ddfc0fc95 100644 --- a/plugins/capabilities/websocket_callback.go +++ b/plugins/capabilities/websocket_callback.go @@ -38,7 +38,7 @@ type OnBinaryMessageRequest struct { // ConnectionID is the unique identifier for the WebSocket connection that received the message. ConnectionID string `json:"connectionId"` // Data is the binary data received from the WebSocket, encoded as base64. - Data string `json:"data"` + Data []byte `json:"data"` } // OnErrorRequest is the request provided when an error occurs on a WebSocket connection. diff --git a/plugins/capabilities/websocket_callback.yaml b/plugins/capabilities/websocket_callback.yaml index 401c77bb4..6cd0cff9f 100644 --- a/plugins/capabilities/websocket_callback.yaml +++ b/plugins/capabilities/websocket_callback.yaml @@ -30,6 +30,7 @@ components: description: ConnectionID is the unique identifier for the WebSocket connection that received the message. data: type: string + format: byte description: Data is the binary data received from the WebSocket, encoded as base64. required: - connectionId diff --git a/plugins/cmd/ndpgen/internal/xtp_schema.go b/plugins/cmd/ndpgen/internal/xtp_schema.go index db30262cc..cc2a7d0e0 100644 --- a/plugins/cmd/ndpgen/internal/xtp_schema.go +++ b/plugins/cmd/ndpgen/internal/xtp_schema.go @@ -246,6 +246,12 @@ func buildProperty(field FieldDef, knownTypes map[string]bool) xtpProperty { return prop } + // Handle primitive types (including []byte which maps to string/byte, not array) + if isPrimitiveGoType(goType) { + prop.Type, prop.Format = goTypeToXTPTypeAndFormat(goType) + return prop + } + // Handle slice types if strings.HasPrefix(goType, "[]") { elemType := goType[2:] @@ -259,7 +265,7 @@ func buildProperty(field FieldDef, knownTypes map[string]bool) xtpProperty { return prop } - // Handle primitive types + // Handle remaining types prop.Type, prop.Format = goTypeToXTPTypeAndFormat(goType) return prop } diff --git a/plugins/cmd/ndpgen/internal/xtp_schema_test.go b/plugins/cmd/ndpgen/internal/xtp_schema_test.go index 5e8a132f2..2e28a75d8 100644 --- a/plugins/cmd/ndpgen/internal/xtp_schema_test.go +++ b/plugins/cmd/ndpgen/internal/xtp_schema_test.go @@ -303,6 +303,45 @@ var _ = Describe("XTP Schema Generation", func() { }) }) + Context("capability with []byte field", func() { + It("should map []byte to string with byte format, not array", func() { + capability := Capability{ + Name: "byte_test", + SourceFile: "byte_test", + Methods: []Export{ + {ExportName: "test", Input: NewParam("input", "Input"), Output: NewParam("output", "Output")}, + }, + Structs: []StructDef{ + { + Name: "Input", + Fields: []FieldDef{ + {Name: "Data", Type: "[]byte", JSONTag: "data"}, + }, + }, + { + Name: "Output", + Fields: []FieldDef{ + {Name: "Value", Type: "string", JSONTag: "value"}, + }, + }, + }, + } + schema, err := GenerateSchema(capability) + Expect(err).NotTo(HaveOccurred()) + Expect(ValidateXTPSchema(schema)).To(Succeed()) + + doc := parseSchema(schema) + components := doc["components"].(map[string]any) + schemas := components["schemas"].(map[string]any) + input := schemas["Input"].(map[string]any) + props := input["properties"].(map[string]any) + data := props["data"].(map[string]any) + Expect(data["type"]).To(Equal("string")) + Expect(data["format"]).To(Equal("byte")) + Expect(data).NotTo(HaveKey("items")) + }) + }) + Context("capability with nullable ref", func() { It("should mark pointer to enum as nullable with $ref", func() { capability := Capability{ diff --git a/plugins/host_websocket.go b/plugins/host_websocket.go index 84b28dd35..74238a422 100644 --- a/plugins/host_websocket.go +++ b/plugins/host_websocket.go @@ -2,7 +2,6 @@ package plugins import ( "context" - "encoding/base64" "errors" "fmt" "maps" @@ -355,7 +354,7 @@ func (s *webSocketServiceImpl) invokeOnTextMessage(ctx context.Context, connecti func (s *webSocketServiceImpl) invokeOnBinaryMessage(ctx context.Context, connectionID string, data []byte) { invokeWebSocketCallback(ctx, s, FuncWebSocketOnBinaryMessage, capabilities.OnBinaryMessageRequest{ ConnectionID: connectionID, - Data: base64.StdEncoding.EncodeToString(data), + Data: data, }, "binary message", connectionID) } diff --git a/plugins/host_websocket_test.go b/plugins/host_websocket_test.go index e8cb9f8fd..83fca9898 100644 --- a/plugins/host_websocket_test.go +++ b/plugins/host_websocket_test.go @@ -5,7 +5,7 @@ package plugins import ( "context" "crypto/sha256" - "encoding/base64" + "encoding/hex" "maps" "net/http" @@ -294,11 +294,13 @@ var _ = Describe("WebSocketService", Ordered, func() { var wsServer *httptest.Server var serverConn *websocket.Conn var serverMessages []string + var serverBinaryMessages [][]byte var serverMu sync.Mutex BeforeEach(func() { serverConn = nil serverMessages = nil + serverBinaryMessages = nil upgrader := websocket.Upgrader{ CheckOrigin: func(r *http.Request) bool { return true }, @@ -314,12 +316,16 @@ var _ = Describe("WebSocketService", Ordered, func() { // Read and store messages for { - _, msg, err := conn.ReadMessage() + msgType, msg, err := conn.ReadMessage() if err != nil { break } serverMu.Lock() - serverMessages = append(serverMessages, string(msg)) + if msgType == websocket.BinaryMessage { + serverBinaryMessages = append(serverBinaryMessages, msg) + } else { + serverMessages = append(serverMessages, string(msg)) + } serverMu.Unlock() } })) @@ -359,13 +365,12 @@ var _ = Describe("WebSocketService", Ordered, func() { serverMu.Unlock() Expect(err).ToNot(HaveOccurred()) - // Plugin echoes binary data back as text prefixed with "binary_echo:" - expectedEcho := "binary_echo:" + base64.StdEncoding.EncodeToString(binaryData) - Eventually(func() []string { + // Plugin echoes binary data back as a binary message + Eventually(func() [][]byte { serverMu.Lock() defer serverMu.Unlock() - return serverMessages - }).Should(ContainElement(expectedEcho)) + return serverBinaryMessages + }).Should(ContainElement(binaryData)) }) It("should invoke OnClose callback when server closes connection", func() { @@ -609,6 +614,3 @@ func findWebSocketService(m *Manager, pluginName string) *webSocketServiceImpl { } return nil } - -// Ensure base64 import is used -var _ = base64.StdEncoding diff --git a/plugins/pdk/go/websocket/websocket.go b/plugins/pdk/go/websocket/websocket.go index 0ad2cb549..47a53b7b3 100644 --- a/plugins/pdk/go/websocket/websocket.go +++ b/plugins/pdk/go/websocket/websocket.go @@ -16,7 +16,7 @@ type OnBinaryMessageRequest struct { // ConnectionID is the unique identifier for the WebSocket connection that received the message. ConnectionID string `json:"connectionId"` // Data is the binary data received from the WebSocket, encoded as base64. - Data string `json:"data"` + Data []byte `json:"data"` } // OnCloseRequest is the request provided when a WebSocket connection is closed. diff --git a/plugins/pdk/go/websocket/websocket_stub.go b/plugins/pdk/go/websocket/websocket_stub.go index 1c808d92f..214118a89 100644 --- a/plugins/pdk/go/websocket/websocket_stub.go +++ b/plugins/pdk/go/websocket/websocket_stub.go @@ -13,7 +13,7 @@ type OnBinaryMessageRequest struct { // ConnectionID is the unique identifier for the WebSocket connection that received the message. ConnectionID string `json:"connectionId"` // Data is the binary data received from the WebSocket, encoded as base64. - Data string `json:"data"` + Data []byte `json:"data"` } // OnCloseRequest is the request provided when a WebSocket connection is closed. diff --git a/plugins/pdk/rust/nd-pdk-capabilities/src/websocket.rs b/plugins/pdk/rust/nd-pdk-capabilities/src/websocket.rs index b077110d3..672233e4b 100644 --- a/plugins/pdk/rust/nd-pdk-capabilities/src/websocket.rs +++ b/plugins/pdk/rust/nd-pdk-capabilities/src/websocket.rs @@ -4,6 +4,29 @@ // It is intended for use in Navidrome plugins built with extism-pdk. use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} // Helper functions for skip_serializing_if with numeric types #[allow(dead_code)] @@ -27,7 +50,8 @@ pub struct OnBinaryMessageRequest { pub connection_id: String, /// Data is the binary data received from the WebSocket, encoded as base64. #[serde(default)] - pub data: String, + #[serde(with = "base64_bytes")] + pub data: Vec, } /// OnCloseRequest is the request provided when a WebSocket connection is closed. #[derive(Debug, Clone, Default, Serialize, Deserialize)] diff --git a/plugins/testdata/test-websocket/main.go b/plugins/testdata/test-websocket/main.go index b85eaa358..270f36a11 100644 --- a/plugins/testdata/test-websocket/main.go +++ b/plugins/testdata/test-websocket/main.go @@ -3,6 +3,7 @@ package main import ( + "encoding/base64" "errors" "github.com/navidrome/navidrome/plugins/pdk/go/host" @@ -45,11 +46,11 @@ func (t *testWebSocket) OnTextMessage(input websocket.OnTextMessageRequest) erro } // OnBinaryMessage is called when a binary message is received. -// Echoes the data back as a text message prefixed with "binary_echo:" so tests -// can observe the callback fired. +// Echoes the data back as a binary message so tests can observe the callback fired. func (t *testWebSocket) OnBinaryMessage(input websocket.OnBinaryMessageRequest) error { - storeReceivedMessage("binary:" + input.Data) - return host.WebSocketSendText(input.ConnectionID, "binary_echo:"+input.Data) + encoded := base64.StdEncoding.EncodeToString(input.Data) + storeReceivedMessage("binary:" + encoded) + return host.WebSocketSendBinary(input.ConnectionID, input.Data) } // OnError is called when an error occurs on a WebSocket connection. From 435fb0b0768b5059992605bb0c0eb2c4812cdbda Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 2 Mar 2026 16:59:05 -0500 Subject: [PATCH 32/60] feat(server): add EnableCoverArtUpload config option Allow administrators to disable playlist cover art upload/removal for non-admin users via the new EnableCoverArtUpload config option (default: true). - Guard uploadPlaylistImage and deletePlaylistImage endpoints (403 for non-admin when disabled) - Set CoverArtRole in Subsonic GetUser/GetUsers responses based on config and admin status - Pass config to frontend and conditionally hide upload/remove UI controls - Admins always retain upload capability regardless of setting --- conf/configuration.go | 2 + server/nativeapi/playlists.go | 12 +++ server/nativeapi/playlists_test.go | 126 +++++++++++++++++++++------- server/serve_index.go | 1 + server/subsonic/users.go | 1 + server/subsonic/users_test.go | 16 ++++ ui/src/config.js | 1 + ui/src/playlist/PlaylistDetails.jsx | 5 +- 8 files changed, 131 insertions(+), 33 deletions(-) diff --git a/conf/configuration.go b/conf/configuration.go index 0d32815ee..b6f65183a 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -76,6 +76,7 @@ type configOptions struct { EnableFavourites bool EnableStarRating bool EnableUserEditing bool + EnableCoverArtUpload bool EnableSharing bool ShareURL string DefaultShareExpiration time.Duration @@ -668,6 +669,7 @@ func setViperDefaults() { viper.SetDefault("enablereplaygain", true) viper.SetDefault("enablecoveranimation", true) viper.SetDefault("enablenowplaying", true) + viper.SetDefault("enablecoverartupload", true) viper.SetDefault("enablesharing", false) viper.SetDefault("shareurl", "") viper.SetDefault("defaultshareexpiration", 8760*time.Hour) diff --git a/server/nativeapi/playlists.go b/server/nativeapi/playlists.go index c7230a209..797654a3b 100644 --- a/server/nativeapi/playlists.go +++ b/server/nativeapi/playlists.go @@ -17,9 +17,11 @@ import ( "github.com/deluan/rest" "github.com/go-chi/chi/v5" + "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" "github.com/navidrome/navidrome/utils/req" _ "golang.org/x/image/webp" ) @@ -237,6 +239,11 @@ const maxImageSize = 10 << 20 // 10MB func uploadPlaylistImage(pls playlists.Playlists) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { ctx := r.Context() + user, _ := request.UserFrom(ctx) + if !conf.Server.EnableCoverArtUpload && !user.IsAdmin { + http.Error(w, "cover art upload is disabled", http.StatusForbidden) + return + } p := req.Params(r) playlistId, _ := p.String(":id") @@ -306,6 +313,11 @@ func uploadPlaylistImage(pls playlists.Playlists) http.HandlerFunc { func deletePlaylistImage(pls playlists.Playlists) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { ctx := r.Context() + user, _ := request.UserFrom(ctx) + if !conf.Server.EnableCoverArtUpload && !user.IsAdmin { + http.Error(w, "cover art upload is disabled", http.StatusForbidden) + return + } p := req.Params(r) playlistId, _ := p.String(":id") diff --git a/server/nativeapi/playlists_test.go b/server/nativeapi/playlists_test.go index 961d10b68..7f0cd7de1 100644 --- a/server/nativeapi/playlists_test.go +++ b/server/nativeapi/playlists_test.go @@ -3,6 +3,7 @@ package nativeapi import ( "context" "encoding/json" + "io" "net/http" "net/http/httptest" "time" @@ -14,50 +15,56 @@ import ( "github.com/navidrome/navidrome/core/auth" "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" "github.com/navidrome/navidrome/server" "github.com/navidrome/navidrome/tests" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ) -type mockPlaylistTrackRepo struct { - model.PlaylistTrackRepository - tracks model.PlaylistTracks -} +var _ = Describe("Playlist Image Endpoints", func() { + BeforeEach(func() { + DeferCleanup(configtest.SetupConfig()) + }) -func (m *mockPlaylistTrackRepo) Count(...rest.QueryOptions) (int64, error) { - return int64(len(m.tracks)), nil -} + DescribeTable("uploadPlaylistImage guard", + func(enableCoverArtUpload, isAdmin bool, expectedStatus int) { + conf.Server.EnableCoverArtUpload = enableCoverArtUpload + handler := uploadPlaylistImage(&mockPlaylistsService{}) -func (m *mockPlaylistTrackRepo) ReadAll(...rest.QueryOptions) (any, error) { - return m.tracks, nil -} + req := httptest.NewRequest("POST", "/playlist/pls-1/image", nil) + ctx := request.WithUser(GinkgoT().Context(), model.User{ID: "user-1", IsAdmin: isAdmin}) + req = req.WithContext(ctx) -func (m *mockPlaylistTrackRepo) EntityName() string { - return "playlist_track" -} + w := httptest.NewRecorder() + handler.ServeHTTP(w, req) + Expect(w.Code).To(Equal(expectedStatus)) + }, + Entry("enabled, regular user passes guard", true, false, http.StatusBadRequest), + Entry("enabled, admin passes guard", true, true, http.StatusBadRequest), + Entry("disabled, admin passes guard", false, true, http.StatusBadRequest), + Entry("disabled, regular user is forbidden", false, false, http.StatusForbidden), + ) -func (m *mockPlaylistTrackRepo) NewInstance() any { - return &model.PlaylistTrack{} -} + DescribeTable("deletePlaylistImage guard", + func(enableCoverArtUpload, isAdmin bool, expectedStatus int) { + conf.Server.EnableCoverArtUpload = enableCoverArtUpload + handler := deletePlaylistImage(&mockPlaylistsService{}) -func (m *mockPlaylistTrackRepo) Read(id string) (any, error) { - for _, t := range m.tracks { - if t.ID == id { - return &t, nil - } - } - return nil, rest.ErrNotFound -} + req := httptest.NewRequest("DELETE", "/playlist/pls-1/image", nil) + ctx := request.WithUser(GinkgoT().Context(), model.User{ID: "user-1", IsAdmin: isAdmin}) + req = req.WithContext(ctx) -type mockPlaylistsService struct { - playlists.Playlists - tracksRepo rest.Repository -} - -func (m *mockPlaylistsService) TracksRepository(_ context.Context, _ string, _ bool) rest.Repository { - return m.tracksRepo -} + w := httptest.NewRecorder() + handler.ServeHTTP(w, req) + Expect(w.Code).To(Equal(expectedStatus)) + }, + Entry("enabled, regular user passes guard", true, false, http.StatusNotFound), + Entry("enabled, admin passes guard", true, true, http.StatusNotFound), + Entry("disabled, admin passes guard", false, true, http.StatusNotFound), + Entry("disabled, regular user is forbidden", false, false, http.StatusForbidden), + ) +}) var _ = Describe("Playlist Tracks Endpoint", func() { var ( @@ -174,3 +181,58 @@ var _ = Describe("Playlist Tracks Endpoint", func() { }) }) }) + +type mockPlaylistTrackRepo struct { + model.PlaylistTrackRepository + tracks model.PlaylistTracks +} + +func (m *mockPlaylistTrackRepo) Count(...rest.QueryOptions) (int64, error) { + return int64(len(m.tracks)), nil +} + +func (m *mockPlaylistTrackRepo) ReadAll(...rest.QueryOptions) (any, error) { + return m.tracks, nil +} + +func (m *mockPlaylistTrackRepo) EntityName() string { + return "playlist_track" +} + +func (m *mockPlaylistTrackRepo) NewInstance() any { + return &model.PlaylistTrack{} +} + +func (m *mockPlaylistTrackRepo) Read(id string) (any, error) { + for _, t := range m.tracks { + if t.ID == id { + return &t, nil + } + } + return nil, rest.ErrNotFound +} + +type mockPlaylistsService struct { + playlists.Playlists + tracksRepo rest.Repository + removeImageFn func(ctx context.Context, id string) error + setImageFn func(ctx context.Context, id string, reader io.Reader, ext string) error +} + +func (m *mockPlaylistsService) RemoveImage(ctx context.Context, id string) error { + if m.removeImageFn != nil { + return m.removeImageFn(ctx, id) + } + return model.ErrNotFound +} + +func (m *mockPlaylistsService) SetImage(ctx context.Context, id string, reader io.Reader, ext string) error { + if m.setImageFn != nil { + return m.setImageFn(ctx, id, reader, ext) + } + return model.ErrNotFound +} + +func (m *mockPlaylistsService) TracksRepository(_ context.Context, _ string, _ bool) rest.Repository { + return m.tracksRepo +} diff --git a/server/serve_index.go b/server/serve_index.go index 92ef47e23..6b0c890a6 100644 --- a/server/serve_index.go +++ b/server/serve_index.go @@ -61,6 +61,7 @@ func serveIndex(ds model.DataStore, fs fs.FS, shareInfo *model.Share) http.Handl "losslessFormats": strings.ToUpper(strings.Join(mime.LosslessFormats, ",")), "devActivityPanel": conf.Server.DevActivityPanel, "enableUserEditing": conf.Server.EnableUserEditing, + "enableCoverArtUpload": conf.Server.EnableCoverArtUpload, "enableSharing": conf.Server.EnableSharing, "shareURL": conf.Server.ShareURL, "defaultDownloadableShare": conf.Server.DefaultDownloadableShare, diff --git a/server/subsonic/users.go b/server/subsonic/users.go index aeac6992b..4f6dccaac 100644 --- a/server/subsonic/users.go +++ b/server/subsonic/users.go @@ -22,6 +22,7 @@ func buildUserResponse(user model.User) responses.User { ScrobblingEnabled: true, DownloadRole: conf.Server.EnableDownloads, ShareRole: conf.Server.EnableSharing, + CoverArtRole: conf.Server.EnableCoverArtUpload || user.IsAdmin, Folder: slice.Map(user.Libraries, func(lib model.Library) int32 { return int32(lib.ID) }), } diff --git a/server/subsonic/users_test.go b/server/subsonic/users_test.go index 95e16590d..1fd5dce71 100644 --- a/server/subsonic/users_test.go +++ b/server/subsonic/users_test.go @@ -63,6 +63,7 @@ var _ = Describe("Users", func() { Expect(userResponse.User.ScrobblingEnabled).To(BeTrue()) Expect(userResponse.User.DownloadRole).To(BeTrue()) Expect(userResponse.User.ShareRole).To(BeTrue()) + Expect(userResponse.User.CoverArtRole).To(BeTrue()) Expect(userResponse.User.Folder).To(ContainElements(int32(10), int32(20))) // Verify GetUsers response structure @@ -81,6 +82,7 @@ var _ = Describe("Users", func() { Expect(singleUser.ScrobblingEnabled).To(Equal(userFromList.ScrobblingEnabled)) Expect(singleUser.DownloadRole).To(Equal(userFromList.DownloadRole)) Expect(singleUser.ShareRole).To(Equal(userFromList.ShareRole)) + Expect(singleUser.CoverArtRole).To(Equal(userFromList.CoverArtRole)) Expect(singleUser.JukeboxRole).To(Equal(userFromList.JukeboxRole)) Expect(singleUser.Folder).To(Equal(userFromList.Folder)) }) @@ -102,6 +104,20 @@ var _ = Describe("Users", func() { Entry("jukebox enabled, admin-only, admin user", true, true, true, true), ) + DescribeTable("CoverArt role permissions", + func(enableCoverArtUpload, isAdmin, expectedCoverArtRole bool) { + conf.Server.EnableCoverArtUpload = enableCoverArtUpload + testUser.IsAdmin = isAdmin + + response := buildUserResponse(testUser) + Expect(response.CoverArtRole).To(Equal(expectedCoverArtRole)) + }, + Entry("enabled, regular user", true, false, true), + Entry("enabled, admin user", true, true, true), + Entry("disabled, regular user", false, false, false), + Entry("disabled, admin user", false, true, true), + ) + Describe("Folder list population", func() { It("should populate Folder field with user's accessible library IDs", func() { testUser.Libraries = model.Libraries{ diff --git a/ui/src/config.js b/ui/src/config.js index 5acf10b69..0672a58f4 100644 --- a/ui/src/config.js +++ b/ui/src/config.js @@ -22,6 +22,7 @@ const defaultConfig = { defaultUIVolume: 100, uiSearchDebounceMs: 200, enableUserEditing: true, + enableCoverArtUpload: true, enableSharing: true, shareURL: '', defaultDownloadableShare: true, diff --git a/ui/src/playlist/PlaylistDetails.jsx b/ui/src/playlist/PlaylistDetails.jsx index b24446cb9..eefed83b4 100644 --- a/ui/src/playlist/PlaylistDetails.jsx +++ b/ui/src/playlist/PlaylistDetails.jsx @@ -20,6 +20,7 @@ import { SizeField, isWritable, } from '../common' +import config from '../config' import subsonic from '../subsonic' import { REST_URL } from '../consts' import { httpClient } from '../dataProvider' @@ -134,7 +135,9 @@ const PlaylistDetails = (props) => { const imageUrl = subsonic.getCoverArtUrl(record, 300, true) const fullImageUrl = subsonic.getCoverArtUrl(record) - const canEdit = isWritable(record.ownerId) + const canEdit = + isWritable(record.ownerId) && + (config.enableCoverArtUpload || localStorage.getItem('role') === 'admin') // Reset image state when playlist changes useEffect(() => { From 157c917ca5bfa268e9247e611830b0ea49c3886d Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 2 Mar 2026 17:01:12 -0500 Subject: [PATCH 33/60] chore(deps): update golang.org/x/net to v0.51.0 Signed-off-by: Deluan --- go.mod | 2 +- go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/go.mod b/go.mod index 1f6fa30a6..7f7e90a7f 100644 --- a/go.mod +++ b/go.mod @@ -69,7 +69,7 @@ require ( go.senan.xyz/taglib v0.11.1 go.uber.org/goleak v1.3.0 golang.org/x/image v0.36.0 - golang.org/x/net v0.50.0 + golang.org/x/net v0.51.0 golang.org/x/sync v0.19.0 golang.org/x/sys v0.41.0 golang.org/x/term v0.40.0 diff --git a/go.sum b/go.sum index 26e4d3925..224b33e90 100644 --- a/go.sum +++ b/go.sum @@ -348,8 +348,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk= golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44= golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= -golang.org/x/net v0.50.0 h1:ucWh9eiCGyDR3vtzso0WMQinm2Dnt8cFMuQa9K33J60= -golang.org/x/net v0.50.0/go.mod h1:UgoSli3F/pBgdJBHCTc+tp3gmrU4XswgGRgtnwWTfyM= +golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo= +golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= From 692f0f99f6184ad108edde85818787181572a255 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 2 Mar 2026 20:48:26 -0500 Subject: [PATCH 34/60] chore(deps): bump actions/upload-artifact in /.github/workflows (#5134) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v6...v7) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pipeline.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/pipeline.yml b/.github/workflows/pipeline.yml index 5fcc9526b..c2e51f47c 100644 --- a/.github/workflows/pipeline.yml +++ b/.github/workflows/pipeline.yml @@ -235,7 +235,7 @@ jobs: CROSS_TAGLIB_VERSION=${{ env.CROSS_TAGLIB_VERSION }} - name: Upload Binaries - uses: actions/upload-artifact@v6 + uses: actions/upload-artifact@v7 with: name: navidrome-${{ env.PLATFORM }} path: ./output @@ -266,7 +266,7 @@ jobs: touch "/tmp/digests/${digest#sha256:}" - name: Upload digest - uses: actions/upload-artifact@v6 + uses: actions/upload-artifact@v7 if: env.IS_LINUX == 'true' && env.IS_DOCKER_PUSH_CONFIGURED == 'true' && env.IS_ARMV5 == 'false' with: name: digests-${{ env.PLATFORM }} @@ -393,7 +393,7 @@ jobs: du -h binaries/msi/*.msi - name: Upload MSI files - uses: actions/upload-artifact@v6 + uses: actions/upload-artifact@v7 with: name: navidrome-windows-installers path: binaries/msi/*.msi @@ -437,7 +437,7 @@ jobs: rm ./dist/*.tar.gz ./dist/*.zip - name: Upload all-packages artifact - uses: actions/upload-artifact@v6 + uses: actions/upload-artifact@v7 with: name: packages path: dist/navidrome_0* @@ -466,7 +466,7 @@ jobs: path: ./dist - name: Upload all-packages artifact - uses: actions/upload-artifact@v6 + uses: actions/upload-artifact@v7 with: name: navidrome_linux_${{ matrix.item }} path: dist/navidrome_0*_linux_${{ matrix.item }} From c8857668549173684622953ad75a998be8981314 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 2 Mar 2026 20:48:36 -0500 Subject: [PATCH 35/60] chore(deps): bump actions/download-artifact in /.github/workflows (#5133) Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 7 to 8. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](https://github.com/actions/download-artifact/compare/v7...v8) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pipeline.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/pipeline.yml b/.github/workflows/pipeline.yml index c2e51f47c..3a24d27b1 100644 --- a/.github/workflows/pipeline.yml +++ b/.github/workflows/pipeline.yml @@ -288,7 +288,7 @@ jobs: - uses: actions/checkout@v6 - name: Download digests - uses: actions/download-artifact@v7 + uses: actions/download-artifact@v8 with: path: /tmp/digests pattern: digests-* @@ -322,7 +322,7 @@ jobs: - uses: actions/checkout@v6 - name: Download digests - uses: actions/download-artifact@v7 + uses: actions/download-artifact@v8 with: path: /tmp/digests pattern: digests-* @@ -374,7 +374,7 @@ jobs: steps: - uses: actions/checkout@v6 - - uses: actions/download-artifact@v7 + - uses: actions/download-artifact@v8 with: path: ./binaries pattern: navidrome-windows* @@ -411,7 +411,7 @@ jobs: fetch-depth: 0 fetch-tags: true - - uses: actions/download-artifact@v7 + - uses: actions/download-artifact@v8 with: path: ./binaries pattern: navidrome-* @@ -460,7 +460,7 @@ jobs: item: ${{ fromJson(needs.release.outputs.package_list) }} steps: - name: Download all-packages artifact - uses: actions/download-artifact@v7 + uses: actions/download-artifact@v8 with: name: packages path: ./dist From ed4c0ef432d6084806e3e179c8bda2b964fb131a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Tue, 3 Mar 2026 07:58:14 -0500 Subject: [PATCH 36/60] fix(scanner): add nil guards to cursor wrapping (#5139) * fix(persistence): add nil guards to cursor wrapping in folder and mediafile repos Prevent SIGSEGV panic when queryWithStableResults yields a zero-value struct on the rows.Err() path (e.g., "database is locked" during concurrent scanning). Extract cursor wrapping into wrapFolderCursor and wrapMediaFileCursor with nil checks matching the existing pattern in album_repository.go. Fixes #5138 * fix(persistence): wrap original cursor error in nil guard messages Use %w to preserve the underlying error (e.g., "database is locked") so callers can use errors.Is/As for root cause analysis. Tests now verify the original error is accessible via errors.Is. * fix(persistence): add nil guards and error wrapping in album, folder, and mediafile cursor functions Signed-off-by: Deluan --------- Signed-off-by: Deluan --- persistence/album_repository.go | 9 +++-- persistence/album_repository_test.go | 41 +++++++++++++++++++++++ persistence/folder_repository.go | 11 ++++++- persistence/folder_repository_test.go | 41 +++++++++++++++++++++++ persistence/mediafile_repository.go | 23 +++++++------ persistence/mediafile_repository_test.go | 42 ++++++++++++++++++++++++ 6 files changed, 152 insertions(+), 15 deletions(-) diff --git a/persistence/album_repository.go b/persistence/album_repository.go index 077e33d17..35bda877c 100644 --- a/persistence/album_repository.go +++ b/persistence/album_repository.go @@ -4,6 +4,7 @@ import ( "context" "encoding/json" "fmt" + "iter" "maps" "slices" "strings" @@ -302,17 +303,21 @@ func (r *albumRepository) GetTouchedAlbums(libID int) (model.AlbumCursor, error) if err != nil { return nil, err } + return wrapAlbumCursor(cursor), nil +} + +func wrapAlbumCursor(cursor iter.Seq2[dbAlbum, error]) model.AlbumCursor { return func(yield func(model.Album, error) bool) { for a, err := range cursor { if a.Album == nil { - yield(model.Album{}, fmt.Errorf("unexpected nil album: %v", a)) + yield(model.Album{}, fmt.Errorf("unexpected nil album (%v): %w", a, err)) return } if !yield(*a.Album, err) || err != nil { return } } - }, nil + } } // RefreshPlayCounts updates the play count and last play date annotations for all albums, based diff --git a/persistence/album_repository_test.go b/persistence/album_repository_test.go index 9fbc6b974..66b6eba9f 100644 --- a/persistence/album_repository_test.go +++ b/persistence/album_repository_test.go @@ -1,6 +1,7 @@ package persistence import ( + "errors" "fmt" "time" @@ -743,6 +744,46 @@ var _ = Describe("AlbumRepository", func() { _, _ = albumRepo.executeSQL(squirrel.Delete("album").Where(squirrel.Eq{"id": album.ID})) }) }) + + Describe("wrapAlbumCursor", func() { + It("does not panic when the cursor yields a dbAlbum with nil Album", func() { + // Simulate what queryWithStableResults does on the rows.Err() path: + // it yields a zero-value dbAlbum (where Album is nil) with an error. + dbErr := fmt.Errorf("database is locked") + cursor := func(yield func(dbAlbum, error) bool) { + var empty dbAlbum // Album pointer is nil + yield(empty, dbErr) + } + + // wrapAlbumCursor should handle the nil Album without panicking + wrappedCursor := wrapAlbumCursor(cursor) + var gotErr error + Expect(func() { + for _, err := range wrappedCursor { + gotErr = err + } + }).ToNot(Panic()) + Expect(gotErr).To(HaveOccurred()) + Expect(gotErr.Error()).To(ContainSubstring("unexpected nil album")) + Expect(errors.Is(gotErr, dbErr)).To(BeTrue(), "should wrap the original cursor error") + }) + + It("yields albums from a valid cursor", func() { + album := &model.Album{ID: "a1", Name: "Test"} + cursor := func(yield func(dbAlbum, error) bool) { + yield(dbAlbum{Album: album}, nil) + } + + wrappedCursor := wrapAlbumCursor(cursor) + var albums []model.Album + for a, err := range wrappedCursor { + Expect(err).ToNot(HaveOccurred()) + albums = append(albums, a) + } + Expect(albums).To(HaveLen(1)) + Expect(albums[0].ID).To(Equal("a1")) + }) + }) }) func _p(id, name string, sortName ...string) model.Participant { diff --git a/persistence/folder_repository.go b/persistence/folder_repository.go index a4e203467..f7bb6a4fe 100644 --- a/persistence/folder_repository.go +++ b/persistence/folder_repository.go @@ -4,6 +4,7 @@ import ( "context" "encoding/json" "fmt" + "iter" "maps" "os" "path/filepath" @@ -218,13 +219,21 @@ func (r folderRepository) GetTouchedWithPlaylists() (model.FolderCursor, error) if err != nil { return nil, err } + return wrapFolderCursor(cursor), nil +} + +func wrapFolderCursor(cursor iter.Seq2[dbFolder, error]) model.FolderCursor { return func(yield func(model.Folder, error) bool) { for f, err := range cursor { + if f.Folder == nil { + yield(model.Folder{}, fmt.Errorf("unexpected nil folder (%v): %w", f, err)) + return + } if !yield(*f.Folder, err) || err != nil { return } } - }, nil + } } func (r folderRepository) purgeEmpty(libraryIDs ...int) error { diff --git a/persistence/folder_repository_test.go b/persistence/folder_repository_test.go index 6c24741c9..7b6a0f764 100644 --- a/persistence/folder_repository_test.go +++ b/persistence/folder_repository_test.go @@ -2,6 +2,7 @@ package persistence import ( "context" + "errors" "fmt" "github.com/navidrome/navidrome/log" @@ -210,4 +211,44 @@ var _ = Describe("FolderRepository", func() { }) }) }) + + Describe("wrapFolderCursor", func() { + It("does not panic when the cursor yields a dbFolder with nil Folder", func() { + // Simulate what queryWithStableResults does on the rows.Err() path: + // it yields a zero-value dbFolder (where Folder is nil) with an error. + dbErr := fmt.Errorf("database is locked") + cursor := func(yield func(dbFolder, error) bool) { + var empty dbFolder // Folder pointer is nil + yield(empty, dbErr) + } + + // wrapFolderCursor should handle the nil Folder without panicking + wrappedCursor := wrapFolderCursor(cursor) + var gotErr error + Expect(func() { + for _, err := range wrappedCursor { + gotErr = err + } + }).ToNot(Panic()) + Expect(gotErr).To(HaveOccurred()) + Expect(gotErr.Error()).To(ContainSubstring("unexpected nil folder")) + Expect(errors.Is(gotErr, dbErr)).To(BeTrue(), "should wrap the original cursor error") + }) + + It("yields folders from a valid cursor", func() { + folder := &model.Folder{ID: "f1", Name: "Test"} + cursor := func(yield func(dbFolder, error) bool) { + yield(dbFolder{Folder: folder}, nil) + } + + wrappedCursor := wrapFolderCursor(cursor) + var folders []model.Folder + for f, err := range wrappedCursor { + Expect(err).ToNot(HaveOccurred()) + folders = append(folders, f) + } + Expect(folders).To(HaveLen(1)) + Expect(folders[0].ID).To(Equal("f1")) + }) + }) }) diff --git a/persistence/mediafile_repository.go b/persistence/mediafile_repository.go index 9034fa8f8..43736e317 100644 --- a/persistence/mediafile_repository.go +++ b/persistence/mediafile_repository.go @@ -3,6 +3,7 @@ package persistence import ( "context" "fmt" + "iter" "slices" "strconv" "strings" @@ -231,17 +232,7 @@ func (r *mediaFileRepository) GetCursor(options ...model.QueryOptions) (model.Me if err != nil { return nil, err } - return func(yield func(model.MediaFile, error) bool) { - for m, err := range cursor { - if m.MediaFile == nil { - yield(model.MediaFile{}, fmt.Errorf("unexpected nil mediafile: %v", m)) - return - } - if !yield(*m.MediaFile, err) || err != nil { - return - } - } - }, nil + return wrapMediaFileCursor(cursor), nil } // FindByPaths finds media files by their paths. @@ -371,13 +362,21 @@ func (r *mediaFileRepository) GetMissingAndMatching(libId int) (model.MediaFileC if err != nil { return nil, err } + return wrapMediaFileCursor(cursor), nil +} + +func wrapMediaFileCursor(cursor iter.Seq2[dbMediaFile, error]) model.MediaFileCursor { return func(yield func(model.MediaFile, error) bool) { for m, err := range cursor { + if m.MediaFile == nil { + yield(model.MediaFile{}, fmt.Errorf("unexpected nil mediafile (%v): %w", m, err)) + return + } if !yield(*m.MediaFile, err) || err != nil { return } } - }, nil + } } // FindRecentFilesByMBZTrackID finds recently added files by MusicBrainz Track ID in other libraries diff --git a/persistence/mediafile_repository_test.go b/persistence/mediafile_repository_test.go index 7a04d79e4..5a866379f 100644 --- a/persistence/mediafile_repository_test.go +++ b/persistence/mediafile_repository_test.go @@ -2,6 +2,8 @@ package persistence import ( "context" + "errors" + "fmt" "time" "github.com/Masterminds/squirrel" @@ -711,4 +713,44 @@ var _ = Describe("MediaRepository", func() { Expect(results).To(BeEmpty()) }) }) + + Describe("wrapMediaFileCursor", func() { + It("does not panic when the cursor yields a dbMediaFile with nil MediaFile", func() { + // Simulate what queryWithStableResults does on the rows.Err() path: + // it yields a zero-value dbMediaFile (where MediaFile is nil) with an error. + dbErr := fmt.Errorf("database is locked") + cursor := func(yield func(dbMediaFile, error) bool) { + var empty dbMediaFile // MediaFile pointer is nil + yield(empty, dbErr) + } + + // wrapMediaFileCursor should handle the nil MediaFile without panicking + wrappedCursor := wrapMediaFileCursor(cursor) + var gotErr error + Expect(func() { + for _, err := range wrappedCursor { + gotErr = err + } + }).ToNot(Panic()) + Expect(gotErr).To(HaveOccurred()) + Expect(gotErr.Error()).To(ContainSubstring("unexpected nil mediafile")) + Expect(errors.Is(gotErr, dbErr)).To(BeTrue(), "should wrap the original cursor error") + }) + + It("yields mediafiles from a valid cursor", func() { + mf := &model.MediaFile{ID: "mf1", Title: "Test"} + cursor := func(yield func(dbMediaFile, error) bool) { + yield(dbMediaFile{MediaFile: mf}, nil) + } + + wrappedCursor := wrapMediaFileCursor(cursor) + var mediafiles []model.MediaFile + for m, err := range wrappedCursor { + Expect(err).ToNot(HaveOccurred()) + mediafiles = append(mediafiles, m) + } + Expect(mediafiles).To(HaveLen(1)) + Expect(mediafiles[0].ID).To(Equal("mf1")) + }) + }) }) From 24ba655dc322050c3d78b15e09f2eb8d9325b17d Mon Sep 17 00:00:00 2001 From: Deluan Date: Tue, 3 Mar 2026 08:14:54 -0500 Subject: [PATCH 37/60] refactor: simplify error handling in updateParticipants and toModels methods Signed-off-by: Deluan --- persistence/album_repository.go | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/persistence/album_repository.go b/persistence/album_repository.go index 35bda877c..7207bf5a2 100644 --- a/persistence/album_repository.go +++ b/persistence/album_repository.go @@ -203,12 +203,11 @@ func (r *albumRepository) Put(al *model.Album) error { } al.ID = id if len(al.Participants) > 0 { - err = r.updateParticipants(al.ID, al.Participants) - if err != nil { + if err = r.updateParticipants(al.ID, al.Participants); err != nil { return err } } - return err + return nil } // TODO Move external metadata to a separated table @@ -242,7 +241,7 @@ func (r *albumRepository) GetAll(options ...model.QueryOptions) (model.Albums, e if err != nil { return nil, err } - return res.toModels(), err + return res.toModels(), nil } func (r *albumRepository) CopyAttributes(fromID, toID string, columns ...string) error { From 668869b6c7737ea443cd2479ad200ea4a23f1792 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Tue, 3 Mar 2026 13:48:49 -0500 Subject: [PATCH 38/60] feat(plugins): add TaskQueue host service for persistent background task queues (#5116) * feat(plugins): define TaskQueue host service interface Add the TaskQueueService interface with CreateQueue, Enqueue, GetTaskStatus, and CancelTask methods plus QueueConfig struct. * feat(plugins): define TaskWorker capability for task execution callbacks * feat(plugins): add taskqueue permission to manifest schema Add TaskQueuePermission with maxConcurrency option. * feat(plugins): implement TaskQueue service with SQLite persistence and workers Per-plugin SQLite database with queues and tasks tables. Worker goroutines dequeue tasks and invoke nd_task_execute callback. Exponential backoff retries, rate limiting via delayMs, automatic cleanup of terminal tasks. * feat(plugins): require TaskWorker capability for taskqueue permission * feat(plugins): register TaskQueue host service in manager * feat(plugins): add test-taskqueue plugin for integration testing * feat(plugins): add integration tests for TaskQueue host service * docs: document TaskQueue module for persistent task queues Signed-off-by: Deluan * fix(plugins): harden TaskQueue host service with validation and safety improvements Add input validation (queue name length, payload size limits), extract status string constants to eliminate raw SQL literals, make CreateQueue idempotent via upsert for crash recovery, fix RetentionMs default check for negative values, cap exponential backoff at 1 hour to prevent overflow, and replace manual mutex-based delay enforcement with rate.Limiter from golang.org/x/time/rate for correct concurrent worker serialization. * refactor(plugins): remove capability check for TaskWorker in TaskQueue host service Signed-off-by: Deluan * fix(plugins): use context-aware database execution in TaskQueue host service Signed-off-by: Deluan * refactor(plugins): streamline task queue configuration and error handling Signed-off-by: Deluan * feat(plugins): increase maxConcurrency for task queue and handle budget exhaustion Signed-off-by: Deluan * refactor(plugins): simplify goroutine management in task queue service Signed-off-by: Deluan * feat(plugins): update TaskWorker interface to return status messages and refactor task queue service Signed-off-by: Deluan * feat(plugins): add ClearQueue function to remove pending tasks from a specified queue Signed-off-by: Deluan * refactor(plugins): use migrateDB for task queue schema and fix constant name collision Replaced the raw db.Exec call in createTaskQueueSchema with migrateDB, matching the pattern used by createKVStoreSchema. This enables version-tracked schema migrations via SQLite's PRAGMA user_version, allowing future schema changes to be appended incrementally. Also renamed cleanupInterval to taskCleanupInterval to resolve a redeclaration conflict with host_kvstore.go. * regenerate PDKs Signed-off-by: Deluan --------- Signed-off-by: Deluan --- plugins/capabilities/taskworker.go | 27 + plugins/capabilities/taskworker.yaml | 38 + plugins/host/task.go | 73 + plugins/host/task_gen.go | 266 ++++ plugins/host_taskqueue.go | 595 ++++++++ plugins/host_taskqueue_test.go | 1221 +++++++++++++++++ plugins/manager_loader.go | 17 + plugins/manifest-schema.json | 20 + plugins/manifest.go | 7 + plugins/manifest_gen.go | 33 + plugins/pdk/go/host/doc.go | 1 + plugins/pdk/go/host/nd_host_task.go | 277 ++++ plugins/pdk/go/host/nd_host_task_stub.go | 105 ++ plugins/pdk/go/taskworker/taskworker.go | 79 ++ plugins/pdk/go/taskworker/taskworker_stub.go | 41 + plugins/pdk/python/host/nd_host_task.py | 188 +++ .../pdk/rust/nd-pdk-capabilities/src/lib.rs | 1 + .../nd-pdk-capabilities/src/taskworker.rs | 102 ++ plugins/pdk/rust/nd-pdk-host/src/lib.rs | 8 + .../pdk/rust/nd-pdk-host/src/nd_host_task.rs | 258 ++++ plugins/testdata/test-taskqueue/go.mod | 16 + plugins/testdata/test-taskqueue/go.sum | 14 + plugins/testdata/test-taskqueue/main.go | 114 ++ plugins/testdata/test-taskqueue/manifest.json | 12 + 24 files changed, 3513 insertions(+) create mode 100644 plugins/capabilities/taskworker.go create mode 100644 plugins/capabilities/taskworker.yaml create mode 100644 plugins/host/task.go create mode 100644 plugins/host/task_gen.go create mode 100644 plugins/host_taskqueue.go create mode 100644 plugins/host_taskqueue_test.go create mode 100644 plugins/pdk/go/host/nd_host_task.go create mode 100644 plugins/pdk/go/host/nd_host_task_stub.go create mode 100644 plugins/pdk/go/taskworker/taskworker.go create mode 100644 plugins/pdk/go/taskworker/taskworker_stub.go create mode 100644 plugins/pdk/python/host/nd_host_task.py create mode 100644 plugins/pdk/rust/nd-pdk-capabilities/src/taskworker.rs create mode 100644 plugins/pdk/rust/nd-pdk-host/src/nd_host_task.rs create mode 100644 plugins/testdata/test-taskqueue/go.mod create mode 100644 plugins/testdata/test-taskqueue/go.sum create mode 100644 plugins/testdata/test-taskqueue/main.go create mode 100644 plugins/testdata/test-taskqueue/manifest.json diff --git a/plugins/capabilities/taskworker.go b/plugins/capabilities/taskworker.go new file mode 100644 index 000000000..c53d50174 --- /dev/null +++ b/plugins/capabilities/taskworker.go @@ -0,0 +1,27 @@ +package capabilities + +// TaskWorker provides task execution handling. +// This capability allows plugins to receive callbacks when their queued tasks +// are ready to execute. Plugins that use the taskqueue host service must +// implement this capability. +// +//nd:capability name=taskworker +type TaskWorker interface { + // OnTaskExecute is called when a queued task is ready to run. + // The returned string is a status/result message stored in the tasks table. + // Return an error to trigger retry (if retries are configured). + //nd:export name=nd_task_execute + OnTaskExecute(TaskExecuteRequest) (string, error) +} + +// TaskExecuteRequest is the request provided when a task is ready to execute. +type TaskExecuteRequest struct { + // QueueName is the name of the queue this task belongs to. + QueueName string `json:"queueName"` + // TaskID is the unique identifier for this task. + TaskID string `json:"taskId"` + // Payload is the opaque data provided when the task was enqueued. + Payload []byte `json:"payload"` + // Attempt is the current attempt number (1-based: first attempt = 1). + Attempt int32 `json:"attempt"` +} diff --git a/plugins/capabilities/taskworker.yaml b/plugins/capabilities/taskworker.yaml new file mode 100644 index 000000000..f10fd0794 --- /dev/null +++ b/plugins/capabilities/taskworker.yaml @@ -0,0 +1,38 @@ +version: v1-draft +exports: + nd_task_execute: + description: |- + OnTaskExecute is called when a queued task is ready to run. + The returned string is a status/result message stored in the tasks table. + Return an error to trigger retry (if retries are configured). + input: + $ref: '#/components/schemas/TaskExecuteRequest' + contentType: application/json + output: + type: string + contentType: application/json +components: + schemas: + TaskExecuteRequest: + description: TaskExecuteRequest is the request provided when a task is ready to execute. + properties: + queueName: + type: string + description: QueueName is the name of the queue this task belongs to. + taskId: + type: string + description: TaskID is the unique identifier for this task. + payload: + type: array + description: Payload is the opaque data provided when the task was enqueued. + items: + type: object + attempt: + type: integer + format: int32 + description: 'Attempt is the current attempt number (1-based: first attempt = 1).' + required: + - queueName + - taskId + - payload + - attempt diff --git a/plugins/host/task.go b/plugins/host/task.go new file mode 100644 index 000000000..dcaf7197e --- /dev/null +++ b/plugins/host/task.go @@ -0,0 +1,73 @@ +package host + +import "context" + +// TaskInfo holds the current state of a task. +type TaskInfo struct { + // Status is the current task status: "pending", "running", + // "completed", "failed", or "cancelled". + Status string `json:"status"` + // Message is the status/result message returned by the plugin callback. + Message string `json:"message"` + // Attempt is the current or last attempt number (1-based). + Attempt int32 `json:"attempt"` +} + +// QueueConfig holds configuration for a task queue. +type QueueConfig struct { + // Concurrency is the max number of parallel workers. Default: 1. + // Capped by the plugin's manifest maxConcurrency. + Concurrency int32 `json:"concurrency"` + + // MaxRetries is the number of times to retry a failed task. Default: 0. + MaxRetries int32 `json:"maxRetries"` + + // BackoffMs is the initial backoff between retries in milliseconds. + // Doubles each retry (exponential: backoffMs * 2^(attempt-1)). Default: 1000. + BackoffMs int64 `json:"backoffMs"` + + // DelayMs is the minimum delay between starting consecutive tasks + // in milliseconds. Useful for rate limiting. Default: 0. + DelayMs int64 `json:"delayMs"` + + // RetentionMs is how long completed/failed/cancelled tasks are kept + // in milliseconds. Default: 3600000 (1h). Min: 60000 (1m). Max: 604800000 (1w). + RetentionMs int64 `json:"retentionMs"` +} + +// TaskService provides persistent task queues for plugins. +// +// This service allows plugins to create named queues with configurable concurrency, +// retry policies, and rate limiting. Tasks are persisted to SQLite and survive +// server restarts. When a task is ready to execute, the host calls the plugin's +// nd_task_execute callback function. +// +//nd:hostservice name=Task permission=taskqueue +type TaskService interface { + // CreateQueue creates a named task queue with the given configuration. + // Zero-value fields in config use sensible defaults. + // If a queue with the same name already exists, returns an error. + // On startup, this also recovers any stale "running" tasks from a previous crash. + //nd:hostfunc + CreateQueue(ctx context.Context, name string, config QueueConfig) error + + // Enqueue adds a task to the named queue. Returns the task ID. + // payload is opaque bytes passed back to the plugin on execution. + //nd:hostfunc + Enqueue(ctx context.Context, queueName string, payload []byte) (string, error) + + // Get returns the current state of a task including its status, + // message, and attempt count. + //nd:hostfunc + Get(ctx context.Context, taskID string) (*TaskInfo, error) + + // Cancel cancels a pending task. Returns error if already + // running, completed, or failed. + //nd:hostfunc + Cancel(ctx context.Context, taskID string) error + + // ClearQueue removes all pending tasks from the named queue. + // Running tasks are not affected. Returns the number of tasks removed. + //nd:hostfunc + ClearQueue(ctx context.Context, queueName string) (int64, error) +} diff --git a/plugins/host/task_gen.go b/plugins/host/task_gen.go new file mode 100644 index 000000000..e4864bb5b --- /dev/null +++ b/plugins/host/task_gen.go @@ -0,0 +1,266 @@ +// Code generated by ndpgen. DO NOT EDIT. + +package host + +import ( + "context" + "encoding/json" + + extism "github.com/extism/go-sdk" +) + +// TaskCreateQueueRequest is the request type for Task.CreateQueue. +type TaskCreateQueueRequest struct { + Name string `json:"name"` + Config QueueConfig `json:"config"` +} + +// TaskCreateQueueResponse is the response type for Task.CreateQueue. +type TaskCreateQueueResponse struct { + Error string `json:"error,omitempty"` +} + +// TaskEnqueueRequest is the request type for Task.Enqueue. +type TaskEnqueueRequest struct { + QueueName string `json:"queueName"` + Payload []byte `json:"payload"` +} + +// TaskEnqueueResponse is the response type for Task.Enqueue. +type TaskEnqueueResponse struct { + Result string `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +// TaskGetRequest is the request type for Task.Get. +type TaskGetRequest struct { + TaskID string `json:"taskId"` +} + +// TaskGetResponse is the response type for Task.Get. +type TaskGetResponse struct { + Result *TaskInfo `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +// TaskCancelRequest is the request type for Task.Cancel. +type TaskCancelRequest struct { + TaskID string `json:"taskId"` +} + +// TaskCancelResponse is the response type for Task.Cancel. +type TaskCancelResponse struct { + Error string `json:"error,omitempty"` +} + +// TaskClearQueueRequest is the request type for Task.ClearQueue. +type TaskClearQueueRequest struct { + QueueName string `json:"queueName"` +} + +// TaskClearQueueResponse is the response type for Task.ClearQueue. +type TaskClearQueueResponse struct { + Result int64 `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +// RegisterTaskHostFunctions registers Task service host functions. +// The returned host functions should be added to the plugin's configuration. +func RegisterTaskHostFunctions(service TaskService) []extism.HostFunction { + return []extism.HostFunction{ + newTaskCreateQueueHostFunction(service), + newTaskEnqueueHostFunction(service), + newTaskGetHostFunction(service), + newTaskCancelHostFunction(service), + newTaskClearQueueHostFunction(service), + } +} + +func newTaskCreateQueueHostFunction(service TaskService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "task_createqueue", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + taskWriteError(p, stack, err) + return + } + var req TaskCreateQueueRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + taskWriteError(p, stack, err) + return + } + + // Call the service method + if svcErr := service.CreateQueue(ctx, req.Name, req.Config); svcErr != nil { + taskWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := TaskCreateQueueResponse{} + taskWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + +func newTaskEnqueueHostFunction(service TaskService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "task_enqueue", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + taskWriteError(p, stack, err) + return + } + var req TaskEnqueueRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + taskWriteError(p, stack, err) + return + } + + // Call the service method + result, svcErr := service.Enqueue(ctx, req.QueueName, req.Payload) + if svcErr != nil { + taskWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := TaskEnqueueResponse{ + Result: result, + } + taskWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + +func newTaskGetHostFunction(service TaskService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "task_get", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + taskWriteError(p, stack, err) + return + } + var req TaskGetRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + taskWriteError(p, stack, err) + return + } + + // Call the service method + result, svcErr := service.Get(ctx, req.TaskID) + if svcErr != nil { + taskWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := TaskGetResponse{ + Result: result, + } + taskWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + +func newTaskCancelHostFunction(service TaskService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "task_cancel", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + taskWriteError(p, stack, err) + return + } + var req TaskCancelRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + taskWriteError(p, stack, err) + return + } + + // Call the service method + if svcErr := service.Cancel(ctx, req.TaskID); svcErr != nil { + taskWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := TaskCancelResponse{} + taskWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + +func newTaskClearQueueHostFunction(service TaskService) extism.HostFunction { + return extism.NewHostFunctionWithStack( + "task_clearqueue", + func(ctx context.Context, p *extism.CurrentPlugin, stack []uint64) { + // Read JSON request from plugin memory + reqBytes, err := p.ReadBytes(stack[0]) + if err != nil { + taskWriteError(p, stack, err) + return + } + var req TaskClearQueueRequest + if err := json.Unmarshal(reqBytes, &req); err != nil { + taskWriteError(p, stack, err) + return + } + + // Call the service method + result, svcErr := service.ClearQueue(ctx, req.QueueName) + if svcErr != nil { + taskWriteError(p, stack, svcErr) + return + } + + // Write JSON response to plugin memory + resp := TaskClearQueueResponse{ + Result: result, + } + taskWriteResponse(p, stack, resp) + }, + []extism.ValueType{extism.ValueTypePTR}, + []extism.ValueType{extism.ValueTypePTR}, + ) +} + +// taskWriteResponse writes a JSON response to plugin memory. +func taskWriteResponse(p *extism.CurrentPlugin, stack []uint64, resp any) { + respBytes, err := json.Marshal(resp) + if err != nil { + taskWriteError(p, stack, err) + return + } + respPtr, err := p.WriteBytes(respBytes) + if err != nil { + stack[0] = 0 + return + } + stack[0] = respPtr +} + +// taskWriteError writes an error response to plugin memory. +func taskWriteError(p *extism.CurrentPlugin, stack []uint64, err error) { + errResp := struct { + Error string `json:"error"` + }{Error: err.Error()} + respBytes, _ := json.Marshal(errResp) + respPtr, _ := p.WriteBytes(respBytes) + stack[0] = respPtr +} diff --git a/plugins/host_taskqueue.go b/plugins/host_taskqueue.go new file mode 100644 index 000000000..283bc9635 --- /dev/null +++ b/plugins/host_taskqueue.go @@ -0,0 +1,595 @@ +package plugins + +import ( + "context" + "database/sql" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "sync" + "time" + + _ "github.com/mattn/go-sqlite3" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model/id" + "github.com/navidrome/navidrome/plugins/capabilities" + "github.com/navidrome/navidrome/plugins/host" + "golang.org/x/time/rate" +) + +const ( + defaultConcurrency int32 = 1 + defaultBackoffMs int64 = 1000 + defaultRetentionMs int64 = 3_600_000 // 1 hour + minRetentionMs int64 = 60_000 // 1 minute + maxRetentionMs int64 = 604_800_000 // 1 week + maxQueueNameLength = 128 + maxPayloadSize = 1 * 1024 * 1024 // 1MB + maxBackoffMs int64 = 3_600_000 // 1 hour + taskCleanupInterval = 5 * time.Minute + pollInterval = 5 * time.Second + shutdownTimeout = 10 * time.Second + + taskStatusPending = "pending" + taskStatusRunning = "running" + taskStatusCompleted = "completed" + taskStatusFailed = "failed" + taskStatusCancelled = "cancelled" +) + +// CapabilityTaskWorker indicates the plugin can receive task execution callbacks. +const CapabilityTaskWorker Capability = "TaskWorker" + +const FuncTaskWorkerCallback = "nd_task_execute" + +func init() { + registerCapability(CapabilityTaskWorker, FuncTaskWorkerCallback) +} + +type queueState struct { + config host.QueueConfig + signal chan struct{} + limiter *rate.Limiter +} + +// notifyWorkers sends a non-blocking signal to wake up queue workers. +func (qs *queueState) notifyWorkers() { + select { + case qs.signal <- struct{}{}: + default: + } +} + +// taskQueueServiceImpl implements host.TaskQueueService with SQLite persistence +// and background worker goroutines for task execution. +type taskQueueServiceImpl struct { + pluginName string + manager *Manager + maxConcurrency int32 + db *sql.DB + ctx context.Context + cancel context.CancelFunc + wg sync.WaitGroup + mu sync.Mutex + queues map[string]*queueState + + // For testing: override how callbacks are invoked + invokeCallbackFn func(ctx context.Context, queueName, taskID string, payload []byte, attempt int32) (string, error) +} + +// newTaskQueueService creates a new taskQueueServiceImpl with its own SQLite database. +func newTaskQueueService(pluginName string, manager *Manager, maxConcurrency int32) (*taskQueueServiceImpl, error) { + dataDir := filepath.Join(conf.Server.DataFolder, "plugins", pluginName) + if err := os.MkdirAll(dataDir, 0700); err != nil { + return nil, fmt.Errorf("creating plugin data directory: %w", err) + } + + dbPath := filepath.Join(dataDir, "taskqueue.db") + db, err := sql.Open("sqlite3", dbPath+"?_busy_timeout=5000&_journal_mode=WAL&_foreign_keys=off") + if err != nil { + return nil, fmt.Errorf("opening taskqueue database: %w", err) + } + + db.SetMaxOpenConns(3) + db.SetMaxIdleConns(1) + + if err := createTaskQueueSchema(db); err != nil { + db.Close() + return nil, fmt.Errorf("creating taskqueue schema: %w", err) + } + + ctx, cancel := context.WithCancel(manager.ctx) + + s := &taskQueueServiceImpl{ + pluginName: pluginName, + manager: manager, + maxConcurrency: maxConcurrency, + db: db, + ctx: ctx, + cancel: cancel, + queues: make(map[string]*queueState), + } + s.invokeCallbackFn = s.defaultInvokeCallback + + s.wg.Go(s.cleanupLoop) + + log.Debug("Initialized plugin taskqueue", "plugin", pluginName, "path", dbPath, "maxConcurrency", maxConcurrency) + return s, nil +} + +// createTaskQueueSchema applies schema migrations to the taskqueue database. +// New migrations must be appended at the end of the slice. +func createTaskQueueSchema(db *sql.DB) error { + return migrateDB(db, []string{ + `CREATE TABLE IF NOT EXISTS queues ( + name TEXT PRIMARY KEY, + concurrency INTEGER NOT NULL DEFAULT 1, + max_retries INTEGER NOT NULL DEFAULT 0, + backoff_ms INTEGER NOT NULL DEFAULT 1000, + delay_ms INTEGER NOT NULL DEFAULT 0, + retention_ms INTEGER NOT NULL DEFAULT 3600000 + )`, + `CREATE TABLE IF NOT EXISTS tasks ( + id TEXT PRIMARY KEY, + queue_name TEXT NOT NULL REFERENCES queues(name), + payload BLOB NOT NULL, + status TEXT NOT NULL DEFAULT 'pending', + attempt INTEGER NOT NULL DEFAULT 0, + max_retries INTEGER NOT NULL, + next_run_at INTEGER NOT NULL, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL, + message TEXT NOT NULL DEFAULT '' + )`, + `CREATE INDEX IF NOT EXISTS idx_tasks_dequeue ON tasks(queue_name, status, next_run_at)`, + }) +} + +// applyConfigDefaults fills zero-value config fields with sensible defaults +// and clamps values to valid ranges, logging warnings for clamped values. +func (s *taskQueueServiceImpl) applyConfigDefaults(ctx context.Context, name string, config *host.QueueConfig) { + if config.Concurrency <= 0 { + config.Concurrency = defaultConcurrency + } + if config.BackoffMs <= 0 { + config.BackoffMs = defaultBackoffMs + } + if config.RetentionMs <= 0 { + config.RetentionMs = defaultRetentionMs + } + + if config.RetentionMs < minRetentionMs { + log.Warn(ctx, "TaskQueue retention clamped to minimum", "plugin", s.pluginName, "queue", name, + "requested", config.RetentionMs, "min", minRetentionMs) + config.RetentionMs = minRetentionMs + } + if config.RetentionMs > maxRetentionMs { + log.Warn(ctx, "TaskQueue retention clamped to maximum", "plugin", s.pluginName, "queue", name, + "requested", config.RetentionMs, "max", maxRetentionMs) + config.RetentionMs = maxRetentionMs + } +} + +// clampConcurrency reduces config.Concurrency if it exceeds the remaining budget. +// Returns an error when the concurrency budget is fully exhausted. +// Must be called with s.mu held. +func (s *taskQueueServiceImpl) clampConcurrency(ctx context.Context, name string, config *host.QueueConfig) error { + var allocated int32 + for _, qs := range s.queues { + allocated += qs.config.Concurrency + } + available := s.maxConcurrency - allocated + if available <= 0 { + log.Warn(ctx, "TaskQueue concurrency budget exhausted", "plugin", s.pluginName, "queue", name, + "allocated", allocated, "maxConcurrency", s.maxConcurrency) + return fmt.Errorf("concurrency budget exhausted (%d/%d allocated)", allocated, s.maxConcurrency) + } + if config.Concurrency > available { + log.Warn(ctx, "TaskQueue concurrency clamped", "plugin", s.pluginName, "queue", name, + "requested", config.Concurrency, "available", available, "maxConcurrency", s.maxConcurrency) + config.Concurrency = available + } + return nil +} + +func (s *taskQueueServiceImpl) CreateQueue(ctx context.Context, name string, config host.QueueConfig) error { + if len(name) == 0 { + return fmt.Errorf("queue name cannot be empty") + } + if len(name) > maxQueueNameLength { + return fmt.Errorf("queue name exceeds maximum length of %d bytes", maxQueueNameLength) + } + + s.applyConfigDefaults(ctx, name, &config) + + s.mu.Lock() + defer s.mu.Unlock() + + if err := s.clampConcurrency(ctx, name, &config); err != nil { + return err + } + + if _, exists := s.queues[name]; exists { + return fmt.Errorf("queue %q already exists", name) + } + + // Upsert into queues table (idempotent across restarts) + _, err := s.db.ExecContext(ctx, ` + INSERT INTO queues (name, concurrency, max_retries, backoff_ms, delay_ms, retention_ms) + VALUES (?, ?, ?, ?, ?, ?) + ON CONFLICT(name) DO UPDATE SET + concurrency = excluded.concurrency, + max_retries = excluded.max_retries, + backoff_ms = excluded.backoff_ms, + delay_ms = excluded.delay_ms, + retention_ms = excluded.retention_ms + `, name, config.Concurrency, config.MaxRetries, config.BackoffMs, config.DelayMs, config.RetentionMs) + if err != nil { + return fmt.Errorf("creating queue: %w", err) + } + + // Reset stale running tasks from previous crash + now := time.Now().UnixMilli() + _, err = s.db.ExecContext(ctx, ` + UPDATE tasks SET status = ?, updated_at = ? WHERE queue_name = ? AND status = ? + `, taskStatusPending, now, name, taskStatusRunning) + if err != nil { + return fmt.Errorf("resetting stale tasks: %w", err) + } + + qs := &queueState{ + config: config, + signal: make(chan struct{}, 1), + } + if config.DelayMs > 0 { + // Rate limit dispatches to enforce delay between tasks. + // Burst of 1 allows one immediate dispatch, then enforces the delay interval. + qs.limiter = rate.NewLimiter(rate.Every(time.Duration(config.DelayMs)*time.Millisecond), 1) + } + s.queues[name] = qs + + for i := int32(0); i < config.Concurrency; i++ { + s.wg.Go(func() { s.worker(name, qs) }) + } + + log.Debug(ctx, "Created task queue", "plugin", s.pluginName, "queue", name, + "concurrency", config.Concurrency, "maxRetries", config.MaxRetries, + "backoffMs", config.BackoffMs, "delayMs", config.DelayMs, "retentionMs", config.RetentionMs) + return nil +} + +func (s *taskQueueServiceImpl) Enqueue(ctx context.Context, queueName string, payload []byte) (string, error) { + s.mu.Lock() + qs, exists := s.queues[queueName] + s.mu.Unlock() + + if !exists { + return "", fmt.Errorf("queue %q does not exist", queueName) + } + if len(payload) > maxPayloadSize { + return "", fmt.Errorf("payload size %d exceeds maximum of %d bytes", len(payload), maxPayloadSize) + } + + taskID := id.NewRandom() + now := time.Now().UnixMilli() + + _, err := s.db.ExecContext(ctx, ` + INSERT INTO tasks (id, queue_name, payload, status, attempt, max_retries, next_run_at, created_at, updated_at) + VALUES (?, ?, ?, ?, 0, ?, ?, ?, ?) + `, taskID, queueName, payload, taskStatusPending, qs.config.MaxRetries, now, now, now) + if err != nil { + return "", fmt.Errorf("enqueuing task: %w", err) + } + + qs.notifyWorkers() + log.Trace(ctx, "Enqueued task", "plugin", s.pluginName, "queue", queueName, "taskID", taskID) + return taskID, nil +} + +// Get returns the current state of a task. +func (s *taskQueueServiceImpl) Get(ctx context.Context, taskID string) (*host.TaskInfo, error) { + var info host.TaskInfo + err := s.db.QueryRowContext(ctx, `SELECT status, message, attempt FROM tasks WHERE id = ?`, taskID). + Scan(&info.Status, &info.Message, &info.Attempt) + if errors.Is(err, sql.ErrNoRows) { + return nil, fmt.Errorf("task %q not found", taskID) + } + if err != nil { + return nil, fmt.Errorf("getting task info: %w", err) + } + return &info, nil +} + +// Cancel cancels a pending task. +func (s *taskQueueServiceImpl) Cancel(ctx context.Context, taskID string) error { + now := time.Now().UnixMilli() + result, err := s.db.ExecContext(ctx, ` + UPDATE tasks SET status = ?, updated_at = ? WHERE id = ? AND status = ? + `, taskStatusCancelled, now, taskID, taskStatusPending) + if err != nil { + return fmt.Errorf("cancelling task: %w", err) + } + + rowsAffected, err := result.RowsAffected() + if err != nil { + return fmt.Errorf("checking cancel result: %w", err) + } + + if rowsAffected == 0 { + // Check if task exists at all + var status string + err := s.db.QueryRowContext(ctx, `SELECT status FROM tasks WHERE id = ?`, taskID).Scan(&status) + if errors.Is(err, sql.ErrNoRows) { + return fmt.Errorf("task %q not found", taskID) + } + if err != nil { + return fmt.Errorf("checking task existence: %w", err) + } + return fmt.Errorf("task %q cannot be cancelled (status: %s)", taskID, status) + } + + log.Trace(ctx, "Cancelled task", "plugin", s.pluginName, "taskID", taskID) + return nil +} + +// ClearQueue removes all pending tasks from the named queue. +// Running tasks are not affected. Returns the number of tasks removed. +func (s *taskQueueServiceImpl) ClearQueue(ctx context.Context, queueName string) (int64, error) { + s.mu.Lock() + _, exists := s.queues[queueName] + s.mu.Unlock() + + if !exists { + return 0, fmt.Errorf("queue %q does not exist", queueName) + } + + now := time.Now().UnixMilli() + result, err := s.db.ExecContext(ctx, ` + UPDATE tasks SET status = ?, updated_at = ? WHERE queue_name = ? AND status = ? + `, taskStatusCancelled, now, queueName, taskStatusPending) + if err != nil { + return 0, fmt.Errorf("clearing queue: %w", err) + } + + cleared, err := result.RowsAffected() + if err != nil { + return 0, fmt.Errorf("checking clear result: %w", err) + } + + if cleared > 0 { + log.Debug(ctx, "Cleared pending tasks from queue", "plugin", s.pluginName, "queue", queueName, "cleared", cleared) + } + return cleared, nil +} + +// worker is the main loop for a single worker goroutine. +func (s *taskQueueServiceImpl) worker(queueName string, qs *queueState) { + // Process any existing pending tasks immediately on startup + s.drainQueue(queueName, qs) + + ticker := time.NewTicker(pollInterval) + defer ticker.Stop() + + for { + select { + case <-s.ctx.Done(): + return + case <-qs.signal: + s.drainQueue(queueName, qs) + case <-ticker.C: + s.drainQueue(queueName, qs) + } + } +} + +func (s *taskQueueServiceImpl) drainQueue(queueName string, qs *queueState) { + for s.ctx.Err() == nil && s.processTask(queueName, qs) { + } +} + +// processTask dequeues and processes a single task. Returns true if a task was processed. +func (s *taskQueueServiceImpl) processTask(queueName string, qs *queueState) bool { + now := time.Now().UnixMilli() + + // Atomically dequeue a task + var taskID string + var payload []byte + var attempt, maxRetries int32 + err := s.db.QueryRowContext(s.ctx, ` + UPDATE tasks SET status = ?, attempt = attempt + 1, updated_at = ? + WHERE id = ( + SELECT id FROM tasks + WHERE queue_name = ? AND status = ? AND next_run_at <= ? + ORDER BY next_run_at, created_at LIMIT 1 + ) + RETURNING id, payload, attempt, max_retries + `, taskStatusRunning, now, queueName, taskStatusPending, now).Scan(&taskID, &payload, &attempt, &maxRetries) + if errors.Is(err, sql.ErrNoRows) { + return false + } + if err != nil { + log.Error(s.ctx, "Failed to dequeue task", "plugin", s.pluginName, "queue", queueName, err) + return false + } + + // Enforce delay between task dispatches using a rate limiter. + // This is done after dequeue so that empty polls don't consume rate tokens. + if qs.limiter != nil { + if err := qs.limiter.Wait(s.ctx); err != nil { + // Context cancelled during wait — revert task to pending for recovery + s.revertTaskToPending(taskID) + return false + } + } + + // Invoke callback + log.Debug(s.ctx, "Executing task", "plugin", s.pluginName, "queue", queueName, "taskID", taskID, "attempt", attempt) + message, callbackErr := s.invokeCallbackFn(s.ctx, queueName, taskID, payload, attempt) + + // If context was cancelled (shutdown), revert task to pending for recovery + if s.ctx.Err() != nil { + s.revertTaskToPending(taskID) + return false + } + + if callbackErr == nil { + s.completeTask(queueName, taskID, message) + } else { + s.handleTaskFailure(queueName, taskID, attempt, maxRetries, qs, callbackErr, message) + } + return true +} + +func (s *taskQueueServiceImpl) completeTask(queueName, taskID, message string) { + now := time.Now().UnixMilli() + if _, err := s.db.ExecContext(s.ctx, `UPDATE tasks SET status = ?, message = ?, updated_at = ? WHERE id = ?`, taskStatusCompleted, message, now, taskID); err != nil { + log.Error(s.ctx, "Failed to mark task as completed", "plugin", s.pluginName, "taskID", taskID, err) + } + log.Debug(s.ctx, "Task completed", "plugin", s.pluginName, "queue", queueName, "taskID", taskID) +} + +func (s *taskQueueServiceImpl) handleTaskFailure(queueName, taskID string, attempt, maxRetries int32, qs *queueState, callbackErr error, message string) { + log.Warn(s.ctx, "Task execution failed", "plugin", s.pluginName, "queue", queueName, + "taskID", taskID, "attempt", attempt, "maxRetries", maxRetries, "err", callbackErr) + + // Use error message as fallback if no message was provided + if message == "" { + message = callbackErr.Error() + } + + now := time.Now().UnixMilli() + if attempt > maxRetries { + if _, err := s.db.ExecContext(s.ctx, `UPDATE tasks SET status = ?, message = ?, updated_at = ? WHERE id = ?`, taskStatusFailed, message, now, taskID); err != nil { + log.Error(s.ctx, "Failed to mark task as failed", "plugin", s.pluginName, "taskID", taskID, err) + } + log.Warn(s.ctx, "Task failed after all retries", "plugin", s.pluginName, "queue", queueName, "taskID", taskID) + return + } + + // Exponential backoff: backoffMs * 2^(attempt-1) + backoff := qs.config.BackoffMs << (attempt - 1) + if backoff <= 0 || backoff > maxBackoffMs { + backoff = maxBackoffMs + } + nextRunAt := now + backoff + if _, err := s.db.ExecContext(s.ctx, ` + UPDATE tasks SET status = ?, next_run_at = ?, updated_at = ? WHERE id = ? + `, taskStatusPending, nextRunAt, now, taskID); err != nil { + log.Error(s.ctx, "Failed to reschedule task for retry", "plugin", s.pluginName, "taskID", taskID, err) + } + + // Wake worker after backoff expires + time.AfterFunc(time.Duration(backoff)*time.Millisecond, func() { + qs.notifyWorkers() + }) +} + +// revertTaskToPending puts a running task back to pending status and decrements the attempt +// counter (used during shutdown to ensure the interrupted attempt doesn't count). +func (s *taskQueueServiceImpl) revertTaskToPending(taskID string) { + now := time.Now().UnixMilli() + _, err := s.db.Exec(`UPDATE tasks SET status = ?, attempt = MAX(attempt - 1, 0), updated_at = ? WHERE id = ? AND status = ?`, taskStatusPending, now, taskID, taskStatusRunning) + if err != nil { + log.Error("Failed to revert task to pending", "plugin", s.pluginName, "taskID", taskID, err) + } +} + +// defaultInvokeCallback calls the plugin's nd_task_execute function. +func (s *taskQueueServiceImpl) defaultInvokeCallback(ctx context.Context, queueName, taskID string, payload []byte, attempt int32) (string, error) { + s.manager.mu.RLock() + p, ok := s.manager.plugins[s.pluginName] + s.manager.mu.RUnlock() + + if !ok { + return "", fmt.Errorf("plugin %s not loaded", s.pluginName) + } + + input := capabilities.TaskExecuteRequest{ + QueueName: queueName, + TaskID: taskID, + Payload: payload, + Attempt: attempt, + } + + message, err := callPluginFunction[capabilities.TaskExecuteRequest, string](ctx, p, FuncTaskWorkerCallback, input) + if err != nil { + return "", err + } + return message, nil +} + +// cleanupLoop periodically removes terminal tasks past their retention period. +func (s *taskQueueServiceImpl) cleanupLoop() { + ticker := time.NewTicker(taskCleanupInterval) + defer ticker.Stop() + + for { + select { + case <-s.ctx.Done(): + return + case <-ticker.C: + s.runCleanup() + } + } +} + +// runCleanup deletes terminal tasks past their retention period. +func (s *taskQueueServiceImpl) runCleanup() { + s.mu.Lock() + queues := make(map[string]*queueState, len(s.queues)) + for k, v := range s.queues { + queues[k] = v + } + s.mu.Unlock() + + now := time.Now().UnixMilli() + for name, qs := range queues { + result, err := s.db.ExecContext(s.ctx, ` + DELETE FROM tasks WHERE queue_name = ? AND status IN (?, ?, ?) AND updated_at + ? < ? + `, name, taskStatusCompleted, taskStatusFailed, taskStatusCancelled, qs.config.RetentionMs, now) + if err != nil { + log.Error(s.ctx, "Failed to cleanup tasks", "plugin", s.pluginName, "queue", name, err) + continue + } + if deleted, _ := result.RowsAffected(); deleted > 0 { + log.Debug(s.ctx, "Cleaned up terminal tasks", "plugin", s.pluginName, "queue", name, "deleted", deleted) + } + } +} + +// Close shuts down the task queue service, stopping all workers and closing the database. +func (s *taskQueueServiceImpl) Close() error { + // Cancel context to signal all goroutines + s.cancel() + + // Wait for goroutines with timeout + done := make(chan struct{}) + go func() { + s.wg.Wait() + close(done) + }() + + select { + case <-done: + case <-time.After(shutdownTimeout): + log.Warn("TaskQueue shutdown timed out", "plugin", s.pluginName) + } + + // Mark running tasks as pending for recovery on next startup + if s.db != nil { + now := time.Now().UnixMilli() + if _, err := s.db.Exec(`UPDATE tasks SET status = ?, updated_at = ? WHERE status = ?`, taskStatusPending, now, taskStatusRunning); err != nil { + log.Error("Failed to reset running tasks on shutdown", "plugin", s.pluginName, err) + } + log.Debug("Closing plugin taskqueue", "plugin", s.pluginName) + return s.db.Close() + } + return nil +} + +// Compile-time verification +var _ host.TaskService = (*taskQueueServiceImpl)(nil) +var _ io.Closer = (*taskQueueServiceImpl)(nil) diff --git a/plugins/host_taskqueue_test.go b/plugins/host_taskqueue_test.go new file mode 100644 index 000000000..c3ab8d119 --- /dev/null +++ b/plugins/host_taskqueue_test.go @@ -0,0 +1,1221 @@ +//go:build !windows + +package plugins + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "net/http" + "os" + "path/filepath" + "sort" + "strings" + "sync" + "sync/atomic" + "time" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/plugins/host" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("TaskQueueService", func() { + var tmpDir string + var service *taskQueueServiceImpl + var ctx context.Context + var manager *Manager + + BeforeEach(func() { + ctx = GinkgoT().Context() + var err error + tmpDir, err = os.MkdirTemp("", "taskqueue-test-*") + Expect(err).ToNot(HaveOccurred()) + + DeferCleanup(configtest.SetupConfig()) + conf.Server.DataFolder = tmpDir + + // Create a mock manager with context + managerCtx, cancel := context.WithCancel(ctx) + manager = &Manager{ + plugins: make(map[string]*plugin), + ctx: managerCtx, + } + DeferCleanup(cancel) + + service, err = newTaskQueueService("test_plugin", manager, 5) + Expect(err).ToNot(HaveOccurred()) + }) + + AfterEach(func() { + if service != nil { + service.Close() + } + os.RemoveAll(tmpDir) + }) + + Describe("CreateQueue", func() { + It("creates a queue successfully", func() { + err := service.CreateQueue(ctx, "my-queue", host.QueueConfig{ + Concurrency: 2, + MaxRetries: 3, + BackoffMs: 2000, + RetentionMs: 7200000, + }) + Expect(err).ToNot(HaveOccurred()) + + service.mu.Lock() + qs, exists := service.queues["my-queue"] + service.mu.Unlock() + Expect(exists).To(BeTrue()) + Expect(qs.config.Concurrency).To(Equal(int32(2))) + Expect(qs.config.MaxRetries).To(Equal(int32(3))) + Expect(qs.config.BackoffMs).To(Equal(int64(2000))) + Expect(qs.config.RetentionMs).To(Equal(int64(7200000))) + }) + + It("returns error for duplicate queue name", func() { + err := service.CreateQueue(ctx, "dup-queue", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + err = service.CreateQueue(ctx, "dup-queue", host.QueueConfig{}) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("already exists")) + }) + }) + + Describe("CreateQueue name validation", func() { + It("rejects empty queue name", func() { + err := service.CreateQueue(ctx, "", host.QueueConfig{}) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("queue name cannot be empty")) + }) + + It("rejects over-length queue name", func() { + longName := strings.Repeat("a", maxQueueNameLength+1) + err := service.CreateQueue(ctx, longName, host.QueueConfig{}) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("exceeds maximum length")) + }) + + It("accepts queue name at maximum length", func() { + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "", nil + } + exactName := strings.Repeat("a", maxQueueNameLength) + err := service.CreateQueue(ctx, exactName, host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + }) + }) + + Describe("CreateQueue defaults", func() { + It("applies defaults for zero-value config", func() { + err := service.CreateQueue(ctx, "defaults-queue", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + service.mu.Lock() + qs := service.queues["defaults-queue"] + service.mu.Unlock() + Expect(qs.config.Concurrency).To(Equal(defaultConcurrency)) + Expect(qs.config.BackoffMs).To(Equal(defaultBackoffMs)) + Expect(qs.config.RetentionMs).To(Equal(defaultRetentionMs)) + }) + }) + + Describe("CreateQueue defaults with negative values", func() { + It("applies default RetentionMs for negative value", func() { + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "", nil + } + err := service.CreateQueue(ctx, "neg-retention", host.QueueConfig{ + RetentionMs: -500, + }) + Expect(err).ToNot(HaveOccurred()) + + service.mu.Lock() + qs := service.queues["neg-retention"] + service.mu.Unlock() + Expect(qs.config.RetentionMs).To(Equal(defaultRetentionMs)) + }) + }) + + Describe("CreateQueue clamping", func() { + It("clamps concurrency exceeding maxConcurrency", func() { + // maxConcurrency is 5; request 10 + err := service.CreateQueue(ctx, "clamped-queue", host.QueueConfig{ + Concurrency: 10, + }) + Expect(err).ToNot(HaveOccurred()) + + service.mu.Lock() + qs := service.queues["clamped-queue"] + service.mu.Unlock() + Expect(qs.config.Concurrency).To(Equal(int32(5))) + }) + + It("returns error when concurrency budget is exhausted", func() { + // maxConcurrency is 5; create a queue that uses all 5 + err := service.CreateQueue(ctx, "full-budget", host.QueueConfig{ + Concurrency: 5, + }) + Expect(err).ToNot(HaveOccurred()) + + // Next queue should fail — no budget remaining + err = service.CreateQueue(ctx, "over-budget", host.QueueConfig{ + Concurrency: 1, + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("concurrency budget exhausted")) + }) + + It("clamps retention below minimum", func() { + err := service.CreateQueue(ctx, "low-retention", host.QueueConfig{ + RetentionMs: 100, // below minRetentionMs + }) + Expect(err).ToNot(HaveOccurred()) + + service.mu.Lock() + qs := service.queues["low-retention"] + service.mu.Unlock() + Expect(qs.config.RetentionMs).To(Equal(minRetentionMs)) + }) + + It("clamps retention above maximum", func() { + err := service.CreateQueue(ctx, "high-retention", host.QueueConfig{ + RetentionMs: 999_999_999_999, // above maxRetentionMs + }) + Expect(err).ToNot(HaveOccurred()) + + service.mu.Lock() + qs := service.queues["high-retention"] + service.mu.Unlock() + Expect(qs.config.RetentionMs).To(Equal(maxRetentionMs)) + }) + }) + + Describe("Enqueue", func() { + BeforeEach(func() { + // Use a no-op callback to prevent actual execution attempts + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "", nil + } + err := service.CreateQueue(ctx, "enqueue-test", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + }) + + It("enqueues a task and returns task ID", func() { + taskID, err := service.Enqueue(ctx, "enqueue-test", []byte("payload")) + Expect(err).ToNot(HaveOccurred()) + Expect(taskID).ToNot(BeEmpty()) + }) + + It("returns error for non-existent queue", func() { + _, err := service.Enqueue(ctx, "no-such-queue", []byte("payload")) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("does not exist")) + }) + + It("rejects payload exceeding maximum size", func() { + bigPayload := make([]byte, maxPayloadSize+1) + _, err := service.Enqueue(ctx, "enqueue-test", bigPayload) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("exceeds maximum")) + }) + + It("accepts payload at maximum size", func() { + exactPayload := make([]byte, maxPayloadSize) + taskID, err := service.Enqueue(ctx, "enqueue-test", exactPayload) + Expect(err).ToNot(HaveOccurred()) + Expect(taskID).ToNot(BeEmpty()) + }) + }) + + Describe("GetTaskStatus", func() { + BeforeEach(func() { + // Use a callback that blocks until context is cancelled so tasks stay pending + service.invokeCallbackFn = func(ctx context.Context, _, _ string, _ []byte, _ int32) (string, error) { + <-ctx.Done() + return "", ctx.Err() + } + }) + + It("returns pending for a new task", func() { + err := service.CreateQueue(ctx, "status-test", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "status-test", []byte("data")) + Expect(err).ToNot(HaveOccurred()) + + // The task may get picked up quickly; check initial status + // Since the callback blocks, it should be either pending or running + info, err := service.Get(ctx, taskID) + Expect(err).ToNot(HaveOccurred()) + Expect(info).ToNot(BeNil()) + Expect(info.Status).To(BeElementOf("pending", "running")) + }) + + It("returns error for unknown task ID", func() { + _, err := service.Get(ctx, "nonexistent-id") + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("not found")) + }) + }) + + Describe("CancelTask", func() { + BeforeEach(func() { + // Block callback so tasks stay in pending/running + service.invokeCallbackFn = func(ctx context.Context, _, _ string, _ []byte, _ int32) (string, error) { + <-ctx.Done() + return "", ctx.Err() + } + }) + + It("cancels a pending task", func() { + // Block the callback so the first task occupies the worker + started := make(chan struct{}) + service.invokeCallbackFn = func(ctx context.Context, _, _ string, _ []byte, _ int32) (string, error) { + close(started) + <-ctx.Done() + return "", ctx.Err() + } + + err := service.CreateQueue(ctx, "cancel-test", host.QueueConfig{ + Concurrency: 1, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue a blocker task to occupy the single worker + _, err = service.Enqueue(ctx, "cancel-test", []byte("blocker")) + Expect(err).ToNot(HaveOccurred()) + + // Wait for the blocker task to start running + Eventually(started).WithTimeout(5 * time.Second).Should(BeClosed()) + + // Enqueue a second task — it stays pending since the worker is busy + taskID, err := service.Enqueue(ctx, "cancel-test", []byte("cancel-me")) + Expect(err).ToNot(HaveOccurred()) + + err = service.Cancel(ctx, taskID) + Expect(err).ToNot(HaveOccurred()) + + info, err := service.Get(ctx, taskID) + Expect(err).ToNot(HaveOccurred()) + Expect(info.Status).To(Equal("cancelled")) + }) + + It("returns error for unknown task ID", func() { + err := service.Cancel(ctx, "nonexistent-id") + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("not found")) + }) + + It("returns error for non-pending task", func() { + // Create a queue where tasks complete immediately + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "", nil + } + err := service.CreateQueue(ctx, "completed-test", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "completed-test", []byte("data")) + Expect(err).ToNot(HaveOccurred()) + + // Wait for task to complete + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + + // Try to cancel completed task + err = service.Cancel(ctx, taskID) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("cannot be cancelled")) + }) + }) + + Describe("ClearQueue", func() { + It("clears all pending tasks from a queue", func() { + // Block the callback so the first task occupies the worker + started := make(chan struct{}) + service.invokeCallbackFn = func(ctx context.Context, _, _ string, _ []byte, _ int32) (string, error) { + close(started) + <-ctx.Done() + return "", ctx.Err() + } + + err := service.CreateQueue(ctx, "clear-test", host.QueueConfig{ + Concurrency: 1, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue a blocker task to occupy the single worker + _, err = service.Enqueue(ctx, "clear-test", []byte("blocker")) + Expect(err).ToNot(HaveOccurred()) + + // Wait for the blocker task to start running + Eventually(started).WithTimeout(5 * time.Second).Should(BeClosed()) + + // Enqueue several more tasks — they stay pending since the worker is busy + var pendingIDs []string + for i := 0; i < 3; i++ { + taskID, err := service.Enqueue(ctx, "clear-test", []byte(fmt.Sprintf("task-%d", i))) + Expect(err).ToNot(HaveOccurred()) + pendingIDs = append(pendingIDs, taskID) + } + + // Clear the queue + cleared, err := service.ClearQueue(ctx, "clear-test") + Expect(err).ToNot(HaveOccurred()) + Expect(cleared).To(Equal(int64(3))) + + // Verify all pending tasks are now cancelled + for _, id := range pendingIDs { + info, err := service.Get(ctx, id) + Expect(err).ToNot(HaveOccurred()) + Expect(info.Status).To(Equal("cancelled")) + } + }) + + It("returns zero when queue has no pending tasks", func() { + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "", nil + } + err := service.CreateQueue(ctx, "empty-clear", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + cleared, err := service.ClearQueue(ctx, "empty-clear") + Expect(err).ToNot(HaveOccurred()) + Expect(cleared).To(Equal(int64(0))) + }) + + It("returns error for non-existent queue", func() { + _, err := service.ClearQueue(ctx, "no-such-queue") + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("does not exist")) + }) + + It("does not affect running tasks", func() { + // Block the callback so tasks stay running + started := make(chan struct{}, 1) + service.invokeCallbackFn = func(ctx context.Context, _, _ string, _ []byte, _ int32) (string, error) { + select { + case started <- struct{}{}: + default: + } + <-ctx.Done() + return "", ctx.Err() + } + + err := service.CreateQueue(ctx, "clear-running", host.QueueConfig{ + Concurrency: 1, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue a task that will start running + runningID, err := service.Enqueue(ctx, "clear-running", []byte("running-task")) + Expect(err).ToNot(HaveOccurred()) + + // Wait for it to start running + Eventually(started).WithTimeout(5 * time.Second).Should(Receive()) + + // Clear the queue — should not affect the running task + cleared, err := service.ClearQueue(ctx, "clear-running") + Expect(err).ToNot(HaveOccurred()) + Expect(cleared).To(Equal(int64(0))) + + // Verify the running task is still running + info, err := service.Get(ctx, runningID) + Expect(err).ToNot(HaveOccurred()) + Expect(info.Status).To(Equal("running")) + }) + }) + + Describe("Worker execution", func() { + It("invokes callback and completes task", func() { + var callCount atomic.Int32 + var receivedQueueName, receivedTaskID string + var receivedPayload []byte + var receivedAttempt int32 + + service.invokeCallbackFn = func(_ context.Context, queueName, taskID string, payload []byte, attempt int32) (string, error) { + callCount.Add(1) + receivedQueueName = queueName + receivedTaskID = taskID + receivedPayload = payload + receivedAttempt = attempt + return "", nil + } + + err := service.CreateQueue(ctx, "worker-test", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "worker-test", []byte("test-payload")) + Expect(err).ToNot(HaveOccurred()) + + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + + Expect(callCount.Load()).To(Equal(int32(1))) + Expect(receivedQueueName).To(Equal("worker-test")) + Expect(receivedTaskID).To(Equal(taskID)) + Expect(receivedPayload).To(Equal([]byte("test-payload"))) + Expect(receivedAttempt).To(Equal(int32(1))) + }) + }) + + Describe("Message storage", func() { + It("stores message on successful completion", func() { + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "task completed successfully", nil + } + + err := service.CreateQueue(ctx, "msg-success", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "msg-success", []byte("data")) + Expect(err).ToNot(HaveOccurred()) + + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + + info, err := service.Get(ctx, taskID) + Expect(err).ToNot(HaveOccurred()) + Expect(info.Message).To(Equal("task completed successfully")) + Expect(info.Attempt).To(Equal(int32(1))) + }) + + It("stores error message on failure", func() { + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "", fmt.Errorf("something went wrong") + } + + err := service.CreateQueue(ctx, "msg-fail", host.QueueConfig{ + MaxRetries: 0, + }) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "msg-fail", []byte("data")) + Expect(err).ToNot(HaveOccurred()) + + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("failed")) + + info, err := service.Get(ctx, taskID) + Expect(err).ToNot(HaveOccurred()) + Expect(info.Message).To(Equal("something went wrong")) + }) + + It("uses explicit message over error message on failure", func() { + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "partial progress made", fmt.Errorf("timeout exceeded") + } + + err := service.CreateQueue(ctx, "msg-fail-with-msg", host.QueueConfig{ + MaxRetries: 0, + }) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "msg-fail-with-msg", []byte("data")) + Expect(err).ToNot(HaveOccurred()) + + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("failed")) + + info, err := service.Get(ctx, taskID) + Expect(err).ToNot(HaveOccurred()) + Expect(info.Message).To(Equal("partial progress made")) + }) + }) + + Describe("Retry on failure", func() { + It("retries and eventually fails after exhausting retries", func() { + var callCount atomic.Int32 + + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + callCount.Add(1) + return "", fmt.Errorf("task failed") + } + + err := service.CreateQueue(ctx, "retry-test", host.QueueConfig{ + MaxRetries: 2, + BackoffMs: 10, // Very short for testing + }) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "retry-test", []byte("retry-payload")) + Expect(err).ToNot(HaveOccurred()) + + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(10 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("failed")) + + // 1 initial attempt + 2 retries = 3 total calls + Expect(callCount.Load()).To(Equal(int32(3))) + }) + }) + + Describe("Retry then succeed", func() { + It("retries and succeeds on second attempt", func() { + var callCount atomic.Int32 + + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, attempt int32) (string, error) { + callCount.Add(1) + if attempt == 1 { + return "", fmt.Errorf("temporary error") + } + return "success", nil + } + + err := service.CreateQueue(ctx, "retry-succeed", host.QueueConfig{ + MaxRetries: 1, + BackoffMs: 10, // Very short for testing + }) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "retry-succeed", []byte("data")) + Expect(err).ToNot(HaveOccurred()) + + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(10 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + + Expect(callCount.Load()).To(Equal(int32(2))) + }) + }) + + Describe("Backoff overflow cap", func() { + It("caps backoff at maxRetentionMs to prevent overflow", func() { + var callCount atomic.Int32 + + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + callCount.Add(1) + return "", fmt.Errorf("always fail") + } + + err := service.CreateQueue(ctx, "backoff-overflow", host.QueueConfig{ + MaxRetries: 3, + BackoffMs: 1_000_000_000, // Very large backoff to trigger overflow on exponentiation + }) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "backoff-overflow", []byte("overflow-test")) + Expect(err).ToNot(HaveOccurred()) + + // Wait for first attempt to fail + Eventually(func() int32 { + return callCount.Load() + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(BeNumerically(">=", int32(1))) + + // Check next_run_at is positive and reasonable (capped at maxRetentionMs from now) + var nextRunAt int64 + err = service.db.QueryRow(`SELECT next_run_at FROM tasks WHERE id = ?`, taskID).Scan(&nextRunAt) + Expect(err).ToNot(HaveOccurred()) + + now := time.Now().UnixMilli() + Expect(nextRunAt).To(BeNumerically(">", int64(0)), "next_run_at should be positive") + Expect(nextRunAt).To(BeNumerically("<=", now+maxBackoffMs+1000), "next_run_at should be at most maxBackoffMs from now") + }) + }) + + Describe("Delay enforcement with concurrent workers", func() { + It("enforces delay between dispatches even with multiple workers", func() { + var mu sync.Mutex + var dispatchTimes []time.Time + + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + mu.Lock() + dispatchTimes = append(dispatchTimes, time.Now()) + mu.Unlock() + return "", nil + } + + err := service.CreateQueue(ctx, "delay-concurrent", host.QueueConfig{ + Concurrency: 3, + DelayMs: 200, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue 5 tasks + for i := 0; i < 5; i++ { + _, err := service.Enqueue(ctx, "delay-concurrent", []byte(fmt.Sprintf("task-%d", i))) + Expect(err).ToNot(HaveOccurred()) + } + + // Wait for all tasks to complete + Eventually(func() int { + mu.Lock() + defer mu.Unlock() + return len(dispatchTimes) + }).WithTimeout(10 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal(5)) + + // Sort dispatch times and verify gaps + mu.Lock() + sort.Slice(dispatchTimes, func(i, j int) bool { + return dispatchTimes[i].Before(dispatchTimes[j]) + }) + times := make([]time.Time, len(dispatchTimes)) + copy(times, dispatchTimes) + mu.Unlock() + + // Consecutive dispatches should have at least ~160ms gap (80% of 200ms) + for i := 1; i < len(times); i++ { + gap := times[i].Sub(times[i-1]) + Expect(gap).To(BeNumerically(">=", 160*time.Millisecond), + fmt.Sprintf("gap between dispatch %d and %d was %v, expected >= 160ms", i-1, i, gap)) + } + }) + }) + + Describe("Shutdown recovery", func() { + It("resets stale running tasks on CreateQueue", func() { + // Create a first service and queue, enqueue a task + service.invokeCallbackFn = func(ctx context.Context, _, _ string, _ []byte, _ int32) (string, error) { + <-ctx.Done() + return "", ctx.Err() + } + err := service.CreateQueue(ctx, "recovery-queue", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + taskID, err := service.Enqueue(ctx, "recovery-queue", []byte("stale-task")) + Expect(err).ToNot(HaveOccurred()) + + // Wait for the task to start running + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("running")) + + // Close the service (simulates crash - tasks left in running state) + service.Close() + + // Create a new service pointing to the same DB + managerCtx2, cancel2 := context.WithCancel(ctx) + DeferCleanup(cancel2) + manager2 := &Manager{ + plugins: make(map[string]*plugin), + ctx: managerCtx2, + } + + service, err = newTaskQueueService("test_plugin", manager2, 5) + Expect(err).ToNot(HaveOccurred()) + + // Override callback to succeed + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { + return "", nil + } + + // Re-create the queue - the upsert handles the existing row from the old service + err = service.CreateQueue(ctx, "recovery-queue", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + // The stale running task should now be reset to pending and eventually completed + Eventually(func() string { + info, err := service.Get(ctx, taskID) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(10 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + }) + }) + + Describe("Close", func() { + It("prevents subsequent operations after close", func() { + err := service.CreateQueue(ctx, "close-test", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + service.Close() + + // After close, operations should fail + _, err = service.Enqueue(ctx, "close-test", []byte("data")) + Expect(err).To(HaveOccurred()) + }) + }) + + Describe("Plugin isolation", func() { + It("uses separate databases for different plugins", func() { + managerCtx2, cancel2 := context.WithCancel(ctx) + DeferCleanup(cancel2) + manager2 := &Manager{ + plugins: make(map[string]*plugin), + ctx: managerCtx2, + } + + service2, err := newTaskQueueService("other_plugin", manager2, 5) + Expect(err).ToNot(HaveOccurred()) + defer service2.Close() + + // Check that separate database files exist + _, err = os.Stat(filepath.Join(tmpDir, "plugins", "test_plugin", "taskqueue.db")) + Expect(err).ToNot(HaveOccurred()) + _, err = os.Stat(filepath.Join(tmpDir, "plugins", "other_plugin", "taskqueue.db")) + Expect(err).ToNot(HaveOccurred()) + + // Both services should be able to create queues with the same name independently + service.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { return "", nil } + service2.invokeCallbackFn = func(_ context.Context, _, _ string, _ []byte, _ int32) (string, error) { return "", nil } + + err = service.CreateQueue(ctx, "shared-name", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + err = service2.CreateQueue(ctx, "shared-name", host.QueueConfig{}) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue to each and verify they work independently + taskID1, err := service.Enqueue(ctx, "shared-name", []byte("plugin1")) + Expect(err).ToNot(HaveOccurred()) + taskID2, err := service2.Enqueue(ctx, "shared-name", []byte("plugin2")) + Expect(err).ToNot(HaveOccurred()) + + Expect(taskID1).ToNot(Equal(taskID2)) + + // Both should complete + Eventually(func() string { + info, err := service.Get(ctx, taskID1) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + + Eventually(func() string { + info, err := service2.Get(ctx, taskID2) + if err != nil || info == nil { + return "" + } + return info.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + }) + }) +}) + +var _ = Describe("TaskQueueService Integration", Ordered, func() { + var manager *Manager + var tmpDir string + + BeforeAll(func() { + var err error + tmpDir, err = os.MkdirTemp("", "taskqueue-integration-test-*") + Expect(err).ToNot(HaveOccurred()) + + // Copy the test-taskqueue plugin + srcPath := filepath.Join(testdataDir, "test-taskqueue"+PackageExtension) + destPath := filepath.Join(tmpDir, "test-taskqueue"+PackageExtension) + data, err := os.ReadFile(srcPath) + Expect(err).ToNot(HaveOccurred()) + err = os.WriteFile(destPath, data, 0600) + Expect(err).ToNot(HaveOccurred()) + + // Compute SHA256 for the plugin + hash := sha256.Sum256(data) + hashHex := hex.EncodeToString(hash[:]) + + // Setup config + DeferCleanup(configtest.SetupConfig()) + conf.Server.Plugins.Enabled = true + conf.Server.Plugins.Folder = tmpDir + conf.Server.Plugins.AutoReload = false + conf.Server.CacheFolder = filepath.Join(tmpDir, "cache") + conf.Server.DataFolder = tmpDir + + // Setup mock DataStore with pre-enabled plugin + mockPluginRepo := tests.CreateMockPluginRepo() + mockPluginRepo.Permitted = true + mockPluginRepo.SetData(model.Plugins{{ + ID: "test-taskqueue", + Path: destPath, + SHA256: hashHex, + Enabled: true, + }}) + dataStore := &tests.MockDataStore{MockedPlugin: mockPluginRepo} + + // Create and start manager + manager = &Manager{ + plugins: make(map[string]*plugin), + ds: dataStore, + metrics: noopMetricsRecorder{}, + subsonicRouter: http.NotFoundHandler(), + } + err = manager.Start(GinkgoT().Context()) + Expect(err).ToNot(HaveOccurred()) + + DeferCleanup(func() { + _ = manager.Stop() + _ = os.RemoveAll(tmpDir) + }) + }) + + // Helper types for calling the test plugin + type testQueueConfig struct { + Concurrency int32 `json:"concurrency,omitempty"` + MaxRetries int32 `json:"maxRetries,omitempty"` + BackoffMs int64 `json:"backoffMs,omitempty"` + DelayMs int64 `json:"delayMs,omitempty"` + RetentionMs int64 `json:"retentionMs,omitempty"` + } + + type testTaskQueueInput struct { + Operation string `json:"operation"` + QueueName string `json:"queueName,omitempty"` + Config *testQueueConfig `json:"config,omitempty"` + Payload []byte `json:"payload,omitempty"` + TaskID string `json:"taskId,omitempty"` + } + + type testTaskQueueOutput struct { + TaskID string `json:"taskId,omitempty"` + Status string `json:"status,omitempty"` + Message string `json:"message,omitempty"` + Attempt int32 `json:"attempt,omitempty"` + Cleared int64 `json:"cleared,omitempty"` + Error *string `json:"error,omitempty"` + } + + callTestTaskQueue := func(ctx context.Context, input testTaskQueueInput) (*testTaskQueueOutput, error) { + manager.mu.RLock() + p := manager.plugins["test-taskqueue"] + manager.mu.RUnlock() + + instance, err := p.instance(ctx) + if err != nil { + return nil, err + } + defer instance.Close(ctx) + + inputBytes, _ := json.Marshal(input) + _, outputBytes, err := instance.Call("nd_test_taskqueue", inputBytes) + if err != nil { + return nil, err + } + + var output testTaskQueueOutput + if err := json.Unmarshal(outputBytes, &output); err != nil { + return nil, err + } + if output.Error != nil { + return nil, errors.New(*output.Error) + } + return &output, nil + } + + Describe("Plugin Loading", func() { + It("should load plugin with taskqueue permission and TaskWorker capability", func() { + manager.mu.RLock() + p, ok := manager.plugins["test-taskqueue"] + manager.mu.RUnlock() + Expect(ok).To(BeTrue()) + Expect(p.manifest.Permissions).ToNot(BeNil()) + Expect(p.manifest.Permissions.Taskqueue).ToNot(BeNil()) + Expect(p.manifest.Permissions.Taskqueue.MaxConcurrency).To(Equal(10)) + Expect(p.capabilities).To(ContainElement(CapabilityTaskWorker)) + }) + }) + + Describe("Create Queue", func() { + It("should create a queue without error", func() { + ctx := GinkgoT().Context() + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-create", + }) + Expect(err).ToNot(HaveOccurred()) + }) + + It("should return error for duplicate queue name", func() { + ctx := GinkgoT().Context() + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-dup", + }) + Expect(err).ToNot(HaveOccurred()) + + _, err = callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-dup", + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("already exists")) + }) + }) + + Describe("Enqueue and Task Completion", func() { + It("should enqueue a task and complete successfully", func() { + ctx := GinkgoT().Context() + + // Create queue + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-complete", + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue task with payload "hello" + output, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "enqueue", + QueueName: "test-complete", + Payload: []byte("hello"), + }) + Expect(err).ToNot(HaveOccurred()) + Expect(output.TaskID).ToNot(BeEmpty()) + + taskID := output.TaskID + + // Poll until completed + Eventually(func() string { + out, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "get_task_status", + TaskID: taskID, + }) + if err != nil { + return "error" + } + return out.Status + }).WithTimeout(5 * time.Second).WithPolling(100 * time.Millisecond).Should(Equal("completed")) + }) + }) + + Describe("Enqueue with Failure, No Retries", func() { + It("should fail when payload is 'fail' and maxRetries is 0", func() { + ctx := GinkgoT().Context() + + // Create queue with no retries + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-fail-no-retry", + Config: &testQueueConfig{ + MaxRetries: 0, + }, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue task that will fail + output, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "enqueue", + QueueName: "test-fail-no-retry", + Payload: []byte("fail"), + }) + Expect(err).ToNot(HaveOccurred()) + + taskID := output.TaskID + + // Poll until failed + Eventually(func() string { + out, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "get_task_status", + TaskID: taskID, + }) + if err != nil { + return "error" + } + return out.Status + }).WithTimeout(5 * time.Second).WithPolling(100 * time.Millisecond).Should(Equal("failed")) + }) + }) + + Describe("Enqueue with Retry Then Success", func() { + It("should retry and eventually succeed with 'fail-then-succeed' payload", func() { + ctx := GinkgoT().Context() + + // Create queue with retries and short backoff + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-retry-succeed", + Config: &testQueueConfig{ + MaxRetries: 2, + BackoffMs: 100, + }, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue task that fails on attempt < 2, then succeeds + output, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "enqueue", + QueueName: "test-retry-succeed", + Payload: []byte("fail-then-succeed"), + }) + Expect(err).ToNot(HaveOccurred()) + + taskID := output.TaskID + + // Poll until completed + Eventually(func() string { + out, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "get_task_status", + TaskID: taskID, + }) + if err != nil { + return "error" + } + return out.Status + }).WithTimeout(5 * time.Second).WithPolling(100 * time.Millisecond).Should(Equal("completed")) + }) + }) + + Describe("Cancel Pending Task", func() { + It("should cancel a pending task", func() { + ctx := GinkgoT().Context() + + // Create queue with concurrency=1 and a large delay between dispatches. + // The first task completes immediately (burst token), the second is dequeued + // but blocks on the rate limiter. Tasks 3+ remain in 'pending' and can be cancelled. + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-cancel", + Config: &testQueueConfig{ + Concurrency: 1, + DelayMs: 60000, + }, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue several tasks - the first will complete immediately, + // the second will be dequeued but block on the rate limiter (status=running), + // the rest will stay pending. + var taskIDs []string + for i := 0; i < 5; i++ { + output, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "enqueue", + QueueName: "test-cancel", + Payload: []byte("hello"), + }) + Expect(err).ToNot(HaveOccurred()) + taskIDs = append(taskIDs, output.TaskID) + } + + // Wait for the first task to complete (it has no delay) + Eventually(func() string { + out, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "get_task_status", + TaskID: taskIDs[0], + }) + if err != nil { + return "error" + } + return out.Status + }).WithTimeout(5 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal("completed")) + + // Give the worker a moment to dequeue the second task (which will + // block on the delay) so tasks 3+ stay in 'pending' + time.Sleep(100 * time.Millisecond) + + // Cancel the last task - it should still be pending + lastTaskID := taskIDs[len(taskIDs)-1] + _, err = callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "cancel_task", + TaskID: lastTaskID, + }) + Expect(err).ToNot(HaveOccurred()) + + // Verify status is cancelled + statusOut, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "get_task_status", + TaskID: lastTaskID, + }) + Expect(err).ToNot(HaveOccurred()) + Expect(statusOut.Status).To(Equal("cancelled")) + }) + }) + + Describe("Enqueue to Non-Existent Queue", func() { + It("should return error when enqueueing to a queue that does not exist", func() { + ctx := GinkgoT().Context() + + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "enqueue", + QueueName: "nonexistent-queue", + Payload: []byte("payload"), + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("does not exist")) + }) + }) + + Describe("Clear Queue", func() { + It("should clear pending tasks and return the count", func() { + ctx := GinkgoT().Context() + + // Create queue with large delay so tasks stay pending after the first completes + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "create_queue", + QueueName: "test-clear", + Config: &testQueueConfig{ + Concurrency: 1, + DelayMs: 60000, + }, + }) + Expect(err).ToNot(HaveOccurred()) + + // Enqueue several tasks + for i := 0; i < 4; i++ { + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "enqueue", + QueueName: "test-clear", + Payload: []byte(fmt.Sprintf("task-%d", i)), + }) + Expect(err).ToNot(HaveOccurred()) + } + + // Wait for the first task to complete (burst token) + time.Sleep(200 * time.Millisecond) + + // Clear the queue — should cancel remaining pending tasks + output, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "clear_queue", + QueueName: "test-clear", + }) + Expect(err).ToNot(HaveOccurred()) + Expect(output.Cleared).To(BeNumerically(">=", int64(1))) + }) + + It("should return error for non-existent queue", func() { + ctx := GinkgoT().Context() + + _, err := callTestTaskQueue(ctx, testTaskQueueInput{ + Operation: "clear_queue", + QueueName: "nonexistent-clear", + }) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("does not exist")) + }) + }) +}) diff --git a/plugins/manager_loader.go b/plugins/manager_loader.go index 610dbd028..59f48453f 100644 --- a/plugins/manager_loader.go +++ b/plugins/manager_loader.go @@ -128,6 +128,23 @@ var hostServices = []hostServiceEntry{ return host.RegisterHTTPHostFunctions(service), nil }, }, + { + name: "Task", + hasPermission: func(p *Permissions) bool { return p != nil && p.Taskqueue != nil }, + create: func(ctx *serviceContext) ([]extism.HostFunction, io.Closer) { + perm := ctx.permissions.Taskqueue + maxConcurrency := int32(1) + if perm.MaxConcurrency > 0 { + maxConcurrency = int32(perm.MaxConcurrency) + } + service, err := newTaskQueueService(ctx.pluginName, ctx.manager, maxConcurrency) + if err != nil { + log.Error("Failed to create Task service", "plugin", ctx.pluginName, err) + return nil, nil + } + return host.RegisterTaskHostFunctions(service), service + }, + }, } // extractManifest reads manifest from an .ndp package and computes its SHA-256 hash. diff --git a/plugins/manifest-schema.json b/plugins/manifest-schema.json index 4e64ca6ea..8daf88ccf 100644 --- a/plugins/manifest-schema.json +++ b/plugins/manifest-schema.json @@ -110,6 +110,9 @@ }, "users": { "$ref": "#/$defs/UsersPermission" + }, + "taskqueue": { + "$ref": "#/$defs/TaskQueuePermission" } } }, @@ -224,6 +227,23 @@ } } }, + "TaskQueuePermission": { + "type": "object", + "description": "Task queue permissions for background task processing", + "additionalProperties": false, + "properties": { + "reason": { + "type": "string", + "description": "Explanation for why task queue access is needed" + }, + "maxConcurrency": { + "type": "integer", + "description": "Maximum total concurrent workers across all queues. Default: 1", + "minimum": 1, + "default": 1 + } + } + }, "UsersPermission": { "type": "object", "description": "Users service permissions for accessing user information", diff --git a/plugins/manifest.go b/plugins/manifest.go index 3ca2657cd..375e73e7f 100644 --- a/plugins/manifest.go +++ b/plugins/manifest.go @@ -72,6 +72,13 @@ func ValidateWithCapabilities(m *Manifest, capabilities []Capability) error { } } + // Task (taskqueue) permission requires TaskWorker capability + if m.Permissions != nil && m.Permissions.Taskqueue != nil { + if !hasCapability(capabilities, CapabilityTaskWorker) { + return fmt.Errorf("'taskqueue' permission requires plugin to export '%s' function", FuncTaskWorkerCallback) + } + } + return nil } diff --git a/plugins/manifest_gen.go b/plugins/manifest_gen.go index 27c3c0677..a565ed3d1 100644 --- a/plugins/manifest_gen.go +++ b/plugins/manifest_gen.go @@ -181,6 +181,9 @@ type Permissions struct { // Subsonicapi corresponds to the JSON schema field "subsonicapi". Subsonicapi *SubsonicAPIPermission `json:"subsonicapi,omitempty" yaml:"subsonicapi,omitempty" mapstructure:"subsonicapi,omitempty"` + // Taskqueue corresponds to the JSON schema field "taskqueue". + Taskqueue *TaskQueuePermission `json:"taskqueue,omitempty" yaml:"taskqueue,omitempty" mapstructure:"taskqueue,omitempty"` + // Users corresponds to the JSON schema field "users". Users *UsersPermission `json:"users,omitempty" yaml:"users,omitempty" mapstructure:"users,omitempty"` @@ -200,6 +203,36 @@ type SubsonicAPIPermission struct { Reason *string `json:"reason,omitempty" yaml:"reason,omitempty" mapstructure:"reason,omitempty"` } +// Task queue permissions for background task processing +type TaskQueuePermission struct { + // Maximum total concurrent workers across all queues. Default: 1 + MaxConcurrency int `json:"maxConcurrency,omitempty" yaml:"maxConcurrency,omitempty" mapstructure:"maxConcurrency,omitempty"` + + // Explanation for why task queue access is needed + Reason *string `json:"reason,omitempty" yaml:"reason,omitempty" mapstructure:"reason,omitempty"` +} + +// UnmarshalJSON implements json.Unmarshaler. +func (j *TaskQueuePermission) UnmarshalJSON(value []byte) error { + var raw map[string]interface{} + if err := json.Unmarshal(value, &raw); err != nil { + return err + } + type Plain TaskQueuePermission + var plain Plain + if err := json.Unmarshal(value, &plain); err != nil { + return err + } + if v, ok := raw["maxConcurrency"]; !ok || v == nil { + plain.MaxConcurrency = 1.0 + } + if 1 > plain.MaxConcurrency { + return fmt.Errorf("field %s: must be >= %v", "maxConcurrency", 1) + } + *j = TaskQueuePermission(plain) + return nil +} + // Enable experimental WebAssembly threads support type ThreadsFeature struct { // Explanation for why threads support is needed diff --git a/plugins/pdk/go/host/doc.go b/plugins/pdk/go/host/doc.go index b801db44b..5781a04c1 100644 --- a/plugins/pdk/go/host/doc.go +++ b/plugins/pdk/go/host/doc.go @@ -43,6 +43,7 @@ The following host services are available: - Library: provides access to music library metadata for plugins. - Scheduler: provides task scheduling capabilities for plugins. - SubsonicAPI: provides access to Navidrome's Subsonic API from plugins. + - Task: provides persistent task queues for plugins. - Users: provides access to user information for plugins. - WebSocket: provides WebSocket communication capabilities for plugins. diff --git a/plugins/pdk/go/host/nd_host_task.go b/plugins/pdk/go/host/nd_host_task.go new file mode 100644 index 000000000..92a41c5bc --- /dev/null +++ b/plugins/pdk/go/host/nd_host_task.go @@ -0,0 +1,277 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains client wrappers for the Task host service. +// It is intended for use in Navidrome plugins built with TinyGo. +// +//go:build wasip1 + +package host + +import ( + "encoding/json" + "errors" + + "github.com/navidrome/navidrome/plugins/pdk/go/pdk" +) + +// QueueConfig represents the QueueConfig data structure. +// QueueConfig holds configuration for a task queue. +type QueueConfig struct { + Concurrency int32 `json:"concurrency"` + MaxRetries int32 `json:"maxRetries"` + BackoffMs int64 `json:"backoffMs"` + DelayMs int64 `json:"delayMs"` + RetentionMs int64 `json:"retentionMs"` +} + +// TaskInfo represents the TaskInfo data structure. +// TaskInfo holds the current state of a task. +type TaskInfo struct { + Status string `json:"status"` + Message string `json:"message"` + Attempt int32 `json:"attempt"` +} + +// task_createqueue is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user task_createqueue +func task_createqueue(uint64) uint64 + +// task_enqueue is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user task_enqueue +func task_enqueue(uint64) uint64 + +// task_get is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user task_get +func task_get(uint64) uint64 + +// task_cancel is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user task_cancel +func task_cancel(uint64) uint64 + +// task_clearqueue is the host function provided by Navidrome. +// +//go:wasmimport extism:host/user task_clearqueue +func task_clearqueue(uint64) uint64 + +type taskCreateQueueRequest struct { + Name string `json:"name"` + Config QueueConfig `json:"config"` +} + +type taskEnqueueRequest struct { + QueueName string `json:"queueName"` + Payload []byte `json:"payload"` +} + +type taskEnqueueResponse struct { + Result string `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +type taskGetRequest struct { + TaskID string `json:"taskId"` +} + +type taskGetResponse struct { + Result *TaskInfo `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +type taskCancelRequest struct { + TaskID string `json:"taskId"` +} + +type taskClearQueueRequest struct { + QueueName string `json:"queueName"` +} + +type taskClearQueueResponse struct { + Result int64 `json:"result,omitempty"` + Error string `json:"error,omitempty"` +} + +// TaskCreateQueue calls the task_createqueue host function. +// CreateQueue creates a named task queue with the given configuration. +// Zero-value fields in config use sensible defaults. +// If a queue with the same name already exists, returns an error. +// On startup, this also recovers any stale "running" tasks from a previous crash. +func TaskCreateQueue(name string, config QueueConfig) error { + // Marshal request to JSON + req := taskCreateQueueRequest{ + Name: name, + Config: config, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := task_createqueue(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse error-only response + var response struct { + Error string `json:"error,omitempty"` + } + if err := json.Unmarshal(responseBytes, &response); err != nil { + return err + } + if response.Error != "" { + return errors.New(response.Error) + } + return nil +} + +// TaskEnqueue calls the task_enqueue host function. +// Enqueue adds a task to the named queue. Returns the task ID. +// payload is opaque bytes passed back to the plugin on execution. +func TaskEnqueue(queueName string, payload []byte) (string, error) { + // Marshal request to JSON + req := taskEnqueueRequest{ + QueueName: queueName, + Payload: payload, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return "", err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := task_enqueue(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse the response + var response taskEnqueueResponse + if err := json.Unmarshal(responseBytes, &response); err != nil { + return "", err + } + + // Convert Error field to Go error + if response.Error != "" { + return "", errors.New(response.Error) + } + + return response.Result, nil +} + +// TaskGet calls the task_get host function. +// Get returns the current state of a task including its status, +// message, and attempt count. +func TaskGet(taskID string) (*TaskInfo, error) { + // Marshal request to JSON + req := taskGetRequest{ + TaskID: taskID, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return nil, err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := task_get(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse the response + var response taskGetResponse + if err := json.Unmarshal(responseBytes, &response); err != nil { + return nil, err + } + + // Convert Error field to Go error + if response.Error != "" { + return nil, errors.New(response.Error) + } + + return response.Result, nil +} + +// TaskCancel calls the task_cancel host function. +// Cancel cancels a pending task. Returns error if already +// running, completed, or failed. +func TaskCancel(taskID string) error { + // Marshal request to JSON + req := taskCancelRequest{ + TaskID: taskID, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := task_cancel(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse error-only response + var response struct { + Error string `json:"error,omitempty"` + } + if err := json.Unmarshal(responseBytes, &response); err != nil { + return err + } + if response.Error != "" { + return errors.New(response.Error) + } + return nil +} + +// TaskClearQueue calls the task_clearqueue host function. +// ClearQueue removes all pending tasks from the named queue. +// Running tasks are not affected. Returns the number of tasks removed. +func TaskClearQueue(queueName string) (int64, error) { + // Marshal request to JSON + req := taskClearQueueRequest{ + QueueName: queueName, + } + reqBytes, err := json.Marshal(req) + if err != nil { + return 0, err + } + reqMem := pdk.AllocateBytes(reqBytes) + defer reqMem.Free() + + // Call the host function + responsePtr := task_clearqueue(reqMem.Offset()) + + // Read the response from memory + responseMem := pdk.FindMemory(responsePtr) + responseBytes := responseMem.ReadBytes() + + // Parse the response + var response taskClearQueueResponse + if err := json.Unmarshal(responseBytes, &response); err != nil { + return 0, err + } + + // Convert Error field to Go error + if response.Error != "" { + return 0, errors.New(response.Error) + } + + return response.Result, nil +} diff --git a/plugins/pdk/go/host/nd_host_task_stub.go b/plugins/pdk/go/host/nd_host_task_stub.go new file mode 100644 index 000000000..4dde0e234 --- /dev/null +++ b/plugins/pdk/go/host/nd_host_task_stub.go @@ -0,0 +1,105 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains mock implementations for non-WASM builds. +// These mocks allow IDE support, compilation, and unit testing on non-WASM platforms. +// Plugin authors can use the exported mock instances to set expectations in tests. +// +//go:build !wasip1 + +package host + +import "github.com/stretchr/testify/mock" + +// QueueConfig represents the QueueConfig data structure. +// QueueConfig holds configuration for a task queue. +type QueueConfig struct { + Concurrency int32 `json:"concurrency"` + MaxRetries int32 `json:"maxRetries"` + BackoffMs int64 `json:"backoffMs"` + DelayMs int64 `json:"delayMs"` + RetentionMs int64 `json:"retentionMs"` +} + +// TaskInfo represents the TaskInfo data structure. +// TaskInfo holds the current state of a task. +type TaskInfo struct { + Status string `json:"status"` + Message string `json:"message"` + Attempt int32 `json:"attempt"` +} + +// mockTaskService is the mock implementation for testing. +type mockTaskService struct { + mock.Mock +} + +// TaskMock is the auto-instantiated mock instance for testing. +// Use this to set expectations: host.TaskMock.On("MethodName", args...).Return(values...) +var TaskMock = &mockTaskService{} + +// CreateQueue is the mock method for TaskCreateQueue. +func (m *mockTaskService) CreateQueue(name string, config QueueConfig) error { + args := m.Called(name, config) + return args.Error(0) +} + +// TaskCreateQueue delegates to the mock instance. +// CreateQueue creates a named task queue with the given configuration. +// Zero-value fields in config use sensible defaults. +// If a queue with the same name already exists, returns an error. +// On startup, this also recovers any stale "running" tasks from a previous crash. +func TaskCreateQueue(name string, config QueueConfig) error { + return TaskMock.CreateQueue(name, config) +} + +// Enqueue is the mock method for TaskEnqueue. +func (m *mockTaskService) Enqueue(queueName string, payload []byte) (string, error) { + args := m.Called(queueName, payload) + return args.String(0), args.Error(1) +} + +// TaskEnqueue delegates to the mock instance. +// Enqueue adds a task to the named queue. Returns the task ID. +// payload is opaque bytes passed back to the plugin on execution. +func TaskEnqueue(queueName string, payload []byte) (string, error) { + return TaskMock.Enqueue(queueName, payload) +} + +// Get is the mock method for TaskGet. +func (m *mockTaskService) Get(taskID string) (*TaskInfo, error) { + args := m.Called(taskID) + return args.Get(0).(*TaskInfo), args.Error(1) +} + +// TaskGet delegates to the mock instance. +// Get returns the current state of a task including its status, +// message, and attempt count. +func TaskGet(taskID string) (*TaskInfo, error) { + return TaskMock.Get(taskID) +} + +// Cancel is the mock method for TaskCancel. +func (m *mockTaskService) Cancel(taskID string) error { + args := m.Called(taskID) + return args.Error(0) +} + +// TaskCancel delegates to the mock instance. +// Cancel cancels a pending task. Returns error if already +// running, completed, or failed. +func TaskCancel(taskID string) error { + return TaskMock.Cancel(taskID) +} + +// ClearQueue is the mock method for TaskClearQueue. +func (m *mockTaskService) ClearQueue(queueName string) (int64, error) { + args := m.Called(queueName) + return args.Get(0).(int64), args.Error(1) +} + +// TaskClearQueue delegates to the mock instance. +// ClearQueue removes all pending tasks from the named queue. +// Running tasks are not affected. Returns the number of tasks removed. +func TaskClearQueue(queueName string) (int64, error) { + return TaskMock.ClearQueue(queueName) +} diff --git a/plugins/pdk/go/taskworker/taskworker.go b/plugins/pdk/go/taskworker/taskworker.go new file mode 100644 index 000000000..5d09a3209 --- /dev/null +++ b/plugins/pdk/go/taskworker/taskworker.go @@ -0,0 +1,79 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains export wrappers for the TaskWorker capability. +// It is intended for use in Navidrome plugins built with TinyGo. +// +//go:build wasip1 + +package taskworker + +import ( + "github.com/navidrome/navidrome/plugins/pdk/go/pdk" +) + +// TaskExecuteRequest is the request provided when a task is ready to execute. +type TaskExecuteRequest struct { + // QueueName is the name of the queue this task belongs to. + QueueName string `json:"queueName"` + // TaskID is the unique identifier for this task. + TaskID string `json:"taskId"` + // Payload is the opaque data provided when the task was enqueued. + Payload []byte `json:"payload"` + // Attempt is the current attempt number (1-based: first attempt = 1). + Attempt int32 `json:"attempt"` +} + +// TaskWorker is the marker interface for taskworker plugins. +// Implement one or more of the provider interfaces below. +// TaskWorker provides task execution handling. +// This capability allows plugins to receive callbacks when their queued tasks +// are ready to execute. Plugins that use the taskqueue host service must +// implement this capability. +type TaskWorker interface{} + +// TaskExecuteProvider provides the OnTaskExecute function. +type TaskExecuteProvider interface { + OnTaskExecute(TaskExecuteRequest) (string, error) +} // Internal implementation holders +var ( + taskExecuteImpl func(TaskExecuteRequest) (string, error) +) + +// Register registers a taskworker implementation. +// The implementation is checked for optional provider interfaces. +func Register(impl TaskWorker) { + if p, ok := impl.(TaskExecuteProvider); ok { + taskExecuteImpl = p.OnTaskExecute + } +} + +// NotImplementedCode is the standard return code for unimplemented functions. +// The host recognizes this and skips the plugin gracefully. +const NotImplementedCode int32 = -2 + +//go:wasmexport nd_task_execute +func _NdTaskExecute() int32 { + if taskExecuteImpl == nil { + // Return standard code - host will skip this plugin gracefully + return NotImplementedCode + } + + var input TaskExecuteRequest + if err := pdk.InputJSON(&input); err != nil { + pdk.SetError(err) + return -1 + } + + output, err := taskExecuteImpl(input) + if err != nil { + pdk.SetError(err) + return -1 + } + + if err := pdk.OutputJSON(output); err != nil { + pdk.SetError(err) + return -1 + } + + return 0 +} diff --git a/plugins/pdk/go/taskworker/taskworker_stub.go b/plugins/pdk/go/taskworker/taskworker_stub.go new file mode 100644 index 000000000..e45054e8e --- /dev/null +++ b/plugins/pdk/go/taskworker/taskworker_stub.go @@ -0,0 +1,41 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file provides stub implementations for non-WASM platforms. +// It allows Go plugins to compile and run tests outside of WASM, +// but the actual functionality is only available in WASM builds. +// +//go:build !wasip1 + +package taskworker + +// TaskExecuteRequest is the request provided when a task is ready to execute. +type TaskExecuteRequest struct { + // QueueName is the name of the queue this task belongs to. + QueueName string `json:"queueName"` + // TaskID is the unique identifier for this task. + TaskID string `json:"taskId"` + // Payload is the opaque data provided when the task was enqueued. + Payload []byte `json:"payload"` + // Attempt is the current attempt number (1-based: first attempt = 1). + Attempt int32 `json:"attempt"` +} + +// TaskWorker is the marker interface for taskworker plugins. +// Implement one or more of the provider interfaces below. +// TaskWorker provides task execution handling. +// This capability allows plugins to receive callbacks when their queued tasks +// are ready to execute. Plugins that use the taskqueue host service must +// implement this capability. +type TaskWorker interface{} + +// TaskExecuteProvider provides the OnTaskExecute function. +type TaskExecuteProvider interface { + OnTaskExecute(TaskExecuteRequest) (string, error) +} + +// NotImplementedCode is the standard return code for unimplemented functions. +const NotImplementedCode int32 = -2 + +// Register is a no-op on non-WASM platforms. +// This stub allows code to compile outside of WASM. +func Register(_ TaskWorker) {} diff --git a/plugins/pdk/python/host/nd_host_task.py b/plugins/pdk/python/host/nd_host_task.py new file mode 100644 index 000000000..5d6e7474c --- /dev/null +++ b/plugins/pdk/python/host/nd_host_task.py @@ -0,0 +1,188 @@ +# Code generated by ndpgen. DO NOT EDIT. +# +# This file contains client wrappers for the Task host service. +# It is intended for use in Navidrome plugins built with extism-py. +# +# IMPORTANT: Due to a limitation in extism-py, you cannot import this file directly. +# The @extism.import_fn decorators are only detected when defined in the plugin's +# main __init__.py file. Copy the needed functions from this file into your plugin. + +from dataclasses import dataclass +from typing import Any + +import extism +import json +import base64 + + +class HostFunctionError(Exception): + """Raised when a host function returns an error.""" + pass + + +@extism.import_fn("extism:host/user", "task_createqueue") +def _task_createqueue(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +@extism.import_fn("extism:host/user", "task_enqueue") +def _task_enqueue(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +@extism.import_fn("extism:host/user", "task_get") +def _task_get(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +@extism.import_fn("extism:host/user", "task_cancel") +def _task_cancel(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +@extism.import_fn("extism:host/user", "task_clearqueue") +def _task_clearqueue(offset: int) -> int: + """Raw host function - do not call directly.""" + ... + + +def task_create_queue(name: str, config: Any) -> None: + """CreateQueue creates a named task queue with the given configuration. +Zero-value fields in config use sensible defaults. +If a queue with the same name already exists, returns an error. +On startup, this also recovers any stale "running" tasks from a previous crash. + + Args: + name: str parameter. + config: Any parameter. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "name": name, + "config": config, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _task_createqueue(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + + +def task_enqueue(queue_name: str, payload: bytes) -> str: + """Enqueue adds a task to the named queue. Returns the task ID. +payload is opaque bytes passed back to the plugin on execution. + + Args: + queue_name: str parameter. + payload: bytes parameter. + + Returns: + str: The result value. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "queueName": queue_name, + "payload": base64.b64encode(payload).decode("ascii"), + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _task_enqueue(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + return response.get("result", "") + + +def task_get(task_id: str) -> Any: + """Get returns the current state of a task including its status, +message, and attempt count. + + Args: + task_id: str parameter. + + Returns: + Any: The result value. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "taskId": task_id, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _task_get(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + return response.get("result", None) + + +def task_cancel(task_id: str) -> None: + """Cancel cancels a pending task. Returns error if already +running, completed, or failed. + + Args: + task_id: str parameter. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "taskId": task_id, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _task_cancel(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + + +def task_clear_queue(queue_name: str) -> int: + """ClearQueue removes all pending tasks from the named queue. +Running tasks are not affected. Returns the number of tasks removed. + + Args: + queue_name: str parameter. + + Returns: + int: The result value. + + Raises: + HostFunctionError: If the host function returns an error. + """ + request = { + "queueName": queue_name, + } + request_bytes = json.dumps(request).encode("utf-8") + request_mem = extism.memory.alloc(request_bytes) + response_offset = _task_clearqueue(request_mem.offset) + response_mem = extism.memory.find(response_offset) + response = json.loads(extism.memory.string(response_mem)) + + if response.get("error"): + raise HostFunctionError(response["error"]) + + return response.get("result", 0) diff --git a/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs b/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs index 0f0daf80f..06c2c5c0d 100644 --- a/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs +++ b/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs @@ -9,4 +9,5 @@ pub mod lifecycle; pub mod metadata; pub mod scheduler; pub mod scrobbler; +pub mod taskworker; pub mod websocket; diff --git a/plugins/pdk/rust/nd-pdk-capabilities/src/taskworker.rs b/plugins/pdk/rust/nd-pdk-capabilities/src/taskworker.rs new file mode 100644 index 000000000..e8aa106a2 --- /dev/null +++ b/plugins/pdk/rust/nd-pdk-capabilities/src/taskworker.rs @@ -0,0 +1,102 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains export wrappers for the TaskWorker capability. +// It is intended for use in Navidrome plugins built with extism-pdk. + +use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} + +// Helper functions for skip_serializing_if with numeric types +#[allow(dead_code)] +fn is_zero_i32(value: &i32) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_u32(value: &u32) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_i64(value: &i64) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_u64(value: &u64) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_f32(value: &f32) -> bool { *value == 0.0 } +#[allow(dead_code)] +fn is_zero_f64(value: &f64) -> bool { *value == 0.0 } +/// TaskExecuteRequest is the request provided when a task is ready to execute. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TaskExecuteRequest { + /// QueueName is the name of the queue this task belongs to. + #[serde(default)] + pub queue_name: String, + /// TaskID is the unique identifier for this task. + #[serde(default)] + pub task_id: String, + /// Payload is the opaque data provided when the task was enqueued. + #[serde(default)] + #[serde(with = "base64_bytes")] + pub payload: Vec, + /// Attempt is the current attempt number (1-based: first attempt = 1). + #[serde(default)] + pub attempt: i32, +} + +/// Error represents an error from a capability method. +#[derive(Debug)] +pub struct Error { + pub message: String, +} + +impl std::fmt::Display for Error { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{}", self.message) + } +} + +impl std::error::Error for Error {} + +impl Error { + pub fn new(message: impl Into) -> Self { + Self { message: message.into() } + } +} + +/// TaskExecuteProvider provides the OnTaskExecute function. +pub trait TaskExecuteProvider { + fn on_task_execute(&self, req: TaskExecuteRequest) -> Result; +} + +/// Register the on_task_execute export. +/// This macro generates the WASM export function for this method. +#[macro_export] +macro_rules! register_taskworker_task_execute { + ($plugin_type:ty) => { + #[extism_pdk::plugin_fn] + pub fn nd_task_execute( + req: extism_pdk::Json<$crate::taskworker::TaskExecuteRequest> + ) -> extism_pdk::FnResult> { + let plugin = <$plugin_type>::default(); + let result = $crate::taskworker::TaskExecuteProvider::on_task_execute(&plugin, req.into_inner())?; + Ok(extism_pdk::Json(result)) + } + }; +} diff --git a/plugins/pdk/rust/nd-pdk-host/src/lib.rs b/plugins/pdk/rust/nd-pdk-host/src/lib.rs index 52a3a86cd..3a31bc489 100644 --- a/plugins/pdk/rust/nd-pdk-host/src/lib.rs +++ b/plugins/pdk/rust/nd-pdk-host/src/lib.rs @@ -40,6 +40,7 @@ //! - [`library`] - provides access to music library metadata for plugins. //! - [`scheduler`] - provides task scheduling capabilities for plugins. //! - [`subsonicapi`] - provides access to Navidrome's Subsonic API from plugins. +//! - [`task`] - provides persistent task queues for plugins. //! - [`users`] - provides access to user information for plugins. //! - [`websocket`] - provides WebSocket communication capabilities for plugins. @@ -99,6 +100,13 @@ pub mod subsonicapi { pub use super::nd_host_subsonicapi::*; } +#[doc(hidden)] +mod nd_host_task; +/// provides persistent task queues for plugins. +pub mod task { + pub use super::nd_host_task::*; +} + #[doc(hidden)] mod nd_host_users; /// provides access to user information for plugins. diff --git a/plugins/pdk/rust/nd-pdk-host/src/nd_host_task.rs b/plugins/pdk/rust/nd-pdk-host/src/nd_host_task.rs new file mode 100644 index 000000000..4f43e165c --- /dev/null +++ b/plugins/pdk/rust/nd-pdk-host/src/nd_host_task.rs @@ -0,0 +1,258 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains client wrappers for the Task host service. +// It is intended for use in Navidrome plugins built with extism-pdk. + +use extism_pdk::*; +use serde::{Deserialize, Serialize}; +use base64::Engine as _; +use base64::engine::general_purpose::STANDARD as BASE64; + +mod base64_bytes { + use serde::{self, Deserialize, Deserializer, Serializer}; + use base64::Engine as _; + use base64::engine::general_purpose::STANDARD as BASE64; + + pub fn serialize(bytes: &Vec, serializer: S) -> Result + where + S: Serializer, + { + serializer.serialize_str(&BASE64.encode(bytes)) + } + + pub fn deserialize<'de, D>(deserializer: D) -> Result, D::Error> + where + D: Deserializer<'de>, + { + let s = String::deserialize(deserializer)?; + BASE64.decode(&s).map_err(serde::de::Error::custom) + } +} + +/// QueueConfig holds configuration for a task queue. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct QueueConfig { + pub concurrency: i32, + pub max_retries: i32, + pub backoff_ms: i64, + pub delay_ms: i64, + pub retention_ms: i64, +} + +/// TaskInfo holds the current state of a task. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TaskInfo { + pub status: String, + pub message: String, + pub attempt: i32, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct TaskCreateQueueRequest { + name: String, + config: QueueConfig, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct TaskCreateQueueResponse { + #[serde(default)] + error: Option, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct TaskEnqueueRequest { + queue_name: String, + #[serde(with = "base64_bytes")] + payload: Vec, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct TaskEnqueueResponse { + #[serde(default)] + result: String, + #[serde(default)] + error: Option, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct TaskGetRequest { + task_id: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct TaskGetResponse { + #[serde(default)] + result: Option, + #[serde(default)] + error: Option, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct TaskCancelRequest { + task_id: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct TaskCancelResponse { + #[serde(default)] + error: Option, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +struct TaskClearQueueRequest { + queue_name: String, +} + +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +struct TaskClearQueueResponse { + #[serde(default)] + result: i64, + #[serde(default)] + error: Option, +} + +#[host_fn] +extern "ExtismHost" { + fn task_createqueue(input: Json) -> Json; + fn task_enqueue(input: Json) -> Json; + fn task_get(input: Json) -> Json; + fn task_cancel(input: Json) -> Json; + fn task_clearqueue(input: Json) -> Json; +} + +/// CreateQueue creates a named task queue with the given configuration. +/// Zero-value fields in config use sensible defaults. +/// If a queue with the same name already exists, returns an error. +/// On startup, this also recovers any stale "running" tasks from a previous crash. +/// +/// # Arguments +/// * `name` - String parameter. +/// * `config` - QueueConfig parameter. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn create_queue(name: &str, config: QueueConfig) -> Result<(), Error> { + let response = unsafe { + task_createqueue(Json(TaskCreateQueueRequest { + name: name.to_owned(), + config: config, + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(()) +} + +/// Enqueue adds a task to the named queue. Returns the task ID. +/// payload is opaque bytes passed back to the plugin on execution. +/// +/// # Arguments +/// * `queue_name` - String parameter. +/// * `payload` - Vec parameter. +/// +/// # Returns +/// The result value. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn enqueue(queue_name: &str, payload: Vec) -> Result { + let response = unsafe { + task_enqueue(Json(TaskEnqueueRequest { + queue_name: queue_name.to_owned(), + payload: payload, + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(response.0.result) +} + +/// Get returns the current state of a task including its status, +/// message, and attempt count. +/// +/// # Arguments +/// * `task_id` - String parameter. +/// +/// # Returns +/// The result value. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn get(task_id: &str) -> Result, Error> { + let response = unsafe { + task_get(Json(TaskGetRequest { + task_id: task_id.to_owned(), + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(response.0.result) +} + +/// Cancel cancels a pending task. Returns error if already +/// running, completed, or failed. +/// +/// # Arguments +/// * `task_id` - String parameter. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn cancel(task_id: &str) -> Result<(), Error> { + let response = unsafe { + task_cancel(Json(TaskCancelRequest { + task_id: task_id.to_owned(), + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(()) +} + +/// ClearQueue removes all pending tasks from the named queue. +/// Running tasks are not affected. Returns the number of tasks removed. +/// +/// # Arguments +/// * `queue_name` - String parameter. +/// +/// # Returns +/// The result value. +/// +/// # Errors +/// Returns an error if the host function call fails. +pub fn clear_queue(queue_name: &str) -> Result { + let response = unsafe { + task_clearqueue(Json(TaskClearQueueRequest { + queue_name: queue_name.to_owned(), + }))? + }; + + if let Some(err) = response.0.error { + return Err(Error::msg(err)); + } + + Ok(response.0.result) +} diff --git a/plugins/testdata/test-taskqueue/go.mod b/plugins/testdata/test-taskqueue/go.mod new file mode 100644 index 000000000..37f857e5a --- /dev/null +++ b/plugins/testdata/test-taskqueue/go.mod @@ -0,0 +1,16 @@ +module test-taskqueue + +go 1.25 + +require github.com/navidrome/navidrome/plugins/pdk/go v0.0.0 + +require ( + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/extism/go-pdk v1.1.3 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/stretchr/objx v0.5.2 // indirect + github.com/stretchr/testify v1.11.1 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) + +replace github.com/navidrome/navidrome/plugins/pdk/go => ../../pdk/go diff --git a/plugins/testdata/test-taskqueue/go.sum b/plugins/testdata/test-taskqueue/go.sum new file mode 100644 index 000000000..af880eb51 --- /dev/null +++ b/plugins/testdata/test-taskqueue/go.sum @@ -0,0 +1,14 @@ +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/extism/go-pdk v1.1.3 h1:hfViMPWrqjN6u67cIYRALZTZLk/enSPpNKa+rZ9X2SQ= +github.com/extism/go-pdk v1.1.3/go.mod h1:Gz+LIU/YCKnKXhgge8yo5Yu1F/lbv7KtKFkiCSzW/P4= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/plugins/testdata/test-taskqueue/main.go b/plugins/testdata/test-taskqueue/main.go new file mode 100644 index 000000000..9b734ed67 --- /dev/null +++ b/plugins/testdata/test-taskqueue/main.go @@ -0,0 +1,114 @@ +// Test TaskQueue plugin for Navidrome plugin system integration tests. +// Build with: tinygo build -o ../test-taskqueue.wasm -target wasip1 -buildmode=c-shared . +package main + +import ( + "fmt" + + "github.com/navidrome/navidrome/plugins/pdk/go/host" + "github.com/navidrome/navidrome/plugins/pdk/go/pdk" + "github.com/navidrome/navidrome/plugins/pdk/go/taskworker" +) + +func init() { + taskworker.Register(&handler{}) +} + +type handler struct{} + +func (h *handler) OnTaskExecute(req taskworker.TaskExecuteRequest) (string, error) { + payload := string(req.Payload) + if payload == "fail" { + return "", fmt.Errorf("task failed as instructed") + } + if payload == "fail-then-succeed" && req.Attempt < 2 { + return "", fmt.Errorf("transient failure") + } + return "completed successfully", nil +} + +// Test helper types +type TestInput struct { + Operation string `json:"operation"` + QueueName string `json:"queueName,omitempty"` + Config *host.QueueConfig `json:"config,omitempty"` + Payload []byte `json:"payload,omitempty"` + TaskID string `json:"taskId,omitempty"` +} + +type TestOutput struct { + TaskID string `json:"taskId,omitempty"` + Status string `json:"status,omitempty"` + Message string `json:"message,omitempty"` + Attempt int32 `json:"attempt,omitempty"` + Cleared int64 `json:"cleared,omitempty"` + Error *string `json:"error,omitempty"` +} + +//go:wasmexport nd_test_taskqueue +func ndTestTaskQueue() int32 { + var input TestInput + if err := pdk.InputJSON(&input); err != nil { + errStr := err.Error() + pdk.OutputJSON(TestOutput{Error: &errStr}) + return 0 + } + + switch input.Operation { + case "create_queue": + config := host.QueueConfig{} + if input.Config != nil { + config = *input.Config + } + err := host.TaskCreateQueue(input.QueueName, config) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestOutput{}) + + case "enqueue": + taskID, err := host.TaskEnqueue(input.QueueName, input.Payload) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestOutput{TaskID: taskID}) + + case "get_task_status": + info, err := host.TaskGet(input.TaskID) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestOutput{Status: info.Status, Message: info.Message, Attempt: info.Attempt}) + + case "cancel_task": + err := host.TaskCancel(input.TaskID) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestOutput{}) + + case "clear_queue": + cleared, err := host.TaskClearQueue(input.QueueName) + if err != nil { + errStr := err.Error() + pdk.OutputJSON(TestOutput{Error: &errStr}) + return 0 + } + pdk.OutputJSON(TestOutput{Cleared: cleared}) + + default: + errStr := "unknown operation: " + input.Operation + pdk.OutputJSON(TestOutput{Error: &errStr}) + } + return 0 +} + +func main() {} diff --git a/plugins/testdata/test-taskqueue/manifest.json b/plugins/testdata/test-taskqueue/manifest.json new file mode 100644 index 000000000..3cd3b0f0b --- /dev/null +++ b/plugins/testdata/test-taskqueue/manifest.json @@ -0,0 +1,12 @@ +{ + "name": "Test TaskQueue Plugin", + "author": "Navidrome Test", + "version": "1.0.0", + "description": "A test plugin for TaskQueue integration testing", + "permissions": { + "taskqueue": { + "reason": "For testing task queue operations", + "maxConcurrency": 10 + } + } +} From eeb1bd5f41e17bfa54724704a19049fcaf87ed9a Mon Sep 17 00:00:00 2001 From: Deluan Date: Tue, 3 Mar 2026 13:54:43 -0500 Subject: [PATCH 39/60] fix(plugins): update payload type to string with byte format for task data Signed-off-by: Deluan --- plugins/capabilities/taskworker.yaml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/plugins/capabilities/taskworker.yaml b/plugins/capabilities/taskworker.yaml index f10fd0794..7aa7126e0 100644 --- a/plugins/capabilities/taskworker.yaml +++ b/plugins/capabilities/taskworker.yaml @@ -23,10 +23,9 @@ components: type: string description: TaskID is the unique identifier for this task. payload: - type: array + type: string + format: byte description: Payload is the opaque data provided when the task was enqueued. - items: - type: object attempt: type: integer format: int32 From f03ca44a8ec4abc06086e84c95ecf7212854d0a7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Tue, 3 Mar 2026 15:48:39 -0500 Subject: [PATCH 40/60] feat(plugins): add lyrics provider plugin capability (#5126) * feat(plugins): add lyrics provider plugin capability Refactor the lyrics system from a static function to an interface-based service that supports WASM plugin providers. Plugins listed in the LyricsPriority config (alongside "embedded" and file extensions) are now resolved through the plugin system. Includes capability definition, Go/Rust PDK, adapter, Wire integration, and tests for plugin fallback behavior. * test(plugins): add lyrics capability integration test with test plugin * fix(plugins): default lyrics language to 'xxx' when plugin omits it Per the OpenSubsonic spec, the server must return 'und' or 'xxx' when the lyrics language is unknown. The lyrics plugin adapter was passing an empty string through when a plugin didn't provide a language value. This defaults the language to 'xxx', consistent with all other callers of model.ToLyrics() in the codebase. * refactor(plugins): rename lyrics import to improve clarity Signed-off-by: Deluan * refactor(lyrics): update TrackInfo description for clarity Signed-off-by: Deluan * fix(lyrics): enhance lyrics plugin handling and case sensitivity Signed-off-by: Deluan * fix(plugins): update payload type to string with byte format for task data Signed-off-by: Deluan --------- Signed-off-by: Deluan --- cmd/wire_gen.go | 8 +- cmd/wire_injectors.go | 2 + core/lyrics/lyrics.go | 34 +++- core/lyrics/lyrics_test.go | 108 ++++++++++++- core/lyrics/sources.go | 24 +++ core/wire_providers.go | 2 + plugins/capabilities/lyrics.go | 26 +++ plugins/capabilities/lyrics.yaml | 115 ++++++++++++++ plugins/capabilities/scrobbler.go | 2 +- plugins/capabilities/scrobbler.yaml | 2 +- plugins/lyrics_adapter.go | 59 +++++++ plugins/lyrics_adapter_test.go | 99 ++++++++++++ plugins/manager.go | 17 ++ plugins/pdk/go/lyrics/lyrics.go | 118 ++++++++++++++ plugins/pdk/go/lyrics/lyrics_stub.go | 82 ++++++++++ plugins/pdk/go/scrobbler/scrobbler.go | 2 +- plugins/pdk/go/scrobbler/scrobbler_stub.go | 2 +- .../pdk/rust/nd-pdk-capabilities/src/lib.rs | 1 + .../rust/nd-pdk-capabilities/src/lyrics.rs | 148 ++++++++++++++++++ .../rust/nd-pdk-capabilities/src/scrobbler.rs | 2 +- plugins/testdata/test-lyrics/go.mod | 16 ++ plugins/testdata/test-lyrics/go.sum | 14 ++ plugins/testdata/test-lyrics/main.go | 42 +++++ plugins/testdata/test-lyrics/manifest.json | 6 + server/e2e/e2e_suite_test.go | 2 + server/subsonic/album_lists_test.go | 2 +- server/subsonic/api.go | 5 +- server/subsonic/media_annotation_test.go | 2 +- server/subsonic/media_retrieval.go | 5 +- server/subsonic/media_retrieval_test.go | 3 +- server/subsonic/opensubsonic_test.go | 2 +- server/subsonic/playlists_test.go | 4 +- server/subsonic/searching_test.go | 2 +- 33 files changed, 930 insertions(+), 28 deletions(-) create mode 100644 plugins/capabilities/lyrics.go create mode 100644 plugins/capabilities/lyrics.yaml create mode 100644 plugins/lyrics_adapter.go create mode 100644 plugins/lyrics_adapter_test.go create mode 100644 plugins/pdk/go/lyrics/lyrics.go create mode 100644 plugins/pdk/go/lyrics/lyrics_stub.go create mode 100644 plugins/pdk/rust/nd-pdk-capabilities/src/lyrics.rs create mode 100644 plugins/testdata/test-lyrics/go.mod create mode 100644 plugins/testdata/test-lyrics/go.sum create mode 100644 plugins/testdata/test-lyrics/main.go create mode 100644 plugins/testdata/test-lyrics/manifest.json diff --git a/cmd/wire_gen.go b/cmd/wire_gen.go index 204d90ba8..e8df9a386 100644 --- a/cmd/wire_gen.go +++ b/cmd/wire_gen.go @@ -1,6 +1,6 @@ // Code generated by Wire. DO NOT EDIT. -//go:generate go run -mod=mod github.com/google/wire/cmd/wire gen -tags "netgo" +//go:generate go run -mod=mod github.com/google/wire/cmd/wire gen -tags "netgo sqlite_fts5" //go:build !wireinject // +build !wireinject @@ -16,6 +16,7 @@ import ( "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/external" "github.com/navidrome/navidrome/core/ffmpeg" + "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playlists" @@ -103,7 +104,8 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router { modelScanner := scanner.New(ctx, dataStore, cacheWarmer, broker, playlistsPlaylists, metricsMetrics) playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager) playbackServer := playback.GetInstance(dataStore) - router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics) + lyricsLyrics := lyrics.NewLyrics(manager) + router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics) return router } @@ -207,7 +209,7 @@ func getPluginManager() *plugins.Manager { // wire_injectors.go: -var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.New, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher))) +var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.New, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher))) func GetPluginManager(ctx context.Context) *plugins.Manager { manager := getPluginManager() diff --git a/cmd/wire_injectors.go b/cmd/wire_injectors.go index 56206feb6..d87a8d6d3 100644 --- a/cmd/wire_injectors.go +++ b/cmd/wire_injectors.go @@ -11,6 +11,7 @@ import ( "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/agents" "github.com/navidrome/navidrome/core/artwork" + "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/scrobbler" @@ -44,6 +45,7 @@ var allProviders = wire.NewSet( plugins.GetManager, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), + wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), diff --git a/core/lyrics/lyrics.go b/core/lyrics/lyrics.go index 858a3ffd8..758053042 100644 --- a/core/lyrics/lyrics.go +++ b/core/lyrics/lyrics.go @@ -9,23 +9,45 @@ import ( "github.com/navidrome/navidrome/model" ) -func GetLyrics(ctx context.Context, mf *model.MediaFile) (model.LyricList, error) { +// Lyrics can fetch lyrics for a media file. +type Lyrics interface { + GetLyrics(ctx context.Context, mf *model.MediaFile) (model.LyricList, error) +} + +// PluginLoader discovers and loads lyrics provider plugins. +type PluginLoader interface { + LoadLyricsProvider(name string) (Lyrics, bool) +} + +type lyricsService struct { + pluginLoader PluginLoader +} + +// NewLyrics creates a new lyrics service. pluginLoader may be nil if no plugin +// system is available. +func NewLyrics(pluginLoader PluginLoader) Lyrics { + return &lyricsService{pluginLoader: pluginLoader} +} + +// GetLyrics returns lyrics for the given media file, trying sources in the +// order specified by conf.Server.LyricsPriority. +func (l *lyricsService) GetLyrics(ctx context.Context, mf *model.MediaFile) (model.LyricList, error) { var lyricsList model.LyricList var err error - for pattern := range strings.SplitSeq(strings.ToLower(conf.Server.LyricsPriority), ",") { + for pattern := range strings.SplitSeq(conf.Server.LyricsPriority, ",") { pattern = strings.TrimSpace(pattern) switch { - case pattern == "embedded": + case strings.EqualFold(pattern, "embedded"): lyricsList, err = fromEmbedded(ctx, mf) case strings.HasPrefix(pattern, "."): - lyricsList, err = fromExternalFile(ctx, mf, pattern) + lyricsList, err = fromExternalFile(ctx, mf, strings.ToLower(pattern)) default: - log.Error(ctx, "Invalid lyric pattern", "pattern", pattern) + lyricsList, err = l.fromPlugin(ctx, mf, pattern) } if err != nil { - log.Error(ctx, "error parsing lyrics", "source", pattern, err) + log.Error(ctx, "error getting lyrics", "source", pattern, err) } if len(lyricsList) > 0 { diff --git a/core/lyrics/lyrics_test.go b/core/lyrics/lyrics_test.go index f4197ccf6..2e495a714 100644 --- a/core/lyrics/lyrics_test.go +++ b/core/lyrics/lyrics_test.go @@ -3,6 +3,7 @@ package lyrics_test import ( "context" "encoding/json" + "fmt" "os" "github.com/navidrome/navidrome/conf" @@ -72,7 +73,8 @@ var _ = Describe("sources", func() { DescribeTable("Lyrics Priority", func(priority string, expected model.LyricList) { conf.Server.LyricsPriority = priority - list, err := lyrics.GetLyrics(ctx, &mf) + svc := lyrics.NewLyrics(nil) + list, err := svc.GetLyrics(ctx, &mf) Expect(err).To(BeNil()) Expect(list).To(Equal(expected)) }, @@ -107,7 +109,8 @@ var _ = Describe("sources", func() { It("should fallback to embedded if an error happens when parsing file", func() { conf.Server.LyricsPriority = ".mp3,embedded" - list, err := lyrics.GetLyrics(ctx, &mf) + svc := lyrics.NewLyrics(nil) + list, err := svc.GetLyrics(ctx, &mf) Expect(err).To(BeNil()) Expect(list).To(Equal(embeddedLyrics)) }) @@ -115,10 +118,109 @@ var _ = Describe("sources", func() { It("should return nothing if error happens when trying to parse file", func() { conf.Server.LyricsPriority = ".mp3" - list, err := lyrics.GetLyrics(ctx, &mf) + svc := lyrics.NewLyrics(nil) + list, err := svc.GetLyrics(ctx, &mf) Expect(err).To(BeNil()) Expect(list).To(BeEmpty()) }) }) }) + + Context("plugin sources", func() { + var mockLoader *mockPluginLoader + + BeforeEach(func() { + mockLoader = &mockPluginLoader{} + }) + + It("should return lyrics from a plugin", func() { + conf.Server.LyricsPriority = "test-lyrics-plugin" + mockLoader.lyrics = unsyncedLyrics + svc := lyrics.NewLyrics(mockLoader) + list, err := svc.GetLyrics(ctx, &mf) + Expect(err).To(BeNil()) + Expect(list).To(Equal(unsyncedLyrics)) + }) + + It("should try plugin after embedded returns nothing", func() { + conf.Server.LyricsPriority = "embedded,test-lyrics-plugin" + mf.Lyrics = "" // No embedded lyrics + mockLoader.lyrics = unsyncedLyrics + svc := lyrics.NewLyrics(mockLoader) + list, err := svc.GetLyrics(ctx, &mf) + Expect(err).To(BeNil()) + Expect(list).To(Equal(unsyncedLyrics)) + }) + + It("should skip plugin if embedded has lyrics", func() { + conf.Server.LyricsPriority = "embedded,test-lyrics-plugin" + mockLoader.lyrics = unsyncedLyrics + svc := lyrics.NewLyrics(mockLoader) + list, err := svc.GetLyrics(ctx, &mf) + Expect(err).To(BeNil()) + Expect(list).To(Equal(embeddedLyrics)) // embedded wins + }) + + It("should skip unknown plugin names gracefully", func() { + conf.Server.LyricsPriority = "nonexistent-plugin,embedded" + mockLoader.notFound = true + svc := lyrics.NewLyrics(mockLoader) + list, err := svc.GetLyrics(ctx, &mf) + Expect(err).To(BeNil()) + Expect(list).To(Equal(embeddedLyrics)) // falls through to embedded + }) + + It("should preserve plugin name case from config", func() { + conf.Server.LyricsPriority = "MyLyricsPlugin" + mockLoader.pluginName = "MyLyricsPlugin" + mockLoader.lyrics = unsyncedLyrics + svc := lyrics.NewLyrics(mockLoader) + list, err := svc.GetLyrics(ctx, &mf) + Expect(err).To(BeNil()) + Expect(list).To(Equal(unsyncedLyrics)) + }) + + It("should handle plugin error gracefully", func() { + conf.Server.LyricsPriority = "test-lyrics-plugin,embedded" + mockLoader.err = fmt.Errorf("plugin error") + svc := lyrics.NewLyrics(mockLoader) + list, err := svc.GetLyrics(ctx, &mf) + Expect(err).To(BeNil()) + Expect(list).To(Equal(embeddedLyrics)) // falls through to embedded + }) + }) }) + +type mockPluginLoader struct { + lyrics model.LyricList + err error + notFound bool + pluginName string // expected plugin name (exact match, like real manager) +} + +func (m *mockPluginLoader) PluginNames(_ string) []string { + if m.notFound { + return nil + } + return []string{"test-lyrics-plugin"} +} + +func (m *mockPluginLoader) LoadLyricsProvider(name string) (lyrics.Lyrics, bool) { + if m.notFound { + return nil, false + } + // If pluginName is set, require exact match (like the real plugin manager) + if m.pluginName != "" && name != m.pluginName { + return nil, false + } + return &mockLyricsProvider{lyrics: m.lyrics, err: m.err}, true +} + +type mockLyricsProvider struct { + lyrics model.LyricList + err error +} + +func (m *mockLyricsProvider) GetLyrics(_ context.Context, _ *model.MediaFile) (model.LyricList, error) { + return m.lyrics, m.err +} diff --git a/core/lyrics/sources.go b/core/lyrics/sources.go index 857dc2eef..82a10ca41 100644 --- a/core/lyrics/sources.go +++ b/core/lyrics/sources.go @@ -49,3 +49,27 @@ func fromExternalFile(ctx context.Context, mf *model.MediaFile, suffix string) ( return model.LyricList{*lyrics}, nil } + +// fromPlugin attempts to load lyrics from a plugin with the given name. +func (l *lyricsService) fromPlugin(ctx context.Context, mf *model.MediaFile, pluginName string) (model.LyricList, error) { + if l.pluginLoader == nil { + log.Debug(ctx, "Invalid lyric source", "source", pluginName) + return nil, nil + } + + provider, ok := l.pluginLoader.LoadLyricsProvider(pluginName) + if !ok { + log.Warn(ctx, "Lyrics plugin not found", "plugin", pluginName) + return nil, nil + } + + lyricsList, err := provider.GetLyrics(ctx, mf) + if err != nil { + return nil, err + } + + if len(lyricsList) > 0 { + log.Trace(ctx, "Retrieved lyrics from plugin", "plugin", pluginName, "count", len(lyricsList)) + } + return lyricsList, nil +} diff --git a/core/wire_providers.go b/core/wire_providers.go index 503feb789..f9b472015 100644 --- a/core/wire_providers.go +++ b/core/wire_providers.go @@ -5,6 +5,7 @@ import ( "github.com/navidrome/navidrome/core/agents" "github.com/navidrome/navidrome/core/external" "github.com/navidrome/navidrome/core/ffmpeg" + "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playlists" @@ -28,4 +29,5 @@ var Set = wire.NewSet( scrobbler.GetPlayTracker, playback.GetInstance, metrics.GetInstance, + lyrics.NewLyrics, ) diff --git a/plugins/capabilities/lyrics.go b/plugins/capabilities/lyrics.go new file mode 100644 index 000000000..6f6d19177 --- /dev/null +++ b/plugins/capabilities/lyrics.go @@ -0,0 +1,26 @@ +package capabilities + +// Lyrics provides lyrics for a given track from external sources. +// +//nd:capability name=lyrics required=true +type Lyrics interface { + //nd:export name=nd_lyrics_get_lyrics + GetLyrics(GetLyricsRequest) (GetLyricsResponse, error) +} + +// GetLyricsRequest contains the track information for lyrics lookup. +type GetLyricsRequest struct { + Track TrackInfo `json:"track"` +} + +// GetLyricsResponse contains the lyrics returned by the plugin. +type GetLyricsResponse struct { + Lyrics []LyricsText `json:"lyrics"` +} + +// LyricsText represents a single set of lyrics in raw text format. +// Text can be plain text or LRC format — Navidrome will parse it. +type LyricsText struct { + Lang string `json:"lang,omitempty"` + Text string `json:"text"` +} diff --git a/plugins/capabilities/lyrics.yaml b/plugins/capabilities/lyrics.yaml new file mode 100644 index 000000000..e4f88476c --- /dev/null +++ b/plugins/capabilities/lyrics.yaml @@ -0,0 +1,115 @@ +version: v1-draft +exports: + nd_lyrics_get_lyrics: + input: + $ref: '#/components/schemas/GetLyricsRequest' + contentType: application/json + output: + $ref: '#/components/schemas/GetLyricsResponse' + contentType: application/json +components: + schemas: + ArtistRef: + description: ArtistRef is a reference to an artist with name and optional MBID. + properties: + id: + type: string + description: ID is the internal Navidrome artist ID (if known). + name: + type: string + description: Name is the artist name. + mbid: + type: string + description: MBID is the MusicBrainz ID for the artist. + required: + - name + GetLyricsRequest: + description: GetLyricsRequest contains the track information for lyrics lookup. + properties: + track: + $ref: '#/components/schemas/TrackInfo' + required: + - track + GetLyricsResponse: + description: GetLyricsResponse contains the lyrics returned by the plugin. + properties: + lyrics: + type: array + items: + $ref: '#/components/schemas/LyricsText' + required: + - lyrics + LyricsText: + description: |- + LyricsText represents a single set of lyrics in raw text format. + Text can be plain text or LRC format — Navidrome will parse it. + properties: + lang: + type: string + text: + type: string + required: + - text + TrackInfo: + description: TrackInfo contains track metadata. + properties: + id: + type: string + description: ID is the internal Navidrome track ID. + title: + type: string + description: Title is the track title. + album: + type: string + description: Album is the album name. + artist: + type: string + description: Artist is the formatted artist name for display (e.g., "Artist1 • Artist2"). + albumArtist: + type: string + description: AlbumArtist is the formatted album artist name for display. + artists: + type: array + description: Artists is the list of track artists. + items: + $ref: '#/components/schemas/ArtistRef' + albumArtists: + type: array + description: AlbumArtists is the list of album artists. + items: + $ref: '#/components/schemas/ArtistRef' + duration: + type: number + format: float + description: Duration is the track duration in seconds. + trackNumber: + type: integer + format: int32 + description: TrackNumber is the track number on the album. + discNumber: + type: integer + format: int32 + description: DiscNumber is the disc number. + mbzRecordingId: + type: string + description: MBZRecordingID is the MusicBrainz recording ID. + mbzAlbumId: + type: string + description: MBZAlbumID is the MusicBrainz album/release ID. + mbzReleaseGroupId: + type: string + description: MBZReleaseGroupID is the MusicBrainz release group ID. + mbzReleaseTrackId: + type: string + description: MBZReleaseTrackID is the MusicBrainz release track ID. + required: + - id + - title + - album + - artist + - albumArtist + - artists + - albumArtists + - duration + - trackNumber + - discNumber diff --git a/plugins/capabilities/scrobbler.go b/plugins/capabilities/scrobbler.go index 300652cb3..8091efe50 100644 --- a/plugins/capabilities/scrobbler.go +++ b/plugins/capabilities/scrobbler.go @@ -38,7 +38,7 @@ type ArtistRef struct { MBID string `json:"mbid,omitempty"` } -// TrackInfo contains track metadata for scrobbling. +// TrackInfo contains track metadata. type TrackInfo struct { // ID is the internal Navidrome track ID. ID string `json:"id"` diff --git a/plugins/capabilities/scrobbler.yaml b/plugins/capabilities/scrobbler.yaml index d8f47c951..5de351a5f 100644 --- a/plugins/capabilities/scrobbler.yaml +++ b/plugins/capabilities/scrobbler.yaml @@ -77,7 +77,7 @@ components: - track - timestamp TrackInfo: - description: TrackInfo contains track metadata for scrobbling. + description: TrackInfo contains track metadata. properties: id: type: string diff --git a/plugins/lyrics_adapter.go b/plugins/lyrics_adapter.go new file mode 100644 index 000000000..aa9930664 --- /dev/null +++ b/plugins/lyrics_adapter.go @@ -0,0 +1,59 @@ +package plugins + +import ( + "context" + + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/plugins/capabilities" +) + +const CapabilityLyrics Capability = "Lyrics" + +const ( + FuncLyricsGetLyrics = "nd_lyrics_get_lyrics" +) + +func init() { + registerCapability( + CapabilityLyrics, + FuncLyricsGetLyrics, + ) +} + +// LyricsPlugin adapts a WASM plugin with the Lyrics capability. +type LyricsPlugin struct { + name string + plugin *plugin +} + +// GetLyrics calls the plugin to fetch lyrics, then parses the raw text responses +// using model.ToLyrics. +func (l *LyricsPlugin) GetLyrics(ctx context.Context, mf *model.MediaFile) (model.LyricList, error) { + req := capabilities.GetLyricsRequest{ + Track: mediaFileToTrackInfo(mf), + } + resp, err := callPluginFunction[capabilities.GetLyricsRequest, capabilities.GetLyricsResponse]( + ctx, l.plugin, FuncLyricsGetLyrics, req, + ) + if err != nil { + return nil, err + } + + var result model.LyricList + for _, lt := range resp.Lyrics { + lang := lt.Lang + if lang == "" { + lang = "xxx" + } + parsed, err := model.ToLyrics(lang, lt.Text) + if err != nil { + log.Warn(ctx, "Error parsing plugin lyrics", "plugin", l.name, err) + continue + } + if parsed != nil && !parsed.IsEmpty() { + result = append(result, *parsed) + } + } + return result, nil +} diff --git a/plugins/lyrics_adapter_test.go b/plugins/lyrics_adapter_test.go new file mode 100644 index 000000000..a1a6c1809 --- /dev/null +++ b/plugins/lyrics_adapter_test.go @@ -0,0 +1,99 @@ +//go:build !windows + +package plugins + +import ( + "github.com/navidrome/navidrome/model" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("LyricsPlugin", Ordered, func() { + var ( + lyricsManager *Manager + provider *LyricsPlugin + ) + + BeforeAll(func() { + lyricsManager, _ = createTestManagerWithPlugins(nil, + "test-lyrics"+PackageExtension, + "test-metadata-agent"+PackageExtension, + ) + + p, ok := lyricsManager.LoadLyricsProvider("test-lyrics") + Expect(ok).To(BeTrue()) + provider = p.(*LyricsPlugin) + }) + + Describe("LoadLyricsProvider", func() { + It("returns a lyrics provider for a plugin with Lyrics capability", func() { + Expect(provider).ToNot(BeNil()) + }) + + It("returns false for a plugin without Lyrics capability", func() { + _, ok := lyricsManager.LoadLyricsProvider("test-metadata-agent") + Expect(ok).To(BeFalse()) + }) + + It("returns false for non-existent plugin", func() { + _, ok := lyricsManager.LoadLyricsProvider("non-existent") + Expect(ok).To(BeFalse()) + }) + }) + + Describe("GetLyrics", func() { + It("successfully returns lyrics from the plugin", func() { + track := &model.MediaFile{ + ID: "track-1", + Title: "Test Song", + Artist: "Test Artist", + } + + result, err := provider.GetLyrics(GinkgoT().Context(), track) + Expect(err).ToNot(HaveOccurred()) + Expect(result).To(HaveLen(1)) + Expect(result[0].Line).ToNot(BeEmpty()) + Expect(result[0].Line[0].Value).To(ContainSubstring("Test Song")) + }) + + It("defaults language to 'xxx' when plugin does not provide one", func() { + manager, _ := createTestManagerWithPlugins(map[string]map[string]string{ + "test-lyrics": {"no_lang": "true"}, + }, "test-lyrics"+PackageExtension) + + p, ok := manager.LoadLyricsProvider("test-lyrics") + Expect(ok).To(BeTrue()) + + track := &model.MediaFile{ID: "track-1", Title: "Test Song", Artist: "Test Artist"} + result, err := p.GetLyrics(GinkgoT().Context(), track) + Expect(err).ToNot(HaveOccurred()) + Expect(result).To(HaveLen(1)) + Expect(result[0].Lang).To(Equal("xxx")) + }) + + It("returns error when plugin returns error", func() { + manager, _ := createTestManagerWithPlugins(map[string]map[string]string{ + "test-lyrics": {"error": "service unavailable"}, + }, "test-lyrics"+PackageExtension) + + p, ok := manager.LoadLyricsProvider("test-lyrics") + Expect(ok).To(BeTrue()) + + track := &model.MediaFile{ID: "track-1", Title: "Test Song"} + _, err := p.GetLyrics(GinkgoT().Context(), track) + Expect(err).To(HaveOccurred()) + }) + }) + + Describe("PluginNames", func() { + It("returns plugin names with Lyrics capability", func() { + names := lyricsManager.PluginNames("Lyrics") + Expect(names).To(ContainElement("test-lyrics")) + }) + + It("does not return metadata agent plugins for Lyrics capability", func() { + names := lyricsManager.PluginNames("Lyrics") + Expect(names).ToNot(ContainElement("test-metadata-agent")) + }) + }) +}) diff --git a/plugins/manager.go b/plugins/manager.go index bf6bab6e8..0c7c91ed8 100644 --- a/plugins/manager.go +++ b/plugins/manager.go @@ -16,6 +16,7 @@ import ( extism "github.com/extism/go-sdk" "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/core/agents" + "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/scrobbler" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -282,6 +283,22 @@ func (m *Manager) LoadScrobbler(name string) (scrobbler.Scrobbler, bool) { }, true } +// LoadLyricsProvider loads and returns a lyrics provider plugin by name. +func (m *Manager) LoadLyricsProvider(name string) (lyrics.Lyrics, bool) { + m.mu.RLock() + plugin, ok := m.plugins[name] + m.mu.RUnlock() + + if !ok || !hasCapability(plugin.capabilities, CapabilityLyrics) { + return nil, false + } + + return &LyricsPlugin{ + name: plugin.name, + plugin: plugin, + }, true +} + // PluginInfo contains basic information about a plugin for metrics/insights. type PluginInfo struct { Name string diff --git a/plugins/pdk/go/lyrics/lyrics.go b/plugins/pdk/go/lyrics/lyrics.go new file mode 100644 index 000000000..4f5aa6302 --- /dev/null +++ b/plugins/pdk/go/lyrics/lyrics.go @@ -0,0 +1,118 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains export wrappers for the Lyrics capability. +// It is intended for use in Navidrome plugins built with TinyGo. +// +//go:build wasip1 + +package lyrics + +import ( + "github.com/navidrome/navidrome/plugins/pdk/go/pdk" +) + +// ArtistRef is a reference to an artist with name and optional MBID. +type ArtistRef struct { + // ID is the internal Navidrome artist ID (if known). + ID string `json:"id,omitempty"` + // Name is the artist name. + Name string `json:"name"` + // MBID is the MusicBrainz ID for the artist. + MBID string `json:"mbid,omitempty"` +} + +// GetLyricsRequest contains the track information for lyrics lookup. +type GetLyricsRequest struct { + Track TrackInfo `json:"track"` +} + +// GetLyricsResponse contains the lyrics returned by the plugin. +type GetLyricsResponse struct { + Lyrics []LyricsText `json:"lyrics"` +} + +// LyricsText represents a single set of lyrics in raw text format. +// Text can be plain text or LRC format — Navidrome will parse it. +type LyricsText struct { + Lang string `json:"lang,omitempty"` + Text string `json:"text"` +} + +// TrackInfo contains track metadata. +type TrackInfo struct { + // ID is the internal Navidrome track ID. + ID string `json:"id"` + // Title is the track title. + Title string `json:"title"` + // Album is the album name. + Album string `json:"album"` + // Artist is the formatted artist name for display (e.g., "Artist1 • Artist2"). + Artist string `json:"artist"` + // AlbumArtist is the formatted album artist name for display. + AlbumArtist string `json:"albumArtist"` + // Artists is the list of track artists. + Artists []ArtistRef `json:"artists"` + // AlbumArtists is the list of album artists. + AlbumArtists []ArtistRef `json:"albumArtists"` + // Duration is the track duration in seconds. + Duration float32 `json:"duration"` + // TrackNumber is the track number on the album. + TrackNumber int32 `json:"trackNumber"` + // DiscNumber is the disc number. + DiscNumber int32 `json:"discNumber"` + // MBZRecordingID is the MusicBrainz recording ID. + MBZRecordingID string `json:"mbzRecordingId,omitempty"` + // MBZAlbumID is the MusicBrainz album/release ID. + MBZAlbumID string `json:"mbzAlbumId,omitempty"` + // MBZReleaseGroupID is the MusicBrainz release group ID. + MBZReleaseGroupID string `json:"mbzReleaseGroupId,omitempty"` + // MBZReleaseTrackID is the MusicBrainz release track ID. + MBZReleaseTrackID string `json:"mbzReleaseTrackId,omitempty"` +} + +// Lyrics requires all methods to be implemented. +// Lyrics provides lyrics for a given track from external sources. +type Lyrics interface { + // GetLyrics + GetLyrics(GetLyricsRequest) (GetLyricsResponse, error) +} // Internal implementation holders +var ( + lyricsImpl func(GetLyricsRequest) (GetLyricsResponse, error) +) + +// Register registers a lyrics implementation. +// All methods are required. +func Register(impl Lyrics) { + lyricsImpl = impl.GetLyrics +} + +// NotImplementedCode is the standard return code for unimplemented functions. +// The host recognizes this and skips the plugin gracefully. +const NotImplementedCode int32 = -2 + +//go:wasmexport nd_lyrics_get_lyrics +func _NdLyricsGetLyrics() int32 { + if lyricsImpl == nil { + // Return standard code - host will skip this plugin gracefully + return NotImplementedCode + } + + var input GetLyricsRequest + if err := pdk.InputJSON(&input); err != nil { + pdk.SetError(err) + return -1 + } + + output, err := lyricsImpl(input) + if err != nil { + pdk.SetError(err) + return -1 + } + + if err := pdk.OutputJSON(output); err != nil { + pdk.SetError(err) + return -1 + } + + return 0 +} diff --git a/plugins/pdk/go/lyrics/lyrics_stub.go b/plugins/pdk/go/lyrics/lyrics_stub.go new file mode 100644 index 000000000..1fdf184e5 --- /dev/null +++ b/plugins/pdk/go/lyrics/lyrics_stub.go @@ -0,0 +1,82 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file provides stub implementations for non-WASM platforms. +// It allows Go plugins to compile and run tests outside of WASM, +// but the actual functionality is only available in WASM builds. +// +//go:build !wasip1 + +package lyrics + +// ArtistRef is a reference to an artist with name and optional MBID. +type ArtistRef struct { + // ID is the internal Navidrome artist ID (if known). + ID string `json:"id,omitempty"` + // Name is the artist name. + Name string `json:"name"` + // MBID is the MusicBrainz ID for the artist. + MBID string `json:"mbid,omitempty"` +} + +// GetLyricsRequest contains the track information for lyrics lookup. +type GetLyricsRequest struct { + Track TrackInfo `json:"track"` +} + +// GetLyricsResponse contains the lyrics returned by the plugin. +type GetLyricsResponse struct { + Lyrics []LyricsText `json:"lyrics"` +} + +// LyricsText represents a single set of lyrics in raw text format. +// Text can be plain text or LRC format — Navidrome will parse it. +type LyricsText struct { + Lang string `json:"lang,omitempty"` + Text string `json:"text"` +} + +// TrackInfo contains track metadata. +type TrackInfo struct { + // ID is the internal Navidrome track ID. + ID string `json:"id"` + // Title is the track title. + Title string `json:"title"` + // Album is the album name. + Album string `json:"album"` + // Artist is the formatted artist name for display (e.g., "Artist1 • Artist2"). + Artist string `json:"artist"` + // AlbumArtist is the formatted album artist name for display. + AlbumArtist string `json:"albumArtist"` + // Artists is the list of track artists. + Artists []ArtistRef `json:"artists"` + // AlbumArtists is the list of album artists. + AlbumArtists []ArtistRef `json:"albumArtists"` + // Duration is the track duration in seconds. + Duration float32 `json:"duration"` + // TrackNumber is the track number on the album. + TrackNumber int32 `json:"trackNumber"` + // DiscNumber is the disc number. + DiscNumber int32 `json:"discNumber"` + // MBZRecordingID is the MusicBrainz recording ID. + MBZRecordingID string `json:"mbzRecordingId,omitempty"` + // MBZAlbumID is the MusicBrainz album/release ID. + MBZAlbumID string `json:"mbzAlbumId,omitempty"` + // MBZReleaseGroupID is the MusicBrainz release group ID. + MBZReleaseGroupID string `json:"mbzReleaseGroupId,omitempty"` + // MBZReleaseTrackID is the MusicBrainz release track ID. + MBZReleaseTrackID string `json:"mbzReleaseTrackId,omitempty"` +} + +// Lyrics requires all methods to be implemented. +// Lyrics provides lyrics for a given track from external sources. +type Lyrics interface { + // GetLyrics + GetLyrics(GetLyricsRequest) (GetLyricsResponse, error) +} + +// NotImplementedCode is the standard return code for unimplemented functions. +const NotImplementedCode int32 = -2 + +// Register is a no-op on non-WASM platforms. +// This stub allows code to compile outside of WASM. +func Register(_ Lyrics) {} diff --git a/plugins/pdk/go/scrobbler/scrobbler.go b/plugins/pdk/go/scrobbler/scrobbler.go index 258b1b4c1..c694f59d8 100644 --- a/plugins/pdk/go/scrobbler/scrobbler.go +++ b/plugins/pdk/go/scrobbler/scrobbler.go @@ -62,7 +62,7 @@ type ScrobbleRequest struct { Timestamp int64 `json:"timestamp"` } -// TrackInfo contains track metadata for scrobbling. +// TrackInfo contains track metadata. type TrackInfo struct { // ID is the internal Navidrome track ID. ID string `json:"id"` diff --git a/plugins/pdk/go/scrobbler/scrobbler_stub.go b/plugins/pdk/go/scrobbler/scrobbler_stub.go index f2fc584ad..6d4afd818 100644 --- a/plugins/pdk/go/scrobbler/scrobbler_stub.go +++ b/plugins/pdk/go/scrobbler/scrobbler_stub.go @@ -59,7 +59,7 @@ type ScrobbleRequest struct { Timestamp int64 `json:"timestamp"` } -// TrackInfo contains track metadata for scrobbling. +// TrackInfo contains track metadata. type TrackInfo struct { // ID is the internal Navidrome track ID. ID string `json:"id"` diff --git a/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs b/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs index 06c2c5c0d..85375b525 100644 --- a/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs +++ b/plugins/pdk/rust/nd-pdk-capabilities/src/lib.rs @@ -6,6 +6,7 @@ //! for implementing Navidrome plugin capabilities in Rust. pub mod lifecycle; +pub mod lyrics; pub mod metadata; pub mod scheduler; pub mod scrobbler; diff --git a/plugins/pdk/rust/nd-pdk-capabilities/src/lyrics.rs b/plugins/pdk/rust/nd-pdk-capabilities/src/lyrics.rs new file mode 100644 index 000000000..16882abae --- /dev/null +++ b/plugins/pdk/rust/nd-pdk-capabilities/src/lyrics.rs @@ -0,0 +1,148 @@ +// Code generated by ndpgen. DO NOT EDIT. +// +// This file contains export wrappers for the Lyrics capability. +// It is intended for use in Navidrome plugins built with extism-pdk. + +use serde::{Deserialize, Serialize}; + +// Helper functions for skip_serializing_if with numeric types +#[allow(dead_code)] +fn is_zero_i32(value: &i32) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_u32(value: &u32) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_i64(value: &i64) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_u64(value: &u64) -> bool { *value == 0 } +#[allow(dead_code)] +fn is_zero_f32(value: &f32) -> bool { *value == 0.0 } +#[allow(dead_code)] +fn is_zero_f64(value: &f64) -> bool { *value == 0.0 } +/// ArtistRef is a reference to an artist with name and optional MBID. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ArtistRef { + /// ID is the internal Navidrome artist ID (if known). + #[serde(default, skip_serializing_if = "String::is_empty")] + pub id: String, + /// Name is the artist name. + #[serde(default)] + pub name: String, + /// MBID is the MusicBrainz ID for the artist. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub mbid: String, +} +/// GetLyricsRequest contains the track information for lyrics lookup. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct GetLyricsRequest { + #[serde(default)] + pub track: TrackInfo, +} +/// GetLyricsResponse contains the lyrics returned by the plugin. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct GetLyricsResponse { + #[serde(default)] + pub lyrics: Vec, +} +/// LyricsText represents a single set of lyrics in raw text format. +/// Text can be plain text or LRC format — Navidrome will parse it. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct LyricsText { + #[serde(default, skip_serializing_if = "String::is_empty")] + pub lang: String, + #[serde(default)] + pub text: String, +} +/// TrackInfo contains track metadata. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TrackInfo { + /// ID is the internal Navidrome track ID. + #[serde(default)] + pub id: String, + /// Title is the track title. + #[serde(default)] + pub title: String, + /// Album is the album name. + #[serde(default)] + pub album: String, + /// Artist is the formatted artist name for display (e.g., "Artist1 • Artist2"). + #[serde(default)] + pub artist: String, + /// AlbumArtist is the formatted album artist name for display. + #[serde(default)] + pub album_artist: String, + /// Artists is the list of track artists. + #[serde(default)] + pub artists: Vec, + /// AlbumArtists is the list of album artists. + #[serde(default)] + pub album_artists: Vec, + /// Duration is the track duration in seconds. + #[serde(default)] + pub duration: f32, + /// TrackNumber is the track number on the album. + #[serde(default)] + pub track_number: i32, + /// DiscNumber is the disc number. + #[serde(default)] + pub disc_number: i32, + /// MBZRecordingID is the MusicBrainz recording ID. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub mbz_recording_id: String, + /// MBZAlbumID is the MusicBrainz album/release ID. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub mbz_album_id: String, + /// MBZReleaseGroupID is the MusicBrainz release group ID. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub mbz_release_group_id: String, + /// MBZReleaseTrackID is the MusicBrainz release track ID. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub mbz_release_track_id: String, +} + +/// Error represents an error from a capability method. +#[derive(Debug)] +pub struct Error { + pub message: String, +} + +impl std::fmt::Display for Error { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{}", self.message) + } +} + +impl std::error::Error for Error {} + +impl Error { + pub fn new(message: impl Into) -> Self { + Self { message: message.into() } + } +} + +/// Lyrics requires all methods to be implemented. +/// Lyrics provides lyrics for a given track from external sources. +pub trait Lyrics { + /// GetLyrics + fn get_lyrics(&self, req: GetLyricsRequest) -> Result; +} + +/// Register all exports for the Lyrics capability. +/// This macro generates the WASM export functions for all trait methods. +#[macro_export] +macro_rules! register_lyrics { + ($plugin_type:ty) => { + #[extism_pdk::plugin_fn] + pub fn nd_lyrics_get_lyrics( + req: extism_pdk::Json<$crate::lyrics::GetLyricsRequest> + ) -> extism_pdk::FnResult> { + let plugin = <$plugin_type>::default(); + let result = $crate::lyrics::Lyrics::get_lyrics(&plugin, req.into_inner())?; + Ok(extism_pdk::Json(result)) + } + }; +} diff --git a/plugins/pdk/rust/nd-pdk-capabilities/src/scrobbler.rs b/plugins/pdk/rust/nd-pdk-capabilities/src/scrobbler.rs index 9dbedd040..2572712d1 100644 --- a/plugins/pdk/rust/nd-pdk-capabilities/src/scrobbler.rs +++ b/plugins/pdk/rust/nd-pdk-capabilities/src/scrobbler.rs @@ -76,7 +76,7 @@ pub struct ScrobbleRequest { #[serde(default)] pub timestamp: i64, } -/// TrackInfo contains track metadata for scrobbling. +/// TrackInfo contains track metadata. #[derive(Debug, Clone, Default, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub struct TrackInfo { diff --git a/plugins/testdata/test-lyrics/go.mod b/plugins/testdata/test-lyrics/go.mod new file mode 100644 index 000000000..fbbb23fc0 --- /dev/null +++ b/plugins/testdata/test-lyrics/go.mod @@ -0,0 +1,16 @@ +module test-lyrics + +go 1.25 + +require github.com/navidrome/navidrome/plugins/pdk/go v0.0.0 + +require ( + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/extism/go-pdk v1.1.3 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/stretchr/objx v0.5.2 // indirect + github.com/stretchr/testify v1.11.1 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) + +replace github.com/navidrome/navidrome/plugins/pdk/go => ../../pdk/go diff --git a/plugins/testdata/test-lyrics/go.sum b/plugins/testdata/test-lyrics/go.sum new file mode 100644 index 000000000..af880eb51 --- /dev/null +++ b/plugins/testdata/test-lyrics/go.sum @@ -0,0 +1,14 @@ +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/extism/go-pdk v1.1.3 h1:hfViMPWrqjN6u67cIYRALZTZLk/enSPpNKa+rZ9X2SQ= +github.com/extism/go-pdk v1.1.3/go.mod h1:Gz+LIU/YCKnKXhgge8yo5Yu1F/lbv7KtKFkiCSzW/P4= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/plugins/testdata/test-lyrics/main.go b/plugins/testdata/test-lyrics/main.go new file mode 100644 index 000000000..0e485ceba --- /dev/null +++ b/plugins/testdata/test-lyrics/main.go @@ -0,0 +1,42 @@ +// Test lyrics plugin for Navidrome plugin system integration tests. +package main + +import ( + "fmt" + + "github.com/navidrome/navidrome/plugins/pdk/go/lyrics" + "github.com/navidrome/navidrome/plugins/pdk/go/pdk" +) + +func init() { + lyrics.Register(&testLyrics{}) +} + +type testLyrics struct{} + +func (t *testLyrics) GetLyrics(input lyrics.GetLyricsRequest) (lyrics.GetLyricsResponse, error) { + // Check for configured error + errMsg, hasErr := pdk.GetConfig("error") + if hasErr && errMsg != "" { + return lyrics.GetLyricsResponse{}, fmt.Errorf("%s", errMsg) + } + + // Check if we should omit language (to test default language handling) + noLang, hasNoLang := pdk.GetConfig("no_lang") + lang := "eng" + if hasNoLang && noLang == "true" { + lang = "" + } + + // Return test lyrics based on track info + return lyrics.GetLyricsResponse{ + Lyrics: []lyrics.LyricsText{ + { + Lang: lang, + Text: "Test lyrics for " + input.Track.Title + "\nBy " + input.Track.Artist, + }, + }, + }, nil +} + +func main() {} diff --git a/plugins/testdata/test-lyrics/manifest.json b/plugins/testdata/test-lyrics/manifest.json new file mode 100644 index 000000000..a61299e92 --- /dev/null +++ b/plugins/testdata/test-lyrics/manifest.json @@ -0,0 +1,6 @@ +{ + "name": "Test Lyrics", + "author": "Navidrome Test", + "version": "1.0.0", + "description": "A test lyrics plugin for integration testing" +} diff --git a/server/e2e/e2e_suite_test.go b/server/e2e/e2e_suite_test.go index 0e0ca606a..484a91cc7 100644 --- a/server/e2e/e2e_suite_test.go +++ b/server/e2e/e2e_suite_test.go @@ -19,6 +19,7 @@ import ( "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/auth" "github.com/navidrome/navidrome/core/external" + "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playlists" @@ -396,6 +397,7 @@ func setupTestDB() { core.NewShare(ds), playback.PlaybackServer(nil), metrics.NewNoopInstance(), + lyrics.NewLyrics(nil), ) } diff --git a/server/subsonic/album_lists_test.go b/server/subsonic/album_lists_test.go index 63c2614cd..aac2d63da 100644 --- a/server/subsonic/album_lists_test.go +++ b/server/subsonic/album_lists_test.go @@ -27,7 +27,7 @@ var _ = Describe("Album Lists", func() { ds = &tests.MockDataStore{} auth.Init(ds) mockRepo = ds.Album(ctx).(*tests.MockAlbumRepo) - router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) w = httptest.NewRecorder() }) diff --git a/server/subsonic/api.go b/server/subsonic/api.go index d0d9bb169..8674a2946 100644 --- a/server/subsonic/api.go +++ b/server/subsonic/api.go @@ -14,6 +14,7 @@ import ( "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/external" + lyricssvc "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" playlistsvc "github.com/navidrome/navidrome/core/playlists" @@ -48,12 +49,13 @@ type Router struct { share core.Share playback playback.PlaybackServer metrics metrics.Metrics + lyrics lyricssvc.Lyrics } func New(ds model.DataStore, artwork artwork.Artwork, streamer core.MediaStreamer, archiver core.Archiver, players core.Players, provider external.Provider, scanner model.Scanner, broker events.Broker, playlists playlistsvc.Playlists, scrobbler scrobbler.PlayTracker, share core.Share, playback playback.PlaybackServer, - metrics metrics.Metrics, + metrics metrics.Metrics, lyrics lyricssvc.Lyrics, ) *Router { r := &Router{ ds: ds, @@ -69,6 +71,7 @@ func New(ds model.DataStore, artwork artwork.Artwork, streamer core.MediaStreame share: share, playback: playback, metrics: metrics, + lyrics: lyrics, } r.Handler = r.routes() return r diff --git a/server/subsonic/media_annotation_test.go b/server/subsonic/media_annotation_test.go index 6f09f5349..57809fbb6 100644 --- a/server/subsonic/media_annotation_test.go +++ b/server/subsonic/media_annotation_test.go @@ -27,7 +27,7 @@ var _ = Describe("MediaAnnotationController", func() { ds = &tests.MockDataStore{} playTracker = &fakePlayTracker{} eventBroker = &fakeEventBroker{} - router = New(ds, nil, nil, nil, nil, nil, nil, eventBroker, nil, playTracker, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, eventBroker, nil, playTracker, nil, nil, nil, nil) }) Describe("Scrobble", func() { diff --git a/server/subsonic/media_retrieval.go b/server/subsonic/media_retrieval.go index c16779e3a..54fcb5e3a 100644 --- a/server/subsonic/media_retrieval.go +++ b/server/subsonic/media_retrieval.go @@ -10,7 +10,6 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/consts" - "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/resources" @@ -109,7 +108,7 @@ func (api *Router) GetLyrics(r *http.Request) (*responses.Subsonic, error) { return response, nil } - structuredLyrics, err := lyrics.GetLyrics(r.Context(), &mediaFiles[0]) + structuredLyrics, err := api.lyrics.GetLyrics(r.Context(), &mediaFiles[0]) if err != nil { return nil, err } @@ -142,7 +141,7 @@ func (api *Router) GetLyricsBySongId(r *http.Request) (*responses.Subsonic, erro return nil, err } - structuredLyrics, err := lyrics.GetLyrics(r.Context(), mediaFile) + structuredLyrics, err := api.lyrics.GetLyrics(r.Context(), mediaFile) if err != nil { return nil, err } diff --git a/server/subsonic/media_retrieval_test.go b/server/subsonic/media_retrieval_test.go index 351b4e591..1a638f066 100644 --- a/server/subsonic/media_retrieval_test.go +++ b/server/subsonic/media_retrieval_test.go @@ -14,6 +14,7 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/core/artwork" + "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/server/subsonic/responses" "github.com/navidrome/navidrome/tests" @@ -33,7 +34,7 @@ var _ = Describe("MediaRetrievalController", func() { MockedMediaFile: mockRepo, } artwork = &fakeArtwork{data: "image data"} - router = New(ds, artwork, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = New(ds, artwork, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, lyrics.NewLyrics(nil)) w = httptest.NewRecorder() DeferCleanup(configtest.SetupConfig()) conf.Server.LyricsPriority = "embedded,.lrc" diff --git a/server/subsonic/opensubsonic_test.go b/server/subsonic/opensubsonic_test.go index 58dca682c..c02b262b9 100644 --- a/server/subsonic/opensubsonic_test.go +++ b/server/subsonic/opensubsonic_test.go @@ -19,7 +19,7 @@ var _ = Describe("GetOpenSubsonicExtensions", func() { ) BeforeEach(func() { - router = subsonic.New(nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = subsonic.New(nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) w = httptest.NewRecorder() r = httptest.NewRequest("GET", "/getOpenSubsonicExtensions?f=json", nil) }) diff --git a/server/subsonic/playlists_test.go b/server/subsonic/playlists_test.go index d99e244b0..86c17b39c 100644 --- a/server/subsonic/playlists_test.go +++ b/server/subsonic/playlists_test.go @@ -24,7 +24,7 @@ var _ = Describe("buildPlaylist", func() { BeforeEach(func() { ds = &tests.MockDataStore{} - router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) ctx = context.Background() }) @@ -224,7 +224,7 @@ var _ = Describe("UpdatePlaylist", func() { BeforeEach(func() { ds = &tests.MockDataStore{} playlists = &fakePlaylists{} - router = New(ds, nil, nil, nil, nil, nil, nil, nil, playlists, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, playlists, nil, nil, nil, nil, nil) }) It("clears the comment when parameter is empty", func() { diff --git a/server/subsonic/searching_test.go b/server/subsonic/searching_test.go index 7f7de381a..d4b7e9702 100644 --- a/server/subsonic/searching_test.go +++ b/server/subsonic/searching_test.go @@ -21,7 +21,7 @@ var _ = Describe("Search", func() { ds = &tests.MockDataStore{} auth.Init(ds) - router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) // Get references to the mock repositories so we can inspect their Options mockAlbumRepo = ds.Album(nil).(*tests.MockAlbumRepo) From 11e4aaed1ba3a1cc6420facb888cbbcf4589b445 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Wed, 4 Mar 2026 22:42:49 -0500 Subject: [PATCH 41/60] feat(server): add percentage-based limits to smart playlists (#5144) * feat(playlists): add percentage-based limits to smart playlists Add a new `limitPercent` JSON field to Criteria that allows smart playlist limits to be expressed as a percentage of matching tracks rather than a fixed number. For example, a playlist matching 450 songs with a 10% limit returns 45 songs, scaling dynamically as the library grows. When `limitPercent` is set, refreshSmartPlaylist runs a COUNT query first to determine the total matching tracks, then resolves the percentage to an absolute LIMIT before executing the main query. The fixed `limit` field takes precedence when both are set. Values are clamped to [0, 100] during JSON unmarshaling. No database migration is needed since rules are stored as a JSON string. * fix(criteria): validate percentage limit range in IsPercentageLimit method Signed-off-by: Deluan * fix(criteria): ensure idempotency of ToSql method for expressions Signed-off-by: Deluan --------- Signed-off-by: Deluan --- core/playlists/parse_nsp_test.go | 15 +++ model/criteria/criteria.go | 91 ++++++++++++++---- model/criteria/criteria_test.go | 142 +++++++++++++++++++++++++++++ model/criteria/fields.go | 25 +++-- model/criteria/operators_test.go | 15 +++ persistence/playlist_repository.go | 54 ++++++++--- 6 files changed, 296 insertions(+), 46 deletions(-) diff --git a/core/playlists/parse_nsp_test.go b/core/playlists/parse_nsp_test.go index 0a7b2727e..516a5355d 100644 --- a/core/playlists/parse_nsp_test.go +++ b/core/playlists/parse_nsp_test.go @@ -122,6 +122,21 @@ var _ = Describe("parseNSP", func() { Expect(pls.Name).To(Equal("Original")) }) + It("parses limitPercent from NSP", func() { + nsp := `{ + "all": [{"is": {"loved": true}}], + "sort": "playCount", + "order": "desc", + "limitPercent": 25 + }` + pls := &model.Playlist{} + err := s.parseNSP(ctx, pls, strings.NewReader(nsp)) + Expect(err).ToNot(HaveOccurred()) + Expect(pls.Rules).ToNot(BeNil()) + Expect(pls.Rules.LimitPercent).To(Equal(25)) + Expect(pls.Rules.Limit).To(Equal(0)) + }) + It("parses criteria with multiple rules", func() { nsp := `{ "all": [ diff --git a/model/criteria/criteria.go b/model/criteria/criteria.go index bc3fe801c..278acf34c 100644 --- a/model/criteria/criteria.go +++ b/model/criteria/criteria.go @@ -15,10 +15,38 @@ type Expression = squirrel.Sqlizer type Criteria struct { Expression - Sort string - Order string - Limit int - Offset int + Sort string + Order string + Limit int + LimitPercent int + Offset int +} + +// EffectiveLimit resolves the effective limit for a query. If a fixed Limit is +// set it takes precedence. Otherwise, if LimitPercent is set, the limit is +// computed as a percentage of totalCount (minimum 1 when totalCount > 0). +// Returns 0 when no limit applies. +func (c Criteria) EffectiveLimit(totalCount int64) int { + if c.Limit > 0 { + return c.Limit + } + if c.LimitPercent > 0 && c.LimitPercent <= 100 { + if totalCount <= 0 { + return 0 + } + result := int(totalCount) * c.LimitPercent / 100 + if result < 1 { + return 1 + } + return result + } + return 0 +} + +// IsPercentageLimit returns true when the criteria uses a valid percentage-based +// limit (i.e. LimitPercent is in [1, 100] and no fixed Limit overrides it). +func (c Criteria) IsPercentageLimit() bool { + return c.Limit == 0 && c.LimitPercent > 0 && c.LimitPercent <= 100 } func (c Criteria) OrderBy() string { @@ -95,6 +123,16 @@ func (c Criteria) ToSql() (sql string, args []any, err error) { return c.Expression.ToSql() } +// ExpressionJoins returns only the JOINs needed by the WHERE-clause expression, +// excluding any JOINs required solely for sorting. This is useful for COUNT +// queries where sort order is irrelevant. +func (c Criteria) ExpressionJoins() JoinType { + if c.Expression == nil { + return JoinNone + } + return extractJoinTypes(c.Expression) +} + // RequiredJoins inspects the expression tree and Sort field to determine which // additional JOINs are needed when evaluating this criteria. func (c Criteria) RequiredJoins() JoinType { @@ -128,17 +166,19 @@ func (c Criteria) ChildPlaylistIds() []string { func (c Criteria) MarshalJSON() ([]byte, error) { aux := struct { - All []Expression `json:"all,omitempty"` - Any []Expression `json:"any,omitempty"` - Sort string `json:"sort,omitempty"` - Order string `json:"order,omitempty"` - Limit int `json:"limit,omitempty"` - Offset int `json:"offset,omitempty"` + All []Expression `json:"all,omitempty"` + Any []Expression `json:"any,omitempty"` + Sort string `json:"sort,omitempty"` + Order string `json:"order,omitempty"` + Limit int `json:"limit,omitempty"` + LimitPercent int `json:"limitPercent,omitempty"` + Offset int `json:"offset,omitempty"` }{ - Sort: c.Sort, - Order: c.Order, - Limit: c.Limit, - Offset: c.Offset, + Sort: c.Sort, + Order: c.Order, + Limit: c.Limit, + LimitPercent: c.LimitPercent, + Offset: c.Offset, } switch rules := c.Expression.(type) { case Any: @@ -153,12 +193,13 @@ func (c Criteria) MarshalJSON() ([]byte, error) { func (c *Criteria) UnmarshalJSON(data []byte) error { var aux struct { - All unmarshalConjunctionType `json:"all"` - Any unmarshalConjunctionType `json:"any"` - Sort string `json:"sort"` - Order string `json:"order"` - Limit int `json:"limit"` - Offset int `json:"offset"` + All unmarshalConjunctionType `json:"all"` + Any unmarshalConjunctionType `json:"any"` + Sort string `json:"sort"` + Order string `json:"order"` + Limit int `json:"limit"` + LimitPercent int `json:"limitPercent"` + Offset int `json:"offset"` } if err := json.Unmarshal(data, &aux); err != nil { return err @@ -174,5 +215,15 @@ func (c *Criteria) UnmarshalJSON(data []byte) error { c.Order = aux.Order c.Limit = aux.Limit c.Offset = aux.Offset + + // Clamp LimitPercent to [0, 100] + if aux.LimitPercent < 0 { + log.Warn("limitPercent value out of range, clamping to 0", "value", aux.LimitPercent) + aux.LimitPercent = 0 + } else if aux.LimitPercent > 100 { + log.Warn("limitPercent value out of range, clamping to 100", "value", aux.LimitPercent) + aux.LimitPercent = 100 + } + c.LimitPercent = aux.LimitPercent return nil } diff --git a/model/criteria/criteria_test.go b/model/criteria/criteria_test.go index 9a4da360c..a76b3fc1f 100644 --- a/model/criteria/criteria_test.go +++ b/model/criteria/criteria_test.go @@ -181,6 +181,28 @@ var _ = Describe("Criteria", func() { }) }) + Describe("ExpressionJoins", func() { + It("excludes sort-only joins", func() { + c := Criteria{ + Expression: All{ + Contains{"title": "love"}, + }, + Sort: "albumRating", + } + gomega.Expect(c.ExpressionJoins()).To(gomega.Equal(JoinNone)) + gomega.Expect(c.RequiredJoins().Has(JoinAlbumAnnotation)).To(gomega.BeTrue()) + }) + + It("includes expression-based joins", func() { + c := Criteria{ + Expression: All{ + Gt{"albumRating": 3}, + }, + } + gomega.Expect(c.ExpressionJoins().Has(JoinAlbumAnnotation)).To(gomega.BeTrue()) + }) + }) + Describe("RequiredJoins", func() { It("returns JoinNone when no annotation fields are used", func() { c := Criteria{ @@ -263,6 +285,126 @@ var _ = Describe("Criteria", func() { }) }) + Describe("LimitPercent", func() { + Describe("JSON round-trip", func() { + It("marshals and unmarshals limitPercent", func() { + goObj := Criteria{ + Expression: All{Contains{"title": "love"}}, + Sort: "title", + Order: "asc", + LimitPercent: 10, + } + j, err := json.Marshal(goObj) + gomega.Expect(err).ToNot(gomega.HaveOccurred()) + gomega.Expect(string(j)).To(gomega.ContainSubstring(`"limitPercent":10`)) + gomega.Expect(string(j)).ToNot(gomega.ContainSubstring(`"limit"`)) + + var newObj Criteria + err = json.Unmarshal(j, &newObj) + gomega.Expect(err).ToNot(gomega.HaveOccurred()) + gomega.Expect(newObj.LimitPercent).To(gomega.Equal(10)) + gomega.Expect(newObj.Limit).To(gomega.Equal(0)) + }) + + It("does not include limitPercent when zero", func() { + goObj := Criteria{ + Expression: All{Contains{"title": "love"}}, + Limit: 50, + } + j, err := json.Marshal(goObj) + gomega.Expect(err).ToNot(gomega.HaveOccurred()) + gomega.Expect(string(j)).To(gomega.ContainSubstring(`"limit":50`)) + gomega.Expect(string(j)).ToNot(gomega.ContainSubstring(`limitPercent`)) + }) + + It("backward compatible: JSON with only limit still works", func() { + jsonStr := `{"all":[{"contains":{"title":"love"}}],"limit":20}` + var c Criteria + err := json.Unmarshal([]byte(jsonStr), &c) + gomega.Expect(err).ToNot(gomega.HaveOccurred()) + gomega.Expect(c.Limit).To(gomega.Equal(20)) + gomega.Expect(c.LimitPercent).To(gomega.Equal(0)) + }) + }) + + Describe("UnmarshalJSON clamping", func() { + It("clamps values above 100 to 100", func() { + jsonStr := `{"all":[{"contains":{"title":"love"}}],"limitPercent":150}` + var c Criteria + err := json.Unmarshal([]byte(jsonStr), &c) + gomega.Expect(err).ToNot(gomega.HaveOccurred()) + gomega.Expect(c.LimitPercent).To(gomega.Equal(100)) + }) + + It("clamps negative values to 0", func() { + jsonStr := `{"all":[{"contains":{"title":"love"}}],"limitPercent":-5}` + var c Criteria + err := json.Unmarshal([]byte(jsonStr), &c) + gomega.Expect(err).ToNot(gomega.HaveOccurred()) + gomega.Expect(c.LimitPercent).To(gomega.Equal(0)) + }) + }) + + Describe("EffectiveLimit", func() { + It("returns fixed limit when Limit is set", func() { + c := Criteria{Limit: 50, LimitPercent: 10} + gomega.Expect(c.EffectiveLimit(1000)).To(gomega.Equal(50)) + }) + + It("returns percentage-based limit", func() { + c := Criteria{LimitPercent: 10} + gomega.Expect(c.EffectiveLimit(450)).To(gomega.Equal(45)) + }) + + It("returns minimum 1 when totalCount > 0 and percentage rounds to 0", func() { + c := Criteria{LimitPercent: 1} + gomega.Expect(c.EffectiveLimit(5)).To(gomega.Equal(1)) + }) + + It("returns 0 when totalCount is 0", func() { + c := Criteria{LimitPercent: 10} + gomega.Expect(c.EffectiveLimit(0)).To(gomega.Equal(0)) + }) + + It("returns 0 when no limit is set", func() { + c := Criteria{} + gomega.Expect(c.EffectiveLimit(1000)).To(gomega.Equal(0)) + }) + + It("returns full count for 100%", func() { + c := Criteria{LimitPercent: 100} + gomega.Expect(c.EffectiveLimit(450)).To(gomega.Equal(450)) + }) + + It("returns 1 for 1% of 50 items", func() { + c := Criteria{LimitPercent: 1} + gomega.Expect(c.EffectiveLimit(50)).To(gomega.Equal(1)) + }) + }) + + Describe("IsPercentageLimit", func() { + It("returns true when LimitPercent is set and Limit is 0", func() { + c := Criteria{LimitPercent: 10} + gomega.Expect(c.IsPercentageLimit()).To(gomega.BeTrue()) + }) + + It("returns false when Limit is set", func() { + c := Criteria{Limit: 50, LimitPercent: 10} + gomega.Expect(c.IsPercentageLimit()).To(gomega.BeFalse()) + }) + + It("returns false when neither is set", func() { + c := Criteria{} + gomega.Expect(c.IsPercentageLimit()).To(gomega.BeFalse()) + }) + + It("returns false when LimitPercent is out of range", func() { + c := Criteria{LimitPercent: 150} + gomega.Expect(c.IsPercentageLimit()).To(gomega.BeFalse()) + }) + }) + }) + Context("with child playlists", func() { var ( topLevelInPlaylistID string diff --git a/model/criteria/fields.go b/model/criteria/fields.go index ed73de9f9..b9d91f087 100644 --- a/model/criteria/fields.go +++ b/model/criteria/fields.go @@ -122,27 +122,24 @@ func mapExpr(expr squirrel.Sqlizer, negate bool, exprFunc func(string, squirrel. log.Fatal(fmt.Sprintf("expr is not a map-based operator: %T", expr)) } - // Extract into a generic map + // Extract the field name and value, then build a new map keyed by "value" + // for the inner condition. The original map is left untouched so that + // ToSql can be called multiple times without corruption. var k string - m := make(map[string]any, rv.Len()) + var v any for _, key := range rv.MapKeys() { - // Save the key to build the expression, and use the provided keyName as the key k = key.String() - m["value"] = rv.MapIndex(key).Interface() + v = rv.MapIndex(key).Interface() break // only one key is expected (and supported) } - // Clear the original map - for _, key := range rv.MapKeys() { - rv.SetMapIndex(key, reflect.Value{}) - } + // Create a new map-based expression with "value" as the key, matching the + // column name inside json_tree subqueries. + newMap := reflect.MakeMap(rv.Type()) + newMap.SetMapIndex(reflect.ValueOf("value"), reflect.ValueOf(v)) + newExpr := newMap.Interface().(squirrel.Sqlizer) - // Write the updated map back into the original variable - for key, val := range m { - rv.SetMapIndex(reflect.ValueOf(key), reflect.ValueOf(val)) - } - - return exprFunc(k, expr, negate) + return exprFunc(k, newExpr, negate) } // mapTagExpr maps a normal field expression to a tag expression. diff --git a/model/criteria/operators_test.go b/model/criteria/operators_test.go index f0681af6b..5f756f97d 100644 --- a/model/criteria/operators_test.go +++ b/model/criteria/operators_test.go @@ -178,6 +178,21 @@ var _ = Describe("Operators", func() { }) }) + DescribeTable("ToSql idempotency", + func(expr Expression) { + sql1, args1, err1 := expr.ToSql() + sql2, args2, err2 := expr.ToSql() + + gomega.Expect(err1).ToNot(gomega.HaveOccurred()) + gomega.Expect(err2).ToNot(gomega.HaveOccurred()) + gomega.Expect(sql2).To(gomega.Equal(sql1)) + gomega.Expect(args2).To(gomega.Equal(args1)) + }, + Entry("tag expression", Is{"genre": "Rock"}), + Entry("role expression", Contains{"artist": "Beatles"}), + Entry("nested criteria", Criteria{Expression: All{Is{"genre": "Rock"}, Contains{"artist": "Beatles"}}}), + ) + DescribeTable("JSON Marshaling", func(op Expression, jsonString string) { obj := And{op} diff --git a/persistence/playlist_repository.go b/persistence/playlist_repository.go index 11b9cd8b4..8d1bbe0f8 100644 --- a/persistence/playlist_repository.go +++ b/persistence/playlist_repository.go @@ -248,22 +248,36 @@ func (r *playlistRepository) refreshSmartPlaylist(pls *model.Playlist) bool { // Conditionally join album/artist annotation tables only when referenced by criteria or sort requiredJoins := rules.RequiredJoins() - if requiredJoins.Has(criteria.JoinAlbumAnnotation) { - sq = sq.LeftJoin("annotation AS album_annotation ON ("+ - "album_annotation.item_id = media_file.album_id"+ - " AND album_annotation.item_type = 'album'"+ - " AND album_annotation.user_id = ?)", usr.ID) - } - if requiredJoins.Has(criteria.JoinArtistAnnotation) { - sq = sq.LeftJoin("annotation AS artist_annotation ON ("+ - "artist_annotation.item_id = media_file.artist_id"+ - " AND artist_annotation.item_type = 'artist'"+ - " AND artist_annotation.user_id = ?)", usr.ID) - } + sq = r.addSmartPlaylistAnnotationJoins(sq, requiredJoins, usr.ID) // Only include media files from libraries the user has access to sq = r.applyLibraryFilter(sq, "media_file") + // Resolve percentage-based limit to an absolute number before applying criteria + if rules.IsPercentageLimit() { + // Use only expression-based joins for the COUNT query (sort joins are unnecessary) + exprJoins := rules.ExpressionJoins() + countSq := Select("count(*) as count").From("media_file"). + LeftJoin("annotation on ("+ + "annotation.item_id = media_file.id"+ + " AND annotation.item_type = 'media_file'"+ + " AND annotation.user_id = ?)", usr.ID) + countSq = r.addSmartPlaylistAnnotationJoins(countSq, exprJoins, usr.ID) + countSq = r.applyLibraryFilter(countSq, "media_file") + countSq = countSq.Where(rules) + + var res struct{ Count int64 } + err = r.queryOne(countSq, &res) + if err != nil { + log.Error(r.ctx, "Error counting matching tracks for percentage limit", "playlist", pls.Name, "id", pls.ID, err) + return false + } + resolvedLimit := rules.EffectiveLimit(res.Count) + log.Debug(r.ctx, "Resolved percentage limit", "playlist", pls.Name, "percent", rules.LimitPercent, "totalMatching", res.Count, "resolvedLimit", resolvedLimit) + rules.Limit = resolvedLimit + rules.LimitPercent = 0 + } + // Apply the criteria rules sq = r.addCriteria(sq, rules) insSql := Insert("playlist_tracks").Columns("id", "playlist_id", "media_file_id").Select(sq) @@ -296,6 +310,22 @@ func (r *playlistRepository) refreshSmartPlaylist(pls *model.Playlist) bool { return true } +func (r *playlistRepository) addSmartPlaylistAnnotationJoins(sq SelectBuilder, joins criteria.JoinType, userID string) SelectBuilder { + if joins.Has(criteria.JoinAlbumAnnotation) { + sq = sq.LeftJoin("annotation AS album_annotation ON ("+ + "album_annotation.item_id = media_file.album_id"+ + " AND album_annotation.item_type = 'album'"+ + " AND album_annotation.user_id = ?)", userID) + } + if joins.Has(criteria.JoinArtistAnnotation) { + sq = sq.LeftJoin("annotation AS artist_annotation ON ("+ + "artist_annotation.item_id = media_file.artist_id"+ + " AND artist_annotation.item_type = 'artist'"+ + " AND artist_annotation.user_id = ?)", userID) + } + return sq +} + func (r *playlistRepository) addCriteria(sql SelectBuilder, c criteria.Criteria) SelectBuilder { sql = sql.Where(c) if c.Limit > 0 { From 627266ec82070bba1ce54ab19330b26a0d59bb7b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 5 Mar 2026 08:01:53 -0500 Subject: [PATCH 42/60] chore(deps): bump immutable from 4.3.7 to 4.3.8 in /ui (#5145) Bumps [immutable](https://github.com/immutable-js/immutable-js) from 4.3.7 to 4.3.8. - [Release notes](https://github.com/immutable-js/immutable-js/releases) - [Changelog](https://github.com/immutable-js/immutable-js/blob/main/CHANGELOG.md) - [Commits](https://github.com/immutable-js/immutable-js/compare/v4.3.7...v4.3.8) --- updated-dependencies: - dependency-name: immutable dependency-version: 4.3.8 dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- ui/package-lock.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ui/package-lock.json b/ui/package-lock.json index b4efed744..bd8a2d2ed 100644 --- a/ui/package-lock.json +++ b/ui/package-lock.json @@ -7183,9 +7183,9 @@ } }, "node_modules/immutable": { - "version": "4.3.7", - "resolved": "https://registry.npmjs.org/immutable/-/immutable-4.3.7.tgz", - "integrity": "sha512-1hqclzwYwjRDFLjcFxOM5AYkkG0rpFPpr1RLPMEuGczoS7YA8gLhy8SWXYRAA/XwfEHpfo3cw5JGioS32fnMRw==", + "version": "4.3.8", + "resolved": "https://registry.npmjs.org/immutable/-/immutable-4.3.8.tgz", + "integrity": "sha512-d/Ld9aLbKpNwyl0KiM2CT1WYvkitQ1TSvmRtkcV8FKStiDoA7Slzgjmb/1G2yhKM1p0XeNOieaTbFZmU1d3Xuw==", "license": "MIT", "optional": true }, From 12f28b9d97b563c238a059cdd8e09259be1283e0 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 5 Mar 2026 19:06:12 -0500 Subject: [PATCH 43/60] chore(deps): bump dompurify in /ui (#5147) Bumps [dompurify](https://github.com/cure53/DOMPurify) to 3.3.2 and updates ancestor dependency . These dependencies need to be updated together. Updates `dompurify` from 3.3.1 to 3.3.2 - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.3.1...3.3.2) Updates `dompurify` from 2.5.8 to 2.5.9 - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.3.1...3.3.2) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.3.2 dependency-type: direct:production - dependency-name: dompurify dependency-version: 2.5.9 dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- ui/package-lock.json | 17 ++++++++++------- ui/package.json | 2 +- 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/ui/package-lock.json b/ui/package-lock.json index bd8a2d2ed..04d6fe07c 100644 --- a/ui/package-lock.json +++ b/ui/package-lock.json @@ -18,7 +18,7 @@ "clsx": "^2.1.1", "connected-react-router": "^6.9.3", "deepmerge": "^4.3.1", - "dompurify": "^3.3.1", + "dompurify": "^3.3.2", "history": "^4.10.1", "inflection": "^3.0.2", "jwt-decode": "^4.0.0", @@ -5503,10 +5503,13 @@ "license": "MIT" }, "node_modules/dompurify": { - "version": "3.3.1", - "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.3.1.tgz", - "integrity": "sha512-qkdCKzLNtrgPFP1Vo+98FRzJnBRGe4ffyCea9IwHB1fyxPOeNTHpLKYGd4Uk9xvNoH0ZoOjwZxNptyMwqrId1Q==", + "version": "3.3.2", + "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.3.2.tgz", + "integrity": "sha512-6obghkliLdmKa56xdbLOpUZ43pAR6xFy1uOrxBaIDjT+yaRuuybLjGS9eVBoSR/UPU5fq3OXClEHLJNGvbxKpQ==", "license": "(MPL-2.0 OR Apache-2.0)", + "engines": { + "node": ">=20" + }, "optionalDependencies": { "@types/trusted-types": "^2.0.7" } @@ -9682,9 +9685,9 @@ "license": "MIT" }, "node_modules/ra-ui-materialui/node_modules/dompurify": { - "version": "2.5.8", - "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-2.5.8.tgz", - "integrity": "sha512-o1vSNgrmYMQObbSSvF/1brBYEQPHhV1+gsmrusO7/GXtp1T9rCS8cXFqVxK/9crT1jA6Ccv+5MTSjBNqr7Sovw==", + "version": "2.5.9", + "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-2.5.9.tgz", + "integrity": "sha512-i6mvVmWN4xo9LrhCOZrDgSs9noW6nOahbrmzjRbPF36YPyj5Ue5lgok0MHDWkG7xzpWFO2OYttXdzM7rJxHvNA==", "license": "(MPL-2.0 OR Apache-2.0)" }, "node_modules/ra-ui-materialui/node_modules/inflection": { diff --git a/ui/package.json b/ui/package.json index 54062c1ac..5b4deb773 100644 --- a/ui/package.json +++ b/ui/package.json @@ -27,7 +27,7 @@ "clsx": "^2.1.1", "connected-react-router": "^6.9.3", "deepmerge": "^4.3.1", - "dompurify": "^3.3.1", + "dompurify": "^3.3.2", "history": "^4.10.1", "inflection": "^3.0.2", "jwt-decode": "^4.0.0", From 1ce561cc8ead1dba3dd68c6e0eb03a2ca789b91c Mon Sep 17 00:00:00 2001 From: Deluan Date: Thu, 5 Mar 2026 19:53:55 -0500 Subject: [PATCH 44/60] refactor(server): remove legacy embedded coverart logic Signed-off-by: Deluan --- conf/configuration.go | 1 - core/artwork/sources.go | 54 ----------------------------------------- go.mod | 10 ++------ go.sum | 2 -- 4 files changed, 2 insertions(+), 65 deletions(-) diff --git a/conf/configuration.go b/conf/configuration.go index b6f65183a..c46879d42 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -132,7 +132,6 @@ type configOptions struct { DevExternalScanner bool DevScannerThreads uint DevSelectiveWatcher bool - DevLegacyEmbedImage bool DevInsightsInitialDelay time.Duration DevEnablePlayerInsights bool DevEnablePluginsInsights bool diff --git a/core/artwork/sources.go b/core/artwork/sources.go index b1b9b5454..0628461e0 100644 --- a/core/artwork/sources.go +++ b/core/artwork/sources.go @@ -15,8 +15,6 @@ import ( "strings" "time" - "github.com/dhowden/tag" - "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/core/external" "github.com/navidrome/navidrome/core/ffmpeg" @@ -86,58 +84,6 @@ var picTypeRegexes = []*regexp.Regexp{ } func fromTag(ctx context.Context, path string) sourceFunc { - if conf.Server.DevLegacyEmbedImage { - return fromTagLegacy(ctx, path) - } - return fromTagGoTaglib(ctx, path) -} - -func fromTagLegacy(ctx context.Context, path string) sourceFunc { - return func() (io.ReadCloser, string, error) { - if path == "" { - return nil, "", nil - } - f, err := os.Open(path) - if err != nil { - return nil, "", err - } - defer f.Close() - - m, err := tag.ReadFrom(f) - if err != nil { - return nil, "", err - } - - types := m.PictureTypes() - if len(types) == 0 { - return nil, "", fmt.Errorf("no embedded image found in %s", path) - } - - var picture *tag.Picture - for _, regex := range picTypeRegexes { - for _, t := range types { - if regex.MatchString(t) { - log.Trace(ctx, "Found embedded image", "type", t, "path", path) - picture = m.Pictures(t) - break - } - } - if picture != nil { - break - } - } - if picture == nil { - log.Trace(ctx, "Could not find a front image. Getting the first one", "type", types[0], "path", path) - picture = m.Picture() - } - if picture == nil { - return nil, "", fmt.Errorf("could not load embedded image from %s", path) - } - return io.NopCloser(bytes.NewReader(picture.Data)), path, nil - } -} - -func fromTagGoTaglib(ctx context.Context, path string) sourceFunc { return func() (io.ReadCloser, string, error) { if path == "" { return nil, "", nil diff --git a/go.mod b/go.mod index 7f7e90a7f..abcadab73 100644 --- a/go.mod +++ b/go.mod @@ -2,13 +2,8 @@ module github.com/navidrome/navidrome go 1.25.0 -replace ( - // Fork to fix https://github.com/navidrome/navidrome/issues/3254 - github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8 => github.com/deluan/tag v0.0.0-20241002021117-dfe5e6ea396d - - // Fork to implement raw tags support - go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 -) +// Fork to implement raw tags support +replace go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 require ( github.com/Masterminds/squirrel v1.5.4 @@ -19,7 +14,6 @@ require ( github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf github.com/deluan/sanitize v0.0.0-20241120162836-fdfd8fdfaa55 github.com/dexterlb/mpvipc v0.0.0-20241005113212-7cdefca0e933 - github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8 github.com/disintegration/imaging v1.6.2 github.com/djherbis/atime v1.1.0 github.com/djherbis/fscache v0.10.2-0.20231127215153-442a07e326c4 diff --git a/go.sum b/go.sum index 224b33e90..e46f1c2e9 100644 --- a/go.sum +++ b/go.sum @@ -42,8 +42,6 @@ github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf h1:tb246l2Zmpt/GpF9EcH github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf/go.mod h1:tSgDythFsl0QgS/PFWfIZqcJKnkADWneY80jaVRlqK8= github.com/deluan/sanitize v0.0.0-20241120162836-fdfd8fdfaa55 h1:wSCnggTs2f2ji6nFwQmfwgINcmSMj0xF0oHnoyRSPe4= github.com/deluan/sanitize v0.0.0-20241120162836-fdfd8fdfaa55/go.mod h1:ZNCLJfehvEf34B7BbLKjgpsL9lyW7q938w/GY1XgV4E= -github.com/deluan/tag v0.0.0-20241002021117-dfe5e6ea396d h1:x/R3+oPEjnisl1zBx2f2v7Gf6f11l0N0JoD6BkwcJyA= -github.com/deluan/tag v0.0.0-20241002021117-dfe5e6ea396d/go.mod h1:apkPC/CR3s48O2D7Y++n1XWEpgPNNCjXYga3PPbJe2E= github.com/dexterlb/mpvipc v0.0.0-20241005113212-7cdefca0e933 h1:r4hxcT6GBIA/j8Ox4OXI5MNgMKfR+9plcAWYi1OnmOg= github.com/dexterlb/mpvipc v0.0.0-20241005113212-7cdefca0e933/go.mod h1:RkQWLNITKkXHLP7LXxZSgEq+uFWU25M5qW7qfEhL9Wc= github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c= From d2db41691e20df4b49e74a676e4d8273f52cdf50 Mon Sep 17 00:00:00 2001 From: Deluan Date: Thu, 5 Mar 2026 20:47:35 -0500 Subject: [PATCH 45/60] fix(ui): conditionally render sync toggle based on screen size Signed-off-by: Deluan --- ui/src/playlist/PlaylistList.jsx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/ui/src/playlist/PlaylistList.jsx b/ui/src/playlist/PlaylistList.jsx index 4ec2d5ca1..67c456f27 100644 --- a/ui/src/playlist/PlaylistList.jsx +++ b/ui/src/playlist/PlaylistList.jsx @@ -181,7 +181,9 @@ const PlaylistList = (props) => { ), comment: , - sync: , + sync: !isXsmall && ( + + ), }), [isDesktop, isXsmall], ) From f102036dc6a4df88ea9e4ef53437f65f2bb7d01d Mon Sep 17 00:00:00 2001 From: Deluan Date: Thu, 5 Mar 2026 20:56:16 -0500 Subject: [PATCH 46/60] fix(server): clear server-managed fields in savePlaylist to prevent injection via REST API Signed-off-by: Deluan --- core/playlists/rest_adapter.go | 8 ++++++- core/playlists/rest_adapter_test.go | 34 +++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/core/playlists/rest_adapter.go b/core/playlists/rest_adapter.go index 3865d97e9..c9cd4c136 100644 --- a/core/playlists/rest_adapter.go +++ b/core/playlists/rest_adapter.go @@ -58,10 +58,16 @@ func (s *playlists) TracksRepository(ctx context.Context, playlistId string, ref } // savePlaylist creates a new playlist, assigning the owner from context. +// Only Name, Comment, Public, and Rules are user-settable via the REST API. func (s *playlists) savePlaylist(ctx context.Context, pls *model.Playlist) (string, error) { usr, _ := request.UserFrom(ctx) pls.OwnerID = usr.ID - pls.ID = "" // Force new creation + pls.ID = "" // Force new creation + pls.Path = "" // Server-managed (M3U file path) + pls.Sync = false // Server-managed (M3U sync flag) + pls.UploadedImage = "" // Managed by image upload endpoint + pls.ExternalImageURL = "" // Managed by M3U import / plugins only + pls.EvaluatedAt = nil // Server-managed err := s.ds.Playlist(ctx).Put(pls) if err != nil { return "", err diff --git a/core/playlists/rest_adapter_test.go b/core/playlists/rest_adapter_test.go index b65095957..29db2fc33 100644 --- a/core/playlists/rest_adapter_test.go +++ b/core/playlists/rest_adapter_test.go @@ -2,10 +2,12 @@ package playlists_test import ( "context" + "time" "github.com/deluan/rest" "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/criteria" "github.com/navidrome/navidrome/model/request" "github.com/navidrome/navidrome/tests" . "github.com/onsi/ginkgo/v2" @@ -56,6 +58,38 @@ var _ = Describe("REST Adapter", func() { Expect(err).ToNot(HaveOccurred()) Expect(pls.ID).ToNot(Equal("should-be-cleared")) }) + + It("clears server-managed fields to prevent injection via REST API", func() { + ctx = request.WithUser(ctx, model.User{ID: "user-1", IsAdmin: false}) + repo = ps.NewRepository(ctx).(rest.Persistable) + now := time.Now() + pls := &model.Playlist{ + Name: "Legit Playlist", + Comment: "A comment", + Public: true, + Rules: &criteria.Criteria{Expression: criteria.Contains{"title": "test"}}, + Path: "/some/path/playlist.m3u", + Sync: true, + UploadedImage: "injected-image-path", + ExternalImageURL: "http://evil.example.com/ssrf", + EvaluatedAt: &now, + } + _, err := repo.Save(pls) + Expect(err).ToNot(HaveOccurred()) + + saved := mockPlsRepo.Last + // User-settable fields are preserved + Expect(saved.Name).To(Equal("Legit Playlist")) + Expect(saved.Comment).To(Equal("A comment")) + Expect(saved.Public).To(BeTrue()) + Expect(saved.Rules).ToNot(BeNil()) + // Server-managed fields are cleared + Expect(saved.Path).To(BeEmpty()) + Expect(saved.Sync).To(BeFalse()) + Expect(saved.UploadedImage).To(BeEmpty()) + Expect(saved.ExternalImageURL).To(BeEmpty()) + Expect(saved.EvaluatedAt).To(BeNil()) + }) }) Describe("Update", func() { From 3cd5d16b0ae615d155d9a1320ee44a977e8bbde3 Mon Sep 17 00:00:00 2001 From: Deluan Date: Fri, 6 Mar 2026 19:23:47 -0500 Subject: [PATCH 47/60] chore: upgrade golangci-lint to 2.11 and fix lint issues Signed-off-by: Deluan --- .golangci.yml | 5 +++++ Makefile | 2 +- plugins/host_taskqueue.go | 2 +- scanner/watcher.go | 2 +- server/nativeapi/playlists.go | 2 +- server/subsonic/middlewares.go | 2 ++ utils/pl/pipelines.go | 4 ++-- 7 files changed, 13 insertions(+), 6 deletions(-) diff --git a/.golangci.yml b/.golangci.yml index 1937c2f77..b6c632dee 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -40,6 +40,11 @@ linters: enable: - nilness exclusions: + rules: + - linters: + - gosec + path: _test\.go + text: "G703" generated: lax presets: - comments diff --git a/Makefile b/Makefile index f7b7b1b05..559a34c3c 100644 --- a/Makefile +++ b/Makefile @@ -21,7 +21,7 @@ DOCKER_TAG ?= deluan/navidrome:develop # Taglib version to use in cross-compilation, from https://github.com/navidrome/cross-taglib CROSS_TAGLIB_VERSION ?= 2.2.0-1 -GOLANGCI_LINT_VERSION ?= v2.10.0 +GOLANGCI_LINT_VERSION ?= v2.11.1 UI_SRC_FILES := $(shell find ui -type f -not -path "ui/build/*" -not -path "ui/node_modules/*") diff --git a/plugins/host_taskqueue.go b/plugins/host_taskqueue.go index 283bc9635..9f2ed85f6 100644 --- a/plugins/host_taskqueue.go +++ b/plugins/host_taskqueue.go @@ -101,7 +101,7 @@ func newTaskQueueService(pluginName string, manager *Manager, maxConcurrency int return nil, fmt.Errorf("creating taskqueue schema: %w", err) } - ctx, cancel := context.WithCancel(manager.ctx) + ctx, cancel := context.WithCancel(manager.ctx) //nolint:gosec // cancel is stored in struct and called in Close() s := &taskQueueServiceImpl{ pluginName: pluginName, diff --git a/scanner/watcher.go b/scanner/watcher.go index 101e3793a..62fcc9341 100644 --- a/scanner/watcher.go +++ b/scanner/watcher.go @@ -158,7 +158,7 @@ func (w *watcher) Watch(ctx context.Context, lib *model.Library) error { } // Start new watcher - watcherCtx, cancel := context.WithCancel(w.mainCtx) + watcherCtx, cancel := context.WithCancel(w.mainCtx) //nolint:gosec // cancel is stored in instance and called on shutdown instance := &libraryWatcherInstance{ library: lib, cancel: cancel, diff --git a/server/nativeapi/playlists.go b/server/nativeapi/playlists.go index 797654a3b..118528f68 100644 --- a/server/nativeapi/playlists.go +++ b/server/nativeapi/playlists.go @@ -247,7 +247,7 @@ func uploadPlaylistImage(pls playlists.Playlists) http.HandlerFunc { p := req.Params(r) playlistId, _ := p.String(":id") - if err := r.ParseMultipartForm(maxImageSize); err != nil { + if err := r.ParseMultipartForm(maxImageSize); err != nil { //nolint:gosec // size is limited by maxImageSize parameter log.Error(ctx, "Error parsing multipart form", err) http.Error(w, "file too large or invalid form", http.StatusBadRequest) return diff --git a/server/subsonic/middlewares.go b/server/subsonic/middlewares.go index 7698a3c7b..2d8b1fd94 100644 --- a/server/subsonic/middlewares.go +++ b/server/subsonic/middlewares.go @@ -31,9 +31,11 @@ import ( func postFormToQueryParams(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + r.Body = http.MaxBytesReader(w, r.Body, 10<<20) // 10MB err := r.ParseForm() if err != nil { sendError(w, r, newError(responses.ErrorGeneric, err.Error())) + return } var parts []string for key, values := range r.Form { diff --git a/utils/pl/pipelines.go b/utils/pl/pipelines.go index 981b86880..df4ee030c 100644 --- a/utils/pl/pipelines.go +++ b/utils/pl/pipelines.go @@ -29,7 +29,7 @@ func Stage[In any, Out any]( limit := int64(maxWorkers) sem1 := semaphore.NewWeighted(limit) - go func() { + go func() { //nolint:gosec // intentional context.Background() below to wait for workers after ctx cancellation defer close(outputChannel) defer close(errorChannel) @@ -58,7 +58,7 @@ func Stage[In any, Out any]( // By using context.Background() here we are assuming the fn will stop when the context // is canceled. This is required so we can wait for the workers to finish and avoid closing // the outputChannel before they are done. - if err := sem1.Acquire(context.Background(), limit); err != nil { + if err := sem1.Acquire(context.Background(), limit); err != nil { //nolint:gosec // intentional: must wait for workers after ctx cancellation log.Error(ctx, "Failed waiting for workers", err) } }() From e1b34129998114220e9236e5877beb075d2dd51a Mon Sep 17 00:00:00 2001 From: Deluan Date: Sat, 7 Mar 2026 12:00:09 -0500 Subject: [PATCH 48/60] fix(scanner): update gotaglib version to reflect actual dependency version Signed-off-by: Deluan --- adapters/gotaglib/gotaglib.go | 13 ++++++++++++- go.mod | 2 +- go.sum | 4 ++-- 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/adapters/gotaglib/gotaglib.go b/adapters/gotaglib/gotaglib.go index f434d1c71..7b827e880 100644 --- a/adapters/gotaglib/gotaglib.go +++ b/adapters/gotaglib/gotaglib.go @@ -44,7 +44,18 @@ func (e extractor) Parse(files ...string) (map[string]metadata.Info, error) { } func (e extractor) Version() string { - return "2.2 WASM" + bi, ok := debug.ReadBuildInfo() + if ok { + for _, dep := range bi.Deps { + if dep.Path == "go.senan.xyz/taglib" { + if dep.Replace != nil { + return dep.Replace.Version + } + return dep.Version + } + } + } + return "unknown" } func (e extractor) extractMetadata(filePath string) (*metadata.Info, error) { diff --git a/go.mod b/go.mod index abcadab73..538b06f89 100644 --- a/go.mod +++ b/go.mod @@ -3,7 +3,7 @@ module github.com/navidrome/navidrome go 1.25.0 // Fork to implement raw tags support -replace go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 +replace go.senan.xyz/taglib => github.com/deluan/go-taglib v0.0.0-20260307161927-168f6e74ada7 require ( github.com/Masterminds/squirrel v1.5.4 diff --git a/go.sum b/go.sum index e46f1c2e9..8f13089a3 100644 --- a/go.sum +++ b/go.sum @@ -36,8 +36,8 @@ github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 h1:5RVFMOWjMyRy8cARdy79nAmgYw3hK/4HUq48LQ6Wwqo= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40= -github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1 h1:seWJmkPAb+M1ysRNGzTGS7FfdrUe9wQTHhB9p2fxDWg= -github.com/deluan/go-taglib v0.0.0-20260225021432-1699562530f1/go.mod h1:sKDN0U4qXDlq6LFK+aOAkDH4Me5nDV1V/A4B+B69xBA= +github.com/deluan/go-taglib v0.0.0-20260307161927-168f6e74ada7 h1:RpRSTEsAdLHx3Ci0d3M5wtpjcBZiKzhnGfnNAxGXrAE= +github.com/deluan/go-taglib v0.0.0-20260307161927-168f6e74ada7/go.mod h1:sKDN0U4qXDlq6LFK+aOAkDH4Me5nDV1V/A4B+B69xBA= github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf h1:tb246l2Zmpt/GpF9EcHCKTtwzrd0HGfEmoODFA/qnk4= github.com/deluan/rest v0.0.0-20211102003136-6260bc399cbf/go.mod h1:tSgDythFsl0QgS/PFWfIZqcJKnkADWneY80jaVRlqK8= github.com/deluan/sanitize v0.0.0-20241120162836-fdfd8fdfaa55 h1:wSCnggTs2f2ji6nFwQmfwgINcmSMj0xF0oHnoyRSPe4= From ae1e0ddb11a3e18660762f6ec7c096732fa77904 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Sun, 8 Mar 2026 23:57:49 -0400 Subject: [PATCH 49/60] feat(subsonic): implement OpenSubsonic Transcoding extension (#4990) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(subsonic): implement transcode decision logic and codec handling for media files Signed-off-by: Deluan * fix(subsonic): update codec limitation structure and decision logic for improved clarity Signed-off-by: Deluan * fix(transcoding): update bitrate handling to use kilobits per second (kbps) across transcode decision logic Signed-off-by: Deluan * refactor(transcoding): simplify container alias handling in matchesContainer function Signed-off-by: Deluan * fix(transcoding): enforce POST method for GetTranscodeDecision and handle non-POST requests Signed-off-by: Deluan * feat(transcoding): add enums for protocol, comparison operators, limitations, and codec profiles in transcode decision logic Signed-off-by: Deluan * refactor(transcoding): streamline limitation checks and applyLimitation logic for improved readability and maintainability Signed-off-by: Deluan * refactor(transcoding): replace strings.EqualFold with direct comparison for protocol and limitation checks Signed-off-by: Deluan * refactor(transcoding): rename token methods to CreateTranscodeParams and ParseTranscodeParams for clarity Signed-off-by: Deluan * refactor(transcoding): enhance logging for transcode decision process and client info conversion Signed-off-by: Deluan * refactor(transcoding): rename TranscodeDecision to Decider and update related methods for clarity Signed-off-by: Deluan * refactor(transcoding): enhance transcoding config lookup logic for audio codecs Signed-off-by: Deluan * refactor(transcoding): enhance transcoding options with sample rate support and improve command handling Signed-off-by: Deluan * refactor(transcoding): add bit depth support for audio transcoding and enhance related logic Signed-off-by: Deluan * refactor(transcoding): enhance AAC command handling and support for audio channels in streaming Signed-off-by: Deluan * refactor(transcoding): streamline transcoding logic by consolidating stream parameter handling and enhancing alias mapping Signed-off-by: Deluan * refactor(transcoding): update default command handling and add codec support for transcoding Signed-off-by: Deluan * fix: implement noopDecider for transcoding decision handling in tests Signed-off-by: Deluan * fix: address review findings for OpenSubsonic transcoding PR Fix multiple issues identified during code review of the transcoding extension: add missing return after error in shared stream handler preventing nil pointer panic, replace dead r.Body nil check with MaxBytesReader size limit, distinguish not-found from other DB errors, fix bpsToKbps integer truncation with rounding, add "pcm" to isLosslessFormat for consistency with model.IsLossless(), add sampleRate/bitDepth/channels to streaming log, fix outdated test comment, and add tests for conversion functions and GetTranscodeStream parameter passing. * feat(transcoding): add sourceUpdatedAt to decision and validate transcode parameters Signed-off-by: Deluan * fix: small issues Updated mock AAC transcoding command to use the new default (ipod with fragmented MP4) matching the migration, ensuring tests exercise the same buildDynamicArgs code path as production. Improved archiver test mock to match on the whole StreamRequest struct instead of decomposing fields, making it resilient to future field additions. Added named constants for JWT claim keys in the transcode token and wrapped ParseTranscodeParams errors with ErrTokenInvalid for consistency. Documented the IsLossless BitDepth fallback heuristic as temporary until Codec column is populated. Signed-off-by: Deluan * fix(transcoding): adapt transcode claims to struct-based auth.Claims Updated transcode token handling to use the struct-based auth.Claims introduced on master, replacing the previous map[string]any approach. Extended auth.Claims with transcoding-specific fields (MediaID, DirectPlay, UpdatedAt, Channels, SampleRate, BitDepth) and added float64 fallback in ClaimsFromToken for numeric claims that lose their Go type during JWT string serialization. Also added the missing lyrics parameter to all subsonic.New() calls in test files. * feat(model): add ProbeData field and UpdateProbeData repository method Add probe_data TEXT column to media_file for caching ffprobe results. Add UpdateProbeData to MediaFileRepository interface and implementations. Use hash:"ignore" tag so probe data doesn't affect MediaFile fingerprints. * feat(ffmpeg): add ProbeAudioStream for authoritative audio metadata Add ProbeAudioStream to FFmpeg interface, using ffprobe to extract codec, profile, bitrate, sample rate, bit depth, and channels. Parse bits_per_raw_sample as fallback for FLAC/ALAC bit depth. Normalize "unknown" profile to empty string. All parseProbeOutput tests use real ffprobe JSON from actual files. * feat(transcoding): integrate ffprobe into transcode decisions Add ensureProbed to probe media files on first transcode decision, caching results in probe_data. Build SourceStream from probe data with fallback to tag-based metadata. Refactor decision logic to pass StreamDetails instead of MediaFile, enabling codec profile limitations (e.g., audioProfile) to use probe data. Add normalizeProbeCodec to map ffprobe codec names (dsd_lsbf_planar, pcm_s16le) to internal names (dsd, pcm). NewDecider now accepts ffmpeg.FFmpeg; wire_gen.go regenerated. * feat(transcoding): add DevEnableMediaFileProbe config flag Add DevEnableMediaFileProbe (default true) to allow disabling ffprobe- based media file probing as a safety fallback. When disabled, the decider uses tag-based metadata from the scanner instead. * test(transcode): add ensureProbed unit tests Test probing when ProbeData is empty, skipping when already set, error propagation from ffprobe, and DevEnableMediaFileProbe flag. * refactor(ffmpeg): use command constant and select_streams for ProbeAudioStream Move ffprobe arguments to a probeAudioStreamCmd constant, following the same pattern as extractImageCmd and probeCmd. Add -select_streams a:0 to only probe the first audio stream, avoiding unnecessary parsing of video and artwork streams. Derive the ffprobe binary path safely using filepath.Dir/Base instead of replacing within the full path string. * refactor(transcode): decouple transcode token claims from auth.Claims Remove six transcode-specific fields (MediaID, DirectPlay, UpdatedAt, Channels, SampleRate, BitDepth) from auth.Claims, which is shared with session and share tokens. Transcode tokens are signed parameter-passing tokens, not authentication tokens, so coupling them to auth created misleading dependencies. The transcode package now owns its own JWT claim serialization via Decision.toClaimsMap() and paramsFromToken(), using generic auth.EncodeToken/DecodeAndVerifyToken wrappers that keep TokenAuth encapsulated. Wire format (JWT claim keys) is unchanged, so in-flight tokens remain compatible. Signed-off-by: Deluan * refactor(transcode): simplify code after review Extract getIntClaim helper to eliminate repeated int/int64/float64 JWT claim extraction pattern in paramsFromToken and ClaimsFromToken. Rewrite checkIntLimitation as a one-liner delegating to applyIntLimitation. Return probe result from ensureProbed to avoid redundant JSON round-trip. Extract toResponseStreamDetails helper and mediaTypeSong constant in the API layer, and use transcode.ProtocolHTTP constant instead of hardcoded string. Signed-off-by: Deluan * fix(ffmpeg): enhance bit_rate parsing logic for audio streams Signed-off-by: Deluan * fix(transcode): improve code review findings across transcode implementation - Fix parseProbeData to return nil on JSON unmarshal failure instead of a zero-valued struct, preventing silent degradation of source stream details - Use probe-resolved codec for lossless detection in buildSourceStream instead of the potentially stale scanner data - Remove MediaFile.IsLossless() (dead code) and consolidate lossless detection in isLosslessFormat(), using codec name only — bit depth is not reliable since lossy codecs like ADPCM report non-zero values - Add "wavpack" to lossless codec list (ffprobe codec_name for WavPack) - Guard bpsToKbps against negative input values - Fix misleading comment in buildTemplateArgs about conditional injection - Avoid leaking internal error details in Subsonic API responses - Add missing test for ErrNotFound branch in GetTranscodeDecision - Add TODO for hardcoded protocol in toResponseStreamDetails * refactor(transcode): streamline transcoding command lookup and format resolution Signed-off-by: Deluan * feat(transcode): implement server-side transcoding override for player formats Signed-off-by: Deluan * fix(transcode): honor bit depth and channel constraints in transcoding selection selectTranscodingOptions only checked sample rate when deciding whether same-format transcoding was needed, ignoring requested bit depth and channel reductions. This caused the streamer to return raw audio when the transcode decision requested downmix or bit-depth conversion. * refactor(transcode): unify streaming decision engine via MakeDecision Move transcoding decision-making out of mediaStreamer and into the subsonic Stream/Download handlers, using transcode.Decider.MakeDecision as the single decision engine. This eliminates selectTranscodingOptions and the mismatch between decision and streaming code paths (decision used LookupTranscodeCommand with built-in fallbacks, while streaming used FindByFormat which only checked the DB). - Add DecisionOptions with SkipProbe to MakeDecision so the legacy streaming path never calls ffprobe - Add buildLegacyClientInfo to translate legacy stream params (format, maxBitRate, DefaultDownsamplingFormat) into a synthetic ClientInfo - Add resolveStreamRequest on the subsonic Router to resolve legacy params into a fully specified StreamRequest via MakeDecision - Simplify DoStream to a dumb executor that receives pre-resolved params - Remove selectTranscodingOptions entirely Signed-off-by: Deluan * refactor(transcode): move MediaStreamer into core/transcode and unify StreamRequest Moved MediaStreamer, Stream, TranscodingCache and related types from core/media_streamer.go into core/transcode/, eliminating the duplicate StreamRequest type. The transcode.StreamRequest now carries all fields (ID, Format, BitRate, SampleRate, BitDepth, Channels, Offset) and ResolveStream returns a fully-populated value, removing manual field copying at every call site. Also moved buildLegacyClientInfo into the transcode package alongside ResolveStream, and unexported ParseTranscodeParams since it was only used internally by ValidateTranscodeParams. * refactor(transcode): rename Decider methods and unexport Params type Rename ResolveStream → ResolveRequest and ValidateTranscodeParams → ResolveRequestFromToken for clarity and consistency. The new ResolveRequestFromToken returns a StreamRequest directly (instead of the intermediate Params type), eliminating manual Params→StreamRequest conversion in callers. Unexport Params to params since it is now only used internally for JWT token parsing. * test(transcode): remove redundant tests and use constants Remove tests that duplicate coverage from integration-level tests (toClaimsMap, paramsFromToken round-trips, applyServerOverride direct call, duplicate 410 handler test). Replace raw "http" strings with ProtocolHTTP constant. Consolidate lossy -sample_fmt tests into DescribeTable. * refactor(transcode): split oversized files into focused modules Split transcode.go and transcode_test.go into focused files by concern: - decider.go: decision engine (MakeDecision, direct play/transcode evaluation, probe) - token.go: JWT token encode/decode (params, toClaimsMap, paramsFromToken, CreateTranscodeParams, ResolveRequestFromToken) - legacy_client.go: legacy Subsonic bridge (buildLegacyClientInfo, ResolveRequest) - codec_test.go: isLosslessFormat and normalizeProbeCodec tests - token_test.go: token round-trip and ResolveRequestFromToken tests Moved the Decider interface from types.go to decider.go to keep it near its implementation, and cleaned up types.go to contain only pure type definitions and constants. No public API changes. * refactor(transcode): reorder parameters in applyServerOverride function Signed-off-by: Deluan * test(e2e): add NewTestStream function and implement spyStreamer for testing Signed-off-by: Deluan --------- Signed-off-by: Deluan --- adapters/gotaglib/gotaglib.go | 1 + cmd/wire_gen.go | 12 +- conf/configuration.go | 2 + consts/consts.go | 8 +- core/archiver.go | 7 +- core/archiver_test.go | 17 +- core/auth/auth.go | 13 + core/auth/claims.go | 8 +- core/ffmpeg/ffmpeg.go | 276 ++++- core/ffmpeg/ffmpeg_test.go | 498 +++++++- core/media_streamer_Internal_test.go | 162 --- core/transcode/aliases.go | 87 ++ core/transcode/codec.go | 77 ++ core/transcode/codec_test.go | 69 ++ core/transcode/decider.go | 425 +++++++ core/transcode/decider_test.go | 1087 +++++++++++++++++ core/transcode/legacy_client.go | 85 ++ core/transcode/legacy_client_test.go | 84 ++ core/transcode/limitations.go | 171 +++ core/{ => transcode}/media_streamer.go | 144 +-- core/{ => transcode}/media_streamer_test.go | 27 +- core/transcode/token.go | 155 +++ core/transcode/token_test.go | 272 +++++ core/transcode/transcode_suite_test.go | 17 + core/transcode/types.go | 134 ++ core/wire_providers.go | 6 +- ...75815_add_codec_and_update_transcodings.go | 73 ++ model/mediafile.go | 60 + model/mediafile_test.go | 54 +- model/metadata/map_mediafile.go | 1 + model/metadata/metadata.go | 1 + persistence/mediafile_repository.go | 5 + server/e2e/e2e_suite_test.go | 65 +- server/e2e/subsonic_media_retrieval_test.go | 124 ++ server/public/handle_streams.go | 6 +- server/public/public.go | 5 +- server/subsonic/album_lists_test.go | 2 +- server/subsonic/api.go | 65 +- server/subsonic/media_annotation_test.go | 2 +- server/subsonic/media_retrieval_test.go | 2 +- server/subsonic/opensubsonic.go | 1 + server/subsonic/opensubsonic_test.go | 5 +- server/subsonic/playlists_test.go | 4 +- server/subsonic/responses/responses.go | 24 + server/subsonic/searching_test.go | 2 +- server/subsonic/stream.go | 15 +- server/subsonic/transcode.go | 381 ++++++ server/subsonic/transcode_test.go | 406 ++++++ tests/mock_ffmpeg.go | 18 +- tests/mock_mediafile_repo.go | 11 + tests/mock_transcoding_repo.go | 4 + 51 files changed, 4828 insertions(+), 352 deletions(-) delete mode 100644 core/media_streamer_Internal_test.go create mode 100644 core/transcode/aliases.go create mode 100644 core/transcode/codec.go create mode 100644 core/transcode/codec_test.go create mode 100644 core/transcode/decider.go create mode 100644 core/transcode/decider_test.go create mode 100644 core/transcode/legacy_client.go create mode 100644 core/transcode/legacy_client_test.go create mode 100644 core/transcode/limitations.go rename core/{ => transcode}/media_streamer.go (56%) rename core/{ => transcode}/media_streamer_test.go (69%) create mode 100644 core/transcode/token.go create mode 100644 core/transcode/token_test.go create mode 100644 core/transcode/transcode_suite_test.go create mode 100644 core/transcode/types.go create mode 100644 db/migrations/20260307175815_add_codec_and_update_transcodings.go create mode 100644 server/subsonic/transcode.go create mode 100644 server/subsonic/transcode_test.go diff --git a/adapters/gotaglib/gotaglib.go b/adapters/gotaglib/gotaglib.go index 7b827e880..9b71cb462 100644 --- a/adapters/gotaglib/gotaglib.go +++ b/adapters/gotaglib/gotaglib.go @@ -77,6 +77,7 @@ func (e extractor) extractMetadata(filePath string) (*metadata.Info, error) { Channels: int(props.Channels), SampleRate: int(props.SampleRate), BitDepth: int(props.BitsPerSample), + Codec: props.Codec, } // Convert normalized tags to lowercase keys (go-taglib returns UPPERCASE keys) diff --git a/cmd/wire_gen.go b/cmd/wire_gen.go index e8df9a386..a7a0769d3 100644 --- a/cmd/wire_gen.go +++ b/cmd/wire_gen.go @@ -21,6 +21,7 @@ import ( "github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/persistence" @@ -94,8 +95,8 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router { agentsAgents := agents.GetAgents(dataStore, manager) provider := external.NewProvider(dataStore, agentsAgents) artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider) - transcodingCache := core.GetTranscodingCache() - mediaStreamer := core.NewMediaStreamer(dataStore, fFmpeg, transcodingCache) + transcodingCache := transcode.GetTranscodingCache() + mediaStreamer := transcode.NewMediaStreamer(dataStore, fFmpeg, transcodingCache) share := core.NewShare(dataStore) archiver := core.NewArchiver(mediaStreamer, dataStore, share) players := core.NewPlayers(dataStore) @@ -105,7 +106,8 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router { playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager) playbackServer := playback.GetInstance(dataStore) lyricsLyrics := lyrics.NewLyrics(manager) - router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics) + decider := transcode.NewDecider(dataStore, fFmpeg) + router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics, decider) return router } @@ -120,8 +122,8 @@ func CreatePublicRouter() *public.Router { agentsAgents := agents.GetAgents(dataStore, manager) provider := external.NewProvider(dataStore, agentsAgents) artworkArtwork := artwork.NewArtwork(dataStore, fileCache, fFmpeg, provider) - transcodingCache := core.GetTranscodingCache() - mediaStreamer := core.NewMediaStreamer(dataStore, fFmpeg, transcodingCache) + transcodingCache := transcode.GetTranscodingCache() + mediaStreamer := transcode.NewMediaStreamer(dataStore, fFmpeg, transcodingCache) share := core.NewShare(dataStore) archiver := core.NewArchiver(mediaStreamer, dataStore, share) router := public.New(dataStore, artworkArtwork, mediaStreamer, share, archiver) diff --git a/conf/configuration.go b/conf/configuration.go index c46879d42..da549ce26 100644 --- a/conf/configuration.go +++ b/conf/configuration.go @@ -139,6 +139,7 @@ type configOptions struct { DevExternalArtistFetchMultiplier float64 DevOptimizeDB bool DevPreserveUnicodeInExternalCalls bool + DevEnableMediaFileProbe bool } type scannerOptions struct { @@ -763,6 +764,7 @@ func setViperDefaults() { viper.SetDefault("devexternalartistfetchmultiplier", 1.5) viper.SetDefault("devoptimizedb", true) viper.SetDefault("devpreserveunicodeinexternalcalls", false) + viper.SetDefault("devenablemediafileprobe", true) } func init() { diff --git a/consts/consts.go b/consts/consts.go index 295abe8a9..2a5fdd94a 100644 --- a/consts/consts.go +++ b/consts/consts.go @@ -153,7 +153,13 @@ var ( Name: "aac audio", TargetFormat: "aac", DefaultBitRate: 256, - Command: "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a aac -f adts -", + Command: "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a aac -f ipod -movflags frag_keyframe+empty_moov -", + }, + { + Name: "flac audio", + TargetFormat: "flac", + DefaultBitRate: 0, + Command: "ffmpeg -i %s -ss %t -map 0:a:0 -v 0 -c:a flac -f flac -", }, } ) diff --git a/core/archiver.go b/core/archiver.go index 63459816e..88b2d5b0e 100644 --- a/core/archiver.go +++ b/core/archiver.go @@ -10,6 +10,7 @@ import ( "strings" "github.com/Masterminds/squirrel" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/utils/slice" @@ -22,13 +23,13 @@ type Archiver interface { ZipPlaylist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error } -func NewArchiver(ms MediaStreamer, ds model.DataStore, shares Share) Archiver { +func NewArchiver(ms transcode.MediaStreamer, ds model.DataStore, shares Share) Archiver { return &archiver{ds: ds, ms: ms, shares: shares} } type archiver struct { ds model.DataStore - ms MediaStreamer + ms transcode.MediaStreamer shares Share } @@ -176,7 +177,7 @@ func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.Med var r io.ReadCloser if format != "raw" && format != "" { - r, err = a.ms.DoStream(ctx, &mf, format, bitrate, 0) + r, err = a.ms.DoStream(ctx, &mf, transcode.StreamRequest{Format: format, BitRate: bitrate}) } else { r, err = os.Open(path) } diff --git a/core/archiver_test.go b/core/archiver_test.go index 37c4ef9ab..bfce641c9 100644 --- a/core/archiver_test.go +++ b/core/archiver_test.go @@ -9,6 +9,7 @@ import ( "github.com/Masterminds/squirrel" "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/model" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" @@ -44,7 +45,7 @@ var _ = Describe("Archiver", func() { }}).Return(mfs, nil) ds.On("MediaFile", mock.Anything).Return(mfRepo) - ms.On("DoStream", mock.Anything, mock.Anything, "mp3", 128, 0).Return(io.NopCloser(strings.NewReader("test")), nil).Times(3) + ms.On("DoStream", mock.Anything, mock.Anything, transcode.StreamRequest{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(3) out := new(bytes.Buffer) err := arch.ZipAlbum(context.Background(), "1", "mp3", 128, out) @@ -73,7 +74,7 @@ var _ = Describe("Archiver", func() { }}).Return(mfs, nil) ds.On("MediaFile", mock.Anything).Return(mfRepo) - ms.On("DoStream", mock.Anything, mock.Anything, "mp3", 128, 0).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) + ms.On("DoStream", mock.Anything, mock.Anything, transcode.StreamRequest{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) out := new(bytes.Buffer) err := arch.ZipArtist(context.Background(), "1", "mp3", 128, out) @@ -104,7 +105,7 @@ var _ = Describe("Archiver", func() { } sh.On("Load", mock.Anything, "1").Return(share, nil) - ms.On("DoStream", mock.Anything, mock.Anything, "mp3", 128, 0).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) + ms.On("DoStream", mock.Anything, mock.Anything, transcode.StreamRequest{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) out := new(bytes.Buffer) err := arch.ZipShare(context.Background(), "1", out) @@ -136,7 +137,7 @@ var _ = Describe("Archiver", func() { plRepo := &mockPlaylistRepository{} plRepo.On("GetWithTracks", "1", true, false).Return(pls, nil) ds.On("Playlist", mock.Anything).Return(plRepo) - ms.On("DoStream", mock.Anything, mock.Anything, "mp3", 128, 0).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) + ms.On("DoStream", mock.Anything, mock.Anything, transcode.StreamRequest{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) out := new(bytes.Buffer) err := arch.ZipPlaylist(context.Background(), "1", "mp3", 128, out) @@ -214,15 +215,15 @@ func (m *mockPlaylistRepository) GetWithTracks(id string, refreshSmartPlaylists, type mockMediaStreamer struct { mock.Mock - core.MediaStreamer + transcode.MediaStreamer } -func (m *mockMediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, reqFormat string, reqBitRate int, reqOffset int) (*core.Stream, error) { - args := m.Called(ctx, mf, reqFormat, reqBitRate, reqOffset) +func (m *mockMediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, req transcode.StreamRequest) (*transcode.Stream, error) { + args := m.Called(ctx, mf, req) if args.Error(1) != nil { return nil, args.Error(1) } - return &core.Stream{ReadCloser: args.Get(0).(io.ReadCloser)}, nil + return &transcode.Stream{ReadCloser: args.Get(0).(io.ReadCloser)}, nil } type mockShare struct { diff --git a/core/auth/auth.go b/core/auth/auth.go index f7ab3ac1b..a75111b35 100644 --- a/core/auth/auth.go +++ b/core/auth/auth.go @@ -120,6 +120,19 @@ func createNewSecret(ctx context.Context, ds model.DataStore) string { return secret } +// EncodeToken creates a signed JWT from an arbitrary claims map. +// It sets the issuer claim automatically. +func EncodeToken(claims map[string]any) (string, error) { + claims[jwt.IssuerKey] = consts.JWTIssuer + _, token, err := TokenAuth.Encode(claims) + return token, err +} + +// DecodeAndVerifyToken verifies a JWT string and returns the parsed token. +func DecodeAndVerifyToken(tokenStr string) (jwt.Token, error) { + return jwtauth.VerifyToken(TokenAuth, tokenStr) +} + func getEncKey() []byte { key := cmp.Or( conf.Server.PasswordEncryptionKey, diff --git a/core/auth/claims.go b/core/auth/claims.go index ca496ae9a..c0e4dea7f 100644 --- a/core/auth/claims.go +++ b/core/auth/claims.go @@ -86,9 +86,11 @@ func ClaimsFromToken(token jwt.Token) Claims { if err := token.Get("f", &f); err == nil { c.Format = f } - var b int - if err := token.Get("b", &b); err == nil { - c.BitRate = b + if err := token.Get("b", &c.BitRate); err != nil { + var bf float64 + if err := token.Get("b", &bf); err == nil { + c.BitRate = int(bf) + } } return c } diff --git a/core/ffmpeg/ffmpeg.go b/core/ffmpeg/ffmpeg.go index d134077ce..7202d02dd 100644 --- a/core/ffmpeg/ffmpeg.go +++ b/core/ffmpeg/ffmpeg.go @@ -2,23 +2,49 @@ package ffmpeg import ( "context" + "encoding/json" "errors" "fmt" "io" "os" "os/exec" + "path/filepath" "strconv" "strings" "sync" "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/log" ) +// TranscodeOptions contains all parameters for a transcoding operation. +type TranscodeOptions struct { + Command string // DB command template (used to detect custom vs default) + Format string // Target format (mp3, opus, aac, flac) + FilePath string + BitRate int // kbps, 0 = codec default + SampleRate int // 0 = no constraint + Channels int // 0 = no constraint + BitDepth int // 0 = no constraint; valid values: 16, 24, 32 + Offset int // seconds +} + +// AudioProbeResult contains authoritative audio stream properties from ffprobe. +type AudioProbeResult struct { + Codec string `json:"codec"` + Profile string `json:"profile,omitempty"` + BitRate int `json:"bitRate"` + SampleRate int `json:"sampleRate"` + BitDepth int `json:"bitDepth"` + Channels int `json:"channels"` +} + type FFmpeg interface { - Transcode(ctx context.Context, command, path string, maxBitRate, offset int) (io.ReadCloser, error) + Transcode(ctx context.Context, opts TranscodeOptions) (io.ReadCloser, error) ExtractImage(ctx context.Context, path string) (io.ReadCloser, error) Probe(ctx context.Context, files []string) (string, error) + ProbeAudioStream(ctx context.Context, filePath string) (*AudioProbeResult, error) CmdPath() (string, error) IsAvailable() bool Version() string @@ -29,21 +55,26 @@ func New() FFmpeg { } const ( - extractImageCmd = "ffmpeg -i %s -map 0:v -map -0:V -vcodec copy -f image2pipe -" - probeCmd = "ffmpeg %s -f ffmetadata" + extractImageCmd = "ffmpeg -i %s -map 0:v -map -0:V -vcodec copy -f image2pipe -" + probeCmd = "ffmpeg %s -f ffmetadata" + probeAudioStreamCmd = "ffprobe -v quiet -select_streams a:0 -print_format json -show_streams -show_format %s" ) type ffmpeg struct{} -func (e *ffmpeg) Transcode(ctx context.Context, command, path string, maxBitRate, offset int) (io.ReadCloser, error) { +func (e *ffmpeg) Transcode(ctx context.Context, opts TranscodeOptions) (io.ReadCloser, error) { if _, err := ffmpegCmd(); err != nil { return nil, err } - // First make sure the file exists - if err := fileExists(path); err != nil { + if err := fileExists(opts.FilePath); err != nil { return nil, err } - args := createFFmpegCommand(command, path, maxBitRate, offset) + var args []string + if isDefaultCommand(opts.Format, opts.Command) { + args = buildDynamicArgs(opts) + } else { + args = buildTemplateArgs(opts) + } return e.start(ctx, args) } @@ -51,7 +82,6 @@ func (e *ffmpeg) ExtractImage(ctx context.Context, path string) (io.ReadCloser, if _, err := ffmpegCmd(); err != nil { return nil, err } - // First make sure the file exists if err := fileExists(path); err != nil { return nil, err } @@ -81,6 +111,91 @@ func (e *ffmpeg) Probe(ctx context.Context, files []string) (string, error) { return string(output), nil } +func (e *ffmpeg) ProbeAudioStream(ctx context.Context, filePath string) (*AudioProbeResult, error) { + if _, err := ffmpegCmd(); err != nil { + return nil, err + } + if err := fileExists(filePath); err != nil { + return nil, err + } + args := createFFmpegCommand(probeAudioStreamCmd, filePath, 0, 0) + log.Trace(ctx, "Executing ffprobe command", "args", args) + cmd := exec.CommandContext(ctx, args[0], args[1:]...) // #nosec + output, err := cmd.Output() + if err != nil { + return nil, fmt.Errorf("running ffprobe on %q: %w", filePath, err) + } + return parseProbeOutput(output) +} + +type probeOutput struct { + Streams []probeStream `json:"streams"` + Format probeFormat `json:"format"` +} + +type probeFormat struct { + BitRate string `json:"bit_rate"` +} + +type probeStream struct { + CodecName string `json:"codec_name"` + CodecType string `json:"codec_type"` + Profile string `json:"profile"` + SampleRate string `json:"sample_rate"` + BitRate string `json:"bit_rate"` + Channels int `json:"channels"` + BitsPerSample int `json:"bits_per_sample"` + BitsPerRawSample string `json:"bits_per_raw_sample"` +} + +func parseProbeOutput(data []byte) (*AudioProbeResult, error) { + var output probeOutput + if err := json.Unmarshal(data, &output); err != nil { + return nil, fmt.Errorf("parsing ffprobe output: %w", err) + } + + for _, s := range output.Streams { + if s.CodecType != "audio" { + continue + } + bitDepth := s.BitsPerSample + if bitDepth == 0 && s.BitsPerRawSample != "" { + bitDepth, _ = strconv.Atoi(s.BitsPerRawSample) + } + result := &AudioProbeResult{ + Codec: s.CodecName, + Channels: s.Channels, + BitDepth: bitDepth, + } + + // Profile: "unknown" → empty + if s.Profile != "" && !strings.EqualFold(s.Profile, "unknown") { + result.Profile = s.Profile + } + + // Sample rate: string → int + if s.SampleRate != "" { + result.SampleRate, _ = strconv.Atoi(s.SampleRate) + } + + // Bit rate: bps string → kbps int + if s.BitRate != "" { + bps, _ := strconv.Atoi(s.BitRate) + result.BitRate = bps / 1000 + } + + // Fallback to format-level bit_rate (needed for FLAC, Opus, etc.) + if result.BitRate == 0 && output.Format.BitRate != "" { + bps, _ := strconv.Atoi(output.Format.BitRate) + result.BitRate = bps / 1000 + } + + return result, nil + } + + return nil, fmt.Errorf("no audio stream found in ffprobe output") +} + func (e *ffmpeg) CmdPath() (string, error) { return ffmpegCmd() } @@ -156,6 +271,141 @@ func (j *ffCmd) wait() { _ = j.out.Close() } +// formatCodecMap maps target format to ffmpeg codec flag. +var formatCodecMap = map[string]string{ + "mp3": "libmp3lame", + "opus": "libopus", + "aac": "aac", + "flac": "flac", +} + +// formatOutputMap maps target format to ffmpeg output format flag (-f). +var formatOutputMap = map[string]string{ + "mp3": "mp3", + "opus": "opus", + "aac": "ipod", + "flac": "flac", +} + +// defaultCommands is used to detect whether a user has customized their transcoding command. +var defaultCommands = func() map[string]string { + m := make(map[string]string, len(consts.DefaultTranscodings)) + for _, t := range consts.DefaultTranscodings { + m[t.TargetFormat] = t.Command + } + return m +}() + +// isDefaultCommand returns true if the command matches the known default for this format. +func isDefaultCommand(format, command string) bool { + return defaultCommands[format] == command +} + +// buildDynamicArgs programmatically constructs ffmpeg arguments for known formats, +// including all transcoding parameters (bitrate, sample rate, channels). +func buildDynamicArgs(opts TranscodeOptions) []string { + cmdPath, _ := ffmpegCmd() + args := []string{cmdPath, "-i", opts.FilePath} + + if opts.Offset > 0 { + args = append(args, "-ss", strconv.Itoa(opts.Offset)) + } + + args = append(args, "-map", "0:a:0") + + if codec, ok := formatCodecMap[opts.Format]; ok { + args = append(args, "-c:a", codec) + } + + if opts.BitRate > 0 { + args = append(args, "-b:a", strconv.Itoa(opts.BitRate)+"k") + } + if opts.SampleRate > 0 { + args = append(args, "-ar", strconv.Itoa(opts.SampleRate)) + } + if opts.Channels > 0 { + args = append(args, "-ac", strconv.Itoa(opts.Channels)) + } + // Only pass -sample_fmt for lossless output formats where bit depth matters. + // Lossy codecs (mp3, aac, opus) handle sample format conversion internally, + // and passing interleaved formats like "s16" causes silent failures. + if opts.BitDepth >= 16 && isLosslessOutputFormat(opts.Format) { + args = append(args, "-sample_fmt", bitDepthToSampleFmt(opts.BitDepth)) + } + + args = append(args, "-v", "0") + + if outputFmt, ok := formatOutputMap[opts.Format]; ok { + args = append(args, "-f", outputFmt) + } + + // For AAC in MP4 container, enable fragmented MP4 for pipe-safe streaming + if opts.Format == "aac" { + args = append(args, "-movflags", "frag_keyframe+empty_moov") + } + + args = append(args, "-") + return args +} + +// buildTemplateArgs handles user-customized command templates, with dynamic injection +// of sample rate, channels, and bit depth when requested by the transcode decision. +// Note: these flags are injected unconditionally when non-zero, even if the template +// already includes them. FFmpeg uses the last occurrence of duplicate flags. +func buildTemplateArgs(opts TranscodeOptions) []string { + args := createFFmpegCommand(opts.Command, opts.FilePath, opts.BitRate, opts.Offset) + + // Dynamically inject -ar, -ac, and -sample_fmt before the output target + if opts.SampleRate > 0 { + args = injectBeforeOutput(args, "-ar", strconv.Itoa(opts.SampleRate)) + } + if opts.Channels > 0 { + args = injectBeforeOutput(args, "-ac", strconv.Itoa(opts.Channels)) + } + if opts.BitDepth >= 16 && isLosslessOutputFormat(opts.Format) { + args = injectBeforeOutput(args, "-sample_fmt", bitDepthToSampleFmt(opts.BitDepth)) + } + return args +} + +// injectBeforeOutput inserts a flag and value before the trailing "-" (stdout output). +func injectBeforeOutput(args []string, flag, value string) []string { + if len(args) > 0 && args[len(args)-1] == "-" { + result := make([]string, 0, len(args)+2) + result = append(result, args[:len(args)-1]...) + result = append(result, flag, value, "-") + return result + } + return append(args, flag, value) +} + +// isLosslessOutputFormat returns true if the format is a lossless audio format +// where preserving bit depth via -sample_fmt is meaningful. +// Note: this covers only formats ffmpeg can produce as output. For the full set of +// lossless formats used in transcoding decisions, see core/transcode/codec.go:isLosslessFormat. +func isLosslessOutputFormat(format string) bool { + switch strings.ToLower(format) { + case "flac", "alac", "wav", "aiff": + return true + } + return false +} + +// bitDepthToSampleFmt converts a bit depth value to the ffmpeg sample_fmt string. +// FLAC only supports s16 and s32; for 24-bit sources, s32 is the correct format +// (ffmpeg packs 24-bit samples into 32-bit containers). +func bitDepthToSampleFmt(bitDepth int) string { + switch bitDepth { + case 16: + return "s16" + case 32: + return "s32" + default: + // 24-bit and other depths: use s32 (the next valid container size) + return "s32" + } +} + // Path will always be an absolute path func createFFmpegCommand(cmd, path string, maxBitRate, offset int) []string { var args []string @@ -196,10 +446,20 @@ func fixCmd(cmd string) []string { if s == "ffmpeg" || s == "ffmpeg.exe" { split[i] = cmdPath } + if s == "ffprobe" || s == "ffprobe.exe" { + split[i] = ffprobePath(cmdPath) + } } return split } +// ffprobePath derives the ffprobe binary path from the resolved ffmpeg path. +func ffprobePath(ffmpegCmd string) string { + dir := filepath.Dir(ffmpegCmd) + base := filepath.Base(ffmpegCmd) + return filepath.Join(dir, strings.Replace(base, "ffmpeg", "ffprobe", 1)) +} + func ffmpegCmd() (string, error) { ffOnce.Do(func() { if conf.Server.FFmpegPath != "" { diff --git a/core/ffmpeg/ffmpeg_test.go b/core/ffmpeg/ffmpeg_test.go index debe0b51e..eebeefe35 100644 --- a/core/ffmpeg/ffmpeg_test.go +++ b/core/ffmpeg/ffmpeg_test.go @@ -2,19 +2,27 @@ package ffmpeg import ( "context" + "os" + "path/filepath" "runtime" sync "sync" "testing" "time" + "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/log" - "github.com/navidrome/navidrome/tests" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ) func TestFFmpeg(t *testing.T) { - tests.Init(t, false) + // Inline test init to avoid import cycle with tests package + //nolint:dogsled + _, file, _, _ := runtime.Caller(0) + appPath, _ := filepath.Abs(filepath.Join(filepath.Dir(file), "..", "..")) + confPath := filepath.Join(appPath, "tests", "navidrome-test.toml") + _ = os.Chdir(appPath) + conf.LoadFromFile(confPath) log.SetLevel(log.LevelFatal) RegisterFailHandler(Fail) RunSpecs(t, "FFmpeg Suite") @@ -70,6 +78,473 @@ var _ = Describe("ffmpeg", func() { }) }) + Describe("isDefaultCommand", func() { + It("returns true for known default mp3 command", func() { + Expect(isDefaultCommand("mp3", "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -f mp3 -")).To(BeTrue()) + }) + It("returns true for known default opus command", func() { + Expect(isDefaultCommand("opus", "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a libopus -f opus -")).To(BeTrue()) + }) + It("returns true for known default aac command", func() { + Expect(isDefaultCommand("aac", "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a aac -f ipod -movflags frag_keyframe+empty_moov -")).To(BeTrue()) + }) + It("returns true for known default flac command", func() { + Expect(isDefaultCommand("flac", "ffmpeg -i %s -ss %t -map 0:a:0 -v 0 -c:a flac -f flac -")).To(BeTrue()) + }) + It("returns false for a custom command", func() { + Expect(isDefaultCommand("mp3", "ffmpeg -i %s -b:a %bk -custom-flag -f mp3 -")).To(BeFalse()) + }) + It("returns false for unknown format", func() { + Expect(isDefaultCommand("wav", "ffmpeg -i %s -f wav -")).To(BeFalse()) + }) + }) + + Describe("buildDynamicArgs", func() { + It("builds mp3 args with bitrate, samplerate, and channels", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "mp3", + FilePath: "/music/file.flac", + BitRate: 256, + SampleRate: 48000, + Channels: 2, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.flac", + "-map", "0:a:0", + "-c:a", "libmp3lame", + "-b:a", "256k", + "-ar", "48000", + "-ac", "2", + "-v", "0", + "-f", "mp3", + "-", + })) + }) + + It("builds flac args without bitrate", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "flac", + FilePath: "/music/file.dsf", + SampleRate: 48000, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.dsf", + "-map", "0:a:0", + "-c:a", "flac", + "-ar", "48000", + "-v", "0", + "-f", "flac", + "-", + })) + }) + + It("builds opus args with bitrate only", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "opus", + FilePath: "/music/file.flac", + BitRate: 128, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.flac", + "-map", "0:a:0", + "-c:a", "libopus", + "-b:a", "128k", + "-v", "0", + "-f", "opus", + "-", + })) + }) + + It("includes offset when specified", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "mp3", + FilePath: "/music/file.mp3", + BitRate: 192, + Offset: 30, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.mp3", + "-ss", "30", + "-map", "0:a:0", + "-c:a", "libmp3lame", + "-b:a", "192k", + "-v", "0", + "-f", "mp3", + "-", + })) + }) + + It("builds aac args with fragmented MP4 container", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "aac", + FilePath: "/music/file.flac", + BitRate: 256, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.flac", + "-map", "0:a:0", + "-c:a", "aac", + "-b:a", "256k", + "-v", "0", + "-f", "ipod", + "-movflags", "frag_keyframe+empty_moov", + "-", + })) + }) + + It("builds flac args with bit depth", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "flac", + FilePath: "/music/file.dsf", + BitDepth: 24, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.dsf", + "-map", "0:a:0", + "-c:a", "flac", + "-sample_fmt", "s32", + "-v", "0", + "-f", "flac", + "-", + })) + }) + + It("omits -sample_fmt when bit depth is 0", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "flac", + FilePath: "/music/file.flac", + BitDepth: 0, + }) + Expect(args).ToNot(ContainElement("-sample_fmt")) + }) + + It("omits -sample_fmt when bit depth is too low (DSD)", func() { + args := buildDynamicArgs(TranscodeOptions{ + Format: "flac", + FilePath: "/music/file.dsf", + BitDepth: 1, + }) + Expect(args).ToNot(ContainElement("-sample_fmt")) + }) + + DescribeTable("omits -sample_fmt for lossy formats even when bit depth >= 16", + func(format string, bitRate int) { + args := buildDynamicArgs(TranscodeOptions{ + Format: format, + FilePath: "/music/file.flac", + BitRate: bitRate, + BitDepth: 16, + }) + Expect(args).ToNot(ContainElement("-sample_fmt")) + }, + Entry("mp3", "mp3", 256), + Entry("aac", "aac", 256), + Entry("opus", "opus", 128), + ) + }) + + Describe("bitDepthToSampleFmt", func() { + It("converts 16-bit", func() { + Expect(bitDepthToSampleFmt(16)).To(Equal("s16")) + }) + It("converts 24-bit to s32 (FLAC only supports s16/s32)", func() { + Expect(bitDepthToSampleFmt(24)).To(Equal("s32")) + }) + It("converts 32-bit", func() { + Expect(bitDepthToSampleFmt(32)).To(Equal("s32")) + }) + }) + + Describe("buildTemplateArgs", func() { + It("injects -ar and -ac into custom template", func() { + args := buildTemplateArgs(TranscodeOptions{ + Command: "ffmpeg -i %s -b:a %bk -v 0 -f mp3 -", + FilePath: "/music/file.flac", + BitRate: 192, + SampleRate: 44100, + Channels: 2, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.flac", + "-b:a", "192k", "-v", "0", "-f", "mp3", + "-ar", "44100", "-ac", "2", + "-", + })) + }) + + It("injects only -ar when channels is 0", func() { + args := buildTemplateArgs(TranscodeOptions{ + Command: "ffmpeg -i %s -b:a %bk -v 0 -f mp3 -", + FilePath: "/music/file.flac", + BitRate: 192, + SampleRate: 48000, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.flac", + "-b:a", "192k", "-v", "0", "-f", "mp3", + "-ar", "48000", + "-", + })) + }) + + It("does not inject anything when sample rate and channels are 0", func() { + args := buildTemplateArgs(TranscodeOptions{ + Command: "ffmpeg -i %s -b:a %bk -v 0 -f mp3 -", + FilePath: "/music/file.flac", + BitRate: 192, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.flac", + "-b:a", "192k", "-v", "0", "-f", "mp3", + "-", + })) + }) + + It("injects -sample_fmt for lossless output format with bit depth", func() { + args := buildTemplateArgs(TranscodeOptions{ + Command: "ffmpeg -i %s -v 0 -c:a flac -f flac -", + Format: "flac", + FilePath: "/music/file.dsf", + BitDepth: 24, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.dsf", + "-v", "0", "-c:a", "flac", "-f", "flac", + "-sample_fmt", "s32", + "-", + })) + }) + + It("does not inject -sample_fmt for lossy output format even with bit depth", func() { + args := buildTemplateArgs(TranscodeOptions{ + Command: "ffmpeg -i %s -b:a %bk -v 0 -f mp3 -", + Format: "mp3", + FilePath: "/music/file.flac", + BitRate: 192, + BitDepth: 16, + }) + Expect(args).To(Equal([]string{ + "ffmpeg", "-i", "/music/file.flac", + "-b:a", "192k", "-v", "0", "-f", "mp3", + "-", + })) + }) + }) + + Describe("injectBeforeOutput", func() { + It("inserts flag before trailing dash", func() { + args := injectBeforeOutput([]string{"ffmpeg", "-i", "file.mp3", "-f", "mp3", "-"}, "-ar", "48000") + Expect(args).To(Equal([]string{"ffmpeg", "-i", "file.mp3", "-f", "mp3", "-ar", "48000", "-"})) + }) + + It("appends when no trailing dash", func() { + args := injectBeforeOutput([]string{"ffmpeg", "-i", "file.mp3"}, "-ar", "48000") + Expect(args).To(Equal([]string{"ffmpeg", "-i", "file.mp3", "-ar", "48000"})) + }) + }) + + Describe("parseProbeOutput", func() { + It("parses MP3 with embedded artwork (real ffprobe output)", func() { + // Real: MP3 file with mjpeg artwork stream after audio + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"mp3","codec_long_name":"MP3 (MPEG audio layer 3)","codec_type":"audio",` + + `"sample_fmt":"fltp","sample_rate":"44100","channels":2,"channel_layout":"stereo",` + + `"bits_per_sample":0,"bit_rate":"198314","tags":{"encoder":"LAME3.99r"}},` + + `{"index":1,"codec_name":"mjpeg","codec_type":"video","profile":"Baseline","width":400,"height":400}]}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("mp3")) + Expect(result.Profile).To(BeEmpty()) // MP3 has no profile field + Expect(result.SampleRate).To(Equal(44100)) + Expect(result.Channels).To(Equal(2)) + Expect(result.BitRate).To(Equal(198)) // 198314 bps -> 198 kbps + Expect(result.BitDepth).To(Equal(0)) // lossy codec + }) + + It("parses AAC-LC in m4a container (real ffprobe output)", func() { + // Real: AAC LC file with profile and artwork + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"aac","codec_long_name":"AAC (Advanced Audio Coding)",` + + `"profile":"LC","codec_type":"audio","sample_fmt":"fltp","sample_rate":"44100",` + + `"channels":2,"channel_layout":"stereo","bits_per_sample":0,"bit_rate":"279958"},` + + `{"index":1,"codec_name":"mjpeg","codec_type":"video","profile":"Baseline"}]}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("aac")) + Expect(result.Profile).To(Equal("LC")) + Expect(result.SampleRate).To(Equal(44100)) + Expect(result.Channels).To(Equal(2)) + Expect(result.BitRate).To(Equal(279)) // 279958 bps -> 279 kbps + }) + + It("parses HE-AACv2 in mp4 container with video stream (real ffprobe output)", func() { + // Real: Fraunhofer HE-AACv2 sample (LFE-SBRstereo.mp4), video stream before audio + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"h264","codec_type":"video","profile":"Main"},` + + `{"index":1,"codec_name":"aac","codec_long_name":"AAC (Advanced Audio Coding)",` + + `"profile":"HE-AACv2","codec_type":"audio","sample_fmt":"fltp",` + + `"sample_rate":"48000","channels":2,"channel_layout":"stereo",` + + `"bits_per_sample":0,"bit_rate":"55999"}]}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("aac")) + Expect(result.Profile).To(Equal("HE-AACv2")) + Expect(result.SampleRate).To(Equal(48000)) + Expect(result.Channels).To(Equal(2)) + Expect(result.BitRate).To(Equal(55)) // 55999 bps -> 55 kbps + }) + + It("parses FLAC using bits_per_raw_sample and format-level bit_rate (real ffprobe output)", func() { + // Real: FLAC reports bit depth in bits_per_raw_sample, not bits_per_sample. + // Stream-level bit_rate is absent; format-level bit_rate is used as fallback. + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"flac","codec_long_name":"FLAC (Free Lossless Audio Codec)",` + + `"codec_type":"audio","sample_fmt":"s16","sample_rate":"44100","channels":2,` + + `"channel_layout":"stereo","bits_per_sample":0,"bits_per_raw_sample":"16"},` + + `{"index":1,"codec_name":"mjpeg","codec_type":"video","profile":"Baseline"}],` + + `"format":{"bit_rate":"906900"}}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("flac")) + Expect(result.SampleRate).To(Equal(44100)) + Expect(result.BitDepth).To(Equal(16)) // from bits_per_raw_sample + Expect(result.BitRate).To(Equal(906)) // format-level: 906900 bps -> 906 kbps + Expect(result.Profile).To(BeEmpty()) // no profile field in real output + }) + + It("parses Opus with format-level bit_rate fallback (real ffprobe output)", func() { + // Real: Opus stream-level bit_rate is absent; format-level is used as fallback. + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"opus","codec_long_name":"Opus (Opus Interactive Audio Codec)",` + + `"codec_type":"audio","sample_fmt":"fltp","sample_rate":"48000","channels":2,` + + `"channel_layout":"stereo","bits_per_sample":0}],` + + `"format":{"bit_rate":"128000"}}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("opus")) + Expect(result.SampleRate).To(Equal(48000)) + Expect(result.Channels).To(Equal(2)) + Expect(result.BitRate).To(Equal(128)) // format-level: 128000 bps -> 128 kbps + Expect(result.BitDepth).To(Equal(0)) + }) + + It("parses WAV/PCM with bits_per_sample (real ffprobe output)", func() { + // Real: WAV uses bits_per_sample directly + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"pcm_s16le","codec_long_name":"PCM signed 16-bit little-endian",` + + `"codec_type":"audio","sample_fmt":"s16","sample_rate":"44100","channels":2,` + + `"bits_per_sample":16,"bit_rate":"1411200"}]}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("pcm_s16le")) + Expect(result.SampleRate).To(Equal(44100)) + Expect(result.Channels).To(Equal(2)) + Expect(result.BitDepth).To(Equal(16)) + Expect(result.BitRate).To(Equal(1411)) + }) + + It("parses ALAC in m4a container (real ffprobe output)", func() { + // Real: Beatles - You Can't Do That (2023 Mix), ALAC 16-bit + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"alac","codec_long_name":"ALAC (Apple Lossless Audio Codec)",` + + `"codec_type":"audio","sample_fmt":"s16p","sample_rate":"44100","channels":2,` + + `"channel_layout":"stereo","bits_per_sample":0,"bit_rate":"1011003",` + + `"bits_per_raw_sample":"16"},` + + `{"index":1,"codec_name":"mjpeg","codec_type":"video","profile":"Baseline"}]}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("alac")) + Expect(result.BitDepth).To(Equal(16)) // from bits_per_raw_sample + Expect(result.SampleRate).To(Equal(44100)) + Expect(result.Channels).To(Equal(2)) + Expect(result.BitRate).To(Equal(1011)) // 1011003 bps -> 1011 kbps + }) + + It("skips video-only streams", func() { + data := []byte(`{"streams":[{"index":0,"codec_name":"mjpeg","codec_type":"video","profile":"Baseline"}]}`) + _, err := parseProbeOutput(data) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("no audio stream")) + }) + + It("returns error for empty streams array", func() { + data := []byte(`{"streams":[]}`) + _, err := parseProbeOutput(data) + Expect(err).To(HaveOccurred()) + }) + + It("returns error for invalid JSON", func() { + data := []byte(`not json`) + _, err := parseProbeOutput(data) + Expect(err).To(HaveOccurred()) + }) + + It("parses HiRes multichannel FLAC with format-level bit_rate (real ffprobe output)", func() { + // Real: Pink Floyd - 192kHz/24-bit/7.1 surround FLAC + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"flac","codec_long_name":"FLAC (Free Lossless Audio Codec)",` + + `"codec_type":"audio","sample_fmt":"s32","sample_rate":"192000","channels":8,` + + `"channel_layout":"7.1","bits_per_sample":0,"bits_per_raw_sample":"24"},` + + `{"index":1,"codec_name":"mjpeg","codec_type":"video","profile":"Progressive"}],` + + `"format":{"bit_rate":"18432000"}}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("flac")) + Expect(result.SampleRate).To(Equal(192000)) + Expect(result.BitDepth).To(Equal(24)) + Expect(result.Channels).To(Equal(8)) + Expect(result.BitRate).To(Equal(18432)) // format-level: 18432000 bps -> 18432 kbps + }) + + It("parses DSD/DSF file (real ffprobe output)", func() { + // Real: Yes - Owner of a Lonely Heart, DSD64 DSF + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"dsd_lsbf_planar",` + + `"codec_long_name":"DSD (Direct Stream Digital), least significant bit first, planar",` + + `"codec_type":"audio","sample_fmt":"fltp","sample_rate":"352800","channels":2,` + + `"channel_layout":"stereo","bits_per_sample":8,"bit_rate":"5644800"},` + + `{"index":1,"codec_name":"mjpeg","codec_type":"video","profile":"Baseline"}]}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Codec).To(Equal("dsd_lsbf_planar")) + Expect(result.BitDepth).To(Equal(8)) // DSD reports 8 bits_per_sample + Expect(result.SampleRate).To(Equal(352800)) // DSD64 sample rate + Expect(result.Channels).To(Equal(2)) + Expect(result.BitRate).To(Equal(5644)) // 5644800 bps -> 5644 kbps + }) + + It("prefers stream-level bit_rate over format-level when both are present", func() { + // ALAC/DSD: stream has bit_rate, format also has bit_rate — stream wins + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"alac","codec_type":"audio","sample_fmt":"s16p",` + + `"sample_rate":"44100","channels":2,"bits_per_sample":0,` + + `"bit_rate":"1011003","bits_per_raw_sample":"16"}],` + + `"format":{"bit_rate":"1050000"}}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.BitRate).To(Equal(1011)) // stream-level: 1011003 bps -> 1011 kbps (not format's 1050) + }) + + It("returns BitRate 0 when neither stream nor format has bit_rate", func() { + data := []byte(`{"streams":[` + + `{"index":0,"codec_name":"flac","codec_type":"audio","sample_fmt":"s16",` + + `"sample_rate":"44100","channels":2,"bits_per_sample":0,"bits_per_raw_sample":"16"}],` + + `"format":{}}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.BitRate).To(Equal(0)) + }) + + It("clears 'unknown' profile to empty string", func() { + data := []byte(`{"streams":[{"index":0,"codec_name":"flac",` + + `"codec_type":"audio","profile":"unknown","sample_rate":"44100",` + + `"channels":2,"bits_per_sample":0}]}`) + result, err := parseProbeOutput(data) + Expect(err).ToNot(HaveOccurred()) + Expect(result.Profile).To(BeEmpty()) + }) + }) + Describe("FFmpeg", func() { Context("when FFmpeg is available", func() { var ff FFmpeg @@ -93,7 +568,12 @@ var _ = Describe("ffmpeg", func() { command := "ffmpeg -f lavfi -i sine=frequency=1000:duration=0 -f mp3 -" // The input file is not used here, but we need to provide a valid path to the Transcode function - stream, err := ff.Transcode(ctx, command, "tests/fixtures/test.mp3", 128, 0) + stream, err := ff.Transcode(ctx, TranscodeOptions{ + Command: command, + Format: "mp3", + FilePath: "tests/fixtures/test.mp3", + BitRate: 128, + }) Expect(err).ToNot(HaveOccurred()) defer stream.Close() @@ -115,7 +595,12 @@ var _ = Describe("ffmpeg", func() { cancel() // Cancel immediately // This should fail immediately - _, err := ff.Transcode(ctx, "ffmpeg -i %s -f mp3 -", "tests/fixtures/test.mp3", 128, 0) + _, err := ff.Transcode(ctx, TranscodeOptions{ + Command: "ffmpeg -i %s -f mp3 -", + Format: "mp3", + FilePath: "tests/fixtures/test.mp3", + BitRate: 128, + }) Expect(err).To(MatchError(context.Canceled)) }) }) @@ -142,7 +627,10 @@ var _ = Describe("ffmpeg", func() { defer cancel() // Start a process that will run for a while - stream, err := ff.Transcode(ctx, longRunningCmd, "tests/fixtures/test.mp3", 0, 0) + stream, err := ff.Transcode(ctx, TranscodeOptions{ + Command: longRunningCmd, + FilePath: "tests/fixtures/test.mp3", + }) Expect(err).ToNot(HaveOccurred()) defer stream.Close() diff --git a/core/media_streamer_Internal_test.go b/core/media_streamer_Internal_test.go deleted file mode 100644 index 44fbf701c..000000000 --- a/core/media_streamer_Internal_test.go +++ /dev/null @@ -1,162 +0,0 @@ -package core - -import ( - "context" - - "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/log" - "github.com/navidrome/navidrome/model" - "github.com/navidrome/navidrome/model/request" - "github.com/navidrome/navidrome/tests" - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" -) - -var _ = Describe("MediaStreamer", func() { - var ds model.DataStore - ctx := log.NewContext(context.Background()) - - BeforeEach(func() { - ds = &tests.MockDataStore{MockedTranscoding: &tests.MockTranscodingRepo{}} - }) - - Context("selectTranscodingOptions", func() { - mf := &model.MediaFile{} - Context("player is not configured", func() { - It("returns raw if raw is requested", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, _ := selectTranscodingOptions(ctx, ds, mf, "raw", 0) - Expect(format).To(Equal("raw")) - }) - It("returns raw if a transcoder does not exists", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, _ := selectTranscodingOptions(ctx, ds, mf, "m4a", 0) - Expect(format).To(Equal("raw")) - }) - It("returns the requested format if a transcoder exists", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "mp3", 0) - Expect(format).To(Equal("mp3")) - Expect(bitRate).To(Equal(160)) // Default Bit Rate - }) - It("returns raw if requested format is the same as the original and it is not necessary to downsample", func() { - mf.Suffix = "mp3" - mf.BitRate = 112 - format, _ := selectTranscodingOptions(ctx, ds, mf, "mp3", 128) - Expect(format).To(Equal("raw")) - }) - It("returns the requested format if requested BitRate is lower than original", func() { - mf.Suffix = "mp3" - mf.BitRate = 320 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "mp3", 192) - Expect(format).To(Equal("mp3")) - Expect(bitRate).To(Equal(192)) - }) - It("returns raw if requested format is the same as the original, but requested BitRate is 0", func() { - mf.Suffix = "mp3" - mf.BitRate = 320 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "mp3", 0) - Expect(format).To(Equal("raw")) - Expect(bitRate).To(Equal(320)) - }) - Context("Downsampling", func() { - BeforeEach(func() { - conf.Server.DefaultDownsamplingFormat = "opus" - mf.Suffix = "FLAC" - mf.BitRate = 960 - }) - It("returns the DefaultDownsamplingFormat if a maxBitrate is requested but not the format", func() { - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "", 128) - Expect(format).To(Equal("opus")) - Expect(bitRate).To(Equal(128)) - }) - It("returns raw if maxBitrate is equal or greater than original", func() { - // This happens with DSub (and maybe other clients?). See https://github.com/navidrome/navidrome/issues/2066 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "", 960) - Expect(format).To(Equal("raw")) - Expect(bitRate).To(Equal(0)) - }) - }) - }) - - Context("player has format configured", func() { - BeforeEach(func() { - t := model.Transcoding{ID: "oga1", TargetFormat: "oga", DefaultBitRate: 96} - ctx = request.WithTranscoding(ctx, t) - }) - It("returns raw if raw is requested", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, _ := selectTranscodingOptions(ctx, ds, mf, "raw", 0) - Expect(format).To(Equal("raw")) - }) - It("returns configured format/bitrate as default", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "", 0) - Expect(format).To(Equal("oga")) - Expect(bitRate).To(Equal(96)) - }) - It("returns requested format", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "mp3", 0) - Expect(format).To(Equal("mp3")) - Expect(bitRate).To(Equal(160)) // Default Bit Rate - }) - It("returns requested bitrate", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "", 80) - Expect(format).To(Equal("oga")) - Expect(bitRate).To(Equal(80)) - }) - It("returns raw if selected bitrate and format is the same as original", func() { - mf.Suffix = "mp3" - mf.BitRate = 192 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "mp3", 192) - Expect(format).To(Equal("raw")) - Expect(bitRate).To(Equal(0)) - }) - }) - - Context("player has maxBitRate configured", func() { - BeforeEach(func() { - t := model.Transcoding{ID: "oga1", TargetFormat: "oga", DefaultBitRate: 96} - p := model.Player{ID: "player1", TranscodingId: t.ID, MaxBitRate: 192} - ctx = request.WithTranscoding(ctx, t) - ctx = request.WithPlayer(ctx, p) - }) - It("returns raw if raw is requested", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, _ := selectTranscodingOptions(ctx, ds, mf, "raw", 0) - Expect(format).To(Equal("raw")) - }) - It("returns configured format/bitrate as default", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "", 0) - Expect(format).To(Equal("oga")) - Expect(bitRate).To(Equal(192)) - }) - It("returns requested format", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "mp3", 0) - Expect(format).To(Equal("mp3")) - Expect(bitRate).To(Equal(160)) // Default Bit Rate - }) - It("returns requested bitrate", func() { - mf.Suffix = "flac" - mf.BitRate = 1000 - format, bitRate := selectTranscodingOptions(ctx, ds, mf, "", 160) - Expect(format).To(Equal("oga")) - Expect(bitRate).To(Equal(160)) - }) - }) - }) -}) diff --git a/core/transcode/aliases.go b/core/transcode/aliases.go new file mode 100644 index 000000000..67a641511 --- /dev/null +++ b/core/transcode/aliases.go @@ -0,0 +1,87 @@ +package transcode + +import ( + "slices" + "strings" +) + +// containerAliasGroups maps each container alias to a canonical group name. +var containerAliasGroups = func() map[string]string { + groups := [][]string{ + {"aac", "adts", "m4a", "mp4", "m4b", "m4p"}, + {"mpeg", "mp3", "mp2"}, + {"ogg", "oga"}, + {"aif", "aiff"}, + {"asf", "wma"}, + {"mpc", "mpp"}, + {"wv"}, + } + m := make(map[string]string) + for _, g := range groups { + canonical := g[0] + for _, name := range g { + m[name] = canonical + } + } + return m +}() + +// codecAliasGroups maps each codec alias to a canonical group name. +// Codecs within the same group are considered equivalent. +var codecAliasGroups = func() map[string]string { + groups := [][]string{ + {"aac", "adts"}, + {"ac3", "ac-3"}, + {"eac3", "e-ac3", "e-ac-3", "eac-3"}, + {"mpc7", "musepack7"}, + {"mpc8", "musepack8"}, + {"wma1", "wmav1"}, + {"wma2", "wmav2"}, + {"wmalossless", "wma9lossless"}, + {"wmapro", "wma9pro"}, + {"shn", "shorten"}, + {"mp4als", "als"}, + } + m := make(map[string]string) + for _, g := range groups { + for _, name := range g { + m[name] = g[0] // canonical = first entry + } + } + return m +}() + +// matchesWithAliases checks if a value matches any entry in candidates, +// consulting the alias map for equivalent names. +func matchesWithAliases(value string, candidates []string, aliases map[string]string) bool { + value = strings.ToLower(value) + canonical := aliases[value] + for _, c := range candidates { + c = strings.ToLower(c) + if c == value { + return true + } + if canonical != "" && aliases[c] == canonical { + return true + } + } + return false +} + +// matchesContainer checks if a file suffix matches any of the container names, +// including common aliases. +func matchesContainer(suffix string, containers []string) bool { + return matchesWithAliases(suffix, containers, containerAliasGroups) +} + +// matchesCodec checks if a codec matches any of the codec names, +// including common aliases. +func matchesCodec(codec string, codecs []string) bool { + return matchesWithAliases(codec, codecs, codecAliasGroups) +} + +func containsIgnoreCase(slice []string, s string) bool { + return slices.ContainsFunc(slice, func(item string) bool { + return strings.EqualFold(item, s) + }) +} diff --git a/core/transcode/codec.go b/core/transcode/codec.go new file mode 100644 index 000000000..aa276d43f --- /dev/null +++ b/core/transcode/codec.go @@ -0,0 +1,77 @@ +package transcode + +import "strings" + +// normalizeProbeCodec maps ffprobe codec_name values to the simplified internal +// codec names used throughout Navidrome (matching inferCodecFromSuffix output). +// Most ffprobe names match directly; this handles the exceptions. +func normalizeProbeCodec(codec string) string { + c := strings.ToLower(codec) + // DSD variants: dsd_lsbf_planar, dsd_msbf_planar, dsd_lsbf, dsd_msbf + if strings.HasPrefix(c, "dsd") { + return "dsd" + } + // PCM variants: pcm_s16le, pcm_s24le, pcm_s32be, pcm_f32le, etc. + if strings.HasPrefix(c, "pcm_") { + return "pcm" + } + return c +} + +// isLosslessFormat returns true if the format is a known lossless audio codec/format. +// Detection is based on codec name only, not bit depth — some lossy codecs (e.g. ADPCM) +// report non-zero bits_per_sample in ffprobe, so bit depth alone is not a reliable signal. +// +// Note: core/ffmpeg has a separate isLosslessOutputFormat that covers only formats +// ffmpeg can produce as output (a smaller set). +func isLosslessFormat(format string) bool { + switch strings.ToLower(format) { + case "flac", "alac", "wav", "aiff", "ape", "wv", "wavpack", "tta", "tak", "shn", "dsd", "pcm": + return true + } + return false +} + +// normalizeSourceSampleRate adjusts the source sample rate for codecs that store +// it differently than PCM. Currently handles DSD (÷8): +// DSD64=2822400→352800, DSD128=5644800→705600, etc. +// For other codecs, returns the rate unchanged. +func normalizeSourceSampleRate(sampleRate int, codec string) int { + if strings.EqualFold(codec, "dsd") && sampleRate > 0 { + return sampleRate / 8 + } + return sampleRate +} + +// normalizeSourceBitDepth adjusts the source bit depth for codecs that use +// non-standard bit depths. Currently handles DSD (1-bit → 24-bit PCM, which is +// what ffmpeg produces). For other codecs, returns the depth unchanged. +func normalizeSourceBitDepth(bitDepth int, codec string) int { + if strings.EqualFold(codec, "dsd") && bitDepth == 1 { + return 24 + } + return bitDepth +} + +// codecFixedOutputSampleRate returns the mandatory output sample rate for codecs +// that always resample regardless of input (e.g., Opus always outputs 48000Hz). +// Returns 0 if the codec has no fixed output rate. +func codecFixedOutputSampleRate(codec string) int { + switch strings.ToLower(codec) { + case "opus": + return 48000 + } + return 0 +} + +// codecMaxSampleRate returns the hard maximum output sample rate for a codec. +// Returns 0 if the codec has no hard limit. +func codecMaxSampleRate(codec string) int { + switch strings.ToLower(codec) { + case "mp3": + return 48000 + case "aac": + return 96000 + } + return 0 +} diff --git a/core/transcode/codec_test.go b/core/transcode/codec_test.go new file mode 100644 index 000000000..6d3fbd78c --- /dev/null +++ b/core/transcode/codec_test.go @@ -0,0 +1,69 @@ +package transcode + +import ( + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("Codec", func() { + Describe("isLosslessFormat", func() { + It("returns true for known lossless codecs", func() { + Expect(isLosslessFormat("flac")).To(BeTrue()) + Expect(isLosslessFormat("alac")).To(BeTrue()) + Expect(isLosslessFormat("pcm")).To(BeTrue()) + Expect(isLosslessFormat("wav")).To(BeTrue()) + Expect(isLosslessFormat("dsd")).To(BeTrue()) + Expect(isLosslessFormat("ape")).To(BeTrue()) + Expect(isLosslessFormat("wv")).To(BeTrue()) + Expect(isLosslessFormat("wavpack")).To(BeTrue()) // ffprobe codec_name for WavPack + }) + + It("returns false for lossy codecs", func() { + Expect(isLosslessFormat("mp3")).To(BeFalse()) + Expect(isLosslessFormat("aac")).To(BeFalse()) + Expect(isLosslessFormat("opus")).To(BeFalse()) + Expect(isLosslessFormat("vorbis")).To(BeFalse()) + }) + + It("returns false for unknown codecs", func() { + Expect(isLosslessFormat("unknown_codec")).To(BeFalse()) + }) + + It("is case-insensitive", func() { + Expect(isLosslessFormat("FLAC")).To(BeTrue()) + Expect(isLosslessFormat("Alac")).To(BeTrue()) + }) + }) + + Describe("normalizeProbeCodec", func() { + It("passes through common codec names unchanged", func() { + Expect(normalizeProbeCodec("mp3")).To(Equal("mp3")) + Expect(normalizeProbeCodec("aac")).To(Equal("aac")) + Expect(normalizeProbeCodec("flac")).To(Equal("flac")) + Expect(normalizeProbeCodec("opus")).To(Equal("opus")) + Expect(normalizeProbeCodec("vorbis")).To(Equal("vorbis")) + Expect(normalizeProbeCodec("alac")).To(Equal("alac")) + Expect(normalizeProbeCodec("wmav2")).To(Equal("wmav2")) + }) + + It("normalizes DSD variants to dsd", func() { + Expect(normalizeProbeCodec("dsd_lsbf_planar")).To(Equal("dsd")) + Expect(normalizeProbeCodec("dsd_msbf_planar")).To(Equal("dsd")) + Expect(normalizeProbeCodec("dsd_lsbf")).To(Equal("dsd")) + Expect(normalizeProbeCodec("dsd_msbf")).To(Equal("dsd")) + }) + + It("normalizes PCM variants to pcm", func() { + Expect(normalizeProbeCodec("pcm_s16le")).To(Equal("pcm")) + Expect(normalizeProbeCodec("pcm_s24le")).To(Equal("pcm")) + Expect(normalizeProbeCodec("pcm_s32be")).To(Equal("pcm")) + Expect(normalizeProbeCodec("pcm_f32le")).To(Equal("pcm")) + }) + + It("lowercases input", func() { + Expect(normalizeProbeCodec("MP3")).To(Equal("mp3")) + Expect(normalizeProbeCodec("AAC")).To(Equal("aac")) + Expect(normalizeProbeCodec("DSD_LSBF_PLANAR")).To(Equal("dsd")) + }) + }) +}) diff --git a/core/transcode/decider.go b/core/transcode/decider.go new file mode 100644 index 000000000..55b451fd6 --- /dev/null +++ b/core/transcode/decider.go @@ -0,0 +1,425 @@ +package transcode + +import ( + "context" + "encoding/json" + "fmt" + "strings" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/consts" + "github.com/navidrome/navidrome/core/ffmpeg" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" +) + +const defaultBitrate = 256 // kbps + +// Decider is the core service interface for making transcoding decisions +type Decider interface { + MakeDecision(ctx context.Context, mf *model.MediaFile, clientInfo *ClientInfo, opts DecisionOptions) (*Decision, error) + CreateTranscodeParams(decision *Decision) (string, error) + ResolveRequestFromToken(ctx context.Context, token string, mediaID string, offset int) (StreamRequest, *model.MediaFile, error) + ResolveRequest(ctx context.Context, mf *model.MediaFile, reqFormat string, reqBitRate int, offset int) StreamRequest +} + +func NewDecider(ds model.DataStore, ff ffmpeg.FFmpeg) Decider { + return &deciderService{ + ds: ds, + ff: ff, + } +} + +type deciderService struct { + ds model.DataStore + ff ffmpeg.FFmpeg +} + +func (s *deciderService) MakeDecision(ctx context.Context, mf *model.MediaFile, clientInfo *ClientInfo, opts DecisionOptions) (*Decision, error) { + decision := &Decision{ + MediaID: mf.ID, + SourceUpdatedAt: mf.UpdatedAt, + } + + var probe *ffmpeg.AudioProbeResult + if !opts.SkipProbe { + var err error + probe, err = s.ensureProbed(ctx, mf) + if err != nil { + return nil, err + } + } + + // Build source stream details (uses probe data if available) + decision.SourceStream = buildSourceStream(mf, probe) + src := &decision.SourceStream + + // Check for server-side player transcoding override + if trc, ok := request.TranscodingFrom(ctx); ok && trc.TargetFormat != "" { + clientInfo = applyServerOverride(ctx, clientInfo, &trc) + } + + log.Trace(ctx, "Making transcode decision", "mediaID", mf.ID, "container", src.Container, + "codec", src.Codec, "bitrate", src.Bitrate, "channels", src.Channels, + "sampleRate", src.SampleRate, "lossless", src.IsLossless, "client", clientInfo.Name) + + // Check global bitrate constraint first. + if clientInfo.MaxAudioBitrate > 0 && src.Bitrate > clientInfo.MaxAudioBitrate { + log.Trace(ctx, "Global bitrate constraint exceeded, skipping direct play", + "sourceBitrate", src.Bitrate, "maxAudioBitrate", clientInfo.MaxAudioBitrate) + decision.TranscodeReasons = append(decision.TranscodeReasons, "audio bitrate not supported") + // Skip direct play profiles entirely — global constraint fails + } else { + // Try direct play profiles, collecting reasons for each failure + for _, profile := range clientInfo.DirectPlayProfiles { + if reason := s.checkDirectPlayProfile(src, &profile, clientInfo); reason == "" { + decision.CanDirectPlay = true + decision.TranscodeReasons = nil // Clear any previously collected reasons + break + } else { + decision.TranscodeReasons = append(decision.TranscodeReasons, reason) + } + } + } + + // If direct play is possible, we're done + if decision.CanDirectPlay { + log.Debug(ctx, "Transcode decision: direct play", "mediaID", mf.ID, "container", src.Container, "codec", src.Codec) + return decision, nil + } + + // Try transcoding profiles (in order of preference) + for _, profile := range clientInfo.TranscodingProfiles { + if ts, transcodeFormat := s.computeTranscodedStream(ctx, src, &profile, clientInfo); ts != nil { + decision.CanTranscode = true + decision.TargetFormat = transcodeFormat + decision.TargetBitrate = ts.Bitrate + decision.TargetChannels = ts.Channels + decision.TargetSampleRate = ts.SampleRate + decision.TargetBitDepth = ts.BitDepth + decision.TranscodeStream = ts + break + } + } + + if decision.CanTranscode { + log.Debug(ctx, "Transcode decision: transcode", "mediaID", mf.ID, + "targetFormat", decision.TargetFormat, "targetBitrate", decision.TargetBitrate, + "targetChannels", decision.TargetChannels, "reasons", decision.TranscodeReasons) + } + + // If neither direct play nor transcode is possible + if !decision.CanDirectPlay && !decision.CanTranscode { + decision.ErrorReason = "no compatible playback profile found" + log.Warn(ctx, "Transcode decision: no compatible profile", "mediaID", mf.ID, + "container", src.Container, "codec", src.Codec, "reasons", decision.TranscodeReasons) + } + + return decision, nil +} + +func buildSourceStream(mf *model.MediaFile, probe *ffmpeg.AudioProbeResult) StreamDetails { + sd := StreamDetails{ + Container: mf.Suffix, + Duration: mf.Duration, + Size: mf.Size, + } + + // Use pre-parsed probe result, or fall back to parsing stored probe data + if probe == nil { + probe, _ = parseProbeData(mf.ProbeData) + } + + // Use probe data if available for authoritative values + if probe != nil { + sd.Codec = normalizeProbeCodec(probe.Codec) + sd.Profile = probe.Profile + sd.Bitrate = probe.BitRate + sd.SampleRate = probe.SampleRate + sd.BitDepth = probe.BitDepth + sd.Channels = probe.Channels + } else { + sd.Codec = mf.AudioCodec() + sd.Bitrate = mf.BitRate + sd.SampleRate = mf.SampleRate + sd.BitDepth = mf.BitDepth + sd.Channels = mf.Channels + } + sd.IsLossless = isLosslessFormat(sd.Codec) + + return sd +} + +// applyServerOverride replaces the client-provided profiles with synthetic ones +// matching the server-forced transcoding format and bitrate. +func applyServerOverride(ctx context.Context, original *ClientInfo, trc *model.Transcoding) *ClientInfo { + maxBitRate := trc.DefaultBitRate + if player, ok := request.PlayerFrom(ctx); ok && player.MaxBitRate > 0 { + maxBitRate = player.MaxBitRate + } + + log.Debug(ctx, "Applying server-side transcoding override", + "targetFormat", trc.TargetFormat, "maxBitRate", maxBitRate, + "client", original.Name) + + return &ClientInfo{ + Name: original.Name, + Platform: original.Platform, + MaxAudioBitrate: maxBitRate, + MaxTranscodingAudioBitrate: maxBitRate, + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{trc.TargetFormat}, AudioCodecs: []string{trc.TargetFormat}, Protocols: []string{ProtocolHTTP}}, + }, + TranscodingProfiles: []Profile{ + {Container: trc.TargetFormat, AudioCodec: trc.TargetFormat, Protocol: ProtocolHTTP}, + }, + } +} + +func parseProbeData(data string) (*ffmpeg.AudioProbeResult, error) { + if data == "" { + return nil, nil + } + var result ffmpeg.AudioProbeResult + if err := json.Unmarshal([]byte(data), &result); err != nil { + return nil, err + } + return &result, nil +} + +// checkDirectPlayProfile returns "" if the profile matches (direct play OK), +// or a typed reason string if it doesn't match. +func (s *deciderService) checkDirectPlayProfile(src *StreamDetails, profile *DirectPlayProfile, clientInfo *ClientInfo) string { + // Check protocol (only http for now) + if len(profile.Protocols) > 0 && !containsIgnoreCase(profile.Protocols, ProtocolHTTP) { + return "protocol not supported" + } + + // Check container + if len(profile.Containers) > 0 && !matchesContainer(src.Container, profile.Containers) { + return "container not supported" + } + + // Check codec + if len(profile.AudioCodecs) > 0 && !matchesCodec(src.Codec, profile.AudioCodecs) { + return "audio codec not supported" + } + + // Check channels + if profile.MaxAudioChannels > 0 && src.Channels > profile.MaxAudioChannels { + return "audio channels not supported" + } + + // Check codec-specific limitations + for _, codecProfile := range clientInfo.CodecProfiles { + if strings.EqualFold(codecProfile.Type, CodecProfileTypeAudio) && matchesCodec(src.Codec, []string{codecProfile.Name}) { + if reason := checkLimitations(src, codecProfile.Limitations); reason != "" { + return reason + } + } + } + + return "" +} + +// computeTranscodedStream attempts to build a valid transcoded stream for the given profile. +// Returns the stream details and the internal transcoding format (which may differ from the +// response container when a codec fallback occurs, e.g., "mp4"→"aac"). +// Returns nil, "" if the profile cannot produce a valid output. +func (s *deciderService) computeTranscodedStream(ctx context.Context, src *StreamDetails, profile *Profile, clientInfo *ClientInfo) (*StreamDetails, string) { + // Check protocol (only http for now) + if profile.Protocol != "" && !strings.EqualFold(profile.Protocol, ProtocolHTTP) { + log.Trace(ctx, "Skipping transcoding profile: unsupported protocol", "protocol", profile.Protocol) + return nil, "" + } + + responseContainer, targetFormat := resolveTargetFormat(profile) + if targetFormat == "" { + return nil, "" + } + + // Verify we have a transcoding command available (DB custom or built-in default) + if LookupTranscodeCommand(ctx, s.ds, targetFormat) == "" { + log.Trace(ctx, "Skipping transcoding profile: no transcoding command available", "targetFormat", targetFormat) + return nil, "" + } + + targetIsLossless := isLosslessFormat(targetFormat) + + // Reject lossy to lossless conversion + if !src.IsLossless && targetIsLossless { + log.Trace(ctx, "Skipping transcoding profile: lossy to lossless not allowed", "targetFormat", targetFormat) + return nil, "" + } + + ts := &StreamDetails{ + Container: responseContainer, + Codec: strings.ToLower(profile.AudioCodec), + SampleRate: normalizeSourceSampleRate(src.SampleRate, src.Codec), + Channels: src.Channels, + BitDepth: normalizeSourceBitDepth(src.BitDepth, src.Codec), + IsLossless: targetIsLossless, + } + if ts.Codec == "" { + ts.Codec = targetFormat + } + + // Apply codec-intrinsic sample rate adjustments before codec profile limitations + if fixedRate := codecFixedOutputSampleRate(ts.Codec); fixedRate > 0 { + ts.SampleRate = fixedRate + } + if maxRate := codecMaxSampleRate(ts.Codec); maxRate > 0 && ts.SampleRate > maxRate { + ts.SampleRate = maxRate + } + + // Determine target bitrate (all in kbps) + if ok := s.computeBitrate(ctx, src, targetFormat, targetIsLossless, clientInfo, ts); !ok { + return nil, "" + } + + // Apply MaxAudioChannels from the transcoding profile + if profile.MaxAudioChannels > 0 && src.Channels > profile.MaxAudioChannels { + ts.Channels = profile.MaxAudioChannels + } + + // Apply codec profile limitations to the TARGET codec + if ok := s.applyCodecLimitations(ctx, src.Bitrate, targetFormat, targetIsLossless, clientInfo, ts); !ok { + return nil, "" + } + + return ts, targetFormat +} + +// LookupTranscodeCommand returns the ffmpeg command for the given format. +// It checks the DB first (for user-customized commands), then falls back to +// the built-in default command. Returns "" if the format is unknown. +func LookupTranscodeCommand(ctx context.Context, ds model.DataStore, format string) string { + t, err := ds.Transcoding(ctx).FindByFormat(format) + if err == nil && t.Command != "" { + return t.Command + } + // Fall back to built-in defaults + for _, dt := range consts.DefaultTranscodings { + if dt.TargetFormat == format { + return dt.Command + } + } + return "" +} + +// resolveTargetFormat determines the response container and internal target format +// from the profile's Container and AudioCodec fields. When an AudioCodec is specified +// it is preferred as targetFormat (e.g. container "mp4" with audioCodec "aac" → targetFormat "aac"). +func resolveTargetFormat(profile *Profile) (responseContainer, targetFormat string) { + responseContainer = strings.ToLower(profile.Container) + targetFormat = responseContainer + + // Prefer the audioCodec as targetFormat when provided (handles container-to-codec + // mapping like "mp4" → "aac", "ogg" → "opus"). + if profile.AudioCodec != "" { + targetFormat = strings.ToLower(profile.AudioCodec) + } + + // If neither container nor audioCodec is set, we can't resolve a format. + if targetFormat == "" { + return "", "" + } + + // When no container was specified, use the targetFormat as container too. + if responseContainer == "" { + responseContainer = targetFormat + } + + return responseContainer, targetFormat +} + +// computeBitrate determines the target bitrate for the transcoded stream. +// Returns false if the profile should be rejected. +func (s *deciderService) computeBitrate(ctx context.Context, src *StreamDetails, targetFormat string, targetIsLossless bool, clientInfo *ClientInfo, ts *StreamDetails) bool { + if src.IsLossless { + if !targetIsLossless { + if clientInfo.MaxTranscodingAudioBitrate > 0 { + ts.Bitrate = clientInfo.MaxTranscodingAudioBitrate + } else { + ts.Bitrate = defaultBitrate + } + } else { + if clientInfo.MaxAudioBitrate > 0 && src.Bitrate > clientInfo.MaxAudioBitrate { + log.Trace(ctx, "Skipping transcoding profile: lossless target exceeds bitrate limit", + "targetFormat", targetFormat, "sourceBitrate", src.Bitrate, "maxAudioBitrate", clientInfo.MaxAudioBitrate) + return false + } + } + } else { + ts.Bitrate = src.Bitrate + } + + // Apply maxAudioBitrate as final cap + if clientInfo.MaxAudioBitrate > 0 && ts.Bitrate > 0 && ts.Bitrate > clientInfo.MaxAudioBitrate { + ts.Bitrate = clientInfo.MaxAudioBitrate + } + return true +} + +// applyCodecLimitations applies codec profile limitations to the transcoded stream. +// Returns false if the profile should be rejected. +func (s *deciderService) applyCodecLimitations(ctx context.Context, sourceBitrate int, targetFormat string, targetIsLossless bool, clientInfo *ClientInfo, ts *StreamDetails) bool { + targetCodec := ts.Codec + for _, codecProfile := range clientInfo.CodecProfiles { + if !strings.EqualFold(codecProfile.Type, CodecProfileTypeAudio) { + continue + } + if !matchesCodec(targetCodec, []string{codecProfile.Name}) { + continue + } + for _, lim := range codecProfile.Limitations { + result := applyLimitation(sourceBitrate, &lim, ts) + if strings.EqualFold(lim.Name, LimitationAudioBitrate) && targetIsLossless && result == adjustAdjusted { + log.Trace(ctx, "Skipping transcoding profile: cannot adjust bitrate for lossless target", + "targetFormat", targetFormat, "codec", targetCodec, "limitation", lim.Name) + return false + } + if result == adjustCannotFit { + log.Trace(ctx, "Skipping transcoding profile: codec limitation cannot be satisfied", + "targetFormat", targetFormat, "codec", targetCodec, "limitation", lim.Name, + "comparison", lim.Comparison, "values", lim.Values) + return false + } + } + } + return true +} + +// ensureProbed runs ffprobe if probe data is missing, persists it, and returns +// the parsed result. Returns (nil, nil) when probing is skipped or data already exists +// (in which case the caller should parse mf.ProbeData). +func (s *deciderService) ensureProbed(ctx context.Context, mf *model.MediaFile) (*ffmpeg.AudioProbeResult, error) { + if mf.ProbeData != "" { + return nil, nil + } + if !conf.Server.DevEnableMediaFileProbe { + return nil, nil + } + + result, err := s.ff.ProbeAudioStream(ctx, mf.AbsolutePath()) + if err != nil { + return nil, fmt.Errorf("probing media file %s: %w", mf.ID, err) + } + + data, err := json.Marshal(result) + if err != nil { + return nil, fmt.Errorf("marshaling probe result for %s: %w", mf.ID, err) + } + mf.ProbeData = string(data) + + if err := s.ds.MediaFile(ctx).UpdateProbeData(mf.ID, mf.ProbeData); err != nil { + log.Error(ctx, "Failed to persist probe data", "mediaID", mf.ID, err) + // Don't fail the decision — we have the data in memory + } + + log.Debug(ctx, "Probed media file", "mediaID", mf.ID, "codec", result.Codec, + "profile", result.Profile, "bitRate", result.BitRate, + "sampleRate", result.SampleRate, "bitDepth", result.BitDepth, "channels", result.Channels) + return result, nil +} diff --git a/core/transcode/decider_test.go b/core/transcode/decider_test.go new file mode 100644 index 000000000..e5ad2f621 --- /dev/null +++ b/core/transcode/decider_test.go @@ -0,0 +1,1087 @@ +package transcode + +import ( + "context" + "encoding/json" + "fmt" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" + "github.com/navidrome/navidrome/core/auth" + "github.com/navidrome/navidrome/core/ffmpeg" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/model/request" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +// withProbe pre-populates ProbeData on a MediaFile from its own fields, +// so ensureProbed short-circuits and tests don't need mock ffprobe results. +func withProbe(mf *model.MediaFile) *model.MediaFile { + probe := ffmpeg.AudioProbeResult{ + Codec: mf.AudioCodec(), + BitRate: mf.BitRate, + SampleRate: mf.SampleRate, + BitDepth: mf.BitDepth, + Channels: mf.Channels, + } + data, _ := json.Marshal(probe) + mf.ProbeData = string(data) + return mf +} + +var _ = Describe("Decider", func() { + var ( + ds *tests.MockDataStore + ff *tests.MockFFmpeg + svc Decider + ctx context.Context + ) + + BeforeEach(func() { + ctx = GinkgoT().Context() + ds = &tests.MockDataStore{ + MockedProperty: &tests.MockedPropertyRepo{}, + MockedTranscoding: &tests.MockTranscodingRepo{}, + } + ff = tests.NewMockFFmpeg("") + auth.Init(ds) + svc = NewDecider(ds, ff) + }) + + Describe("MakeDecision", func() { + Context("Direct Play", func() { + It("allows direct play when profile matches", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 320, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp3"}, AudioCodecs: []string{"mp3"}, Protocols: []string{ProtocolHTTP}, MaxAudioChannels: 2}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + Expect(decision.CanTranscode).To(BeFalse()) + Expect(decision.TranscodeReasons).To(BeEmpty()) + }) + + It("rejects direct play when container doesn't match", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp3"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.TranscodeReasons).To(ContainElement("container not supported")) + }) + + It("rejects direct play when codec doesn't match", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "m4a", Codec: "ALAC", BitRate: 1000, Channels: 2}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"m4a"}, AudioCodecs: []string{"aac"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.TranscodeReasons).To(ContainElement("audio codec not supported")) + }) + + It("rejects direct play when channels exceed limit", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 6}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}, MaxAudioChannels: 2}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.TranscodeReasons).To(ContainElement("audio channels not supported")) + }) + + It("handles container aliases (aac -> m4a)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "m4a", Codec: "AAC", BitRate: 256, Channels: 2}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"aac"}, AudioCodecs: []string{"aac"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + It("handles container aliases (mp4 -> m4a)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "m4a", Codec: "AAC", BitRate: 256, Channels: 2}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp4"}, AudioCodecs: []string{"aac"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + It("handles codec aliases (adts -> aac)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "m4a", Codec: "AAC", BitRate: 256, Channels: 2}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"m4a"}, AudioCodecs: []string{"adts"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + It("allows when protocol list is empty (any protocol)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, AudioCodecs: []string{"flac"}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + It("allows when both container and codec lists are empty (wildcard)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 128, Channels: 2}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{}, AudioCodecs: []string{}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + }) + + Context("MaxAudioBitrate constraint", func() { + It("revokes direct play when bitrate exceeds maxAudioBitrate", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1500, Channels: 2}) + ci := &ClientInfo{ + MaxAudioBitrate: 500, // kbps + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeReasons).To(ContainElement("audio bitrate not supported")) + }) + }) + + Context("Transcoding", func() { + It("selects transcoding when direct play isn't possible", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 256, // kbps + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp3"}, Protocols: []string{ProtocolHTTP}}, + }, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP, MaxAudioChannels: 2}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + Expect(decision.TargetBitrate).To(Equal(256)) // kbps + Expect(decision.TranscodeReasons).To(ContainElement("container not supported")) + }) + + It("rejects lossy to lossless transcoding", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 320, Channels: 2}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "flac", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeFalse()) + }) + + It("uses default bitrate when client doesn't specify", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, BitDepth: 16}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetBitrate).To(Equal(defaultBitrate)) // 256 kbps + }) + + It("preserves lossy bitrate when under max", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "ogg", BitRate: 192, Channels: 2}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 256, // kbps + TranscodingProfiles: []Profile{ + {Container: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetBitrate).To(Equal(192)) // source bitrate in kbps + }) + + It("rejects format with no transcoding command available", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "wav", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeFalse()) + }) + + It("applies maxAudioBitrate as final cap on transcoded stream", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 320, Channels: 2}) + ci := &ClientInfo{ + MaxAudioBitrate: 96, // kbps + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetBitrate).To(Equal(96)) // capped by maxAudioBitrate + }) + + It("selects first valid transcoding profile in order", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 48000, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp3"}, Protocols: []string{ProtocolHTTP}}, + }, + TranscodingProfiles: []Profile{ + {Container: "opus", AudioCodec: "opus", Protocol: ProtocolHTTP}, + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP, MaxAudioChannels: 2}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("opus")) + }) + }) + + Context("Lossless to lossless transcoding", func() { + It("allows lossless to lossless when samplerate needs downsampling", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "dsf", Codec: "DSD", BitRate: 5644, Channels: 2, SampleRate: 176400, BitDepth: 1}) + ci := &ClientInfo{ + MaxAudioBitrate: 1000, + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + }) + + It("sets IsLossless=true on transcoded stream when target is lossless", func() { + // Transcoding to mp3 (lossy) should result in IsLossless=false. + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 96000, BitDepth: 24}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.IsLossless).To(BeFalse()) // mp3 is lossy + }) + }) + + Context("No compatible profile", func() { + It("returns error when nothing matches", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 6}) + ci := &ClientInfo{} + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.CanTranscode).To(BeFalse()) + Expect(decision.ErrorReason).To(Equal("no compatible playback profile found")) + }) + }) + + Context("Codec limitations on direct play", func() { + It("rejects direct play when codec limitation fails (required)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 512, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp3"}, AudioCodecs: []string{"mp3"}, Protocols: []string{ProtocolHTTP}}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "mp3", + Limitations: []Limitation{ + {Name: LimitationAudioBitrate, Comparison: ComparisonLessThanEqual, Values: []string{"320"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.TranscodeReasons).To(ContainElement("audio bitrate not supported")) + }) + + It("allows direct play when optional limitation fails", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 512, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp3"}, AudioCodecs: []string{"mp3"}, Protocols: []string{ProtocolHTTP}}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "mp3", + Limitations: []Limitation{ + {Name: LimitationAudioBitrate, Comparison: ComparisonLessThanEqual, Values: []string{"320"}, Required: false}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + It("handles Equals comparison with multiple values", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "flac", + Limitations: []Limitation{ + {Name: LimitationAudioChannels, Comparison: ComparisonEquals, Values: []string{"1", "2"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + It("rejects when Equals comparison doesn't match any value", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 6, SampleRate: 44100}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "flac", + Limitations: []Limitation{ + {Name: LimitationAudioChannels, Comparison: ComparisonEquals, Values: []string{"1", "2"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + }) + + It("rejects direct play when audioProfile limitation fails (required)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "m4a", Codec: "AAC", BitRate: 256, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"m4a"}, AudioCodecs: []string{"aac"}, Protocols: []string{ProtocolHTTP}}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "aac", + Limitations: []Limitation{ + {Name: LimitationAudioProfile, Comparison: ComparisonEquals, Values: []string{"LC"}, Required: true}, + }, + }, + }, + } + // Source profile is empty (not yet populated from scanner), so Equals("LC") fails + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.TranscodeReasons).To(ContainElement("audio profile not supported")) + }) + + It("allows direct play when audioProfile limitation is optional", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "m4a", Codec: "AAC", BitRate: 256, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"m4a"}, AudioCodecs: []string{"aac"}, Protocols: []string{ProtocolHTTP}}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "aac", + Limitations: []Limitation{ + {Name: LimitationAudioProfile, Comparison: ComparisonEquals, Values: []string{"LC"}, Required: false}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + It("rejects direct play due to samplerate limitation", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 96000, BitDepth: 24}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "flac", + Limitations: []Limitation{ + {Name: LimitationAudioSamplerate, Comparison: ComparisonLessThanEqual, Values: []string{"48000"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.TranscodeReasons).To(ContainElement("audio samplerate not supported")) + }) + }) + + Context("Codec limitations on transcoded output", func() { + It("applies bitrate limitation to transcoded stream", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 192, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + MaxAudioBitrate: 96, // force transcode + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "mp3", + Limitations: []Limitation{ + {Name: LimitationAudioBitrate, Comparison: ComparisonLessThanEqual, Values: []string{"96"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.Bitrate).To(Equal(96)) + }) + + It("applies channel limitation to transcoded stream", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 6, SampleRate: 48000, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "mp3", + Limitations: []Limitation{ + {Name: LimitationAudioChannels, Comparison: ComparisonLessThanEqual, Values: []string{"2"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.Channels).To(Equal(2)) + }) + + It("applies samplerate limitation to transcoded stream", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 96000, BitDepth: 24}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "mp3", + Limitations: []Limitation{ + {Name: LimitationAudioSamplerate, Comparison: ComparisonLessThanEqual, Values: []string{"48000"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.SampleRate).To(Equal(48000)) + }) + + It("applies bitdepth limitation to transcoded stream", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 96000, BitDepth: 24}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "flac", AudioCodec: "flac", Protocol: ProtocolHTTP}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "flac", + Limitations: []Limitation{ + {Name: LimitationAudioBitdepth, Comparison: ComparisonLessThanEqual, Values: []string{"16"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.BitDepth).To(Equal(16)) + Expect(decision.TargetBitDepth).To(Equal(16)) + }) + + It("preserves source bit depth when no limitation applies", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 24}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "flac", AudioCodec: "flac", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.BitDepth).To(Equal(24)) + Expect(decision.TargetBitDepth).To(Equal(24)) + }) + + It("rejects transcoding profile when GreaterThanEqual cannot be satisfied", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "mp3", + Limitations: []Limitation{ + {Name: LimitationAudioSamplerate, Comparison: ComparisonGreaterThanEqual, Values: []string{"96000"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeFalse()) + }) + }) + + Context("DSD sample rate conversion", func() { + It("converts DSD sample rate to PCM-equivalent in decision", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "dsf", Codec: "DSD", BitRate: 5644, Channels: 2, SampleRate: 2822400, BitDepth: 1}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + // DSD64 2822400 / 8 = 352800, capped by MP3 max of 48000 + Expect(decision.TranscodeStream.SampleRate).To(Equal(48000)) + Expect(decision.TargetSampleRate).To(Equal(48000)) + // DSD 1-bit → 24-bit PCM + Expect(decision.TranscodeStream.BitDepth).To(Equal(24)) + Expect(decision.TargetBitDepth).To(Equal(24)) + }) + + It("converts DSD sample rate for FLAC target without codec limit", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "dsf", Codec: "DSD", BitRate: 5644, Channels: 2, SampleRate: 2822400, BitDepth: 1}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "flac", AudioCodec: "flac", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("flac")) + // DSD64 2822400 / 8 = 352800, FLAC has no hard max + Expect(decision.TranscodeStream.SampleRate).To(Equal(352800)) + Expect(decision.TargetSampleRate).To(Equal(352800)) + // DSD 1-bit → 24-bit PCM + Expect(decision.TranscodeStream.BitDepth).To(Equal(24)) + Expect(decision.TargetBitDepth).To(Equal(24)) + }) + + It("applies codec profile limit to DSD-converted FLAC sample rate", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "dsf", Codec: "DSD", BitRate: 5644, Channels: 2, SampleRate: 2822400, BitDepth: 1}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "flac", AudioCodec: "flac", Protocol: ProtocolHTTP}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "flac", + Limitations: []Limitation{ + {Name: LimitationAudioSamplerate, Comparison: ComparisonLessThanEqual, Values: []string{"48000"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + // DSD64 2822400 / 8 = 352800, capped by codec profile limit of 48000 + Expect(decision.TranscodeStream.SampleRate).To(Equal(48000)) + Expect(decision.TargetSampleRate).To(Equal(48000)) + // DSD 1-bit → 24-bit PCM + Expect(decision.TranscodeStream.BitDepth).To(Equal(24)) + Expect(decision.TargetBitDepth).To(Equal(24)) + }) + + It("applies audioBitdepth limitation to DSD-converted bit depth", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "dsf", Codec: "DSD", BitRate: 5644, Channels: 2, SampleRate: 2822400, BitDepth: 1}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "flac", AudioCodec: "flac", Protocol: ProtocolHTTP}, + }, + CodecProfiles: []CodecProfile{ + { + Type: CodecProfileTypeAudio, + Name: "flac", + Limitations: []Limitation{ + {Name: LimitationAudioBitdepth, Comparison: ComparisonLessThanEqual, Values: []string{"16"}, Required: true}, + }, + }, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + // DSD 1-bit → 24-bit PCM, then capped by codec profile limit to 16-bit + Expect(decision.TranscodeStream.BitDepth).To(Equal(16)) + Expect(decision.TargetBitDepth).To(Equal(16)) + }) + }) + + Context("Probe-based lossless detection", func() { + It("uses probe codec name for lossless detection", func() { + // WavPack files: ffprobe reports codec as "wavpack", suffix is ".wv" + mf := &model.MediaFile{ID: "1", Suffix: "wv", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16} + probe := ffmpeg.AudioProbeResult{ + Codec: "wavpack", BitRate: 1000, SampleRate: 44100, BitDepth: 16, Channels: 2, + } + data, _ := json.Marshal(probe) + mf.ProbeData = string(data) + + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + MaxTranscodingAudioBitrate: 256, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.SourceStream.IsLossless).To(BeTrue()) + Expect(decision.SourceStream.Codec).To(Equal("wavpack")) + // Lossless source transcoding to MP3 should use MaxTranscodingAudioBitrate + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.Bitrate).To(Equal(256)) + }) + + It("detects lossy from probe codec name", func() { + mf := &model.MediaFile{ID: "1", Suffix: "ogg", BitRate: 192, Channels: 2, SampleRate: 48000} + probe := ffmpeg.AudioProbeResult{ + Codec: "vorbis", BitRate: 192, SampleRate: 48000, BitDepth: 0, Channels: 2, + } + data, _ := json.Marshal(probe) + mf.ProbeData = string(data) + + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"ogg"}, AudioCodecs: []string{"vorbis"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.SourceStream.IsLossless).To(BeFalse()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + }) + + Context("Opus fixed sample rate", func() { + It("sets Opus output to 48000Hz regardless of input", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 128, + TranscodingProfiles: []Profile{ + {Container: "opus", AudioCodec: "opus", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("opus")) + // Opus always outputs 48000Hz + Expect(decision.TranscodeStream.SampleRate).To(Equal(48000)) + Expect(decision.TargetSampleRate).To(Equal(48000)) + }) + + It("sets Opus output to 48000Hz even for 96kHz input", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1500, Channels: 2, SampleRate: 96000, BitDepth: 24}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 128, + TranscodingProfiles: []Profile{ + {Container: "opus", AudioCodec: "opus", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.SampleRate).To(Equal(48000)) + }) + }) + + Context("Container vs format separation", func() { + It("preserves mp4 container when falling back to aac format", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 256, + TranscodingProfiles: []Profile{ + {Container: "mp4", AudioCodec: "aac", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + // TargetFormat is the internal format used for transcoding ("aac") + Expect(decision.TargetFormat).To(Equal("aac")) + // Container in the response preserves what the client asked ("mp4") + Expect(decision.TranscodeStream.Container).To(Equal("mp4")) + Expect(decision.TranscodeStream.Codec).To(Equal("aac")) + }) + + It("uses container as format when container matches transcoding config", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 256, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + Expect(decision.TranscodeStream.Container).To(Equal("mp3")) + }) + }) + + Context("MP3 max sample rate", func() { + It("caps sample rate at 48000 for MP3", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1500, Channels: 2, SampleRate: 96000, BitDepth: 24}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.SampleRate).To(Equal(48000)) + }) + + It("preserves sample rate at 44100 for MP3", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TranscodeStream.SampleRate).To(Equal(44100)) + }) + }) + + Context("AAC max sample rate", func() { + It("caps sample rate at 96000 for AAC", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "dsf", Codec: "DSD", BitRate: 5644, Channels: 2, SampleRate: 2822400, BitDepth: 1}) + ci := &ClientInfo{ + MaxTranscodingAudioBitrate: 320, + TranscodingProfiles: []Profile{ + {Container: "aac", AudioCodec: "aac", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + // DSD64 2822400 / 8 = 352800, capped by AAC max of 96000 + Expect(decision.TranscodeStream.SampleRate).To(Equal(96000)) + }) + }) + + Context("Typed transcode reasons from multiple profiles", func() { + It("collects reasons from each failed direct play profile", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "ogg", Codec: "Vorbis", BitRate: 128, Channels: 2, SampleRate: 48000}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + {Containers: []string{"mp3"}, AudioCodecs: []string{"mp3"}, Protocols: []string{ProtocolHTTP}}, + {Containers: []string{"m4a", "mp4"}, AudioCodecs: []string{"aac"}, Protocols: []string{ProtocolHTTP}}, + }, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.TranscodeReasons).To(HaveLen(3)) + Expect(decision.TranscodeReasons[0]).To(Equal("container not supported")) + Expect(decision.TranscodeReasons[1]).To(Equal("container not supported")) + Expect(decision.TranscodeReasons[2]).To(Equal("container not supported")) + }) + }) + + Context("Source stream details", func() { + It("populates source stream correctly with kbps bitrate", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 96000, BitDepth: 24, Duration: 300.5, Size: 50000000}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.SourceStream.Container).To(Equal("flac")) + Expect(decision.SourceStream.Codec).To(Equal("flac")) + Expect(decision.SourceStream.Bitrate).To(Equal(1000)) // kbps + Expect(decision.SourceStream.SampleRate).To(Equal(96000)) + Expect(decision.SourceStream.BitDepth).To(Equal(24)) + Expect(decision.SourceStream.Channels).To(Equal(2)) + }) + }) + + Context("Server-side player transcoding override", func() { + It("forces transcoding when override targets a different format", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + Name: "TestClient", + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + } + // Set server override in context + overrideCtx := request.WithTranscoding(ctx, model.Transcoding{TargetFormat: "mp3", DefaultBitRate: 192}) + overrideCtx = request.WithPlayer(overrideCtx, model.Player{MaxBitRate: 0}) + + decision, err := svc.MakeDecision(overrideCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + Expect(decision.TargetBitrate).To(Equal(192)) + }) + + It("allows direct play when source matches forced format and bitrate is within cap", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 128, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + Name: "TestClient", + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + } + overrideCtx := request.WithTranscoding(ctx, model.Transcoding{TargetFormat: "mp3", DefaultBitRate: 256}) + + decision, err := svc.MakeDecision(overrideCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + Expect(decision.CanTranscode).To(BeFalse()) + }) + + It("transcodes when source bitrate exceeds the forced cap", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 320, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + Name: "TestClient", + } + overrideCtx := request.WithTranscoding(ctx, model.Transcoding{TargetFormat: "mp3", DefaultBitRate: 192}) + + decision, err := svc.MakeDecision(overrideCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + Expect(decision.TargetBitrate).To(Equal(192)) + }) + + It("uses player MaxBitRate over transcoding DefaultBitRate", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + Name: "TestClient", + } + overrideCtx := request.WithTranscoding(ctx, model.Transcoding{TargetFormat: "mp3", DefaultBitRate: 192}) + overrideCtx = request.WithPlayer(overrideCtx, model.Player{MaxBitRate: 320}) + + decision, err := svc.MakeDecision(overrideCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + Expect(decision.TargetBitrate).To(Equal(320)) + }) + + It("applies no bitrate cap when both MaxBitRate and DefaultBitRate are 0", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + Name: "TestClient", + } + overrideCtx := request.WithTranscoding(ctx, model.Transcoding{TargetFormat: "mp3", DefaultBitRate: 0}) + overrideCtx = request.WithPlayer(overrideCtx, model.Player{MaxBitRate: 0}) + + decision, err := svc.MakeDecision(overrideCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetFormat).To(Equal("mp3")) + // With no cap, lossless→lossy uses defaultBitrate (256) + Expect(decision.TargetBitrate).To(Equal(defaultBitrate)) + }) + + It("does not apply override when no transcoding is in context", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + Name: "TestClient", + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac"}, Protocols: []string{ProtocolHTTP}}, + }, + } + // No override in context — client profiles used as-is + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + + }) + }) + + Describe("ensureProbed", func() { + var mockMFRepo *tests.MockMediaFileRepo + + BeforeEach(func() { + mockMFRepo = tests.CreateMockMediaFileRepo() + ds.MockedMediaFile = mockMFRepo + }) + + It("calls ffprobe and populates ProbeData when empty", func() { + mf := &model.MediaFile{ID: "probe-1", Suffix: "mp3", BitRate: 320, Channels: 2} + mockMFRepo.SetData(model.MediaFiles{*mf}) + + ff.ProbeAudioResult = &ffmpeg.AudioProbeResult{ + Codec: "mp3", BitRate: 320, SampleRate: 44100, Channels: 2, + } + + svc := NewDecider(ds, ff).(*deciderService) + probe, err := svc.ensureProbed(ctx, mf) + Expect(err).ToNot(HaveOccurred()) + Expect(mf.ProbeData).ToNot(BeEmpty()) + Expect(probe).ToNot(BeNil()) + Expect(probe.Codec).To(Equal("mp3")) + Expect(probe.BitRate).To(Equal(320)) + Expect(probe.SampleRate).To(Equal(44100)) + Expect(probe.Channels).To(Equal(2)) + + // Verify persisted to DB + stored := mockMFRepo.Data["probe-1"] + Expect(stored.ProbeData).To(Equal(mf.ProbeData)) + }) + + It("skips ffprobe when ProbeData is already set", func() { + mf := withProbe(&model.MediaFile{ID: "probe-2", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2}) + + // Set error on mock — if ffprobe were called, this would fail + ff.Error = fmt.Errorf("should not be called") + + svc := NewDecider(ds, ff).(*deciderService) + probe, err := svc.ensureProbed(ctx, mf) + Expect(err).ToNot(HaveOccurred()) + Expect(probe).To(BeNil()) + }) + + It("returns error when ffprobe fails", func() { + mf := &model.MediaFile{ID: "probe-3", Suffix: "mp3"} + ff.Error = fmt.Errorf("ffprobe not found") + + svc := NewDecider(ds, ff).(*deciderService) + _, err := svc.ensureProbed(ctx, mf) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("probing media file")) + Expect(mf.ProbeData).To(BeEmpty()) + }) + + It("skips ffprobe when DevEnableMediaFileProbe is false", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.DevEnableMediaFileProbe = false + + mf := &model.MediaFile{ID: "probe-4", Suffix: "mp3"} + // Set a result — if ffprobe were called, ProbeData would be populated + ff.ProbeAudioResult = &ffmpeg.AudioProbeResult{Codec: "mp3"} + + svc := NewDecider(ds, ff).(*deciderService) + probe, err := svc.ensureProbed(ctx, mf) + Expect(err).ToNot(HaveOccurred()) + Expect(probe).To(BeNil()) + Expect(mf.ProbeData).To(BeEmpty()) + }) + }) + +}) diff --git a/core/transcode/legacy_client.go b/core/transcode/legacy_client.go new file mode 100644 index 000000000..83190ec92 --- /dev/null +++ b/core/transcode/legacy_client.go @@ -0,0 +1,85 @@ +package transcode + +import ( + "context" + + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" +) + +// buildLegacyClientInfo translates legacy Subsonic stream/download parameters +// into a ClientInfo for use with MakeDecision. +// It does NOT read request.TranscodingFrom(ctx) — that is handled by +// MakeDecision's applyServerOverride. +func buildLegacyClientInfo(mf *model.MediaFile, reqFormat string, reqBitRate int) *ClientInfo { + ci := &ClientInfo{Name: "legacy"} + + // Determine target format for transcoding + var targetFormat string + switch { + case reqFormat != "": + targetFormat = reqFormat + case reqBitRate > 0 && reqBitRate < mf.BitRate && conf.Server.DefaultDownsamplingFormat != "": + targetFormat = conf.Server.DefaultDownsamplingFormat + } + + if targetFormat != "" { + ci.DirectPlayProfiles = []DirectPlayProfile{ + {Containers: []string{mf.Suffix}, AudioCodecs: []string{mf.AudioCodec()}, Protocols: []string{ProtocolHTTP}}, + } + ci.TranscodingProfiles = []Profile{ + {Container: targetFormat, AudioCodec: targetFormat, Protocol: ProtocolHTTP}, + } + if reqBitRate > 0 { + ci.MaxAudioBitrate = reqBitRate + ci.MaxTranscodingAudioBitrate = reqBitRate + } + } else { + // No transcoding requested — direct play everything + ci.DirectPlayProfiles = []DirectPlayProfile{ + {Protocols: []string{ProtocolHTTP}}, + } + } + + return ci +} + +// ResolveRequest uses MakeDecision to resolve legacy Subsonic stream parameters +// into a fully specified StreamRequest. +func (s *deciderService) ResolveRequest(ctx context.Context, mf *model.MediaFile, reqFormat string, reqBitRate int, offset int) StreamRequest { + var req StreamRequest + req.ID = mf.ID + req.Offset = offset + + if reqFormat == "raw" { + req.Format = "raw" + return req + } + + clientInfo := buildLegacyClientInfo(mf, reqFormat, reqBitRate) + decision, err := s.MakeDecision(ctx, mf, clientInfo, DecisionOptions{SkipProbe: true}) + if err != nil { + log.Error(ctx, "Error making transcode decision, falling back to raw", "id", mf.ID, err) + req.Format = "raw" + return req + } + + if decision.CanDirectPlay { + req.Format = "raw" + return req + } + + if decision.CanTranscode { + req.Format = decision.TargetFormat + req.BitRate = decision.TargetBitrate + req.SampleRate = decision.TargetSampleRate + req.BitDepth = decision.TargetBitDepth + req.Channels = decision.TargetChannels + return req + } + + // No compatible profile — fallback to raw + req.Format = "raw" + return req +} diff --git a/core/transcode/legacy_client_test.go b/core/transcode/legacy_client_test.go new file mode 100644 index 000000000..9628764f4 --- /dev/null +++ b/core/transcode/legacy_client_test.go @@ -0,0 +1,84 @@ +package transcode + +import ( + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" + "github.com/navidrome/navidrome/model" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("buildLegacyClientInfo", func() { + var mf *model.MediaFile + + BeforeEach(func() { + mf = &model.MediaFile{Suffix: "flac", BitRate: 960} + }) + + It("sets transcoding profile for explicit format without bitrate", func() { + ci := buildLegacyClientInfo(mf, "mp3", 0) + + Expect(ci.Name).To(Equal("legacy")) + Expect(ci.TranscodingProfiles).To(HaveLen(1)) + Expect(ci.TranscodingProfiles[0].Container).To(Equal("mp3")) + Expect(ci.TranscodingProfiles[0].AudioCodec).To(Equal("mp3")) + Expect(ci.TranscodingProfiles[0].Protocol).To(Equal(ProtocolHTTP)) + Expect(ci.MaxAudioBitrate).To(BeZero()) + Expect(ci.MaxTranscodingAudioBitrate).To(BeZero()) + Expect(ci.DirectPlayProfiles).To(HaveLen(1)) + Expect(ci.DirectPlayProfiles[0].Containers).To(Equal([]string{"flac"})) + Expect(ci.DirectPlayProfiles[0].AudioCodecs).To(Equal([]string{mf.AudioCodec()})) + Expect(ci.DirectPlayProfiles[0].Protocols).To(Equal([]string{ProtocolHTTP})) + }) + + It("sets transcoding profile and bitrate for explicit format with bitrate", func() { + ci := buildLegacyClientInfo(mf, "mp3", 192) + + Expect(ci.TranscodingProfiles).To(HaveLen(1)) + Expect(ci.TranscodingProfiles[0].Container).To(Equal("mp3")) + Expect(ci.TranscodingProfiles[0].AudioCodec).To(Equal("mp3")) + Expect(ci.MaxAudioBitrate).To(Equal(192)) + Expect(ci.MaxTranscodingAudioBitrate).To(Equal(192)) + Expect(ci.DirectPlayProfiles).To(HaveLen(1)) + Expect(ci.DirectPlayProfiles[0].Containers).To(Equal([]string{"flac"})) + }) + + It("returns direct play profile when no format and no bitrate", func() { + ci := buildLegacyClientInfo(mf, "", 0) + + Expect(ci.DirectPlayProfiles).To(HaveLen(1)) + Expect(ci.DirectPlayProfiles[0].Containers).To(BeEmpty()) + Expect(ci.DirectPlayProfiles[0].AudioCodecs).To(BeEmpty()) + Expect(ci.DirectPlayProfiles[0].Protocols).To(Equal([]string{ProtocolHTTP})) + Expect(ci.TranscodingProfiles).To(BeEmpty()) + Expect(ci.MaxAudioBitrate).To(BeZero()) + }) + + It("uses default downsampling format for bitrate-only downsampling", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.DefaultDownsamplingFormat = "opus" + + ci := buildLegacyClientInfo(mf, "", 128) + + Expect(ci.TranscodingProfiles).To(HaveLen(1)) + Expect(ci.TranscodingProfiles[0].Container).To(Equal("opus")) + Expect(ci.TranscodingProfiles[0].AudioCodec).To(Equal("opus")) + Expect(ci.TranscodingProfiles[0].Protocol).To(Equal(ProtocolHTTP)) + Expect(ci.MaxAudioBitrate).To(Equal(128)) + Expect(ci.MaxTranscodingAudioBitrate).To(Equal(128)) + Expect(ci.DirectPlayProfiles).To(HaveLen(1)) + Expect(ci.DirectPlayProfiles[0].Containers).To(Equal([]string{"flac"})) + Expect(ci.DirectPlayProfiles[0].AudioCodecs).To(Equal([]string{mf.AudioCodec()})) + }) + + It("returns direct play when bitrate >= source bitrate", func() { + ci := buildLegacyClientInfo(mf, "", 960) + + Expect(ci.DirectPlayProfiles).To(HaveLen(1)) + Expect(ci.DirectPlayProfiles[0].Containers).To(BeEmpty()) + Expect(ci.DirectPlayProfiles[0].AudioCodecs).To(BeEmpty()) + Expect(ci.DirectPlayProfiles[0].Protocols).To(Equal([]string{ProtocolHTTP})) + Expect(ci.TranscodingProfiles).To(BeEmpty()) + Expect(ci.MaxAudioBitrate).To(BeZero()) + }) +}) diff --git a/core/transcode/limitations.go b/core/transcode/limitations.go new file mode 100644 index 000000000..aefc87d97 --- /dev/null +++ b/core/transcode/limitations.go @@ -0,0 +1,171 @@ +package transcode + +import ( + "strconv" + "strings" +) + +// adjustResult represents the outcome of applying a limitation to a transcoded stream value +type adjustResult int + +const ( + adjustNone adjustResult = iota // Value already satisfies the limitation + adjustAdjusted // Value was changed to fit the limitation + adjustCannotFit // Cannot satisfy the limitation (reject this profile) +) + +// checkLimitations checks codec profile limitations against source stream details. +// Returns "" if all limitations pass, or a typed reason string for the first failure. +func checkLimitations(src *StreamDetails, limitations []Limitation) string { + for _, lim := range limitations { + var ok bool + var reason string + + switch lim.Name { + case LimitationAudioChannels: + ok = checkIntLimitation(src.Channels, lim.Comparison, lim.Values) + reason = "audio channels not supported" + case LimitationAudioSamplerate: + ok = checkIntLimitation(src.SampleRate, lim.Comparison, lim.Values) + reason = "audio samplerate not supported" + case LimitationAudioBitrate: + ok = checkIntLimitation(src.Bitrate, lim.Comparison, lim.Values) + reason = "audio bitrate not supported" + case LimitationAudioBitdepth: + ok = checkIntLimitation(src.BitDepth, lim.Comparison, lim.Values) + reason = "audio bitdepth not supported" + case LimitationAudioProfile: + ok = checkStringLimitation(src.Profile, lim.Comparison, lim.Values) + reason = "audio profile not supported" + default: + continue + } + + if !ok && lim.Required { + return reason + } + } + return "" +} + +// applyLimitation adjusts a transcoded stream parameter to satisfy the limitation. +// Returns the adjustment result. +func applyLimitation(sourceBitrate int, lim *Limitation, ts *StreamDetails) adjustResult { + switch lim.Name { + case LimitationAudioChannels: + return applyIntLimitation(lim.Comparison, lim.Values, ts.Channels, func(v int) { ts.Channels = v }) + case LimitationAudioBitrate: + current := ts.Bitrate + if current == 0 { + current = sourceBitrate + } + return applyIntLimitation(lim.Comparison, lim.Values, current, func(v int) { ts.Bitrate = v }) + case LimitationAudioSamplerate: + return applyIntLimitation(lim.Comparison, lim.Values, ts.SampleRate, func(v int) { ts.SampleRate = v }) + case LimitationAudioBitdepth: + if ts.BitDepth > 0 { + return applyIntLimitation(lim.Comparison, lim.Values, ts.BitDepth, func(v int) { ts.BitDepth = v }) + } + case LimitationAudioProfile: + // TODO: implement when audio profile data is available + } + return adjustNone +} + +// applyIntLimitation applies a limitation comparison to a value. +// If the value needs adjusting, calls the setter and returns the result. +func applyIntLimitation(comparison string, values []string, current int, setter func(int)) adjustResult { + if len(values) == 0 { + return adjustNone + } + + switch comparison { + case ComparisonLessThanEqual: + limit, ok := parseInt(values[0]) + if !ok { + return adjustNone + } + if current <= limit { + return adjustNone + } + setter(limit) + return adjustAdjusted + case ComparisonGreaterThanEqual: + limit, ok := parseInt(values[0]) + if !ok { + return adjustNone + } + if current >= limit { + return adjustNone + } + // Cannot upscale + return adjustCannotFit + case ComparisonEquals: + // Check if current value matches any allowed value + for _, v := range values { + if limit, ok := parseInt(v); ok && current == limit { + return adjustNone + } + } + // Find the closest allowed value below current (don't upscale) + var closest int + found := false + for _, v := range values { + if limit, ok := parseInt(v); ok && limit < current { + if !found || limit > closest { + closest = limit + found = true + } + } + } + if found { + setter(closest) + return adjustAdjusted + } + return adjustCannotFit + case ComparisonNotEquals: + for _, v := range values { + if limit, ok := parseInt(v); ok && current == limit { + return adjustCannotFit + } + } + return adjustNone + } + + return adjustNone +} + +func checkIntLimitation(value int, comparison string, values []string) bool { + return applyIntLimitation(comparison, values, value, func(int) {}) == adjustNone +} + +// checkStringLimitation checks a string value against a limitation. +// Only Equals and NotEquals comparisons are meaningful for strings. +// LessThanEqual/GreaterThanEqual are not applicable and always pass. +func checkStringLimitation(value string, comparison string, values []string) bool { + switch comparison { + case ComparisonEquals: + for _, v := range values { + if strings.EqualFold(value, v) { + return true + } + } + return false + case ComparisonNotEquals: + for _, v := range values { + if strings.EqualFold(value, v) { + return false + } + } + return true + } + return true +} + +func parseInt(s string) (int, bool) { + v, err := strconv.Atoi(s) + if err != nil || v < 0 { + return 0, false + } + return v, true +} diff --git a/core/media_streamer.go b/core/transcode/media_streamer.go similarity index 56% rename from core/media_streamer.go rename to core/transcode/media_streamer.go index c741ed476..88fb61e2d 100644 --- a/core/media_streamer.go +++ b/core/transcode/media_streamer.go @@ -1,4 +1,4 @@ -package core +package transcode import ( "context" @@ -6,6 +6,7 @@ import ( "io" "mime" "os" + "strings" "sync" "time" @@ -19,8 +20,8 @@ import ( ) type MediaStreamer interface { - NewStream(ctx context.Context, id string, reqFormat string, reqBitRate int, offset int) (*Stream, error) - DoStream(ctx context.Context, mf *model.MediaFile, reqFormat string, reqBitRate int, reqOffset int) (*Stream, error) + NewStream(ctx context.Context, req StreamRequest) (*Stream, error) + DoStream(ctx context.Context, mf *model.MediaFile, req StreamRequest) (*Stream, error) } type TranscodingCache cache.FileCache @@ -36,44 +37,53 @@ type mediaStreamer struct { } type streamJob struct { - ms *mediaStreamer - mf *model.MediaFile - filePath string - format string - bitRate int - offset int + ms *mediaStreamer + mf *model.MediaFile + filePath string + format string + bitRate int + sampleRate int + bitDepth int + channels int + offset int } func (j *streamJob) Key() string { - return fmt.Sprintf("%s.%s.%d.%s.%d", j.mf.ID, j.mf.UpdatedAt.Format(time.RFC3339Nano), j.bitRate, j.format, j.offset) + return fmt.Sprintf("%s.%s.%d.%d.%d.%d.%s.%d", j.mf.ID, j.mf.UpdatedAt.Format(time.RFC3339Nano), j.bitRate, j.sampleRate, j.bitDepth, j.channels, j.format, j.offset) } -func (ms *mediaStreamer) NewStream(ctx context.Context, id string, reqFormat string, reqBitRate int, reqOffset int) (*Stream, error) { - mf, err := ms.ds.MediaFile(ctx).Get(id) +func (ms *mediaStreamer) NewStream(ctx context.Context, req StreamRequest) (*Stream, error) { + mf, err := ms.ds.MediaFile(ctx).Get(req.ID) if err != nil { return nil, err } - return ms.DoStream(ctx, mf, reqFormat, reqBitRate, reqOffset) + return ms.DoStream(ctx, mf, req) } -func (ms *mediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, reqFormat string, reqBitRate int, reqOffset int) (*Stream, error) { +func (ms *mediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, req StreamRequest) (*Stream, error) { var format string var bitRate int var cached bool defer func() { log.Info(ctx, "Streaming file", "title", mf.Title, "artist", mf.Artist, "format", format, "cached", cached, - "bitRate", bitRate, "user", userName(ctx), "transcoding", format != "raw", + "bitRate", bitRate, "sampleRate", req.SampleRate, "bitDepth", req.BitDepth, "channels", req.Channels, + "user", userName(ctx), "transcoding", format != "raw", "originalFormat", mf.Suffix, "originalBitRate", mf.BitRate) }() - format, bitRate = selectTranscodingOptions(ctx, ms.ds, mf, reqFormat, reqBitRate) + format = req.Format + bitRate = req.BitRate + if format == "" || format == "raw" { + format = "raw" + bitRate = 0 + } s := &Stream{ctx: ctx, mf: mf, format: format, bitRate: bitRate} filePath := mf.AbsolutePath() if format == "raw" { log.Debug(ctx, "Streaming RAW file", "id", mf.ID, "path", filePath, - "requestBitrate", reqBitRate, "requestFormat", reqFormat, "requestOffset", reqOffset, + "requestBitrate", req.BitRate, "requestFormat", req.Format, "requestOffset", req.Offset, "originalBitrate", mf.BitRate, "originalFormat", mf.Suffix, "selectedBitrate", bitRate, "selectedFormat", format) f, err := os.Open(filePath) @@ -87,12 +97,15 @@ func (ms *mediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, reqF } job := &streamJob{ - ms: ms, - mf: mf, - filePath: filePath, - format: format, - bitRate: bitRate, - offset: reqOffset, + ms: ms, + mf: mf, + filePath: filePath, + format: format, + bitRate: bitRate, + sampleRate: req.SampleRate, + bitDepth: req.BitDepth, + channels: req.Channels, + offset: req.Offset, } r, err := ms.cache.Get(ctx, job) if err != nil { @@ -105,7 +118,7 @@ func (ms *mediaStreamer) DoStream(ctx context.Context, mf *model.MediaFile, reqF s.Seeker = r.Seeker log.Debug(ctx, "Streaming TRANSCODED file", "id", mf.ID, "path", filePath, - "requestBitrate", reqBitRate, "requestFormat", reqFormat, "requestOffset", reqOffset, + "requestBitrate", req.BitRate, "requestFormat", req.Format, "requestOffset", req.Offset, "originalBitrate", mf.BitRate, "originalFormat", mf.Suffix, "selectedBitrate", bitRate, "selectedFormat", format, "cached", cached, "seekable", s.Seekable()) @@ -130,56 +143,15 @@ func (s *Stream) EstimatedContentLength() int { return int(s.mf.Duration * float32(s.bitRate) / 8 * 1024) } -// TODO This function deserves some love (refactoring) -func selectTranscodingOptions(ctx context.Context, ds model.DataStore, mf *model.MediaFile, reqFormat string, reqBitRate int) (format string, bitRate int) { - format = "raw" - if reqFormat == "raw" { - return format, 0 +// NewTestStream creates a Stream for testing purposes. +func NewTestStream(mf *model.MediaFile, format string, bitRate int) *Stream { + return &Stream{ + ctx: context.Background(), + mf: mf, + format: format, + bitRate: bitRate, + ReadCloser: io.NopCloser(strings.NewReader("")), } - if reqFormat == mf.Suffix && reqBitRate == 0 { - bitRate = mf.BitRate - return format, bitRate - } - trc, hasDefault := request.TranscodingFrom(ctx) - var cFormat string - var cBitRate int - if reqFormat != "" { - cFormat = reqFormat - } else { - if hasDefault { - cFormat = trc.TargetFormat - cBitRate = trc.DefaultBitRate - if p, ok := request.PlayerFrom(ctx); ok { - cBitRate = p.MaxBitRate - } - } else if reqBitRate > 0 && reqBitRate < mf.BitRate && conf.Server.DefaultDownsamplingFormat != "" { - // If no format is specified and no transcoding associated to the player, but a bitrate is specified, - // and there is no transcoding set for the player, we use the default downsampling format. - // But only if the requested bitRate is lower than the original bitRate. - log.Debug("Default Downsampling", "Using default downsampling format", conf.Server.DefaultDownsamplingFormat) - cFormat = conf.Server.DefaultDownsamplingFormat - } - } - if reqBitRate > 0 { - cBitRate = reqBitRate - } - if cBitRate == 0 && cFormat == "" { - return format, bitRate - } - t, err := ds.Transcoding(ctx).FindByFormat(cFormat) - if err == nil { - format = t.TargetFormat - if cBitRate != 0 { - bitRate = cBitRate - } else { - bitRate = t.DefaultBitRate - } - } - if format == mf.Suffix && bitRate >= mf.BitRate { - format = "raw" - bitRate = 0 - } - return format, bitRate } var ( @@ -199,9 +171,9 @@ func NewTranscodingCache() TranscodingCache { consts.TranscodingCacheDir, consts.DefaultTranscodingCacheMaxItems, func(ctx context.Context, arg cache.Item) (io.Reader, error) { job := arg.(*streamJob) - t, err := job.ms.ds.Transcoding(ctx).FindByFormat(job.format) - if err != nil { - log.Error(ctx, "Error loading transcoding command", "format", job.format, err) + command := LookupTranscodeCommand(ctx, job.ms.ds, job.format) + if command == "" { + log.Error(ctx, "No transcoding command available", "format", job.format) return nil, os.ErrInvalid } @@ -217,7 +189,16 @@ func NewTranscodingCache() TranscodingCache { transcodingCtx = request.AddValues(context.Background(), ctx) } - out, err := job.ms.transcoder.Transcode(transcodingCtx, t.Command, job.filePath, job.bitRate, job.offset) + out, err := job.ms.transcoder.Transcode(transcodingCtx, ffmpeg.TranscodeOptions{ + Command: command, + Format: job.format, + FilePath: job.filePath, + BitRate: job.bitRate, + SampleRate: job.sampleRate, + BitDepth: job.bitDepth, + Channels: job.channels, + Offset: job.offset, + }) if err != nil { log.Error(ctx, "Error starting transcoder", "id", job.mf.ID, err) return nil, os.ErrInvalid @@ -225,3 +206,12 @@ func NewTranscodingCache() TranscodingCache { return out, nil }) } + +// userName extracts the username from the context for logging purposes. +func userName(ctx context.Context) string { + if user, ok := request.UserFrom(ctx); !ok { + return "UNKNOWN" + } else { + return user.UserName + } +} diff --git a/core/media_streamer_test.go b/core/transcode/media_streamer_test.go similarity index 69% rename from core/media_streamer_test.go rename to core/transcode/media_streamer_test.go index f5175495b..f49dcb8d8 100644 --- a/core/media_streamer_test.go +++ b/core/transcode/media_streamer_test.go @@ -1,4 +1,4 @@ -package core_test +package transcode_test import ( "context" @@ -7,7 +7,7 @@ import ( "github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf/configtest" - "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/tests" @@ -16,7 +16,7 @@ import ( ) var _ = Describe("MediaStreamer", func() { - var streamer core.MediaStreamer + var streamer transcode.MediaStreamer var ds model.DataStore ffmpeg := tests.NewMockFFmpeg("fake data") ctx := log.NewContext(context.TODO()) @@ -29,9 +29,9 @@ var _ = Describe("MediaStreamer", func() { ds.MediaFile(ctx).(*tests.MockMediaFileRepo).SetData(model.MediaFiles{ {ID: "123", Path: "tests/fixtures/test.mp3", Suffix: "mp3", BitRate: 128, Duration: 257.0}, }) - testCache := core.NewTranscodingCache() + testCache := transcode.NewTranscodingCache() Eventually(func() bool { return testCache.Available(context.TODO()) }).Should(BeTrue()) - streamer = core.NewMediaStreamer(ds, ffmpeg, testCache) + streamer = transcode.NewMediaStreamer(ds, ffmpeg, testCache) }) AfterEach(func() { _ = os.RemoveAll(conf.Server.CacheFolder) @@ -39,34 +39,29 @@ var _ = Describe("MediaStreamer", func() { Context("NewStream", func() { It("returns a seekable stream if format is 'raw'", func() { - s, err := streamer.NewStream(ctx, "123", "raw", 0, 0) + s, err := streamer.NewStream(ctx, transcode.StreamRequest{ID: "123", Format: "raw"}) Expect(err).ToNot(HaveOccurred()) Expect(s.Seekable()).To(BeTrue()) }) - It("returns a seekable stream if maxBitRate is 0", func() { - s, err := streamer.NewStream(ctx, "123", "mp3", 0, 0) - Expect(err).ToNot(HaveOccurred()) - Expect(s.Seekable()).To(BeTrue()) - }) - It("returns a seekable stream if maxBitRate is higher than file bitRate", func() { - s, err := streamer.NewStream(ctx, "123", "mp3", 320, 0) + It("returns a seekable stream if no format is specified (direct play)", func() { + s, err := streamer.NewStream(ctx, transcode.StreamRequest{ID: "123"}) Expect(err).ToNot(HaveOccurred()) Expect(s.Seekable()).To(BeTrue()) }) It("returns a NON seekable stream if transcode is required", func() { - s, err := streamer.NewStream(ctx, "123", "mp3", 64, 0) + s, err := streamer.NewStream(ctx, transcode.StreamRequest{ID: "123", Format: "mp3", BitRate: 64}) Expect(err).To(BeNil()) Expect(s.Seekable()).To(BeFalse()) Expect(s.Duration()).To(Equal(float32(257.0))) }) It("returns a seekable stream if the file is complete in the cache", func() { - s, err := streamer.NewStream(ctx, "123", "mp3", 32, 0) + s, err := streamer.NewStream(ctx, transcode.StreamRequest{ID: "123", Format: "mp3", BitRate: 32}) Expect(err).To(BeNil()) _, _ = io.ReadAll(s) _ = s.Close() Eventually(func() bool { return ffmpeg.IsClosed() }, "3s").Should(BeTrue()) - s, err = streamer.NewStream(ctx, "123", "mp3", 32, 0) + s, err = streamer.NewStream(ctx, transcode.StreamRequest{ID: "123", Format: "mp3", BitRate: 32}) Expect(err).To(BeNil()) Expect(s.Seekable()).To(BeTrue()) }) diff --git a/core/transcode/token.go b/core/transcode/token.go new file mode 100644 index 000000000..e110320d0 --- /dev/null +++ b/core/transcode/token.go @@ -0,0 +1,155 @@ +package transcode + +import ( + "context" + "errors" + "fmt" + "time" + + "github.com/lestrrat-go/jwx/v3/jwt" + "github.com/navidrome/navidrome/core/auth" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" +) + +const tokenTTL = 12 * time.Hour + +// params contains the parameters extracted from a transcode token. +// TargetBitrate is in kilobits per second (kbps). +type params struct { + MediaID string + DirectPlay bool + TargetFormat string + TargetBitrate int + TargetChannels int + TargetSampleRate int + TargetBitDepth int + SourceUpdatedAt time.Time +} + +// toClaimsMap converts a Decision into a JWT claims map for token encoding. +// Only non-zero transcode fields are included. +func (d *Decision) toClaimsMap() map[string]any { + m := map[string]any{ + "mid": d.MediaID, + "ua": d.SourceUpdatedAt.Truncate(time.Second).Unix(), + jwt.ExpirationKey: time.Now().Add(tokenTTL).UTC().Unix(), + } + if d.CanDirectPlay { + m["dp"] = true + } + if d.CanTranscode && d.TargetFormat != "" { + m["f"] = d.TargetFormat + if d.TargetBitrate != 0 { + m["b"] = d.TargetBitrate + } + if d.TargetChannels != 0 { + m["ch"] = d.TargetChannels + } + if d.TargetSampleRate != 0 { + m["sr"] = d.TargetSampleRate + } + if d.TargetBitDepth != 0 { + m["bd"] = d.TargetBitDepth + } + } + return m +} + +// paramsFromToken extracts and validates Params from a parsed JWT token. +// Returns an error if required claims (media ID, source timestamp) are missing. +func paramsFromToken(token jwt.Token) (*params, error) { + var p params + var mid string + if err := token.Get("mid", &mid); err == nil { + p.MediaID = mid + } + if p.MediaID == "" { + return nil, fmt.Errorf("%w: missing media ID", ErrTokenInvalid) + } + + var dp bool + if err := token.Get("dp", &dp); err == nil { + p.DirectPlay = dp + } + + ua := getIntClaim(token, "ua") + if ua != 0 { + p.SourceUpdatedAt = time.Unix(int64(ua), 0) + } + if p.SourceUpdatedAt.IsZero() { + return nil, fmt.Errorf("%w: missing source timestamp", ErrTokenInvalid) + } + + var f string + if err := token.Get("f", &f); err == nil { + p.TargetFormat = f + } + p.TargetBitrate = getIntClaim(token, "b") + p.TargetChannels = getIntClaim(token, "ch") + p.TargetSampleRate = getIntClaim(token, "sr") + p.TargetBitDepth = getIntClaim(token, "bd") + return &p, nil +} + +// getIntClaim extracts an int claim from a JWT token, handling the case where +// the value may be stored as int64 or float64 (common in JSON-based JWT libraries). +func getIntClaim(token jwt.Token, key string) int { + var v int + if err := token.Get(key, &v); err == nil { + return v + } + var v64 int64 + if err := token.Get(key, &v64); err == nil { + return int(v64) + } + var f float64 + if err := token.Get(key, &f); err == nil { + return int(f) + } + return 0 +} + +func (s *deciderService) CreateTranscodeParams(decision *Decision) (string, error) { + return auth.EncodeToken(decision.toClaimsMap()) +} + +func (s *deciderService) parseTranscodeParams(tokenStr string) (*params, error) { + token, err := auth.DecodeAndVerifyToken(tokenStr) + if err != nil { + return nil, err + } + return paramsFromToken(token) +} + +func (s *deciderService) ResolveRequestFromToken(ctx context.Context, token string, mediaID string, offset int) (StreamRequest, *model.MediaFile, error) { + p, err := s.parseTranscodeParams(token) + if err != nil { + return StreamRequest{}, nil, errors.Join(ErrTokenInvalid, err) + } + if p.MediaID != mediaID { + return StreamRequest{}, nil, fmt.Errorf("%w: token mediaID %q does not match %q", ErrTokenInvalid, p.MediaID, mediaID) + } + mf, err := s.ds.MediaFile(ctx).Get(mediaID) + if err != nil { + if errors.Is(err, model.ErrNotFound) { + return StreamRequest{}, nil, ErrMediaNotFound + } + return StreamRequest{}, nil, err + } + if !mf.UpdatedAt.Truncate(time.Second).Equal(p.SourceUpdatedAt) { + log.Info(ctx, "Transcode token is stale", "mediaID", mediaID, + "tokenUpdatedAt", p.SourceUpdatedAt, "fileUpdatedAt", mf.UpdatedAt) + return StreamRequest{}, nil, ErrTokenStale + } + + req := StreamRequest{ID: mediaID, Offset: offset} + if !p.DirectPlay && p.TargetFormat != "" { + req.Format = p.TargetFormat + req.BitRate = p.TargetBitrate + req.SampleRate = p.TargetSampleRate + req.BitDepth = p.TargetBitDepth + req.Channels = p.TargetChannels + } + return req, mf, nil +} diff --git a/core/transcode/token_test.go b/core/transcode/token_test.go new file mode 100644 index 000000000..b9b74c8fc --- /dev/null +++ b/core/transcode/token_test.go @@ -0,0 +1,272 @@ +package transcode + +import ( + "context" + "time" + + "github.com/go-chi/jwtauth/v5" + "github.com/navidrome/navidrome/core/auth" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("Token", func() { + var ( + ds *tests.MockDataStore + ff *tests.MockFFmpeg + svc Decider + ctx context.Context + ) + + BeforeEach(func() { + ctx = GinkgoT().Context() + ds = &tests.MockDataStore{ + MockedProperty: &tests.MockedPropertyRepo{}, + MockedTranscoding: &tests.MockTranscodingRepo{}, + } + ff = tests.NewMockFFmpeg("") + auth.Init(ds) + svc = NewDecider(ds, ff) + }) + + Describe("Token round-trip", func() { + var ( + sourceTime time.Time + impl *deciderService + ) + + BeforeEach(func() { + sourceTime = time.Date(2025, 6, 15, 10, 30, 0, 0, time.UTC) + impl = svc.(*deciderService) + }) + + It("creates and parses a direct play token", func() { + decision := &Decision{ + MediaID: "media-123", + CanDirectPlay: true, + SourceUpdatedAt: sourceTime, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + Expect(token).ToNot(BeEmpty()) + + params, err := impl.parseTranscodeParams(token) + Expect(err).ToNot(HaveOccurred()) + Expect(params.MediaID).To(Equal("media-123")) + Expect(params.DirectPlay).To(BeTrue()) + Expect(params.TargetFormat).To(BeEmpty()) + Expect(params.SourceUpdatedAt.Unix()).To(Equal(sourceTime.Unix())) + }) + + It("creates and parses a transcode token with kbps bitrate", func() { + decision := &Decision{ + MediaID: "media-456", + CanDirectPlay: false, + CanTranscode: true, + TargetFormat: "mp3", + TargetBitrate: 256, // kbps + TargetChannels: 2, + SourceUpdatedAt: sourceTime, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + + params, err := impl.parseTranscodeParams(token) + Expect(err).ToNot(HaveOccurred()) + Expect(params.MediaID).To(Equal("media-456")) + Expect(params.DirectPlay).To(BeFalse()) + Expect(params.TargetFormat).To(Equal("mp3")) + Expect(params.TargetBitrate).To(Equal(256)) // kbps + Expect(params.TargetChannels).To(Equal(2)) + Expect(params.SourceUpdatedAt.Unix()).To(Equal(sourceTime.Unix())) + }) + + It("creates and parses a transcode token with sample rate", func() { + decision := &Decision{ + MediaID: "media-789", + CanDirectPlay: false, + CanTranscode: true, + TargetFormat: "flac", + TargetBitrate: 0, + TargetChannels: 2, + TargetSampleRate: 48000, + SourceUpdatedAt: sourceTime, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + + params, err := impl.parseTranscodeParams(token) + Expect(err).ToNot(HaveOccurred()) + Expect(params.MediaID).To(Equal("media-789")) + Expect(params.DirectPlay).To(BeFalse()) + Expect(params.TargetFormat).To(Equal("flac")) + Expect(params.TargetSampleRate).To(Equal(48000)) + Expect(params.TargetChannels).To(Equal(2)) + }) + + It("creates and parses a transcode token with bit depth", func() { + decision := &Decision{ + MediaID: "media-bd", + CanDirectPlay: false, + CanTranscode: true, + TargetFormat: "flac", + TargetBitrate: 0, + TargetChannels: 2, + TargetBitDepth: 24, + SourceUpdatedAt: sourceTime, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + + params, err := impl.parseTranscodeParams(token) + Expect(err).ToNot(HaveOccurred()) + Expect(params.MediaID).To(Equal("media-bd")) + Expect(params.TargetBitDepth).To(Equal(24)) + }) + + It("omits bit depth from token when 0", func() { + decision := &Decision{ + MediaID: "media-nobd", + CanDirectPlay: false, + CanTranscode: true, + TargetFormat: "mp3", + TargetBitrate: 256, + TargetBitDepth: 0, + SourceUpdatedAt: sourceTime, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + + params, err := impl.parseTranscodeParams(token) + Expect(err).ToNot(HaveOccurred()) + Expect(params.TargetBitDepth).To(Equal(0)) + }) + + It("omits sample rate from token when 0", func() { + decision := &Decision{ + MediaID: "media-100", + CanDirectPlay: false, + CanTranscode: true, + TargetFormat: "mp3", + TargetBitrate: 256, + TargetSampleRate: 0, + SourceUpdatedAt: sourceTime, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + + params, err := impl.parseTranscodeParams(token) + Expect(err).ToNot(HaveOccurred()) + Expect(params.TargetSampleRate).To(Equal(0)) + }) + + It("truncates SourceUpdatedAt to seconds", func() { + timeWithNanos := time.Date(2025, 6, 15, 10, 30, 0, 123456789, time.UTC) + decision := &Decision{ + MediaID: "media-trunc", + CanDirectPlay: true, + SourceUpdatedAt: timeWithNanos, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + + params, err := impl.parseTranscodeParams(token) + Expect(err).ToNot(HaveOccurred()) + Expect(params.SourceUpdatedAt.Unix()).To(Equal(timeWithNanos.Truncate(time.Second).Unix())) + }) + + It("rejects an invalid token", func() { + _, err := impl.parseTranscodeParams("invalid-token") + Expect(err).To(HaveOccurred()) + }) + }) + + Describe("ResolveRequestFromToken", func() { + var ( + mockMFRepo *tests.MockMediaFileRepo + sourceTime time.Time + ) + + BeforeEach(func() { + sourceTime = time.Date(2025, 6, 15, 10, 30, 0, 0, time.UTC) + mockMFRepo = &tests.MockMediaFileRepo{} + ds.MockedMediaFile = mockMFRepo + }) + + createTokenForMedia := func(mediaID string, updatedAt time.Time) string { + decision := &Decision{ + MediaID: mediaID, + CanDirectPlay: true, + SourceUpdatedAt: updatedAt, + } + token, err := svc.CreateTranscodeParams(decision) + Expect(err).ToNot(HaveOccurred()) + return token + } + + It("returns stream request and media file for valid token", func() { + mockMFRepo.SetData(model.MediaFiles{ + {ID: "song-1", UpdatedAt: sourceTime}, + }) + token := createTokenForMedia("song-1", sourceTime) + + req, mf, err := svc.ResolveRequestFromToken(ctx, token, "song-1", 0) + Expect(err).ToNot(HaveOccurred()) + Expect(req.ID).To(Equal("song-1")) + Expect(req.Format).To(BeEmpty()) // direct play has no target format + Expect(mf.ID).To(Equal("song-1")) + }) + + It("returns ErrTokenInvalid for invalid token", func() { + _, _, err := svc.ResolveRequestFromToken(ctx, "bad-token", "song-1", 0) + Expect(err).To(MatchError(ContainSubstring(ErrTokenInvalid.Error()))) + }) + + It("returns ErrTokenInvalid when mediaID does not match token", func() { + token := createTokenForMedia("song-1", sourceTime) + + _, _, err := svc.ResolveRequestFromToken(ctx, token, "song-2", 0) + Expect(err).To(MatchError(ContainSubstring(ErrTokenInvalid.Error()))) + }) + + It("returns ErrMediaNotFound when media file does not exist", func() { + token := createTokenForMedia("gone-id", sourceTime) + + _, _, err := svc.ResolveRequestFromToken(ctx, token, "gone-id", 0) + Expect(err).To(MatchError(ErrMediaNotFound)) + }) + + It("returns ErrTokenStale when media file has changed", func() { + newTime := sourceTime.Add(1 * time.Hour) + mockMFRepo.SetData(model.MediaFiles{ + {ID: "song-1", UpdatedAt: newTime}, + }) + token := createTokenForMedia("song-1", sourceTime) + + _, _, err := svc.ResolveRequestFromToken(ctx, token, "song-1", 0) + Expect(err).To(MatchError(ErrTokenStale)) + }) + }) + + Describe("paramsFromToken", func() { + It("returns error when media ID is missing", func() { + tokenAuth := jwtauth.New("HS256", []byte("test-secret"), nil) + token, _, err := tokenAuth.Encode(map[string]any{"ua": int64(1700000000)}) + Expect(err).NotTo(HaveOccurred()) + + _, err = paramsFromToken(token) + Expect(err).To(MatchError(ContainSubstring("missing media ID"))) + }) + + It("returns error when source timestamp is missing", func() { + tokenAuth := jwtauth.New("HS256", []byte("test-secret"), nil) + token, _, err := tokenAuth.Encode(map[string]any{"mid": "song-5"}) + Expect(err).NotTo(HaveOccurred()) + + _, err = paramsFromToken(token) + Expect(err).To(MatchError(ContainSubstring("missing source timestamp"))) + }) + }) +}) diff --git a/core/transcode/transcode_suite_test.go b/core/transcode/transcode_suite_test.go new file mode 100644 index 000000000..e35471b05 --- /dev/null +++ b/core/transcode/transcode_suite_test.go @@ -0,0 +1,17 @@ +package transcode + +import ( + "testing" + + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +func TestTranscode(t *testing.T) { + tests.Init(t, false) + log.SetLevel(log.LevelFatal) + RegisterFailHandler(Fail) + RunSpecs(t, "Transcode Suite") +} diff --git a/core/transcode/types.go b/core/transcode/types.go new file mode 100644 index 000000000..d7a63fbc4 --- /dev/null +++ b/core/transcode/types.go @@ -0,0 +1,134 @@ +package transcode + +import ( + "errors" + "time" +) + +var ( + ErrTokenInvalid = errors.New("invalid or expired transcode token") + ErrMediaNotFound = errors.New("media file not found") + ErrTokenStale = errors.New("transcode token is stale: media file has changed") +) + +// DecisionOptions controls optional behavior of MakeDecision. +type DecisionOptions struct { + // SkipProbe prevents MakeDecision from running ffprobe on the media file. + // When true, source stream details are derived from tag metadata only. + SkipProbe bool +} + +// StreamRequest contains the resolved parameters for creating a media stream. +type StreamRequest struct { + ID string + Format string + BitRate int // kbps + SampleRate int + BitDepth int + Channels int + Offset int // seconds +} + +// ClientInfo represents client playback capabilities. +// All bitrate values are in kilobits per second (kbps) +type ClientInfo struct { + Name string + Platform string + MaxAudioBitrate int + MaxTranscodingAudioBitrate int + DirectPlayProfiles []DirectPlayProfile + TranscodingProfiles []Profile + CodecProfiles []CodecProfile +} + +// DirectPlayProfile describes a format the client can play directly +type DirectPlayProfile struct { + Containers []string + AudioCodecs []string + Protocols []string + MaxAudioChannels int +} + +// Profile describes a transcoding target the client supports +type Profile struct { + Container string + AudioCodec string + Protocol string + MaxAudioChannels int +} + +// CodecProfile describes codec-specific limitations +type CodecProfile struct { + Type string + Name string + Limitations []Limitation +} + +// Limitation describes a specific codec limitation +type Limitation struct { + Name string + Comparison string + Values []string + Required bool +} + +// Protocol values (OpenSubsonic spec enum) +const ( + ProtocolHTTP = "http" + ProtocolHLS = "hls" +) + +// Comparison operators (OpenSubsonic spec enum) +const ( + ComparisonEquals = "Equals" + ComparisonNotEquals = "NotEquals" + ComparisonLessThanEqual = "LessThanEqual" + ComparisonGreaterThanEqual = "GreaterThanEqual" +) + +// Limitation names (OpenSubsonic spec enum) +const ( + LimitationAudioChannels = "audioChannels" + LimitationAudioBitrate = "audioBitrate" + LimitationAudioProfile = "audioProfile" + LimitationAudioSamplerate = "audioSamplerate" + LimitationAudioBitdepth = "audioBitdepth" +) + +// Codec profile types (OpenSubsonic spec enum) +const ( + CodecProfileTypeAudio = "AudioCodec" +) + +// Decision represents the internal decision result. +// All bitrate values are in kilobits per second (kbps). +type Decision struct { + MediaID string + CanDirectPlay bool + CanTranscode bool + TranscodeReasons []string + ErrorReason string + TargetFormat string + TargetBitrate int + TargetChannels int + TargetSampleRate int + TargetBitDepth int + SourceStream StreamDetails + SourceUpdatedAt time.Time + TranscodeStream *StreamDetails +} + +// StreamDetails describes audio stream properties. +// Bitrate is in kilobits per second (kbps). +type StreamDetails struct { + Container string + Codec string + Profile string // Audio profile (e.g., "LC", "HE-AACv2"). Populated from ffprobe data. + Bitrate int + SampleRate int + BitDepth int + Channels int + Duration float32 + Size int64 + IsLossless bool +} diff --git a/core/wire_providers.go b/core/wire_providers.go index f9b472015..20b5eb9a5 100644 --- a/core/wire_providers.go +++ b/core/wire_providers.go @@ -10,11 +10,12 @@ import ( "github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" + "github.com/navidrome/navidrome/core/transcode" ) var Set = wire.NewSet( - NewMediaStreamer, - GetTranscodingCache, + transcode.NewMediaStreamer, + transcode.GetTranscodingCache, NewArchiver, NewPlayers, NewShare, @@ -22,6 +23,7 @@ var Set = wire.NewSet( NewLibrary, NewUser, NewMaintenance, + transcode.NewDecider, agents.GetAgents, external.NewProvider, wire.Bind(new(external.Agents), new(*agents.Agents)), diff --git a/db/migrations/20260307175815_add_codec_and_update_transcodings.go b/db/migrations/20260307175815_add_codec_and_update_transcodings.go new file mode 100644 index 000000000..4e8b1b7f5 --- /dev/null +++ b/db/migrations/20260307175815_add_codec_and_update_transcodings.go @@ -0,0 +1,73 @@ +package migrations + +import ( + "context" + "database/sql" + + "github.com/navidrome/navidrome/model/id" + "github.com/pressly/goose/v3" +) + +func init() { + goose.AddMigrationContext(upAddCodecAndUpdateTranscodings, downAddCodecAndUpdateTranscodings) +} + +func upAddCodecAndUpdateTranscodings(_ context.Context, tx *sql.Tx) error { + // Add codec column to media_file. + _, err := tx.Exec(`ALTER TABLE media_file ADD COLUMN codec VARCHAR(255) DEFAULT '' NOT NULL`) + if err != nil { + return err + } + _, err = tx.Exec(`CREATE INDEX IF NOT EXISTS media_file_codec ON media_file(codec)`) + if err != nil { + return err + } + + // Update old AAC default (adts) to new default (ipod with fragmented MP4). + // Only affects users who still have the unmodified old default command. + _, err = tx.Exec( + `UPDATE transcoding SET command = ? WHERE target_format = 'aac' AND command = ?`, + "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a aac -f ipod -movflags frag_keyframe+empty_moov -", + "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a aac -f adts -", + ) + if err != nil { + return err + } + + // Add FLAC transcoding for existing installations that were seeded before FLAC was added. + var count int + err = tx.QueryRow("SELECT COUNT(*) FROM transcoding WHERE target_format = 'flac'").Scan(&count) + if err != nil { + return err + } + if count == 0 { + _, err = tx.Exec( + "INSERT INTO transcoding (id, name, target_format, default_bit_rate, command) VALUES (?, ?, ?, ?, ?)", + id.NewRandom(), "flac audio", "flac", 0, + "ffmpeg -i %s -ss %t -map 0:a:0 -v 0 -c:a flac -f flac -", + ) + if err != nil { + return err + } + } + + // Add probe_data column for caching ffprobe results. + _, err = tx.Exec(`ALTER TABLE media_file ADD COLUMN probe_data TEXT DEFAULT NULL`) + if err != nil { + return err + } + return nil +} + +func downAddCodecAndUpdateTranscodings(_ context.Context, tx *sql.Tx) error { + _, err := tx.Exec(`ALTER TABLE media_file DROP COLUMN probe_data`) + if err != nil { + return err + } + _, err = tx.Exec(`DROP INDEX IF EXISTS media_file_codec`) + if err != nil { + return err + } + _, err = tx.Exec(`ALTER TABLE media_file DROP COLUMN codec`) + return err +} diff --git a/model/mediafile.go b/model/mediafile.go index 103b02639..20532bfb9 100644 --- a/model/mediafile.go +++ b/model/mediafile.go @@ -56,6 +56,8 @@ type MediaFile struct { SampleRate int `structs:"sample_rate" json:"sampleRate"` BitDepth int `structs:"bit_depth" json:"bitDepth"` Channels int `structs:"channels" json:"channels"` + Codec string `structs:"codec" json:"codec"` + ProbeData string `structs:"probe_data" json:"-" hash:"ignore"` Genre string `structs:"genre" json:"genre"` Genres Genres `structs:"-" json:"genres,omitempty"` SortTitle string `structs:"sort_title" json:"sortTitle,omitempty"` @@ -168,6 +170,63 @@ func (mf MediaFile) AbsolutePath() string { return filepath.Join(mf.LibraryPath, mf.Path) } +// AudioCodec returns the audio codec for this file. +// Uses the stored Codec field if available, otherwise infers from Suffix and audio properties. +func (mf MediaFile) AudioCodec() string { + // If we have a stored codec from scanning, normalize and return it + if mf.Codec != "" { + return strings.ToLower(mf.Codec) + } + // Fallback: infer from Suffix + BitDepth + return mf.inferCodecFromSuffix() +} + +// inferCodecFromSuffix infers the codec from the file extension when Codec field is empty. +func (mf MediaFile) inferCodecFromSuffix() string { + switch strings.ToLower(mf.Suffix) { + case "mp3", "mpga": + return "mp3" + case "mp2": + return "mp2" + case "ogg", "oga": + return "vorbis" + case "opus": + return "opus" + case "mpc": + return "mpc" + case "wma": + return "wma" + case "flac": + return "flac" + case "wav": + return "pcm" + case "aif", "aiff", "aifc": + return "pcm" + case "ape": + return "ape" + case "wv", "wvp": + return "wv" + case "tta": + return "tta" + case "tak": + return "tak" + case "shn": + return "shn" + case "dsf", "dff": + return "dsd" + case "m4a": + // AAC if BitDepth==0, ALAC if BitDepth>0 + if mf.BitDepth > 0 { + return "alac" + } + return "aac" + case "m4b", "m4p", "m4r": + return "aac" + default: + return "" + } +} + type MediaFiles []MediaFile // ToAlbum creates an Album object based on the attributes of this MediaFiles collection. @@ -363,6 +422,7 @@ type MediaFileRepository interface { CountBySuffix(options ...QueryOptions) (map[string]int64, error) Exists(id string) (bool, error) Put(m *MediaFile) error + UpdateProbeData(id string, data string) error Get(id string) (*MediaFile, error) GetWithParticipants(id string) (*MediaFile, error) GetAll(options ...QueryOptions) (MediaFiles, error) diff --git a/model/mediafile_test.go b/model/mediafile_test.go index 0b9191fe5..207d3c155 100644 --- a/model/mediafile_test.go +++ b/model/mediafile_test.go @@ -497,7 +497,7 @@ var _ = Describe("MediaFile", func() { Entry("returns just album name when tag is absent", true, Tags{}, "Album"), Entry("returns just album name when tag is an empty slice", true, Tags{TagAlbumVersion: []string{}}, "Album"), ) - Describe("CoverArtId()", func() { + Describe("CoverArtId", func() { It("returns its own id if it HasCoverArt", func() { mf := MediaFile{ID: "111", AlbumID: "1", HasCoverArt: true} id := mf.CoverArtID() @@ -518,6 +518,58 @@ var _ = Describe("MediaFile", func() { Expect(id.ID).To(Equal(mf.AlbumID)) }) }) + + Describe("AudioCodec", func() { + It("returns normalized stored codec when available", func() { + mf := MediaFile{Codec: "AAC", Suffix: "m4a"} + Expect(mf.AudioCodec()).To(Equal("aac")) + }) + + It("returns stored codec lowercased", func() { + mf := MediaFile{Codec: "ALAC", Suffix: "m4a"} + Expect(mf.AudioCodec()).To(Equal("alac")) + }) + + DescribeTable("infers codec from suffix when Codec field is empty", + func(suffix string, bitDepth int, expected string) { + mf := MediaFile{Suffix: suffix, BitDepth: bitDepth} + Expect(mf.AudioCodec()).To(Equal(expected)) + }, + Entry("mp3", "mp3", 0, "mp3"), + Entry("mpga", "mpga", 0, "mp3"), + Entry("mp2", "mp2", 0, "mp2"), + Entry("ogg", "ogg", 0, "vorbis"), + Entry("oga", "oga", 0, "vorbis"), + Entry("opus", "opus", 0, "opus"), + Entry("mpc", "mpc", 0, "mpc"), + Entry("wma", "wma", 0, "wma"), + Entry("flac", "flac", 0, "flac"), + Entry("wav", "wav", 0, "pcm"), + Entry("aif", "aif", 0, "pcm"), + Entry("aiff", "aiff", 0, "pcm"), + Entry("aifc", "aifc", 0, "pcm"), + Entry("ape", "ape", 0, "ape"), + Entry("wv", "wv", 0, "wv"), + Entry("wvp", "wvp", 0, "wv"), + Entry("tta", "tta", 0, "tta"), + Entry("tak", "tak", 0, "tak"), + Entry("shn", "shn", 0, "shn"), + Entry("dsf", "dsf", 0, "dsd"), + Entry("dff", "dff", 0, "dsd"), + Entry("m4a with BitDepth=0 (AAC)", "m4a", 0, "aac"), + Entry("m4a with BitDepth>0 (ALAC)", "m4a", 16, "alac"), + Entry("m4b", "m4b", 0, "aac"), + Entry("m4p", "m4p", 0, "aac"), + Entry("m4r", "m4r", 0, "aac"), + Entry("unknown suffix", "xyz", 0, ""), + ) + + It("prefers stored codec over suffix inference", func() { + mf := MediaFile{Codec: "ALAC", Suffix: "m4a", BitDepth: 0} + Expect(mf.AudioCodec()).To(Equal("alac")) + }) + }) + }) func t(v string) time.Time { diff --git a/model/metadata/map_mediafile.go b/model/metadata/map_mediafile.go index c64e8c724..824cad7c2 100644 --- a/model/metadata/map_mediafile.go +++ b/model/metadata/map_mediafile.go @@ -65,6 +65,7 @@ func (md Metadata) ToMediaFile(libID int, folderID string) model.MediaFile { mf.SampleRate = md.AudioProperties().SampleRate mf.BitDepth = md.AudioProperties().BitDepth mf.Channels = md.AudioProperties().Channels + mf.Codec = md.AudioProperties().Codec mf.Path = md.FilePath() mf.Suffix = md.Suffix() mf.Size = md.Size() diff --git a/model/metadata/metadata.go b/model/metadata/metadata.go index 954505c98..48928f989 100644 --- a/model/metadata/metadata.go +++ b/model/metadata/metadata.go @@ -35,6 +35,7 @@ type AudioProperties struct { BitDepth int SampleRate int Channels int + Codec string } type Date string diff --git a/persistence/mediafile_repository.go b/persistence/mediafile_repository.go index 43736e317..264778ea0 100644 --- a/persistence/mediafile_repository.go +++ b/persistence/mediafile_repository.go @@ -163,6 +163,11 @@ func (r *mediaFileRepository) Put(m *model.MediaFile) error { return r.updateParticipants(m.ID, m.Participants) } +func (r *mediaFileRepository) UpdateProbeData(id string, data string) error { + _, err := r.executeSQL(Update(r.tableName).Set("probe_data", data).Where(Eq{"id": id})) + return err +} + func (r *mediaFileRepository) selectMediaFile(options ...model.QueryOptions) SelectBuilder { sql := r.newSelect(options...).Columns("media_file.*", "library.path as library_path", "library.name as library_name"). LeftJoin("library on media_file.library_id = library.id") diff --git a/server/e2e/e2e_suite_test.go b/server/e2e/e2e_suite_test.go index 484a91cc7..02b66f4b7 100644 --- a/server/e2e/e2e_suite_test.go +++ b/server/e2e/e2e_suite_test.go @@ -3,6 +3,7 @@ package e2e import ( "context" "encoding/json" + "errors" "io" "net/http" "net/http/httptest" @@ -19,12 +20,14 @@ import ( "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/auth" "github.com/navidrome/navidrome/core/external" + "github.com/navidrome/navidrome/core/ffmpeg" "github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" "github.com/navidrome/navidrome/core/storage/storagetest" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" @@ -69,6 +72,7 @@ var ( ctx context.Context ds *tests.MockDataStore router *subsonic.Router + spy *spyStreamer lib model.Library // Snapshot paths for fast DB restore @@ -224,17 +228,50 @@ func (n noopArtwork) GetOrPlaceholder(_ context.Context, _ string, _ int, _ bool return io.NopCloser(io.LimitReader(nil, 0)), time.Time{}, nil } -// noopStreamer implements core.MediaStreamer -type noopStreamer struct{} +// spyStreamer captures the StreamRequest passed to DoStream for test assertions, +// then returns a minimal fake Stream so the handler completes without error. +type spyStreamer struct { + LastRequest transcode.StreamRequest + LastMediaFile *model.MediaFile +} -func (n noopStreamer) NewStream(context.Context, string, string, int, int) (*core.Stream, error) { +func (s *spyStreamer) NewStream(ctx context.Context, req transcode.StreamRequest) (*transcode.Stream, error) { return nil, model.ErrNotFound } -func (n noopStreamer) DoStream(context.Context, *model.MediaFile, string, int, int) (*core.Stream, error) { - return nil, model.ErrNotFound +func (s *spyStreamer) DoStream(_ context.Context, mf *model.MediaFile, req transcode.StreamRequest) (*transcode.Stream, error) { + s.LastRequest = req + s.LastMediaFile = mf + format := req.Format + if format == "" || format == "raw" { + format = mf.Suffix + } + return transcode.NewTestStream(mf, format, req.BitRate), nil } +// noopFFmpeg implements ffmpeg.FFmpeg with no-op methods. +type noopFFmpeg struct{} + +func (n noopFFmpeg) Transcode(context.Context, ffmpeg.TranscodeOptions) (io.ReadCloser, error) { + return nil, errors.New("noop ffmpeg: transcode not supported") +} + +func (n noopFFmpeg) ExtractImage(context.Context, string) (io.ReadCloser, error) { + return nil, errors.New("noop ffmpeg: extract image not supported") +} + +func (n noopFFmpeg) Probe(context.Context, []string) (string, error) { + return "", nil +} + +func (n noopFFmpeg) ProbeAudioStream(context.Context, string) (*ffmpeg.AudioProbeResult, error) { + return nil, errors.New("noop ffmpeg: probe not supported") +} + +func (n noopFFmpeg) CmdPath() (string, error) { return "", nil } +func (n noopFFmpeg) IsAvailable() bool { return false } +func (n noopFFmpeg) Version() string { return "noop" } + // noopArchiver implements core.Archiver type noopArchiver struct{} @@ -298,11 +335,12 @@ func (n noopPlayTracker) Submit(context.Context, []scrobbler.Submission) error { // Compile-time interface checks var ( - _ artwork.Artwork = noopArtwork{} - _ core.MediaStreamer = noopStreamer{} - _ core.Archiver = noopArchiver{} - _ external.Provider = noopProvider{} - _ scrobbler.PlayTracker = noopPlayTracker{} + _ artwork.Artwork = noopArtwork{} + _ transcode.MediaStreamer = &spyStreamer{} + _ core.Archiver = noopArchiver{} + _ external.Provider = noopProvider{} + _ scrobbler.PlayTracker = noopPlayTracker{} + _ ffmpeg.FFmpeg = noopFFmpeg{} ) var _ = BeforeSuite(func() { @@ -380,13 +418,15 @@ func setupTestDB() { ds = &tests.MockDataStore{RealDS: persistence.New(db.Db())} auth.Init(ds) - // Create the Subsonic Router with real DS + noop stubs + // Create the Subsonic Router with real DS, spy streamer, and real Decider + spy = &spyStreamer{} + decider := transcode.NewDecider(ds, noopFFmpeg{}) s := scanner.New(ctx, ds, artwork.NoopCacheWarmer(), events.NoopBroker(), playlists.NewPlaylists(ds), metrics.NewNoopInstance()) router = subsonic.New( ds, noopArtwork{}, - noopStreamer{}, + spy, noopArchiver{}, core.NewPlayers(ds), noopProvider{}, @@ -398,6 +438,7 @@ func setupTestDB() { playback.PlaybackServer(nil), metrics.NewNoopInstance(), lyrics.NewLyrics(nil), + decider, ) } diff --git a/server/e2e/subsonic_media_retrieval_test.go b/server/e2e/subsonic_media_retrieval_test.go index c36713dbb..465082acb 100644 --- a/server/e2e/subsonic_media_retrieval_test.go +++ b/server/e2e/subsonic_media_retrieval_test.go @@ -3,6 +3,9 @@ package e2e import ( "net/http" + "github.com/navidrome/navidrome/conf" + "github.com/navidrome/navidrome/conf/configtest" + "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/server/subsonic/responses" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" @@ -14,21 +17,142 @@ var _ = Describe("Media Retrieval Endpoints", Ordered, func() { }) Describe("Stream", func() { + var trackID string + + BeforeAll(func() { + // All test tracks are mp3 at 320kbps + songs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{Max: 1, Sort: "title"}) + Expect(err).ToNot(HaveOccurred()) + Expect(songs).ToNot(BeEmpty()) + trackID = songs[0].ID + }) + It("returns error when id parameter is missing", func() { resp := doReq("stream") Expect(resp.Status).To(Equal(responses.StatusFailed)) Expect(resp.Error).ToNot(BeNil()) }) + + It("streams raw when no format or bitrate specified", func() { + w := doRawReq("stream", "id", trackID) + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("raw")) + }) + + It("streams raw when format=raw", func() { + w := doRawReq("stream", "id", trackID, "format", "raw") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("raw")) + }) + + It("transcodes to different format with bitrate", func() { + w := doRawReq("stream", "id", trackID, "format", "opus", "maxBitRate", "128") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("opus")) + Expect(spy.LastRequest.BitRate).To(Equal(128)) + }) + + It("downsamples when only maxBitRate is specified (lower than source)", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.DefaultDownsamplingFormat = "opus" + + w := doRawReq("stream", "id", trackID, "maxBitRate", "128") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("opus")) + Expect(spy.LastRequest.BitRate).To(Equal(128)) + }) + + It("streams raw when maxBitRate is higher than source", func() { + w := doRawReq("stream", "id", trackID, "maxBitRate", "999") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("raw")) + }) + + It("streams raw when format matches source and no bitrate reduction", func() { + w := doRawReq("stream", "id", trackID, "format", "mp3", "maxBitRate", "320") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("raw")) + }) + + It("transcodes when same format but lower bitrate", func() { + w := doRawReq("stream", "id", trackID, "format", "mp3", "maxBitRate", "128") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("mp3")) + Expect(spy.LastRequest.BitRate).To(Equal(128)) + }) + + It("falls back to raw for unknown format", func() { + w := doRawReq("stream", "id", trackID, "format", "xyz") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("raw")) + }) + + It("passes timeOffset through", func() { + w := doRawReq("stream", "id", trackID, "format", "opus", "maxBitRate", "128", "timeOffset", "30") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("opus")) + Expect(spy.LastRequest.Offset).To(Equal(30)) + }) }) Describe("Download", func() { + var trackID string + + BeforeAll(func() { + // All test tracks are mp3 at 320kbps + songs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{Max: 1, Sort: "title"}) + Expect(err).ToNot(HaveOccurred()) + Expect(songs).ToNot(BeEmpty()) + trackID = songs[0].ID + }) + It("returns error when id parameter is missing", func() { resp := doReq("download") Expect(resp.Status).To(Equal(responses.StatusFailed)) Expect(resp.Error).ToNot(BeNil()) }) + + It("downloads raw when no format specified and AutoTranscodeDownload is false", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.EnableDownloads = true + conf.Server.AutoTranscodeDownload = false + + w := doRawReq("download", "id", trackID) + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("raw")) + }) + + It("downloads with explicit format and bitrate", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.EnableDownloads = true + + w := doRawReq("download", "id", trackID, "format", "opus", "bitrate", "128") + + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("opus")) + Expect(spy.LastRequest.BitRate).To(Equal(128)) + }) + + It("returns error when downloads are disabled", func() { + DeferCleanup(configtest.SetupConfig()) + conf.Server.EnableDownloads = false + + resp := doReq("download", "id", trackID) + + Expect(resp.Status).To(Equal(responses.StatusFailed)) + }) }) Describe("GetCoverArt", func() { diff --git a/server/public/handle_streams.go b/server/public/handle_streams.go index d6819974b..a147a2ac8 100644 --- a/server/public/handle_streams.go +++ b/server/public/handle_streams.go @@ -7,6 +7,7 @@ import ( "strconv" "github.com/navidrome/navidrome/core/auth" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/utils/req" ) @@ -22,10 +23,13 @@ func (pub *Router) handleStream(w http.ResponseWriter, r *http.Request) { return } - stream, err := pub.streamer.NewStream(ctx, info.id, info.format, info.bitrate, 0) + stream, err := pub.streamer.NewStream(ctx, transcode.StreamRequest{ + ID: info.id, Format: info.format, BitRate: info.bitrate, + }) if err != nil { log.Error(ctx, "Error starting shared stream", err) http.Error(w, "invalid request", http.StatusInternalServerError) + return } // Make sure the stream will be closed at the end, to avoid leakage diff --git a/server/public/public.go b/server/public/public.go index ebccb01d2..7d8a4e007 100644 --- a/server/public/public.go +++ b/server/public/public.go @@ -11,6 +11,7 @@ import ( "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core/publicurl" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/server" @@ -20,14 +21,14 @@ import ( type Router struct { http.Handler artwork artwork.Artwork - streamer core.MediaStreamer + streamer transcode.MediaStreamer archiver core.Archiver share core.Share assetsHandler http.Handler ds model.DataStore } -func New(ds model.DataStore, artwork artwork.Artwork, streamer core.MediaStreamer, share core.Share, archiver core.Archiver) *Router { +func New(ds model.DataStore, artwork artwork.Artwork, streamer transcode.MediaStreamer, share core.Share, archiver core.Archiver) *Router { p := &Router{ds: ds, artwork: artwork, streamer: streamer, share: share, archiver: archiver} shareRoot := path.Join(conf.Server.BasePath, consts.URLPathPublic) p.assetsHandler = http.StripPrefix(shareRoot, http.FileServer(http.FS(ui.BuildAssets()))) diff --git a/server/subsonic/album_lists_test.go b/server/subsonic/album_lists_test.go index aac2d63da..ae4ef9bb9 100644 --- a/server/subsonic/album_lists_test.go +++ b/server/subsonic/album_lists_test.go @@ -27,7 +27,7 @@ var _ = Describe("Album Lists", func() { ds = &tests.MockDataStore{} auth.Init(ds) mockRepo = ds.Album(ctx).(*tests.MockAlbumRepo) - router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) w = httptest.NewRecorder() }) diff --git a/server/subsonic/api.go b/server/subsonic/api.go index 8674a2946..6f355d161 100644 --- a/server/subsonic/api.go +++ b/server/subsonic/api.go @@ -19,6 +19,7 @@ import ( "github.com/navidrome/navidrome/core/playback" playlistsvc "github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/scrobbler" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/server" @@ -36,42 +37,44 @@ type handlerRaw = func(http.ResponseWriter, *http.Request) (*responses.Subsonic, type Router struct { http.Handler - ds model.DataStore - artwork artwork.Artwork - streamer core.MediaStreamer - archiver core.Archiver - players core.Players - provider external.Provider - playlists playlistsvc.Playlists - scanner model.Scanner - broker events.Broker - scrobbler scrobbler.PlayTracker - share core.Share - playback playback.PlaybackServer - metrics metrics.Metrics - lyrics lyricssvc.Lyrics + ds model.DataStore + artwork artwork.Artwork + streamer transcode.MediaStreamer + archiver core.Archiver + players core.Players + provider external.Provider + playlists playlistsvc.Playlists + scanner model.Scanner + broker events.Broker + scrobbler scrobbler.PlayTracker + share core.Share + playback playback.PlaybackServer + metrics metrics.Metrics + lyrics lyricssvc.Lyrics + transcodeDecision transcode.Decider } -func New(ds model.DataStore, artwork artwork.Artwork, streamer core.MediaStreamer, archiver core.Archiver, +func New(ds model.DataStore, artwork artwork.Artwork, streamer transcode.MediaStreamer, archiver core.Archiver, players core.Players, provider external.Provider, scanner model.Scanner, broker events.Broker, playlists playlistsvc.Playlists, scrobbler scrobbler.PlayTracker, share core.Share, playback playback.PlaybackServer, - metrics metrics.Metrics, lyrics lyricssvc.Lyrics, + metrics metrics.Metrics, lyrics lyricssvc.Lyrics, transcodeDecision transcode.Decider, ) *Router { r := &Router{ - ds: ds, - artwork: artwork, - streamer: streamer, - archiver: archiver, - players: players, - provider: provider, - playlists: playlists, - scanner: scanner, - broker: broker, - scrobbler: scrobbler, - share: share, - playback: playback, - metrics: metrics, - lyrics: lyrics, + ds: ds, + artwork: artwork, + streamer: streamer, + archiver: archiver, + players: players, + provider: provider, + playlists: playlists, + scanner: scanner, + broker: broker, + scrobbler: scrobbler, + share: share, + playback: playback, + metrics: metrics, + lyrics: lyrics, + transcodeDecision: transcodeDecision, } r.Handler = r.routes() return r @@ -176,6 +179,8 @@ func (api *Router) routes() http.Handler { h(r, "getLyricsBySongId", api.GetLyricsBySongId) hr(r, "stream", api.Stream) hr(r, "download", api.Download) + hr(r, "getTranscodeDecision", api.GetTranscodeDecision) + hr(r, "getTranscodeStream", api.GetTranscodeStream) }) r.Group(func(r chi.Router) { // configure request throttling diff --git a/server/subsonic/media_annotation_test.go b/server/subsonic/media_annotation_test.go index 57809fbb6..fc767b0ff 100644 --- a/server/subsonic/media_annotation_test.go +++ b/server/subsonic/media_annotation_test.go @@ -27,7 +27,7 @@ var _ = Describe("MediaAnnotationController", func() { ds = &tests.MockDataStore{} playTracker = &fakePlayTracker{} eventBroker = &fakeEventBroker{} - router = New(ds, nil, nil, nil, nil, nil, nil, eventBroker, nil, playTracker, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, eventBroker, nil, playTracker, nil, nil, nil, nil, nil) }) Describe("Scrobble", func() { diff --git a/server/subsonic/media_retrieval_test.go b/server/subsonic/media_retrieval_test.go index 1a638f066..7f64fb47f 100644 --- a/server/subsonic/media_retrieval_test.go +++ b/server/subsonic/media_retrieval_test.go @@ -34,7 +34,7 @@ var _ = Describe("MediaRetrievalController", func() { MockedMediaFile: mockRepo, } artwork = &fakeArtwork{data: "image data"} - router = New(ds, artwork, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, lyrics.NewLyrics(nil)) + router = New(ds, artwork, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, lyrics.NewLyrics(nil), nil) w = httptest.NewRecorder() DeferCleanup(configtest.SetupConfig()) conf.Server.LyricsPriority = "embedded,.lrc" diff --git a/server/subsonic/opensubsonic.go b/server/subsonic/opensubsonic.go index a364651c5..353cf1077 100644 --- a/server/subsonic/opensubsonic.go +++ b/server/subsonic/opensubsonic.go @@ -13,6 +13,7 @@ func (api *Router) GetOpenSubsonicExtensions(_ *http.Request) (*responses.Subson {Name: "formPost", Versions: []int32{1}}, {Name: "songLyrics", Versions: []int32{1}}, {Name: "indexBasedQueue", Versions: []int32{1}}, + {Name: "transcoding", Versions: []int32{1}}, } return response, nil } diff --git a/server/subsonic/opensubsonic_test.go b/server/subsonic/opensubsonic_test.go index c02b262b9..92d1c3e84 100644 --- a/server/subsonic/opensubsonic_test.go +++ b/server/subsonic/opensubsonic_test.go @@ -19,7 +19,7 @@ var _ = Describe("GetOpenSubsonicExtensions", func() { ) BeforeEach(func() { - router = subsonic.New(nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = subsonic.New(nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) w = httptest.NewRecorder() r = httptest.NewRequest("GET", "/getOpenSubsonicExtensions?f=json", nil) }) @@ -35,11 +35,12 @@ var _ = Describe("GetOpenSubsonicExtensions", func() { err := json.Unmarshal(w.Body.Bytes(), &response) Expect(err).NotTo(HaveOccurred()) Expect(*response.Subsonic.OpenSubsonicExtensions).To(SatisfyAll( - HaveLen(4), + HaveLen(5), ContainElement(responses.OpenSubsonicExtension{Name: "transcodeOffset", Versions: []int32{1}}), ContainElement(responses.OpenSubsonicExtension{Name: "formPost", Versions: []int32{1}}), ContainElement(responses.OpenSubsonicExtension{Name: "songLyrics", Versions: []int32{1}}), ContainElement(responses.OpenSubsonicExtension{Name: "indexBasedQueue", Versions: []int32{1}}), + ContainElement(responses.OpenSubsonicExtension{Name: "transcoding", Versions: []int32{1}}), )) }) }) diff --git a/server/subsonic/playlists_test.go b/server/subsonic/playlists_test.go index 86c17b39c..41701b4de 100644 --- a/server/subsonic/playlists_test.go +++ b/server/subsonic/playlists_test.go @@ -24,7 +24,7 @@ var _ = Describe("buildPlaylist", func() { BeforeEach(func() { ds = &tests.MockDataStore{} - router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) ctx = context.Background() }) @@ -224,7 +224,7 @@ var _ = Describe("UpdatePlaylist", func() { BeforeEach(func() { ds = &tests.MockDataStore{} playlists = &fakePlaylists{} - router = New(ds, nil, nil, nil, nil, nil, nil, nil, playlists, nil, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, playlists, nil, nil, nil, nil, nil, nil) }) It("clears the comment when parameter is empty", func() { diff --git a/server/subsonic/responses/responses.go b/server/subsonic/responses/responses.go index 0fdbf1be6..be59e5851 100644 --- a/server/subsonic/responses/responses.go +++ b/server/subsonic/responses/responses.go @@ -61,6 +61,7 @@ type Subsonic struct { OpenSubsonicExtensions *OpenSubsonicExtensions `xml:"openSubsonicExtensions,omitempty" json:"openSubsonicExtensions,omitempty"` LyricsList *LyricsList `xml:"lyricsList,omitempty" json:"lyricsList,omitempty"` PlayQueueByIndex *PlayQueueByIndex `xml:"playQueueByIndex,omitempty" json:"playQueueByIndex,omitempty"` + TranscodeDecision *TranscodeDecision `xml:"transcodeDecision,omitempty" json:"transcodeDecision,omitempty"` } const ( @@ -617,3 +618,26 @@ func marshalJSONArray[T any](v []T) ([]byte, error) { } return json.Marshal(v) } + +// TranscodeDecision represents the response for getTranscodeDecision (OpenSubsonic transcoding extension) +type TranscodeDecision struct { + CanDirectPlay bool `xml:"canDirectPlay,attr" json:"canDirectPlay"` + CanTranscode bool `xml:"canTranscode,attr" json:"canTranscode"` + TranscodeReasons []string `xml:"transcodeReason,omitempty" json:"transcodeReason,omitempty"` + ErrorReason string `xml:"errorReason,attr,omitempty" json:"errorReason,omitempty"` + TranscodeParams string `xml:"transcodeParams,attr,omitempty" json:"transcodeParams,omitempty"` + SourceStream *StreamDetails `xml:"sourceStream,omitempty" json:"sourceStream,omitempty"` + TranscodeStream *StreamDetails `xml:"transcodeStream,omitempty" json:"transcodeStream,omitempty"` +} + +// StreamDetails describes audio stream properties for transcoding decisions +type StreamDetails struct { + Protocol string `xml:"protocol,attr,omitempty" json:"protocol,omitempty"` + Container string `xml:"container,attr,omitempty" json:"container,omitempty"` + Codec string `xml:"codec,attr,omitempty" json:"codec,omitempty"` + AudioChannels int32 `xml:"audioChannels,attr,omitempty" json:"audioChannels,omitempty"` + AudioBitrate int32 `xml:"audioBitrate,attr,omitempty" json:"audioBitrate,omitempty"` + AudioProfile string `xml:"audioProfile,attr,omitempty" json:"audioProfile,omitempty"` + AudioSamplerate int32 `xml:"audioSamplerate,attr,omitempty" json:"audioSamplerate,omitempty"` + AudioBitdepth int32 `xml:"audioBitdepth,attr,omitempty" json:"audioBitdepth,omitempty"` +} diff --git a/server/subsonic/searching_test.go b/server/subsonic/searching_test.go index d4b7e9702..ab40a726f 100644 --- a/server/subsonic/searching_test.go +++ b/server/subsonic/searching_test.go @@ -21,7 +21,7 @@ var _ = Describe("Search", func() { ds = &tests.MockDataStore{} auth.Init(ds) - router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) + router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil) // Get references to the mock repositories so we can inspect their Options mockAlbumRepo = ds.Album(nil).(*tests.MockAlbumRepo) diff --git a/server/subsonic/stream.go b/server/subsonic/stream.go index d0cbe2086..753e408c1 100644 --- a/server/subsonic/stream.go +++ b/server/subsonic/stream.go @@ -9,7 +9,7 @@ import ( "strings" "github.com/navidrome/navidrome/conf" - "github.com/navidrome/navidrome/core" + "github.com/navidrome/navidrome/core/transcode" "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model/request" @@ -17,7 +17,7 @@ import ( "github.com/navidrome/navidrome/utils/req" ) -func (api *Router) serveStream(ctx context.Context, w http.ResponseWriter, r *http.Request, stream *core.Stream, id string) { +func (api *Router) serveStream(ctx context.Context, w http.ResponseWriter, r *http.Request, stream *transcode.Stream, id string) { if stream.Seekable() { http.ServeContent(w, r, stream.Name(), stream.ModTime(), stream) } else { @@ -60,7 +60,13 @@ func (api *Router) Stream(w http.ResponseWriter, r *http.Request) (*responses.Su format, _ := p.String("format") timeOffset := p.IntOr("timeOffset", 0) - stream, err := api.streamer.NewStream(ctx, id, format, maxBitRate, timeOffset) + mf, err := api.ds.MediaFile(ctx).Get(id) + if err != nil { + return nil, err + } + + streamReq := api.transcodeDecision.ResolveRequest(ctx, mf, format, maxBitRate, timeOffset) + stream, err := api.streamer.DoStream(ctx, mf, streamReq) if err != nil { return nil, err } @@ -129,7 +135,8 @@ func (api *Router) Download(w http.ResponseWriter, r *http.Request) (*responses. switch v := entity.(type) { case *model.MediaFile: - stream, err := api.streamer.NewStream(ctx, id, format, maxBitRate, 0) + streamReq := api.transcodeDecision.ResolveRequest(ctx, v, format, maxBitRate, 0) + stream, err := api.streamer.DoStream(ctx, v, streamReq) if err != nil { return nil, err } diff --git a/server/subsonic/transcode.go b/server/subsonic/transcode.go new file mode 100644 index 000000000..ffc4cfcd7 --- /dev/null +++ b/server/subsonic/transcode.go @@ -0,0 +1,381 @@ +package subsonic + +import ( + "encoding/json" + "errors" + "fmt" + "net/http" + "slices" + "strconv" + + "github.com/navidrome/navidrome/core/transcode" + "github.com/navidrome/navidrome/log" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/server/subsonic/responses" + "github.com/navidrome/navidrome/utils/req" +) + +// API-layer request structs for JSON unmarshaling (decoupled from core structs) + +// clientInfoRequest represents client playback capabilities from the request body +type clientInfoRequest struct { + Name string `json:"name,omitempty"` + Platform string `json:"platform,omitempty"` + MaxAudioBitrate int `json:"maxAudioBitrate,omitempty"` + MaxTranscodingAudioBitrate int `json:"maxTranscodingAudioBitrate,omitempty"` + DirectPlayProfiles []directPlayProfileRequest `json:"directPlayProfiles,omitempty"` + TranscodingProfiles []transcodingProfileRequest `json:"transcodingProfiles,omitempty"` + CodecProfiles []codecProfileRequest `json:"codecProfiles,omitempty"` +} + +// directPlayProfileRequest describes a format the client can play directly +type directPlayProfileRequest struct { + Containers []string `json:"containers,omitempty"` + AudioCodecs []string `json:"audioCodecs,omitempty"` + Protocols []string `json:"protocols,omitempty"` + MaxAudioChannels int `json:"maxAudioChannels,omitempty"` +} + +// transcodingProfileRequest describes a transcoding target the client supports +type transcodingProfileRequest struct { + Container string `json:"container,omitempty"` + AudioCodec string `json:"audioCodec,omitempty"` + Protocol string `json:"protocol,omitempty"` + MaxAudioChannels int `json:"maxAudioChannels,omitempty"` +} + +// codecProfileRequest describes codec-specific limitations +type codecProfileRequest struct { + Type string `json:"type,omitempty"` + Name string `json:"name,omitempty"` + Limitations []limitationRequest `json:"limitations,omitempty"` +} + +// limitationRequest describes a specific codec limitation +type limitationRequest struct { + Name string `json:"name,omitempty"` + Comparison string `json:"comparison,omitempty"` + Values []string `json:"values,omitempty"` + Required bool `json:"required,omitempty"` +} + +// toCoreClientInfo converts the API request struct to the transcode.ClientInfo struct. +// The OpenSubsonic spec uses bps for bitrate values; core uses kbps. +func (r *clientInfoRequest) toCoreClientInfo() *transcode.ClientInfo { + ci := &transcode.ClientInfo{ + Name: r.Name, + Platform: r.Platform, + MaxAudioBitrate: bpsToKbps(r.MaxAudioBitrate), + MaxTranscodingAudioBitrate: bpsToKbps(r.MaxTranscodingAudioBitrate), + } + + for _, dp := range r.DirectPlayProfiles { + ci.DirectPlayProfiles = append(ci.DirectPlayProfiles, transcode.DirectPlayProfile{ + Containers: dp.Containers, + AudioCodecs: dp.AudioCodecs, + Protocols: dp.Protocols, + MaxAudioChannels: dp.MaxAudioChannels, + }) + } + + for _, tp := range r.TranscodingProfiles { + ci.TranscodingProfiles = append(ci.TranscodingProfiles, transcode.Profile{ + Container: tp.Container, + AudioCodec: tp.AudioCodec, + Protocol: tp.Protocol, + MaxAudioChannels: tp.MaxAudioChannels, + }) + } + + for _, cp := range r.CodecProfiles { + coreCP := transcode.CodecProfile{ + Type: cp.Type, + Name: cp.Name, + } + for _, lim := range cp.Limitations { + coreLim := transcode.Limitation{ + Name: lim.Name, + Comparison: lim.Comparison, + Values: lim.Values, + Required: lim.Required, + } + // Convert audioBitrate limitation values from bps to kbps + if lim.Name == transcode.LimitationAudioBitrate { + coreLim.Values = convertBitrateValues(lim.Values) + } + coreCP.Limitations = append(coreCP.Limitations, coreLim) + } + ci.CodecProfiles = append(ci.CodecProfiles, coreCP) + } + + return ci +} + +// bpsToKbps converts bits per second to kilobits per second (rounded). +func bpsToKbps(bps int) int { + if bps < 0 { + return 0 + } + return (bps + 500) / 1000 +} + +// kbpsToBps converts kilobits per second to bits per second. +func kbpsToBps(kbps int) int { + return kbps * 1000 +} + +// convertBitrateValues converts a slice of bps string values to kbps string values. +func convertBitrateValues(bpsValues []string) []string { + result := make([]string, len(bpsValues)) + for i, v := range bpsValues { + n, err := strconv.Atoi(v) + if err == nil { + result[i] = strconv.Itoa(bpsToKbps(n)) + } else { + result[i] = v // preserve unparseable values as-is + } + } + return result +} + +// validate checks that all enum fields in the request contain valid values per the OpenSubsonic spec. +func (r *clientInfoRequest) validate() error { + for _, dp := range r.DirectPlayProfiles { + for _, p := range dp.Protocols { + if !isValidProtocol(p) { + return fmt.Errorf("invalid protocol: %s", p) + } + } + } + for _, tp := range r.TranscodingProfiles { + if tp.Protocol != "" && !isValidProtocol(tp.Protocol) { + return fmt.Errorf("invalid protocol: %s", tp.Protocol) + } + } + for _, cp := range r.CodecProfiles { + if !isValidCodecProfileType(cp.Type) { + return fmt.Errorf("invalid codec profile type: %s", cp.Type) + } + for _, lim := range cp.Limitations { + if !isValidLimitationName(lim.Name) { + return fmt.Errorf("invalid limitation name: %s", lim.Name) + } + if !isValidComparison(lim.Comparison) { + return fmt.Errorf("invalid comparison: %s", lim.Comparison) + } + } + } + return nil +} + +// Only support songs for now +var validMediaTypes = []string{ + "song", +} + +func isValidMediaType(mediaType string) bool { + return slices.Contains(validMediaTypes, mediaType) +} + +var validProtocols = []string{ + transcode.ProtocolHTTP, + transcode.ProtocolHLS, +} + +func isValidProtocol(p string) bool { + return slices.Contains(validProtocols, p) +} + +var validCodecProfileTypes = []string{ + transcode.CodecProfileTypeAudio, +} + +func isValidCodecProfileType(t string) bool { + return slices.Contains(validCodecProfileTypes, t) +} + +var validLimitationNames = []string{ + transcode.LimitationAudioChannels, + transcode.LimitationAudioBitrate, + transcode.LimitationAudioProfile, + transcode.LimitationAudioSamplerate, + transcode.LimitationAudioBitdepth, +} + +func isValidLimitationName(n string) bool { + return slices.Contains(validLimitationNames, n) +} + +var validComparisons = []string{ + transcode.ComparisonEquals, + transcode.ComparisonNotEquals, + transcode.ComparisonLessThanEqual, + transcode.ComparisonGreaterThanEqual, +} + +func isValidComparison(c string) bool { + return slices.Contains(validComparisons, c) +} + +// toResponseStreamDetails converts a core StreamDetails to the API response type. +func toResponseStreamDetails(sd *transcode.StreamDetails) *responses.StreamDetails { + return &responses.StreamDetails{ + Protocol: transcode.ProtocolHTTP, // TODO: derive from decision when HLS support is added + Container: sd.Container, + Codec: sd.Codec, + AudioBitrate: int32(kbpsToBps(sd.Bitrate)), + AudioProfile: sd.Profile, + AudioSamplerate: int32(sd.SampleRate), + AudioBitdepth: int32(sd.BitDepth), + AudioChannels: int32(sd.Channels), + } +} + +// GetTranscodeDecision handles the OpenSubsonic getTranscodeDecision endpoint. +// It receives client capabilities and returns a decision on whether to direct play or transcode. +func (api *Router) GetTranscodeDecision(w http.ResponseWriter, r *http.Request) (*responses.Subsonic, error) { + if r.Method != http.MethodPost { + w.Header().Set("Allow", "POST") + http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed) + return nil, nil + } + + ctx := r.Context() + p := req.Params(r) + + mediaID, err := p.String("mediaId") + if err != nil { + return nil, newError(responses.ErrorMissingParameter, "missing required parameter: mediaId") + } + + mediaType, err := p.String("mediaType") + if err != nil { + return nil, newError(responses.ErrorMissingParameter, "missing required parameter: mediaType") + } + + if !isValidMediaType(mediaType) { + return nil, newError(responses.ErrorGeneric, "mediaType '%s' is not yet supported", mediaType) + } + + // Parse and validate ClientInfo from request body (required per OpenSubsonic spec) + var clientInfoReq clientInfoRequest + r.Body = http.MaxBytesReader(w, r.Body, 1<<20) // 1 MB limit + if err := json.NewDecoder(r.Body).Decode(&clientInfoReq); err != nil { + return nil, newError(responses.ErrorGeneric, "invalid JSON request body") + } + if err := clientInfoReq.validate(); err != nil { + return nil, newError(responses.ErrorGeneric, "%v", err) + } + clientInfo := clientInfoReq.toCoreClientInfo() + + // Get media file + mf, err := api.ds.MediaFile(ctx).Get(mediaID) + if err != nil { + if errors.Is(err, model.ErrNotFound) { + return nil, newError(responses.ErrorDataNotFound, "media file not found: %s", mediaID) + } + log.Error(ctx, "Error retrieving media file", "mediaID", mediaID, err) + return nil, newError(responses.ErrorGeneric, "error retrieving media file") + } + + // Make the decision + decision, err := api.transcodeDecision.MakeDecision(ctx, mf, clientInfo, transcode.DecisionOptions{}) + if err != nil { + log.Error(ctx, "Failed to make transcode decision", "mediaID", mediaID, err) + return nil, newError(responses.ErrorGeneric, "failed to make transcode decision") + } + + // Only create a token when there is a valid playback path + var transcodeParams string + if decision.CanDirectPlay || decision.CanTranscode { + transcodeParams, err = api.transcodeDecision.CreateTranscodeParams(decision) + if err != nil { + log.Error(ctx, "Failed to create transcode token", "mediaID", mediaID, err) + return nil, newError(responses.ErrorGeneric, "failed to create transcode token") + } + } + + // Build response (convert kbps from core to bps for the API) + response := newResponse() + response.TranscodeDecision = &responses.TranscodeDecision{ + CanDirectPlay: decision.CanDirectPlay, + CanTranscode: decision.CanTranscode, + TranscodeReasons: decision.TranscodeReasons, + ErrorReason: decision.ErrorReason, + TranscodeParams: transcodeParams, + SourceStream: toResponseStreamDetails(&decision.SourceStream), + } + + if decision.TranscodeStream != nil { + response.TranscodeDecision.TranscodeStream = toResponseStreamDetails(decision.TranscodeStream) + } + + return response, nil +} + +// GetTranscodeStream handles the OpenSubsonic getTranscodeStream endpoint. +// It streams media using the decision encoded in the transcodeParams JWT token. +// All errors are returned as proper HTTP status codes (not Subsonic error responses). +func (api *Router) GetTranscodeStream(w http.ResponseWriter, r *http.Request) (*responses.Subsonic, error) { + ctx := r.Context() + p := req.Params(r) + + mediaID, err := p.String("mediaId") + if err != nil { + http.Error(w, "Bad Request", http.StatusBadRequest) + return nil, nil + } + + mediaType, err := p.String("mediaType") + if err != nil { + http.Error(w, "Bad Request", http.StatusBadRequest) + return nil, nil + } + + transcodeParamsToken, err := p.String("transcodeParams") + if err != nil { + http.Error(w, "Bad Request", http.StatusBadRequest) + return nil, nil + } + + if !isValidMediaType(mediaType) { + http.Error(w, "Bad Request", http.StatusBadRequest) + return nil, nil + } + + // Validate the token and resolve streaming parameters + streamReq, mf, err := api.transcodeDecision.ResolveRequestFromToken(ctx, transcodeParamsToken, mediaID, p.IntOr("offset", 0)) + if err != nil { + switch { + case errors.Is(err, transcode.ErrMediaNotFound): + http.Error(w, "Not Found", http.StatusNotFound) + case errors.Is(err, transcode.ErrTokenInvalid), errors.Is(err, transcode.ErrTokenStale): + http.Error(w, "Gone", http.StatusGone) + default: + log.Error(ctx, "Error validating transcode params", err) + http.Error(w, "Internal Server Error", http.StatusInternalServerError) + } + return nil, nil + } + + // Create stream (use DoStream to avoid duplicate DB fetch) + stream, err := api.streamer.DoStream(ctx, mf, streamReq) + if err != nil { + log.Error(ctx, "Error creating stream", "mediaID", mediaID, err) + http.Error(w, "Internal Server Error", http.StatusInternalServerError) + return nil, nil + } + + // Make sure the stream will be closed at the end + defer func() { + if err := stream.Close(); err != nil && log.IsGreaterOrEqualTo(log.LevelDebug) { + log.Error("Error closing stream", "id", mediaID, "file", stream.Name(), err) + } + }() + + w.Header().Set("X-Content-Type-Options", "nosniff") + + api.serveStream(ctx, w, r, stream, mediaID) + + return nil, nil +} diff --git a/server/subsonic/transcode_test.go b/server/subsonic/transcode_test.go new file mode 100644 index 000000000..717eeb1f5 --- /dev/null +++ b/server/subsonic/transcode_test.go @@ -0,0 +1,406 @@ +package subsonic + +import ( + "bytes" + "context" + "errors" + "net/http" + "net/http/httptest" + + "github.com/navidrome/navidrome/core/transcode" + "github.com/navidrome/navidrome/model" + "github.com/navidrome/navidrome/tests" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var _ = Describe("Transcode endpoints", func() { + var ( + router *Router + ds *tests.MockDataStore + mockTD *mockTranscodeDecision + w *httptest.ResponseRecorder + mockMFRepo *tests.MockMediaFileRepo + ) + + BeforeEach(func() { + mockMFRepo = &tests.MockMediaFileRepo{} + ds = &tests.MockDataStore{MockedMediaFile: mockMFRepo} + mockTD = &mockTranscodeDecision{} + router = New(ds, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, mockTD) + w = httptest.NewRecorder() + }) + + Describe("GetTranscodeDecision", func() { + It("returns 405 for non-POST requests", func() { + r := newGetRequest("mediaId=123", "mediaType=song") + resp, err := router.GetTranscodeDecision(w, r) + Expect(err).ToNot(HaveOccurred()) + Expect(resp).To(BeNil()) + Expect(w.Code).To(Equal(http.StatusMethodNotAllowed)) + Expect(w.Header().Get("Allow")).To(Equal("POST")) + }) + + It("returns error when mediaId is missing", func() { + r := newJSONPostRequest("mediaType=song", "{}") + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + }) + + It("returns error when mediaType is missing", func() { + r := newJSONPostRequest("mediaId=123", "{}") + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + }) + + It("returns error for unsupported mediaType", func() { + r := newJSONPostRequest("mediaId=123&mediaType=podcast", "{}") + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("not yet supported")) + }) + + It("returns ErrorDataNotFound when media file does not exist", func() { + // mockMFRepo has no data set, so Get() returns model.ErrNotFound + r := newJSONPostRequest("mediaId=nonexistent&mediaType=song", "{}") + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("media file not found")) + }) + + It("returns error when media file retrieval fails", func() { + mockMFRepo.SetError(true) + r := newJSONPostRequest("mediaId=song-1&mediaType=song", "{}") + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("error retrieving media file")) + }) + + It("returns error when body is empty", func() { + r := newJSONPostRequest("mediaId=song-1&mediaType=song", "") + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + }) + + It("returns error when body contains invalid JSON", func() { + r := newJSONPostRequest("mediaId=song-1&mediaType=song", "not-json{{{") + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + }) + + It("returns error for invalid protocol in direct play profile", func() { + body := `{"directPlayProfiles":[{"containers":["mp3"],"audioCodecs":["mp3"],"protocols":["ftp"]}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid protocol")) + }) + + It("returns error for invalid comparison operator", func() { + body := `{"codecProfiles":[{"type":"AudioCodec","name":"mp3","limitations":[{"name":"audioBitrate","comparison":"InvalidOp","values":["320"]}]}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid comparison")) + }) + + It("returns error for invalid limitation name", func() { + body := `{"codecProfiles":[{"type":"AudioCodec","name":"mp3","limitations":[{"name":"unknownField","comparison":"Equals","values":["320"]}]}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid limitation name")) + }) + + It("returns error for invalid codec profile type", func() { + body := `{"codecProfiles":[{"type":"VideoCodec","name":"mp3"}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid codec profile type")) + }) + + It("rejects wrong-case protocol", func() { + body := `{"directPlayProfiles":[{"containers":["mp3"],"audioCodecs":["mp3"],"protocols":["HTTP"]}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid protocol")) + }) + + It("rejects wrong-case codec profile type", func() { + body := `{"codecProfiles":[{"type":"audiocodec","name":"mp3"}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid codec profile type")) + }) + + It("rejects wrong-case comparison operator", func() { + body := `{"codecProfiles":[{"type":"AudioCodec","name":"mp3","limitations":[{"name":"audioBitrate","comparison":"lessthanequal","values":["320"]}]}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid comparison")) + }) + + It("rejects wrong-case limitation name", func() { + body := `{"codecProfiles":[{"type":"AudioCodec","name":"mp3","limitations":[{"name":"AudioBitrate","comparison":"Equals","values":["320"]}]}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + _, err := router.GetTranscodeDecision(w, r) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("invalid limitation name")) + }) + + It("returns a valid decision response", func() { + mockMFRepo.SetData(model.MediaFiles{ + {ID: "song-1", Suffix: "mp3", Codec: "MP3", BitRate: 320, Channels: 2, SampleRate: 44100}, + }) + mockTD.decision = &transcode.Decision{ + MediaID: "song-1", + CanDirectPlay: true, + SourceStream: transcode.StreamDetails{ + Container: "mp3", Codec: "mp3", Bitrate: 320, + SampleRate: 44100, Channels: 2, + }, + } + mockTD.token = "test-jwt-token" + + body := `{"directPlayProfiles":[{"containers":["mp3"],"protocols":["http"]}]}` + r := newJSONPostRequest("mediaId=song-1&mediaType=song", body) + resp, err := router.GetTranscodeDecision(w, r) + + Expect(err).ToNot(HaveOccurred()) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeParams).To(Equal("test-jwt-token")) + Expect(resp.TranscodeDecision.SourceStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.SourceStream.Protocol).To(Equal("http")) + Expect(resp.TranscodeDecision.SourceStream.Container).To(Equal("mp3")) + Expect(resp.TranscodeDecision.SourceStream.AudioBitrate).To(Equal(int32(320_000))) + }) + + It("includes transcode stream when transcoding", func() { + mockMFRepo.SetData(model.MediaFiles{ + {ID: "song-2", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 96000, BitDepth: 24}, + }) + mockTD.decision = &transcode.Decision{ + MediaID: "song-2", + CanDirectPlay: false, + CanTranscode: true, + TargetFormat: "mp3", + TargetBitrate: 256, + TranscodeReasons: []string{"container not supported"}, + SourceStream: transcode.StreamDetails{ + Container: "flac", Codec: "flac", Bitrate: 1000, + SampleRate: 96000, BitDepth: 24, Channels: 2, + }, + TranscodeStream: &transcode.StreamDetails{ + Container: "mp3", Codec: "mp3", Bitrate: 256, + SampleRate: 96000, Channels: 2, + }, + } + mockTD.token = "transcode-token" + + r := newJSONPostRequest("mediaId=song-2&mediaType=song", "{}") + resp, err := router.GetTranscodeDecision(w, r) + + Expect(err).ToNot(HaveOccurred()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeReasons).To(ConsistOf("container not supported")) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Container).To(Equal("mp3")) + }) + }) + + Describe("GetTranscodeStream", func() { + It("returns 400 when mediaId is missing", func() { + r := newGetRequest("mediaType=song", "transcodeParams=abc") + resp, err := router.GetTranscodeStream(w, r) + Expect(err).ToNot(HaveOccurred()) + Expect(resp).To(BeNil()) + Expect(w.Code).To(Equal(http.StatusBadRequest)) + }) + + It("returns 400 when transcodeParams is missing", func() { + r := newGetRequest("mediaId=123", "mediaType=song") + resp, err := router.GetTranscodeStream(w, r) + Expect(err).ToNot(HaveOccurred()) + Expect(resp).To(BeNil()) + Expect(w.Code).To(Equal(http.StatusBadRequest)) + }) + + It("returns 410 for invalid or mismatched token", func() { + mockTD.resolveErr = transcode.ErrTokenInvalid + r := newGetRequest("mediaId=123", "mediaType=song", "transcodeParams=bad-token") + resp, err := router.GetTranscodeStream(w, r) + Expect(err).ToNot(HaveOccurred()) + Expect(resp).To(BeNil()) + Expect(w.Code).To(Equal(http.StatusGone)) + }) + + It("returns 404 when media file not found", func() { + mockTD.resolveErr = transcode.ErrMediaNotFound + r := newGetRequest("mediaId=gone-id", "mediaType=song", "transcodeParams=valid-token") + resp, err := router.GetTranscodeStream(w, r) + Expect(err).ToNot(HaveOccurred()) + Expect(resp).To(BeNil()) + Expect(w.Code).To(Equal(http.StatusNotFound)) + }) + + It("returns 410 when media file has changed (stale token)", func() { + mockTD.resolveErr = transcode.ErrTokenStale + r := newGetRequest("mediaId=song-1", "mediaType=song", "transcodeParams=stale-token") + resp, err := router.GetTranscodeStream(w, r) + Expect(err).ToNot(HaveOccurred()) + Expect(resp).To(BeNil()) + Expect(w.Code).To(Equal(http.StatusGone)) + }) + + It("builds correct StreamRequest for direct play", func() { + fakeStreamer := &fakeMediaStreamer{} + router = New(ds, nil, fakeStreamer, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, mockTD) + mockTD.resolvedReq = transcode.StreamRequest{ID: "song-1"} + mockTD.resolvedMF = &model.MediaFile{ID: "song-1"} + + r := newGetRequest("mediaId=song-1", "mediaType=song", "transcodeParams=valid-token") + _, _ = router.GetTranscodeStream(w, r) + + Expect(fakeStreamer.captured).ToNot(BeNil()) + Expect(fakeStreamer.captured.ID).To(Equal("song-1")) + Expect(fakeStreamer.captured.Format).To(BeEmpty()) + Expect(fakeStreamer.captured.BitRate).To(BeZero()) + Expect(fakeStreamer.captured.SampleRate).To(BeZero()) + Expect(fakeStreamer.captured.BitDepth).To(BeZero()) + Expect(fakeStreamer.captured.Channels).To(BeZero()) + }) + + It("builds correct StreamRequest for transcoding", func() { + fakeStreamer := &fakeMediaStreamer{} + router = New(ds, nil, fakeStreamer, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil, mockTD) + mockTD.resolvedReq = transcode.StreamRequest{ + ID: "song-2", + Format: "mp3", + BitRate: 256, + SampleRate: 44100, + BitDepth: 16, + Channels: 2, + } + mockTD.resolvedMF = &model.MediaFile{ID: "song-2"} + + r := newGetRequest("mediaId=song-2", "mediaType=song", "transcodeParams=valid-token", "offset=10") + _, _ = router.GetTranscodeStream(w, r) + + Expect(fakeStreamer.captured).ToNot(BeNil()) + Expect(fakeStreamer.captured.ID).To(Equal("song-2")) + Expect(fakeStreamer.captured.Format).To(Equal("mp3")) + Expect(fakeStreamer.captured.BitRate).To(Equal(256)) + Expect(fakeStreamer.captured.SampleRate).To(Equal(44100)) + Expect(fakeStreamer.captured.BitDepth).To(Equal(16)) + Expect(fakeStreamer.captured.Channels).To(Equal(2)) + Expect(fakeStreamer.captured.Offset).To(Equal(10)) + }) + }) + + Describe("bpsToKbps", func() { + It("converts standard bitrates", func() { + Expect(bpsToKbps(128000)).To(Equal(128)) + Expect(bpsToKbps(320000)).To(Equal(320)) + Expect(bpsToKbps(256000)).To(Equal(256)) + }) + It("returns 0 for 0", func() { + Expect(bpsToKbps(0)).To(Equal(0)) + }) + It("rounds instead of truncating", func() { + Expect(bpsToKbps(999)).To(Equal(1)) + Expect(bpsToKbps(500)).To(Equal(1)) + Expect(bpsToKbps(499)).To(Equal(0)) + }) + It("returns 0 for negative values", func() { + Expect(bpsToKbps(-1)).To(Equal(0)) + Expect(bpsToKbps(-1000)).To(Equal(0)) + Expect(bpsToKbps(-1000000)).To(Equal(0)) + }) + }) + + Describe("kbpsToBps", func() { + It("converts standard bitrates", func() { + Expect(kbpsToBps(128)).To(Equal(128000)) + Expect(kbpsToBps(320)).To(Equal(320000)) + }) + It("returns 0 for 0", func() { + Expect(kbpsToBps(0)).To(Equal(0)) + }) + }) + + Describe("convertBitrateValues", func() { + It("converts valid bps strings to kbps", func() { + Expect(convertBitrateValues([]string{"128000", "320000"})).To(Equal([]string{"128", "320"})) + }) + It("preserves unparseable values", func() { + Expect(convertBitrateValues([]string{"128000", "bad", "320000"})).To(Equal([]string{"128", "bad", "320"})) + }) + It("handles empty slice", func() { + Expect(convertBitrateValues([]string{})).To(Equal([]string{})) + }) + }) +}) + +// newJSONPostRequest creates an HTTP POST request with JSON body and query params +func newJSONPostRequest(queryParams string, jsonBody string) *http.Request { + r := httptest.NewRequest("POST", "/getTranscodeDecision?"+queryParams, bytes.NewBufferString(jsonBody)) + r.Header.Set("Content-Type", "application/json") + return r +} + +// mockTranscodeDecision is a test double for transcode.Decider +type mockTranscodeDecision struct { + decision *transcode.Decision + token string + tokenErr error + resolvedReq transcode.StreamRequest + resolvedMF *model.MediaFile + resolveErr error +} + +func (m *mockTranscodeDecision) MakeDecision(_ context.Context, _ *model.MediaFile, _ *transcode.ClientInfo, _ transcode.DecisionOptions) (*transcode.Decision, error) { + if m.decision != nil { + return m.decision, nil + } + return &transcode.Decision{}, nil +} + +func (m *mockTranscodeDecision) ResolveRequest(_ context.Context, _ *model.MediaFile, _ string, _ int, _ int) transcode.StreamRequest { + return transcode.StreamRequest{Format: "raw"} +} + +func (m *mockTranscodeDecision) CreateTranscodeParams(_ *transcode.Decision) (string, error) { + return m.token, m.tokenErr +} + +func (m *mockTranscodeDecision) ResolveRequestFromToken(_ context.Context, _ string, _ string, offset int) (transcode.StreamRequest, *model.MediaFile, error) { + if m.resolveErr != nil { + return transcode.StreamRequest{}, nil, m.resolveErr + } + req := m.resolvedReq + req.Offset = offset + return req, m.resolvedMF, nil +} + +// fakeMediaStreamer captures the StreamRequest and returns a sentinel error, +// allowing tests to verify parameter passing without constructing a real Stream. +var errStreamCaptured = errors.New("stream request captured") + +type fakeMediaStreamer struct { + captured *transcode.StreamRequest +} + +func (f *fakeMediaStreamer) NewStream(_ context.Context, req transcode.StreamRequest) (*transcode.Stream, error) { + f.captured = &req + return nil, errStreamCaptured +} + +func (f *fakeMediaStreamer) DoStream(_ context.Context, _ *model.MediaFile, req transcode.StreamRequest) (*transcode.Stream, error) { + f.captured = &req + return nil, errStreamCaptured +} diff --git a/tests/mock_ffmpeg.go b/tests/mock_ffmpeg.go index a792ae9d3..a35defeae 100644 --- a/tests/mock_ffmpeg.go +++ b/tests/mock_ffmpeg.go @@ -6,6 +6,8 @@ import ( "strings" "sync" "sync/atomic" + + "github.com/navidrome/navidrome/core/ffmpeg" ) func NewMockFFmpeg(data string) *MockFFmpeg { @@ -14,16 +16,17 @@ func NewMockFFmpeg(data string) *MockFFmpeg { type MockFFmpeg struct { io.Reader - lock sync.Mutex - closed atomic.Bool - Error error + lock sync.Mutex + closed atomic.Bool + Error error + ProbeAudioResult *ffmpeg.AudioProbeResult } func (ff *MockFFmpeg) IsAvailable() bool { return true } -func (ff *MockFFmpeg) Transcode(context.Context, string, string, int, int) (io.ReadCloser, error) { +func (ff *MockFFmpeg) Transcode(_ context.Context, _ ffmpeg.TranscodeOptions) (io.ReadCloser, error) { if ff.Error != nil { return nil, ff.Error } @@ -43,6 +46,13 @@ func (ff *MockFFmpeg) Probe(context.Context, []string) (string, error) { } return "", nil } +func (ff *MockFFmpeg) ProbeAudioStream(context.Context, string) (*ffmpeg.AudioProbeResult, error) { + if ff.Error != nil { + return nil, ff.Error + } + return ff.ProbeAudioResult, nil +} + func (ff *MockFFmpeg) CmdPath() (string, error) { if ff.Error != nil { return "", ff.Error diff --git a/tests/mock_mediafile_repo.go b/tests/mock_mediafile_repo.go index 1d4527c88..01eacae30 100644 --- a/tests/mock_mediafile_repo.go +++ b/tests/mock_mediafile_repo.go @@ -109,6 +109,17 @@ func (m *MockMediaFileRepo) Put(mf *model.MediaFile) error { return nil } +func (m *MockMediaFileRepo) UpdateProbeData(id string, data string) error { + if m.Err { + return errors.New("error") + } + if d, ok := m.Data[id]; ok { + d.ProbeData = data + return nil + } + return model.ErrNotFound +} + func (m *MockMediaFileRepo) Delete(id string) error { if m.Err { return errors.New("error") diff --git a/tests/mock_transcoding_repo.go b/tests/mock_transcoding_repo.go index 12db0d7be..796e84111 100644 --- a/tests/mock_transcoding_repo.go +++ b/tests/mock_transcoding_repo.go @@ -18,6 +18,10 @@ func (m *MockTranscodingRepo) FindByFormat(format string) (*model.Transcoding, e return &model.Transcoding{ID: "oga1", TargetFormat: "oga", DefaultBitRate: 128}, nil case "opus": return &model.Transcoding{ID: "opus1", TargetFormat: "opus", DefaultBitRate: 96}, nil + case "flac": + return &model.Transcoding{ID: "flac1", TargetFormat: "flac", DefaultBitRate: 0, Command: "ffmpeg -i %s -ss %t -map 0:a:0 -v 0 -c:a flac -f flac -"}, nil + case "aac": + return &model.Transcoding{ID: "aac1", TargetFormat: "aac", DefaultBitRate: 256, Command: "ffmpeg -i %s -ss %t -map 0:a:0 -b:a %bk -v 0 -c:a aac -f ipod -movflags frag_keyframe+empty_moov -"}, nil default: return nil, model.ErrNotFound } From 928741ef253645f61c81a7cccdaeb2e1eb18adad Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 9 Mar 2026 08:06:06 -0400 Subject: [PATCH 50/60] fix(db): recreate probe_data column as NOT NULL with empty string default The probe_data column was added with DEFAULT NULL in migration 20260307175815, which causes sql.Scan errors when reading into Go string fields. This migration drops and recreates the column with DEFAULT '' NOT NULL to prevent NULL scan errors. --- .../20260309120007_fix_probe_data_null.go | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 db/migrations/20260309120007_fix_probe_data_null.go diff --git a/db/migrations/20260309120007_fix_probe_data_null.go b/db/migrations/20260309120007_fix_probe_data_null.go new file mode 100644 index 000000000..a7e7366ed --- /dev/null +++ b/db/migrations/20260309120007_fix_probe_data_null.go @@ -0,0 +1,28 @@ +package migrations + +import ( + "context" + "database/sql" + + "github.com/pressly/goose/v3" +) + +func init() { + goose.AddMigrationContext(upFixProbeDataNull, downFixProbeDataNull) +} + +func upFixProbeDataNull(_ context.Context, tx *sql.Tx) error { + // Recreate probe_data column as NOT NULL with empty string default. + // The previous migration created it with DEFAULT NULL, which causes + // scan errors when reading into Go string fields. + _, err := tx.Exec(`ALTER TABLE media_file DROP COLUMN probe_data`) + if err != nil { + return err + } + _, err = tx.Exec(`ALTER TABLE media_file ADD COLUMN probe_data TEXT DEFAULT '' NOT NULL`) + return err +} + +func downFixProbeDataNull(_ context.Context, tx *sql.Tx) error { + return nil +} From 7c5aa1fafaae0b4cc3af7fe1958ad9e1cc9bf943 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 9 Mar 2026 09:43:55 -0400 Subject: [PATCH 51/60] test(e2e): add transcode endpoint e2e tests and clean up test helpers Add comprehensive e2e tests for getTranscodeDecision and getTranscodeStream endpoints covering direct play, transcoding, error handling, and round-trip token validation. Refactor buildPostReq to reuse buildReq for auth params, remove unused WAV/AAC test tracks, and consolidate duplicate test assertions. --- core/storage/storagetest/fake_storage.go | 3 + server/e2e/e2e_suite_test.go | 57 +++ server/e2e/subsonic_album_lists_test.go | 24 +- server/e2e/subsonic_browsing_test.go | 2 +- server/e2e/subsonic_multilibrary_test.go | 2 +- server/e2e/subsonic_searching_test.go | 6 +- server/e2e/subsonic_transcode_test.go | 464 +++++++++++++++++++++++ 7 files changed, 542 insertions(+), 16 deletions(-) create mode 100644 server/e2e/subsonic_transcode_test.go diff --git a/core/storage/storagetest/fake_storage.go b/core/storage/storagetest/fake_storage.go index 79ed3193d..1b0d1a6c1 100644 --- a/core/storage/storagetest/fake_storage.go +++ b/core/storage/storagetest/fake_storage.go @@ -284,6 +284,9 @@ func (ffs *FakeFS) parseFile(filePath string) (*metadata.Info, error) { p.AudioProperties.BitDepth = getInt("bitdepth") p.AudioProperties.SampleRate = getInt("samplerate") p.AudioProperties.Channels = getInt("channels") + if codec, ok := data["codec"].(string); ok { + p.AudioProperties.Codec = codec + } for k, v := range data { p.Tags[k] = []string{fmt.Sprintf("%v", v)} } diff --git a/server/e2e/e2e_suite_test.go b/server/e2e/e2e_suite_test.go index 02b66f4b7..e55130ffd 100644 --- a/server/e2e/e2e_suite_test.go +++ b/server/e2e/e2e_suite_test.go @@ -1,6 +1,7 @@ package e2e import ( + "bytes" "context" "encoding/json" "errors" @@ -55,6 +56,7 @@ type _t = map[string]any var template = storagetest.Template var track = storagetest.Track +var file = storagetest.File // MusicBrainz ID constants for test data (valid UUID v4 values) const ( @@ -122,6 +124,9 @@ func buildTestFS() storagetest.FakeFS { popTrack := template(_t{"albumartist": "Various", "artist": "Various", "album": "Pop", "year": 2020, "genre": "Pop"}) cowboyBebop := template(_t{"albumartist": "シートベルツ", "artist": "シートベルツ", "album": "COWBOY BEBOP", "year": 1998, "genre": "Jazz"}) + // Template for diverse-format transcode test tracks + tcBase := _t{"albumartist": "Test Artist", "artist": "Test Artist", "album": "Transcode Formats", "year": 2024, "genre": "Test"} + return createFS(fstest.MapFS{ // Rock / The Beatles / Abbey Road (with MBIDs) // Note: "musicbrainz_trackid" is an alias for the musicbrainz_recordingid tag (populates MbzRecordingID), @@ -140,6 +145,33 @@ func buildTestFS() storagetest.FakeFS { "Pop/01 - Standalone Track.mp3": popTrack(track(1, "Standalone Track")), // CJK / シートベルツ / COWBOY BEBOP (Japanese artist, for CJK search tests) "CJK/シートベルツ/COWBOY BEBOP/01 - プラチナ・ジェット.mp3": cowboyBebop(track(1, "プラチナ・ジェット")), + + // Diverse audio format tracks for transcode e2e tests + "Test/Transcode Formats/01 - TC FLAC Standard.flac": file(tcBase, _t{ + "title": "TC FLAC Standard", "track": 1, "suffix": "flac", + "bitrate": 900, "samplerate": 44100, "bitdepth": 16, "channels": 2, "duration": int64(240), + }), + "Test/Transcode Formats/02 - TC FLAC HiRes.flac": file(tcBase, _t{ + "title": "TC FLAC HiRes", "track": 2, "suffix": "flac", + "bitrate": 3000, "samplerate": 96000, "bitdepth": 24, "channels": 2, "duration": int64(180), + }), + "Test/Transcode Formats/03 - TC ALAC Track.m4a": file(tcBase, _t{ + "title": "TC ALAC Track", "track": 3, "suffix": "m4a", + "bitrate": 900, "samplerate": 44100, "bitdepth": 16, "channels": 2, "duration": int64(200), + }), + "Test/Transcode Formats/04 - TC DSD Track.dsf": file(tcBase, _t{ + "title": "TC DSD Track", "track": 4, "suffix": "dsf", + "bitrate": 5645, "samplerate": 2822400, "bitdepth": 1, "channels": 2, "duration": int64(300), + }), + "Test/Transcode Formats/05 - TC Opus Track.opus": file(tcBase, _t{ + "title": "TC Opus Track", "track": 5, "suffix": "opus", + "bitrate": 128, "samplerate": 48000, "bitdepth": 0, "channels": 2, "duration": int64(210), + }), + "Test/Transcode Formats/06 - TC MKA Opus.mka": file(tcBase, _t{ + "title": "TC MKA Opus", "track": 6, "suffix": "mka", "codec": "opus", + "bitrate": 128, "samplerate": 48000, "bitdepth": 0, "channels": 2, "duration": int64(220), + }), + // _empty folder (directory with no audio) "_empty/.keep": &fstest.MapFile{Data: []byte{}, ModTime: time.Now()}, }) @@ -207,6 +239,30 @@ func buildReq(user model.User, endpoint string, params ...string) *http.Request return httptest.NewRequest("GET", "/"+endpoint+"?"+q.Encode(), nil) } +// buildPostReq creates a POST request with a JSON body and Subsonic auth params in the query string. +func buildPostReq(user model.User, endpoint string, body string, params ...string) *http.Request { + getReq := buildReq(user, endpoint, params...) + r := httptest.NewRequest("POST", getReq.URL.RequestURI(), bytes.NewReader([]byte(body))) + r.Header.Set("Content-Type", "application/json") + return r +} + +// doPostReq makes a POST round-trip as admin and returns the parsed Subsonic response. +func doPostReq(endpoint string, body string, params ...string) *responses.Subsonic { + w := httptest.NewRecorder() + r := buildPostReq(adminUser, endpoint, body, params...) + router.ServeHTTP(w, r) + return parseJSONResponse(w) +} + +// doRawPostReq makes a POST round-trip as admin and returns the raw recorder. +func doRawPostReq(endpoint string, body string, params ...string) *httptest.ResponseRecorder { + w := httptest.NewRecorder() + r := buildPostReq(adminUser, endpoint, body, params...) + router.ServeHTTP(w, r) + return w +} + // parseJSONResponse parses the JSON response body into a Subsonic response struct. func parseJSONResponse(w *httptest.ResponseRecorder) *responses.Subsonic { Expect(w.Code).To(Equal(http.StatusOK)) @@ -411,6 +467,7 @@ func setupTestDB() { }) conf.Server.MusicFolder = "fake:///music" conf.Server.DevExternalScanner = false + conf.Server.DevEnableMediaFileProbe = false // Restore DB to golden state (no scan needed) restoreDB() diff --git a/server/e2e/subsonic_album_lists_test.go b/server/e2e/subsonic_album_lists_test.go index d3d24a7c5..d41d17dbc 100644 --- a/server/e2e/subsonic_album_lists_test.go +++ b/server/e2e/subsonic_album_lists_test.go @@ -19,7 +19,7 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.AlbumList).ToNot(BeNil()) - Expect(resp.AlbumList.Album).To(HaveLen(6)) + Expect(resp.AlbumList.Album).To(HaveLen(7)) }) It("type=alphabeticalByName sorts albums by name", func() { @@ -27,14 +27,15 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.AlbumList).ToNot(BeNil()) albums := resp.AlbumList.Album - Expect(albums).To(HaveLen(6)) - // Verify alphabetical order: Abbey Road, COWBOY BEBOP, Help!, IV, Kind of Blue, Pop + Expect(albums).To(HaveLen(7)) + // Verify alphabetical order: Abbey Road, COWBOY BEBOP, Help!, IV, Kind of Blue, Pop, Transcode Formats Expect(albums[0].Title).To(Equal("Abbey Road")) Expect(albums[1].Title).To(Equal("COWBOY BEBOP")) Expect(albums[2].Title).To(Equal("Help!")) Expect(albums[3].Title).To(Equal("IV")) Expect(albums[4].Title).To(Equal("Kind of Blue")) Expect(albums[5].Title).To(Equal("Pop")) + Expect(albums[6].Title).To(Equal("Transcode Formats")) }) It("type=alphabeticalByArtist sorts albums by artist name", func() { @@ -42,22 +43,23 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.AlbumList).ToNot(BeNil()) albums := resp.AlbumList.Album - Expect(albums).To(HaveLen(6)) + Expect(albums).To(HaveLen(7)) // Articles like "The" are stripped for sorting, so "The Beatles" sorts as "Beatles" - // Non-compilations first: Beatles (x2), Led Zeppelin, Miles Davis, then compilations: Various, then CJK: シートベルツ + // Non-compilations first: Beatles (x2), Led Zeppelin, Miles Davis, Test Artist, then compilations: Various, then CJK: シートベルツ Expect(albums[0].Artist).To(Equal("The Beatles")) Expect(albums[1].Artist).To(Equal("The Beatles")) Expect(albums[2].Artist).To(Equal("Led Zeppelin")) Expect(albums[3].Artist).To(Equal("Miles Davis")) - Expect(albums[4].Artist).To(Equal("Various")) - Expect(albums[5].Artist).To(Equal("シートベルツ")) + Expect(albums[4].Artist).To(Equal("Test Artist")) + Expect(albums[5].Artist).To(Equal("Various")) + Expect(albums[6].Artist).To(Equal("シートベルツ")) }) It("type=random returns albums", func() { resp := doReq("getAlbumList", "type", "random") Expect(resp.AlbumList).ToNot(BeNil()) - Expect(resp.AlbumList.Album).To(HaveLen(6)) + Expect(resp.AlbumList.Album).To(HaveLen(7)) }) It("type=byGenre filters by genre parameter", func() { @@ -188,7 +190,7 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.AlbumList2).ToNot(BeNil()) albums := resp.AlbumList2.Album - Expect(albums).To(HaveLen(6)) + Expect(albums).To(HaveLen(7)) // Verify AlbumID3 format fields Expect(albums[0].Name).To(Equal("Abbey Road")) Expect(albums[0].Id).ToNot(BeEmpty()) @@ -199,7 +201,7 @@ var _ = Describe("Album List Endpoints", func() { resp := doReq("getAlbumList2", "type", "newest") Expect(resp.AlbumList2).ToNot(BeNil()) - Expect(resp.AlbumList2.Album).To(HaveLen(6)) + Expect(resp.AlbumList2.Album).To(HaveLen(7)) }) }) @@ -244,7 +246,7 @@ var _ = Describe("Album List Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.RandomSongs).ToNot(BeNil()) Expect(resp.RandomSongs.Songs).ToNot(BeEmpty()) - Expect(resp.RandomSongs.Songs).To(HaveLen(7)) + Expect(resp.RandomSongs.Songs).To(HaveLen(10)) }) It("respects size parameter", func() { diff --git a/server/e2e/subsonic_browsing_test.go b/server/e2e/subsonic_browsing_test.go index 5a2da8737..55aeb8e9e 100644 --- a/server/e2e/subsonic_browsing_test.go +++ b/server/e2e/subsonic_browsing_test.go @@ -288,7 +288,7 @@ var _ = Describe("Browsing Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.Genres).ToNot(BeNil()) - Expect(resp.Genres.Genre).To(HaveLen(3)) + Expect(resp.Genres.Genre).To(HaveLen(4)) }) It("includes correct genre names", func() { diff --git a/server/e2e/subsonic_multilibrary_test.go b/server/e2e/subsonic_multilibrary_test.go index aa4a0c626..f59187d00 100644 --- a/server/e2e/subsonic_multilibrary_test.go +++ b/server/e2e/subsonic_multilibrary_test.go @@ -141,7 +141,7 @@ var _ = Describe("Multi-Library Support", Ordered, func() { resp := doReqWithUser(adminWithLibs, "getAlbumList", "type", "alphabeticalByName", "musicFolderId", fmt.Sprintf("%d", lib.ID)) Expect(resp.AlbumList).ToNot(BeNil()) - Expect(resp.AlbumList.Album).To(HaveLen(6)) + Expect(resp.AlbumList.Album).To(HaveLen(7)) for _, a := range resp.AlbumList.Album { Expect(a.Title).ToNot(Equal("Symphony No. 9")) } diff --git a/server/e2e/subsonic_searching_test.go b/server/e2e/subsonic_searching_test.go index bfcbbc8ee..7f6aaf57a 100644 --- a/server/e2e/subsonic_searching_test.go +++ b/server/e2e/subsonic_searching_test.go @@ -115,9 +115,9 @@ var _ = Describe("Search Endpoints", func() { Expect(resp.Status).To(Equal(responses.StatusOK)) Expect(resp.SearchResult3).ToNot(BeNil()) - Expect(resp.SearchResult3.Artist).To(HaveLen(5)) - Expect(resp.SearchResult3.Album).To(HaveLen(6)) - Expect(resp.SearchResult3.Song).To(HaveLen(7)) + Expect(resp.SearchResult3.Artist).To(HaveLen(6)) + Expect(resp.SearchResult3.Album).To(HaveLen(7)) + Expect(resp.SearchResult3.Song).To(HaveLen(13)) }) It("finds across all entity types simultaneously", func() { diff --git a/server/e2e/subsonic_transcode_test.go b/server/e2e/subsonic_transcode_test.go new file mode 100644 index 000000000..16a884bf3 --- /dev/null +++ b/server/e2e/subsonic_transcode_test.go @@ -0,0 +1,464 @@ +package e2e + +import ( + "net/http" + "time" + + "github.com/navidrome/navidrome/server/subsonic/responses" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +// Client profile JSON bodies for getTranscodeDecision requests. +// All bitrate values are in bps (per OpenSubsonic spec). +const ( + // mp3OnlyClient can direct-play mp3 and transcode to mp3 + mp3OnlyClient = `{ + "name": "test-mp3-only", + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "mp3", "audioCodec": "mp3", "protocol": "http"} + ] + }` + + // flacAndMp3Client can direct-play flac and mp3, transcode to mp3 + flacAndMp3Client = `{ + "name": "test-flac-mp3", + "directPlayProfiles": [ + {"containers": ["flac"], "audioCodecs": ["flac"], "protocols": ["http"]}, + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "mp3", "audioCodec": "mp3", "protocol": "http"} + ] + }` + + // universalClient can direct-play most formats + universalClient = `{ + "name": "test-universal", + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]}, + {"containers": ["flac"], "audioCodecs": ["flac"], "protocols": ["http"]}, + {"containers": ["m4a"], "audioCodecs": ["alac", "aac"], "protocols": ["http"]}, + {"containers": ["opus", "ogg"], "audioCodecs": ["opus"], "protocols": ["http"]}, + {"containers": ["wav"], "audioCodecs": ["pcm"], "protocols": ["http"]}, + {"containers": ["dsf"], "audioCodecs": ["dsd"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "mp3", "audioCodec": "mp3", "protocol": "http"} + ] + }` + + // bitrateCapClient has maxAudioBitrate set to 320000 bps (320 kbps) + bitrateCapClient = `{ + "name": "test-bitrate-cap", + "maxAudioBitrate": 320000, + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]}, + {"containers": ["flac"], "audioCodecs": ["flac"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "mp3", "audioCodec": "mp3", "protocol": "http"} + ] + }` + + // opusTranscodeClient can direct-play mp3, transcode to opus + opusTranscodeClient = `{ + "name": "test-opus-transcode", + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "opus", "audioCodec": "opus", "protocol": "http"} + ] + }` + + // flacOnlyClient can direct-play flac, transcode to flac (no mp3 support at all) + flacOnlyClient = `{ + "name": "test-flac-only", + "directPlayProfiles": [ + {"containers": ["flac"], "audioCodecs": ["flac"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "flac", "audioCodec": "flac", "protocol": "http"} + ] + }` + + // maxTranscodeBitrateClient has maxTranscodingAudioBitrate set + maxTranscodeBitrateClient = `{ + "name": "test-max-transcode-bitrate", + "maxTranscodingAudioBitrate": 192000, + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "mp3", "audioCodec": "mp3", "protocol": "http"} + ] + }` + + // dsdToFlacClient can direct-play mp3, transcode to flac + dsdToFlacClient = `{ + "name": "test-dsd-to-flac", + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]} + ], + "transcodingProfiles": [ + {"container": "flac", "audioCodec": "flac", "protocol": "http"} + ] + }` +) + +var _ = Describe("Transcode Endpoints", Ordered, func() { + // Track IDs resolved in BeforeAll + var ( + mp3TrackID string // Come Together (mp3, 320kbps) + flacTrackID string // TC FLAC Standard (flac, 900kbps) + flacHiResTrackID string // TC FLAC HiRes (flac, 3000kbps) + alacTrackID string // TC ALAC Track (m4a, alac) + dsdTrackID string // TC DSD Track (dsf, dsd) + opusTrackID string // TC Opus Track (opus, 128kbps) + mkaOpusTrackID string // TC MKA Opus (mka, opus via codec tag) + ) + + BeforeAll(func() { + setupTestDB() + + songs, err := ds.MediaFile(ctx).GetAll() + Expect(err).ToNot(HaveOccurred()) + byTitle := map[string]string{} + for _, s := range songs { + byTitle[s.Title] = s.ID + } + ensureGetTrackID := func(title string) string { + id := byTitle[title] + Expect(id).ToNot(BeEmpty()) + return id + } + mp3TrackID = ensureGetTrackID("Come Together") + flacTrackID = ensureGetTrackID("TC FLAC Standard") + flacHiResTrackID = ensureGetTrackID("TC FLAC HiRes") + alacTrackID = ensureGetTrackID("TC ALAC Track") + dsdTrackID = ensureGetTrackID("TC DSD Track") + opusTrackID = ensureGetTrackID("TC Opus Track") + mkaOpusTrackID = ensureGetTrackID("TC MKA Opus") + }) + + Describe("getTranscodeDecision", func() { + Describe("error cases", func() { + It("returns 405 for GET request", func() { + w := doRawReq("getTranscodeDecision", "mediaId", mp3TrackID, "mediaType", "song") + Expect(w.Code).To(Equal(http.StatusMethodNotAllowed)) + }) + + It("returns error when mediaId is missing", func() { + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(responses.ErrorMissingParameter)) + }) + + It("returns error when mediaType is missing", func() { + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mp3TrackID) + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(responses.ErrorMissingParameter)) + }) + + It("returns error for unsupported mediaType", func() { + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mp3TrackID, "mediaType", "video") + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(responses.ErrorGeneric)) + }) + + It("returns error for invalid JSON body", func() { + resp := doPostReq("getTranscodeDecision", "{invalid-json", "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + + It("returns error for empty JSON body", func() { + w := doRawPostReq("getTranscodeDecision", "", "mediaId", mp3TrackID, "mediaType", "song") + Expect(w.Code).To(Equal(http.StatusOK)) // Subsonic errors are returned as 200 with error status + resp := parseJSONResponse(w) + Expect(resp.Status).To(Equal(responses.StatusFailed)) + }) + + It("returns error for non-existent media ID", func() { + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", "non-existent-id", "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + Expect(resp.Error.Code).To(Equal(responses.ErrorDataNotFound)) + }) + + It("returns error for invalid protocol in body", func() { + invalidBody := `{ + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["invalid-protocol"]} + ] + }` + resp := doPostReq("getTranscodeDecision", invalidBody, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + + It("returns error for invalid comparison operator in body", func() { + invalidBody := `{ + "directPlayProfiles": [ + {"containers": ["mp3"], "audioCodecs": ["mp3"], "protocols": ["http"]} + ], + "codecProfiles": [{ + "type": "AudioCodec", "name": "mp3", + "limitations": [{"name": "audioBitrate", "comparison": "InvalidOp", "values": ["320000"]}] + }] + }` + resp := doPostReq("getTranscodeDecision", invalidBody, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusFailed)) + Expect(resp.Error).ToNot(BeNil()) + }) + }) + + Describe("direct play decisions", func() { + It("allows MP3 direct play when client supports mp3", func() { + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).To(BeNil()) + Expect(resp.TranscodeDecision.TranscodeParams).ToNot(BeEmpty()) + }) + + It("allows FLAC direct play when client supports flac", func() { + resp := doPostReq("getTranscodeDecision", flacAndMp3Client, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + }) + + It("allows ALAC direct play via m4a container + alac codec matching", func() { + resp := doPostReq("getTranscodeDecision", universalClient, "mediaId", alacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + }) + + It("allows Opus direct play when client supports opus", func() { + resp := doPostReq("getTranscodeDecision", universalClient, "mediaId", opusTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + }) + + It("denies direct play when container mismatches", func() { + // mp3OnlyClient cannot play FLAC container + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeFalse()) + }) + + It("denies direct play when codec mismatches", func() { + // MKA container with opus codec — client only supports mp3 + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mkaOpusTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeFalse()) + }) + + It("denies direct play when maxAudioBitrate exceeded", func() { + // bitrateCapClient caps at 320kbps, FLAC is 900kbps + resp := doPostReq("getTranscodeDecision", bitrateCapClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeFalse()) + }) + }) + + Describe("transcode decisions", func() { + It("transcodes FLAC to MP3 when client only supports MP3", func() { + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeFalse()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Container).To(Equal("mp3")) + Expect(resp.TranscodeDecision.TranscodeStream.Codec).To(Equal("mp3")) + Expect(resp.TranscodeDecision.TranscodeParams).ToNot(BeEmpty()) + }) + + It("transcodes FLAC hi-res to Opus with correct sample rate", func() { + resp := doPostReq("getTranscodeDecision", opusTranscodeClient, "mediaId", flacHiResTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Codec).To(Equal("opus")) + // Opus always outputs 48000 Hz + Expect(resp.TranscodeDecision.TranscodeStream.AudioSamplerate).To(Equal(int32(48000))) + }) + + It("transcodes DSD to FLAC with normalized sample rate and bit depth", func() { + resp := doPostReq("getTranscodeDecision", dsdToFlacClient, "mediaId", dsdTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Codec).To(Equal("flac")) + // DSD sample rate normalized: 2822400 / 8 = 352800 + Expect(resp.TranscodeDecision.TranscodeStream.AudioSamplerate).To(Equal(int32(352800))) + // DSD 1-bit → 24-bit PCM + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitdepth).To(Equal(int32(24))) + }) + + It("refuses lossy to lossless transcoding: MP3 to FLAC", func() { + // flacOnlyClient can't direct-play mp3, and lossy→lossless transcode is rejected + resp := doPostReq("getTranscodeDecision", flacOnlyClient, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + // MP3 is lossy, FLAC is lossless — should not allow transcoding + Expect(resp.TranscodeDecision.CanTranscode).To(BeFalse()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeFalse()) + Expect(resp.TranscodeDecision.TranscodeParams).To(BeEmpty()) + }) + + It("caps transcode bitrate via maxTranscodingAudioBitrate", func() { + resp := doPostReq("getTranscodeDecision", maxTranscodeBitrateClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + // maxTranscodingAudioBitrate is 192000 bps = 192 kbps → response in bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(192000))) + }) + }) + + Describe("response structure", func() { + It("has correct sourceStream details", func() { + resp := doPostReq("getTranscodeDecision", universalClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + src := resp.TranscodeDecision.SourceStream + Expect(src).ToNot(BeNil()) + Expect(src.Container).To(Equal("flac")) + Expect(src.Codec).To(Equal("flac")) + // AudioBitrate is in bps: 900 kbps * 1000 = 900000 bps + Expect(src.AudioBitrate).To(Equal(int32(900000))) + Expect(src.AudioSamplerate).To(Equal(int32(44100))) + Expect(src.AudioChannels).To(Equal(int32(2))) + Expect(src.Protocol).To(Equal("http")) + }) + + It("reports audioBitrate in bps (kbps * 1000)", func() { + resp := doPostReq("getTranscodeDecision", universalClient, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + src := resp.TranscodeDecision.SourceStream + Expect(src).ToNot(BeNil()) + // MP3 is 320 kbps → 320000 bps + Expect(src.AudioBitrate).To(Equal(int32(320000))) + }) + }) + }) + + Describe("getTranscodeStream", func() { + Describe("error cases", func() { + It("returns 400 when mediaId is missing", func() { + w := doRawReq("getTranscodeStream", "mediaType", "song", "transcodeParams", "some-token") + Expect(w.Code).To(Equal(http.StatusBadRequest)) + }) + + It("returns 400 when mediaType is missing", func() { + w := doRawReq("getTranscodeStream", "mediaId", mp3TrackID, "transcodeParams", "some-token") + Expect(w.Code).To(Equal(http.StatusBadRequest)) + }) + + It("returns 400 when transcodeParams is missing", func() { + w := doRawReq("getTranscodeStream", "mediaId", mp3TrackID, "mediaType", "song") + Expect(w.Code).To(Equal(http.StatusBadRequest)) + }) + + It("returns 400 for unsupported mediaType", func() { + w := doRawReq("getTranscodeStream", "mediaId", mp3TrackID, "mediaType", "video", "transcodeParams", "some-token") + Expect(w.Code).To(Equal(http.StatusBadRequest)) + }) + + It("returns 410 for malformed token", func() { + w := doRawReq("getTranscodeStream", "mediaId", mp3TrackID, "mediaType", "song", "transcodeParams", "invalid-token") + Expect(w.Code).To(Equal(http.StatusGone)) + }) + + It("returns 410 for stale token (media file updated after token issued)", func() { + // Get a valid decision token + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + token := resp.TranscodeDecision.TranscodeParams + Expect(token).ToNot(BeEmpty()) + + // Save original UpdatedAt and restore after test + mf, err := ds.MediaFile(ctx).Get(mp3TrackID) + Expect(err).ToNot(HaveOccurred()) + originalUpdatedAt := mf.UpdatedAt + + // Update the media file's UpdatedAt to simulate a change after token issuance + mf.UpdatedAt = time.Now().Add(time.Hour) + Expect(ds.MediaFile(ctx).Put(mf)).To(Succeed()) + + // Attempt to stream with the now-stale token + w := doRawReq("getTranscodeStream", "mediaId", mp3TrackID, "mediaType", "song", "transcodeParams", token) + Expect(w.Code).To(Equal(http.StatusGone)) + + // Restore original UpdatedAt + mf.UpdatedAt = originalUpdatedAt + Expect(ds.MediaFile(ctx).Put(mf)).To(Succeed()) + }) + }) + + Describe("round-trip: decision then stream", func() { + It("streams direct play for MP3", func() { + // Get decision + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + token := resp.TranscodeDecision.TranscodeParams + Expect(token).ToNot(BeEmpty()) + + // Stream using the token + w := doRawReq("getTranscodeStream", "mediaId", mp3TrackID, "mediaType", "song", "transcodeParams", token) + Expect(w.Code).To(Equal(http.StatusOK)) + // Direct play: format should be "raw" or empty + Expect(spy.LastRequest.Format).To(BeElementOf("raw", "")) + }) + + It("streams transcoded FLAC to MP3", func() { + // Get decision + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + token := resp.TranscodeDecision.TranscodeParams + Expect(token).ToNot(BeEmpty()) + + // Stream using the token + w := doRawReq("getTranscodeStream", "mediaId", flacTrackID, "mediaType", "song", "transcodeParams", token) + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("mp3")) + }) + + It("passes offset through to stream request", func() { + // Get decision + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + token := resp.TranscodeDecision.TranscodeParams + Expect(token).ToNot(BeEmpty()) + + // Stream with offset + w := doRawReq("getTranscodeStream", "mediaId", mp3TrackID, "mediaType", "song", + "transcodeParams", token, "offset", "30") + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Offset).To(Equal(30)) + }) + }) + }) +}) From a25306f2c14f97e9f4e178bfb47f4532d03571ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 9 Mar 2026 10:52:13 -0400 Subject: [PATCH 52/60] fix(artwork): search parent folders for album cover art in multi-disc layouts (#5157) * fix(artwork): search parent folders for album cover art in multi-disc layouts When albums have tracks in subdirectories (e.g., CD1/, CD2/), Navidrome only searched those subdirectories for cover images. This meant cover art placed in the album's root folder (e.g., "Artist/Album/cover.jpg") was not found. Now loadAlbumFoldersPaths also queries parent folders of the album's media folders, so cover art in the album root is discovered. * fix(artwork): simplify parent folder detection for album cover art lookup Signed-off-by: Deluan * fix(album): propagate non-ErrNotFound errors from parent folder lookup Signed-off-by: Deluan --------- Signed-off-by: Deluan --- core/artwork/reader_album.go | 42 +++++++ core/artwork/reader_album_test.go | 177 +++++++++++++++++++++++++++++ core/artwork/reader_artist_test.go | 18 ++- 3 files changed, 235 insertions(+), 2 deletions(-) diff --git a/core/artwork/reader_album.go b/core/artwork/reader_album.go index 9fc9262cb..36b2fff05 100644 --- a/core/artwork/reader_album.go +++ b/core/artwork/reader_album.go @@ -4,6 +4,7 @@ import ( "cmp" "context" "crypto/md5" + "errors" "fmt" "io" "path/filepath" @@ -17,6 +18,7 @@ import ( "github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core/external" "github.com/navidrome/navidrome/core/ffmpeg" + "github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/model" ) @@ -103,6 +105,28 @@ func loadAlbumFoldersPaths(ctx context.Context, ds model.DataStore, albums ...mo if err != nil { return nil, nil, nil, err } + + folderIDSet := make(map[string]bool, len(folderIDs)) + for _, id := range folderIDs { + folderIDSet[id] = true + } + + // For multi-disc albums (2+ folders), check if all folders share a common parent + // that is not already included. This finds cover art in the album root folder + // (e.g., "Artist/Album/cover.jpg" when tracks are in "Artist/Album/CD1/" and "Artist/Album/CD2/"). + // We skip single-folder albums to avoid pulling images from the artist folder. + if commonParentID := commonParentFolder(folders, folderIDSet); commonParentID != "" { + parentFolder, err := ds.Folder(ctx).Get(commonParentID) + if errors.Is(err, model.ErrNotFound) { + log.Warn(ctx, "Parent folder not found for album cover art lookup", "parentID", commonParentID) + } else if err != nil { + return nil, nil, nil, err + } + if parentFolder != nil { + folders = append(folders, *parentFolder) + } + } + var paths []string var imgFiles []string var updatedAt time.Time @@ -125,6 +149,24 @@ func loadAlbumFoldersPaths(ctx context.Context, ds model.DataStore, albums ...mo return paths, imgFiles, &updatedAt, nil } +// commonParentFolder returns the shared parent folder ID when all folders have the +// same parent and that parent is not already in folderIDSet. Returns "" otherwise. +func commonParentFolder(folders []model.Folder, folderIDSet map[string]bool) string { + if len(folders) < 2 { + return "" + } + parentID := folders[0].ParentID + if parentID == "" || folderIDSet[parentID] { + return "" + } + for _, f := range folders[1:] { + if f.ParentID != parentID { + return "" + } + } + return parentID +} + // compareImageFiles compares two image file paths for sorting. // It extracts the base filename (without extension) and compares case-insensitively. // This ensures that "cover.jpg" sorts before "cover.1.jpg" since "cover" < "cover.1". diff --git a/core/artwork/reader_album_test.go b/core/artwork/reader_album_test.go index fd5f8a2be..a8a0eae3e 100644 --- a/core/artwork/reader_album_test.go +++ b/core/artwork/reader_album_test.go @@ -2,6 +2,7 @@ package artwork import ( "context" + "errors" "path/filepath" "time" @@ -116,5 +117,181 @@ var _ = Describe("Album Artwork Reader", func() { Expect(imgFiles[1]).To(Equal(filepath.FromSlash("Artist/Album/cover.jpg"))) Expect(imgFiles[2]).To(Equal(filepath.FromSlash("Artist/Album/Folder.jpg"))) }) + + It("includes images from parent folder for multi-disc albums", func() { + // Simulates: Artist/Album/cover.jpg with tracks in Artist/Album/CD1/ and Artist/Album/CD2/ + repo.result = []model.Folder{ + { + ID: "folder1", + Path: "Artist/Album", + Name: "CD1", + ParentID: "parentFolder", + ImagesUpdatedAt: now, + ImageFiles: []string{}, + }, + { + ID: "folder2", + Path: "Artist/Album", + Name: "CD2", + ParentID: "parentFolder", + ImagesUpdatedAt: now, + ImageFiles: []string{}, + }, + } + repo.parentResult = &model.Folder{ + ID: "parentFolder", + Path: "Artist", + Name: "Album", + ImagesUpdatedAt: expectedAt, + ImageFiles: []string{"cover.jpg", "back.jpg"}, + } + + _, imgFiles, imagesUpdatedAt, err := loadAlbumFoldersPaths(ctx, ds, album) + + Expect(err).ToNot(HaveOccurred()) + Expect(*imagesUpdatedAt).To(Equal(expectedAt)) + Expect(imgFiles).To(HaveLen(2)) + Expect(imgFiles[0]).To(Equal(filepath.FromSlash("Artist/Album/back.jpg"))) + Expect(imgFiles[1]).To(Equal(filepath.FromSlash("Artist/Album/cover.jpg"))) + }) + + It("does not query parent when parent ID is already in album folders", func() { + // When the parent folder is already one of the album's folders, skip it + repo.result = []model.Folder{ + { + ID: "folder1", + Path: "Artist", + Name: "Album", + ParentID: "folder2", + ImagesUpdatedAt: now, + ImageFiles: []string{"cover.jpg"}, + }, + { + ID: "folder2", + Path: "", + Name: "Artist", + ImagesUpdatedAt: now, + ImageFiles: []string{}, + }, + } + + _, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album) + + Expect(err).ToNot(HaveOccurred()) + Expect(imgFiles).To(HaveLen(1)) + Expect(imgFiles[0]).To(Equal(filepath.FromSlash("Artist/Album/cover.jpg"))) + // Get should not have been called (parent already in folder set) + Expect(repo.getCallCount).To(Equal(0)) + }) + + It("does not query parent when folders have different parents", func() { + // When album folders span different parents, don't search any parent + repo.result = []model.Folder{ + { + ID: "folder1", + Path: "Artist1/Album", + Name: "part1", + ParentID: "parentA", + ImagesUpdatedAt: now, + ImageFiles: []string{"cover.jpg"}, + }, + { + ID: "folder2", + Path: "Artist2/Album", + Name: "part2", + ParentID: "parentB", + ImagesUpdatedAt: now, + ImageFiles: []string{}, + }, + } + + _, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album) + + Expect(err).ToNot(HaveOccurred()) + Expect(imgFiles).To(HaveLen(1)) + Expect(imgFiles[0]).To(Equal(filepath.FromSlash("Artist1/Album/part1/cover.jpg"))) + // Get should not have been called (different parents) + Expect(repo.getCallCount).To(Equal(0)) + }) + + It("does not query parent for single-folder albums", func() { + // A single-folder album's parent is typically the artist folder, + // which should not be searched for cover art + repo.result = []model.Folder{ + { + ID: "folder1", + Path: "Artist", + Name: "Album", + ParentID: "artistFolder", + ImagesUpdatedAt: now, + ImageFiles: []string{"cover.jpg"}, + }, + } + + _, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album) + + Expect(err).ToNot(HaveOccurred()) + Expect(imgFiles).To(HaveLen(1)) + Expect(imgFiles[0]).To(Equal(filepath.FromSlash("Artist/Album/cover.jpg"))) + // Get should not have been called (single folder, no parent lookup) + Expect(repo.getCallCount).To(Equal(0)) + }) + + It("propagates non-ErrNotFound errors from parent folder lookup", func() { + repo.result = []model.Folder{ + { + ID: "folder1", + Path: "Artist/Album", + Name: "CD1", + ParentID: "parentFolder", + ImagesUpdatedAt: now, + ImageFiles: []string{"cover.jpg"}, + }, + { + ID: "folder2", + Path: "Artist/Album", + Name: "CD2", + ParentID: "parentFolder", + ImagesUpdatedAt: now, + ImageFiles: []string{}, + }, + } + repo.getErr = errors.New("db connection failed") + + _, _, _, err := loadAlbumFoldersPaths(ctx, ds, album) + + Expect(err).To(MatchError("db connection failed")) + Expect(repo.getCallCount).To(Equal(1)) + }) + + It("continues gracefully when parent folder is not found", func() { + // Parent folder may have been deleted; should log a warning and continue + repo.result = []model.Folder{ + { + ID: "folder1", + Path: "Artist/Album", + Name: "CD1", + ParentID: "missingParent", + ImagesUpdatedAt: now, + ImageFiles: []string{"cover.jpg"}, + }, + { + ID: "folder2", + Path: "Artist/Album", + Name: "CD2", + ParentID: "missingParent", + ImagesUpdatedAt: now, + ImageFiles: []string{}, + }, + } + // parentResult is nil, so Get will return ErrNotFound + + _, imgFiles, _, err := loadAlbumFoldersPaths(ctx, ds, album) + + Expect(err).ToNot(HaveOccurred()) + Expect(imgFiles).To(HaveLen(1)) + Expect(imgFiles[0]).To(Equal(filepath.FromSlash("Artist/Album/CD1/cover.jpg"))) + Expect(repo.getCallCount).To(Equal(1)) + }) }) }) diff --git a/core/artwork/reader_artist_test.go b/core/artwork/reader_artist_test.go index e6a0168f8..4aa71c9ca 100644 --- a/core/artwork/reader_artist_test.go +++ b/core/artwork/reader_artist_test.go @@ -417,14 +417,28 @@ var _ = Describe("artistArtworkReader", func() { type fakeFolderRepo struct { model.FolderRepository - result []model.Folder - err error + result []model.Folder + parentResult *model.Folder + getErr error + getCallCount int + err error } func (f *fakeFolderRepo) GetAll(...model.QueryOptions) ([]model.Folder, error) { return f.result, f.err } +func (f *fakeFolderRepo) Get(id string) (*model.Folder, error) { + f.getCallCount++ + if f.getErr != nil { + return nil, f.getErr + } + if f.parentResult != nil { + return f.parentResult, nil + } + return nil, model.ErrNotFound +} + type fakeDataStore struct { model.DataStore folderRepo *fakeFolderRepo From 957130ca386bb88f9a5c23785cd07629851cfb0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 9 Mar 2026 11:06:31 -0400 Subject: [PATCH 53/60] feat(ui): integrate transcode decision into web player (#5155) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(ui): add browser audio profile detection for transcoding Detect browser codec capabilities via canPlayType() to build a client profile for the getTranscodeDecision API. Only codecs returning "probably" are treated as supported for conservative compatibility. * feat(ui): add transcode decision service with caching and pre-fetch Standalone service that fetches getTranscodeDecision results, caches them with an 11-hour TTL (1h buffer before 12h token expiry), and supports bulk pre-fetching for upcoming queue items. Includes invalidateAll() for handling stale tokens and getCachedDecision() for synchronous cache reads. * feat(ui): add fetch helper for getTranscodeDecision endpoint POST-based Subsonic API call that sends the browser's codec profile and returns the transcode decision including the JWT transcodeParams token for subsequent streaming. * feat(ui): wire transcode decision service singleton Module index that creates the service singleton with the real fetch function and re-exports the browser profile detector. * feat(ui): add Redux transcoding reducer for browser profile state Store the detected browser codec profile in Redux so it's available globally. The profile is set once at startup and used by the decision service when calling getTranscodeDecision. * feat(ui): integrate transcode decision into player musicSrc Replace static stream URLs with lazy musicSrc functions that fetch a transcode decision before playback. Falls back to the old stream endpoint if the decision fetch fails or if no browser profile is set. * feat(ui): detect browser profile and pre-fetch transcode decisions Run codec detection once when the Player mounts, storing the profile in both the decision service and Redux. Pre-fetch decisions for the next 3 songs when the queue or play position changes. * feat(ui): handle stale tokens and replace audio preload with decision pre-fetch On audio playback error, invalidate all cached transcode decisions and pre-fetch fresh decisions for upcoming songs. Replace the old Audio element preload with decision pre-fetching to warm the cache for instant playback transitions. * feat(ui): show transcode format in QualityInfo chip When transcode decision data is available, QualityInfo now shows "FLAC → OPUS 128" instead of just the source format. The new props are optional, so existing usages in song lists, album songs, playlists, and shares are unaffected. * feat(ui): display transcode status in player quality badge AudioTitle now reads the cached transcode decision for the current track and passes it to QualityInfo, showing "FLAC → OPUS 128" when transcoding or the normal format when direct playing. * chore(ui): format and lint transcode decision integration * refactor(ui): use JWT exp claim for decision cache expiry Replace the hardcoded 11-hour TTL with actual token expiration decoded from the JWT's exp claim. Each cache entry is now validated against its own token's lifetime, adapting automatically to server configuration changes. Tokens without an exp claim are treated as expired and re-fetched immediately. * fix(ui): resolve transcode URLs eagerly on browser refresh Instead of setting musicSrc to a function on queue refresh (which breaks the player's identity matching and can't survive JSON serialization), resolve transcode decisions for the current and next few tracks before dispatching, passing string URLs to the reducer. Also simplifies code: extract makeMusicSrc helper, add resolveStreamUrl to decisionService, use httpClient instead of raw fetch, and remove barrel file test. * chore(ui): fix prettier formatting in Player.jsx * fix(ui): use ref to avoid stale closure in mount-only transcode effect Split the mount effect into profile detection + URL resolution, using a ref for playerState so the effect correctly reads the latest queue without needing playerState in the dependency array (which would cause it to re-run on every queue/position change). * fix(ui): address code review feedback on transcode integration - Use jwt-decode for JWT parsing instead of manual atob (handles base64url) - Guard resolveStreamUrl to fall back to direct stream when decision is null - Fix savedPlayIndex -1 bug in PLAYER_REFRESH_QUEUE (findIndex returns -1) * docs: improve comments on JWT exp claim decoding in decision service Signed-off-by: Deluan --------- Signed-off-by: Deluan --- ui/src/App.jsx | 2 + ui/src/actions/player.js | 12 + ui/src/audioplayer/AudioTitle.jsx | 10 + ui/src/audioplayer/Player.jsx | 93 +++++++- ui/src/common/QualityInfo.jsx | 26 ++- ui/src/common/QualityInfo.test.jsx | 26 +++ ui/src/reducers/index.js | 1 + ui/src/reducers/playerReducer.js | 28 ++- ui/src/reducers/playerReducer.test.js | 52 +++++ ui/src/reducers/transcodingReducer.js | 14 ++ ui/src/reducers/transcodingReducer.test.js | 23 ++ ui/src/transcode/browserProfile.js | 40 ++++ ui/src/transcode/browserProfile.test.js | 76 ++++++ ui/src/transcode/decisionService.js | 111 +++++++++ ui/src/transcode/decisionService.test.js | 256 +++++++++++++++++++++ ui/src/transcode/fetchDecision.js | 23 ++ ui/src/transcode/fetchDecision.test.js | 92 ++++++++ ui/src/transcode/index.js | 5 + 18 files changed, 883 insertions(+), 7 deletions(-) create mode 100644 ui/src/reducers/playerReducer.test.js create mode 100644 ui/src/reducers/transcodingReducer.js create mode 100644 ui/src/reducers/transcodingReducer.test.js create mode 100644 ui/src/transcode/browserProfile.js create mode 100644 ui/src/transcode/browserProfile.test.js create mode 100644 ui/src/transcode/decisionService.js create mode 100644 ui/src/transcode/decisionService.test.js create mode 100644 ui/src/transcode/fetchDecision.js create mode 100644 ui/src/transcode/fetchDecision.test.js create mode 100644 ui/src/transcode/index.js diff --git a/ui/src/App.jsx b/ui/src/App.jsx index 2dbe72421..35eaee3eb 100644 --- a/ui/src/App.jsx +++ b/ui/src/App.jsx @@ -33,6 +33,7 @@ import { replayGainReducer, downloadMenuDialogReducer, shareDialogReducer, + transcodingReducer, } from './reducers' import createAdminStore from './store/createAdminStore' import { i18nProvider } from './i18n' @@ -72,6 +73,7 @@ const adminStore = createAdminStore({ activity: activityReducer, settings: settingsReducer, replayGain: replayGainReducer, + transcoding: transcodingReducer, }, }) diff --git a/ui/src/actions/player.js b/ui/src/actions/player.js index acef2e9b2..9056abeb6 100644 --- a/ui/src/actions/player.js +++ b/ui/src/actions/player.js @@ -7,6 +7,8 @@ export const PLAYER_PLAY_TRACKS = 'PLAYER_PLAY_TRACKS' export const PLAYER_CURRENT = 'PLAYER_CURRENT' export const PLAYER_SET_VOLUME = 'PLAYER_SET_VOLUME' export const PLAYER_SET_MODE = 'PLAYER_SET_MODE' +export const TRANSCODING_SET_PROFILE = 'TRANSCODING_SET_PROFILE' +export const PLAYER_REFRESH_QUEUE = 'PLAYER_REFRESH_QUEUE' export const setTrack = (data) => ({ type: PLAYER_SET_TRACK, @@ -102,3 +104,13 @@ export const setPlayMode = (mode) => ({ type: PLAYER_SET_MODE, data: { mode }, }) + +export const setTranscodingProfile = (profile) => ({ + type: TRANSCODING_SET_PROFILE, + data: profile, +}) + +export const refreshQueue = (resolvedUrls) => ({ + type: PLAYER_REFRESH_QUEUE, + data: resolvedUrls, +}) diff --git a/ui/src/audioplayer/AudioTitle.jsx b/ui/src/audioplayer/AudioTitle.jsx index 093bb53fb..df37edfbb 100644 --- a/ui/src/audioplayer/AudioTitle.jsx +++ b/ui/src/audioplayer/AudioTitle.jsx @@ -3,6 +3,7 @@ import { useMediaQuery } from '@material-ui/core' import { Link } from 'react-router-dom' import clsx from 'clsx' import { QualityInfo } from '../common' +import { decisionService } from '../transcode' import useStyle from './styles' import { useDrag } from 'react-dnd' import { DraggableTypes } from '../consts' @@ -35,6 +36,14 @@ const AudioTitle = React.memo(({ audioInfo, gainInfo, isMobile }) => { rgTrackPeak: song.rgTrackPeak, } + const decision = decisionService.getCachedDecision(audioInfo.trackId) + const transcodeProps = decision + ? { + transcodeStream: decision.transcodeStream || null, + isDirectPlay: decision.canDirectPlay, + } + : {} + const subtitle = song.tags?.['subtitle'] const title = song.title + (subtitle ? ` (${subtitle})` : '') @@ -53,6 +62,7 @@ const AudioTitle = React.memo(({ audioInfo, gainInfo, isMobile }) => { record={qi} className={classes.qualityInfo} {...gainInfo} + {...transcodeProps} /> )} diff --git a/ui/src/audioplayer/Player.jsx b/ui/src/audioplayer/Player.jsx index 7d086172b..eba3b82d7 100644 --- a/ui/src/audioplayer/Player.jsx +++ b/ui/src/audioplayer/Player.jsx @@ -1,4 +1,4 @@ -import React, { useCallback, useEffect, useMemo, useState } from 'react' +import React, { useCallback, useEffect, useMemo, useRef, useState } from 'react' import { useDispatch, useSelector } from 'react-redux' import { useMediaQuery } from '@material-ui/core' import { ThemeProvider } from '@material-ui/core/styles' @@ -19,7 +19,9 @@ import AudioTitle from './AudioTitle' import { clearQueue, currentPlaying, + refreshQueue, setPlayMode, + setTranscodingProfile, setVolume, syncQueue, } from '../actions' @@ -30,6 +32,7 @@ import locale from './locale' import { keyMap } from '../hotkeys' import keyHandlers from './keyHandlers' import { calculateGain } from '../utils/calculateReplayGain' +import { detectBrowserProfile, decisionService } from '../transcode' const Player = () => { const theme = useCurrentTheme() @@ -49,6 +52,61 @@ const Player = () => { ) const { authenticated } = useAuthState() + + // Keep a ref to playerState so the mount effect can read the latest value + // without re-triggering on every queue/position change + const playerStateRef = useRef(playerState) + playerStateRef.current = playerState + + // Detect browser codec profile and eagerly resolve transcode URLs for the + // persisted queue once on mount (e.g. after a browser refresh) + useEffect(() => { + const profile = detectBrowserProfile() + decisionService.setProfile(profile) + dispatch(setTranscodingProfile(profile)) + + const state = playerStateRef.current + const currentIdx = state.savedPlayIndex || 0 + const trackIds = state.queue + .slice(currentIdx, currentIdx + 4) + .filter((item) => !item.isRadio && item.trackId) + .map((item) => item.trackId) + + if (trackIds.length === 0) { + dispatch(refreshQueue()) + return + } + + Promise.allSettled( + trackIds.map((id) => + decisionService.resolveStreamUrl(id).then((url) => [id, url]), + ), + ).then((results) => { + const resolvedUrls = {} + results.forEach((r) => { + if (r.status === 'fulfilled') { + resolvedUrls[r.value[0]] = r.value[1] + } + }) + dispatch(refreshQueue(resolvedUrls)) + }) + }, [dispatch]) + + // Pre-fetch transcode decisions for next 2-3 songs when queue or position changes + useEffect(() => { + if (!playerState.queue.length) return + + const currentIdx = playerState.savedPlayIndex || 0 + const nextSongIds = playerState.queue + .slice(currentIdx + 1, currentIdx + 4) + .filter((item) => !item.isRadio) + .map((item) => item.trackId) + + if (nextSongIds.length > 0) { + decisionService.prefetchDecisions(nextSongIds) + } + }, [playerState.queue, playerState.savedPlayIndex]) + const visible = authenticated && playerState.queue.length > 0 const isRadio = playerState.current?.isRadio || false const classes = useStyle({ @@ -151,7 +209,9 @@ const Player = () => { ...defaultOptions, audioLists: playerState.queue.map((item) => item), playIndex: playerState.playIndex, - autoPlay: playerState.clear || playerState.playIndex === 0, + autoPlay: + playerState.autoPlay !== false && + (playerState.clear || playerState.playIndex === 0), clearPriorAudioLists: playerState.clear, extendsContent: ( @@ -190,9 +250,9 @@ const Player = () => { if (!preloaded) { const next = nextSong() - if (next != null) { - const audio = new Audio() - audio.src = next.musicSrc + if (next != null && !next.isRadio) { + // Trigger decision pre-fetch (this also warms the cache) + decisionService.prefetchDecisions([next.trackId]) } setPreload(true) return @@ -284,6 +344,28 @@ const Player = () => { } }, []) + const onAudioError = useCallback( + (error, currentPlayId, audioLists, audioInfo) => { + // Invalidate all cached decisions — token may be stale + decisionService.invalidateAll() + + // Pre-fetch decisions for upcoming songs with fresh tokens + const currentIdx = playerState.queue.findIndex( + (item) => item.uuid === currentPlayId, + ) + if (currentIdx >= 0) { + const nextSongIds = playerState.queue + .slice(currentIdx + 1, currentIdx + 4) + .filter((item) => !item.isRadio) + .map((item) => item.trackId) + if (nextSongIds.length > 0) { + decisionService.prefetchDecisions(nextSongIds) + } + } + }, + [playerState.queue], + ) + const onBeforeDestroy = useCallback(() => { return new Promise((resolve, reject) => { dispatch(clearQueue()) @@ -320,6 +402,7 @@ const Player = () => { onPlayModeChange={(mode) => dispatch(setPlayMode(mode))} onAudioEnded={onAudioEnded} onCoverClick={onCoverClick} + onAudioError={onAudioError} onBeforeDestroy={onBeforeDestroy} getAudioInstance={setAudioInstance} /> diff --git a/ui/src/common/QualityInfo.jsx b/ui/src/common/QualityInfo.jsx index 171f5e0f0..57a8251a4 100644 --- a/ui/src/common/QualityInfo.jsx +++ b/ui/src/common/QualityInfo.jsx @@ -20,7 +20,15 @@ const useStyle = makeStyles( }, ) -export const QualityInfo = ({ record, size, gainMode, preAmp, className }) => { +export const QualityInfo = ({ + record, + size, + gainMode, + preAmp, + className, + transcodeStream, + isDirectPlay, +}) => { const classes = useStyle() let { suffix, bitRate, rgAlbumGain, rgAlbumPeak, rgTrackGain, rgTrackPeak } = record @@ -34,6 +42,20 @@ export const QualityInfo = ({ record, size, gainMode, preAmp, className }) => { } } + // Show transcode target when transcoding (not direct play) + if (transcodeStream && !isDirectPlay) { + const targetCodec = (transcodeStream.codec || '').toUpperCase() + const targetBitrate = transcodeStream.audioBitrate + ? Math.round(transcodeStream.audioBitrate / 1000) + : 0 + let targetInfo = targetCodec + if (targetBitrate > 0) { + targetInfo += ' ' + targetBitrate + } + const sourceSuffix = suffix || placeholder + info = `${sourceSuffix} → ${targetInfo}` + } + const extra = useMemo(() => { if (gainMode !== 'none') { const gainValue = calculateGain( @@ -63,6 +85,8 @@ QualityInfo.propTypes = { size: PropTypes.string, className: PropTypes.string, gainMode: PropTypes.string, + transcodeStream: PropTypes.object, + isDirectPlay: PropTypes.bool, } QualityInfo.defaultProps = { diff --git a/ui/src/common/QualityInfo.test.jsx b/ui/src/common/QualityInfo.test.jsx index ae1874715..174ee8a85 100644 --- a/ui/src/common/QualityInfo.test.jsx +++ b/ui/src/common/QualityInfo.test.jsx @@ -77,4 +77,30 @@ describe('', () => { ) expect(screen.getByText('FLAC (0.00 dB)')).toBeInTheDocument() }) + + it('shows transcode arrow when transcodeStream is provided', () => { + const info = { suffix: 'FLAC', bitRate: 1008 } + const transcodeStream = { codec: 'opus', audioBitrate: 128000 } + render() + expect(screen.getByText('FLAC → OPUS 128')).toBeInTheDocument() + }) + + it('shows transcode with lossy source including bitrate', () => { + const info = { suffix: 'FLAC', bitRate: 1008 } + const transcodeStream = { codec: 'mp3', audioBitrate: 320000 } + render() + expect(screen.getByText('FLAC → MP3 320')).toBeInTheDocument() + }) + + it('does not show arrow when isDirectPlay is true', () => { + const info = { suffix: 'MP3', bitRate: 320 } + render() + expect(screen.getByText('MP3 320')).toBeInTheDocument() + }) + + it('behaves normally when no transcode props are passed', () => { + const info = { suffix: 'MP3', bitRate: 320 } + render() + expect(screen.getByText('MP3 320')).toBeInTheDocument() + }) }) diff --git a/ui/src/reducers/index.js b/ui/src/reducers/index.js index 3db0b1dff..64a0049b7 100644 --- a/ui/src/reducers/index.js +++ b/ui/src/reducers/index.js @@ -6,3 +6,4 @@ export * from './albumView' export * from './activityReducer' export * from './settingsReducer' export * from './replayGainReducer' +export * from './transcodingReducer' diff --git a/ui/src/reducers/playerReducer.js b/ui/src/reducers/playerReducer.js index 0392736e5..b7086e6c8 100644 --- a/ui/src/reducers/playerReducer.js +++ b/ui/src/reducers/playerReducer.js @@ -1,5 +1,6 @@ import { v4 as uuidv4 } from 'uuid' import subsonic from '../subsonic' +import { decisionService } from '../transcode' import { PLAYER_ADD_TRACKS, PLAYER_CLEAR_QUEUE, @@ -10,6 +11,7 @@ import { PLAYER_SET_VOLUME, PLAYER_SYNC_QUEUE, PLAYER_SET_MODE, + PLAYER_REFRESH_QUEUE, } from '../actions' import config from '../config' @@ -30,6 +32,14 @@ const pad = (value) => { } } +const makeMusicSrc = (trackId) => + decisionService.getProfile() + ? () => + decisionService + .resolveStreamUrl(trackId) + .catch(() => subsonic.streamUrl(trackId)) + : subsonic.streamUrl(trackId) + const mapToAudioLists = (item) => { // If item comes from a playlist, trackId is mediaFileId const trackId = item.mediaFileId || item.id @@ -76,7 +86,7 @@ const mapToAudioLists = (item) => { lyric: lyricText, singer: item.artist, duration: item.duration, - musicSrc: subsonic.streamUrl(trackId), + musicSrc: makeMusicSrc(trackId), cover: subsonic.getCoverArtUrl( { id: trackId, @@ -210,6 +220,22 @@ export const playerReducer = (previousState = initialState, payload) => { return reduceCurrent(previousState, payload) case PLAYER_SET_MODE: return reduceMode(previousState, payload) + case PLAYER_REFRESH_QUEUE: { + const resolvedUrls = payload.data || {} + return { + ...previousState, + queue: previousState.queue.map((item) => ({ + ...item, + musicSrc: item.isRadio + ? item.musicSrc + : resolvedUrls[item.trackId] || subsonic.streamUrl(item.trackId), + })), + clear: true, + autoPlay: false, + playIndex: + previousState.savedPlayIndex >= 0 ? previousState.savedPlayIndex : 0, + } + } default: return previousState } diff --git a/ui/src/reducers/playerReducer.test.js b/ui/src/reducers/playerReducer.test.js new file mode 100644 index 000000000..9e3b03b1e --- /dev/null +++ b/ui/src/reducers/playerReducer.test.js @@ -0,0 +1,52 @@ +import { describe, it, expect } from 'vitest' +import { playerReducer } from './playerReducer' +import { PLAYER_REFRESH_QUEUE } from '../actions' + +describe('playerReducer', () => { + describe('PLAYER_REFRESH_QUEUE', () => { + it('clamps negative savedPlayIndex to 0', () => { + const state = { + queue: [ + { trackId: 'song-1', musicSrc: 'old-url', uuid: 'a' }, + { trackId: 'song-2', musicSrc: 'old-url', uuid: 'b' }, + ], + savedPlayIndex: -1, + current: {}, + clear: false, + volume: 1, + } + const action = { type: PLAYER_REFRESH_QUEUE, data: {} } + const result = playerReducer(state, action) + expect(result.playIndex).toBe(0) + }) + + it('preserves valid savedPlayIndex', () => { + const state = { + queue: [ + { trackId: 'song-1', musicSrc: 'old-url', uuid: 'a' }, + { trackId: 'song-2', musicSrc: 'old-url', uuid: 'b' }, + ], + savedPlayIndex: 1, + current: {}, + clear: false, + volume: 1, + } + const action = { type: PLAYER_REFRESH_QUEUE, data: {} } + const result = playerReducer(state, action) + expect(result.playIndex).toBe(1) + }) + + it('uses savedPlayIndex of 0 correctly', () => { + const state = { + queue: [{ trackId: 'song-1', musicSrc: 'old-url', uuid: 'a' }], + savedPlayIndex: 0, + current: {}, + clear: false, + volume: 1, + } + const action = { type: PLAYER_REFRESH_QUEUE, data: {} } + const result = playerReducer(state, action) + expect(result.playIndex).toBe(0) + }) + }) +}) diff --git a/ui/src/reducers/transcodingReducer.js b/ui/src/reducers/transcodingReducer.js new file mode 100644 index 000000000..db7a3708a --- /dev/null +++ b/ui/src/reducers/transcodingReducer.js @@ -0,0 +1,14 @@ +import { TRANSCODING_SET_PROFILE } from '../actions' + +const initialState = { + browserProfile: null, +} + +export const transcodingReducer = (state = initialState, { type, data }) => { + switch (type) { + case TRANSCODING_SET_PROFILE: + return { ...state, browserProfile: data } + default: + return state + } +} diff --git a/ui/src/reducers/transcodingReducer.test.js b/ui/src/reducers/transcodingReducer.test.js new file mode 100644 index 000000000..eb3e7a490 --- /dev/null +++ b/ui/src/reducers/transcodingReducer.test.js @@ -0,0 +1,23 @@ +import { describe, it, expect } from 'vitest' +import { transcodingReducer } from './transcodingReducer' +import { TRANSCODING_SET_PROFILE } from '../actions' + +describe('transcodingReducer', () => { + const initialState = { browserProfile: null } + + it('returns initial state', () => { + expect(transcodingReducer(undefined, {})).toEqual(initialState) + }) + + it('handles TRANSCODING_SET_PROFILE', () => { + const profile = { + name: 'NavidromeUI', + directPlayProfiles: [{ containers: ['mp3'] }], + } + const state = transcodingReducer(initialState, { + type: TRANSCODING_SET_PROFILE, + data: profile, + }) + expect(state.browserProfile).toEqual(profile) + }) +}) diff --git a/ui/src/transcode/browserProfile.js b/ui/src/transcode/browserProfile.js new file mode 100644 index 000000000..4ee114e45 --- /dev/null +++ b/ui/src/transcode/browserProfile.js @@ -0,0 +1,40 @@ +// Each entry: { codec name for the server, container, MIME to probe } +export const CODEC_PROBES = [ + { codec: 'mp3', container: 'mp3', mime: 'audio/mpeg' }, + { codec: 'aac', container: 'mp4', mime: 'audio/mp4; codecs="mp4a.40.2"' }, + { codec: 'opus', container: 'ogg', mime: 'audio/ogg; codecs="opus"' }, + { codec: 'vorbis', container: 'ogg', mime: 'audio/ogg; codecs="vorbis"' }, + { codec: 'flac', container: 'flac', mime: 'audio/flac' }, + { codec: 'wav', container: 'wav', mime: 'audio/wav' }, + { codec: 'alac', container: 'mp4', mime: 'audio/mp4; codecs="alac"' }, +] + +// Default transcoding targets — ordered by preference. +// These are attempted if direct play is not possible. +const DEFAULT_TRANSCODING_PROFILES = [ + { container: 'ogg', audioCodec: 'opus', protocol: 'http' }, + { container: 'mp3', audioCodec: 'mp3', protocol: 'http' }, +] + +export function detectBrowserProfile() { + const audio = new Audio() + const directPlayProfiles = [] + + for (const { codec, container, mime } of CODEC_PROBES) { + if (audio.canPlayType(mime) === 'probably') { + directPlayProfiles.push({ + containers: [container], + audioCodecs: [codec], + protocols: ['http'], + }) + } + } + + return { + name: 'NavidromeUI', + platform: navigator.userAgent, + directPlayProfiles, + transcodingProfiles: DEFAULT_TRANSCODING_PROFILES, + codecProfiles: [], + } +} diff --git a/ui/src/transcode/browserProfile.test.js b/ui/src/transcode/browserProfile.test.js new file mode 100644 index 000000000..9764a0046 --- /dev/null +++ b/ui/src/transcode/browserProfile.test.js @@ -0,0 +1,76 @@ +import { describe, it, expect, beforeEach, vi } from 'vitest' +import { detectBrowserProfile, CODEC_PROBES } from './browserProfile' + +describe('detectBrowserProfile', () => { + let mockCanPlayType + + beforeEach(() => { + mockCanPlayType = vi.fn() + vi.stubGlobal( + 'Audio', + class { + canPlayType = mockCanPlayType + }, + ) + }) + + it('includes codecs that return "probably"', () => { + mockCanPlayType.mockImplementation((mime) => { + if (mime === 'audio/mpeg') return 'probably' + if (mime === 'audio/ogg; codecs="opus"') return 'probably' + return '' + }) + + const profile = detectBrowserProfile() + + expect(profile.name).toBe('NavidromeUI') + expect(profile.directPlayProfiles.length).toBe(2) + + const codecs = profile.directPlayProfiles.flatMap((p) => p.audioCodecs) + expect(codecs).toContain('mp3') + expect(codecs).toContain('opus') + }) + + it('excludes codecs that return "maybe"', () => { + mockCanPlayType.mockReturnValue('maybe') + + const profile = detectBrowserProfile() + expect(profile.directPlayProfiles).toEqual([]) + }) + + it('excludes codecs that return empty string', () => { + mockCanPlayType.mockReturnValue('') + + const profile = detectBrowserProfile() + expect(profile.directPlayProfiles).toEqual([]) + }) + + it('sets protocol to "http" for all direct play profiles', () => { + mockCanPlayType.mockReturnValue('probably') + + const profile = detectBrowserProfile() + profile.directPlayProfiles.forEach((p) => { + expect(p.protocols).toEqual(['http']) + }) + }) + + it('includes transcoding profiles for common formats', () => { + mockCanPlayType.mockReturnValue('') + + const profile = detectBrowserProfile() + expect(profile.transcodingProfiles.length).toBeGreaterThan(0) + expect(profile.transcodingProfiles[0].protocol).toBe('http') + }) + + it('sets codecProfiles to empty array', () => { + mockCanPlayType.mockReturnValue('probably') + + const profile = detectBrowserProfile() + expect(profile.codecProfiles).toEqual([]) + }) + + it('includes platform info', () => { + const profile = detectBrowserProfile() + expect(typeof profile.platform).toBe('string') + }) +}) diff --git a/ui/src/transcode/decisionService.js b/ui/src/transcode/decisionService.js new file mode 100644 index 000000000..9228cc882 --- /dev/null +++ b/ui/src/transcode/decisionService.js @@ -0,0 +1,111 @@ +import { jwtDecode } from 'jwt-decode' +import subsonic from '../subsonic' +import { baseUrl } from '../utils' + +// Decode the exp claim from a JWT token (no signature verification needed client-side). +// The JWT token is meant to be opaque to the client, we are only allowing ourselves to do +// this here because the UI is tightly integrated with the server; normally we would +// need to rely on the getTranscodeStream returning an error on stale tokens. +export function decodeJwtExp(token) { + try { + if (!token) return null + const payload = jwtDecode(token) + return typeof payload.exp === 'number' ? payload.exp : null + } catch { + return null + } +} + +export function createDecisionService(fetchFn) { + const cache = new Map() + let currentProfile = null + + function isFresh(entry) { + const exp = decodeJwtExp(entry.decision?.transcodeParams) + if (exp == null) return false + // exp is in seconds, Date.now() in milliseconds; 60s buffer avoids mid-request expiry + return Date.now() < (exp - 60) * 1000 + } + + function setProfile(profile) { + currentProfile = profile + } + + function getProfile() { + return currentProfile + } + + async function getDecision(songId, browserProfile) { + const profile = browserProfile || currentProfile + if (!profile) return null + + const cached = cache.get(songId) + if (cached && isFresh(cached)) { + return cached.decision + } + + const decision = await fetchFn(songId, profile) + cache.set(songId, { decision }) + return decision + } + + async function prefetchDecisions(songIds, browserProfile) { + const profile = browserProfile || currentProfile + if (!profile) return + + const uncached = songIds.filter((id) => { + const entry = cache.get(id) + return !entry || !isFresh(entry) + }) + + await Promise.allSettled( + uncached.map(async (id) => { + const decision = await fetchFn(id, profile) + cache.set(id, { decision }) + }), + ) + } + + function invalidateAll() { + cache.clear() + } + + function buildStreamUrl(songId, transcodeParams, offset) { + const params = { + mediaId: songId, + mediaType: 'song', + transcodeParams, + } + if (offset != null && offset > 0) { + params.offset = offset + } + return baseUrl(subsonic.url('getTranscodeStream', null, params)) + } + + async function resolveStreamUrl(songId) { + const decision = await getDecision(songId) + if (!decision?.transcodeParams) { + return baseUrl(subsonic.streamUrl(songId)) + } + return buildStreamUrl(songId, decision.transcodeParams) + } + + function getCachedDecision(songId) { + const entry = cache.get(songId) + if (entry && isFresh(entry)) { + return entry.decision + } + return null + } + + return { + getDecision, + getCachedDecision, + prefetchDecisions, + resolveStreamUrl, + invalidateAll, + buildStreamUrl, + setProfile, + getProfile, + } +} diff --git a/ui/src/transcode/decisionService.test.js b/ui/src/transcode/decisionService.test.js new file mode 100644 index 000000000..a2718ade9 --- /dev/null +++ b/ui/src/transcode/decisionService.test.js @@ -0,0 +1,256 @@ +import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest' +import { createDecisionService, decodeJwtExp } from './decisionService' + +// Helper: create a fake JWT with a given exp (seconds since epoch) +function fakeJwt(expSeconds) { + const header = btoa(JSON.stringify({ alg: 'HS256', typ: 'JWT' })) + const payload = btoa(JSON.stringify({ exp: expSeconds })) + return `${header}.${payload}.fake-signature` +} + +// Helper: create a fake JWT with no exp claim +function fakeJwtNoExp() { + const header = btoa(JSON.stringify({ alg: 'HS256', typ: 'JWT' })) + const payload = btoa(JSON.stringify({ sub: 'test' })) + return `${header}.${payload}.fake-signature` +} + +describe('decodeJwtExp', () => { + it('extracts exp from a valid JWT', () => { + const exp = 1700000000 + expect(decodeJwtExp(fakeJwt(exp))).toBe(exp) + }) + + it('returns null for JWT without exp claim', () => { + expect(decodeJwtExp(fakeJwtNoExp())).toBeNull() + }) + + it('returns null for non-JWT string', () => { + expect(decodeJwtExp('not-a-jwt')).toBeNull() + }) + + it('returns null for empty string', () => { + expect(decodeJwtExp('')).toBeNull() + }) + + it('returns null for null/undefined', () => { + expect(decodeJwtExp(null)).toBeNull() + expect(decodeJwtExp(undefined)).toBeNull() + }) +}) + +describe('decisionService', () => { + let service + let mockFetchFn + + const fakeProfile = { + name: 'NavidromeUI', + platform: 'test', + directPlayProfiles: [], + transcodingProfiles: [], + codecProfiles: [], + } + + // Token that expires 1 hour from "now" (will be relative to fake timers) + function makeFakeDecision(expiresInMs = 3600 * 1000) { + const expSeconds = Math.floor((Date.now() + expiresInMs) / 1000) + return { + canDirectPlay: true, + canTranscode: false, + transcodeParams: fakeJwt(expSeconds), + sourceStream: { codec: 'mp3', container: 'mp3' }, + } + } + + beforeEach(() => { + localStorage.setItem('username', 'testuser') + localStorage.setItem('subsonic-token', 'testtoken') + localStorage.setItem('subsonic-salt', 'testsalt') + mockFetchFn = vi.fn().mockImplementation(() => { + return Promise.resolve(makeFakeDecision()) + }) + service = createDecisionService(mockFetchFn) + }) + + afterEach(() => { + vi.restoreAllMocks() + localStorage.clear() + }) + + describe('getDecision', () => { + it('fetches and caches a decision', async () => { + const result = await service.getDecision('song-1', fakeProfile) + expect(result.canDirectPlay).toBe(true) + expect(mockFetchFn).toHaveBeenCalledTimes(1) + expect(mockFetchFn).toHaveBeenCalledWith('song-1', fakeProfile) + + // Second call uses cache + const result2 = await service.getDecision('song-1', fakeProfile) + expect(result2).toEqual(result) + expect(mockFetchFn).toHaveBeenCalledTimes(1) + }) + + it('re-fetches after token expires', async () => { + vi.useFakeTimers() + + // Token expires in 1 hour + mockFetchFn.mockResolvedValue(makeFakeDecision(3600 * 1000)) + await service.getDecision('song-1', fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(1) + + // Advance past expiration + vi.advanceTimersByTime(3600 * 1000 + 1000) + await service.getDecision('song-1', fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(2) + vi.useRealTimers() + }) + + it('does not re-fetch before token expires', async () => { + vi.useFakeTimers() + + // Token expires in 1 hour + mockFetchFn.mockResolvedValue(makeFakeDecision(3600 * 1000)) + await service.getDecision('song-1', fakeProfile) + + // 30 minutes later — still fresh + vi.advanceTimersByTime(1800 * 1000) + await service.getDecision('song-1', fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(1) + vi.useRealTimers() + }) + + it('re-fetches immediately when token has no exp claim', async () => { + const noExpDecision = { + canDirectPlay: true, + canTranscode: false, + transcodeParams: fakeJwtNoExp(), + sourceStream: { codec: 'mp3', container: 'mp3' }, + } + mockFetchFn.mockResolvedValue(noExpDecision) + await service.getDecision('song-1', fakeProfile) + + // Should re-fetch because token has no exp + await service.getDecision('song-1', fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(2) + }) + + it('caches different songs independently', async () => { + await service.getDecision('song-1', fakeProfile) + await service.getDecision('song-2', fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(2) + }) + }) + + describe('getCachedDecision', () => { + it('returns null when song is not cached', () => { + expect(service.getCachedDecision('song-1')).toBeNull() + }) + + it('returns cached decision after getDecision', async () => { + await service.getDecision('song-1', fakeProfile) + const cached = service.getCachedDecision('song-1') + expect(cached).not.toBeNull() + expect(cached.canDirectPlay).toBe(true) + }) + + it('returns null after cache is invalidated', async () => { + await service.getDecision('song-1', fakeProfile) + service.invalidateAll() + expect(service.getCachedDecision('song-1')).toBeNull() + }) + + it('returns null after token expires', async () => { + vi.useFakeTimers() + mockFetchFn.mockResolvedValue(makeFakeDecision(3600 * 1000)) + await service.getDecision('song-1', fakeProfile) + + vi.advanceTimersByTime(3600 * 1000 + 1000) + expect(service.getCachedDecision('song-1')).toBeNull() + vi.useRealTimers() + }) + }) + + describe('prefetchDecisions', () => { + it('fetches decisions for uncached songs', async () => { + await service.prefetchDecisions(['song-1', 'song-2'], fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(2) + }) + + it('skips already cached songs', async () => { + await service.getDecision('song-1', fakeProfile) + mockFetchFn.mockClear() + + await service.prefetchDecisions(['song-1', 'song-2'], fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(1) + expect(mockFetchFn).toHaveBeenCalledWith('song-2', fakeProfile) + }) + + it('silently ignores fetch errors', async () => { + mockFetchFn.mockRejectedValue(new Error('network error')) + await expect( + service.prefetchDecisions(['song-1'], fakeProfile), + ).resolves.not.toThrow() + }) + }) + + describe('invalidateAll', () => { + it('clears cache so next getDecision re-fetches', async () => { + await service.getDecision('song-1', fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(1) + + service.invalidateAll() + + await service.getDecision('song-1', fakeProfile) + expect(mockFetchFn).toHaveBeenCalledTimes(2) + }) + }) + + describe('resolveStreamUrl', () => { + it('fetches decision and returns built URL', async () => { + service.setProfile(fakeProfile) + const url = await service.resolveStreamUrl('song-1') + expect(url).toContain('getTranscodeStream') + expect(url).toContain('mediaId=song-1') + expect(mockFetchFn).toHaveBeenCalledTimes(1) + }) + + it('falls back to stream URL when decision has no transcodeParams', async () => { + service.setProfile(fakeProfile) + mockFetchFn.mockResolvedValue({ + canDirectPlay: true, + canTranscode: false, + }) + const url = await service.resolveStreamUrl('song-1') + expect(url).toContain('stream') + expect(url).not.toContain('getTranscodeStream') + }) + + it('falls back to stream URL when decision is null', async () => { + service.setProfile(fakeProfile) + mockFetchFn.mockResolvedValue(null) + const url = await service.resolveStreamUrl('song-1') + expect(url).toContain('stream') + expect(url).not.toContain('getTranscodeStream') + }) + }) + + describe('buildStreamUrl', () => { + it('builds URL with required parameters', () => { + const url = service.buildStreamUrl('song-1', 'jwt-token-123') + expect(url).toContain('getTranscodeStream') + expect(url).toContain('mediaId=song-1') + expect(url).toContain('mediaType=song') + expect(url).toContain('transcodeParams=jwt-token-123') + }) + + it('includes offset when provided', () => { + const url = service.buildStreamUrl('song-1', 'jwt-token-123', 30) + expect(url).toContain('offset=30') + }) + + it('omits offset when not provided', () => { + const url = service.buildStreamUrl('song-1', 'jwt-token-123') + expect(url).not.toContain('offset') + }) + }) +}) diff --git a/ui/src/transcode/fetchDecision.js b/ui/src/transcode/fetchDecision.js new file mode 100644 index 000000000..1c794082b --- /dev/null +++ b/ui/src/transcode/fetchDecision.js @@ -0,0 +1,23 @@ +import subsonic from '../subsonic' +import { httpClient } from '../dataProvider' + +export async function fetchTranscodeDecision(songId, browserProfile) { + const fetchUrl = subsonic.url('getTranscodeDecision', null, { + mediaId: songId, + mediaType: 'song', + }) + + const { json } = await httpClient(fetchUrl, { + method: 'POST', + body: JSON.stringify(browserProfile), + }) + + const subsonicResponse = json['subsonic-response'] + + if (subsonicResponse.status !== 'ok') { + const err = subsonicResponse.error || {} + throw new Error(`getTranscodeDecision error: ${err.code} ${err.message}`) + } + + return subsonicResponse.transcodeDecision +} diff --git a/ui/src/transcode/fetchDecision.test.js b/ui/src/transcode/fetchDecision.test.js new file mode 100644 index 000000000..83f934f33 --- /dev/null +++ b/ui/src/transcode/fetchDecision.test.js @@ -0,0 +1,92 @@ +import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest' + +// Mock httpClient before importing module under test +vi.mock('../dataProvider', () => ({ + httpClient: vi.fn(), +})) + +import { fetchTranscodeDecision } from './fetchDecision' +import { httpClient } from '../dataProvider' + +describe('fetchTranscodeDecision', () => { + const fakeProfile = { + name: 'NavidromeUI', + platform: 'test', + directPlayProfiles: [ + { containers: ['mp3'], audioCodecs: ['mp3'], protocols: ['http'] }, + ], + transcodingProfiles: [], + codecProfiles: [], + } + + const fakeJson = { + 'subsonic-response': { + status: 'ok', + transcodeDecision: { + canDirectPlay: true, + canTranscode: false, + transcodeParams: 'jwt-token', + sourceStream: { codec: 'mp3' }, + }, + }, + } + + beforeEach(() => { + localStorage.setItem('username', 'testuser') + localStorage.setItem('subsonic-token', 'testtoken') + localStorage.setItem('subsonic-salt', 'testsalt') + + httpClient.mockResolvedValue({ json: fakeJson }) + }) + + afterEach(() => { + vi.restoreAllMocks() + localStorage.clear() + }) + + it('makes a POST request to getTranscodeDecision with correct URL', async () => { + await fetchTranscodeDecision('song-1', fakeProfile) + + expect(httpClient).toHaveBeenCalledTimes(1) + const [url, options] = httpClient.mock.calls[0] + expect(url).toContain('getTranscodeDecision') + expect(url).toContain('mediaId=song-1') + expect(url).toContain('mediaType=song') + expect(options.method).toBe('POST') + }) + + it('sends the browser profile as JSON body', async () => { + await fetchTranscodeDecision('song-1', fakeProfile) + + const [, options] = httpClient.mock.calls[0] + expect(JSON.parse(options.body)).toEqual(fakeProfile) + }) + + it('returns the transcodeDecision from response', async () => { + const result = await fetchTranscodeDecision('song-1', fakeProfile) + expect(result).toEqual(fakeJson['subsonic-response'].transcodeDecision) + }) + + it('throws on HTTP error (httpClient rejects)', async () => { + httpClient.mockRejectedValue(new Error('Server Error')) + + await expect( + fetchTranscodeDecision('song-1', fakeProfile), + ).rejects.toThrow() + }) + + it('throws on Subsonic error response', async () => { + httpClient.mockResolvedValue({ + json: { + 'subsonic-response': { + status: 'failed', + error: { code: 70, message: 'not found' }, + }, + }, + }) + + await expect( + fetchTranscodeDecision('song-1', fakeProfile), + ).rejects.toThrow() + }) +}) diff --git a/ui/src/transcode/index.js b/ui/src/transcode/index.js new file mode 100644 index 000000000..aa0cb216c --- /dev/null +++ b/ui/src/transcode/index.js @@ -0,0 +1,5 @@ +import { createDecisionService } from './decisionService' +import { fetchTranscodeDecision } from './fetchDecision' +export { detectBrowserProfile } from './browserProfile' + +export const decisionService = createDecisionService(fetchTranscodeDecision) From 09e1cf6ae75d7f6e30cd7505acf474f373fbd623 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 9 Mar 2026 11:22:43 -0400 Subject: [PATCH 54/60] chore(deps): update TagLib to 2.2.1 Signed-off-by: Deluan --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 559a34c3c..be3c221ba 100644 --- a/Makefile +++ b/Makefile @@ -20,7 +20,7 @@ PLATFORMS ?= $(SUPPORTED_PLATFORMS) DOCKER_TAG ?= deluan/navidrome:develop # Taglib version to use in cross-compilation, from https://github.com/navidrome/cross-taglib -CROSS_TAGLIB_VERSION ?= 2.2.0-1 +CROSS_TAGLIB_VERSION ?= 2.2.1-1 GOLANGCI_LINT_VERSION ?= v2.11.1 UI_SRC_FILES := $(shell find ui -type f -not -path "ui/build/*" -not -path "ui/node_modules/*") From e08d4bef16586256e70640d2f1f6078f1bb5bdf3 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 9 Mar 2026 12:44:19 -0400 Subject: [PATCH 55/60] fix(ui): preserve pending track selection through queue sync and premature callbacks When clicking a song while another was playing, PLAYER_SYNC_QUEUE and PLAYER_CURRENT would fire before the music player switched tracks, wiping the playIndex set by PLAYER_PLAY_TRACKS. This caused the player to stay on the old track instead of switching to the clicked one. Now reduceSyncQueue and reduceCurrent preserve a pending playIndex until the music player confirms it actually reached the requested track. --- ui/src/reducers/playerReducer.js | 17 +++-- ui/src/reducers/playerReducer.test.js | 95 ++++++++++++++++++++++++++- 2 files changed, 107 insertions(+), 5 deletions(-) diff --git a/ui/src/reducers/playerReducer.js b/ui/src/reducers/playerReducer.js index b7086e6c8..d3291633c 100644 --- a/ui/src/reducers/playerReducer.js +++ b/ui/src/reducers/playerReducer.js @@ -173,8 +173,11 @@ const reduceSyncQueue = (state, { data: { audioInfo, audioLists } }) => { return { ...state, queue: audioLists, - clear: false, - playIndex: undefined, + // Keep clear and playIndex alive so the music player can still + // pick up a pending track selection set by PLAYER_PLAY_TRACKS. + // They will be consumed by the next PLAYER_CURRENT dispatch. + clear: state.playIndex != null ? state.clear : false, + playIndex: state.playIndex != null ? state.playIndex : undefined, } } @@ -183,11 +186,17 @@ const reduceCurrent = (state, { data }) => { const savedPlayIndex = state.queue.findIndex( (item) => item.uuid === current.uuid, ) + // When a track selection is pending (playIndex is set), keep it alive + // until the music player confirms it actually switched to the requested + // track. Without this, a premature onAudioPlay callback for the + // still-playing old track would overwrite the pending selection. + const pending = state.playIndex != null && savedPlayIndex !== state.playIndex return { ...state, current, - playIndex: undefined, - savedPlayIndex, + playIndex: pending ? state.playIndex : undefined, + clear: pending ? state.clear : false, + savedPlayIndex: pending ? state.savedPlayIndex : savedPlayIndex, volume: data.volume, } } diff --git a/ui/src/reducers/playerReducer.test.js b/ui/src/reducers/playerReducer.test.js index 9e3b03b1e..10e9512d7 100644 --- a/ui/src/reducers/playerReducer.test.js +++ b/ui/src/reducers/playerReducer.test.js @@ -1,8 +1,101 @@ import { describe, it, expect } from 'vitest' import { playerReducer } from './playerReducer' -import { PLAYER_REFRESH_QUEUE } from '../actions' +import { + PLAYER_SYNC_QUEUE, + PLAYER_CURRENT, + PLAYER_REFRESH_QUEUE, +} from '../actions' describe('playerReducer', () => { + describe('pending track selection survives SYNC_QUEUE and premature CURRENT', () => { + // Simulates the real sequence when clicking a new song while one is playing: + // 1. PLAYER_PLAY_TRACKS sets playIndex and clear + // 2. PLAYER_SYNC_QUEUE fires when music player syncs its internal queue + // 3. PLAYER_CURRENT fires for the OLD still-playing track + // 4. PLAYER_CURRENT fires for the NEW track (player switched) + const stateAfterPlayTracks = { + queue: [ + { trackId: 's1', uuid: 'aaa', name: 'Song 1' }, + { trackId: 's2', uuid: 'bbb', name: 'Song 2' }, + { trackId: 's3', uuid: 'ccc', name: 'Song 3' }, + ], + current: { uuid: 'ccc', name: 'Song 3' }, + playIndex: 0, // user clicked Song 1 + savedPlayIndex: 2, // Song 3 was playing + clear: true, + volume: 1, + } + + it('SYNC_QUEUE preserves pending playIndex and clear', () => { + const newQueue = [ + { trackId: 's1', uuid: 'xxx', name: 'Song 1' }, + { trackId: 's2', uuid: 'yyy', name: 'Song 2' }, + { trackId: 's3', uuid: 'zzz', name: 'Song 3' }, + ] + const action = { + type: PLAYER_SYNC_QUEUE, + data: { audioInfo: {}, audioLists: newQueue }, + } + const result = playerReducer(stateAfterPlayTracks, action) + expect(result.playIndex).toBe(0) + expect(result.clear).toBe(true) + expect(result.queue).toBe(newQueue) + }) + + it('SYNC_QUEUE clears playIndex when no pending selection', () => { + const stateNoPending = { ...stateAfterPlayTracks, playIndex: undefined } + const action = { + type: PLAYER_SYNC_QUEUE, + data: { audioInfo: {}, audioLists: stateNoPending.queue }, + } + const result = playerReducer(stateNoPending, action) + expect(result.playIndex).toBeUndefined() + expect(result.clear).toBe(false) + }) + + it('CURRENT for old track preserves pending playIndex', () => { + // After SYNC_QUEUE, queue has new UUIDs. The old track's UUID (zzz) + // is at index 2, but playIndex is 0. This is a premature callback. + const stateAfterSync = { + ...stateAfterPlayTracks, + queue: [ + { trackId: 's1', uuid: 'xxx', name: 'Song 1' }, + { trackId: 's2', uuid: 'yyy', name: 'Song 2' }, + { trackId: 's3', uuid: 'zzz', name: 'Song 3' }, + ], + } + const action = { + type: PLAYER_CURRENT, + data: { uuid: 'zzz', name: 'Song 3', volume: 1 }, + } + const result = playerReducer(stateAfterSync, action) + expect(result.playIndex).toBe(0) + expect(result.clear).toBe(true) + expect(result.savedPlayIndex).toBe(2) // preserved from before + }) + + it('CURRENT for correct track consumes pending playIndex', () => { + const stateAfterSync = { + ...stateAfterPlayTracks, + queue: [ + { trackId: 's1', uuid: 'xxx', name: 'Song 1' }, + { trackId: 's2', uuid: 'yyy', name: 'Song 2' }, + { trackId: 's3', uuid: 'zzz', name: 'Song 3' }, + ], + } + // Player switched to Song 1 (uuid 'xxx', index 0 == playIndex) + const action = { + type: PLAYER_CURRENT, + data: { uuid: 'xxx', name: 'Song 1', volume: 1 }, + } + const result = playerReducer(stateAfterSync, action) + expect(result.playIndex).toBeUndefined() + expect(result.clear).toBe(false) + expect(result.savedPlayIndex).toBe(0) + expect(result.current.name).toBe('Song 1') + }) + }) + describe('PLAYER_REFRESH_QUEUE', () => { it('clamps negative savedPlayIndex to 0', () => { const state = { From d4b2499e1ee1cd6ca135af08758e9a96e0d7966d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 9 Mar 2026 14:19:53 -0400 Subject: [PATCH 56/60] fix(server): return correct scanType in startScan response (#5159) * fix(api): return correct scanType in startScan response The startScan endpoint launches the scan in a goroutine and immediately calls GetScanStatus to build the response. Because the scanner hasn't had time to initialize and write its state to the database, the response contained stale data from the previous scan (e.g., scanType "quick" when fullScan=true was requested). Add a polling loop that waits briefly (up to 3s, polling every 50ms) for the scanner to report Scanning=true before returning the status. If the timeout expires, it falls back to the current behavior (no regression). Fixes #5158 * fix(api): use ticker/timer with context cancellation for scan polling Replace time.Sleep loop with proper ticker, timer, and ctx.Done() handling so the poll exits cleanly on timeout or client disconnect. * fix(api): handle fast scan completion in poll loop Add a channel to detect when the scan goroutine finishes before the poll loop observes Scanning=true, avoiding a 3s timeout on very fast scans. Use defer close to handle both success and error paths. --- server/subsonic/library_scanning.go | 32 +++++++++++++ server/subsonic/library_scanning_test.go | 60 ++++++++++++++++++++++++ tests/mock_scanner.go | 23 +++++++++ 3 files changed, 115 insertions(+) diff --git a/server/subsonic/library_scanning.go b/server/subsonic/library_scanning.go index c9dd64968..bac27f821 100644 --- a/server/subsonic/library_scanning.go +++ b/server/subsonic/library_scanning.go @@ -80,7 +80,9 @@ func (api *Router) StartScan(r *http.Request) (*responses.Subsonic, error) { } } + fastScanCompleted := make(chan struct{}) go func() { + defer close(fastScanCompleted) start := time.Now() var err error @@ -99,5 +101,35 @@ func (api *Router) StartScan(r *http.Request) (*responses.Subsonic, error) { log.Info(ctx, "On-demand scan complete", "user", loggedUser.UserName, "elapsed", time.Since(start)) }() + // Wait briefly for the scanner to start and update its status, so the response + // reflects the current scan (not stale data from a previous scan). + const ( + pollInterval = 50 * time.Millisecond + pollTimeout = 3 * time.Second + ) + ticker := time.NewTicker(pollInterval) + defer ticker.Stop() + timer := time.NewTimer(pollTimeout) + defer timer.Stop() + +loop: + for { + status, err := api.scanner.Status(ctx) + if err == nil && status.Scanning { + break + } + select { + case <-fastScanCompleted: + log.Info(ctx, "Fast scan completed", "user", loggedUser.UserName) + break loop + case <-timer.C: + log.Warn(ctx, "Timed out waiting for scanner to start; response may be stale") + break loop + case <-ctx.Done(): + return nil, newError(responses.ErrorGeneric, "Request cancelled while waiting for scanner to start") + case <-ticker.C: + } + } + return api.GetScanStatus(r) } diff --git a/server/subsonic/library_scanning_test.go b/server/subsonic/library_scanning_test.go index d8eba296b..c62c156bc 100644 --- a/server/subsonic/library_scanning_test.go +++ b/server/subsonic/library_scanning_test.go @@ -365,6 +365,66 @@ var _ = Describe("LibraryScanning", func() { Expect(targets[0].LibraryID).To(Equal(1)) Expect(targets[0].FolderPath).To(Equal("")) }) + + It("returns correct scanType in response when fullScan=false", func() { + // Setup mock to update status when scan starts (simulating the real scanner) + ms.SetScanStatusFunc(func(fullScan bool, targets []model.ScanTarget) *model.ScannerStatus { + scanType := "quick" + if fullScan { + scanType = "full" + } + return &model.ScannerStatus{ + Scanning: true, + ScanType: scanType, + } + }) + + ctx := request.WithUser(context.Background(), model.User{ + ID: "admin-id", + IsAdmin: true, + }) + + r := httptest.NewRequest("GET", "/rest/startScan", nil) + r = r.WithContext(ctx) + + response, err := api.StartScan(r) + + Expect(err).ToNot(HaveOccurred()) + Expect(response).ToNot(BeNil()) + Expect(response.ScanStatus).ToNot(BeNil()) + Expect(response.ScanStatus.Scanning).To(BeTrue()) + Expect(response.ScanStatus.ScanType).To(Equal("quick")) + }) + + It("returns correct scanType in response when fullScan=true", func() { + // Setup mock to update status when scan starts (simulating the real scanner) + ms.SetScanStatusFunc(func(fullScan bool, targets []model.ScanTarget) *model.ScannerStatus { + scanType := "quick" + if fullScan { + scanType = "full" + } + return &model.ScannerStatus{ + Scanning: true, + ScanType: scanType, + } + }) + + ctx := request.WithUser(context.Background(), model.User{ + ID: "admin-id", + IsAdmin: true, + }) + + r := httptest.NewRequest("GET", "/rest/startScan?fullScan=true", nil) + r = r.WithContext(ctx) + + response, err := api.StartScan(r) + + Expect(err).ToNot(HaveOccurred()) + Expect(response).ToNot(BeNil()) + Expect(response.ScanStatus).ToNot(BeNil()) + Expect(response.ScanStatus.Scanning).To(BeTrue()) + Expect(response.ScanStatus.ScanType).To(Equal("full")) + }) }) Describe("GetScanStatus", func() { diff --git a/tests/mock_scanner.go b/tests/mock_scanner.go index 52396723f..495e8fe53 100644 --- a/tests/mock_scanner.go +++ b/tests/mock_scanner.go @@ -14,6 +14,7 @@ type MockScanner struct { scanFoldersCalls []ScanFoldersCall scanningStatus bool statusResponse *model.ScannerStatus + scanStatusFunc func(fullScan bool, targets []model.ScanTarget) *model.ScannerStatus } type ScanAllCall struct { @@ -38,6 +39,13 @@ func (m *MockScanner) ScanAll(_ context.Context, fullScan bool) ([]string, error m.scanAllCalls = append(m.scanAllCalls, ScanAllCall{FullScan: fullScan}) + // Simulate the scanner updating its status when the scan starts + if m.scanStatusFunc != nil { + m.statusResponse = m.scanStatusFunc(fullScan, nil) + } else { + m.scanningStatus = true + } + return nil, nil } @@ -54,6 +62,13 @@ func (m *MockScanner) ScanFolders(_ context.Context, fullScan bool, targets []mo Targets: targetsCopy, }) + // Simulate the scanner updating its status when the scan starts + if m.scanStatusFunc != nil { + m.statusResponse = m.scanStatusFunc(fullScan, targetsCopy) + } else { + m.scanningStatus = true + } + return nil, nil } @@ -118,3 +133,11 @@ func (m *MockScanner) SetStatusResponse(status *model.ScannerStatus) { defer m.mu.Unlock() m.statusResponse = status } + +// SetScanStatusFunc sets a function that will be called when ScanAll/ScanFolders is invoked, +// simulating the scanner updating its status when the scan starts. +func (m *MockScanner) SetScanStatusFunc(fn func(fullScan bool, targets []model.ScanTarget) *model.ScannerStatus) { + m.mu.Lock() + defer m.mu.Unlock() + m.scanStatusFunc = fn +} From d7c3a50f86f96ee21da999604a3eb3c276e8d525 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Deluan=20Quint=C3=A3o?= Date: Mon, 9 Mar 2026 16:47:34 -0400 Subject: [PATCH 57/60] fix: player MaxBitRate cap, format-aware defaults, browser profile filtering (#5165) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(transcode): apply player MaxBitRate cap and use format-aware default bitrates Add player MaxBitRate cap to the transcode decider so server-side player bitrate limits are respected when making OpenSubsonic transcode decisions. The player cap is applied only when it is more restrictive than the client's maxAudioBitrate (or when the client has no limit). Also replace the hardcoded 256 kbps default with a format-aware lookup that checks the DB first (for user-customized values), then built-in defaults, and finally falls back to 256 kbps. For lossless→lossy transcoding, prefer maxTranscodingAudioBitrate over maxAudioBitrate when available. * test(e2e): add tests for player MaxBitRate cap and format-aware default bitrates Add e2e tests covering: - Player MaxBitRate forcing transcode when source exceeds cap - Player MaxBitRate having no effect when source is under cap - Client limit winning when more restrictive than player MaxBitRate - Player MaxBitRate winning when more restrictive than client limit - Player MaxBitRate=0 having no effect - Format-aware defaults: mp3 (192kbps), opus (128kbps) instead of hardcoded 256 - maxAudioBitrate fallback for lossless→lossy when no maxTranscodingAudioBitrate - maxTranscodingAudioBitrate taking priority over maxAudioBitrate - Combined player + client limits flowing correctly through decision→stream * feat(transcode): update transcoding profiles to add flac, filter by supported codecs, and ensure mp3 fallback Signed-off-by: Deluan * fix(db): ensure all default transcodings exist on upgrade Older installations that were seeded before aac/flac were added to DefaultTranscodings may be missing these entries. The previous migration only added flac; this one ensures all default transcodings are present without touching user-customized entries. * test: remove duplication Signed-off-by: Deluan --------- Signed-off-by: Deluan --- core/transcode/decider.go | 28 ++- core/transcode/decider_test.go | 97 ++++++++- ...60309203355_ensure_default_transcodings.go | 41 ++++ server/e2e/subsonic_transcode_test.go | 188 ++++++++++++++++++ ui/src/transcode/browserProfile.js | 42 ++-- ui/src/transcode/browserProfile.test.js | 39 +++- 6 files changed, 413 insertions(+), 22 deletions(-) create mode 100644 db/migrations/20260309203355_ensure_default_transcodings.go diff --git a/core/transcode/decider.go b/core/transcode/decider.go index 55b451fd6..e870e9af7 100644 --- a/core/transcode/decider.go +++ b/core/transcode/decider.go @@ -14,7 +14,7 @@ import ( "github.com/navidrome/navidrome/model/request" ) -const defaultBitrate = 256 // kbps +const fallbackBitrate = 256 // kbps // Decider is the core service interface for making transcoding decisions type Decider interface { @@ -58,6 +58,13 @@ func (s *deciderService) MakeDecision(ctx context.Context, mf *model.MediaFile, // Check for server-side player transcoding override if trc, ok := request.TranscodingFrom(ctx); ok && trc.TargetFormat != "" { clientInfo = applyServerOverride(ctx, clientInfo, &trc) + } else if player, ok := request.PlayerFrom(ctx); ok && player.MaxBitRate > 0 { + if clientInfo.MaxAudioBitrate == 0 || player.MaxBitRate < clientInfo.MaxAudioBitrate { + modified := *clientInfo + modified.MaxAudioBitrate = player.MaxBitRate + clientInfo = &modified + log.Debug(ctx, "Applied player MaxBitRate cap", "playerMaxBitRate", player.MaxBitRate, "client", clientInfo.Name) + } } log.Trace(ctx, "Making transcode decision", "mediaID", mf.ID, "container", src.Container, @@ -291,6 +298,21 @@ func (s *deciderService) computeTranscodedStream(ctx context.Context, src *Strea return ts, targetFormat } +// lookupDefaultBitrate returns the default bitrate for the given format. +// It checks the DB first (for user-customized values), then falls back to +// the built-in defaults, and finally to fallbackBitrate. +func lookupDefaultBitrate(ctx context.Context, ds model.DataStore, format string) int { + if t, err := ds.Transcoding(ctx).FindByFormat(format); err == nil && t.DefaultBitRate > 0 { + return t.DefaultBitRate + } + for _, dt := range consts.DefaultTranscodings { + if dt.TargetFormat == format && dt.DefaultBitRate > 0 { + return dt.DefaultBitRate + } + } + return fallbackBitrate +} + // LookupTranscodeCommand returns the ffmpeg command for the given format. // It checks the DB first (for user-customized commands), then falls back to // the built-in default command. Returns "" if the format is unknown. @@ -341,8 +363,10 @@ func (s *deciderService) computeBitrate(ctx context.Context, src *StreamDetails, if !targetIsLossless { if clientInfo.MaxTranscodingAudioBitrate > 0 { ts.Bitrate = clientInfo.MaxTranscodingAudioBitrate + } else if clientInfo.MaxAudioBitrate > 0 { + ts.Bitrate = clientInfo.MaxAudioBitrate } else { - ts.Bitrate = defaultBitrate + ts.Bitrate = lookupDefaultBitrate(ctx, s.ds, targetFormat) } } else { if clientInfo.MaxAudioBitrate > 0 && src.Bitrate > clientInfo.MaxAudioBitrate { diff --git a/core/transcode/decider_test.go b/core/transcode/decider_test.go index e5ad2f621..280060fbf 100644 --- a/core/transcode/decider_test.go +++ b/core/transcode/decider_test.go @@ -229,7 +229,7 @@ var _ = Describe("Decider", func() { decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) Expect(err).ToNot(HaveOccurred()) Expect(decision.CanTranscode).To(BeTrue()) - Expect(decision.TargetBitrate).To(Equal(defaultBitrate)) // 256 kbps + Expect(decision.TargetBitrate).To(Equal(160)) // mp3 default from mock transcoding repo }) It("preserves lossy bitrate when under max", func() { @@ -993,8 +993,8 @@ var _ = Describe("Decider", func() { Expect(err).ToNot(HaveOccurred()) Expect(decision.CanTranscode).To(BeTrue()) Expect(decision.TargetFormat).To(Equal("mp3")) - // With no cap, lossless→lossy uses defaultBitrate (256) - Expect(decision.TargetBitrate).To(Equal(defaultBitrate)) + // With no cap, lossless→lossy uses format default bitrate (160 for mp3 from mock) + Expect(decision.TargetBitrate).To(Equal(160)) }) It("does not apply override when no transcoding is in context", func() { @@ -1012,6 +1012,97 @@ var _ = Describe("Decider", func() { }) }) + + Context("Player MaxBitRate cap", func() { + It("applies player MaxBitRate cap when client has no limit", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + Name: "TestClient", + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"flac", "mp3"}, AudioCodecs: []string{"flac", "mp3"}, Protocols: []string{ProtocolHTTP}}, + }, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + playerCtx := request.WithPlayer(ctx, model.Player{MaxBitRate: 320}) + + decision, err := svc.MakeDecision(playerCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + // Source bitrate 1000 > player cap 320, so direct play is not possible + Expect(decision.CanDirectPlay).To(BeFalse()) + Expect(decision.CanTranscode).To(BeTrue()) + // Lossless→lossy should use MaxAudioBitrate (320) as target, not format default + Expect(decision.TargetBitrate).To(Equal(320)) + }) + + It("uses client limit when it is more restrictive than player MaxBitRate", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + Name: "TestClient", + MaxAudioBitrate: 256, + MaxTranscodingAudioBitrate: 256, + TranscodingProfiles: []Profile{ + {Container: "mp3", AudioCodec: "mp3", Protocol: ProtocolHTTP}, + }, + } + playerCtx := request.WithPlayer(ctx, model.Player{MaxBitRate: 500}) + + decision, err := svc.MakeDecision(playerCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + // Client limit 256 < player cap 500, so player cap doesn't apply; client limit wins + Expect(decision.TargetBitrate).To(Equal(256)) + }) + + It("does not cap when player MaxBitRate is 0", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "mp3", Codec: "MP3", BitRate: 320, Channels: 2, SampleRate: 44100}) + ci := &ClientInfo{ + Name: "TestClient", + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"mp3"}, AudioCodecs: []string{"mp3"}, Protocols: []string{ProtocolHTTP}}, + }, + } + playerCtx := request.WithPlayer(ctx, model.Player{MaxBitRate: 0}) + + decision, err := svc.MakeDecision(playerCtx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + }) + + Context("Format-aware default bitrate", func() { + It("uses opus default bitrate from DB", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 48000, BitDepth: 16}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "opus", AudioCodec: "opus", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetBitrate).To(Equal(96)) // opus default from mock + }) + + It("uses aac default bitrate from DB", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "flac", Codec: "FLAC", BitRate: 1000, Channels: 2, SampleRate: 44100, BitDepth: 16}) + ci := &ClientInfo{ + TranscodingProfiles: []Profile{ + {Container: "aac", AudioCodec: "aac", Protocol: ProtocolHTTP}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanTranscode).To(BeTrue()) + Expect(decision.TargetBitrate).To(Equal(256)) // aac default from mock + }) + + It("falls back to 256 for unknown format", func() { + bitrate := lookupDefaultBitrate(ctx, ds, "xyz") + Expect(bitrate).To(Equal(fallbackBitrate)) + }) + }) }) Describe("ensureProbed", func() { diff --git a/db/migrations/20260309203355_ensure_default_transcodings.go b/db/migrations/20260309203355_ensure_default_transcodings.go new file mode 100644 index 000000000..ff3838222 --- /dev/null +++ b/db/migrations/20260309203355_ensure_default_transcodings.go @@ -0,0 +1,41 @@ +package migrations + +import ( + "context" + "database/sql" + + "github.com/navidrome/navidrome/consts" + "github.com/navidrome/navidrome/model/id" + "github.com/pressly/goose/v3" +) + +func init() { + goose.AddMigrationContext(upEnsureDefaultTranscodings, downEnsureDefaultTranscodings) +} + +func upEnsureDefaultTranscodings(_ context.Context, tx *sql.Tx) error { + // Older installations may be missing default transcodings that were added + // after the initial seeding (e.g., aac was added later than mp3/opus). + // Insert any missing defaults without touching user-customized entries. + for _, t := range consts.DefaultTranscodings { + var count int + err := tx.QueryRow("SELECT COUNT(*) FROM transcoding WHERE target_format = ?", t.TargetFormat).Scan(&count) + if err != nil { + return err + } + if count == 0 { + _, err = tx.Exec( + "INSERT INTO transcoding (id, name, target_format, default_bit_rate, command) VALUES (?, ?, ?, ?, ?)", + id.NewRandom(), t.Name, t.TargetFormat, t.DefaultBitRate, t.Command, + ) + if err != nil { + return err + } + } + } + return nil +} + +func downEnsureDefaultTranscodings(_ context.Context, tx *sql.Tx) error { + return nil +} diff --git a/server/e2e/subsonic_transcode_test.go b/server/e2e/subsonic_transcode_test.go index 16a884bf3..64d788b30 100644 --- a/server/e2e/subsonic_transcode_test.go +++ b/server/e2e/subsonic_transcode_test.go @@ -146,6 +146,25 @@ var _ = Describe("Transcode Endpoints", Ordered, func() { }) Describe("getTranscodeDecision", func() { + // setPlayerMaxBitRate ensures a player exists for the test-client and sets its MaxBitRate. + // It makes a dummy request to register the player, then updates it via the repository. + setPlayerMaxBitRate := func(maxBitRate int) { + doReq("ping") + player, err := ds.Player(ctx).FindMatch(adminUser.ID, "test-client", "") + Expect(err).ToNot(HaveOccurred()) + player.MaxBitRate = maxBitRate + Expect(ds.Player(ctx).Put(player)).To(Succeed()) + } + + AfterEach(func() { + // Reset player MaxBitRate to 0 after each test + player, err := ds.Player(ctx).FindMatch(adminUser.ID, "test-client", "") + if err == nil { + player.MaxBitRate = 0 + _ = ds.Player(ctx).Put(player) + } + }) + Describe("error cases", func() { It("returns 405 for GET request", func() { w := doRawReq("getTranscodeDecision", "mediaId", mp3TrackID, "mediaType", "song") @@ -360,6 +379,175 @@ var _ = Describe("Transcode Endpoints", Ordered, func() { Expect(src.AudioBitrate).To(Equal(int32(320000))) }) }) + + Describe("player MaxBitRate cap", func() { + It("forces transcode when source bitrate exceeds player MaxBitRate", func() { + setPlayerMaxBitRate(320) // 320 kbps cap + + // FLAC is 900kbps, client has no bitrate limit but player cap is 320 + resp := doPostReq("getTranscodeDecision", flacAndMp3Client, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeFalse()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Container).To(Equal("mp3")) + // Target bitrate should be capped at player's 320kbps = 320000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(320000))) + }) + + It("does not affect direct play when source bitrate is under player MaxBitRate", func() { + setPlayerMaxBitRate(500) // 500 kbps cap + + // MP3 is 320kbps, under the 500kbps player cap → direct play + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", mp3TrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + }) + + It("uses client limit when more restrictive than player MaxBitRate", func() { + setPlayerMaxBitRate(500) // 500 kbps player cap + + // Client caps at 320kbps (bitrateCapClient), which is more restrictive than 500 + // FLAC is 900kbps → exceeds both limits → transcode + resp := doPostReq("getTranscodeDecision", bitrateCapClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + // Client limit (320kbps) is more restrictive → 320000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(320000))) + }) + + It("uses player MaxBitRate when more restrictive than client limit", func() { + setPlayerMaxBitRate(192) // 192 kbps player cap + + // Client caps at 320kbps (bitrateCapClient), player is more restrictive at 192 + // FLAC is 900kbps → transcode at 192kbps + resp := doPostReq("getTranscodeDecision", bitrateCapClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + // Player limit (192kbps) is more restrictive → 192000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(192000))) + }) + + It("has no effect when player MaxBitRate is 0", func() { + setPlayerMaxBitRate(0) // No player cap + + // FLAC with flac+mp3 client → direct play (no bitrate constraint) + resp := doPostReq("getTranscodeDecision", flacAndMp3Client, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanDirectPlay).To(BeTrue()) + }) + }) + + Describe("format-aware default bitrate", func() { + It("uses mp3 format default (192kbps) for lossless-to-mp3 with no bitrate limits", func() { + // mp3OnlyClient has no maxAudioBitrate or maxTranscodingAudioBitrate + // FLAC → MP3 should use the mp3 default bitrate (192kbps), not hardcoded 256 + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Container).To(Equal("mp3")) + // mp3 default is 192kbps = 192000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(192000))) + }) + + It("uses opus format default (128kbps) for lossless-to-opus with no bitrate limits", func() { + // opusTranscodeClient has no maxAudioBitrate or maxTranscodingAudioBitrate + // FLAC → Opus should use the opus default bitrate (128kbps) + resp := doPostReq("getTranscodeDecision", opusTranscodeClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Codec).To(Equal("opus")) + // opus default is 128kbps = 128000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(128000))) + }) + + It("uses maxAudioBitrate as fallback for lossless-to-lossy when no maxTranscodingAudioBitrate", func() { + // bitrateCapClient has maxAudioBitrate=320000 but no maxTranscodingAudioBitrate + // FLAC → MP3: maxAudioBitrate (320kbps) should be used as the target + resp := doPostReq("getTranscodeDecision", bitrateCapClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + // maxAudioBitrate is 320kbps = 320000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(320000))) + }) + + It("prefers maxTranscodingAudioBitrate over maxAudioBitrate for lossless-to-lossy", func() { + // maxTranscodeBitrateClient has maxTranscodingAudioBitrate=192000 + // FLAC → MP3: should use 192kbps, not format default or maxAudioBitrate + resp := doPostReq("getTranscodeDecision", maxTranscodeBitrateClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + // maxTranscodingAudioBitrate is 192kbps = 192000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(192000))) + }) + }) + + Describe("player MaxBitRate + client limits combined", func() { + It("player MaxBitRate injects maxAudioBitrate, format default used for transcode target", func() { + setPlayerMaxBitRate(320) + + // opusTranscodeClient has no client bitrate limits + // Player cap injects maxAudioBitrate=320 + // FLAC (900kbps) → exceeds 320 → transcode to opus + // Lossless→lossy: maxTranscodingAudioBitrate=0, so falls back to maxAudioBitrate=320 + resp := doPostReq("getTranscodeDecision", opusTranscodeClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + Expect(resp.TranscodeDecision.TranscodeStream.Codec).To(Equal("opus")) + // maxAudioBitrate=320 used as fallback → 320000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(320000))) + }) + + It("player MaxBitRate + client maxTranscodingAudioBitrate work together", func() { + setPlayerMaxBitRate(320) + + // maxTranscodeBitrateClient: maxTranscodingAudioBitrate=192000 (192kbps), no maxAudioBitrate + // Player cap injects maxAudioBitrate=320 + // FLAC (900kbps) → exceeds 320 → transcode to mp3 + // Lossless→lossy: maxTranscodingAudioBitrate=192 takes priority + resp := doPostReq("getTranscodeDecision", maxTranscodeBitrateClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision).ToNot(BeNil()) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + Expect(resp.TranscodeDecision.TranscodeStream).ToNot(BeNil()) + // maxTranscodingAudioBitrate=192 is preferred → 192000 bps + Expect(resp.TranscodeDecision.TranscodeStream.AudioBitrate).To(Equal(int32(192000))) + }) + + It("streams with correct bitrate after player MaxBitRate-triggered transcode", func() { + setPlayerMaxBitRate(128) + + // Get decision: FLAC (900kbps) with player cap 128 → transcode + resp := doPostReq("getTranscodeDecision", mp3OnlyClient, "mediaId", flacTrackID, "mediaType", "song") + Expect(resp.Status).To(Equal(responses.StatusOK)) + Expect(resp.TranscodeDecision.CanTranscode).To(BeTrue()) + token := resp.TranscodeDecision.TranscodeParams + Expect(token).ToNot(BeEmpty()) + + // Stream using the token + w := doRawReq("getTranscodeStream", "mediaId", flacTrackID, "mediaType", "song", "transcodeParams", token) + Expect(w.Code).To(Equal(http.StatusOK)) + Expect(spy.LastRequest.Format).To(Equal("mp3")) + Expect(spy.LastRequest.BitRate).To(Equal(128)) + }) + }) }) Describe("getTranscodeStream", func() { diff --git a/ui/src/transcode/browserProfile.js b/ui/src/transcode/browserProfile.js index 4ee114e45..5a4cde20f 100644 --- a/ui/src/transcode/browserProfile.js +++ b/ui/src/transcode/browserProfile.js @@ -9,32 +9,44 @@ export const CODEC_PROBES = [ { codec: 'alac', container: 'mp4', mime: 'audio/mp4; codecs="alac"' }, ] -// Default transcoding targets — ordered by preference. -// These are attempted if direct play is not possible. -const DEFAULT_TRANSCODING_PROFILES = [ - { container: 'ogg', audioCodec: 'opus', protocol: 'http' }, - { container: 'mp3', audioCodec: 'mp3', protocol: 'http' }, -] +// Transcoding targets in preference order (lossless first, then lossy). +// Derived from CODEC_PROBES to avoid duplicating MIME strings. +// MP3 is always included as a universal fallback. +const TRANSCODE_CODECS = ['flac', 'opus', 'mp3'] + +function probeSupported(audio, probes) { + return probes.filter(({ mime }) => audio.canPlayType(mime) === 'probably') +} export function detectBrowserProfile() { const audio = new Audio() - const directPlayProfiles = [] - for (const { codec, container, mime } of CODEC_PROBES) { - if (audio.canPlayType(mime) === 'probably') { - directPlayProfiles.push({ - containers: [container], - audioCodecs: [codec], - protocols: ['http'], + const directPlayProfiles = probeSupported(audio, CODEC_PROBES).map( + ({ codec, container }) => ({ + containers: [container], + audioCodecs: [codec], + protocols: ['http'], + }), + ) + + // Build transcoding profiles from supported codecs, always keeping mp3 as fallback + const transcodingProfiles = TRANSCODE_CODECS.reduce((profiles, codec) => { + const probe = CODEC_PROBES.find((p) => p.codec === codec) + if (audio.canPlayType(probe.mime) === 'probably' || codec === 'mp3') { + profiles.push({ + container: probe.container, + audioCodec: codec, + protocol: 'http', }) } - } + return profiles + }, []) return { name: 'NavidromeUI', platform: navigator.userAgent, directPlayProfiles, - transcodingProfiles: DEFAULT_TRANSCODING_PROFILES, + transcodingProfiles, codecProfiles: [], } } diff --git a/ui/src/transcode/browserProfile.test.js b/ui/src/transcode/browserProfile.test.js index 9764a0046..360ae7885 100644 --- a/ui/src/transcode/browserProfile.test.js +++ b/ui/src/transcode/browserProfile.test.js @@ -54,14 +54,49 @@ describe('detectBrowserProfile', () => { }) }) - it('includes transcoding profiles for common formats', () => { + it('filters transcoding profiles by canPlayType', () => { + mockCanPlayType.mockImplementation((mime) => { + if (mime === 'audio/mpeg') return 'probably' + if (mime === 'audio/ogg; codecs="opus"') return 'probably' + return '' + }) + + const profile = detectBrowserProfile() + const codecs = profile.transcodingProfiles.map((p) => p.audioCodec) + expect(codecs).toEqual(['opus', 'mp3']) + expect(codecs).not.toContain('flac') + profile.transcodingProfiles.forEach((p) => { + expect(p.protocol).toBe('http') + }) + }) + + it('always includes mp3 fallback in transcoding profiles', () => { mockCanPlayType.mockReturnValue('') const profile = detectBrowserProfile() - expect(profile.transcodingProfiles.length).toBeGreaterThan(0) + expect(profile.transcodingProfiles.length).toBe(1) + expect(profile.transcodingProfiles[0].audioCodec).toBe('mp3') expect(profile.transcodingProfiles[0].protocol).toBe('http') }) + it('does not duplicate mp3 when canPlayType supports it', () => { + mockCanPlayType.mockReturnValue('probably') + + const profile = detectBrowserProfile() + const mp3Count = profile.transcodingProfiles.filter( + (p) => p.audioCodec === 'mp3', + ).length + expect(mp3Count).toBe(1) + }) + + it('preserves transcoding profile preference order', () => { + mockCanPlayType.mockReturnValue('probably') + + const profile = detectBrowserProfile() + const codecs = profile.transcodingProfiles.map((p) => p.audioCodec) + expect(codecs).toEqual(['flac', 'opus', 'mp3']) + }) + it('sets codecProfiles to empty array', () => { mockCanPlayType.mockReturnValue('probably') From 94894fd511a02068a038bb112d6c162a63534a4c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 9 Mar 2026 17:19:03 -0400 Subject: [PATCH 58/60] chore(deps): bump docker/build-push-action in /.github/workflows (#5164) Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6 to 7. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/v6...v7) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pipeline.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/pipeline.yml b/.github/workflows/pipeline.yml index 3a24d27b1..eb7523d4e 100644 --- a/.github/workflows/pipeline.yml +++ b/.github/workflows/pipeline.yml @@ -221,7 +221,7 @@ jobs: hub_password: ${{ secrets.DOCKER_HUB_PASSWORD }} - name: Build Binaries - uses: docker/build-push-action@v6 + uses: docker/build-push-action@v7 with: context: . file: Dockerfile @@ -244,7 +244,7 @@ jobs: - name: Build and push image by digest id: push-image if: env.IS_LINUX == 'true' && env.IS_DOCKER_PUSH_CONFIGURED == 'true' && env.IS_ARMV5 == 'false' - uses: docker/build-push-action@v6 + uses: docker/build-push-action@v7 with: context: . file: Dockerfile From d76b49c6d1e2e262f65848ce5e81af0b5b36b805 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 9 Mar 2026 17:18:38 -0400 Subject: [PATCH 59/60] chore(deps): update golang.org/x/sync, golang.org/x/sys, golang.org/x/time, and go.opentelemetry.io/proto/otlp to latest versions Signed-off-by: Deluan --- go.mod | 8 ++++---- go.sum | 16 ++++++++-------- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/go.mod b/go.mod index 538b06f89..d50f738b7 100644 --- a/go.mod +++ b/go.mod @@ -64,11 +64,11 @@ require ( go.uber.org/goleak v1.3.0 golang.org/x/image v0.36.0 golang.org/x/net v0.51.0 - golang.org/x/sync v0.19.0 - golang.org/x/sys v0.41.0 + golang.org/x/sync v0.20.0 + golang.org/x/sys v0.42.0 golang.org/x/term v0.40.0 golang.org/x/text v0.34.0 - golang.org/x/time v0.14.0 + golang.org/x/time v0.15.0 gopkg.in/yaml.v3 v3.0.1 ) @@ -131,7 +131,7 @@ require ( github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834 // indirect github.com/valyala/fastjson v1.6.10 // indirect github.com/zeebo/xxh3 v1.1.0 // indirect - go.opentelemetry.io/proto/otlp v1.9.0 // indirect + go.opentelemetry.io/proto/otlp v1.10.0 // indirect go.uber.org/multierr v1.11.0 // indirect go.yaml.in/yaml/v2 v2.4.3 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect diff --git a/go.sum b/go.sum index 8f13089a3..d92075234 100644 --- a/go.sum +++ b/go.sum @@ -305,8 +305,8 @@ github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ= github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0= github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs= github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s= -go.opentelemetry.io/proto/otlp v1.9.0 h1:l706jCMITVouPOqEnii2fIAuO3IVGBRPV5ICjceRb/A= -go.opentelemetry.io/proto/otlp v1.9.0/go.mod h1:xE+Cx5E/eEHw+ISFkwPLwCZefwVjY+pqKg1qcK03+/4= +go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g= +go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= @@ -355,8 +355,8 @@ golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= -golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= +golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180926160741-c2ed4eda69e7/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -372,8 +372,8 @@ golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= -golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo= +golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE= golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4 h1:bTLqdHv7xrGlFbvf5/TXNxy/iUwwdkjhqQTJDjW7aj0= golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4/go.mod h1:g5NllXBEermZrmR51cJDQxmJUHUOfRAaNyWBM+R+548= @@ -399,8 +399,8 @@ golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= -golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= -golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U= +golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= From 844dffa2f1f45e8ac8ddd1b84420784c85bbea38 Mon Sep 17 00:00:00 2001 From: Deluan Date: Mon, 9 Mar 2026 18:26:07 -0400 Subject: [PATCH 60/60] fix: add 'opus' to the container aliases for improved direct play detection Signed-off-by: Deluan --- core/transcode/aliases.go | 2 +- core/transcode/decider_test.go | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/core/transcode/aliases.go b/core/transcode/aliases.go index 67a641511..0c9bfcb44 100644 --- a/core/transcode/aliases.go +++ b/core/transcode/aliases.go @@ -10,7 +10,7 @@ var containerAliasGroups = func() map[string]string { groups := [][]string{ {"aac", "adts", "m4a", "mp4", "m4b", "m4p"}, {"mpeg", "mp3", "mp2"}, - {"ogg", "oga"}, + {"ogg", "oga", "opus"}, {"aif", "aiff"}, {"asf", "wma"}, {"mpc", "mpp"}, diff --git a/core/transcode/decider_test.go b/core/transcode/decider_test.go index 280060fbf..cc9b5fb60 100644 --- a/core/transcode/decider_test.go +++ b/core/transcode/decider_test.go @@ -129,6 +129,18 @@ var _ = Describe("Decider", func() { Expect(decision.CanDirectPlay).To(BeTrue()) }) + It("handles container aliases (opus -> ogg)", func() { + mf := withProbe(&model.MediaFile{ID: "1", Suffix: "opus", Codec: "Opus", BitRate: 165, Channels: 2, SampleRate: 48000}) + ci := &ClientInfo{ + DirectPlayProfiles: []DirectPlayProfile{ + {Containers: []string{"ogg"}, AudioCodecs: []string{"opus"}, Protocols: []string{ProtocolHTTP}}, + }, + } + decision, err := svc.MakeDecision(ctx, mf, ci, DecisionOptions{}) + Expect(err).ToNot(HaveOccurred()) + Expect(decision.CanDirectPlay).To(BeTrue()) + }) + It("handles codec aliases (adts -> aac)", func() { mf := withProbe(&model.MediaFile{ID: "1", Suffix: "m4a", Codec: "AAC", BitRate: 256, Channels: 2}) ci := &ClientInfo{