mirror of
https://github.com/navidrome/navidrome.git
synced 2026-10-18 07:30:29 +00:00
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Deluan Quintão
|
758e64c999
|
feat(scanner): per-library PID configuration (#6252)
* feat(model): add per-library PID config columns * refactor(metadata): pass PID config to ToMediaFile and add spec validation * feat(scanner): rescan only libraries whose PID config changed * feat(server): validate library PID config and rescan on change * feat(ui): edit per-library PID config * fix(ui): label the PID mode selects * fix: tighten per-library PID rescan edge cases An interrupted PID rescan no longer upgrades every library to a full scan, a save that loses the race for the scanner logs at debug, the confirm dialog only shows when the effective PID spec changes, and it now gets translation keys. * refactor(metadata): pass the library to ToMediaFile ToMediaFile and core.Inspect took the library ID and its PID config as separate arguments, so a caller could mix values from two libraries. They now take the model.Library and resolve the effective PID config from it. * chore: tidy per-library PID comments, PropTypes and migration Trim comments that restated the code, add PropTypes to the new UI components, and recreate the migration with make migration-sql. * fix(ui): show the PID spec help under its input * feat(cmd): make inspect use the file's library PID config inspect always used the global PID config, so it showed different IDs than the scanner for files in a library with an override. It now finds the file's library in the DB and uses its effective config, falling back to the global config when there is no DB or the file is outside every library. It never creates a DB. The library path matcher moves from core/playlists to model so both can use it. * refactor: simplify per-library PID code Share the DB-file check between CLI commands, move ErrAlreadyScanning to model so core no longer imports scanner, read the libraries once for insights, and let ValidatePIDSpec accept an empty spec and look tags up directly. In the scanner, use FullScanInProgress instead of a second flag, and skip recomputing album IDs when the album spec did not change. In the UI, share the PID inputs between Create and Edit, and use docsUrl. * feat(ui): add section titles to Library Create and pre-fill Custom PID specs Custom now starts from the global spec, so admins edit a working spec instead of typing one from scratch. * fix(inspect): map files with the library-relative path the scanner uses Inspect gave metadata the file's directory as typed, so folder-based PIDs never matched the DB. It now uses the path relative to the library root, through the scanner's helper, which moves to model. * fix(scanner): say when a PID rescan only covers target folders * fix: reject tag aliases in album PID specs and match root libraries Tags are stored under canonical names, so an alias in a spec always reads as empty. In an album spec that gives every album the same ID, so album specs now require the tag name. Track specs keep accepting aliases, since the default one uses them. LibraryMatcher now matches paths under a library at the filesystem root. * refactor(model): move the tag alias lookup to tag_mappings.go * test: run the library matcher and inspect tests on Windows Build test paths with filepath instead of Unix literals, so they use the OS separator like filepath.Abs output, and drop the Windows skips. * feat(ui): add pt-BR translations for per-library PID settings |
||
|
Deluan Quintão
|
b293b96256
|
refactor(persistence): stateless repositories with per-call context (#6149)
* refactor(persistence): adopt generic deluan/rest repository API Pin deluan/rest to the refactor branch. REST-facing repository methods take a context and return typed values. Drop DataStore.Resource and ResourceRepository; the native API names typed repositories directly through a per-request adapter that later commits remove. * refactor(persistence): base repository helpers take a context * refactor(persistence): LibraryRepository takes a context per call * refactor(persistence): PropertyRepository takes a context per call * refactor(persistence): UserPropsRepository takes a context per call * refactor(persistence): TranscodingRepository takes a context per call * refactor(persistence): ShareRepository takes a context per call * refactor(persistence): PlayerRepository takes a context per call * refactor(persistence): RadioRepository takes a context per call * refactor(persistence): PlayQueueRepository takes a context per call * refactor(persistence): Tag and Genre repositories take a context per call * refactor(persistence): PluginRepository takes a context per call * refactor(persistence): Scrobble repositories take a context per call * refactor(persistence): FolderRepository takes a context per call * refactor(persistence): Artwork repositories take a context per call * refactor(persistence): UserRepository takes a context per call * refactor(persistence): ArtistRepository takes a context per call ReadAll no longer rewrites the shared sort mappings for the role filter; it works on a per-call copy. * test(persistence): assert artist role sort sanitization in ReadAll * refactor(persistence): AlbumRepository takes a context per call * test(persistence): pass the test context to album repository helpers * refactor(persistence): MediaFileRepository takes a context per call * refactor(persistence): Playlist repositories take a context per call * refactor(persistence): build all repositories once per store * refactor(core): REST repository wrappers are built once * refactor(persistence): repositories are stateless Remove the context field from the base repository and the per-request REST adapter. Enable the containedctx linter so no repository can hold a request context again. * chore(lint): skip containedctx in test files * refactor: share simplifications from the stateless repositories sweep Add deleteOwnedAll on sqlRepository and use it in player/share Delete to remove the duplicated bulk-delete loop; have Share.Repository() return model.ShareRepository so subsonic sharing.go drops its repeated type assertions. * chore(core): assert REST wrappers implement Persistable * chore: reformat imports * perf(persistence): build repositories on first use Each transaction store used to construct all 21 repositories up front, paying for filter and sort mapping setup the block never touched. Fields are now sync.OnceValue thunks, so a store only builds what it uses. * fix(persistence): clean plugin references per deleted user A bulk user delete that fails on a later id had already removed the earlier rows but skipped their plugin cleanup. Cleanup now runs right after each successful delete. * fix(core): unload disabled plugins even when a user delete fails A bulk delete can fail on a later id after earlier users were removed and their plugins auto-disabled. The wrapper returned before unloading, leaving those plugins running until the next successful delete or a restart. * chore(deps): pin deluan/rest to v1.0.1 Replaces the pseudo-version of the refactor branch with the tagged release. REST error messages now name the bare type (Artist, not model.Artist). * test: use the spec context instead of context.Background() Replace the context.Background()/context.TODO() calls this branch added to tests with the spec's ctx, GinkgoT().Context(), or t/b.Context(), so repository calls are bound to the running spec's lifetime. * test: declare the spec context once per Describe Set ctx from GinkgoT().Context() first in each top-level BeforeEach and reuse it, building user contexts on top of it instead of repeating inline calls. |
||
|
Deluan Quintão
|
8b4125267e
|
fix(artwork): only use image files as local artwork sources (#6180)
* fix(playlists): limit local cover paths to images in owner's libraries A local #EXTALBUMARTURL path (absolute or file://) was only checked against the union of all libraries. The artwork resolver then opened it with no further check and served the bytes as the playlist cover, undecoded. Any user who can upload an M3U could read any file under any library root, including libraries they were not granted, through getCoverArt (GHSA-vwq6-xrw5-phpg). resolveImageURL now requires an image extension, and for uploaded playlists (no folder) the library holding the cover must pass the owner's HasLibraryAccess. Scanner and CLI imports keep the all-libraries check, since those files are admin-controlled. resolveLocalFile, used by every file-backed artwork source, now ignores paths without an image extension, which covers playlists stored before this fix that were not resolved yet. openOriginal refuses a stored file-backed row whose path is not an image, so the existing dangling path re-resolves it and the playlist falls back to the generated grid. No migration is needed. * fix(artwork): skip non-image files matched by folder cover patterns Album and disc folder sources opened any file in the folder's image list that matched a cover pattern, without checking its extension. openOriginal now refuses to serve file-backed rows whose path is not an image, so a stored row like that would be refused, re-resolved to the same file, and refused again on every view. The list comes from the scanner, which only records image files, but a database scanned where the OS mime table knows more image types than the serving process could still reach this. Both fromExternalFile variants now skip matches that are not image files, so the album falls back to its next source instead. Also correct the parser comment: a playlist without a folder can come from an API upload or from a CLI import of a file outside all libraries. * fix(artwork): check stored source type before using the resize cache The image-extension check for file-backed rows ran inside openOriginal, which the resize cache skips on a hit. Before the fix, a resized request for a playlist pointing at a non-image file cached the raw bytes, because a failed resize falls back to the original data. After the upgrade the same request still hit that entry and returned the file. serveHash now refuses a file-backed row whose path is not an image before calling serveSource, so both full-size and resized requests go through dangling and re-resolve the item. The stale cache entry is keyed by the old hash and is no longer reachable once the row changes. * test: register mime_types.yaml in test binaries Artwork resolution now skips candidates that are not image files, and model.IsImageFile answers from the process mime table. The server registers the extra image types from resources/mime_types.yaml through a conf hook, but a test binary only does that if it links conf/mime, so the artwork e2e suite fell back to the host table: .jxl resolves on macOS and Linux and does not on Windows, where the #5950 cover spec then found no source. tests.Init now imports conf/mime for its side effect, so every suite that loads the test config sees the same image types as the server. * fix(artwork): drop the image-file guard from the disc art reader The guard was added to both fromExternalFile variants, but disc artwork keeps no state row and is never queued, so it cannot hit the refuse-and-re-resolve loop the guard exists to prevent. The only case where it can fire is a real image whose extension this process's mime table does not know, and there it drops a disc cover that used to work. The album variant keeps the guard, since those resolutions are stored and re-served. |
||
|
Deluan Quintão
|
dbd26ba2e7
|
perf(scanner): improve playlist importing on large libraries (#6055)
* perf(persistence): avoid a full media_file scan when resolving playlist paths FindByPaths built one OR-ed equality term per path. On the real media_file schema SQLite abandons the path index at just two OR-ed terms and falls back to SCAN media_file, re-testing every term against every row, so the cost grows with (rows x terms). Group the candidates by library and emit one IN list per library instead, which plans as SEARCH media_file USING INDEX media_file_path_nocase. The NOCASE collation is kept so ASCII case-insensitive matching still works. This is the dominant cost of M3U playlist import, which resolves every track on every scan. Measured with a 1000-track playlist against a migrated DB: 100k media_file rows: 397 -> 51,414 tracks/sec 500k media_file rows: 78.5 -> 47,174 tracks/sec The rate no longer degrades as the table grows, which is the expected shape for an index lookup. Reported in #6043, where an 8 hour scan of a 2M-song library spent 7h52m in the playlist phase. * docs(playlists): correct the stale reason for the M3U lookup chunk size The expression-tree depth ceiling applied to the old OR-per-path query, which capped a batch at roughly 500 terms. The IN form is bound by SQLite's 32766 variable limit instead, which the 400 candidates per chunk sit far below. |
||
|
Deluan Quintão
|
5d1c9530ab
|
feat(cli): add pls export/import subcommands for bulk playlist management (#5412)
* refactor: rename ImportFile to ImportFromFolder in playlists service * feat: add ImportFile method with library/folder resolution * feat: allow sync flag upgrade on re-import of non-synced playlists * feat: add pls export subcommand with bulk and single export Add `navidrome pls export` command that supports: - Single playlist export to stdout (-p flag only) - Single playlist export to directory (-p and -o flags) - Bulk export of all playlists to a directory (-o flag only) - Filtering by user (-u flag) - Automatic filename sanitization and collision detection Also extracts findPlaylist helper from runExporter for reuse. * feat: add pls import subcommand with sync flag support * fix: improve error message for export without output directory * test: add tests for ImportFile sync flag and sync upgrade behavior * refactor: streamline export and import logic by removing redundant comments and improving library path matching Signed-off-by: Deluan <deluan@navidrome.org> * feat: update ImportFile method to include sync flag for playlist imports Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement fetchPlaylists function to streamline playlist retrieval Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace inline filename sanitization with centralized utility function Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor playlist import logic to consolidate sync handling and improve method signatures Signed-off-by: Deluan <deluan@navidrome.org> * fix: address code review feedback on playlist import/export - Fix duplicate playlist creation on non-sync re-import: only reconcile sync flag when the playlist was actually persisted (has an ID) - Distinguish "not in any library" from real errors in resolveFolder using a sentinel error, so DB/folder errors propagate instead of falling back to ImportM3U - Use bufio.Scanner in countM3UTrackLines instead of reading entire file * feat: replace bufio.Scanner with UTF8Reader and LinesFrom utility for improved file reading Signed-off-by: Deluan <deluan@navidrome.org> * fix: record path for outside-library imports to prevent duplicates Files outside all libraries now go through updatePlaylist with the absolute path recorded, so re-importing the same file updates the existing playlist instead of creating a duplicate. * refactor: name guard condition in updatePlaylist for readability Extracted the compound boolean expression into a named local variable `alreadyImportedAndNotSynced` to make the intent of the early-return guard clearer at a glance. * add godocs Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan
|
d91b5e8f4d | refactor: simplify playlist name extraction using strings.CutPrefix | ||
|
Deluan Quintão
|
acd69f6a4f
|
feat(playlist): support #EXTALBUMARTURL directive and sidecar images (#5131)
* feat(playlist): add migration for playlist image field rename and external URL * refactor(playlist): rename ImageFile to UploadedImage and ArtworkPath to UploadedImagePath Rename playlist model fields and methods for clarity in preparation for adding external image URL and sidecar image support. Add the new ExternalImageURL field to the Playlist model. * feat(playlist): parse #EXTALBUMARTURL directive in M3U imports * feat(playlist): always sync ExternalImageURL on re-scan, preserve UploadedImage * feat(artwork): add sidecar image discovery and cache invalidation for playlists Add playlist sidecar image support to the artwork reader fallback chain. A sidecar image (e.g., MyPlaylist.jpg next to MyPlaylist.m3u) is discovered via case-insensitive base name matching using model.IsImageFile(). Cache invalidation uses max(playlist.UpdatedAt, imageFile.ModTime()) to bust stale artwork when sidecar or ExternalImageURL local files change. * feat(artwork): add external image URL source to playlist artwork reader Add fromPlaylistExternalImage source function that resolves playlist cover art from ExternalImageURL, supporting both HTTP(S) URLs (via the existing fromURL helper) and local file paths (via os.Open). Insert it in the Reader() fallback chain between sidecar and tiled cover. * refactor(artwork): simplify playlist artwork source functions Extract shared fromLocalFile helper, use url.Parse for scheme check, and collapse sidecar directory scan conditions. * test(artwork): remove redundant fromPlaylistSidecar tests These tests duplicated scenarios already covered by findPlaylistSidecarPath tests combined with fromLocalFile (tested via fromPlaylistExternalImage). After refactoring fromPlaylistSidecar to a one-liner composing those two functions, the wrapper tests add no value. * fix(playlist): address security review comments from PR #5131: - Use url.PathUnescape instead of url.QueryUnescape for file:// URLs so that '+' in filenames is preserved (not decoded as space). - Validate all local image paths (file://, absolute, relative) against known library boundaries via libraryMatcher, rejecting paths outside any configured library. - Harden #EXTALBUMARTURL against path traversal and SSRF by adding EnableM3UExternalAlbumArt config flag (default false, also disabled by EnableExternalServices=false) to gate HTTP(S) URL storage at parse time and fetching at read time (defense in depth). - Log a warning when os.ReadDir fails in findPlaylistSidecarPath for diagnosability. - Extract resolveLocalPath helper to simplify resolveImageURL. Signed-off-by: Deluan <deluan@navidrome.org> * feat(playlist): implement human-friendly filename generation for uploaded playlist cover images Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
7ad2907719
|
refactor: move playlist business logic from repositories to service layer (#5027)
* refactor: move playlist business logic from repositories to core.Playlists service Move authorization, permission checks, and orchestration logic from playlist repositories to the core.Playlists service, following the existing pattern used by core.Share and core.Library. Changes: - Expand core.Playlists interface with read, mutation, track management, and REST adapter methods - Add playlistRepositoryWrapper for REST Save/Update/Delete with permission checks (follows Share/Library pattern) - Simplify persistence/playlist_repository.go: remove isWritable(), auth checks from Delete()/Put()/updatePlaylist() - Simplify persistence/playlist_track_repository.go: remove isTracksEditable() and permission checks from Add/Delete/Reorder - Update Subsonic API handlers to route through service - Update Native API handlers to accept core.Playlists instead of model.DataStore * test: add coverage for playlist service methods and REST wrapper Add 30 new tests covering the service methods added during the playlist refactoring: - Delete: owner, admin, denied, not found - Create: new playlist, replace tracks, admin bypass, denied, not found - AddTracks: owner, admin, denied, smart playlist, not found - RemoveTracks: owner, smart playlist denied, non-owner denied - ReorderTrack: owner, smart playlist denied - NewRepository wrapper: Save (owner assignment, ID clearing), Update (owner, admin, denied, ownership change, not found), Delete (delegation with permission checks) Expand mockedPlaylistRepo with Get, Delete, Tracks, GetWithTracks, and rest.Persistable methods. Add mockedPlaylistTrackRepo for track operation verification. * fix: add authorization check to playlist Update method Added ownership verification to the Subsonic Update endpoint in the playlist service layer. The authorization check was present in the old repository code but was not carried over during the refactoring to the service layer, allowing any authenticated user to modify playlists they don't own via the Subsonic API. Also added corresponding tests for the Update method's permission logic. * refactor: improve playlist permission checks and error handling, add e2e tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename core.Playlists to playlists package and update references Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename playlists_internal_test.go to parse_m3u_test.go and update tests; add new parse_nsp.go and rest_adapter.go files Signed-off-by: Deluan <deluan@navidrome.org> * fix: block track mutations on smart playlists in Create and Update Create now rejects replacing tracks on smart playlists (pre-existing gap). Update now uses checkTracksEditable instead of checkWritable when track changes are requested, restoring the protection that was removed from the repository layer during the refactoring. Metadata-only updates on smart playlists remain allowed. * test: add smart playlist protection tests to ensure readonly behavior and mutation restrictions * refactor: optimize track removal and renumbering in playlists Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement track reordering in playlists with SQL updates Signed-off-by: Deluan <deluan@navidrome.org> * refactor: wrap track deletion and reordering in transactions for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unused getTracks method from playlistTrackRepository Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize playlist track renumbering with CTE-based UPDATE Replace the DELETE + re-INSERT renumbering strategy with a two-step UPDATE approach using a materialized CTE and ROW_NUMBER() window function. The previous approach (SELECT all IDs, DELETE all tracks, re-INSERT in chunks of 200) required 13 SQL operations for a 2000-track playlist. The new approach uses just 2 UPDATEs: first negating all IDs to clear the positive space, then assigning sequential positions via UPDATE...FROM with a CTE. This avoids the UNIQUE constraint violations that affected the original correlated subquery while reducing per-delete request time from ~110ms to ~12ms on a 2000-track playlist. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename New function to NewPlaylists for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update mock playlist repository and tests for consistency Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |