mirror of
https://github.com/navidrome/navidrome.git
synced 2026-10-18 07:30:29 +00:00
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Deluan Quintão
|
4b60b21316
|
fix(scanner): read file birth time via statx on Linux (#6046)
* fix(scanner): read file birth time via statx on Linux On Linux the file birth time is only reachable through statx(2). We were reading it with times.Get(), which looks only at the plain stat() result, where the field does not exist: djherbis/times declares HasBirthTime=false for Linux, so the check was always false and every file fell back to time.Now(). This has been the case since #2553 introduced the feature, which means that PR was a no-op on Linux from day one. macOS and Windows were never affected, as there the birth time does come back from plain stat. BirthTime() now tries times.Get() first, which costs no syscall and is already correct on macOS, Windows and BSD, and only falls back to times.Stat() on the path when that comes back empty. Ordering matters: on Windows times.Stat() opens the file asking for FILE_WRITE_ATTRIBUTES, which fails on a read-only share before falling back. Not every filesystem stores a birth time. Measured with a probe over real mounts: ext4, SMB/CIFS and mergerfs report one, while NFS and rclone/FUSE never do. Asking those on every file is pure overhead, so a miss is remembered per device on the localFS and skipped from then on. The memo is keyed by device rather than by library, so a library spanning two mounts does not lose birth times on the mount that does support them. Cost of the extra call is ~2us per file against ~52us just to open a file for tag reading, so 0.23s across a 97k-file library, and only for files whose tags are actually read. Existing rows keep their current birth_time: the repository drops that column on update, so only newly added files get the real value. * fix(scanner): return the device id opaquely to satisfy unconvert st.Dev is uint64 on Linux and int32 on darwin, so a uint64() cast is redundant on one and required on the other. Returning it as an opaque value drops the cast entirely, which also removes the gosec suppression that came with it. The value is only ever used as a sync.Map key. |
||
|
Deluan Quintão
|
7fa13761d7
|
fix(scanner): resolve file symlinks with the production local storage FS (#5755)
* fix(scanner): resolve file symlinks with the production local storage FS The symlink classification added for GHSA-r5qr-m328-qcf4 relied on fs.ReadLink, but the local storage FS wraps os.DirFS behind the fs.FS interface, hiding its ReadLinkFS implementation. Every resolution failed at the first hop, so the scanner silently skipped ALL file symlinks, regardless of target or the FollowSymlinks setting. Libraries made of symlinks (e.g. shared-pool setups) lost all their tracks after upgrading to 0.63. The local storage now exposes full OS-level resolution (EvalSymlinks) through a new optional storage.SymlinkResolverFS interface, which the scanner prefers over the fs.ReadLink hop loop. This also classifies a chain by its FINAL target even when it passes through an audio-named intermediate outside the library, closing a bypass the hop loop had. Regular (non-symlink) entries keep the same early-return path, so scan performance is unaffected for normal libraries. Fixes #5752 * fix(test): keep watcher specs off the real local storage The watcher specs spawn watchLibrary goroutines that are not joined on spec teardown. Now that the scanner test binary registers the file:// storage, those leaked goroutines reached newLocalStorage, which reads conf.Server on construction, racing with the configtest cleanup that restores the config snapshot (caught by CI's race detector). Point the mock libraries at a fake storage scheme, which never touches the config and does not support watching, so the goroutine exits immediately. * fix(storage): reject invalid fs paths in ResolveSymlink Defense-in-depth for the SymlinkResolverFS contract: names must be valid fs.FS paths. A lexical ".." in the name would otherwise escape the library root via filepath.Join cleaning. No current caller can produce such a name (they come from ReadDir walks), but the guard enforces the documented contract at the boundary. |
||
|
Deluan Quintão
|
06993a8e04
|
test(storage): re-enable local storage tests on Windows (#5654)
* refactor(storage): extract LocalPathToURL from storage.For * test(storage): re-enable local storage tests on Windows (#5381) * test(storage): fix Windows drive-letter path expectation The 'should handle Windows drive letters correctly' test was gated behind the now-removed SkipOnWindows BeforeEach, so its expectation had never run. On Windows, newLocalStorage re-joins u.Host+u.Path via filepath.Join, which yields a backslash path (C:\music), not C:/music. Assert against filepath.Join so the expectation matches the OS-native result. * test(storage): probe Windows drive-letter path in LocalPathToURL Three review bots flagged that LocalPathToURL escapes the drive-letter colon (C: -> C%3A), which url.Parse rejects. There are no Windows bug reports, so add a Windows-gated test that exercises the real conversion on a drive-letter path and let CI decide whether the bug is real before changing production code. |
||
|
Deluan Quintão
|
64c8d3f4c5
|
ci: run Go tests on Windows (#5380)
* ci(windows): add skeleton go-windows job (compile-only smoke test)
* ci(windows): fix comment to reference Task 7 not Task 6
* ci(windows): harden PATH visibility and set explicit bash shell
* ci(windows): enable full go test suite and ndpgen check
* test(gotaglib): skip Unix-only permission tests on Windows
* test(lyrics): skip Windows-incompatible tests
* test(utils): skip Windows-incompatible tests
* test(mpv): skip Windows-incompatible playback tests
Skip 3 subprocess-execution tests that rely on Unix-style mpv
invocation; .bat output includes \r-terminated lines that break
argument parsing (#TBD-mpv-windows).
* test(storage): skip Windows-incompatible tests
Skip relative-path test where filepath.Join uses backslash but the
storage implementation returns a forward-slash URL path
(#TBD-path-sep-storage).
* test(storage/local): skip Windows-incompatible tests
Skip 13 tests that fail because url.Parse("file://" + windowsPath)
treats the drive letter colon as an invalid port; also skip the
Windows drive-letter path test that exposes a backslash vs
forward-slash normalisation bug (#TBD-path-sep-storage-local).
* test(playlists): skip Windows-incompatible tests
* test(model): skip Windows-incompatible tests
* test(model/metadata): skip Windows-incompatible tests
* test(core): skip Windows-incompatible tests
AbsolutePath uses filepath.Join which produces OS-native path separators;
skip the assertion test on Windows until the production code is fixed
(#TBD-path-sep-core).
* test(artwork): skip Windows-incompatible tests
Artwork readers produce OS-native path separators on Windows while tests
assert forward-slash paths; skip 11 affected tests pending a fix in
production code (#TBD-path-sep-artwork).
* test(persistence): skip Windows-incompatible tests
Skip flaky timestamp comparison (#TBD-flake-persistence) and path-separator
real-bugs (#TBD-path-sep-persistence) in FolderRepository.GetFolderUpdateInfo
which uses filepath.Clean/os.PathSeparator converting stored forward-slash paths
to backslashes on Windows.
* test(scanner): skip Windows-incompatible tests
Skip symlink tests (Unix-assumption), ndignore path-separator bugs
(#TBD-path-sep-scanner) in processLibraryEvents/resolveFolderPath where
filepath.Rel/filepath.Split return backslash paths incompatible with fs.FS
forward-slash expectations, error message mismatch on Windows, and file
format upgrade detection (#TBD-path-sep-scanner).
* test(plugins): skip Windows-incompatible tests
Add //go:build !windows tags to test files that reference the suite
bootstrap (testManager, testdataDir, createTestManager) which is only
compiled on non-Windows. Add a Windows-only suite stub that skips all
specs via BeforeEach to prevent [build failed] on Windows CI.
* test(server): skip Windows-incompatible tests
Skip createUnixSocketFile tests that rely on Unix file permission bits
(chmod/fchmod) which are not supported on Windows.
* test(nativeapi): skip Windows-incompatible tests
Skip the i18n JSON validation test that uses filepath.Join to build
embedded-FS paths; filepath.Join produces backslashes on Windows which
breaks fs.Open (embedded FS always uses forward slashes).
* test(e2e): skip Windows-incompatible tests
On Windows, SQLite holds file locks that prevent the Ginkgo TempDir
DeferCleanup from deleting the DB file. Register an explicit db.Close
DeferCleanup (LIFO before TempDir cleanup) on Windows so the file lock
is released before the temp directory is removed.
* test(windows): fix e2e AfterSuite and skip remaining scanner path test
* test(scanner): skip another Windows path-sep test (#TBD-path-sep-scanner)
* test(subsonic): skip timing-flaky test on Windows (#TBD-flake-time-resolution-subsonic)
* test(scanner): skip 'detects file moved to different folder' on Windows
* test(scanner): consolidate 'Library changes' Windows skips into BeforeEach
* test(scanner): close DB before TempDir cleanup to fix Windows file lock
* test(scanner): skip ScanFolders suite on Windows instead of closing shared DB
* ci: retrigger for Windows soak run 2/3
* ci: retrigger for Windows soak run 3/3
* ci: retrigger for Windows soak run 3/3 (take 2)
* test(scanner): skip Multi-Library suite on Windows (SQLite file lock)
* ci(windows): promote go-windows to blocking status check
* test(plugins): run platform-neutral specs on Windows, drop blanket Skip
* test(windows): make tests cross-platform instead of skipping
- subsonic: back-date submissionTime baseline by 1s so
BeTemporally(">") passes under millisecond clock resolution
- persistence: sleep briefly between Put calls so UpdatedAt is
strictly after CreatedAt on low-resolution clocks
- utils/files: close tempFile before os.Remove so the test works on
Windows (where an open handle holds a file lock)
- tests.TempFile: close the handle before returning; metadata tests
no longer leak the open file into Ginkgo's TempDir cleanup
Resolves Copilot review comments on #5380.
* test(tests): add SkipOnWindows helper to reduce boilerplate
Introduces tests.SkipOnWindows(reason) that wraps the 3-line
runtime.GOOS guard pattern used in every Windows-skipped spec.
* test(adapters): use tests.SkipOnWindows helper
* test(core): use tests.SkipOnWindows helper
* test(model): use tests.SkipOnWindows helper
* test(persistence): use tests.SkipOnWindows helper
* test(scanner): use tests.SkipOnWindows helper
* test(server): use tests.SkipOnWindows helper
* test(plugins): run pure-Go unit tests on Windows
config_validation_test, manager_loader_test, and migrate_test have no
WASM/exec dependencies and don't rely on the make-built test plugins
from plugins_suite_test.go. Let them run on Windows too.
|
||
|
Deluan Quintão
|
0a6b5519cc
|
refactor(scanner): remove C++ taglib adapter (#5349)
* refactor(build): remove CPP taglib adapter Remove the CGO-based TagLib adapter (adapters/taglib/) and all cross-taglib build infrastructure. The WASM-based go-taglib adapter (adapters/gotaglib/) is now the sole metadata extractor. - Delete adapters/taglib/ (CPP/CGO wrapper) - Delete .github/actions/download-taglib/ - Remove CROSS_TAGLIB_VERSION, CGO_CFLAGS_ALLOW, and all taglib-related references from Dockerfile, Makefile, CI pipeline, and devcontainer * fix(scanner): gracefully fallback to default extractor instead of crashing Replace log.Fatal with a graceful fallback when the configured scanner extractor is not found. Instead of terminating the process, the code now warns and falls back to the default taglib extractor using the existing consts.DefaultScannerExtractor constant. A fatal log is retained only for the case where the default extractor itself is not registered, which indicates a broken build. * test(scanner): cover default extractor fallback and suppress redundant warn Address review feedback on the extractor fallback in newLocalStorage: - Only log the "using default" warning when the configured extractor differs from the default, so a broken build (default extractor itself missing) logs only the fatal — not a misleading "falling back" warn followed immediately by the fatal. - Add a unit test that registers a mock under consts.DefaultScannerExtractor, sets the configured extractor to an unknown name, and asserts the local storage is constructed using the default extractor's constructor. |
||
|
Deluan Quintão
|
0da2352907
|
fix: improve URL path handling in local storage for special characters (#4378)
* refactor: improve URL path handling in local storage system Enhanced the local storage implementation to properly handle URL-decoded paths and fix issues with file paths containing special characters. Added decodedPath field to localStorage struct to separate URL parsing concerns from file system operations. Key changes: - Added decodedPath field to localStorage struct for proper URL decoding - Modified newLocalStorage to use decoded path instead of modifying original URL - Fixed Windows path handling to work with decoded paths - Improved URL escaping in storage.For() to handle special characters - Added comprehensive test suite covering URL decoding, symlink resolution, Windows paths, and edge cases - Refactored test extractor to use mockTestExtractor for better isolation This ensures that file paths with spaces, hash symbols, and other special characters are handled correctly throughout the storage system. Signed-off-by: Deluan <deluan@navidrome.org> * fix(tests): fix test file permissions and add missing tests.Init call * refactor(tests): remove redundant test Signed-off-by: Deluan <deluan@navidrome.org> * fix: URL building for Windows and remove redundant variable Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify URL path escaping in local storage Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |