23 Commits

Author SHA1 Message Date
Deluan Quintão
e6560ccb40
fix(cache): don't serve partially-written transcodes after a crash (#5657)
* feat(cache): add completion marker helpers to spreadFS

* feat(cache): write completion marker after successful cache write

* fix(cache): adopt only complete files on reload, grandfather existing caches

* test(cache): regression tests for partial-transcode crash leftover (#5636)

* test(cache): guard concurrent in-progress streaming with completion marker

* test(cache): make concurrent-streaming guard actually attach a second reader mid-write

The previous test obtained s2 only after pw.Close(), so no reader ever
attached to the in-progress entry. Now pw.Write("hello ") is called
synchronously before the second Get — io.Pipe's blocking write gives a
deterministic happens-before — then both s1 and s2 are drained in parallel
goroutines while the producer writes the rest and closes the pipe.

* style(cache): clarify best-effort intent of cleanup os.Remove calls

* refactor(cache): lift one-time grandfather pass out of Reload's steady-state loop

* refactor(cache): have MarkComplete take the key, owning path mapping in spreadFS

* test(cache): assert no completion marker is written when the write fails

* refactor(cache): rename migration sentinel to generic .nd-migrated

* refactor(cache): rename grandfather migration to migrateExistingFiles

* refactor(cache): single-pass Reload with safer marker-error handling

Address PR review feedback:
- Merge the one-time migration into Reload's single directory walk,
  avoiding a second full walk on first boot.
- Only delete a data file when its marker is definitively absent
  (os.IsNotExist); skip on other stat errors to avoid destroying valid
  entries under transient I/O failures.
- Write the migration sentinel only after a clean walk, so a partial
  walk can't strand valid-but-unmarked files for later deletion.
- Return early from walkDataFiles on a WalkDir error.
- Assert fs.Create error in the marker-removal test.
2026-06-23 22:23:54 -04:00
Deluan Quintão
8f0b4930ff
refactor(conf): replace eager dir creation with lazy Dir type (#5495)
* feat(conf): add Dir type with lazy directory creation

Introduces the Dir type that wraps a directory path string and defers
os.MkdirAll until the first call to Path() or MustPath(), using sync.Once
to ensure the creation happens exactly once. Implements fmt.Stringer,
encoding.TextMarshaler, and encoding.TextUnmarshaler for config integration.
Includes Ginkgo/Gomega tests covering all methods and error paths.

* refactor(conf): replace eager dir creation with lazy Dir type

Change DataFolder, CacheFolder, Plugins.Folder, and Backup.Path from
string to Dir. Remove all os.MkdirAll calls from Load() so directories
are created lazily on first Path()/MustPath() call. Artwork folder
creation was already handled at point-of-use in image_upload.go.

Add SnapshotConfig() to conf package for safe test config save/restore
that avoids copying sync.Once inside Dir fields. Fix copy-lock vet
warning in nativeapi/config.go by marshalling pointer instead of value.

* refactor(conf): migrate tests and db init to lazy Dir type

Update all test files to use conf.NewDir() for Dir field assignments.
Ensure DataFolder is created lazily when the database is first opened
in db.Db(). Remove eager directory creation from conf.Load() tests.

* fix(conf): address review findings for Dir type

- Use os.ModePerm for DataFolder/CacheFolder (was 0700, should match
  original behavior). Add NewDirWithPerm for PluginsFolder (0700).
- Use Path() instead of MustPath() in db.Prune() to avoid logFatal
  from background cron job.
- Panic on marshal/unmarshal errors in SnapshotConfig (test helper).
- Clean up redundant String()/MustPath() calls in plugin manager.
- Remove dead code in dir_test.go.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(conf): add GoString to Dir for clean config dump output

Implement fmt.GoStringer on Dir so pretty.Sprintf shows the path
string instead of internal struct fields (sync.Once, perm, err).
Also add TODO comment to configtest about removing the indirection.

* fix(dir): improve error logging in MustPath method

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(tests): remove redundant tests for unwritable DataFolder and CacheFolder

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(conf): address PR review feedback

- Ensure Plugins.Folder always uses 0700, even when user-configured
  (previously only the derived default got restrictive permissions).
- Create LogFile parent directory before opening, so LogFile paths
  inside a not-yet-created DataFolder work correctly.

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-05-13 17:44:22 -03:00
Deluan
de698918ac Revert "fix(server): failed transcoded files should not be cached (#4124)"
This reverts commit 9dd5a8c3347f5223d138d4279e3d51e5def4edce.
2025-05-27 19:53:10 -04:00
Deluan Quintão
9dd5a8c334
fix(server): failed transcoded files should not be cached (#4124)
* Close stream on caching errors

* fix(test): replace errPartialReader with errFakeReader to fix lint error

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(test): update error assertion to check for substring in closed file error

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-05-26 20:30:26 -04:00
Deluan Quintão
6731787053
fix(server): memory leak in cache warmer (#4095)
* Prevent cache warmer memory leak when cache disabled

* refactor(tests): replace disabledCache with mockFileCache in CacheWarmer tests

Signed-off-by: Deluan <deluan@navidrome.org>

* test(cache): enhance CacheWarmer tests for initialization, buffer management, and error handling

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-05-21 21:48:49 -04:00
Deluan Quintão
cd0cf7c12b
feat: cache login background images (#3462)
* feat: use direct links to unsplash for background images

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: cache images from unsplash

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: use cache.HTTPClient to reduce complexity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove magic numbers

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2024-11-08 20:23:30 -05:00
Deluan
c95fa11a2f Remove potential integer overflow conversion uint64 -> int64 2024-08-22 19:28:22 -04:00
tomleb
257ccc5f43
Allow configuring cache folder (#2357)
* Set all clients to dev_download for make get-music

* Use multiple TranscodingCache instances in tests

This fixes flaky tests. The issue is that the TranscodingCache object
was being reused in tests from media_stream_Internal_test.go and
media_stream_test.go. If tests from the former was run first, the cache
would be filled up, so that when running tests from the latter, the `NON
seekable` test would fail.

* Allow configuring cache folder

This commit introduces a new configuration option to configure the cache
folder. This allows the cache to be in a separate folder such as
/var/cache/navidrome on Linux distributions.

* Fix tests

* Removed unused test setup code

---------

Co-authored-by: Deluan <deluan@deluan.com>
Co-authored-by: Deluan <deluan@navidrome.org>
2023-06-02 17:14:11 -04:00
Deluan
bd402fb2a8 Fix IntelliJ warning 2023-04-04 13:01:32 -04:00
Deluan
6dce4b2478 Remove custom atomic.Bool, we are now at Go 1.19 2023-02-15 21:18:24 -05:00
Deluan
bfaf4a3388 Add logs to cache hunter 2023-02-06 10:41:33 -05:00
Deluan
4a7e86e989 Fix file descriptor leaking. 2023-02-02 10:36:49 -05:00
Deluan
580e9ae4bd Fix timer going awry 2023-01-17 22:04:09 -05:00
Deluan
cc14485194 When trying to PreCache, wait for ImageCache to be available 2022-12-28 23:26:39 -05:00
Deluan
52a4721c91 Remove empty (invalid) entries from the cache 2022-12-28 15:31:56 -05:00
Deluan
9ec349dce0 Make sure album is updated if external cover changes 2022-12-28 15:31:56 -05:00
Deluan
73bb0104f0 Cache original images 2022-12-28 15:31:56 -05:00
Deluan Quintão
24d520882e
Don't cache transcoded files if the request was cancelled (#2041)
* Don't cache transcoded files if the request was cancelled (or there was a transcoding error)

* Add context to logs

* Simplify Wait error handling

* Fix flaky test

* Change log level for "populating cache" error message

* Small cleanups
2022-12-18 12:22:12 -05:00
Deluan
94e36d7f60 Remove old feature flag for cache layout 2021-07-02 10:04:41 -04:00
Deluan
452c8dc44b Fixed the enduring nasty "too many files open" bug!! Fix #446 2021-07-01 12:07:32 -04:00
Deluan
467eb345ad Don't panic if fscache could not be initialized due to a FS error 2021-04-21 23:39:23 -04:00
Deluan
31b553e972 Add missing error log message in fscache initialization 2021-04-21 14:15:42 -04:00
Deluan
5fdd8b32d7 Move utilitarian/generic packages to utils: lastfm, spotify, gravatar, cache, and pool 2021-02-09 15:33:33 -05:00