508 Commits

Author SHA1 Message Date
Deluan
e222d8d8db refactor(scanner): enhance walkDirTree to support target folder scanning
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
08bccad020 docs(scanner): comments
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
587249bf81 feat(watcher): enhance .ndignore handling for folder deletions and file changes
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
7ed817c1c0 feat(configuration): add DevSelectiveWatcher configuration option
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
71acd02d1b refactor(tests): rename selective scan test file to scanner_selective_test.go
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
78904257df refactor(scanner): move initialization logic from phase_1 to the scanner itself
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
b9f368ad0d refactor(scanner): streamline phase folder and album creation by removing unnecessary library parameter
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
fadaaf3ad1 refactor(scanner): simplify selective scan logic using slice.Filter
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
bfa31a246f refactor(scanner): update scanner interface and implementations to use model.Scanner
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
ed781e8da0 refactor: move scanner types to model,implement MockScanner
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
54f19c598a refactor(scanner): move scanner.ScanTarget to model.ScanTarget
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
dec658c235 refactor(scanner): streamline scanning logic by removing scanAll method
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
7e396fd6ce refactor(scanner): update CallScan to handle specific library/folder pairs
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
265a6973bc feat(subsonic): allow selective scan in the /startScan endpoint
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
847cc92e88 feat(scanner): update folder scanning to include all descendants of specified folders
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
33704edc1c refactor(scanner): simplify GC method by removing library ID parameter
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
f64b51f161 refactor(scanner): enhance ScanTarget struct with String method for better target representation
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
b06d90a4db refactor(ignore_checker): rename scanner to lineScanner for clarity
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
0b06ecb034 refactor(scanner): implement IgnoreChecker for managing .ndignore patterns
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
3cbbb440c2 feat(watcher): implement path ignoring based on .ndignore patterns
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
b361837f50 refactor(watcher): add resolveFolderPath function for testability
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
1bd58a14ab feat(watcher): implement folder scanning notifications with deduplication
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
712bf2815e refactor(scanner): simplify scanner methods
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
1250647dc3 refactor(folder): enhance external scanner to support target-specific scanning
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
f1dbd88024 feat(folder): replace GetByPaths with GetFolderUpdateInfo for improved folder updates retrieval
Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-12 16:19:18 -05:00
Deluan
8f1a6116fe feat: Add selective folder scanning capability
Implement targeted scanning of specific library/folder pairs without
full recursion. This enables efficient rescanning of individual folders
when changes are detected, significantly reducing scan time for large
libraries.

Key changes:
- Add ScanTarget struct and ScanFolders API to Scanner interface
- Implement CLI flag --targets for specifying libraryID:folderPath pairs
- Add FolderRepository.GetByPaths() for batch folder info retrieval
- Create loadSpecificFolders() for non-recursive directory loading
- Scope GC operations to affected libraries only (with TODO for full impl)
- Add comprehensive tests for selective scanning behavior

The selective scan:
- Only processes specified folders (no subdirectory recursion)
- Maintains library isolation
- Runs full maintenance pipeline scoped to affected libraries
- Supports both full and quick scan modes

Examples:
  navidrome scan --targets "1:Music/Rock,1:Music/Jazz"
  navidrome scan --full --targets "2:Classical"
2025-11-12 16:19:18 -05:00
Deluan Quintão
b2019da999
chore(deps): update all dependencies (#4618)
* chore: update to Go 1.25.3

Signed-off-by: Deluan <deluan@navidrome.org>

* chore: update to golangci-lint

Signed-off-by: Deluan <deluan@navidrome.org>

* chore: update go dependencies

Signed-off-by: Deluan <deluan@navidrome.org>

* chore: update vite dependencies in package.json and improve EventSource mock in tests

- Upgraded @vitejs/plugin-react to version 5.1.0 and @vitest/coverage-v8 to version 4.0.3.
- Updated vite to version 7.1.12 and vite-plugin-pwa to version 1.1.0.
- Enhanced the EventSource mock implementation in eventStream.test.js for better test isolation.

* ci: remove coverage flag from Go test command in pipeline

* chore: update Node.js version to v24 in devcontainer, pipeline, and .nvmrc

* chore: prettier

Signed-off-by: Deluan <deluan@navidrome.org>

* chore: update actions/checkout from v4 to v5 in pipeline and update-translations workflows

* chore: update JS dependencies remove unused jest-dom import in Linkify.test.jsx

* chore: update actions/download-artifact from v4 to v5 in pipeline

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-10-25 17:05:16 -04:00
Deluan Quintão
00c83af170
feat: Multi-library support (#4181)
* feat(database): add user_library table and library access methods

Signed-off-by: Deluan <deluan@navidrome.org>

# Conflicts:
#	tests/mock_library_repo.go

* feat(database): enhance user retrieval with library associations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): implement library management and user-library association endpoints

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): restrict access to library and config endpoints to admin users

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(library): implement library management service and update API routes

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(database): add library filtering to album, folder, and media file queries

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor library service to use REST repository pattern and remove CRUD operations

Signed-off-by: Deluan <deluan@navidrome.org>

* add total_duration column to library and update user_library table

Signed-off-by: Deluan <deluan@navidrome.org>

* fix migration file name

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): add library management features including create, edit, delete, and list functionalities - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): enhance library validation and management with path checks and normalization - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): improve library path validation and error handling - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* use utils/formatBytes

Signed-off-by: Deluan <deluan@navidrome.org>

* simplify DeleteLibraryButton.jsx

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): enhance validation messages and error handling for library paths

Signed-off-by: Deluan <deluan@navidrome.org>

* lint

Signed-off-by: Deluan <deluan@navidrome.org>

* test(scanner): add tests for multi-library scanning and validation

Signed-off-by: Deluan <deluan@navidrome.org>

* test(scanner): improve handling of filesystem errors and ensure warnings are returned

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(controller): add function to retrieve the most recent scan time across all libraries

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): add additional fields and restructure LibraryEdit component for enhanced statistics display

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): enhance LibraryCreate and LibraryEdit components with additional props and styling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(mediafile): add LibraryName field and update queries to include library name

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(missingfiles): add library filter and display in MissingFilesList component

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): implement scanner interface for triggering library scans on create/update

Signed-off-by: Deluan <deluan@navidrome.org>

# Conflicts:
#	cmd/wire_gen.go
#	cmd/wire_injectors.go

# Conflicts:
#	cmd/wire_gen.go

# Conflicts:
#	cmd/wire_gen.go
#	cmd/wire_injectors.go

* feat(library): trigger scan after successful library deletion to clean up orphaned data

Signed-off-by: Deluan <deluan@navidrome.org>

* rename migration file for user library table to maintain versioning order

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: move scan triggering logic into a helper method for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): add library path and name fields to album and mediafile models

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): add/remove watchers on demand, not only when server starts

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scanner): streamline library handling by using state-libraries for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: track processed libraries by updating state with scan timestamps

Signed-off-by: Deluan <deluan@navidrome.org>

* prepend libraryID for track and album PIDs

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(repository): apply library filtering in CountAll methods for albums, folders, and media files

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(user): add library selection for user creation and editing

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): implement library selection functionality with reducer and UI component

Signed-off-by: Deluan <deluan@navidrome.org>

# Conflicts:
#	.github/copilot-instructions.md

# Conflicts:
#	.gitignore

* feat(library): add tests for LibrarySelector and library selection hooks

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add unit tests for file utility functions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): add library ID filtering for album resources

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): streamline library ID filtering in repositories and update resource filtering logic

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(repository): add table name handling in filter functions for SQL queries

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): add refresh functionality on LibrarySelector close

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(artist): add library ID filtering for artists in repository and update resource filtering logic

Signed-off-by: Deluan <deluan@navidrome.org>

# Conflicts:
#	persistence/artist_repository.go

* Add library_id field support for smart playlists

- Add library_id field to smart playlist criteria system
- Supports Is and IsNot operators for filtering by library ID
- Includes comprehensive test coverage for single values and lists
- Enables creation of library-specific smart playlists

* feat(subsonic): implement user-specific library access in GetMusicFolders

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(library): enhance LibrarySelectionField to extract library IDs from record

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): update GetIndexes and GetArtists method to support library ID filtering

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: ensure LibrarySelector dropdown refreshes on button close

Added refresh() call when closing the dropdown via button click to maintain
consistency with the ClickAwayListener behavior. This ensures the UI
updates properly regardless of how the dropdown is closed, fixing an
inconsistent refresh behavior between different closing methods.

The fix tracks the previous open state and calls refresh() only when
the dropdown was open and is being closed by the button click.

* refactor: simplify getUserAccessibleLibraries function and update related tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance selectedMusicFolderIds function to handle valid music folder IDs and improve fallback logic

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: change ArtistRepository.GetIndex to accept multiple library IDs

Updated the GetIndex method signature to accept a slice of library IDs instead of a single ID, enabling support for filtering artists across multiple libraries simultaneously.

Changes include:
- Modified ArtistRepository interface in model/artist.go
- Updated implementation in persistence/artist_repository.go with improved library filtering logic
- Refactored Subsonic API browsing.go to use new selectedMusicFolderIds helper
- Added comprehensive test coverage for multiple library scenarios
- Updated mock repository implementation for testing

This change improves flexibility for multi-library operations while maintaining backward compatibility through the selectedMusicFolderIds helper function.

* feat: add library access validation to selectedMusicFolderIds

Enhanced the selectedMusicFolderIds function to validate musicFolderId parameters
against the user's accessible libraries. Invalid library IDs (those the user
doesn't have access to) are now silently filtered out, improving security by
preventing users from accessing libraries they don't have permission for.

Changes include:
- Added validation logic to check musicFolderId parameters against user's accessible libraries
- Added slices package import for efficient validation
- Enhanced function documentation to clarify validation behavior
- Added comprehensive test cases covering validation scenarios
- Maintains backward compatibility with existing behavior

* feat: implement multi-library support for GetAlbumList and GetAlbumList2 endpoints

- Enhanced selectedMusicFolderIds helper to validate and filter library IDs
- Added ApplyLibraryFilter function in filter/filters.go for library filtering
- Updated getAlbumList to support musicFolderId parameter filtering
- Added comprehensive tests for multi-library functionality
- Supports single and multiple musicFolderId values
- Falls back to all accessible libraries when no musicFolderId provided
- Validates library access permissions for user security

* feat: implement multi-library support for GetRandomSongs, GetSongsByGenre, GetStarred, and GetStarred2

- Added multi-library filtering to GetRandomSongs endpoint using musicFolderId parameter
- Added multi-library filtering to GetSongsByGenre endpoint using musicFolderId parameter
- Enhanced GetStarred and GetStarred2 to filter artists, albums, and songs by library
- Added Options field to MockMediaFileRepo and MockArtistRepo for test compatibility
- Added comprehensive Ginkgo/Gomega tests for all new multi-library functionality
- All tests verify proper SQL filter generation and library access validation
- Supports single/multiple musicFolderId values with fallback to all accessible libraries

* refactor: optimize starred items queries with parallel execution and fix test isolation

Refactored starred items functionality by extracting common logic into getStarredItems()
method that executes artist, album, and media file queries in parallel for better performance.
This eliminates code duplication between GetStarred and GetStarred2 methods while improving
response times through concurrent database queries using run.Parallel().

Also fixed test isolation issues by adding missing auth.Init(ds) call in album lists test setup.
This resolves nil pointer dereference errors in GetStarred and GetStarred2 tests when run independently.

* fix: add ApplyArtistLibraryFilter to filter artists by associated music folders

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add library access methods to User model

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement library access filtering for artist queries based on user permissions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance artist library filtering based on user permissions and optimize library ID retrieval

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: return error when any musicFolderId is invalid or inaccessible

Changed behavior from silently filtering invalid library IDs to returning
ErrorDataNotFound (code 70) when any provided musicFolderId parameter
is invalid or the user doesn't have access to it.

The error message includes the specific library number for better debugging.
This affects album/song list endpoints (getAlbumList, getRandomSongs,
getSongsByGenre, getStarred) to provide consistent error handling
across all Subsonic API endpoints.

Updated corresponding tests to expect errors instead of silent filtering.

* feat: add musicFolderId parameter support to Search2 and Search3 endpoints

Implemented musicFolderId parameter support for Subsonic API Search2 and Search3 endpoints, completing multi-library functionality across all Subsonic endpoints.

Key changes:
- Added musicFolderId parameter handling to Search2 and Search3 endpoints
- Updated search logic to filter results by specified library or all accessible libraries when parameter not provided
- Added proper error handling for invalid/inaccessible musicFolderId values
- Refactored SearchableRepository interface to support library filtering with variadic QueryOptions
- Updated repository implementations (Album, Artist, MediaFile) to handle library filtering in search operations
- Added comprehensive test coverage with robust assertions verifying library filtering works correctly
- Enhanced mock repositories to capture QueryOptions for test validation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: refresh LibraryList on scan end

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: allow editing name of main library

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement SendBroadcastMessage method for event broadcasting

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add event broadcasting for library creation, update, and deletion

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add useRefreshOnEvents hook for custom refresh logic on event changes

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance library management with refresh event broadcasting

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: replace AddUserLibrary and RemoveUserLibrary with SetUserLibraries for better library management

Signed-off-by: Deluan <deluan@navidrome.org>

* chore: remove commented-out genre repository code from persistence tests

* feat: enhance library selection with master checkbox functionality

Added a master checkbox to the SelectLibraryInput component, allowing users to select or deselect all libraries at once. This improves user experience by simplifying the selection process when multiple libraries are available. Additionally, updated translations in the en.json file to include a new message for selecting all libraries, ensuring consistency in user interface messaging.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add default library assignment for new users

Introduced a new column `default_new_users` in the library table to
facilitate automatic assignment of default libraries to new regular users.
When a new user is created, they will now be assigned to libraries marked
as default, enhancing user experience by ensuring they have immediate access
to essential resources. Additionally, updated the user repository logic
to handle this new functionality and modified the user creation validation
to reflect that library selection is optional for non-admin users.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: correct updated_at assignment in library repository

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: improve cache buffering logic

Refactored the cache buffering logic to ensure thread safety when checking
the buffer length

Signed-off-by: Deluan <deluan@navidrome.org>

* fix formating

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement per-library artist statistics with automatic aggregation

Implemented comprehensive multi-library support for artist statistics that
automatically aggregates stats from user-accessible libraries. This fundamental
change moves artist statistics from global scope to per-library granularity
while maintaining backward compatibility and transparent operation.

Key changes include:
- Migrated artist statistics from global artist.stats to per-library library_artist.stats
- Added automatic library filtering and aggregation in existing Get/GetAll methods
- Updated role-based filtering to work with per-library statistics storage
- Enhanced statistics calculation to process and store stats per library
- Implemented user permission-aware aggregation that respects library access control
- Added comprehensive test coverage for library filtering and restricted user access
- Created helper functions to ensure proper library associations in tests

This enables users to see statistics that accurately reflect only the content
from libraries they have access to, providing proper multi-tenant behavior
while maintaining the existing API surface and UI functionality.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add multi-library support with per-library tag statistics - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: genre and tag repositories. add comprehensive tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add multi-library support to tag repository system

Implemented comprehensive library filtering for tag repositories to support the multi-library feature. This change ensures that users only see tags from libraries they have access to, while admin users can see all tags.

Key changes:
- Enhanced TagRepository.Add() method to accept libraryID parameter for proper library association
- Updated baseTagRepository to implement library-aware queries with proper joins
- Added library_tag table integration for per-library tag statistics
- Implemented user permission-based filtering through user_library associations
- Added comprehensive test coverage for library filtering scenarios
- Updated UI data provider to include tag filtering by selected libraries
- Modified scanner to pass library ID when adding tags during folder processing

The implementation maintains backward compatibility while providing proper isolation between libraries for tag-based operations like genres and other metadata tags.

* refactor: simplify artist repository library filtering

Removed conditional admin logic from applyLibraryFilterToArtistQuery method
and unified the library filtering approach to match the tag repository pattern.
The method now always uses the same SQL join structure regardless of user role,
with admin access handled automatically through user_library associations.

Added artistLibraryIdFilter function to properly qualify library_id column
references and prevent SQL ambiguity errors when multiple tables contain
library_id columns. This ensures the filter targets library_artist.library_id
specifically rather than causing ambiguous column name conflicts.

* fix: resolve LibrarySelectionField validation error for non-admin users

Fixed validation error 'At least one library must be selected for non-admin users' that appeared even when libraries were selected. The issue was caused by a data format mismatch between backend and frontend.

The backend sends user data with libraries as an array of objects, but the LibrarySelectionField component expects libraryIds as an array of IDs. Added data transformation in the data provider's getOne method to automatically convert libraries array to libraryIds format when fetching user records.

Also extracted validation logic into a separate userValidation module for better code organization and added comprehensive test coverage to prevent similar issues.

* refactor: remove unused library access functions and related tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename search_test.go to searching_test.go for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: add user context to scrobble buffer getParticipants call

Added user context handling to scrobbleBufferRepository.Next method to resolve
SQL error 'no such column: library_artist.library_id' when processing scrobble
entries in multi-library environments. The artist repository now requires user
context for proper library filtering, so we fetch the user and temporarily
inject it into the context before calling getParticipants. This ensures
background scrobbling operations work correctly with multi-library support.

* feat: add cross-library move detection for scanner

Implemented cross-library move detection for the scanner phase 2 to properly handle files moved between libraries. This prevents users from losing play counts, ratings, and other metadata when moving files across library boundaries.

Changes include:
- Added MediaFileRepository methods for two-tier matching: FindRecentFilesByMBZTrackID (primary) and FindRecentFilesByProperties (fallback)
- Extended scanner phase 2 pipeline with processCrossLibraryMoves stage that processes files unmatched within their library
- Implemented findCrossLibraryMatch with MusicBrainz Release Track ID priority and intrinsic properties fallback
- Updated producer logic to handle missing tracks without matches, ensuring cross-library processing
- Updated tests to reflect new producer behavior and cross-library functionality

The implementation uses existing moveMatched function for unified move operations, automatically preserving all user data through database foreign key relationships. Cross-library moves are detected using the same Equals() and IsEquivalent() matching logic as within-library moves for consistency.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add album annotation reassignment for cross-library moves

Implemented album annotation reassignment functionality for the scanner's missing tracks phase. When tracks move between libraries and change album IDs, the system now properly reassigns album annotations (starred status, ratings) from the old album to the new album. This prevents loss of user annotations when tracks are moved across library boundaries.

The implementation includes:
- Thread-safe annotation reassignment using mutex protection
- Duplicate reassignment prevention through processed album tracking
- Graceful error handling that doesn't fail the entire move operation
- Comprehensive test coverage for various scenarios including error conditions

This enhancement ensures data integrity and user experience continuity during cross-library media file movements.

* fix: address PR review comments for multi-library support

Fixed several issues identified in PR review:

- Removed unnecessary artist stats initialization check since the map is already initialized in PostScan()
- Improved code clarity in user repository by extracting isNewUser variable to avoid checking count == 0 twice
- Fixed library selection logic to properly handle initial library state and prevent overriding user selections

These changes address code quality and logic issues identified during the multi-library support PR review.

* feat: add automatic playlist statistics refreshing

Implemented automatic playlist statistics (duration, size, song count) refreshing
when tracks are modified. Added new refreshStats() method to recalculate
statistics from playlist tracks, and SetTracks() method to update tracks
and refresh statistics atomically. Modified all track manipulation methods
(RemoveTracks, AddTracks, AddMediaFiles) to automatically refresh statistics.
Updated playlist repository to use the new SetTracks method for consistent
statistics handling.

* refactor: rename AddTracks to AddMediaFilesByID for clarity

Renamed the AddTracks method to AddMediaFilesByID throughout the codebase
to better reflect its purpose of adding media files to a playlist by their IDs.
This change improves code readability and makes the method name more descriptive
of its actual functionality. Updated all references in playlist model, tests,
core playlist logic, and Subsonic API handlers to use the new method name.

* refactor: consolidate user context access in persistence layer

Removed duplicate helper functions userId() and isAdmin() from sql_base_repository.go and consolidated all user context access to use loggedUser(r.ctx).ID and loggedUser(r.ctx).IsAdmin consistently across the persistence layer.

This change eliminates code duplication and provides a single, consistent pattern for accessing user context information in repository methods. All functionality remains unchanged - this is purely a code cleanup refactoring.

* refactor: eliminate MockLibraryService duplication using embedded struct

- Replace 235-line MockLibraryService with 40-line embedded struct pattern
- Enhance MockLibraryRepo with service-layer methods (192→310 lines)
- Maintain full compatibility with existing tests
- All 72 nativeapi specs pass with proper error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: cleanup

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-07-18 18:41:12 -04:00
Deluan Quintão
b63630fa6e
fix(scanner) artist stats not refreshing during quick scan and after missing file deletion (#4269)
* Fix artist not being marked as touched during quick scans

When a new album is added during quick scans, artists were not being
marked as 'touched' due to media files having older modification times
than the scan completion time.

Changes:
- Add 'updated_at' to artist Put() columns in scanner to ensure
  timestamp is set when artists are processed
- Simplify RefreshStats query to check artist.updated_at directly
  instead of complex media file joins
- Artists from new albums now properly get refreshed in later phases

This fixes the issue where newly added albums would have incomplete
artist information after quick scans.

* fix(missing): refresh artist stats in background after deleting missing files

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(request): add InternalAuth to user context

Signed-off-by: Deluan <deluan@navidrome.org>

* Add comprehensive test for artist stats update during quick scans

- Add test that verifies artist statistics are correctly updated when new files are added during incremental scans
- Test ensures both overall stats (AlbumCount, SongCount) and role-specific stats are properly refreshed
- Validates fix for artist stats not being refreshed during quick scans when new albums are added
- Uses real artist repository instead of mock to verify actual stats calculation behavior

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-06-26 15:50:56 -04:00
Deluan Quintão
e5e2d860ef
fix(scanner): ensure full scans update the DB (#4252)
* fix: ensure full scan refreshes all artist stats

After PR #4059, full scans were not forcing a refresh of all artists.
This change ensures that during full scans, all artist stats are refreshed
instead of only those with recently updated media files.

Changes:
- Set changesDetected=true at start of full scans to ensure maintenance operations run
- Add allArtists parameter to RefreshStats() method
- Pass fullScan state to RefreshStats to control refresh scope
- Update mock repository to match new interface

Fixes #4246
Related to PR #4059

* fix: add tests for full and incremental scans

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-06-23 13:26:48 -04:00
Deluan
65029968ab refactor: rename chain package to run and update references
Signed-off-by: Deluan <deluan@navidrome.org>
2025-06-14 17:19:06 -04:00
Deluan Quintão
5667f6ab75
feat(scanner): add library stats to DB (#4229)
* Combine library stats migrations

* test: verify full library stats

* Fix total_songs calculation

* Fix library stats migration

* fix(scanner): log elapsed time and number of libraries updated during scan

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): refresh library stats conditionally, only if changes were detected

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): refresh library stats conditionally, only if changes were detected

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): update queries to exclude missing entries in library stats

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-06-14 15:58:33 -04:00
Deluan
44834204de fix(scanner): improve folderEntry methods and hashing logic for better change detection
Signed-off-by: Deluan <deluan@navidrome.org>
2025-06-14 12:35:28 -04:00
Deluan Quintão
0d74d36cec
feat(scanner): add folder hash for smarter quick scan change detection (#4220)
* Simplify folder hash migration

* fix hashing lint

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* Update scanner/folder_entry.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-12 13:17:34 -04:00
Kendall Garner
b79e84a535
fix(scanner): update prometheus at the end of the scan (#4163)
* fix(scannner): use prometheus instance over noop if configured properly

* Real Fix: move `WriteAfterScanMetrics` outside gofunc

* refactor: remove unused artwork.CacheWarmer param from CallScan function

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-06-02 20:13:54 -04:00
Deluan
3350e6c115 fix(ui): elapsed time for scans
Signed-off-by: Deluan <deluan@navidrome.org>
2025-05-23 23:28:02 -04:00
Deluan
1e4c759d93 test: fix flaky scanner tests by setting maximum open connections to 1
Signed-off-by: Deluan <deluan@navidrome.org>
2025-05-23 15:39:44 -04:00
Copilot
992c78376c
feat(scanner): add Scanner.PurgeMissing configuration option (#4107)
* Initial plan for issue

* Add Scanner.PurgeMissing configuration option

Co-authored-by: deluan <331353+deluan@users.noreply.github.com>

* Remove GC call from phaseMissingTracks.purgeMissing method

Co-authored-by: deluan <331353+deluan@users.noreply.github.com>

* Address PR comments for Scanner.PurgeMissing feature

Co-authored-by: deluan <331353+deluan@users.noreply.github.com>

* Address PR comments and add DeleteAllMissing method

Co-authored-by: deluan <331353+deluan@users.noreply.github.com>

* refactor(scanner): simplify purgeMissing logic and improve error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* fix configuration test

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: deluan <331353+deluan@users.noreply.github.com>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-05-22 20:50:15 -04:00
Deluan Quintão
6880cffd16
feat(ui): add scan progress and error reporting to UI (#4094)
* feat(scanner): add LastScanError tracking to scanner status

- Introduced LastScanErrorKey constant for error tracking.
- Updated StatusInfo struct to include LastError field.
- Modified scanner logic to store and retrieve last scan error.
- Enhanced ScanStatus response to include error information.
- Updated UI components to display last scan error when applicable.
- Added tests to verify last scan error functionality.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): enhance scan status with type and elapsed time tracking

- Added LastScanTypeKey and LastScanStartTimeKey constants for tracking scan type and start time.
- Updated StatusInfo struct to include ScanType and ElapsedTime fields.
- Implemented getScanInfo method to retrieve scan type, elapsed time, and last error.
- Modified scanner logic to store scan type and start time during scans.
- Enhanced ScanStatus response and UI components to display scan type and elapsed time.
- Added formatShortDuration utility for better elapsed time representation.
- Updated activity reducer to handle new scan status fields.

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(tests): consolidate controller status tests into a single file

- Removed the old controller_status_test.go file.
- Merged relevant tests into the new controller_test.go file for better organization and maintainability.
- Ensured all existing test cases for controller status are preserved and functional.

Signed-off-by: Deluan <deluan@navidrome.org>

* Fix formatting issues

* refactor(scanner): update getScanInfo method documentation

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-05-21 09:30:23 -04:00
Deluan Quintão
19d443ec7f
feat(scanner): add Scanner.FollowSymlinks option (#4061)
* Add Scanner.FollowSymlinks option (default: true) - Fixes #4060

* fix(mockMusicFS): improve symlink handling in Open, Stat, and ReadDir methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(tests): enhance walkDirTree tests with symlink handling and cleanup

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-05-15 10:33:28 -04:00
Deluan
dfa217ab51 docs(scanner): add overview README document
Signed-off-by: Deluan <deluan@navidrome.org>
2025-04-25 12:54:29 -04:00
Deluan Quintão
55ce28b2c6
fix(bfr): force upgrade to read all folders. (#3871)
* chore(scanner): add trace logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(bfr): force upgrade to read all folders. It was skipping folders for certain timezones

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-24 15:22:59 -04:00
Deluan
3a0ce6aafa fix(scanner): elapsed time for folder processing is wrong in the logs
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-23 12:36:38 -04:00
Deluan
223e88d481 chore: remove some BFR-related TODOs that are not valid anymore
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-23 11:37:20 -04:00
Deluan
b386981b7f fix(scanner): better log message when AutoImportPlaylists is disabled
Fix #3861

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-22 15:08:26 -04:00
Deluan
cd552a55ef fix(scanner): pass datafolder and cachefolder to scanner subprocess
Fix #3831

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-19 22:15:20 -04:00
Deluan
422ba2284e chore(scanner): add logs to .ndignore processing
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-14 17:44:11 -04:00
Deluan
5fb1db6031 fix(scanner): watcher not working with relative MusicFolder
Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-12 18:13:22 -04:00
Deluan
226be78bf5 fix(scanner): full_text not being updated on scan
Fixes #3813

Signed-off-by: Deluan <deluan@navidrome.org>
2025-03-12 17:51:36 -04:00
Kendall Garner
36ed880e61
fix(scanner): always refresh folder image time when adding first image (#3764)
* fix(scanner): Always refresh folder image time when adding first image

Currently, the `images_updated_at` field is only set to the image modification time.
However, in cases where a new image is added _and_ said image is older than the folder mod time, the field is not updated properly.

In this the case where `images_updated_at` is null (no images were ever added) and a new images is found, use the folder modification time instead of image modification time.

**Note**, this doesn't handle cases such as replacing a newer image with an older one.

* simplify image update at

* we don't want to set imagesUpdatedAt when there's no images in the folder

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>
2025-03-06 22:16:37 -05:00