mirror of
https://github.com/navidrome/navidrome.git
synced 2026-02-02 06:24:14 +00:00
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Deluan Quintão
|
03a45753e9
|
feat(plugins): New Plugin System with multi-language PDK support (#4833)
* chore(plugins): remove the old plugins system implementation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement new plugin system with using Extism Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add capability detection for plugins based on exported functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): standardize variable names and remove superfluous wrapper functions Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): improve error handling and logging in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement plugin function call helper and refactor MetadataAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): race condition in plugin manager * tests(plugins): change BeforeEach to BeforeAll in MetadataAgent tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): optimize tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): more optimizations Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): ignore goroutine leaks from notify library in tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Wikimedia plugin for Navidrome to fetch artist metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin logging and set User-Agent header Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement scrobbler plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate logs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): clean up manifest struct and improve plugin loading logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add metadata agent and scrobbler schemas for bootstrapping plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(hostgen): add hostgen tool for generating Extism host function wrappers - Implemented hostgen tool to generate wrappers from annotated Go interfaces. - Added command-line flags for input/output directories and package name. - Introduced parsing and code generation logic for host services. - Created test data for various service interfaces and expected generated code. - Added documentation for host services and annotations for code generation. - Implemented SubsonicAPI service with corresponding generated code. * feat(subsonicapi): update Call method to return JSON string response Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement SubsonicAPI host function integration with permissions Signed-off-by: Deluan <deluan@navidrome.org> * fix(generator): error-only methods in response handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate client wrappers for host functions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(generator): remove error handling for response.Error in client templates Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add Scheduler service interface with host function wrappers for scheduling tasks * feat(plugins): add WASI build constraints to client wrapper templates, to avoid lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler service with one-time and recurring scheduling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor(manifest): remove unused ConfigPermission from permissions schema Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add scheduler callback schema and implementation for plugins Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): streamline scheduling logic and remove unused callback tracking Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): add Close method for resource cleanup on plugin unload Signed-off-by: Deluan <deluan@navidrome.org> * docs(scheduler): clarify SchedulerCallback requirement for scheduling functions Signed-off-by: Deluan <deluan@navidrome.org> * fix: update wasm build rule to include all Go files in the directory Signed-off-by: Deluan <deluan@navidrome.org> * feat: rewrite the wikimedia plugin using the XTP CLI Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): replace uuid with id.NewRandom for schedule ID generation Signed-off-by: Deluan <deluan@navidrome.org> * refactor: capabilities registration Signed-off-by: Deluan <deluan@navidrome.org> * test: add scheduler service isolation test for plugin instances Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin manager initialization and encapsulate logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add WebSocket service definitions for plugin communication Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WebSocket service for plugin integration and connection management Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Crypto Ticker example plugin for real-time cryptocurrency price updates via Coinbase WebSocket API Also add the lifecycle capability Signed-off-by: Deluan <deluan@navidrome.org> * fix: use context.Background() in invokeCallback for scheduled tasks Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin.create() to plugin.instance() Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginInstance to plugin for consistency across the codebase Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify schedule cloning in Close method and enhance plugin cleanup error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Artwork service for generating artwork URLs in Navidrome plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor: moved public URL builders to avoid import cycles Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cache service for in-memory TTL-based caching in plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Discord Rich Presence example plugin for Navidrome integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor: host function wrappers to use structured request and response types - Updated the host function signatures in `nd_host_artwork.go`, `nd_host_scheduler.go`, `nd_host_subsonicapi.go`, and `nd_host_websocket.go` to accept a single parameter for JSON requests. - Introduced structured request and response types for various cache operations in `nd_host_cache.go`. - Modified cache functions to marshal requests to JSON and unmarshal responses, improving error handling and code clarity. - Removed redundant memory allocation for string parameters in favor of JSON marshaling. - Enhanced error handling in WebSocket and cache operations to return structured error responses. * refactor: error handling in various plugins to convert response.Error to Go errors - Updated error handling in `nd_host_scheduler.go`, `nd_host_websocket.go`, `nd_host_artwork.go`, `nd_host_cache.go`, and `nd_host_subsonicapi.go` to convert string errors from responses into Go errors. - Removed redundant error checks in test data plugins for cleaner code. - Ensured consistent error handling across all plugins to improve reliability and maintainability. * refactor: rename fake plugins to test plugins for clarity in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add help target to Makefile for plugin usage instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cover Art Archive plugin as an example of Python plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Makefile and README to clarify Go plugin usage Signed-off-by: Deluan <deluan@navidrome.org> * feat: include plugin capabilities in loading log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add trace logging for plugin availability and error handling in agents Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Now Playing Logger plugin to showcase calling host functions from Python plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: generate Python client wrappers for various host services Signed-off-by: Deluan <deluan@navidrome.org> * feat: add generated host function wrappers for Scheduler and SubsonicAPI services Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Python plugin documentation and usage instructions for host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Webhook Scrobbler plugin in Rust to send HTTP notifications on scrobble events Signed-off-by: Deluan <deluan@navidrome.org> * feat: enable parallel loading of plugins during startup Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include WebSocket callback schema in plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin watcher with improved logging and debounce duration adjustment Signed-off-by: Deluan <deluan@navidrome.org> * add trace message for plugin recompiles Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin cache purging functionality Signed-off-by: Deluan <deluan@navidrome.org> * test: move purgeCacheBySize unit tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin repository and database support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin management routes and middleware Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): implement plugin synchronization with database for add, update, and remove actions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add PluginList and PluginShow components with plugin management functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): optimize plugin change detection Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins UI): improve PluginList structure Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): enhance PluginShow with author, website, and permissions display Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): refactor to use MUI and RA components Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add error handling for plugin enable/disable actions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): inject PluginManager into native API Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update GetManager to accept DataStore parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add subsonicRouter to Manager and refactor host service registration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): enhance debug logging for plugin actions and recompile logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): break manager.go into smaller, focused files Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline error handling and improve plugin retrieval logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update newWebSocketService to use WebSocketPermission for allowed hosts Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): introduce ToggleEnabledSwitch for managing plugin enable/disable state Signed-off-by: Deluan <deluan@navidrome.org> * docs: update READMEs Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add Library service for metadata access and filesystem integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Library Inspector plugin for periodic library inspection and file size logging Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect JSON configuration format for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix(build): update target to wasm32-wasip1 for improved WASI support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement configuration management UI with key-value pairs support Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): adjust grid layout in InfoRow component for improved responsiveness Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename ErrorIndicator to EnabledOrErrorField and enhance error handling logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(i18n): add Portuguese translations for plugin management and notifications Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for .ndp plugin packages and update build process Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README for .ndp plugin packaging and installation instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement KVStore service for persistent key-value storage Signed-off-by: Deluan <deluan@navidrome.org> * docs: enhance README with Extism plugin development resources and recommendations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate event broker into plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update config handling in PluginShow to track last record state Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Rust host function library and example implementation of Discord Rich Presence plugin in Rust Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Rust lib.rs file to expose host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update JSON field names to camelCase for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reduce cyclomatic complexity by refactoring main function Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance Rust code generation with typed struct support and improved type handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Go client library with host function wrappers and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Go client stubs for non-WASM platforms Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update client template file names for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add initial implementation of the Navidrome Plugin Development Kit code generator - Pahse 1 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 (2) Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 4 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 5 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consistent naming/types across PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline plugin function signatures and error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update scrobbler interface to return errors directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * test: make all test plugins use the PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize and sort type definitions for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update error handling for methods to return errors directly Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update function signatures to return values directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update request/response types to use private naming conventions Signed-off-by: Deluan <deluan@navidrome.org> * build: mark .wasm files as intermediate for cleanup after building .ndp Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consolidate PDK module path and update Go version to 1.25 Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Rust PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize Rust output structure to follow standard conventions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update Discord Rich Presence and Library Inspector plugins to use nd-pdk for service calls and implement lifecycle management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update macro names for websocket and metadata registration to improve clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename scheduler callback methods for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update export wrappers to use `//go:wasmexport` for WebAssembly compatibility Signed-off-by: Deluan <deluan@navidrome.org> * docs: update plugin registration docs Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): generate host wrappers Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): conditionally run goleak checks based on CI environment Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect changes in plugin import paths Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update plugin instance creation to accept context for cancellation support Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update return types in metadata interfaces to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): enhance type handling for Rust and XTP output in capability generation Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update IsAuthorized method to return boolean instead of response object Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add unit tests for rustOutputType and isPrimitiveRustType functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement XTP JSONSchema validation for generated schemas Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update response types in testMetadataAgent methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * docs: update Go and Rust plugin developer sections for clarity Signed-off-by: Deluan <deluan@navidrome.org> * docs: correct example link for library inspector in README Signed-off-by: Deluan <deluan@navidrome.org> * docs: clarify artwork URL generation capabilities in service descriptions Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include Rust PDK crate information for plugin developers Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle URL parsing errors and use atomic upsert in plugin repository Added proper error handling for url.Parse calls in PublicURL and AbsoluteURL functions. When parsing fails, PublicURL now falls back to AbsoluteURL, and AbsoluteURL logs the error and returns an empty string, preventing malformed URLs from being generated. Replaced the non-atomic UPDATE-then-INSERT pattern in plugin repository Put method with a single atomic INSERT ... ON CONFLICT statement. This eliminates potential race conditions and improves consistency with the upsert pattern already used in host_kvstore.go. * feat: implement mock service instances for non-WASM builds using testify/mock Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Discord RPC struct to encapsulate WebSocket logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for experimental WebAssembly threads Signed-off-by: Deluan <deluan@navidrome.org> * feat: add PDK abstraction layer with mock support for non-WASM builds Signed-off-by: Deluan <deluan@navidrome.org> * feat: add unit tests for Discord plugin and RPC functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: update return types in minimalPlugin and wikimediaPlugin methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix: context cancellation and implement WebSocket callback timeout for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: conditionally include error handling in generated client code templates Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ConfigService for plugin configuration management Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance plugin manager to support metrics recording Signed-off-by: Deluan <deluan@navidrome.org> * refactor: make MockPDK private Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update interface types to use 'any' in plugin repository methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename List method to Keys for clarity in configuration management Signed-off-by: Deluan <deluan@navidrome.org> * test: add ndpgen plugin tests in the pipeline and update Makefile Signed-off-by: Deluan <deluan@navidrome.org> * feat: add users permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * refactor: streamline users integration tests and enhance plugin user management Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove UserID from scrobbler request structure Signed-off-by: Deluan <deluan@navidrome.org> * test: add integration tests for UsersService enable gate behavior Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user permissions for SubsonicAPI and scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: show proper error in the UI when enabling a plugin fails Signed-off-by: Deluan <deluan@navidrome.org> * feat: add library permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * feat: add user permission for processing scrobbles in Discord Rich Presence plugin Signed-off-by: Deluan <deluan@navidrome.org> * fix: implement dynamic loading for buffered scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add GetAdmins method to retrieve admin users from the plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Portuguese translations for user and library permissions Signed-off-by: Deluan <deluan@navidrome.org> * reorder migrations Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove unnecessary bulkActionButtons prop from PluginList component * feat: add manual plugin rescan functionality and corresponding UI action Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user/library and plugin management integration with cleanup on deletion Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace core mock services with test-specific implementations to avoid import cycles * feat: add ID fields to Artist and Song structs and enhance track loading logic by prioritizing ID matches Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin permissions from allowedHosts to requiredHosts for better clarity and consistency * feat: refactor plugin host permissions to use RequiredHosts directly for improved clarity * fix: don't record metrics for plugin calls that aren't implemented at all Signed-off-by: Deluan <deluan@navidrome.org> * fix: enhance connection management with improved error handling and cleanup logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: introduce ArtistRef struct for better artist representation and update track metadata handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: update user configuration handling to use user key prefix for improved clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance ConfigCard input fields with multiline support and vertical resizing Signed-off-by: Deluan <deluan@navidrome.org> * fix: rust plugin compilation error Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement IsOptionPattern method for better return type handling in Rust PDK generation Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
5ce6e16d96
|
fix: album statistics not updating after deleting missing files (#4668)
* feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
00c83af170
|
feat: Multi-library support (#4181)
* feat(database): add user_library table and library access methods Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # tests/mock_library_repo.go * feat(database): enhance user retrieval with library associations Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): implement library management and user-library association endpoints Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): restrict access to library and config endpoints to admin users Signed-off-by: Deluan <deluan@navidrome.org> * refactor(library): implement library management service and update API routes Signed-off-by: Deluan <deluan@navidrome.org> * feat(database): add library filtering to album, folder, and media file queries Signed-off-by: Deluan <deluan@navidrome.org> * refactor library service to use REST repository pattern and remove CRUD operations Signed-off-by: Deluan <deluan@navidrome.org> * add total_duration column to library and update user_library table Signed-off-by: Deluan <deluan@navidrome.org> * fix migration file name Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add library management features including create, edit, delete, and list functionalities - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance library validation and management with path checks and normalization - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): improve library path validation and error handling - WIP Signed-off-by: Deluan <deluan@navidrome.org> * use utils/formatBytes Signed-off-by: Deluan <deluan@navidrome.org> * simplify DeleteLibraryButton.jsx Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance validation messages and error handling for library paths Signed-off-by: Deluan <deluan@navidrome.org> * lint Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): add tests for multi-library scanning and validation Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): improve handling of filesystem errors and ensure warnings are returned Signed-off-by: Deluan <deluan@navidrome.org> * feat(controller): add function to retrieve the most recent scan time across all libraries Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add additional fields and restructure LibraryEdit component for enhanced statistics display Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance LibraryCreate and LibraryEdit components with additional props and styling Signed-off-by: Deluan <deluan@navidrome.org> * feat(mediafile): add LibraryName field and update queries to include library name Signed-off-by: Deluan <deluan@navidrome.org> * feat(missingfiles): add library filter and display in MissingFilesList component Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): implement scanner interface for triggering library scans on create/update Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # cmd/wire_gen.go # cmd/wire_injectors.go # Conflicts: # cmd/wire_gen.go # Conflicts: # cmd/wire_gen.go # cmd/wire_injectors.go * feat(library): trigger scan after successful library deletion to clean up orphaned data Signed-off-by: Deluan <deluan@navidrome.org> * rename migration file for user library table to maintain versioning order Signed-off-by: Deluan <deluan@navidrome.org> * refactor: move scan triggering logic into a helper method for clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add library path and name fields to album and mediafile models Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add/remove watchers on demand, not only when server starts Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline library handling by using state-libraries for consistency Signed-off-by: Deluan <deluan@navidrome.org> * fix: track processed libraries by updating state with scan timestamps Signed-off-by: Deluan <deluan@navidrome.org> * prepend libraryID for track and album PIDs Signed-off-by: Deluan <deluan@navidrome.org> * feat(repository): apply library filtering in CountAll methods for albums, folders, and media files Signed-off-by: Deluan <deluan@navidrome.org> * feat(user): add library selection for user creation and editing Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): implement library selection functionality with reducer and UI component Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # .github/copilot-instructions.md # Conflicts: # .gitignore * feat(library): add tests for LibrarySelector and library selection hooks Signed-off-by: Deluan <deluan@navidrome.org> * test: add unit tests for file utility functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add library ID filtering for album resources Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): streamline library ID filtering in repositories and update resource filtering logic Signed-off-by: Deluan <deluan@navidrome.org> * fix(repository): add table name handling in filter functions for SQL queries Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add refresh functionality on LibrarySelector close Signed-off-by: Deluan <deluan@navidrome.org> * feat(artist): add library ID filtering for artists in repository and update resource filtering logic Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # persistence/artist_repository.go * Add library_id field support for smart playlists - Add library_id field to smart playlist criteria system - Supports Is and IsNot operators for filtering by library ID - Includes comprehensive test coverage for single values and lists - Enables creation of library-specific smart playlists * feat(subsonic): implement user-specific library access in GetMusicFolders Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance LibrarySelectionField to extract library IDs from record Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): update GetIndexes and GetArtists method to support library ID filtering Signed-off-by: Deluan <deluan@navidrome.org> * fix: ensure LibrarySelector dropdown refreshes on button close Added refresh() call when closing the dropdown via button click to maintain consistency with the ClickAwayListener behavior. This ensures the UI updates properly regardless of how the dropdown is closed, fixing an inconsistent refresh behavior between different closing methods. The fix tracks the previous open state and calls refresh() only when the dropdown was open and is being closed by the button click. * refactor: simplify getUserAccessibleLibraries function and update related tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance selectedMusicFolderIds function to handle valid music folder IDs and improve fallback logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor: change ArtistRepository.GetIndex to accept multiple library IDs Updated the GetIndex method signature to accept a slice of library IDs instead of a single ID, enabling support for filtering artists across multiple libraries simultaneously. Changes include: - Modified ArtistRepository interface in model/artist.go - Updated implementation in persistence/artist_repository.go with improved library filtering logic - Refactored Subsonic API browsing.go to use new selectedMusicFolderIds helper - Added comprehensive test coverage for multiple library scenarios - Updated mock repository implementation for testing This change improves flexibility for multi-library operations while maintaining backward compatibility through the selectedMusicFolderIds helper function. * feat: add library access validation to selectedMusicFolderIds Enhanced the selectedMusicFolderIds function to validate musicFolderId parameters against the user's accessible libraries. Invalid library IDs (those the user doesn't have access to) are now silently filtered out, improving security by preventing users from accessing libraries they don't have permission for. Changes include: - Added validation logic to check musicFolderId parameters against user's accessible libraries - Added slices package import for efficient validation - Enhanced function documentation to clarify validation behavior - Added comprehensive test cases covering validation scenarios - Maintains backward compatibility with existing behavior * feat: implement multi-library support for GetAlbumList and GetAlbumList2 endpoints - Enhanced selectedMusicFolderIds helper to validate and filter library IDs - Added ApplyLibraryFilter function in filter/filters.go for library filtering - Updated getAlbumList to support musicFolderId parameter filtering - Added comprehensive tests for multi-library functionality - Supports single and multiple musicFolderId values - Falls back to all accessible libraries when no musicFolderId provided - Validates library access permissions for user security * feat: implement multi-library support for GetRandomSongs, GetSongsByGenre, GetStarred, and GetStarred2 - Added multi-library filtering to GetRandomSongs endpoint using musicFolderId parameter - Added multi-library filtering to GetSongsByGenre endpoint using musicFolderId parameter - Enhanced GetStarred and GetStarred2 to filter artists, albums, and songs by library - Added Options field to MockMediaFileRepo and MockArtistRepo for test compatibility - Added comprehensive Ginkgo/Gomega tests for all new multi-library functionality - All tests verify proper SQL filter generation and library access validation - Supports single/multiple musicFolderId values with fallback to all accessible libraries * refactor: optimize starred items queries with parallel execution and fix test isolation Refactored starred items functionality by extracting common logic into getStarredItems() method that executes artist, album, and media file queries in parallel for better performance. This eliminates code duplication between GetStarred and GetStarred2 methods while improving response times through concurrent database queries using run.Parallel(). Also fixed test isolation issues by adding missing auth.Init(ds) call in album lists test setup. This resolves nil pointer dereference errors in GetStarred and GetStarred2 tests when run independently. * fix: add ApplyArtistLibraryFilter to filter artists by associated music folders Signed-off-by: Deluan <deluan@navidrome.org> * feat: add library access methods to User model Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement library access filtering for artist queries based on user permissions Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance artist library filtering based on user permissions and optimize library ID retrieval Signed-off-by: Deluan <deluan@navidrome.org> * fix: return error when any musicFolderId is invalid or inaccessible Changed behavior from silently filtering invalid library IDs to returning ErrorDataNotFound (code 70) when any provided musicFolderId parameter is invalid or the user doesn't have access to it. The error message includes the specific library number for better debugging. This affects album/song list endpoints (getAlbumList, getRandomSongs, getSongsByGenre, getStarred) to provide consistent error handling across all Subsonic API endpoints. Updated corresponding tests to expect errors instead of silent filtering. * feat: add musicFolderId parameter support to Search2 and Search3 endpoints Implemented musicFolderId parameter support for Subsonic API Search2 and Search3 endpoints, completing multi-library functionality across all Subsonic endpoints. Key changes: - Added musicFolderId parameter handling to Search2 and Search3 endpoints - Updated search logic to filter results by specified library or all accessible libraries when parameter not provided - Added proper error handling for invalid/inaccessible musicFolderId values - Refactored SearchableRepository interface to support library filtering with variadic QueryOptions - Updated repository implementations (Album, Artist, MediaFile) to handle library filtering in search operations - Added comprehensive test coverage with robust assertions verifying library filtering works correctly - Enhanced mock repositories to capture QueryOptions for test validation Signed-off-by: Deluan <deluan@navidrome.org> * feat: refresh LibraryList on scan end Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow editing name of main library Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement SendBroadcastMessage method for event broadcasting Signed-off-by: Deluan <deluan@navidrome.org> * feat: add event broadcasting for library creation, update, and deletion Signed-off-by: Deluan <deluan@navidrome.org> * feat: add useRefreshOnEvents hook for custom refresh logic on event changes Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance library management with refresh event broadcasting Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace AddUserLibrary and RemoveUserLibrary with SetUserLibraries for better library management Signed-off-by: Deluan <deluan@navidrome.org> * chore: remove commented-out genre repository code from persistence tests * feat: enhance library selection with master checkbox functionality Added a master checkbox to the SelectLibraryInput component, allowing users to select or deselect all libraries at once. This improves user experience by simplifying the selection process when multiple libraries are available. Additionally, updated translations in the en.json file to include a new message for selecting all libraries, ensuring consistency in user interface messaging. Signed-off-by: Deluan <deluan@navidrome.org> * feat: add default library assignment for new users Introduced a new column `default_new_users` in the library table to facilitate automatic assignment of default libraries to new regular users. When a new user is created, they will now be assigned to libraries marked as default, enhancing user experience by ensuring they have immediate access to essential resources. Additionally, updated the user repository logic to handle this new functionality and modified the user creation validation to reflect that library selection is optional for non-admin users. Signed-off-by: Deluan <deluan@navidrome.org> * fix: correct updated_at assignment in library repository Signed-off-by: Deluan <deluan@navidrome.org> * fix: improve cache buffering logic Refactored the cache buffering logic to ensure thread safety when checking the buffer length Signed-off-by: Deluan <deluan@navidrome.org> * fix formating Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement per-library artist statistics with automatic aggregation Implemented comprehensive multi-library support for artist statistics that automatically aggregates stats from user-accessible libraries. This fundamental change moves artist statistics from global scope to per-library granularity while maintaining backward compatibility and transparent operation. Key changes include: - Migrated artist statistics from global artist.stats to per-library library_artist.stats - Added automatic library filtering and aggregation in existing Get/GetAll methods - Updated role-based filtering to work with per-library statistics storage - Enhanced statistics calculation to process and store stats per library - Implemented user permission-aware aggregation that respects library access control - Added comprehensive test coverage for library filtering and restricted user access - Created helper functions to ensure proper library associations in tests This enables users to see statistics that accurately reflect only the content from libraries they have access to, providing proper multi-tenant behavior while maintaining the existing API surface and UI functionality. Signed-off-by: Deluan <deluan@navidrome.org> * feat: add multi-library support with per-library tag statistics - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor: genre and tag repositories. add comprehensive tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add multi-library support to tag repository system Implemented comprehensive library filtering for tag repositories to support the multi-library feature. This change ensures that users only see tags from libraries they have access to, while admin users can see all tags. Key changes: - Enhanced TagRepository.Add() method to accept libraryID parameter for proper library association - Updated baseTagRepository to implement library-aware queries with proper joins - Added library_tag table integration for per-library tag statistics - Implemented user permission-based filtering through user_library associations - Added comprehensive test coverage for library filtering scenarios - Updated UI data provider to include tag filtering by selected libraries - Modified scanner to pass library ID when adding tags during folder processing The implementation maintains backward compatibility while providing proper isolation between libraries for tag-based operations like genres and other metadata tags. * refactor: simplify artist repository library filtering Removed conditional admin logic from applyLibraryFilterToArtistQuery method and unified the library filtering approach to match the tag repository pattern. The method now always uses the same SQL join structure regardless of user role, with admin access handled automatically through user_library associations. Added artistLibraryIdFilter function to properly qualify library_id column references and prevent SQL ambiguity errors when multiple tables contain library_id columns. This ensures the filter targets library_artist.library_id specifically rather than causing ambiguous column name conflicts. * fix: resolve LibrarySelectionField validation error for non-admin users Fixed validation error 'At least one library must be selected for non-admin users' that appeared even when libraries were selected. The issue was caused by a data format mismatch between backend and frontend. The backend sends user data with libraries as an array of objects, but the LibrarySelectionField component expects libraryIds as an array of IDs. Added data transformation in the data provider's getOne method to automatically convert libraries array to libraryIds format when fetching user records. Also extracted validation logic into a separate userValidation module for better code organization and added comprehensive test coverage to prevent similar issues. * refactor: remove unused library access functions and related tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename search_test.go to searching_test.go for consistency Signed-off-by: Deluan <deluan@navidrome.org> * fix: add user context to scrobble buffer getParticipants call Added user context handling to scrobbleBufferRepository.Next method to resolve SQL error 'no such column: library_artist.library_id' when processing scrobble entries in multi-library environments. The artist repository now requires user context for proper library filtering, so we fetch the user and temporarily inject it into the context before calling getParticipants. This ensures background scrobbling operations work correctly with multi-library support. * feat: add cross-library move detection for scanner Implemented cross-library move detection for the scanner phase 2 to properly handle files moved between libraries. This prevents users from losing play counts, ratings, and other metadata when moving files across library boundaries. Changes include: - Added MediaFileRepository methods for two-tier matching: FindRecentFilesByMBZTrackID (primary) and FindRecentFilesByProperties (fallback) - Extended scanner phase 2 pipeline with processCrossLibraryMoves stage that processes files unmatched within their library - Implemented findCrossLibraryMatch with MusicBrainz Release Track ID priority and intrinsic properties fallback - Updated producer logic to handle missing tracks without matches, ensuring cross-library processing - Updated tests to reflect new producer behavior and cross-library functionality The implementation uses existing moveMatched function for unified move operations, automatically preserving all user data through database foreign key relationships. Cross-library moves are detected using the same Equals() and IsEquivalent() matching logic as within-library moves for consistency. Signed-off-by: Deluan <deluan@navidrome.org> * feat: add album annotation reassignment for cross-library moves Implemented album annotation reassignment functionality for the scanner's missing tracks phase. When tracks move between libraries and change album IDs, the system now properly reassigns album annotations (starred status, ratings) from the old album to the new album. This prevents loss of user annotations when tracks are moved across library boundaries. The implementation includes: - Thread-safe annotation reassignment using mutex protection - Duplicate reassignment prevention through processed album tracking - Graceful error handling that doesn't fail the entire move operation - Comprehensive test coverage for various scenarios including error conditions This enhancement ensures data integrity and user experience continuity during cross-library media file movements. * fix: address PR review comments for multi-library support Fixed several issues identified in PR review: - Removed unnecessary artist stats initialization check since the map is already initialized in PostScan() - Improved code clarity in user repository by extracting isNewUser variable to avoid checking count == 0 twice - Fixed library selection logic to properly handle initial library state and prevent overriding user selections These changes address code quality and logic issues identified during the multi-library support PR review. * feat: add automatic playlist statistics refreshing Implemented automatic playlist statistics (duration, size, song count) refreshing when tracks are modified. Added new refreshStats() method to recalculate statistics from playlist tracks, and SetTracks() method to update tracks and refresh statistics atomically. Modified all track manipulation methods (RemoveTracks, AddTracks, AddMediaFiles) to automatically refresh statistics. Updated playlist repository to use the new SetTracks method for consistent statistics handling. * refactor: rename AddTracks to AddMediaFilesByID for clarity Renamed the AddTracks method to AddMediaFilesByID throughout the codebase to better reflect its purpose of adding media files to a playlist by their IDs. This change improves code readability and makes the method name more descriptive of its actual functionality. Updated all references in playlist model, tests, core playlist logic, and Subsonic API handlers to use the new method name. * refactor: consolidate user context access in persistence layer Removed duplicate helper functions userId() and isAdmin() from sql_base_repository.go and consolidated all user context access to use loggedUser(r.ctx).ID and loggedUser(r.ctx).IsAdmin consistently across the persistence layer. This change eliminates code duplication and provides a single, consistent pattern for accessing user context information in repository methods. All functionality remains unchanged - this is purely a code cleanup refactoring. * refactor: eliminate MockLibraryService duplication using embedded struct - Replace 235-line MockLibraryService with 40-line embedded struct pattern - Enhance MockLibraryRepo with service-layer methods (192→310 lines) - Maintain full compatibility with existing tests - All 72 nativeapi specs pass with proper error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor: cleanup Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |