mirror of
https://github.com/navidrome/navidrome.git
synced 2026-08-31 07:30:32 +00:00
* fix(jellyfin): honor Filters=IsFavorite on /Artists and /Artists/AlbumArtists listArtistsByRole hand-built its itemsQuery and never set favOnly, so the favorites filter was silently dropped on both artist routes while /Items honored it. Finamp's home screen asks for favorite artists once per load and was served the entire artist list instead: 10,298 artists, 6.15 MB, 2.7s on a real library, and the wrong data on screen. Extract the favOnly parsing that parseItemsQuery already did into parseFavOnly and use it in both places. listArtists now adds the starred predicate to notMissing rather than replacing it, matching listAlbums and listSongs, so a favorite artist whose files are gone stays excluded. * fix(jellyfin): map SortBy=Runtime to duration for albums and songs sortColumnsByType had no runtime/runtimeticks key for any type, so Finamp's "Duration" sort silently misbehaved in two different ways. Albums: Finamp sends a bare SortBy=Runtime. Nothing matched, opts.Sort stayed empty, and applyOptions skips OrderBy entirely when Sort is empty — so the query ran with no ORDER BY at all and Ascending and Descending returned identical lists. Songs: Finamp sends SortBy=Runtime,AlbumArtist,Album,SortName. applySort takes the first *recognized* key, so Runtime was skipped and the list came back sorted by album artist while looking correct. Both repos already accept a duration sort (mediafile_repository maps it explicitly; album_repository falls through to the column name), so no migration is needed. Sorting 97k songs by duration costs a temp B-tree (~114ms on a prod-sized copy) — the same cost the Subsonic and UI duration sorts already pay, and correct where the previous behaviour was merely fast. * fix(jellyfin): apply the played/unplayed filters and MaxHeight image bound Filters was matched with a substring test for IsFavorite, so every other token Jellyfin defines was silently dropped and the response kept rows it should have excluded. Finamp sends Filters=IsUnplayed in normal use. Replace the bool with a parsed itemFilters carrying nullable favorite and played flags, so isFavorite=false and isPlayed=false are real filters rather than indistinguishable from an absent param. Standalone params are read first and the Filters list overrides them, the precedence real Jellyfin has. IsFavoriteOrLikes now maps to favorites deliberately instead of by substring accident; Likes, Dislikes, IsFolder, IsNotFolder and IsResumable have no Navidrome equivalent and are dropped rather than half-applied. The negative cases match NULL as well, since annotations are LEFT JOINed and an untouched item has no row. getItemImage read only maxwidth, so a client sending just MaxHeight got the full-size original: measured against a real cover, maxHeight=100 returned 82,570 bytes where maxWidth=100 returned 3,316. Use the tighter of the two bounds. * refactor(jellyfin): share the plain-param parser between /Items and /Artists listArtistsByRole hand-listed the itemsQuery fields it happened to need, which is exactly how the favorites filter went missing: the literal has been amended in four of the five commits that touched it. Extract listParams for the fields that come straight from query params so both paths read one parser, and the next supported param reaches every list path instead of only /Items. Also from the cleanup pass: collapse imageSize to a single clamped comparison and read its bounds through req.Params like the rest of the package, which drops the strconv import; build the artist and playlist filter lists with the flat append shape the album and song paths already use, instead of re-wrapping opts.Filters into a nested And per predicate; drop a nil guard in listPlaylists that no caller can reach, since both paths into queryItemsOfType build QueryOptions without Filters. applySort now logs when no SortBy key resolves at all — a miss inside a fallback list is normal, but none matching means a silently ignored sort, the failure mode that hid the Runtime bug. Its doc comment records why the remaining keys cannot simply be joined. Folds three duplicated test bodies into the tables that already parameterize them, and covers the artist-parent album branch, which reaches notMissing through filter.AlbumsByArtistID rather than the default branch. * docs(jellyfin): correct how applySort describes Jellyfin's SortBy semantics The comment claimed SortBy is a comma-separated fallback list. It is not: RequestHelpers.GetOrderBy (10.10) builds one (ItemSortBy, SortOrder) pair per key, so Jellyfin orders by every key in turn. Navidrome applies only the first recognized one, which is a real divergence — secondary keys never break ties — not the intended reading of the parameter. The assertion that the keys cannot be joined was also wrong. buildSortOrder does split its input on commas; what it maps is the whole string, so joining raw Jellyfin key names misses the mappings. Mapping each key first and joining the results would work, which makes multi-key sorting a real option rather than a blocked one. Documenting the current behaviour as a known divergence until then. * fix(jellyfin): order by every recognized SortBy key, not just the first Jellyfin orders by each SortBy key in turn, so "DatePlayed,SortName" means break ties by name. Navidrome applied only the first recognized key and dropped the rest, which is 28% of the sort traffic on a real server (23 of 82 requests in 12h carry 2-5 keys). Most were harmless because the primary key dominates, but PremiereDate,Album,ParentIndexNumber,IndexNumber,SortName came back unordered within a year. The keys cannot simply be joined: sortMapping keyed on the whole Sort string, so a joined value missed every mapping and fell through to raw column names. Make it resolve a comma list per part, but only when every part is a known key — the four existing callers that pass raw column lists (core/matcher, core/lyrics, core/maintenance, subsonic/browsing) all carry a part that is not a mapping key, several with their own direction, so they keep falling through exactly as before. Verified each one. applySort now collects every recognized key, skipping duplicates so ParentIndexNumber,IndexNumber does not repeat a column. random stays alone: the repo matches it by exact string equality, so joining it would both break that path and emit a bare 'random' column into the ORDER BY. Verified against a prod-sized copy: every multi-key combination seen in real traffic returns 200, and a secondary key now changes the order within a tied year for songs. Albums are unchanged there, because their max_year mapping already ended in ", name". * fix(persistence): resolve sort mappings exactly once Making sortMapping resolve a comma list per part broke an invariant it had been relying on: idempotence. sanitizeSort mapped the sort key up front and applyOptions then ran buildSortOrder over the result, so sortMapping was already being handed its own output. That was harmless only while a mapped value could never look like a key list. media_file's rated_at maps to "rating, rated_at", and both parts are keys, so the second pass expanded it to "rating, rating, rated_at". Found by round-tripping every mapping in all four repositories; it was the only collision, and the duplicate sort key was benign in SQL, but any future mapping of that shape would silently change meaning. sanitizeSort now validates without resolving, leaving buildSortOrder as the single mapping point. The generated SQL is unchanged — the whole suite passes apart from the two specs that asserted the old return value, which are updated and joined by a round-trip guard covering exactly the rated_at shape. Also use the paren-aware splitFunc that buildSortOrder already uses, so an expression carrying commas inside its parentheses cannot be split apart. * refactor(jellyfin,persistence): flatten the sort resolution paths Cleanup pass over the branch, no behavior change. sortMapping loses the len(parts)>1 guard, which existed only to pick between two identical toSnakeCase exits; the single-key case now falls through the same loop. lookupSortMapping hands back the snake_case form it had to derive so the fallback stops recomputing it — toSnakeCase is two regexps, and on a miss it was running twice per call. sanitizeSort now asks lookupSortMapping instead of probing the map itself, so "is this a known sort key" has one answer; the two had already drifted, since sanitizeSort tried one casing where the resolver tries three. applySort folds the nested random branch into the skip condition and the two trailing length tests into one switch. setSortMappings documents the invariant the comma-list rule depends on, where someone adding a mapping will read it. The README line describing SortBy still said only the first key applied, which the commit before last made false. Tests: the twelve near-identical sorting specs become one DescribeTable of (itemType, SortBy, want) triples, 124 lines to 36, and the applyOptions round-trip assertion collapses to the buildSortOrder call its sibling uses. * fix(jellyfin): keep annotation filters out of search, resolve sorts per part Two findings from the Codex review on #5981. The played/unplayed filters turned working requests into 500s when combined with SearchTerm. Search runs a two-phase FTS query whose first phase selects rowids with no annotation join, so a starred or play_count predicate there is "no such column", not a filter. Measured against master: MusicAlbum with SearchTerm and Filters=IsUnplayed went 200 -> 500, likewise IsPlayed and the Audio equivalents. listAlbums and listSongs now skip those predicates on the search path, matching what listArtists already did. That also clears the same 500 master already had for Filters=IsFavorite with SearchTerm. sortMapping resolved a comma list only while every part was a known key, so a list mixing a plain column with a mapped key kept neither: MusicAlbum SortBy=Runtime,SortName arrives as "duration, name", and duration is a plain album column, so name stayed raw instead of expanding to order_album_name. Albums whose name differs from its sort form — 1,366 of 6,987 on a real library — then ordered by the wrong secondary key, and PreferSortTags was ignored. Each part is now resolved on its own, which is what setSortMappings already documents for a single field. Verified every in-tree caller that passes a raw column list still produces its original ORDER BY. Codex also asked for the artist search path to apply the same filters. It would 500 for the reason above, and wrapping the library scope in a compound filter makes requestedLibraryIDs stop recognizing it, silently widening the search past the requested ParentId. * fix(jellyfin): honor the first SortOrder value for a multi-key sort applySort compared the whole SortOrder string with "Descending", so a per-key list like SortOrder=Descending,Ascending failed the match and every key, including the primary, sorted ascending — the exact opposite of the request. Take the first comma-separated value, which Jellyfin also uses for any key past the end of the SortOrder list. True per-key directions can't be expressed through the single opts.Sort string and are left out; no observed client sends a SortOrder list.
180 lines
4.6 KiB
Go
180 lines
4.6 KiB
Go
package persistence
|
|
|
|
import (
|
|
"cmp"
|
|
"context"
|
|
"fmt"
|
|
"reflect"
|
|
"strings"
|
|
"sync"
|
|
|
|
. "github.com/Masterminds/squirrel"
|
|
"github.com/deluan/rest"
|
|
"github.com/fatih/structs"
|
|
"github.com/navidrome/navidrome/log"
|
|
"github.com/navidrome/navidrome/model"
|
|
)
|
|
|
|
type filterFunc = func(field string, value any) Sqlizer
|
|
|
|
func (r *sqlRepository) parseRestFilters(ctx context.Context, options rest.QueryOptions) Sqlizer {
|
|
if len(options.Filters) == 0 {
|
|
return nil
|
|
}
|
|
filters := And{}
|
|
for f, v := range options.Filters {
|
|
// Ignore filters with empty values
|
|
if v == "" {
|
|
continue
|
|
}
|
|
// Look for a custom filter function
|
|
f = strings.ToLower(f)
|
|
if ff, ok := r.filterMappings[f]; ok {
|
|
if filter := ff(f, v); filter != nil {
|
|
filters = append(filters, filter)
|
|
}
|
|
continue
|
|
}
|
|
// Ignore invalid filters (not based on a field or filter function)
|
|
if r.isFieldWhiteListed != nil && !r.isFieldWhiteListed(f) {
|
|
log.Warn(ctx, "Ignoring filter not whitelisted", "filter", f, "table", r.tableName)
|
|
continue
|
|
}
|
|
// For fields ending in "id", use an exact match
|
|
if strings.HasSuffix(f, "id") {
|
|
filters = append(filters, eqFilter(f, v))
|
|
continue
|
|
}
|
|
// Default to a "starts with" filter
|
|
filters = append(filters, Like{f: fmt.Sprintf("%s%%", v)})
|
|
}
|
|
return filters
|
|
}
|
|
|
|
func (r *sqlRepository) parseRestOptions(ctx context.Context, options ...rest.QueryOptions) model.QueryOptions {
|
|
qo := model.QueryOptions{}
|
|
if len(options) > 0 {
|
|
qo.Sort, qo.Order = r.sanitizeSort(options[0].Sort, options[0].Order)
|
|
qo.Max = options[0].Max
|
|
qo.Offset = options[0].Offset
|
|
if seed, ok := options[0].Filters["seed"].(string); ok {
|
|
qo.Seed = seed
|
|
delete(options[0].Filters, "seed")
|
|
}
|
|
qo.Filters = r.parseRestFilters(ctx, options[0])
|
|
}
|
|
return qo
|
|
}
|
|
|
|
func (r sqlRepository) sanitizeSort(sort, order string) (string, string) {
|
|
if sort != "" {
|
|
sort = toSnakeCase(sort)
|
|
// Validate only: buildSortOrder resolves the mapping later, and mapping here as well would
|
|
// feed sortMapping its own output.
|
|
if _, _, known := r.lookupSortMapping(sort); !known && !r.isFieldWhiteListed(sort) {
|
|
log.Warn(r.ctx, "Ignoring sort not whitelisted", "sort", sort, "table", r.tableName)
|
|
sort = ""
|
|
}
|
|
}
|
|
if order != "" {
|
|
order = strings.ToLower(order)
|
|
if order != "desc" {
|
|
order = "asc"
|
|
}
|
|
}
|
|
return sort, order
|
|
}
|
|
|
|
func eqFilter(field string, value any) Sqlizer {
|
|
return Eq{field: value}
|
|
}
|
|
|
|
func startsWithFilter(field string) func(string, any) Sqlizer {
|
|
return func(_ string, value any) Sqlizer {
|
|
return Like{field: fmt.Sprintf("%s%%", value)}
|
|
}
|
|
}
|
|
|
|
func containsFilter(field string) func(string, any) Sqlizer {
|
|
return func(_ string, value any) Sqlizer {
|
|
return Like{field: fmt.Sprintf("%%%s%%", value)}
|
|
}
|
|
}
|
|
|
|
func booleanFilter(field string, value any) Sqlizer {
|
|
v := strings.ToLower(value.(string))
|
|
return Eq{field: v == "true"}
|
|
}
|
|
|
|
func fullTextFilter(tableName string, mbidFields ...string) func(string, any) Sqlizer {
|
|
return func(field string, value any) Sqlizer {
|
|
v := strings.ToLower(value.(string))
|
|
return cmp.Or[Sqlizer](
|
|
mbidExpr(tableName, v, mbidFields...),
|
|
getSearchStrategy(tableName, v),
|
|
)
|
|
}
|
|
}
|
|
|
|
func substringFilter(field string, value any) Sqlizer {
|
|
parts := strings.Fields(value.(string))
|
|
filters := And{}
|
|
for _, part := range parts {
|
|
filters = append(filters, Like{field: "%" + part + "%"})
|
|
}
|
|
return filters
|
|
}
|
|
|
|
func idFilter(tableName string) func(string, any) Sqlizer {
|
|
return func(field string, value any) Sqlizer { return Eq{tableName + ".id": value} }
|
|
}
|
|
|
|
func invalidFilter(ctx context.Context) func(string, any) Sqlizer {
|
|
return func(field string, value any) Sqlizer {
|
|
log.Warn(ctx, "Invalid filter", "fieldName", field, "value", value)
|
|
return Eq{"1": "0"}
|
|
}
|
|
}
|
|
|
|
var (
|
|
whiteList = map[string]map[string]struct{}{}
|
|
mutex sync.RWMutex
|
|
)
|
|
|
|
func registerModelWhiteList(instance any) fieldWhiteListedFunc {
|
|
name := reflect.TypeOf(instance).String()
|
|
registerFieldWhiteList(name, instance)
|
|
return getFieldWhiteListedFunc(name)
|
|
}
|
|
|
|
func registerFieldWhiteList(name string, instance any) {
|
|
mutex.Lock()
|
|
defer mutex.Unlock()
|
|
if whiteList[name] != nil {
|
|
return
|
|
}
|
|
m := structs.Map(instance)
|
|
whiteList[name] = map[string]struct{}{}
|
|
for k := range m {
|
|
whiteList[name][toSnakeCase(k)] = struct{}{}
|
|
}
|
|
ma := structs.Map(model.Annotations{})
|
|
for k := range ma {
|
|
whiteList[name][toSnakeCase(k)] = struct{}{}
|
|
}
|
|
}
|
|
|
|
type fieldWhiteListedFunc func(field string) bool
|
|
|
|
func getFieldWhiteListedFunc(tableName string) fieldWhiteListedFunc {
|
|
return func(field string) bool {
|
|
mutex.RLock()
|
|
defer mutex.RUnlock()
|
|
if _, ok := whiteList[tableName]; !ok {
|
|
return false
|
|
}
|
|
_, ok := whiteList[tableName][field]
|
|
return ok
|
|
}
|
|
}
|