mirror of
https://github.com/navidrome/navidrome.git
synced 2026-08-31 07:30:32 +00:00
* fix(jellyfin): stream /Items responses to prevent OOM on large libraries
Finamp's library sync issues an unbounded GET /Items?IncludeItemTypes=Audio
with Fields=MediaSources and no Limit. On a large library this built the whole
result set — every MediaFile and every BaseItemDto, each fat with MediaSources —
in memory, and json.Encoder then buffered the entire ~200MB response before
writing a byte. Measured on a 96k-track library, one such request peaked near
1.6GB RSS; in a memory-limited container the resulting slowness made clients
retry, stacking concurrent full-library builds until the process was OOM-killed.
Stream the song listing straight from a DB cursor (MediaFileRepository.GetCursor),
mapping and encoding one row at a time, so peak memory is bounded to about one
item regardless of library size. The cursor is opened lazily, when streaming
begins, so it doesn't hold a DB connection across the CountAll and ServerId
lookups that run first (ServerId can write on first use and would deadlock
against an open reader). Pagination still works — the cursor query carries
LIMIT/OFFSET/ORDER BY from StartIndex/Limit/SortBy — and TotalRecordCount stays
the full count. Other materialized responses stream their JSON too, avoiding the
encoder's buffer-the-whole-output cost.
Also bound artwork image concurrency with the same server.ThrottleBacklog that
Subsonic's getCoverArt uses, so a burst of image requests during sync can't
exhaust memory through concurrent decode/resize.
Verified against a copy of a 96k-track production database: byte-for-byte
identical response, peak RSS 1593MB -> 53MB, time-to-first-byte 8.1s -> 0.2s.
* fix(jellyfin): fail loudly on /Items streaming errors instead of a truncated 200
Addresses code review: a streamed /Items response commits HTTP 200 before an
error can occur, so failures were surfacing as misleadingly successful bodies.
- A cursor-open failure (e.g. a busy DB under connection pressure) is now
surfaced before the first byte is written: the cursor open is deferred into
the item source and run in writeItems after the ServerId lookup but before the
envelope, returning a clean 500 the client can retry — not a 200 with an empty
item list.
- A mid-stream (row-scan) error now aborts without closing the JSON envelope,
leaving the body malformed. A truncated-but-valid response would let a sync
client (Finamp) treat the short list as the whole library and prune local
tracks; malformed JSON forces the client's parser to fail and retry.
* refactor(jellyfin): stream every collection endpoint from a DB cursor
Streaming was applied only to the song listing, which left two write paths, two
ServerId stamping mechanisms and a listXxx return-type split ("songs is
special") that made the code hard to follow.
Add GetCursor to the album, artist, genre and playlist repositories, mirroring
MediaFileRepository.GetCursor: same select builder as GetAll, so each cursor
yields identical, fully-hydrated rows (hydration happens per-row in PostScan,
and toModels is only a deref loop). The playlist cursor keeps GetAll's
owner/public visibility filter. Each repository test asserts the cursor yields
exactly what GetAll returns, including Max/Offset.
With cursors everywhere, every listXxx returns itemsResult and every collection
streams through one writer:
- listAlbums, listArtists and listPlaylists now stream from their cursors;
search paths stay materialized (Search returns a slice).
- listGenres stays materialized: its total is the length of the full list and it
paginates in memory, so there is nothing for a cursor to page over.
- api.ok's QueryResult case delegates to writeItems, so the ~9 inline callers
funnel into the same writer; streamQueryResult and wrapResult are gone.
- /Items/Latest streams as a bare JSON array (its Jellyfin wire shape) via
writeItemsArray, which shares the item loop and stamping. That also closes the
same unbounded hole /Items had: limit=0 disabled its LIMIT.
- ServerId is now stamped in exactly one place, so api.ok only handles single,
non-collection payloads.
Verified against a copy of a 96k-track production database: every endpoint
byte-for-byte identical to master. Unbounded /Items peak RSS 1691MB -> 54MB;
time-to-first-byte 6.3s -> 0.19s, /Artists 1.14s -> 0.13s.
* refactor(jellyfin): route every response through api.ok
Handlers were split between api.ok and api.writeItems with no clear rule for
which to call. api.ok now accepts itemsResult too, so it is the single entry
point: callers hand it whatever they have and it routes collections (cursor
-backed or materialized) to the streaming writer. writeItems is reached only
through api.ok now. The one exception is /Items/Latest, which returns a bare
JSON array rather than a QueryResult envelope and so writes directly — noted in
both doc comments.
Also drop GenreRepository.GetCursor: listGenres derives its total from the
length of the full list and paginates in memory, so there is nothing for a
cursor to page over, leaving the method unused.
* fix(jellyfin): stream the unbounded multi-type /Items merge
The multi-type merge capped each per-type query at offset+limit only when a
Limit was given. Without one (Finamp's favorites screen sends multi-type), every
type ran an unbounded query and collect() drained each cursor into a slice — so
/Items?IncludeItemTypes=MusicAlbum,Audio with no Limit materialized every album
and every song, the same OOM class this branch set out to fix. The comment
claiming the set was "capped at offset+limit per type" was wrong for limit=0.
Without a limit the merged page is just each type's rows in order minus the
first offset, which is exactly what chaining the per-type cursors yields, so
stream that instead of merging in memory. The bounded path is unchanged: with a
Limit each type holds at most offset+limit rows, so merging and paginating
across the combined list is safe. Cursors are opened one at a time (each pins a
DB connection until drained), with the first opened eagerly so the usual failure
is still a clean error before any byte is written.
Measured on a 96k-track library, /Items?IncludeItemTypes=MusicAlbum,Audio with
no Limit (103,393 items): peak RSS 612MB -> 53MB, time-to-first-byte 4.9s ->
0.6s, response byte-for-byte identical.
* refactor(jellyfin): parse /Items params into a struct
The /Items dispatcher was a single 90-line block that parsed a dozen params and
then threaded them positionally through three layers — queryItemsOfType took 11
arguments, listSongs and listAlbums 9 each — so reading any one of them meant
decoding a long argument list.
Parse once into an itemsQuery and pass that instead. queryItems now reads as the
four things it actually does: parse, the id/playlists-folder/playlist-parent
special cases, single-type dispatch, multi-type merge — with the playlist-parent
and merge bodies moved to playlistTracks and mergeTypes. Every listXxx takes
(ctx, opts, q).
No behavior change: parsing, ordering and the entityParent rule are unchanged,
and /Artists still passes favOnly=false explicitly by building the subset of the
query it uses.
* docs(jellyfin): trim comments on the streaming path
Cut the comments back to the non-obvious why: the deferred cursor open (the
ServerId write would deadlock against an open reader), the deliberate malformed
JSON on a mid-stream error, why chained opens one cursor at a time, why
listGenres stays materialized, and why the cursor helpers take the underlying
func type. Dropped the rest — restatements of the code, doc comments that
repeated a test's own name, and the GetCursor interface comments that the
existing MediaFileRepository.GetCursor does without.
* refactor(persistence): fold the cursor wrappers into a generic wrapCursor
wrapAlbumCursor, wrapArtistCursor, wrapMediaFileCursor and wrapFolderCursor were
the same twelve lines four times over, differing only in the type name, and the
playlist cursor had its own inline copy of the loop.
Add one generic wrapCursor. It takes an extractor func rather than a method on
an interface: a type parameter can't reach an embedded field, and methods that
only satisfy a generic constraint are reported by the unused linter. The
extractor also lets both type parameters be inferred, so call sites need no
explicit instantiation. Each entity keeps its named wrapper as a one-liner,
since the model cursor types are defined types and need the conversion — and the
existing wrapper tests call them directly.
The nil-row error now names the model type via %T ("unexpected nil model.Album")
instead of a hand-written per-entity string; the three tests asserting that
message are updated.
* fix(jellyfin): bound concurrent collection streams
A streamed collection holds a DB cursor — and its pooled connection — for the whole client-paced
response, where the old materialize-then-write path released it as soon as the query finished. The
pool is shared with the scanner, Subsonic, the native API and the UI, so enough slow clients take
every connection and everything else blocks waiting for one. Measured against a copy of a 96k-track
production DB, 20 concurrent unbounded streams against a pool of 16 stalled a write for 16.2s (the
other 14 writes in the run took ~2ms — the signature of connection starvation, not lock contention).
Cap concurrent streams at half the pool. Excess requests queue rather than fail, so no client is
rejected: the same 20 streams now all complete and the worst write is 2.5ms.
- conf.MaxOpenConns() now owns the pool sizing (db calls it). It belongs in conf: the pool is a
tunable, db already imports conf, and putting it in db would force server/jellyfin to import db
just to size the cap against it. Expressing the cap as MaxOpenConns()/2 also keeps the two from
drifting apart.
- Uses chi's ThrottleBacklog, not server.ThrottleBacklog: the latter buffers the whole response to
release its token early, which is right for artwork but would undo the streaming. chi's panics on a
non-positive limit, so throttleStreams guards it — setting MaxConcurrentStreams=0 disables the cap
instead of crashing the server at startup.
* refactor(jellyfin): move throttleStreams to middlewares.go
It's a middleware, so it belongs beside normalizeQueryKeys, authenticate and
withPlayer rather than in api.go. Its tests move to middlewares_test.go with it,
keeping one test file per production file.
* fix(jellyfin): abandon the scan when the client stops reading
encodeItems discarded its write errors and relied on the final Flush to report
them, so once bufio's buffer filled and the flush failed, the loop still pulled
every remaining row through the cursor and serialized it for a client that was
gone. A test with a failing writer confirms it: all 20k items were drained.
That wastes CPU, and holds the cursor's pooled DB connection and a stream slot
(now a capped resource) for the length of a full scan nobody is reading. Check
the per-item writes so the first failure ends the scan; the fixed envelope
writes stay unchecked, since bufio latches for them anyway.
795 lines
29 KiB
Go
795 lines
29 KiB
Go
package jellyfin
|
|
|
|
import (
|
|
"context"
|
|
"io"
|
|
"iter"
|
|
"net/http"
|
|
"slices"
|
|
"strconv"
|
|
"strings"
|
|
|
|
"github.com/Masterminds/squirrel"
|
|
"github.com/go-chi/chi/v5"
|
|
"github.com/navidrome/navidrome/log"
|
|
"github.com/navidrome/navidrome/model"
|
|
"github.com/navidrome/navidrome/model/request"
|
|
"github.com/navidrome/navidrome/server/filter"
|
|
"github.com/navidrome/navidrome/server/jellyfin/dto"
|
|
"github.com/navidrome/navidrome/utils/req"
|
|
"github.com/navidrome/navidrome/utils/slice"
|
|
)
|
|
|
|
// notMissing excludes items whose backing files are all gone ("missing" is a real column on
|
|
// album, artist and media_file).
|
|
var notMissing = squirrel.Eq{"missing": false}
|
|
|
|
func (api *Router) getItems(w http.ResponseWriter, r *http.Request) {
|
|
res, err := api.queryItems(r.Context(), r)
|
|
if err != nil {
|
|
api.internalError(w, r, err)
|
|
return
|
|
}
|
|
api.ok(w, r, res)
|
|
}
|
|
|
|
// itemsResult is the outcome of a collection query: a materialized page, or a cursor opener so a
|
|
// full-library response never builds every DTO at once. Exactly one of items/openCursor is set.
|
|
//
|
|
// openCursor is deferred rather than opened here: it must run after the ServerId lookup, which
|
|
// writes to the DB on first use and would deadlock against an open reader, but before the first
|
|
// response byte, so a failed open is still a clean error rather than a truncated 200.
|
|
type itemsResult struct {
|
|
items []dto.BaseItemDto
|
|
openCursor func() (iter.Seq2[dto.BaseItemDto, error], error)
|
|
total int
|
|
start int
|
|
}
|
|
|
|
func materialized(q dto.QueryResult) itemsResult {
|
|
return itemsResult{items: q.Items, total: q.TotalRecordCount, start: q.StartIndex}
|
|
}
|
|
|
|
func streamed(open func() (iter.Seq2[dto.BaseItemDto, error], error), total, start int) itemsResult {
|
|
return itemsResult{openCursor: open, total: total, start: start}
|
|
}
|
|
|
|
// chained streams several results back to back, skipping the first skip items — the unbounded
|
|
// multi-type merge, where paginate(items, offset, 0) is just the concatenation minus its head.
|
|
func chained(results []itemsResult, total, skip int) itemsResult {
|
|
open := func() (iter.Seq2[dto.BaseItemDto, error], error) {
|
|
if len(results) == 0 {
|
|
return sliceItems(nil), nil
|
|
}
|
|
// Only the first opens eagerly (so the usual failure is still a clean error); the rest open as
|
|
// the stream reaches them, so only one cursor pins a DB connection at a time.
|
|
first, err := results[0].seq()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return func(yield func(dto.BaseItemDto, error) bool) {
|
|
n := 0
|
|
emit := func(seq iter.Seq2[dto.BaseItemDto, error]) bool {
|
|
for it, err := range seq {
|
|
if err != nil {
|
|
yield(dto.BaseItemDto{}, err)
|
|
return false
|
|
}
|
|
if n < skip {
|
|
n++
|
|
continue
|
|
}
|
|
if !yield(it, nil) {
|
|
return false
|
|
}
|
|
}
|
|
return true
|
|
}
|
|
if !emit(first) {
|
|
return
|
|
}
|
|
for _, res := range results[1:] {
|
|
seq, err := res.seq()
|
|
if err != nil {
|
|
yield(dto.BaseItemDto{}, err)
|
|
return
|
|
}
|
|
if !emit(seq) {
|
|
return
|
|
}
|
|
}
|
|
}, nil
|
|
}
|
|
return streamed(open, total, skip)
|
|
}
|
|
|
|
// streamCursor builds a deferred opener that maps each row as it's yielded. It takes the cursor's
|
|
// underlying func type, so callers wrap repo.GetCursor for the named type to infer T.
|
|
func streamCursor[T any](openCursor func() (func(func(T, error) bool), error), toItem func(T) dto.BaseItemDto) func() (iter.Seq2[dto.BaseItemDto, error], error) {
|
|
return func() (iter.Seq2[dto.BaseItemDto, error], error) {
|
|
cursor, err := openCursor()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return func(yield func(dto.BaseItemDto, error) bool) {
|
|
for row, err := range cursor {
|
|
if err != nil {
|
|
yield(dto.BaseItemDto{}, err)
|
|
return
|
|
}
|
|
if !yield(toItem(row), nil) {
|
|
return
|
|
}
|
|
}
|
|
}, nil
|
|
}
|
|
}
|
|
|
|
// seq returns the items as one sequence, opening the cursor if there is one.
|
|
func (ir itemsResult) seq() (iter.Seq2[dto.BaseItemDto, error], error) {
|
|
if ir.openCursor != nil {
|
|
return ir.openCursor()
|
|
}
|
|
return sliceItems(ir.items), nil
|
|
}
|
|
|
|
// collect drains the result into a slice, for the merge that combines types before paginating.
|
|
func (ir itemsResult) collect() ([]dto.BaseItemDto, error) {
|
|
if ir.openCursor == nil {
|
|
return ir.items, nil
|
|
}
|
|
seq, err := ir.openCursor()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
var out []dto.BaseItemDto
|
|
for it, err := range seq {
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
out = append(out, it)
|
|
}
|
|
return out, nil
|
|
}
|
|
|
|
func (api *Router) writeItems(w http.ResponseWriter, r *http.Request, res itemsResult) {
|
|
api.streamResult(w, r, res, func(w io.Writer, items iter.Seq2[dto.BaseItemDto, error]) error {
|
|
return streamItemsEnvelope(w, items, res.total, res.start)
|
|
})
|
|
}
|
|
|
|
// writeItemsArray writes the bare-array shape (/Items/Latest), which has no QueryResult envelope.
|
|
func (api *Router) writeItemsArray(w http.ResponseWriter, r *http.Request, res itemsResult) {
|
|
api.streamResult(w, r, res, streamItemsArray)
|
|
}
|
|
|
|
// streamResult stamps every item's ServerId (constant per request, so it's set here rather than in
|
|
// each mapper). The cursor opens before the first byte, so a failed open is still a clean 500.
|
|
func (api *Router) streamResult(w http.ResponseWriter, r *http.Request, res itemsResult,
|
|
write func(io.Writer, iter.Seq2[dto.BaseItemDto, error]) error) {
|
|
sid := api.serverID(r.Context())
|
|
seq, err := res.seq()
|
|
if err != nil {
|
|
api.internalError(w, r, err)
|
|
return
|
|
}
|
|
stamped := func(yield func(dto.BaseItemDto, error) bool) {
|
|
for it, err := range seq {
|
|
if err != nil {
|
|
yield(dto.BaseItemDto{}, err)
|
|
return
|
|
}
|
|
it.ServerId = sid
|
|
if !yield(it, nil) {
|
|
return
|
|
}
|
|
}
|
|
}
|
|
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
|
if err := write(w, stamped); err != nil {
|
|
log.Error(r.Context(), "Jellyfin API: error streaming response", err)
|
|
}
|
|
}
|
|
|
|
// itemsQuery is a parsed /Items request, so the dispatch and every listXxx take one value instead
|
|
// of a long positional parameter list.
|
|
type itemsQuery struct {
|
|
fields dto.Fields
|
|
ids []string
|
|
rawTypes string
|
|
types []string
|
|
search string
|
|
sortBy string
|
|
sortOrder string
|
|
offset int
|
|
limit int
|
|
favOnly bool
|
|
// parentId scopes the query. entityParent is the same id only when it names an entity (an artist
|
|
// for MusicAlbum, an album for Audio) rather than a library.
|
|
parentId string
|
|
entityParent string
|
|
isLibraryParent bool
|
|
scopeIDs []int
|
|
// artistId selects that artist's own discography; contributingOnly means albums they merely
|
|
// appear on (Jellyfin's "Featured On"), which must exclude that discography.
|
|
artistId string
|
|
contributingOnly bool
|
|
genreIds []string
|
|
}
|
|
|
|
// parseItemsQuery also resolves the entity types (inferring them from the parent when
|
|
// IncludeItemTypes is absent) and the library scope. Query keys are read lowercase because
|
|
// normalizeQueryKeys folded them (Jellyfin binds case-insensitively).
|
|
func (api *Router) parseItemsQuery(ctx context.Context, r *http.Request) itemsQuery {
|
|
p := req.Params(r)
|
|
q := itemsQuery{
|
|
fields: dto.ParseFields(p.StringOr("fields", "")),
|
|
ids: decodedQueryIDs(r, "ids"),
|
|
rawTypes: p.StringOr("includeitemtypes", ""),
|
|
search: p.StringOr("searchterm", ""),
|
|
sortBy: p.StringOr("sortby", ""),
|
|
sortOrder: p.StringOr("sortorder", ""),
|
|
offset: p.IntOr("startindex", 0),
|
|
limit: p.IntOr("limit", 0),
|
|
// Clients express "favorites only" two ways: Filters=IsFavorite and the standalone
|
|
// isFavorite=true param (Finamp's "Favourite tracks" widget uses the latter).
|
|
favOnly: strings.Contains(p.StringOr("filters", ""), "IsFavorite") || p.BoolOr("isfavorite", false),
|
|
parentId: dto.DecodeID(p.StringOr("parentid", "")),
|
|
// Finamp's genre screen sends ParentId=<libraryId> for scoping plus GenreIds for the genre.
|
|
genreIds: decodedQueryIDs(r, "genreids"),
|
|
}
|
|
// An artist's page filters by artist, not ParentId: Finamp sends ParentId=<libraryId> for scoping
|
|
// plus AlbumArtistIds/ArtistIds/contributingArtistIds for the artist.
|
|
albumArtistScope := firstNonEmpty(p.StringOr("albumartistids", ""), p.StringOr("artistids", ""))
|
|
contributingScope := p.StringOr("contributingartistids", "")
|
|
q.artistId = firstDecodedID(firstNonEmpty(albumArtistScope, contributingScope))
|
|
q.contributingOnly = albumArtistScope == "" && contributingScope != ""
|
|
|
|
q.types = parseTypes(q.rawTypes)
|
|
q.scopeIDs, q.isLibraryParent = resolveLibraryScope(ctx, q.parentId)
|
|
|
|
// With no item type, Jellyfin infers the child type from the parent: album parent -> its tracks
|
|
// (Jellify opens albums this way). An artist parent keeps parseTypes' MusicAlbum default (browse
|
|
// its albums).
|
|
if q.rawTypes == "" && q.parentId != "" && !q.isLibraryParent {
|
|
if q.parentId == playlistsFolderID {
|
|
// Browsing into the synthetic playlists folder lists the user's playlists.
|
|
q.types = []string{"Playlist"}
|
|
} else if _, err := api.ds.Album(ctx).Get(q.parentId); err == nil {
|
|
q.types = []string{"Audio"}
|
|
}
|
|
}
|
|
// ParentId-as-entity-id only makes sense for a single type; a multi-type query has no natural
|
|
// parent entity, so there ParentId is only library scoping.
|
|
q.entityParent = q.parentId
|
|
if q.isLibraryParent || len(q.types) > 1 {
|
|
q.entityParent = ""
|
|
}
|
|
return q
|
|
}
|
|
|
|
// queryItems is the /Items dispatcher: it resolves the request to entity types and queries each via
|
|
// the matching listXxx, merging multi-type results into one paginated list (as Finamp's favorites
|
|
// screen requests).
|
|
func (api *Router) queryItems(ctx context.Context, r *http.Request) (itemsResult, error) {
|
|
q := api.parseItemsQuery(ctx, r)
|
|
switch {
|
|
// /Items?ids= is a batch-fetch-by-id that bypasses the type dispatch.
|
|
case len(q.ids) > 0:
|
|
return materialized(api.itemsByIDs(ctx, q.ids, q.fields)), nil
|
|
// A ManualPlaylistsFolder query asks for the synthetic "playlists library" container, not real items.
|
|
case strings.Contains(q.rawTypes, "ManualPlaylistsFolder"):
|
|
return materialized(result([]dto.BaseItemDto{playlistsFolder()}, 1, 0)), nil
|
|
}
|
|
if res, ok := api.playlistTracks(ctx, q); ok {
|
|
return res, nil
|
|
}
|
|
if len(q.types) == 1 {
|
|
opts := model.QueryOptions{Offset: q.offset, Max: q.limit}
|
|
applySort(&opts, q.types[0], q.sortBy, q.sortOrder)
|
|
return api.queryItemsOfType(ctx, q.types[0], opts, q)
|
|
}
|
|
return api.mergeTypes(ctx, q)
|
|
}
|
|
|
|
// playlistTracks resolves a playlist parent to its tracks, whatever IncludeItemTypes says: Jellify
|
|
// opens a playlist with ParentId=<playlist>&IncludeItemTypes=Audio, and routing that through
|
|
// listSongs would treat the playlist id as an album id and return nothing.
|
|
func (api *Router) playlistTracks(ctx context.Context, q itemsQuery) (itemsResult, bool) {
|
|
if q.parentId == "" || q.isLibraryParent || q.parentId == playlistsFolderID {
|
|
return itemsResult{}, false
|
|
}
|
|
pls, err := api.playlists.GetWithTracks(ctx, q.parentId)
|
|
if err != nil {
|
|
return itemsResult{}, false
|
|
}
|
|
// GetWithTracks enforces visibility (public or owned by the current user).
|
|
items := slice.Map(pls.Tracks, func(t model.PlaylistTrack) dto.BaseItemDto { return trackToBaseItem(t, q.fields) })
|
|
return materialized(result(paginate(items, q.offset, q.limit), len(items), q.offset)), true
|
|
}
|
|
|
|
func (api *Router) mergeTypes(ctx context.Context, q itemsQuery) (itemsResult, error) {
|
|
// Each per-type query needs at most offset+limit rows (the worst case where one type fills the
|
|
// whole [offset, offset+limit) window). Totals are unaffected — they come from CountAll.
|
|
var results []itemsResult
|
|
total := 0
|
|
for _, itemType := range q.types {
|
|
var opts model.QueryOptions
|
|
if q.limit > 0 {
|
|
opts.Max = q.offset + q.limit
|
|
}
|
|
applySort(&opts, itemType, q.sortBy, q.sortOrder)
|
|
res, err := api.queryItemsOfType(ctx, itemType, opts, q)
|
|
if err != nil {
|
|
return itemsResult{}, err
|
|
}
|
|
results = append(results, res)
|
|
total += res.total
|
|
}
|
|
if q.limit == 0 {
|
|
// No cap above, so merging in memory would pull every row of every type. The merged page is
|
|
// just their rows in order minus the first offset — what chaining the cursors yields.
|
|
return chained(results, total, q.offset), nil
|
|
}
|
|
var items []dto.BaseItemDto
|
|
for _, res := range results {
|
|
typeItems, err := res.collect()
|
|
if err != nil {
|
|
return itemsResult{}, err
|
|
}
|
|
items = append(items, typeItems...)
|
|
}
|
|
return materialized(result(paginate(items, q.offset, q.limit), total, q.offset)), nil
|
|
}
|
|
|
|
func (api *Router) queryItemsOfType(ctx context.Context, itemType string, opts model.QueryOptions, q itemsQuery) (itemsResult, error) {
|
|
switch itemType {
|
|
case "Audio":
|
|
return api.listSongs(ctx, opts, q)
|
|
case "MusicArtist":
|
|
// The MusicArtist browse hierarchy (UserViews -> artists -> albums) means album artists.
|
|
return api.listArtists(ctx, opts, q, model.RoleAlbumArtist)
|
|
case "MusicGenre":
|
|
return api.listGenres(ctx, opts)
|
|
case "Playlist":
|
|
return api.listPlaylists(ctx, opts, q)
|
|
default: // MusicAlbum
|
|
return api.listAlbums(ctx, opts, q)
|
|
}
|
|
}
|
|
|
|
// firstNonEmpty returns the first non-empty string, or "".
|
|
func firstNonEmpty(vals ...string) string {
|
|
for _, v := range vals {
|
|
if v != "" {
|
|
return v
|
|
}
|
|
}
|
|
return ""
|
|
}
|
|
|
|
// firstDecodedID decodes the first id from a (possibly comma-separated) Jellyfin id list.
|
|
func firstDecodedID(s string) string {
|
|
if s == "" {
|
|
return ""
|
|
}
|
|
first, _, _ := strings.Cut(s, ",")
|
|
return dto.DecodeID(strings.TrimSpace(first))
|
|
}
|
|
|
|
// decodedQueryIDs reads an id-list param in both client spellings (see queryIDs), decoding each id.
|
|
func decodedQueryIDs(r *http.Request, key string) []string {
|
|
return slice.Map(queryIDs(r, key), dto.DecodeID)
|
|
}
|
|
|
|
// parseTypes returns the recognized entries in IncludeItemTypes in order, defaulting to
|
|
// {"MusicAlbum"} when none are recognized (so ParentId=<artistId> browses that artist's albums).
|
|
func parseTypes(types string) []string {
|
|
var recognized []string
|
|
for t := range strings.SplitSeq(types, ",") {
|
|
t = strings.TrimSpace(t)
|
|
switch t {
|
|
case "Audio", "MusicArtist", "MusicAlbum", "MusicGenre", "Playlist":
|
|
recognized = append(recognized, t)
|
|
}
|
|
}
|
|
if len(recognized) == 0 {
|
|
return []string{"MusicAlbum"}
|
|
}
|
|
return recognized
|
|
}
|
|
|
|
// paginate applies StartIndex/Limit to an in-memory item list, for the multi-type merge path only
|
|
// (single-type queries push Offset/Max down to SQL instead).
|
|
func paginate(items []dto.BaseItemDto, offset, limit int) []dto.BaseItemDto {
|
|
if offset >= len(items) {
|
|
return []dto.BaseItemDto{}
|
|
}
|
|
items = items[offset:]
|
|
if limit > 0 && limit < len(items) {
|
|
items = items[:limit]
|
|
}
|
|
return items
|
|
}
|
|
|
|
// searchPage runs a repository Search fetching one extra row to derive TotalRecordCount, since the
|
|
// Search API returns no match count and CountAll can't see the search term. offset+len(rows) is
|
|
// exact once matches end (and a growing lower bound before), so paging terminates at the last match.
|
|
func searchPage[S ~[]E, E any](opts model.QueryOptions, search func(model.QueryOptions) (S, error)) (S, int, error) {
|
|
fetch := opts
|
|
if fetch.Max > 0 {
|
|
fetch.Max++
|
|
}
|
|
rows, err := search(fetch)
|
|
if err != nil {
|
|
return nil, 0, err
|
|
}
|
|
total := opts.Offset + len(rows)
|
|
if opts.Max > 0 && len(rows) > opts.Max {
|
|
rows = rows[:opts.Max]
|
|
}
|
|
return rows, total, nil
|
|
}
|
|
|
|
func (api *Router) listAlbums(ctx context.Context, opts model.QueryOptions, q itemsQuery) (itemsResult, error) {
|
|
repo := api.ds.Album(ctx)
|
|
filters := squirrel.And{}
|
|
// For albums, ParentId (browse an artist) and AlbumArtistIds/ArtistIds both mean "this artist's
|
|
// albums"; contributingArtistIds means "albums they only appear on" (Featured On).
|
|
switch {
|
|
case q.contributingOnly && q.artistId != "":
|
|
filters = append(filters, filter.AlbumsByContributingArtistID(q.artistId).Filters)
|
|
case firstNonEmpty(q.artistId, q.entityParent) != "":
|
|
filters = append(filters, filter.AlbumsByArtistID(firstNonEmpty(q.artistId, q.entityParent)).Filters)
|
|
default:
|
|
filters = append(filters, notMissing)
|
|
}
|
|
if len(q.genreIds) > 0 {
|
|
filters = append(filters, filter.ByGenreID(q.genreIds))
|
|
}
|
|
if q.favOnly {
|
|
filters = append(filters, filter.ByStarred().Filters)
|
|
}
|
|
opts.Filters = filters
|
|
opts = filter.ApplyLibraryFilter(opts, q.scopeIDs)
|
|
|
|
if q.search != "" {
|
|
albums, total, err := searchPage(opts, func(o model.QueryOptions) (model.Albums, error) {
|
|
return repo.Search(q.search, o)
|
|
})
|
|
if err != nil {
|
|
return itemsResult{}, err
|
|
}
|
|
return materialized(result(slice.Map(albums, dto.AlbumToBaseItem), total, opts.Offset)), nil
|
|
}
|
|
total, _ := repo.CountAll(model.QueryOptions{Filters: opts.Filters})
|
|
open := streamCursor(func() (func(func(model.Album, error) bool), error) {
|
|
return repo.GetCursor(opts)
|
|
}, dto.AlbumToBaseItem)
|
|
return streamed(open, int(total), opts.Offset), nil
|
|
}
|
|
|
|
func (api *Router) listSongs(ctx context.Context, opts model.QueryOptions, q itemsQuery) (itemsResult, error) {
|
|
toItem := func(mf model.MediaFile) dto.BaseItemDto { return dto.SongToBaseItem(mf, q.fields) }
|
|
repo := api.ds.MediaFile(ctx)
|
|
filters := squirrel.And{}
|
|
// For songs, ArtistIds/AlbumArtistIds selects an artist's tracks; ParentId selects an album's.
|
|
switch {
|
|
case q.artistId != "":
|
|
filters = append(filters, filter.SongsByArtistID(q.artistId).Filters)
|
|
case q.entityParent != "":
|
|
filters = append(filters, filter.SongsByAlbum(q.entityParent).Filters)
|
|
default:
|
|
filters = append(filters, notMissing)
|
|
}
|
|
if len(q.genreIds) > 0 {
|
|
filters = append(filters, filter.ByGenreID(q.genreIds))
|
|
}
|
|
if q.favOnly {
|
|
filters = append(filters, filter.ByStarred().Filters)
|
|
}
|
|
opts.Filters = filters
|
|
opts = filter.ApplyLibraryFilter(opts, q.scopeIDs)
|
|
|
|
if q.search != "" {
|
|
mfs, total, err := searchPage(opts, func(o model.QueryOptions) (model.MediaFiles, error) {
|
|
return repo.Search(q.search, o)
|
|
})
|
|
if err != nil {
|
|
return itemsResult{}, err
|
|
}
|
|
return materialized(result(slice.Map(mfs, toItem), total, opts.Offset)), nil
|
|
}
|
|
// When browsing an album's tracks, default to disc+track order (like Subsonic's GetAlbum); an
|
|
// explicit client SortBy still wins, since applySort already set opts.Sort.
|
|
if q.artistId == "" && q.entityParent != "" && opts.Sort == "" {
|
|
opts.Sort = filter.SongsByAlbum(q.entityParent).Sort
|
|
}
|
|
// A full-library request (Finamp's sync, with MediaSources) is tens of thousands of fat rows.
|
|
total, _ := repo.CountAll(model.QueryOptions{Filters: opts.Filters})
|
|
open := streamCursor(func() (func(func(model.MediaFile, error) bool), error) {
|
|
return repo.GetCursor(opts)
|
|
}, toItem)
|
|
return streamed(open, int(total), opts.Offset), nil
|
|
}
|
|
|
|
// listArtists lists artists in the given role: RoleAlbumArtist for the "album artists" views,
|
|
// RoleArtist for performing artists (/Artists). Without the role filter both lists would be identical.
|
|
// genreIds isn't applied to search — a name lookup, like role (see below).
|
|
func (api *Router) listArtists(ctx context.Context, opts model.QueryOptions, q itemsQuery, role model.Role) (itemsResult, error) {
|
|
repo := api.ds.Artist(ctx)
|
|
|
|
// Artist Search does its own library scoping: it consumes a sole Eq{"library_id": ...} filter as a
|
|
// search scope (artists have no library_id column). A compound or join-based filter
|
|
// (ApplyArtistLibraryFilter) would leak into the FTS query and 500, so search and browse build
|
|
// filters differently. Role isn't applied to search for the same reason — it's a name lookup.
|
|
if q.search != "" {
|
|
if len(q.scopeIDs) > 0 {
|
|
opts.Filters = squirrel.Eq{"library_id": q.scopeIDs}
|
|
}
|
|
artists, total, err := searchPage(opts, func(o model.QueryOptions) (model.Artists, error) {
|
|
return repo.Search(q.search, o)
|
|
})
|
|
if err != nil {
|
|
return itemsResult{}, err
|
|
}
|
|
return materialized(result(slice.Map(artists, dto.ArtistToBaseItem), total, opts.Offset)), nil
|
|
}
|
|
|
|
if q.favOnly {
|
|
opts.Filters = filter.ArtistsByStarred().Filters
|
|
} else {
|
|
opts.Filters = notMissing
|
|
}
|
|
if len(q.genreIds) > 0 {
|
|
opts.Filters = squirrel.And{opts.Filters, filter.ArtistsByGenreID(q.genreIds)}
|
|
}
|
|
opts = filter.ArtistsByRole(opts, role)
|
|
opts = filter.ApplyArtistLibraryFilter(opts, q.scopeIDs)
|
|
total, _ := repo.CountAll(model.QueryOptions{Filters: opts.Filters})
|
|
open := streamCursor(func() (func(func(model.Artist, error) bool), error) {
|
|
return repo.GetCursor(opts)
|
|
}, dto.ArtistToBaseItem)
|
|
return streamed(open, int(total), opts.Offset), nil
|
|
}
|
|
|
|
// listGenres is intentionally unscoped: genres are global tags, not per-library entities. It's also
|
|
// the one listXxx that stays materialized: GenreRepository has no CountAll, so the total is the
|
|
// length of the full list and paging is in-memory — nothing for a cursor to page over.
|
|
func (api *Router) listGenres(ctx context.Context, opts model.QueryOptions) (itemsResult, error) {
|
|
genres, err := api.ds.Genre(ctx).GetAll(model.QueryOptions{Sort: opts.Sort, Order: opts.Order})
|
|
if err != nil {
|
|
return itemsResult{}, err
|
|
}
|
|
items := slice.Map(genres, dto.GenreToBaseItem)
|
|
return materialized(result(paginate(items, opts.Offset, opts.Max), len(items), opts.Offset)), nil
|
|
}
|
|
|
|
// listPlaylists lists playlists visible to the current user. Visibility (public or owned) is
|
|
// enforced by playlistRepository, not scopeIDs.
|
|
func (api *Router) listPlaylists(ctx context.Context, opts model.QueryOptions, q itemsQuery) (itemsResult, error) {
|
|
if q.favOnly {
|
|
starred := squirrel.Eq{"starred": true}
|
|
if opts.Filters == nil {
|
|
opts.Filters = starred
|
|
} else {
|
|
opts.Filters = squirrel.And{opts.Filters, starred}
|
|
}
|
|
}
|
|
repo := api.ds.Playlist(ctx)
|
|
total, err := repo.CountAll(model.QueryOptions{Filters: opts.Filters})
|
|
if err != nil {
|
|
return itemsResult{}, err
|
|
}
|
|
open := streamCursor(func() (func(func(model.Playlist, error) bool), error) {
|
|
return repo.GetCursor(opts)
|
|
}, dto.PlaylistToBaseItem)
|
|
return streamed(open, int(total), opts.Offset), nil
|
|
}
|
|
|
|
// resolveItemByID resolves a decoded navidrome id to its BaseItemDto, trying library view, album,
|
|
// artist, song and playlist in turn. Albums and songs report not-found when the user lacks access
|
|
// to their library, so an id can't probe content outside the user's libraries.
|
|
func (api *Router) resolveItemByID(ctx context.Context, id string, fields dto.Fields) (dto.BaseItemDto, bool) {
|
|
// The synthetic playlists folder must resolve by the id we advertised, not 404.
|
|
if id == playlistsFolderID {
|
|
return playlistsFolder(), true
|
|
}
|
|
u, _ := request.UserFrom(ctx)
|
|
// Finamp resolves a /UserViews entry (Id=library id) by fetching it as a plain item; without this
|
|
// the home screen and library tabs 404.
|
|
if libID, err := strconv.Atoi(id); err == nil && u.HasLibraryAccess(libID) {
|
|
for _, lib := range u.Libraries {
|
|
if lib.ID == libID {
|
|
return libraryView(lib), true
|
|
}
|
|
}
|
|
// Admin bypass: Libraries is empty but all access is granted, so fetch the real library.
|
|
if lib, err := api.ds.Library(ctx).Get(libID); err == nil {
|
|
return libraryView(*lib), true
|
|
}
|
|
}
|
|
if al, err := api.ds.Album(ctx).Get(id); err == nil {
|
|
if !u.HasLibraryAccess(al.LibraryID) {
|
|
return dto.BaseItemDto{}, false
|
|
}
|
|
return dto.AlbumToBaseItem(*al), true
|
|
}
|
|
if ar, err := api.ds.Artist(ctx).Get(id); err == nil {
|
|
// TODO: an artist spans multiple libraries (library_artist), so there's no single
|
|
// LibraryID to gate here; artist access relies on list-time scoping and persistence.
|
|
return dto.ArtistToBaseItem(*ar), true
|
|
}
|
|
if mf, err := api.ds.MediaFile(ctx).Get(id); err == nil {
|
|
if !u.HasLibraryAccess(mf.LibraryID) {
|
|
return dto.BaseItemDto{}, false
|
|
}
|
|
return dto.SongToBaseItem(*mf, fields), true
|
|
}
|
|
// api.playlists.Get enforces ownership/visibility, so a non-owned or missing id falls through.
|
|
if pl, err := api.playlists.Get(ctx, id); err == nil {
|
|
return dto.PlaylistToBaseItem(*pl), true
|
|
}
|
|
return dto.BaseItemDto{}, false
|
|
}
|
|
|
|
// songsByIDs fetches the media files among ids with chunked IN queries instead of a Get per id.
|
|
func (api *Router) songsByIDs(ctx context.Context, ids []string) map[string]model.MediaFile {
|
|
songs := make(map[string]model.MediaFile, len(ids))
|
|
// Chunked to stay under SQLITE_MAX_VARIABLE_NUMBER, like playqueue's loadTracks.
|
|
for chunk := range slice.CollectChunks(slices.Values(ids), 500) {
|
|
mfs, err := api.ds.MediaFile(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"media_file.id": chunk}})
|
|
if err != nil {
|
|
log.Error(ctx, "Jellyfin API: error fetching songs by id", err)
|
|
continue
|
|
}
|
|
for _, mf := range mfs {
|
|
songs[mf.ID] = mf
|
|
}
|
|
}
|
|
return songs
|
|
}
|
|
|
|
// itemsByIDs resolves a decoded id list, keeping input order and skipping unresolvable ids.
|
|
// A Finamp-truncated id is resolved by prefix but echoed as requested — Finamp matches restored
|
|
// queue items against its stored (truncated) ids.
|
|
func (api *Router) itemsByIDs(ctx context.Context, ids []string, fields dto.Fields) dto.QueryResult {
|
|
u, _ := request.UserFrom(ctx)
|
|
fullIDs := api.resolveItemIDs(ctx, ids)
|
|
songs := api.songsByIDs(ctx, fullIDs)
|
|
var items []dto.BaseItemDto
|
|
for i, id := range fullIDs {
|
|
var item dto.BaseItemDto
|
|
if mf, ok := songs[id]; ok {
|
|
if !u.HasLibraryAccess(mf.LibraryID) {
|
|
continue
|
|
}
|
|
item = dto.SongToBaseItem(mf, fields)
|
|
} else if item, ok = api.resolveItemByID(ctx, id, fields); !ok {
|
|
continue
|
|
}
|
|
if id != ids[i] {
|
|
item.Id = dto.EncodeID(ids[i])
|
|
}
|
|
items = append(items, item)
|
|
}
|
|
return result(items, len(items), 0)
|
|
}
|
|
|
|
func (api *Router) getItem(w http.ResponseWriter, r *http.Request) {
|
|
id := api.resolveItemID(r.Context(), dto.DecodeID(chi.URLParam(r, "itemId")))
|
|
fields := dto.ParseFields(req.Params(r).StringOr("fields", ""))
|
|
if item, ok := api.resolveItemByID(r.Context(), id, fields); ok {
|
|
api.ok(w, r, item)
|
|
return
|
|
}
|
|
http.Error(w, "Not Found", http.StatusNotFound)
|
|
}
|
|
|
|
// deleteItem handles DELETE /Items/{id}. Only playlists are deletable here (albums/songs come from
|
|
// scanning), so a non-playlist id 404s. core/playlists.Delete enforces ownership.
|
|
func (api *Router) deleteItem(w http.ResponseWriter, r *http.Request) {
|
|
ctx := r.Context()
|
|
id := dto.DecodeID(chi.URLParam(r, "itemId"))
|
|
if err := api.playlists.Delete(ctx, id); err != nil {
|
|
api.playlistError(w, r, err)
|
|
return
|
|
}
|
|
w.WriteHeader(http.StatusNoContent)
|
|
}
|
|
|
|
// getLatest returns a bare array, not a QueryResult envelope — real Jellyfin's shape for
|
|
// /Items/Latest, and why it writes directly instead of going through api.ok.
|
|
func (api *Router) getLatest(w http.ResponseWriter, r *http.Request) {
|
|
ctx := r.Context()
|
|
opts := filter.AlbumsByNewest()
|
|
opts.Max = req.Params(r).IntOr("limit", 20)
|
|
opts = filter.ApplyLibraryFilter(opts, accessibleLibraryIDs(ctx))
|
|
repo := api.ds.Album(ctx)
|
|
open := streamCursor(func() (func(func(model.Album, error) bool), error) {
|
|
return repo.GetCursor(opts)
|
|
}, dto.AlbumToBaseItem)
|
|
api.writeItemsArray(w, r, streamed(open, 0, 0))
|
|
}
|
|
|
|
func result(items []dto.BaseItemDto, total, start int) dto.QueryResult {
|
|
if items == nil {
|
|
items = []dto.BaseItemDto{}
|
|
}
|
|
return dto.QueryResult{Items: items, TotalRecordCount: total, StartIndex: start}
|
|
}
|
|
|
|
// applySort translates Jellyfin's SortBy/SortOrder into a valid model.QueryOptions sort key for the
|
|
// item type. Clients send SortBy as a comma-separated fallback list (e.g. "DateCreated,SortName");
|
|
// this uses the first recognized key. An unrecognized SortBy is left untouched (the repo's default),
|
|
// not passed through raw where it could produce an invalid ORDER BY.
|
|
func applySort(opts *model.QueryOptions, itemType, sortBy, order string) {
|
|
for key := range strings.SplitSeq(sortBy, ",") {
|
|
if col, ok := sortColumn(itemType, strings.TrimSpace(key)); ok {
|
|
opts.Sort = col
|
|
break
|
|
}
|
|
}
|
|
if strings.EqualFold(order, "Descending") {
|
|
opts.Order = "desc"
|
|
}
|
|
}
|
|
|
|
// sortColumnsByType maps lowercased-SortBy -> repo-sort-key per item type. Each repository maps
|
|
// logical fields to different real columns (e.g. media_file has "title" not "name"; artist has no
|
|
// "random").
|
|
var sortColumnsByType = map[string]map[string]string{
|
|
"Audio": {
|
|
"sortname": "title", "name": "title",
|
|
"album": "album",
|
|
// Finamp's album view sorts by ParentIndexNumber,IndexNumber (disc, track); Navidrome's
|
|
// "album" sort key is disc+track order within an album, so map both to it.
|
|
"indexnumber": "album",
|
|
"parentindexnumber": "album",
|
|
"artist": "artist",
|
|
"albumartist": "album_artist",
|
|
"datecreated": "recently_added",
|
|
"playcount": "play_count",
|
|
"dateplayed": "play_date",
|
|
"communityrating": "rating",
|
|
"random": "random",
|
|
// Finamp's "Latest Releases" sorts by PremiereDate; "year" matches songs' ProductionYear.
|
|
"premieredate": "year",
|
|
"productionyear": "year",
|
|
},
|
|
"MusicArtist": {
|
|
"sortname": "name", "name": "name",
|
|
"albumcount": "album_count",
|
|
"songcount": "song_count",
|
|
"datecreated": "created_at",
|
|
"playcount": "play_count",
|
|
"dateplayed": "play_date",
|
|
"communityrating": "rating",
|
|
},
|
|
"MusicAlbum": {
|
|
"sortname": "name", "name": "name", "album": "name",
|
|
"artist": "artist",
|
|
"albumartist": "album_artist",
|
|
"datecreated": "recently_added",
|
|
"random": "random",
|
|
"playcount": "play_count",
|
|
"dateplayed": "play_date",
|
|
"communityrating": "rating",
|
|
"premieredate": "max_year", "productionyear": "max_year",
|
|
},
|
|
"MusicGenre": {
|
|
"sortname": "name", "name": "name",
|
|
},
|
|
"Playlist": {
|
|
"sortname": "name", "name": "name",
|
|
"datecreated": "created_at",
|
|
},
|
|
}
|
|
|
|
// sortColumn maps a single (non comma-list) Jellyfin SortBy key to the repo sort key for
|
|
// itemType, reporting false when it isn't recognized for that type.
|
|
func sortColumn(itemType, sortBy string) (string, bool) {
|
|
col, ok := sortColumnsByType[itemType][strings.ToLower(sortBy)]
|
|
return col, ok
|
|
}
|