mirror of
https://github.com/navidrome/navidrome.git
synced 2026-08-31 07:30:32 +00:00
feat(jellyfin): send a synthetic placeholder blurhash for unresolved artwork (#5941)
* refactor(artwork): let callers pin the blurhash component counts * feat(artwork): synthesize a unique placeholder blurhash from a seed * fix(artwork): base the synthetic-hash no-collision guarantee on the prefix, not length The prior comment and test claimed a synthetic value could never collide with a real one because it's always shorter. That's false: components() targets ~16 tiles by scaling one axis down as the other hits the 9 cap, so an extreme aspect ratio (e.g. 10x200) collapses to 1x9 = 9 components, which encodes to the same 22 characters as a 3x3 synthetic hash. The old test only exercised a square 64x64 gradient, so it never caught this. The real guarantee is structural, not length-based: the first character encodes shape as (xComp-1)+(yComp-1)*9, and components() derives xf*yf = 16 exactly before flooring/capping (xf = sqrt(16w/h), yf = xf*h/w = sqrt(16h/w), so xf*yf = sqrt(256) = 16). Two factors both in [2,3) can't multiply to 16, so Encode can never derive 3x3 - the synthetic prefix 'K' is structurally exclusive to Synthetic. Replaced the length-based test with one that sweeps extreme aspect ratios and asserts no real encode ever produces prefix 'K', alongside the assertion that Synthetic always does. * feat(artwork): tint a synthetic blurhash from a base colour Parses baseColor into HSL and clamps saturation/lightness so the tint stays muted; per-cell hashing (unchanged) is what keeps a shared tint across an album's tracks from colliding, as the new 1M-seed spec proves under a single fixed colour. * fix(artwork): trim over-long comment in synthetic blurhash test Review flagged the collision spec's comment for exceeding the 2-line budget; the Finamp rationale it restated already lives in the design doc and commit history. * feat(jellyfin): send a synthetic blurhash for unresolved artwork Clients render nothing where a placeholder belongs when ImageBlurHashes is omitted for pending artwork. primaryImage now synthesizes a value (seeded on the tag, so a cover swap re-keys it) whenever a tag is present but no blurhash has been computed yet. Known-absent artwork (ImageAbsent) is unaffected: it still emits neither tag nor blurhash, since GetOrPlaceholder would otherwise pin a shared placeholder under a distinct cache key for a year. * fix(jellyfin): avoid computing a synthetic blurhash when a real one exists cmp.Or evaluates both arguments before choosing between them, so blurhash.Synthetic ran (and was discarded) on every call even when img.BlurHash was already set. That's the common case in production: artwork_repository.go populates BlurHash from persisted values once a scan resolves it, so a healthy library paid the synthesis cost (xxh3 hashing, HSL conversion, image alloc, DCT encode) on every mapped item for a value it never used. Branch on emptiness first instead. * feat(jellyfin): tint a pending track's placeholder with its album's colour primaryImage never reaches the embeddedArtPending branch of SongToBaseItem, since there is no resolved image yet to feed it. Seed the synthetic blurhash on mf.ID (so the value stays unique and the client still issues the read-through request) but tint it with the album's DominantColor, which is already hydrated on MediaFile at no extra cost. * style(artwork): trim synthetic blurhash comments to the budget * docs(jellyfin): fix stale blurhash README bullet + two review nits The "Blurhashes are synthetic" bullet under Known limitations described dto/blurhash.go, which was deleted when the real core/artwork-computed blurhash + synthetic-fallback pipeline landed; every claim in it was false. Replaced it with an accurate paragraph in the Images section, since the described behaviour is now the finished design, not a gap. Also: fix a doc/body comment mismatch in blurhash.go (component counts are 2..9, not 1..9), and deduplicate the inline DC-extraction logic in synthetic_test.go by reusing the existing dcOf helper. * refactor(artwork): slice the synthetic cell jitter on byte boundaries The three perturbations came off one hash with mismatched masks and shifts (0xFF at 0, 0x3F at 8, 0x3F at 14), so a reader had to do the arithmetic to confirm the fields did not overlap. Only 20 of the 64 bits were in use either way, so the narrower fields bought nothing. Uniform byte slices at 0/8/16 are non-overlapping by inspection and give each field the full 8 bits. Both 1,000,000-seed collision specs still measure 1,000,000 distinct values. Also drops the local `n` alias, which was a second name for synthComponents inside a 15-line function. * docs(jellyfin): fix the blurhash paragraph's opening sentence It opened with "follow the same principle", pointing back at the preceding paragraph on admin-context artwork resolution — an unrelated subject, so the reader looks for a connection that is not there. * fix(artwork): render the synthetic grid larger than its component count The 3x3 cell grid was handed straight to the encoder as a 3x3 image, so the source had exactly as many samples as basis functions. Blurhash normalises its coefficients by 1/(w*h) and 2/(w*h), which assumes many samples per component, so the AC terms came out far too large. At the bottom-right corner the x and y bases are both [1, -0.5, -0.5], everything lines up negative, and the result clamped to black — a dark blob on every synthetic placeholder. Rendering the same nine colours bilinearly at 8x8 first removes it: measured over three seeds, the darkest corner goes from 13 to 74 and the darkest pixel from 1 to 63. 8px is the smallest size that clears the artefact; 12 and 16 are visually indistinguishable and cost 1.7x and 2.7x more. The interpolation is hand-rolled rather than x/image's scaler, which allocated 528 times per call against 14 for this. Both 1,000,000-seed collision specs still measure 1,000,000 distinct values. * refactor(artwork): tidy the synthetic upscale helpers cellWeight nudged its upper bound with a 1e-9 epsilon so int() could never land on the last cell. Clamping the coordinate and then the index says the same thing without a magic constant, and makes the clamp-don't-extrapolate intent explicit — edge pixels map outside the cell centres, so the fraction would otherwise run past 1. The grid type is spelled once as colorGrid rather than repeated in the local and the upscale signature, and encodeAt's doc now states the source-size contract that Synthetic depends on, so the next caller sees it at the function rather than only in synthetic.go. Output is unchanged: all seven sample hashes match byte for byte. * perf(artwork): make the synthetic upscale separable Both axes are square and constant-sized, so the per-pixel cell index and weight were the same 8 values recomputed 64 times per call. They are now built once by sync.OnceValue, matching the srgbToLinearTable pattern. The interpolation is also separable: stretching each of the 3 grid rows horizontally once and then blending rows vertically does 264 lerps where the per-pixel form did 576. upscale drops from 347ns to 260ns, Synthetic from 1780ns to 1669ns. Output is byte-identical across all seven sample hashes — same operations in the same order, only hoisted. * refactor(artwork): let a caller supply the cosine basis encodeAt built its cosine tables inline, so Synthetic rebuilt bit-identical ones on every call — its shape is always 3 components over 8 pixels. Splitting the table construction into cosBasis and the encoder proper into encodePixels lets Synthetic build the basis once via sync.OnceValue, and leaves encodeAt's signature and behaviour untouched. Synthetic drops from 14 allocations to 6 (1669ns to 1508ns). The time saving is small and this path only runs while artwork is still unresolved; the allocation cut is the point, alongside a shorter encodeAt. Every hash is unchanged: nine real Encode outputs spanning square, 10x200, 200x10, 1x50 and a real JPEG, plus all seven synthetic samples, all byte for byte identical before and after.
This commit is contained in:
parent
9e95b19a4f
commit
036c9cab96
@ -32,23 +32,32 @@ func Encode(img image.Image) (string, error) {
|
||||
}
|
||||
// Pre-downscale: its rounding can flip a component count, and the hash is a client cache key.
|
||||
xComp, yComp := components(img.Bounds().Dx(), img.Bounds().Dy())
|
||||
src := pixelsOf(downscale(img))
|
||||
w, h := src.w, src.h
|
||||
return encodeAt(img, xComp, yComp), nil
|
||||
}
|
||||
|
||||
cosX := make([][]float64, xComp)
|
||||
for i := range cosX {
|
||||
cosX[i] = make([]float64, w)
|
||||
for x := range cosX[i] {
|
||||
cosX[i][x] = math.Cos(math.Pi * float64(i) * float64(x) / float64(w))
|
||||
}
|
||||
}
|
||||
cosY := make([][]float64, yComp)
|
||||
for j := range cosY {
|
||||
cosY[j] = make([]float64, h)
|
||||
for y := range cosY[j] {
|
||||
cosY[j][y] = math.Cos(math.Pi * float64(j) * float64(y) / float64(h))
|
||||
// encodeAt encodes img at the given component counts (each 2..9). Callers guarantee non-empty
|
||||
// bounds, and a source far larger than the counts: too few samples overshoot the AC terms.
|
||||
func encodeAt(img image.Image, xComp, yComp int) string {
|
||||
src := pixelsOf(downscale(img))
|
||||
return encodePixels(src, xComp, yComp, cosBasis(xComp, src.w), cosBasis(yComp, src.h))
|
||||
}
|
||||
|
||||
// cosBasis is the cosine basis for comp components sampled across n pixels. It depends only
|
||||
// on its arguments, so a caller with a fixed shape can build it once and reuse it.
|
||||
func cosBasis(comp, n int) [][]float64 {
|
||||
basis := make([][]float64, comp)
|
||||
for i := range basis {
|
||||
basis[i] = make([]float64, n)
|
||||
for x := range basis[i] {
|
||||
basis[i][x] = math.Cos(math.Pi * float64(i) * float64(x) / float64(n))
|
||||
}
|
||||
}
|
||||
return basis
|
||||
}
|
||||
|
||||
// encodePixels is the encoder proper; cosX and cosY must match src's dimensions.
|
||||
func encodePixels(src pixels, xComp, yComp int, cosX, cosY [][]float64) string {
|
||||
w, h := src.w, src.h
|
||||
|
||||
lin := srgbToLinearTable()
|
||||
factors := make([][3]float64, xComp*yComp)
|
||||
@ -86,7 +95,7 @@ func Encode(img image.Image) (string, error) {
|
||||
var sb strings.Builder
|
||||
sb.WriteString(encode83((xComp-1)+(yComp-1)*9, 1))
|
||||
|
||||
// Derived counts are at least 1x9, so there is always at least one AC factor.
|
||||
// Every caller passes at least 2 components, so there is always at least one AC factor.
|
||||
ac := factors[1:]
|
||||
actualMax := 0.0
|
||||
for _, f := range ac {
|
||||
@ -101,7 +110,7 @@ func Encode(img image.Image) (string, error) {
|
||||
for _, f := range ac {
|
||||
sb.WriteString(encode83(quantAC(f[0], maxVal)*19*19+quantAC(f[1], maxVal)*19+quantAC(f[2], maxVal), 2))
|
||||
}
|
||||
return sb.String(), nil
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// pixels is direct Pix access for the pixel loop, avoiding a per-pixel allocation via image.At.
|
||||
|
||||
172
core/artwork/blurhash/synthetic.go
Normal file
172
core/artwork/blurhash/synthetic.go
Normal file
@ -0,0 +1,172 @@
|
||||
package blurhash
|
||||
|
||||
import (
|
||||
"image"
|
||||
"math"
|
||||
"strconv"
|
||||
"sync"
|
||||
|
||||
"github.com/zeebo/xxh3"
|
||||
)
|
||||
|
||||
// A real hash never has 3x3 components: components() always targets ~16 tiles.
|
||||
const synthComponents = 3
|
||||
|
||||
// Encoding a source no bigger than the component grid overshoots the AC coefficients,
|
||||
// clamping the corners to black; the encoder assumes many samples per component.
|
||||
const synthSourceSize = 8
|
||||
|
||||
type colorGrid = [synthComponents * synthComponents][3]float64
|
||||
|
||||
// Synthetic returns a blurhash unique to seed, for artwork whose real hash does not exist
|
||||
// yet. baseColor ("#rrggbb") sets the hue family; "" derives it from the seed.
|
||||
func Synthetic(seed, baseColor string) string {
|
||||
hue, sat, light := baseTone(baseColor, xxh3.HashStringSeed(seed, 0))
|
||||
|
||||
var grid colorGrid
|
||||
for i := range grid {
|
||||
// Each cell hashes the seed separately, so one tint shared by many items still
|
||||
// yields one value per item.
|
||||
bits := xxh3.HashStringSeed(seed, uint64(i)+1)
|
||||
dh := (byteFrac(bits, 0) - 0.5) * 60
|
||||
ds := (byteFrac(bits, 8) - 0.5) * 0.16
|
||||
dl := (byteFrac(bits, 16) - 0.5) * 0.30
|
||||
r, g, b := hslToRGB(hue+dh, clamp01(sat+ds), clamp01(light+dl))
|
||||
grid[i] = [3]float64{float64(r), float64(g), float64(b)}
|
||||
}
|
||||
// The shape is fixed, so the cosine basis is reused instead of rebuilt per call.
|
||||
basis := synthBasis()
|
||||
return encodePixels(pixelsOf(upscale(&grid)), synthComponents, synthComponents, basis, basis)
|
||||
}
|
||||
|
||||
// synthBasis is the cosine basis for the fixed synthetic shape. Square, so one serves both axes.
|
||||
var synthBasis = sync.OnceValue(func() [][]float64 {
|
||||
return cosBasis(synthComponents, synthSourceSize)
|
||||
})
|
||||
|
||||
// upscale renders the cell grid bilinearly at synthSourceSize. Hand-rolled because
|
||||
// x/image's scaler allocates per call, and this runs once per mapped item.
|
||||
func upscale(grid *colorGrid) *image.NRGBA {
|
||||
const n, size = synthComponents, synthSourceSize
|
||||
axis := axisWeights()
|
||||
|
||||
// Separable: each grid row is stretched horizontally once, then rows blend vertically.
|
||||
// Doing it per pixel instead would repeat every horizontal lerp `size` times.
|
||||
var rows [n][size][3]float64
|
||||
for r := range n {
|
||||
for x, a := range axis {
|
||||
for c := range 3 {
|
||||
rows[r][x][c] = grid[r*n+a.cell][c]*(1-a.frac) + grid[r*n+a.cell+1][c]*a.frac
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
img := image.NewNRGBA(image.Rect(0, 0, size, size))
|
||||
for y, a := range axis {
|
||||
top, bot := &rows[a.cell], &rows[a.cell+1]
|
||||
for x := range size {
|
||||
p := img.Pix[y*img.Stride+x*4:]
|
||||
for c := range 3 {
|
||||
p[c] = uint8(top[x][c]*(1-a.frac) + bot[x][c]*a.frac + 0.5)
|
||||
}
|
||||
p[3] = 255
|
||||
}
|
||||
}
|
||||
return img
|
||||
}
|
||||
|
||||
// axisWeights maps each source pixel to its lower cell and the fraction toward the next.
|
||||
// Both axes are square and constant-sized, so one table serves both and outlives the call.
|
||||
var axisWeights = sync.OnceValue(func() *[synthSourceSize]struct {
|
||||
cell int
|
||||
frac float64
|
||||
} {
|
||||
var t [synthSourceSize]struct {
|
||||
cell int
|
||||
frac float64
|
||||
}
|
||||
for i := range t {
|
||||
// Edge pixels fall outside the cell centres, so both ends clamp rather than extrapolate.
|
||||
f := min(max((float64(i)+0.5)*synthComponents/synthSourceSize-0.5, 0), synthComponents-1)
|
||||
t[i].cell = min(int(f), synthComponents-2)
|
||||
t[i].frac = f - float64(t[i].cell)
|
||||
}
|
||||
return &t
|
||||
})
|
||||
|
||||
func byteFrac(bits uint64, shift int) float64 {
|
||||
return float64(bits>>shift&0xFF) / 255
|
||||
}
|
||||
|
||||
// baseTone clamps saturation and lightness away from the extremes, so a placeholder
|
||||
// never reads as neon or as solid black.
|
||||
func baseTone(baseColor string, hueBits uint64) (hue, sat, light float64) {
|
||||
if r, g, b, ok := parseHex(baseColor); ok {
|
||||
h, s, l := rgbToHSL(r, g, b)
|
||||
return h, min(max(s, 0.10), 0.35), min(max(l, 0.15), 0.75)
|
||||
}
|
||||
return float64(hueBits&0x1FF) / 512 * 360, 0.22, 0.40
|
||||
}
|
||||
|
||||
func parseHex(s string) (r, g, b uint8, ok bool) {
|
||||
if len(s) != 7 || s[0] != '#' {
|
||||
return 0, 0, 0, false
|
||||
}
|
||||
v, err := strconv.ParseUint(s[1:], 16, 32)
|
||||
if err != nil {
|
||||
return 0, 0, 0, false
|
||||
}
|
||||
return uint8(v >> 16), uint8(v >> 8), uint8(v), true
|
||||
}
|
||||
|
||||
func rgbToHSL(r, g, b uint8) (h, s, l float64) {
|
||||
rf, gf, bf := float64(r)/255, float64(g)/255, float64(b)/255
|
||||
mx, mn := max(rf, gf, bf), min(rf, gf, bf)
|
||||
l = (mx + mn) / 2
|
||||
if mx == mn {
|
||||
return 0, 0, l
|
||||
}
|
||||
d := mx - mn
|
||||
s = d / (1 - math.Abs(2*l-1))
|
||||
switch mx {
|
||||
case rf:
|
||||
h = math.Mod((gf-bf)/d, 6)
|
||||
case gf:
|
||||
h = (bf-rf)/d + 2
|
||||
default:
|
||||
h = (rf-gf)/d + 4
|
||||
}
|
||||
h *= 60
|
||||
if h < 0 {
|
||||
h += 360
|
||||
}
|
||||
return h, s, l
|
||||
}
|
||||
|
||||
func hslToRGB(h, s, l float64) (uint8, uint8, uint8) {
|
||||
h = math.Mod(math.Mod(h, 360)+360, 360)
|
||||
c := (1 - math.Abs(2*l-1)) * s
|
||||
hp := h / 60
|
||||
x := c * (1 - math.Abs(math.Mod(hp, 2)-1))
|
||||
var r, g, b float64
|
||||
switch int(hp) {
|
||||
case 0:
|
||||
r, g, b = c, x, 0
|
||||
case 1:
|
||||
r, g, b = x, c, 0
|
||||
case 2:
|
||||
r, g, b = 0, c, x
|
||||
case 3:
|
||||
r, g, b = 0, x, c
|
||||
case 4:
|
||||
r, g, b = x, 0, c
|
||||
default:
|
||||
r, g, b = c, 0, x
|
||||
}
|
||||
m := l - c/2
|
||||
return to8(r + m), to8(g + m), to8(b + m)
|
||||
}
|
||||
|
||||
func clamp01(v float64) float64 { return min(max(v, 0), 1) }
|
||||
|
||||
func to8(v float64) uint8 { return uint8(math.Round(clamp01(v) * 255)) }
|
||||
103
core/artwork/blurhash/synthetic_test.go
Normal file
103
core/artwork/blurhash/synthetic_test.go
Normal file
@ -0,0 +1,103 @@
|
||||
package blurhash_test
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/navidrome/navidrome/core/artwork/blurhash"
|
||||
"github.com/zeebo/xxh3"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("Synthetic", func() {
|
||||
It("returns a well-formed 3x3 blurhash", func() {
|
||||
h := blurhash.Synthetic("alb-1", "")
|
||||
// 3x3 encodes as (3-1)+(3-1)*9 = 20, which is base83 'K'; length is 1+1+4+8*2.
|
||||
Expect(h).To(HaveLen(22))
|
||||
Expect(h).To(HavePrefix("K"))
|
||||
for _, c := range h {
|
||||
Expect(strings.ContainsRune(alphabet, c)).To(BeTrue(), "unexpected char %q", c)
|
||||
}
|
||||
})
|
||||
|
||||
It("keeps its prefix exclusive to real encodes, across aspect ratios", func() {
|
||||
Expect(blurhash.Synthetic("alb-1", "")).To(HavePrefix("K"))
|
||||
|
||||
ratios := []struct{ w, h int }{
|
||||
{10, 200}, {200, 10}, {1, 50}, {50, 1}, {64, 64}, {100, 300}, {300, 100},
|
||||
}
|
||||
for _, r := range ratios {
|
||||
encoded, err := blurhash.Encode(gradientImage(r.w, r.h))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(encoded).ToNot(HavePrefix("K"), "real encode at %dx%d produced the synthetic prefix", r.w, r.h)
|
||||
}
|
||||
})
|
||||
|
||||
It("is deterministic for one seed", func() {
|
||||
Expect(blurhash.Synthetic("alb-1", "")).To(Equal(blurhash.Synthetic("alb-1", "")))
|
||||
})
|
||||
|
||||
It("differs across seeds", func() {
|
||||
Expect(blurhash.Synthetic("alb-1", "")).ToNot(Equal(blurhash.Synthetic("alb-2", "")))
|
||||
})
|
||||
|
||||
dcOf := func(hash string) (int, int, int) {
|
||||
dc := decode83(hash[2:6])
|
||||
return dc >> 16 & 0xFF, dc >> 8 & 0xFF, dc & 0xFF
|
||||
}
|
||||
|
||||
It("decodes to a muted DC colour", func() {
|
||||
r, g, b := dcOf(blurhash.Synthetic("alb-1", ""))
|
||||
spread := max(r, g, b) - min(r, g, b)
|
||||
Expect(spread).To(BeNumerically("<", 120), "saturation is capped, so no channel should run away")
|
||||
Expect(max(r, g, b)).To(BeNumerically("<", 240), "lightness is capped below white")
|
||||
Expect(min(r, g, b)).To(BeNumerically(">", 10), "lightness is capped above black")
|
||||
})
|
||||
|
||||
It("keeps 1,000,000 seeds effectively distinct", func() {
|
||||
const count = 1_000_000
|
||||
seen := make(map[uint64]struct{}, count)
|
||||
for i := range count {
|
||||
seen[xxh3.HashString(blurhash.Synthetic(fmt.Sprintf("item-%d", i), ""))] = struct{}{}
|
||||
}
|
||||
Expect(len(seen)).To(BeNumerically(">=", 999_900))
|
||||
})
|
||||
|
||||
It("leans the DC towards a red tint", func() {
|
||||
r, g, b := dcOf(blurhash.Synthetic("alb-1", "#c04040"))
|
||||
Expect(r).To(BeNumerically(">", g))
|
||||
Expect(r).To(BeNumerically(">", b))
|
||||
})
|
||||
|
||||
It("leans the DC towards a blue tint", func() {
|
||||
r, g, b := dcOf(blurhash.Synthetic("alb-1", "#4040c0"))
|
||||
Expect(b).To(BeNumerically(">", r))
|
||||
Expect(b).To(BeNumerically(">", g))
|
||||
})
|
||||
|
||||
It("changes the value when only the tint changes", func() {
|
||||
Expect(blurhash.Synthetic("alb-1", "#c04040")).ToNot(Equal(blurhash.Synthetic("alb-1", "#4040c0")))
|
||||
})
|
||||
|
||||
It("falls back to the seed hue for an unparseable tint", func() {
|
||||
Expect(blurhash.Synthetic("alb-1", "not-a-colour")).To(Equal(blurhash.Synthetic("alb-1", "")))
|
||||
})
|
||||
|
||||
It("tracks a dark tint's lightness", func() {
|
||||
dark, _, _ := dcOf(blurhash.Synthetic("alb-1", "#101820"))
|
||||
light, _, _ := dcOf(blurhash.Synthetic("alb-1", "#e8f0f8"))
|
||||
Expect(dark).To(BeNumerically("<", light))
|
||||
})
|
||||
|
||||
// A shared tint must not become a shared value: the seed alone carries uniqueness.
|
||||
It("keeps 1,000,000 seeds distinct under a single fixed tint", func() {
|
||||
const count = 1_000_000
|
||||
seen := make(map[uint64]struct{}, count)
|
||||
for i := range count {
|
||||
seen[xxh3.HashString(blurhash.Synthetic(fmt.Sprintf("track-%d", i), "#3a5f7d"))] = struct{}{}
|
||||
}
|
||||
Expect(len(seen)).To(BeNumerically(">=", 999_900))
|
||||
})
|
||||
})
|
||||
@ -180,6 +180,14 @@ warmer uses — so user-scoped items like private playlists still resolve their
|
||||
falling back to the placeholder. Album, artist, media-file and playlist ids are all resolved to
|
||||
their Navidrome `ArtworkID`.
|
||||
|
||||
Blurhashes come in three tiers. A real blurhash is computed once in
|
||||
`core/artwork` from the decoded image and stored per artwork row; the mappers read it whenever it
|
||||
exists. While artwork is still unresolved, `dto/mappers.go` instead emits a synthetic 3x3 blurhash
|
||||
(`core/artwork/blurhash.Synthetic`), seeded on the image tag so it's effectively unique per image
|
||||
and can never collide with a real hash's leading byte, giving clients a valid cache key while art
|
||||
loads. A track awaiting embedded-art extraction has its synthetic hash tinted from the parent
|
||||
album's dominant colour. Known-absent artwork emits no tag and no blurhash at all.
|
||||
|
||||
## Finamp saved-queue id truncation
|
||||
|
||||
Real Jellyfin item ids are GUIDs — 128-bit values, always 32 hex characters. Finamp relies on that
|
||||
@ -330,16 +338,6 @@ make test PKG=./server/jellyfin/...
|
||||
Access control for artists is enforced by scoping the `Artists`/`Items?IncludeItemTypes=MusicArtist`
|
||||
*list* to the user's libraries, plus the persistence layer's own defense-in-depth; a client
|
||||
that already has an artist id from elsewhere is not re-checked against library membership.
|
||||
- **Blurhashes are synthetic, not computed from the artwork (follow-up).** `ImageBlurHashes` is
|
||||
populated by `dto/blurhash.go`, which derives a well-formed **1-component (solid color)**
|
||||
blurhash by hashing the item id — it never looks at the actual image. Real Jellyfin computes a
|
||||
multi-component blurhash from the cover's pixels (downscaled to 128×128) once at scan time and
|
||||
stores it per image, so its placeholder approximates the art. Ours satisfies the protocol
|
||||
(Finamp gets a valid value to use as a de-dup key and a placeholder, no missing-blurhash
|
||||
warning) but renders as a flat color while art loads. A proper implementation would compute the
|
||||
real blurhash in the `core/artwork` pipeline (where the image is already decoded), cache it
|
||||
keyed like the artwork, and have the mappers read it — keeping the synthetic value as a fallback
|
||||
for art that hasn't been rendered yet.
|
||||
- **The WebSocket only keep-alives; it pushes no events (follow-up).** `GET socket` sends a
|
||||
`ForceKeepAlive` and answers `KeepAlive` pings so real-time clients (Finamp) settle into a
|
||||
working session instead of 404-loop-reconnecting, but it never pushes anything. A follow-up
|
||||
|
||||
@ -6,6 +6,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
"github.com/navidrome/navidrome/core/artwork/blurhash"
|
||||
"github.com/navidrome/navidrome/model"
|
||||
"github.com/navidrome/navidrome/utils/slice"
|
||||
)
|
||||
@ -212,6 +213,8 @@ func SongToBaseItem(mf model.MediaFile, fields Fields) BaseItemDto {
|
||||
// Nothing enqueues media files: advertising the id is what makes a client ask, and that
|
||||
// request is what extracts the embedded art and queues the track.
|
||||
item.ImageTags = map[string]string{"Primary": mf.ID}
|
||||
hash := blurhash.Synthetic(mf.ID, mf.AlbumImage.DominantColor)
|
||||
item.ImageBlurHashes = map[string]map[string]string{"Primary": {mf.ID: hash}}
|
||||
} else if mf.AlbumID != "" {
|
||||
if tag, blurs, ratio := primaryImage(mf.AlbumImage, mf.AlbumID, fields); tag != "" {
|
||||
item.AlbumPrimaryImageTag = tag
|
||||
@ -227,15 +230,19 @@ func embeddedArtPending(mf model.MediaFile) bool {
|
||||
mf.ImageHash == "" && !mf.ItemImage.ImageAbsent
|
||||
}
|
||||
|
||||
// primaryImage never fakes a blurhash: clients key their cover cache on the value, which would
|
||||
// pin a stale cover forever.
|
||||
// primaryImage synthesizes a blurhash when none was computed yet: clients key their cover
|
||||
// cache on the value, so it must be unique per image, never shared or reused.
|
||||
func primaryImage(img model.ItemImage, fallback string, fields Fields) (tag string, blurs map[string]map[string]string, ratio *float64) {
|
||||
if img.ImageAbsent {
|
||||
return "", nil, nil
|
||||
}
|
||||
tag = cmp.Or(img.ImageHash, fallback)
|
||||
if img.BlurHash != "" {
|
||||
blurs = map[string]map[string]string{"Primary": {tag: img.BlurHash}}
|
||||
if tag != "" {
|
||||
hash := img.BlurHash
|
||||
if hash == "" {
|
||||
hash = blurhash.Synthetic(tag, "")
|
||||
}
|
||||
blurs = map[string]map[string]string{"Primary": {tag: hash}}
|
||||
}
|
||||
if fields.Has("PrimaryImageAspectRatio") {
|
||||
ratio = img.AspectRatio()
|
||||
|
||||
@ -6,6 +6,7 @@ import (
|
||||
|
||||
"github.com/navidrome/navidrome/conf"
|
||||
"github.com/navidrome/navidrome/conf/configtest"
|
||||
"github.com/navidrome/navidrome/core/artwork/blurhash"
|
||||
"github.com/navidrome/navidrome/model"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
@ -37,7 +38,7 @@ var _ = Describe("mappers", func() {
|
||||
Expect(item.UserData.Key).To(Equal(EncodeID("song-1")))
|
||||
Expect(item.UserData.ItemId).To(Equal(EncodeID("song-1")))
|
||||
Expect(item.AlbumPrimaryImageTag).To(Equal("alb-1"))
|
||||
Expect(item.ImageBlurHashes).To(BeNil())
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(HaveKeyWithValue("alb-1", blurhash.Synthetic("alb-1", "")))
|
||||
Expect(item.Genres).To(Equal([]string{"genre 1", "genre 2"}))
|
||||
Expect(item.GenreItems).To(Equal([]NameGuidPair{{Id: EncodeID("1"), Name: "genre 1"}, {Id: EncodeID("2"), Name: "genre 2"}}))
|
||||
})
|
||||
@ -298,7 +299,7 @@ var _ = Describe("mappers", func() {
|
||||
Expect(*item.ProductionYear).To(Equal(1999))
|
||||
Expect(*item.ChildCount).To(Equal(10))
|
||||
Expect(item.ImageTags).To(HaveKeyWithValue("Primary", "alb-1"))
|
||||
Expect(item.ImageBlurHashes).To(BeNil())
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(HaveKeyWithValue("alb-1", blurhash.Synthetic("alb-1", "")))
|
||||
Expect(item.Genres).To(Equal([]string{"genre 1", "genre 2"}))
|
||||
Expect(item.GenreItems).To(Equal([]NameGuidPair{{Id: EncodeID("1"), Name: "genre 1"}, {Id: EncodeID("2"), Name: "genre 2"}}))
|
||||
})
|
||||
@ -472,7 +473,7 @@ var _ = Describe("mappers", func() {
|
||||
Expect(item.UserData.PlayCount).To(Equal(2))
|
||||
Expect(*item.UserData.Rating).To(Equal(8.0))
|
||||
Expect(item.ImageTags).To(HaveKeyWithValue("Primary", "pl-1"))
|
||||
Expect(item.ImageBlurHashes).To(BeNil())
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(HaveKeyWithValue("pl-1", blurhash.Synthetic("pl-1", "")))
|
||||
})
|
||||
|
||||
It("changes the playlist image tag when the cover content changes", func() {
|
||||
@ -507,13 +508,37 @@ var _ = Describe("mappers", func() {
|
||||
It("advertises the track id so the client triggers the read-through", func() {
|
||||
mf := model.MediaFile{ID: "mf-1", AlbumID: "alb-1", HasCoverArt: true}
|
||||
mf.AlbumImage.ImageHash = "0123456789abcdef"
|
||||
mf.AlbumImage.DominantColor = "#3a5f7d"
|
||||
|
||||
item := SongToBaseItem(mf, nil)
|
||||
Expect(item.ImageTags).To(HaveKeyWithValue("Primary", "mf-1"))
|
||||
Expect(item.ImageBlurHashes).To(BeNil(), "no resolved image means no blurhash to send")
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(
|
||||
HaveKeyWithValue("mf-1", blurhash.Synthetic("mf-1", "#3a5f7d")))
|
||||
Expect(item.AlbumPrimaryImageTag).To(BeEmpty())
|
||||
})
|
||||
|
||||
// The client's image request is what extracts the embedded art. Reusing the album's
|
||||
// value would let it answer from cache and never send it.
|
||||
It("gives the track a value distinct from its album's", func() {
|
||||
mf := model.MediaFile{ID: "mf-3", AlbumID: "alb-1", HasCoverArt: true}
|
||||
mf.AlbumImage.ImageHash = "0123456789abcdef"
|
||||
mf.AlbumImage.BlurHash = "LEHV6nWB2yk8"
|
||||
mf.AlbumImage.DominantColor = "#3a5f7d"
|
||||
|
||||
track := SongToBaseItem(mf, nil).ImageBlurHashes["Primary"]["mf-3"]
|
||||
album := AlbumToBaseItem(model.Album{ID: "alb-1", ItemImage: mf.AlbumImage}, nil).
|
||||
ImageBlurHashes["Primary"]["0123456789abcdef"]
|
||||
Expect(track).ToNot(BeEmpty())
|
||||
Expect(track).ToNot(Equal(album))
|
||||
})
|
||||
|
||||
It("still emits a value when the album has no dominant colour", func() {
|
||||
mf := model.MediaFile{ID: "mf-4", AlbumID: "alb-1", HasCoverArt: true}
|
||||
|
||||
item := SongToBaseItem(mf, nil)
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(HaveKeyWithValue("mf-4", blurhash.Synthetic("mf-4", "")))
|
||||
})
|
||||
|
||||
It("falls back to the album when the track has no art of its own", func() {
|
||||
mf := model.MediaFile{ID: "mf-2", AlbumID: "alb-1", HasCoverArt: false}
|
||||
mf.AlbumImage.ImageHash = "0123456789abcdef"
|
||||
@ -555,13 +580,14 @@ var _ = Describe("mappers", func() {
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(HaveKeyWithValue("0123456789abcdef", "LEHV6nWB2yk8"))
|
||||
})
|
||||
|
||||
It("omits the blurhash entirely when none was computed", func() {
|
||||
It("synthesizes a blurhash when none was computed", func() {
|
||||
al := model.Album{ID: "alb-2", Name: "Album"}
|
||||
al.ImageHash = "0123456789abcdef"
|
||||
|
||||
item := AlbumToBaseItem(al, nil)
|
||||
Expect(item.ImageTags).To(HaveKeyWithValue("Primary", "0123456789abcdef"))
|
||||
Expect(item.ImageBlurHashes).To(BeNil(), "a synthesized blurhash pins stale covers in Finamp")
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(
|
||||
HaveKeyWithValue("0123456789abcdef", blurhash.Synthetic("0123456789abcdef", "")))
|
||||
})
|
||||
|
||||
It("omits tags for known-absent artwork", func() {
|
||||
@ -573,10 +599,19 @@ var _ = Describe("mappers", func() {
|
||||
Expect(item.ImageBlurHashes).To(BeNil())
|
||||
})
|
||||
|
||||
It("keeps known-absent artwork free of any synthesized value", func() {
|
||||
al := model.Album{ID: "alb-5", Name: "Album"}
|
||||
al.ImageAbsent = true
|
||||
|
||||
item := AlbumToBaseItem(al, nil)
|
||||
Expect(item.ImageBlurHashes).To(BeNil(),
|
||||
"there is no image to key a cache on, and nothing will ever re-key it")
|
||||
})
|
||||
|
||||
It("falls back to the entity id while artwork is still unresolved", func() {
|
||||
item := AlbumToBaseItem(model.Album{ID: "alb-4", Name: "Album"}, nil)
|
||||
Expect(item.ImageTags).To(HaveKeyWithValue("Primary", "alb-4"))
|
||||
Expect(item.ImageBlurHashes).To(BeNil())
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(HaveKeyWithValue("alb-4", blurhash.Synthetic("alb-4", "")))
|
||||
})
|
||||
|
||||
It("versions an artist's tag by content hash", func() {
|
||||
@ -601,13 +636,14 @@ var _ = Describe("mappers", func() {
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(HaveKeyWithValue("0123456789abcdef", "LEHV6nWB2yk8"))
|
||||
})
|
||||
|
||||
It("never synthesizes a song blurhash when the album has none", func() {
|
||||
It("synthesizes a song's album blurhash when the album has none", func() {
|
||||
mf := model.MediaFile{ID: "song-2", Title: "Song", AlbumID: "alb-2"}
|
||||
mf.AlbumImage.ImageHash = "0123456789abcdef"
|
||||
|
||||
item := SongToBaseItem(mf, nil)
|
||||
Expect(item.AlbumPrimaryImageTag).To(Equal("0123456789abcdef"))
|
||||
Expect(item.ImageBlurHashes).To(BeNil())
|
||||
Expect(item.ImageBlurHashes["Primary"]).To(
|
||||
HaveKeyWithValue("0123456789abcdef", blurhash.Synthetic("0123456789abcdef", "")))
|
||||
})
|
||||
|
||||
It("omits a song's album tag when the album art is known absent", func() {
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user