Unlinking before re-creating fixes the premature-EOF spin on unix, but Windows
refuses to remove a file another handle still has open and returns a sharing
violation. Because Create surfaces that error, every cache miss on a path with a
live reader would have failed outright - worse than the spin it was meant to fix.
Split the create behind a build tag: unix unlinks for a fresh inode, Windows keeps
truncating in place and stays exposed to the spin, which is the behaviour it
already had. The unix-only spec is skipped there.
Create re-opened the path with O_TRUNC, which shrinks the file out from under an
older stream that may still be serving readers. stream.Reader then hits EOF from
the OS at the new, shorter length while the broadcaster still reports the original
size, so Wait() reports 'more data exists' and the reader retries forever - a tight
pread loop that burns a core and never releases its handle, which in turn blocks
Stream.Remove() indefinitely.
Unlink first and create with O_EXCL so the new entry gets a fresh inode. Existing
readers keep their descriptor on the old inode, see its full contents, and reach a
clean EOF.
Note this does not address the deferred unlink deleting the re-created file, which
is handled separately by the re-fetch in fileCache.Get.
* feat(cache): add completion marker helpers to spreadFS
* feat(cache): write completion marker after successful cache write
* fix(cache): adopt only complete files on reload, grandfather existing caches
* test(cache): regression tests for partial-transcode crash leftover (#5636)
* test(cache): guard concurrent in-progress streaming with completion marker
* test(cache): make concurrent-streaming guard actually attach a second reader mid-write
The previous test obtained s2 only after pw.Close(), so no reader ever
attached to the in-progress entry. Now pw.Write("hello ") is called
synchronously before the second Get — io.Pipe's blocking write gives a
deterministic happens-before — then both s1 and s2 are drained in parallel
goroutines while the producer writes the rest and closes the pipe.
* style(cache): clarify best-effort intent of cleanup os.Remove calls
* refactor(cache): lift one-time grandfather pass out of Reload's steady-state loop
* refactor(cache): have MarkComplete take the key, owning path mapping in spreadFS
* test(cache): assert no completion marker is written when the write fails
* refactor(cache): rename migration sentinel to generic .nd-migrated
* refactor(cache): rename grandfather migration to migrateExistingFiles
* refactor(cache): single-pass Reload with safer marker-error handling
Address PR review feedback:
- Merge the one-time migration into Reload's single directory walk,
avoiding a second full walk on first boot.
- Only delete a data file when its marker is definitively absent
(os.IsNotExist); skip on other stat errors to avoid destroying valid
entries under transient I/O failures.
- Write the migration sentinel only after a clean walk, so a partial
walk can't strand valid-but-unmarked files for later deletion.
- Return early from walkDataFiles on a WalkDir error.
- Assert fs.Create error in the marker-removal test.
