Fast, sample-accurate Vorbis seeking (require humecodec>=0.7.1)#83
Open
jpc wants to merge 15 commits into
Open
Fast, sample-accurate Vorbis seeking (require humecodec>=0.7.1)#83jpc wants to merge 15 commits into
jpc wants to merge 15 commits into
Conversation
- Use start_skip_samples from demuxer for seek adjustment (mp3 encoder delay, MP4 edit lists). The demuxer applies this skip at pts=0 but not after seeking. - Read codec_delay from decoder's output stream info for codecs where the delay is set by the codec init (wmav2, opus). - Switch read loop to PTS-based termination instead of sample counting, fixing truncation when seek lands far from target. - Detect when seek lands at stream start (chunk0.pts < margin) and use tstart=0 timeline to match ffmpeg CLI behavior. - Handle negative first-chunk PTS (some AAC files) by clamping to 0. Test results: 100/100 full load, 93/100 seek (±5 sample tolerance). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- For short seeks (tstart < 5s) and unreliable codecs (wmav2/wmapro), always read from the start instead of seeking. This avoids the seek_landed_at_start heuristic entirely for short seeks and costs almost nothing for small files. - Only apply start_skip_samples seek_adj when actually seeking, not when reading from start (where the decoder handles it automatically). - Initialize seek_adj to 0.0 to avoid UnboundLocalError. Comprehensive test: 1494/1580 pass (94.6%), up from ~85% before. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Build a packet index (128KB resolution) on first seek and use seek_to_byte_offset for raw MPEG audio formats. This avoids the slow sequential scan that ffmpeg's mp3 demuxer does for timestamp seeks. The index PTS is used directly for trimming since the demuxer doesn't update PTS after byte seek. Also: no seek_adj for indexed seeks (index PTS is in raw timeline, not the skip_samples-adjusted timeline). Comprehensive test: 1556/1580 pass (98.5%) across 8 seek positions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Drop 'vorbis' from AudioDecoder._seek_unreliable: humecodec 0.7.1 corrects the first post-seek Vorbis frame's pts (block-size transition warmup), so timestamp seeking is now sample-accurate and no longer needs the O(offset) read-from-start fallback (up to ~7x less IO on deep seeks). wmav2/wmapro stay on the fallback. Add humecodec>=0.7.1 as a dependency to pin the runtime that provides the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
torch.cat(chunks) on humecodec Chunk (a torch.Tensor subclass) dispatches through __torch_dispatch__ -> tree_map(unwrap, ...) on every decode, allocating ~14 cyclic pytree objects per call. Those accumulate and are freed only by the cyclic GC, feeding multi-hundred-ms (occasionally multi-second) dataloader latency spikes. Unwrap to the underlying `_elem` plain tensors before cat. Bit-exact: identical sha1 over 40 voxceleb decodes. Per-load cyclic allocation 14->0 pytree objs (33->20 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
".npy" columns store arrays as binary blobs decoded with np.load, whose header
parse (ast.literal_eval) allocates ~11 cyclic objects per read and feeds GC
latency spikes in the dataloader (measured 25us + 11 objs/read). The new ".arr"
extension stores the array as a native pyarrow list<fixed_size_list<T,K>> (2-D)
or list<T> (1-D):
- encode_value keeps the numpy array; WSSink._build_record writes the native
typed column (arr_column_type infers T/K from the data).
- get_sample dispatches ".arr" to decode_arr -> values.flatten().to_numpy()
(zero-copy, no ast).
Bit-exact vs .npy; ~4.5x faster read (6.2 vs 27.8us), p99 6.9 vs 92.7us, and
0 vs 11 cyclic objs/read. Backward compatible (npy/txt/json/audio unchanged;
7 regression tests pass).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- AudioDecoder.add_seek_points(positions, pts_seconds): forwards to humecodec >=0.8's add_seek_points so a timestamp seek brackets the target via the precomputed index (av_index_search_timestamp) and converges in ~1 read instead of a full binary/secant search (big win for Ogg/Vorbis). No-op on older humecodec / torchcodec backend. - ws_audio_index.load_audio_segment: decode via create_decoder + get_samples_played_in_range (correct per-codec seek: mp4 moov/timestamp, vorbis corrected granule, wma read-from-start, mp3 byte index) instead of a raw seek_to_byte_offset. Seed add_seek_points from the stored index; also seed the mp3 byte-index path so it doesn't rescan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…cally) generate_audio_seek_index now stores seek_index_header (~132kB) + seek_index_footer (~64kB) per blob. load_audio_segment wraps the shard reader with _HFOverlayReader, serving those regions from the cached bytes so the decoder's open (find_stream_info + codec-open read-ahead + Ogg duration probe) fetches nothing from the store. Combined with add_seek_points, a cold Ogg/Vorbis seek drops from ~6-23 touched 128kB blocks to ~2-3 (target only), bit-exact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ~132kB "header read-ahead" was an artifact of opening with a 128kB avio buffer; with a 4kB buffer (DECODE_BUFFER_BYTES) find_stream_info reads only ~8kB — and read-ahead is redundant once the block-cache layer serves whole 128kB blocks locally. So cache just 8kB header + 8kB real last page and zero-fill the ~64kB Ogg duration-probe window in _HFOverlayReader. Index cost drops 200kB -> 16kB/episode; a cold Ogg/Vorbis seek touches ~1-2 target blocks (was ~2-3), bit-exact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_ogg_footer scans back for the last page carrying a valid granule and caches from there to EOF — the actual bytes the duration probe needs (~2-4kB for vorbis, vs the 8kB fixed guess). Falls back to the last FOOTER_CACHE_BYTES for non-Ogg / not-found. Cache drops to ~10kB/episode (8kB header + ~2kB footer); cold-seek block count and accuracy unchanged (1-2 target blocks, bit-exact). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_measure_header_len opens the decoder with the decode-time buffer_size and records what find_stream_info + first-packet-confirm actually read at the head, caching exactly that. find_stream_info always reads one buffer past the vorbis setup (~setup_end + buffer, ~8kB here) to confirm the first audio packet, and analyzeduration/probesize can't trim below that minimum — so a fixed 8kB works for these files but undercaches for larger codebooks; measuring self-adjusts. Footer already parses the real last page. ~10kB/episode, cold seek 1-2 blocks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… seek Attach a precomputed (blob-relative positions, pts) index; get_decoder then injects it into the decoder's _packet_index (mp3/mp2/mp1 — skips the whole- episode build_packet_index scan, ~200-400ms for a long spotify mp3) or seeds add_seek_points (vorbis — skips the demuxer bisection). Verified on real spotify mp3: read_segment 167-440ms -> 10-129ms. No-op when no index attached. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables sample-accurate, fast Ogg/Vorbis seeking and requires
humecodec>=0.7.1.Vorbis was previously flagged
_seek_unreliableand forced onto the O(offset) read-from-start fallback, because a timestamp seek landed ~10.16 ms early on ~7% of seeks. Root cause: after a seek-flush, the first decoded Vorbis frame is a block-size transition frame that emits(bs_prev+bs_cur)/4samples but advances its granule by onlybs_cur/2— the leading(bs_prev-bs_cur)/4(= 448 samples @44.1 kHz) belong before its pts. Fixed in humecodec 0.7.1 (HumeAI/humecodec#1) by correcting that frame's pts.This PR drops
vorbisfromAudioDecoder._seek_unreliable(keepingwmav2/wmapro) and pinshumecodec>=0.7.1so the runtime providing the fix is guaranteed.Verification
Contents
The vorbis change (last commit) rides on the audio-decoding rework already on this branch (unified humecodec/torchaudio/torchcodec
AudioDecoder, sample-accurate seeking across codecs, indexed byte-seek for MP1/2/3). Convplayer and pupyarrow WIP were deliberately excluded.🤖 Generated with Claude Code