Visitar URL original
Drain FFmpeg video output before end of stream by anilbeesetti · Pull Request #211 · anilbeesetti/nextlib · GitHub
Skip to content

Drain FFmpeg video output before end of stream - #211

Open
anilbeesetti wants to merge 1 commit into
mainfrom
codex/drain-ffmpeg-video
Open

anilbeesetti wants to merge 1 commit into
mainfrom
codex/drain-ffmpeg-video

Conversation

@anilbeesetti

Copy link
Copy Markdown
Owner

Media3's SimpleDecoder publishes EOS without invoking the decoder and assumes one output per input. This can discard delayed H.264, HEVC, and MPEG-2 frames and classify reordered output against the wrong timestamp after a seek.

Implement the video Decoder contract directly with fixed input/output pools and one worker. Java EOS sends FFmpeg's null packet, and output EOS follows receive-EOF after every delayed frame. Retain unaccepted packets across send-EAGAIN, bound the VP9 hidden-frame retry exception, and use decoded timestamps for presentation and seek filtering. Flush invalidates in-flight work; release reclaims outstanding native frames. Existing output modes, AV1 low-delay behavior, codec initialization data, and audio behavior are preserved.

Add runnable Java/JNI regressions for delayed and multiple outputs, hidden VP9 frames, missing PTS, repeated seeks, flush/EOS races, callbacks, and frame ownership. YUV frames are checked against reference timestamps and pixel hashes. Only consumed test assets are checked in; generated playback files and machine-specific validation artifacts remain outside Git.

Validation

  • Built debug/release AARs for ARM64, ARMv7, x86, and x86_64 against Media3 1.11.0 / FFmpeg 9.0.1. All 11 library JVM tests and library lintDebug passed.
  • All 11 Android Java/JNI and renderer tests passed on a task-owned API 36 ARM64 emulator. Native VP9 backpressure/hidden-frame, reset, error-origin, color-conversion, and surface-failure checks passed.
  • An isolated Next Player checkout at 3cab1e90, using -PnextlibPath, passed assembleDebug, 205 JVM tests, and ktlintCheck, with test failures enforced. Builds also passed on JDK 17; the final Robolectric API 36 tests used the app's checked-in Java 21 runtime because Robolectric rejects Java 17.
  • After removing redundant assets, regenerated all 20 packet/reference assets byte-for-byte identically, rebuilt the instrumentation APK, and verified that it packages every required asset and none of the removed playback files.

Explicit FFmpeg playback in Next Player rendered all expected frames and reached STATE_ENDED:

Clip Expected / rendered Final timestamp (µs)
H.264 MP4 48 / 48 3916666
HEVC MP4 48 / 48 3916666
MPEG-2 Matroska 48 / 48 3917000
AV1 MP4 12 / 12 916666
VP9 alt-ref WebM 72 / 72 2958000

Every rendered timestamp matched host ffprobe within 1 µs. The installed APK's six JNI/FFmpeg libraries matched the local builds byte-for-byte; libmedia3ext.so build ID was 408d2e25e8451cff8e601a5905293436d4506d08.

Interactive H.264 checks also covered forward/backward seeks, FFmpeg → hardware → FFmpeg on the same surface, pause/resume near EOS, the labelled final frame, and replay. Those captures preceded the final missing-PTS rounding/callback adjustment; the final APK repeated the five-codec counts above and the final Java/JNI suite repeated the seek/ownership cases. The disposable emulator was stopped and removed; both Next Player checkouts were left clean.

Reproduction

With FFmpeg dependencies already built and ANDROID_SERIAL set to a disposable ARM64 emulator:

./gradlew assembleDebug assembleRelease test :media3ext:lintDebug -x ffmpegSetup
bash media3ext/src/androidTest/run_video_decoder_test.sh -x ffmpegSetup
bash media3ext/src/test/cpp/run_ffvideo_test.sh

Fixture generation and required SDK/NDK environment variables are documented with the test runners and fixtures.

Known limitations

  • Runtime coverage is ARM64/API 36; the other ABIs were built only.
  • Next Player lintDebug fails on 507 existing MissingTranslation errors in core/ui.
  • The existing AV1 native probe passed its 8-bit low-delay/flush cases. Its 10-bit film-grain case crashes with SIGILL, reproduced using unchanged 1df6554 decoder source and the same dependencies; its cause is unresolved.
  • An early hardware-only H.264 control run stalled and later errored; it is not counted as successful EOS validation. Android CLI layout-server setup failures were recovered before rerunning the affected checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant