Add Video OCR: extract burned-in (hardcoded) subtitles#12270
Merged
Conversation
New "Video -> OCR burned-in subtitle..." feature (#10944), inspired by timminator's VideOCR pipeline: - ffmpeg samples frames from a user-selected scan area (draggable/resizable crop rectangle over a video frame preview, with bottom-third default and presets) at a configurable rate - consecutive near-identical frames are collapsed before OCR by comparing bright-pixel masks (Jaccard overlap), so each on-screen subtitle is OCR'ed once; frames with no bright pixels are skipped as blank - OCR engines: Paddle OCR (standalone/Python, reusing the existing batch engine via a new decode-free file input path), Ollama vision (e.g. glm-ocr), and a new GLM API engine (Z.ai / bigmodel.cn, OpenAI-compatible) - consecutive OCR results are merged into subtitle lines by Levenshtein text similarity with majority-vote text selection, max-gap bridging and minimum-duration filtering; vision-LLM hallucinations on textless frames are sanitized away - optional ASSA position tag ({\anX}) derived from the scan area position - shared Paddle install prompts extracted to PaddleOcrInstallHelper Verified end-to-end against a synthetic video with burned-in subtitles over a moving background (Ollama + glm-ocr): exact texts and timings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements video OCR of hardcoded/burned-in subtitles (#10944), inspired by VideOCR's pipeline.
New feature: Video → OCR burned-in subtitle...
![screenshot placeholder — drag & drop a screenshot here]
How it works
CropAreaSelectorcontrol: move, edge/corner resize, draw-new, dimmed outside area, rule-of-thumb presets for bottom third / bottom half / full frame). A slider scrubs the preview through the video. Default: bottom third.fps+crop+ optionalscaledown to 720 px).PaddleOcrbatch engine via a new decode-free file-input path (PaddleOcrBatchInput.SourceFileName), so the whole video is one batch: no per-image decode/re-encode, flat memory, one model load.glm-ocrmodel (reuses the existingOllamaOcrengine).{\anX}tag derived from the scan area position (e.g.{\an8}for a top scan area; bottom-center emits no tag).Lines appear live in the results grid during OCR; OK loads them into the main window (with ASSA formatting when the current format is Advanced Sub Station Alpha).
Also in this PR
PaddleOcrInstallHelper— the Paddle engine/models download prompts extracted to a shared helper (previously duplicated across view models), used by the new feature.SeVideoOcr, crop stored as percentages so it applies across resolutions), localization (LanguageVideoOcr), and docs (docs/features/video-ocr.md, F1 help).Verification
testsrc2background) through the real pipeline with Ollama +glm-ocr: output was exactly 3 lines with exact texts and frame-accurate timings (1–4 s, 5–8 s, 10–14 s).Closes #10944
🤖 Generated with Claude Code