2026-11-05 · America/Los_Angeles · 论文 · #13

VidDub: Replacement-Aware Distillation for Two-Step Video Dubbing

Video dubbing aims to synthesize lip movements that match target speech while preserving the speaker identity, head pose, and background of a source video. Existing editors are predominantly trained via self-reconstruction under original paired audio, even though the source mouth becomes a conflicting target when driving speech is replaced. We introduce replacement-aware cross-audio supervision: paired-audio samples use full-frame reconstruction, whereas unpaired replacement-audio samples mask out source-mouth reconstruction losses and supervise the mouth solely through audio-visual…

热度 39.3 / 100；排名与评分保留该期记录。