Lip Sync

Our model operates at the phoneme level — it analyzes the exact sound shapes in your audio track and maps them to the corresponding lip positions, jaw openings, and facial muscle configurations for each sound. This produces frame-accurate sync that maintains accuracy even for languages with phoneme structures very different from English, and for audio with varying speech rates and emphasis patterns. The model handles natural speech variation well — it produces accurate results with human-recorded voiceover, AI TTS, and recorded creator audio alike. Sync accuracy is highest when the audio is clear and free of background noise or competing music tracks.

Yes. This is a popular workflow — generate a video using an AI actor and AI voice, then replace the AI voice with a custom human-recorded voiceover and use Lip Sync to re-sync the actor's lips to the human voice. This gives you the visual quality of a BasedUGC-generated video with the authenticity of a real human voice — useful for brands that have an existing voice actor, a founder who records their own audio, or specific pronunciation requirements that the AI voice doesn't meet precisely. The custom voiceover can be recorded by anyone and uploaded in any standard audio format.

Yes. Lip Sync is the final step in the BasedUGC translation workflow. After using Translate Video to generate translated audio in any of 74 supported languages, Lip Sync re-processes the video's lip movements to match the new language's phoneme timing. This produces a video where the actor appears to speak the target language natively rather than having dubbed audio playing over mismatched lip movements from the original language. The combination of Translate Video and Lip Sync is the most complete localization workflow on the platform, producing fully native-looking multilingual content from a single original video at scale.

Input audio can be uploaded in MP3 (most common, works well for all use cases), WAV (lossless, highest quality — recommended for professional voiceover and studio recording), M4A (Apple audio format, fully supported), and AAC (compressed audio, same quality as MP3 at equivalent bitrates). Output video is exported as MP4 with the new audio embedded. The original video's audio is replaced completely by the uploaded audio track. For best results, upload audio that closely matches the duration of your video — if your audio is shorter, the remaining portion of the video will have no audio in the output.