Subaligner
Automatically synchronize and translate subtitles, or create new ones by transcribing, using pre-trained DNNs, Forced Alignments and Transformers. https://subaligner.readthedocs.io/
Install / Use
/learn @baxtree/SubalignerREADME
<div align="center">
<img src="https://raw.githubusercontent.com/baxtree/subaligner/master/figures/subaligner.png" alt="subaligner" width="300" />
</div>
Supported Formats
Subtitle: SubRip, TTML, WebVTT, (Advanced) SubStation Alpha, MicroDVD, MPL2, TMP, EBU STL, SAMI, SCC and SBV.
Video/Audio: MP4, WebM, Ogg, 3GP, FLV, MOV, Matroska, MPEG TS, WAV, MP3, AAC, FLAC, etc.
:information_source: <small style="line-height: 1.2;">Subaligner relies on file extensions as default hints to process a wide range of audiovisual or subtitle formats. It is recommended to use extensions widely acceppted by the community to ensure compatibility.</small>
Dependant package
Required by the basic installation: FFmpeg
<details> <summary>Install FFmpeg</summary> <pre><code>apt-get install ffmpeg</code></pre> <pre><code>brew install ffmpeg</code></pre> </details>Basic Installation
<details> <summary>Install from PyPI</summary> <pre><code>pip install -U pip && pip install -U setuptools wheel</code></pre> <pre><code>pip install subaligner</code></pre> </details> <details> <summary>Install from source</summary> <pre><code>git clone git@github.com:baxtree/subaligner.git && cd subaligner</code></pre> <pre><code>pip install -U pip && pip install -U setuptools</code></pre> <pre><code>pip install .</code></pre> </details> :information_source: <small style="line-height: 1.2;">It is highly recommended creating a virtual environment prior to installation.</small>Installation with Optional Packages Supporting Additional Features
<details> <summary>Install dependencies for enabling translation and transcription</summary> <pre><code>pip install 'subaligner[llm]'</code></pre> </details> <details> <summary>Install dependencies for enabling forced alignment</summary> <pre><code>pip install 'setuptools<65.0.0'</code></pre> <pre><code>pip install 'subaligner[stretch]'</code></pre> </details> <details> <summary>Install dependencies for setting up the development environment</summary> <pre><code>pip install 'setuptools<65.0.0'</code></pre> <pre><code>pip install 'subaligner[dev]'</code></pre> </details> <details> <summary>Install all extra dependencies</summary> <pre><code>pip install 'setuptools<65.0.0'</code></pre> <pre><code>pip install 'subaligner[harmony]'</code></pre> </details>Note that subaligner[stretch], subaligner[dev] and subaligner[harmony] require eSpeak to be pre-installed:
Container Support
If you prefer using a containerised environment over installing everything locally:
<details> <summary>Run subaligner with a container</summary> <pre><code>docker run -v `pwd`:`pwd` -w `pwd` -it baxtree/subaligner bash</code></pre> </details>For Windows users, you can use Windows Subsystem for Linux (WSL) to install Subaligner.
Alternatively, you can use Docker Desktop to pull and run the image.
Assuming your media assets are stored under d:\media, open built-in command prompt, PowerShell, or Windows Terminal:
Usage
<details> <summary>Single-stage alignment (high-level shift with lower latency)</summary> <pre><code>subaligner -m single -v video.mp4 -s subtitle.srt</code></pre> <pre><code>subaligner -m single -v https://example.com/video.mp4 -s https://example.com/subtitle.srt -o subtitle_aligned.srt</code></pre> </details> <details> <summary>Dual-stage alignment (low-level shift with higher latency)</summary> <pre><code>subaligner -m dual -v video.mp4 -s subtitle.srt</code></pre> <pre><code>subaligner -m dual -v https://example.com/video.mp4 -s https://example.com/subtitle.srt -o subtitle_aligned.srt</code></pre> </details> <details> <summary>Generate subtitles by transcribing audiovisual files</summary> <pre><code>subaligner -m transcribe -v video.mp4 -ml eng -mr whisper -mf small -o subtitle_aligned.srt</code></pre> <pre><code>subaligner -m transcribe -v video.mp4 -ml zho -mr whisper -mf medium -o subtitle_aligned.srt</code></pre> </details> <details> <summary>Pass in a global prompt for the entire audio transcription</summary> <pre><code>subaligner -m transcribe -v video.mp4 -ml eng -mr whisper -mf turbo -ip "your initial prompt" -o subtitle_aligned.srt</code></pre> </details> <details> <summary>Use the full subtitle content as a prompt</summary> <pre><code>subaligner -m transcribe -v video.mp4 -s subtitle.srt -ml eng -mr whisper -mf turbo -o subtitle_aligned.srt</code></pre> </details> <details> <summary>Use the previous subtitle segment as the prompt when transcribing the following segment</summary> <pre><code>subaligner -m transcribe -v video.mp4 -s subtitle.srt --use_prior_prompting -ml eng -mr whisper -mf turbo -o subtitle_aligned.srt</code></pre> </details>(For details on the prompt crafting for transcription, please refer to Whisper prompting guide.)
<details> <summary>Alignment on segmented plain texts (double newlines as the delimiter)</summary> <pre><code>subaligner -m script -v video.mp4 -s subtitle.txt -o subtitle_aligned.srt</code></pre> <pre><code>subaligner -m script -v https://example.com/video.mp4 -s https://example.com/subtitle.txt -o subtitle_aligned.srt</code></pre> </details> <details> <summary>Generate JSON raw subtitle with per-word timings</summary> <pre><code>subaligner -m transcribe -v video.mp4 -ml eng -mr whisper -mf turbo -ip "your initial prompt" --word_time_codes -o raw_subtitle.json</code></pre> <pre><code>subaligner -m script -v video.mp4 -s subtitle.txt --word_time_codes -o raw_subtitle.json</code></pre> </details> <details> <summary>Alignment on multiple subtitles against the single media file</summary> <pre><code>subaligner -m script -v video.mp4 -s subtitle_lang_1.txt -s subtitle_lang_2.txt</code></pre> <pre><code>subaligner -m script -v video.mp4 -s subtitle_lang_1.txt subtitle_lang_2.txt</code></pre> </details> <details> <summary>Alignment on embedded subtitles</summary> <pre><code>subaligner -m single -v video.mkv -s embedded:stream_index=0 -o subtitle_aligned.srt</code></pre> <pre><code>subaligner -m dual -v video.mkv -s embedded:stream_index=0 -o subtitle_aligned.srt</code></pre> </details> <details> <summary>Translative alignment with the ISO 639-3 language code pair (src,tgt)</summary> <pre><code>subaligner --languages</code></pre> <pre><code>subaligner -m single -v video.mp4 -s subtitle.srt -t src,tgt</code></pre> <pre><code>subaligner -m dual -v video.mp4 -s subtitle.srt -t src,tgt</code></pre> <pre><code>subaligner -m script -v video.mp4 -s subtitle.txt -o subtitle_aligned.srt -t src,tgt</code></pre> <pre><code>subaligner -m dual -v video.mp4 -s subtitle.srt -tr helsinki-nlp -o subtitle_aligned.srt -t src,tgt</code></pre> <pre><code>subaligner -m dual -v video.mp4 -s subtitle.srt -tr facebook-mbart -tf large -o subtitle_aligned.srt -t src,tgt</code></pre> <pre><code>subaligner -m dual -v video.mp4 -s subtitle.srt -tr facebook-m2m100 -tf small -o subtitle_aligned.srt -t src,tgt</code></pre> <pre><code>subaligner -m dual -v video.mp4 -s subtitle.srt -tr whisper -tf small -o subtitle_aligned.srt -t src,tgt</code></pre> </details> <details> <summary>Transcribe audiovisual files and generate translated subtitles</summary> <pre><code>subaligner -m transcribe -v video.mp4 -ml src -mr whisper -mf small -tr helsinki-nlp -o subtitle_aligned.srt -t src,tgt</code></pre> </details> <details> <summary>Shift subtitle manually by offset in seconds</summary> <pre><code>subaligner -m shift --subtitle_path subtitle.srt -os 5.5</code></pre> <pre><code>subaligner -m shift --subtitle_path subtitle.srt -os -5.5 -o subtitle_shifted.srt</code></pre> </details> <details> <summary>Run batch alignment against directories</summary> <pre><code>subaligner_batch -m single -vd videos/ -sd subtitles/ -od aligned_subtitles/</code></pre> <pre><code>subaligner_batch -m dual -vd videos/ -sd subtitles/ -od aligned_subtitles/</code></pre> <pre><code>subaligner_batch -m dual -vd videos/ -sd subtitles/ -od aligned_subtitles/ -of ttml</code></pre> </details> <details> <summary>Run alignments with pipx</summary> <pre><code>pipx run subaligner -m single -v video.mp4 -s subtitl