pyugt - Python Universal Game Translator

pyugt is a universal game translator coded in Python: it takes screenshots from a region you select on your screen, uses OCR (via Tesseract v5) to extract the characters, then feeds them to a machine translator (multiple backends are included) to then show you a translated text.

Since it works directly on images, there is no need to hack the game or anything to access the text. It is also cross-platform (support for Windows and Linux and experimentally on MacOSX).

Here is a demo:

Several machine translation backends are available: from free online APIs such as Google Translate, to DeepL with a free or paid subscription, and also an offline translator that runs directly on your computer without needing internet access thanks to Argos Translate and OpenNMT.

Of course, since the translation is done by a machine, don't expect a very nice translation, but for games where no translation is available, it can be sufficient to understand the gist and be able to play.

The software can also be useful to human translators, as it is possible to enable logging of OCR'ed text in the config file, so that all captured text will be saved in a log file that can later be used as the source for a manual translation.

The software is also not limited to games, but can be applied to anything that displays text on screen.

This software was inspired by the amazing work of Seth Robinson on UGT (Universal Game Translator).

How to install & update

1. First, install Tesseract v5 (an open-source OCR engine), installers are provided by UB Mannheim. Make sure to install the additional languages you want to translate from (eg, Japanese, there is support for both horizontal and vertical Kanji). Alternatively, on most platform, Tesseract can be installed with the default package manager, eg on Debian/Ubuntu: apt-get install tesseract. On Windows, Chocolatey can be used: choco install --pre tesseract.

2. Then install pyugt:

   • Either on Windows OSes, there is a prepackaged binary you can download here (look for pyugt_vx.x.x_bin_win64.zip for 64-bit or win32 for 32-bit).

   • Either for other platform or if you want to run from sourcecode, you need to install a Python interpreter. Anaconda is a good one, and Miniconda3 is a smaller alternative that works too.

     Then, install this software:

     pip install --upgrade pyugt

     Or for developers who want to run it locally, after downloading the archive from Github, unzip it anywhere, cd in the folder and type:

     pip install py3make
     py3make installdev
     

     Note the software was tested on Windows 10 x64 with Python 3.10 (Anaconda) and 3.11. It was tested on Linux by other users but is not regularly tested.

Language packs for the Tesseract are downloadable directly from the installer. Language packs for the offline machine translator Argos are downloaded on-the-fly when required by the user, but can also be downloaded beforehand from this index, which also provides IPFS links that are future-proof, in case the on-the-fly downloads fail.

How to use

• First, you need to configure the config file config.ini. A sample config file is provided with the software that should work fine on Windows, but on other platforms or in some cases you may need to edit it, particularly to setup the path to the Tesseract binaries. The config file also allows you to change the hotkeys and the monitor to screenshot from, and a few other things such as the source and target languages (by default, the source is japanese and target is english).

• Then, you can launch the script from a terminal/console:

pyugt

or:

python -m pyugt

• Then, use the hotkey to select a region to capture from (default hotkey: CTRL+SHIFT+F3). The selected region does not need to be very precise, but need to contain the text to translate.

• Finally, use the hotkey to translate what is shown in the region (default: CTRL+F3). This will display a window with the original text and the translated text. Repeat as many times as you need, you don't need to reselect the region to translate again.

• Tip: if the software has difficulties in recognizing the characters (you get gibberish and non-letters characters instead of words), first try to redefine the region with CTRL+F2 and make sure the region includes all text with some margin but not too much of the background (the tighter around the text, the less the OCR will be confused by the background, this can help a lot!). You can use the region selection and translation hotkey to do both in a streamlined fashion (default: CTRL+F2).

• Tip2: Try to make the game screen bigger. The bigger the characters, the easier for the OCR to work.

• Tip3: You can specify the path to a config file by using the -c or --config argument: pyugt -c <path_to_config_file>

• Tip4: In the translation box, it's possible to manually edit the OCR'ed text and force a new translation by clicking on the "Translate again" button. This can be useful when the OCR has wrongly detected non-letters characters.

• Tip5: If you use blue light filtering softwares, disable them all before using the OCR, it will improve the contrast and hence the accuracy.

IMPORTANT NOTE: The software is still in alpha stage (and may forever stay in this state). It IS working, but sometimes the hotkeys glitch and they do not work anymore. If this happens, simply focus the Python console and hit CTRL+C to force quit the app, then launch it again. The selected region is saved in the config file, so you don't have to redo this step everytime.

Options

Here is a sample configuration file, with comments for the various additional options (such as logs for the OCR'ed text and translated text):

[USER]
# User parameters to configure the pyUGT program.
# Note that parameters can be modified on-the-fly while the app is running, and changes will be reflected in realtime. For example, translators can be changed on-the-fly.

# Path to Tesseract v5 binary. Easily install it from UB Mannheim installers: https://github.com/UB-Mannheim/tesseract/wiki
path_tesseract_bin = C:\Program Files\Tesseract-OCR\tesseract.exe
# Source language to translate from, for OCR. Both the Optical Recognition Character and the translator will search specifically for strings in this language, this reduces the amount of false positives (eg, translating strings in other languages that are more prominent or bigger on-screen). Language code can be found inside Tesseract tessdata folder (depends on what languages you chose in the installer).
lang_source_ocr = jpn
# Source language to translate from.
lang_source_trans = ja
# Target language to translate to. Must be a language code for the target machine translator: either Google Translate language code (NOT a Tesseract code! See: https://readthedocs.org/projects/py-googletrans/downloads/pdf/latest/ ) or DeepL code (eg, en for Google Translator, Argos and most others, or EN-US for DeepL).
lang_target = en
# Machine translator library to use. Can be online_free to use free online APIs but which can be throttled (eg, Google Translate, DeepL free, Baidu, etc) ; deepl to use DeepL API with your own authkey (not throttled but limited number of translations in free plan, then need to pay for more, but it's best in class japanese->english machine translator) ; offline_argos for offline translation using Argos based on OpenNMT, which produces less accurate translations but is free, unlimited and does not require an internet connection.
translator_lib = online_free
# If online_free is the selected translator_lib, we can specify here the translator service to use. For a list of available services, see: https://github.com/UlionTse/translators#more-about-translators
translator_lib_online_free_service = google
# If translator_lib is set to deepl, the API authorization key must be set here
translator_lib_deepl_authkey = fa14ef6c-d...
# Hotkey to set the region on screen to capture future screenshots from. The region does not need to be precise, but must contain the region where text is likely to be found.
hotkey_set_region_capture = ctrl+shift+F3
# Hotkey to translate from the selected region
hotkey_translate_region_capture = ctrl+F3
# Hotkey to set a region AND translate it directly on mouse click release. This is useful for games where the contrast between the text and background is bad (eg, transparent dialog box), so reselecting a tight region for each dialogue may yield better results, this is a faster way to do that with one shortcut instead of 2.
hotkey_set_and_translate_region_capture = ctrl+F2
# Hotkey to preview in a window the postprocessed screenshot that is fed to OCR, this helps with tweaking parameters here and see how it improve the text contrast
hotkey_show_ocr_preview = ctrl+p
# On which monitor the screen region capture should display? If you have only one screen, leave this to 0 (first monitor)
monitor = 0
# Save all OCR'ed text into a log file? Set a path or file name different than None to activate (exemple: log_ocr = log_ocr.txt). This can be very useful for human translators to gather game text data.
log_ocr = None
# Save all translated text into a log file? Set a path or file