easieRnmt

The goal of easieRnmt is to provide a user-friendly R wrapper around the 'EasyNMT' python library, which provides “Easy to use, state-of-the-art Neural Machine Translation for 100+ languages” - on a local machine.

Installation

You can install the development version of easieRnmt from GitHub with:

# install.packages("pak")
pak::pak("thieled/easieRnmt")

From version 0.0.3 onwards, the package’s python backend that runs the 'EasyNMT' library is managed by 'uv' via 'reticulate'.¹ This function initializes this backend, automatically installs the correct pytorch version - supporting CUDA (Nvidia GPU) integration if this is available on your machine. It also works around the 'fasttext' dependency that 'EasyNMT' declares but never needs here: 'fasttext' ships no installable binaries for Windows (and only a source distribution for Linux), so the backend removes it from the environment entirely and detects languages in R instead.

easieRnmt::initialize_easynmt()

Note that the package requires a C++ compiler (e.g. g++). If you are a Windows user, please make sure to install a RTools version that matches your R version, from here.

Example

The package easieRnmt completely takes care of preprocessing your text data - from sentence tokenization, careful cleaning, emoji-replacement, language detection, and handling ambiguous cases.

To avoid depending on the fasttext python library, which has no installable binaries on most platforms, it uses the fastText R package for language detection.

It supports efficient batch-processing, and takes care that only language-homogeneous batches are processed – as the models assume that languages is consistent within batches.

Finally, it glues all translated sentences back together to the input format, sorts the translations as the input, and returns either a data.table (including the cleaned text and additional information) or the string only.

# Minimal example
sentences = c('Dies ist ein Satz in Deutsch. Und noch ein Satz.',   # This is a German sentence
              'Esta es una oración en español.', # This is a Spanish sentence
              "هذه جملة باللغة العربية!!!")       # This is an Arabic sentence

library(easieRnmt)

# Translate
res <- easieRnmt::translate(sentences,
                     model = 'opus-mt',
                     targ_lang = "en",
                     return_string = T)
#> Running fastText language detection...
#> Tokenizing texts after language detection...
#> Tokenizing texts into sentences and chunks...
#> Processing language: ar
#> Processing language: de
#> Processing language: es
# Print results
print(res)
#> [1] "This is a sentence in German. And another sentence."
#> [2] "This is a sentence in Spanish."                     
#> [3] "That's a sentence in Arabic!"

Footnotes

1. Thanks to JBGruber, who provided this python backend setup for my other package sentiner. ↩