Machine Translation Python Packages

ctranslate2

Fast inference engine for Transformer models

sacremoses

Python port of Moses tokenizer, truecaser and normalizer

spark-nlp

State of the Art Natural Language Processing

nemo-toolkit

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

unbabel-comet

A Neural Framework for MT Evaluation

argostranslate

Open-source offline translation library written in Python

pyonmttok

Fast and customizable text tokenization library with BPE and SentencePiece support

espnet

End-to-End Speech Processing Toolkit

subword-nmt

Unsupervised Word Segmentation for Neural Machine Translation and Text Generation

opennmt-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch

nlpcloud

NLP Cloud serves high performance pre-trained or custom models for NER, sentiment-analysis, classification, summarization, paraphrasing, intent classification, product description and ad generation, chatbot, grammar and spelling correction, keywords and keyphrases extraction, text generation, image generation, code generation, and more...

subset2evaluate

Find informative examples to efficiently (human)-evaluate NLG models.

argos-translate-lt

Open-source offline translation library written in Python

opennmt-tf

Neural machine translation and sequence learning using TensorFlow

dynabatch

PyTorch/Hugging Face batching utility that sorts variable-length text by difficulty, then dynamically increases batch size on easier samples using a pre-trained VRAM predictor to improve GPU utilization and throughput while reducing OOM risk with fallback handling.

tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

rusty-manga-image-translator

Translate manga/image.

mtdata

mtdata is a tool to download datasets for machine translation

sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

ruby

A tool for translating text from source grammar to target grammar (context-free) with corresponding dictionary.

pearmut

Platform for Evaluating and Reviewing of Multilingual Tasks

texar-pytorch

Integrating the Best of TF into PyTorch, for Machine Learning, Natural Language Processing, and Text Generation. This is part of the CASL project: http://casl-project.ai/

opustrainer

Curriculum training

opuscleaner

OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.