Language Identification Python Packages

lingua-language-detector

The most accurate natural language detection library for Python, suitable for short text and mixed-language text

fasttext-predict

fasttext with wheels and no external dependency, but only the predict method (<1MB)

py3langid

Faster, modernized fork of the language identification tool langid.py

simplemma

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

split-lang

✨ Split text by languages (e.g. 你喜欢看アニメ吗 -> 你喜欢看 | アニメ | 吗) for NLP tasks (e.g. parse, TTS). Powered by fasttext and budoux

eld

Fast and accurate natural language detection. Detector written in Python. Nito-ELD, ELD.

fastlangid

fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)

rosette-api

Babel Street Analytics Client Library for Python

glotscript

[LREC 2024] 🖋 Resource and Tool for Writing System Identification

langdetect-py

Language detection library ported from Google's language-detection.

fastspell

Targetted language identifier, based on FastText and Hunspell.

polars-whichlang

Language identification plugin for polars

fastspell-dictionaries

Targetted language identifier, based on FastText and Hunspell.

textpipe

textpipe: clean and extract metadata from text

lidbox

Spoken language identification out of the box with TensorFlow

seqtolang

Multi-Langauge Identification

whatlangid

This project is build on top of whatthelang and langid

pylade

PyLaDe - Language Detection tool.

codeswitch

CodeSwitch is a NLP tool, can use for language identification, pos tagging, name entity recognition, sentiment analysis of code mixed data.

afrolid

AfroLID, a powerful neural toolkit for African languages identification which covers 517 African languages.

geolid

Geographically-informed language identification