Text Processing Python Packages

pyparsing

Python library for creating PEG parsers

pymupdf

PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.

pymupdfb

PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.

nameparser

A simple Python module for parsing human names into their individual components

jaconv

Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku

pythainlp

Thai natural language processing in Python

html-to-markdown

High performance and CommonMark compliant HTML to Markdown converter. Maintained by the Kreuzberg team. Kreuzberg is a fast, polyglot document intelligence engine with a Rust core. It extracts structured data from 56+ document formats using streaming parsers and built-in OCR.

proces

🐨 text preprocess.

humanreadable

humanreadable is a Python library to convert human-readable values to other units.

wetextprocessing

Text Normalization & Inverse Text Normalization

matcher-py

A high-performance matcher designed to solve LOGICAL and TEXT VARIATIONS problems in word matching, implemented in Rust.

nostril-detector

Nostril: Nonsense String Evaluator

tfkit

🤖📇 handling multiple nlp task in one pipeline

twitter-text-parser

Twitter Text Libraries for Python

daachorse

🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure. (Python wrapper for daachorse)

rustling

A high-performance library for computational linguistics

nlpo3

Thai natural language processing library in Rust, with Python and Node bindings.

hazm

Persian NLP Toolkit

flashtext2

Flashtext implementation in Rust

deduce

Deduce: de-identification method for Dutch medical text

wordninja-enhanced

Probabilistically split concatenated words. Now with more functionality and languages!

funread

文档阅读和解析工具包 - 支持多种文档格式的读取和解析

zhtw

Taiwan Traditional Chinese quality tool for AI-generated content (CLI + 6-language SDK, 31K terms / 100M-char zero-mistranslation validation)

pynlpl

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).