Nlp Python Packages

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

datasets

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

nltk

NLTK Source

thinc

🔮 A refreshing functional take on deep learning, compatible with your favorite libraries

spacy

💫 Industrial-strength Natural Language Processing (NLP) in Python

spacy-loggers

📟 Logging utilities for spaCy

htmldate

Fast and robust date extraction from web pages, with Python or on the command-line

trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

usaddress

:us: a python library for parsing unstructured United States address strings into address components

unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

gensim

Topic Modelling for Humans

presidio-analyzer

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

banks

LLM prompt language based on Jinja. Banks provides tools and functions to build prompts text and chat messages from generic blueprints. It allows attaching metadata to prompts to ease their management, and versioning is first-class citizen. Banks provides ways to store prompts on disk along with their metadata.

modelscope

ModelScope: bring the notion of Model-as-a-Service to life.

datefinder

Find dates inside text using Python and get back datetime objects

presidio-anonymizer

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

semchunk

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

sacremoses

Python port of Moses tokenizer, truecaser and normalizer

flashtext

Extract Keywords from sentence or Replace keywords in sentences.

textblob

Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.

lingua-language-detector

The most accurate natural language detection library for Python, suitable for short text and mixed-language text

openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

word2number

Convert number words (eg. twenty one) to numeric digits (21)