Corpus Python Packages

speach

🐍🍑 Python 3 library for managing, annotating, and converting natural language corpuses using popular formats (CoNLL, ELAN, Praat, CSV, JSON, SQLite, VTT, Audacity, TTL, TIG, ISF, etc.)

chatterbot-corpus

A multilingual dialog corpus

fundus

A very simple news crawler with a funny name

ms3

A parser for annotated MuseScore 3 files.

german-nouns

A list of ~100,000 German nouns and their grammatical properties compiled from WiktionaryDE as CSV file. Plus a module to look up the data and parse compound words.

corpus-replicator

A corpus generation tool

korpora

Korean corpus repository

sadedegel

A General Purpose NLP library for Turkish

maalfrid-toolkit

Toolkit for the Målfrid project

neosca

L2SCA & LCA fork: cross-platform, GUI, without Java dependency

demeuk

Demeuk is a simple tool to clean up corpora (like dictionaries) or any dataset containing plain text strings.

expanda

The universal integrated corpus-building environment.

ekorpkit

eKorpkit provides a flexible interface for NLP and ML research pipelines such as extraction, transformation, tokenization, training, and visualization.

aovec

Make Word2Vec from aozorabunko/aozorabunko

opus-api

OPUS (opus.nlpl.eu) Python3 API

krwordrank

Korean Corpus Downloader

rnc

API for Russian National Corpus

npvcc2016

Python loader of npVCC2016 corpus

keywords

This is a simple library for extracting keywords from data with/without using a corpus.

pyclue

Python toolkit for Chinese Language Understanding(CLUE) Evaluation benchmark

textdirectory

TextDirectory allows you to filter, transform, and combine multiple text files into one aggregated file.

wpextract

Create datasets from WordPress sites

texttaglib

Python library for managing and annotating text corpuses in different formats (ELAN, TIG, TTL, et cetera)

ua-gec

UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language