Data Curation Python Packages

fiftyone

Refine high-quality datasets and visual AI models

fiftyone-db

Refine high-quality datasets and visual AI models

cleanlab

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

fastdup

fastdup is a powerful, free tool designed to rapidly generate valuable insights from image and video datasets. It helps enhance the quality of both images and labels, while significantly reducing data operation costs, all with unmatched scalability.

cleanlab-studio

Client interface to Cleanlab Studio

fiftyone-db-ubuntu2204

Refine high-quality datasets and visual AI models

selfclean

[NeurIPS 2024] 🧼🔎 A holistic self-supervised data cleaning strategy to detect irrelevant samples, near duplicates and label errors.

fiftyone-desktop

FiftyOne Desktop

synrbl

Rebalancing chemical reaction

sliceguard

A library for detecting problematic data segments in structured and unstructured data with few lines of code.

distributed-curator

Partition-aware MinHash LSH deduplication library for large-scale text data curation on Apache Spark.

cubids

Curation of BIDS (CuBIDS): A sanity-preserving software package for processing BIDS datasets.

annotate-ez

High-throughput curation and visualization of large-scale single-cell microscopy images, in a lightweight GUI.

fiftyone-db-ubuntu2004

Refine high-quality datasets and visual AI models

cleanlab-cli

Client interface to Cleanlab Studio

ldcoolp-figshare

Python tool using the Figshare API for data curation

urlgenie

Python package to make URL extraction, generalization, validation, and filtration easy.

example-package-elisno

The standard package for data-centric AI, machine learning with label errors, and automatically finding and fixing dataset issues in Python.

docta-ai

Docta.ai

invisible-rabbit

Scalable data pre processing and curation toolkit for LLMs

fiftyone-db-debian9

FiftyOne DB

fiftyone-db-ubuntu1604

Project FiftyOne database

fiftyone-db-rhel7

Refine high-quality datasets and visual AI models

learn2clean

Learn2Clean: Optimizing the Sequence of Tasks for Data Preparation and Cleaning