Search Packages

dingo-python

Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool

distributed-curator

Partition-aware MinHash LSH deduplication library for large-scale text data curation on Apache Spark.

ccdown

A rust based, resumable downloader cli and python library for Common Crawl data

gau-python

This library gets urls from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl. Inspired by Corbin Leo's gau

dingo-client

Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool