Search Packages

removedup

Remove duplicates from parallel corpora

pyunique

Pyunique helps you get rid of duplicate files

deplicate

Advanced Duplicate File Finder for Python. Nothing is impossible to solve.

thebear

Bear - the decluttering deduplicator

imageduplicatefinder

Simple duplication finder for Images, matches on names and then compares image hashes.

yadupe

Recursively scan one or more given directories for duplicate files.

arrayhascher

Fast hash in 2D Arrays (Numpy/Pandas/lists/tuples)

screwduplicates

provides a simple and efficient way to remove duplicates from an iterable (even with unhashable elements, optional order preservation)

dropduplicatesplanb

Drops duplicates in DataFrames with tedious dtypes

a-pandas-ex-duplicates-to-df

Creates a DataFrame/Series from duplicates

deplicate-cli

Command Line Interface for deplicate.

drop-duplicates-nested-list

Drops duplicates from nested list

duplicateindexer

Find duplicates in multiple lists and return their indices and values.

listset

remove duplicates from lists

stridesduplicatefinder

Calculate overlapping values between two arrays and return the results as a DataFrame

dupe-eraser

A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.

searchdups

Search for duplicate files