PyPI Stats

Search Packages

Find Python packages by name, description, GitHub topic, or filter by metrics
apify
crawlee

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

538K 9K 712
omkarcloud
botasaurus

The All in One Framework to Build Undefeatable Scrapers

36K 4K 388
omkarcloud
bota

The All in One Framework to Build Undefeatable Scrapers

28K 4K 388
omkarcloud
javascript-fixes

The All in One Framework to Build Undefeatable Scrapers

27K 4K 388
omkarcloud
botasaurus-humancursor

The All in One Framework to Build Undefeatable Scrapers

26K 4K 388
omkarcloud
botasaurus-server

The All in One Framework to Build Undefeatable Scrapers

1K 4K 388
Changwanseo
genmine

GenBank Record downloader for taxonomists

981 8 0
INNOVINATI
microwler

A micro-framework for asynchronous deep crawls and web scraping with Python

535 13 1
omkarcloud
bose

The All in One Framework to Build Undefeatable Scrapers

499 4K 385
thesp0nge
nightcrawler-mitm

A python program that crawls a website and tries to stress it, polluting forms with bogus data

321 26 1
mike-gee
webtranspose

Web scraping API for building AI applications.

245 40 2
r0botsorg
agentsearchcli

Give any AI agent the ability to search, crawl, and extract the web.

227 2 0
alyakhtar
katastrophe

Download torrents from kat.ph directly through terminal

180 82 12
omkarcloud
pg-cache-storage

The All in One Framework to Build Undefeatable Scrapers

145 4K 388
heleusbrands
insite

A lightning fast tool for crawling websites and compiling PDFs of their pages

136 1 0
Thordata
thordata-firecrawl

Thordata Firecrawl – Firecrawl-compatible web crawling & scraping API built on Thordata, turning any website into AI-ready Markdown/JSON/HTML/screenshots.

128 2 0
michellepellon
pricetag

Extract price and currency information from unstructured text. Pure Python library for e-commerce data extraction and web scraping.

125 0 0
sadiuysal
iflow-mcp-sadiuysal-crawl4ai-mcp-server

A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents.

114 72 10
innovinati
scrapy-googlechat

Send crawl reports from Scrapy spiders to Google Chat

104 1 1
HuberTRoy
seen

Supported JavaScript Web crawling framework for everyone.

100 13 3
William-Fernandes252
astel

An asyncronous web crawling library for Python.

92 0 0
omkarcloud
sqlite-cache-storage

The All in One Framework to Build Undefeatable Scrapers

68 4K 388
ZeroCool940711
new-frontera

A scalable frontier for web crawlers

59 0 0