Search Packages

crawlee

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

pydude

dude uncomplicated data extraction: A simple framework for writing web scrapers using Python decorators

microwler

A micro-framework for asynchronous deep crawls and web scraping with Python

parselcli

cli for evaluating css and xpath selectors

topscrape

Declarative, resilient, and typed web scraping. Define what you want — topscrape figures out how to get it, even when the site changes.