Scraper Python Packages

firecrawl-py

🔥 The API to search, scrape, and interact with the web for AI

selectolax

Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.

newspaper3k

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

firecrawl

🔥 The API to search, scrape, and interact with the web for AI

google-play-scraper

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

spotifyscraper

Spotify Scraper to extract all the information from spotify, download mp3 with cover of the song

crawlee

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

rnet

An ergonomic Python HTTP Client with TLS fingerprint

spider-client

Python, Javascript, and Rust libraries for the Spider Cloud API.

scrapfly-sdk

Official Python SDK for the Scrapfly platform: web scraping, screenshots, AI extraction, crawling, and a remote anti-bot browser. Integrates with Scrapy, LlamaIndex, and LangChain.

twikit

Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot

zenrows

SDK to access ZenRows API directly from Python. We handle proxies rotation, headless browsers and CAPTCHAs for you.

snscrape

A social networking service scraper in Python

cinemagoer

Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people, characters and companies

twscrape

2025! X / Twitter API scrapper with authorization support. Allows you to scrape search results, User's profiles (followers/following), Tweets (favoriters/retweeters) and more.

perplexity-webui-scraper

An advanced, high-performance Python client, MCP server, and REST API for reverse-engineering Perplexity AI's WebUI.

scrapetube

A YouTube scraper for scraping channels, playlists, and searching 🔎

outscraper

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

tiktoklive

The definitive Python library to receive livestream events (comments, gifts, etc.) in realtime from TikTok LIVE.

wreq

An ergonomic Python HTTP Client with TLS fingerprint

imdbpy

Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people, characters and companies

is-crawler

Crawler detection from User-Agent strings in 50 ns. Issues and pull requests welcome!

misoreports

A comprehensive Python library for downloading Midcontinent Independent System Operator (MISO) public reports into pandas dataframes.

app-store-scraper

Single API ☝ App Store Review Scraper 🧹