Crawling Python Python Packages

scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

wrapper-tls-requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

tibia-py

API to parse tibia.com content into python objects.

scrapy-proxy-headers

Handle custom proxy headers when making HTTPS requests through proxies in scrapy

endabyss

Recon Crawling Endpoint and Parameter Scan Tool

crawlipt

The script for selenium in python

proxy-randomizer

This library helps you sfetly crawle apis and web pages

facebook-page-info-scraper

Free Facebook pages MetaData Scraping Library - Unlimited Calls

crawlist

A universal solution for web crawling lists. 抓取网页列表的通用解决方案

kafka-scrapy-connect

Integrating Scrapy with kafka using the confluent-kafka python client

filecrawler

File Crawler index files and search hard-coded credentials.

webtranspose

Web scraping API for building AI applications.

pw-simple-scraper

simple web scraper using by playwright

mcqueen

A Collection Of Information-Gathering Tools 🌐

solidscraper

Easy to use JQuery-Like API for Web Scraping/Crawling.

musubi-scrape

Musubi: A convenient crawling tool for collecting web text data in Python.

iflow-mcp-d4vinci-scrapling

Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be!

newscrap

NewsCrap adalah alat scraping berita Google berbasis Command Line Interface (CLI) yang dirancang untuk riset, investigasi, dan pengumpulan data OSINT. Dengan fitur canggih seperti rotation proxy, scheduling otomatis, dan multi-format export, alat ini memudahkan pengumpulan data berita secara efisien dan andal.

scrapling-zhng

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!