Search Packages

is-crawler

Crawler detection from User-Agent strings in 50 ns. Issues and pull requests welcome!

mlscraper

🤖 Scrape data from HTML websites automatically by just providing examples

quick-crawler

A toolkit for quickly performing crawler functions

airspider

A Fast and Light Python Spider Framework 🕷️

crawlist

A universal solution for web crawling lists

sitecrawl

Simple Python3 module to crawl a website and extract URLs

sneakpeek-py

Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis

tezzcrawler

A web crawler that converts web pages to markdown and prepares them for LLM consumption

extracto-scraper

AI-powered web scraper. Extract structured data using plain English.

baikes

简单的百度百科爬虫

fcrawler

Python application that can be used to copy files of a given file type from a folder directory.

weibo-api-sdk

对微博m站API进行封装，免登陆获取微博数据