Search Packages

spidermon

Scrapy Extension for monitoring spiders execution.

scrapy-proxy-headers

Handle custom proxy headers when making HTTPS requests through proxies in scrapy

crawlerutils

Utils for programming web crawler

cumtbscore

Student score interface for CUMTB

shub-cli

A CLI at your hands to deal with the features of ScrapingHub.

scrapy-dynamic-spiders

Dynamically generate spider subclasses based on existing spiders, including editing crawl rules and per-spider custom settings. Run crawls sequentially using crochet. Do both at the same time.