Search Packages

trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

news-please

news-please - an integrated web crawler and information extractor for news that just works

fundus

A very simple news crawler with a funny name

koreanewscrawler

A korean news crawler built to ingest large amounts of news data.

shutterstock-analysis

The internet's very first python package supports analyzing the Shutterstock public data, which helps creators optimize their creative portfolio and earn more income with less effort.

ur-gadget

A Python package that helps capture news updates from top Vietnamese news sites

newshound

This Python package can be used to systematically extract multiple data elements (e.g., title, keywords, text) from news sources around the world in over 50 languages.

webcorpus

Generate large textual corpora for almost any language by crawling the web

vnnews

A Python package that helps capture news updates from top Vietnamese news sites