crawling-tool

List of libraries, tools and APIs for web scraping and data processing.

SpideyX a multipurpose Web Penetration Testing tool with asynchronous concurrent performance with multiple mode and configurations.

📱 A utility for downloading Android apps from the Google Play Store and Xiaomi App Store (the Chinese App Store).

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

An declarative and easy to use web crawler and scraper in C#

Web crawling & scraping framework for Node.js on top of headless Chrome browser

🕷️ Easily scrap the web for torrent and media files.

AI Scraper : scrap and extract data from website in any format (CSV, JSON, HTML...) using Selenium or Crawl4ai, and using Ollama or Sambanova API, and using Streamlit for UI as chatbot

Firetastic - a tool for automating interaction with FireTV apps

🚀 THIS WEB SCRAPING TEMPLATE PROVIDES YOU WITH A GREAT STARTING POINT WHEN CREATING WEB SCRAPING BOTS. 🤖

List of best web crawlers to extract data from the web. Find web crawling tools for different needs.

This python script crawls course title, ratings, description and instructors from coursera.org

Web scraping API examples in Python, Node, Go, Java, PHP, Ruby, cURL and CLI. Proxies, JS rendering, AI extraction, 9 scrapers.

Google AI Mode scraper API in Python and Node using ScrapingBee. Pull the AI-generated answer, citations, and sources as JSON.

A web Crowler design , basic setup c++

A Website Spider/Crawler, Python 3.x

Recon Crawling Endpoint and Parameter Scan Tool

Google popular times API alternative: place opening hours, ratings and review counts from Google Maps via ScrapingBee.

SILKFANG - is a website recon engine and a powerful crawler. It crawls targets, mines JS/HTML/JSON for secrets, fingerprints the full stack, audits headers/TLS/CORS/cookies, hunts subdomain takeovers & dangling CNAMEs, and fires active probes.

Firetastic - a tool for automating interaction with FireTV apps