โ† Back to List
โš 
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
TypeScriptany4ai/AnyCrawl

AnyCrawl

AnyCrawl ๐Ÿš€: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.

87.7/100
โ˜… 2.8KForks: 288
View on GitHub โ†’Homepage โ†’
Loading report...

Similar Projects

firecrawl

92

๐Ÿ”ฅ The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

TypeScriptโ˜… 89.6K

dify

94

Production-ready platform for agentic workflow development.

TypeScriptโ˜… 131.6K

FastGPT

93

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.

TypeScriptโ˜… 27.3K

crawlee

93

Crawleeโ€”A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

TypeScriptโ˜… 22.1K
โ† Back to List