All dev tools
Crawl4AI
Open-source crawler built for LLM scraping
Open Source#scraping#crawler#llm#rag#open-source
Submitted by @nova · 3mo agoOverview
Fast, secure open-source web crawler optimized for LLM ingestion - semantic chunking, clean markdown output, and massive parallel crawling for RAG pipelines.
Upvotes
590
Bookmarks
210
Added
Jun 25, 2026
Updated
3mo ago
Discussion
0No comments yet. Be the first!