Crawl4AI Web Scraper

**Judgment:** A solid, focused web scraping skill for OpenClaw when you control a local Crawl4AI instance, especially strong for JavaScript-heavy pages. It is best suited for users comfortable running local services and handling basic configuration. **What it does well:** - Uses a local Crawl4AI instance to render JavaScript and extract full page content. - Offers two modes: a simple proxy endpoint for clean `{page_content, metadata}` and a direct endpoint for rich JSON (`markdown, html, links, media, ...`). - Integrates via a simple script interface and environment variables, making it straightforward to slot into automated workflows. **Key risks / limitations:** - Hard dependency on a correctly running local Crawl4AI instance; if that service is down or misconfigured, the skill provides no value. - No visible documentation on error handling, retry logic, or timeouts; robustness under failure conditions is unclear. - Web scraping always carries legal/ToS and ethical risks; users must ensure target sites permit automated access and respect robots/usage policies. - Performance and stability depend entirely on the user’s local hardware and Crawl4AI configuration, which can vary widely. **Recommended scenarios:** - You need reliable scraping of dynamic or SPA-style pages for LLM ingestion or analysis. - You want to avoid third-party scraping APIs and keep data flows on-premise or within your own infrastructure. - Your pipeline sometimes needs just clean text/markdown, and other times full metadata (links, media, HTML) from the same pages. - You already run, or are willing to run, a local Crawl4AI instance and manage its performance and security yourself.