Crawlee

Web scraping and browser-automation library (Node and Python) built to extract data for AI, LLMs, and RAG.

Connects to: Playwright, Puppeteer · TypeScript · Apache-2.0 23,800★

Use it with an AI agent

Loadbay is an MCP server, so an agent can search the catalog and find this harness:

claude mcp add --transport http loadbay https://loadbay.xyz/api/mcp

• Source: https://github.com/apify/crawlee
• This harness as JSON: /api/harnesses/crawlee
• Agent setup: /setup.md
• Browse all 370+ harnesses on Loadbay