Scrapfly MCP Server

Key Features & Capabilities

👉 Live web data access 👉 Anti-bot handling & proxy infrastructure 👉 Structured data extraction 👉 Screenshots capture 👉 Secure Auth options

Works with your favorite AI tools: Claude Desktop, Cursor, Cline, Windsurf, LangChain, LlamaIndex, CrewAI, OpenAI function calling, n8n, Make, and Zapier.

Practical Use Cases

🕸️ Job aggregation 🕸️ Price monitoring 🕸️ Content research & aggregation 🕸️ Competitor intelligence & market analysis 🕸️ Real-estate analysis 🕸️ RAG & LLM data ingestion 🕸️ Automated multi-step workflows

Tools

instructions Purpose: Provide guidance and required parameters for successful scraping calls. Features: Supplies best-practice recommendations, parameter selection hints, error-handling guidance, and the mandatory pow value needed before calling web_get_page or web_scrape. Example usage:

{
  "tool": "web_get_page",
  "parameters": {
    "url": "https://news.ycombinator.com",
    "pow": "obtained_from_instruction_tool",
    "format": "markdown",
    "format_options": ["only_content"]
  }
}

web_scrape Purpose: Provide full-control, customizable scraping for complex sites and workflows. Features: Supports browser automation, login/auth flows, custom HTTP methods and headers, cookie management, multi-step interactions, and AI-powered data extraction. Example usage:

{
  "tool": "web_scrape",
  "parameters": {
    "url": "https://web-scraping.dev/login",
    "pow": "obtained_from_instruction_tool",
    "render_js": true,
    "js_scenario": [
      { "fill": { "selector": "input[name='username']", "value": "myuser" } },
      { "fill": { "selector": "input[name='password']", "value": "mypass" } },
      { "click": { "selector": "button[type='submit']" } },
      { "wait_for_navigation": { "timeout": 5000 } }
    ]
  }
}

screenshot Purpose: Capture visual snapshots of webpages. Features: Supports full-page screenshots or element-level captures using CSS selectors. Example usage:

{
  "tool": "screenshot",
  "parameters": {
    "url": "https://web-scraping.dev/pricing",
    "capture": ".pricing-table",
    "format": "png",
    "options": ["load_images", "block_banners"]
  }
}

info_account Purpose: Retrieve real-time details about your Scrapfly account and project. Features: Returns account metadata, project configuration, subscription status, credit usage, remaining quota, and concurrency limits. Example usage: { "tool": "info_account" }

Setup Guide

Claude Desktop

1. Locate Your Configuration File Claude Desktop stores its configuration in a JSON file. Open the file for your operating system: macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows: %APPDATA%\Claude\claude_desktop_config.json

2. Choose Authentication Method Select your preferred authentication method: OAuth2 (recommended): Add the following configuration to your claude_desktop_config.json file:

{
  "mcpServers": {
    "scrapfly": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.scrapfly.io/mcp"
      ]
    }
  }
}

API Key: Add the following configuration to your claude_desktop_config.json file:

{
  "mcpServers": {
    "scrapfly": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.scrapfly.io/mcp?key=YOUR-API-KEY"
      ]
    }
  }
}

3. Restart Claude Desktop After saving the configuration file, completely quit and restart Claude Desktop to apply the changes.

4. Verify the Integration After restarting, check that the MCP tools are available:

• Look for the hammer icon (🔨) in the bottom right corner of the chat window
• Click the hammer icon to see available MCP tools - you should see Scrapfly tools listed
• Try a test prompt: "Can you scrape https://news.ycombinator.com and show me the top 5 posts?"

Documentation

https://scrapfly.io/docs/mcp/getting-started