Web Scraping MCP Server

<!-- mcp-name: com.hasdata/web-scraping -->

A hosted Model Context Protocol (MCP) server that gives Claude, Cursor, Windsurf and any other MCP client one read-only tool for fetching any public web page. It goes out through managed proxies, renders JavaScript when a page needs it, and returns clean markdown, plain text, raw HTML or structured JSON, with nothing to host and no browser in your stack.

This is the fallback for sites with no dedicated API. When a site does have one in the HasData catalogue, that tool returns parsed fields and this one returns a page.

1,000 free credits every month, no card required. A plain fetch costs 1 credit, so the free tier covers 1,000 of them.

https://mcp.hasdata.com/mcp?apis=web_scraping

• Prompts and resources

Contents

• What you need
• Quick start
• Example prompts
• Tools
• Output formats
• Errors and failure paths
• Pricing, free tier and limits
• How it compares
• FAQ
• HasData links
• Development
• Contributing
• License

What you need

An MCP client and a HasData API key from the dashboard, free to create with no card. This is a remote server, so the simplest path is a URL and an x-api-key header, with no container to run. A client that only speaks stdio reaches it through a thin launcher, published as @hasdata/web-scraping-mcp on npm and hasdata-web-scraping-mcp on PyPI, shown below.

Quick start

The server URL is the same for every client. We run it hands-on in Claude Code and Claude Desktop. The other blocks follow each client's own documented format for a remote server.

| Field | Value | | :--- | :--- | | URL | https://mcp.hasdata.com/mcp?apis=web_scraping | | Transport | HTTP, streamable | | Auth header | x-api-key: HASDATA_API_KEY |

Clients with OAuth support can add the same URL as a connector and sign in without putting a key in a config file.

<details> <summary><b>Claude Code</b></summary>

claude mcp add --transport http web-scraping "https://mcp.hasdata.com/mcp?apis=web_scraping" \
  --header "x-api-key: HASDATA_API_KEY"

</details> <details> <summary><b>Claude Desktop</b></summary>

Settings, then Connectors, then Add custom connector, then paste https://mcp.hasdata.com/mcp?apis=web_scraping and sign in.

For the config-file route, Claude Desktop loads only local (stdio) servers, so it reaches a remote server through a stdio launcher. The @hasdata/web-scraping-mcp package is that launcher, and it reads the key from the environment. Add this to claude_desktop_config.json:

{
  "mcpServers": {
    "web-scraping": {
      "command": "npx",
      "args": ["-y", "@hasdata/web-scraping-mcp"],
      "env": { "HASDATA_API_KEY": "YOUR_KEY" }
    }
  }
}

For Python instead of Node, swap the launcher for the PyPI package, which uvx runs without a manual install:

{
  "mcpServers": {
    "web-scraping": {
      "command": "uvx",
      "args": ["hasdata-web-scraping-mcp"],
      "env": { "HASDATA_API_KEY": "YOUR_KEY" }
    }
  }
}

</details> <details> <summary><b>Cursor</b></summary>

~/.cursor/mcp.json for every project, or .cursor/mcp.json for one:

{
  "mcpServers": {
    "web-scraping": {
      "url": "https://mcp.hasdata.com/mcp?apis=web_scraping",
      "headers": { "x-api-key": "HASDATA_API_KEY" }
    }
  }
}

</details> <details> <summary><b>Windsurf</b></summary>

~/.codeium/windsurf/mcp_config.json. Windsurf calls the field serverUrl, not url:

{
  "mcpServers": {
    "web-scraping": {
      "serverUrl": "https://mcp.hasdata.com/mcp?apis=web_scraping",
      "headers": { "x-api-key": "HASDATA_API_KEY" }
    }
  }
}

</details> <details> <summary><b>VS Code</b></summary>

.vscode/mcp.json in the workspace:

{
  "servers": {
    "web-scraping": {
      "type": "http",
      "url": "https://mcp.hasdata.com/mcp?apis=web_scraping",
      "headers": { "x-api-key": "HASDATA_API_KEY" }
    }
  }
}

</details>

Example prompts

• Fetch this page as markdown and summarise it.
• Read this documentation page and pull out every code block.
• Get the titles and links from the front page of this site as JSON.
• This page loads its content with JavaScript, so render it and wait for the results list before reading.
• Fetch this page through a German residential proxy and tell me whether the pricing differs.
• Take a screenshot of this page.

One call answers each of these. What changes between them is how much of the browser you asked for, and that is what the call costs.

Tools

| Tool | What it returns | | --- | --- | | hasdata_web_scraping_web_scraping_scrapeWebPage | HTML, text, markdown, and/or JSON along with status code, extracted emails and links, CSS-selector extractions, and AI-structured fields per schema. 1 credit a call for a plain fetch, 10 with JS rendering |

One tool. The cost depends on what you turn on, and the table is in Pricing.

Scrape web page

hasdata_web_scraping_web_scraping_scrapeWebPage

Fetch one URL.

| Parameter | Type | Required | Notes | | :--- | :--- | :--- | :--- | | url | string | yes | The page to fetch | | outputFormat | array | | Any of markdown, text, html, json. See Output formats | | jsRendering | boolean | | Render the page in a browser. On by default, and the main cost lever | | proxyType | string | | datacenter or residential | | proxyCountry | string | | US, UK, DE, IE, FR, IT, SE, BR, CA, JP, SG, IN or ID | | headers | object | | Custom request headers | | wait | number | | Milliseconds to wait after load | | waitFor | string | | CSS selector to wait for before reading | | jsScenario | array | | Actions to run on the page. See below | | extractRules | object | | CSS selectors to pull named fields | | aiExtractRules | object | | A typed schema an LLM fills from the page | | extractLinks | boolean | | Collect the page's links | | extractEmails | boolean | | Collect email addresses on the page | | screenshot | boolean | | Capture the rendered page | | blockResources | boolean | | Skip images and stylesheets | | blockAds | boolean | | Skip ad requests | | blockUrls | array | | Skip these URLs | | includeOnlyTags | array | | Keep only elements matching these selectors | | excludeTags | array | | Drop elements matching these selectors | | removeBase64Images | boolean | | Strip inline base64 images from the output |

extractRules maps a field name to a CSS selector, with @attr to read an attribute rather than text.

{ "title": "h1", "link_href": "a#link @href", "page_text": "body" }

jsScenario is an array of actions run in order, covering click, wait, waitFor, waitForAndClick, scrollX, scrollY, fill and evaluate for arbitrary JavaScript. It needs jsRendering on.

aiExtractRules describes the shape you want and lets a model fill it from the HTML. Each key is an output field, typed as string, number, boolean, list or item for a nested object.

Prompts and resources

The server ships one prompt, a ready-made workflow a client can offer instead of making the user compose a tool call.

| Prompt | What it does | | --- | --- | | scrape_web_page | Fetch any public web page and summarize its content. |

Alongside them the server exposes 2 resources, one per parameter whose accepted values are a fixed list. Reading one is cheaper than learning the vocabulary from a rejected call, and it costs no credits. Each URI is hasdata://web_scraping/<parameter>.

| Parameter | Values | What it selects | | --- | ---: | --- | | proxyType | 2 | Type of proxy to use. | | proxyCountry | 13 | Optional proxy country code. |

Both lists are served without an API key, so a client can read them before a user has signed up.

Output formats

This is the part worth reading before your first call, because the response shape moves with outputFormat.

Ask for exactly one of markdown, text or html, and the content arrives as a plain string in text at the top level.

{
  "url": "https://api.hasdata.com/scrape/web/",
  "status": 200,
  "json": null,
  "text": "# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\n[Learn more](https://iana.org/domains/example)\n"
}

Include json, alone or alongside another format, and everything moves inside json, the top-level text becomes null, and the requested formats become keys in there next to the page metadata.

{
  "json": {
    "requestMetadata": { "id": "7764031a-43f7-4102-8561-a7b7a6f1cbf5", "status": "ok" },
    "statusCode": 200,
    "statusText": "OK",
    "headers": { "server": "nginx", "content-type": "text/html; charset=utf-8" },
    "extractedData": { "title": "Hacker News", "firstStory": ["iPhone Duo", "apple.com", "Show HN: What if the speed of light was 5 km/h?"] }
  },
  "text": null
}

extractedData holds the extractRules results. A selector matching several elements returns all of them as an array, so .titleline a on a listing page returns every match rather than the first.

Errors and failure paths

Plan for these rather than assuming a happy path.

extractLinks and extractEmails do nothing unless outputFormat includes json. They put links and emails inside the json object, and there is nowhere for them to go in a plain markdown response. Asking for them with outputFormat: ["text"] returns the text and silently no links.

A 404 or a 403 on the target page is a successful call. The page's own status comes back as statusCode inside json, or as status at the top level, and the request is billed either way. Check the status before you parse the body.

jsRendering is on by default, and it is what the call costs. Turning it off takes a fetch from 10 credits to 1. Most static pages, documentation, articles and anything server-rendered do not need it. Turn it on when the content arrives empty without it.

waitFor beats wait. A fixed delay is a guess that is either too short on a slow load or wasted on a fast one. A CSS selector waits for the thing you actually need and returns as soon as it appears.

A residential proxy is five to fifteen times the price of a datacenter one.