# A web scraping API that deals with the blocks for you

> Send a URL, get the page back as HTML, markdown or text. When a site answers with a block, a rate limit or a bot check, the request is retried automatically, and you pay only for a page that comes back with content.

Canonical page: https://scrapercompany.com/web-scraping-api. Endpoints and prices checked against the API on 2026-10-26.

Most scraping projects stall on the same three problems: the site starts refusing requests, rate limits kick in as volume grows, and half the pages are built with JavaScript, so the HTML you fetch is empty. Solving them means running browsers, managing IP addresses and writing retry logic before you have parsed a single field. This API takes that work off your hands. One POST with a URL returns the page and the status the site answered with.

Pricing follows the work. A plain fetch is the cheapest call; render runs the page’s JavaScript first and returns the finished DOM; premium uses the premium network from the first attempt, for the sites that refuse standard requests. Whichever you pick, a block, an error page, an empty page or a timeout costs nothing, so a retry loop on your side never burns credits. For the sites we cover with a ready-made source (Amazon, Google, Booking.com, LinkedIn Jobs and more) use that instead: it returns typed fields rather than a page to parse.

## What you get

- `content`: the page as HTML, markdown or text, as you asked; JSON, XML and CSV come back as served
- `status_code`: the status the site answered with, so you can tell a missing page from a working one
- `final_url`: where the page ended up after redirects
- `headers.location`: every URL a redirect pointed at, in order
- `content_type, format`: the site’s Content-Type and the format of content
- `screenshot`: a base64 PNG, when you ask for one
- `meta`: whether render and premium were used, the time taken and the page size

## Endpoints and price per call

| Endpoint | Method and path | What it returns | Credits per successful call | Price per call |
| --- | --- | --- | --- | --- |
| [Scrape a URL](https://scrapercompany.com/docs/reference/scrape) | `POST /v1/scrape` | Any public page as HTML, markdown or text, with optional JavaScript rendering, screenshot, country and session | 1 credit | $0.0012 on Starter, $0.0010 on Growth, $0.0008 on Scale, $0.0007 on Business |

Pricing: free trial of 50 calls or 14 days, whichever ends first, no card needed; Starter $49/month for 40,000 credits; Growth $149/month for 150,000 credits; Scale $399/month for 500,000 credits; Business $999/month for 1,500,000 credits; credit packs $15 for 7,500, $60 for 35,000, $200 for 125,000 (valid 12 months). Failed requests (4xx or 5xx) and empty results cost 0 credits. https://scrapercompany.com/pricing

## Example

`POST /v1/scrape`, 1 credit per successful call.

```bash
curl -X POST "https://api.scrapercompany.com/v1/scrape" \
  -H "x-api-key: $SCRAPERCOMPANY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com"
  }'
```

Response (trimmed):

```json
{
  "url": "https://example.com/",
  "final_url": "https://example.com/",
  "status_code": 200,
  "content_type": "text/html; charset=UTF-8",
  "format": "markdown",
  "content": "# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\n[Learn more](https://iana.org/domains/example)",
  "headers": {
    "content-type": "text/html; charset=UTF-8",
    "last-modified": "Mon, 05 Oct 2026 18:43:12 GMT",
    "etag": "\"84238dfc8092e5d9c0dac8ef93371a07:1736799080.121134\"",
    "location": []
  },
  "meta": {
    "render": false,
    "premium": false,
    "elapsed_s": 0.412,
    "bytes": 1256
  }
}
```

## FAQ

### Do you solve CAPTCHAs?

The API detects block pages, rate limits and bot checks and retries the request until a real page comes back or the time budget runs out; for most sites that is what gets you past them. It does not type answers into a CAPTCHA you are shown. If every attempt is refused, the answer is a 503 that says so, and it costs nothing.

### What do I pay for?

Only a page that comes back with a 2xx status and content: 1 credit for a plain fetch, 5 with render, 10 with premium and 75 with both. A screenshot adds nothing. Blocks, error pages, empty pages, timeouts and refused URLs are free.

### When do I need render or premium?

Start with a plain fetch. Use render when the content you want is built by JavaScript after the page loads (the plain HTML is missing it). Use premium when a site keeps refusing plain requests: the 503 tells you it was blocked. Each answer’s meta says which were used, so you can keep the cheapest setting that works per site.

### Can I scrape pages behind a login?

You can pass your own Cookie or Authorization header, which goes only to the host you named, and keep one address across requests with session_id. Only scrape accounts and data you are allowed to access; the Terms of Service apply.

### Which URLs are refused?

Private, loopback and cloud-metadata addresses, ports other than 80, 443, 8080 and 8443, and hosts on a short denylist, including when a redirect or a rendered page points at one. Those answer 422 and cost nothing. Binary files such as images and PDFs are refused too.

### How fast is it, and what are the limits?

A plain fetch usually takes a second or two; a render takes longer. timeout_s is the budget for every attempt together, up to 60 seconds, or 90 with render. Requests per minute and concurrency follow your plan.

### Can AI agents use it?

Yes. format: markdown returns a page an LLM can read directly, and the same endpoint is a tool on the MCP server, so Claude and other MCP clients can fetch pages with your key.

### Do failed requests cost credits?

No. A failed or blocked call (any 4xx or 5xx response) costs 0 credits, and so does a search page that comes back empty. You pay only for successful results.

### How do I get an API key?

Create an account on the sign-up page and verify your email. You get a free trial of 50 calls or 14 days, whichever ends first, no card needed on the travel endpoints (hotels, rentals, flights, reviews); for everything else, and after the trial, buy a credit pack (from $15) or pick a plan. You create and rotate keys yourself in the dashboard.

## Read next

- [Web scraping API reference](https://scrapercompany.com/docs/reference/scrape)
- [Web scraping for AI and LLMs](https://scrapercompany.com/llm-web-scraping-api)
- [Price monitoring API](https://scrapercompany.com/price-monitoring-api)
- [Solutions by industry](https://scrapercompany.com/solutions)
- [All APIs](https://scrapercompany.com/apis.md)