Datasets and the free weather API

Overview#

Most of the API fetches a page when you ask. Datasets work the other way round: we collect the data on a schedule and keep every change, so you can read today’s values or the history without running a scraper. Each dataset also has a free page with a sample of its latest build.

Find a dataset and its columns#

GET /v1/datasets lists every dataset with its refresh interval, sources and row counts. GET /v1/datasets/{name} adds the column descriptions, the columns you can filter on and a sample. Both are free.

curl -H "x-api-key: $SCRAPERCOMPANY_API_KEY" \
  "https://api.scrapercompany.com/v1/datasets/weather"

Read the latest values#

GET /v1/datasets/{name}/latest returns the current value of every item, 1 credit per call. Filter on any column the dataset lists under filters with f.<column>=<value>, an exact match. Page with limit and offset. value_since is the day the values last changed.

curl -H "x-api-key: $SCRAPERCOMPANY_API_KEY" \
  "https://api.scrapercompany.com/v1/datasets/weather/latest?f.country=CA&limit=1"

{
  "dataset": "weather",
  "filters": { "country": "CA" },
  "rows": [
    {
      "item_id": "city/calgary",
      "name": "Calgary",
      "forecast_date": "2026-11-06",
      "tomorrow_high_c": 9.9,
      "tomorrow_low_c": -0.7,
      "tomorrow_condition": "partly_cloudy",
      "source": "met_norway",
      "value_since": "2026-11-05"
    }
  ]
}

Read one item’s history#

GET /v1/datasets/{name}/history takes an item_id from a latest row and returns every period its values held, 1 credit per call. History is change-only: a value is stored when it differs from the last one, with the first_seen and last_seen days, so a price that sits still for a month is one row. History starts on the day we began collecting that dataset.

curl -H "x-api-key: $SCRAPERCOMPANY_API_KEY" \
  "https://api.scrapercompany.com/v1/datasets/weather/history?item_id=city/toronto"

Download the whole file#

GET /v1/datasets/{name}/download returns a signed link, good for 15 minutes, to the dataset as a gzipped SQLite file, 500 credits per call. The link is tried before you are charged, so a link that does not answer costs nothing. The file has the tables items and observations and a view latest; open it with sqlite3, DuckDB, pandas or Datasette. Each file holds the whole history, and day asks for an older build while it is kept (the newest 7 days and the first of each of the last 12 months).

curl -o weather.sqlite.gz "$SIGNED_URL"
gunzip weather.sqlite.gz
sqlite3 weather.sqlite \
  "SELECT name, tomorrow_high_c FROM latest WHERE country = 'CA'"

The free weather API#

GET /v1/weather/forecast is a live forecast for any point on Earth: current conditions, 48 hours hourly by default and up to 10 days daily, with a simple condition code. It costs no credits, needs an API key, and takes lat and lon, with optional hours, days, units and tz.

curl -H "x-api-key: $SCRAPERCOMPANY_API_KEY" \
  "https://api.scrapercompany.com/v1/weather/forecast?lat=43.65&lon=-79.38&days=3"

Limits: your plan’s per-minute limit, 1,000 calls an hour per account and 200 an hour for points not recently asked for. Answers are cached until the source’s next update, and cached, fetched_at and expires_at say which you got.

The weather dataset is a different thing: it records tomorrow’s forecast for about 200 cities every day, so you can look back at what was forecast. Its values are forecasts, not station readings.

What it costs#

Failed requests are free. A dataset that is not built yet answers 503 with retry-after and costs nothing.