Scrape

Scrape a single URL and return normalized content in your requested formats.

Execution model

Live request

Runtime depends on endpoint, target, pagination, rendering mode, and active plan limits.

Credit weight

Live catalog

Current weights are managed from the Data API Weights admin table and shown on pricing before use.

Parameters

Name	Type	Requirement	Description
url	string	Required	The URL to scrape. Protocol is auto-prepended if missing.
formats	Array<string \| object>	Optional	Output formats. Strings remain supported. Object configs support markdown mode ("reader", "raw", "prune") and screenshot options: { "type": "screenshot", "capture": "viewport\|full_page\|after_actions", "full_page": true, "width": 1440, "height": 900 }.
only_main_content	boolean	Optional	Extract only the main content, removing navs, footers, sidebars.
mobile	boolean	Optional	Emulate a mobile device viewport.
mobile_device	string	Optional	Device preset name: "iphone_14", "pixel_7", "ipad_pro".
timeout	number	Optional	Request timeout in milliseconds.
wait_for	number	Optional	Wait this many ms after page load before extracting.
css_selector	string	Optional	Only extract content matching this CSS selector.
xpath	string	Optional	XPath expression for targeted extraction.
include_tags	string[]	Optional	Only include these HTML tags in extraction.
exclude_tags	string[]	Optional	Exclude these HTML tags from extraction.
headers	object	Optional	Custom HTTP headers to send (e.g. { "Cookie": "session=abc" }).
cookies	object	Optional	Custom cookies to send as name/value pairs.
actions	ActionStep[]	Optional	Browser actions to perform before extraction: click, wait, scroll, type, screenshot, hover, press, select, fill_form, evaluate.
extract	object	Optional	LLM extraction config: { prompt: string, schema: JSONSchema }.
webhook_url	string	Optional	Webhook URL for job completion notification.
webhook_secret	string	Optional	HMAC secret for webhook signature verification.
capture_network	boolean	Optional	Capture browser network requests/responses.

Response fields

Field	Type	Description
data.status	string	"success", "empty", "thin", or "blocked" — the outcome classification for this result.
data.quality	object	Content quality metrics: markdown_len, word_count, links_count, headings_count, images_count, has_title, and empty_reason (set when the result is empty/thin/blocked/skipped).
data.time_taken	number	Elapsed processing time in seconds for this page result.

Request and response

curl -X POST "https://api.datablue.dev/v1/scrape" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://news.ycombinator.com",
  "formats": [
    "markdown",
    "links"
  ],
  "only_main_content": true
}'

Example response

{
  "success": true,
  "data": {
    "markdown": "# Hacker News\n\n1. Show HN: I built an open-source web scraper with strategy caching\n2. Why Rust is eating the world...",
    "links": [
      "https://news.ycombinator.com/item?id=39912345",
      "https://news.ycombinator.com/item?id=39912346",
      "https://news.ycombinator.com/newest"
    ],
    "status": "success",
    "quality": {
      "markdown_len": 12340,
      "word_count": 1847,
      "links_count": 42,
      "headings_count": 8,
      "images_count": 3,
      "has_title": true,
      "empty_reason": null
    },
    "metadata": {
      "title": "Hacker News",
      "description": null,
      "language": "en",
      "source_url": "https://news.ycombinator.com",
      "status_code": 200,
      "word_count": 1847,
      "reading_time_seconds": 7,
      "content_length": 12340
    },
    "time_taken": 0.48
  }
}