Docs / API reference

POST/v1/extract

Extract

Extract structured data from web pages or raw content using LLM. Accepts URLs to scrape first, or raw markdown/HTML content directly. Returns typed JSON matching your schema.

Execution model

Live request

Runtime depends on endpoint, target, pagination, rendering mode, and active plan limits.

Credit weight

Live catalog

Current weights are managed from the Data API Weights admin table and shown on pricing before use.

Extract Endpoint

Turn raw content, HTML, one URL, or multiple URLs into an answer or schema-shaped JSON.

Getting Started

Send your API key as a bearer token. Start with the smallest request below.

Extract from Content

Send content plus a prompt for an immediate natural-language answer.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "content": "The Acme Pro plan costs $49 per month and includes 10 seats.",
  "prompt": "What does the Pro plan cost and how many seats are included?"
}'
Example response
{
  "success": true,
  "data": {
    "answer": "The Pro plan costs $49 per month and includes 10 seats.",
    "content_length": 63
  },
  "job_id": "550e8400-e29b-41d4-a716-446655440010"
}

Input Modes

Raw HTML

Send html when you already have page markup and do not need DataBlue to fetch a URL.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "html": "<article><h1>Widget</h1><p>Price: $49</p></article>",
  "prompt": "Extract the product name and price"
}'
Example response
{
  "success": true,
  "data": {
    "answer": "Widget costs $49.",
    "content_length": 19
  },
  "job_id": "550e8400-e29b-41d4-a716-446655440011"
}

Single URL

Send url to scrape one page and extract from it synchronously.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://example.com/products/widget",
  "prompt": "Extract the product name and price"
}'
Example response
{
  "success": true,
  "data": {
    "url": "https://example.com/products/widget",
    "answer": "Widget costs $49.",
    "content_length": 1240
  },
  "job_id": "550e8400-e29b-41d4-a716-446655440012"
}

Multiple URLs

Send more than one URL to start an asynchronous extraction job.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "urls": [
    "https://example.com/products/widget",
    "https://example.com/products/gadget"
  ],
  "prompt": "Extract the product name and price"
}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440013",
  "status": "started",
  "message": "Extraction started for 2 URLs",
  "total_urls": 2
}

Check Extraction Status

Poll a multi-URL job until it completes, fails, or is cancelled.

curl -X GET "https://api.datablue.dev/v1/extract/JOB_ID" \
  -H "Authorization: Bearer YOUR_API_KEY"
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440013",
  "status": "completed",
  "total_urls": 2,
  "completed_urls": 2,
  "data": [
    {
      "url": "https://example.com/products/widget",
      "answer": "Widget costs $49.",
      "content_length": 1240
    },
    {
      "url": "https://example.com/products/gadget",
      "answer": "Gadget costs $79.",
      "content_length": 1180
    }
  ]
}

Extraction Options

Prompt and Schema

Use a prompt for an answer, or add schema to return validated structured fields.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "content": "Widget — $49 — In stock",
  "prompt": "Extract the product",
  "schema": {
    "type": "object",
    "properties": {
      "name": {
        "type": "string"
      },
      "price": {
        "type": "number"
      },
      "in_stock": {
        "type": "boolean"
      }
    },
    "required": [
      "name",
      "price",
      "in_stock"
    ]
  }
}'
Example response
{
  "success": true,
  "data": {
    "extract": {
      "name": "Widget",
      "price": 49,
      "in_stock": true
    },
    "content_length": 23
  },
  "job_id": "550e8400-e29b-41d4-a716-446655440014"
}

Provider and Model

Set provider and an optional model to use a configured BYOK provider.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "content": "Widget — $49",
  "prompt": "Extract the product and price",
  "provider": "openai",
  "model": "gpt-4.1-mini"
}'
Example response
{
  "success": true,
  "data": {
    "answer": "Widget costs $49.",
    "content_length": 12
  },
  "job_id": "550e8400-e29b-41d4-a716-446655440015"
}

URL Content Settings

For URL inputs, control main-content cleanup, wait time, timeout, headers, and cookies.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://example.com/account",
  "prompt": "Extract the account plan",
  "only_main_content": true,
  "wait_for": 1000,
  "timeout": 30000,
  "headers": {
    "Accept-Language": "en-US"
  },
  "cookies": {
    "session": "SESSION_VALUE"
  }
}'
Example response
{
  "success": true,
  "data": {
    "url": "https://example.com/account",
    "answer": "The account uses the Pro plan.",
    "content_length": 860
  },
  "job_id": "550e8400-e29b-41d4-a716-446655440016"
}

Completion

Webhook

Add a webhook to a multi-URL request to receive the completed or failed job event.

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "urls": [
    "https://example.com/products/widget",
    "https://example.com/products/gadget"
  ],
  "prompt": "Extract the product name and price",
  "webhook_url": "https://api.example.net/webhooks/datablue",
  "webhook_secret": "WEBHOOK_SECRET"
}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440013",
  "status": "started",
  "message": "Extraction started for 2 URLs",
  "total_urls": 2
}

Response

Prompt-only extraction returns data.answer. Schema extraction returns data.extract. Multi-URL requests return a job ID first.

Error Handling

Handle 401 for authentication, 422 for invalid input, 429 for limits, and retry only transient failures.

Extract vs Scrape

Use Extract for specific answers or typed fields. Use Scrape when you need the page content itself.

Parameters

NameTypeRequirementDescription
urlstringOptionalSingle URL to scrape then extract from.
urlsstring[]OptionalMultiple URLs to scrape and extract from (async job).
contentstringOptionalRaw markdown/text content to extract from (no scraping needed).
htmlstringOptionalRaw HTML to convert and extract from.
promptstringOptionalNatural language extraction instruction (e.g. 'Extract all product names and prices').
schemaobjectOptionalJSON Schema for structured output. The LLM will return data matching this schema.
providerstringOptionalLLM provider: "openai", "anthropic", "gemini", "groq", "openrouter", etc.
modelstringOptionalOptional model override for the selected provider.
only_main_contentbooleanOptionalExtract only main content before LLM processing.
wait_fornumberOptionalWait ms after page load (for URLs).
timeoutnumberOptionalScrape timeout in ms (for URLs).
headersobjectOptionalCustom HTTP headers.
cookiesobjectOptionalCustom cookies.
webhook_urlstringOptionalWebhook URL for extraction completion notification.
webhook_secretstringOptionalHMAC secret for webhook signature verification.

Response fields

FieldTypeDescription
data.extractobjectThe structured JSON matching your schema (or free-form when only a prompt is given).
data.time_takennumberElapsed processing time in seconds for this extraction result.

Request and response

curl -X POST "https://api.datablue.dev/v1/extract" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://openai.com/pricing",
  "prompt": "Extract all pricing tiers with name, price per million tokens, and context window",
  "schema": {
    "type": "object",
    "properties": {
      "tiers": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "name": {
              "type": "string"
            },
            "input_price": {
              "type": "string"
            },
            "output_price": {
              "type": "string"
            },
            "context_window": {
              "type": "string"
            }
          }
        }
      }
    }
  }
}'
Example response
{
  "success": true,
  "data": {
    "url": "https://openai.com/pricing",
    "extract": {
      "tiers": [
        {
          "name": "GPT-4o",
          "input_price": "$2.50/1M",
          "output_price": "$10.00/1M",
          "context_window": "128K"
        },
        {
          "name": "GPT-4o mini",
          "input_price": "$0.15/1M",
          "output_price": "$0.60/1M",
          "context_window": "128K"
        },
        {
          "name": "GPT-4.1",
          "input_price": "$2.00/1M",
          "output_price": "$8.00/1M",
          "context_window": "1M"
        }
      ]
    },
    "content_length": 48230,
    "time_taken": 2.39
  }
}