Docs / API reference

POST/v1/search

Search

Discover ranked organic Google results, then optionally scrape every result page into LLM-ready content. Search runs asynchronously and preserves result rank across the full job lifecycle.

Execution model

Live request

Runtime depends on endpoint, target, pagination, rendering mode, and active plan limits.

Credit weight

Live catalog

Current weights are managed from the Data API Weights admin table and shown on pricing before use.

Search Endpoint

Search the web for ranked organic results. Return URLs and snippets, or scrape every result into LLM-ready content.

Getting Started

Send your API key as a bearer token. Search runs asynchronously, so start a job and then read its status.

Start a Search

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"best web scraping tools","num_results":10,"result_mode":"serp"}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"best web scraping tools\""
}

Check Search Status

Use the returned job_id until the status is completed, failed, or cancelled.

curl "https://api.datablue.dev/v1/search/JOB_ID" \
  -H "Authorization: Bearer YOUR_API_KEY"
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "completed",
  "query": "best web scraping tools",
  "total_results": 1,
  "completed_results": 1,
  "data": [
    {
      "url": "https://example.com/guide",
      "rank": 1,
      "title": "Web Scraping Guide",
      "snippet": "A practical guide to web scraping.",
      "success": true,
      "scraped": false
    }
  ]
}

Result Types

SERP Results

Set result_mode to serp to return ranked organic URLs, titles, and snippets without scraping the destination pages.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "latest Python releases",
    "num_results": 10,
    "result_mode": "serp",
    "language": "en",
    "country": "US",
    "safe_search": true
  }'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"latest Python releases\""
}

Scraped Results

Set result_mode to scrape to process every discovered URL with DataBlue's single-page Scrape engine. Scrape is the default mode and defaults to the markdown format.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "web scraping documentation",
    "num_results": 5,
    "result_mode": "scrape",
    "formats": ["markdown", "links", "images"],
    "only_main_content": true
  }'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"web scraping documentation\""
}

Choosing a result type.

NeedModeReturned data
Find and rank sourcesserpRank, URL, title, snippet
Build a research corpusscrapeSERP metadata plus requested page formats
Extract typed factsscrapePage content plus extract output

Search Options

Use these fields to control how many results Search returns and where those results come from.

Options at a Glance

OptionDefaultPurpose
queryrequiredGoogle organic search terms
num_results5Total unique organic results, from 1 through 50
result_modescrapeChoose metadata-only SERP results or scraped page content
languageenPreferred result language
countrynoneOptional two-letter geographic context
safe_searchfalseEnable filtered Google results

Result Count

Set num_results from 1 to 50. Search stops after collecting enough unique organic URLs and may return fewer when fewer valid results are available.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"web scraping tools","num_results":20,"result_mode":"serp"}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"web scraping tools\""
}

Language and Country

Set language for the preferred result language and country for two-letter geographic context, such as en with IN.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"best laptops","language":"en","country":"IN","result_mode":"serp"}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"best laptops\""
}

Safe Search

Set safe_search to true to request filtered Google results.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"family travel videos","safe_search":true,"result_mode":"serp"}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"family travel videos\""
}

Content Scraping

These options apply only when result_mode is scrape.

Output Formats

Request only the outputs you need. Supported formats are markdown, html, raw_html, links, images, headings, structured_data, and screenshot.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "web scraping documentation",
    "result_mode": "scrape",
    "formats": ["markdown", "links", "screenshot"],
    "format_options": {
      "markdown": {"mode": "reader"},
      "screenshot": {"capture": "full_page", "full_page": true}
    }
  }'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"web scraping documentation\""
}

Main Content Only

only_main_content defaults to true. Set it to false when navigation, footers, and surrounding page content are required.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"company documentation","result_mode":"scrape","only_main_content":false}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"company documentation\""
}

Mobile View

Set mobile to true to scrape mobile page variants. Add mobile_device when a specific device preset is needed.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"mobile shopping pages","result_mode":"scrape","mobile":true,"mobile_device":"iPhone 13"}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"mobile shopping pages\""
}

Headers and Cookies

Use headers and cookies for the destination pages scraped from the results. They do not change Google result discovery.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "product documentation",
    "result_mode": "scrape",
    "headers": {"Accept-Language": "en-IN"},
    "cookies": {"region": "IN"}
  }'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"product documentation\""
}

Structured Extraction

In Scrape mode, provide extract.prompt to transform each successfully scraped page into focused JSON. Add extract.schema to request a stable shape. The result is returned in each item's extract field.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Python web frameworks",
    "num_results": 5,
    "result_mode": "scrape",
    "formats": ["markdown"],
    "extract": {
      "prompt": "Extract the framework name and its main use case",
      "schema": {
        "type": "object",
        "properties": {
          "name": {"type": "string"},
          "use_case": {"type": "string"}
        },
        "required": ["name", "use_case"]
      }
    }
  }'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "completed",
  "query": "Python web frameworks",
  "total_results": 1,
  "completed_results": 1,
  "time_taken": 7.106,
  "data": [
    {
      "id": "18c8bb75-32dc-48e0-a2c1-d973e2a41a62",
      "url": "https://example.com/framework",
      "rank": 1,
      "title": "Example Framework",
      "success": true,
      "scraped": true,
      "extract": {
        "name": "Example Framework",
        "use_case": "Building web applications"
      }
    }
  ]
}

Response

Job and Result Fields

The status response contains the job lifecycle and the results collected so far. Fields that do not apply to the selected mode or requested formats are omitted.

FieldMeaning
statusCurrent job state
total_resultsUnique organic results accepted for the job
completed_resultsResult records completed so far
data[]Ranked result items in organic order
data[].successWhether this result completed without an item-level error
data[].scrapedWhether the destination page was processed in Scrape mode
data[].errorDestination-page failure for this result, when present
time_takenMeasured job processing time in seconds, when available
errorTop-level job failure, when present

Result items. Each data[] item contains id and url and can include rank, title, snippet, requested output fields, extract, metadata, and error.

Partial Failures

If one destination page cannot be scraped, its search result is retained with success: false, scraped: false, and an error. Other results continue processing.

Example
{
  "url": "https://example.com/unavailable",
  "rank": 3,
  "title": "Unavailable Result",
  "snippet": "The organic search snippet is preserved.",
  "success": false,
  "scraped": false,
  "error": "Page could not be scraped"
}

Webhooks

Set webhook_url to receive job.completed or job.failed. Add webhook_secret to sign deliveries with X-DataBlue-Signature. See the Webhooks guide for verification and retries.

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "web scraping documentation",
    "num_results": 10,
    "result_mode": "serp",
    "webhook_url": "https://your-app.example/webhooks/datablue",
    "webhook_secret": "YOUR_WEBHOOK_SECRET"
  }'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search started for \"web scraping documentation\""
}

Manage Jobs

Cancel a Search

Cancel a pending or running job with POST /v1/jobs/{job_id}/cancel. Cancelling an already-cancelled job returns success.

curl -X POST "https://api.datablue.dev/v1/jobs/JOB_ID/cancel" \
  -H "Authorization: Bearer YOUR_API_KEY"
Example response
{
  "success": true,
  "message": "Job cancelled",
  "job_id": "550e8400-e29b-41d4-a716-446655440001"
}

Retry a Failed Search

Retry a failed or cancelled job with POST /v1/jobs/{job_id}/retry. The retry creates a new job with the same configuration.

curl -X POST "https://api.datablue.dev/v1/jobs/JOB_ID/retry" \
  -H "Authorization: Bearer YOUR_API_KEY"
Example response
{
  "success": true,
  "message": "Job retried successfully",
  "original_job_id": "550e8400-e29b-41d4-a716-446655440001",
  "new_job_id": "550e8400-e29b-41d4-a716-446655440002",
  "type": "search"
}

Export Results

JSON, CSV, and ZIP. Download completed Search results from GET /v1/search/{job_id}/export?format=json. The supported export values are json, csv, and zip.

curl "https://api.datablue.dev/v1/search/JOB_ID/export?format=json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -o search-results.json
Example response
{
  "query": "latest Python releases",
  "results": [
    {
      "url": "https://www.python.org/downloads/",
      "title": "Download Python",
      "snippet": "Download the latest version of Python.",
      "success": true
    }
  ]
}

Change format to csv or zip and use the matching output filename to download those export types. ZIP exports include an index, full JSON data, and per-result folders for available content and metadata.

Retention. Raw Search results are available for 48 hours. Export before that window closes. After expiry, the status response returns an empty data array with result_expired: true, retention_hours, and expired_at. An expired export returns 410; a job with no exportable results returns 404.

Error Handling

SignalMeaningClient action
400Cancel or retry is invalid for the current job stateRead status before changing job state
401Missing or invalid bearer credentialReplace the API key; do not retry unchanged credentials
404Job is missing, belongs to another user, or has no exportable resultsVerify the job ID and authenticated account
422Request validation failedCorrect the query, result count, mode, or field types
429Quota or active-job concurrency limit reachedHonor Retry-After and avoid duplicate submissions
410Raw export data passed its retention windowRun a new Search job
status: failedGoogle result discovery or the job failedRead the top-level error; retry only when appropriate
data[].success: falseOne destination page failed in Scrape modeKeep successful items and inspect the item-level error

Search vs Google SERP API

Use Search for optional page content. Use Search when you need organic result discovery and may also need markdown, HTML, links, images, screenshots, or extraction from the destination pages.

Use Google SERP for search-page data. Use the dedicated Google SERP API when you need its richer Google search response contract instead of optional destination-page scraping.

Parameters

NameTypeRequirementDescription
querystringRequiredThe search query.
num_resultsnumberOptionalTotal organic results to return (1-50).
result_mode"serp" | "scrape"OptionalReturn ranked search metadata only, or scrape every discovered result page.
languagestringOptionalPreferred search-result language.
countrystringOptionalOptional two-letter country code for geographic result context.
safe_searchbooleanOptionalRequest filtered search results.
formatsstring[]OptionalScrape-mode outputs: markdown, html, raw_html, links, screenshot, structured_data, headings, and images. SERP mode omits page formats.
format_optionsobjectOptionalPer-format settings, such as markdown.mode or screenshot.capture/full_page.
only_main_contentbooleanOptionalExtract only the primary content from each result page.
mobilebooleanOptionalEmulate a mobile device viewport for scraping.
mobile_devicestringOptionalMobile device preset name.
headersobjectOptionalCustom HTTP headers sent while scraping result pages.
cookiesobjectOptionalCustom cookies sent while scraping result pages.
extractobjectOptionalLLM extraction config applied to each result: { prompt, schema, provider, model }.
webhook_urlstringOptionalWebhook URL for search completion notification.
webhook_secretstringOptionalHMAC secret for webhook signature verification.

Response fields

FieldTypeDescription
successbooleanWhether the API response itself was produced successfully.
job_idstringSearch job identifier used by status, export, cancel, and retry endpoints.
statusstringCurrent job status: started, pending, running, completed, failed, or cancelled.
querystringOriginal Search query.
total_resultsnumberNumber of unique organic results accepted for the job.
completed_resultsnumberNumber of result records completed so far.
time_takennumberMeasured job processing time in seconds, when available.
data[].idstringPersisted result identifier.
data[].ranknumberOne-based organic result rank.
data[].urlstringCanonical result URL.
data[].titlestringOrganic result title.
data[].snippetstringOrganic result snippet.
data[].successbooleanWhether this result completed without an item-level error.
data[].scrapedbooleanWhether the result page was processed in scrape mode.
data[].requested outputsobjectRequested page formats, extraction output, and page metadata added in Scrape mode.
data[].errorstringItem-level destination-page failure, when present.
errorstringTop-level job failure, when present.

Request and response

curl -X POST "https://api.datablue.dev/v1/search" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "query": "best python web scraping libraries 2026",
  "num_results": 10,
  "result_mode": "scrape",
  "formats": [
    "markdown"
  ]
}'
Example response
{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440001",
  "status": "started",
  "message": "Search job started"
}