Sandbox::https://help.decodo.com/docs/web-data-api-crawl-parameters

Visitar URL original

Parameters - Introduction

POST endpoint

https://data.decodo.com/v1/crawl

Input parameters

Parameter	Type	Required	Description
`url`	string	✅	The URL to start crawling from.
`limit`	integer		Maximum number of pages to scrape. Accepts `1` to `10000`.
`max_depth`	integer		Maximum number of link hops from the starting URL. Pages at this depth are scraped, but their links aren’t followed.
`sitemap`	string		Possible values: `include`, `only`, `skip`. `include` - collect URLs from both the sitemap and page links. `only` - crawl only URLs listed in the sitemap. `skip` - ignore the sitemap and follow page links only. Sitemap indexes are followed into their child sitemaps. With `only`, the starting URL itself is crawled only if the sitemap lists it. Default value is `include`.
`select_paths`	string or array		Regular expression, or a list of them. Only URLs whose path matches at least one pattern are crawled. The starting URL is crawled even if it doesn’t match. Paths are matched without a trailing slash, so use `/?$` rather than `/$` to match the end of a path. Example: `["^/blog"]`
`exclude_paths`	string or array		Regular expression, or a list of them. URLs whose path matches any pattern are skipped. Example: `["^/case-studies"]`
`domain_filter`	string		Which domains the crawler follows links to. Possible values: `domain`, `subdomain`, `all`. `domain` - only the starting URL’s host. `subdomain` - the host and its subdomains. Starting from `github.com`, links to `docs.github.com` are followed. `all` - links to any domain. Default value is `domain`.
`output`	string		Format of each page’s content. Possible values: `markdown`, `html`. Default value is `markdown`.
`max_duration`	integer		Maximum crawl time in seconds. When it’s reached, the crawl stops with status `truncated`, and the pages collected so far stay available.
`follow_redirects`	boolean		When `true`, redirects are followed. When `false`, a page that redirects fails. Default value is `true`.
`JS_retry`	boolean		When set to `true`, a page that needs JavaScript is retried once with JavaScript rendering. See Billing for how retried pages are charged.
`geo`	string		Country the pages are requested from. Accepts a full country name in any case (`Germany`, `germany`) or an uppercase 2-letter country code (`DE`). A lowercase code or an unrecognized country is ignored without an error, and pages are scraped from a random location.
`locale`	string		Language and region sent to websites in the `Accept-Language` header. Example: `en-US`, `de-DE`
`callback_url`	string		An `http(s)` URL that receives a `POST` request when the crawl finishes. See Webhooks.
`passthrough`	string		Any text you want returned unchanged in the webhook payload, for example to confirm the webhook came from Decodo.

Output

Starting a crawl returns 202 Accepted with its crawl_id and links to its status and results. Use them to check status, fetch results, or cancel.

{
  "target": "crawl",
  "url": "https://decodo.com/",
  "created_at": "2026-10-08 07:14:45",
  "crawl_id": "3b9ed512-74a8-4470-a76b-2494a12a70ed",
  "status": "pending",
  "crawl_status": "https://data.decodo.com/v1/crawl/3b9ed512-74a8-4470-a76b-2494a12a70ed/status",
  "results_tasks": "https://data.decodo.com/v1/crawl/3b9ed512-74a8-4470-a76b-2494a12a70ed",
  "results_content": "https://data.decodo.com/v1/crawl/3b9ed512-74a8-4470-a76b-2494a12a70ed?type=content"
}

If a parameter is invalid, the request returns 400 with an error message, for example {"error": "limit must be between 1 and 10000"}.