Extract API with Scroll Parameter

Automatically extract clean article text and other data from news articles, blog posts and other text-heavy pages.

byDiffbot Extract API (Web Extraction)

3192calls

744mslatency

Insufficient datasuccess rate

Discover -> Inspect -> Call

Call setup

Tool IDdiffbot.article.retrieve.v1

Request parametersJSON

Before this Call

1 credit / call

The final charge is settled from actual provider usage and will appear in Usage.

Field reference

urlstringRequired

Target URL to extract

fieldsstringOptional

Specify optional fields to be returned from any fully-extracted pages (e.g. `fields=querystring,links`)

timeoutintegerOptional

Sets a value in milliseconds to wait for the retrieval/fetch of content from the requested URL. The default timeout for the third-party response is 30 seconds (30000).

callbackstringOptional

Use for jsonp requests. Needed for cross-domain ajax.

proxystringOptional

Specify an IP address of a [custom proxy](https://docs.diffbot.com/reference/using-proxies#how-to-use-proxies) that will be used to fetch the target page. (Ex: `&proxy` or `&proxy=0.0.0.0`)

proxyAuthstringOptional

Used to specify the authentication parameters that will be used with a custom proxy specified in the &proxy parameter. (Ex: `proxyAuth=username:password`)

useProxystringOptional

Set to `default` to use [Diffbot's datacenter proxy](https://docs.diffbot.com/reference/using-proxies#how-to-use-proxies) for this request. `none` will instruct Extract to not use proxies, even if proxies have been enabled for this particular URL globally.

pagingbooleanOptional

Pass `paging=false` to disable automatic concatenation multiple-page articles.

maxTagsintegerOptional

Set the maximum number of automatically-generated tags to return. (Default: 10)

tagConfidencenumberOptional

Set the minimum relevance `score` of tags to return, between 0.0 and 1.0. By default only tags with a `score` equal to or above 0.5 will be returned.

categoryConfidencenumberOptional

Set the minimum relevance `score` of categories to return, between 0.0 and 1.0. By default only categories with a `score` equal to or above 0.5 will be returned.

discussionbooleanOptional

Pass `discussion=false` to disable automatic extraction of article comments.

naturalLanguagestringOptional

Run extracted text and title through the Diffbot Natural Language API. Example: `&naturalLanguage=entities,facts,categories,sentiment`.

summaryNumSentencesintegerOptional

Sets the maximum number of sentences for summary generation when using naturalLanguage=summary (Default: 3).

scrollstringOptional

Enables page scrolling to load lazy content; valid value is 'slow'.