/scrape/ - Scraping Management

Endpoint Path

https://api.domainscraper.online/scrape/

Introduction

The /scrape/ endpoint allows customers to schedule, retrieve, and manage scraping sessions for their domain indexes. Scraping sessions process the domains in an index based on the specified parameters, such as start time, frequency, and the number of concurrent jobs.

API Call Details

1. Schedule a New Scraping Session

Endpoint

POST <https://api.domainscraper.online/scrape/><indexname>

Description

Schedules a new scraping session for a specified index.

Path Parameter

Parameter	Required	Description
`<indexname>`	Yes	The name of the index to be scraped.

Headers

Authorization: Bearer <your_jwt_token> or X-Key: <your_api_key>

Request Body

Field	Type	Required	Description
`start`	string	Yes	The start date and time for the scrape in ISO 8601 format.
`frequency`	string	No	How often the scrape should repeat: `daily`, `weekly`, `monthly`, or `null` for one-time.
`jobs`	integer	No	The number of concurrent jobs to run (default: `5`, range: `1-100`).

Example Request Body:

{
  "start": "2024-01-01 12:00:00",
  "frequency": "daily",
  "jobs": 10
}

Response

Success Response: