Batch Mode Operations in ShoppingScraper API
Create, manage, and retrieve batch operations for large-scale data extraction across multiple e-commerce sites.
Creating a Batch
Submit a batch of queries to the ShoppingScraper API to process a large volume of EANs or URLs simultaneously. You can make use of multiple sites at the same time and will get the data returned in one CSV file.
POST https://api.shoppingscraper.com/batches?api_key=your_api_key_here
Payload:
{
"sites": ["shopping.google.nl", "shopping.google.be", "shopping.google.de", "shopping.google.fr", "bol.com"],
"eans": [
"8426265138470",
"8426265138487",
"8426265138548",
"8426265142071",
"8426265142088",
"8426265142477"
]
}
Batch Submission Response
Upon successful batch submission, the API returns the following payload:
{
"scheduler_id": null,
"batch_id": "unique_batch_id_here",
"total_query_no": 30,
"estimated_processing_time": 15.0,
"malformed_queries": []
}
Checking Batch Statuses
To check the status of all batches:
GET https://api.shoppingscraper.com/batches?api_key=your_api_key_here
To check the status of a specific batch:
GET https://api.shoppingscraper.com/batches/batch_id_here/status?api_key=your_api_key_here
Batch Status Payload
Example payload for a specific batch status request:
{
"status": "FINISHED",
"processing_time": 4.2616,
"queued_queries": 0,
"processed_queries": 80,
"failed_queries": 0,
"successful_queries": 80,
"not_found_queries": 0,
"malformed_queries": 0,
"written_to_db_queries": 80,
"written_to_json_queries": 80,
"written_to_csv_queries": 80,
"written_to_excel_queries": 0,
"queued_eans": 0,
"processed_eans": 80,
"failed_eans": 0,
"successful_eans": 80,
"not_found_eans": 0,
"malformed_eans": 0,
"written_to_db_eans": 80,
"written_to_json_eans": 80,
"written_to_csv_eans": 80,
"written_to_excel_eans": 0
}
Downloading Batch Results
To download the results of a batch in CSV format:
GET https://api.shoppingscraper.com/batches/batch_id_here/results?api_key=your_api_key_here&output_format=csv
Note: Replace your_api_key_here and batch_id_here with your actual API key and batch ID respectively.