# Webhooks

Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/webhooks/

**Webhooks** allow you to receive real-time notifications when scraping operations complete, fail, or start. Instead of polling for results, configure a webhook endpoint to receive instant updates about your scraping jobs.

---

## Overview

Webhooks are HTTP callbacks that deliver event notifications to your configured URL. Use webhooks to:

- **Get notified when scrapes complete** — Receive results automatically
- **Handle failures gracefully** — React to errors in real-time
- **Integrate with chat platforms** — Send notifications to Discord or Slack
- **Build event-driven workflows** — Trigger downstream processes automatically

---

## Supported Platforms

| Platform | Description |
| :--- | :--- |
| **Discord** | Send rich embeds to Discord channels via webhook URLs |
| **Slack** | Post messages to Slack channels using incoming webhooks |
| **Custom** | Send JSON payloads to any HTTP endpoint with HMAC signature verification |

---

## Webhook Configuration

### Parameters

| Parameter | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| `url` | string | Yes | Your webhook endpoint URL |
| `type` | string | Yes | Webhook type: `"discord"`, `"slack"`, or `"custom"` |
| `events` | array | Yes | List of events to subscribe to |
| `secret` | string | No | Secret key for HMAC signature (custom webhooks only) |

### Available Events

| Event | Description |
| :--- | :--- |
| `started` | Job has been queued and started |
| `completed` | Job finished successfully |
| `failed` | Job encountered an error and failed |

---

## Usage Examples

### With Scrape Requests

```bash
curl -X POST "https://scrape.evomi.com/api/v1/scraper/realtime" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "mode": "browser",
    "webhook": {
      "url": "https://your-server.com/webhook",
      "type": "custom",
      "events": ["completed", "failed"],
      "secret": "your-secret-key"
    }
  }'
```

### With Crawl Requests

```bash
curl -X POST "https://scrape.evomi.com/api/v1/scraper/crawl" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "max_urls": 100,
    "webhook": {
      "url": "https://discord.com/api/webhooks/...",
      "type": "discord",
      "events": ["completed"]
    }
  }'
```

### With Map Requests

```bash
curl -X POST "https://scrape.evomi.com/api/v1/scraper/map" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "sources": ["sitemap", "commoncrawl"],
    "webhook": {
      "url": "https://hooks.slack.com/services/...",
      "type": "slack",
      "events": ["completed", "failed"]
    }
  }'
```

### With Search Requests

```bash
curl -X POST "https://scrape.evomi.com/api/v1/scraper/search" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "e-commerce platforms",
    "max_urls": 20,
    "webhook": {
      "url": "https://your-server.com/webhook",
      "type": "custom",
      "events": ["completed"]
    }
  }'
```

### With Schedules

```bash
curl -X POST "https://scrape.evomi.com/api/v1/account/schedule" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Daily Price Monitor",
    "config_id": "cfg_abc123",
    "interval_minutes": 1440,
    "webhook": {
      "url": "https://discord.com/api/webhooks/...",
      "type": "discord",
      "events": ["completed", "failed"]
    }
  }'
```

---

## Platform-Specific Integration

### Discord Webhooks

Discord webhooks post rich embeds to a channel. Get your webhook URL from channel settings:

1. Go to **Channel Settings** → **Integrations** → **Webhooks**
2. Click **New Webhook** and copy the URL

```bash
# Discord webhook example
"webhook": {
  "url": "https://discord.com/api/webhooks/123456789/abc123...",
  "type": "discord",
  "events": ["completed", "failed"]
}
```

Discord embeds include:
- Job status with color coding (green for success, red for failure)
- URL scraped
- Credits used
- Timestamp

### Slack Webhooks

Slack webhooks post messages using incoming webhook URLs. Create one in your Slack app:

1. Create a **Slack App** or use an existing one
2. Enable **Incoming Webhooks**
3. Create a new webhook URL

```bash
# Slack webhook example
"webhook": {
  "url": "https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXX",
  "type": "slack",
  "events": ["completed"]
}
```

Slack messages include:
- Status emoji (✅ or ❌)
- Job details in attachment format
- Link to view results

### Custom Webhooks

Custom webhooks send JSON payloads to any HTTP endpoint. Use the `secret` parameter for HMAC signature verification:

```bash
# Custom webhook with HMAC signature
"webhook": {
  "url": "https://your-server.com/api/webhooks/evomi",
  "type": "custom",
  "events": ["started", "completed", "failed"],
  "secret": "your-webhook-secret"
}
```

---

## Payload Structure

### Custom Webhook Payload

When using custom webhooks, Evomi sends a POST request with the following JSON structure:

```json
{
  "event": "completed",
  "timestamp": "2026-03-09T12:00:00Z",
  "data": {
    "task_id": "task_abc123",
    "task_type": "scrape",
    "url": "https://example.com",
    "status": "success",
    "credits_used": 5,
    "result": {
      "content": "...",
      "metadata": {}
    }
  }
}
```

### Payload Fields

| Field | Type | Description |
| :--- | :--- | :--- |
| `event` | string | Event type: `started`, `completed`, or `failed` |
| `timestamp` | string | ISO 8601 timestamp of the event |
| `data.task_id` | string | Unique identifier for the task |
| `data.task_type` | string | Type: `scrape`, `crawl`, `map`, or `search` |
| `data.url` | string | URL that was scraped (for scrape tasks) |
| `data.status` | string | `success` or `failed` |
| `data.credits_used` | number | Credits consumed by the operation |
| `data.result` | object | Scraping results (for completed events) |
| `data.error` | string | Error message (for failed events) |

---

## HMAC Signature Verification

For custom webhooks, verify the request authenticity using HMAC-SHA256 signatures.

### How It Works

1. Evomi creates a signature using your `secret` and the request body
2. The signature is sent in the `X-Evomi-Signature` header
3. Your server verifies the signature before processing

### Verification Example (Node.js)

```javascript
const crypto = require('crypto');

function verifyWebhook(req, secret) {
  const signature = req.headers['x-evomi-signature'];
  const payload = JSON.stringify(req.body);

  const expectedSignature = 'sha256=' + 
    crypto.createHmac('sha256', secret)
      .update(payload)
      .digest('hex');

  return signature === expectedSignature;
}

// Usage
app.post('/webhooks/evomi', (req, res) => {
  if (!verifyWebhook(req, 'your-webhook-secret')) {
    return res.status(401).send('Invalid signature');
  }

  // Process webhook
  console.log('Event:', req.body.event);
  res.status(200).send('OK');
});
```

### Verification Example (Python)

```python
import hmac
import hashlib
import json

def verify_webhook(headers, body, secret):
    signature = headers.get('X-Evomi-Signature', '')
    expected = 'sha256=' + hmac.new(
        secret.encode(),
        json.dumps(body).encode(),
        hashlib.sha256
    ).hexdigest()

    return hmac.compare_digest(signature, expected)

# Usage in Flask
@app.route('/webhooks/evomi', methods=['POST'])
def handle_webhook():
    if not verify_webhook(request.headers, request.json, 'your-webhook-secret'):
        return 'Invalid signature', 401

    # Process webhook
    event = request.json.get('event')
    print(f'Event: {event}')
    return 'OK', 200
```

---

## Best Practices

### 1. Respond Quickly

Return a `200 OK` response within 5 seconds. Long-running processes should be handled asynchronously:

```python
@app.route('/webhook', methods=['POST'])
def webhook():
    # Queue the work for background processing
    process_webhook_async.delay(request.json)
    return 'OK', 200
```

### 2. Implement Retry Logic

If your endpoint is temporarily unavailable, Evomi will retry:
- Up to 5 attempts
- Exponential backoff
- 24-hour retention

### 3. Verify Signatures

Always verify HMAC signatures for custom webhooks to ensure requests come from Evomi.

### 4. Handle Duplicate Events

Events may be delivered more than once. Use the `task_id` to deduplicate:

```python
processed_tasks = set()

def handle_webhook(payload):
    task_id = payload['data']['task_id']
    if task_id in processed_tasks:
        return  # Already processed
    processed_tasks.add(task_id)
    # Process the event
```

### 5. Use HTTPS

Webhook URLs must use HTTPS with a valid SSL certificate.

---

## Troubleshooting

### Webhooks Not Received

1. **Check firewall settings** — Ensure your server accepts connections from Evomi IPs
2. **Verify SSL certificate** — HTTPS is required with a valid certificate
3. **Review server logs** — Look for incoming requests from Evomi
4. **Test manually** — Use curl to test your endpoint

### Test Your Endpoint

```bash
curl -X POST "https://your-server.com/webhook" \
  -H "Content-Type: application/json" \
  -H "X-Evomi-Signature: sha256=..." \
  -d '{
    "event": "completed",
    "timestamp": "2026-03-09T12:00:00Z",
    "data": {
      "task_id": "test_123",
      "task_type": "scrape",
      "status": "success"
    }
  }'
```

---

## What's Next?

- **[Schedules](/scraping-products-instructions/scraper-api/schedules/)** — Create scheduled scraping jobs
- **[Scraper Configs](/scraping-products-instructions/scraper-api/scraper-config/)** — Save reusable configurations
- **[Usage Examples](/scraping-products-instructions/scraper-api/usage-examples/)** — See complete code examples