JS Automation

Scraper API

Parameters

JS Automation

Execute custom interactions on web pages before extracting content. Click buttons, fill forms, and wait for dynamic elements—all without managing browsers yourself.

JavaScript automation requires mode=browser or mode=auto. It’s automatically available when browser rendering is used.

Structured Actions Permalink to Structured Actions

Use js_instructions to define a sequence of actions to perform:

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": "#search-input"},
    {"fill": ["#search-input", "web scraping"]},
    {"click": "#search-button"},
    {"wait": 2000}
  ]
}

Supported Actions Permalink to Supported Actions

Click Permalink to Click

Click any element on the page.

Syntax: {"click": "selector"}

{"click": "#submit-button"}
{"click": ".cookie-accept"}
{"click": "button[type='submit']"}

Example:

curl -X POST "https://scrape.evomi.com/api/v1/scraper/realtime" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "mode": "browser",
    "js_instructions": [
      {"click": ".show-more-button"}
    ]
  }'

Fill Permalink to Fill

Fill form fields with text.

Syntax: {"fill": ["selector", "value"]}

{"fill": ["#email", "user@example.com"]}
{"fill": ["input[name='username']", "johndoe"]}
{"fill": ["#search-box", "product search term"]}

Example:

{
  "js_instructions": [
    {"fill": ["#username", "john@example.com"]},
    {"fill": ["#password", "secret123"]},
    {"click": "#login-button"}
  ]
}

Wait Permalink to Wait

Pause execution for a specified duration.

Syntax: {"wait": milliseconds}

{"wait": 1000}    // 1 second
{"wait": 2500}    // 2.5 seconds
{"wait": 5000}    // 5 seconds

Example:

{
  "js_instructions": [
    {"click": "#load-more"},
    {"wait": 3000},  // Wait for content to load
    {"click": "#load-more"}
  ]
}

Wait For Permalink to Wait For

Wait until an element appears on the page.

Syntax: {"wait_for": "selector"}

{"wait_for": ".results"}
{"wait_for": "#dynamic-content"}
{"wait_for": "div.loaded"}

Example:

{
  "js_instructions": [
    {"click": "#search-button"},
    {"wait_for": ".search-results"}  // Wait until results appear
  ]
}

CSS Selectors Permalink to CSS Selectors

Use standard CSS selectors to target elements:

Selector	Description	Example
`#id`	Element by ID	`#submit-btn`
`.class`	Elements by class	`.accept-cookies`
`tag`	By tag name	`button`, `input`
`[attribute]`	By attribute	`[name="email"]`, `[type="submit"]`
`parent > child`	Direct child	`.nav > a`
`ancestor descendant`	Descendant	`.container button`
`:pseudo`	Pseudo-selector	`:first-child`, `:last-of-type`

Common Patterns Permalink to Common Patterns

1. Search Form Interaction Permalink to 1. Search Form Interaction

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": "#search-input"},
    {"fill": ["#search-input", "web scraping"]},
    {"click": "#search-button"},
    {"wait": 2000},
    {"wait_for": ".search-results"}
  ]
}

2. Login Flow Permalink to 2. Login Flow

{
  "url": "https://example.com/login",
  "mode": "browser",
  "js_instructions": [
    {"fill": ["input[name='username']", "user@example.com"]},
    {"fill": ["input[name='password']", "password123"]},
    {"click": "button[type='submit']"},
    {"wait_for": ".dashboard"}
  ]
}

3. Accept Cookie Banner Permalink to 3. Accept Cookie Banner

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": ".cookie-banner"},
    {"click": ".accept-cookies"},
    {"wait": 500}
  ]
}

4. Load More Content Permalink to 4. Load More Content

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"click": ".load-more"},
    {"wait": 2000},
    {"wait_for": ".new-content"},
    {"click": ".load-more"},
    {"wait": 2000}
  ]
}

5. Navigate Menu Permalink to 5. Navigate Menu

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": "nav.main-menu"},
    {"click": "nav a[href='/products']"},
    {"wait": 1000},
    {"wait_for": ".product-grid"}
  ]
}

6. Multi-Step Form Permalink to 6. Multi-Step Form

{
  "url": "https://example.com/form",
  "mode": "browser",
  "js_instructions": [
    {"fill": ["#firstName", "John"]},
    {"fill": ["#lastName", "Doe"]},
    {"fill": ["#email", "john@example.com"]},
    {"fill": ["#phone", "555-1234"]},
    {"click": "#submit"},
    {"wait_for": ".success-message"}
  ]
}

Raw JavaScript Execution Permalink to Raw JavaScript Execution

For advanced use cases, execute custom JavaScript code with execute_js:

{
  "url": "https://example.com",
  "mode": "browser",
  "execute_js": "return document.title + ' - ' + window.location.href;"
}

The result is available in the js_result field:

{
  "success": true,
  "js_result": "Example Domain - https://example.com/"
}

Extract Data with JavaScript Permalink to Extract Data with JavaScript

{
  "execute_js": "return Array.from(document.querySelectorAll('.product')).map(p => ({name: p.querySelector('.name').textContent, price: p.querySelector('.price').textContent}));"
}

Response:

{
  "js_result": [
    {"name": "Product 1", "price": "$19.99"},
    {"name": "Product 2", "price": "$29.99"}
  ]
}

Modify Page Before Scraping Permalink to Modify Page Before Scraping

{
  "execute_js": "document.querySelectorAll('.ad').forEach(ad => ad.remove()); return 'Ads removed';"
}

Response Format Permalink to Response Format

Instructions Result Permalink to Instructions Result

When using js_instructions, the response includes execution details:

{
  "js_instructions_result": [
    {"action": "wait_for", "selector": "#button", "status": "success"},
    {"action": "click", "selector": "#button", "status": "success"},
    {"action": "wait", "duration": 2000, "status": "success"}
  ]
}

Execute JS Result Permalink to Execute JS Result

When using execute_js, the return value is in js_result:

{
  "js_result": "Return value from your JavaScript code"
}

Best Practices Permalink to Best Practices

1. Always Wait for Dynamic Elements Permalink to 1. Always Wait for Dynamic Elements

Bad:

[
  {"click": "#dynamic-button"}
]

Good:

[
  {"wait_for": "#dynamic-button"},
  {"click": "#dynamic-button"}
]

2. Add Delays After Actions Permalink to 2. Add Delays After Actions

Give pages time to respond to interactions:

[
  {"click": "#submit"},
  {"wait": 1000},        // Wait for response
  {"wait_for": ".result"}
]

3. Use Specific Selectors Permalink to 3. Use Specific Selectors

Bad (too generic):

{"click": "button"}

Good (specific):

{"click": "#submit-btn"}
{"click": "button.primary-action"}

4. Test Selectors First Permalink to 4. Test Selectors First

Before automating, test your selectors in the browser console:

document.querySelector("#your-selector")

5. Handle Errors Gracefully Permalink to 5. Handle Errors Gracefully

If an element isn’t found, the API will retry automatically. Use wait_for to ensure elements exist:

[
  {"wait_for": ".ajax-content"},  // Ensures element loads
  {"click": ".ajax-content button"}
]

6. Choose Appropriate Wait Times Permalink to 6. Choose Appropriate Wait Times

Scenario	Wait Time
Fast interactions	500-1000ms
API/AJAX calls	2000-3000ms
Page transitions	3000-5000ms
Heavy animations	2000ms+

7. Combine with Other Features Permalink to 7. Combine with Other Features

With Screenshots:

{
  "mode": "browser",
  "screenshot": true,
  "js_instructions": [
    {"wait_for": ".modal"},
    {"click": ".close-modal"}
  ]
}

With Resource Blocking:

{
  "mode": "browser",
  "block_resources": ["image", "font"],
  "js_instructions": [
    {"click": "#load-more"}
  ]
}

With AI Enhancement:

{
  "mode": "browser",
  "js_instructions": [
    {"click": ".show-full-article"}
  ],
  "ai_enhance": true,
  "ai_source": "markdown",
  "ai_prompt": "Summarize in 3 bullet points"
}

Limitations Permalink to Limitations

Maximum Actions Permalink to Maximum Actions

Keep instruction sequences reasonable (< 20 actions). Very long sequences may timeout.

Timeout Limits Permalink to Timeout Limits

Browser mode has a 45-second timeout. Ensure your instruction sequence completes within this time.

Element Visibility Permalink to Element Visibility

Elements must be visible on the page to be clicked. Hidden or off-screen elements may fail.

Dynamic Content Permalink to Dynamic Content

For heavily dynamic sites, increase wait_seconds at the page level:

{
  "mode": "browser",
  "wait_seconds": 5,
  "js_instructions": [...]
}

Troubleshooting Permalink to Troubleshooting

Element Not Found Permalink to Element Not Found

{"action": "click", "selector": "#button", "status": "failed", "error": "Element not found"}

Solutions:

Add {"wait_for": "#button"} before clicking
Check selector spelling in browser DevTools
Ensure element isn’t in an iframe
Use more specific selector

Click Not Working Permalink to Click Not Working

Possible causes:

Element is covered by another element
Element is outside viewport
Element requires specific wait condition

Solutions:

[
  {"wait_for": "#button"},
  {"wait": 500},  // Give page time to settle
  {"click": "#button"}
]

Form Fill Not Working Permalink to Form Fill Not Working

Solutions:

Ensure input field is visible and enabled
Wait for field to be ready:

[
  {"wait_for": "#email"},
  {"wait": 200},
  {"fill": ["#email", "user@example.com"]}
]

Timeout Errors Permalink to Timeout Errors

If instructions timeout, optimize your sequence:

Remove unnecessary waits
Use wait_for instead of fixed wait
Block resources to speed up page load
Consider splitting into multiple requests

JavaScript automation requires browser mode, which only works with residential proxies. Each automated request costs 5 credits (flat rate). Add +1 credit for screenshots and +30 credits for AI enhancement.

Scraping Modes Network Capture