JavaScript Automation

Scraper API

Execute custom interactions on web pages before extracting content. Click buttons, fill forms, and wait for dynamic elements—all without managing browsers yourself.

ℹ️

JavaScript automation requires mode=browser or mode=auto. It’s automatically available when browser rendering is used.

Structured Actions

Use js_instructions to define a sequence of actions to perform:

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": "#search-input"},
    {"fill": ["#search-input", "web scraping"]},
    {"click": "#search-button"},
    {"wait": 2000}
  ]
}

Supported Actions

Click

Click any element on the page.

Syntax: {"click": "selector"}

{"click": "#submit-button"}
{"click": ".cookie-accept"}
{"click": "button[type='submit']"}

Example:

curl -X POST "https://scrape.evomi.com/api/v1/scraper/realtime" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "mode": "browser",
    "js_instructions": [
      {"click": ".show-more-button"}
    ]
  }'

Fill

Fill form fields with text.

Syntax: {"fill": ["selector", "value"]}

{"fill": ["#email", "[email protected]"]}
{"fill": ["input[name='username']", "johndoe"]}
{"fill": ["#search-box", "product search term"]}

Example:

{
  "js_instructions": [
    {"fill": ["#username", "[email protected]"]},
    {"fill": ["#password", "secret123"]},
    {"click": "#login-button"}
  ]
}

Wait

Pause execution for a specified duration.

Syntax: {"wait": milliseconds}

{"wait": 1000}    // 1 second
{"wait": 2500}    // 2.5 seconds
{"wait": 5000}    // 5 seconds

Example:

{
  "js_instructions": [
    {"click": "#load-more"},
    {"wait": 3000},  // Wait for content to load
    {"click": "#load-more"}
  ]
}

Wait For

Wait until an element appears on the page.

Syntax: {"wait_for": "selector"}

{"wait_for": ".results"}
{"wait_for": "#dynamic-content"}
{"wait_for": "div.loaded"}

Example:

{
  "js_instructions": [
    {"click": "#search-button"},
    {"wait_for": ".search-results"}  // Wait until results appear
  ]
}

CSS Selectors

Use standard CSS selectors to target elements:

Selector	Description	Example
`#id`	Element by ID	`#submit-btn`
`.class`	Elements by class	`.accept-cookies`
`tag`	By tag name	`button`, `input`
`[attribute]`	By attribute	`[name="email"]`, `[type="submit"]`
`parent > child`	Direct child	`.nav > a`
`ancestor descendant`	Descendant	`.container button`
`:pseudo`	Pseudo-selector	`:first-child`, `:last-of-type`

Common Patterns

1. Search Form Interaction

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": "#search-input"},
    {"fill": ["#search-input", "web scraping"]},
    {"click": "#search-button"},
    {"wait": 2000},
    {"wait_for": ".search-results"}
  ]
}

2. Login Flow

{
  "url": "https://example.com/login",
  "mode": "browser",
  "js_instructions": [
    {"fill": ["input[name='username']", "[email protected]"]},
    {"fill": ["input[name='password']", "password123"]},
    {"click": "button[type='submit']"},
    {"wait_for": ".dashboard"}
  ]
}

3. Accept Cookie Banner

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": ".cookie-banner"},
    {"click": ".accept-cookies"},
    {"wait": 500}
  ]
}

4. Load More Content

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"click": ".load-more"},
    {"wait": 2000},
    {"wait_for": ".new-content"},
    {"click": ".load-more"},
    {"wait": 2000}
  ]
}

5. Navigate Menu

{
  "url": "https://example.com",
  "mode": "browser",
  "js_instructions": [
    {"wait_for": "nav.main-menu"},
    {"click": "nav a[href='/products']"},
    {"wait": 1000},
    {"wait_for": ".product-grid"}
  ]
}

6. Multi-Step Form

{
  "url": "https://example.com/form",
  "mode": "browser",
  "js_instructions": [
    {"fill": ["#firstName", "John"]},
    {"fill": ["#lastName", "Doe"]},
    {"fill": ["#email", "[email protected]"]},
    {"fill": ["#phone", "555-1234"]},
    {"click": "#submit"},
    {"wait_for": ".success-message"}
  ]
}

Raw JavaScript Execution

For advanced use cases, execute custom JavaScript code with execute_js:

{
  "url": "https://example.com",
  "mode": "browser",
  "execute_js": "return document.title + ' - ' + window.location.href;"
}

The result is available in the js_result field:

{
  "success": true,
  "js_result": "Example Domain - https://example.com/"
}

Extract Data with JavaScript

{
  "execute_js": "return Array.from(document.querySelectorAll('.product')).map(p => ({name: p.querySelector('.name').textContent, price: p.querySelector('.price').textContent}));"
}

Response:

{
  "js_result": [
    {"name": "Product 1", "price": "$19.99"},
    {"name": "Product 2", "price": "$29.99"}
  ]
}

Modify Page Before Scraping

{
  "execute_js": "document.querySelectorAll('.ad').forEach(ad => ad.remove()); return 'Ads removed';"
}

Response Format

Instructions Result

When using js_instructions, the response includes execution details:

{
  "js_instructions_result": [
    {"action": "wait_for", "selector": "#button", "status": "success"},
    {"action": "click", "selector": "#button", "status": "success"},
    {"action": "wait", "duration": 2000, "status": "success"}
  ]
}

Execute JS Result

When using execute_js, the return value is in js_result:

{
  "js_result": "Return value from your JavaScript code"
}

Best Practices

1. Always Wait for Dynamic Elements

Bad:

[
  {"click": "#dynamic-button"}
]

Good:

[
  {"wait_for": "#dynamic-button"},
  {"click": "#dynamic-button"}
]

2. Add Delays After Actions

Give pages time to respond to interactions:

[
  {"click": "#submit"},
  {"wait": 1000},        // Wait for response
  {"wait_for": ".result"}
]

3. Use Specific Selectors

Bad (too generic):

{"click": "button"}

Good (specific):

{"click": "#submit-btn"}
{"click": "button.primary-action"}

4. Test Selectors First

Before automating, test your selectors in the browser console:

document.querySelector("#your-selector")

5. Handle Errors Gracefully

If an element isn’t found, the API will retry automatically. Use wait_for to ensure elements exist:

[
  {"wait_for": ".ajax-content"},  // Ensures element loads
  {"click": ".ajax-content button"}
]

6. Choose Appropriate Wait Times

Scenario	Wait Time
Fast interactions	500-1000ms
API/AJAX calls	2000-3000ms
Page transitions	3000-5000ms
Heavy animations	2000ms+

7. Combine with Other Features

With Screenshots:

{
  "mode": "browser",
  "screenshot": true,
  "js_instructions": [
    {"wait_for": ".modal"},
    {"click": ".close-modal"}
  ]
}

With Resource Blocking:

{
  "mode": "browser",
  "block_resources": ["image", "font"],
  "js_instructions": [
    {"click": "#load-more"}
  ]
}

With AI Enhancement:

{
  "mode": "browser",
  "js_instructions": [
    {"click": ".show-full-article"}
  ],
  "ai_enhance": true,
  "ai_source": "markdown",
  "ai_prompt": "Summarize in 3 bullet points"
}

Limitations

Maximum Actions

Keep instruction sequences reasonable (< 20 actions). Very long sequences may timeout.

Timeout Limits

Browser mode has a 45-second timeout. Ensure your instruction sequence completes within this time.

Element Visibility

Elements must be visible on the page to be clicked. Hidden or off-screen elements may fail.

Dynamic Content

For heavily dynamic sites, increase wait_seconds at the page level:

{
  "mode": "browser",
  "wait_seconds": 5,
  "js_instructions": [...]
}

Troubleshooting

Element Not Found

{"action": "click", "selector": "#button", "status": "failed", "error": "Element not found"}

Solutions:

Add {"wait_for": "#button"} before clicking
Check selector spelling in browser DevTools
Ensure element isn’t in an iframe
Use more specific selector

Click Not Working

Possible causes:

Element is covered by another element
Element is outside viewport
Element requires specific wait condition

Solutions:

[
  {"wait_for": "#button"},
  {"wait": 500},  // Give page time to settle
  {"click": "#button"}
]

Form Fill Not Working

Solutions:

Ensure input field is visible and enabled
Wait for field to be ready:

[
  {"wait_for": "#email"},
  {"wait": 200},
  {"fill": ["#email", "[email protected]"]}
]

Timeout Errors

If instructions timeout, optimize your sequence:

Remove unnecessary waits
Use wait_for instead of fixed wait
Block resources to speed up page load
Consider splitting into multiple requests

⚠️

JavaScript automation counts toward browser mode costs. Each automated request costs 5× the base rate (plus proxy multiplier).

Output Formats AI Enhancement