Integration Guides
Selenium

Selenium

This guide provides instructions on how to integrate Evomi’s proxies with Selenium, a popular web browser automation tool, using SeleniumWire for enhanced proxy management.

Prerequisites

Before you begin, ensure you have the following:

  1. Python installed on your system.
  2. Selenium and SeleniumWire installed in your project.
  3. Your Evomi proxy credentials (username and password).

Installation

If you haven’t already installed Selenium and SeleniumWire, you can do so using pip:

pip install selenium selenium-wire

You’ll also need to install a WebDriver for your preferred browser. For this guide, we’ll use ChromeDriver:

  1. Download ChromeDriver from the official website (https://chromedriver.chromium.org/downloads).
  2. Ensure the ChromeDriver executable is in your system’s PATH or specify its location in your script.

Configuration

To use Evomi proxies with SeleniumWire, you need to configure the WebDriver with the proxy settings. SeleniumWire makes it easy to handle proxies that require authentication.

Here’s how you can do it:

from seleniumwire import webdriver  # Import from seleniumwire
from selenium.webdriver.chrome.options import Options

# Proxy configuration
proxy_host = "rp.evomi.com"
proxy_port = "1000"  # Port should be a string for selenium-wire options
proxy_username = "your_username"
proxy_password = "your_password_session-anychars_mode-speed"

# SeleniumWire options
options = {
    'proxy': {
        'http': f'http://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}',
        'https': f'https://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}',
        'no_proxy': 'localhost,127.0.0.1'  # Add domains to bypass proxy if needed
    }
}

# Set up Chrome options (optional, e.g., for headless mode)
chrome_options = Options()
# chrome_options.add_argument('--headless') # Example: run in headless mode

# Create a new Chrome driver instance with SeleniumWire
driver = webdriver.Chrome(seleniumwire_options=options, options=chrome_options)

# Navigate to a page to test the proxy
driver.get("https://ip.evomi.com/s")
print(driver.page_source)

# Close the browser
driver.quit()

Replace your_username with your actual Evomi proxy username and your_password with your actual proxy password.

Explanation

Let’s break down the key parts of this script:

  1. We import webdriver from seleniumwire instead of selenium.
  2. We set up the proxy configuration with your Evomi proxy details: proxy_host, proxy_port, proxy_username, and proxy_password.
  3. We create a seleniumwire_options dictionary. This dictionary configures the HTTP and HTTPS proxies, including the authentication details (username and password) directly in the proxy URL.
    • The format for the proxy string is protocol://username:password@host:port.
    • no_proxy can be used to specify any hosts that should bypass the proxy.
  4. We initialize webdriver.Chrome, passing the seleniumwire_options to the seleniumwire_options parameter. You can also pass standard Selenium Options (like chrome_options for headless mode, etc.) to the options parameter.
  5. We navigate to https://ip.evomi.com/s to verify the proxy connection. This page displays the IP address seen by the server.
  6. We print the page source, which should show the IP address of the Evomi proxy you’re using.
  7. Finally, we close the browser using driver.quit().

Evomi Proxy Endpoints

Depending on the Evomi product and protocol you’re using, you’ll need to adjust the proxy_host and proxy_port in your code. Ensure the protocol in the options dictionary (http/https) also matches your needs.

Residential Proxies

  • HTTP Proxy: rp.evomi.com:1000
  • HTTPs Proxy: rp.evomi.com:1001 (Use this for HTTPS traffic if specifying separate proxy types, or ensure your main proxy supports HTTPS pass-through)
  • SOCKS5 Proxy: rp.evomi.com:1002 (SeleniumWire also supports SOCKS proxies, the format would be socks5://user:pass@host:port)

Mobile Proxies

  • HTTP Proxy: mp.evomi.com:3000
  • HTTPs Proxy: mp.evomi.com:3001
  • SOCKS5 Proxy: mp.evomi.com:3002

Datacenter Proxies

  • HTTP Proxy: dcp.evomi.com:2000
  • HTTPs Proxy: dcp.evomi.com:2001
  • SOCKS5 Proxy: dcp.evomi.com:2002

To use a different product or protocol:

  1. Replace the proxy_host and proxy_port variables.
  2. Adjust the proxy URLs in the options dictionary. For SOCKS5, it would look like: 
    options = {
    'proxy': {
        'http': f'socks5://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}',
        'https': f'socks5://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}',
    }
}

Running the Script

Save the script to a file (e.g., selenium_wire_proxy_test.py) and run it using Python:

python selenium_wire_proxy_test.py

If everything is set up correctly, you should see the content of https://ip.evomi.com/s printed in your console, showing the IP address of the Evomi proxy you’re using.

Tips and Troubleshooting

  • Credentials: Double-check your proxy username, password, host, and port.
  • Password Format: The proxy password format (your_password_session-anychars_mode-speed) includes additional parameters. Make sure to replace your_password with your actual password while keeping the _session-anychars_mode-speed part (or any other required session parameters) intact.
  • SeleniumWire Import: Ensure you’re importing webdriver from seleniumwire (from seleniumwire import webdriver).
  • SSL Errors: If you encounter SSL certificate errors for specific sites (not the proxy itself), you can configure Chrome to ignore these using chrome_options: 
    chrome_options = Options()
chrome_options.add_argument('--ignore-certificate-errors')
# Potentially also:
# chrome_options.add_argument('--allow-running-insecure-content')
# driver = webdriver.Chrome(seleniumwire_options=options, options=chrome_options)
    Use this with caution, especially in production.
  • WebDriver Path: If ChromeDriver (or your chosen WebDriver) is not in your system PATH, you may need to specify its executable path when creating the driver instance: 
    # from selenium.webdriver.chrome.service import Service as ChromeService
# service = ChromeService(executable_path='/path/to/chromedriver')
# driver = webdriver.Chrome(service=service, seleniumwire_options=options, options=chrome_options)
  • Check SeleniumWire Backend: SeleniumWire uses an underlying mitmproxy backend. If you have specific network configurations (VPNs, firewalls), ensure they don’t interfere with mitmproxy’s ability to capture and forward traffic.
  • Debug Headers: You can inspect requests and responses made by the browser through SeleniumWire: 
    # After driver.get(...)
for request in driver.requests:
    if request.response:
        print(
            request.url,
            request.response.status_code,
            request.response.headers.get('Content-Type')
        )

By following this guide, you should now be able to successfully integrate Evomi’s proxies with your Selenium scripts using SeleniumWire for robust web automation and scraping tasks.

Scrapy