Selenium
This guide provides instructions on how to integrate Evomi’s proxies with Selenium, a popular web browser automation tool, using SeleniumWire for enhanced proxy management.
Prerequisites
Before you begin, ensure you have the following:
- Python installed on your system.
- Selenium and SeleniumWire installed in your project.
- Your Evomi proxy credentials (username and password).
Installation
If you haven’t already installed Selenium and SeleniumWire, you can do so using pip:
pip install selenium selenium-wire
You’ll also need to install a WebDriver for your preferred browser. For this guide, we’ll use ChromeDriver:
- Download ChromeDriver from the official website (https://chromedriver.chromium.org/downloads).
- Ensure the ChromeDriver executable is in your system’s PATH or specify its location in your script.
Configuration
To use Evomi proxies with SeleniumWire, you need to configure the WebDriver with the proxy settings. SeleniumWire makes it easy to handle proxies that require authentication.
Here’s how you can do it:
from seleniumwire import webdriver # Import from seleniumwire
from selenium.webdriver.chrome.options import Options
# Proxy configuration
proxy_host = "rp.evomi.com"
proxy_port = "1000" # Port should be a string for selenium-wire options
proxy_username = "your_username"
proxy_password = "your_password_session-anychars_mode-speed"
# SeleniumWire options
options = {
'proxy': {
'http': f'http://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}',
'https': f'https://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}',
'no_proxy': 'localhost,127.0.0.1' # Add domains to bypass proxy if needed
}
}
# Set up Chrome options (optional, e.g., for headless mode)
chrome_options = Options()
# chrome_options.add_argument('--headless') # Example: run in headless mode
# Create a new Chrome driver instance with SeleniumWire
driver = webdriver.Chrome(seleniumwire_options=options, options=chrome_options)
# Navigate to a page to test the proxy
driver.get("https://ip.evomi.com/s")
print(driver.page_source)
# Close the browser
driver.quit()
Replace your_username
with your actual Evomi proxy username and your_password
with your actual proxy password.
Explanation
Let’s break down the key parts of this script:
- We import
webdriver
fromseleniumwire
instead ofselenium
. - We set up the proxy configuration with your Evomi proxy details:
proxy_host
,proxy_port
,proxy_username
, andproxy_password
. - We create a
seleniumwire_options
dictionary. This dictionary configures the HTTP and HTTPS proxies, including the authentication details (username and password) directly in the proxy URL.- The format for the proxy string is
protocol://username:password@host:port
. no_proxy
can be used to specify any hosts that should bypass the proxy.
- The format for the proxy string is
- We initialize
webdriver.Chrome
, passing theseleniumwire_options
to theseleniumwire_options
parameter. You can also pass standard SeleniumOptions
(likechrome_options
for headless mode, etc.) to theoptions
parameter. - We navigate to
https://ip.evomi.com/s
to verify the proxy connection. This page displays the IP address seen by the server. - We print the page source, which should show the IP address of the Evomi proxy you’re using.
- Finally, we close the browser using
driver.quit()
.
Evomi Proxy Endpoints
Depending on the Evomi product and protocol you’re using, you’ll need to adjust the proxy_host
and proxy_port
in your code. Ensure the protocol in the options
dictionary (http/https) also matches your needs.
Residential Proxies
- HTTP Proxy:
rp.evomi.com:1000
- HTTPs Proxy:
rp.evomi.com:1001
(Use this for HTTPS traffic if specifying separate proxy types, or ensure your main proxy supports HTTPS pass-through) - SOCKS5 Proxy:
rp.evomi.com:1002
(SeleniumWire also supports SOCKS proxies, the format would besocks5://user:pass@host:port
)
Mobile Proxies
- HTTP Proxy:
mp.evomi.com:3000
- HTTPs Proxy:
mp.evomi.com:3001
- SOCKS5 Proxy:
mp.evomi.com:3002
Datacenter Proxies
- HTTP Proxy:
dcp.evomi.com:2000
- HTTPs Proxy:
dcp.evomi.com:2001
- SOCKS5 Proxy:
dcp.evomi.com:2002
To use a different product or protocol:
- Replace the
proxy_host
andproxy_port
variables. - Adjust the proxy URLs in the
options
dictionary. For SOCKS5, it would look like:options = { 'proxy': { 'http': f'socks5://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}', 'https': f'socks5://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}', } }
Running the Script
Save the script to a file (e.g., selenium_wire_proxy_test.py
) and run it using Python:
python selenium_wire_proxy_test.py
If everything is set up correctly, you should see the content of https://ip.evomi.com/s
printed in your console, showing the IP address of the Evomi proxy you’re using.
Tips and Troubleshooting
- Credentials: Double-check your proxy username, password, host, and port.
- Password Format: The proxy password format (
your_password_session-anychars_mode-speed
) includes additional parameters. Make sure to replaceyour_password
with your actual password while keeping the_session-anychars_mode-speed
part (or any other required session parameters) intact. - SeleniumWire Import: Ensure you’re importing
webdriver
fromseleniumwire
(from seleniumwire import webdriver
). - SSL Errors: If you encounter SSL certificate errors for specific sites (not the proxy itself), you can configure Chrome to ignore these using
chrome_options
:chrome_options = Options() chrome_options.add_argument('--ignore-certificate-errors') # Potentially also: # chrome_options.add_argument('--allow-running-insecure-content') # driver = webdriver.Chrome(seleniumwire_options=options, options=chrome_options)
- WebDriver Path: If ChromeDriver (or your chosen WebDriver) is not in your system PATH, you may need to specify its executable path when creating the driver instance:
# from selenium.webdriver.chrome.service import Service as ChromeService # service = ChromeService(executable_path='/path/to/chromedriver') # driver = webdriver.Chrome(service=service, seleniumwire_options=options, options=chrome_options)
- Check SeleniumWire Backend: SeleniumWire uses an underlying mitmproxy backend. If you have specific network configurations (VPNs, firewalls), ensure they don’t interfere with mitmproxy’s ability to capture and forward traffic.
- Debug Headers: You can inspect requests and responses made by the browser through SeleniumWire:
# After driver.get(...) for request in driver.requests: if request.response: print( request.url, request.response.status_code, request.response.headers.get('Content-Type') )
By following this guide, you should now be able to successfully integrate Evomi’s proxies with your Selenium scripts using SeleniumWire for robust web automation and scraping tasks.