Implementing Proxy Rotation: Code Examples

Proxy Rotation Implementation

For scraping and high-volume tasks, rotating proxies prevents blocks.

TL;DR: Maintain a pool of proxies, rotate on each request or on failure, implement retry logic with fresh proxies.

Python Implementation

import requests
import random
proxy_pool = [
    'http://user:pass@proxy1:port',
    'http://user:pass@proxy2:port',
    'http://user:pass@proxy3:port',
]
def get_random_proxy():
    proxy = random.choice(proxy_pool)
    return {'http': proxy, 'https': proxy}
def fetch_with_rotation(url, max_retries=3):
    for attempt in range(max_retries):
        try:
            proxy = get_random_proxy()
            response = requests.get(url, proxies=proxy, timeout=10)
            if response.status_code == 200:
                return response
        except Exception as e:
            print(f"Attempt {attempt + 1} failed: {e}")
    return None
# Usage
response = fetch_with_rotation('https://example.com')

Node.js Implementation

const axios = require('axios');
const HttpsProxyAgent = require('https-proxy-agent');
const proxyPool = [
  'http://user:pass@proxy1:port',
  'http://user:pass@proxy2:port',
  'http://user:pass@proxy3:port',
];
function getRandomProxy() {
  return proxyPool[Math.floor(Math.random() * proxyPool.length)];
}
async function fetchWithRotation(url, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      const agent = new HttpsProxyAgent(getRandomProxy());
      const response = await axios.get(url, { httpsAgent: agent });
      return response.data;
    } catch (error) {
      console.log(Attempt ${i + 1} failed);
    }
  }
  return null;
}

Smart Rotation Strategies

Round Robin

class ProxyRotator:
    def __init__(self, proxies):
        self.proxies = proxies
        self.index = 0
    def get_next(self):
        proxy = self.proxies[self.index]
        self.index = (self.index + 1) % len(self.proxies)
        return proxy

Weighted by Success

class WeightedRotator:
    def __init__(self, proxies):
        self.proxy_scores = {p: 1.0 for p in proxies}
    def get_proxy(self):
        # Higher score = more likely to be selected
        return max(self.proxy_scores, key=self.proxy_scores.get)
    def report_success(self, proxy):
        self.proxy_scores[proxy] = min(self.proxy_scores[proxy] + 0.1, 2.0)
    def report_failure(self, proxy):
        self.proxy_scores[proxy] = max(self.proxy_scores[proxy] - 0.3, 0.1)

For Outreach

Remember: Outreach needs static proxies, not rotation. Use rotation only for scraping tasks.