Python:Selenium + Chrome添加认证代理
生活随笔
收集整理的這篇文章主要介紹了
Python:Selenium + Chrome添加认证代理
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
添加無認證代理,以參數形式添加
chromeOptions = webdriver.ChromeOptions() chromeOptions.add_argument('--proxy-server=http://ip:port') driver = webdriver.Chrome(chrome_options=chromeOptions)添加認證代理,以插件的形式添加
1、新建擴展文件夾 Chrome-proxy-helper添加兩個文件
Chrome-proxy-helper/background.jsmanifest.json內容如下
(1)background.js
var config = {mode: "fixed_servers",rules: {singleProxy: {scheme: "http",host: "%proxy_host",port: parseInt(%proxy_port)},bypassList: ["foobar.com"]}};chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});function callbackFn(details) {return {authCredentials: {username: "%username",password: "%password"}}; }chrome.webRequest.onAuthRequired.addListener(callbackFn,{urls: ["<all_urls>"]},['blocking'] );(2)manifest.json
{"version": "1.0.0","manifest_version": 2,"name": "Chrome Proxy","permissions": ["proxy","tabs","unlimitedStorage","storage","<all_urls>","webRequest","webRequestBlocking"],"background": {"scripts": ["background.js"]},"minimum_chrome_version":"22.0.0" }2、動態生成插件并使用
# -*- coding: utf-8 -*-import os import re import time import zipfile from selenium import webdriver# Chrome代理模板插件(https://github.com/RobinDev/Selenium-Chrome-HTTP-Private-Proxy)目錄 CHROME_PROXY_HELPER_DIR = 'Chrome-proxy-helper' # 存儲自定義Chrome代理擴展文件的目錄 CUSTOM_CHROME_PROXY_EXTENSIONS_DIR = 'chrome-proxy-extensions'def get_chrome_proxy_extension(proxy):"""獲取一個Chrome代理擴展,里面配置有指定的代理(帶用戶名密碼認證)proxy - 指定的代理,格式: username:password@ip:port"""m = re.compile('([^:]+):([^\@]+)\@([\d\.]+):(\d+)').search(proxy)if m:# 提取代理的各項參數username = m.groups()[0]password = m.groups()[1]ip = m.groups()[2]port = m.groups()[3]# 創建一個定制Chrome代理擴展(zip文件)if not os.path.exists(CUSTOM_CHROME_PROXY_EXTENSIONS_DIR):os.mkdir(CUSTOM_CHROME_PROXY_EXTENSIONS_DIR)extension_file_path = os.path.join(CUSTOM_CHROME_PROXY_EXTENSIONS_DIR, '{}.zip'.format(proxy.replace(':', '_')))if not os.path.exists(extension_file_path):# 擴展文件不存在,創建zf = zipfile.ZipFile(extension_file_path, mode='w')zf.write(os.path.join(CHROME_PROXY_HELPER_DIR, 'manifest.json'), 'manifest.json')# 替換模板中的代理參數background_content = open(os.path.join(CHROME_PROXY_HELPER_DIR, 'background.js')).read()background_content = background_content.replace('%proxy_host', ip)background_content = background_content.replace('%proxy_port', port)background_content = background_content.replace('%username', username)background_content = background_content.replace('%password', password)zf.writestr('background.js', background_content)zf.close()return extension_file_pathelse:raise Exception('Invalid proxy format. Should be username:password@ip:port')if __name__ == '__main__':# 測試options = webdriver.ChromeOptions()# 添加一個自定義的代理插件(配置特定的代理,含用戶名密碼認證)options.add_extension(get_chrome_proxy_extension(proxy="username:password@ip:port"))driver = webdriver.Chrome(options=options)# 訪問一個IP回顯網站,查看代理配置是否生效了driver.get('http://httpbin.org/ip')print(driver.page_source)time.sleep(60)driver.close()driver.quit()參考:
總結
以上是生活随笔為你收集整理的Python:Selenium + Chrome添加认证代理的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 区块链网络搭建
- 下一篇: websocket python爬虫_p