🔍 Proxy Ultimate
Author: mrak66 | Telegram channel: «allwersia» {xdkill} | End Way - Форум программирования и сливов различных скриптов
Parse 130+ sources, validate with 500+ concurrent connections, enrich with GeoIP, export by country
Parse 130+ sources, validate with 500+ concurrent connections, enrich with GeoIP, export by country
✨ Features
| Category | Details |
|---|---|
| 130+ Sources | GitHub raw lists, ProxyScrape, GeoNode, PubProxy, HideMyName, Spys.one, Pastebin, and more |
| Multi-Protocol | HTTP, HTTPS, SOCKS4, SOCKS5 — parse and validate all types |
| Blazing Fast | Up to 500+ concurrent checks via asyncio + aiohttp |
| GeoIP Enrichment | 3-stage lookup: ip-api.com batch → single → ipwho.is fallback with rate-limit handling |
| Anonymity Detection | Deep check: Elite, Anonymous, Transparent classification |
| Smart Export | Auto-export by country, protocol, and speed ranking |
| SQLite Database | Persistent storage with index optimization |
| Web API Server | REST API for stats, proxy lists, and random proxy |
| Rich Terminal UI | Progress bars, tables, colored output via rich |
| Zero Dependencies | Works with only aiohttp (optional: rich, aiosqlite) |
| Cross-Platform | Windows, Linux, macOS — with ProactorEventLoop fix |
| Premium Support | Oxylabs & BrightData via environment variables |
| Scheduler | Auto-run collection at configurable intervals |
📖 Usage
Basic Modes
# Full pipeline: collect + check + exportpython proxy_ultimate.py
# Only collect from sources (no validation)
python proxy_ultimate.py --mode collect
# Only validate previously collected proxies
python proxy_ultimate.py --mode check
# Start web API server
python proxy_ultimate.py --mode serve --web-port 9090
Protocol Filtering
# Only HTTP proxiespython proxy_ultimate.py --protocol http
# Only SOCKS5 proxies
python proxy_ultimate.py --protocol socks5
# Only HTTPS proxies
python proxy_ultimate.py --protocol https
Performance Tuning
# 500 concurrent connections, 15s timeoutpython proxy_ultimate.py --concurrent 500 --timeout 15
# Check only first 1000 proxies
python proxy_ultimate.py --max-check 1000
# Use only 50 sources (faster collection)
python proxy_ultimate.py --sources 50
Deep Analysis
# Full analysis: anonymity check + GeoIP enrichmentpython proxy_ultimate.py --mode all --deep --geoip
# Filter by specific countries
python proxy_ultimate.py --geoip --countries "US,GB,DE,RU"
# Elite proxies only from specific countries
python proxy_ultimate.py --deep --anonymity elite --geoip --countries "US,DE"
Advanced
# Load proxies from file instead of collectingpython proxy_ultimate.py --mode check --input-file my_proxies.txt
# Custom output directory
python proxy_ultimate.py --output-dir ./results
# Schedule auto-run every 30 minutes
python proxy_ultimate.py --schedule 30
# Verbose logging
python proxy_ultimate.py --log-level DEBUG --verbose
⚙️ CLI Arguments
| Argument | Default | Description |
|---|---|---|
| --mode | all | collect, check, serve, all |
| --sources | 0 | Limit number of sources (0 = all) |
| --concurrent | 200 | Max concurrent validation threads |
| --timeout | 10.0 | Connection timeout in seconds |
| --protocol | all | http, https, socks4, socks5, all |
| --anonymity | None | transparent, anonymous, elite |
| --max-speed | 9999 | Max response time filter (ms) |
| --max-check | 0 | Max proxies to check (0 = all) |
| --deep | false | Enable anonymity level detection |
| --geoip | false | Enable GeoIP country/city/ISP lookup |
| --countries | "" | Filter by country codes (comma-separated) |
| --output-dir | ./output | Output directory for results |
| --db-path | ./proxies.db | SQLite database path |
| --web-port | 8080 | Web API server port |
| --schedule | 0 | Auto-run interval in minutes |
| --no-color | false | Disable colored output |
| --verbose | false | Verbose terminal output |
| --log-level | WARNING | DEBUG, INFO, WARNING, ERROR |
| --input-file | None | Load proxies from file |
🌐 Web API
Start the REST API server to access proxies programmatically:python proxy_ultimate.py --mode serve --web-port 9090
Endpoints
| Endpoint | Method | Description |
|---|---|---|
| /api/health | GET | Health check + timestamp |
| /api/stats | GET | Full statistics (valid, by protocol, by country) |
| /api/proxies | GET | List all valid proxies (filter: ?protocol=http&format=json) |
| /api/random | GET | Get one random proxy |
Examples
# Get statisticscurl http://localhost:9090/api/stats
# Get only SOCKS5 proxies as text
curl http://localhost:9090/api/proxies?protocol=socks5
# Get all proxies as JSON
curl http://localhost:9090/api/proxies?format=json
# Random proxy
curl http://localhost:9090/api/random
📁 Output Structure
After running, results are saved to ./output/:output/
├── http.txt # All valid HTTP proxies
├── socks5.txt # All valid SOCKS5 proxies
├── fastest.txt # Top 200 fastest proxies
└── by_country/ # Proxies grouped by country code
├── US.txt
├── DE.txt
├── RU.txt
├── GB.txt
└── ... # One file per country
All files are in ip:port format, one proxy per line.
📡 Sources (130+)
| Type | Count | Examples |
|---|---|---|
| GitHub Raw | 53 | TheSpeedX, monosans, jetkai, clarketm, mmpx12, prxchk... |
| API (Text) | 24 | ProxyScrape, proxy-list.download, PubProxy, ProxySpace, FoxTools |
| HTML Tables | 20 | free-proxy-list.net, sslproxies.org, hidemy.name, geonode.com |
| JSON APIs | 9 | GeoNode, FreeProxy, GimmeProxy, OpenTrack |
| PubProxy Pages | 15 | Paginated requests (5 pages x 3 protocols) |
| Special Parsers | 4 | Spys.one, HideMyName, GimmeProxy, Pastebin search |
| Extra Raw | 28 | Additional GitHub sources |
| Premium | 2 | Oxylabs, BrightData (via API keys) |
Premium Providers
# Set API keys as environment variablesexport OXYLABS_KEY="your_key_here"
export BRIGHTDATA_KEY="your_key_here"
# Run — premium sources will be included automatically
python proxy_ultimate.py
🏗 Architecture
proxy_ultimate.py # Single-file, zero-dependency core├── ProxyParser # Source collection & parsing
│ ├── GitHub raw lists
│ ├── API text endpoints
│ ├── HTML table scraping
│ ├── JSON API parsing
│ ├── Pastebin search
│ └── Premium providers
├── ProxyChecker # Validation engine
│ ├── HTTP/HTTPS check # Via test URLs (Google, httpbin, etc.)
│ ├── SOCKS4 check # Raw SOCKS4 handshake
│ ├── SOCKS5 check # Raw SOCKS5 handshake
│ ├── Anonymity detect # httpbin.org/headers analysis
│ └── GeoIP enrich # 3-stage: batch → single → fallback
├── ProxyManager # In-memory proxy store
│ ├── Deduplication
│ ├── Filtering (protocol, country, speed, anonymity)
│ └── Statistics
├── DatabaseManager # SQLite persistence
├── Exporter # Smart file export
├── WebServer # REST API (aiohttp.web)
├── Scheduler # Interval-based auto-run
└── ConsoleUI # Rich terminal output
🔧 How It Works
Validation Pipeline
┌──────────┐ ┌───────────┐ ┌────────────┐ ┌──────────┐│ Collect │───>│ Deduplicate│───>│ Validate │───>│ Export │
│ 130+ src │ │ by key │ │ concurrent │ │ by group │
└──────────┘ └───────────┘ └────────────┘ └──────────┘
│
┌───────────┼───────────┐
▼ ▼ ▼
HTTP/HTTPS SOCKS4 SOCKS5
test URLs handshake handshake
GeoIP Pipeline
Stage 1: ip-api.com/batch (50 per request, rate-limited)↓ missed IPs
Stage 2: ip-api.com/single (4 concurrent, rate-limited)
↓ still missed
Stage 3: ipwho.is + ipwhois.app fallback (6 concurrent)
↓
100% coverage target
📊 Example Output
___ ___ _ _ _____ _ _____| \/ |(_) (_)_ _| | | _ |
| . . | _ _ __ _ | | | | | | | | |
| |\/| || || '_ \| || | | | | | | | |
| | | || || | | || | |_| | \_/\_/
\_| |_/|_||_| |_||_/ \___/ \___/
Ultimate Proxy Parser & Validator v3.1
130+ sources | asyncio+aiohttp
Collecting from 130 sources...
Collected 48231 new (total: 48231)
Checking 48231 proxies (500 concurrent)...
██████████████████████████████ 100% 12m 34s
[OK] Checked 48231: 3847 valid, 12 auth, 41231 dead, 3141 timeout
┌────────────────── Results ──────────────────┐
│ Total 48231 │
│ Valid (no auth) 3847 │
│ Auth Required 12 │
│ Invalid 41231 │
│ Elite 1203 │
│ Avg Speed 847ms │
└─────────────────────────────────────────────┘
Top Countries Count
US 842
DE 634
RU 521
FR 412
GB 387
🛠 Requirements
| Package | Required | Purpose |
|---|---|---|
| aiohttp | Yes | Async HTTP client for collection & validation |
| rich | No | Colored terminal output, progress bars, tables |
| aiosqlite | No | Async SQLite database persistence |
pip install aiohttp
# Recommended
pip install aiohttp rich aiosqlite
Python 3.8+ required. Tested on Windows, Linux, and macOS.
📄 License
MIT License. Use freely for personal and commercial projects. Для просмотра скрытого содержимого вы должны войти или зарегистрироваться.
Последнее редактирование: