Cloudscraper

该介绍目前仅提供英文版本。

Overview

Cloudscraper is a specialized Python module designed to bypass Cloudflare’s anti-bot protection, commonly known as “I’m Under Attack Mode” (IUAM). Built on top of the Requests library, it allows developers to access content on websites protected by Cloudflare without being blocked by their challenge page, effectively simulating a real browser solving the JavaScript challenges.

Key Features

• Cloudflare Challenge Bypass: Automatically solves JavaScript challenges presented by Cloudflare’s protection
• Captcha Support: Integrates with multiple CAPTCHA solving services (2captcha, CapMonster, Anti-Captcha, etc.)
• Session Handling: Maintains cookies and user-agent consistency to prevent detection
• Proxy Support: Works with various proxy configurations, including SOCKS proxies
• Requests Compatibility: Provides a drop-in replacement for the Requests module with identical API
• Integrated Debugging: Includes a debugging mode to troubleshoot connection issues
• Custom Browser Emulation: Supports custom user-agents and browser fingerprints
• SSL/TLS Configuration: Allows customization of cryptographic settings for challenging environments

How It Works

Cloudscraper operates by:

1. Detecting Challenges: Identifying when a website is protected by Cloudflare’s anti-bot system
2. Solving JavaScript Puzzles: Executing and solving the JavaScript challenges that Cloudflare presents
3. Managing Tokens: Properly handling the cookies and tokens required for authentication
4. Maintaining Sessions: Ensuring that subsequent requests use the same tokens and user-agent
5. Handling CAPTCHAs: Supporting various CAPTCHA solving services when challenges require human verification

Usage Examples

The library is designed to be as simple to use as the Requests library:

import cloudscraper

# Create a scraper instance
scraper = cloudscraper.create_scraper()

# Make a request
response = scraper.get('https://protected-site.com')
print(response.text)

For more complex scenarios, it supports various options:

# With proxy support
scraper = cloudscraper.create_scraper(
    proxies={'http': 'http://localhost:8080', 'https': 'http://localhost:8080'}
)

# With CAPTCHA solving
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',
        'api_key': 'your_2captcha_api_key'
    }
)

Integration Capabilities

Cloudscraper offers robust integration with other tools and libraries:

• Cookie Extraction: Methods to extract cookies for use with other HTTP clients
• User-Agent Synchronization: Functions to maintain the same user-agent across different requests
• External Tool Support: Easy integration with tools like curl via command-line interfaces
• Session Management: Functions to retrieve and manage Cloudflare session tokens

Technical Details

The project is built with:

• Python (99.2%)
• Requests library as the foundation
• JavaScript parsing and execution capabilities
• Support for modern cryptographic standards
• Compatibility with Python 2.7 and 3.x

Ethical Use Notice

This tool is provided for legitimate research, testing, and development purposes. It should be used responsibly and in compliance with website terms of service and applicable laws regarding web scraping and automated access.