HTTPX — An Overview

HTTPX is a next-generation HTTP client for Python, designed to be fully compatible with the standard requests library while adding support for asynchronous requests. It allows for synchronous and asynchronous HTTP requests, making it versatile for both blocking and non-blocking operations. HTTPX supports HTTP/1.1 and HTTP/2, and includes features like connection pooling, cookie management, and proxy support. It's ideal for modern web scraping and API interaction tasks, offering advanced functionality and performance. For more information, visit HTTPX documentation.

Prerequisites

Install Python from the official website.

python --version

If you receive an error or see a version number starting with 2.x, you need to download Python 3.x and follow the installation instructions to set it up

Next, create a new Python project and install the HTTPX package

pip install httpx[cli]

After installing the HTTPX library, let's make a request to retrieve our IP address. This can be achieved by sending a GET request to a service that returns your public IP. Here’s how you can do it:

import asyncio
import httpx

async def main():
    url = 'http://httpbin.io/ip'

    async with httpx.AsyncClient() as client:
        req = await client.get(url)
        if req.status_code == 200:
            try:
                res = req.json()
                print(res)
            except Exception as e:
                print(f"Failed to decode JSON: {e}")
        else:
            print(f"Request failed with status code: {req.status_code}")

asyncio.run(main())

After running the script, a GET request is sent to http://httpbin.io/ip, retrieving your public IP address from the response. The IP address is then displayed in JSON format.

After setting up HTTPX, let's configure it to integrate with ProxyPanel proxies for secure and anonymous web scraping.

Configuring the HTTPXs Package with ProxyPanel

To integrate the request with our proxy, we need to first get the proxy configuration from the ProxyPanel dashboard. This includes obtaining the proxy server address, port, and any necessary authentication details

Go to your Dashboard panel and navigate to the "My Proxy" section to view your IP information.

Click on the "Show Password" button and enter your account password to display your proxy password.

Once we have this information, we can add the proxies parameter to the asyncClient function as shown in the code.

import asyncio
import httpx
from httpx import Proxy

async def main():
    url = 'http://httpbin.io/ip'
    proxy_url = "http://154.198.34.247:8083"
    proxy_auth = ("john.MyProxy-1", "Proxypanel!123")

    proxies = {
        "http://": Proxy(url=proxy_url, auth=proxy_auth),
        "https://": Proxy(url=proxy_url, auth=proxy_auth),
    }

    async with httpx.AsyncClient(proxies=proxies) as client:
        req = await client.get(url)
        if req.status_code == 200:
            try:
                res = req.json()
                print(res)
            except Exception as e:
                print(f"Failed to decode JSON: {e}")
        else:
            print(f"Request failed with status code: {req.status_code}")

asyncio.run(main())

										

Replace username, password, proxy_address, and port with your ProxyPanel credentials and proxy details.

You will notice that the IP address with proxy configuration differs from the one without a proxy. Now you have successfully integrated the HTTPX package with ProxyPanel.