Skip to content

R1 — SNI/DNS override: requests 2.34.2 + urllib3 2.7.0 — VERIFIED EMPIRICALLY

Method: recipe written against the INSTALLED urllib3 2.7.0 source (mechanism traced file:line), then executed against local TLS servers on 127.0.0.1/127.0.0.2/[::1] with a wildcard resolver pointing the hostname elsewhere — 38/38 checks passed, twice (deterministic).

Binding recipe for wpt/dnsoverride.py (drop-in, empirically proven)

Design: keep the urllib3 pool keyed by the URL hostname (Host header, SNI, and cert-match all derive from it automatically) and override ONLY the TCP dial (HTTPSConnection._new_conn). No global monkeypatching; scoped to one adapter.

import sys, socket
import requests
import urllib3
import urllib3.connection, urllib3.connectionpool
from requests.adapters import HTTPAdapter
from urllib3.poolmanager import PoolManager
from urllib3.exceptions import ConnectTimeoutError, NameResolutionError, NewConnectionError
from urllib3.util.connection import create_connection

class IPDialHTTPSConnection(urllib3.connection.HTTPSConnection):
    """TCP dials dial_ip; Host/SNI/cert-target stay self.host (URL hostname)."""
    def __init__(self, *args, **kwargs):
        self.dial_ip = kwargs.pop("dial_ip", None)
        super().__init__(*args, **kwargs)

    def _new_conn(self):
        if self.dial_ip is None:
            return super()._new_conn()
        try:
            sock = create_connection((self.dial_ip, self.port), self.timeout,
                                     source_address=self.source_address,
                                     socket_options=self.socket_options)
        except socket.gaierror as e:
            raise NameResolutionError(self.host, self, e) from e
        except socket.timeout as e:
            raise ConnectTimeoutError(self, f"Connection to {self.host} timed out.") from e
        except OSError as e:
            raise NewConnectionError(self, f"Failed to establish a new connection: {e}") from e
        sys.audit("http.client.connect", self, self.dial_ip, self.port)
        return sock

class IPDialHTTPSConnectionPool(urllib3.connectionpool.HTTPSConnectionPool):
    ConnectionCls = IPDialHTTPSConnection

class IPDialPoolManager(PoolManager):
    def __init__(self, target_host, target_ip, **kwargs):
        self._target_host = target_host.rstrip(".").lower()
        self._target_ip = target_ip
        super().__init__(**kwargs)
        # PoolManager.__init__ assigns the module-level dict BY REFERENCE — copy
        # before mutating or you globally patch every PoolManager.
        self.pool_classes_by_scheme = dict(self.pool_classes_by_scheme)
        self.pool_classes_by_scheme["https"] = IPDialHTTPSConnectionPool

    def _new_pool(self, scheme, host, port, request_context=None):
        if scheme == "https" and host and host.rstrip(".").lower() == self._target_host:
            ctx = dict(request_context) if request_context is not None else {}
            ctx["dial_ip"] = self._target_ip
            return super()._new_pool(scheme, host, port, ctx)
        return super()._new_pool(scheme, host, port, request_context)

class IPDialHTTPAdapter(HTTPAdapter):
    def __init__(self, hostname, ip, **kwargs):
        self._hostname, self._ip = hostname, ip
        super().__init__(**kwargs)

    def init_poolmanager(self, connections, maxsize, block=False, **pool_kwargs):
        self._pool_connections, self._pool_maxsize, self._pool_block = connections, maxsize, block
        self.poolmanager = IPDialPoolManager(self._hostname, self._ip,
                                             num_pools=connections, maxsize=maxsize,
                                             block=block, **pool_kwargs)

def resolve_session(hostname, ip, *, verify=False, timeout=30.0,
                    pool_maxsize=10, pool_connections=4, max_retries=0) -> requests.Session:
    s = requests.Session()
    s.default_timeout = timeout  # or subclass Session to enforce (see W3 impl)
    s.verify = verify
    s.trust_env = False          # hostile HTTPS_PROXY/ALL_PROXY/netrc cannot hijack
    s.mount("https://", IPDialHTTPAdapter(hostname, ip, pool_connections=pool_connections,
                                          pool_maxsize=pool_maxsize, max_retries=max_retries))
    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)  # process-global, called once at module init
    return s

Mechanism (installed source, requests 2.34.2 / urllib3 2.7.0)

  • Do NOT use the server_hostname= approach: it requires the pool host to be the IP, forcing manual Host headers and assert_same_host=False. Host-keyed pool + dial override avoids all of it. requests never sets Host itself; http.client derives it from the connection host (connection.py:461 skip_host = "host" in header_keys).
  • Flow: HTTPAdapter.get_connection_with_tls_context → poolmanager._new_pool (poolmanager.py:243-279, the documented customization hook) injects dial_ip into request_context → pool conn_kw → ConnectionCls (connectionpool.py:1080-1095) → our _new_conn replaces the ONLY DNS resolution point (connection.py:198-225, create_connection at :204). SNI: HTTPSConnection.connect uses self.host (connection.py:760,780). Numeric IP literals never hit a resolver.
  • verify=False → cert_reqs="CERT_NONE" set at pool build + per-request cert_verify.

Pitfalls (all verified)

  • InsecureRequestWarning fires PER REQUEST (connectionpool._validate_conn:1098) and urllib3 installs simplefilter("always", SecurityWarning) at import: call urllib3.disable_warnings(InsecureRequestWarning) ONCE (process-global; no per-session scoping exists).
  • trust_env=False required (env proxies hijack otherwise); also disables netrc/env CA bundles — acceptable since verify=False in this mode.
  • Host-gated: only the configured hostname is overridden; same-host redirects keep the override; cross-host redirects fall back to real DNS (document this behavior).
  • Pool rebuilds (poolmanager.clear(), adapter unpickling) re-inject dial_ip — idempotent.
  • IPv6 dial literals work (bare + bracketed); Host header never shows the IP.
  • Port: URL port flows to dial; Host gets :port appended for non-443 automatically.
  • Pin urllib3>=2.7,<3 (set_cert() already FutureWarns in 3.0; our extension surface is the same one urllib3's SOCKS contrib uses — moderate 3.0 risk).

Verification performed (38/38, re-run deterministic)

Local self-signed TLS servers on 127.0.0.1/127.0.0.2/[::1]; wildcard resolver pointing the hostname to a wrong IP; server-side SNI capture (sni_callback) + Host assertion; keep-alive pool reuse; redirect-following; unroutable-IP error path; hostile proxy env; verify=True still failing correctly; global-state untouched assertions.