[rohanbuilds-ai]

Summary

Fixes #506.

This PR provides comprehensive SSRF hardening for the User Profile avatar update and share card fetch endpoints in finbot/apps/ctf/routes/profile.py and finbot/apps/ctf/routes/share.py, addressing maintainer feedback from @c-thiam regarding IPv6 handling, multi-answer DNS, DNS rebinding / TOCTOU defenses, and preserving strict TLS verification (verify=True).

---

Threat Model & Addressed Vulnerabilities

1. Validation Bypass on Partial Profile Updates:
   In update_profile, avatar URL validation was previously gated on if request.avatar_type == "url":. If a client updated avatar_url while omitting avatar_type (None), validation was bypassed even when the stored profile already had avatar_type == "url".
2. Missing Address Family & Metadata Coverage:
   Previous checks did not cover IPv6 loopback (::1), link-local (fe80::/10), unique local (fc00::/7, fd00::/8), unspecified (::), cloud metadata (169.254.169.254), CGNAT (100.64.0.0/10), IPv4-mapped IPv6, or hostnames resolving to mixed public/private DNS records.
3. DNS Rebinding & Redirect Pivots in Share Card:
   _fetch_avatar_b64 previously used follow_redirects=True and had no connection-time IP pinning. A public URL could redirect to internal endpoints, or a DNS rebinding service with a short TTL could resolve to internal IPs at TCP connect time. Prior attempts (fix(security): resolve profile update validation bypass and SSRF #507) tried connecting directly to IP addresses with verify=False, which disabled TLS certificate verification.
4. Unhandled Server Errors on Malformed Inputs:
   Malformed URLs (e.g. out-of-range ports like :99999999 or non-integer ports) raised unhandled ValueError inside urlparse, escaping the route handler as an unexpected HTTP 500 error instead of a controlled HTTP 400.
5. Event Loop Blocking:
   Synchronous DNS resolution (socket.getaddrinfo) directly on the event loop thread in async route handlers could freeze the worker process during slow DNS responses.

---

Implementation Details

1. Comprehensive IP & CIDR Filtering (is_ip_allowed):
   • Covers IPv4 loopback (127.0.0.0/8), RFC-1918 private subnets (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), link-local / cloud metadata (169.254.0.0/16), carrier-grade NAT (100.64.0.0/10), and 0.0.0.0/8.
   • Covers IPv6 loopback (::1/128), unique local (fc00::/7), link-local (fe80::/10), and unspecified (::/128).
   • Automatically unpacks and inspects IPv4-mapped IPv6 addresses.
2. Asynchronous Non-Blocking URL Validation (validate_avatar_url):
   • Safely parses URLs and ports, converting ValueError into HTTP 400 client errors.
   • Enforces the HTTPS scheme and rejects embedded credentials or empty hostnames.
   • Performs DNS resolution asynchronously via loop.getaddrinfo(...) in the thread pool executor, preventing event loop blocking.
   • Checks every resolved IP and rejects the request if any address is prohibited.
3. Connection-Time IP Pinning with Preserved TLS (SSRFProtectedBackend):
   • Subclasses httpcore._backends.anyio.AnyIOBackend.
   • In connect_tcp(), asynchronously resolves the target hostname and fails closed if any address is prohibited or malformed.
   • Directly connects to verified safe IP literals, preventing DNS rebinding without a second hostname lookup.
   • Implements sequential dual-stack fallback across verified safe IPs if the initial address (e.g. IPv6) is unreachable.
   • Retains the original target hostname in httpcore's start_tls call for SNI and CA certificate validation (verify=True).
4. Avatar Fetcher Hardening (_fetch_avatar_b64):
   • Pre-validates the URL via validate_avatar_url.
   • Disables HTTP redirects (follow_redirects=False).
   • Routes outbound requests through make_ssrf_safe_transport().

---

Verification & Test Coverage

A dedicated test suite has been added in tests/unit/ctf/test_avatar_ssrf.py (30 test cases):

• Validation bypass reproduction and prevention when omitting avatar_type.
• Rejection of non-HTTPS schemes (http://, ftp://).
• Rejection of direct and resolved private/metadata addresses across IPv4, IPv6, and IPv4-mapped IPv6.
• Mixed public/private DNS answers fail closed.
• HTTP redirect suppression in the share card avatar fetcher.
• Connection-time DNS rebinding prevention via SSRFProtectedBackend.
• Preservation of valid public HTTPS avatar URLs.
• Rejection of malformed URLs, out-of-range ports, non-integer ports, embedded credentials, and empty hostnames with HTTP 400.
• Dual-stack safe IP failover.
• Explicit verification that server_hostname in TLS negotiation retains the original domain name.

Verified Test Results:

• tests/unit/ctf/test_avatar_ssrf.py: 30 passed in 4.12s
• tests/unit/ctf/: 57 passed, 1 skipped in 6.87s
• tests/unit/apps/web/test_routes.py: 13 passed in 37.34s
• Formatting and whitespace verified with isort, black, and git diff --check.

---

Compatibility & Architectural Considerations

• make_ssrf_safe_transport() configures transport._pool._network_backend = SSRFProtectedBackend(). While httpcore.AsyncConnectionPool exposes network_backend in its constructor, httpx.AsyncHTTPTransport.__init__ does not currently expose it as a top-level parameter. This integration operates cleanly on the repository's pinned httpx 0.28.1 and httpcore 1.0.9 while maintaining complete connection pooling and TLS verification pipelines.

[prince-shakyaa]

Hi @rohanbuilds-ai, thanks for working on this! Just a heads-up that #507 already covers #506 (I filed the issue and opened that PR back in May). I've just updated it to keep TLS verification on with IP pinning, using httpx's public sni_hostname option instead of the transport internals. It also handles CGNAT, IPv4-mapped IPv6, bad ports and non-blocking DNS.

Would you mind taking a look at #507? If you spot anything missing, suggestions there are very welcome. I'll leave it to the maintainers to decide how to go forward.