Skip to content

Check signup emails with UserCheck directly and the disposable-email-domains list - #658

Merged
bpepple merged 10 commits into
masterfrom
656-check-signup-emails-with-the-usercheck-api-directly-instead-of-through-rapidapi
Oct 17, 2026
Merged

bpepple merged 10 commits into
masterfrom
656-check-signup-emails-with-the-usercheck-api-directly-instead-of-through-rapidapi

Conversation

@bpepple

@bpepple bpepple commented Oct 17, 2026

Copy link
Copy Markdown
Member

Closes #656

Signup and email-change checks now go straight to UserCheck instead of MailCheck through RapidAPI. Before any lookup, the address is checked against the disposable-email-domains list, as @jyggen suggested, so known throwaway domains are rejected without spending a UserCheck credit.

⚠️ Deploy note

Add USERCHECK_API_KEY to the production env before deploying. RAPID_API_KEY and RAPID_API_HOST are no longer read and can be removed. The app won't start without the new key.

How an address is checked

  1. Local blocklists, no credit spent. The address is rejected if its domain, or any domain above it (so x.mailinator.com is caught too), is on the hand-kept EMAIL_DOMAIN_BLOCKLIST or the package's list of about 9,900 domains. The package also lists common typo domains such as gmial.com.

  2. Allowlist. gmail.com, yahoo.com and proton.me skip the lookup, as before.

  3. UserCheck lookup (GET /email/{email}):

    Signal Result
    disposable or spam "Temporary email addresses are not allowed."
    relay_domain "Email forwarding addresses are not allowed." (new)
    mx is false, with a did_you_mean suggestion "Did you mean x@gmail.com?" (new)
    mx is false, no suggestion "Email address is not valid."
    400 (malformed address) "Email address is not valid."

    An unknown mx result does not reject the address.

Decisions on the issue's open questions

  • Relay addresses are rejected. This matches the existing blocklist, which already blocks duck.com, simplelogin.com and passmail.net.
  • Fail open. If UserCheck can't answer (timeout, 5xx, or a 429 that's still there after one retry), the address is allowed, since it has already passed both local lists. It's logged, and a Pushover alert goes out at most once an hour. Before this change, any API error blocked the signup with "Error creating account. Contact the site administrator."
  • Results are cached per domain in Redis for 30 days, so a domain costs a credit only once a month. Only successful lookups are cached. Every check we act on is about the domain, not the mailbox, so caching loses nothing. The downside is that a domain that turns disposable mid-window gets through until its cache entry expires.
  • 429 handling. The free plan allows 1 request per second, so a 429 gets one retry after a second.

Other changes

  • Domain matching now ignores case. Before, Duck.com got past the blocklist.
  • A failed connection to the API no longer crashes the error handling.
  • Italian translations for the two new messages. The unused "Error creating account…" string is removed.

Tests

  • An automatic test setup now keeps every test off the real API and gives each test its own cache keys.
  • tests/users/test_utils.py is rewritten for the new client: path encoding, auth header, caching, retry, alerts and failures.
  • New tests/users/test_email_validation.py covers the order of checks and each UserCheck result.
  • The full suite passes. These changes haven't been run against the real UserCheck API yet.

Drop RAPID_API_* env variables and USERCHECK_API_KEY
Replace the MailCheck lookup through RapidAPI with UserCheck's own API, and check the disposable-email-domains blocklist first so known throwaway domains are rejected without spending a credit.

- Reject an address if its domain, or any domain above it, is on the hand-kept blocklist or the package's list. Domain matching now ignores case, so Duck.com no longer gets past the blocklist.
- Reject disposable, spam and relay domains, and domains without MX records. When UserCheck has a typo fix, suggest it ("Did you mean...?").
- Cache UserCheck results per domain in Redis for 30 days to save credits. Only successful lookups are cached.
- On a 429, retry once after a second to cover the 1 request/second limit.
- Fail open when UserCheck can't answer: allow the address, log it and send at most one Pushover alert an hour.
- Add Italian translations for the new messages.
- Keep tests off the real API and give each test its own cache keys.

Closes #656
@bpepple bpepple linked an issue Oct 17, 2026 that may be closed by this pull request
7 tasks done
@bpepple bpepple self-assigned this Oct 17, 2026
@bpepple bpepple added the enhancement New feature or request label Oct 17, 2026
- Catch Pushover errors in the UserCheck failure alert, and give send_pushover a timeout and close its connections, so an alert that fails or hangs can't turn a signup into a 500.
- Make USERCHECK_API_KEY optional. Without it, the remote check is skipped and signups are checked against the local blocklists only, so a deploy with an old env file still starts.
- Handle 200 responses whose body isn't a JSON object, or whose did_you_mean isn't a string, by alerting and allowing the signup instead of raising AttributeError.
A domain with mx False may just have had a brief DNS failure. Caching that verdict for the full 30 days blocked every address on the domain until the entry expired.
Lookups run inside the signup request, so a slow UserCheck tied up a worker for up to about 21s (two 10s timeouts plus the 429 retry delay). A healthy lookup takes well under a second, and slower ones fail open anyway.
validate_email_allowed already extracts and lowercases the domain to check the blocklists. check_email_domain was parsing it again with slightly different code, so the cache key and the blocklist check could drift apart. Parse it in one place and pass it in.
rate-limited lookups while credits remain

The email lookup ran during form validation, before the captcha was checked. Unauthenticated POSTs could spend metered UserCheck credits and force 429s, which the form treated as "allow without the check".

- Check the captcha first, so every lookup costs a solved captcha. On an hCaptcha outage, re-render an unbound form so the errors shown don't trigger validation.
- When lookups still get a 429 after the retry, use /status to tell the per-second limit from an exhausted quota. While credits remain, ask the user to retry instead of allowing the signup. When the quota is used up, keep allowing signups and send the alert.
failed signup captcha

- The profile form ran the UserCheck lookup in clean_email, before clean() checked the current password, so any logged-in session could spend credits by posting new addresses. Run only the local checks there, and the remote check once the password is confirmed.
- With the captcha now checked before form validation, a failed captcha redirected to "activation email sent" even when the form had errors. Re-render the form with "Please complete the captcha and try again." instead, without validating it.
Every 400 hard-rejected the address, with no alert. A 400 caused by the request itself, or by an API change, would silently block real users. Reject only on the error UserCheck documents for a malformed address. Alert on any other 400 and allow the signup, as for other unexpected responses.
@bpepple
bpepple merged commit edc6223 into master Oct 17, 2026
2 checks passed
@bpepple
bpepple deleted the 656-check-signup-emails-with-the-usercheck-api-directly-instead-of-through-rapidapi branch October 17, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check signup emails with the UserCheck API directly instead of through RapidAPI

1 participant