Canceled subscriptions are not being backfilled

[th0rgall]

I added this integration to Supabase recently, and noticed that the count of subscriptions objects was off. It was virtually equal to our confirmed count of active subscriptions, but it included just a few canceled subscriptions.

This leads me to suspect that the initial subscriptions backfill did not include canceled objects, which seems plausible considering that the listing API for subscriptions mentions:

By default, returns a list of subscriptions that have not been canceled. In order to list canceled subscriptions, specify status=canceled.

My guess that this API is used by this engine without status parameter, and that the engine does not perform a second pass to list and backfill canceled subscriptions. The few canceled subscriptions that did get synced came from webhook events.

[janpio]

I can confirm that we are not seeing all the subscriptions being added to the subscriptions table. Roughly the number of active and in-trial ones.

We mostly looked at this integration because it would enable us to avoid having to backfill the full subscription history via the API - seems it is not ready for that yet.

[kennethkeim]

Seeing the same thing for my account. It's a non-issue for me since I'm not in prod yet, but it would be good to document or fix it.

[quyumkehinde]

Since #338 has been sitting for a while, here's a workaround that doesn't need an engine patch: run a second pass over GET /v1/subscriptions?status=canceled and upsert the results into the existing tables. Same row shape, so no schema change; mind starting_after pagination if the cancelled set is large.

Disclosure: I maintain Driftless, an open-source alternative that lists with status=all on backfill and can import an existing sync-engine database, if anyone wants a different route.