Skip to content

v10 slice 5: example app on v10, test seam, CI, docs, version 10.0.0 #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dmurphy5 wants to merge 1 commit into from
Open

v10 slice 5: example app on v10, test seam, CI, docs, version 10.0.0 #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/node.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,42 @@ jobs:
run: |
cd example/RNBGUExample/android
./gradlew :react-native-background-upload:testDebugUnitTest

# Host-side unit tests for the pure Swift half of the iOS module.
# They run on macOS with Xcode's Swift toolchain. ios/Package.swift needs
# swift-tools-version 5.9. The macos-15 image ships Xcode 16 or later
# (Swift 6 or later), which meets that.
#
# This job does not build the example app. Do that by hand before a
# release, because a simulator build is too slow for every push:
# cd example/RNBGUExample/ios && pod install
# xcodebuild -workspace RNBGUExample.xcworkspace -scheme RNBGUExample \
# -sdk iphonesimulator -destination 'generic/platform=iOS Simulator' \
# -configuration Debug build
ios-host-tests:
runs-on: macos-15
timeout-minutes: 30
if: "!contains(github.event.head_commit.message, '[skip ci]')"

steps:
- name: checkout
uses: actions/checkout@v4

- name: swift version
id: swift
run: |
swift --version
echo "version=$(swift --version 2>&1 | head -n 1 | shasum | cut -c1-12)" >> "$GITHUB_OUTPUT"

- name: cache swift build
uses: actions/cache@v4
with:
path: ios/.build
key: swiftpm-${{ runner.os }}-${{ steps.swift.outputs.version }}-${{ hashFiles('ios/*.swift', 'ios/Tests/**/*.swift') }}
restore-keys: |
swiftpm-${{ runner.os }}-${{ steps.swift.outputs.version }}-

- name: ios host tests
run: |
cd ios
swift test
193 changes: 146 additions & 47 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,12 @@
## 10.0.0 (unreleased)
## 10.0.0

The library now owns a durable request queue. A consumer describes each
request kind one time with `define()`, enqueues instances with `mutate()`,
and receives every outcome through the definition's handlers, on this launch
or a later one. Outcomes are journaled natively before JS hears about them
and acknowledged only after the handler's promise resolves. See the README's
"Usage" and "Reliable delivery" sections.

This release is built in slices. The JS layer, the codegen spec, and native
stubs land first; the Android and iOS queues follow. Until they land, every
queue method rejects with `E_NOT_IMPLEMENTED`.
"Usage" and "Reliable delivery" sections. "Upgrading from v9" there covers
the migration.

Breaking:
- **`startUpload` and `getAllUploads` are removed.** `define()` + `mutate()`
Expand All @@ -29,63 +26,165 @@ Breaking:
- **`progress` carries `{ id, bytesSent, totalBytes }`** instead of a
percentage.
- **`configure()` must be called at boot, after every `define()`.** It starts
the replay of journaled outcomes. It also takes `lifetimeMs`, `retry`, a
`headers` provider that runs at `mutate()`, `maxVarsBytes` (default 1 MB),
and `enqueueTimeoutMs` (default 10 s): `mutate()` rejects with a named
error and warns when the native write has not settled by then, so a native
bug cannot hang a caller in silence.
the replay of journaled outcomes. Its `android` notification options are
unchanged from v9.
- **`ErrorKind` gains `'truncated'`.** With a `response` parser set and a body
over the 1 MB cap, `onError` fires with it instead of `onSuccess`.
- **Removed exports:** `startUpload`, `cancelUpload`, `removeUpload`,
`getAllUploads`, the public `getUnacknowledgedEvents` / `ackEvents`, the
`error` / `completed` / `cancelled` event names, and the
`ProgressData`, `CompletedData`, `ErrorData`, `CancelledData`, `EventData`,
`TerminalEventData`, `JournaledEvent`, `UploadSnapshot`, `UploadOptions`,
`ChunkedUploadOptions`, `StartUploadOptions`, `AndroidOnlyUploadOptions`,
`RawUploadOptions`, and `UploadId` types. The native `startChunkedUpload` is
internal: a descriptor with `parts` routes to it.

Added:
- **`createUploadClient()`**: builds a client with its own definitions and
settings. The default export is one client.
- **`define({ key, request, response?, onSuccess?, onError? })`**: `vars`
infer from the `request` parameter, the handler data type from the
`response` return. `response(raw, vars)` also receives the entry's `vars`;
a one-argument parser such as `schema.parse` still fits. A duplicate key
replaces the definition and warns in development.
a one-argument parser such as `schema.parse` still fits. Without
`response`, `onSuccess` receives the `RawResponse`. A duplicate key
replaces the definition and warns in development. A `request` that takes
no parameter gives a `mutate()` that takes no arguments.
- **`mutate(vars, { id? })`**: runs `request(vars)` once, merges the configured
headers under the descriptor's, validates the descriptor (at most one of
`data` / `form` / `file`, none for a bodiless DELETE; no body on a GET;
`parts` only with `file`; parts must tile the file; no field outside the
descriptor shape), defaults `expiresAt` to now +
`lifetimeMs`, and resolves when the entry is durable. `vars` is any
JSON-serializable object, so generated API request types work as they are;
`mutate()` rejects vars or `data` that do not serialize (a cycle, a function,
a BigInt) and caps `vars` at `configure().maxVarsBytes`, 1 MB by default. A
definition whose `request` takes no vars calls `mutate()` with no arguments.
It resolves after the row and every staged body copy are on disk, so the
caller may delete its source file then; native failures reject with
`E_INVALID`, `E_RUNNING`, `E_FILE_MISSING`, or `E_STORAGE`.
`data` / `form` / `file`; no body on a GET; `parts` only with `file`; parts
must tile the file from 0; no field outside the descriptor shape, with a
"did you mean" hint), defaults `expiresAt` to now + `lifetimeMs`, and
resolves when the entry is durable: after the row and every staged body
copy are on disk, so the caller may delete its source file then. `vars` is
any JSON-serializable object, so generated API request types work as they
are; `mutate()` rejects vars or `data` that do not serialize (a cycle, a
function, a BigInt) and caps `vars` at `configure().maxVarsBytes`, 1 MB by
default. `id` defaults to a UUID.
- **Same id, again.** The body is `data`, `form`, `file`, or `parts`, and a
different `url` or `method` counts as a different body. Same body: resume,
with new headers, `expiresAt`, and `vars`. Different body on an entry that
is not running: replace and reopen; it settles once more. Different body
on a running entry: reject `E_RUNNING`. Cancelled but not yet acknowledged:
a fresh generation. Completed but not yet acknowledged: the journaled
outcome is emitted again and nothing is re-sent.
- **Request bodies**: JSON (`data`), multipart (`form`), whole file (`file`),
and chunked (`file` + `parts`). All under one entry shape and one id.
and chunked (`file` + `parts`). All under one entry shape and one id. A
bodiless request (a GET, a DELETE, a POST whose meaning is in the URL) sets
none. `data: null` sends the JSON body `null`.
- **`configure()` options**: `lifetimeMs` (default 14 days), `retry`
(backoff base 1 s, max 2 h, jitter 0.2; `terminalHttp.exempt` default
`[404]`), a `headers` provider that runs at `mutate()`, `maxVarsBytes`
(default 1 MB), `enqueueTimeoutMs` (default 10 s: `mutate()` rejects with
a named error and warns when the native write has not settled by then, so
a native bug cannot hang a caller in silence), and the v9 `android`
notification options.
- **Queue control**: `pause()` / `resume()` for the whole queue, `cancel(id)`,
`setWifiOnly(enabled)`, and `updateHeaders(patch)` to re-auth entries
parked on 401 or 403. Each `updateHeaders()` call bumps a header
generation, so a 401 from an attempt sent under older headers re-issues at
once instead of parking.
- **`getRequests(filter?)`**: synchronous, from native's in-memory index.
Returns every entry native has not yet forgotten, so completed and
cancelled rows appear until their ack and `error` rows until `cancel()` or
a same-id `mutate()`. `filter` is `{ key?, id? }`.
- **`RequestRow`**: `{ id, key, vars, state, bytesSent, totalBytes, attempts,
updatedAt, nextAttemptAt? }`. `state` is `queued`, `running`,
`awaiting-auth`, `paused`, `completed`, `error`, or `cancelled`.
`nextAttemptAt` (epoch ms) is set while an entry waits out a retry
backoff. `attempts` counts the current generation.
- **`Meta`** for handlers: `{ id, key, at, attempts, requestId?, deliveries }`.
`deliveries` counts deliveries that reached a JS listener: 1 on the first,
+1 per replay. A handler that keeps throwing sees it grow; the library
never gives up on its own, so the app decides a poison policy.
- **Events**: `state` (a full `RequestRow` per transition, plus
`reason: 'unhandled-key'` for an outcome whose key has no definition),
`progress` (byte-weighted across a chunked upload's parts, throttled to
1 s in the foreground), and `attempt` (one row per HTTP attempt with
`requestId`, `httpCode`, a 4 KB response body, `responseHeaders`, and
`errorKind`). `attempt.outcome` is `completed` or `error`; pause, cancel,
and supersede emit none. `attempt` events are live-only, never journaled.
- **Delivery rules**: dedupe by event id; the outcomes of one id deliver in
order, one handler at a time; an outcome for an id waits for that id's
in-flight `mutate()`; an outcome whose key has no definition stays
unacknowledged and reaches `state` listeners with `reason: 'unhandled-key'`;
a handler that has not settled after 30 s logs a warning. No ordering is
promised between different ids.
- **`Meta.deliveries`**: counts deliveries that reached a JS listener: 1 on
the first, +1 per replay. A handler that keeps throwing sees it grow; the
library never gives up on its own, so the app decides a poison policy.
- **`RequestRow.nextAttemptAt`**: epoch ms, set while an entry waits out a
retry backoff. `getRequests()` returns every entry native has not yet
forgotten, so completed and cancelled rows appear until their ack.
- **`pause()` / `resume()`** for the whole queue, **`updateHeaders(patch)`** to
re-auth parked entries, and the **`attempt`** event with one row per HTTP
attempt. Its `outcome` is `completed` or `error`; pause, cancel, and
supersede emit none.

Removed:
- `startUpload`, `startChunkedUpload` (native), `cancelUpload`,
`removeUpload`, `getAllUploads`, the public `getUnacknowledgedEvents` /
`ackEvents`, and the `progress` / `error` / `completed` / `cancelled` event
names, with their `ProgressData`, `CompletedData`, `ErrorData`,
`CancelledData`, `EventData`, `TerminalEventData`, `JournaledEvent`,
`UploadSnapshot`, `UploadOptions`, `ChunkedUploadOptions`,
`StartUploadOptions`, `AndroidOnlyUploadOptions`, `RawUploadOptions`, and
`UploadId` types.
a handler that has not settled after 30 s logs a warning and keeps
waiting; a malformed journal entry is dropped with a warning. No ordering
is promised between different ids.
- **Error codes** on the native promises: `E_INVALID` (input native cannot
send: a non-http(s) URL, a header name or value the platform HTTP client
rejects, a GET with a body, parts that do not tile the file),
`E_RUNNING`, `E_FILE_MISSING`, and `E_STORAGE`. `cancel()` rejects with
`E_STORAGE` and changes nothing when the journal or the store cannot be
written. `updateHeaders()` rejects with `E_INVALID` for a bad header name
or value.
- **`X-Request-Id`** on every attempt, minted per attempt. `Meta.requestId`
and `attempt.requestId` carry it.
- **`chunkPlan`** stays a module export and is also on the client.
- **Testing seam**: `createUploadClient({ native })` takes a fake native
module, and `createFakeNative()` in `src/testing` builds one that keeps
rows, journals outcomes, and acks, with `settle()`, `failNext()`,
`seedRows()`, and `seedUnacknowledged()` to script native behavior. The
real validation and delivery run on top of it.

Native:
- **Android queue.** One durable store under `files/rnbgupload-chunked/`,
one directory per entry (`entry.json` plus the staged body), written
tmp + fsync + rename. The journal of settled outcomes moves to
`files/rnbgupload-settled/`. Queue settings (`wifiOnly`, `paused`, header
generation, retry defaults) persist next to the entries. An in-memory
index serves `getRequests()`. One WorkManager worker per id; a long
backoff waits on a separate wake job so a same-id `mutate()` or
`updateHeaders()` can run the entry at once. The backoff streak is stored
on the entry, so the wait grows toward 2 h across runs. A run stopped by
WorkManager's timeout takes one more backoff step instead of restarting
at once. The global cap of 4 concurrent requests and the window of 3
parts per chunked upload carry over from v9. A boot sweep repairs a settle
whose entry save was lost, applies a cancel whose save was lost, and
finishes a pause or resume cut short by process death. A settle whose
journal write fails holds the record in memory and retries the write
(5 s, doubling to 10 min). Response bodies are capped at 1 MB while they
stream, counted in UTF-8 bytes and cut on a character boundary.
- **iOS queue.** The v9 chunked store is generalized in place into the queue
store. Every body is staged to a file in the entry directory, because a
background `URLSession` uploads from files only. Store, journal, and task
map writes are tmp + fsync + rename, and the directories are excluded from
backup. Retries are delayed session tasks (`earliestBeginDate`) for simple
entries and for chunked parts, so a backoff keeps running while the app is
suspended or dead. Each attempt owns its task through the task
description, so a stale completion cannot settle a newer attempt. At
relaunch, an entry whose task is gone but whose completion may still be
pending waits up to 10 s for the replay before it re-issues, so a request
the daemon finished while the app was dead is not sent twice; the
background completion handler is released as soon as that replay lands.
Two background sessions, one that allows cellular and one Wi-Fi only, each
with `httpMaximumConnectionsPerHost = 4`. A system cancel, including a
force-quit, is a transient failure: no outcome, the entry retries. A
multipart body always uses its own `Content-Type`, because a caller's has
no boundary. Response bodies are capped at 1 MB while they stream. The v9
per-part budget of 3 HTTP rejections is gone; the retry policy decides.
- **v9 import, both platforms.** The first v10 launch imports each v9 journal
entry as a read-only settled row with key `legacy`, the v9 upload id, and
0/0 bytes. Nothing is delivered for them: the app reads them with
`getRequests({ key: 'legacy' })` and calls `cancel(id)`. In-flight v9 work
is cancelled (Android WorkManager rows, iOS session tasks). v9 chunked
manifests and their bytes stay: a same-id `mutate()` with the same parts
resumes from the accepted parts, and one with different parts starts over
on the kept bytes. Wi-Fi only starts off.
- **Platform limits.** Android, API 31 and later: a WorkManager run started
from the background usually cannot start its foreground service and then
has JobScheduler's limit of about 10 minutes; a single body that does not
finish in that time restarts from byte 0 after a backoff, so large bodies
need `parts`. Android: the store and journal hold auth headers and staged
bodies, so the host app sets `android:allowBackup="false"` or excludes the
two directories. iOS: no global in-flight cap, and the AppDelegate
`handleEventsForBackgroundURLSession` hook is required for outcomes to be
journaled when the system relaunches the app.

Fixed:
- iOS: a chunked part's retry backoff no longer stalls while the app is
dead. v9 ran the cooldown on an in-process timer.
- iOS: task-map keys whose completion never arrives are pruned at the end of
the relaunch grace wait and when their entry is gone. v9 kept them forever.

## 9.0.0

Expand Down
Loading
Loading