Skip to content

integrity-blob: move the Azure backend to object_store (RUSTSEC-2026-0275) - #55

Merged
captjt merged 2 commits into
mainfrom
mic-84-replace-the-legacy-azure-sdk-in-integrity-blob-azure_core
Oct 28, 2026
Merged

captjt merged 2 commits into
mainfrom
mic-84-replace-the-legacy-azure-sdk-in-integrity-blob-azure_core

Conversation

@captjt

@captjt captjt commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Fixes MIC-84. blob-azure was built on the legacy Azure SDK (azure_storage / azure_storage_blobs 0.21). Its azure_core 0.21 logs each request's authorization header, a replayable Shared Key signature, at debug and trace level (RUSTSEC-2026-0275). The legacy SDK is unsupported, and Microsoft closed the upstream issue as not planned, so the fix is to move off it.

Change

integrity-blob/src/blob_store/azure_blob.rs now uses object_store 0.14 (Apache Arrow, MIT/Apache-2.0) and its MicrosoftAzureBuilder, as MIC-84 recommends.

  • Same API and config. AzureBlob::new(account, key, container) and the FFI's ig_blob_store_azure_blob_new are unchanged, and auth is still the storage account key. No FFI ABI or header change.
  • Same behavior. A 404 returns Ok(None) from get and false from exists. put writes a block blob with content type text/plain, overwriting any existing blob. Other errors are logged at warn and returned.
  • Timeouts and retries. object_store limits each whole request to 30s by default. The legacy SDK had no such limit, and a large blob could exceed it, so init disables that limit. object_store's 5s connect timeout and its retries on 5xx, 429 and timeouts stay on (up to 10 retries, at most 3 minutes).
  • One deliberate difference. A key that isn't valid base64 now fails at init. Before, the first request failed.

Cargo.lock drops azure_storage, azure_storage_blobs, azure_svc_blobstorage and RustyXML. It adds object_store 0.14.2 and a few small crates (rand 0.10, getrandom 0.4, base64 0.23, itertools 0.15, quick-xml 0.31 → 0.41). reqwest 0.13 was already in the tree.

Logging check (MIC-84's "verify before switching")

  • object_store's only log lines are info-level retry messages (status code, backoff, transport error). It never logs request headers and doesn't turn on reqwest's connection_verbose.
  • The builder's Debug prints only the account and container.
  • Error messages show method, URI, status and response body. With account-key auth the URI has no secret in it.

Still on azure_core 0.21

signer-akv (integrity-signer: azure_identity and azure_security_keyvault 0.21) still depends on azure_core 0.21, so azure_core stays in this repo's Cargo.lock. With blob-azure,blob-s3,blob-gcs, the set Guardian's integrity-service enables, cargo tree -i azure_core finds nothing. Moving signer-akv off the legacy SDK is separate work.

Testing

Against Azurite 3.35.0. I ran the same harness on main and on this branch, with each copy of AzureBlob pointed at the emulator for the test only, and trace logging on for every crate:

main (azure_storage 0.21) this branch (object_store 0.14)
get / exists, missing blob None / false None / false
put, put again, put with wrong CID ok, ok, error ok, ok, error
exists / get, stored blob true / "Hello World" true / "Hello World"
8 MiB put/get round trip equal equal
stored content type text/plain text/plain
wrong key: get / exists 403 error 403 error
key that isn't valid base64: init ok error
SharedKey signatures in the logs 16 0

On this branch, the only log lines containing "authorization" are Azurite's 403 body ("Make sure the value of the Authorization header is formed correctly").

Locally on Rust 1.94.1:

  • just fmt-check (also clean under nightly rustfmt for integrity-blob)
  • just lint, just lint-docs, just test, just build (--all-features)
  • cargo check --locked -p integrity-ffi --features blob-all,signer-all
  • cargo clippy -p integrity-blob --no-default-features --features blob-azure --all-targets -- -Dwarnings. just lint uses default features, so it doesn't compile this backend.

Not run:

  • The wasm32 check: the target isn't installed for 1.94.1 here. object_store only applies to non-wasm targets, and CI runs that check.
  • A real Azure storage account (step 4 of MIC-84).
  • Trivy on the final lockfile (CI covers it).
  • cargo clippy -p integrity-blob --features blob-all --all-targets fails on main too: the s3.rs tests use #[tokio::test], but tokio isn't a dev-dependency.

After merge

MIC-84 steps 2 and 3: cut an integrity release, bump the tag in Guardian's services/integrity-service, and confirm cargo deny check advisories no longer reports RUSTSEC-2026-0275. Deployments that ran the Azure backend with debug or trace logging for the Azure crates should treat those logs as holding credentials and rotate the storage account key.

🤖 Generated with Claude Code

captjt and others added 2 commits October 28, 2026 12:26
…0275)

azure_core 0.21, which azure_storage / azure_storage_blobs 0.21 bring in,
logs each request's authorization header at debug and trace level. The
legacy SDK is unsupported upstream, so the fix is to move off it.

AzureBlob now uses object_store's MicrosoftAzureBuilder with the same
account-key auth, so callers and the FFI are unchanged. A missing blob
still maps to None, and puts still store a text/plain block blob.
object_store's 30s whole-request timeout is disabled so large blobs
behave as before; its connect timeout and retries are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@captjt captjt self-assigned this Oct 28, 2026
@captjt
captjt requested review from TJbrunk and cameronfyfe October 28, 2026 17:25
@captjt
captjt marked this pull request as ready for review October 28, 2026 17:26

@TJbrunk TJbrunk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@captjt
captjt merged commit ca6b7f6 into main Oct 28, 2026
6 checks passed
@captjt
captjt deleted the mic-84-replace-the-legacy-azure-sdk-in-integrity-blob-azure_core branch October 28, 2026 18:03
@captjt captjt mentioned this pull request Oct 28, 2026
captjt added a commit that referenced this pull request Oct 28, 2026
Release prep for `v0.0.20`, following CONTRIBUTING.md.

- **CHANGELOG:** the `Unreleased` entries move to `## [0.0.20] -
2026-10-28`. Since `v0.0.19` that is #55: the `blob-azure` backend moves
from the legacy Azure SDK to `object_store`, clearing RUSTSEC-2026-0275
(MIC-84). `Unreleased` stays at the top, empty. The comparison links are
advanced: `[Unreleased]` compares `v0.0.20...main`, and `[0.0.20]`
compares `v0.0.19...v0.0.20`.
- **Version:** `[workspace.package].version` moves from `0.0.19` to
`0.0.20`. `Cargo.lock` changes only the 12 workspace crates' versions.
The FFI ABI version (0.5.0) is a separate contract and is unchanged; #55
didn't change the ABI.

These pass on Rust 1.94.1:
- `./scripts/check-changelog.sh v0.0.20`
- `just fmt-check`, `just lint`, `just test` (90 passed)
- `cargo check --locked -p integrity-ffi --features blob-all,signer-all`

**After merging:** tag the merge commit on `main` and push the tag. That
starts the native FFI release workflow, which checks for the `##
[0.0.20]` heading and then builds and publishes the artifacts.

```bash
git tag v0.0.20 <merge-commit>
git push origin v0.0.20
```

Then Guardian's `services/integrity-service` bumps its integrity tag to
`v0.0.20`, and `cargo deny check advisories` should no longer report
RUSTSEC-2026-0275 (MIC-84 step 3).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants