Skip to content

HYPERFLEET-1340 - feat: support custom CA certificates for JWKS endpoint TLS#292

Open
pnguyen44 wants to merge 2 commits into
openshift-hyperfleet:mainfrom
pnguyen44:HYPERFLEET-1340
Open

HYPERFLEET-1340 - feat: support custom CA certificates for JWKS endpoint TLS#292
pnguyen44 wants to merge 2 commits into
openshift-hyperfleet:mainfrom
pnguyen44:HYPERFLEET-1340

Conversation

@pnguyen44

@pnguyen44 pnguyen44 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

HYPERFLEET-1340

When deployed inside a Kubernetes cluster, the JWT handler cannot fetch JWKS from https://kubernetes.default.svc.cluster.local/openid/v1/jwks because Go's default HTTP client does not trust the cluster CA. This adds a jwk_cert_ca_file config field that lets operators point to a custom CA certificate (e.g. /var/run/secrets/kubernetes.io/serviceaccount/ca.crt), enabling TLS-verified JWKS fetches without resorting to the static-file workaround.

Changes

  • Added jwk_cert_ca_file to JWTIssuerConfig with config validation ensuring it is only set when jwk_cert_url is also present
  • When jwk_cert_ca_file is set, buildKeyfunc constructs a custom HTTP client with a TLS config that trusts the specified CA, used for both URL-only and combined (file + URL) JWKS configurations
  • Helm configmap template renders jwk_cert_ca_file when present, with matching values schema and documentation updates

Test plan

  • Unit tests added/updated
  • make test-all passes
  • make lint passes
  • Helm chart changes validated with make test-helm (if applicable)
  • Deployed to a development cluster and verified (if Helm/config changes)
  • E2E tests passed (if cross-component or major changes) -- N/A

@openshift-ci openshift-ci Bot requested review from mbrudnoy and sherine-k July 10, 2026 19:00
@openshift-ci

openshift-ci Bot commented Jul 10, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mbrudnoy for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 9e4dcdbe-75d9-40b1-b56f-6919ab542d70

📥 Commits

Reviewing files that changed from the base of the PR and between 48db2a4 and 1f16762.

📒 Files selected for processing (2)
  • pkg/auth/jwt_handler.go
  • pkg/auth/jwt_handler_test.go
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • openshift-hyperfleet/architecture (manual)
  • openshift-hyperfleet/hyperfleet-api (manual)
  • openshift-hyperfleet/hyperfleet-sentinel (manual)
  • openshift-hyperfleet/hyperfleet-adapter (manual)
  • openshift-hyperfleet/hyperfleet-broker (manual)
🚧 Files skipped from review as they are similar to previous changes (2)
  • pkg/auth/jwt_handler_test.go
  • pkg/auth/jwt_handler.go

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added support for an optional CA certificate file (jwk_cert_ca_file) to trust TLS when fetching JWKS.
    • Configuration output now reports whether a custom CA is configured for each JWT issuer.
  • Bug Fixes

    • Added cross-field validation: if jwk_cert_ca_file is set, jwk_cert_url must also be provided.
  • Documentation

    • Updated authentication and configuration examples to include the new jwk_cert_ca_file option.
  • Tests

    • Expanded unit and Helm template coverage for the new TLS/CA behavior.

Walkthrough

Adds optional custom CA certificate support for TLS JWKS retrieval. JWT configuration validates that jwk_cert_ca_file requires jwk_cert_url. The JWT handler builds CA-aware HTTP clients for URL-only and combined file-plus-URL verification. Helm schemas, templates, examples, documentation, configuration dumps, and tests now expose and validate the field.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant JWTHandler
  participant CAFile
  participant JWKSClient
  participant JWKS_Endpoint
  JWTHandler->>CAFile: Read CA certificate PEM
  JWTHandler->>JWKSClient: Configure TLS roots and JWKS storage
  JWKSClient->>JWKS_Endpoint: Fetch JWKS over TLS
  JWKS_Endpoint-->>JWKSClient: Return JWKS keys
  JWKSClient-->>JWTHandler: Provide signing keys
Loading
🚥 Pre-merge checks | ✅ 10 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (10 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: custom CA support for JWKS endpoint TLS.
Description check ✅ Passed The description matches the changeset, covering the new CA config, validation, TLS client behavior, and Helm/docs updates.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Sec-02: Secrets In Log Output ✅ Passed No non-test/non-example log statements contain token/password/credential/secret fields or interpolations; new JWT logs only use url/ca_file. CWE-532 not triggered.
No Hardcoded Secrets ✅ Passed No hardcoded creds, embedded-auth URLs, or long base64 blobs in touched files; the only secret-like literal is a test HMAC fixture, not production.
No Weak Cryptography ✅ Passed No banned primitives or secret comparisons in touched files; the new CA path uses crypto/tls and crypto/x509 only, with no SHA1/MD5/DES/RC4 or custom crypto.
No Injection Vectors ✅ Passed Touched code adds CA-file TLS JWKS support and YAML/docs only; no exec.Command, template.HTML, yaml.Unmarshal, or SQL concatenation in changed paths (CWE-89/78/79/502).
No Privileged Containers ✅ Passed No privileged settings in changed manifests; chart defaults are non-root/false, and Dockerfile root is build-stage only before USER 65532.
No Pii Or Sensitive Data In Logs ✅ Passed No PII, session IDs, raw bodies, or credential-bearing logs were added; the new JWT CA logs only emit URL/path metadata.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
✨ Simplify code
  • Create PR with simplified code

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
pkg/auth/jwt_handler.go (2)

175-261: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

buildKeyfunc exceeds decomposition threshold; CA-storage logic duplicated across branches.

The function spans ~86 lines with branching on hasFile/hasURL/CA-presence in two places (196-222 and 235-247) that each independently decide between CA-aware storage and a plain/memory fallback. Extracting a shared helper (e.g. remoteJWKSStorage(ctx, url, caFile string) (jwkset.Storage, error)) would remove the duplication and shrink buildKeyfunc below the flagged size/branch threshold.

♻️ Sketch of consolidated helper
func remoteJWKSStorage(ctx context.Context, url, caFile string) (jwkset.Storage, error) {
	if caFile == "" {
		return jwkset.NewMemoryStorage(), nil
	}
	return newStorageWithCA(ctx, url, caFile)
}

Both branches would then call remoteJWKSStorage(ctx, issuer.JWKCertURL, issuer.JWKCertCAFile) instead of duplicating the if issuer.JWKCertCAFile != "" check.

As per path instructions, "Functions >50 lines or >5 branching paths — flag for decomposition."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/auth/jwt_handler.go` around lines 175 - 261, Decompose buildKeyfunc by
extracting the repeated CA-aware remote storage selection into a helper such as
remoteJWKSStorage(ctx, url, caFile string) (jwkset.Storage, error). Have it
return newStorageWithCA when caFile is set and an appropriate memory storage
otherwise, then use it in both URL-only and combined file/URL branches while
preserving existing error context and behavior.

Source: Path instructions


285-306: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Split buildKeyfunc into smaller helpers. It mixes source selection, CA-backed storage setup, and keyfunc construction in one 80+ line function, and the URL+CA wiring is duplicated across branches.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/auth/jwt_handler.go` around lines 285 - 306, Refactor buildKeyfunc into
focused helpers: isolate key-source selection, CA-backed HTTP client/storage
setup, and final keyfunc construction, while preserving existing behavior.
Consolidate duplicated URL-and-CA wiring through a shared helper that uses
buildHTTPClientWithCA, and have buildKeyfunc orchestrate these helpers with
clear error propagation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@charts/values.schema.json`:
- Around line 166-169: The Helm schema currently allows jwk_cert_ca_file without
its required jwk_cert_url. Update the relevant schema object containing
jwk_cert_ca_file with an item-level if/then rule: if jwk_cert_ca_file is
present, then require jwk_cert_url, matching the validation enforced by server
configuration.

In `@docs/authentication.md`:
- Line 181: Update the `jwk_cert_ca_file` entry in the authentication
configuration table to state that it is valid only when `jwk_cert_url` is
configured, rejected when used only with `jwk_cert_file`, and must reference an
existing PEM file available at the configured path inside the API container.

In `@pkg/config/dump.go`:
- Around line 82-83: Handle the ignored error in formatIssuers by replacing the
fmt.Fprintf call with a non-error-returning write, or propagate its error
through formatIssuers and DumpConfig; ensure issuer output cannot be silently
incomplete.

In `@scripts/test-helm.sh`:
- Around line 149-151: Update the CA path assertion in the test script to use
fixed-string matching rather than regular-expression matching. Modify
assert_contains or add a helper using grep -Fq --, and ensure the assertion for
/var/run/secrets/kubernetes.io/serviceaccount/ca.crt invokes that fixed-string
behavior.

---

Nitpick comments:
In `@pkg/auth/jwt_handler.go`:
- Around line 175-261: Decompose buildKeyfunc by extracting the repeated
CA-aware remote storage selection into a helper such as remoteJWKSStorage(ctx,
url, caFile string) (jwkset.Storage, error). Have it return newStorageWithCA
when caFile is set and an appropriate memory storage otherwise, then use it in
both URL-only and combined file/URL branches while preserving existing error
context and behavior.
- Around line 285-306: Refactor buildKeyfunc into focused helpers: isolate
key-source selection, CA-backed HTTP client/storage setup, and final keyfunc
construction, while preserving existing behavior. Consolidate duplicated
URL-and-CA wiring through a shared helper that uses buildHTTPClientWithCA, and
have buildKeyfunc orchestrate these helpers with clear error propagation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 517d74d0-134e-477e-b723-4c96ff92e902

📥 Commits

Reviewing files that changed from the base of the PR and between 1a6df40 and 47a1bac.

📒 Files selected for processing (12)
  • charts/templates/configmap.yaml
  • charts/values.schema.json
  • charts/values.yaml
  • configs/config.yaml.example
  • docs/authentication.md
  • pkg/auth/jwt_handler.go
  • pkg/auth/jwt_handler_test.go
  • pkg/config/dump.go
  • pkg/config/dump_test.go
  • pkg/config/server.go
  • pkg/config/server_test.go
  • scripts/test-helm.sh
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • openshift-hyperfleet/architecture (manual)
  • openshift-hyperfleet/hyperfleet-api (manual)
  • openshift-hyperfleet/hyperfleet-sentinel (manual)
  • openshift-hyperfleet/hyperfleet-adapter (manual)
  • openshift-hyperfleet/hyperfleet-broker (manual)

Comment thread charts/values.schema.json
Comment thread docs/authentication.md Outdated
Comment thread pkg/config/dump.go
Comment thread scripts/test-helm.sh
@hyperfleet-ci-bot

hyperfleet-ci-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown

Risk Score: 1 — risk/low

Signal Detail Points
PR size 403 lines (>200) +1
Sensitive paths none +0
Test coverage Tests cover changed packages +0

Computed by hyperfleet-risk-scorer

Comment thread pkg/auth/jwt_handler.go
Comment thread pkg/auth/jwt_handler.go Outdated
Comment thread pkg/auth/jwt_handler_test.go Outdated
@tirthct

tirthct commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

/retest

@openshift-ci

openshift-ci Bot commented Jul 10, 2026

Copy link
Copy Markdown

@pnguyen44: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/images 1f16762 link true /test images
ci/prow/helm-test 1f16762 link true /test helm-test
ci/prow/unit 1f16762 link true /test unit
ci/prow/lint 1f16762 link true /test lint

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants