Skip to content

chore: merge upstream cloud-custodian and restamp 0.9.53a1+bf0b50dbc - #24

Merged
jtroup merged 35 commits into
stacklet/integrationfrom
elmo/firm-toad
Sep 16, 2026
Merged

jtroup merged 35 commits into
stacklet/integrationfrom
elmo/firm-toad

Conversation

@jtroup

@jtroup jtroup commented Sep 15, 2026 •

Copy link
Copy Markdown

ENG-8864

what

Merges upstream cloud-custodian e4c099e03..bf0b50dbc into
stacklet/integration and restamps the workspace onto the new
upstream hash. The base and alpha are unchanged at 0.9.53a1; only
the local version segment moves, to +bf0b50dbc. All thirteen
workspace packages are stamped uniformly, uv.lock agrees with
them, and no reference to the old hash remains.

Two commits: the upstream merge on its own, then the stamp. The
merge's combined diff (git show --cc) is empty, so there is no
hand-written content in it to review — every conflict was resolved
by taking upstream's side verbatim.

Conflicts were tests/test_tags.py,
tools/c7n_kube/c7n_kube/query.py,
tools/c7n_kube/tests/test_query_pagination.py and uv.lock. In
the three source files upstream was a strict superset of the branch:
the k8s pagination fix (cloud-custodian#11081) picked up a second
continuation-token key in cloud-custodian#11083, and test_tags.py gained a
ClientError import. uv.lock was rebuilt by just append-hash bf0b50dbc.

Upstream content picked up:

  • core: a --vars-file parameter for injecting policy variables.
  • aws: tag actions tolerate resources deleted mid-run;
    cloudwatch-dashboard tagging moves to the native cloudwatch APIs;
    a new apigwv2 route resource for authorization-type filtering; a
    deployments filter on bedrock-custom-model; kms and route53
    actions now skip AWS-managed resources.
  • gcp: audit mode for dataproc-clusters, and an update action on
    kms-cryptokey.
  • k8s: a custom-resource type for dynamic CRD discovery.
  • releng: a python-requirements dependency group bump.

No Stacklet code changed.

why

Keeps stacklet/integration current with upstream so the prerelease
builds carry the latest fixes.

git log stacklet/integration..HEAD lists 34 commits, but only the
ten summarised above are new content. The previous upstream merge
(#23) was squash-merged, so upstream history never entered the
ancestry and the merge base was still a83dda5c9 — everything #23
had already brought in looked like an independent addition on both
sides, which is why files carrying no Stacklet changes conflicted at
all.

Landing this as a merge commit rather than a squash is what stops
that repeating: the first commit's second parent is bf0b50dbc, and
preserving it makes the next merge base bf0b50dbc instead of
a83dda5c9. Rebase is already disabled on the repo, so merge commit
and squash are the only two buttons.

testing

The upstream commits bring their own tests, which arrive with the
merge: placebo and terraform fixtures for the new apigwv2 route, the
bedrock deployments filter and native cloudwatch-dashboard tagging;
ec2 and elasticache cases for tagging a resource deleted mid-run;
and flight data for k8s custom-resource discovery and the gcp
dataproc and kms additions. No Stacklet code changed, so Stacklet CI
and upstream CI running on this PR is the verification.

docs

No docs needed.


🤖 Generated with Claude Code

jimfulton and others added 30 commits August 20, 2026 21:26
Signed-off-by: 1fanwang <1fannnw@gmail.com>
* releng - dependency updates 2026-08

* AzureResource security hub support

* accept MIT-0 (MIT no attribution) license from cffi
…10872)

---------

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
---------

Co-authored-by: Caleb Syring <caleb.syring@sixfeetup.com>
---------

Co-authored-by: Daniel Lindsley <daniel@sixfeetup.com>
Co-authored-by: AJ Kerrigan <aj@speckledmonkey.com>
…n#11053)

Adds azure.private-dns-zone and azure.private-record-set, bringing Private
DNS to parity with the existing dnszone/recordset support.

Private DNS record sets carry metadata rather than tags, so they join
public DNS record sets in arm_tags_unsupported.

Also fixes two test harness gaps found while recording the cassettes: the
subscription id sanitizer missed response bodies that escape slashes as
\/, and x-ms-operation-identifier response headers were recorded verbatim,
carrying the recording tenant and user object ids into cassettes.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…cloud-custodian#10991)

Allow configuring server-side encryption for policy log uploads to the
S3 output via query parameters on the output url, eg
s3://bucket/prefix?sse=aws:kms&kms-key=alias/custodian. Supplying
kms-key implies aws:kms. Defaults to AES256 to preserve existing
behavior; aws:kms without a key falls back to the account's AWS managed
aws/s3 key.
…n is provided (cloud-custodian#11009)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…cloud-custodian#11008)

* azure - functions - use Entra ID token for SCM deploy instead of basic auth
* azure - update function app runtime to python 3.12

---------

Co-authored-by: AJ Kerrigan <aj@stacklet.io>
…loud-custodian#11081)

ResourceQuery._invoke_client_enum made a single unpaginated API call
per resource type and discarded the continuation token, so passing
limit silently truncated results to just the first page. It now loops
until the server stops returning metadata._continue, and raises
PolicyExecutionError instead of silently dropping already-fetched
items if a later page's response shape is unexpected.

Verified against a real kube-apiserver (recorded as the new flight
data fixture) and independently against a real k3s cluster.
* Centralize dynamic tag lookup logic where possible
* Extend the "default-value with no key" pattern from AWS to GCP & Azure
* Fix aws.asg-specific tag logic, and extract shared behavior between ASG & generic tag logic into a mixin
…d-custodian#11050)

- kms-key: AWS managed keys (KeyManager == 'AWS', e.g. the keys behind
  alias/aws/s3, alias/aws/ebs) reject tag mutations, key policy changes,
  rotation changes, and ScheduleKeyDeletion - AWS owns their full
  lifecycle.
- resolver-rule: Route 53's global auto-defined rules (e.g.
  rslvr-autodefined-rr-internet-resolver) have no owning account and
  reject tag mutations the same way.

Fixes cloud-custodian#11027

Co-authored-by: Usman Qureshi <usman.nja.14@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
… 17 updates (cloud-custodian#11088)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: AJ Kerrigan <aj@stacklet.io>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dian#11083)

* k8s - add custom-resource type for dynamic CRD discovery

This adds a k8s.custom-resource type. It discovers installed CustomResourceDefinitions through the typed ApiextensionsV1Api. For each served and storage version, it lists instances through the existing list_cluster_custom_object and list_custom_object_for_all_namespaces calls. It needs no per-CRD configuration and works with any CRD a cluster has installed. Each instance carries its own kind, group, version, plural, and scope. If discovery or listing fails for one CRD, the type logs the failure and continues with the rest.

This also fixes a pagination bug in ResourceQuery._invoke_client_enum. The code checked metadata._continue only for a next-page token. That key exists only on the typed response path. continue is a Python reserved word, so the client renames it to _continue after a call to to_dict(). list_cluster_custom_object and list_custom_object_for_all_namespaces return raw dicts that skip to_dict() and keep the real wire key, continue. Pagination stopped after the first page for exactly the calls this new type depends on.

I verified both changes against a real cluster. A recorded VCR cassette proves CRD discovery and instance listing end to end. For pagination, a CRD with 6 real instances and page_size=2 returned 2 of 6 instances before the fix and 6 of 6 after.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLgk1oh2BzoxfPd28ZtecH

* refactor: nest resource_type directly, matching every other c7n_kube type

Collapses the standalone CustomResourceTypeInfo class plus a trivial
pass subclass into a single nested resource_type class, matching the
convention every other resource type in this package already uses.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
…10944)

---------

Co-authored-by: AJ Kerrigan <aj@speckledmonkey.com>
@jtroup
jtroup requested a review from a team as a code owner September 15, 2026 23:45
@greptile-apps

greptile-apps Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 1/5

The PR is not safe to merge until the Azure compatibility mismatch and the three policy-execution correctness gaps are addressed.

Fix All in Claude CodeFindings

  1. P1 Cosmos Version Range Is Incompatible ▶
  2. P1 Managed Keys Remain Mutable ▶
  3. P1 CRD Failures Return Partial Results ▶
  4. P1 Dashboard Deletion Aborts Tagging ▶
Fix with agent prompt
### Issue 1
tools/c7n_azure/pyproject.toml:12
The dependency still allows azure-cosmos 3.x, but the updated implementation imports the v4 exception module and uses the v4 client interface. An unlocked installation can legally resolve 3.x and then fail while importing `cosmos_db.py`, making Cosmos DB resources unusable. The lower bound should require 4.x.

```suggestion
    "azure-cosmos>=4.0.0,<5.0.0",
```

### Issue 2
c7n/resources/kms.py:162-176
The new AWS-managed-key guard does not cover the existing `rename-tag` action. Because `kms-key` remains universally taggable, that action directly uses unguarded `UniversalTag` and `UniversalUntag` operations. If a policy selects an AWS-managed key, `rename-tag` still attempts forbidden mutations and fails instead of skipping the key like the newly wrapped tag actions.

### Issue 3
tools/c7n_kube/c7n_kube/resources/custom_resource.py:54-64
This catches every error while listing an individual CRD, including authorization failures, transient API failures, and implementation errors. The policy then reports success with only the resources collected before or after that failure, so governance checks can silently miss noncompliant custom resources. Only explicitly tolerated conditions should be skipped; unexpected failures need to surface.

### Issue 4
c7n/resources/cw.py:759-761
If a dashboard is deleted after augmentation but before mutation, the new `TagResource` call propagates `ResourceNotFound` and aborts the entire policy run. The new read path already treats this race as benign, but both the tag action here and the remove-tag action at lines 784–787 leave it unhandled. These actions should skip a dashboard that disappeared and continue processing the rest.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Summary

This PR merges upstream Cloud Custodian changes and restamps the prerelease workspace to +bf0b50dbc.

  • Adds policy-variable file support and new AWS API Gateway v2 route, Bedrock deployment, tagging, KMS, and Route 53 behavior.
  • Adds GCP Dataproc audit lookup and KMS CryptoKey updates.
  • Adds dynamic Kubernetes custom-resource discovery.
  • Updates Azure Cosmos and Terraform/HCL dependency integrations.
  • Four correctness gaps remain in Azure dependency compatibility, KMS managed-key handling, Kubernetes failure reporting, and CloudWatch dashboard mutation races.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  U[Upstream bf0b50dbc] --> M[Merge into stacklet/integration]
  M --> C[Core and AWS features]
  M --> G[GCP provider updates]
  M --> K[Kubernetes CRD discovery]
  M --> A[Azure and dependency updates]
  M --> V[Workspace restamp]
  V --> P[0.9.53a1+bf0b50dbc packages]
  V --> L[uv.lock]
Loading

Reviews (1) · Last reviewed commit: "chore: bump version suffix to a1+bf0b50d..."

"applicationinsights<1.0.0,>=0.11.9",
"apscheduler<4.0.0,>=3.6.3",
"azure-cosmos<4.0.0,>=3.1.2",
"azure-cosmos>=3.1.2,<5.0.0",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Cosmos version range is incompatible

The dependency still allows azure-cosmos 3.x, but the updated implementation imports the v4 exception module and uses the v4 client interface. An unlocked installation can legally resolve 3.x and then fail while importing cosmos_db.py, making Cosmos DB resources unusable. The lower bound should require 4.x.

Suggested change
"azure-cosmos>=3.1.2,<5.0.0",
"azure-cosmos>=4.0.0,<5.0.0",
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/c7n_azure/pyproject.toml
Line: 12

Comment:
**Cosmos version range is incompatible**

The dependency still allows azure-cosmos 3.x, but the updated implementation imports the v4 exception module and uses the v4 client interface. An unlocked installation can legally resolve 3.x and then fail while importing `cosmos_db.py`, making Cosmos DB resources unusable. The lower bound should require 4.x.

```suggestion
    "azure-cosmos>=4.0.0,<5.0.0",
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

Comment thread c7n/resources/kms.py
Comment on lines +162 to +176
@Key.action_registry.register('mark')
@Key.action_registry.register('tag')
class KeyTag(SkipAwsManagedKey, tagmod.UniversalTag):
pass


@Key.action_registry.register('unmark')
@Key.action_registry.register('untag')
@Key.action_registry.register('remove-tag')
class KeyRemoveTag(SkipAwsManagedKey, tagmod.UniversalUntag):
pass


@Key.action_registry.register('mark-for-op')
class KeyMarkForOp(SkipAwsManagedKey, tagmod.UniversalTagDelayedAction):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Managed keys remain mutable

The new AWS-managed-key guard does not cover the existing rename-tag action. Because kms-key remains universally taggable, that action directly uses unguarded UniversalTag and UniversalUntag operations. If a policy selects an AWS-managed key, rename-tag still attempts forbidden mutations and fails instead of skipping the key like the newly wrapped tag actions.

Prompt To Fix With AI
This is a comment left during a code review.
Path: c7n/resources/kms.py
Line: 162-176

Comment:
**Managed keys remain mutable**

The new AWS-managed-key guard does not cover the existing `rename-tag` action. Because `kms-key` remains universally taggable, that action directly uses unguarded `UniversalTag` and `UniversalUntag` operations. If a policy selects an AWS-managed key, `rename-tag` still attempts forbidden mutations and fails instead of skipping the key like the newly wrapped tag actions.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

Comment on lines +54 to +64
try:
instances.extend(self._list_instances(session, crd))
except Exception:
log.exception(
"failed to list instances of CRD %s/%s %s (group=%s); "
"continuing with other discovered CRDs",
crd["plural"],
crd["version"],
crd["kind"],
crd["group"],
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 CRD failures return partial results

This catches every error while listing an individual CRD, including authorization failures, transient API failures, and implementation errors. The policy then reports success with only the resources collected before or after that failure, so governance checks can silently miss noncompliant custom resources. Only explicitly tolerated conditions should be skipped; unexpected failures need to surface.

Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/c7n_kube/c7n_kube/resources/custom_resource.py
Line: 54-64

Comment:
**CRD failures return partial results**

This catches every error while listing an individual CRD, including authorization failures, transient API failures, and implementation errors. The policy then reports success with only the resources collected before or after that failure, so governance checks can silently miss noncompliant custom resources. Only explicitly tolerated conditions should be skipped; unexpected failures need to surface.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

Comment thread c7n/resources/cw.py
Comment on lines +759 to +761
def process_resource_set(self, client, resource_set, tags):
for arn in self.manager.get_arns(resource_set):
self.manager.retry(client.tag_resource, ResourceARN=arn, Tags=tags)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Dashboard deletion aborts tagging

If a dashboard is deleted after augmentation but before mutation, the new TagResource call propagates ResourceNotFound and aborts the entire policy run. The new read path already treats this race as benign, but both the tag action here and the remove-tag action at lines 784–787 leave it unhandled. These actions should skip a dashboard that disappeared and continue processing the rest.

Prompt To Fix With AI
This is a comment left during a code review.
Path: c7n/resources/cw.py
Line: 759-761

Comment:
**Dashboard deletion aborts tagging**

If a dashboard is deleted after augmentation but before mutation, the new `TagResource` call propagates `ResourceNotFound` and aborts the entire policy run. The new read path already treats this race as benign, but both the tag action here and the remove-tag action at lines 784–787 leave it unhandled. These actions should skip a dashboard that disappeared and continue processing the rest.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

16 issues found across 115 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="tests/terraform/ec2_instance_tag/main.tf">

<violation number="1" location="tests/terraform/ec2_instance_tag/main.tf:3">
P1: When the functional suite runs, this provider inherits `AWS_DEFAULT_REGION=us-east-2`, but `test_native_tag_skips_gone_instance` queries `us-east-1`. The test then finds zero of the provisioned instances and fails its `len(resources) == 2` assertion; pin the fixture to `us-east-1` to match the test and recorded state.</violation>
</file>

<file name="tests/terraform/bedrock_deployable_custom_model/README.md">

<violation number="1" location="tests/terraform/bedrock_deployable_custom_model/README.md:86">
P2: When rebuilding while `KEEP-c7n-deployable-test-fixture` still exists, `setup.py` submits the same `customModelName` and Bedrock rejects the duplicate name. Document deleting the existing model with `DeleteCustomModel` before rerunning the setup command.</violation>
</file>

<file name="tests/test_tags.py">

<violation number="1" location="tests/test_tags.py:840">
P2: This test does not verify that the surviving instances are actually included in either API call. The `DescribeInstances` responses are prerecorded state, so the tag assertions pass independently of `tag.process` and `remove_tag.process`; assert the `CreateTags`/`DeleteTags` request IDs or use a mutable/mock client for these calls.</violation>
</file>

<file name="tests/test_bedrock.py">

<violation number="1" location="tests/test_bedrock.py:1765">
P2: When the deployment does not become `Active`, the fixture still tries to delete it while it may be `Creating`, which Bedrock rejects and leaves the shared test deployment behind. Make teardown wait for a deletable terminal state and handle cleanup failures without masking the original test error.</violation>
</file>

<file name="tests/data/placebo/test_native_tag_skips_gone_instance/ec2.DescribeInstances_2.json">

<violation number="1" location="tests/data/placebo/test_native_tag_skips_gone_instance/ec2.DescribeInstances_2.json:145">
P2: After `tag.process`, this is the replay response used by the test's verification `describe_instances` call, but neither instance includes the `App=Custodian` tag. Add the expected `Tags` block to both instances so the fixture reflects the successful `CreateTags_2` response.</violation>
</file>

<file name="tests/terraform/bedrock_deployable_custom_model/setup.py">

<violation number="1" location="tests/terraform/bedrock_deployable_custom_model/setup.py:46">
P2: When the caller lacks `bedrock:CreateModelCustomizationJob`, AWS returns an `AccessDeniedException` containing `is not authorized`, so this predicate waits five minutes before surfacing the permanent failure. Match only assume-role propagation messages so permission errors fail immediately.</violation>
</file>

<file name="c7n/resources/cw.py">

<violation number="1" location="c7n/resources/cw.py:761">
P1: Handle `ResourceNotFound`/`ResourceNotFoundException` around both `tag_resource` and `untag_resource` calls. A dashboard deleted after augmentation currently aborts the action instead of skipping it and processing the remaining dashboards.</violation>

<violation number="2" location="c7n/resources/cw.py:791">
P2: Policies using `rename-tag` now fail validation for `cloudwatch-dashboard` because the explicit native action registrations omit it. Add a dashboard-specific rename action that performs the native tag and untag calls, or preserve equivalent rename support.</violation>
</file>

<file name="c7n/resources/kms.py">

<violation number="1" location="c7n/resources/kms.py:150">
P2: When `DescribeKey` is denied, `KeyManager` is absent and this guard treats the key as customer-managed. Treat missing or unknown `KeyManager` values as non-actionable, or resolve the metadata before allowing these mutations.</violation>

<violation number="2" location="c7n/resources/kms.py:164">
P2: When a `rename-tag` action receives an AWS-managed key carrying a stale or externally supplied old tag, it bypasses `SkipAwsManagedKey` and calls the tagging APIs, which reject the key. Register `rename-tag` with a guarded KMS wrapper as well.</violation>
</file>

<file name="tools/c7n_azure/c7n_azure/resources/cosmos_db.py">

<violation number="1" location="tools/c7n_azure/c7n_azure/resources/cosmos_db.py:8">
P1: The package still permits `azure-cosmos` 3.x, but this migration uses the v4 client and exception surface, so 3.x installations can no longer run Cosmos resources. Raise the dependency floor to `azure-cosmos>=4` or retain a v3-compatible code path.</violation>
</file>

<file name="c7n/element.py">

<violation number="1" location="c7n/element.py:42">
P2: When the package is type-checked, this closed annotation makes existing element subclasses invalid: several schemas use top-level `anyOf`/`oneOf`, and `AgeFilter.schema` is intentionally `None`. Broaden `ElementJSONSchema` to cover the supported JSON Schema keywords and nullable schemas, or use a general JSON-schema mapping type for `Element.schema`.</violation>
</file>

<file name="c7n/commands.py">

<violation number="1" location="c7n/commands.py:54">
P2: When a vars file supplies a typed policy value such as `max-resources: "{limit}"`, schema validation rejects the placeholder before `p.expand_variables` can restore the integer. Pass `vars=vars` to `policy_load`, or validate only after variable expansion.</violation>
</file>

<file name="tools/c7n_gcp/c7n_gcp/resources/kms.py">

<violation number="1" location="tools/c7n_gcp/c7n_gcp/resources/kms.py:218">
P2: When more than 20 keys are selected and an invalid-purpose key appears after the first chunk, this check raises only after earlier keys have been updated. Move the purpose validation before `MethodAction.process` starts chunking so the action fails without partially changing the selection.</violation>
</file>

<file name="tests/terraform/cloudwatch_dashboard_tag/main.tf">

<violation number="1" location="tests/terraform/cloudwatch_dashboard_tag/main.tf:3">
P2: When the Terraform environment selects a region other than `us-east-1`, this fixture creates the dashboard there but `test_cw_dashboard_tag` queries `us-east-1`, so the policy can find no dashboard. Set the provider region explicitly to `us-east-1` to match the test and fixture conventions.</violation>
</file>

<file name="tools/c7n_kube/c7n_kube/resources/custom_resource.py">

<violation number="1" location="tools/c7n_kube/c7n_kube/resources/custom_resource.py:56">
P1: Do not swallow every exception from `_list_instances`. RBAC, transient API, or implementation failures currently become partial success, so policies can miss noncompliant custom resources; let unexpected errors propagate and skip only known benign conditions.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

@@ -0,0 +1,36 @@
# Copyright The Cloud Custodian Authors.
# SPDX-License-Identifier: Apache-2.0
provider "aws" {}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: When the functional suite runs, this provider inherits AWS_DEFAULT_REGION=us-east-2, but test_native_tag_skips_gone_instance queries us-east-1. The test then finds zero of the provisioned instances and fails its len(resources) == 2 assertion; pin the fixture to us-east-1 to match the test and recorded state.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/terraform/ec2_instance_tag/main.tf, line 3:

<comment>When the functional suite runs, this provider inherits `AWS_DEFAULT_REGION=us-east-2`, but `test_native_tag_skips_gone_instance` queries `us-east-1`. The test then finds zero of the provisioned instances and fails its `len(resources) == 2` assertion; pin the fixture to `us-east-1` to match the test and recorded state.</comment>

<file context>
@@ -0,0 +1,36 @@
+# Copyright The Cloud Custodian Authors.
+# SPDX-License-Identifier: Apache-2.0
+provider "aws" {}
+
+data "aws_ami" "amazon_linux" {
</file context>


from azure.cosmos.cosmos_client import CosmosClient
from azure.cosmos.errors import HTTPFailure
from azure.cosmos.exceptions import CosmosHttpResponseError

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The package still permits azure-cosmos 3.x, but this migration uses the v4 client and exception surface, so 3.x installations can no longer run Cosmos resources. Raise the dependency floor to azure-cosmos>=4 or retain a v3-compatible code path.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tools/c7n_azure/c7n_azure/resources/cosmos_db.py, line 8:

<comment>The package still permits `azure-cosmos` 3.x, but this migration uses the v4 client and exception surface, so 3.x installations can no longer run Cosmos resources. Raise the dependency floor to `azure-cosmos>=4` or retain a v3-compatible code path.</comment>

<file context>
@@ -5,7 +5,7 @@
 
 from azure.cosmos.cosmos_client import CosmosClient
-from azure.cosmos.errors import HTTPFailure
+from azure.cosmos.exceptions import CosmosHttpResponseError
 from azure.mgmt.cosmosdb.models import VirtualNetworkRule
 
</file context>

Comment thread c7n/resources/cw.py

def process_resource_set(self, client, resource_set, tags):
for arn in self.manager.get_arns(resource_set):
self.manager.retry(client.tag_resource, ResourceARN=arn, Tags=tags)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Handle ResourceNotFound/ResourceNotFoundException around both tag_resource and untag_resource calls. A dashboard deleted after augmentation currently aborts the action instead of skipping it and processing the remaining dashboards.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At c7n/resources/cw.py, line 761:

<comment>Handle `ResourceNotFound`/`ResourceNotFoundException` around both `tag_resource` and `untag_resource` calls. A dashboard deleted after augmentation currently aborts the action instead of skipping it and processing the remaining dashboards.</comment>

<file context>
@@ -698,14 +727,71 @@ class resource_type(TypeInfo):
+
+    def process_resource_set(self, client, resource_set, tags):
+        for arn in self.manager.get_arns(resource_set):
+            self.manager.retry(client.tag_resource, ResourceARN=arn, Tags=tags)
+
+
</file context>

for crd in self._discover_crds(session):
try:
instances.extend(self._list_instances(session, crd))
except Exception:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Do not swallow every exception from _list_instances. RBAC, transient API, or implementation failures currently become partial success, so policies can miss noncompliant custom resources; let unexpected errors propagate and skip only known benign conditions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tools/c7n_kube/c7n_kube/resources/custom_resource.py, line 56:

<comment>Do not swallow every exception from `_list_instances`. RBAC, transient API, or implementation failures currently become partial success, so policies can miss noncompliant custom resources; let unexpected errors propagate and skip only known benign conditions.</comment>

<file context>
@@ -0,0 +1,132 @@
+        for crd in self._discover_crds(session):
+            try:
+                instances.extend(self._list_instances(session, crd))
+            except Exception:
+                log.exception(
+                    "failed to list instances of CRD %s/%s %s (group=%s); "
</file context>

cd tests/terraform/bedrock_deployable_custom_model
tofu init
tofu apply # prerequisites only; no model yet
../../../.venv/bin/python setup.py # submits the customization job and exits

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When rebuilding while KEEP-c7n-deployable-test-fixture still exists, setup.py submits the same customModelName and Bedrock rejects the duplicate name. Document deleting the existing model with DeleteCustomModel before rerunning the setup command.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/terraform/bedrock_deployable_custom_model/README.md, line 86:

<comment>When rebuilding while `KEEP-c7n-deployable-test-fixture` still exists, `setup.py` submits the same `customModelName` and Bedrock rejects the duplicate name. Document deleting the existing model with `DeleteCustomModel` before rerunning the setup command.</comment>

<file context>
@@ -0,0 +1,139 @@
+cd tests/terraform/bedrock_deployable_custom_model
+tofu init
+tofu apply     # prerequisites only; no model yet
+../../../.venv/bin/python setup.py    # submits the customization job and exits
+```
+
</file context>

Comment thread c7n/resources/kms.py
warning so operators know why they were skipped."""

def process(self, resources):
excluded = [r for r in resources if r.get('KeyManager') == 'AWS']

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When DescribeKey is denied, KeyManager is absent and this guard treats the key as customer-managed. Treat missing or unknown KeyManager values as non-actionable, or resolve the metadata before allowing these mutations.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At c7n/resources/kms.py, line 150:

<comment>When `DescribeKey` is denied, `KeyManager` is absent and this guard treats the key as customer-managed. Treat missing or unknown `KeyManager` values as non-actionable, or resolve the metadata before allowing these mutations.</comment>

<file context>
@@ -136,6 +137,46 @@ def alias_map(self):
+    warning so operators know why they were skipped."""
+
+    def process(self, resources):
+        excluded = [r for r in resources if r.get('KeyManager') == 'AWS']
+        if excluded:
+            self.manager.log.warning(
</file context>

Comment thread c7n/element.py
executor_factory = ThreadPoolExecutor

schema = {'type': 'object'}
schema: ElementJSONSchema = {'type': 'object'}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When the package is type-checked, this closed annotation makes existing element subclasses invalid: several schemas use top-level anyOf/oneOf, and AgeFilter.schema is intentionally None. Broaden ElementJSONSchema to cover the supported JSON Schema keywords and nullable schemas, or use a general JSON-schema mapping type for Element.schema.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At c7n/element.py, line 42:

<comment>When the package is type-checked, this closed annotation makes existing element subclasses invalid: several schemas use top-level `anyOf`/`oneOf`, and `AgeFilter.schema` is intentionally `None`. Broaden `ElementJSONSchema` to cover the supported JSON Schema keywords and nullable schemas, or use a general JSON-schema mapping type for `Element.schema`.</comment>

<file context>
@@ -16,7 +39,8 @@ class Element:
     executor_factory = ThreadPoolExecutor
 
-    schema = {'type': 'object'}
+    schema: ElementJSONSchema = {'type': 'object'}
+
     # schema aliases get hoisted into a jsonschema definition
</file context>

Comment thread c7n/commands.py
for fp in options.configs:
try:
collection = policy_load(options, fp, validate=validate, vars=vars)
collection = policy_load(options, fp, validate=validate)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a vars file supplies a typed policy value such as max-resources: "{limit}", schema validation rejects the placeholder before p.expand_variables can restore the integer. Pass vars=vars to policy_load, or validate only after variable expansion.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At c7n/commands.py, line 54:

<comment>When a vars file supplies a typed policy value such as `max-resources: "{limit}"`, schema validation rejects the placeholder before `p.expand_variables` can restore the integer. Pass `vars=vars` to `policy_load`, or validate only after variable expansion.</comment>

<file context>
@@ -50,7 +51,7 @@ def _load_policies(options):
         for fp in options.configs:
             try:
-                collection = policy_load(options, fp, validate=validate, vars=vars)
+                collection = policy_load(options, fp, validate=validate)
             except IOError:
                 log.error('policy file does not exist ({})'.format(fp))
</file context>
Suggested change
collection = policy_load(options, fp, validate=validate)
collection = policy_load(options, fp, validate=validate, vars=vars)

return self

def process_resource_set(self, client, model, resources):
if self.rotation_fields & self.data.keys():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When more than 20 keys are selected and an invalid-purpose key appears after the first chunk, this check raises only after earlier keys have been updated. Move the purpose validation before MethodAction.process starts chunking so the action fails without partially changing the selection.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tools/c7n_gcp/c7n_gcp/resources/kms.py, line 218:

<comment>When more than 20 keys are selected and an invalid-purpose key appears after the first chunk, this check raises only after earlier keys have been updated. Move the purpose validation before `MethodAction.process` starts chunking so the action fails without partially changing the selection.</comment>

<file context>
@@ -134,6 +135,112 @@ def _get_location(cls, resource):
+        return self
+
+    def process_resource_set(self, client, model, resources):
+        if self.rotation_fields & self.data.keys():
+            invalid = [
+                r['name'] for r in resources
</file context>

@@ -0,0 +1,26 @@
# Copyright The Cloud Custodian Authors.
# SPDX-License-Identifier: Apache-2.0
provider "aws" {}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When the Terraform environment selects a region other than us-east-1, this fixture creates the dashboard there but test_cw_dashboard_tag queries us-east-1, so the policy can find no dashboard. Set the provider region explicitly to us-east-1 to match the test and fixture conventions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/terraform/cloudwatch_dashboard_tag/main.tf, line 3:

<comment>When the Terraform environment selects a region other than `us-east-1`, this fixture creates the dashboard there but `test_cw_dashboard_tag` queries `us-east-1`, so the policy can find no dashboard. Set the provider region explicitly to `us-east-1` to match the test and fixture conventions.</comment>

<file context>
@@ -0,0 +1,26 @@
+# Copyright The Cloud Custodian Authors.
+# SPDX-License-Identifier: Apache-2.0
+provider "aws" {}
+
+resource "random_id" "id" {
</file context>

@tvansteenburgh tvansteenburgh left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jtroup

@jtroup
jtroup merged commit 42572e6 into stacklet/integration Sep 16, 2026
4 checks passed
@jtroup
jtroup deleted the elmo/firm-toad branch September 16, 2026 15:06
kevin-birkett added a commit that referenced this pull request Oct 23, 2026
…NG-9376) (#27)

[ENG-9376](https://stacklet.atlassian.net/browse/ENG-9376)

### what

Merges upstream cloud-custodian `bf0b50dbc..0fa7d19` into
`stacklet/integration` and stamps a new alpha series: `c7n` becomes
`0.9.54a1+0fa7d19f5`, and each workspace tool ticks one patch (e.g.
`c7n-azure` 0.7.53a1, `c7n-left` 0.3.40a1).

Two commits: the upstream merge on its own, then the stamp. The merge's
combined diff (`git show --cc`) is empty, so there is no hand-written
content in it to review.

Conflicts were the `version` fields in `pyproject.toml`, the twelve
`tools/*/pyproject.toml` and `c7n/version.py`, the root `[tool.uv]`
cooldown block, and `uv.lock`. The versions and the 7-day cooldown keep
the branch's side, and `uv.lock` takes upstream's. Upstream's other
edits to those files (an `azure-mgmt-eventgrid` floor bump and new
`GOOGLE_WORKSPACE_*` test env unsets) merged cleanly and are kept. No
source file conflicted: #24 landing as a merge commit left the merge
base at `bf0b50dbc`.

The stamp commit also drops the `tfparse` `exclude-newer-package`
override added in #3. It admitted tfparse 0.6.20 past the 7-day
cooldown, but as a fixed date it now capped tfparse at releases before
2026-06-02. Without it the lock resolves tfparse 0.6.22, matching
upstream. Apart from that, the lock differs from upstream's only in the
cooldown span and the workspace versions.

Upstream content picked up:

- core: a `duration` value_type; resolver URI caching keys on request
  headers and skips blank CSV rows.
- aws: new resources for s3tables (table-bucket, table), s3files
  (file-system, mount-target, access-point), Q Business, AIOps
  investigation groups, rds-global-cluster, eventbridge event-connection
  and kafka-broker, plus expanded IoT support; new actions including
  account `set-access-analyzer`, ami `set-disabled`, cloudtrail
  `set-event-selectors` and sagemaker-space update; SSM tagging for
  managed instances, ops items, patch baselines, maintenance windows and
  associations; security-group listing now paginates; fixes for
  invoke-lambda qualifiers, secrets-manager ARN lookup, and tag actions
  on CFN `ROLLBACK_COMPLETE` stacks and CloudTrail shadow trails.
- azure: machine learning compute cluster, environment version and
  schedule resources, and ML job cancel / data container archive
actions; disk snapshot and NSG remove-rules actions; set-firewall-rules
  no longer corrupts rules across resources; API call stats.
- gcp: workspace-user (MFA reporting) and Vertex AI metadata store
  artifact resources; set-public-access on buckets, set-ip-forwarding on
instances, set-iam-policy on KMS keys with auditConfig support; project
  metrics scoped to one project at a time; API call stats.
- k8s: resource cache keys include the resource type; API call stats.
- releng: upstream's 0.9.53 release prep.

No Stacklet code changed.

### why

The October c7n upgrade (ENG-9376) needs upstream's latest changes.
Upstream released 0.9.53.0 on 2026-10-01, so per the versioning
convention the base moves to the next unreleased version, 0.9.54, and
the alpha restarts at a1.

This should land as a merge commit, not a squash, so that `0fa7d19f5`
stays in the ancestry and the next upstream merge has the right merge
base (see #24).

### testing

The upstream commits bring their own tests, which arrive with the
merge. No Stacklet code changed, so Stacklet CI and upstream CI running
on this PR is the verification of the code.

For the stamp: all 13 workspace packages are stamped uniformly,
`c7n/version.py` is regenerated, no reference to the old hash remains,
and the cumulative diff against upstream `main` contains only the known
Stacklet deltas (versions, cooldown, CODEOWNERS, `stacklet-*.yml`, the
ci.yml trigger retarget, justfile, release tooling).
`make pkg-build-wheel` builds all 13 wheels and passes
`twine check --strict`, `tools/dev/check_release_wheels.py` accepts
them, and `make lint` passes.

### docs

No docs needed.

---------------------------

🤖 Generated with [Claude Code](https://claude.com/claude-code)


[ENG-9376]:
https://stacklet.atlassian.net/browse/ENG-9376?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.