Skip to content

chore: resync 2026.08.26 c7n with July's tagging fixes - #26

Merged
jtroup merged 7 commits into
release/2026-08-26from
elmo/open-martin
Sep 30, 2026
Merged

jtroup merged 7 commits into
release/2026-08-26from
elmo/open-martin

Conversation

@jtroup

@jtroup jtroup commented Sep 28, 2026 •

Copy link
Copy Markdown

ENG-8974

what

Cherry-picks six changes carried by the July release line onto
release/2026-08-26 and stamps the workspace
0.9.52a5.post2+a83dda5c9 — fc8d0ac72 already took .post1.

Two are Stacklet-authored and will never arrive from upstream:

Four are upstream:

A seventh candidate, cloud-custodian#11016 (s3 augment() connection errors),
needed no pick: it reached integration before August's base was cut,
so the content is already present under a different SHA.

why

August was cut from a later integration point, which is not a
superset of July's. Its base predates most of the upstream tagging
work July picked up, and none of that has been applied since, so a
deployment moving from the July line to August loses all six changes.
The two Stacklet-authored ones are the same pair the June-to-July
port existed to rescue; without this, they would be lost a second
time.

testing

Each pick brings its own coverage: per-bucket tolerance on the s3 tag
action, including an end-to-end case through collection and tagging;
cloud run replace bodies rejecting root fields absent from the
schema; dynamic tag resolution across AWS, GCP and Azure and on the
asg-specific path; AWS-managed kms keys and auto-defined resolver
rules skipped by their actions; cloudwatch dashboard tag and
remove-tag against the native apis; and batched ec2 tagging dropping
the ids an error names as gone.

What the diff cannot settle is whether cloud-custodian#10942, already on the August
line, and cloud-custodian#10968 cohere when applied in this order — the reverse of
the order July received them in. That is the question the suite run
on this branch answers.

docs

No docs needed.


🤖 Generated with Claude Code

ajkerrigan and others added 7 commits September 26, 2026 02:14
…1089)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…loud-custodian#11090)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d-custodian#11050)

- kms-key: AWS managed keys (KeyManager == 'AWS', e.g. the keys behind
  alias/aws/s3, alias/aws/ebs) reject tag mutations, key policy changes,
  rotation changes, and ScheduleKeyDeletion - AWS owns their full
  lifecycle.
- resolver-rule: Route 53's global auto-defined rules (e.g.
  rslvr-autodefined-rr-internet-resolver) have no owning account and
  reject tag mutations the same way.

Fixes cloud-custodian#11027

Co-authored-by: Usman Qureshi <usman.nja.14@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Centralize dynamic tag lookup logic where possible
* Extend the "default-value with no key" pattern from AWS to GCP & Azure
* Fix aws.asg-specific tag logic, and extract shared behavior between ASG & generic tag logic into a mixin
### what

`modify_bucket_tags()` tolerates the same transport failures
`BucketAssembly.assemble()` already does, on both the tag re-fetch and
the write. A bucket whose endpoint doesn't answer is logged and
skipped; the rest of the batch is tagged as normal.

`ConnectionClosedError` joins that set in both places. A degraded
endpoint can accept a connection and then drop it rather than refusing
or hanging, and that class was in neither handler - so it would still
abort the tag action, and still abort collection for the whole
account, which is what cd79871 set out to stop.

`ProxyConnectionError`, `SSLError` and `ResponseStreamingError` are
deliberately left to raise: the first two say something is wrong with
the path to s3 rather than with one bucket, and the third can't
plausibly happen on a response this small.

### why

One bucket whose endpoint hangs silently costs healthy buckets their
tags, on every run.

`modify_bucket_tags()` caught only `ClientError`, so a connect timeout
raised out of it and abandoned the rest of its batch.
`_common_tag_processer` chunks at `Tag.batch_size` - 25 for `tag`, 100
for `remove-tag` - and walks each chunk serially, so the loss is every
bucket after the hung one in its own chunk. How many that is depends
on where the hung bucket sorts in `list_buckets` order, so it is
silent, it varies, and it repeats every run. Measured against an
account matching the reported one, 76 buckets with one timing out on
every call:

| hung bucket's position | before | after |
| ---------------------- | ------ | ----- |
| last | run errors, 75 of 75 tagged | succeeds, 75 of 75 |
| eleventh | run errors, 61 of 75 tagged | succeeds, 75 of 75 |

The run also reported error either way, so an account that was in fact
being tagged still looked like a failure to anything gating on run
status.

This is what remains after cd79871, which stopped the same bucket
from aborting resource collection outright. `list_buckets` is global,
so every run in every region enumerates the whole account - one bucket
whose endpoint hangs is on the path of every s3 policy, not only ones
targeting its region.

The bucket that couldn't be read is skipped rather than written to:
tags are set atomically for the whole set, so writing one we couldn't
re-read would drop whatever tags it has.

### testing

New tests cover a hung bucket on the re-fetch and on the write leaving
the rest of the batch tagged; a connection closed mid response
surviving both collection and the tag action; and an end to end case -
collection then tag action, with one bucket timing out on every call -
asserting that a hung bucket at the head of a chunk costs none of the
29 buckets behind it. That last one fails against the previous
behaviour with the timeout propagating out of the action, which is the
reported symptom.

### docs

No docs needed.

---------------------------

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@jtroup
jtroup requested a review from a team as a code owner September 28, 2026 02:32
@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 2/5

[High risk] Resync of tagging system with resource-gone error handling.

The PR is not ready to merge because valid tag policies can overwrite existing Azure values or silently omit an ASG tag.

Fix All in Claude CodeFindings

  1. P1 Existing Azure tag overwritten ▶
  2. P1 Single-tag default overwrites values ▶
  3. P1 Falsy ASG values discarded ▶
Fix with agent prompt
### Issue 1
tools/c7n_azure/c7n_azure/actions/tagging.py:112
If a resource already has an `owner` tag and the policy sets `Owner` with a no-key default, this case-sensitive check treats the tag as absent. Azure tag names are case-insensitive, so the default can replace the existing value instead of preserving it.

### Issue 2
tools/c7n_azure/c7n_azure/actions/tagging.py:109-110
In the single `tag`/`value` form, a no-key default is accepted by the schema, but `Lookup.extract` returns that default without checking whether the tag exists. If it does, the action overwrites its value rather than leaving it untouched.

### Issue 3
c7n/resources/asg.py:1176
The single-value schema now accepts numbers and booleans, but this truthiness check drops `0` and `false` before they become tag values. A policy using either value validates yet silently writes no requested tag.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Summary

The PR resynchronizes July tagging and resource-action fixes onto the August release line and advances workspace package versions.

  • Centralizes lookup and conditional-default handling across AWS, Azure, and GCP.
  • Adds CloudWatch dashboard tagging, Cloud Run replacement-body cleanup, and handling for deleted or unmodifiable AWS resources.
  • The Azure conditional-default paths and ASG’s newly accepted falsy values need correction.

Reviews (1) · Last reviewed commit: "chore: cherry pick tagging fixes and bum..."

Comment thread c7n/resources/asg.py Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 66 files

Architecture diagram
sequenceDiagram
    participant Policy as Policy Engine
    participant Lookup as c7n.lookup
    participant ASG as ASG Tag Action
    participant EC2 as EC2 CreateTags
    participant RGTA as Resource Groups Tagging API
    participant CW as CloudWatch API
    participant S3 as S3 API
    participant KMS as KMS API
    participant R53 as Route53 API
    participant GCP as GCP/Knative API
    participant Azure as Azure API

    Note over Policy,Lookup: Dynamic Tag Value Resolution (shared across providers)

    Policy->>Lookup: resolve_value(spec, resource, name, current_tags)
    alt Lookup with key
        Lookup->>Lookup: Extract value via jmespath
        alt Key found
            Lookup-->>Policy: Resolved value
        else Key not found, default-value present
            Lookup-->>Policy: Default value
        else Key not found, no default
            Lookup-->>Policy: Raise error
        end
    else Conditional default (no key)
        alt Tag already present
            Lookup-->>Policy: SKIP (omit tag)
        else Tag absent
            Lookup-->>Policy: Default value
        end
    else Static scalar
        Lookup-->>Policy: Value as-is
    end

    Note over Policy,EC2: EC2 Native Tag Action (deleted resource handling)

    Policy->>EC2: Process resource batch
    EC2->>EC2: retry_skip_bad_ids()
    EC2->>EC2: call CreateTags with resource IDs
    alt Success
        EC2-->>Policy: Tags applied
    else ClientError with InvalidXXX.NotFound
        EC2->>EC2: Extract bad IDs from error message
        EC2->>EC2: Remove bad IDs from batch
        opt Remaining IDs
            EC2->>EC2: Retry with remaining IDs
            EC2-->>Policy: Tags applied to survivors
        end
    end

    Note over Policy,RGTA: Universal Tag Action (KMS, Route53, Elasticache)

    Policy->>RGTA: Process resources with tags
    alt AWS-managed KMS key
        Policy->>Policy: Skip (KeyManager == 'AWS')
    else Auto-defined resolver rule
        Policy->>Policy: Skip (Id starts with rslvr-autodefined-rr-)
    else Normal resource
        RGTA->>RGTA: Call TagResources
        alt Resource already deleted
            RGTA->>RGTA: Skip failed ARN from FailedResourcesMap
            RGTA-->>Policy: Partial success
        else Success
            RGTA-->>Policy: Tags applied
        end
    end

    Note over Policy,CW: CloudWatch Dashboard Tagging

    Policy->>CW: ListDashboards
    CW-->>Policy: Dashboard list
    Policy->>CW: ListTagsForResource (per dashboard ARN)
    alt Dashboard deleted between calls
        Policy->>Policy: Skip dashboard
    else Success
        CW-->>Policy: Existing tags
    end
    Policy->>CW: TagResource / UntagResource (native API)
    CW-->>Policy: Confirmation

    Note over Policy,S3: S3 Bucket Tagging with Transport Error Tolerance

    Policy->>S3: get_bucket_tagging
    alt Transport error (timeout, connection closed)
        Policy->>Policy: Skip bucket, continue batch
    else Success
        S3-->>Policy: Existing tags
    end
    Policy->>S3: put_bucket_tagging
    alt Transport error on write
        Policy->>Policy: Log error, continue batch
    else Success
        S3-->>Policy: Confirmation
    end

    Note over Policy,GCP: GCP Cloud Run Label Actions

    Policy->>GCP: Get resource
    GCP-->>Policy: Service/Job object
    Policy->>Policy: knative_body() - filter to schema fields
    alt Unrecognized root field
        Policy->>Policy: Raise PolicyExecutionError
    else Valid body
        Policy->>Policy: Strip c7n annotations
        Policy->>GCP: Replace service/job (whole-object)
        GCP-->>Policy: Updated object
    end

    Note over Policy,Azure: Azure Tag Actions

    Policy->>Azure: Get resource
    Azure-->>Policy: Resource with existing tags
    Policy->>Azure: Resolve tag values via Lookup
    alt Conditional default and tag exists
        Policy->>Policy: Skip tag write
    else Resolved tags to write
        Policy->>Azure: Update resource tags
        Azure-->>Policy: Confirmation
    end
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread pyproject.toml Resolved
Comment thread tools/c7n_gcp/tests/test_gcp_labels.py Resolved
Comment thread tests/test_cw_dashboard.py Resolved
Comment thread c7n/resources/kms.py Resolved

@kevin-birkett kevin-birkett left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jtroup

@jtroup
jtroup merged commit e05b23e into release/2026-08-26 Sep 30, 2026
4 checks passed
@jtroup
jtroup deleted the elmo/open-martin branch September 30, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants