Repository navigation
Conversation
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Use literal exclusion lookups and skip directory metadata reads before writing tar entries. Preserve archive bytes and keep the Argo CD compressor for full-checkout fallbacks. Co-Authored-By: Codex <noreply@openai.com>
Owner
|
@ak-a Thank you for the PR! Unfortunately, I don't have time to review it before KubeCon next week :) |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kustomize sources currently stream the full checkout for each render. The archive cache avoids repeated compression, but the repo server still receives and extracts unrelated files.
Use
argocd.argoproj.io/manifest-generate-pathsto stream the source directory and its declared dependencies. Preserve the repository layout and both literal and glob matches. Cache archives by their final exclusions, so full-checkout fallbacks share one archive. Write scoped archives with literal path lookups and skip excluded directories before reading metadata.Narrowing requires an explicit Kustomize source (
kustomize: {}is sufficient). Sources that use automatic discovery, lack the annotation, or retain symlinks keep the full checkout. Source override files also disable narrowing. Annotation whitespace is normalized as in application selection. All required shared bases, patches, and value files must be declared.In one comparison of the initial implementation, a monorepo with about 18,000 files rendered 24 applications at two revisions, with concurrency 20 and an existing Argo CD instance:
All 48 rendered manifest files were byte-for-byte identical. Both runs used the same input commits; the candidate used
mainplus this change. The baseline ran first and backend caches were not reset. The observed 79% reduction applies to this workload and predates the review fixes.A local synthetic benchmark of the final archive writer, selecting one file among 16,000 sibling directories, reduced selection and archive creation from 8.09 s to 38 ms. This does not measure total CI runtime.
Validation: full unit suite with race detection and coverage, lint, and CLI build passed. Tests verify byte-for-byte archive equality, shared dependencies, cache reuse, literal/glob paths, renderer guards, annotation whitespace, and error cleanup. A benchmark covers 1,000–16,000 sibling directories.