Skip to content

docs: describe how transforms reach the pod today #107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
catinspace-au merged 1 commit into from
Oct 6, 2026
Merged

docs: describe how transforms reach the pod today #107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/DESIGN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ Vector is powerful but opaque. This wrapper makes it behave like every other DFE
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ Volumes: │
│ • /etc/dfe/config.yaml (ConfigMap — big-dial config) │
│ • /etc/dfe/transforms/ (init container — from Artifactory) │
│ • /etc/dfe-transform-vector/config.yaml (ConfigMap, big dials) │
│ • /etc/dfe-transform-vector/transforms/ (empty dir from image) │
│ • /var/run/vector/config/ (emptyDir — assembled Vector config) │
└──────────────────────────────────────────────────────────────────┘
```
Expand Down Expand Up @@ -837,7 +837,7 @@ affinity: {}
| Workload type | StatefulSet | StatefulSet | Deployment |
| Health probes | /livez, /readyz | /livez, /readyz | /livez, /readyz |
| Metrics port | 9090 | 9090 | 9090 |
| ConfigMap | config.yaml | config.yaml | config.yaml + transforms/ |
| ConfigMap | config.yaml | config.yaml | config.yaml |
| PVC | data dir | data dir | None (Deployment) |
| KEDA ScaledObject | Kafka lag | Kafka lag | Kafka lag |
| PodMonitor | Yes | Yes | Yes |
Expand Down
6 changes: 3 additions & 3 deletions docs/MIGRATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ keda:

Key differences:

- **No inline VRL** — transforms are fetched by the init container from Artifactory
- **No inline VRL** — transforms are separate YAML files the wrapper loads from `config.transforms.dir` (the image's `/etc/dfe-transform-vector/transforms`) or lists in `config.transforms.files`
- **No persistence** — Deployment (not StatefulSet), data dir loss just re-reads from Kafka
- **SASL/TLS** inherited from `common.yaml` — only override pipeline-specific values
- **KEDA** configured declaratively, inherits defaults from common.yaml
Expand All @@ -102,10 +102,9 @@ Key differences:

### What Stays the Same

- **Secrets**: the chart creates none. The Kafka SASL secret and any artefact-store credential come from wherever your deployment already supplies them
- **Secrets**: the chart creates none. The Kafka SASL secret comes from wherever your deployment already supplies it
- **KEDA TriggerAuthentication**: keeps pointing at the same Kafka credential secret
- **Node placement**: set the chart's `nodeSelector` and `tolerations` to match your existing pool
- **Transform YAML delivery**: Same Artifactory ZIP download pattern via init container

### What Changes

Expand All @@ -117,6 +116,7 @@ Key differences:
| Health endpoint | `/health` on :8686 | `/livez`, `/readyz` on :9090, beside `/metrics` |
| Metrics | Vector native on :9090 | Wrapper + merged Vector on :9090 |
| Config format | Full Vector YAML (customConfig) | Big-dial config (source/sink/transforms) |
| Transform delivery | Inline VRL in `customConfig` | YAML files in `transforms.dir` or listed in `transforms.files`. The chart has no init container or download step: bake the files into your image or mount them at that path |
| DAG wiring | Manual (inline in values) | Auto-wired by wrapper |
| Hot-reload | Not supported | Poll-based file watcher + SIGHUP |
| Crash recovery | K8s pod restart only | Wrapper restarts Vector with backoff |
Expand Down
Loading