diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a923c59..01626a2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -26,3 +26,16 @@ jobs: - uses: azure/setup-helm@v4 - run: helm lint deploy/helm/vbilling - run: helm template vbilling deploy/helm/vbilling --set adapters='{stripe}' --set stripe.existingSecret=vbilling-stripe > /dev/null + + gke-stripe: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Check for accidentally committed demo credentials + run: python3 scripts/check-demo-secrets.py --tracked + - name: Test the secret check with synthetic fixtures + run: python3 scripts/check-demo-secrets.py --self-test + - name: Test demo helpers without cloud credentials + run: python3 -m unittest discover -s examples/gke-stripe/tests -v + - name: Check local launcher syntax + run: bash -n examples/gke-stripe/scripts/run-local.sh diff --git a/.gitignore b/.gitignore index 014f4a5..d34ee04 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,24 @@ stripe-webhook-secret metronome-token lago-api-key webhook-secret + +# Local demo credentials and generated runtime state (never commit) +.env.* +!.env.example +*.kubeconfig +*.kubeconfig.* +kubeconfig +api-token +private-tenants.json +/examples/gke-gpu/join.txt +/examples/gke-gpu/startup.sh +/examples/gke-gpu/agent.yaml +/examples/gke-stripe/.runtime/ + +# Demo helper bytecode and CLI authentication files +__pycache__/ +*.py[cod] +vcluster-cli.json +platform-cli.json +private-join.txt +private-startup.sh diff --git a/README.md b/README.md index 250ef14..456523c 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ vBilling is the **metering pipe**, not the billing engine. It turns what runs in > vBilling lives in `vClusterLabs-Experiments`: it is an experimental project, not a supported vCluster Labs product. +**[GPU metering with Stripe](examples/gke-stripe/README.md)** provides a step-by-step setup for shared L4 GPUs, optional private A100 nodes, usage delivery to Stripe test mode, and resource cleanup. Credentials and runtime data stay outside the repository. + ## What's new in v0.2 - **Stripe and Metronome adapters.** Use Stripe Billing Meters on their own, or Metronome for rating with Stripe for payment, invoicing and tax. vBilling creates the Stripe customer and links it in Metronome. @@ -290,7 +292,7 @@ Override any mapping with `ENFORCEMENT_RULES`, for example `alerts.spend_thresho - **`annotate`**: also annotate the tenant cluster, label its namespace `vbilling.vcluster.com/billing-state`, and emit a Kubernetes event. - **`enforce`**: also label suspended namespaces `vbilling.vcluster.com/suspended=true`. With `enforcement.admissionPolicy.enabled=true` (Kubernetes 1.30+), a ValidatingAdmissionPolicy then denies **new** pods in those namespaces. Running workloads keep running, the tenant sees the reason as a sync error inside their tenant cluster, and a paid invoice lifts it automatically. The policy acts in the control plane cluster, so it holds tenant clusters on shared or dedicated nodes; tenant clusters with their own nodes (Private Nodes, Auto Nodes, Standalone) run their pods only inside the tenant cluster, so act on their billing state with your own automation. -Payment failures on invoices that Stripe collects arrive through Stripe's webhooks, so in Metronome + Stripe mode, send Stripe's webhooks to vBilling as well. Billing states are enabled once `STRIPE_WEBHOOK_SECRET` or `METRONOME_WEBHOOK_SECRET` is set. +Payment failures on invoices that Stripe collects arrive through Stripe's webhooks, so in Metronome + Stripe mode, send Stripe's webhooks to vBilling as well. Billing states are always available: Stripe and Metronome webhooks set them once `STRIPE_WEBHOOK_SECRET` or `METRONOME_WEBHOOK_SECRET` is set, and `PUT /api/v1/billing-states/{tenant}` sets them from anything else, such as Lago or a custom billing platform. ## Custom usage: tokens, Slurm, storage, network diff --git a/cmd/vbilling/main.go b/cmd/vbilling/main.go index bb1b1cc..74a9b82 100644 --- a/cmd/vbilling/main.go +++ b/cmd/vbilling/main.go @@ -198,11 +198,11 @@ func main() { log.Println("vBilling stopped") } -// newEnforcer enables billing-state enforcement when a webhook secret is set. +// newEnforcer sets up billing states. They are always available: Stripe and +// Metronome webhooks set them once their secrets are configured, and +// PUT /api/v1/billing-states/{tenant} sets them from anywhere else, such as +// Lago or a custom billing platform. func newEnforcer(cfg *config.Config, kube kubernetes.Interface, ctrl *controller.Controller, dests []destinations.Destination, tel *telemetry.Registry) *enforcement.Enforcer { - if cfg.StripeWebhookSecret == "" && cfg.MetronomeWebhookSecret == "" { - return nil - } rules, err := enforcement.ParseRules(cfg.EnforcementRules) if err != nil { log.Fatalf("ENFORCEMENT_RULES: %v", err) @@ -225,7 +225,8 @@ func newEnforcer(cfg *config.Config, kube kubernetes.Interface, ctrl *controller if err != nil { log.Fatalf("open billing state: %v", err) } - log.Printf("Billing-state enforcement enabled (mode=%s)", cfg.EnforcementMode) + log.Printf("Billing states: mode=%s, Stripe webhooks %s, Metronome webhooks %s", cfg.EnforcementMode, + onOff(cfg.StripeWebhookSecret != ""), onOff(cfg.MetronomeWebhookSecret != "")) return enforcement.New(kube, enforcement.Options{ Mode: cfg.EnforcementMode, StripeWebhookSecret: cfg.StripeWebhookSecret, @@ -269,3 +270,10 @@ func getKubeConfig() (*rest.Config, error) { log.Printf("Using kubeconfig (%s)", os.Getenv("KUBECONFIG")) return cfg, nil } + +func onOff(b bool) string { + if b { + return "on" + } + return "off" +} diff --git a/docs/custom-adapter.html b/docs/custom-adapter.html index 6800698..0b7cf1a 100644 --- a/docs/custom-adapter.html +++ b/docs/custom-adapter.html @@ -378,7 +378,7 @@

Responses, retries and dead letters

!
-
Deduplicate on id. A batch can arrive twice: after a timeout, a restart before the cursor was saved, a dead-letter replay or a cursor rewind. A usage event keeps its ID for the same window, so upsert on it. Tenant and catalog IDs carry a timestamp: keep the latest. Return 2xx for types you ignore, because vBilling holds all delivery to the endpoint until it accepts the catalog event.
+
Deduplicate on id. A batch can arrive twice: after a timeout, a restart before the cursor was saved, a dead-letter replay or a cursor rewind. A usage event keeps its ID for the same window, so upsert on it. Tenant and catalog IDs carry a timestamp: keep the latest. Return 2xx for types you ignore. A catalog event your endpoint rejects with 400, 413 or 422 is logged and skipped; any other failure holds delivery to the endpoint until the catalog event is accepted.

Dead letters stay on the ledger volume until a replay clears them. Add -H "Authorization: Bearer $API_TOKEN" when the API is protected:

curl -s localhost:8080/api/v1/destinations/webhook/dead-letters \ @@ -436,7 +436,7 @@

Go adapter

EventFilterAccepts(metric string) boolSkip metrics the platform should not see. destinations.BillableOnly drops the informational ones. Skipped events are counted, never retried. ReconcilerRecordedTotals(ctx context.Context, q TotalsQuery) ([]Total, error)Report what the platform recorded per tenant and metric, so GET /api/v1/reconcile can compare it with the ledger. - CustomerResolverTenantForCustomer(ctx context.Context, customerID string) (string, bool)Map a platform customer ID back to a tenant for inbound billing webhooks. Only the Stripe and Metronome adapters' webhooks (/webhooks/stripe, /webhooks/metronome) consult it today. + CustomerResolverTenantForCustomer(ctx context.Context, customerID string) (string, bool)Map a platform customer ID back to a tenant for inbound billing webhooks. Only the Stripe and Metronome adapters' webhooks (/webhooks/stripe, /webhooks/metronome) consult it today; any other billing platform sets billing states with PUT /api/v1/billing-states/{tenant}.
diff --git a/docs/example-gke.html b/docs/example-gke.html index 395e5d0..8587375 100644 --- a/docs/example-gke.html +++ b/docs/example-gke.html @@ -277,6 +277,8 @@

Example: GKE with shared and private GPU nodes

Both vCluster tenancy models on GPUs, metered by vBilling and invoiced by Stripe test mode, from an empty Google Cloud project. The manifests, values files and commands are the ones from a recorded run, with every key, token and account ID left out.

+

For a setup without Prometheus, see GPU metering with Stripe. It uses two L4 sharing slots, local vBilling, and paused Stripe test subscriptions. This example uses four slots, Prometheus/DCGM, and automatic subscriptions.

+

What you build