Skip to content

feat: add bounded game performance and relay monitoring - #771

Open
Baylem wants to merge 20 commits into
GameplanePanel:masterfrom
Baylem:fix/audit-observability
Open

Baylem wants to merge 20 commits into
GameplanePanel:masterfrom
Baylem:fix/audit-observability

Conversation

@Baylem

@Baylem Baylem commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Operators need to distinguish missing measurements, game degradation, resource pressure and relay failures. This adds cached numeric game-performance measurements and relay metrics, with a complete Prometheus configuration and a real policy-enforced scrape scenario in CI.

Modules can explicitly declare tick/frame rates or tick duration, units and an optional degradation threshold. One bounded source feeds both dashboard routes and Prometheus; scrapes never execute RCON. Unknown, invalid and stale readings have no value series, while genuine zero remains visible. Source RCON receives an aggregate response limit and an absolute exchange deadline so a streaming response cannot defeat the collector's resource bounds.

Tunnel metrics consume the same sanitized health evidence as readiness. The chart adds scrape-policy validation, tunnel discovery, pressure/game/relay alerts and a workload dashboard. The documented Prometheus example aligns monitor labels, discovery namespaces and ingress permissions; explicitly managed external policies remain supported. The CI fixture uses Prometheus Operator and Cilium, verifies actual agent/tunnel scraping, revokes/restores ingress and checks denial from an unapproved namespace. Promtool fixtures cover alert firing/recovery and duplicate operator readings.

No game module is automatically opted in. Playit's pinned client still reports unknown live connectivity, and infrastructure rules require kubelet/cAdvisor and kube-state-metrics collectors as documented.

Includes #768 relay health and its #765/#762 prerequisites, #763 network isolation, and #759 shared web build correction.

Validation: Go production/regression-source and web type compilation passed; required actionlint/offline zizmor and whitespace checks passed. Runtime tests, Prometheus policy enforcement and rule evaluation remain for CI under the repository's compile-only local policy.

The NuclearOption RCON client also applies its absolute command deadline before writing the request, preventing a peer that stops reading from holding the shared client indefinitely. Both blocked-write stages have regression coverage. Monitoring selector validation follows the operator NetworkPolicy gate, so standalone panel monitoring remains usable without a game scrape selector.

Closes #725.

Baylem added 14 commits October 21, 2026 10:59
Signed-off-by: Baylem <zelathis@gmail.com>
…cement

Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 122 files, which is 22 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Check out review usage here.

⚙️ Run configuration
  • Configuration used: Repository: GameplanePanel/Gameplane/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 73f690d0-82fa-4837-93f6-fb70a0cc6c77
📥 Commits

Reviewing files that changed from the base of the PR and between df1e371 and 7b794f2.

⛔ Files ignored due to path filters: 3
  • go.work is excluded by !**/*.work
  • go.work.sum is excluded by !**/*.sum
  • tunnel/go.sum is excluded by !**/*.sum
📒 Files selected for processing: 122
  • .github/workflows/ci.yaml
  • agent/cmd/main.go
  • agent/internal/caps/caps.go
  • agent/internal/rcon/nuclearoption.go
  • agent/internal/rcon/nuclearoption_deadline_test.go
  • agent/internal/rcon/nuclearoption_test.go
  • agent/internal/rcon/rcon.go
  • agent/internal/rcon/rcon_bounds_test.go
  • agent/internal/status/performance.go
  • agent/internal/status/performance_test.go
  • agent/internal/status/status.go
  • agent/specs.md
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_backups.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_backupschedules.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_clusters.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_gameservers.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_gametemplates.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_modules.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_modulesources.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_networkcaptures.yaml
  • charts/gameplane/charts/gameplane-crds/crds/gameplane.local_restores.yaml
  • charts/gameplane/crd-manifests/gameplane.local_backups.yaml
  • charts/gameplane/crd-manifests/gameplane.local_backupschedules.yaml
  • charts/gameplane/crd-manifests/gameplane.local_clusters.yaml
  • charts/gameplane/crd-manifests/gameplane.local_gameservers.yaml
  • charts/gameplane/crd-manifests/gameplane.local_gametemplates.yaml
  • charts/gameplane/crd-manifests/gameplane.local_modules.yaml
  • charts/gameplane/crd-manifests/gameplane.local_modulesources.yaml
  • charts/gameplane/crd-manifests/gameplane.local_networkcaptures.yaml
  • charts/gameplane/crd-manifests/gameplane.local_restores.yaml
  • charts/gameplane/dashboards/gameplane-workloads.json
  • charts/gameplane/templates/NOTES.txt
  • charts/gameplane/templates/_helpers.tpl
  • charts/gameplane/templates/api.yaml
  • charts/gameplane/templates/audit-syslog-bridge.yaml
  • charts/gameplane/templates/crd-apply-hook.yaml
  • charts/gameplane/templates/gateway.yaml
  • charts/gameplane/templates/grafana-dashboard.yaml
  • charts/gameplane/templates/mcp-server.yaml
  • charts/gameplane/templates/monitoring-rules.yaml
  • charts/gameplane/templates/monitoring-validation.yaml
  • charts/gameplane/templates/networkpolicies.yaml
  • charts/gameplane/templates/operator-strategy-migrate.yaml
  • charts/gameplane/templates/operator.yaml
  • charts/gameplane/templates/poddisruptionbudgets.yaml
  • charts/gameplane/templates/servicemonitors.yaml
  • charts/gameplane/templates/telemetry-receiver.yaml
  • charts/gameplane/templates/validate.yaml
  • charts/gameplane/templates/web.yaml
  • charts/gameplane/tests/monitoring-rules.sh
  • charts/gameplane/tests/monitoring-rules.test.yaml
  • charts/gameplane/values-production.example.yaml
  • charts/gameplane/values.yaml
  • charts/tests/check_gateway.py
  • charts/tests/check_network_isolation.py
  • charts/tests/check_standalone.py
  • charts/tests/check_workloads.py
  • deploy/kind/e2e.sh
  • deploy/kind/monitoring.sh
  • docs/examples/monitoring/gameplane-values.yaml
  • docs/examples/monitoring/prometheus.yaml
  • docs/module-authoring.md
  • docs/monitoring.md
  • docs/networking.md
  • docs/production-placement.md
  • docs/relay-health.md
  • docs/release-notes.md
  • docs/tailscale-forwarding-integration.md
  • docs/tunnels.md
  • hack/sync-chart-crds.sh
  • i18n/locales/en/app.json
  • operator/api/v1alpha1/gameserver_types.go
  • operator/api/v1alpha1/gametemplate_types.go
  • operator/api/v1alpha1/zz_generated.deepcopy.go
  • operator/cmd/main.go
  • operator/config/crd/gameplane.local_gameservers.yaml
  • operator/config/crd/gameplane.local_gametemplates.yaml
  • operator/hack/condition-ids/main.go
  • operator/internal/controller/auxiliary_resources.go
  • operator/internal/controller/auxiliary_resources_test.go
  • operator/internal/controller/backup_controller.go
  • operator/internal/controller/backup_forget.go
  • operator/internal/controller/gameserver_controller.go
  • operator/internal/controller/gameserver_relay_health_test.go
  • operator/internal/controller/gameserver_sentinel.go
  • operator/internal/controller/gameserver_status.go
  • operator/internal/controller/gameserver_tailscale_test.go
  • operator/internal/controller/gameserver_tunnel.go
  • operator/internal/controller/gameserver_tunnel_envtest_test.go
  • operator/internal/controller/gametemplate_performance_envtest_test.go
  • operator/internal/controller/restore_controller.go
  • operator/internal/controller/sentinel_isolation.go
  • operator/internal/controller/sentinel_isolation_test.go
  • operator/specs.md
  • specs/done_018-v0-3-release-readiness/OPEN-DECISIONS.md
  • test/e2e/buckets.sh
  • test/e2e/helm_install_e2e_test.go
  • test/e2e/management_placement_e2e_test.go
  • test/e2e/monitoring_e2e_test.go
  • test/e2e/network_isolation_e2e_test.go
  • test/e2e/optional_components_e2e_test.go
  • test/e2e/relay_health_e2e_test.go
  • test/e2e/specs.md
  • test/e2e/upgrade_e2e_test.go
  • tunnel/Dockerfile.frp
  • tunnel/Dockerfile.playit
  • tunnel/Dockerfile.tailscale
  • tunnel/go.mod
  • tunnel/main.go
  • tunnel/main_test.go
  • tunnel/playit_reporter.go
  • tunnel/relay_health.go
  • tunnel/relay_health_test.go
  • tunnel/relay_metrics.go
  • tunnel/relay_metrics_test.go
  • tunnel/specs.md
  • tunnel/tailscale.go
  • tunnel/tailscale_forward.go
  • tunnel/tailscale_forward_test.go
  • tunnel/tailscale_test.go
  • web/src/routes/CreateServer.tsx
  • web/src/types.ts

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added area: agent in-pod sidecar area: chart Helm chart and deployment area: e2e kind-based end-to-end suite area: modules game template OCI bundles area: operator controller-runtime operator and CRDs area: optional-components sentinel, capture-sidecar, mcp-server, tunnel, audit-syslog-bridge, telemetry-receiver area: shared netguard, gameaction, gameproto, svcutil and other shared packages area: specs spec-kit feature specs and design docs area: web React dashboard type: feature New capability or user-facing addition labels Oct 21, 2026
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: agent in-pod sidecar area: chart Helm chart and deployment area: e2e kind-based end-to-end suite area: modules game template OCI bundles area: operator controller-runtime operator and CRDs area: optional-components sentinel, capture-sidecar, mcp-server, tunnel, audit-syslog-bridge, telemetry-receiver area: shared netguard, gameaction, gameproto, svcutil and other shared packages area: specs spec-kit feature specs and design docs area: web React dashboard type: feature New capability or user-facing addition

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(observability): add game-performance and relay signals with a complete scrape setup

1 participant