From f6aecfca4c9de8220f2dce815b31e3eec0029f72 Mon Sep 17 00:00:00 2001 From: Russell Trow Date: Thu, 8 Oct 2026 10:54:42 +0100 Subject: [PATCH] Consolidate CPU utilization patterns into 'Match CPU capacity to average and peak utilization' Merges optimize-avg-cpu-utilization and optimize-peak-cpu-utilization (same author/date, verbatim-identical Description) into one pattern with average and peak techniques, rewritten Assumptions and Considerations, a new Cost Impact section, and cross-references to the shedding and queuing patterns. Originals moved to archive; redirects and persona data updated. Part of #434 (see #385). Co-Authored-By: Claude Sonnet 5.5 --- .../optimize-avg-cpu-utilization.md | 0 .../optimize-peak-cpu-utilization.md | 0 ...apacity-to-average-and-peak-utilization.md | 69 +++++++++++++++++++ docusaurus.config.js | 6 +- src/data/personas.ts | 8 +-- 5 files changed, 76 insertions(+), 7 deletions(-) rename {docs => archive}/operations/capacity-management/optimize-avg-cpu-utilization.md (100%) rename {docs => archive}/operations/capacity-management/optimize-peak-cpu-utilization.md (100%) create mode 100644 docs/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization.md diff --git a/docs/operations/capacity-management/optimize-avg-cpu-utilization.md b/archive/operations/capacity-management/optimize-avg-cpu-utilization.md similarity index 100% rename from docs/operations/capacity-management/optimize-avg-cpu-utilization.md rename to archive/operations/capacity-management/optimize-avg-cpu-utilization.md diff --git a/docs/operations/capacity-management/optimize-peak-cpu-utilization.md b/archive/operations/capacity-management/optimize-peak-cpu-utilization.md similarity index 100% rename from docs/operations/capacity-management/optimize-peak-cpu-utilization.md rename to archive/operations/capacity-management/optimize-peak-cpu-utilization.md diff --git a/docs/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization.md b/docs/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization.md new file mode 100644 index 000000000..7f2541ac1 --- /dev/null +++ b/docs/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization.md @@ -0,0 +1,69 @@ +--- +version: 1.0 +submitted_by: dubrie +published_date: 2022-11-10 +category: Operations +description: Provisioned CPU capacity has to cover peak demand, so the gap between average and peak utilization determines how much hardware sits idle. Matching CPU capacity to both average and peak utilization reduces idle resources while protecting performance at peak load. +tags: + - compute + - monitoring + - size:medium + - persona:devops-engineer + - persona:infrastructure-engineer + - persona:solution-architect +personas: DevOps Engineer, Infrastructure Engineer, Solution Architect +--- + +# Match CPU capacity to average and peak utilization + +## Description + +CPU demand varies through the day, sometimes sharply. Capacity must be available to absorb the highest spikes, so the larger the gap between average and peak CPU utilization, the more resources are held in stand-by and sit underused for most of the day. Idle servers still draw a large share of their peak power and carry embodied carbon, so a wide gap between average and peak means energy and hardware spent on capacity that does little useful work. + +The gap can be narrowed from both ends: by running typical load at a more efficient utilization level, and by flattening the spikes that force capacity to be provisioned. + +## Solution + +Start by measuring CPU utilization over a window that includes normal peaks, and record both the average and the peak. Then work on each end of the gap. + +**Raise average utilization to an efficient level.** Choose a target based on how quickly your system can add capacity. If it scales out in seconds or minutes, a higher average target is appropriate because spikes can be absorbed on demand. If scaling is slow or manual, a lower target is needed to keep a buffer. Consolidate or resize workloads so typical utilization sits near that target rather than far below it. + +**Reduce and flatten peaks.** Identify which processes and actions drive CPU spikes, for example a request that triggers many database queries, data processing and rendering, and reduce their cost. Typical techniques include adding caching, reducing the volume of data processed or transmitted, and deferring non-urgent work to quieter periods. Where demand itself exceeds capacity, shed or queue lower-priority work rather than provisioning for it. + +Use more hardware to spread load only when software-level efficiencies have been exhausted, since each additional server adds embodied carbon. + +## SCI Impact + +`SCI = (E * I) + M per R` +[Software Carbon Intensity Spec](https://grnsft.org/sci) + +- `E`: Decreases. Higher average utilization makes better use of the energy that idle capacity would otherwise draw, and lower peaks reduce the CPU work needed to serve the same traffic. +- `M`: Decreases. A smaller gap between average and peak means less stand-by hardware is provisioned, so less embodied carbon is attributed to the workload. This holds only if the work is not offset by adding extra servers to reduce load. + +## Cost Impact + +- **Compute costs decrease.** Fewer or smaller instances are needed when average utilization is higher and peaks are lower. Savings grow with the size of the average-to-peak gap. +- **Engineering and monitoring effort increases.** Profiling CPU hotspots, tuning scaling policies and maintaining utilization dashboards take ongoing effort. +- **Supporting infrastructure may add cost.** Caching layers and queues used to flatten peaks carry their own running costs, which should be weighed against the compute saved. +- **Poorly set targets can cost more than they save.** Too little headroom risks performance incidents and emergency over-provisioning. + +## Assumptions + +- Traffic fluctuates through normal production use, so average and peak utilization differ meaningfully. +- CPU utilization is monitored at a granularity and over a time window (for example, several weeks at one-minute resolution) that captures peaks as well as averages. +- The time your system takes to add capacity is known, so a utilization target can be set against it. +- Spikes caused by the system itself, such as runaway processes or inefficient jobs, are diagnosed and fixed separately rather than treated as demand. + +## Considerations + +- **Higher average utilization leaves less headroom.** Without fast scaling, raising the average increases the risk of degraded performance during unexpected spikes. Raise the target gradually and validate against real traffic. +- **No universal target exists.** The right utilization level depends on workload, scaling speed and latency requirements, so set it from your own measurements. +- **Shedding and queuing can degrade user experience.** Reducing peaks by delaying or dropping lower-priority requests means some users wait longer or are turned away. Apply it only to work that tolerates it. +- **CPU is not the only constraint.** Memory or I/O may limit consolidation before CPU does, so check them before raising targets. +- **Adding servers can defeat the purpose.** Spreading load across more hardware lowers per-server utilization and adds embodied carbon. + +## References + +- [Hardware Efficiency Principle](https://learn.greensoftware.foundation/practitioner/hardware-efficiency) +- [Energy Efficiency Principle](https://learn.greensoftware.foundation/practitioner/energy-efficiency) +- Related patterns: [Shed lower-priority traffic](/requirements/shed-lower-priority-traffic), [Queue non-urgent requests](/architecture/system-topology/queue-non-urgent-requests), [Scale infrastructure with user load](/operations/capacity-management/scale-infrastructure-with-user-load), [Use a circuit breaker](/operations/capacity-management/use-circuit-breaker) diff --git a/docusaurus.config.js b/docusaurus.config.js index 2604d7306..846d81d60 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -251,8 +251,10 @@ const config = { // Cloud → Operations: Capacity Management { from: "/catalog/cloud/match-utilization-requirements-of-vm", to: "/operations/capacity-management/match-utilization-requirements-of-vm" }, { from: "/catalog/cloud/match-utilization-requirements-with-pre-configured-server", to: "/operations/capacity-management/match-utilization-requirements-with-pre-configured-server" }, - { from: "/catalog/cloud/optimize-avg-cpu-utilization", to: "/operations/capacity-management/optimize-avg-cpu-utilization" }, - { from: "/catalog/cloud/optimize-peak-cpu-utilization", to: "/operations/capacity-management/optimize-peak-cpu-utilization" }, + { from: "/catalog/cloud/optimize-avg-cpu-utilization", to: "/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization" }, + { from: "/catalog/cloud/optimize-peak-cpu-utilization", to: "/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization" }, + { from: "/operations/capacity-management/optimize-avg-cpu-utilization", to: "/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization" }, + { from: "/operations/capacity-management/optimize-peak-cpu-utilization", to: "/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization" }, { from: "/catalog/cloud/scale-down-kubernetes-workloads", to: "/operations/capacity-management/scale-down-kubernetes-workloads" }, { from: "/catalog/cloud/scale-down-unused-applications", to: "/operations/capacity-management/scale-down-unused-applications" }, { from: "/catalog/cloud/scale-infrastructure-with-user-load", to: "/operations/capacity-management/scale-infrastructure-with-user-load" }, diff --git a/src/data/personas.ts b/src/data/personas.ts index 9237468c2..35a016eef 100644 --- a/src/data/personas.ts +++ b/src/data/personas.ts @@ -73,8 +73,7 @@ export const personas: Persona[] = [ { title: "Match utilization requirements of virtual machines (VMs)", permalink: "/operations/capacity-management/match-utilization-requirements-of-vm", description: "It's better to have one VM running at a higher utilization than two running at low utilization rates, not only in terms of energy proportionality but also in terms of embodied carbon. Two servers running at low utilization rates will consume more energy than one running at a high utilization rate. In addition, the unused capacity on the underutilized server could be more efficiently used for another task or process.", category: "Operations", tags: ["compute","size:small"] }, { title: "Match utilization requirements with pre-configured servers", permalink: "/operations/capacity-management/match-utilization-requirements-with-pre-configured-server", description: "It's better to have one VM running at a higher utilization than two running at low utilization rates, not only in terms of energy proportionality but also in terms of embodied carbon. Two servers running at low utilization rates will consume more energy than one running at a high utilization rate. In addition, the unused capacity on the underutilized server could be more efficiently used for another task or process.", category: "Operations", tags: ["compute","size:small"] }, { title: "Optimise storage utilization", permalink: "/operations/resource-lifecycle/optimise-storage-resource-utilisation", description: "It's better to maximise storage utilisation so the storage layer is optimised for the task, not only in terms of energy proportionality but also in terms of embodied carbon. Two storage units running at low utilization rates will consume more energy than one running at a high utilization rate. In addition, the unused capacity on the underutilised storage unit could be more efficiently used for another task or process.", category: "Operations", tags: ["storage","size:small"] }, - { title: "Optimize average CPU utilization", permalink: "/operations/capacity-management/optimize-avg-cpu-utilization", description: "CPU usage and utilization varies throughout the day, sometimes wildly for different computational requirements. The larger the variance between the average and peak CPU utilization values, the more resources need to be provisioned in stand-by mode to absorb those spikes in traffic.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, - { title: "Optimize peak CPU utilization", permalink: "/operations/capacity-management/optimize-peak-cpu-utilization", description: "CPU usage and utilization varies throughout the day, sometimes wildly for different computational requirements. The larger the variance between the average and peak CPU utilization values, the more resources need to be provisioned in stand-by mode to absorb those spikes in traffic.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, + { title: "Match CPU capacity to average and peak utilization", permalink: "/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization", description: "Provisioned CPU capacity has to cover peak demand, so the gap between average and peak utilization determines how much hardware sits idle. Matching CPU capacity to both average and peak utilization reduces idle resources while protecting performance at peak load.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, { title: "Remove unused assets", permalink: "/operations/resource-lifecycle/remove-unused-assets", description: "Unused cloud resources such as databases, storage buckets, and compute instances continue to consume energy and generate embodied carbon; identifying and decommissioning them eliminates unnecessary waste.", category: "Operations", tags: ["cloud","size:small"] }, { title: "Scale down applications when not in use", permalink: "/operations/capacity-management/scale-down-unused-applications", description: "Applications consume CPU even when they are not actively in use. For example, background timers, garbage collection, health checks, etc. Even when the application is shut down, the underlying hardware is consuming idle power.", category: "Operations", tags: ["cloud","size:small"] }, { title: "Scale down kubernetes applications when not in use", permalink: "/operations/capacity-management/scale-down-kubernetes-workloads", description: "In order to reduce carbon emissions and costs, Dev&Test Kubernetes clusters can turn off nodes out of office hours. Thereby, optimization is implemented at the cluster level. For production clusters, where nodes need to stay up and running, optimization needs to be implemented at the application level.", category: "Operations", tags: ["kubernetes","size:small"] }, @@ -125,7 +124,7 @@ export const personas: Persona[] = [ { title: "Use sustainable regions for AI/ML training", permalink: "/development/leverage-sustainable-regions", description: "Depending on the model parameters and training iterations, training an AI/ML model consumes a lot of power and requires many servers which contribute to embodied emissions.", category: "Development", tags: ["ai","machine-learning","size:small"] }, { title: "Match utilization requirements of virtual machines (VMs)", permalink: "/operations/capacity-management/match-utilization-requirements-of-vm", description: "It's better to have one VM running at a higher utilization than two running at low utilization rates, not only in terms of energy proportionality but also in terms of embodied carbon. Two servers running at low utilization rates will consume more energy than one running at a high utilization rate. In addition, the unused capacity on the underutilized server could be more efficiently used for another task or process.", category: "Operations", tags: ["compute","size:small"] }, { title: "Match utilization requirements with pre-configured servers", permalink: "/operations/capacity-management/match-utilization-requirements-with-pre-configured-server", description: "It's better to have one VM running at a higher utilization than two running at low utilization rates, not only in terms of energy proportionality but also in terms of embodied carbon. Two servers running at low utilization rates will consume more energy than one running at a high utilization rate. In addition, the unused capacity on the underutilized server could be more efficiently used for another task or process.", category: "Operations", tags: ["compute","size:small"] }, - { title: "Optimize average CPU utilization", permalink: "/operations/capacity-management/optimize-avg-cpu-utilization", description: "CPU usage and utilization varies throughout the day, sometimes wildly for different computational requirements. The larger the variance between the average and peak CPU utilization values, the more resources need to be provisioned in stand-by mode to absorb those spikes in traffic.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, + { title: "Match CPU capacity to average and peak utilization", permalink: "/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization", description: "Provisioned CPU capacity has to cover peak demand, so the gap between average and peak utilization determines how much hardware sits idle. Matching CPU capacity to both average and peak utilization reduces idle resources while protecting performance at peak load.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, { title: "Scale down applications when not in use", permalink: "/operations/capacity-management/scale-down-unused-applications", description: "Applications consume CPU even when they are not actively in use. For example, background timers, garbage collection, health checks, etc. Even when the application is shut down, the underlying hardware is consuming idle power.", category: "Operations", tags: ["cloud","size:small"] }, { title: "Scale down kubernetes applications when not in use", permalink: "/operations/capacity-management/scale-down-kubernetes-workloads", description: "In order to reduce carbon emissions and costs, Dev&Test Kubernetes clusters can turn off nodes out of office hours. Thereby, optimization is implemented at the cluster level. For production clusters, where nodes need to stay up and running, optimization needs to be implemented at the application level.", category: "Operations", tags: ["kubernetes","size:small"] }, { title: "Time-shift Kubernetes cron jobs", permalink: "/operations/time-shift-kubernetes-cron-jobs", description: "The carbon emissions of a software system depends on the power consumed by that software, but also on the Carbon intensity of the electricity it is powered on. For this reason, running energy-efficient software on carbon intensive electricity grid, might be inefficient to reduce its global carbon emissions. Carbon aware time scheduling, is about scheduling workloads to execute, when electricity carbon intensity is low.", category: "Operations", tags: ["kubernetes","size:medium"] }, @@ -181,8 +180,7 @@ export const personas: Persona[] = [ { title: "Use serverless cloud services", permalink: "/architecture/technology-selection/use-serverless", description: "Serverless cloud services scale dynamically with demand and share infrastructure across many applications, reducing idle resource consumption and lowering embodied carbon emissions.", category: "Architecture", tags: ["cloud","size:small","serverless"] }, { title: "Minimize the total number of deployed environments", permalink: "/development/minimizing-deployed-environments", description: "In a given application, there may be a need to utilize multiple environments in the application workflow. Typically, a development environment is used for regular updates, while staging or testing enviroments are used to make sure there are no issues before code reaches a production environment where users may have access. Each added environment has an increasing energy impact, which in turn creates more emissions. As such, it is important to understand the necessity of each enviroment and it's environmental impact.", category: "Development", tags: ["cloud","deployment"] }, { title: "Optimise storage utilization", permalink: "/operations/resource-lifecycle/optimise-storage-resource-utilisation", description: "It's better to maximise storage utilisation so the storage layer is optimised for the task, not only in terms of energy proportionality but also in terms of embodied carbon. Two storage units running at low utilization rates will consume more energy than one running at a high utilization rate. In addition, the unused capacity on the underutilised storage unit could be more efficiently used for another task or process.", category: "Operations", tags: ["storage","size:small"] }, - { title: "Optimize average CPU utilization", permalink: "/operations/capacity-management/optimize-avg-cpu-utilization", description: "CPU usage and utilization varies throughout the day, sometimes wildly for different computational requirements. The larger the variance between the average and peak CPU utilization values, the more resources need to be provisioned in stand-by mode to absorb those spikes in traffic.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, - { title: "Optimize peak CPU utilization", permalink: "/operations/capacity-management/optimize-peak-cpu-utilization", description: "CPU usage and utilization varies throughout the day, sometimes wildly for different computational requirements. The larger the variance between the average and peak CPU utilization values, the more resources need to be provisioned in stand-by mode to absorb those spikes in traffic.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, + { title: "Match CPU capacity to average and peak utilization", permalink: "/operations/capacity-management/match-cpu-capacity-to-average-and-peak-utilization", description: "Provisioned CPU capacity has to cover peak demand, so the gap between average and peak utilization determines how much hardware sits idle. Matching CPU capacity to both average and peak utilization reduces idle resources while protecting performance at peak load.", category: "Operations", tags: ["compute","monitoring","size:medium"] }, { title: "Scale infrastructure with user load", permalink: "/operations/capacity-management/scale-infrastructure-with-user-load", description: "Demand for resources depends on user load at any given time. However, most applications run without taking this into consideration. As a result,resources are underused and inefficient.", category: "Operations", tags: ["compute","cloud","size:medium"] }, { title: "Scale Kubernetes workloads based on relevant demand metrics", permalink: "/operations/capacity-management/scale-kubernetes-workloads-based-on-events", description: "By default, Kubernetes scales workloads based on CPU and RAM utilization. In practice, however, it's difficult to correlate your application's demand drivers with CPU and RAM utilization. Scaling your workload based on relevant demand metrics that drive scaling of your applications, such as HTTP requests, queue length, and cloud alerting events can help reduce resource utilization, and therefore also your carbon emissions.", category: "Operations", tags: ["cloud","serverless","kubernetes","role:software-engineer","role:cloud-engineer","size:medium"] }, { title: "Match your service level objectives to business needs", permalink: "/requirements/match-slo", description: "If service downtimes are acceptable it's better to not strive for highest availability but to design the solution according to real business needs. Lower availability guarantees can help reduce energy consumption by using less infrastructure components.", category: "Requirements", tags: ["Cloud","Reliability/Availability","Monitoring"] },