K8s Resource Management

Kubernetes resource management is built around a declarative model: users describe what an application needs, and the control plane continuously reconciles the declared state with the cluster’s actual capacity. This model makes workloads portable, schedulable, and easier to operate at scale.

The central idea is to separate resource requests, which guide scheduling, from resource limits, which enforce runtime boundaries. When these settings are aligned with real application behavior, Kubernetes can improve workload density without turning resource contention into outages.
Resource management operates at two connected levels:

• Node level: Each kubelet reports the node’s capacity and allocatable resources to the control plane. Allocatable capacity excludes resources reserved for system workloads and the kubelet itself.
• Pod level: The scheduler evaluates each Pod’s resource requests against the allocatable capacity remaining on candidate nodes. It then selects a node where the Pod can fit according to its requests, affinity rules, topology constraints, and other scheduling conditions.

Requests and limits serve different purposes. A request helps the scheduler make a placement decision; a limit is enforced by the container runtime after the workload is running.

For example:

apiVersion: v1
kind: Pod
metadata:
  name: reporting-service
spec:
  containers:
    - name: application
      image: company/reporting-service:1.4.0
      resources:
        requests:
          cpu: 250m
          memory: 512Mi
        limits:
          cpu: "1"
          memory: 1Gi

Here, 250m represents 250 millicores. The scheduler treats the Pod as requiring 250 millicores of CPU and 512 MiB of memory when choosing a node. The CPU limit permits short bursts above the request, while the memory limit establishes a firm upper boundary. Setting the limit too close to normal demand can create throttling; setting memory limits too generously may reduce node density.

Quality of Service and Eviction Priority

Kubernetes assigns a Quality of Service (QoS) class based on how CPU and memory requests and limits are configured.

• Guaranteed: Every container defines equal CPU and memory requests and limits. These workloads receive the strongest protection from node resource pressure.
• Burstable: The Pod does not meet the Guaranteed criteria but defines at least one resource request. It can use additional resources when they are available and may be throttled or reclaimed when necessary.
• BestEffort: No CPU or memory requests or limits are defined. These Pods have the lowest resource guarantees and are typically the first to face eviction pressure.

QoS influences eviction behavior, but it is not the only factor. Pod priority, node pressure, disruption policies, and available resources also matter. Even so, avoiding BestEffort workloads in production is usually prudent because a single unconstrained process can consume an entire node and affect every other workload running there.

Controlling Resource Consumption by Namespace

For shared clusters, namespace-level policies provide a practical governance layer.

A ResourceQuota caps the aggregate resources and object counts that a namespace may consume. It can limit total CPU, memory, persistent storage, and the number of Pods or other objects created within the namespace. Quotas prevent one tenant from exhausting shared cluster capacity, although they do not physically reserve resources for that tenant.

A LimitRange establishes defaults and validation rules for individual containers. It can automatically apply requests and limits, require certain resource declarations, and enforce minimum or maximum values. Together, these controls create predictable workload behavior even when application teams do not provide complete resource specifications.

For example, an organization might allow application teams to request CPU freely but cap a namespace’s memory at 100 GiB. A LimitRange could also inject a default request and limit into containers that omit them, ensuring that workloads do not silently become BestEffort.

Storage, Network, and Specialized Hardware

Resource management extends beyond CPU and memory.

• Storage: PersistentVolumeClaims, or PVCs, allow applications to request persistent storage without depending on the implementation details of a particular storage system. StorageClasses can define how volumes are dynamically provisioned, while access modes and capacity requirements express the application’s needs.
• Networking: Services expose applications through stable virtual addresses and load balancing. Ingress resources route external HTTP or HTTPS traffic to Services. These abstractions separate application consumers from the addresses of individual Pods.
• Extended resources: Device plugins advertise specialized hardware such as GPUs or FPGAs as extended resources. Pods request these resources in the same scheduling model used for CPU and memory, allowing the scheduler to place workloads only on nodes with the required devices.

Extended resources are generally allocated as indivisible units. This differs from CPU, which can be requested in small increments and shared over time.

Production Best Practices

Base Resource Settings on Measurements

Avoid copying arbitrary values from templates. Use load tests and production telemetry to establish a baseline for steady-state demand and peak demand.

A useful starting point is to set CPU requests near the application’s typical usage and memory requests near its stable working set. Limits should tolerate expected bursts without allowing a workload to monopolize the node.

Set Both Requests and Limits

Requests and limits protect different parts of the system. Requests support intelligent placement and fair CPU sharing, while limits contain runaway workloads. Neither should be treated as optional in a production namespace.

CPU limits should be high enough to avoid frequent throttling. Memory limits should leave enough headroom for traffic spikes, garbage collection, and temporary allocations. For highly variable workloads, horizontal scaling may be preferable to setting an excessively high per-Pod limit.

Combine Requests with Autoscaling

The Horizontal Pod Autoscaler (HPA) adjusts a workload’s replica count according to CPU, memory, or custom metrics. Resource-based utilization is calculated relative to requests, so accurate requests are essential for meaningful scaling decisions.

An HPA increases capacity when demand rises and can remove replicas when demand falls. To avoid oscillation, configure suitable scaling policies and stabilization windows. Resource requests must still be set correctly; autoscaling cannot compensate for missing or unrealistic baseline values.

Isolate Teams and Workloads

Use namespaces to separate teams, environments, or business domains. Apply ResourceQuota and LimitRange policies consistently so that developers receive sensible defaults without being able to bypass capacity boundaries.

For workloads with different business importance, combine quotas with PriorityClass resources. Priority affects preemption decisions and helps ensure that critical services retain capacity when a node becomes overloaded.

Monitor Saturation, Not Just Usage

Track CPU throttling, memory working sets, OOM terminations, eviction events, scheduling latency, and pending Pods. These signals reveal the difference between apparent resource usage and actual resource pressure.

It is also important to leave node-level headroom. A cluster scheduled to exactly 100% of its allocatable capacity has no room for rolling updates, node maintenance, or sudden traffic increases. Sustainable density usually requires deliberate reserve.

A Balanced Resource Strategy

Effective Kubernetes resource management is not simply about minimizing consumption. It is about establishing enforceable boundaries while preserving enough flexibility for workloads to operate efficiently.

Requests enable the scheduler to place workloads intelligently. Limits contain runtime behavior. QoS classes, namespace policies, autoscaling, storage abstractions, and device plugins extend that control across the platform. By grounding every setting in measured demand and monitoring the resulting behavior, teams can build clusters that are denser, more resilient, and easier to operate at scale.