Autoscaling your NKE workload

Autoscaling enables you to worry less about capacity planning and ensures the uptime of your services during load peaks. All this while you only pay what resources are needed at any given moment.

Details

With autoscaling configured, NKE automatically adds new node(s) to your cluster if you've created new Pods that don't have enough capacity to run. Conversely, if a node in your cluster is underutilized and its Pods can be run on other nodes, the autoscaler can delete the node.

Keep in mind that when resources are deleted or moved in the course of autoscaling your cluster, your services can experience some disruption. For example, if your service consists of a controller with a single replica, that replica's Pod might be restarted on a different node if its current node is deleted. Before enabling autoscaling, ensure that your services can tolerate potential disruption or that they are designed and configured so that downscaling does not disrupt Pods that cannot be interrupted.

Availability

Autoscaling can be configured on a node pool level in Cockpit. But there are a few more things that have to be configured in order to automatically scale your workload.

Usage

Scaling your workloads horizontally

Configure the minimum and maximum node count: By default we won't just scale your cluster to an infinite amount of nodes to guard you from unexpected costs. You can set the min and max count of nodes on a per node pool basis and the autoscaler will scale within these boundaries.
Set CPU requests on your pods: The cluster autoscaler is using this as a base to calculate how much free capacity a node has. Without setting CPU requests the cluster autoscaler does not function. Plus this is good practice regardless if you make use of the autoscaler or not.
Setup a Horizontal Pod Autoscaler: To scale your pods with the incoming load you can setup a Horizontal Pod Autoscaler (HPA) to scale the pods on the CPU utilization. As soon as your nodes are full this will in turn trigger the cluster autoscaler to add more nodes. The Kubernetes documentation has a great walkthrough to help you setup a HPA. On a vCluster, you must manually install metrics-server before configuring CPU- or memory-based HPAs.

Scaling on custom metrics

There is the possibility to scale horizontally by leveraging an optional managed installation of KEDA. KEDA allows to get metrics from various backends (called 'scalers' in KEDA terms) and to scale based on them. You can deploy KEDA on your cluster using Cockpit.

You can find the available scalers in KEDA's documentation. KEDA also allows to scale on own custom metrics, by providing a self written external scaler.

Scaling on Metrics Agent Metrics

If you collect metrics with a Metrics Agent, you can use the Prometheus scaler to scale on them. KEDA queries the metrics directly from the central metrics store, so you don't need to run a Prometheus instance in your cluster.

The metrics store requires OAuth2 authentication. KEDA authenticates with the credentials of an API service account with the metrics-viewer role.

Create an API service account with the metrics-viewer role in the project your Metrics Agent belongs to:

apiVersion: iam.nine.ch/v1alpha1
kind: APIServiceAccount
metadata:
  name: keda
  namespace: <project>
spec:
  forProvider:
    role: metrics-viewer

nctl apply -f apiserviceaccount.yaml

Store the credentials of the API service account in a Secret in the namespace of the workload you want to scale:

kubectl -n <namespace> create secret generic keda-metrics \
  --from-literal=client_id="$(nctl get apiserviceaccount keda --print-client-id)" \
  --from-literal=client_secret="$(nctl get apiserviceaccount keda --print-client-secret)" \
  --from-literal=token_url="$(nctl get apiserviceaccount keda --print-token-url)"

Create a TriggerAuthentication that references the Secret, and a ScaledObject that uses it:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-metrics
  namespace: <namespace>
spec:
  secretTargetRef:
    - parameter: oauthTokenURI
      name: keda-metrics
      key: token_url
    - parameter: clientID
      name: keda-metrics
      key: client_id
    - parameter: clientSecret
      name: keda-metrics
      key: client_secret
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: my-app
  namespace: <namespace>
spec:
  scaleTargetRef:
    name: my-app
  minReplicaCount: 1
  maxReplicaCount: 10
  triggers:
    - type: prometheus
      metadata:
        serverAddress: https://metrics.nineapis.ch/select/organization/<organization>/project/<project>/prometheus
        query: sum(rate(http_requests_total{namespace="<namespace>", service="my-app"}[2m]))
        threshold: "10"
        authModes: oauth
      authenticationRef:
        name: keda-metrics

Replace <organization> and <project> with the organization and project of your Metrics Agent. KEDA requests a new access token whenever the current one expires.

All clusters of a project send their metrics to the same store. If your project contains more than one cluster, filter your query on the nine_resource_name label, which contains the name of the cluster, for example http_requests_total{nine_resource_name="my-cluster"}.

To troubleshoot a ScaledObject, run kubectl describe scaledobject <name>. Test your query in Grafana first to make sure it returns a single value.

note

Scaling on Metrics Agent metrics requires KEDA version 2.21 or later.

Details​

Availability​

Usage​

Scaling your workloads horizontally​

Scaling on custom metrics​

Scaling on Metrics Agent Metrics​