Cluster Autoscaler

Introduction

Cluster Autoscaler is a component that automatically adjusts the size of a Kubernetes Cluster so that all pods have a place to run and there are no unneeded nodes.

The cluster autoscaler for CloudBit scales worker nodes within any specified CloudBit Kubernetes cluster.

Installation

As there is no concept of a node group within CloudBit Cloud's Kubernetes offering, the configuration required is quite simple. You need to set:

• Your CloudBit Application Token

• The Kubernetes Cluster's ID (not the name)

• The minimum and maximum number of worker nodes you want (the master is excluded)

1. Please adjust the following bold values in the yaml file below:

• Minimum & Maximum of worker nodes (for example minimum 3 and maximal nine): nodes=3:9:workers

• Generate an application token in https://my.cloudbit.ch/#/organization/applications and convert it to a base64 string and replace the **api-token** value.

• Retrieve the cluster ID from your Kubernetes Cluster, convert it to a base64 string and replace the **cluster-id** value.

cluster-autoscaler.yaml
 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
  name: cluster-autoscaler
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cluster-autoscaler
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
rules:
  - apiGroups: [""]
    resources:
      [
        "pods",
        "services",
        "replicationcontrollers",
        "persistentvolumeclaims",
        "persistentvolumes",
        "nodes",
        "endpoints",
        "namespaces",
        "configmaps",
      ]
    verbs: ["watch", "list", "get", "update", "create", "delete"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["watch", "list", "get", "create", "update", "delete", "patch"]
  - apiGroups: ["extensions"]
    resources: ["replicasets", "daemonsets"]
    verbs: ["watch", "list", "get"]
  - apiGroups: ["policy"]
    resources: ["poddisruptionbudgets"]
    verbs: ["watch", "list"]
  - apiGroups: ["apps"]
    resources: ["statefulsets", "replicasets", "daemonsets"]
    verbs: ["watch", "list", "get"]
  - apiGroups: ["storage.k8s.io"]
    resources:
      ["storageclasses", "csinodes", "csistoragecapacities", "csidrivers"]
    verbs: ["watch", "list", "get"]
  - apiGroups: ["batch", "extensions"]
    resources: ["jobs"]
    verbs: ["get", "list", "watch", "patch"]
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["get", "create", "update"]
  - apiGroups: [""]
    resources: ["pods/eviction"]
    verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-autoscaler
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-autoscaler
subjects:
  - kind: ServiceAccount
    name: cluster-autoscaler
    namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    k8s-addon: cluster-autoscaler.addons.k8s.io
    k8s-app: cluster-autoscaler
rules:
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["create", "get", "update"]
    resourceNames: ["cluster-autoscaler"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["create", "list", "watch", "update"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    app: cluster-autoscaler
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8085"
    spec:
      serviceAccountName: cluster-autoscaler
      containers:
        - image: flowswiss/cluster-autoscaler:cloudbit 
          name: cluster-autoscaler
          imagePullPolicy: Always
          resources:
            limits:
              cpu: 100m
              memory: 300Mi
            requests:
              cpu: 100m
              memory: 300Mi
          command:
            - ./cluster-autoscaler
            - --v=4
            - --stderrthreshold=info
            - --cloud-provider=cloudbit
            - --nodes=1:6:workers
            - --skip-nodes-with-local-storage=false
            - --skip-nodes-with-system-pods=false
          env:
            - name: CLOUDBIT_API_TOKEN
              valueFrom:
                secretKeyRef:
                  key: **api-token** #base64_encoded_api_token
                  name: cluster-autoscaler-secrets
            - name: CLOUDBIT_CLUSTER_ID
              valueFrom:
                secretKeyRef:
                  name: cluster-autoscaler-secrets
                  key: **cluster-id** #base64_encoded_cluster_id
            - name: CLOUDBIT_API_URL
              valueFrom:
                secretKeyRef:
                  name: cluster-autoscaler-secrets
                  key: aHR0cHM6Ly9hcGkuY2xvdWRiaXQuY2gv #base64_encoded_api_url

2. Apply the yaml file with kubectl in your desired Kubernetes Cluster.

3. Follow the official documentation to configure the behavior of the cluster autoscaler: https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler