Kubernetes

Kubernetes, also known as K8s, is a container orchestration service by Google.
It supposedly has a harder learning curve than docker-swarm but is heavily inspired by Google's internal borg system.

Getting Started

Background

Kubernetes runs applications across nodes which are physical or virtual machines.
Each node contains a kubelet process, a container runtime, and possibly one or more pods.
Pods contain resources needed to host your application including volumes and one or more containers.

Installation

For local development, you can install minikube.
Otherwise, install kubeadm.

kubeadm

kubeadm install

Install Commands

# Setup docker repos and install containerd.io
sudo apt update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update && sudo apt install containerd.io

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system

sudo apt-get install -y apt-transport-https ca-certificates curl
sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl



# Configure containerd
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo systemctl restart containerd

# Systemd cgroup
sudo vim /etc/containerd/config.toml

# Under this line, add the line below.
#  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
#    SystemdCgroup = true
sudo systemctl restart containerd

Control Plane Init

# Disable swap
sudo swapoff -a
sudo kubeadm init \
  --cri-socket=/run/containerd/containerd.sock \
  --pod-network-cidr=192.168.4.0/24
<br />
# Setup calico networking
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

# (Optional) Remove taint on control-node to allow job scheduling
kubectl taint nodes --all node-role.kubernetes.io/master-

Add worker nodes

Run the following on worker nodes.

sudo swapoff -a
# Add the line to join the cluster here
# kubeadm join <ip>:6443 --token <...> --discovery-token-ca-cert-hash <...>

kubectl

nodes

kubectl get nodes

pods

kubectl get pods
kubectl describe pods

deployment

kubectl get deployments

# For one-off deployments of an image.
kubectl create deployment <name> --image=<image>

proxy

kubectl proxy

containers

kubectl logs $POD_NAME
kubectl exec -it $POD_NAME -- bash

service

Services handle routing to your pods.

kubectl get services

kubectl expose deployment/<name> --type=<type> --port <port>
kubectl describe services/<name>

Services

Documentation

Services handle networking.
For self-hosted/bare metal deployments, there are two types of services.

• ClusterIP - This creates an IP address on the internal cluster which nodes and pods on the cluster can access. (Default)
• NodePort - This exposes the port on every node. It implicitly creates a ClusterIP and every node will route to that. This allows access from outside the cluster.

On managed deployments (e.g. AWS EKS, GKE) you also have

• LoadBalancer - fires up the provider's load balancer
• ExternalName

By default, ClusterIP is provided by kube-proxy and performs round-robin load-balancing to pods.

Ingress

Ingress is equivalent to having a load-balancer / reverse-proxy pod with a NodePort service.

Variants

minikube

minikube is a tool to quickly set up a local Kubernetes cluster on your PC.

k3s

k3s is a lighter-weight Kubernetes by Rancher Labs.

Resources

• Kubernetes Basics