Kubernetes: Difference between revisions
| (11 intermediate revisions by the same user not shown) | |||
| Line 14: | Line 14: | ||
In addition to standard Kubernetes objects, '''operators''' watch for and allow you to instantiate custom resources (CR). | In addition to standard Kubernetes objects, '''operators''' watch for and allow you to instantiate custom resources (CR). | ||
==Administration== | ==Kubeadm Administration== | ||
Notes on administering kubernetes clusters. | Notes on administering kubernetes clusters. | ||
Kuberenetes has many parts and administration is very tedious which is why K3S exists. I'd recommend against using kubeadm for a homelab. | |||
===Installation=== | ===Installation=== | ||
| Line 205: | Line 207: | ||
To renew the certificates manually, run <code>kubeadm certs renew all</code> and restart your control plane services. | To renew the certificates manually, run <code>kubeadm certs renew all</code> and restart your control plane services. | ||
Note that if you lets the certificates expire, you will need to setup kubectl again. | Note that if you lets the certificates expire, you will need to setup kubectl again. | ||
Issues connecting with etcd | |||
I ran into this when trying to kubeadm upgrade | |||
;context deadline exceeded remote error: tls: bad certificate | |||
Kubeadm stores etcd certificates in <code>/etc/kubernetes/pki/etcd/</code>. | |||
Follow this to generate new certificates: https://github.com/etcd-io/etcd/issues/9785#issuecomment-432438748 | |||
You will need to create a temporary files for ca-config.json and server.json to generate new keys. | |||
Make sure in the server.json to set the key algo to "rsa" and size to 2048. In the same file, set your CN to 127.0.0.1 and the hosts to [127.0.0.1, your local IP]. | |||
;cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs | |||
This means your hosts in server.json is not correct when you generated the new keys. | |||
===Pods per node=== | ===Pods per node=== | ||
| Line 211: | Line 226: | ||
You may increase this up to a limit of 255 with the default networking subnet.<br> | You may increase this up to a limit of 255 with the default networking subnet.<br> | ||
For reference, GCP GKE uses 110 pods per node and AWS EKS uses 250 pods per node. | For reference, GCP GKE uses 110 pods per node and AWS EKS uses 250 pods per node. | ||
===Changing Master Address=== | |||
See https://ystatit.medium.com/how-to-change-kubernetes-kube-apiserver-ip-address-402d6ddb8aa2 | |||
==kubectl== | ==kubectl== | ||
| Line 366: | Line 384: | ||
==Ingress== | ==Ingress== | ||
[https://kubernetes.io/docs/concepts/services-networking/ingress/ Ingress | Kubernetes]<br> | [https://kubernetes.io/docs/concepts/services-networking/ingress/ Ingress | Kubernetes]<br> | ||
An ingress is an http endpoint. This configures an ingress controller which is a load-balancer or reverse-proxy pod that integrates with Kubernetes. | |||
A common ingress controller is [https://github.com/kubernetes/ingress-nginx ingress-nginx] which is maintained by the Kubernetes team. Alternatives include [https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/ nginx-ingress] [https://doc.traefik.io/traefik/providers/kubernetes-ingress/ traefik], [https://haproxy-ingress.github.io/ haproxy-ingress], and [https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/ others]. | |||
===Installing ingress-nginx=== | ===Installing ingress-nginx=== | ||
| Line 467: | Line 487: | ||
# Then you can simply add the following annotations to your ingresses to protect them: | # Then you can simply add the following annotations to your ingresses to protect them: | ||
#:<syntaxhighlight lang="yaml"> | #:<syntaxhighlight lang="yaml"> | ||
nginx.ingress.kubernetes.io/auth-url: " | nginx.ingress.kubernetes.io/auth-url: "http://oauth2proxy.default.svc.cluster.local/oauth2/auth?allowed_emails=myemail@gmail.com" | ||
nginx.ingress.kubernetes.io/auth-signin: "https://oauth2proxy.davidl.me/oauth2/start?rd=$scheme://$host$request_uri" | nginx.ingress.kubernetes.io/auth-signin: "https://oauth2proxy.davidl.me/oauth2/start?rd=$scheme://$host$request_uri" | ||
</syntaxhighlight> | </syntaxhighlight> | ||
;Additional things to look into | |||
* Pomerium | |||
* Keycloak | |||
** https://www.talkingquickly.co.uk/webapp-authentication-keycloak-OAuth2-proxy-nginx-ingress-kubernetes | |||
* Authelia - only supports username/password as the first factor | |||
* Authentik - tried this but had too complicated and buggy for me. | |||
If you use Cloudflare, you can also use Cloudflare access, though make sure you prevent other sources from accessing the service directly. | |||
==Autoscaling== | ==Autoscaling== | ||
| Line 608: | Line 637: | ||
===k3s=== | ===k3s=== | ||
[https://k3s.io/ k3s] is a lighter-weight Kubernetes by Rancher Labs. | [https://k3s.io/ k3s] is a lighter-weight Kubernetes by Rancher Labs. | ||
It includes Flannel CNI and Traefik Ingress Controller. | |||
==KubeVirt== | ==KubeVirt== | ||
{{main | KubeVirt}} | {{main | KubeVirt}} | ||
KubeVirt allows you to run virtual machines | KubeVirt allows you to run virtual machines on your Kubernetes cluster. | ||
==Resources== | ==Resources== | ||