![ApptioCareers-Employee[1]](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyMjUgMjc1IiB3aWR0aD0iMjI1IiBoZWlnaHQ9IjI3NSIgZGF0YS11PSJodHRwcyUzQSUyRiUyRnd3dy5hcHB0aW8uY29tJTJGd3AtY29udGVudCUyRnVwbG9hZHMlMkZlbGVtZW50b3IlMkZ0aHVtYnMlMkZBcHB0aW9DYXJlZXJzLUVtcGxveWVlMS1xZzVuenB5MHdoaWJtbzZvdmJqNGM3MHh0aTUzdmo1OHM0endwajh0dHMucG5nIiBkYXRhLXc9IjIyNSIgZGF0YS1oPSIyNzUiIGRhdGEtYmlwPSIiPjwvc3ZnPg== "ApptioCareers-Employee[1]")
IBM ApptioKubernetes Multi-Cloud
Chapter 2: Distributions
Like this article?
Subscribe to our LinkedIn newsletter to receive more educational content.
Kubernetes is the de facto standard for container orchestration today. From simple test deployments on a Raspberry Pi, to highly distributed systems running across multiple regions, you can find Kubernetes just about everywhere. That’s because Kubernetes is extensible, reliable, scalable, and feature-rich. It supports many use cases including load balancing, failover, and automatic scheduling.
However, getting started with Kubernetes can result in questions with no single answer. There are multiple Kubernetes distributions with their own sets of pros and cons, and so hoping for one right option isn’t recommended. For example, minikube makes it easy to test and develop but isn’t the right choice for enterprise workloads at scale. Take a look at just how many choices there are in the Kubernetes distribution landscape:
Summary of key concepts
As Kubernetes gained popularity, many companies started providing add-on services, features, and capabilities to help their customers achieve their business goals more quickly. Each of these offerings is known as a distribution; a modified version of the upstream Kubernetes distribution or bundled with additional tools and features. The current list maintained by CNCF includes dozens of distributions (see the full list here) though many are specialized and not relevant to most users.
The table below summarizes the types of distributions that are explained in more detail in this article.
| Kubernetes Distribution Type | Examples | Description |
|---|---|---|
| Upstream / open-source | Canonical, Rancher, OKD | Open-source, free to use, easy to set up, optional enterprise support. |
| Opinionated | Openshift, Tanzu | Kubernetes distribution with additional features to support developers, operations specialists and administrators. |
| Cloud managed | AKS, EKS, GKE | Fully managed, Fast cluster creation, low operation overheads, automatic updates, use of other PaaS and IaaS supporting services. |
| Single-node | Microsk8s , minikube | Excellent choice to run kubernetes on edge devices, IoT or for learning. |
Kubernetes components
Control plane
The control plane is the brain, making important decisions within and about the cluster and responding to instructions. There are four main components within the control plane. It is highly recommended that the control plane components run on the same server or node, and these nodes should not run any workloads or user containers on them. The following table summarizes their use.| Component | Function |
|---|---|
| etcd | Containers are transient and stateless, because of which they are often destroyed and created. Someone has to maintain information like which container is running on which node and details regarding its creation or destruction timelines, among other things. It is essential to have a record of the state of the nodes and the cluster itself. This someone is etcd. |
| Scheduler | The Scheduler identifies the right nodes to place a container into based on the nodes' capacity (CPU, storage, ram), policies, constraints, and other filters. The Scheduler also tracks the containers and their status. A key concept to understand here is that the Scheduler Identifies the node where a container should run but does not actually deploy the container in the identified node. |
| Controller manager | The controller manager is tasked with monitoring the state of the cluster, ensuring that the current state is always equal to the desired state. The controller manager has a set of controllers responsible for specific tasks. |
| API server | The API server's primary function is related to authentication, and it is the single point of contact for all API calls and external clients. |
Additionally, there is a cloud controller manager, which is similar to the controller manager. The main difference is the cloud controller manager allows you to link to the cloud-specific APIs. Kubernetes clusters that run on-premises may not have a cloud controller manager.
Worker node
The worker node is responsible for running containers and has three main components, as listed in the table below.
| Component | Function |
|---|---|
| kubelet | Each worker node has a kubelet agent that runs on each of the nodes where containers are deployed, and it uses the underlying container runtime environment (Docker, rkt and containerd) to run the containers. |
| Container runtime environment | A container runtime environment is a set of applications that lets you run containers. |
| Kube proxy | Kube proxy is responsible for the network connectivity among the containers. |
These components are made available on GitHub in a Kubernetes open-source project. One of the ways to install, configure, and set up is to download these source codes and compile them. This method works across all infrastructures, servers, and virtual machines in the cloud or on-premises.
The downside of this approach is that it is complex and time-consuming. If you fail at some stage or want to update, you will have to rebuild it. Following this method of setting up a cluster is called vanilla Kubernetes or a vanilla distribution.
Kubernetes distributions
- Opinionated distributions build the vendor’s opinions and best practices related to installation, configuration, and operation into the distribution.
- Upstream and open-source distributions are simpler open-source Kubernetes distributions that make installation easier without add-ons. Some of these distributions use upstream Kubernetes and others do not.
- Cloud-managed distributions are offered by cloud platforms like AWS, GCP, and Azure. With these distributions, the provider manages the control plane and users focus on development and productivity.
- Single-node distributions have control plane and worker nodes on the same server. These Kubernetes distributions are often used as learning environments and can be beneficial for use cases like running an application in Raspberry Pi, edge, and IoT devices.
| Distribution Type | Examples | Key Features |
|---|---|---|
| Upstream / Open source | Canonical, Rancher, OKD |
|
| Opinionated | Openshift, Tanzu |
|
| Cloud-managed | AKS, EKS, GKE |
|
| Single-node | Microsk8s, minikube |
|
VMware Tanzu
Tanzu is a portfolio of products by VMware that allows administrators and developers familiar with the VMware architecture to quickly provision, manage and run Kubernetes clusters. Tanzu Kubernetes Grid is an upstream Kubernetes-compatible distribution that integrates with the VMware ecosystem with 24×7 support from the VMware Global support services.
Here are four key Tanzu offerings from VMware:
- Tanzu Kubernetes Grid (TKG): TKG is a multi-cloud Kubernetes service by VMware to run on VSphere or public cloud virtual machines.
- Tanzu Kubernetes Grid Integrated (TKGi): TKGi is a production-grade solution based on Pivotal Kubernetes Service (PKS), and it integrates BOSH and Ops manager. TKGi is helpful if you want to create a multi-tenant Kubernetes Environment.
- Tanzu Mission Control (TMC): TMC is a service related to VMware Cloud that lets you attach clusters from multi-cloud and allows you to monitor and manage them from a single console.
- Tanzu Kubernetes Grid Service (TKGS): TKGS is Kubernetes with vSphere integration which means that administrators can provision Kubernetes clusters using the vSphere platform and the associated vSphere services.
Red Hat OpenShift
RedHat has two distributions of Kubernetes, one of them being OpenShift, an enterprise-ready distribution with a robust SLA and paid professional support. While OpenShift gets new Kubernetes features later than upstream Kubernetes, there is a strong focus on stability, security, and integration with other Red Hat technologies that help the developer community to be more productive.
Redhat Enterprise Linux is the base operating system, and CRI-O is the container runtime environment. OpenShift has a multi-cloud architecture and works well with AWS (Red Hat OpenShift Service on AWS) and Microsoft Azure (Azure Red Hat OpenShift). `oc` is the administrative tool for Openshift similar to `kubectl` that Kubernetes uses.
OpenShift features a simple and flexible installer that deploys the cluster on any infrastructure and comes with an update service.
The free community distribution of OpenShift is OKD or Origin. OKD uses Fedora as the underlying operating system. It is an excellent option if you want to try or have a local cluster on your machine using the same APIs as OpenShift.
Canonical Kubernetes
Charmed Kubernetes is an enterprise-grade, upstream Kubernetes by Canonical which offers cluster lifecycle management and is capable of multi-cloud deployment. Canonical offers managed Kubernetes, where Canonical engineers will build and operate the cluster 24/7 on bare metal, OpenStack, or any public clouds. Canonical is the publisher of Ubuntu, a popular Linux distribution. In addition to the distribution, Canonical also offers Enterprise support for a cost. Regarding the container runtime environment, Charmed Kubernetes supports Containerd, Kata containers and CRI-O.
Canonical offers a single-node Kubernetes distribution known as MicroK8. It is a lightweight Kubernetes distribution that can run on your laptop or any edge of IoT devices. Installation of MicroK8s is straightforward, with a single command to install followed by three commands to add your user to the microk8s group, as shown below.
sudo snap install microk8s --classic --channel=1.25
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
Rancher Kubernetes
Rancher is a multi-cloud Kubernetes management platform that allows you to deploy clusters on private cloud, public cloud or vSphere and provides centralized authentication and observability. For IT operators, this means that they can use Rancher as a single pane of glass to manage on-premises clusters as well as those hosted on clouds like Azure Kubernetes Service (AKS), Elastic Kubernetes Service (EKS) and Google Kubernetes Engine (GKE).
Rancher also provides enterprises with a hosted Kubernetes service, a premium service where Rancher takes care of the installation, upgrades, and operations of the control plane with an SLA of 99.9%. Customers can add support for tools like Grafana, Prometheus, Istio, and Fluentd by paying additional subscription costs.
Additionally, Rancher has a product called Rancher Kubernetes Engine (RKE). This CNCF-certified Kubernetes distribution runs within Docker containers and K3s, a lightweight Kubernetes distribution with very low resource requirements.
Cloud-managed Kubernetes
Kubernetes has become so popular in the last few years, major cloud providers have come up with their own cloud-managed Kubernetes distributions. The three most popular options are:
- AWS Elastic Kubernetes Service (EKS)
- Azure Kubernetes Service (AKS)
- CGP Google Kubernetes Engine (GKE)
These managed Kubernetes services integrate with the cloud provider’s services and technologies. Cloud providers deploy, run and maintain the control plane and let you shift your focus to the workload.
EKS charges a few cents per hour for the control plane, whereas AKS and GKE offer it for free. While most of these services have similar offerings and technologies built around them, there are minor differences, and the choice of managed Kubernetes distribution depends on which cloud is preferred. They all have solutions for multi-cloud and hybrid-cloud deployments like Anthos from Google, Arc from Azure, and EKS Anywhere from AWS.
Which Kubernetes distribution to choose?
With so many distributions available, how does anyone choose a particular distribution? The following five questions can guide you towards the best solution:
- Do you want your cluster on-premises or on the cloud?
- Are you looking to manage and operate the cluster yourself?
- Do you have requirements where you need to deploy your application(s) in multiple clouds or multiple regions?
- Do you currently use AWS, Azure, or GCP?
- Do you plan to use multiple Kubernetes Clusters?
Conclusion
There are more than 120 distributions of CNCF-certified Kubernetes available, and any of them *could* be an excellent choice. What matters is the available tools and services surrounding the distribution and if the maintainers offer enterprise support to the level you require.
Consider your staff’s skill set and align technology decisions accordingly. Another metric is the popularity based on the number of posts, issues and discussions in forums like Stack Overflow, Discord, and Reddit.
Kubernetes has erased the line between cloud and on-premise. Public cloud providers are offering their services within the data center using Google Anthos or Azure Arc, and on-premises providers are establishing their presence on the cloud like Microsoft Azure, Red Hat OpenShift, and Red Hat OpenShift Service on AWS.
With so many distributions available, there is a lot to consider when choosing a Kubernetes distribution. The concepts and questions covered in this article should help you make an informed choice.
