Join 80,000+ fellow learners

Hi, and welcome back to lesson three. In this lesson, we are going to talk about what are some of the storage considerations to keep in mind when building a Kubernetes stack that is optimal for your developers. In lesson two, we spoke about networking and how some of the plugin model within Kubernetes handles the rollout of different networking services. In this chapter, we're going to focus more on storage and how Kubernetes is going to handle storage for us.

Now, if you take a look at a Kubernetes cluster, a cluster essentially is a collection of different nodes. So a cluster can have multiple nodes, from two to three or one to however many, based on the requirement. But essentially, what tends to happen is these nodes provide the Kubernetes cluster with compute and memory and other resources. And when applications land on top of these clusters, essentially what's going to happen, is there will be pods that get deployed on a container runtime, running within these nodes.

Now, Kubernetes by default, has a capability of self-healing. So what that really means is, it is constantly watching out for the pods, for the node status, et cetera. And if it sees, for example, that a pod is not operating optimally, or it's not responding, it is going to kill the pod, and spin up a new one. Now at the node level, it is also doing similar things. So for example, if a node completely is unreachable, for some reason, it will reschedule the pods that are running on particular node, onto a new node.

Now, as a result of this, you will see that pods tend to move around or jump around different nodes from time to time. At the same time, Kubernetes is also watch the cluster to make sure that workloads are distributed evenly, and that the compute and memory consumption on a particular node is not overloaded, while their other nodes are sitting idle or less utilized.

So Kubernetes is going to load balance the pods that are running on these nodes to make sure that the cluster is constantly in a balanced state. And as a result of that, again, the pods tend to shift around. A lot of the applications that run within these pods may have different storage requirements. For example, some of the pods may need storage during the lifecycle of that particular pod, and they might need to share the data that they have with other pods running in the cluster. Or they might have a need to persist the data that they are storing or creating, beyond the life cycle of the pod.

Pods running within Kubernetes have ephemeral storage associated with it. So, which means that the storage that is provision for a particular pod or mounted on a particular pod, is available during the lifecycle of that pod. And when development teams provision this particular pod, while creating the deployment spec, they can define how much storage the pod needs, in megabytes or gigabytes. And then Kubernetes is going to provision the storage for them. But again, these are ephemeral storage. If a pod needs persistent storage, then there is another way of defining that. We'll get into that in a bit.

But the key idea here to keep in mind, is number one, because the pods are constantly moving between nodes, the storage that Kubernetes is using to create these different volumes for these pods, has to be a shareable storage. First of all, because the pod can jump around, and as such the volumes that are associated with these pods need to be remounted and mounted constantly. Now, the way Kubernetes is going to mount a particular volume, there are different volumes that you could mount on a particular pod, and we'll get into the types of volumes in a bit, but essentially there are different volumes.

There are some of ephemeral volumes and some are persistent volumes. Either way, whether it's an ephemeral or a persistent volume, what tends to happen is if you're on a Cloud or an on-prem infrastructure, you will have a large chunk of storage available to you, either through a block file system, or the network file system, or something like that. And then from that particular storage, you're going to carve out a volume that you can associate with the Kubernetes cluster. And then once Kubernetes understands what that volume is, what types of volumes are available, then a pod is scheduled on a particular node, it is going to utilize those volumes to provide data storage capabilities to the pods running on that particular node.

And so this is overall how nested storage is in Kubernetes again, and some of the requirements or some of the things that tend to happen when storage is mounted on a particular pod. Also, another thing to note over here is the Kubernetes storage provider, which is the plugin for Kubernetes, you can deploy any kind of storage provider plugin, depending upon the infrastructure that you run on, and Kubernetes is going to support a vast set of different volume types, all the way. For example, from vSphere volumes, the data store volumes, to Amazon EBS, to Azure, and so on and so forth. So again, the storage provider of plugin comes into play over here, and you need to deploy the appropriate plugins within the Kubernetes cluster as part of the run time of that particular cluster, so that humanity's API understands how to talk to the backend infrastructure, to request for a volume when a pod needs it.

And these volumes have to be dynamic in nature, because at any given point in time, a developer can come in and say, "I need a persistent volume, or I need a volume of any sort to mount to my pod. And here's my requirements." So if that request has to pass through, your volumes will have to be dynamic in nature for them to be attached to a particular port and so on and so forth. And you'll have to figure out what kind of capacity needs are required by your development teams, to make sure that there are enough volumes with enough storage available to the Kubernetes cluster.

Now getting into Kubernetes storage abstractions. There are two different storage's, like I mentioned, for the pods. One, our regular volumes, which are most likely ephemeral. And then there are persistent volumes that Kubernetes is going to use. Let's take you look at individual volumes. Now, volumes provide a mechanism to attach storage to the pods. Some volumes are ephemeral, and they last as long as the pod they are attached to. Examples of ephemeral volumes or ConfigMap or emptyDir. So what is a ConfigMap?

A ConfigMap in Kubernetes is basically a way for Kubernetes to provide runtime environment variables, to runtime environment variables, to a pod running in a particular cluster. So let's say you have an application that is running in a pod and the application has requirements to reach a database service. Now at runtime, what is the database service that it can access? What is the username password? And things like that. So ConfigMaps help provide those runtime variables into the pod. Essentially, for a particular application or a pod for a particular workload, that data need not persist. So that's something that is considered as an ephemeral storage.

Now some volumes, on the other hand, are persistent and they outlive of pod. So volumes are attached to pod and not the container, which is really important to keep in mind. Now, volume types can be blocked by NFS. They could be block storage, they could be Windows shares, or persistent disks provided by the... It could be anything that can be provided by a Cloud vendor. These volumes could be blocked storage on network file systems, as well.

They provide durable storage beyond the life-cycle of a pod. A failing node or a pod does not affect these volumes. If a pod fails, the volume is simply unmounted from that particular pod. And similarly, if the node fails, then the volume is unmounted from the underlying node, as well, although they might already exist before the creation of a pod. A pod could consist of multiple containers. However, in most cases, there is one container per pod. Volumes allow containers within a pod to share data and are required for pods to be stateful.

Now there are different volume types. For example, these are some of the volume types that are listed over here, emptyDir, The first one listed is an ephemeral volume type that can be mounted on a pod. And as the name implies, emptyDir is simply an empty directory that allows containers inside the pod to read and write, to and from it. emptyDir is created when a pod is assigned a node, and it exists as long as the pod exists. emptyDir will be removed if the pod is removed from the node for any reason. Hence, so we do not use emptyDir for critical data, applications use emptyDir for short term purposes. Kubernetes creates emptyDir from nodes local disk, or memory back-filed system.

Another data volume type is a ConfigMap. ConfigMap provides a way to inject configuration data into pods. We just spoke about this in the last slide, and this data basically stores any kind of object, data reference, data that the pod needs during its life cycle. Another volume type is called Secret, which is very similar to ConfigMap, but Secrets are leveraged to store sensitive information such as passwords, tokens, or SSH keys. And a Secret is used to pass sensitive information to the pod.

downwardApi volume time is used to download API data available to applications. It is a way containers can learn about the pod environment. A use case could be, your containerized application needs to construct a unique identifier within the cluster. It would be very easy to create that identifier based on the name of its pod, since pod names are unique within the cluster. downwardApi is the way applications can fetch name of the pod it's running on.

Another type of volume is PersistentVolumeClaims. And we're going to talk a little bit about that in the next slide. Now, when it comes to persistent volumes, there are different methods to how persistent volumes can be attached to a pod, and the implementation really is dependent upon how the team operates between the person who is operating, or the team who is managing the Kubernetes cluster and storage associated with that cluster, and how development teams are required to access any storage requests.

So the two methods are basically, essentially the developer ultimately really wants to attach a PersistentVolumeClaim to its pod, so that it can process data or store data, into a service like MySQL, et cetera. There are two methods developer can do that. They can either do it dynamically, wherein they directly request a PersistentVolumeClaim to do a Storage Class. And a Storage Class, essentially, is a definition of what is required... And we'll get into the details of this in a little bit. But essentially a Storage Class defines what kind of a storage you have, what kind of policies you have to find for that storage, and things like that.

If you have multiple storage types, let's say you have a block storage, you have a NFS storage, or different kinds of storage, then a Storage Class essentially defines the properties for that particular storage subsystem. And when a developer request a PersistentVolumeClaim, they can do that directly by defining the Storage Class. But in order to do so, the development teams need to have understanding of the different Storage Classes that are available, and to create a dynamic PersistentVolumeClaim. Now, on the other hand, as an operator, if you want to pre-build those PersistentVolume templates, for your development teams to different Storage Classes, then you can do so by a PersistentVolume.

So when you create a PersistentVolume, you're basically defining the Storage Class that it is based on, and what that persistent volume is. And when a development team needs a claim, it will just request that, using that particular persistent volume. So these are just the way you can implement how PersistentVolumes get created by a particular dev team. Like I mentioned, a persistent volume essentially, is something that the operator is going to provision using a particular Storage Class type. The life-cycle of a PersistentVolume is independent of the pod that it gets eventually attached to.

The idea behind having a PersistentVolume, is that it abstracts the storage provisioning from the storage consumption. So a PersistentVolume allows an operator to provision a variety of volume types. The cluster operator simply provisions the storage, and does not have to worry about its consumption. An application development, on the other hand, can claim and use this provision storage, using PersistentVolumeClaims, without creating and maintaining storage volumes. This allows clear separation of concerns. Operators make persistent volumes available, and developers leverage those volumes to create claims.

Now, when application developers use PersistentVolumeClaims, they do not need to worry about where the application is deployed, provision storage capacity can be claimed by the application, regardless of whether it's running on-prem or a particular Cloud vendor, et cetera. Now, PersistentVolumeClaims, on the other hand, is essentially particular volume that gets mounted on a pod. And the PersistentVolumeClaim can be dynamically created using a Storage Class, or it could be created using a PersistentVolume that an operator pre-created. PersistentVolumeClaim is similar to a pod. A pod consumes nodes resources, whereas PersistentVolume consumes PersistentVolume resources.

Now then we come to Storage Classes. Now, Storage Class provides a way for operators or administrators to describe the class of the storage that they offer. So let's say you have different subsystems of storage, like NFS, block, et cetera, and you want to define certain qualities for that particular storage type without necessarily explaining a ton to your development teams. You can create a different Storage Class templates within that cluster and then development teams can claim PersistentVolumes using these Storage Classes. There's always a default Storage Class that needs to be created if the team is expecting a dynamically created PersistentVolume Claim, then you will have to create a default Storage Class. For example, if dev team is requesting a PersistentVolumeClaim, and there is no Storage Class, then the default is the one that gets picked up.

Now, different Storage Classes could map to different levels of quality of service, or to backup policies or to arbitrary policies determined by the cluster administrator. This concept is sometimes called Profiles in other storage systems. Now each Storage Class contains a couple of fields, like the provisioner, the parameters, what is the reclamation policy which I use when a PersistentVolume belonging to the class, needs to be dynamically provisioned? The name of the Storage Class object is significant, and is how users can request a particular class.

Now, an administrator can set the name of other parameters of a class when first creating the Storage Class objects, and the objects can not be updated once they are created. Administrators can specify a default Storage Class just for PersistentVolumeClaims that don't request for any particular class to bind to. So you do need to create a particular Storage Class, which is called the default, in case development teams do not specify a particular Storage Class.

Essentially, it's a way of defining different types of storage available to that cluster, and bringing it up to your development teams for them to consume through PersistentVolumeClaims. Now, there are different ways of provisioning volumes. One is manual where, if you are going with the PersistentVolume route, you are going to, as an operator, pre-create some of these PersistentVolumes, and that is a manual method. And then development teams are going to use those PersistentVolumes to create PersistentVolumeClaims.

Another method is Dynamically Provisioning, this allows volumes to be created on demand, and essentially it eliminates the need to have that back and forth between cluster operators and development teams to provision storage. And ideally you want to go with this kind of notion, where you're allowing your development teams to define storage claims or PersistentVolumeClaims dynamically so that they can operate their pods, their applications, the way they need to, at any time without having to file a ticket or without having to talk to a operator to create a PersistentVolumes.

Now, when it comes to using dynamic provisioning, there's always this question back in the head of operators like, "Okay, I can give that to a development team, but then again, they can start claiming PersistentVolumeClaims that are big chunks of volumes, like one terabyte, five terabyte. And what happens if they exhaust my backend storage system? Or how do I maintain and manage that?"

And that is where this concept of Storage Resource Quota has come into play. You can define limits and quotas on how many PersistentVolumeClaims can be created. What is the total size of all the different storage that can be requested for these PersistentVolumeClaims? Et cetera, using these different Storage Resource Quota definitions. And these can be applied at a specific namespace within that Kubernetes cluster. So resource quotas are not cluster wide. You can create a resource quotas based on namespaces, and that is because namespaces are essentially a way to dissect a Kubernetes clusters for multiple teams or different products or different workloads that get created within a Kubernetes cluster.

And so you can define what the storage quota is for a particular namespace. So you can define, for example, through request or storage parameter, you can say, "What is the sum of total storage requests across all PersistentVolumeClaims that can be requested?" So it cannot exceed that particular value, and so on and so forth. So you can create this using the Storage Resource Quota.

We are going to talk a little bit more about resource quotas in our next lesson, as well, because this applies to a lot more than just storage. So in conclusion, this is mostly about how storage should be configured for your development teams in order to request a storage capacity, what were the different storage types to think about, and things like that. And we are going to cover resources in our next lesson. Thank you.