Hi, I'm Eric Shanks, a senior field engineer with VMware. And in this video, we'll show you how to use some of the capabilities of the vSphere cloud provider with your Kubernetes cluster. The first piece of added functionality that the cloud provider gives us is the use of zone aware clusters. Let's look at why we might want this functionality.

In this example, we have a three node cluster and a replica set is being created with two pods. The Kubernetes scheduler will try to put those pods on different nodes, assuming there's no resource contention going on or something like that. This is great for availability purposes, but what if the environment has different fault domains?

For example, let's assume that my three ESXi nodes are part of different zones. Maybe my even nodes are in one rack and my odd nodes are in another rack and I've got redundant power for each individual rack. What would happen if my odd number rack had a power issue and we lost all the nodes in that rack. Well, in this example, we'd experienced an outage while the pods were restarting. Zone aware clusters however, can take this type of environment into account when we schedule our pods.

Let's take a peak at my lab environment for this video. You can see here, I've got a tag on the data center and it's a region tag, and I've also created three zone tags and put them each one on an ESXi host. You can see that I've got AZ1 on host one and three, and I've also got AZ2 on host number two. Now these were set up prior to configuring our cloud provider and setting that up and deploying our Kubernetes cluster, which I did in preparation for this video. Let's hop over to a terminal and let's see what effect that those tags had on our Kubernetes cluster.

First things first, let's let you see what I'm working with, with my Kubernetes cluster. So when I run kubectl get nodes, you can see I've got three controllers. I've also got three worker nodes. Now, if I run another command, just to see some of the labels that are on those nodes, you can see I'm outputting my three worker nodes here, and each one of them has a region label. And it also has a zone label. And you can see here that my AZ1 and AZ2 are set on each one of my worker nodes. Now this is done automatically as part of the cloud provider configuration and deployment. The Kubernetes kubelet and the API server are reading these tags from our vSphere infrastructure, and they're applying them on the Kubernetes nodes. Let's take a look and see what effect zone awareness will have on our Kubernetes cluster.

First thing is let's take a look at a deployment I've already got created. Here you can see I've got six replicas in my deployment file, and it's just an nginex deployment. Let's go ahead and deploy this to our Kubernetes cluster. And then once all of our Kubernetes pods have been deployed, we can run another command to see what node that those pods are living on. And you can see here, I've got one pod on worker zero, three pods on worker one, and two pods on worker two. Does that make sense? Let's hop back over to our PowerPoint and see what's happening.

Now in this first example, we'll assume we've got a three node cluster like we were using in our lab, except it's not zone aware. Here we've got, again, a replica set with six pods in it. And if we went to deploy those, the Kubernetes scheduler would try to evenly distribute those across all of our nodes. Assuming that the nodes were the same size and there wasn't resource contention. But as soon as we make them zone aware, things change a little bit. Now the Kubernetes scheduler wants to make sure that we're evenly distributing our pods across zones, not nodes. So here, when we go to deploy this, the pods will be deployed on multiple nodes in one zone. And since the other zone only has one node in it, it will take the rest. So you can see, we have even amounts of pods in each zone. Thus, the Kubernetes cloud provider is using zone aware deployments.

Shifting gears for just a moment, let's take a look at another piece of functionality added by the cloud provider and that's storage. Now in a normal cluster that doesn't have a Kubernetes cloud provider configured, if I needed a persistent volume, that volume would typically live on one of our Kubernetes nodes file systems. As you can see here, in my example, I have a persistent volume claim, that's attached to a persistent volume, and that volume lives on one of my worker nodes. Now with the Kubernetes cloud provider, I can move this persistent volume to a VMware data store, which is typically shared between all of the nodes. And that gives us some flexibility here. As you can see in my next example, where that persistent volume claim that still lives on worker node one for our application, connects to persistent volume that lives on the VMware data store. This lets us do things like if we had to coordinate our node or move our pods around, the persistent volume can stay right where it is. No changes need to be made there. But then as the new pod gets spun up, it connects to that persistent volume on the VMware data store. Let's take a look at this in the lab.

Okay, here we are back in our terminal. And I want to show you that before we get too far, I've already got a storage class created and ready to use with our vSphere cloud provider. You can see that the provisioner type is of vSphere volume, which means it's going to create us a VMware virtual disc and attach it to our nodes as we need it. So the next thing we'll do is we'll apply a persistent volume claim and that persistent volume claim will tell our storage class to create a persistent volume and attach those. So you can see here, our PVC is of status bound and it's of 10 gig.

Now, the next thing I want to do is we're going to apply an nginex manifest. We can exact into that pod, into the etc test directory, which is how I've mounted the file system. And I'm just going to create a simplified file here called test.file. And then we're going to exit. Now the next step is to take a look and see what node this is running on. Here you can see it's running on worker zero. And what we're going to do now is we're going to cordon off worker zero so that it can't schedule any pods. And then we're going to drain worker zero so that the pods that are running on it have been removed .and that's going to move our pod to a different node.

Now, if we had our persistent volume on the same node, this would be a problem. But since our persistent volume actually lives in the vSphere data store, which is shared across nodes, we should be able to connect to the same persistent volume when the pod starts up on a different node. To test that, let's check out the new pod that was created on a different node and we'll exec into it as well. And then we'll just check and see if that file is still listed under our etc test directory. And there it is.

Hopefully after watching this video, you've seen a little bit about how you can get some added functionality out of your Kubernetes clusters by using the vSphere cloud provider in your environment. Thank you for watching.