Join 80,000+ fellow learners

Hi and welcome back to lesson five. So far we have talked about different components but in regards to storage, networking, compute and how to configure these individual elements. So, in this lecture, we're going to talk about how to manage access to the Kubernetes API itself and what are some of the resource quotas that you can set up for development teams.

Now, a couple of things to keep in mind is that the Kubernetes API is pretty powerful, like if the development teams have access to that, that means they can create infrastructure objects, like they could bind multiple storage volumes, they could create load balancing services, things like that. While that is what the development teams expect, the idea is that they have the access to the Kubernetes API and hence they can start building the necessary objects to deploy their application.

But at the same time, it also becomes pretty important to think about what are the limits and challenges within the infrastructure. How much workload that the infrastructure can handle? And then how do we balance resource management that development teams can utilize? That's one thing and then the API itself is pretty powerful, so you'll have to think about what are some of the secure ways that you can give access to user groups to that particular API.

Now, by default, before we get into resource and API Management, the way Kubernetes clusters are deployed, there're mostly two design high level design methods. One is where a team or an organization will have just a single giant Kubernetes cluster and then multiple teams will be operating within that Kubernetes cluster and they will be dissecting the cluster on a namespaces basis. So a Kubernetes cluster will have multiple namespaces and maybe each namespace is mapped to an individual team or each namespaces map to individual workloads and things like that. So that is one way to think about how Kubernetes clusters would be used.

Another way is that they will, we have seen teams or organizations create multiple Kubernetes clusters on their various teams. So each team will get their own Kubernetes cluster or each product or workload that they're working on might get an individual Kubernetes cluster. So these are some of the two high level design considerations to think about when designing your resource quotas or how do you manage API access to different Kubernetes clusters. When it comes to a single Kubernetes cluster that everybody is using, of course, you have to think about that single Kubernetes cluster is going to probably consume a lot of your infrastructure resources versus if you have multiple, then how can I divide or how can I slice my resources to those different Kubernetes clusters.? So at the high level, from an infrastructure perspective, you'll have to think a little bit about that.

Now, within Kubernetes, there is a construct called resource oh sorry, quota management. So Kubernetes has this concept of defining quota management or defining quota limits through quota management. And this is available on a per namespace basis, so it's not cluster-wide. The idea being if there're different teams, you might want to have quotas set on namespaces basis. So you can apply quota management to different namespaces or on a per namespace basis.

Before we get into the actual resource quota management, why is this important? So when dev team says it's going to create, let's say, a pod or a deployment type, they can specify how much memory and CPU that the pod is going to consume. And there are two things that they can specify particularly. One is the requested limit. So you can or the dev teams can specify, "This is my pod and I need 64 megabytes of memory and then 215 megabytes or 250 cycles of CPU," things like that. Or "Megahertz of cycles for CPU." And at the same time, they can also specify the limit to how much memory or CPU that pod can consume.

So in this example, over here, you'll see this is a spec for a pod deployment. And the pod has two containers. One is the app container and the other one is the log aggregator. And the developer has defined resources requested and limits for individual containers. So when this pod is going to run on that particular Kubernetes cluster, the pod will at least get 64 megabytes of memory and it cannot use more than 128 megabytes, so the pod knows that's the bandwidth it has.

Now, a lot of times development teams may not specify this, right? They may not specify any memory resource request. So in that case, it becomes imperative to define a default namespace wide resources limits both in terms of CPU and memory, and that's where resource quota management comes into play. So for example, for CPU and memory, these are the four resource quota definitions that you can add.

One is CPU.limit. This is basically the, since the pod spec can define how much limit I can have in my memory and CPU and how much requested CPU and memory I can have. The quota basically limits stored CPU define how much or what is the maximum number of limit that all the pods can have in a given namespace. The sum of total CPU limits cannot exceed the CPU.limit defined in your resource quota spec. Similarly, slimstock memory will limit some of the memory that is requested or limited, it cannot exceed what you specify in the limit stored memory spec. Now the request.CPU is basically the total number of CPU request that cannot exceed this value. So let's say if there are 10 pods requesting 250, 250 megahertz of CPU and you set a limit of 500, then only two of them are going to start running.

So this is how you can define different CPU memory resource quota limits at the namespace level on a Kubernetes cluster. Now, this is how you actually define that. So this is the spec file, an example spec file for resource quota management. So for example, in this case, we are setting up a resource quota and the name is mem CPU demo. What you're setting is those requests.CPU cannot exceed one megahertz, their memory cannot exceed one gigabyte, the limit cannot be more than two CPUs or the limit on memory cannot exceed more than two gigabytes. So if a pod on the other hand, let's say this is a developer creating the sports pack and they define a memory of 800 megabytes, CPU of 800 megahertz and memory a limit of sorry, 800 megabytes and requests of 600, this is actually within the requested memory and requested limit for that particular namespace, so this pod is going to start running.

So similar to CPU and memory quota management, you can even define resource quotas for storage and we covered this in our storage lessons earlier. But again, you can define the total number of persistent volume claims that is allowed on a namespace basis. Then you can even define the total sum of storage requests on all pods persistent volume claims. Within a storage class, you can define what are the limits on the requested storage and a namespace cannot exceed that when you create a request.storage limit, and so on and so forth. So, that is another way you can create a resource quota for your CPU and memory and then you can create one more for your storage.

Now, so far, what we looked at was limits in terms of overall the mid start, any number of pods that are running within that namespace cannot exceed. So let's say you have a pod that your overall limit for memory for a namespace is one gigabyte and there are two pods, one is running at 200mb and the other one is running at 800 megabytes, they both will keep running as long as there is no individual limits set within the pod spec for these pods. But what about where, in this case, it's really between the two pods there might be race conditions where one pod really needs more memory but because the other pod is hogging most of the memory it cannot get that.

So there is another way to set limits on a per pod or a per container basis. So these are more aggregated request limits or resource quotas that we saw. There is another concept called limit ranges where you can define or what are the limits that are applicable to a per pod or a per container basis. And again, what this does is it makes sure that there is not one pod or one container that is hogging too much memory or too much CPU. And there's equal number of resources available across different CPUs, across different pods.

So, limit range is another spec you can define, and this is an example. In this case, we are setting that a CPU can have 800 megahertz of max and a minimum of 200 on a per container basis. So if there is a pod spec file where the container says, "I need 500 megahertz of CPU," that pod will not run. And every pod or container running within that namespace cannot exceed these limits. So, so far, we talked about resource quotas. And now we get into how do you allow users and groups to access the Kubernetes API?

Again, the API is pretty important to secure because if anybody has access to the Kubernetes API, then they can essentially create a lot of objects within your Kubernetes cluster, like load balancing services or they can start deploying things like volume drives and things like that. So, it's really important to authenticate and authorize your Kubernetes API.

Now, within the Kubernetes cluster, there is a set of a default role-based access control that you can define. And you can also authorize users based on those roles and definitions using an identity server that you might have, like Active Directory or LDAP Server. Again, to keep the role-based access control, these are a default set of roles that can be defined at the cluster level within Kubernetes and then you can bind the role-based access control authorization through an identity provider. And so you can define a set of group of users or individual users that can log in and authenticate to have access to the Kubernetes API.

Keep in mind, if you have multiple Kubernetes clusters, then you will have to define these RBAC policies or map users to these RBAC policies on each and every individual cluster, because roles are not going to flow into individual clusters, so these are on a per cluster basis. Again, Kubernetes has different ways of defining role-based access control.

There are two kinds of access control lists that you can define or roles that you can define. One is called Role, very appropriately named. And then the other one is called ClusterRole. The difference between the two is the scope of the role. Just a Role type is used when you want to create role-based access control that is limited to a specific note namespace. So let's say you have a set of users or a group of users that you want to bind them to a specific role that is attached to a namespace within Kubernetes, then you use the Role type RBAC control or authorization.

Now, on the other hand, if you have a set of users that need access to every namespace or those who need cluster-wide access control, then you can define ClusterRole, which gives them access to every resource type within that Kubernetes cluster, irrespective of the namespace that they are in. So the major difference between Role and ClusterRole is that a Role is defined for a specific set of user or group of users on objects that live within a namespace. So for example, pods, services, persistent volume claims, et cetera. On the other hand, the ClusterRole are those for users that need cluster-wide access to resources like for example, storage classes, right?

And the way these roles are created is you define a resource object that you create a rule for. And then you get the final word for that. So for example, get, list, watch and then the corresponding role type. So for example, in this case, this RoleBinding is applicable to the namespace test and the role name is role a and it is applicable to resources called pods. But then that namespace test. Whatever user has this role bound to their name can only get or list the pods within that namespace, so this is the idea of an individual Role.

On the other hand, a cluster-wide role for example, we're here in this case, the spec is basically specifying that for example, the resource storage classes across the cluster. This particular user who is bound to this role can get list and watch all the storage classes within that cluster. The difference between the two is the scope, Role is namespace bound and ClusterRole is cluster-wide. Now, once you have defined roles, then you can define role binding or cluster role binding. What this does is it attaches a role to a specific set of users or groups. So in case of a role bindings, let's say the role binding that we created or saw in the previous example, you want a user [email protected] to use this particular role binding, to use that particular role, you'll create a RoleBinding for that. So, by creating a RoleBinding, you're essentially saying the user [email protected] is allowed the Role, roll binding a, which means they are allowed to only watch or list pods in the namespace test.

So this is how you can create a RoleBinding. And similarly, the same applies to a ClusterRoleBinding. If you have users that you want to define to a specific ClusterRole, you create a ClusterRoleBinding. So in this case, the user [email protected] is going to have a ClusterRoleBinding that binds it to a ClusterRole that gives them access to cluster wide resources. So this was just a quick overview of how resource quotas can be applied, how you can create different roles, how you can bind those roles to different users within a Kubernetes cluster. Thank you for watching this lesson. See you in the recap.