Join 80,000+ fellow learners

Hi, my name's Eric Smalling, and I'm a staff field engineer at VMware. In this lesson we'll go over how Kubernetes Pod networking works at a high level, and we'll touch on the container networking interface or CNI. As you know, in Kubernetes, a Pod is a logical collection of one or more containers that will always be co-located and scheduled together on the same note in a cluster. Among other things, the containers in these Pods share a network namespace. To those of you that are new to containers or Linux networking details, in the context of networking, you can think of a namespace as basically a representation of a single network stack to be used by one or more processes that are running in the containers in a Pod.

The Kubernetes network model as described at their website here, stipulates a few rules that every Kubernetes network implementation must adhere to. First, every Pod gets its own IP address. This is in the network namespace we referred to earlier. Pods on a node can communicate with all Pods on all nodes, without network address translation, or NAT. Next, agents on a node. System demons, kubelet, things like that, can communicate with all Pods on that node. And finally, Pods in the host network of a node for platforms that support that like Linux, can communicate with all Pods on all nodes without NAT. Let's take a look at what's happening from a network point of view when a Pod is created by Kubernetes.

In this diagram, we see a pair of Pods, alpha and beta, each with two containers. The Pod gets a network namespace or stack with an IP address allocated to it, and both of the containers are started, attached to it. For containers within the same Pod to communicate with each other, A to B for instance, they simply can use local hosts, just like non-containerized processes running on the same machine would do. Note, we're talking about local host within the Pods network namespace here, not the local host of the route or node network namespace. If however, containers in different Pods need to communicate, let's say from A to C, they would need to use the IP address of the other Pod. To see how this works, let's step out of the Pods and get a little more detailed about what Kubernetes configured on the node.

Now, every Linux machine, a VM or otherwise, has a root network namespace that all processes on the machine can get access to. Every Pod gets a pair of virtual network devices, one in the Pod's namespace and the other in the root namespace. When a Pod needs to connect to the IP of another Pod on the same machine, the packet is sent via it's eth zero adapter through it's paired veth. Then the packet flows through to the destination veth to that Pods eth zero, where the container or process is hopefully listening. The details of how this is implemented can differ depending on what CNI plugin provider you're using. And we'll talk more about CNI shortly, but for now let's see a demonstration. Okay. To demonstrate this, let's deploy a pair of Pods somewhat similar to that diagram I showed you.

So as you can see, I've got a single node Kubernetes cluster, and I'm going to deploy pair Pods. Let me show you what I've got here. In this manifest, you see I've got an alpha Pod that has two containers, netshoot, which is a network troubleshooting type container that my buddy Nico wrote, and Genex, which is an open source web server. Nothing deployed to it, just the Genex server. And secondly, we have a Pod named beta that has also the netshoot container and Apache Tomcat with no web app or anything, just the Tomcat server. Simple stuff. So let's go ahead and kubectl, apply Pods. I've cashed these, so they should start up pretty fast. Let's do kubectl, get Pods, format wide. There we go. So both Pods are running, and both of them have two containers as indicated by the two out of two, under the ready. And you can see IP addresses were assigned to both Pods.

We have a dot nine and a dot ten address assigned. Lets exec into the alpha Pod, kubectl exec, interactive with TTY. And we want to connect to the netshoot named container on the alpha Pod, and we want to run Bash. There we go. We're inside that container now. And I'm going to show you that I can curl localhost. And sure enough, I get the NGINX server back. This is the default page, right? And that works. That's trying to hit that Tomcat server. So if I do a curl of the IP of the other, let me go ahead and copy it. So this is the beta Pod. Paste that in, and I know the default port of Tomcat is 8080, and I get a 404, which is valid because Tomcat doesn't have an app, anything deployed to it. So it's giving a 404. If it wasn't there, and if I couldn't hit it, I'd be getting some kind of action failure. So that worked. Get out of that.

So what happens when the destination Pod is on another node? Well, in this case, the root namespace will not have a matching port for the ARP lookup. So the packet will traverse out the default route, which in this example, is eth zero on the route machine. Assuming each node has a CIDR block for what Pod IPs it's capable of hosting, the network will route that traffic to the appropriate node, where the destination machine gets and will give it to the right veth and Pod. Now you might be saying, "Wait a minute, wait a minute. You skipped over a huge detail there with the assumption of this node to node routing." And you're right. This is another area where CNI plugins come into play. Now CNI plugins abstract away the network's specific details about how this and other implementations work.

There are dozens of CNI plugins, and a list of popular ones are available on the same Kubernetes documentation page we talked about earlier. Some of the most common ones include open source projects like Calico, Antrea, Flannel and Weave. There are also commercial plugins, or you could be using one pre-installed by your Cloud provider if you're in a managed or hosted solution. Which CNI plugin you choose is a big decision, and that varies depending on your network and feature needs. For example, Flannel is a popular CNI provider that's been around for a long time. Many choose it because of its relative simplicity and maturity. Flannel does not however provide network policy functionality. So if you're wanting to leverage that, you'd need to pick a different plugin that does. No. Network policy is a big topic and we're dedicating an entire lesson to it later in the course.

Okay. Let's see this in action. Looking at my terminal now, I've got three worker nodes in this cluster. Control plane and three workers. So let's deploy the same Pod. So I do kubectl, apply, Pods YAML, and we'll do kubectl get Pod, go wide, and it's taken a minute. So I cached these, but it's just taking a moment to start back up. So let's do that again, and we'll put a watch on this. Okay, there we go. So we have both the alpha and the beta running. Let me clean that up to... There we go. And the alpha is on a 1.4 and beta is on a 3.3. Notice those IPs are a little different, because we've got different nodes in play now, instead of all on the same node. So let's do similar to our last time. We're going to exec in. Yup. Let's go ahead and hit the Tomcat server on beta from here, across from node to node.

So we're going to grab the IP address of that, colon 8080. And sure enough, it works. So I've talked about how Kubernetes Pods work, but let's step back and think about why some of the decisions were made. Specifically, why does the model mandate that every Pod gets its own IP address? One of the main reasons is that it eliminates the problem of having multiple processes running on shared hosts that are contending for the same ports. Let's say, for example, we've got a travel booking application and over time we've been breaking up this monolith into separate services. We've got a couple JEE apps on Tomcat, a spring boot one and a Node.js one for this example. We also have a lot of large servers that we want to utilize as much as possible.

In order to be able to run these services alongside each other, we'll need to assign them unique port numbers so they don't collide. Additionally, if we want to run multiple instances of the same service, we have to manage that as well. So perhaps we decide to assign each service a range of ports, or maybe you have some kind of middleware or some external tooling that keeps track of what's running there. That's getting a bit complicated. So instead, we can use virtual machines to carve up the servers and utilize them better. You're still having to deal with multiple services in their ports on each VM though. Well, what if we separate the services up and have a separate VM for each one? Well, that does simplify the application deployment per VM, and supports elasticity more easily. But now we have the overhead of an OS for every instance of every service, including all the resource issues that come with them.

CPU, Ram, startup time, not to mention management of the service discovery and things like that. When Kubernetes, by giving every Pod its own IP address and managing them via the Kubernetes services, which we'll get into more in later lessons, you solve all these issues as well as the configuration complexities for developers and operators of the applications. To you use our travel app again, as an example, now every instance can run in its own Pod, and simply use its default port or whatever port we want. From the point of view of the applications, it looks like it's the only one on the machine, rather than having to keep separate server configurations for which instance is running. This looks similar to the one VM per service model, but without the resource and performance issues.

Plus, Kubernetes provides other benefits that you'll see in other lessons in this course, including a built-in service discovery mechanism. Speaking of service discovery, in a container orchestrated environment like Kubernetes, Pods could be scheduled anywhere in the cluster and stopped, restarted, moved around at will. So the IP addresses of them could be different from moment to moment. This raises the question of how do Pods and external clients know where to send traffic to? This is where the concept of services comes into play. For example, you can have an avail service that'll keep track of running avail Pods, and a check-in service to keep track of check-in Pods and so on. In the next lesson, you'll learn all about services and how to use them to enable access to your applications. Thanks for listening. We'll see you in that next lesson.