KubeAcademy by VMware
Demo: Logs with EFK
Next Lesson

In this lesson, we'll walk through a complete logging stack running on Kubernetes cluster so you can see what kinds of data we can collect and visualize.

Hart Hoover

Senior Field Engineer at Kong

Hart Hoover is a Senior Field Engineer at Kong. His expertise lies in technical training, consulting, community building, Linux-based operating systems, computing automation, and cloud application architecture.

View Profile

Hello. My name is Hart Hoover, Manager of Kubernetes Education at VMware, and in this lesson, we'll be going through a demo of what an Elasticsearch Fluent Bit and Kibana stack look like in a Kubernetes cluster, as well as look at some sample logs and see what data is collected in a cluster.

First, I'd like to show you what we're actually deploying. We're using the Elasticsearch operator in Kubernetes to provide Elasticsearch and Kibana as a service to my cluster, meaning I can use a custom resource definition to say, "I need an Elasticsearch. I need a Kibana," and a set will be deployed for me.

I'm also using the Fluent Bit DaemonSet, installation steps right off their website, which will deploy a DaemonSet, which means a pod on every node in the cluster to collect logs from VAR log container on every single node in the cluster.

Looking at what is actually deployed in my cluster, I have a namespace called logging, that's going to hold all of my logging information or logging services. And you can see here that I've got a StatefulSet for Elasticsearch with just a single pod. Granted, this is a demo, this is not production. You would definitely want more than one if this was a production set up. But for now, one's fine.

That StatefulSet deploys a single pod. Notice it is numbered as a StatefulSet is. That StatefulSet is a pod backed by persistent storage. I also have a DaemonSet for Fluent Bit. Because it's a DaemonSet, it runs a single pod on every node. And here, those are listed here. This is running on a three node cluster. So, I have three pods.

And then finally, I have a deployment for my Kibana service. And Kibana is used to visualize log data that's stored in Elasticsearch. So, high-level Fluent Bit running as a DaemonSet on every node in my cluster, feeding data into Elasticsearch as a data store, and then Kibana being used to visualize that data.

I also have some services in here so these things can talk to each other. For example, Fluent Bit uses the Elasticsearch service to feed in data, instead of talking to this pod directly. Kibana also uses the Elasticsearch service to read from, to read its data.

Let's take a look at data in an actual cluster. This is Kibana, and I'm visualizing log data that's stored in my cluster while Fluent Bit is collecting all these logs of containers running in my cluster. I don't really have that much running in here, because again, this is a demo cluster, not much going on. But there are some logs I can see.

As this data loads here, we'll take a look at a sample log entry. You can see that Fluent Bit, because it can talk to the Kubernetes API, gets all sorts of metadata about this log entry that it's writing. I've got a docker ID for the container that it was running from, I've got a container name in a pod, I have a pod ID down here, and a pod name, as well as what host this workload was running on, and things like if it was using a pod security policy, that's there as well. And these log messages are of course timestamped. And then finally, I've got the log message itself.

You can use these Kubernetes labels here, or tags, to search for data about services running in your cluster. And you can do it even on things like labels or if it's a control plane service or not, all sorts of things. And Fluent Bit grabs this data for you by talking to the Kubernetes API.

This has been a quick demo of a logging infrastructure running in a Kubernetes cluster. We saw that there was a DaemonSet for Fluent Bit, a StatefulSet for Elasticsearch running from the operator, and finally, a deployment of Kibana for visualization of logs. Information about how to install these services can be found online very easily. We'll also have a link to resources in this lesson below. Thank you.

Give Feedback

Help us improve by sharing your thoughts.

Links

Share