Kubernetes Storage By Example: Part 1

Exploring Kubernetes storage topics through example.

It is assumed that the reader has a basic understanding of Kubernetes, e.g., in particular Pods.

Also, if one wants to follow along, all the examples run on MicroK8s; although the concepts generally apply to all Kubernetes implementations. The examples are also available for download.

Motivation

Generally, I only write articles when I find learning a topic particularly challenging. Through this process, I re-enforce my learning as well as produce something that I will inevitably will turn to in the future.

In my recent study of Kubernetes, I struggled to learn the topic of storage from the official documentation and found little else generally available.

Writable Container Layer

As containers are ephemeral, it is proper to think of containers as immutable. At the same time, we remind ourselves that they themselves do have a writable container layer that is our simplest form of storage.

When you create a new container, you add a new writable layer on top of the underlying layers. This layer is often called the “container layer”. All changes made to the running container, such as writing new files, modifying existing files, and deleting files, are written to this thin writable container layer.

— Docker — About Storage Drivers

This writable container layer is typically used for temporary files and such; not suitable for writing either persistent or large volumes of data.

Volumes (Temporary)

On-disk files in a Container are ephemeral, which presents some problems for non-trivial applications when running in Containers. First, when a Container crashes, kubelet will restart it, but the files will be lost — the Container starts with a clean state. Second, when running Containers together in a Pod it is often necessary to share files between those Containers. The Kubernetes Volume abstraction solves both of these problems.

— Kubernetes — Volumes

We start our exploration of Kubernetes Volumes with the emptyDir Volume type in much the same way we used writable container layers, i.e., creating temporary files (neither persistent or large volumes of data).

An emptyDir volume is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node. As the name says, it is initially empty. Containers in the Pod can all read and write the same files in the emptyDir volume, though that volume can be mounted at the same or different paths in each Container. When a Pod is removed from a node for any reason, the data in the emptyDir is deleted forever.

— Kubernetes — Volumes

Here we use an emptyDir Volume to share files between two containers in a Pod.

empty-dir/pod.yaml

Unless otherwise noted, all the examples in this article are provided as Kubernetes configuration files organized into folders; one per section. For example, we can apply this section’s configuration with the following command.

kubectl apply -f empty-dir

note: We can use this same command, replacing apply with delete, to clean up after ourselves.

We write a file to the Volume from the busybox-a container:

kubectl exec empty-dir --container busybox-a -- sh -c "echo \"Hello World\" > /cache/hello.txt"

And read from that same file from the busybox-b container:

kubectl exec empty-dir --container busybox-b -- cat /cache/hello.txt

Volumes (Persistent)

Not to be confused with a PersistentVolume (covered in the next section), here we explore connecting Volumes to Pods with persistent data. A representative example of this is an awsElasticBlockStore Volume:

An awsElasticBlockStore volume mounts an Amazon Web Services (AWS) EBS Volume into your Pod. Unlike emptyDir, which is erased when a Pod is removed, the contents of an EBS volume are preserved and the volume is merely unmounted. This means that an EBS volume can be pre-populated with data, and that data can be “handed off” between Pods.

— Kubernetes — Volumes

For simplicity, however, in this article we will use a hostPath Volume:

A hostPath volume mounts a file or directory from the host node’s filesystem into your Pod. This is not something that most Pods will need, but it offers a powerful escape hatch for some applications.

— Kubernetes — Volumes

In preparation for this example, we need to create a folder named data in the root of our development machine with a file named hello.txt in it. With this in place, we create our configuration:

host-path/pod.yaml

After applying this configuration, we can see that we can read the hello.txt file:

kubectl exec host-path -- cat /data/hello.txt

Next Steps

We introduce PersistentVolumes and StorageClasses in the next article: Kubernetes Storage By Example: Part 2.