Exploring Kubernetes monitoring on Google Cloud Platform (GCP) through a concrete example.

Image for post
Image for post

Please note: For exploring Kubernetes logging on GCP, there is another article: Google Kubernetes Engine Logging by Example.

When we create a Kubernetes cluster on Google Kubernetes Engine (GKE), Google Cloud Operations for GKE is enabled by default.

Google Kubernetes Engine (GKE) includes native integration with Cloud Monitoring and Cloud Logging. When you create a GKE cluster, Cloud Operations for GKE is enabled by default and provides a monitoring dashboard specifically tailored for Kubernetes.

— GCP — Overview of Google Cloud’s operations suite for GKE

Please note: As Legacy Logging and Monitoring is deprecated and will be decommissioned on March 31, 2021, this article does not cover it. …


Exploring Grafana through a concrete example using a Prometheus data source.

Image for post
Image for post

What is Grafana?

Grafana is open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.

— Grafana Labs — Getting Started

In order to make sense of Grafana, we need to understand what a time-series is and have access to one; for this, we build off the learning environment in a separate article Prometheus by Example.


The article that I did not think I needed to write but did.

Image for post
Image for post

It turns out that understanding how to authenticate to Google Cloud on your workstation is more complicated than one would think.

Please note: This article is not about authenticating a user account to the Google Cloud Console. It is also not about authenticating a service account on a GCE instance.

First, we need to understand that there are two separate components that are authenticated separately: Google Cloud SDK Command Line Tools and Google Cloud Client Libraries.

First, we have Google Cloud SDK Command Line Tools:

The gcloud CLI manages authentication, local configuration, developer workflow, interactions with Google Cloud APIs. With the gcloud command-line tool, it’s easy to perform many common cloud tasks, like creating a Compute Engine VM instance, managing a Google Kubernetes Engine cluster, and deploying an App Engine application, either from the command line or in scripts and other automations.

A collection of command-line tools comes packaged with Cloud SDK, including gsutil, bq, and kubectl. The gsutil tool allows you to manage Cloud Storage buckets and objects using the command line. Run queries and manipulate datasets, tables, and entities in BigQuery through the command line with bq. With kubectl, you can deploy and manage Kubernetes container clusters using the command line. …


Exploring Prometheus through a familiar example.

Image for post
Image for post

First, what is Prometheus?

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company.

— Prometheus — Overview

One key feature of Prometheus is that it collects metrics from target systems, systems being monitored, by pulling over HTTP.

Why do you pull rather than push?
Pulling over HTTP offers a number of advantages:

You can run your monitoring on your laptop when developing changes.
You can more easily tell if a target is down.
You can manually go to a target and inspect its health with a web browser.
Overall, we believe that pulling is slightly better than pushing, but it should not be considered a major point when considering a monitoring system. …


Exploring Google Kubernetes Engine (GKE) native integration with GCP Cloud Logging.

Image for post
Image for post

Google Kubernetes Engine (GKE) includes native integration with Cloud Monitoring and Cloud Logging. When you create a GKE cluster, Cloud Operations for GKE is enabled by default and provides a monitoring dashboard specifically tailored for Kubernetes.

— GCP — Overview of Google Cloud’s operations suite for GKE

Prerequisites

If you wish to follow along, you will need administrative access to a GCP project with a GKE cluster and Kubectl CLI configured for the cluster. One way to accomplish this is to have:


A simple example of using Kubernetes watches.

Image for post
Image for post

Recently I ran into a situation where I needed to maintain a list of endpoints — specifically the IP addresses of a Kubernetes service. Because the service was backed by a deployment with a horizontal pod autoscaler, the list of endpoints (pods) was updated regularly and unpredictably.

One approach to maintaining this list of endpoints is to regularly — say every minute — query the endpoints URL. To illustrate this, we start a kubectl proxy on our workstation with:

$ kubectl proxy --port 8080

And then we can use a browser on our workstation to query the URL, here a deployment and service are already running in the my-namespace namespace. …


More on Hiera and a quick bit on files/templates.

Image for post
Image for post

This is part of a series of articles starting with Puppet Code by Example: Part 1. The final body of Puppet Code developed through this series is available for download.

Parameterized Modules

In the previous article, we parameterized a class, my_parameters::my_class, in a module. The obvious problem with this approach is that this class is an implementation detail of the module and as such should not be exposed outside of the module, i.e., we had to supply the parameter using Hiera as
my_parameters::my_class::greeting: Hola Mundo.

Here we will rather parameterize the module and use that parameter in the class. Let us first create a parametrized module by creating a my_parameters_refactor module with the classes my_class and the usual main (my_parameters_refactor) class. …


Exploring variables, facts, and parameters.

Image for post
Image for post

This is part of a series of articles starting with Puppet Code by Example: Part 1. The final body of Puppet Code developed through this series is available for download.

Variables

So far we have only been using hard-coded values, e.g., the string Hello World. Here we introduce Puppet variables (which behave more like constants) to help eliminate duplicated hard-coded values.

Variables store values so that those values can be accessed in code later.

After you’ve assigned a variable a value, you cannot reassign it. …


Learning Puppet Code through example.

Image for post
Image for post

Yes… Puppet, for many, is a fading technology as we have collectively moved towards immutable infrastructure. At the same time, there are plenty of legacy systems that still use it and I recently happened upon such a system.

While Puppet has been around for some time, I could not find a tutorial that synchronized with me. The closest I could find was a paid course Puppet Quick Start by A Cloud Guru and even that was a little rough. Thus, I was motivated to write this series of articles.

Another important observation is that — because Puppet has been around for some time — most of the tutorials I found used dated Puppet Code patterns. This series of articles is based on the most recent Puppet Code 6.19, using up-to-date recommended patterns. …


Wrapping up our step-by-step walk-through by deploying an application to a GKE cluster.

Image for post
Image for post

This is part of a series (starting with Spinnaker by Example: Part 1) providing a step-by-step walk-through for installing and using Spinnaker to deploy applications to a Google Kubernetes Engine (GKE) cluster. The final set of configuration files provided throughout this series of articles is available for download. So far our focus has been on the installation and configuration aspects of Spinnaker. In this article, we wrap up this series with a simple example of deploying an application to a GKE cluster.

The bad news is that using Spinnaker is surprisingly confusing; I believe it is related to how flexible and powerful it is. The good news, however, is that Spinnaker provides a number of Codelabs that walk one through particular scenarios. The better news is that there is a Codelab, Kubernetes Source To Prod, that walks through our particular scenario of deploying an application to a Kubernetes cluster. This article is closely aligned with this Codelab; just provides a bit more detail and is updated for the latest version of Spinnaker (1.23.1). …

About

John Tucker

Broad infrastructure, development, and soft-skill background

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store