Exploring Grafana through a concrete example using a Prometheus data source.

What is Grafana?

Grafana is open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.

— Grafana Labs — Getting Started

In order to make sense of Grafana, we need to understand what a time-series is and have access to one; for this, we build off the learning environment in a separate article Prometheus by Example. …

The article that I did not think I needed to write but did.

It turns out that understanding how to authenticate to Google Cloud on your workstation is more complicated than one would think.

Please note: This article is not about authenticating a user account to the Google Cloud Console. It is also not about authenticating a service account on a GCE instance.

First, we need to understand that there are two separate components that are authenticated separately: Google Cloud SDK Command Line Tools and Google Cloud Client Libraries.

First, we have Google Cloud SDK Command Line Tools:

The gcloud CLI manages authentication, local configuration, developer workflow, interactions with Google Cloud APIs. With the gcloud command-line tool, it’s easy to perform many common cloud tasks, like creating a Compute Engine VM instance, managing a Google Kubernetes Engine cluster, and deploying an App Engine application, either from the command line or in scripts and other automations.
…
A collection of command-line tools comes packaged with Cloud SDK, including gsutil, bq, and kubectl. The gsutil tool allows you to manage Cloud Storage buckets and objects using the command line. Run queries and manipulate datasets, tables, and entities in BigQuery through the command line with bq. With kubectl, you can deploy and manage Kubernetes container clusters using the command line. …

Exploring Prometheus through a familiar example.

First, what is Prometheus?

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company.

— Prometheus — Overview

One key feature of Prometheus is that it collects metrics from target systems, systems being monitored, by pulling over HTTP.

Why do you pull rather than push?
Pulling over HTTP offers a number of advantages:

You can run your monitoring on your laptop when developing changes.
You can more easily tell if a target is down.
You can manually go to a target and inspect its health with a web browser.
Overall, we believe that pulling is slightly better than pushing, but it should not be considered a major point when considering a monitoring system. …

Exploring Google Kubernetes Engine (GKE) native integration with GCP Cloud Logging.

Google Kubernetes Engine (GKE) includes native integration with Cloud Monitoring and Cloud Logging. When you create a GKE cluster, Cloud Operations for GKE is enabled by default and provides a monitoring dashboard specifically tailored for Kubernetes.

— GCP — Overview of Google Cloud’s operations suite for GKE

Prerequisites

If you wish to follow along, you will need administrative access to a GCP project with a GKE cluster and Kubectl CLI configured for the cluster. One way to accomplish this is to have:

• Gcloud CLI installed and configured for your GCP…

A simple example of using Kubernetes watches.

Recently I ran into a situation where I needed to maintain a list of endpoints — specifically the IP addresses of a Kubernetes service. Because the service was backed by a deployment with a horizontal pod autoscaler, the list of endpoints (pods) was updated regularly and unpredictably.

One approach to maintaining this list of endpoints is to regularly — say every minute — query the endpoints URL. To illustrate this, we start a kubectl proxy on our workstation with:

$ kubectl proxy --port 8080

And then we can use a browser on our workstation to query the URL, here a deployment and service are already running in the my-namespace namespace. …

More on Hiera and a quick bit on files/templates.

This is part of a series of articles starting with Puppet Code by Example: Part 1. The final body of Puppet Code developed through this series is available for download.

Parameterized Modules

In the previous article, we parameterized a class, my_parameters::my_class, in a module. The obvious problem with this approach is that this class is an implementation detail of the module and as such should not be exposed outside of the module, i.e., we had to supply the parameter using Hiera as
my_parameters::my_class::greeting: Hola Mundo.

Here we will rather parameterize the module and use that parameter in the class. Let us first create a parametrized module by creating a my_parameters_refactor module with the classes my_class and the usual main (my_parameters_refactor) class. …

Exploring variables, facts, and parameters.

This is part of a series of articles starting with Puppet Code by Example: Part 1. The final body of Puppet Code developed through this series is available for download.

Variables

So far we have only been using hard-coded values, e.g., the string Hello World. Here we introduce Puppet variables (which behave more like constants) to help eliminate duplicated hard-coded values.

Variables store values so that those values can be accessed in code later.

After you’ve assigned a variable a value, you cannot reassign it. …

Learning Puppet Code through example.

Yes… Puppet, for many, is a fading technology as we have collectively moved towards immutable infrastructure. At the same time, there are plenty of legacy systems that still use it and I recently happened upon such a system.

While Puppet has been around for some time, I could not find a tutorial that synchronized with me. The closest I could find was a paid course Puppet Quick Start by A Cloud Guru and even that was a little rough. Thus, I was motivated to write this series of articles.

Another important observation is that — because Puppet has been around for some time — most of the tutorials I found used dated Puppet Code patterns. This series of articles is based on the most recent Puppet Code 6.19, using up-to-date recommended patterns. …