We explore core Terraform concepts using a concrete Heroku example.
The real reason for this article is that I wanted to explore Terraform using a simpler infrastructure than Amazon Web Services, Microsoft Azure, or Google Cloud Platform; Heroku is definitely simpler.
One common challenge with developing with Heroku is ensuring that the Heroku App infrastructure is synchronized with the code that it is running. For example, if we add PostgreSQL database support to the code, we need to provision a PostgreSQL database in the Heroku App before deploying the code.
In this article, we will use Terraform to manage both the Heroku App infrastructure and code deployment and thus address our challenge.
Only if one wants to follow-along…
note: The final solution provided in this article is available for download.
The first Terraform concept that we need to understand is a provider:
A provider is responsible for creating and managing resources. A provider is a plugin that Terraform uses to translate the API interactions with the service. A provider is responsible for understanding API interactions and exposing resources.
— Terraform — Providers
In our particular case, we will be using the heroku provider.
To get started, we pick a globally unique name for our application (using letters, numbers, and dashes), e.g., I used larkintuckerllc-my-app .
We generate a Heroku authorization token for this application using the following command (used the application’s name in the description simply for consistency):
heroku authorizations:create --description larkintuckerllc-my-app
We set the following environment variables using the created Heroku authorization token and the email associated with our Heroku account with these commands:
note: These environment variables are only valid for the current terminal. To set in a new terminal, we can display our authorization ids using the heroku authorizations command and heroku authorizations:info <authorization_id> to display the token.
We create a project folder using the application’s name (again for consistency) with a single file in it; main.tf:
note: This file is written in a format called Terraform Configuration Language consisting of blocks. Blocks have a type, e.g., provider, with zero or more labels, e.g., heroku, and zero or more arguments, e.g,. version = “~> 2.0”. Other than the .tf extension, the name of the file is arbitrary.
Finally, we run the command from within the project folder:
This command downloads the providers’ plugins; in this case just the heroku plugin. These plugins are stored in a new created hidden folder, .terraform, in the project folder.
Given that providers create resources, we now need to understand what a resource is:
The resource block defines a resource that exists within the infrastructure. A resource might be a physical component such as an EC2 instance, or it can be a logical resource such as a Heroku application.
The resource block has two strings before opening the block: the resource type and the resource name.
— Terraform — Resources
To get started, we will create a Heroku App resource.
A web application on Heroku. Has a set of dynos that run the application’s source code. Has a unique .herokuapp.com URL and release history.
— Heroku — App
We update our main.tf file:
When I first wrote this example, my inclination was to ensure that resource names were globally unique and representative of the resource type, e.g, something long like larkintuckerllc_my_app_heroku_app. I later found out that Terraform has documented Naming conventions that would discourage this pattern. Following these conventions, we use this as the name.
We can run the following command to validate the syntax of the configuration file.
note: Another command, terraform fmt standardizes the configuration file’s formatting.
The following command describes what applying this change would do:
Once we validate that the changes are what we expect, e.g., in this case it will create a Heroku application named larkintuckerllc-my-app, we use the following command to apply the changes:
Logging into Heroku, we indeed see that Terraform has created the application.
Also notice that by applying the change, Terraform creates two files (one is a backup) that captures state the state of the managed infrastructure:
Terraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures.
This state is stored by default in a local file named “terraform.tfstate”, but it can also be stored remotely, which works better in a team environment.
While the format of the state files are just JSON, direct file editing of the state is discouraged. Terraform provides the terraform state command to perform basic modifications of the state using the CLI.
— Terraform — State
note: If you are committing your work to a version control system (VCS) such as Git, do not commit the .terraform, terraform.tfstate, or terraform.tfstate.backup files.
Hello World Node.js Application
Our goal is to deploy a basic Node.js application to the Heroku App that was just created; we first, however, need be able to run it locally first.
We create an folder named app in our project folder and run the following command in it to initialize it as a Node.js project; accepting all defaults:
note: If you are committing your work to a version control system (VCS) such as Git, do not commit the app/node_modules folder.
In this same app folder, we create a file, index.js, with the Node.js Getting Started Guide example:
We can start this example by executing the following command and opening the provided url, http://127.0.0.1:3000/, in a web browser.
We need to make three changes to prepare this application for Heroku deployment.
First, we update index.js to have the application listen on the port provided by the PORT environment variable and two, we update it to listen on any hostname / address:
Third, we create a Procfile in the
app folder that instructs Heroku on how to start the application:
With these changes in place, we now can run our application locally with the following command; this command sets the PORT environment variable among other things:
Deploy to Heroku App (and Resource Dependencies)
We can now add a second resource, heroku_build, that deploys the app folder to the Heroku App.
One important observation here is that this second resource depends on first resource; this is an example of an implicit dependency in that the second resource references the first in the app attribute.
note: One can also explicitly define resource dependencies using the depends_on attribute; supplying an array of resource names; we will use this later.
As before, we can describe the change with:
and apply the change with:
Logging into Heroku, we can see that indeed the application is built and deployed.
By pressing the Open app button, we can indeed see that the application is deployed.
If one runs either terraform plan or terraform apply again, Terraform will indicate the infrastructure is up-to-date. The heroku_build resource, using a checksum, determines if the contents of the app folder changes and thus whether this resource is up-to-date.
One minor improvement to the Terraform configuration would be to output the Heroku App URL when the terraform apply command is executed. This is accomplished using the output block type:
Now let us say that we want to ensure the Heroku App is not only deployed, but actually responding to requests before the heroku_build resource completes. Here we can use a provisioner:
But if you need to do some initial setup on your instances, then provisioners let you upload files, run shell scripts, or install and trigger other software like configuration management tools, etc.
— Terraform — Provision
Here we can create a bash script, scripts/health-check:
note: This script was provided in the Heroku article, Using Terraform with Heroku.
And then add it to the heroku_build resource:
With this in place, when the heroku_build is created, the provisioner executes; does not execute on resource updates. It is important, however, to note that this particular resource is destroyed and re-created each time there is a change in the source folder.
Remote State Storage
You have now seen how to build, change, and destroy infrastructure from a local machine. This is great for testing and development, but in production environments it is considered a best practice to store state elsewhere than your local machine. The best way to do this is by running Terraform in a remote environment with shared access to state.
Terraform supports team-based workflows with a feature known as remote backends. Remote backends allow Terraform to use a shared storage space for state data, so any member of your team can use Terraform to manage the same infrastructure.
— Terraform — Remote State Storage
To enable the remote backend, we add a terraform block to the Terraform configuration file; main.tf:
and then run the command:
We then remove the local terraform.tfstate and terraform.tfstate.backup files as the state now resides in Terraform Cloud.
At the same time, we will want to continue to have the plans and applies occur locally (not remotely on Terraform Cloud). This is because the heroku_build resource needs access to the files in the app and scripts folders. This is a setting available in Heroku Cloud under the workspace> Settings > General > Execution Mode > Local.
Now our terraform plan and terraform apply commands use the state from Heroku Cloud but still run locally.
Hello Database Node.js Application
Now that we have our deployment solution in place, let us use it to update our application to use a PostgreSQL database. First, however, let us get everything working locally.
The first thing that we need to know is that adding a PostgreSQL database to a Heroku App automatically creates an environment variable, DATABASE_URL, that our application can use to connect to the database. To develop this locally then, we need to manually create this same environment variable with a valid connection string to a PostgreSQL database.
While we could have used Docker to relatively easily setup a local PostgreSQL database, an even easier solution is to create a separate Heroku App, e.g., larkintuckerllc-db, that has a single Heroku Postgres resource.
note: The connection string can be found in the Heroku App’s Settings > Reveal Config Vars menu. Also want to emphasize that this database, while on Heroku, is our local development database.
We then create a file app/.env with this connection string.
note: If you are committing your work to a version control system (VCS) such as Git, do not commit the app/.env file.
We then add support for PostgreSQL in our application by running the command (in the app folder).
npm install pg
We then update app/index.js to use the PostgreSQL database.
With this in place, we can run our application locally (using the .env file) using the following command:
Deploying the Hello Database Node.js Application
With our code updated, we can now simply update our Terraform configuration with a heroku_addon resource (and having our heroku_build resource depend on it).
We deploy with the command:
(Bonus) Continuous Integration with Travis CI
So far we have been doing our Terraform deployment from our local development environment. Thought to quickly explore how one can deploy using a continuous integration system, e.g., Travis CI.
In this case, we can create a Travis CI configuration file; .travis.yml:
The referenced script; scripts/create-config:
By activating a GitHub repository with Travis CI and setting the following environment variables in Travis CI we have our automated deployment:
Hope you found this useful.