Provision a Cluster on Google Cloud with Terraform

Building a cluster in the cloud is becoming a mundane step that you have to do several times during the life cycle of a project. There are multiple driving forces behind this fact:

• Initial set up
• Redundancy
• Setting up an environment similar to the one in production
• Scaling to support a given number of request per second

Cloud providers (Amazon, Google, Digital Ocean) have a web UI that allows you to provision your services. However being able to completely script this process is a huge time saver and lets you build/ test/tear-down a cluster with a single instruction. The self-documentation aspect of such script has proven to be extremely valuable as well.

This example will use Terraform to build a small cluster on Google Cloud.

Terraform script

Terraform is easy to take in hand, the following subcommands do what you expect:

• terraform plan – Displays what would be executed
• terraform apply – Applies the changes
• terraform destroy – Wipes out what have been created

In this small tutorial we are going to create a Salt master server and two minions ready to be accepted and configured. Salt can be ignored at that point but it is a natural companion for a tool like Terraform. There are some overlap between the 2 tools but their core missions are different (over simplified):

• Salt configures the software installed on your cloud servers
• Terraform provisions your infrastructure (network, DNS, firewall, cloud servers)

If you follow along at the end of this article you should get:

• Network properly configure so serve an http app
• 1 Salt master to distribute and maintain the software configuration
• 2 salt minions ready to configure to serve your HTTP app

In order to do so we are going to create four files:

• main.tf – contains the definition of what we want to achieve
• variables.tf – contains the variables definition.
• terraform.tfvars – contains the values for variables.
• output.tf – contains the output that you want to see.

Variables

variabless.tf holds the definition of the elements that can be configured in your deployment script.

variable "region" {
  default = "us-central1"
}

variable "region_zone" {
  default = "us-central1-f"
}

variable "project_name" {
  description = "The ID of the Google Cloud project"
}

variable "account_file_path" {
  description = "Path to the JSON file used to describe your account credentials"
}

From now on every time you run a terraform commands {plan|apply|destroy|…} you will need to provide the required variables. Without further information terraform will enter an interactive mode requesting each variables one by one at the prompt:

$ terraform plan
var.account_file_path
  Path to the JSON file used to describe your account credentials

  Enter a value: /home/yml/google-service-accounts/gwadeloop.json

var.project_name
  The ID of the Google Cloud project

  Enter a value: gwadeloop
...

If you do not want to set these values on every run you can create a file called terraform.tfvars

region = "us-central1"
region_zone = "us-central1-a"
project_name = "gwadeloop"
account_file_path = "/home/yml/google-service-accounts/gwadeloop.json"

If you prefer you can also pass the variables directly in the command line:

$ terraform apply \
        -var="region=us-central1" \
        -var="region_zone=us-central1-f" \
        -var="project_name=my-project-id-123" \
        -var="account_file_path=~/.gcloud/gwadeloop.json"

Architecture

Now that we know what we want to build and also how we want to parametrize our script we are ready to build the main.tf. The code snippets below are extracted from this file.

First we define that we are going to work with Google Cloud Services I assume that you have already created a project in which you want to create your infrastructure.

provider "google" {
  region = "${var.region}"
  project = "${var.project_name}"
  credentials = "${file(var.account_file_path)}"
}

Networking

In this section we are going to provision a public IP address and two forwarding rules to route the traffic from this public address to our www cluster. The first rule will manage the HTTP traffic on port 80 and the second rule will do the same but for HTTPS on port 443.

resource "google_compute_address" "www" {
    name = "tf-www-address"
}

resource "google_compute_target_pool" "www" {
  name = "tf-www-target-pool"
  instances = ["${google_compute_instance.www.*.self_link}"]
  health_checks = ["${google_compute_http_health_check.http.name}"]
}

resource "google_compute_forwarding_rule" "http" {
  name = "tf-www-http-forwarding-rule"
  target = "${google_compute_target_pool.www.self_link}"
  ip_address = "${google_compute_address.www.address}"
  port_range = "80"
}

resource "google_compute_forwarding_rule" "https" {
  name = "tf-www-https-forwarding-rule"
  target = "${google_compute_target_pool.www.self_link}"
  ip_address = "${google_compute_address.www.address}"
  port_range = "443"
}

resource "google_compute_http_health_check" "http" {
  name = "tf-www-http-basic-check"
  request_path = "/"
  check_interval_sec = 1
  healthy_threshold = 1
  unhealthy_threshold = 10
  timeout_sec = 1
}

Cloud servers

In this section we are going to provision two kinds of servers:

• Salt master
• Salt minions that will compose our www cluster

The example below runs a shell script when the instance starts up that configures Salt and copies my states for configuration.

# Salt master server is used to configure and manage the minions
resource "google_compute_instance" "salt" {
  count = 1
  name = "tf-salt"
  machine_type = "f1-micro"
  zone = "${var.region_zone}"
  tags = ["salt", "letsencrypt"]

  disk {
    image = "ubuntu-1510-wily-v20160315"
  }
  network_interface {
    network = "default"
    access_config {
        # Ephemeral
    }
  }
  service_account {
    scopes = ["https://www.googleapis.com/auth/compute.readonly"]
  }

  metadata_startup_script = <<SCRIPT
aptitude -y update
#aptitude -y safe-upgrade
aptitude -y install salt-master salt-minion salt-ssh salt-cloud salt-doc
echo -e "master: $HOSTNAME" > /etc/salt/minion.d/master.conf
echo -e "grains:\n  roles:\n    - salt\n    - letsencrypt" > /etc/salt/minion.d/grains.conf
echo -e "file_roots:\n  base:\n    - /srv/gwadeloop-states/salt\npillar_roots:\n  base:\n    - /srv/gwadeloop-states/pillar" > /etc/salt/master.d/path_roots.conf
mkdir /srv/gwadeloop-states/
chown -R ubuntu:root /srv/gwadeloop-states/
SCRIPT

  provisioner "file" {
    connection {
      user = "ubuntu"
   }

    source = "/srv/gwadeloop-states"
    destination = "/home/ubuntu/"
  }

  provisioner "remote-exec" {
    connection {
     user = "ubuntu"
    }
   inline = [
     "sudo cp -r /home/ubuntu/gwadeloop-states /srv/gwadeloop-states",
     "sudo chown -R ubuntu:root /srv/gwadeloop-states"
   ]
  }

  metadata {
    sshKeys = "ubuntu:${file("~/.ssh/id_rsa.pub")}"
   }
}

# Salt minions (2) part of the www cluster
resource "google_compute_instance" "www" {
  count = 2
  name = "tf-www-${count.index}"
  machine_type = "f1-micro"
  zone = "${var.region_zone}"
  tags = ["www-node"]

  disk {
    image = "ubuntu-1510-wily-v20160315"
  }
  network_interface {
    network = "default"
    access_config {
        # Ephemeral
    }
  }
  service_account {
    scopes = ["https://www.googleapis.com/auth/compute.readonly"]
  }

  metadata_startup_script = <<SCRIPT
aptitude -y update
#aptitude -y safe-upgrade
aptitude install -y salt-minion
echo -e "master: ${google_compute_instance.salt.name}" > /etc/salt/minion.d/master.conf
echo -e "grains:\n  roles:\n    - webserver" > /etc/salt/minion.d/grains.conf
service salt-minion restart
SCRIPT

  metadata {
    sshKeys = "ubuntu:${file("~/.ssh/id_rsa.pub")}"
  }
}

Firewall

The last step of our main.tf consists of two firewall rules that allow incoming TCP traffic on port 80 and 443 for all the members of www cluster.

resource "google_compute_firewall" "www" {
  name = "tf-www-firewall"
  network = "default"

  allow {
    protocol = "tcp"
    ports = ["80", "443"]
  }

  source_ranges = ["0.0.0.0/0"]
  target_tags = ["www-node"]
}

Usage

At any point of time you can see what Terraform would do with the plan subcommand:

• terraform plan

To actually provision your cluster you can run:

• terraform apply

The command above will print out a detailed report about what has been done.

Define a custom output

Important information like IP addresses might end up being buried among less important pieces of information in the default output. If we add a file called output.tf with the following content:

output "public_ip" {
  value = "${google_compute_address.www.address}"
}

output "salt_ip" {
  value = "${join(" ", google_compute_instance.salt.*.network_interface.0.access_config.0.assigned_nat_ip)}"
}


output "instance_ips" {
  value = "${join(" ", google_compute_instance.www.*.network_interface.0.access_config.0.assigned_nat_ip)}"
}

Every time we run terraform apply or terraform output we are going to get a compact informative piece of information:

Outputs:

  instance_ips = 104.197.2.121 108.59.83.61
  public_ip    = 104.154.99.65
  salt_ip      = 104.154.66.25

Next steps

You should now have a basic understanding of how you can use Terraform to provision your own cluster and set up the network. We have just covered the basics, but the Terraform documentation will help you to take it from here.

One closing note: don’t forget to tear down your cluster when you are done experimenting with terraform destroy to avoid a surprise bill at the end of the month :)