Deploying Acpera to GCE

This document describes how to create GCE resources using Terraform and how to deploy an Apcera cluster to GCE using Orchestrator.

Complete prerequisites


Edit the file with the following information.

GCE AUTH Variables

variable "auth-project"            { default = "Your `project_id` from JSON file or Google Cloud Console" }
variable "auth-region"             { default = "YOUR-GCE-REGION" }
variable "auth-account-file"       { default = "Local PATH to JSON file" }

For example:

variable "auth-project"            { default = "apcera-ee-gce" }
variable "auth-region"             { default = "us-central1" }
variable "auth-account-file"       { default = "gce_auth.json" }


variable "cluster"                   { default = "CLUSTER-NAME" }
variable "apcera-base-image"         { default = "CONTINUUM-BASE-IMAGE" }
variable "apcera-orchestrator-image" { default = "ORCHESTRATOR-IMAGE" }
variable "all-cluster-cidrs"         { default = [ "xx.xx.xx.xx/24" ] }

The cluster name must contain alphanumeric characters. You set the cluster domain in the cluster.conf.erb file described below.

For example:

variable "cluster"                   { default = "my-cluster" }
variable "apcera-base-image"         { default = "continuum-base-1434669182" }
variable "apcera-orchestrator-image" { default = "apcera-orchestrator-1444764646" }
variable "all-cluster-cidrs"         { default = [ "" ] }

NOTE: Your image names will differ.

Apcera module sources

Update the module sources to point to your local Terraform module files that you downloaded from the support site.

module "subnets" {
  source = "/Users/user/gce/network_with_subnets"
module "subnet0_hosts" {
  source = "/Users/user/gce/servers_for_subnet"

Component count

Adjust the component count, if necessary.

For example, the following adjustment deploys a singleton NFS server instead of a 3-node Gluster cluster for persistence.

  monitoring-count          = 1
  orchestrator-count        = 1
  auditlog-count            = 1
  exposed-router-count      = 1
  metricslogs-count         = 1

  # multiple hosts of the same hosttype are automtically spread out between the zones
  central-count             = 3
  riak-count                = 3
  instance-manager-count    = 3
  # gluster-count           = 3 # must be a multiple of 3
  nfs-count                 = 1

Note that the GCE module does not include any of the following optional components:

  • ip-manager
  • tcp-router
  • splunk-indexer
  • splunk-search
  • (non exposed) HTTP routers for use with load balancing (not supported for this release)

Such components can be enabled by adding the following to the file:

  • ip-manager-count = 1
  • tcp-router-count = 1

If you want to integrate with Splunk, see the configuring Splunk documentation.

Other customizations

You can make further customizations by uncommenting one or more options as indicated in the file.

Run Terraform

  1. CD to the /gce directory you extracted the Terraform GCE module and example files.

    cd $HOME/gce

  2. Run the command terraform version and check the version.

    If necessary install the supported Terraform version.

    See also

  3. Run the command terraform get.

    The get command downloads and caches in the working directory the modules used by this particular Terraform configuration. This must be run again with --update to import changes to the upstream modules, so without that flag it is safe to automate this step.

  4. Run the command terraform plan --module-depth -1.

    The plan command displays the changes Terraform will attempt. If you receive any errors, debug as necessary.

  5. Run the command terraform apply.

    Use the apply command to apply and run the changes. This command may take some time to complete.

    If you receive any errors, rerun the terraform apply command a second time.

  6. Verify creation of GCE resources

    When the terraform apply command completes successfully, log in to the GCE Console for your account.

    From the Dashboard select the Resource panel.


    You should see the GCE resources created by Terraform:


Configure DNS

You need to configure DNS before you deploy the cluster.

Get the public IP address for the <cluster-name>-net0-exposed-router-0. For example:

You can run the command terraform output to get the address of the HTTP router.

You must create or update a DNS "A" record for the HTTP server that points to the base_domain entry. See Configuring DNS for guidance. If you are using multiple routers, update DNS to point to the load balancer that fronts the routers.

For example:

  • <cluster-name>.<domain>.<tld> An "A" record with the public IP address of the exposed-router-public-addresses host.
  • *.<cluster-name>.<domain>.<tld> A CNAME pointing to <cluster-name>.<domain>.<tld>.
  • <cluster-name>.clustermon.<domain>.<tld> An "A" record with the value for monitoring-public-addresses.

Configure SSH and Connect to Orchestrator

  1. Add your private SSH key to your local SSH agent:

     ssh-add apcera.pem

    You should see the message "Identity added: apcera.pem (apcera.pem)," indicating success.

    Run the following command and confirm that your key was successfully added:

     ssh-add -l

    If you get an error saying that "Permissions 0644 for 'apcera.pem' are too open," change the permission on the file so that only you can read it, then re-run the ssh-add command: chmod 400 apcera.pem.

  2. Connect to the Orchestrator host via SSH.

    Obtain the public IP address for Orchestrator in the Azure portal by selecting the -orchestrator VM and looking at the "Essentials" pane.

    Connect remotely to Orchestrator as follows:

         ssh -A orchestrator@
         orchestrator@'s password:

    Successful login:

         Last login: Thu Nov  3 17:05:04 2016 from
  3. Check the Orchestrator version and upgrade if necessary.

     orchestrator-cli version

    If you are not running Orchestrator 0.5.3 or later, update Orchestrator software and perform a kernel upgrade followed by a reboot.

Edit cluster.conf.erb

Edit the cluster.conf.erb file with the following information:

1) The cluster_name and base_domain.

For example:

cluster_name  = 'my-cluster'
base_domain   = cluster_name + ''

2) The cluster admin user names and passwords.

For example:

chef: {
  "continuum": {
    "auth_server": {
      "identity": {
        "basic": {
          "enabled": true,
          "users": [
            { "name": "admin",    "password": "PASSWORD" }
            { "name": "user1",    "password": "PASSWORD" }
            { "name": "someone",  "password": "PASSWORD" }

      },        # auth_server -> identity

      "admins": [

    },          # continuum -> auth_server
  },            # continuum

3) The Monitoring user names and passwords.

For example:

chef: {
  "apzabbix": {
    "db": {
      "hostport": "localhost:5432",
      "master_user": "apcera_ops",
      "master_pass": "PASSWORD",
      "zdb_user": "zabbix",
      "zdb_pass": "PASSWORD"
    "users": {
      "guest": { "user": "MONITOR", "pass": "PASSWORD" },
      "admin": { "user": "ADMIN", "pass": "PASSWORD", "delete": false }
    "web_hostnames": [
      "<%= cluster_name %>-bastion",
      "<%= cluster_name %>.clustermon.<domain-name>.<tld>"

    #Defined as a PD integration of type 'Zabbix'
    #"pagerduty": {
    "email": {
      #"sendto": "",
      "sendto":      "",
      "smtp_server": "localhost",
      "smtp_helo":   "localhost",
      "smtp_email":  ""
  }     # apzabbix

NOTE: The initial deploy is HTTP. To enable HTTPS, add your SSL keys to the chef.continuum.router section of cluster.conf. Typically you will perform an initial deploy and then update it for HTTPS.

Generate cluster.conf

  1. Run the following command to generate the cluster.conf file:

     erb cluster.conf.erb > cluster.conf

    The cluster.conf file is used to deploy the cluster. If successful this command should exit silently.

    Verify that the generated cluster.conf file is output to your directory. If you encounter an error, run the command again.

    If you want to update your cluster, you should update the cluster.conf.erb file and generate an updated cluster.conf file. In other words, you should avoid editing the cluster.conf file directly.

  2. Copy cluster.conf to the Orchestrator host.

    Use SCP to copy cluster.conf to the orchestrator user home directory using the Orchestrator IP address, for example:

     scp cluster.conf orchestrator@

    If you see the message, “Are you sure you want to continue connecting (yes/no)?” Type “yes” to proceed.

    Verify that the cluster.conf file is copied to the Orchestrator host:

     cluster.conf 100% 9537 9.3KB/s 00:00
  3. Copy the release bundle to Orchestrator (optional).

    During the deployment (described below) you can pull the latest release from cloud storage. Alternatively, to speed deployment time and prevent various round-trips between the Orchestrator server and the internet, you can download the release bundle and copy it to the Orchestrator host. To download release bundle, contact Apcera Support.

    To securely copy the release bundle tarball to the Orchestrator host, for example:

scp release-3.0.0.tar.gz orchestrator@ orchestrator@'s password: release-3.0.0.tar.gz 100% 581MB 283.2KB/s 35:02

Deploy the cluster

  1. SSH to the Orchestrator host.

     ssh -A orchestrator@

    You should be connected, indicated by orchestrator@<host>-orchestrator:~#.

  2. Initialize the Orchestrator DB.

    For first time deploys only (not updates), you must initialize the Orchestrator database:

     orchestrator-cli init

    If you do not initialize the DB for initial deploys, you receive the error Error: no tenant records found!.

  3. Perform a dry run to validate cluster.conf:

     orchestrator-cli deploy -c cluster.conf --update-latest-release --dry

    Or using a release bundle you copied to Orchestrator:

     orchestrator-cli deploy -c cluster.conf --release-bundle release-3.0.0.tar.gz --dry-run

    Use ls to verify that the graph.png file is generated in the home directory.

    Type exit to log out of the Orchestrator host.

    Copy the graph.png file to your local computer and examine it for errors:

     scp orchestrator@ ~/apcera-azure
     orchestrator@'s password:
     graph.png                                             	100%  487KB 243.6KB/s   00:02
  4. Deploy the cluster.

    To deploy the latest promoted release from the internet:

     orchestrator-cli deploy -c cluster.conf --update-latest-release

    To deploy a release bundle you copied to Orchestrator:

     orchestrator-cli deploy -c cluster.conf --release-bundle release-3.0.0.tar.gz

    Read and acknowledge any reboot warning messages.

    Successful deployment is indicated by the message "Done with cluster updates."

    NOTE: If this is an initial deploy, you may have to run the deploy a second time to ensure all hosts are monitored by Zabbix.

  5. Test the deployment

    See Post Installation Tasks.