Configuring Hybrid Deployments

This section describes how to configure an Apcera cluster for hybrid cloud deployments using component-level tagging. For more information about Apcera's hybrid cloud features and capabilities, see Job Scheduling.

Overview

Apcera supports the use of component tag_rules for the Instance Manager (IM) and HTTP Router (router) components. You can tag these components with one or more user-defined strings and schedule jobs to run on tagged components.

In addition, you can use datacenter tags to logically associate IMs, HTTP routers, and Package Managers (PMs) (beta feature) in one or more virtual datacenters.

Component tags

You use tag_rules to specify one or more tags for a component.

Usage

tag_rules
    { "filter": "filter-value", "tag-type": ["tag1", "tag2", "tagN"] }

Arguments

  • The filter parameter controls what type of matching is performed
    and must be either network_cidr or datacenter. The
    filter-value is then matched against each host in the cluster to
    determine whether this tag rule should be applied to the host.

  • The tag-type is one of the supported tag types:

    • extra_tags are in addition to the default tags that are auto-defined when the host is created, such as "hostname_provider"

    • remove_tags removes the tag you are specifying

    • override_tags removes all existing tags and replaces them with
      the list of tags specified

    • tag_group is used for HTTP Routers

NOTE: Datacenter and component tags are unrelated to the suitable_tags: [ "component-name" ] parameter set in the machines.instance_manager block. Suitable tags are reserved for internal purposes.

IM tag rules

You can tag IMs using the tag_rules parameter. The tag_rules parameter supports an array of rules. Each rule specifies a filter used to select which hosts to apply tags to, and one or more tag changes to apply. Each rule is applied in order.

Values specified in the override_tags replace all existing tags. Values specified in the extra_tags are in addition to the existing tag list, while remove_tags specifies a list of tags to remove.

The following example shows how to tag IMs. The filter is used to specify the IP address of the IM host where to apply the tag.

chef: {
  "continuum": {
    "instance_manager": {
      "tag_rules": [
          { "network_cidr": "10.0.0.0/8",  "override_tags": ["aws", "us-west-2", "DMZ"] },
          { "network_cidr": "10.11.0.0/16", "extra_tags":    ["dev"] },
      ],
    }
  }
}

Once IMs are tagged, you can affinitize jobs to IMs using matching tags. Refer to Job Scheduling for more information.

HTTP Router tag rules

The Apcera HTTP router supports using tags to choose which job routes
each router should use when forwarding client connections. Similar to
IM tags, you use tag_rules to set the tags used by the HTTP Routers.

To enable this feature you must set the router configuration parameter
enable_route_tagging to true.

The tag list specified for a router is compared against the tags
configured on the Instance Manager which is announcing a route to
determine whether the route should be used. By default, all routers
will use their datacenter configuration as a route selection
preference to prefer "local" routes over remote ones, but will still
route to any available job.

To set all routers to only accept routes that match their tags, using
the tags as a filter instead of as a preference, set the router
configuration parameter default_route_tagging_behavior to
filter. (Default setting: prefer)

Some internal components like the API server generate routes that have
no tags at all. By default all routers accept those routes. To set
all routers to ignore those routes set the router configuration
parameter default_route_untagged_behavior to ignore. (Default
setting: allow) Note: If you set the default to ignore you must
use tag rules (below) to set this behavior to allow on the routers
which your API and Auth server routes connect through, or the API and
Auth servers will be unreachable.

For example:

chef: {
  "continuum": {
    "router": {
      "enable_route_tagging": true,
      "tag_rules": [
          { "network_cidr": "10.10.0.0/8",  "extra_tags": ["dc1", "dc2"] },
          { "network_cidr": "10.11.0.0/8",  "override_tags": ["aws", "us-west-2", "DMZ"] },
      ],
    }
  }
}

In addition, HTTP Routers support the use of non-default tag rules by
specifying the tag_group in a tag rule. Using tag groups you can
specify primary and secondary regions for your routers.

For example, continuum-lb-secondary-tag can be added as the
tag_group and allows you to control the HTTP Router's primary and
secondary route selection order independently.

chef: {
  "continuum": {
    "router": {
      "tag_rules": [
        # This adds "dc1" and "dc2" to the primary tag list for routers in 10.10.x.x
        { "network_cidr": "10.10.0.0/16",  "extra_tags": ["dc1", "dc2"] },
        # This adds "dc3" and "dc4" as secondary tags that the routers
	# will use if no routes exist with one of the primary tags
        { "network_cidr": "10.10.0.0/16",  "tag_group": "continuum-lb-secondary-tag", "override_tags":    ["dc3", "dc4"] },
      ],
    }
  }
}

Other tag groups and values that are supported for HTTP Routers are:

  • continuum-lb-tagging-behavior: Allows per-router control of the route tagging behavior. Can be set to prefer or filter.
  • continuum-lb-untagged-behavior: Allows per-router control of the usage of untagged system routes like the API and Auth Servers. Can be set to allow or ignore.

Datacenter tags

The datacenter parameter is a single value attribute on a node indicating a logical name for the datacenter the node resides in. The datacenter tag may be used by supported Apcera components (IMs, PMs, routers) to configure cooperation within a virtual datacenter.

Within the datacenter block you can specify one or more tag_rules. Each rule specifies the IP address filter and the datacenter_name parameter and tag. The datacenter_name can then be used as a matching filter in the tag_rules section of other components.

For example, you can apply datacenter tag rules to IMs and routers as shown below. For this example, assume the cluster has the following topology:

  • Four datacenters, DC1 - DC4, each with a /24 of IP space allocated to the Apcera platform installation.
  • All datacenters have Instance Managers and Package Managers
  • Instance managers in the first half (X.X.X.0-127) of the IP space in DC1 and DC2 should be tagged "prod", and the other half tagged "QA"
  • All Instance managers in DC3 and DC4 should be tagged "dev"
  • Four specific IMs in DC1 & 2 should get the special "High-Memory" tag, and the DC name, but NO OTHER TAGS
  • All datacenters have Apcera routers
  • Connections coming in to any DC should route to jobs in that DC by default
  • Connections coming in to DC1 should route to DC2 before considering DC3 or DC4 routes
  • Connections coming in to DC2 should route to DC1 before considering DC3 or DC4 routes
chef: {
  "continuum": {
    "datacenter": {
      "tag_rules": [
          # Note: Instance Managers will download packages from a Package Manager in the same datacenter by default
          { "network_cidr": "10.0.1.0/24",  "datacenter_name": "dc1" },
          { "network_cidr": "10.0.2.0/24",  "datacenter_name": "dc2" },
          { "network_cidr": "10.0.3.0/24",  "datacenter_name": "dc3" },
          { "network_cidr": "10.0.4.0/24",  "datacenter_name": "dc4" },
      ],
    }
    "instance_manager": {
      "tag_rules": [
          # Note: All IMs get the datacenter name applied as a tag by default
          # Apply the prod and QA tags by IP matching
          { "network_cidr": "10.0.1.0/25",   "extra_tags":    ["prod"] },
          { "network_cidr": "10.0.1.128/25", "extra_tags":    ["QA"] },
          { "network_cidr": "10.0.2.0/25",   "extra_tags":    ["prod"] },
          { "network_cidr": "10.0.2.128/25", "extra_tags":    ["QA"] },
    # Apply the dev tags to all IMs in these datacenters
          { "datacenter":   "dc3",           "extra_tags":    ["dev"] },
          { "datacenter":   "dc4",           "extra_tags":    ["dev"] },
    # These four hosts get all tags stripped and replaced with this exact list only
          { "network_cidr": "10.0.1.10/32",  "override_tags": ["High-Memory", "dc1"] },
          { "network_cidr": "10.0.1.11/32",  "override_tags": ["High-Memory", "dc1"] },
          { "network_cidr": "10.0.2.10/32",  "override_tags": ["High-Memory", "dc2"] },
          { "network_cidr": "10.0.2.11/32",  "override_tags": ["High-Memory", "dc2"] },
      ],
    }
    "router": {
      "enable_route_tagging": true,
      "tag_rules": [
          # Note: All routers default to having their datacenter name as the primary tag for route matching
    # DC1 should prefer DC2 before falling back to DC3 or DC4
          { "datacenter":   "dc1",     "tag_group": "continuum-lb-secondary-tag", "override_tags":    ["dc2"] },
    # DC2 should prefer DC1 before falling back to DC3 or DC4
          { "datacenter":   "dc2",     "tag_group": "continuum-lb-secondary-tag", "override_tags":    ["dc1"] },
      ],
    }
  }
}