Creating and Using Virtual Networks
This documentation describes how to create, use, and manage virtual networks.
- Creating Virtual Networks
- Working with Subnet Pools
- Joining Jobs to a Virtual Network
- Removing Jobs from a Virtual Network
- Managing Virtual Networks
- Enabling Broadcast and Multicast Routes
- Policy and Virtual Networks
- Virtual Network Reachability Example
- Network Discovery
Creating Virtual Networks
To create a virtual network you use the
apc network create <network> command, where
<network> is the network's local name or fully-qualified name.
For example, the following creates a new virtual network named
prod-net in the current namespace:
apc network create prod-net
The following command creates a network with the same local name in the
apc network create /prod::prod-net
Working with Subnet Pools
Assuming the standard cluster configuration for release 3.0, with layer 2 networks and subnet pools, by default each network you create is made a member of the default subnet pool and allocated its own ethernet address at 10.224.0.0/12.
To specify an existing subnet pool for the network, use the
apc network create net1 --pool my-pool-1
To define a custom default pool, see using subnet pools.
If you are using legacy layer 3 networks, each network you create is allocated its own subnet in the range 192.168.x.x/16.
Joining Jobs to a Virtual Network
apc network join <network> --job <job-name> command joins an existing job to a virtual network. For example, the following joins the job named
testapp to the network
test-network located in the
apc network join /networks::test-network --job testapp
You can optionally assign a discovery address to the job using the
apc network join <network> --job <job-name> --discovery-address <string> command. The discovery address is used to configure a DNS address through which instances can be discovered by other instances belonging to the same virtual network. For instance, the following joins the
testapp job to the
test-network and assigns the job the discovery address of
apc network join test-network --job testapp --discovery-address test-app
Any other jobs that are members of
test-network can access this job at
test-app.apcera.local. For more information, see Network discovery.
You can also join a job to a network when creating or updating a job or application. To do this, include the
--network parameter in the
apc app create (or
apc app update) command to specify the network to which the job should be added. For example, the following command creates an application named
app2 and also adds it to the network
apc app create testapp --network test-network
Jobs that join a network are assigned virtual IPs from the network's subnet. To see which jobs belong to a network and their associated IP addresses, you use the
apc network show command (see Showing Virtual Networks).
A job can be a member of one virtual network at a time. If a job is already a member of a virtual network, APC will display an error message if you try to add it to another network:
apc network join other-network --job testapp Error: job is already a member of a network.
If a job has multiple instances, all instances will be allocated their own IPs and be visible on the network.
Removing Jobs from a Virtual Network
To remove a job from a virtual network, you can use the
apc network leave command or
apc app update command. For example, the following are equivalent:
apc network leave /networks::test-network --job testapp apc app update testapp --network /networks::test-network -leave
Managing Virtual Networks
To view a virtual network's details you can use the
apc network show <network> or
apc network list commands.
apc network list command lists available virtual networks, including the name, namespace, and the network's assigned subnet. For example, the following shows that there are two virtual networks (
prod) that are assigned subnets of 192.168.1.0/24 and 192.168.2.0/24, respectively.
apc network list Working in "/sandbox/joe.user" ╭──────────────┬──────────────────────┬────────────────╮ │ Network Name │ Namespace │ Subnet │ ├──────────────┼──────────────────────┼────────────────┤ │ prod │ /apcera/prod │ 192.168.2.0/24 │ │ net1 │ /dev/tests │ 192.168.1.0/24 │ ╰──────────────┴──────────────────────┴────────────────╯
To view details about a specific virtual network, use the
apc network show <network> command.
For L2 networks, the output shows the subnet pool UUID that is used to segment the network, as well as the network address which shows the system default subnet pool address (10.224.0.0/12).
apc network show my-network ╭────────────────────┬──────────────────────────────────────╮ │ my-network │ │ ├────────────────────┼──────────────────────────────────────┤ │ Name: │ my-network │ │ FQN: │ network::/sandbox/admin::my-network │ │ UUID: │ 636f7c4c-8e34-417a-818e-323fbc22e47f │ │ Description: │ │ │ Pool UUID: │ system-default-pool-key │ │ Created By: │ firstname.lastname@example.org │ │ Updated By: │ │ │ Network Address: │ 10.224.0.0/12 │ │ Network Endpoints: │ │ ╰────────────────────┴──────────────────────────────────────╯
Or, if you have specified a custom subnet pool:
apc network show my-new-network ╭────────────────────┬─────────────────────────────────────────╮ │ my-new-network │ │ ├────────────────────┼─────────────────────────────────────────┤ │ Name: │ my-new-network │ │ FQN: │ network::/sandbox/admin::my-new-network │ │ UUID: │ 4aa7fdbc-348d-49ca-96f2-0d01a38f4fed │ │ Description: │ │ │ Pool UUID: │ a20524f0-3a69-4c0a-802a-12ffa79140c5 │ │ Created By: │ email@example.com │ │ Updated By: │ │ │ Network Address: │ 10.192.0.0/12 │ │ Network Endpoints: │ │ ╰────────────────────┴─────────────────────────────────────────╯
For L3 networks, the output displays the network's assigned subnet, as well as any member jobs and their assigned IP addresses and MAC addresses. The Network Address field shown below indicates that the
net1 network is assigned a subnet of 192.168.1.0/24. The Network Endpoints field indicates the network has two member jobs named
node-2, which are assigned IP addresses of 192.168.1.1 and 192.168.1.2, respectively.
apc network show network::/dev::net1 ╭────────────────────┬──────────────────────────────────────╮ │ net1 │ │ ├────────────────────┼──────────────────────────────────────┤ │ Name: │ net1 │ │ FQN: │ network::/dev/tests::net1 │ │ UUID: │ 773fd48c-241d-4975-8379-cca21ed3e8c8 │ │ Description: │ │ │ Created By: │ firstname.lastname@example.org │ │ Updated By: │ │ │ Network Address: │ 192.168.1.0/24 │ │ Network Endpoints: │ │ │ Job FQN: │ job::/dev::node-1 │ │ Interface: │ │ │ IPv4 address: │ 192.168.1.1/24 │ │ MAC: │ da:dc:a3:64:51:30 │ │ Job FQN: │ job::/dev::node-2 │ │ Interface: │ │ │ IPv4 address: │ 192.168.1.2/24 │ │ MAC: │ ba:44:f8:b7:4a:5a │ ╰────────────────────┴──────────────────────────────────────╯
To delete a virtual network, use the
apc network delete <network> command:
apc network delete /prod::prod-net
If you are using subnet pools, you must delete any member networks before you delete a subnet pool.
Enabling Broadcast and Multicast Routes
By default, network broadcast and multicast networks are not enabled in a virtual network. You can optionally enable broadcast and multicast routes on jobs in a virtual network. Enabling broadcast on a job provides a well-defined IP address (255.255.255.255) that all jobs in the same virtual network that also have broadcast can access. Enabling a multicast route enables the job to to join the multicast group you specified (220.127.116.11/24, for example). All jobs in the same multicast group can communicate with each other.
To enable broadcast or multicast support for a job, the job must belong to a virtual network, for example:
apc network join devnet --job app1
To enable the broadcast route on a job, update the job definition with the
-be) flag, for example:
apc job update app1 --network devnet --broadcast-enable
255.255.255.255 to the job's routing table. You can confirm this by SSH'ing into the job instance and running the Linux
route -n command, for example:
apc app update app1 --allow-ssh apc app connect app1 # route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default 169.254.0.6 0.0.0.0 UG 0 0 0 veth_b0681fbe 169.254.0.6 * 255.255.255.254 U 0 0 0 veth_b0681fbe 192.168.1.0 * 255.255.255.0 U 0 0 0 vi-21e8-066ea5 255.255.255.255 * 255.255.255.255 UH 0 0 0 vi-21e8-066ea5
To enable multicast group support for a job, update the job with the
--multicast-add flag and specify the CIDR of the network subnet to enable for multicast communication, for example:
apc job update app1 --network devnet --multicast-add 18.104.22.168/24
Note: The CIDR must specify a valid multicast address range between 22.214.171.124 and 126.96.36.199.
To remove the broadcast route from a job, you update the job with
apc job update app1 --network devnet --broadcast-disable
To remove a multicast group support from a job, you update the job with the
--multicast-delete flag, specifying the same CIDR used to add the multicast route:
apc job update app1 --network devnet --multicast-delete 188.8.131.52/24
For a complete example of using the broadcast route to communicate between jobs , see the broadcast-virtual-net sample application. The sample consists of two Go applications: one app (the server) listens on its broadcast address (255.255.255.255) for UDP connections, and a client application that sends UDP messages to the broadcast address.
Policy and Virtual Networks
Virtual networks are resources within an Apcera cluster (like jobs or packages) and have their own resource type ("network"). For instance, the fully-qualified name of the virtual network named
retail_network in the
/prod namespace is
network::/prod::retail_network. Policy can be written against virtual networks to control the following:
- If a particular user is allowed to create or join a virtual network in a given namespace.
- If a particular job is allowed to join a virtual network in a given namespace.
- The number of virtual networks that can be created in a a given namespace.
For more information on managing policy and virtual networks, see the following:
- Network Permissions lists and describes available permissions (claims) for network resources.
- Network Policy Example provides example policy documents for virtual networks.
Virtual Network Reachability Example
This example demonstrates basic network reachability and discovery between two capsules in the same virtual network. Both capsules are assigned network discovery addresses. You then connect (SSH) to each capsule and ping the other using its assigned discovery address.
To ping a capsule from another capsule in the same network:
Create a new virtual network in the current namespace:
apc network create ping-net
Create two capsules,
apc capsule create cap-1 --image linux --batch apc capsule create cap-2 --image linux --batch
Join each capsule to
ping-netand assign a network discovery address to each one:
apc network join ping-net --job cap-1 --discovery-address cap-1 apc network join ping-net --job cap-2 --discovery-address cap-2
This results two DNS entries being added to the virtual network,
cap-2.apcera.local, that are assigned to the
apc capsule connect cap-1
cap-2at its DNS address,
root@ip-169-254-0-41:/root$ ping cap-2.apcera.local PING cap-2.apcera.local (192.168.3.2) 56(84) bytes of data. 64 bytes from 192.168.3.2: icmp_seq=1 ttl=64 time=0.353 ms 64 bytes from 192.168.3.2: icmp_seq=2 ttl=64 time=0.080 ms 64 bytes from 192.168.3.2: icmp_seq=3 ttl=64 time=0.085 ms
In another terminal, connect to
apc capsule connect cap-2 root@ip-169-254-0-41:/root$ ping cap-1.apcera.local PING cap-1.apcera.local (192.168.3.1) 56(84) bytes of data. 64 bytes from 192.168.3.1: icmp_seq=1 ttl=64 time=0.353 ms 64 bytes from 192.168.3.1: icmp_seq=2 ttl=64 time=0.080 ms 64 bytes from 192.168.3.1: icmp_seq=3 ttl=64 time=0.085 ms
When joining a job to a virtual network you can optionally specify the job's discovery address. This address is used to create a DNS address through which job instances can be discovered by other instances belonging to the same virtual network. The DNS address is composed of the discovery address you provide concatenated with
.apcera.local. For example, if you assign the discovery address "myapp" to a job, the DNS address for the job instance will be
You assign a network discovery address to a job with the
--discovery-address flag when joining the job to a network.
The syntax is as follows:
apc network join <network> --job <job-name> --discovery-address <string>
Or, using shortcut flags:
apc network join <network> -j <job-name> -da <string>
For example, the following command adds the job
app1 to the
testnet network and assigns the discovery address
my-app to the job:
apc network join testnet --job myapp --discovery-address my-app
This results in the DNS address
my-app.apcera.local being assigned to the job. Any other job that's a member of the same virtual network can access the job at this address. See Virtual Network Reachability Example for an example of this. Jobs outside a virtual network cannot lookup or connect to jobs in that network.
Note: If a job has more than one instance, one is picked randomly when it is looked up.
Discovery addresses are case-insensitive and may consist of letters and numbers. Dots and hyphens are also allowed, except at the beginning and end of the discovery address.
If a job hasn't been assigned a discovery address, it can still be discovered by its UUID. For example, a job with the UUID of "f81d4fae-7dec-11d0-a765-00a0c91e6bf6" can be discovered at the address of
You can find a job's UUID in a number of ways:
- At runtime, job instances can read the
CNTM_JOB_UUIDenvironment variable to get the job's UUID.
- Using APC, you can find a job's UUID with the
apc job list -lcommand. The
-loption includes each job's UUID in the command output.
When viewing a job's details view in the web console, it's UUID is displayed in the browser's location field.