Required Ports for Deployment

This documentation lists the ports in use by the Apcera platform as well as ports that would typically be in use by other processes when deployed.

For supported provisioners, during installation Apcera will create the required network rules and security groups for cluster host connectivity and component communcations. The exception is vSphere where we only create the virtual machines, not the networks or security groups. For vSphere you will need to set up your own firewall rules using the information provided here. See also Network requirements and segmentation guidelines.

All Machines

All machines within an Apcera cluster will be listening on the following ports:

  • 22 - SSH
  • 7778 - Orchestrator Agent
  • 8089 - Splunk Forwarder (optional, if the cluster is instrumenting Splunk)
  • 10050 - Zabbix Agent

Individual Cluster Processes

This section lists the ports we require for Apcera processes in the cluster.

NATS

  • 4222 - NATS client port
  • 4242 - NATS cluster port
  • 8222 - NATS http/debug port

Auth Server

  • 5678 - HTTP API port
  • Randomly chosen port is used for debugging

Flex Auth Servers

  • APP_AUTH_SERVER = 5705
  • BASIC_AUTH_SERVER = 5702
  • CROWD_AUTH_SERVER = 5703 (deprecated)
  • GOOGLE_AUTH_SERVER = 5700
  • LDAP_AUTH_SERVER = 5701
  • KERBEROS_AUTH_SERVER = 5704 (deprecated)
  • KEYCLOAK_AUTH_SERVER = 5706

API Server

  • 8790 - HTTP API port
  • Randomly chosen port is used for debugging

Consul

  • 8300 - Used by servers to handle incoming requests from other agents.
  • 8301 - This is used to handle gossip in the LAN. Required by all agents. TCP and UDP.
  • 8302 - This is used by servers to gossip over the WAN to other servers. TCP and UDP.
  • 8500 - HTTP API Port (never used outside of localhost)

Cluster Monitor

  • 6768 - HTTP API port
  • Randomly chosen port is used for debugging.

Metrics Manager

  • 6767 - HTTP API port
  • Randomly chosen port is used for debugging.

Package Manager

  • 8989 - HTTP API port
  • Randomly chosen port is used for debugging

Instance Manager

  • 8686 - HTTP API port
  • 4789/udp - For vXLan virtual networks, GRE otherwise
  • Randomly chosen port is used for debugging

NOTE: IMs need outbound connectivity to NATS and the PMs, inbound connectivity from the routers on effectively all ports.

HTTP Router/Nginx

The nginx router will listen on different ports, depending on the cluster configuration and infrastructure deployed to.

On AWS, the routers lie behind an Elastic Load Balancer, which will receive traffic on 80/443 typically and forward it to the routers on 8080/8181.

  • 8080 - Standard HTTP port
  • 8181 - HTTPS port
  • 8282 - Nginx status port, only queryable by localhost

On other clusters, it will often use the following ports:

  • 80 - Standard HTTP port
  • 443 - HTTPS port
  • 6104 - Outside of AWS, Riak is used for package persistence, and the router proxies requests to Riak. Those requests will come from the PM host.
  • 8282 - Nginx status port, only queryable by localhost

IP Manager

  • 8787 - HTTP API port
  • Randomly chosen port is used for debugging

Vault

  • 8200 - HTTP API port
  • 8201 - Used for server-to-server communication

Job Manager

Job Manager will listen on two randomly chosen ports. One is for an HTTP API which is automatically bound, however the Job Manager doesn't have any handlers associated with it (no endpoints). The other is the standard component debug handler.

Cluster Monitor

Cluster Monitor will listen on two randomly chosen ports. One is for an HTTP API which is automatically bound, however the Cluster Monitor doesn't have any handlers associated with it (no endpoints). The other is the standard component debug handler.

Health Manager

Health Manager will listen on two randomly chosen ports. One is for an HTTP API which is automatically bound, however the Health Manager doesn't have any handlers associated with it (no endpoints). The other is the standard component debug handler.

Metrics Manager

Metrics Manager will listen on two randomly chosen ports. One is for an HTTP API which is automatically bound, however the Metrics Manager doesn't have any handlers associated with it (no endpoints). The other is the standard component debug handler.

Events Server

The Events Server uses port 8585 for the /healthz endpoint.

TCP Router

TCP Router will listen on two randomly chosen ports. One is for an HTTP API which is automatically bound, however the TCP Router doesn't have any handlers associated with it (no endpoints). The other is the standard component debug handler.

Cluster monitoring and management (bastion zone)

  • Orchestrator host needs connectivity to every host in the cluster via port 7777 on the Orchestrator server host.
  • Zabbix-server
    • Zabbix host needs connectivity to every host in the cluster 10015 - 10051
    • If IPsec is enabled, we make us use of Zabbix traps, which are inbound 10051/tcp to the Zabbix server.
  • splunk-server

NFS Singleton

We open ports the following ports for the NFS singleton server: 32765, 32766, 32767, and 32768.

Container Processes

Several pieces of Apcera run as containers inside of a cluster. They listen on ports, however they are in their own network namespace to avoid collisions with the host, and are not directly accessible from outside by the same port.

  • 5602 - Semantic Pipeline Update Port

File Server (NFS non-HA)

If you deploy a file server (NFS non-HA) it will open these ports to the other servers within the subnet.

  • 111 - NFS (rpcbind)
  • 2049 - NFS

File Server Cluster (NFS HA)

If you deploy a file server cluster (NFS HA) it will open these ports to the other servers within the subnet.

  • 49152 - GlusterFS

IPsec Server

If you enable IPsec, all components must have the following ports open:

  • 500/udp
  • 4500/udp

In addition, you must have the ESP(50) protocol and AH(51) protocol open.

Dependent Services

Apcera leverages a number of third party applications for things like databases. The following is a list of ports used by non-Apcera owned processes:

  • 22 - SSH
  • 111 - NFS (rpcbind)
  • 2003 - Graphite's carbon process
  • 2004 - Graphite's carbon process
  • 2049 - NFS
  • 5432 - PostgreSQL
  • 6100-6106 Riak
    • 6100 Riak PB (ProtoBuff) (default: 8087)
    • 6101 Riak HTTP (default: 8098)
    • 6102 Riak HTTPS (default: 9098)
    • 6103 Riak Cluster Manager (default: 9085)
    • 6107 Riak CS (default: 8080)
    • 6105 Riak Stanchion (default: 8085)
    • 6106 Riak CS Control (default: 8000)
  • 6379 - Redis
  • 7002 - Graphite's carbon process
  • 8082 - Nginx, which proxies requests to graphite
  • 8089 - Splunk Forwarder (optional, if the cluster is instrumenting Splunk)
  • 8126 - statsd
  • 8200 - Vault HTTP API port
  • 8201 - Vault server-to-server communication
  • 8300-8301 - Consul internal communication
  • 8500 - Consul HTTP API Port
  • 10050 - Zabbix monitoring agent
  • 11211 - Memcached, used by graphite
  • 49152 - GlusterFS

Aggregated List

This is the aggregated list of ports that would typically be in use at the host level and seen when doing a port scan.

  • 22 - SSH
  • 80 - Apcera HTTP Router (non-AWS)
  • 111 - NFS (rpcbind)
  • 443 - Apcera HTTP Router (non-AWS)
  • 2003 - Graphite's carbon process
  • 2004 - Graphite's carbon process
  • 2049 - NFS
  • 32765, 32766, 32767, and 32768 - NFS singleton server
  • 4222 - NATS client port
  • 4242 - NATS cluster port
  • 5432 - PostgreSQL
  • 5678 - Apcera Auth Server
  • 6100-6106 Riak
    • 6100 Riak PB (ProtoBuf?) (default: 8087)
    • 6101 Riak HTTP (default: 8098)
    • 6102 Riak HTTPS (default: 9098)
    • 6103 Riak Cluster Manager (default: 9085)
    • 6104 - Apcera HTTP Router (when Riak is used)
    • 6107 Riak CS (default: 8080)
    • 6105 Riak Stanchion (default: 8085)
    • 6106 Riak CS Control (default: 8000)
  • 6379 - Redis
  • 7002 - Graphite's carbon process
  • 8080 - Apcera HTTP Router (AWS)
  • 8082 - Nginx, which proxies requests to graphite
  • 8089 - Splunk Forwarder (optional, if the cluster is instrumenting Splunk)
  • 8126 - statsd
  • 8181 - Apcera HTTP Router (AWS)
  • 8200 - Vault HTTP API port
  • 8201 - Vault server-to-server communication
  • 8222 - NATS http/debug port
  • 8282 - Apcera HTTP Router
  • 8500 - Consul
  • 8686 - Apcera Instance Manager
  • 8787 - Apcera IP Manager
  • 8790 - Apcera API Server
  • 8989 - Apcera Package Manager
  • 10050 - Zabbix monitoring agent
  • 11211 - Memcached, used by graphite
  • 49152 - GlusterFS

Datacenter-aware PMs

If you want to put a Package Manager component in each datacenter or workload zone with the IMs, the following list of component ports are required for the PM to communicate with the central host (brain) and other components.

  • PM to Orchestrator
  • PM to NATS on ports 4222 and 4242
  • PM to Router on port 6104
  • PM to Component DB on port 5432
  • PM to Audit Log DB on port 5432
  • API Server to PM on the PM HTTP port
  • Zabbix Server to PM on port 10050
  • All IMs to each PM on port 8989

IMs will default to trying to download packages from the PM in their local datacenter, but will fallback to any PM if the "local" PM doesn't respond.

The typical package upload flow is as follows:

(1) apc → (2) Nginx → (3) API server → (4) Package Manager → (5) Storage backend

However, if Riak is used then the flow is:

(1) apc → (2) Nginx → (3) API server → (4) Package Manager → (5) Nginx → (6) Riak store