Events System API Overview

The Events System enables Apcera Platform users to obtain life-cycle information and resource usage metrics from their cluster in real-time. You can subscribe to events using APC or using a custom event client you create that streams events using Web Application Messaging Protocol (WAMP).

The Event Reference describes the structure and content of event payloads.

There are several potential use cases for the events system:

  • Auto-scaling – A process could react to resource usage spikes for a job by increasing the number of job instances or allocating additional memory by calling Apcera REST API endpoints.
  • Billing – Events generated from the system could be used to analyze customer usage or to predict future usage of resources for future or peak conditions.
  • Route management – A monitoring agent could watch for changes to route endpoints on a job or namespace and update an external proxy management system with the new endpoint.

If you are using Enterprise Edition, to use the events system you must deploy an Event Server by editing the cluster.conf file. Specifically you need to place "events-server" for the "suitable_tags" list of the central machine. And you need to edit the "components" section to include the "events-server". See Populating cluster.conf.

Demo: Subscribing to events with APC

The apc event subscribe <fqn> subscribes to events for a job and displays events in the terminal as they arrive. In this tutorial you will create two APC terminal windows: one to run APC commands to create and modify the job at the specified FQN, and the other to subscribe and view events that are generated for that FQN.

To subscribe to events using APC:

  1. Open two terminal windows (or tabs) arranged next to each other.
  2. In one terminal (the "event viewer") enter the following command to subscribe to events for a job with the FQN of job::/sandbox/admin::testapp. Replace /sandbox/admin with your user's sandbox namespace, if necessary.

     apc event subscribe job::/sandbox/admin::testapp
     Success!
    

    A success message indicates that the event subscription was successful and APC is waiting for events to be published. At this point, no events are being published because this job doesn't exist yet.

  3. In the second terminal window use APC to create (but not start) a new application with the same FQN (/sandbox/admin::testapp):

     apc app create /sandbox/admin::testapp -p ~/sample-apps/example-static --batch
     Packaging... done
     ...
     App may be started with:
       > apc app start testapp
     Success!
    
  4. Return to first terminal window where you should see a JSON-encoded job update event.

    Job create event

    Notice the event object field named action whose value is job_create; this specifies the action that caused this event to be generated. Below is an example JSON object formatted for easier readability:

     {
       "event_source": "api_server@apcera.me",
       "payload": {
         "action": "job_create",
         "create_time": 1.4667196166355507E+18,
         "created_by": "admin@apcera.me",
         "job_uuid": "230609f8-483a-42d0-8752-abb40f3ac049",
         "num_instances": 1,
         "tags": {
           "app": "testapp"
         },
         "update_time": 1.4667196166355507E+18,
         "user": "admin@apcera.me"
       },
       "resource": "job::/sandbox/admin::testapp",
       "time": 1.4667196166369388E+18,
       "type": 0
     }
    
  5. In the second terminal window again, start the job you just created:

     apc app start testapp
    
  6. In your event viewer terminal notice the job_update event that appears, corresponding to the job being started.

     {"event_source":"api_server@apcera.me","payload":{"action":"job_update","create_time":1.4667196166355507e+18,"created_by":"admin@apcera.me","job_uuid":"230609f8-483a-42d0-8752-abb40f3ac049","num_instances":1,"tags":{"app":"testapp"},"update_time":1.4667200668742285e+18,"updated_by":"admin@apcera.me","user":"admin@apcera.me"},"resource":"job::/sandbox/admin::testapp","time":1.466720066875815e+18,"type":0}
    

    Momentarily, a resource usage event should appear containing resource usage for CPU, memory, network and other job resource metrics. A new resource usage event is published automatically every 10 seconds.

     {"event_source":"","payload":{"cpu":1.803383e+06,"disk_total":1.073741824e+09,"disk_used":1.51552e+06,"instance_uuid":"50a5fe18-bd85-462e-9ce8-3296446d6f70","job_fqn":"job::/sandbox/admin::testapp","job_uuid":"230609f8-483a-42d0-8752-abb40f3ac049","memory_total":2.68435456e+08,"memory_used":4.702208e+06,"network_total":5e+06,"network_used":{"veth-50a5fe18":{"RxBytes":36,"TxBytes":52}},"timestamp":1.466720077e+09},"resource":"job::/sandbox/admin::testapp","time":1.466720077467392e+18,"type":0}
     {"event_source":"","payload":{"cpu":10283,"disk_total":1.073741824e+09,"disk_used":1.51552e+06,"instance_uuid":"50a5fe18-bd85-462e-9ce8-3296446d6f70","job_fqn":"job::/sandbox/admin::testapp","job_uuid":"230609f8-483a-42d0-8752-abb40f3ac049","memory_total":2.68435456e+08,"memory_used":4.644864e+06,"network_total":5e+06,"network_used":{"veth-50a5fe18":{}},"timestamp":1.466720087e+09},"resource":"job::/sandbox/admin::testapp","time":1.466720087468278e+18,"type":0}
    
  7. Lastly, delete testapp using APC:

     apc app delete /sandbox/admin::testapp
    

    In the event viewer terminal notice the job_delete event that appears:

     {"event_source":"api_server@apcera.me","payload":{"action":"job_delete","create_time":1.4667217265730588e+18,"created_by":"admin@apcera.me","job_uuid":"f4d806fc-4ff5-41ad-9666-216aa7371d5e","num_instances":1,"tags":{"app":"testapp"},"update_time":1.4667217265730588e+18,"user":"admin@apcera.me"},"resource":"job::/sandbox/admin::testapp","time":1.4667228883550372e+18,"type":0}
    
  8. To unsubscribe from the event stream close the event viewer terminal window, or press Control+C.

Subscribing to events with WAMP

In addition to streaming events using APC, you can create your own application to stream events from an Apcera cluster over WAMP (Web Application Messaging Protocol). WAMP is a WebSocket sub-protocol that provides a PubSub messaging pattern. A WAMP client subscribes to receive events from an Apcera cluster for one or more FQNs. Apcera includes a WAMP router which streams new events as they arrive to subscribed clients.

There are a number of WAMP client libraries available for various languages, such as the Go Turnpike library used in the Go Event Client sample application, or the AutobahnJS library used with the Node.js Event Client sample application.

Connecting to the WAMP router

To subscribe to an events stream you first connect your WAMP client to the cluster's WAMP router. WAMP client libraries provide a method to connect to a WAMP router, which takes two parameters:

  • The URL of the WAMP server to connect to. For the Apcera event system, the connection URL is wss://api.<cluster-domain>/v1/wamp (if your cluster is configured with TLS) or ws://api.<cluster-domain>/v1/wamp (no TLS).
  • The connection "realm". This is always com.apcera.api.es.

For example, the following JavaScript uses the AutobahnJS library to connect to the WAMP router on the example.com cluster:

var connection = new autobahn.Connection({
    url: "wss://api.example.com/v1/wamp",
    realm: "com.apcera.api.es"
});

Your event client must authenicate with the API server for the connection to succeed. The recommended way to do this is using app tokens, as demonstrated in the Node.js sample application. Alternatively, you can obtain an authentication token manually and add it as a query parameter named authorization, as shown below:

var token = "Bearer eyJ0eXAiOiI...";
var connection = new autobahn.Connection({
    url: "wss://api.example.com/v1/wamp?authorization=" + token
    realm: "com.apcera.api.es"
});

Once your app made a connection it can subscribe to events.

Subscribing to events with WAMP

Once your client has established a connection to the WAMP router, you can subscribe to receive events for a given resource FQN. To subscribe to an event stream for a resource or namespace, you must have read permission on that resource.

In WAMP parlance, the FQN string (job::/ns::my-app) is the subscription topic. WAMP client libraries provide a method you call to subscribe to a topic and begin receiving messages. The : and / characters used in FQNs hold special meaning for WAMP client libraries, which may try to validate subscription topic strings before attempting to subscribe. To avoid validation errors related to special characters, it's recommended that you escape : and / characters in subscription FQNs (standardized into UTF-8 characters) before using the client's subscribe API. For example, if you want to subscribe to events for job::/apcera::continuum-guide you should pass the string job%3A%3A%2Fapcera%3A%3Acontinuum-guide to your WAMP client's subscribe method.

As another example, the Node.js Event Client sample application uses the JavaScript encodeURIComponent() method to escape the topic FQN before passing it to the Autobahn|JS library's subscribe() method, as shown below:

var topicFQN = encodeURIComponent("job::/apcera::continuum-guide");
ws.subscribe(topicFQN, {
   onEvent: function(data) {
       console.log(data[0].resource, data[0].payload);
   },
   onSuccess: function() {
       console.log('Successfully subscribed to topic: ' + topicFQN);
   },
   onError: function(err) {
       console.log('Subscription error:' + err);
   },
});

Subscription FQNs and event types

To subscribe to event using APC a WAMP client you provide the fully-qualified name (FQN) of the resource for which you're interested in receiving events. This is called the subscription FQN. Like all FQNs, a subscription FQN consists of resource type, a namespace and a local name, for example:

<resource-type>::/<name>/<space>::<local-name>

The type of event that is published to a client depends on the resource type specified in the subscription FQN. You can also specify "*" as the resource type to subscribe to events for all resource types.

The following lists valid values for <resource-type> in a subscription FQN, and the type of events published for that resource type:

For example, below are some valid subscription FQNs:

  • job::/testing::myapp – Subscribes to job events for the /testing::myapp job, only.
  • service::/dev – Subscribes to service events for all job resources in the /dev namespace (child namespaces of /dev are not included).
  • *::/testing – Subscribes to events for all resource types in the /testing namespace (child namespaces not included).
  • /apcera – Subscribe to events for all resource types in the /apcera namespace (child namespaces not included).
  • *::/apcera or *::/apcera::* – Equivalent to /apcera (above).

Partial or wildcard namespaces (job::/apcera/*::myapp, for example) are not allowed.

Policy enforcement of event streams

Policy is enforced on outgoing event streams, not on subscription requests. This means that a user (represented by their API token) can successfully subscribe to events for any resource or namespace, but the published event stream they receive only includes events for resources that they have read access to.

For instance, suppose a user with limited cluster permissions uses APC to subscribe to events on a namespace they do not have read permission on:

apc event subscribe /private
Success!

Although the subscription request succeeds no events are streamed to APC because the user making the request doesn't have read permissions on any resources in the "/private" namespace.

In other words, if a job is listed when you run apc <resource-type> list --namespace / against your cluster, then you can stream events for that job. The same applies to all resources (services, providers, and so forth).