How Apcera Works

This section provides a high-level view of how Acpera components work together to achieve the goals of the system.

The architectural goals that underlie Apcera are as follows:

Enable rapid innovation and agile response to changing conditions and safe, efficient, managed, policy-controlled operations.

At the center of implementing those goals is the idea of a job.

Jobs

A job is a workload that runs, manages, monitors, and enables you to update, report on, and govern. Uploaded code can be anything from a small application to a complex operating system. They are all jobs within the system. Everything that runs in the Apcera is a job.

Apcera ensures that your code has an appropriate execution environment, packages it with that environment so it's ready to run, and deploys it in an isolated container. This standardized form enables Apcera to deploy and orchestrate execution units in a uniform, policy-controlled way. Fine-grained policies, set by you, control a job's communication with other entities - within or outside the system.

Apcera uses a process called staging to ready your code for execution as a job. It examines the code and the metadata that accompanies it to determine which of the available operating systems and runtime environments to package it with. It can also perform other staging tasks that you specify - such as testing or virus scanning. The result of staging is a package that is ready to run. Apcera saves that package and uses it to instantiate the same environment every time.

A job usually needs to collaborate with other jobs and data repositories. Apcera manages these interactions and ensures that you can trust them. The system balances loads, automatically remaps moved jobs, and restarts and reconnects failed jobs. The Apcera policy engine is integral to all runtime components of the platform, so it provides precise control.

Apps

By default, Apcera lets you deploy, run, and manage apps written in a variety of languages. It provides packages for Go, Ruby, Java, Python, PHP, Node.js, and static sites. You can easily support other languages by building new packages. Apcera also supports running Docker images, and provides robust features for Docker, including caching and whitelisting.

An application is a collection of source code written in a supported language, perhaps using a framework and optionally enough description of its dependencies for stagers to produce a component that is ready to be executed.

Mechanisms for declaring dependencies vary across languages: Ruby uses a Gemfile, Python uses requirements.txt files, Node.js uses package.json files, Java uses pom.xml files, and so on. You can also declare dependencies in a manifest.

Staging

Staging prepares code to be runnable as a job. Staging starts when new code, say an application, enters the system. Apcera determines that code's dependencies (usually a specific set of operating systems and language runtimes) and creates a package consisting of:

  • The code.
  • Other packages that provide the dependencies.
  • Metadata that helps the system work with the resulting package.

drawing

In the picture, your configuration files specify the versions of the operating system and language runtime that your code depends on. Your policies allow you (assuming your name is Jon) to use the Ruby package. The stager bundles your code with the specified runtime and OS packages to produce a complete application package that is ready to run.

A stager can modify package contents and metadata. If a package can satisfy other packages' dependencies, its metadata reflects that fact.

An ordered sequence of stagers forms a staging pipeline. You can specify which staging pipeline to use when you deploy an application. If you don't specify a staging pipeline, the system determines one automatically by examining the code and other files accompanying it.

You can write a custom stager in any language to implement any of your lifecycle management processes. Apcera runs stagers as jobs, and a stager can participate in more than one staging pipeline. The final output of staging is a package. Apcera saves the package, so it can use it each time it deploys the underlying application.

Isolating a Job

To turn the package that results from staging into a job, Apcera places it into a lightweight, secure, virtualized Linux container that is isolated from all other system components. Its CPU, memory, and file system are invisible to other components, and by default it is invisible on the network. It can only communicate with endpoints that are allowed and defined by policy.

To be useful, most jobs need outside access, so Apcera enables you to specify exactly how that access should take place. You can add routes to allow ingress to any services that the job provides. You can link to another job or bind the job to a service to allow the job to communicate to the outside. Isolating jobs enables workload security, portability, fault tolerance, load balancing, and other kinds of automation.

Service Gateways

A service is any backend that provides data over an API (for example, HTTP or a database protocol). Examples of services include a database, a queue, a cache, or an account on a third-party SaaS provider. Apcera provides built-in support for connecting applications to services like MySQL and Postgres.

Service gateways are jobs that manage services and configure connections between services and other jobs in the system. Every service type has an associated service gateway. Apcera includes several built-in service gateways including MySQL and Postgres. Working with Services provides more details about services.

You can write service gateways to add new types of services. Writing and deploying a service gateway is similar to writing a web application and binding it to a persistent service. Service Gateway API provides details on how to write and deploy a service gateway.

Semantic Pipelines

A semantic pipeline is a job that enables Apcera to monitor, inject policy, and control behavior within a communication channel in real time. Semantic pipelines speak wire protocols – they accept client connections, authenticate the clients locally, and establish connections to real servers using separate credentials. The client cannot communicate directly with the server using the credentials it receives from the semantic pipeline, so the effect of accidental exposure of those credentials is minimal.

drawing

In the picture, the job requests a service of type t (for example, postgres). The service gateway obtains a URI for the job to use and tells Apcera which protocol the service uses. Apcera finds a semantic pipeline for that type of protocol and places it between the job and the service.

Policy

Policy is built into every component of the system; it can be injected and enforced anywhere.

Every resource (job, service, binding, or package) has a unique identifier called a fully qualified name (FQN). An FQN consists of a resource type, a namespace, and a local name. A policy is composed of rules attached to specified FQNs. Working with Policies provides more details.