Writing Package Scripts

This section describes how to author package scripts for creating custom packages. To learn how packages are used in Apcera, see working with packages.

Example package script

Below is a package script named node-5.12.0.conf. This script file creates a package that provides the Node.js v5.12.0 runtime.

Check out the Apcera Package Scripts repository on GitHub for additional package scripts.

name:      "node-5.12.0"
namespace: "/apcera/pkg/runtimes"

sources [
  { url: "https://s3.amazonaws.com/apcera-sources/node/node-v5.12.0-linux-x64.tar.gz",
    sha256: "c0f459152aa87aba8a019a95899352170db0d8d52c860715c88356cb253fe2c4" },
]

build_depends [ { package: "build-essential" } ]
depends       [ { os: "ubuntu" } ]
provides      [ { runtime: "node" },
                { runtime: "node-5" },
                { runtime: "node-5.11" },
                { runtime: "node-5.12.0" } ]

environment { "PATH": "/opt/apcera/node-5.12.0/bin:$PATH" }

cleanup [ "/usr/lib/python2.7" ] # this will come from it using python
                                 # it will generate *.pyc files

build (
      export INSTALLPATH=/opt/apcera/node-5.12.0
      mkdir -p $INSTALLPATH

      sudo ln -f /usr/bin/python2.7 /usr/bin/python

      tar xzvf node-v5.12.0-linux-x64.tar.gz
      mv node-v5.12.0-linux-x64/* $INSTALLPATH

      sudo mkdir -p ${INSTALLPATH}/etc
      echo 'cache = /app/.npm' > npmrc
      sudo chown root:root npmrc
      sudo mv npmrc ${INSTALLPATH}/etc/npmrc

      sudo rm -f /usr/bin/python
)

Package script elements

The following table lists and describes the top-level elements of an Apcera package configuration file. See also the examples.

Attribute Usage Description Data Type Default
name Required The package's local name. If not specified, defaults to name of package script file without suffix. String None
namespace Optional The package's namespace. This value is ignored if you specify a namespace with the --namespace (or -ns) command-line parameter. Otherwise, the namespace is set to APC's current default namespace. String None
version Optional The package version. Can be any string. String None
sources Optional List of URLs of remote files to be downloaded and verified. Each URL in the array is a set of attribute-value pairs, where the attributes are url, name, sha256, sha1, and md5. If used the url attribute is required, the others are optional. The URL must resolve to a publicly available file. It can include a username and password for basic authentication. Attributes sha256, sha1, and md5 denote file verification methods, and the associated value is the corresponding key. The build fails if the verification fails. When used within the sources array, the name is the name to use for the file in the build script. The default name is the final path element of the URL. See sources example below. Array None
include_files Optional List of file paths that should exist on your local system. When APC runs a build, it uploads those files to the Apcera cluster and makes them available to the build script. The uploaded files have the same names as on the local system. APC generates the MD5, SHA1, and SHA256 locally and validates all of those hashes on the remote system after uploading them. Each file is capped at 128MB in size (see examples below). Array none
build_depends Optional List of package dependencies used only during the building process. These dependencies are automatically removed from the package after the build script finishes successfully. As with depends, Apcera uses dependency resolution for these dependencies.    
depends Required List of package dependencies that are available before executing the build script. These dependencies remain in the package and are required for any job that needs to use the package. If there are ambiguities or policy conflicts with the specified dependencies, Apcera uses dependency resolution to resolve them. Array None
provides Optional Each provides creates a provides on the package. Apcera dependency resolution checks these. Array None
environment Optional Lets you specify environment variables that should be set on the package after the build script has completed. The block is a set of colon-separated key value pairs: environment variable names and their values. Environment variables can come from multiple packages and from the running job. If you set environment variables that might also be set on other packages, have them expand on each other. For example, if you want to add a path to PATH, set it as /my/path:$PATH so that your part is added to the front of the existing PATH. Array None
cleanup Optional List of paths to be removed after the build script is finished, before it takes the final snapshot of the filesystem. Array None
snapshot_path Optional Directory of the image to snapshot. Typically a snapshot is taken of a currently running job. However, with a package script you want the snapshot to be for the image that is being created (not the snapshot of the stager that created the image). Array None
build Required The build element contains Bash commands to turn the package source files into the final package. Note that within the build script, any text between a # and the end of the line is a comment. See Build script example. Array None

Package script element examples

This section provides additional examples for key package script elements.

sources example

The sources attribute is an array of URLs of remote files to be downloaded and verified. Each URL in the array is a set of attribute-value pairs, where the attributes are url (required), name, sha256, sha1, and md5. When used within the sources array, name is the name to use for the file in the build script. The default name is the final path element of the URL.

sources [
  { url: "https://apcera-sources.s3.amazonaws.com/ruby/ruby-1.9.3-p448.tar.gz",
  sha256: "2f35e186543a03bec5e603296d6d8828b94ca58bab049b67b1ceb61d381bc8a7" },
  { url: "https://apcera-sources.s3.amazonaws.com/ruby/customyaml-0.1.0.tar.gz",
  name: "yaml",
  sha256: "7bf81554ae5ab2d9b6977da398ea789722e0db75b86bffdaeb4e66d961de6a37" },
  { url: "https://apcera-sources.s3.amazonaws.com/ruby/rubygems-2.1.7.tgz",
  sha256: "76288eb4238378189d1ef6b27a2f94241d3d21ba96ce5c9a6b1534a33ec1609f" },
  { url: "https://apcera-sources.s3.amazonaws.com/ruby/bundler-1.3.5.gem",
  sha256: "08b89047f7b829f3e197a28fb1bde74c3f5cfea1552f99dfba237fee30eaffe4" },
]

include_files example

When you build a package using APC, these files are uploaded to the Apcera cluster and made available to the build script. The uploaded files have the same names as on the local system. APC generates the MD5, SHA1, and SHA256 locally and validates all of those hashes on the remote system after uploading them.

include_files [
  "nginx-1.4.3.tar.gz",
  "patches/request_start_variable.patch",
  "patches/unused-variable.patch"
]

The include_files size is capped at 128MB per file. It is not designed for processing large files into packages. Although you can work around this limitation to some extent by increasing the memory size allocated to the Compiler Stager when you create a job or package, for large packages the recommended approach is to use apc package from file or the sources.url parameter in the package configuration file. See creating large packages for details.

build_depends example

The build_depends block contains a list of package dependencies used only during the building process. These dependencies are automatically removed from the package after the build script finishes successfully. As with depends, Apcera uses dependency resolution for these dependencies.

build_depends [ { package: "build-essential" } ]

depends example

The depends block (required) contains a list of package dependencies that are available before executing the build script. They remain in the package and are required for any job that wishes to use the package. If there are ambiguities or policy conflicts with the specified dependencies, Apcera uses dependency resolution to resolve them.

depends       [ { os: "linux" },
              { package: "apache-2.2" } ]

provides example

Each provides in the block creates a provides on the package. Apcera dependency resolution checks each provides.

provides      [ { runtime: "python" },
              { runtime: "python-3.3" },
              { runtime: "python-3.3.2" } ]

environment example

The environment block enables you specify environment variables that should be set on the package after the build script has completed. The block is a set of colon-separated key value pairs: environment variable names and their values.

Environment variables can come from multiple packages and from the running job. If you set environment variables that might also be set on other packages, have them expand on each other. For example, if you want to add a path to PATH, set it as /my/path:$PATH so that your part is added to the front of the existing PATH.

environment
  { "PATH": "/opt/apcera/apache-2.2.25/bin:$PATH",
    "APACHE_ROOT": "/opt/apcera/apache-2.2.25",
    "APACHE_MODULES_PATH": "/opt/apcera/apache-2.2.25/modules" }

cleanup example

The cleanup section is a list of paths that should be removed after the build script is finished, before it takes the final snapshot of the filesystem.

cleanup [
  "/var/lib/dpkg",
  "/etc"
]

snapshot_path example

The snapshot_path block specifies the directory of the image to take the snapshot of. Normally, a snapshot is taken of a currently running job. However, in case of a package script, you want the snapshot to be for the image that is being created (not the snapshot of the stager that created the image).

snapshot_path "/rootfs"

For a more complete example, see the Ubuntu 14.04.3 package script.

build script example

The build (required) element holds the build script that generates the package. The build script is a bash script that takes the source files and turns them into the final package.

Within the build script, any text between a # and the end of the line is a comment.

The working directory at the start of the script is a temporary Apcera directory created under /tmp, and containing the files specified in the sources block and the include_files list.

The build script is executed as a non-root user, but it does have access to passwordless sudo for gaining root access. The temporary directory that is created and the files in it are owned by the non-root user as well.

The build script is expected to take the sources to a final result. If the script exits with non-zero status, Apcera assumes the build failed. Otherwise it succeeded.

The build script can alter any parts of the isolated filesystem in Apcera. At the end of the build, the system cleans up the paths specified by the cleanup list and removes anything in /tmp. It then uses Apcera's snapshot functionality to capture the files that changed while the build script ran.

The build block can contain any Bash statements without extra escape characters. The only requirement is that a closing parenthesis is the first and only character of the final line of the block. You can format arbitrarily within the the block, but it's a good idea to indent the build script to prevent the closing parenthesis of an internal structure from accidentally terminating the block.

The following example illustrates this format:

build (
      export BUILDPATH=`pwd`
      export INSTALLPATH=/opt/apcera/ruby-1.9.3-p448

      tar xzvf yaml-0.1.4.tar.gz
      (
          cd yaml-0.1.4
          ./configure --disable-shared --with-pic
          make
      )

      tar xzvf ruby-1.9.3-p448.tar.gz
      (
        cd ruby-1.9.3-p448
        export LDFLAGS="-L${BUILDPATH}/yaml-0.1.4/src/.libs $LDFLAGS"
        export CPPFLAGS="-I${BUILDPATH}/yaml-0.1.4/include $CPPFLAGS"
        ./configure --prefix=${INSTALLPATH} --enable-shared --disable-install-doc
        make
        sudo make install
      )

      tar zxvf rubygems-2.1.7.tgz
      (
        cd rubygems-2.1.7
        sudo ${INSTALLPATH}/bin/ruby setup.rb
      )

      sudo ${INSTALLPATH}/bin/gem install bundler-1.3.5.gem --no-ri --no-rdoc
)