Configuring Job Manager

This document describes the Job Manager Configuration parameters and their default values across various releases.

Job Manager Configuration

Job Manager is the system of record for the desired state of the cluster. Desired state is persisted in the consul kv-store. JM records are persisted as a external representation (JSON) in the kv-store. The cache minimizes processing overhead (unmarshalling) of record retrieval which improves response time. The cache_size configures the maximum number of records to maintain in-memory. A cache_size is defined for each Schema. Any non-positive value disables the cache. When the number of cache records is exceeded a random record is evicted.

Prior to 3.2.2, the number of records to cache for each JM keyspace was fixed.

As of 3.2.2:

  1. default_cache_size increased from 100 to 300
  2. mFQN is independently configurable, with a default of 2000
  3. cache sizes are configurable for each cache config
Cache Config 3.2.1 defaults 3.2.2 defaults Job Manager schema
job_cache_size 1000 1000 job
mapped_fqn_cache_size default_cache_size 2000 mappedFQN - Providers/Services/Bindings
allocator_cache_size 1 1 simpleIPV4Allocator, simpleIPV4AllocatorV1
default_cache_size 100 300 spEventRule, stagingPipeline, network, route, subnetpool, task, secret, certificateDomain

Configuration value recommendation

A Schema does not share cache at runtime, only the configuration value is shared. Increasing the cache size can prevent cache churn. The trade off is that this will increase memory pressure on the Job Manager, which is generally co-located with consul and all of the other "central" services. "Centrals" are under additional memory pressure when Package Manager is co-located with the JM and consul. PM utilizes the Linux VFS buffer cache to service package get/put operations.

When increasing these values, monitor system logs for Linux OOM killer activity on "centrals" as well as JM/Consul restarts/stability.