All configuration data is stored entirely as structured documents, for which schemas must be registered. Documents satisfy the following use cases:
- layering - helps reduce duplication in configuration while maintaining auditability across many sites
- substitution - provides separation between secret data and other configuration data, while allowing a simple interface for clients
- revision history - improves auditability and enables services to provide functional validation of a well-defined collection of documents that are meant to operate together
- validation - allows services to implement and register different kinds of validations and report errors
The document format is modeled loosely after Kubernetes practices. The top
level of each document is a dictionary with 3 keys:
schema- Defines the name of the JSON schema to be used for validation. Must have the form:
<namespace>/<kind>/<version>, where the meaning of each component is:
namespace- Identifies the owner of this type of document. The values
metadataare reserved for internal use.
kind- Identifies a type of configuration resource in the namespace.
version- Describe the version of this resource, e.g.
metadata- Defines details that Deckhand will inspect and understand. There are multiple schemas for this section as discussed below. All the various types of metadata include a
metadata.namefield which must be unique for each document
data- Data to be validated by the schema described by the
schemafield. Deckhand only interacts with content here as instructed to do so by the
metadatasection. The form of this section is considered to be completely owned by the
At the database level, documents are uniquely identified by the combination of:
This means that raw revision documents – which are persisted in Deckhand’s database – require that the combination of all 3 parameters be unique.
However, post-rendered documents are only uniquely identified by the combination of:
Because collisions with respect to the third parameter –
metadata.layeringDefinition.layer – can only occur with
Document Replacement. But after document rendering, the replacement-parent
documents are never returned.
Below is a fictitious example of a complete document, which illustrates all the
valid fields in the
--- schema: some-service/ResourceType/v1 metadata: schema: metadata/Document/v1 name: unique-name-given-schema storagePolicy: cleartext labels: genesis: enabled master: enabled layeringDefinition: abstract: true layer: region parentSelector: required_key_a: required_label_a required_key_b: required_label_b actions: - method: merge path: .path.to.merge.into.parent - method: delete path: .path.to.delete substitutions: - dest: path: .substitution.target src: schema: another-service/SourceType/v1 name: name-of-source-document path: .source.path data: path: to: merge: into: parent: foo: bar ignored: data: here substitution: target: null
There are 2 supported kinds of document metadata. Documents with
metadata are the most common, and are used for normal configuration data.
Control metadata are used to customize the behavior of
This type of metadata allows the following metadata hierarchy:
name- string, required - Unique within a revision for a given
storagePolicy- string, required - Either
encyrptedis specified, then the
datasection of the document will be stored in a secure backend (likely via OpenStack Barbican).
schemafields are always stored in cleartext. More information on document encryption is available here.
layeringDefinition- dict, required - Specifies layering details. See the Layering section below for details.
abstract- boolean, required - An abstract document is not expected to pass schema validation after layering and substitution are applied. Non-abstract (concrete) documents are.
layer- string, required - References a layer in the
parentSelector- labels, optional - Used to construct document chains for executing merges.
actions- list, optional - A sequence of actions to apply this documents data during the merge process. *
method- string, required - How to layer this content. *
path- string, required - What content in this document to layer onto parent content.
substitutions- list, optional - A sequence of substitutions to apply. See the Substitutions section for additional details.
dest- dict, required - A description of the inserted content destination.
path- string, required - The JSON path where the data will be placed into the
datasection of this document.
pattern- string, optional - A regex to search for in the string specified at
pathin this document and replace with the source data
src- dict, required - A description of the inserted content source.
schema- string, required - The
schemaof the source document.
name- string, required - The
metadata.nameof the source document.
path- string, required - The JSON path from which to extract data in the source document relative to its
This schema is the same as the
Document schema, except it omits the
substitutions keys, as these
actions are not supported on
The complete list of valid
Control document kinds is specified below along
with descriptions of each document kind.
Document abstraction can be compared to an abstract class in programming languages: The idea is to declare an abstract base class used for declaring common data to be overridden and customized by subclasses. In fact, this is the predominant use case for document abstraction: Defining base abstract documents that other concrete (non-abstract) documents can layer with.
An abstract document is a document whose
metadata.abstract property is
True. A concrete document is a document whose
is False. Concrete and non-abstract are terms that are used interchangeably.
In Deckhand, document abstraction has certain implications:
- An abstract document, like all other documents, will be persisted in Deckhand’s database and will be subjected to Revision History.
- However, abstract documents are not returned by Deckhand’s
rendered-documentsendpoint: That is, rendered documents never include abstract documents.
- Concrete documents can layer with abstract documents – and this is encouraged.
- Abstract documents can layer with other documents as well – but unless a concrete document layers with or substitutes from the resultant abstract document, no meaningful data will be returned via rendering, as only concrete documents are returned.
- Likewise, abstract documents can substitute from other documents. The same reasoning as the bullet point above applies.
- However, abstract documents cannot be used as substitution sources. Only concrete documents may be used as substitution sources.