Spec Core Functions
v2023-04-20
This document contains the definitions of the core ${..}
substitution utility functions that should be supported by all implementations of the spec.
fromjson
The fromjson
function is used to extract values from a serialised json string.
Implementations should use json pointer notation to allow for the extraction of values from complex serialised structures.
This only works for extracting values when the root of the json string is an object.
Parameters:
string
- A valid string literal, reference or function call yielding the json string to extract values from.string
- A valid json pointer expression to extract the value from the json string.
Returns:
any
The value extracted from the json string. This could be a primitive value, an array, or a mapping.
Examples:
With a reference:
${fromjson(variables.cacheClusterConfig, "/host")}
With a function call:
${fromjson(trim(variables.cacheClusterConfig), "/host")}
With a string literal:
${fromjson("{\"host\":\"localhost\"}", "/host")}
fromjson_g
The fromjson_g
is a composable version of the fromjson
function that takes a json pointer expression as a static argument and returns a function that takes the json string to extract values from.
Like with fromjson
, implementations should use json pointer notation to allow for the extraction of values from complex serialised structures.
Parameters:
string
- A valid json pointer expression to extract the value from the json string.
Returns:
func (string) -> any
A function that takes a json string and extracts a value from it using the pre-configured json pointer expression.
Examples:
${map(values.cacheClusterConfigDefs, fromjson_g("/host"))}
jsondecode
The jsondecode
function is used to decode a serialised json string into a primitive value, array or mapping.
Parameters:
string
- A valid string literal, reference or function call yielding the json string to decode.
Returns:
any
The decoded json string. This could be a primitive value, an array, or a mapping.
Example:
${jsondecode(values.cacheClusterConfig)}
len
The len
function is used to get the length of a string, array, or mapping.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string, array, or mapping to get the length of.
Returns:
integer
The length of the string, array, or mapping. For a string, the length is the number of characters. For a mapping, the length is the number of key value pairs.
Example:
${len(values.cacheClusterConfig.endpoints)}
substr
The substr
function is used to get a substring from a string.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to get the substring from.integer
- The index of the first character to include in the substring.integer
(optional) - The index of the last character to include in the substring. If not provided the substring will include all characters from the start index to the end of the string.
Returns:
string
The substring from the string.
Example:
${substr(values.cacheClusterConfig.host, 0, 3)}
substr_g
substr_g
is a composable version of the substr
function that takes the start and end indexes as static arguments and returns a function that takes the string to get the substring from.
Parameters:
integer
- The index of the first character to include in the substring.integer
(optional) - The index of the last character to include in the substring. If not provided the substring will include all characters from the start index to the end of the string.
Returns:
func (string) -> string
A function that takes a string and returns the substring from the string using the pre-configured start and end positions.
Example:
${map(values.cacheClusterConfig.hosts, substr_g(0, 3))}
replace
The replace
function is used to replace all occurrences of a substring in a string with another substring.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing an input string that contains a substring that needs replacing.string
- The "search" substring to replace.string
- The substring to replace the "search" substring with.
Returns:
string
The input string with all occurrences of the "search" substring replaced with the "replace" substring.
Example:
${replace(values.cacheClusterConfig.host, "http://", "https://")}
replace_g
replace_g
is a composable version of the replace
function that takes the search and replace substrings as static arguments and returns a function that takes the string to replace the substrings in.
Parameters:
string
- The "search" substring to replace.string
- The substring to replace the "search" substring with.
Returns:
func (string) -> string
The input string with all occurrences of the "search" substring replaced with the "replace" substring.
Example:
${map(values.cacheClusterConfig.hosts, replace_g("http://", "https://"))}
trim
The trim
function is used to remove all leading and trailing whitespace from a string.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to trim.
Returns:
The input string with all leading and trailing whitespace removed.
Example:
${trim(values.cacheClusterConfig.host)}
trimprefix
The trimprefix
function is used to remove a prefix from a string.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to remove the prefix from.string
- The prefix to remove from the string.
Returns:
The input string with the prefix removed.
Example:
${trimprefix(values.cacheClusterConfig.host, "http://")}
trimprefix_g
trimprefix_g
is a composable version of the trimprefix
function that takes the prefix as a static argument and returns a function that takes the string to remove the prefix from.
Parameters:
string
- The prefix to remove from the string.
Returns:
func (string) -> string
A function that takes a string and returns the input string with the prefix removed.
Example:
${map(variables,cacheClusterConfig.hosts, trimprefix_g("http://"))}
trimsuffix
The trimsuffix
function is used to remove a suffix from a string.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to remove the suffix from.string
- The suffix to remove from the string.
Returns:
The input string with the suffix removed.
Example:
${trimsuffix(values.cacheClusterConfig.host, ":3000")}
trimsuffix_g
trimsuffix_g
is a composable version of the trimsuffix
function that takes the suffix as a static argument and returns a function that takes the string to remove the suffix from.
Parameters:
string
- The suffix to remove from the string.
Returns:
func (string) -> string
A function that takes a string and returns the input string with the suffix removed.
Example:
${map(values.cacheClusterConfig.hosts, trimsuffix_g("/config"))}
split
The split
function is used to split a string into an array of substrings based on a delimiter.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to split.string
- The delimiter to split the string by.
Returns:
array
An array of substrings that have been split by the delimiter.
Example:
${split(values.cacheClusterConfig.hosts, ",")}
split_g
split_g
is a function that takes the delimiter as a static argument and returns a function that takes the string to split.
Parameters:
string
- The delimiter to split the string by.
Returns:
func (string) -> array
A function that takes a string and returns an array of substrings that have been split by the delimiter.
Example:
${flatmap(values.cacheClusterConfig.multiClusterHosts, split_g(","))}
join
The join
function is used to join an array of strings into a single string using a delimiter.
Parameters:
array
- A reference or function call yielding a return value representing an array of strings to join together.string
- The delimiter to join the strings with.
Returns:
string
A single string that is the result of joining the array of strings with the delimiter.
Example:
${join(values.cacheClusterConfig.hosts, ",")}
index
The index
function is used to get the first index of a substring in a given string.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to search for the substring in.string
- The substring to search for in the string.
Returns:
integer
The index of the first occurrence of the substring in the string. This will be -1 if the substring is not found in the string.
Example:
${index(values.cacheClusterConfig.host, ":3000")}
last_index
The last_index
function is used to get the last index of a substring in a given string.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to search for the substring in.string
- The substring to search for in the string.
Returns:
integer
The index of the last occurrence of the substring in the string. This will be -1 if the substring is not found in the string.
Example:
${last_index(values.cacheClusterConfig.host, ":3000")}
to_upper
The to_upper
function converts all characters of a string to upper case.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to convert to upper case.
Returns:
string
The input string with all characters converted to upper case.
Example:
${to_upper(values.cacheClusterConfig.hostName)}
to_lower
The to_lower
function converts all characters of a string to lower case.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to convert to lower case.
Returns:
string
The input string with all characters converted to lower case.
Example:
${to_lower(values.cacheClusterConfig.hostId)}
has_prefix
The has_prefix
function is used to check if a string starts with a given substring.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to check.string
- The prefix to check for at the start of the string.
Returns:
bool
True, if the string starts with the prefix, false otherwise.
Example:
${has_prefix(values.cacheClusterConfig.host, "http://")}
has_prefix_g
has_prefix_g
is a composable version of the has_prefix
function that takes the prefix as a static argument and returns a function that takes the string to check.
Parameters:
string
- The prefix to check for at the start of the string.
Returns:
func (string) -> bool
A function that takes a string and returns true if the string starts with the prefix, false otherwise.
Example:
${filter(
values.cacheClusterConfig.hosts,
has_prefix_g("http://")
)}
has_suffix
The has_suffix
function is used to check if a string ends with a given substring.
Parameters:
string
- A valid string literal, reference or function call yielding a return value representing the string to check.string
- The suffix to check for at the end of the string.
Returns:
bool
True, if the string ends with the suffix, false otherwise.
Example:
${has_suffix(values.cacheClusterConfig.host, "/config")}
has_suffix_g
has_suffix_g
is a composable version of the has_suffix
function that takes the suffix as a static argument and returns a function that takes the string to check.
Parameters:
string
- The suffix to check for at the end of the string.
Returns:
func (string) -> bool
A function that takes a string and returns true if the string ends with the suffix, false otherwise.
Example:
${filter(
values.cacheClusterConfig.hosts,
has_suffix_g("/config")
)}
contains
The contains
function is used to check if a string contains a given substring or if an array contains a given value.
Parameters:
string | array
- A valid string literal, reference or function call yielding a return value representing a string or array to search.string | ArrayElem
- The substring or value to search for in the string or array.
Returns:
bool
True, if the substring or value is found in the string or array, false otherwise.
Example:
${contains(values.cacheClusterConfig.host, "celerityframework.com")}
contains_g
contains_g
is a composable version of the contains
function that takes the substring or value as a static argument and returns a function that takes the string or array to check.
Parameters:
string | ArrayElem
- The substring or value to search for in the string or array.
Returns:
func (string | array) -> bool
A function that takes a string or array and returns true if the substring or value is found in the string or array, false otherwise.
Example:
${filter(
values.cacheClusterConfig.hosts,
contains_g("celerityframework.com")
)}
list
Creates a list of values from arguments of the same type.
Parameters:
N arguments of the same type that will be used to create a list.
Returns:
array
An array of values that have been passed as arguments.
Example:
${list(
"item1",
"item2",
"item3",
"item4"
)}
object
Creates an object from named arguments.
Parameters:
N named arguments that will be used to create an object/mapping. When no arguments are passed, an empty object should be returned.
Named function arguments are a special case that are only used in the object
function.
It is not syntactically invalid to use named arguments in other functions but the names will be
ignored and the values passed in will be treated as positional arguments.
Returns:
object
An object containing attributes that have been passed as named arguments.
Example:
${object(
id = "subnet-1234",
label = "Subnet 1234"
)}
keys
Produces an array of keys from a mapping or attribute names from an object.
Parameters:
mapping
- A mapping to extract keys from. (This can also be an object with known attributes, the two are interchangeable up to the point of validating parameters and return values)
Returns:
array
An array of keys or attributes from the mapping or object.
Example:
${keys(datasources.network.subnets)}
vals
Produces an array of values from a mapping or object with known attributes.
This function is named vals
to avoid conflicting with the values
keyword used for blueprint values.
Parameters:
mapping
- A mapping or object to extract values from.
Returns:
array
An array of values extracted from the provided mapping or object.
Example:
${vals(datasources.network.subnets)}
map
Maps a list of values to a new list of values using a function.
Parameters:
array
- An array of items where all items are of the same type to map.func<Item, NewItem>(Item, integer?) -> NewItem
- A function that will be applied to each item in the array, the function must match the type of the items in the array. A runtime error should be returned if the function does not match this type. This function can optionally take an index as a second argument.
Returns:
array
An array of values that have been transformed by the function.
Example:
${map(
datasources.network.subnets,
compose(to_upper, getattr("id"))
)}
filter
Filters a list of values based on a predicate function.
Parameters:
array
- An array of items where all items are of the same type to filter.func<Item>(Item) -> bool
- A function that will be applied to each item in the array, the function must return a boolean value.
Returns:
array
An array of filtered values based on the predicate function.
Example:
${filter(
datasources.network.subnets,
hasprefix_g("subnet-402948-")
)}
reduce
Reduces a list of values to a single value using a function.
There are no core functions that have the signature of a reducer function,
the reduce
function is in the core spec to provide the tools for implementations and end users
to be able to apply more complex transformations of data in blueprints.
Parameters:
array
- An array of items where all items are of the same type to reduce over.func<Item, Accum>(Accum, Item, integer?) -> Accum
- A function that will be applied to each item in the array, accumulating a single value. This function can optionally take an index as a third argument.Accum
- The initial value to start the accumulation with.
Returns:
any
The accumulated value as a result of applying the reducer function to each item in the array.
Example:
${reduce(
datasources.network.subnets,
extract_crucial_network_info,
object()
)}
sort
Sorts a list of values based on a comparison function.
There are no core functions that have the signature of a comparison function,
the sort
function is in the core spec to provide the tools for implementations and end users
to sort arrays of values based on a custom comparison function.
Parameters:
array
- An array of items where all items are of the same type to sort.func<Item>(Item, Item) -> integer
- The comparison function that is expected to return a negative integer if the first item should come before the second item, a positive integer if the first item should come after the second item, and zero if the items are equal.
Returns:
array
An array of sorted values based on the comparison function.
Example:
${sort(
datasources.network.subnets,
compare_cidr_ranges,
)}
flatmap
Maps a list of values to a new list of values using a function and flattens the result.
One example where this is useful would be where you have a list of strings where each string is a comma separated list of values and you want to split each string into a list of values and then flatten the result into a single output list.
To illustrate this, for input:
values.hosts = ["host1,example.com:3049", "host2,example.com:4095"]
Applying the flat map function like so:
${flatmap(
values.hosts,
split_g(",")
)}
Would result in the following output:
["host1", "example.com:3049", "host2", "example.com:4095"]
Parameters:
array
- An array of items where all items are of the same type to map.func<Item, NewItem extends array>(Item, integer?) -> NewItem
- A function that will be applied to each item in the array, the function must match the type of the items in the array. A runtime error should be returned if the function does not match this type. This function can optionally take an index as a second argument.
Returns:
array
An array of values that have been transformed by the function and flattened into a one-dimensional array.
Example:
${flatmap(
values.cacheClusterConfig.hosts,
split_g(",")
)}
compose
compose
is a higher-order function that combines N functions into a single function,
where the output of one function is passed in as the input to the previous function.
The call order of the functions is from right to left.
Parameters:
N functions that will be composed together from right to left. The return type of each function must match the input type of the function to the left.
Returns:
func (any) -> any
A function that takes the input value of the right-most function and returns the output value of the left-most function.
Example:
${map(
datasources.network.subnets,
compose(to_upper, getattr("id"))
)}
pipe
pipe
is a higher-order function that combines N functions into a single function,
where the output of one function is passed in as the input to the next function.
The call order of the functions is from left to right, which is generally seen as more intuitive than the right to left order of compose
.
Parameters:
N functions that will be piped together from left to right. The return type of each function must match the input type of the function to the right.
Returns:
func (any) -> any
A function that takes the input value of the left-most function and returns the output value of the right-most function.
Example:
${map(
datasources.network.subnets,
pipe(getattr("id"), to_upper)
)}
getattr
A higher-order function that returns a function that extracts a named attribute from an object or a mapping. This is useful in situations where you want to map an array of objects to an array of values of a specific attribute such as IDs.
Parameters:
string
- The name of the attribute to extract from the object or mapping.
Returns:
func (object | mapping) -> any
A function that takes an object or mapping and returns the value of the named attribute.
Example:
${map(
datasources.network.subnets,
compose(getattr("id"), getattr("definition"))
)}
This example would take a list of subnets that would be of the following form:
[{
"definition": {
"id": "subnet-1234"
}
}, {
"definition": {
"id": "subnet-5678"
}
}]
And return a list of ids:
["subnet-1234", "subnet-5678"]
getelem
A higher-order function that returns a function that extracts an element from an array. This is useful in situations where you want to map a two-dimensional array to an array of values of a specific element.
Parameters:
integer
- The position of the element to extract from the array.
Returns:
func (array) -> any
A function that takes an array and returns the value at the provided position.
Example:
${map(
datasources.network.subnets,
compose(get_attr("id"), getelem(0))
)}
This example would take a list of subnets that would be of the following form:
[[
{
"id": "subnet-1234",
"label": "Subnet 1234"
},
"10.0.0.0/16"
], [
{
"id": "subnet-5678",
"label": "Subnet 5678"
},
"172.31.0.0/16"
]]
And return a list of ids:
["subnet-1234", "subnet-5678"]
link
A function to retrieve the state of a link between two resources.
Parameters:
string | ResourceRef
- Resource A in the relationship, can be a string literal of the resource name or a reference to a resource.string | ResourceRef
- Resource B in the relationship, can be a string literal of the resource name or a reference to a resource.
Returns:
object
An object containing all the information about the link between the two resources made available by the provider that powers the link.
Example:
Using resource references:
${link(resources.orderApi, resources.createOrderFunction)}
Using implict resource references (identifiers without a namespace are either resources or functions):
${link(orderApi, listOrdersFunction)}
Using string literals:
${link("orderApi", "deleteOrderFunction")}
and
A function that acts as a logical AND operator on two boolean values.
Parameters:
boolean
- The result of boolean expression A, the left-hand side of the AND operation.boolean
- The result of boolean expression B, the right-hand side of the AND operation.
Returns:
boolean
The result of the logical AND operation on the two boolean values.
Example:
${and(resources.orderApi.spec.isProd, eq(variables.environment, "prod"))}
or
A function that acts as a logical OR operator on two boolean values.
Parameters:
boolean
- The result of boolean expression A the left-hand side of the OR operation.boolean
- The result of boolean expression B the right-hand side of the OR operation.
Returns:
boolean
The result of the logical OR operation on the two boolean values.
Example:
${or(resources.orderApi.spec.isDev, eq(variables.environment, "dev"))}
not
A function that negates a given boolean value.
Parameters:
boolean
- The result of a boolean expression to negate.
Returns:
boolean
The result of negating the provided boolean value.
Example:
${not(eq(variables.environment, "prod"))}
eq
A function that determines whether two values of the same type are equal.
Parameters:
any
- The left-hand side of the equality comparison.any
- The right-hand side of the equality comparison.
Returns:
boolean
True, if the two values are equal, false otherwise.
Example:
${eq(variables.environment, "prod")}
gt
A function that determines whether a number is greater than another number.
Parameters:
integer | float
- "a" in the expression "a > b".integer | float
- "b" in the expression "a > b".
Returns:
boolean
True, if the first number is greater than the second number, false otherwise.
Example:
${gt(len(datasources.network.subnets), 10)}
ge
A function that determines whether a number is greater than or equal to another number.
Parameters:
integer | float
- "a" in the expressiona >= b
.integer | float
- "b" in the expressiona >= b
.
Returns:
boolean
True, if the first number is greater than or equal to the second number, false otherwise.
Example:
${ge(len(datasources.network.subnets), 10)}
lt
A function that determines whether a number is less than another number.
Parameters:
integer | float
- "a" in the expressiona < b
.integer | float
- "b" in the expressiona < b
.
Returns:
boolean
True, if the first number is less than the second number, false otherwise.
Example:
${lt(len(datasources.network.subnets), 10)}
le
A function that determines whether a number is less than or equal to another number.
Parameters:
integer | float
- "a" in the expressiona <= b
.integer | float
- "b" in the expressiona <= b
.
Returns:
boolean
True, if the first number is less than or equal to the second number, false otherwise.
Example:
${le(len(datasources.network.subnets), 10)}
cwd
A function that returns the user's current working directory as set on the host system, by a tool implementing the spec or by the user.
Parameters:
This function does not take any parameters.
Returns:
string
The current working directory of the user.
Example:
${cwd()}/blueprints/core-infra.blueprint.yaml
datetime
A function that returns the current date/time as per the host system's clock in the specified format. All times are normalised to UTC.
The datetime
function supports the following date/time formats:
unix
- The number of seconds since the Unix epoch. (e.g. 1611312000)rfc3339
- The date/time in RFC3339 format. (e.g. 2023-01-02T15:04:05Z07:00)tag
- The date/time in a format suitable for use as a tag. (e.g. 2023-01-02--15-04-05)tagcompact
- The date/time in a compact format suitable for use as a tag. (e.g. 20230102150405)
rfc3339
is a format derived from ISO 8601.
This function is useful for generating timestamps that can be used for tagging and versioning resources. (e.g. Docker image tags, S3 object keys, etc.)
Parameters:
string
- The date/time format to return the current date/time in. The format should be one of the valid formats defined above.
Returns:
string
A string representing the current time in the requested format.
Example:
${datetime("tag")}
Clarification on _g
functions
A lot of the core functions have two definitions in the spec, one for direct function calls and one for function composition. The _g
1 suffix is used to denote a function to be used for composition that takes static arguments and produces a function that takes the dynamic value to be transformed.
A direct usage example would be:
${trimprefix(values.cacheClusterConfig.host, "http://")}
A composition usage example would be:
${map(values.cacheClusterConfig.hosts, trimprefix_g("http://"))}
Only some of the core string manipulation functions have a composable version, this is because they are the most likely to be used in a composition context. Implementations are free to add composable versions of other functions as they see fit. A plugin system for functions could also be useful in allowing users of a tool implementing the spec to add their own functions.
⚠️ Function usage caveats
Higher-order functions
Functions that are returned when calling named functions, such as the result of getattr("id")
, can not be called directly.
For example, getattr("id")(resource.spec)
is not a valid expression. In this example, you would instead use resource.spec.id
to access the id attribute of the spec object.
Higher-order functions in the blueprint substitution language can only be used as arguments to other functions that accept functions as parameters. A valid usage of the getattr
function would be as follows:
${map(datasources.network.subnets, getattr("id"))}
This example would map over a list of subnets and extract the id
attribute from each subnet object, producing a list of IDs.
Footnotes
-
The reasoning for the
_g
suffix for these functions is thatg
is commonly used in mathematics to denote a function in composition. ↩