The need
I went into some troubles when I wanted to implement NSXT rules. My aim was to keep the order of the rules as intended by the user when he wrote his data without asking him to enter a rule ID manually. If the order is kept then it’s easy to prioritize the rules according to their placement. With the NSX-T Terraform provider the rules are in the form below :
resource "nsxt_policy_security_policy" "policy1" {
display_name = "policy1"
rule {
...
}
rule {
....
}
}
You can see that it’s made of rule blocks. These blocks will be generated dynamically. We need a way to always have the same order while generating the rule blocks. If we don’t keep the order, Terraform complains that the rule has changed. Every time we will add a rule they will be scrambled and Terraform will modify the order.
Why does this happen ?
The data model I use to represent the rules is a list of map. Every map is a rule. We can sum up the data structure as below :
listOfRule = [
{
display_name = "ruleA"
source_groups = "10.0.0.1"
destination_groups = "10.0.0.2"
},
{
display_name = "ruleD"
source_groups = "10.1.0.1"
destination_groups = "10.1.0.2"
}
]
To understand the behavior, we will simplify this data structure to an object with 2 attributes. id which is an integer and name which is a string. In this example, the ids are not sorted neither the names.
variable "listOfRule" {
default = [
{
id = 3
name = "ruleA"
},
{
id = 2
name = "ruleD"
},
{
id = 10
name = "ruleB"
},
{
id = 1
name = "ruleC"
}
]
}
for_each only accept a set or a map. When you have a list and you want to iterate it over a for_each, you need to either transform it into a set with the toset() function or create a map with a key thanks to the for loop.
{ for key, value in var.myvar : key => value }
- When transforming this list of object into a set, the order is not kept and is difficult to predict.
- When transforming this list of object into a map, the map will be sorted according to the key which MUST be a string. The ordering is then done alphabetically.
To illustrate the ordering we will do different examples. You can copy the original variable in a main.tf file then play with it thanks to the “terraform console” command.
The examples
Transform list into set
With this transformation we see that the order is not easy to predict. In that case, it’s sorted numerically but with the longest id first.
| Original | Sorted |
|
|
Transform list into map with name as index
Here this is sorted alphabetically.
| Original | Sorted |
|
|
Transform list into a map with id as index
Here the numerical index is transformed into a string and then it’s sorted alphabetically. 10 is before 2 because 1 is bigger than 2.
| Original | Sorted |
|
|
Transform list into a map using the list key as index.
With a list, there is an implicit numerical index starting from 0. We can try to use this index as a key for the map.
| Original | Sorted |
|
|
That looks good until we start to have more than 10 values. for_each accept only string as index. When we use the numerical list index as key for the map it’s automatically transformed into a string. If we have more than 10 values, the index 10 is then put before 2. The reason is because when it’s sorted alphabetically, the first character is compared first and this is independent of the length of the string.
| Original | Sorted |
|
|
Transform list into a map with numerical index as a string
As for_each accept only string as key we need to find a way to emulate the numerical ordering with strings. To do that we need to convert the integer into strings of the same length. This way, we add 0 in front of shorter integer, as 0 is lower than 1 we prevent the previous behavior and we have the intended result. In my case I used the format function and chose a 4 characters length.
| Original | Sorted |
|
> { for key,rule in var.listOfRule : format("%.4d",key) => rule }
{
"0000" = {
"id" = 3
"name" = "ruleA"
}
"0001" = {
"id" = 2
"name" = "ruleD"
}
"0002" = {
"id" = 10
"name" = "ruleB"
}
"0003" = {
"id" = 1
"name" = "ruleC"
}
"0004" = {
"id" = 4
"name" = "ruleE"
}
"0005" = {
"id" = 5
"name" = "ruleF"
}
"0006" = {
"id" = 6
"name" = "ruleG"
}
"0007" = {
"id" = 7
"name" = "ruleH"
}
"0008" = {
"id" = 8
"name" = "ruleI"
}
"0009" = {
"id" = 9
"name" = "ruleJ"
}
"0010" = {
"id" = 20
"name" = "ruleK"
}
}
|
Finally we got what we wanted, the order of the rules exactly the same as intended by the user after giving the list to the for_each Meta-Argument.