Ansible:- Variable format, Precedence, jinja2 and various method to define the variables in playbook

9 min readAug 5, 2020

While automation exists to make it easier to make things repeatable, it also require to develop the reusable code by defining the variables.

All systems are not exactly alike; some may require configuration that is slightly different from others.

Example: Assume that you have a playbook which will install oracle binary and configure Oracle on prod and non-prod environment. While configuration of Oracle need to define an unique oracle home specific to environment(prod, uat and dev -: it may be same or unique for your environment).

Now this can be achieved in two ways.

Create multiple environment specific playbook to install and configure oracle in respective environment(Here you will define oracle home for each environment) — But this is not the right way to use Ansible.

Example:

##Created Oracle roles to create a oracle prod playbook.
    oracle for prod env = "/u01/product/19.0.0/prod/dbhome"##Created Oracle roles to create a oracle uat playbook.
    oracle for prod env = "/u01/product/19.0.0/uat/dbhome"##Created Oracle roles to create a oracle dev playbook.
    oracle for prod env = "/u01/product/19.0.0/dev/dbhome"

2. Create a single playbook by defining the variable inside the playbook, that make you to create a reusable playbook for any environment.

Now, you can see the variable importance when you define the variable key and value in the playbook as mentioned below.

Define oracle_environment variable in ansible playbook : /u01/product/19.0.0/{{ oracle_environment }}/dbhome” where can be oracle_environment variable value is — test/dev/uat/prod.

oracle_environment → variable key

test/dev/uat/prod → variable value

##Created Oracle roles to create a oracle playbook.[root@mycontrolnode ~]# ansible-galaxy role init oracle
- Role oracle was created successfully
[root@mycontrolnode ~]# tree oracle
oracle
├── defaults
│   └── main.yml
├── files
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── README.md
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml8 directories, 8 files
[root@mycontrolnode ~]#

Where to define variable, in my case I am defining the oracle home variable in vars->main.yml

Execute the oracle playbook by changing the “oracle_environment” that will set oracle home for dev environment as mentioned below.

[root@mycontrolnode ~]# vim oracle/vars/main.yml
---
# vars file for oracle
oracle_environment: dev
oracle_install_user: oracle
oracle_install_group: oinstall

Now, i want to execute the same playbook for prod then replace the variable value from dev to prod for prod environment as mentioned below.

[root@mycontrolnode ~]# vim oracle/vars/main.yml
---
# vars file for oracle
oracle_environment: prod
oracle_install_user: oracle
oracle_install_group: oinstall

create an oracle home path to configure oracle home by adding {{ oracle_environment }} variable value in path.

[root@mycontrolnode oracle]# vim tasks/main.yml
---
# tasks file for oracle
- name: create directories for oracle_home
  file:
    path: "/u01/product/19.0.0/{{ oracle_environment }}/dbhome"
    state: directory
    owner: "{{ oracle_user }}"
    group: "{{ oracle_group }}"
    mode: 0755

Part-1

Ansible variables format:

Before you start variables, it’s impotent to know what are the valid variable names.
Note: Variables should always start with a letter. Variables names should be letters, numbers and underscores.

oracle_listener_port is a great variable. servername01 is fine too.

oracle_listener_port, oracle listener port, oracle.listener.port and 01 are not valid variable names.

Part-2

Ansible variables Precedence: where should I put variables to consume in playbook?

If multiple variables of the same name are defined in different places, they get overwritten in a certain order.

Here is the order of precedence from least to greatest (the last listed variables winning prioritization):

https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable

command line values (eg “-u user”)

role defaults [1]

inventory file or script group vars [2]

inventory group_vars/all [3]

playbook group_vars/all [3]

inventory group_vars/* [3]

playbook group_vars/* [3]

inventory file or script host vars [2]

inventory host_vars/* [3]

playbook host_vars/* [3]

host facts / cached set_facts [4]

play vars

play vars_prompt

play vars_files

role vars (defined in role/vars/main.yml)

block vars (only for tasks in block)

task vars (only for the task)

include_vars

set_facts / registered vars

role (and include_role) params

include params

extra vars (always win precedence)

Basically, anything that goes into “role defaults” (the defaults folder inside the role) is the most malleable and easily overridden. Anything in the vars directory of the role overrides previous versions of that variable in namespace. The idea here to follow is that the more explicit you get in scope, the more precedence it takes with command line -e extra vars always winning. Host and/or inventory variables can win over role defaults, but not explicit includes like the vars directory or an include_vars task.

So far i have explained correct why variable is impotent in playbook, variables format and variable precedence.

Now, I will explain multiple way to consume the variables in Ansible Playbook.

1) Defining variables in a playbook:-

You can define variables directly in a playbook:

- hosts: all
  vars:
    oracle_listener_port: 1567

2) Defining variables in included files and roles:-

As described variable above by creating oracle roles,

Include_vars can be used in a playbook or roles to call a variables from file.

[root@mycontrolnode ~]# vim site.yml
---
- name: Include vars of stuff.yaml into the 'stuff' variable (2.2).
  include_vars:
    file: stuff.yaml
    name: stuff
- debug:
    msg: {{ print_variabe }}_this is a test play

stuff.yaml file is located in same directory

[root@mycontrolnode ~]# vim stuff.yaml
---
print_variabe: Playbook_output

Playbook output

ok: [mycontrolnode] => {
    "msg": "Playbook_output_this is a test play"
}

3) Using variables with Jinja2:-

Once you’ve defined variables, you can use them in your playbooks using the Jinja2 templating system. Here’s a simple Jinja2 template:

here “number_of_huge_pages” variable value need to replace with 256

[root@mycontrolnode ~]# vim oracle/vars/main.yml
---
# vars file for oracle
oracle_environment: dev
oracle_install_user: oracle
oracle_install_group: oinstall#System Variables
number_of_huge_pages: 256

here “number_of_huge_pages” variable value will be assigned to vm.nr_hugepages in sysctl.conf file

[root@mycontrolnode oracle]# vim templates/sysctl.conf.j2
# sysctl settings are defined through files in
#  /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
#  Vendors settings live in /usr/lib/sysctl.d/.
#  To override a whole file, create a new file with the same in
#  /etc/sysctl.d/ and put new settings there. To override
#  only specific settings, add a file with a lexically later
#  name in /etc/sysctl.d/ and put new settings there.
#
#  For more information, see sysctl.conf(5) and sysctl.d(5).
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.# Controls IP packet forwarding
net.ipv4.ip_forward = 0# Controls source route verification
net.ipv4.conf.default.rp_filter = 1# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1
# Controls the default maxmimum size of a mesage queue
kernel.msgmnb = 65536# Controls the maximum size of a message, in bytes
kernel.msgmax = 65536# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296# semaphores
kernel.sem = 250 32000 100 128
# shared memory
kernel.shmmni = 4096# file handles
fs.file-max = 6815744# asynchronous i/o
fs.aio-max-nr = 1048576# disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1# default range of IP port numbers that are allowed for TCP and UDP traffic
net.ipv4.ip_local_port_range = 9000 65535# increase TCP max buffer size (depending on the type of NIC and the round-trip time these values can be changed)
# Maximum TCP Receive Window
net.core.rmem_max = 8388608
net.core.rmem_default = 8388608
# Maximum TCP Send Window
net.core.wmem_max = 8388608
net.core.wmem_default = 8388608
#  memory reserved for TCP receive buffers (vector of 3 integers: [min, default, max])
net.ipv4.tcp_rmem = 4096 87380 8388608
# memory reserved for TCP send buffers (vector of 3 integers: [min, default, max])
net.ipv4.tcp_wmem = 4096 87380 8388608# increase the length of the processor input queue
net.core.netdev_max_backlog = 30000
# maximum amount of memory buffers (could be set equal to net.core.rmem_max and net.core.wmem_max)
net.core.optmem_max = 20480
# socket of the listen backlog
net.core.somaxconn = 1024# tcp selective acknowledgements (disable them on high-speed networks)
net.ipv4.tcp_sack = 1
net.ipv4.tcp_dsack = 1
# Timestamps add 12 bytes to the TCP header
net.ipv4.tcp_timestamps = 1
# Support for large TCP Windows - Needs to be set to 1 if the Max TCP Window is over 65535
net.ipv4.tcp_window_scaling = 1
# The interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe
net.ipv4.tcp_keepalive_time = 1800
# The interval between subsequential keepalive probes, regardless of what the connection has exchanged in the meantime
net.ipv4.tcp_keepalive_intvl = 30
# The number of unacknowledged probes to send before considering the connection dead and notifying the application layer
net.ipv4.tcp_keepalive_probes = 5
# The time that must elapse before TCP/IP can release a closed connection and reuse its resources.
net.ipv4.tcp_fin_timeout = 30
# Size of the backlog connections queue.
net.ipv4.tcp_max_syn_backlog = 4096
# The tcp_tw_reuse setting is particularly useful in environments where numerous short connections are open and left in TIME_WAIT state, such as web servers.
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1# The percentage of how aggressively memory pages are swapped to disk
vm.swappiness = 0
# The percentage of main memory the pdflush daemon should write data out to the disk.
vm.dirty_background_ratio = 25
# The percentage of main memory the actual disk writes will take place.
vm.dirty_ratio = 20# set the number of huge pages based on the Hugepagesize, i.e., 2048kB
vm.nr_hugepages = {{ number_of_huge_pages }}

When template will copy from src to dest then jinja2 templating will replace the variable to variable value

- name: set sysctl for oracle configuration
  template: 
    src: sysctl_conf.j2 
    dest: /etc/sysctl.d/sysctl.conf
    owner: root
    group: root
    mode: 0644  
    backup: yes
  register: sysctl_changed

4) Variables discovered from systems:- Facts

To use facts you need to enable gather fact in the playbook by add a parameter called “gather_facts: yes” (more details you can get from given link)

Now, print OS distribution of server by using facts {{ ansible_distribution }}

- name: os distribution of server 
  shell: echo os distribution of server is {{ ansible_distribution }}

5) Registering variables:-

Another major use of variables is running a command and registering the result of that command as a variable. When you execute a task and save the return value in a variable for use in later tasks, you create a registered variable.

For example:

- hosts: all  tasks:     - shell: /usr/bin/df
       register: disk_result    - shell: /usr/bin/lsblk
       when: disk_result.rc == 5

6) Accessing complex variable data:-

We already described facts a little higher up in the documentation.

Some provided facts, like networking information, are made available as nested data structures. Here’s how we get an IP address:

{{ ansible_facts["eth0"]["ipv4"]["address"] }}

OR alternatively:

{{ ansible_facts.eth0.ipv4.address }}

7) Accessing information about other hosts with magic variables:-

Magic variable names are reserved — do not set variables with these names. The variable environment is also reserved.

The most commonly used magic variables are hostvars, groups, group_names, and inventory_hostname.

hostvars lets you access variables for another host, including facts that have been gathered about that host. You can access host variables at any point in a playbook. Even if you haven’t connected to that host yet in any play in the playbook or set of playbooks, you can still get the variables, but you will not be able to see the facts.

If your database server wants to use the value of a ‘fact’ from another node, or an inventory variable assigned to another node, it’s easy to do so within a template or even an action line:

[webservers]
192.168.10.100
192.168.10.101

[dbservers]
192.168.20.100---
- name: Create hosts file in all servers
  hosts: all
  become: true
  gather_facts: true

tasks:
    - name: Create /etc/hosts
      lineinfile: 
        dest: /etc/hosts
        regexp: '.*{{ item }}$'
        line: "{{ hostvars['item']['ansible_default_ipv4']['address'] }} {{ hostvars['item']['ansible_fqdn'] }} {{ hostvars['item']['ansible_hostname'] }}"                                                                                                       state: present
        state: present 
      with_items: "{{ groups['dbservers'] }}"

groups is a list of all the groups (and hosts) in the inventory. This can be used to enumerate all hosts within a group. For example:

If a server needs to use the value of a “fact” from another node, it’s easy to do so within a template templates/hosts.j2

{% for host in groups['webservers'] %}
{{ hostvars[host]['ansible_facts']['default_ipv4']['address'] }} {{ hostvars[host]['ansible_facts']['fqdn'] }} {{ hostvars[host]['ansible_facts']['hostname'] }}
{% endfor %}

The playbook:

---
- name: Generate Hosts File
  hosts: all
  become: true
  gather_facts: true

  tasks:
    - name: Create "/etc/hosts"
      template:
        src: "hosts.j2"
        dest: "/etc/hosts"
        owner: root
        group: root
        mode: "0644"

inventory_hostname is the name of the hostname as configured in Ansible’s inventory host file. This can be useful when you’ve disabled fact-gathering, or you don’t want to rely on the discovered hostname ansible_hostname. If you have a long FQDN, you can use inventory_hostname_short, which contains the part up to the first period, without the rest of the domain.

8. Defining variables in files

You can do this by using an external variables file, or files, just like this:

---- hosts: all
  remote_user: root
  vars:
    favcolor: blue
  vars_files:
    - /vars/external_vars.yml  tasks:- debug: 
      mgs: {{somevar}} and {{ magic }}

This removes the risk of sharing sensitive data with others when sharing your playbook source with them.

The contents of each variables file is a simple YAML dictionary, like this:

---
# in the above example, this would be vars/external_vars.yml
somevar: somevalue
password: magic

9. Passing variables on the command line

In addition to vars_prompt and vars_files, it is possible to set variables at the command line using the --extra-vars (or -e) argument. Variables can be defined using a single quoted string (containing one or more variables) using one of the formats below

key=value format:

ansible-playbook release.yml --extra-vars "version=1.23.45 other_variable=foo"

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇

Reference:

Using Variables - Ansible Documentation

While automation exists to make it easier to make things repeatable, all systems are not exactly alike; some may…

docs.ansible.com