Automating Azure Deployments with Ansible on OpenBSD: Part 1
It took me four hours to give up on Terraform when I tried it. The reasons why I loathed every aspect of my Terraform experiences are well-documented, but my dissatisfaction boiled down to one unavoidable point: it didn't work.
This is largely due to a much lower level of maturity in the Terraform community's Azure platform modules. They are more primitive than the AWS modules and a key indicator of that is a lack of filtering in Terraform's Azure data resources. If you already created a widget named "foo" in the "West US" region and one named "bar" in "East US", you can't ask Terraform to assemble a list of every widget and then simply use the name of the West US one in your West US deployment. You have to know it's named "foo", or else you need to write a separate script, with a separate authentication method and a separate cloud API and a separate everything else, and call it from your Terraform plan.
Terraform didn't just disappoint me. It depressed me. I like Ansible's clean, (mostly) consistent declarative syntax and Terraform seemed like it could have been a good cloud-ready component in anyone's cloud service toolkit. Deploy your virtual resources with Terraform, manage them at scale with Ansible. It would have been glorious. But sadly, Terraform just isn't ready for prime time.
Imagine my surprise and delight when I discovered that Ansible can handle cloud deployments. Why did no one tell me?
https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html
Ansible can be configured to deploy and manage cloud resources across a number of platforms: AWS, Google, and Azure are all represented, as are many of the smaller players in the field.
When I discovered this, I almost passed out.
Let's do this. You can install Ansible with Azure CLI on an OpenBSD machine, but there are some undocumented caveats and a lot of Python package incompatibilities that you need to take into consideration. You may not have the same problems if you make a Linux machine your Ansible master box, but why would you ever do that?
There's a pre-built Ansible port in OpenBSD packages you can install (pkg_add ansible), but I don't use it. I find that it's always behind, often significantly behind, the latest stable Ansible release. On my Ansible controllers, I run Ansible directly from the main git repo.
I do this not just because it's always going to be more current than the OpenBSD port, but because it's going to always give me a consistently current Ansible platform... on any OS. If I setup Ansible to run from source, I can use the same version of Ansible on BSD or Linux. I began setting up Ansible this way after a version syntax incompatibility between two Ansible hosts with two different packaged versions installed that required me to rewrite most of my playbooks to get them to work on both machines.
(A quick note about OpenBSD: OpenBSD has a wxallowed mount flag that disables W^X support on given partitions. We're going to make virtual Python environments under our home directory and W^X will prevent you from running Python from anywhere in your /home partition unless you make sure that wxallowed is in /etc/fstab. If you install everything into / you don't have to worry about editing /etc/fstab, but you really shouldn't do that.)
ANSIBLE_GIT_PATH=~/ansible.git
PYTHON_VERSION=python-3.6.8p0
# install pre-reqs as root:
doas pkg_add bash git ${PYTHON_VERSION} py3-jinja2 py3-setuptools py3-yaml
(cd /usr/local/bin; test -f python || doas ln -sf python3 python)Ansible freaks out if you aren't running a Unicode-compatible shell. For OpenBSD, Unicode support is a constant work in progress:
cp -p ~/.profile ~/.profile.$(date +"%F_%T").backup cp -p ~/.profile ~/.profile.new echo export LC_CTYPE="en_US.UTF-8" >> ~/.profile.new mv ~/.profile.new ~/.profile . ~/.profile
Clone the Ansible repository and checkout the latest stable version.
git clone https://github.com/ansible/ansible.git "${ANSIBLE_GIT_PATH}" ANSIBLE_VERSION=`(cd "${ANSIBLE_GIT_PATH}"; git branch -a | grep stable | tail -1 | awk -F'/' '{print $3}')` (cd "${ANSIBLE_GIT_PATH}"; git checkout ${ANSIBLE_VERSION})
That's it. You could stop here if you just want to use vanilla Ansible to manage one machine or a thousand.
To actually run Ansible, start Bash and source the Ansible environment:
cd "${ANSIBLE_GIT_PATH}"
bash
bash-5.0$ source hacking/env-setupI used to install Ansible and then install the Azure bits separately in a tmux session. Eventually, like that ape at the start of 2001: A Space Odyssey, it dawned on me that once I got Ansible installed, I didn't need to do any extra work if I had an "install the Azure bits" playbook. So I wrote one.
bash-5.0$ cat install-azure-packages.yml
---
- hosts: localhost
vars:
ansible_become_method: doas
async_runtime_max: 3600
pip_path: /usr/local/bin/pip3.6
pipenv_azure_module_path: "{{ansible_user_dir}}/azure-ansible-modules"
pipenv_azure_cli_path: "{{ansible_user_dir}}/cli-azure"
tasks:
- name: install py3-pip
become: yes
openbsd_pkg:
name: py3-pip
state: present
- name: upgrade pip
become: yes
command: "{{pip_path}} install --upgrade pip"
- name: pip install pipenv
become: yes
command: "{{pip_path}} install pipenv"
- name: remove {{pipenv_azure_module_path}}
file:
path: "{{pipenv_azure_module_path}}"
state: absent
- name: create {{pipenv_azure_module_path}}
file:
path: "{{pipenv_azure_module_path}}"
recurse: yes
state: directory
- name: pip install 'ansible[azure]' (async)
command: pipenv install 'ansible[azure]'
args:
chdir: "{{pipenv_azure_module_path}}"
async: "{{async_runtime_max}}"
poll: 0
register: job_mod
- name: remove {{pipenv_azure_cli_path}}
file:
path: "{{pipenv_azure_cli_path}}"
state: absent
- name: create {{pipenv_azure_cli_path}}
file:
path: "{{pipenv_azure_cli_path}}"
recurse: yes
state: directory
- name: pip install azure-cli (async)
command: pipenv install --pre azure-cli
args:
chdir: "{{pipenv_azure_cli_path}}"
async: "{{async_runtime_max}}"
poll: 0
register: job_cli
- name: wait for azure-cli
async_status:
jid: "{{job_cli.ansible_job_id}}"
register: job_cli_result
until: job_cli_result.finished
delay: 120
retries: 30
- name: wait for 'ansible[azure]'
async_status:
jid: "{{job_mod.ansible_job_id}}"
register: job_mod_result
until: job_mod_result.finished
delay: 120
retries: 30
# END
bash-5.0$ ansible-playbook install-azure-packages.ymlThis playbook basically just runs pipenv install 'ansible[azure]' and pipenv install --pre azure-cli simultaneously and waits for them to finish. The --pre argument is necessary because the azure-cli package sometimes requires a release candidate version of a dependency (such as azure-mgmt-containerregistry version 3.0.0rc5) and by default pipenv will skip pre-release versions and therefore throw an error and die.
(A note about Azure authentication: The Ansible documentation states that Azure authentication supports an Azure Active Directory config (a username and a password) or an Azure Service Principal (an app ID, a tenant ID, and a password). Technically, service principals can also handle management certificates, but this is more complex and Ansible doesn't support it. You can set up a service principal instead of installing Azure CLI, but it is a chore. With Azure CLI installed you can simply run az login once and Ansible will leverage that instead. The service principal option is a more convoluted authentication method, but its advantages are more useful for an enterprise team/production environment. As an individual I prefer the Azure CLI option, but your actual mileage may vary.)
You only have to do this once: start Azure CLI inside the pipenv virtual environment to log into Azure. My Ansible machine does not have a browser installed, so I have to submit the login request from the Ansible machine, then authenticate that request from a different box with a GUI on it.
cd ~/cli-azure pipenv run az login --use-device-code
The --use-device-code option gives you an alphanumeric code to enter. Follow the instructions on a machine with a browser (i.e., go to https://microsoft.com/devicelogin) and enter the code when prompted.
After authenticating your machine with Azure CLI, set your default subscription. For example:
cd ~/cli-azure pipenv run az account set --subscription 49f575c7-bdad-4e13-98ff-b546adf5d0b1
You're done with Azure CLI now. (Plenty of Azure deployment work can be done with Azure CLI directly, so I like to keep it around for running queries and one-offs.) From here on out, all of our Ansible playbooks will NOT mention our cloud credentials or subscription information, so if you are doing deployments between multiple components in different subscriptions and with different accounts, you will need to take this into consideration with Azure CLI, not with Ansible.
From within your Ansible bash session, fire up your Ansible+Azure virtual environment:
# if you haven't sourced your Ansible session yet: cd ~/ansible.git bash bash-5.0$ source hacking/env-setup # then start your virtual env: bash-5.0$ cd ~/azure-ansible-modules bash-5.0$ pipenv shell
Using pipenv shell instead of pipenv run lets you keep your virtual environment open for more than just one command. This way you can run ansible-playbook multiple times and it can continually leverage the Azure modules.
Your Python virtual environment will look something like this:
(azure-ansible-modules) bash-5.0$
That's it. You're done. From here, you can write an Ansible cloud-resource playbook and start building things in Azure.
Next time: Writing an Ansible cloud-resource playbook and building things in Azure.
No comments:
Post a Comment