BOSH is an effective way to manage your infrastructure and Azure is a great infrastructure to manage, but before you have a BOSH director, you need to bootstrap Azure to a point where you can deploy a BOSH Director to it. Namely, we need an Azure App Registration to use as an automation account, a properly configured virtual network, and a virtual machine inside the network from which we can deploy the BOSH director.
Both Terraform and BOSH perform tasks without user input, and therefore they require an automation account that can access the resources they need to manipulate. Azure implements these in the form of what they call App Registrations.
You will need to create an App Registration, generate a client secret for
authenticating to that App Registration, and then grant said App
Registration Contributor access to the subscription in which you would like
resources to be created. During this process, you will need to collect a few
details:
Now that we have automation credentials, we can start standing up infrastructure using Terraform. Terraform binaries and documentation can be found at https://terraform.io.
We've already made a basic Terraform file for you that can stand up your control plane if given a few extra parameters. You can set up your working directory for terraform as follows:
~/work/repos → git clone git@github.com:genesis-community/terraforms.git Cloning into 'terraforms'... remote: Enumerating objects: 26, done. remote: Counting objects: 100% (26/26), done. remote: Compressing objects: 100% (18/18), done. remote: Total 113 (delta 7), reused 23 (delta 4), pack-reused 87 Receiving objects: 100% (113/113), 1.38 MiB | 3.32 MiB/s, done. Resolving deltas: 100% (41/41), done. ~/work/repos → mkdir terraform-work ~/work/repos → cp terraforms/controlplane/azure/azure.tf terraform-work/ ~/work/repos → cp terraforms/controlplane/azure/terraform.tfvars.example terraform-work/terraform.tfvars ~/work/repos → cd terraform-work/ ~/work/repos/terraform-work → terraform init
Initializing the backend...
Initializing provider plugins... - Checking for available provider plugins... - Downloading plugin for provider "azurerm" (terraform-providers/azurerm) 1.32.1...
The following providers do not have any version constraints in configuration, so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below.
provider.azurerm: version = "~> 1.32"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.
If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.
Input variables for this deployment can be provided by putting their values
into the terraform.tfvars file. Minimally, you'll need to edit the
subscription_id, tenant_id, client_id, and client_secret.
These are all values you should have taken note earlier during the app
registration steps.
Once configured, run terraform apply from the directory with your azure.tf and
terraform.tfvars file.
~/work/repos/terraform-work → pwd /Users/tom/work/repos/terraform-work ~/work/repos/terraform-work → terraform apply
An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create
Terraform will perform the following actions:
...all of the resources to be created...
Plan: 13 to add, 0 to change, 0 to destroy.
Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve.
Enter a value: yes
... many creation lines ...
Apply complete! Resources: 12 added, 0 changed, 0 destroyed.
Outputs:
bastion-box-ip-address = 13.82.181.177 bastion-box-username = ubuntu bastion-box-password = abCdEfGhiJKLmnoPqRStUvWXyz123456 dns-servers = [ "1.1.1.1", "1.0.0.1", ] network-security-group-name = genesis-sg recommended-BOSH-director-IP = 10.0.0.5 resource-group-name = genesis subnet-CIDR = 10.0.0.0/24 subnet-gateway = 10.0.0.1 subnet-name = genesis-controlplane-subnet virtual-network-name = genesis-network
Following this command, Terraform will have stood up a resource group, virtual network, network security groups, and most importantly, a bastion box virtual machine from which you can continue your work. The terraform script will also install various tools and libraries on the bastion box that Genesis, BOSH, and Cloud Foundry need.
You can see from the example output above (and similarly in your actual output) that some output variables are printed at the end of the apply. You will want to copy those output values somewhere for now, as they contain the answers to many of the questions that Genesis will ask you later.
Once your bastion box is created, you can ssh into it to begin working from it. The user, password, and IP address can be found in the terraform output variables.
~/work/repos/terraform-work → ssh ubuntu@13.82.181.177 The authenticity of host '13.82.181.177 (13.82.181.177)' can't be established. ECDSA key fingerprint is SHA256:vvBs1voA6+R6fDyrPf0+YpH1go9HEv3ThxE4ionswR8. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '13.82.181.177' (ECDSA) to the list of known hosts. ubuntu@12.82.181.177's password: Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 4.18.0-1024-azure x86_64)
Last login: Mon Aug 12 15:44:03 2019 from 71.186.231.155
Before proceeding, you will need to set up the basics of your Git config. Run the following with the proper values substituted:
git config --global user.name "YOURNAME" git config --global user.email "YOUREMAIL"
Genesis stores deployment secrets in Vault, a secure, encrypted-at-rest credentials store from Hashicorp. Genesis has opinions about how the Vault should be organized, so it's not suggested to colocate the Genesis secret store on a pre-existing Vault.
We use the Safe CLI to interact
with Vault in general. The Terraform script has installed it on your bastion
host. We're going to use safe to quickly set up a temporary local Vault
instance that Genesis can use until we are able to set up a more permanent
Vault later, via BOSH.
safe will run this Vault as a foreground process, so you will need to open
up a separate shell session / tmux pane on the jumpbox and run:
safe local --memory --as temp-vault
Safe will output the target name of the new Vault, target it, and authenticate to it automatically.
First, create an deployments directory in which to store your various
Genesis deployment folders. Then, change to that directory and initialize
a new Genesis deployment repository:
ubuntu@bastion:~$ mkdir ~/deployments ubuntu@bastion:~$ cd ~/deployments ubuntu@bastion:~/deployments$ genesis init bosh -k bosh --vault temp-vault
Verifying availability of selected vault...ok
Downloading Genesis kit bosh (latest version)...
Initialized empty Genesis repository in ./bosh-deployments - using vault at http://127.0.0.1:8201 (insecure) - using the bosh/1.4.0 kit.
cd into the newly created directory. Now we need to set up a new
proto-BOSH director deployment using genesis new. Genesis new takes a
positional argument which is the name of the environment that you're
creating the deployment in. For the purpose of this tutorial, we'll call ours
test-eastus-controlplane. Genesis will walk you through a series of
questions about the deployment you're about to make. Answers to most of
these questions were output at the end of your terraform apply. Azure
credential information can be found in the terraform.tfvars file that you
set up in your terraform-work directory outside of the bastion box. An
example of a successful genesis new call is seen below.
ubuntu@bastion:~/deployments$ cd bosh-deployments ubuntu@bastion:~/deployments/bosh-deployments$ genesis new test-eastus-controlplane Setting up new environment test-eastus-controlplane...
Using vault at http://127.0.0.1:8201. Verifying availability...ok
Is this a proto-BOSH director? [y|n] > y
We are deploying this BOSH without the aid of an existing BOSH, so this is a Proto-BOSH director
What static IP do you want to deploy this BOSH director on?
10.0.0.5
What network should this BOSH director exist in (in CIDR notation)?
10.0.0.0/24
What default gateway (IP address) should this BOSH director use?
10.0.0.1
This network information is output from your Terraform script. Assuming that
your Terraform script is configured to use the 10.0.0.0/24 CIDR, then the
gateway would be 10.0.0.1, and the first available IP (after the network
address, gateway, Azure reserved IPs, and bastion host) would be 10.0.0.5.
What DNS servers should BOSH use? (leave value empty to end) 1st value > 1.1.1.1 2nd value > 1.0.0.1 3rd value >
Select the DNS servers that you would like this BOSH director and all VMs deployed by this BOSH director to use. For this example, we're using Cloudflare's public DNS.
What IaaS will this BOSH director orchestrate? 1) VMWare vSphere 2) Amazon Web Services 3) Microsoft Azure 4) Google Cloud Platform 5) OpenStack 6) BOSH Warden
Select choice > Microsoft Azure
What is your Azure Client ID?
01234567-89ab-cdef-0123-456789abcdef What is your Azure Client Secret? clientsecret [hidden]: clientsecret [confirm]:
What is your Azure Tenant ID?
01234567-89ab-cdef-0123-456789abcdef
What is your Azure Subscription ID?
01234567-89ab-cdef-0123-456789abcdef
If you've gotten this far into this guide, then you should most likely
select Microsoft Azure as your infrastructure. The Azure Client ID, Client
Secret, Tenant ID, and Subscription ID were all things you should have taken
note of earlier when setting up your App Registration. Also, you should have
placed this information in your terraform.tfvars.
What Azure Resource Group will BOSH be deploying VMs into?
genesis
What security group should be used as the BOSH default security group?
genesis-sg
What is the name of your Azure Virtual Network?
genesis-network
What is the name of the Azure subnet that the BOSH will be placed in?
genesis-controlplane-subnet
These resources should all have been created by your Terraform script. The
names of the created resources were output at the end of the terraform apply.
All of the names are prefixed with the name you chose for your resource
group.
Would you like to edit the environment file? [y|n] > n - auto-generating credentials (in secret/test/eastus/controlplane/bosh)... - auto-generating certificates (in secret/test/eastus/controlplane/bosh)... New environment test-eastus-controlplane provisioned!
To deploy, run this:
genesis deploy 'test-eastus-controlplane'
The command will then spend some time generating certificates and secrets
for the deployment to use. These will get stored in the local Vault and made
accessible by the safe utility.
An environment file named after your environment name (e.g.
test-eastus-controlplane.yml) will be created. This contains non-sensitive
information about your deployment. It is safe to store this file in version
control. To deploy your newly-configured BOSH deployment, you can now run:
ubuntu@bastion:~/deployments/bosh-deployments$ genesis deploy test-eastus-controlplane Compiling package 'ruby-2.4-r4/0cdc60ed7fdb326e605479e9275346200af30a25'... Finished (00:05:47) Compiling package 'boshazurecpi/afc0bf467dd0d4489f757d788065ab4d4f977ce4'... Finished (00:01:55) Installing packages... Finished (00:00:01) Rendering job templates... Finished (00:00:01) Installing job 'azure_cpi'... Finished (00:00:00) Finished installing CPI (00:07:45)
Starting registry... Finished (00:00:00) Uploading stemcell 'bosh-azure-hyperv-ubuntu-xenial-go_agent/315.26'... Finished (00:02:24)
Started deploying Creating VM for instance 'bosh/0' from stemcell 'bosh-stemcell-90cb8ae1-9202-4555-8416-18fe08a1cb19'... Finished (00:02:20) Waiting for the agent on VM 'agentid:16d9e7ca-f62d-442c-43b2-9dd270517140;resourcegroupname:genesis' to be ready... Finished (00:01:00) Creating disk... Finished (00:00:13) Attaching disk 'caching:None;diskname:bosh-disk-data-612ab183-3042-4ec3-a077-e58a4e01ada8;resourcegroupname:genesis' to VM 'agentid:16d9e7ca-f62d-442c-43b2-9dd270517140;resourcegroupname:genesis'... Finished (00:01:07) Rendering job templates... Finished (00:00:36) Compiling package 'golang/b348840d6ee6de87fdd43e4792fdbe1af734039b'... Finished (00:00:49) Compiling package 'bpm-runc/cb758414ed267820880c6bf1ca57d7467c2d70bf'... Finished (00:02:02) Compiling package 'ruby-2.4-r5/726cbb2214e138b576700db6a30698edb2b994e2'... Finished (00:02:12) Compiling package 'mysql/de7350b5d6d835f758a62c0b3a8f07d3f0150d67abfa0da3967bd1127fada4c6'... Finished (00:00:47) Compiling package 'libpq/b3fead55dd7c9fa83be2412b24db84d2b04dbdb5dd0583ee7e9afddbb200659a'... Finished (00:00:23) Compiling package 'ruby-2.4-r4/0cdc60ed7fdb326e605479e9275346200af30a25'... Finished (00:02:05) Compiling package 'openjdk1.8.0/d2d85d44c6f0ee050ef1fb0149ce3da575234ad6'... Finished (00:00:12) Compiling package 'gonats/a32f561770ebdc422c2f9955c7614c23900b9edc62aec51a6b48adc636516802'... Finished (00:00:01) Compiling package 's3cli/4b6a4e65f277e1e4cdd9bfd8330de14ea28e9e5762a304f2d3a35f47958d7376'... Finished (00:00:01) Compiling package 'nginx/d9c13905876ac2bc3f62657048750257794ecc53121ad7cd92583d3ef292b8b8'... Finished (00:00:48) Compiling package 'bpm/d7c8de76cea8201f6cdf37d4f848b0a52c9c5798'... Finished (00:00:12) Compiling package 'uaautils/90097ea98715a560867052a2ff0916ec3460aabb'... Finished (00:00:00) Compiling package 'director/35121e0b2f3354dcfccbed5ea52ea2ccb14dd0d0b90cdd5c4be0787cbdd96ca6'... Finished (00:01:10) Compiling package 'boshazurecpi/afc0bf467dd0d4489f757d788065ab4d4f977ce4'... Finished (00:00:48) Compiling package 'uaa/58ad7562f1d34ad24256fff92b3a241f8e101a2a'... Finished (00:00:56) Compiling package 'postgres-9.4/ee3022bfae2cb7f7454aed9e28fe7715765b9a0086ef2045219f7a4e16baca81'... Finished (00:03:49) Compiling package 'bosh-gcscli/a03b1fc29fd357b8e3023193c87ae9ee49db052965efe5b3d9bff3538ed2df4b'... Finished (00:00:01) Compiling package 'registry/1d74c7cbca2216cdf7f51bc74eca2c831012dc25b5b53d66b3412042edb032ef'... Finished (00:00:50) Compiling package 'credhub/0481a83e36b6179f4fc9f28088c4cd2ac3aa870a'... Finished (00:00:23) Compiling package 'verifymultidigest/2405293bf44933ac48362d7e109fc38ea4469b962cf62715533ecca53c28bf7b'... Finished (00:00:01) Compiling package 'davcli/100c52066493ef0d766c464c9cdd811c902a555013543bd16b62cf59ba5c72eb'... Finished (00:00:01) Compiling package 'lunaclient/b922e045db5246ec742f0c4d1496844942d6167a'... Finished (00:00:03) Compiling package 'health_monitor/8f3a84d055a292d385f4f2f0df42ba25eb424ccff6a3854e1209668fec263295'... Finished (00:00:26) Updating instance 'bosh/0'... Finished (00:01:58) Waiting for instance 'bosh/0' to be running... Finished (00:02:34) Running the post-start scripts 'bosh/0'... Finished (00:00:02) Finished deploying (00:28:04)
Stopping registry... Finished (00:00:00) Cleaning up rendered CPI jobs... Finished (00:00:00)
Succeeded
test-eastus-controlplane BOSH Director deployed!
For details about the deployment, run
genesis info test-eastus-controlplane
To set up a named alias for this director, run
genesis do test-eastus-controlplane -- alias
To log into the BOSH director, as the admin user, run
genesis do test-eastus-controlplane -- login
If this is a new BOSH director, you can upload stemcells by running
genesis do test-eastus-controlplane -- upload-stemcells
To log in to the BOSH director, there is a hook script provided for you by
genesis. Hook scripts can be run with genesis do:
ubuntu@bastion:~/deployments/bosh-deployments$ genesis do test-eastus-controlplane -- login
The BOSH cloud config contains shared information about all the deployments you'll be deploying to your control plane environment. This includes the network IP allocations, VM sizes, disk sizes, availability zone configurations, and compilation VM configuration.
The terraform script, as a part of its creation of the bastion host, has put
a cloud config file for you at $HOME/deployments/director-configs. All you
need to do is upload it.
First, make sure that you're logged into bosh. Then run
bosh -e test-eastus-controlplane update-cloud-config $HOME/deployments/director-configs/controlplane.yml
Now your cloud config should be uploaded and ready to go.