I’ve written a whole bunch of blogs about running Cloud Foundry on Kubernetes. They all work, to varying degrees of success and have the support of a smattering of communities (read: companies with large budgets + engineers + open source ideals).
Sometimes you just want BOSH. That sweet, sweet, familiar BOSH. Enter
bosh-lite works by spinning up a VM and placing BOSH on it. From there, you can use it to spin
zookeeper, or anything else that has a BOSH manifest.
The BOSH director is configured with runtime configs, cloud configs, stemcells, releases, and all the other BOSH-like functionality you’ve come to expect.
The tool has been around forever, with a very good tutorial on installing and configuring
bosh-lite located at https://bosh.io/docs/bosh-lite/. The rest of this blog assumes that you have followed these instructions up to step #7.
I had an issue with a newer version of VirtualBox (6.1.28), what worked for me is version 6.1.26 which is available here.
It ends with an example deployment of
zookeeper which I guess is cool, but I’m guessing most BOSH directors are associated with a Cloud Foundry deployment. Read on for tips that maybe aren’t obvious from the documentation!
cf-deployment-land there is documentation for using BBL to deploy
bosh-lite and CF to GCP or AWS. I point these instructions out in case you have an unlimited IAAS budget, however, I’ll show you how to deploy this to your Mac.
There is also a Readme within
cf-deployment For Operators Deploying CF to local bosh-lite. I’ll be using this as the basis for the scripting below.
For these examples, I’ve assumed you will clone the CF repo to your home folder:
cd ~ git clone https://github.com/cloudfoundry/cf-deployment.git
This is spread across a few places in the documentation, I’ve gathered them all together:
export CREDHUB_SERVER=https://192.168.56.6:8844 export CREDHUB_CLIENT=credhub-admin export CREDHUB_SECRET=$(bosh interpolate ~/deployments/vbox/creds.yml --path=/credhub_admin_client_secret) export CREDHUB_CA_CERT="$(bosh interpolate ~/deployments/vbox/creds.yml --path=/credhub_tls/ca )"$'\n'"$( bosh interpolate ~/deployments/vbox/creds.yml --path=/uaa_ssl/ca)" export BOSH_CLIENT=admin export BOSH_CLIENT_SECRET="$(bosh int ~/deployments/vbox/creds.yml --path /admin_password)" export BOSH_CA_CERT="$(bosh interpolate ~/deployments/vbox/creds.yml --path /director_ssl/ca)" export BOSH_ENVIRONMENT=vbox bosh alias-env vbox -e 192.168.56.6 --ca-cert <(bosh int ~/deployments/vbox/creds.yml --path /director_ssl/ca)
You might consider adding this to your bash/zsh profile.
bosh upload-stemcell --sha1 f399044d2ebe3351f0f1b0b3f97ef11464d283b4 "https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-xenial-go_agent?v=621.125" bosh update-runtime-config ~/workspace/bosh-deployment/runtime-configs/dns.yml --name dns bosh update-cloud-config ~/cf-deployment/iaas-support/bosh-lite/cloud-config.yml
As time goes by, CF may complain of wanting a newer stemcell version, update the
bosh upload-stemcell command with the requested version. The version and
sha1 are listed here, note that
This part will take around 45 minutes to complete, unless you’ve changed the IP address of the BOSH director, the following command will deploy CF.
cd ~/cf-deployment bosh -e 192.168.56.6 -d cf deploy \ cf-deployment.yml \ -o operations/bosh-lite.yml \ -v system_domain=bosh-lite.com
This is done in two steps, targeting the API and then logging in, pulling the admin password out of credhub without printing it to the screen:
cf api https://api.bosh-lite.com --skip-ssl-validation cf login -u admin -p $(credhub get -n $(credhub find -n admin | grep cf_admin | cut -d: -f2) | grep value | cut -d: -f2) -o system -s test
Create a space and start pushing apps.
Why? I like to be able to SSH to my BOSH Director to poke around. I’m weird. This is not required for the CF deployment, but nice to have:
bosh int ~/deployments/vbox/creds.yml --path /jumpbox_ssh/private_key > ~/deployments/vbox/jumpbox.key chmod 600 ~/deployments/vbox/jumpbox.key ssh [email protected] -i ~/deployments/vbox/jumpbox.key
Rebooting your Mac causes
bosh-lite to misbehave, badly, unless you follow a couple of easy steps.
Even performing these steps I can’t always bring the BOSH Director back online.
If you have accidentally rebooted the host you can recreate the BOSH director using the state file and use
bosh cck to recover the broken CF deployment:
cd ~/deployments/vbox bosh create-env ~/workspace/bosh-deployment/bosh.yml \ --state ./state.json \ -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \ -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \ -o ~/workspace/bosh-deployment/bosh-lite.yml \ -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \ -o ~/workspace/bosh-deployment/uaa.yml \ -o ~/workspace/bosh-deployment/credhub.yml \ -o ~/workspace/bosh-deployment/jumpbox-user.yml \ --vars-store ./creds.yml \ -v director_name=bosh-lite \ -v internal_ip=192.168.56.6 \ -v internal_gw=192.168.56.1 \ -v internal_cidr=192.168.50.0/24 \ -v outbound_network_name=NatNetwork --recreate bosh cck
If that doesn’t work, you can completely wipe and recreate the BOSH and CF deployment by:
Delete all files
bosh create-envcommand then redeploy CF.
If you get:
Deploying: Creating instance 'bosh/0': Waiting until instance is ready: Post "https://mbus:<redacted>@192.168.56.6:6868/agent": dial tcp 192.168.56.6:6868: connect: connection refused Exit code 1
You probably skipped the step to add the host routing:
sudo route add -net 10.244.0.0/16 192.168.56.6
We all deserve nice things.
bosh-lite is one of those nice things for folks who enjoy BOSH and want to use it on their own computer.
Also published here.