Photo by on Flickr Alex Gorzen More Limes, More Coconuts In previous blog posts, I reviewed how to , which gives you a nice stable deployment of , the downside is this costs you money for every hour it is run on AWS. deploy KubeCF on EKS KubeCF I’m used to giving Amazon money, but I typically get a small cardboard box in exchange every few days. So, how do you run on your Mac for free(ish)? Tune in below. KubeCF There Be Dragons Ahead You need at least 16GB of memory installed on your Apple MacOS device, the install will use around 11GB of the memory once it is fully spun up. The installation is fragile and frustrating at times, this is geared more towards operators who are trying out skunkworks on the platform such as testing custom buildpacks, hacking db queries, and other potentially destructive activities. The install does NOT survive reboots and becomes extra brittle after 24+ hours of running. This is not KubeCF’s fault, when run on EKS it will happily continue to run without issues. You’ve been warned! Overview of Install Install Homebrew and Docker Desktop Install and start the shim tuntap Install and deploy a cluster kind Install and configure metallb Install and deploy cf-operator kubecf Log in and marvel at your creation The next few sections are borrowed heavily from , I encourage you to skim this document to understand why the shim is needed and how to verify the configuration for . https://www.thehumblelab.com/kind-and-metallb-on-mac/ tuntap metallb Install Homebrew and Docker Desktop I won’t go into great detail as these tools are likely already installed: Homebrew (full details are here: ) or just run: https://docs.brew.sh/Installation /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" Docker Desktop (full details for installation are at: ) or run: https://docs.docker.com/desktop/mac/install/ brew install --cask docker # Then launch docker from Applications to complete the install and start docker Install and start the shim tuntap Running on MacOS has some “deficiencies” that can be overcome by installing a networking shim, to perform this install: docker brew install git brew install --cask tuntap git clone https://github.com/AlmirKadric-Published/docker-tuntap-osx.git cd docker-tuntap-osx ./sbin/docker_tap_install.sh ./sbin/docker_tap_up.sh Install and deploy a cluster kind Sweet’n’Simple: brew install kind kind create cluster Install and configure metallb We’ll be using as a LoadBalancer resource and open up a local route so that MacOS can route traffic locally to the cluster. metallb sudo route -v add -net 172.18.0.1 -netmask 255.255.0.0 10.0.75.2 kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)" kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml cat << EOF > metallb-config.yaml apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: default protocol: layer2 addresses: - 172.18.0.150-172.18.0.200 EOF kubectl create -f metallb-config.yaml Install and deploy cf-operator kubecf brew install wget brew install helm brew install watch wget https://github.com/cloudfoundry-incubator/kubecf/releases/download/v2.7.13/kubecf-bundle-v2.7.13.tgz tar -xvzf kubecf-bundle-v2.7.13.tgz kubectl create namespace cf-operator helm install cf-operator \ --namespace cf-operator \ --set "global.singleNamespace.name=kubecf" \ cf-operator.tgz \ --wait helm install kubecf \ --namespace kubecf \ --set system_domain=172.18.0.150.nip.io \ --set features.eirini.enabled=false \ --set features.ingress.enabled=false \ --set services.router.externalIPs={172.18.0.150}\ https://github.com/cloudfoundry-incubator/kubecf/releases/download/v2.7.13/kubecf-v2.7.13.tgz watch kubectl get pods -A Now, go take a walk, it will take 30-60 minutes for the helm chart to be fully picked up by the CRDs and are scheduled for running. When complete, you should see output similar to: kubecf cf-operator Every 2.0s: kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE cf-operator quarks-cd9d4b96f-rtkbt 1/1 Running 0 40m cf-operator quarks-job-6d8d744bc6-pmfnd 1/1 Running 0 40m cf-operator quarks-secret-7d76f854dc-9wp2f 1/1 Running 0 40m cf-operator quarks-statefulset-f6dc85fb8-x6jfb 1/1 Running 0 40m kube-system coredns-558bd4d5db-ncmh5 1/1 Running 0 41m kube-system coredns-558bd4d5db-zlpgg 1/1 Running 0 41m kube-system etcd-kind-control-plane 1/1 Running 0 41m kube-system kindnet-w4m9n 1/1 Running 0 41m kube-system kube-apiserver-kind-control-plane 1/1 Running 0 41m kube-system kube-controller-manager-kind-control-plane 1/1 Running 0 41m kube-system kube-proxy-ln6hb 1/1 Running 0 41m kube-system kube-scheduler-kind-control-plane 1/1 Running 0 41m kubecf api-0 17/17 Running 1 19m kubecf auctioneer-0 6/6 Running 2 20m kubecf cc-worker-0 6/6 Running 0 20m kubecf cf-apps-dns-76947f98b5-tbfql 1/1 Running 0 39m kubecf coredns-quarks-7cf8f9f58d-msq9m 1/1 Running 0 38m kubecf coredns-quarks-7cf8f9f58d-pbkt9 1/1 Running 0 38m kubecf credhub-0 8/8 Running 0 20m kubecf database-0 2/2 Running 0 38m kubecf database-seeder-0bc49e7bcb1f9453-vnvjm 0/2 Completed 0 38m kubecf diego-api-0 9/9 Running 2 20m kubecf diego-cell-0 12/12 Running 2 20m kubecf doppler-0 6/6 Running 0 20m kubecf log-api-0 9/9 Running 0 20m kubecf log-cache-0 10/10 Running 0 19m kubecf nats-0 7/7 Running 0 20m kubecf router-0 7/7 Running 0 20m kubecf routing-api-0 6/6 Running 1 20m kubecf scheduler-0 12/12 Running 2 19m kubecf singleton-blobstore-0 8/8 Running 0 20m kubecf tcp-router-0 7/7 Running 0 20m kubecf uaa-0 9/9 Running 0 20m local-path-storage local-path-provisioner-547f784dff-r8trf 1/1 Running 1 41m metallb-system controller-fb659dc8-dhpnb 1/1 Running 0 41m metallb-system speaker-h9lh9 1/1 Running 0 41m Log in and marvel at your creation To login with the uaa user account: admin cf api --skip-ssl-validation "https://api.172.18.0.150.nip.io" acp=$(kubectl get secret \ --namespace kubecf var-cf-admin-password \ -o jsonpath='{.data.password}' \ | base64 --decode) cf auth admin "${acp}" cf create-space test -o system cf target -o system -s test Or, to use uaa client account (because you’re a rebel or something): smoke_tests cf api --skip-ssl-validation "https://api.172.18.0.150.nip.io" myclient=$(kubectl get secret \ --namespace kubecf var-uaa-clients-cf-smoke-tests-secret \ -o jsonpath='{.data.password}' \ | base64 --decode) cf auth cf_smoke_tests "${myclient}" --client-credentials cf create-space test -o system cf target -o system -s test Cleaning Up and Putting Your Toys Away If you are done with the deployment of KubeCF you have two options: Put your creation to sleep. Start Docker Desktop > Dashboard > Select and click “Stop”. Go have fun, when you come back, click “Start”. After a few minutes, the pods will recreate and become healthy kind-control-place Clean up. To remove the cluster and custom routing: kind delete cluster sudo route delete 172.18.0.0 ./sbin/docker_tap_uninstall.sh Debugging Issues Get used to seeing: Request error: Get “https://api.172.18.0.150.nip.io”: dial tcp 172.18.0.150:443: i/o timeout The api server is flaky, try whatever you were doing again after verifying all pods are running as seen in the section. Install cf-operator and deploy kubecf Follow up Have questions? There is an excellent community for KubeCF which can be found at as in Slack. You can ping me there via . https://cloudfoundry.slack.com/archives/CQ2U3L6DC kubecf-dev @cweibel I also have Terraform code which will spin up a VPC + EKS + KubeCF for a more permanent solution to running not on a Mac, check out for more details. KubeCF https://github.com/cweibel/example-terraform-eks/tree/main/eks_for_kubecf_v2 Enjoy! Also published here