Patrick Lee Scott

Read my story at <a href="https://patscott.io/home">https://patscott.io/home</a>

Automatic TLS/SSL/HTTPS for Jenkins X Previews

Sanely Enabling TLS using CertManager/LetsEncrypt with Jenkins X's ExposeController on every Preview build

If you're on the Jenkins X train, chances are you're loving the preview builds.
With every single Pull Request you make, as part of Jenkins X's integration with Kubernetes, a "Preview Environment" is spun up, and posted to your open PR with a comment and a link that you can click to see the running version of the changes you just made.
Great for fast visual validation that things are looking, and working as intended.
Under the hood, these preview environments are powered by Helm. specifically, without the use of the server side component Tiller. Jenkins X is nice enough to wire this up, and create Helm charts for you.
Chances are, you've also wanted to set up HTTPS.
No problem, there's a command for that.
jx upgrade ingress --cluster
After a rather long series of saying yes over and over, you have certificates!
"Success!" you shout, as you make a winning motion with your fist.
And so you go on with life, but only for a short while.
As you're happily awaiting your magical preview link something happens!
Which something that is depends on your configuration and version, but it's not a working SSL cert, that's for sure!
Either you get a build error that the preview status could not be determined and something about a certificate, or you get a successful build, only to open up chrome to an invalid cert.
error: error checking if preview application https://*.com is available
Or maybe the dreaded "Privacy Error":
What happened!? I thought we upgraded the certs!
And so, you begin to dig.
You find closed PRs that say it's fixed, and references to a
tls-acme
or
tlsacme
or
tlsAcme
or maybe even
TLSAcme
setting and try several variations of that set to true and none of those work, and what about turning this http setting off? Nothing works.
Eventually you look more closely at the logs and see it never used any of your values anyway.
So then you go read the source code and you see
tls-acme
and
tlsacme
and
TLSAcme
so you're just more confused now cause where do go modules even come from anyway?
So then you go learn Go and how it's module system works, at least, enough to read it, and figure out that make preview is calling this and has a flag called
tls-acme
that you can set to true from the command line, so you try that!
You modify your pipeline to append
--tls-acme=\"true\"
to it.
A few minutes pass as you anxiously await the results of your pipeline. 9th times a charm, right?
Annnnd! DEPRECATED!
WOMP WOMP!
You saw it in the code, but hoped it would work anyway.
It does leave a clue, however.
Flag --tls-acme has been deprecated, please use `jx upgrade ingress` after install instead
Feeling defeated you head over to your Jenkinsfile, and add the line:
sh "jx upgrade ingress"
Alas, it needs user input, as you already knew.
To solve this, we need the
--batch-mode
flag.
And with that, we will find one more issue, a missing helm repository:
jenkins-x
.
That can be fixed easy enough by adding:
sh "helm repo add jenkins-x https://storage.googleapis.com/chartmuseum.jenkins-x.io"
Using batch mode, turns out, it does get some config from a ConfigMap that's been created in the development namespace, which is
jx
by default, and a team name if you are using that feature. But not everything.
apiVersion: v1
data:
  clusterissuer: "false"
  domain: yourdomain.com
  email: youremail@yourdomain.com
  exposer: Ingress
  issuer: letsencrypt-prod
  tls: "true"
  urltemplate: ""
kind: ConfigMap
For the rest, we can pass in some flags. Here's the full final Jenkinsfile stage.
stage('CI Build and push snapshot') {
  when {
    branch 'PR-*'
  }
  environment {
    TEAM = "jx"
    PREVIEW_VERSION = "0.0.0-SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER"
    PREVIEW_NAMESPACE = "$APP_NAME-$BRANCH_NAME".toLowerCase()
    HELM_RELEASE = "$PREVIEW_NAMESPACE".toLowerCase()
    FULL_PREVIEW_NAMESPACE = "$TEAM-$ORG-$APP_NAME-$BRANCH_NAME".toLowerCase()
  }
  steps {
    container('nodejs') {
      sh "export VERSION=$PREVIEW_VERSION && skaffold build -f skaffold.yaml"
      sh "jx step post build --image $DOCKER_REGISTRY/$ORG/$APP_NAME:$PREVIEW_VERSION"
      dir('./charts/preview') {
        sh "make preview"
        sh "jx preview --app $APP_NAME --dir ../.."
        sh "helm repo add jenkins-x https://storage.googleapis.com/chartmuseum.jenkins-x.io"
        sh "jx upgrade ingress --batch-mode --namespaces=$FULL_PREVIEW_NAMESPACE --services=$APP_NAME --wait-for-certs=false --skip-resources-update=true --skip-certmanager=true"
      }
    }
  }
}
And with that, the preview pull request now creates a cert after the preview environment is successfully created! It may take around 60 seconds or so for the cert to be created after the completion of the build.
There are a few caveats to this, such as when you are using a custom urlTemplate for the preview environment. In that case you'd need to adjust the environment variable I've added
FULL_PREVIEW_NAMESPACE
. Another one is if you are using the
teams
feature, you'll need to make sure you update that environment variable as well.
Happy Hacking!
Make sure to follow me on here if that's still a thing, and on twitter @pat_scott!
If you're looking to learn for yourself, the best place to start is reading about my own journey, which I'm imagining is quite similar to yours! At the end you'll find a FREE 21 day email course called "Getting Dangerous with DevOps!"

Tags

Comments

More by Patrick Lee Scott

Kubernetes
Softwareengineering
Startup
Engineering
Money
Kubernetes
Docker
React
Javascript
Docker
Software Architecture
Microservices
Devops
Javascript
React
React
Startup
Microservice Architecture
Microservices
Docker
Topics of interest