Automatic TLS/SSL/HTTPS for Jenkins X Previews

Sanely Enabling TLS using CertManager/LetsEncrypt with Jenkins X's ExposeController on every Preview build

If you're on the Jenkins X train, chances are you're loving the preview builds.

With every single Pull Request you make, as part of Jenkins X's integration with Kubernetes, a "Preview Environment" is spun up, and posted to your open PR with a comment and a link that you can click to see the running version of the changes you just made.

Great for fast visual validation that things are looking, and working as intended.

Under the hood, these preview environments are powered by Helm. specifically, without the use of the server side component Tiller. Jenkins X is nice enough to wire this up, and create Helm charts for you.

Chances are, you've also wanted to set up HTTPS.

No problem, there's a command for that.

jx upgrade ingress --cluster

After a rather long series of saying yes over and over, you have certificates!

"Success!" you shout, as you make a winning motion with your fist.

And so you go on with life, but only for a short while.

As you're happily awaiting your magical preview link something happens!

Which something that is depends on your configuration and version, but it's not a working SSL cert, that's for sure!

Either you get a build error that the preview status could not be determined and something about a certificate, or you get a successful build, only to open up chrome to an invalid cert.

error: error checking if preview application https://*.com is available

Or maybe the dreaded "Privacy Error":

What happened!? I thought we upgraded the certs!

And so, you begin to dig.

You find closed PRs that say it's fixed, and references to a

tls-acme

tlsacme

tlsAcme

or maybe even

TLSAcme

setting and try several variations of that set to true and none of those work, and what about turning this http setting off? Nothing works.

Eventually you look more closely at the logs and see it never used any of your values anyway.

So then you go read the source code and you see

tls-acme

and

tlsacme

and

TLSAcme

so you're just more confused now cause where do go modules even come from anyway?

So then you go learn Go and how it's module system works, at least, enough to read it, and figure out that make preview is calling this and has a flag called

tls-acme

that you can set to true from the command line, so you try that!

You modify your pipeline to append

--tls-acme=\"true\"

to it.

A few minutes pass as you anxiously await the results of your pipeline. 9th times a charm, right?

Annnnd! DEPRECATED!

WOMP WOMP!

You saw it in the code, but hoped it would work anyway.

It does leave a clue, however.

Flag --tls-acme has been deprecated, please use `jx upgrade ingress` after install instead

Feeling defeated you head over to your Jenkinsfile, and add the line:

sh "jx upgrade ingress"

Alas, it needs user input, as you already knew.

To solve this, we need the

--batch-mode

flag.

And with that, we will find one more issue, a missing helm repository:

jenkins-x

That can be fixed easy enough by adding:

sh "helm repo add jenkins-x https://storage.googleapis.com/chartmuseum.jenkins-x.io"

Using batch mode, turns out, it does get some config from a ConfigMap that's been created in the development namespace, which is

jx

by default, and a team name if you are using that feature. But not everything.

apiVersion: v1
data:
  clusterissuer: "false"
  domain: yourdomain.com
  email: [email protected]
  exposer: Ingress
  issuer: letsencrypt-prod
  tls: "true"
  urltemplate: ""
kind: ConfigMap

For the rest, we can pass in some flags. Here's the full final Jenkinsfile stage.

stage('CI Build and push snapshot') {
  when {
    branch 'PR-*'
  }
  environment {
    TEAM = "jx"
    PREVIEW_VERSION = "0.0.0-SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER"
    PREVIEW_NAMESPACE = "$APP_NAME-$BRANCH_NAME".toLowerCase()
    HELM_RELEASE = "$PREVIEW_NAMESPACE".toLowerCase()
    FULL_PREVIEW_NAMESPACE = "$TEAM-$ORG-$APP_NAME-$BRANCH_NAME".toLowerCase()
  }
  steps {
    container('nodejs') {
      sh "export VERSION=$PREVIEW_VERSION && skaffold build -f skaffold.yaml"
      sh "jx step post build --image $DOCKER_REGISTRY/$ORG/$APP_NAME:$PREVIEW_VERSION"
      dir('./charts/preview') {
        sh "make preview"
        sh "jx preview --app $APP_NAME --dir ../.."
        sh "helm repo add jenkins-x https://storage.googleapis.com/chartmuseum.jenkins-x.io"
        sh "jx upgrade ingress --batch-mode --namespaces=$FULL_PREVIEW_NAMESPACE --services=$APP_NAME --wait-for-certs=false --skip-resources-update=true --skip-certmanager=true"
      }
    }
  }
}

And with that, the preview pull request now creates a cert after the preview environment is successfully created! It may take around 60 seconds or so for the cert to be created after the completion of the build.

There are a few caveats to this, such as when you are using a custom urlTemplate for the preview environment. In that case you'd need to adjust the environment variable I've added

FULL_PREVIEW_NAMESPACE

. Another one is if you are using the

teams

feature, you'll need to make sure you update that environment variable as well.

Happy Hacking!

Make sure to follow me on here if that's still a thing, and on twitter @pat_scott!

~ Patrick Lee Scott

If you're looking to learn for yourself, the best place to start is reading about my own journey, which I'm imagining is quite similar to yours! At the end you'll find a FREE 21 day email course called "Getting Dangerous with DevOps!"