Automatic TLS/SSL/HTTPS for Jenkins X Previews by@patrickleet

Automatic TLS/SSL/HTTPS for Jenkins X Previews

Jenkins X's "Preview Environment" is spun up as part of Jenkins X integration with Kubernetes. Jenkins X is nice enough to wire this up, and create Helm charts for you. The preview environments are powered by Helm, without the use of the server side component Tiller. There are a few caveats to this, such as when you are using a custom url template for the preview environment, you'll need to adjust that variable to make sure you're using it properly. Hacking Jenkins X with Jenkins X will be a free course called "Getting Dangerous with DevOps!"
image
Patrick Lee Scott Hacker Noon profile picture

Patrick Lee Scott

Read my story at https://patscott.io/

Sanely Enabling TLS using CertManager/LetsEncrypt with Jenkins X's ExposeController on every Preview build

If you're on the Jenkins X train, chances are you're loving the preview builds.

With every single Pull Request you make, as part of Jenkins X's integration with Kubernetes, a "Preview Environment" is spun up, and posted to your open PR with a comment and a link that you can click to see the running version of the changes you just made.

image

Great for fast visual validation that things are looking, and working as intended.

Under the hood, these preview environments are powered by Helm. specifically, without the use of the server side component Tiller. Jenkins X is nice enough to wire this up, and create Helm charts for you.

Chances are, you've also wanted to set up HTTPS.

No problem, there's a command for that.

jx upgrade ingress --cluster

After a rather long series of saying yes over and over, you have certificates!

"Success!" you shout, as you make a winning motion with your fist.

And so you go on with life, but only for a short while.

As you're happily awaiting your magical preview link something happens!

Which something that is depends on your configuration and version, but it's not a working SSL cert, that's for sure!

Either you get a build error that the preview status could not be determined and something about a certificate, or you get a successful build, only to open up chrome to an invalid cert.

error: error checking if preview application https://*.com is available

Or maybe the dreaded "Privacy Error":

image

What happened!? I thought we upgraded the certs!

And so, you begin to dig.

You find closed PRs that say it's fixed, and references to a

tls-acme
or
tlsacme
or
tlsAcme
or maybe even
TLSAcme
setting and try several variations of that set to true and none of those work, and what about turning this http setting off? Nothing works.

Eventually you look more closely at the logs and see it never used any of your values anyway.

So then you go read the source code and you see

tls-acme
and
tlsacme
and
TLSAcme
so you're just more confused now cause where do go modules even come from anyway?

image
image

So then you go learn Go and how it's module system works, at least, enough to read it, and figure out that make preview is calling this and has a flag called

tls-acme
that you can set to true from the command line, so you try that!

You modify your pipeline to append

--tls-acme=\"true\"
to it.

A few minutes pass as you anxiously await the results of your pipeline. 9th times a charm, right?

Annnnd! DEPRECATED!

WOMP WOMP!

You saw it in the code, but hoped it would work anyway.

It does leave a clue, however.

Flag --tls-acme has been deprecated, please use `jx upgrade ingress` after install instead

Feeling defeated you head over to your Jenkinsfile, and add the line:

sh "jx upgrade ingress"

Alas, it needs user input, as you already knew.

To solve this, we need the

--batch-mode
flag.

And with that, we will find one more issue, a missing helm repository:

jenkins-x
.

That can be fixed easy enough by adding:

sh "helm repo add jenkins-x https://storage.googleapis.com/chartmuseum.jenkins-x.io"

Using batch mode, turns out, it does get some config from a ConfigMap that's been created in the development namespace, which is

jx
by default, and a team name if you are using that feature. But not everything.

apiVersion: v1
data:
  clusterissuer: "false"
  domain: yourdomain.com
  email: [email protected]
  exposer: Ingress
  issuer: letsencrypt-prod
  tls: "true"
  urltemplate: ""
kind: ConfigMap

For the rest, we can pass in some flags. Here's the full final Jenkinsfile stage.

stage('CI Build and push snapshot') {
  when {
    branch 'PR-*'
  }
  environment {
    TEAM = "jx"
    PREVIEW_VERSION = "0.0.0-SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER"
    PREVIEW_NAMESPACE = "$APP_NAME-$BRANCH_NAME".toLowerCase()
    HELM_RELEASE = "$PREVIEW_NAMESPACE".toLowerCase()
    FULL_PREVIEW_NAMESPACE = "$TEAM-$ORG-$APP_NAME-$BRANCH_NAME".toLowerCase()
  }
  steps {
    container('nodejs') {
      sh "export VERSION=$PREVIEW_VERSION && skaffold build -f skaffold.yaml"
      sh "jx step post build --image $DOCKER_REGISTRY/$ORG/$APP_NAME:$PREVIEW_VERSION"
      dir('./charts/preview') {
        sh "make preview"
        sh "jx preview --app $APP_NAME --dir ../.."
        sh "helm repo add jenkins-x https://storage.googleapis.com/chartmuseum.jenkins-x.io"
        sh "jx upgrade ingress --batch-mode --namespaces=$FULL_PREVIEW_NAMESPACE --services=$APP_NAME --wait-for-certs=false --skip-resources-update=true --skip-certmanager=true"
      }
    }
  }
}

And with that, the preview pull request now creates a cert after the preview environment is successfully created! It may take around 60 seconds or so for the cert to be created after the completion of the build.

There are a few caveats to this, such as when you are using a custom urlTemplate for the preview environment. In that case you'd need to adjust the environment variable I've added

FULL_PREVIEW_NAMESPACE
. Another one is if you are using the
teams
feature, you'll need to make sure you update that environment variable as well.

Happy Hacking!

Make sure to follow me on here if that's still a thing, and on twitter @pat_scott!

~ Patrick Lee Scott

If you're looking to learn for yourself, the best place to start is reading about my own journey, which I'm imagining is quite similar to yours! At the end you'll find a FREE 21 day email course called "Getting Dangerous with DevOps!"

Tags

Join Hacker Noon

Create your free account to unlock your custom reading experience.