Building a JIRA integration with Clojure & Atlassian Connect

is the market leading issue-tracking and agile project management tool, and is in heavy use at companies around the globe. Atlassian JIRA For I wanted to build a slick integration to to help my customers to search their JIRA issues alongside their Slack messages, Trello cards, files, GitHub issues and emails. CTX, my team search app JIRA When it came to actually building the integration, I was pleasantly surprised by the , and especially the and the simple integration patterns it promotes. APIs available Atlassian Connect API While there’s — and even a short , I couldn’t find a single, simple tutorial that covered Connect, the webhook interface and the main API in one place. lots of documentation blog series about integrating Bitbucket with a Clojure application JIRA So I wrote one. . The full source of the application we’re building is available on GitHub here What are we building? We’re going to build a JIRA integration in Clojure, that listens for issues being created and mutated and creates a basic activity feed of changes. To do this, we need a Clojure web application, listening on an HTTPS endpoint. We’ll register the application with JIRA, and every time a user creates or updates a ticket, JIRA will send us an event on a Webhook. We’ll record these in an in-memory database, and write a little front-end application that polls our app to display a list of changes to issues. What is Atlassian Connect? Connect provides a simple framework for integrating with Atlassian applications like JIRA and Confluence. It leverages OAuth2 and JWT, along with some simple integration patterns, to give developers a nice and consistent way to talk to the Atlassian API suite. In its’ most basic form, all you need to do is publish a Descriptor listing the endpoints you provide and the services you require, point your Atlassian application at it, and then deploy some code on the listed endpoints that talks HTTP / JSON. Pre-conditions First, install , and make sure it’s in your PATH. And get yourself a Java JDK if you haven’t got one. Leiningen Check it all works: $ lein --versionLeiningen 2.8.1 on Java 9.0.4 Java HotSpot(TM) 64-Bit Server VM Basic project setup Let’s create a basic Clojure app skeleton using the . Compojure template $ lein new compojure connect-example$ cd connect-example$ ls-rw-r--r-- 1 rory Users 273 20 Mar 16:21 README.md-rw-r--r-- 1 rory Users 483 20 Mar 16:21 project.cljdrwxr-xr-x 3 rory Users 102 20 Mar 16:21 resources drwxr-xr-x 3 rory Users 102 20 Mar 16:21 src drwxr-xr-x 3 rory Users 102 20 Mar 16:21 test$ lein ring server-headlessRetrieving lein-ring/lein-ring/0.9.7/lein-ring-0.9.7.pom from clojars ...2018-03-20 16:24:40.501:INFO:oejs.Server:jetty-7.6.13.v20130916 2018-03-20 16:24:40.550:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:3000 In a second tab, use to test it's running: curl $ curl http://localhost:3000Hello World Great! We’ve got a running app. Import basic dependencies Now we need to add some libraries to our Clojure project to let us do the basics of HTTP comms, the Connect flow and so on. Open up the file in an editor of your choice, and make it look like this: project.clj (defproject connect-example "0.1.0-SNAPSHOT" :min-lein-version "2.0.0" :dependencies [[org.clojure/clojure "1.9.0"] [compojure "1.6.0"] [ring/ring-defaults "0.3.1"] [ring/ring-json "0.4.0"] [clj-connect "0.2.4"]] :plugins [[lein-ring "0.9.7"]] :ring {:handler connect-example.handler/app} :resource-paths ["resources"] :profiles {:dev {:dependencies [[javax.servlet/servlet-api "2.5"] [ring/ring-mock "0.3.0"]]}}) The libraries in the clause give us HTTP serving ( ), routing ( ) and a simple wrapper around the Atlassian Connect API & underlying JWT technology ( ) :dependencies ring compojure clj-connect Connecting to JIRA JIRA will only talk to our application over HTTPS, so the first hurdle is how to get it on the internet. Without setting up a server, or using something like Heroku, this can be annoyingly time consuming, so here’s a simple way round it: . ngrok ngrok lets you create an externally accessible SSL endpoint to test your integration software. You just run it like this; $ ngrok tls 3000ngrok by @inconshreveable Session Status online Account Rory Gibson Version 2.2.8 Region United States (us) Web Interface http://127.0.0.1:4040 Forwarding http://9a69173f.ngrok.io -> localhost:3000 Forwarding https://9a69173f.ngrok.io -> localhost:3000Connections ttl opn rt1 rt5 p50 p90 0 0 0.00 0.00 0.00 0.00 and it will create a tunnel, and give you a URL — e.g. - you can use to access your software, running on your development machine but accessible over SSL on the internet. https://9a69173f.ngrok.io Now that we have a service that’s discoverable externally, we can generate our OAuth credentials in JIRA. Serving the Descriptor Now we’ve got all the plumbing sorted, we can serve up the Connect Descriptor. This is a simple JSON file that declares the application’s capabilities and dependencies, so that JIRA (and other Atlassian applications) know how to talk to it. Create a file at and paste the following content into it (ensure that the baseUrl property matches your ngrok URL): /resources/public/connect.json { "key": "connect-example", "name": "connect-example", "description": "Demo JIRA / Clojure integration", "baseUrl": "https://9a69173f.ngrok.io", "authentication": { "type": "jwt" }, "lifecycle": { "installed": "/installed" }, "modules": { "oauthConsumer": { "clientId": "PLACEHOLDER" }, "webhooks": [ { "event": "*", "url": "/recv" } ] }, "scopes": ["READ"]} Handling lifecycle callbacks The Connect API works by loading your descriptor, then making a call to on your app as a "lifecycle callback" (there are others, like Uninstalled, that we're ignoring here). /installed All it wants is a 200 response. We’ll use Compojure and Ring to create a simple handler that serves this function (and provides basic API plumbing) (ns connect-example.handler (:require [compojure.core :refer :all] [compojure.route :as route] [ring.middleware.defaults :refer [wrap-defaults api-defaults]] [ring.middleware.json :refer [wrap-json-body wrap-json-response]] [ring.util.response :refer [response]]))(defroutes app-routes (POST "/installed" [] {:status 200 :body "OK"}))(def app (-> app-routes wrap-json-body wrap-json-response (wrap-defaults api-defaults))) Now that we have this much, we can wire it up to JIRA in the Cloud. Save everything and restart the app ( ) lein ring server-headless Go and login to your JIRA instance and click (the cog icon) . Settings > Add-Ons Go to the tiny link at the bottom of the page. Settings and ensure that and are both ticked. Enable development mode Enable private listings Now you should see a link to — click this and paste in the fully-qualified URL to your descriptor (e.g. ) Upload add-on HTTPS https://abcd1234.ngrok.com/connect.json Handling events Once Connect has hooked up to your app, you’ll start receiving the events we specified in the Descriptor whenever they occur. All we need is some handlers and some logic on our side. Add 2 routes to the routing block — one for JIRA to call when an event happens, and one for us to call to list events. (POST "/recv" req (recv req))(GET "/events" req (response (events req))) and the functions for them to call. gets the payload, extracts the data we want and stores it in an in the server (instead of a DB, this is only a demo :) recv atom (def event-store (atom '()))(defn persist! "Save event to local in-memory structure" [e] (swap! event-store conj e) (println e) ;; debug info for development e)(defn transform "Get the fields we care about from the JIRA payload" [e1] (let [id (-> e1 :issue :key) user (-> e1 :user :displayName) timestamp (-> e1 :timestamp) type (-> e1 :webhookEvent) summary (-> e1 :issue :fields :summary)] {:user user :id id :timestamp timestamp :summary summary :type type}))(defn handle-event [raw] (-> raw clojure.walk/keywordize-keys transform persist!))(defn events [_] @event-store)(defn recv "Handler function for all events" [req] (handle-event (:body req)) {:status 200 :body "OK"}) Restart your app Hit then again. Ctrl-C lein ring server-headless If you change an Issue in your JIRA (say by moving it from To Do to Done in the agile view) you should see some output in your terminal as JIRA calls our app and we save the data, and dump it to screen with the above. println Hit the endpoint to check we're saving the data and you should see something like this: /events $ curl http://localhost:3000/events[{"summary":"My issue 1,"timestamp":1521891438142,"type":"jira:issue_updated","id":"TEST-1"}] Adding a UI That’s great! We have an inbound and outbound API. But most non-developers don’t like to use for everything, so we'd better add a UI. curl We’ll build something in ClojureScript, so that we can use the same paradigms and dialect across front and back-end codebases. $ mkdir -p src/clj$ mv src/connect-example src/clj$ mkdir -p src/cljs/connect-example$ touch resources/public/index.html Now edit your to the following (pulling in the new directories and adding the ClojureScript libraries and build configuration) project.clj (defproject connect-example "0.1.0-SNAPSHOT" :min-lein-version "2.0.0" :dependencies [[org.clojure/clojure "1.9.0"] [org.clojure/clojurescript "1.10.217"] [compojure "1.6.0"] [ring/ring-defaults "0.3.1"] [ring/ring-json "0.4.0"] [rum "0.11.2"] [clj-connect "0.2.4"] [cljs-ajax "0.7.3"]] :plugins [[lein-ring "0.9.7"] [lein-figwheel "0.5.13"]] :source-paths ["src/clj"] :resource-pahts ["resources" "target/cljsbuild"] :ring {:handler connect-example.handler/app} :profiles {:dev {:dependencies [[javax.servlet/servlet-api "2.5"] [ring/ring-mock "0.3.2"]]}} :clean-targets ^{:protect false} [:target-path "out" "resources/public/cljs"] :figwheel {:css-dirs ["resources/public/css"] :server-port 3000 :ring-handler connect-example.handler/app} :cljsbuild { :builds [{:id "dev" :source-paths ["src/cljs"] :figwheel true :compiler {:main "connect-example.core" :asset-path "cljs/out" :output-to "resources/public/cljs/main.js" :output-dir "resources/public/cljs/out"} }] }) Create and fill in your resources/public/index.html and ... src/cljs/connect-example/core.cljs (ns connect-example.core)(let [el (.getElementById js/document "content")] (set! (.-innerText el) "Hello World")) Which will, when it’s loaded, find the with ID and put an informative message into it. div content Now let’s try to run it, using the ; awesome Figwheel plugin $ lein figwheelRetrieving figwheel/figwheel/0.5.13/figwheel-0.5.13.pom from clojars Retrieving org/clojure/core.async/0.3.443/core.async-0.3.443.pom from central ...Figwheel: Validating the configuration found in project.clj Spec Warning: missing an :output-to option - you probably will want this ... Figwheel: Configuration Valid ;) Figwheel: Starting server at http://0.0.0.0:3449 Figwheel: Watching build - dev Figwheel: Cleaning build - dev Figwheel: Starting server at http://0.0.0.0:3449 Figwheel: Watching build - dev Compiling "main.js" from ["src/cljs"]... Successfully compiled "main.js" in 1.95 seconds. Launching ClojureScript REPL for build: dev Figwheel Controls: (stop-autobuild) ;; stops Figwheel autobuilder (start-autobuild [id ...]) ;; starts autobuilder focused on optional ids (switch-to-build id ...) ;; switches autobuilder to different build (reset-autobuild) ;; stops, cleans, and starts autobuilder (reload-config) ;; reloads build config and resets autobuild (build-once [id ...]) ;; builds source one time (clean-builds [id ..]) ;; deletes compiled cljs target files (print-config [id ...]) ;; prints out build configurations (fig-status) ;; displays current state of system (figwheel.client/set-autoload false) ;; will turn autoloading off (figwheel.client/set-repl-pprint false) ;; will turn pretty printing off Switch REPL build focus: :cljs/quit ;; allows you to switch REPL to another build Docs: (doc function-name-here) Exit: Control+C or :cljs/quit Results: Stored in vars *1, *2, *3, *e holds last exception objectPrompt will show when Figwheel connects to your application Load up a browser tab pointed to and you should see, in your browser, the message "Hello World". http://localhost:3000 Hello World Some React magic ClojureScript web development is heavily focussed on the React ecosystem. Several React bindings exist; one of the easiest to get started with is . Rum Add the following to the key in your : :dependencies project.clj [cljs-ajax "0.7.3"][rum "0.11.2"] Modify so it includes the for Rum and a component definition (which creates a React component under the hood), then mounts it on the document body. core.cljs require (ns connect-example.core (:require [rum.core :as rum :refer [defc mount]))(defc label [text] [:div {:class "label"} text])(mount (label "Hello again") js/document.body) Restart Figwheel (Ctrl-C in your terminal then again) and refresh your browser, and that's it - you're seeing a React rendered application! lein figwheel A simple list-based UI Now we have the basic structure to put a React UI on screen, let’s fill it in with something that’s actually useful. To match the wireframe above, we’ll have an H1 for the title: (defc heading [] [:h1 "JIRA Activity Feed"]) And a component to iterate over a list of JIRA issues — so we’ll call the display of each result an “issue”. We’ll create some dummy data to render initially, too. (defc issue [{:keys [id summary type timestamp] :as issue}] [:div.issue [:div.id id] [:div.info [:div.title summary] [:div.detail type] [:div.date timestamp]]])(defc issues [is] [:div.issues (map issue is)])(def fake-issues '({:id "FOO-1" :summary "The Title" :type "jira:issue_updated" :timestamp 123456789}))(defc content [data] [:div.container (heading) (if data (issues data) [:div "No activity found in feed."])])(mount (content fake-issues) (.getElementById js/document "content")) Save everything and reload your browser and you should see a very basic list and title. It could probably do with some styling, so let’s tweak our as follows: index.html Create a file with some simple styles: resources/public/css/site.css body { font-family: sans-serif;}.container { margin-left: auto; margin-right: auto; padding-top: 30px; width: 800px;}div.issue { font-size: 16px; width: 600px; border: solid black 1px; padding: 1em; margin-bottom: 1em;}.issue .id { font-weight: bold; float: left; width: 75px; height: 50px; text-align: center; padding: 1em .5em 0 0;}.issue .info { height: 100%;}.issue .summary { font-weight: bold; padding-bottom: 0.5em} Restart again and reload and everything should be perfect. lein figwheel NB: once you get to this stage, Figwheel’s most awesome benefit kicks in — hot auto-reload of ClojureScript and CSS files. Just save in your editor and the browser updates on its’ own! AJAX comms Now let’s get the UI to query the server side for our JIRA information… Add a new so the top of looks like this: require core.cljs (ns connect-example.core (:require [rum.core :as rum] [ajax.core :refer [GET]])) and then let’s add a function to fetch and display the data (and actually call that function to do the display) Remove the existing form and add this to the end of : mount core.cljs (defn handler [data] (mount (content data) (.getElementById js/document "content")))(fetch-issues handler) Finally, let’s get the browser app to poll the server every 5 seconds to update the feed. (js/setInterval #(fetch-issues handler) 5000) What have we got? We’ve built a dynamically-updating ClojureScript React application that’s showing live changes from JIRA. It turns out it’s surprisingly easy to integrate a Clojure app with JIRA through Atlassian Connect. Evidently, what we’ve built here is only the barest beginnings of a production app; in real life you’d want to check the validity of inbound events using the JWT functionality built into the Connect API, and you’d at least want a proper database of some sort to persist your data into. And remember — the full working source to this application is on GitHub here for your reference: https://github.com/getctx/connect-example One more thing… If you like what you see here, please check out our application, like JIRA, Trello, Slack, GitHub, Google Drive, email and more in one shot. We offer a free trial with no credit card required. Thanks! CTX, which lets you and your team search your apps Originally published at getctx.io on March 24, 2018.