Verisoulis in the Blockchain Infrastructure Industry

Verisoul

verisoul

Build a Lightning-fast SMS Reminder System With Vonage SMS API and FastAPI Backend!

Bacteria: Chapter II - BACTERIA IN WATER

Do You Know How to Fail Fast With Your Next Idea?

Introduction Recently I have decided to make one of my side projects - - public. After some initial versions, I got bored with repeating the manual steps again and again so I have realized, I need a CI/CD pipeline to do the work instead of me. webdevtools.io But where should I begin? I don’t want to run a dedicated Jenkins for this job, it would be overkill. Luckily my code is stored in GitLab so the GitLab CI/CD was given, and it is free to use for a limited amount of minutes/month. Configuring the AWS stack The project is a pretty simple web application. It does not use any server-side logic or database layer, just pure frontend code. To keep it simple I used a simple bucket to store and server the application’s code. S3 can be slow sometimes by itself and can be expensive to retrieve data on a larger scale so I used as a CDN service. It can also provide me a free SSL certification. To register my domain, I simply used . It is simple and works with CloudFront as a charm. S3 CloudFront Route 53 The stack looks like this: S3 - to store the actual code CloudFront - to make things faster R53 - as a domain name registrar To keep this article brief, I am just going to cover the GitLab CI configuration deeper. Configuring GitLab CI You can simply enable GitLab CI/CD by pushing a file to repository’s root. I will just cover my use case in this article, for further reference please read the . .gitlab-ci.yml documentation To getting started, I defined two-stage for the job. stages:
  - build
  - deploy

variables:

create_dist:
  stage: build
  image: node:12.20.2-alpine3.10
  allow_failure: false
  script: |
    echo "Hello build"
  only:
   refs:
     - master

deploy_to_storage:
  stage: deploy
  image: python:alpine
  when: manual
  allow_failure: false
  script: |
    echo "Hello deploy"
  only:
    refs:
      - master The stage will be triggered when a commit is pushed to the master branch. Since is using NPM as a package manager, I am using node:12 with alpine as a root image for the job. create_dist webdevtools.io The other stage, will only check out the master branch but it will need a manual trigger to run, to avoid unwanted deployments. It will use as a base image because I want to utilize the AWS CLI pip package. deploy_to_storage python:alpine Prerequisities To access AWS with GitLab CI, you have to create an IAM user with programmatic access type and with the required permissions. I have added and rights to my IAM user. After the user is created, you will receive an id and a secret. You should save them to GitLab in the Settings/CI/CD/Variables menu. You have to declare these variables in the variables section in your build script. AmazonS3FullAccess CloudFrontFullAccess To create a you have to a add an Access Token to you profile in Settings/Access Tokens. After it is done, simply add it to the variables like the AWS access keys. GITLAB_ACCESS_TOKEN variables:
  APP_NAME: ${CI_PROJECT_NAME}
  S3_BUCKET: ${AWS_BUCKET_NAME}
  CDN_DISTRIBUTION_ID: ${CLOUDFRONT_DIST_ID}
  AWS_ID: ${MY_AWS_ID}
  AWS_ACCESS_KEY_ID: ${MY_AWS_ACCESS_KEY_ID}
  AWS_SECRET_ACCESS_KEY: ${MY_AWS_SECRET_ACCESS_KEY}
  AWS_REGION: ${AWS_REGION_NAME} Let’s build it The build will be triggered inside the job, the actual build script will look like this. create_dist First I am setting up the environment dependencies. After that, I am creating a git tag with the version number from the and then I am installing the npm packages. As a next step, I just simply building a distribution with . package.json ng build --prod After that, I just simply create a new tarball with the build output in it and I upload the output to GitLabs’s package registry as a generic package. With this step, I will have a nice collection of previous versions. create_dist:
  stage: build
  image: node:12.20.2-alpine3.10
  allow_failure: false
  script: |
    echo "Installing curl"
    apk --no-cache add curl

    echo "Installing JQ"
    apk --no-cache add jq

    echo "Installing git"
    apk --no-cache add git

    echo "Creating version"
    APP_VERSION=$(cat ./package.json | jq -r '.version')
    echo "$APP_VERSION"

    echo "Tagging build"
    git config user.email "${GITLAB_USER_EMAIL}"
    git config user.name "${GITLAB_USER_NAME}"
    git remote add api-origin https://oauth2:${GITLAB_ACCESS_TOKEN}@gitlab.com/${CI_PROJECT_PATH}
    git tag -a "$APP_VERSION" -m "Version $APP_VERSION"
    git push api-origin "$APP_VERSION"

    echo "Installing dependencies"
    npm install -g @angular/cli@10.0.8

    echo "Installing npm modules"
    npm install

    echo "Building distribution"
    ng build --prod

    echo "Creating artifact..."
    tar -vzcf ${APP_NAME}_${APP_VERSION}.tar.gz dist
    curl --header "JOB-TOKEN: $CI_JOB_TOKEN" \
      --upload-file ${APP_NAME}_${APP_VERSION}.tar.gz \
      "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/frontend/${APP_VERSION}/${APP_NAME}_${APP_VERSION}.tar.gz"
  only:
    refs:
      - master Deploy to S3 For this stage, I am employing the as a root image for the job, because I want to utilize the AWS CLI. As a first step, I am installing this and after that, I am downloading the previously created package from GitLab’s package registry. To start with a clean sheet I am deleting the previous version from the S3 bucket before I am uploading the new version.
After the upload, I have to invalidate the CloudFront distribution. Without this step, caching issues can occur. pyton:alpine deploy_to_storage:
  stage: deploy
  image: python:alpine
  when: manual
  allow_failure: false
  script: |
    pip install awscli

    echo "Unpacking artifact"
    APP_VERSION=$(cat ./package.json |  python3 -c "import sys, json; print(json.load(sys.stdin)['version'])")
    wget --header="JOB-TOKEN: $CI_JOB_TOKEN" \
      ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/frontend/${APP_VERSION}/${APP_NAME}_${APP_VERSION}.tar.gz
    mkdir ./package
    tar -vxzf  ${APP_NAME}_${APP_VERSION}.tar.gz  --directory ./package

    echo "Delete previous version"
    aws s3 rm s3://${S3_BUCKET} --recursive --region ${AWS_REGION}

    echo "Uploading verison to S3..."
    aws s3 cp ./package/dist/dev-tools s3://${S3_BUCKET}/ --recursive --region ${AWS_REGION}

    echo "Creating CDN invalidation"
    aws cloudfront create-invalidation \
          --distribution-id ${CDN_DISTRIBUTION_ID} \
          --paths "/*"
  only:
    refs:
      - master To sum up As you can see, you can speed up your deployment processes with these easy steps. I hope you found it useful, and I could help you to speed up your deployment. If you have any questions or suggestions, please leave a comment. If you are interested, you can find the while script . here

CI/CD Pipelines on GitLab and AWS: Speed Up Your Deployments

How Can We Democratize Publishing Online?

Sentiment Analysis using Deep Learning techniques with India Elections 2019 — A Case study

If you have not read the first blog on the why, how, and hope of this series, check out the first here Python 2 vs Python 3 🐍: Which one?!?! I think the biggest reason why I have chosen Python 3 over Python 2 is that starting this year, there will be no maintaining of Python 2 and almost no more bug fixes (see more here on Python’s official ). Thus, I’ve decided to go ahead and code in Python 3 for the remainder of this series. Now, with that comes an interesting question: what’s changed between versions? What I’ve seen on the internet and many of blog posts, it’s a few things that come to mind: documentation Hello World print( )
Hello World # Python 2 verison of print >>> print "Hello World" # Python 3 version of print - print is now a function >>> "Hello World" print statements in Python3 a = range( , ) % a
[ , , , ,... ] b = xrange( , ) print(b)
xrange( , ) i b: % i
[ , , , ,... ] c = range( , ) print(c)
range( , ) new_c = list(c) print(new_c)
[ , , , ,... ] #Python 2 - range([start - optional], stop, [step - optional]) vs. xrange([start - optional], stop, [step - optional]) #range gives you a python list of values that you can iterate through immediately >>> 1 100 >>> print "%s" 0 1 2 3 .99 #xrange gives you back an object that evaluates lazily 👉🏾 (as you need it/on-demand) - good for memory >>> 1 100 >>> 1 100 >>> for in >>> print "%d" 0 1 2 3 .99 #Python 3 - there is no more xrange! Just 1 range to rule them all (range in Python 3 includes xrange) >>> 1 100 >>> 1 100 # if you want to make a list out of the range object >>> >>> 0 1 2 3 .99 range vs xrange in Python -> see more explanation at this stack Overflow post 👉🏾 https://bit.ly/2SPX4c5 num1 = / num2 = / print(num2) #Python 2 - watch out for values that evaluate to float data types! >>> 1 2 >>> print "num" 0 #Python 3 - you'll get a float value >>> 1 2 >>> 0.5 float handling in Python3 While this is not at all an exhaustive , I hope this gives an idea of somethings that have changed from Python 2 to Python 3. Ultimately, if you are working on a legacy system of some sort, you might have to switch to Python 2 so the best case scenario is to be comfortable with both versions 😃. list What’s a Class? Seeing as the blog posts will focus on the understanding and implementation of data structures and algorithms, I think it’s important to explain a fundamental part of any object-oriented programming language, and that’s . Rather than give you a definition first, it may be easier to think about a picture. Let’s think about your standard calculator. classes Photo by on StellrWeb Unsplash A calculator has many different that a person has access to. A person can come to the calculator with their given numbers and add, subtract, multiply, divide and (depending on if it’s a fancy calculator) also plot graphs, utilize charts, and much more. A class in python is no different than a calculator. functions You can think of a class as (the calculator’s job is not to tell you the weather 👉🏾 it’s to crunch numbers) (your finger would be the object, in this case, accessing the functionality of the calculator, providing the input in numbers and operations and the calculator gives you the desired output). an interface with defined functionality and attributes to be accessed through an object This can take you down a rabbit hole of examples when you think it through (a car can be considered a class, the remote you use for you tv can be considered a class, the possibilities and internet are endless with examples 😂). We’ll choose to stay above ground and see what this simple example looks like in code 😃 Building a Calculator From Scratch self.num1 = num1
    self.num2 = num2 : class Calculator : def __init__ (self, num1, num2) Consider this like the laying of groundwork for the Class The def __init__ is considered to initialize the class¹. Most, if not all classes have this at the top of the class file. You might be wondering about the in the argument list. This took me a while to figure out when first learning object-oriented programming but in a nutshell, the self is “used to being created at the moment” of initialization¹. Num1 and Num2 are both considered as attributes and more so represent how you can extend your class (you don’t have to use the same two numbers every single time you use your calculator). self refer to the object (n1 + n2) (n1 - n2) (n1 * n2) (n1/n2) : def add (self, n1, n2) return : def subtract (self, n1, n2) return : def multiply (self, n1, n2) return #remember in Python3 from above, you'll get out floats if you pass two integers : def divide (self, n1, n2) return Rest of the functionality of the Calculator class The add, subtract, multiply, and divide functions above would be considered the actual “buttons” that a user of the calculator would physically press for the desired usage. These functions work in the exact same way! Once you create your object, you can then access the class functions via the object as you can see below 👇🏾 self.num1 = num1
    self.num2 = num2 (self.num1 + self.num2) (self.num1 - self.num2) (self.num1 * self.num2) (self.num1/self.num2) : class Calculator : def __init__ (self, num1, num2) : def add (self) return : def subtract (self) return : def multiply (self) return : def divide (self) return Final Calculator.py file you would save cal_object1 = Calculator( , ) cal_object1.add() cal_object1.multiply() cal_object1.subtract() cal_object1.divide() cal_object1.num1 cal_object1.num2 cal_object2 = Calculator( , ) cal_object2.add() # assuming you have saved the above code to a .py file in the same directory of launching python interpreter or ipython >>> 4 5 #cal_object1 is now the object that you will use to access the functions/methods in the Calculator class >>> 9 >>> 20 >>> -1 >>> 0.8 >>> 4 >>> 5 # The flexible part about creating a class is assigning different attributes (or in our case numbers) # all you have to do is create a new object! >>> 10 20 >>> 30 Object creation using Calculator.py You Did It! GIF from https://giphy.com/ If this example makes sense and you are solid on the information, you did it! Pat yourself on the back 😃. Being comfortable with classes moving through the series is very important as the data structures and algorithms that we will be invoking are not native to Python (or really any other language), therefore they will need to be built before implemented. Seeing as the only way to get better is through practice, I will be posting some programming problems sometime next week via online flashcards that I will share publicly so you can practice your programming anywhere. Until next time! 👋🏾 If your interest was peaked and/or have been in encouraged/helped by this blog post, follow me on Twitter ( )! @jeff_ridgeway4 References: [1] Shovic, John C, and Alan Simpson. “Doing Python with Class.” , John Wiley & Sons, Inc., 2019, pp. 213–220. Python All-In-One For Dummies Photo by on Hitesh Choudhary Unsplash Previously published at https://medium.com/@walkingtruth146/re-learning-data-structures-and-algorithms-series-python-3-classes-95095e1c70c7

Re-Learning Data Structures and Algorithms Series: Python 3 And Classes

How to know which feedback to listen to for your startup. If you’re starting a new business, launching a new idea, or doing anything at all really, you’re going to be getting a lot of feedback. People are going to tell you if they love it. And you’re inevitably going to hear some verison of: “That’s not a good idea.” The fact is, you’re probably going to hear that one a lot, and from a lot of different people. Some of them perfectly well-meaning. So, how do we determine which feedback matters the most? When we’re overwhelmed with messages about our ideas being too risky or not good enough, how do we filter out the noise and get to the feedback that really matters? How do we know whether to listen to the people saying yes, or the people saying no? I’ve dealt with this problem a lot. Over the course of my entrepreneurial journey, I’ve had more than a few ideas. Some of them were winners. Many, not so much. Some I never really got the chance to find out if they would be successful until it was too late. Some I left behind because I listened to the feedback. wrong This is the story of an idea that I wish I’d done more with, and how it taught me which feedback to listen to, and which to ignore. Looking back at this, I realize what I should have done differently (and that I potentially could have made a lot of money!) had I only known then what I’m telling you now. One weekend, my buddy Krish managed to get us a spot in a lean startup machine workshop. Normally, this would have cost us a few thousand dollars. Somehow though, he managed to get us in for free. For three days, we’d be attending this workshop and working on our own ideas for a startup. I was pretty excited for this opportunity. I'd recently read The Lean Startup, and was interested in getting more hands on with it. Inside the workshop, we divided up into groups of three to five people. Each group was tasked with coming up with a potential startup idea. Once we had one, it was time to hit the streets. Having the idea wasn’t enough. We needed proof that our idea had value, and we needed to get that by getting people to sign onto it in one form or another. We needed to hear from our potential customers. Now, a lot of the people in my group lived in the upper east side. This workshop? It was in downtown Manhattan. This was before the days of Uber. We had two options for getting there. Either we cough up the money for a expensive cab ride, or we use the subway for a much cheaper, but also much slower, commute. very This was the problem that we tackled with our startup idea. We called our solution “CabPool.” A clever twist on the phrase “carpool.” People would use an app to find others going to the same place as them and share a cab, splitting the cost. very If you’re drawing parallels between what we were thinking about and rideshare apps? Trust me, I’ve been doing that for a while, too. When it came time to test our idea, we planted ourselves in front of a Whole Foods market. The plan was simple. We’d wait to see someone hailing a cab and ask them if they’d be interested in a program like ours. We even had a little application. When someone said they were interested, we’d ask them where they were going so we could find a match for them. Sort of. For the purposes of this test, I was the person waiting for the ride, and trust me, I just so happened to be going the same direction. They’d be matched up with me, I’d hop in the cab, and off we’d go. We made small talk along the way, and then, at the end of the ride, we split the cost. The people I talked to seemed to genuinely enjoy this idea. Not surprising, right? Who doesn’t want cheaper transportation? Whether that was the idea itself or just my winning personality, you be the judge. As we did this, we got people to give us money. So, when it came time for the judges to determine which group had the best idea? We were pretty sure that we were going to win. We did not, in fact, win. The judges told us that, despite the fact that we’d made some money, our plan wasn’t a good idea. It simply wasn’t going to work. The winner of the competition had proposed an app that would let people know when a parking spot opened up and where it was. I’m sure it’s a complete surprise that a lot of people in NYC were thinking about traffic, commuting, and parking. Their group hadn’t gotten anyone to give them money, like ours had. Instead, they had gotten people to sign a letter of intent to use the app. The judges decided this was the better approach. Now, I don’t want to harp on their idea, but I do feel like it is worth pointing out that a parking spot in NYC is probably only open for about 60 seconds on a good day – according to the team whom presented their findings... By the time you check an app, find an open spot, and drive to it? That spot isn’t empty anymore. That’s probably the reason we’re not seeing this kind of app around much. Nevertheless, we took the advice of the judges. After all, they were experts on these things, right? This had only been a weekend idea for us, not a serious venture. And the judges had told us it hadn’t been good enough. Why continue on with it? Obviously, given the success of rideshare apps, we probably should have ignored them. The judges, the experts on the issue? They were the feedback we chose to listen to. They weren’t the feedback we have been listening to, though. should The customers that we had over the course of the trial had been giving us feedback too. They’d been doing it with their wallets, and they’d been doing it in a positive way. was the feedback we should have been listening to. That Our customers voted with their wallets, and we decided to listen to the people who had no financial interest in the matter. When you launch a new idea, that idea is always going to have some detractors. Those detractors might even be the experts. They might tell you that your idea isn’t a good one. That there isn’t a market for what you’re thinking of doing. That data? It really isn’t all that useful. The useful data comes from your customers, and how they choose to vote with their wallets. We had people voting yes, putting their dollars down on our idea. We didn’t listen to them. We listened to the experts instead. That was our mistake. Don’t let it be yours. When you’re starting something new, the main feedback you need to listen to is the one that comes from the customers. Are they willing to spend money on what you’re proposing? That answer matters more than any other, and is going to really tell you if your idea can work or not. When people vote with their wallets, listen to them. Do you have an idea or product that you want to turn into a full-fledged business? Click here to learn more about me and my agency, Unbounded, and how I can put my experience to work for you. Images licensed by Adobe Stock Photos. First image by fuzzbones . Second image by Семен Саливанчук

Have You Ever Been Told Your Business Idea Sucks?

is the market leading issue-tracking and agile project management tool, and is in heavy use at companies around the globe. Atlassian JIRA For I wanted to build a slick integration to to help my customers to search their JIRA issues alongside their Slack messages, Trello cards, files, GitHub issues and emails. CTX, my team search app JIRA When it came to actually building the integration, I was pleasantly surprised by the , and especially the and the simple integration patterns it promotes. APIs available Atlassian Connect API While there’s — and even a short , I couldn’t find a single, simple tutorial that covered Connect, the webhook interface and the main API in one place. lots of documentation blog series about integrating Bitbucket with a Clojure application JIRA So I wrote one. . The full source of the application we’re building is available on GitHub here What are we building? We’re going to build a JIRA integration in Clojure, that listens for issues being created and mutated and creates a basic activity feed of changes. To do this, we need a Clojure web application, listening on an HTTPS endpoint.  We’ll register the application with JIRA, and every time a user creates or updates a ticket, JIRA will send us an event on a Webhook. We’ll record these in an in-memory database, and write a little front-end application that polls our app to display a list of changes to issues. What is Atlassian Connect? Connect provides a simple framework for integrating with Atlassian applications like JIRA and Confluence. It leverages OAuth2 and JWT, along with some simple integration patterns, to give developers a nice and consistent way to talk to the Atlassian API suite. In its’ most basic form, all you need to do is publish a Descriptor listing the endpoints you provide and the services you require, point your Atlassian application at it, and then deploy some code on the listed endpoints that talks HTTP / JSON. Pre-conditions First, install , and make sure it’s in your PATH.  And get yourself a Java JDK if you haven’t got one. Leiningen Check it all works: $ lein --versionLeiningen 2.8.1 on Java 9.0.4 Java HotSpot(TM) 64-Bit Server VM Basic project setup Let’s create a basic Clojure app skeleton using the . Compojure template $ lein new compojure connect-example$ cd connect-example$ ls-rw-r--r--  1 rory  Users  273 20 Mar 16:21 README.md-rw-r--r--  1 rory  Users  483 20 Mar 16:21 project.cljdrwxr-xr-x  3 rory  Users  102 20 Mar 16:21 resources  drwxr-xr-x  3 rory  Users  102 20 Mar 16:21 src  drwxr-xr-x  3 rory  Users  102 20 Mar 16:21 test$ lein ring server-headlessRetrieving lein-ring/lein-ring/0.9.7/lein-ring-0.9.7.pom from clojars  ...2018-03-20 16:24:40.501:INFO:oejs.Server:jetty-7.6.13.v20130916  2018-03-20 16:24:40.550:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:3000 In a second tab, use to test it's running: curl $ curl http://localhost:3000Hello World Great! We’ve got a running app. Import basic dependencies Now we need to add some libraries to our Clojure project to let us do the basics of HTTP comms, the Connect flow and so on. Open up the file in an editor of your choice, and make it look like this: project.clj (defproject connect-example "0.1.0-SNAPSHOT"  :min-lein-version "2.0.0"  :dependencies [[org.clojure/clojure "1.9.0"]                 [compojure "1.6.0"]                 [ring/ring-defaults "0.3.1"]                 [ring/ring-json "0.4.0"]                 [clj-connect "0.2.4"]]  :plugins [[lein-ring "0.9.7"]]  :ring {:handler connect-example.handler/app}  :resource-paths ["resources"]  :profiles  {:dev {:dependencies [[javax.servlet/servlet-api "2.5"]                        [ring/ring-mock "0.3.0"]]}}) The libraries in the clause give us HTTP serving ( ), routing ( ) and a simple wrapper around the Atlassian Connect API & underlying JWT technology ( ) :dependencies ring compojure clj-connect Connecting to JIRA JIRA will only talk to our application over HTTPS, so the first hurdle is how to get it on the internet. Without setting up a server, or using something like Heroku, this can be annoyingly time consuming, so here’s a simple way round it: . ngrok ngrok lets you create an externally accessible SSL endpoint to test your integration software. You just run it like this; $ ngrok tls 3000ngrok by @inconshreveable    Session Status                online  Account                       Rory Gibson  Version                       2.2.8  Region                        United States (us)  Web Interface                 http://127.0.0.1:4040  Forwarding                    http://9a69173f.ngrok.io -> localhost:3000  Forwarding                    https://9a69173f.ngrok.io -> localhost:3000Connections                   ttl     opn     rt1     rt5     p50     p90                                0       0       0.00    0.00    0.00    0.00 and it will create a tunnel, and give you a URL — e.g. - you can use to access your software, running on your development machine but accessible over SSL on the internet. https://9a69173f.ngrok.io Now that we have a service that’s discoverable externally, we can generate our OAuth credentials in JIRA. Serving the Descriptor Now we’ve got all the plumbing sorted, we can serve up the Connect Descriptor. This is a simple JSON file that declares the application’s capabilities and dependencies, so that JIRA (and other Atlassian applications) know how to talk to it. Create a file at and paste the following content into it (ensure that the baseUrl property matches your ngrok URL): <project>/resources/public/connect.json {    "key": "connect-example",    "name": "connect-example",    "description": "Demo JIRA / Clojure integration",    "baseUrl": "https://9a69173f.ngrok.io",    "authentication": {        "type": "jwt"    },    "lifecycle": {        "installed": "/installed"    },    "modules": {        "oauthConsumer": {            "clientId": "PLACEHOLDER"        },        "webhooks": [            {                "event": "*",                "url": "/recv"            }        ]            },    "scopes": ["READ"]} Handling lifecycle callbacks The Connect API works by loading your descriptor, then making a call to on your app as a "lifecycle callback" (there are others, like Uninstalled, that we're ignoring here). /installed All it wants is a 200 response. We’ll use Compojure and Ring to create a simple handler that serves this function (and provides basic API plumbing) (ns connect-example.handler  (:require [compojure.core :refer :all]            [compojure.route :as route]            [ring.middleware.defaults :refer [wrap-defaults api-defaults]]            [ring.middleware.json :refer [wrap-json-body wrap-json-response]]            [ring.util.response :refer [response]]))(defroutes app-routes  (POST "/installed" [] {:status 200 :body "OK"}))(def app  (-> app-routes    wrap-json-body    wrap-json-response    (wrap-defaults api-defaults))) Now that we have this much, we can wire it up to JIRA in the Cloud. Save everything and restart the app ( ) lein ring server-headless Go and login to your JIRA instance and click (the cog icon) . Settings > Add-Ons Go to the tiny link at the bottom of the page. Settings and ensure that and are both ticked. Enable development mode Enable private listings Now you should see a link to — click this and paste in the fully-qualified URL to your descriptor (e.g. ) Upload add-on HTTPS https://abcd1234.ngrok.com/connect.json Handling events Once Connect has hooked up to your app, you’ll start receiving the events we specified in the Descriptor whenever they occur.  All we need is some handlers and some logic on our side. Add 2 routes to the routing block — one for JIRA to call when an event happens, and one for us to call to list events. (POST "/recv" req (recv req))(GET "/events" req (response (events req))) and the functions for them to call. gets the payload, extracts the data we want and stores it in an in the server (instead of a DB, this is only a demo :) recv atom (def event-store (atom '()))(defn persist!  "Save event to local in-memory structure"  [e]  (swap! event-store conj e)  (println e) ;; debug info for development  e)(defn transform  "Get the fields we care about from the JIRA payload"  [e1]  (let [id        (-> e1 :issue :key)        user      (-> e1 :user :displayName)        timestamp (-> e1 :timestamp)        type      (-> e1 :webhookEvent)        summary   (-> e1 :issue :fields :summary)]    {:user user :id id :timestamp timestamp :summary summary :type type}))(defn handle-event  [raw]  (-> raw    clojure.walk/keywordize-keys    transform    persist!))(defn events  [_]  @event-store)(defn recv  "Handler function for all events"  [req]  (handle-event (:body req))  {:status 200 :body "OK"}) Restart your app Hit then again. Ctrl-C lein ring server-headless If you change an Issue in your JIRA (say by moving it from To Do to Done in the agile view) you should see some output in your terminal as JIRA calls our app and we save the data, and dump it to screen with the above. println Hit the endpoint to check we're saving the data and you should see something like this: /events $ curl http://localhost:3000/events[{"summary":"My issue 1,"timestamp":1521891438142,"type":"jira:issue_updated","id":"TEST-1"}] Adding a UI That’s great! We have an inbound and outbound API.  But most non-developers don’t like to use for everything, so we'd better add a UI. curl We’ll build something in ClojureScript, so that we can use the same paradigms and dialect across front and back-end codebases. $ mkdir -p src/clj$ mv src/connect-example src/clj$ mkdir -p src/cljs/connect-example$ touch resources/public/index.html Now edit your to the following (pulling in the new directories and adding the ClojureScript libraries and build configuration) project.clj (defproject connect-example "0.1.0-SNAPSHOT"  :min-lein-version "2.0.0"  :dependencies [[org.clojure/clojure "1.9.0"]                 [org.clojure/clojurescript "1.10.217"]                 [compojure "1.6.0"]                 [ring/ring-defaults "0.3.1"]                 [ring/ring-json "0.4.0"]                 [rum "0.11.2"]                 [clj-connect "0.2.4"]                 [cljs-ajax "0.7.3"]]  :plugins [[lein-ring "0.9.7"]            [lein-figwheel "0.5.13"]]  :source-paths ["src/clj"]  :resource-pahts ["resources" "target/cljsbuild"]  :ring {:handler connect-example.handler/app}  :profiles  {:dev {:dependencies [[javax.servlet/servlet-api "2.5"]                        [ring/ring-mock "0.3.2"]]}}  :clean-targets ^{:protect false} [:target-path "out" "resources/public/cljs"]  :figwheel {:css-dirs ["resources/public/css"]             :server-port 3000             :ring-handler connect-example.handler/app}  :cljsbuild {              :builds [{:id "dev"                        :source-paths ["src/cljs"]                        :figwheel true                        :compiler {:main "connect-example.core"                                   :asset-path "cljs/out"                                   :output-to  "resources/public/cljs/main.js"                                   :output-dir "resources/public/cljs/out"}                        }]              }) Create and fill in your resources/public/index.html <!DOCTYPE html>  <html>    <head></head>  <body>    <div id="content"></div>    <script src="app.js" type="text/javascript"></script>  </body></html> and ... src/cljs/connect-example/core.cljs (ns connect-example.core)(let [el (.getElementById js/document "content")]  (set! (.-innerText el) "Hello World")) Which will, when it’s loaded, find the with ID and put an informative message into it. div content Now let’s try to run it, using the ; awesome Figwheel plugin $ lein figwheelRetrieving figwheel/figwheel/0.5.13/figwheel-0.5.13.pom from clojars  Retrieving org/clojure/core.async/0.3.443/core.async-0.3.443.pom from central  ...Figwheel: Validating the configuration found in project.clj  Spec Warning:  missing an :output-to option - you probably will want this ...  Figwheel: Configuration Valid ;)  Figwheel: Starting server at http://0.0.0.0:3449  Figwheel: Watching build - dev  Figwheel: Cleaning build - dev  Figwheel: Starting server at http://0.0.0.0:3449  Figwheel: Watching build - dev  Compiling "main.js" from ["src/cljs"]...  Successfully compiled "main.js" in 1.95 seconds.  Launching ClojureScript REPL for build: dev  Figwheel Controls:            (stop-autobuild)                ;; stops Figwheel autobuilder          (start-autobuild [id ...])      ;; starts autobuilder focused on optional ids          (switch-to-build id ...)        ;; switches autobuilder to different build          (reset-autobuild)               ;; stops, cleans, and starts autobuilder          (reload-config)                 ;; reloads build config and resets autobuild          (build-once [id ...])           ;; builds source one time          (clean-builds [id ..])          ;; deletes compiled cljs target files          (print-config [id ...])         ;; prints out build configurations          (fig-status)                    ;; displays current state of system          (figwheel.client/set-autoload false)    ;; will turn autoloading off          (figwheel.client/set-repl-pprint false) ;; will turn pretty printing off  Switch REPL build focus:          :cljs/quit                      ;; allows you to switch REPL to another build    Docs: (doc function-name-here)    Exit: Control+C or :cljs/quit Results: Stored in vars *1, *2, *3, *e holds last exception objectPrompt will show when Figwheel connects to your application Load up a browser tab pointed to and you should see, in your browser, the message "Hello World". http://localhost:3000 Hello World Some React magic ClojureScript web development is heavily focussed on the React ecosystem. Several React bindings exist; one of the easiest to get started with is . Rum Add the following to the key in your : :dependencies project.clj [cljs-ajax "0.7.3"][rum "0.11.2"] Modify so it includes the for Rum and a component definition (which creates a React component under the hood), then mounts it on the document body. core.cljs require (ns connect-example.core  (:require [rum.core :as rum :refer [defc mount]))(defc label [text]  [:div {:class "label"} text])(mount (label "Hello again") js/document.body) Restart Figwheel (Ctrl-C in your terminal then again) and refresh your browser, and that's it - you're seeing a React rendered application! lein figwheel A simple list-based UI Now we have the basic structure to put a React UI on screen, let’s fill it in with something that’s actually useful. To match the wireframe above, we’ll have an H1 for the title: (defc heading []  [:h1 "JIRA Activity Feed"]) And a component to iterate over a list of JIRA issues — so we’ll call the display of each result an “issue”. We’ll create some dummy data to render initially, too. (defc issue  [{:keys [id summary type timestamp] :as issue}]  [:div.issue   [:div.id id]   [:div.info    [:div.title summary]    [:div.detail type]    [:div.date timestamp]]])(defc issues  [is]  [:div.issues   (map issue is)])(def fake-issues '({:id "FOO-1" :summary "The Title" :type "jira:issue_updated" :timestamp 123456789}))(defc content  [data]  [:div.container   (heading)   (if data     (issues data)     [:div "No activity found in feed."])])(mount (content fake-issues) (.getElementById js/document "content")) Save everything and reload your browser and you should see a very basic list and title. It could probably do with some styling, so let’s tweak our as follows: index.html <!DOCTYPE html>  <html>    <head>    <link rel="stylesheet" href="/site.css" />  </head>  <body>    <div id="content"></div>    <script src="cljs/main.js" type="text/javascript"></script>  </body></html> Create a file with some simple styles: resources/public/css/site.css body {    font-family: sans-serif;}.container {  margin-left: auto;  margin-right: auto;  padding-top: 30px;  width: 800px;}div.issue {    font-size: 16px;  width: 600px;  border: solid black 1px;  padding: 1em;  margin-bottom: 1em;}.issue .id {  font-weight: bold;  float: left;  width: 75px;  height: 50px;  text-align: center;  padding: 1em .5em 0 0;}.issue .info {  height: 100%;}.issue .summary {  font-weight: bold;  padding-bottom: 0.5em} Restart again and reload and everything should be perfect. lein figwheel NB: once you get to this stage, Figwheel’s most awesome benefit kicks in — hot auto-reload of ClojureScript and CSS files. Just save in your editor and the browser updates on its’ own! AJAX comms Now let’s get the UI to query the server side for our JIRA information… Add a new so the top of looks like this: require core.cljs (ns connect-example.core  (:require [rum.core :as rum]            [ajax.core :refer [GET]])) and then let’s add a function to fetch and display the data (and actually call that function to do the display) Remove the existing form and add this to the end of : mount core.cljs (defn handler [data]  (mount (content data) (.getElementById js/document "content")))(fetch-issues handler) Finally, let’s get the browser app to poll the server every 5 seconds to update the feed. (js/setInterval  #(fetch-issues handler)  5000) What have we got? We’ve built a dynamically-updating ClojureScript React application that’s showing live changes from JIRA. It turns out it’s surprisingly easy to integrate a Clojure app with JIRA through Atlassian Connect. Evidently, what we’ve built here is only the barest beginnings of a production app; in real life you’d want to check the validity of inbound events using the JWT functionality built into the Connect API, and you’d at least want a proper database of some sort to persist your data into. And remember — the full working source to this application is on GitHub here for your reference: https://github.com/getctx/connect-example One more thing… If you like what you see here, please check out our application, like JIRA, Trello, Slack, GitHub, Google Drive, email and more in one shot. We offer a free trial with no credit card required.  Thanks! CTX, which lets you and your team search your apps Originally published at getctx.io on March 24, 2018.

Building a JIRA integration with Clojure & Atlassian Connect

KC Chng (CEO) ETC Refund We are happy that the refund process has executed as expected. To any users who still have issues redeeming their rightful ownership of ETC due to the unintended consequences of the hard fork, please ping us on Slack or reddit, and we will do our best to respond to you as soon as we can. Some of the common issues encountered and clarifications: not enough ETC gas for calling the redemption function. Please make sure you have enough ETC to do so. DGD-Redemption tokens are distributed to the same address (where DGDs are kept on the main chain at block #3,800,000) on the classic chain. When you generate your private keys on the main chain, you also have ownership of the same keys and public address on the classic chain. If you are using Mew with a Trezor or Ledger Nano , please make sure your HD Derivation path is similar on the ETC chain as it is on the ETH chain. Refund will be ongoing for a year before DGD token holders decide what to do with the unclaimed balances. Please see the guide if you have not already done so at https://github.com/DigixGlobal/etc-redemption/blob/master/guide/GUIDE.md Regular development updates will resume as the main message for the past 2 weeks was to have everyone aware and focused on the ETC refund. We have also included what was done for the past 2 weeks as well in this week’s update. Path forward Concurrently with the security audit done by Smartpool founders, we are working on integrating the front end of our marketplace with our core contracts. We expect minor changes, if not, none, to our core contracts. Digix has also begun looking for additional core developers to join our team. Stay tuned for further announcement of new hires within the next 2 months. Several new partnerships are in the works in preparation for the launch of our marketplace. We will follow up with some new announcements this coming week. Community building efforts Shaun Djie, Digix’s CCO, took the time to present Digix’s journey and tech stack at a Microsoft’s Start Up Day sponsored event last week at BASH@71 Ayer Rajah. We were very happy to see that the number of blockchain enthusiasts growing in Singapore. http://www.spiffy.sg/startup/leveraging-blockchain-technologies-with-digix-global-at-bash-chope-microsoft-startup-days/ We also hosted Indorse and Status.im at the SGI offices to a full turnout last week. At SGInnovate This coming monday, Michael Wheuler from & would be presenting their product at a ConsenSys Asia Pacific sponsored event. Please catch the live stream at INFURA MetaMask https://www.youtube.com/watch?v=N784-1G6ZZY&ab_channel=DigixGlobal Anthony Eufemio (CTO) Security Audits The security audits are currently ongoing with . We have been in private discussions about the codebase and working closely together to complete this very important step before our launch. Smartpool.io Code Quality Audits We are going to be engaging DappHub to perform a separate audit of our codebase that deals with the code quality. The focus of this audit is not to find security vulnerabilities which is the scope of the security audits performed by SmartPool but to ensure that our codebase is coherent, has good test coverage, and bug free. Marketplace Update This week we will be describing the set of user stories that will be needed to complete the development of the Digix Gold Marketplace. The Marketplace consists of a few server side APIs that allow us to handle customer physical gold bar orders, uploading and approval of KYC documents, as well as the back-office services which will be consumed by the Spectrum enhanced version of our Marketplace. Chris Hitchcott (Digix Core Dev) MAY 7 What a fun packed week it’s been. Many projects worked on, including the ETC Redemption Dry Run deployment and code review (thanks Vincent), Digix Explorer Interface (+ integration with Vendor & Custodian views), IPFS coolness and general tweaking and testing. Let’s start off with IPFS: one nice improvement is that we’re figured out a better way of storing proof of asset docs on IPFS. Admittedly kind of obvious, instead of saving the hash of the PoA document on chain, we’re now saving the hash of a JSON file containing metadata to (potentially many) other data fields, including other IPFS content hashes. This linked data structure means we get infinite storage and flexibility in terms of what is uploaded as proof of asset (such as the ability to store picture galleries, videos, etc) with infinite future-poofing and no extra Solidity. Nice pattern! One more IPFS related thing is that we’re going to be publishing deployments of spectrum on IPFS now and then as we roll out test DGX 2.0 features (like the assets explorer). Which brings me to the assets explorer: I’ve completed an MVP for the explorer view and actions in the DGX 2.0 PoA system. Multiple categories, ID searching, Pagination, Sorting, it has it all. It’s also more closely integrated with the other Vendor and Custodian views, which also had an API update, and inherits the pagination and sorting. IPFS DEPLOYED DEMO RELEASE: https://gateway.ipfs.io/ipfs/QmPFrQEtrF9gtYAMFFdGRnPw3Bos5aMPCVEeZQjTdoxBeN/ Next week I’ll be working on integrating a multisig wallet into Spectrum, not just as a dapplet, but as an *account type*, which means it’ll be cleanly integrated into just about every transaction type, and provide a really neat UI for multisig transactions, which can be initiated by any of the existing keystore types (including hardware wallets). MAY 19 This week I have been implementing the MultiSig keystore type for Spectrum, using Consensys’ contract (which is being used by the Golem team to store ~440,000+ ETH). https://github.com/ConsenSys/MultiSigWallet This feature allows for the deployment and management of this contract, and seamlessly integrates into Spectrum’s existing accounts system. Uniquely, the multisig keystore type will automatically ‘proxy’ a given transaction, automatically finding a relevant owner’s signing address (to initiate the transaction) and transforming the transaction data formatted to submit to the Multisig. This makes signing transactions via a the multisig contract wallet a lot simpler to end users, as they never have to meddle around with contract data. It also works in combination with the Ledger nano, so you can easily sign securely (if you are paranoid you could even sign via ledger via a cold account via a multisig). super insanely Whilst this integration is is functionally complete, unfortunately I’ve not been able to finish the styling for everything (still have some JSON output that needs to be tabulated) and have an event on Friday, so I will leave the polishing until Sunday (and will release another distribution on IPFS). In the meantime, please see the WIP screenshots below: New Keystore Creation Menu MultiSigs are added to keystores list like a normal account Click the account config to manage Multisig Contract Config, Owners and Transactions (paginated and categorised) When signing with the Multisig keystore type, users can select which ‘owner’ account initiates the transaction, and UI is displayed based on the selected address’ keystore type. Finally, I also had a bit of an experiment day last Sunday and came up with a very simple proof of concept called DGXi — the “i” stands for interface. This contract is a token wrapper for DGX, that allows people to hold an value that decreases in value over time, but keeps the same number — kind of the opposite of DGX, which decreases in number but remains 1:1 gold value. There are no plans to do anything with this concept as of yet, but it could be very useful for exchanges and dapps that don’t want to implement their own demurrage fee calculations (as it allows people to deposit and withdraw with a like-for-like token that simply decreases in value deterministically via fixed ‘inflation’, paying the fees on withdrawal). If anyone is interested in looking at the contract code, it’s at . https://github.com/DigixGlobal/DGXi/blob/master/contracts/DGXi.sol MAY 26 This week started by wrapping up the Multisig wallet UI talked about last week, which will now be included in future release of spectrum. I then deployed the final version of the ETC refund contract (which required a forked version of truffle-compile to disable the optimiser as per the suggestion of the code review), and made some small tweaks to the ETC redemption dapplet. It now awaits block 3800000, when the snapshot will take place. I also spent some time on a faucet service, which uses a registry on Kovan to whitelist users and enable DGD holders to withdraw a small amount of ETC in order to execute the redemption step. This service is a one time service, available to DGD holders at <enter_your_address_here> https://digixparity04.digix.io/faucet/0x JUNE 3 The majority of this week was taken up by the ETC redemption process, which included various tweaks to the UI, updating the guide, providing support and importantly actually executing the redemption process steps — deploying, capturing, minting, etc. I also made some updates to the faucet that helped DGD holders get a small amount of ETC to make their claim. I spent a small amount of time on Spectrum; implementing a neat little addition for that extra bit of security when generating private keys: the option of providing random entropy via dice rolling — optionally replacing `crypto.randomBytes`. This also implements an XOR transform to reduce the effect of biased dice. ( ) https://www.reddit.com/r/Bitcoin/comments/22nezb/just_got_my_16_sided_dice_extra_secure_paper/cgopcvx/ ## doxity ### git@github.com:DigixGlobal/doxity.git - 78d1fa1 — Merge pull request #15 from yudilevi/patch-1 Added a missing comma in the sample .doxityrc file - 76b64ac — Added a missing comma in the sample .doxityrc file ## etc-refund ### git@github.com:DigixGlobal/etc-refund.git - eb4c575 — [maint] deploy to classic - 28fbb75 — [maint] remove multisig, keep multisig integration tests - f8b964a — Merge commit ‘b143daaae1e458d2d2544cb08a03483919230d33’ - 2e9804b — [maint] disable optimizer, rebuild - 6913b0d — [maint] contract data checking - 22eb84d — [feature] spectrum: ui tweaks ## spectrum ### git@github.com:DigixGlobal/Spectrum.git - f05ebe1 — [maint] minor filesystem organisation - a449552 — [maint] ipfs dists - 82e1e71 — [maint] fix deps - 8803329 — [maint] cleanup, deps - 182f2c0 — [feature] allow changing owner in transaction - 40a0919 — [feature] multisig deployer - 0c482f5 — [feature] show network selector if web3 not passed to transaction modal - 1b99227 — [maint] random ipfs deploy ## truffle-gnosis-multisig ### git@github.com:DigixGlobal/multisig-dapplet.git - e5fb8d6 — [maint] bump verison, remove spectrum dep - 1f14474 — [maint] update readme - 0a0885b — [feature] tabulate json data in tx info - 4c8d39d — [feature] sepectrum: dynamic contract, UI tweaks ## whitelist-faucet ### git@github.com:DigixGlobal/whitelist-faucet.git - 540bfc3 — [maint] fixes - 1d2663a — [maint] fix script, redeploy - e67e728 — [maint] redeploy - 023c0eb — [wip] cleanup, text edits, fix deploy - bacb461 — [feature] allow passing `classic` to provide faucet to classic - 0540ce3 — [feature] population script - 32926bb — [feature] add onboarding script - 9a1b6cf — [wip] minor text update - f7bb43c — [feature] faucet http server - e39402b — [feature] initial commit; registry contract + tests ## core2 ### git@github.com:DigixGlobal/core2-storage-library-contracts.git - e833c710 — [test] AssetInfoStorage ## etc-refund ### git@github.com:DigixGlobal/etc-redemption.git - cea8832 — Update GUIDE.md - 8e9e724 — Update GUIDE.md - 5ee3666 — Update GUIDE.md - 0b929d9 — Update GUIDE.md - 31e1ccc — Update GUIDE.md - 955656f — Merge commit ‘b0ff47c44b58187b6ecad09acbb294353c1fd07b’ - d423f51 — [maint] update guide - b0ff47c — Update GUIDE.md - 9ff80dc — Update GUIDE.md - 7018792 — Update GUIDE.md - 4971a50 — [maint] fix time estimate - 4cf7c34 — [maint] update IPFS hash - 58fe6c4 — [maint] UI updates for live contract - 5c4dae5 — Update GUIDE.md - 748eca1 — Merge commit ‘f02dde3254fcdcdc9d6cf525f0b91ebb3df8dfb7’ - 8d38286 — [maint] add reports - f02dde3 — Update GUIDE.md - fe8c5cb — Update GUIDE.md - fdc624d — Update GUIDE.md - d720ed7 — Update GUIDE.md - 62b3083 — Update GUIDE.md - 44eece3 — Update GUIDE.md - eeadeee — [maint] update addresses - 702290f — [maint] add accounts - c60dff4 — [maint] add source code link - bece299 — [maint] fix readme link - a509167 — [maint] build, update ipfs hash - 83ff7b9 — [maint] add app to guide - 64d102f — [maint] minor ui fix + formatting - b774ac9 — [wip] comment out wallet test (not used) - 0f8f67f — [maint] fix formatting - 9654092 — [maint] add exchanges info - be66536 — [maint] add faucet info - 98ca3fe — [maint] update readme - 448a3dc — [maint] rebuild docs - 689e138 — [maint] rebuild docs, update readme, guide - cf9e2e0 — [maint] fix versions & shrinkwrap - d4405f9 — [maint] built & migrate without optimizer - e4f27e4 — [maint] more step-1 data - e072f23 — [maint] update heading text - 3d47d52 — [maint] update to use live contract, use top-up keys - 467ceeb — [maint] whitelist npm publish dirs - 49d6e00 — [maint] update readme regarding multisigs ## spectrum ### git@github.com:DigixGlobal/Spectrum.git - 5814f01 — [maint] add entropy fields for randomData generation - e4a843d — [maint] remove redemption + overlay - eee380d — [maint] build - fb0f2b6 — [maint] redeploy - c125ba6 — [maint] update token address in tokens list - b1b4c0e — [maint] new build - 200e752 — [maint] ipfs build - 71f74cf — [maint] bump etc-redemption and build - 0b3c677 — [maint] update overlay, ipfs deploy - 9493503 — [maint] fixed deps and added shrinkwrap - c94a0e6 — [maint] ipfs deploys - eae25aa — [maint] add ipfs deploy script - 960f4fc — [maint] publish IPFS dists - 0683f2b — [maint] only use ETC - eee0d28 — [maint] fix linked deps, whitelist published to `src` ## whitelist-faucet ### git@github.com:DigixGlobal/whitelist-faucet.git - dcf140e — [maint] logging - af68d4b — [hotfix] temp hardcode the list - 9cf6865 — [hotfix] temp fix for faucet - 9135a69 — [maint] use kovan - c43bcba — [bugfix] set gas - f3e150a — [maint] redeploy - 24f47b7 — [maint] make populating much more efficient

Digix Dev Update Jun 3 2017 — Unabridged

You might not be aware of it, but the engine stores the credentials you use with in plain text in a file called . Well, granted, it’s not ASCII, it’s base64, which may look like encryption on first glance, . Base64 is a scheme for encoding, not encryption. You can simply copy the base64 string and convert it to ASCII in a matter of seconds. docker docker login $HOME/.docker/config.json but it’s not OMG, I’m strongly concerned about the secrecy of my passwords, what should I do? I’m glad you asked. Thankfully, docker offers , which will use your operating system’s native store instead of . credential helpers credential config.json containing the helper for your operating system from and . Don’t forget to . Download the archive Github unpack it to **/usr/local/bin** make it executable If you are on this should work for you to install verison 0.4.2 of the helper (you will need to be root): Linux wget && tar -xf && chmod +x docker-credential-secretservice && mv docker-credential-secretservice /usr/local/bin/ https://github.com/docker/docker-credential-helpers/releases/download/v0.4.2/docker-credential-secretservice-v0.4.2-amd64.tar.gz docker-credential-secretservice-v0.4.2-amd64.tar.gz Once that’s done, key to the JSON object’s root. The . This is basically the word after the last hyphen of the executable you just installed. For Linux, the executable is called , so the service name is . open **~/.docker/config.json** and add a **credsStore** value will be the name of the credential storage service docker-credential-secretservice secretservice If you are on and trust my skills you could try and run Linux sed sed -i '0,/{/s/{/{\n\t"credsStore": "secretservice",/' ~/.docker/config.json From this point forward, docker will use the external service to store your credentials for . To get rid of the ones that are already in the file, simply . docker login <url> docker logout <url> **Reference** Official Documentation on docker login. is how hackers start their afternoons. We’re a part of the family. We are now and happy to opportunities. Hacker Noon @AMI accepting submissions discuss advertising & sponsorship To learn more, , , or simply, read our about page like/message us on Facebook tweet/DM @HackerNoon. If you enjoyed this story, we recommend reading our and . Until next time, don’t take the realities of the world for granted! latest tech stories trending tech stories

Getting rid of Docker plain text credentials

Execution is responsible for executing the bytecode. In this paper, we’ll talk about the workflow from S to E and several kernel functions. Our test case: function JsPrint(a){
	if(a >5){
		return "Debug";
	}
}
console.log(JsPrint(6)); Workflow The GetSharedFunctionInfoForScript() compiles JS source code into bytecodes. 0.  MaybeHandle<SharedFunctionInfo> Compiler::GetSharedFunctionInfoForScript(
1.      Isolate* isolate, Handle<String> source,
2.      const Compiler::ScriptDetails& script_details,
3.      ScriptOriginOptions origin_options, v8::Extension* extension,
4.      ScriptData* cached_data, ScriptCompiler::CompileOptions compile_options,
5.      ScriptCompiler::NoCacheReason no_cache_reason, NativesFlag natives) {
6.  //.omit.............
7.    // Do a lookup in the compilation cache but not for extensions.
8.    MaybeHandle<SharedFunctionInfo> maybe_result;
9.    IsCompiledScope is_compiled_scope;
10.    if (extension == nullptr) {
11.      bool can_consume_code_cache =
12.          compile_options == ScriptCompiler::kConsumeCodeCache;
13.      if (can_consume_code_cache) {
14.        compile_timer.set_consuming_code_cache();
15.      }
16.      // First check per-isolate compilation cache.
17.      maybe_result = compilation_cache->LookupScript(
18.          source, script_details.name_obj, script_details.line_offset,
19.          script_details.column_offset, origin_options, isolate->native_context(),
20.          language_mode);
21.      if (!maybe_result.is_null()) {
22.        compile_timer.set_hit_isolate_cache();
23.      } 
24.    }
25.  //.....omit..............
26.    if (maybe_result.is_null()) {
27.      ParseInfo parse_info(isolate);
28.      // No cache entry found compile the script.
29.      NewScript(isolate, &parse_info, source, script_details, origin_options,
30.                natives);
31.      // Compile the function and add it to the isolate cache.
32.      if (origin_options.IsModule()) parse_info.set_module();
33.      parse_info.set_extension(extension);
34.      parse_info.set_eager(compile_options == ScriptCompiler::kEagerCompile);
35.      parse_info.set_language_mode(
36.          stricter_language_mode(parse_info.language_mode(), language_mode));
37.      maybe_result = CompileToplevel(&parse_info, isolate, &is_compiled_scope);
38.      Handle<SharedFunctionInfo> result;
39.      if (extension == nullptr && maybe_result.ToHandle(&result)) {
40.        DCHECK(is_compiled_scope.is_compiled());
41.        compilation_cache->PutScript(source, isolate->native_context(),
42.                                     language_mode, result);
43.      } else if (maybe_result.is_null() && natives != EXTENSION_CODE) {
44.        isolate->ReportPendingMessages();
45.      }
46.    }
47.    return maybe_result;
48.  } In the 17th line of code, lookup compilation_cache, cache hit indicates that there are already bytecodes, and don’t need to recompile. The 26th line of code indicates that cache miss, so call the CompileToplevel on the 37th line to generate bytecodes. Bytecode needs to be bound to a context before execution, as below: 0.  Local<Script> UnboundScript::BindToCurrentContext() {
1.    auto function_info =
2.        i::Handle<i::SharedFunctionInfo>::cast(Utils::OpenHandle(this));
3.    i::Isolate* isolate = function_info->GetIsolate();
4.    i::Handle<i::JSFunction> function =
5.        isolate->factory()->NewFunctionFromSharedFunctionInfo(
6.            function_info, isolate->native_context());
7.    return ToApiHandle<Script>(function);
8.  } After binding, we get an important member — JSFunction, which is the input expected by the execution. 0.  bool Shell::ExecuteString(Isolate* isolate, Local<String> source,
1.                            Local<Value> name, PrintResult print_result,
2.                            ReportExceptions report_exceptions,
3.                            ProcessMessageQueue process_message_queue) {
4.    bool success = true;
5.    {
6.      PerIsolateData* data = PerIsolateData::Get(isolate);
7.      Local<Context> realm =
8.          Local<Context>::New(isolate, data->realms_[data->realm_current_]);
9.      Context::Scope context_scope(realm);
10.      MaybeLocal<Script> maybe_script;
11.      Local<Context> context(isolate->GetCurrentContext());
12.      ScriptOrigin origin(name);
13.      if (options.compile_options == ScriptCompiler::kConsumeCodeCache) {
14.  //.......omit
15.      } else if (options.stress_background_compile) {
16.  //.......omit
17.      } else {
18.        ScriptCompiler::Source script_source(source, origin);
19.        maybe_script = ScriptCompiler::Compile(context, &script_source,
20.                                               options.compile_options);
21.      }
22.      maybe_result = script->Run(realm);
23.  //......omit
24.      if (options.code_cache_options ==
25.          ShellOptions::CodeCacheOptions::kProduceCacheAfterExecute) {
26.        // Serialize and store it in memory for the next execution.
27.        ScriptCompiler::CachedData* cached_data =
28.            ScriptCompiler::CreateCodeCache(script->GetUnboundScript());
29.        StoreInCodeCache(isolate, source, cached_data);
30.        delete cached_data;
31.      }
32.      if (process_message_queue && !EmptyMessageQueues(isolate)) success = false;
33.      data->realm_current_ = data->realm_switch_;
34.    }
35.    DCHECK(!try_catch.HasCaught());
36.    return success;
37.  } The above function is the entrance for executing JS. Line 19 compiles the JS to bytecode. Line 22 executes the bytecode. Debug line 22 to step into the Run(realm). 0.  MaybeLocal<Value> Script::Run(Local<Context> context) {
1.    auto isolate = reinterpret_cast<i::Isolate*>(context->GetIsolate());
2.    TRACE_EVENT_CALL_STATS_SCOPED(isolate, "v8", "V8.Execute");
3.    ENTER_V8(isolate, context, Script, Run, MaybeLocal<Value>(),
4.             InternalEscapableScope);
5.    i::HistogramTimerScope execute_timer(isolate->counters()->execute(), true);
6.    i::AggregatingHistogramTimerScope timer(isolate->counters()->compile_lazy());
7.    i::TimerEventScope<i::TimerEventExecute> timer_scope(isolate);
8.    auto fun = i::Handle<i::JSFunction>::cast(Utils::OpenHandle(this));
9.    i::Handle<i::Object> receiver = isolate->global_proxy();
10.    Local<Value> result;
11.    has_pending_exception = !ToLocal<Value>(
12.        i::Execution::Call(isolate, fun, receiver, 0, nullptr), &result);
13.    RETURN_ON_FAILED_EXECUTION(Value);
14.    RETURN_ESCAPED(result);
15.  } The 8th line casts this pointer to JSFunction. For our case, the variable fun is console.log(JsPrint(6)), then call the 12th line — call(). In call(), the Invoke() will be called which is below. 0.  V8_WARN_UNUSED_RESULT MaybeHandle<Object> Invoke(Isolate* isolate,
1.                                                   const InvokeParams& params) {
2.  //...............omit
3.    Object value;
4.    Handle<Code> code =
5.        JSEntry(isolate, params.execution_target, params.is_construct);
6.    {
7.      SaveContext save(isolate);
8.      SealHandleScope shs(isolate);
9.      if (FLAG_clear_exceptions_on_js_entry) isolate->clear_pending_exception();
10.      if (params.execution_target == Execution::Target::kCallable) {
11.        // clang-format off
12.        // {new_target}, {target}, {receiver}, return value: tagged pointers
13.        // {argv}: pointer to array of tagged pointers
14.        using JSEntryFunction = GeneratedCode<Address(
15.            Address root_register_value, Address new_target, Address target,
16.            Address receiver, intptr_t argc, Address** argv)>;
17.        // clang-format on
18.        JSEntryFunction stub_entry =
19.            JSEntryFunction::FromAddress(isolate, code->InstructionStart());
20.        Address orig_func = params.new_target->ptr();
21.        Address func = params.target->ptr();
22.        Address recv = params.receiver->ptr();
23.        Address** argv = reinterpret_cast<Address**>(params.argv);
24.        RuntimeCallTimerScope timer(isolate, RuntimeCallCounterId::kJS_Execution);
25.        value = Object(stub_entry.Call(isolate->isolate_data()->isolate_root(),
26.                                       orig_func, func, recv, params.argc, argv));
27.      } else {
28.  //..................omit     
29.      }
30.    } Line 18 gets the address of the builtin, its index is 40, and its name is JSEntry. This builtin builds the execution environment for bytecodes. Then step into line 25, where the bytecode is executed. After that, we can only debug the assembly code, but c++. Figure 1 is the call stack. Before stepping into bytecodes, Generate_JSEntryVariant will be called. 0.  void Generate_JSEntryVariant(MacroAssembler* masm, StackFrame::Type type,
1.                               Builtins::Name entry_trampoline) {
2.    Label invoke, handler_entry, exit;
3.    Label not_outermost_js, not_outermost_js_2;
4.    {  // NOLINT. Scope block confuses linter.
5.      NoRootArrayScope uninitialized_root_register(masm);
6.      // Set up frame.
7.      __ pushq(rbp);
8.      __ movq(rbp, rsp);
9.      // Push the stack frame type.
10.      __ Push(Immediate(StackFrame::TypeToMarker(type)));
11.      // Reserve a slot for the context. It is filled after the root register has
12.      // been set up.
13.      __ AllocateStackSpace(kSystemPointerSize);
14.      // Save callee-saved registers (X64/X32/Win64 calling conventions).
15.      __ pushq(r12);
16.      __ pushq(r13);
17.      __ pushq(r14);
18.      __ pushq(r15);
19.  #ifdef _WIN64
20.      __ pushq(rdi);  // Only callee save in Win64 ABI, argument in AMD64 ABI.
21.      __ pushq(rsi);  // Only callee save in Win64 ABI, argument in AMD64 ABI.
22.  #endif
23.      __ pushq(rbx);
24.  //.....................omit............................
25.    __ ret(0);
26.  }
27.  }  // namespace
28.  //==================separation===============================
29.  void Builtins::Generate_JSEntry(MacroAssembler* masm) {
30.    Generate_JSEntryVariant(masm, StackFrame::ENTRY,
31.                            Builtins::kJSEntryTrampoline);
32.  } Look at the 30th line of code, the generator function of the 40th builtin is B, and its analysis method refers to my previous paper. Figure 2 shows the corresponding assembly code. You can also see a bit of C++ in Figure 2, but this is the last chance to see C++. After that, only the assembly code. Our case uses the console.log which is a global function. It is below. 1.  #define RUNTIME_FUNCTION_RETURNS_TYPE(Type, InternalType, Convert, Name)      \
2.    static V8_INLINE InternalType __RT_impl_##Name(Arguments args,              \
3.                                                   Isolate* isolate);           \
4.                                                                                \
5.    V8_NOINLINE static Type Stats_##Name(int args_length, Address* args_object, \
6.                                         Isolate* isolate) {                    \
7.      RuntimeCallTimerScope timer(isolate, RuntimeCallCounterId::k##Name);      \
8.      TRACE_EVENT0(TRACE_DISABLED_BY_DEFAULT("v8.runtime"),                     \
9.                   "V8.Runtime_" #Name);                                        \
10.     Arguments args(args_length, args_object);                                 \
11.      return Convert(__RT_impl_##Name(args, isolate));                          \
12.    }                                                                           \
13.                                                                                \
14.    Type Name(int args_length, Address* args_object, Isolate* isolate) {        \
15.      DCHECK(isolate->context().is_null() || isolate->context().IsContext());   \
16.      CLOBBER_DOUBLE_REGISTERS();                                               \
17.      if (V8_UNLIKELY(TracingFlags::is_runtime_stats_enabled())) {              \
18.        return Stats_##Name(args_length, args_object, isolate);                 \
19.      }                                                                         \
20.      Arguments args(args_length, args_object);                                 \
21.      return Convert(__RT_impl_##Name(args, isolate));                          \
22.    }                                                                           \
23.                                                                                \
24.    static InternalType __RT_impl_##Name(Arguments args, Isolate* isolate)
25.  #define CONVERT_OBJECT(x) (x).ptr()
26.  #define CONVERT_OBJECTPAIR(x) (x)
27.  #define RUNTIME_FUNCTION(Name) \
28.    RUNTIME_FUNCTION_RETURNS_TYPE(Address, Object, CONVERT_OBJECT, Name)
29.  //=================separation==============================
30.  //==================separation===============================
31.  Object DeclareGlobals(Isolate* isolate, Handle<FixedArray> declarations,
32.                        int flags, Handle<JSFunction> closure) {
33.    HandleScope scope(isolate);
34.    Handle<JSGlobalObject> global(isolate->global_object());
35.    Handle<Context> context(isolate->context(), isolate);
36.    Handle<FeedbackVector> feedback_vector = Handle<FeedbackVector>::null();
37.    Handle<ClosureFeedbackCellArray> closure_feedback_cell_array =
38.        Handle<ClosureFeedbackCellArray>::null();
39.    if (closure->has_feedback_vector()) {
40.      feedback_vector =
41.          Handle<FeedbackVector>(closure->feedback_vector(), isolate);
42.      closure_feedback_cell_array = Handle<ClosureFeedbackCellArray>(
43.          feedback_vector->closure_feedback_cell_array(), isolate);
44.    } else {
45.      closure_feedback_cell_array = Handle<ClosureFeedbackCellArray>(
46.          closure->closure_feedback_cell_array(), isolate);
47.    }
48.    // Traverse the name/value pairs and set the properties.
49.    int length = declarations->length();
50.    FOR_WITH_HANDLE_SCOPE(isolate, int, i = 0, i, i < length, i += 4, {
51.      Handle<String> name(String::cast(declarations->get(i)), isolate);
52.      FeedbackSlot slot(Smi::ToInt(declarations->get(i + 1)));
53.      Handle<Object> possibly_feedback_cell_slot(declarations->get(i + 2),
54.                                                 isolate);
55.      Handle<Object> initial_value(declarations->get(i + 3), isolate);
56.      bool is_var = initial_value->IsUndefined(isolate);
57.      bool is_function = initial_value->IsSharedFunctionInfo();
58.      DCHECK_NE(is_var, is_function);
59.      Handle<Object> value;
60.      if (is_function) {
61.        DCHECK(possibly_feedback_cell_slot->IsSmi());
62.        Handle<FeedbackCell> feedback_cell =
63.            closure_feedback_cell_array->GetFeedbackCell(
64.                Smi::ToInt(*possibly_feedback_cell_slot));
65.        // Copy the function and update its context. Use it as value.
66.        Handle<SharedFunctionInfo> shared =
67.            Handle<SharedFunctionInfo>::cast(initial_value);
68.        Handle<JSFunction> function =
69.            isolate->factory()->NewFunctionFromSharedFunctionInfo(
70.                shared, context, feedback_cell, AllocationType::kOld);
71.        value = function;
72.      } else {
73.        value = isolate->factory()->undefined_value();
74.      }
75.      // Compute the property attributes. According to ECMA-262,
76.      // the property must be non-configurable except in eval.
77.      bool is_eval = DeclareGlobalsEvalFlag::decode(flags);
78.      int attr = NONE;
79.      if (!is_eval) attr |= DONT_DELETE;
80.      // ES#sec-globaldeclarationinstantiation 5.d:
81.      // If hasRestrictedGlobal is true, throw a SyntaxError exception.
82.      Object result = DeclareGlobal(isolate, global, name, value,
83.                                    static_cast<PropertyAttributes>(attr), is_var,
84.                                    is_function, RedeclarationType::kSyntaxError,
85.                                    feedback_vector, slot);
86.      if (isolate->has_pending_exception()) return result;
87.    });
88.    return ReadOnlyRoots(isolate).undefined_value();
89.  }
90.  //==================separation===============================
91.  //==================separation===============================
92.  RUNTIME_FUNCTION(Runtime_DeclareGlobals) {
93.    HandleScope scope(isolate);
94.    DCHECK_EQ(3, args.length());
95.    CONVERT_ARG_HANDLE_CHECKED(FixedArray, declarations, 0);
96.    CONVERT_SMI_ARG_CHECKED(flags, 1);
97.    CONVERT_ARG_HANDLE_CHECKED(JSFunction, closure, 2);
98.    return DeclareGlobals(isolate, declarations, flags, closure);
99.  } RUNTIME_FUNCTION(Runtime_DeclareGlobals) is a runtime function defined by a macro template. The function checks the parameters and then calls DeclareGlobals() to perform the corresponding operation. In our case, DeclareGlobals() lookup the global object console, then takes out the log() from the console, and gets the address of console.log(). Finally, pass the parameter JsPrint(6) into log() which will start a new compilation and execution. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : v8blink@outlook.com WeChat Email This story was first published here.

Stories, News & Mentions about VERISOUL

Verisoul’s Stories on HackerNoon