Hackapost: how hackathons can be the best learning tool under the sun

Reflecting back to the Hackapost hackathon, I wanted to share a couple elements on how the technology stack worked.

There are indeed quite a couple of moving pieces involved.

First of all, let’s talk about sensing the world.

In order to do that, one needs sensors.

To make that easy on us, we have been provided with hardware and tools from AllThingsTalk (ATT for short) in the form of their LoRa development kit.

Hum, wait, what’s LoRa?

Lora stands for a longer thing actually: “LoRaWAN”.

“LoRaWAN™ is a Low Power Wide Area Network (LPWAN) specification intended for wireless battery operated Things in regional, national or global network.”

The LoRa Alliance website has a lot of stuff about it: https://www.lora-alliance.org

As a piece of hardware, it has the form of a Microchip component (actually some Semtech licensed thing). One needs to plug this component somewhere so that we can control it. In the LoRa devkit, we can plug this into an Arduino board named the “Mbili Sodaq”. Sodaq stands for Solar Data Acquisition. This is kind of useful as a solar panel and a battery can keep the system powered up without an external source (if one wants an external source, it works still). More about it on: http://support.sodaq.com/mbili/

Let’s get back to sensors. Arduino boards usually require some soldering or play with breadboards etc. Not so with this one, as it has what is called “Grove sockets”. These sockets are making it much much easier to connect sensors in a modular fashion. And the LoRa development kit contains quite a couple of such sensors.

There is something we discovered with this board during the hackathon: there is only one I2C connector and this means that one has to choose between the GPS module and the 3-Axis accelerometer. You can’t have both at the same time.

Here is a view on the bits and pieces making up the kit.

So, connecting sensors is easy, getting the LoRa modem is easy as well.

Now, there is a need for some software to make all of this do something. Like sending data out so that we can use it in an app or website.

First, the radio side of things

That’s where Proximus EnCo comes in handy. They operate the LoRa network, and have antennas deployed in the country.

Some areas are still spotty at this point.

In order to be able to talk with the network, our little device has to be recognized.

When using the LoRa development kit, one automatically gets a device registered in the system after having activated the development kit on the https://devs.enabling.be website.

That translates into having a device id.

The device id uniquely identifies the board (I do not really know if this is the board, or the modem itself).

There is another important concept, namely the “cloud channels”.

For the development kit, a channel is automagically created that forwards the sensor data to AllThingsTalk.

The channel is using MQTT. The reference site for MQTT is http://mqtt.org

What’s MQTT? It is an IoT protocol invented at IBM in 1999 by Andy Standford-Clark (@andysc on Twitter). So much for being new!

There is super cool interview of him on The Pi Podcast #14, that was out this february. http://thepipodcast.com/the-pi-podcast-14-andy-stanford-clark/

For the interested, the protocol specification is accessible here: https://www.ibm.com/developerworks/webservices/library/ws-mqtt/

So, MQTT stands for “MQ Telemetry Transport”. It is a “lightweight publish/subscribe protocol flowing over TCP/IP for remote sensors and control devices through low bandwidth, unreliable or intermittent communications.”

Ah, perfect for the IoT space. MQ stands for “Message Queuing”.

There is a lot of available software to deal with MQTT: https://github.com/mqtt/mqtt.github.io/wiki/software?id=software

Client libraries are also plenty: https://github.com/mqtt/mqtt.github.io/wiki/libraries

For the server side of things, one can use RabbitMQ along with its MQTT Adapter (https://www.rabbitmq.com/mqtt.html ).

I think that it is what AllThingsTalk use at their side BTW.

There is also a very widely used broker named Mosquitto: http://mosquitto.org/, and the code lives on GitHub from this month on: https://github.com/eclipse/mosquitto

Back to our Cloud Channel Subscription, here is the details of ours:

Hum, well, but how is our device sending things?

Retrieving data from Enabling.co is a matter of using the API.

But that’s not what we want here. We want to send data sensed from our sensors on the Arduino board.

Let’s pay a visit to how one programs the Arduino board to actually send data.

In order to program the Arduino, one has to use the Arduino IDE.

This IDE usually has a light background and it kills my eyes when coding. As you can see it is actually possible to get it with a dark theme. Just check https://github.com/jeffThompson/DarkArduinoTheme

For setting things up for the LoRa development kit, let’s pay a visit to AllThingsTalk.

The site itself isn’t telling much as the real meat is on http://www.smartliving.io/

Err, yeah, one step even further actually: https://maker.smartliving.io/

There are docs in there about how it works: http://docs.smartliving.io/kits/lora/

After having registered (“claiming”) the device, there is a nice “Playground” available:

The interesting bit about this is that one can add other devices to a playground.

Let’s have a look at ours, containing our single nice little device.

As we had some fun before writing this, you can see a couple of values for the various sensors. This will obviously be empty for a new kit.

Back to the hardware/Arduino front there are some things to do.

Basically, there are some keys things to do:

• Add the type of board to the Arduino IDE
• Add the libraries for the board, LoRa, and sensors to the Arduino IDE
• Get a copy of the “Keys” that make it possible for the device to be recognized when it sends radio stuff, and that will make messages come into the Enabling system

The libraries are also living on GitHub and you may want to stay up to date:

https://github.com/allthingstalk/arduino-lora

There is also a set of example sketches in there. A sketch is what an Arduino IDE helping you edit, compile and transfer into an Arduino board.

There is a keys.h file as well, where one puts the keys retrieved from the smartliving.io cloud settings.

These are things that are used by the code.

Let’s have a view on the sketch content:

https://github.com/allthingstalk/arduino-lora/blob/master/ATT_Lora_IOT/examples/experiments/environmentalSensing/environmentalSensing.ino

As you can see, the sketch is using your keys and also has an ATTDevice using the MicrochipLoRaModem.

This looks like easy, and it is from the developer perspective.

Underneath, yeah, there are lots of things going on, by we do not mind.

So, get the board connected to your PC via USB… (provided the correct COM port is selected in the Arduino IDE)

Just compile and upload.

From then on, the system will run its program loop and send data according to its logic.

Sending pushes the data to the ATT platform (through Enabling but that’s under the covers).

Data is now flowing through, from the sensor to LoRa, through Enabling to Smartliving.

That’s where we can start talking about actually using the data in an application of ours.

How?

By leveraging APIs.

There are actually several ways to do just that.

There is the easy way, that doesn’t even involve Smartliving. Or just could too. More on this later.

Let’s talk Waylay.io

Waylay actually does a lot of things:

The fun part is on https://playground.waylay.io

I got the change to chat with Waylay’s CTO, Veselin Pizurica. Great guy, super smart. I got a blast from our chat. I got an account on the playground and started having fun.

Waylay docs are at: http://docs.waylay.io/ if you want to dig more.

There is a sensor that can actually directly hook into Enabling’s API.

The code is there, ready to be used.

It is all Javascript.

And it is crazy what one can do with it. But that will be for another post.

But that’s not the API for Smartliving.

If we want to use the API of Smartliving (and that’s what I wanted to do during the hackathon, not sure I would have done that if I had had more clues about Waylay…)

Let’s have a look at the Smartliving API. It is documented at http://docs.smartliving.io/api/

First thing I jump onto when dealing with APIs is: “How the hell am I going to authenticate myself with this thing”? APIs have often their own views on this.

So, we need ids, keys etc.

Where are those?

Just here:

And then, the fun begins with doing the right dance with headers and so on.

I actually wanted to implement the API using my favorite development platform, which happens to be Pharo (http://pharo.org)

These days, I am also using Python and Javascript|Node but they do not have the “Pharo feel”.

If you want to know more about Pharo, here is a couple of nice articles about it on Medium.com.

For example:

https://medium.com/concerning-pharo/elegant-pharo-code-bb590f0856d0#.sjqdgmtsc

It is actually a very capable language. I made a solution for dealing with CMTSes and cable modems with it: http://pharo.org/success/CableExpertise

It looks like this:

Pretty much something that could make good use of all of the items in the IoT space as there is nothing that limits it to doing only CMTS work. It is a generic data collection and presentation + alerting thing.

So, I wanted to use Pharo for accessing the API.

And started with a little test…

The client being an AttClient instance.

An AttClient authentication looks like this:

One nice thing is that one doens’t have to create a myriad of classes to organize things as, like in Objective-C, there are protocols, which are grouping methods.

Long story short, the API invocation code looked like this at the end;

The explore thing shouldn’t be in a test, but hey, we were hacking.

It shows an object explorer like this:

This makes it very handy to explore the results from the API and drill down, all without leaving the development environment.

Once the API client was working nicely, I turned into making a server so that we could access the data in the format that was required by the front end developer.

In Pharo, we have a little framework named “Teapot” which can do that pretty easily:

With the xml entity returned being produced by the #sensors message, which actually sent ‘#sensors:’ with our actual device id.

Giving:

We needed XML out, that why there is the last line converting the data on its way out.

It was actually easy as well:

As you can see, Pharo code is pretty clean.

At this point, we now have the possibility to use the data in the way we want.

Of course we could have this same thing in a JS frontend app right away.

But I was more interested in getting a Pharo based component that would help me created IoT apps than the business case.

We had the opportunity to work with very useful hardware, test all sensors, get data through all of the path, discover new services and features, integrate APIs, and meet new cool people.

For me, that’s what mattered about it, not being part of yet another startup. To be frank I am already busy with a couple ventures, and the key point was to understand all of this in more depth.

If there is another hackathon around IoT, I am sure I’ll be better equipped to deal with it.

And I am left with a super cool kit, for which I have new ideas!