Ask Gourav Das

Tech Enthusiast and Clouder. AWS 4x Certified. & I still watch One piece and spongebob squarepants

Cloud Home Automation Series Part 1 : Connect ESP32 to AWS IoT with Arduino code

Welcome to 
SIMPLE LEARNING
AWS Cloud Home AutomationZero to Hero Series, where the readership, essentially Embedded System Engineer & Cloud Engineer will able to learn to create Automated IoT Solutions using ESP32 Module (Hardware), AWS IoT portfolio services (MQTT & Integration assistance) & Arduino sketch (Code). The entire series mainly focuses on the Hardware integration with AWS Cloud to make a system of interrelated computing devices.
The following series split into four parts (refer below) with very simple and clear instructions to provision a home automation system to control house appliances through the web.
Everything covered from scratch you won't face any difficulty understanding. In case of any clarification, drop me a note on LinkedIn. Feel free to explore them with ease, skip to the one which is relevant to you.
  1. Part I - Connect your device (ESP32) to AWS cloud.
  2. Part II - Use Device Shadow Service (AWS IoT) to control ESP32 inbuilt led using MQTT client.
  3. Part III - Create a secure web client hosted in Node-RED to control ESP32 inbuilt led.
  4. Part IV - The Real Deal: Create an automated system to make the light bulb connected to the web using AWS, an ESP32 board and a relay module.
We are going to use ESP32 MCU which is a
low-cost & low-power device
, with inbuilt Wi-Fi module, thus acts as a Cloud Communication Module (CCM) & we can create wonders with the conjunction of sensors & modules if used in ESP32.
    Security of connected devices is of predominant concern. The widely used Transport Layer Security (TLS) version 1.2 is the de-facto standard used on the web for secure connections including banking and financial institute. There are very few micro-controllers that supports TLS 1.2 and ESP32 is one of them. 
    Good news is that AWS IoT Core maintains TLS 1.2 and above, thus making the security robust 
    .
    Prerequisites:
    1. Arduino IDE (arduino.cc/en/main/software).
    2. ESP32 Board (Easily available in Amazon.com/Amazon.in/Aliexpress or Local Robotics Shop), they cost around 6$ or 350-450 INR.
    3. AWS Free-Tier Account. (Services used here will cover up in free tier limit). Choose the N. Virginia Region only throughout the series.
    4. This tutorial mostly covers hands-on, prior knowledge of AWS IoT is an advantage. Click here to check AWS Official IoT Documentation.
    Learning Objectives:
    1. Install ESP32 espressif/arduino-esp32 library , which the main source library of ESP32 board and contains all necessary functions.  [Assuming Arduino IDE preinstalled]
    2. Install AWS ESP32 Hornbill Library which binds ESP32 with AWS.
    3. In AWS Console, will create a Device thing (for ESP32), Certificate (for Authentication) & Policy (for Authorization).
    4. Upload the Certificates & Arduino sketch in ESP32 to send Temperature & Humidity sensor data to AWS MQTT Broker.
    5. Bonus Challenge: Create IoT rule to set a notification whenever temperature crosses the threshold.
    So let's see something happen now
    . 
    Kindly look for the step-by-step screenshots to follow the instructions vividly, kept it separate to make the document minimalistic. 
    1. Install ESP32 essential Libraries
    To Install the espressif/arduino-esp32 library in windows, click here
     for step-by-step instructions
    . To install in other platforms, click here and select the required platform in the Installation Instructions. Too lazy to follow the instructions, follow the below steps for Windows (Easy Hack).
    • Create the last three highlighted folders : C:\Users\{UserName}\Documents\Arduino\hardware\espressif\esp32  and make the complete folder structure as shown.
    • Either clone the repository, select esp32 as the target folder or click here and download the master zip. Extract arduino-esp32-master.zip inside esp32 folder, make sure all the extracted files inside the arduino-esp32-master folder copied to esp32 folder.
    • Open the following folder: C:\Users\{Username}\Documents\Arduino\hardware\espressif\esp32\tools and open get.exe (Ran as an Admin). When get.exe finishes, you should see the following files in the directory → Click here .
    • Plug your ESP32 board to your Desktop/Laptop using USB cable and wait for the drivers to install.
    2. Install Hornbill Library to Connect ESP32 with AWS
    In this part, we will install the Hornbill AWS library which integrates the micro-controller, ESP32 with AWS Cloud, securely and anchored the communication channel with TLS 1.2. Click here to download the AWS_IOT Hornbill library. Exact the Hornbill-Examples-master.zip & go-to Hornbill-Examples-master\arduino-esp32, copy AWS_IOT and paste it to C:\Users\{Your User Name}\Documents\Arduino\libraries.
    3. AWS Integration
    From the AWS region selector in the navigation bar (top right corner), choose the US East (NVirginiaRegion only throughout the series.
    Create Policy
    Click Here to go to AWS IoT Console we will select Policies under the
    Secure
    option from the Sidebar and click Create [Top Right Corner]. In the next step, we will give a name to this policy (e.g. ESP32Policy), under Action copy-paste the following; iot:Publish, iot:Connect, iot:Subscribe, iot:Receive, under Resource ARN put *, under Effect mark allow and click Create to provision the policy. In short, we are trying to create an AWS policy which will provide limited access to ESP32 to AWS resources. Click-here
     to refer the step-by-step screenshots to create policy
    .
    Register the Thing
    Lastly, we will select Things under the
    Manage
    option from the Sidebar and click Create [Top Right Corner] and follow the below steps. Click-here
    to refer the step-by-step screenshots to register the Thing.
    1. Next, select Create a single Thing
    2. In the next step, give a name for Example; ESP32 and keep everything Default & click next
    3. In the Next step, Select Create Certificate. 
    4. In the final step, we will download the Thing Certificate and the Private Key and keep it safe as we need it in the next section (No need to download the Public Key), select activate and click Attach a policy (Note: Don't press Done or else you need to repeat the step) and select the policy (ESP32Policy) created in the previous section. Thus, we are done provisioning the digital representation of ESP32 board in AWS. Next, we will do the hardware integration.
    4. Hardware Integration
    Here comes the fun part where we can see our real progress, firstly we will register the certificates and private key to our AWS_IOT library. Go to the following directory, C:\Users\YourUsername\Documents\Arduino\libraries\AWS_IOT\src, open the file aws_iot_certficates.c in an editor (Notepad++).
    The Certificates and the Private Key
    Here we need to attach the Thing Certificate and Private Key which we have downloaded in the previous section along with the CA certificate, click here to copy or download the CA certificate. Now the tricky part, as we can see the blank spaces, represented with multiple:x, here we will inscribe the certificates and the private key. In aws_root_ca_pem[] we will set the CA certificate, in private_pem_key[] we will set the private key and  certificate_pem_crt[] we will set the Thing Certificate. Click here
    to refer the step-by-step screenshots to enrol the Certificates and the Private Key.
    Arduino Code in ESP32
    Next, we will work on the Arduino IDE and upload the Arduino code in the ESP32 Board. First, restart the Arduino IDE after following the instructions in section 1 & Plug your ESP32 board to your Desktop/Laptop using a USB cable. In the code, we are sending dummy Humidity and Temperature data, replicating a DHT11/22 Sensor to Topic ESP32/DHT11. Click here 
    to refer the step-by-step screenshots to implement the Arduino Code in ESP32 Board.
    1. Click here to download Arduino Code and open it using Arduino IDE.
    2. Select Tools (from the Toolbar) and choose the following options; for Board select ESP32 Dev Module, for Upload Speed select 115200 & for Port select COM3 (Though I will suggest check COM Port No. from device manager → Ports (COM & LPT) as it varies).
    3. In the code, you need to add your Wifi Name & Password and Host_Address which you will find in AWS IoT Core main dashboard (Click Here to go to AWS IoT Main Dashboard), go to Settings (Second last option in the Sidebar) & under Custom endpoint, you will find the Endpoint (Host_Address) copy this and add this as the HOST_ADDRESS[] in the code.
    4. Click Upload (Look for the right arrow at the Top) in Arduino IDE to start the code transfer to ESP32. Hold the Boot Button in ESP32 until the code gets uploaded completely (You will able to see a message "Done Uploading" below, once done).
    5. Open the Serial Monitor from Tools, a separate console will open. Select 115200 baud below. If you can see the publish messages you are almost done.
    6. Final mission: Check the messages coming to AWS, Click Here to go to AWS IoT Console, Region: N. Virginia. Choose Test from the Sidebar it will connect the MQTT client. Under Subscription topic, put ESP32/DHT11 and keep everything default and Click Subscribe to topic and after that, you be able to see all the published messages coming from your ESP32.
      Voila! You did the integration successfully :)

    Bonus Challenge (Optional): Create IoT rule to set a notification whenever temperature crosses the threshold ?
    Complete this challenge. Refer the following Document for more details. If you find any difficulties. Connect me @ https://linkedin.com/in/dasgourav
    In the next part,
    we will turn ON/OFF the inbuilt led of ESP32 using AWS Device Shadow.Hope to see you again :)

    Tags

    Comments

    February 9th, 2020

    Like all the AWS IOT apps and Alexa but first time my Alexa alarms failed when internet went Down I realized its not just lack of security but Resilence it lacks. i have to fix all the Smart home apps or create new ones to work in a smart home while its offline internet or grid power is down. Emergencies like last year in 2019 had our home systems paralized without internet. Added an old 555 timer for .25 cents is all it took for home to work on a battery Alarm,Alert and timer system in our Home of Things Fog network. AWS could still offer us IPFS archives and backup but will add a FOG server to my home as some AI needs to operate local for Security Systems and even cooking otherwise we burn the food or cause fires when internet drops out.

    February 9th, 2020

    Hi Alan, I agree that power outage and internet connectivity are major roadblocks. I would suggest using AWS IoT Greengrass for local execution. For resiliency, you may have a second channel to interact with your devices either use The Things Network (global open LoRaWAN network) or LoRa modules.

    February 9th, 2020

    Thanks gourav-das, your suggestions are most helpful. This should keep me busy this week. I also found a local company https://www.semtech.com/company/press/semtech-announces-open-source-software-for-lora-based-gateways-at-the-things-conference that makes an Open Source LoRa packet-forwarder.
    I have a Buoy 3.6 mile offshore i need communication with will try this.

    February 18th, 2020

    Hey gourav-das, great tutorial. Was able to follow part 1 together with the bonus lesson. However I ran into one problem. After uploading the code I get an error message Error (-28) connecting to the AWS endpoint via port 8883 … trying to reconnect.

    When I go to the AWS console / Settings the custom endpoint says ‘ENABLED’ in green (top right) so not sure what’s going on.

    Also why do you publish random values and not the actual temp from the sensor?

    February 18th, 2020

    Thanks Thomas for reading and great job for following all the steps :slight_smile: .

    1. Your error seems like there is some issue with your policy. Try to replicate the same policy as shown below, go to IoT Core Main Dashboard , In Side Bar Select Secure --> Polices -->The policy you created for this exercise.
      image

    2. As I want to keep the implementation minimalistic thus I haven’t integrated it with DHT module. If you are interested to add a DHT module to this project, I uploaded the code which receives data from DHT module, [click here] to download the ino file.

    If you are still facing the same issue , let’s connect on LinkedIn.

    February 18th, 2020

    Thanks for the quick response.

    (1) The policy looks as per your extract although iot:Connect and iot:Publish were in the other order - changing of which didn’t have any impact.

    What I don’t understand is the message in the console “Trying to reconnect” doesn’t seem to come from the sketch. If the connection is not successful I would expect the message “AWS connection failed, Check the HOST Address”

    if(hornbill.connect(HOST_ADDRESS,CLIENT_ID)== 0) // Connect to AWS using Host Address and Client ID
    {
    Serial.println(“Connected to AWS”);
    delay(1000);
    }
    else
    {
    Serial.println(“AWS connection failed, Check the HOST Address”);
    while(1);
    }
    }

    Instead what I get is …

    Connected to WiFi
    192.168.2.168
    E (25565) AWS_IOT: Error(-28) connecting to axxxxxxxxmvady-ats.iot.us-east-1.amazonaws.com:8883,

    Trying to reconnect

    1. The DHT module needs the DHT library. Where can I find that?
    February 20th, 2020

    Hi Thomas,
    As discussed, by activating the Certificate, it has started working.
    Thanks for your valuable feedback. I have updated all my documents so that there is no room for error.

    Also I forgot to mention you need DHT libraries to make your DHT sensor working. Open your Arduino IDE and go to Sketch > Include Library > Manage Libraries. The Library Manager should open.
    Search for “DHT” on the Search box and install the DHT library from Adafruit.

    More by Ask Gourav Das

    Topics of interest