Opportunities for IoT Development with MQTT v5.0

is among the top 10 most used protocols in the Internet of Things (IoT) industry. It has a vast legacy behind it, having been used for over two decades before. Even today, it receives significant attention thanks to its lightweight, open-source nature and accessibility. IoT devices can communicate through it despite spotty network conditions. Summary: Message Queuing Telemetry Transport (MQTT) Even though the latest version, , officially became OASIS standard in 2019, not many users have transitioned from the previous v3.1.1 MQTT. The primary concern may be the limited documentation available regarding MQTT 5-compatible broker installation and configuration. MQTT 5 This article aims to fill you in on . It will also go through a practical use case involving a Python-based system deployment. We will implement and configure an MQTT 5-compatible broker installation for an IoT system on LAN. MQTT 5’s feature set What Is MQTT? serves as a tool to connect many types of IoT devices in deployments of all magnitudes. It originally started in 1999 for oil and gas pipelines to communicate over remote satellites. MQTT Operating above TCP/IP, MQTT is a network protocol that functions on a communication model. It’s lightweight enough to be used for a wide variety of IoT devices yet powerful enough to work through unstable network conditions. publisher-subscriber HOW MQTT COMMUNICATION WORKS A few entities work together to make up an MQTT protocol: This device sends messages to subscribers via the Topic. Publisher: Each resource has this unique identifier. The publisher sends a message to the Topic, which then passes it on to the Subscriber. Topic: The subscriber is the end device that receives the messages from the Publisher via the Topic. Subscriber: This is the server, a central hub responsible for organizational communication between publishers and subscribers. Broker: The following diagram, which excludes the provided platform, shows how a simple MQTT communication works. If a cloud platform isn’t suitable for the development of a system using MQTT communication, we can use hbmqtt, gmqtt, paho-mqtt lib instead. level, a key feature of the MQTT protocol, is an agreement between the sender and receiver of a message. It defines how confidently the system can deliver a specific message. Quality of Service (QoS) QoS gives the client the power to choose a level of service that matches its network reliability and application logic. Because MQTT manages the re-transmission of messages and guarantees delivery (even when the underlying transport is not reliable), QoS makes communication in unreliable networks a lot safer and secure. Why Is MQTT Used in IoT Development? Thanks to its energy-efficient method of delivering data, MQTT is common for low-powered devices with limited CPU power or RAM. Some examples of use cases include: MQTT’s low data and energy usage make it a prime candidate for a real-time, text-based messaging application. Efficient communication. In a home security system, for example, the QoS feature of MQTT systems can determine whether an important message has been delivered successfully, ensuring that the inhabitants are properly alerted of the danger. Security. MQTT allows organizations to collect data from multiple sources efficiently, such as smartphones or car sensors. Intel gathering. For instance, multiple fire alarm detectors can communicate to determine whether a potential hazard is dangerous. Synchronizing sensors. Multiple sensors can monitor the health parameters of patients leaving a hospital. Medical IoT applications. Unlike HTTP, MQTT is capable of keeping up a persistent session between the client and broker. This feature is especially useful in connected cars. When a vehicle moves through a cellular network dead zone, the session can continue smoothly once reception returns. There is no need for a complicated HTTP handshake upon reconnection. Connected vehicles. WHAT ARE THE BENEFITS OF MQTT? is the selling point of MQTT, as its architecture runs even on devices with low-power CPUs and low RAM. The result is smoother data transfers over competing protocols like AMQP. The lightweight architecture allows MQTT users to implement the protocol quickly and easily. Efficiency being sent results in reduced network usage. Fewer data packets makes it ideal for connected devices. The reduced power usage MQTT can achieve a more efficient distribution of data. and control is easier through the protocol. Implementation of remote sensing WHAT ARE THE DRAWBACKS OF MQTT? MQTT might not be ideal in all circumstances, however. Some of the drawbacks to the protocol include: compared to Constrained Application Protocol (CoAP). Fast cycles are critical for systems with more than 250 devices. Slower transmit cycles . MQTT operates on a flexible topic subscription system. CoAP uses a stable resource discovery system. Resource discovery While MQTT uses Transport Layer Security/Secure Sockets Layer (TLS/SSL), it is primarily unencrypted. Lack of security encryption. Creating a globally scalable network is more difficult with an MQTT protocol compared to other competitors. Scalability. MQTT is the ideal solution for many applications but consult with an regarding its needs and preferences. The challenges are primarily related to security and interoperability. IoT development company MQTT v5.0 Features Overview The new features of MQTT version 5.0 aim to achieve the following goals: Communication among thousands or even millions of devices is more streamlined now. There are no protocol limitations, however, to orchestrate this number of devices, a proper architecture is required. Performance for large-scale systems. The MQTT v5.0 protocol renames the return code to a reason code, which can indicate more types of errors. Error reporting. Repeated ways devices interacted with one another are now formalized in the current version of MQTT. Defining the capabilities of interacting devices and how they respond to requests is now built into the system. Implementation of common interactions. New functionalities include adding custom properties and specifying the content type or payload format. Addition of extensibility mechanisms. , especially for smaller users who desire better productivity through MQTT. Better support MQTT 5.0 vs. MQTT 3.1.1 1. COMMUNICATION FEATURES , which can be implemented via Authentication Method and Authentication Data properties inside the payload. Enhanced authentication , which can be implemented with the Session Expiry Interval property. For instance, the Topic could include the subscription time, the amount of time the message will be stored. Session expiration . Limitations can be built-in, including maximum packet size (number of bytes to transmit) and maximum to receive (number of messages to be sent simultaneously for a client or server). Client and Server restrictions , a property to send messages on a delay. Will Delay Interval , properties that can help transfer packets to different brokers or servers. Server Reference or Server Redirect 2. POSTING FEATURES , the time to keep messages stored. Message Expiry Interval This attribute defines the type of payload flags that can be used: bytes (binary), UTF-8, or MIME types. Payload format indicator and content type. For instance, the topic/v1/device/ has an alias 1. This feature can minimize the number of data packages needed. Topic Aliases. With the use of Response Topics, the MQTT protocol can work similarly to an HTTP protocol with a response-request scheme. Response Topics. 3. SUBSCRIPTION FEATURES Users can optionally opt-out of receiving messages that the client published. Non-local publishing. This parameter controls message sorting. Retained message control. , which is used for server identification of the subscription. Subscription identifier to enable more flexible subscriptions with additional symbols and filtering features. Shared subscriptions 4. GENERAL FEATURES Errors can occur at any stage. In MQTTv3.1.1, not much guidance from the server was available as to what went wrong at different stages, such as establishing communication, posting a message, or subscriptions to topics. Reason codes on all ACK messages. Unlike MQTT 3.1, packets regarding disconnections can be delivered from both client and server sides in MQTT 5.0. Server disconnections. Keys can be used as value storage for different properties. User properties. An Example of an MQTT 5.0 Small System Deployment Let’s look at a case where we need to organize a local MQTT v5.0 network with Python-based clients. We will describe the challenges, issues, and pros and cons along the way. We will conclude by comparing it with an MQTT v3.1.1 network. DESCRIPTION: We have a building with several rooms containing a local area network (LAN). One room contains three standalone devices (e.g. activity standalone sensor, photo camera sensor, or audio sensor). The host device is located inside the LAN and connected to the router wirelessly or via cable. It must provide data gathering (and processing) functions from the standalone device in a period and must store this data locally in the database. For the current scope, an SQLLite database or simpler alternative is usable. The photo camera sensor and audio sensor must activate only after receiving the message from the activity sensor. GOAL: To ensure communication between the host device and standalone devices, and provide local database deployment and communication on the host side. REQUIREMENTS: All messages going from the sensors to the host device must be constrained with the additional properties of MQTT 5.0 (e.g. byte size of the message transferred to the topic). Messages from topics must contain a MIME type for easier encoding on the host side. Messages must be stored inside a database instance locally. NOTATIONS: x86 or ARM-based (e.g. Raspberry Pi) with connected sensors and access to a local network. Standalone device: x86 or ARM-based (e.g. Raspberry Pi) which hosts MQTT broker and handles messages from standalone devices. Host device: WHICH CLIENTS SUPPORT MQTT 5.0 AND PYTHON? Currently, we have two options to work with: and . However, these options do not have a built-in MQTT 5.0 broker and therefore are not preferable with a local deployment of a network. There is a non-Python implementation of the broker called Mosquitto that does support MQTT 5.0. paho-mqtt gmqtt Documentation can be found . It is able to support up to 50 000 devices per broker. Mosquitto has an “inflight queue” that can be configured in size (typical setting: 1000 messages) so that even under heavy load conditions like thousands of messages per sec or thousands of connected clients no connection or message gets lost. here HOW A PYTHON-BASED SYSTEM WORKS WITH MQTT 5.0 Not many libraries and documentation exist for the MQTT v5.0 protocol, especially from a Python developer’s point of view. The only current v5.0 clients for Python are gmqtt and paho-mqtt. PROS AND CONS OF AN MQTT V5.0 LOCAL NETWORK Pros: Fully autonomous device interaction inside the local area network. There is no need for cloud providers like GCP or AWS and no need for a WAN connection for local IoT system functioning. Network latency and data transfer speed. Transfer speeds depend only on the hardware capability of local devices. The disposition of devices within a LAN environment enables minimal latency. Energy efficiency with MQTT compared to the competition. Network security. Since the local network is not exposed to WAN, packets with messages cannot be captured or traced by an entity outside of the local network. The MQTT v5.0 protocol provides authentication of clients by servers and of servers by clients. MQTT also may use TLS certificates for secure connections and data transferring. Packet restrictions can be applied to brokers inside the network. Containerization. Easier containerization makes simulation and debugging much easier. Cons: Process and thread management for receiving messages and parallel works must be done beforehand. Threads for processing messages should be parallelized and managed properly for your devices to operate properly. WAN connection. Developers must debug and troubleshoot regularly, and having a proper connection between the host and standalone devices must be organized first, usually with a secure SSH connection. No support for stream transfer with the MQTT protocol. Look to other protocols if your organization needs it. Transfer of large files not available on MQTT. Consider bucket uploads or the HTTP protocol. Brokers cannot intelligently manage data. However, data can be stored for a limited time during a disconnection. Major Practical Differences Between MQTT v3.1.1 and v5.0 Properties for storing additional data Payload format indicators (bytes, UTF-8, or UTF-8 string pair) Request/Response PatternReason codes for client connection and disconnection Session expirations and control The upgraded protocol version allows to simplify data payload processing and parsing. Brings the ability to separate and precise control over messages, connections, and sessions. And it allows the transfer of additional data through properties, which could lead to creating more sophisticated IoT solutions. MQTT 5 Challenges Process/thread management for parallel publishing and listening for messages on standalone devices. You need to pay attention in a production environment. Limited documentation available and the implementation process of classes inside the package (paho-mqtt) is not obvious. Installation of a broker and its upgrade to MQTT v5.0 is difficult because of a lack of documentation. To recognize a device in the network, we need an IP discoverer to be added to the system. MQTT v5.0 Is a Big Improvement MQTT v5.0 is a suitable option for local IoT device communication if you have a central device that can host a message broker for communication between devices and/or the host. Despite its drawbacks (most of which were eliminated in MQTT v5.0), this protocol can be used for communication between small-to-medium sized networks of IoT devices. The author - Daniil Liadov , Python engineer at MobiDev Previously published at https://mobidev.biz/blog/mqtt-5-protocol-features-iot-development