Connecting to Cloud

Connecting devices to internet and their long term maintenance requires minimum two key features:

• Remote server allowing devices to securely publish data, send control and configuration information.
• Remote firmware update to fix issues and add new features.

One can write a custom server and design optimized protocols for communicating with devices in field however as number of devices grow and spread geographically, it makes economical sense to go with commercial cloud platforms offering IoT services. In this article we will look at two such IoT service providers, Amazon Web Services (AWS) IoT Core^[1] and Google Cloud Platform (GCP) IoT Core^[2]. This article will be released in two parts, this one will cover connecting to IoT cloud services (both of them), publishing data and controlling the device. Second part will cover Over-the-Air (OTA) firmware updates over cellular network.

The sample project uses Nordic Semiconductor's nRF52840 SoC^[3] based EBYTE module^[4] to create BLE or Thread mesh network, SIMCOM SIM7600E Multi-band LTE/4G/3G/2G based AND Technologies module^[5] for cellular connectivity and any CMSIS DAP Debugger. The reason for selecting a powerful 4G LTE module is: NB-IoT service is not yet commercially available here and telecom operators are going to shut down 2G networks within few years anyway.

Before we proceed further, this article is not about which cloud platform is better than the other one, and neither it compares the cost or free quota as these data can change any time, also costing calculators are available on their respective websites. Both cloud platforms are fantastic and very well documented with loads of examples, however they are not same. Each platform has a different impact on your application's architecture and the required capabilities of your selected microcontroller. Which one is more suitable for the given application is a better question.

Telemetry with MQTT

Message Queuing Telemetry Transport (MQTT) is a very simple protocol specification^[6] allowing memory constrained devices to communicate among each other with the concept of Topics. Devices can create, Publish and Subscribe to any arbitrary number of topics. As of now AWS and GCP IoT Core only supports MQTT specification 3.1.1 with some respective limitations. The topic messages can have different reliability levels controlled with QoS parameter:

• QoS 0: Send and forget, message loss is acceptable.
• QoS 1: Ensure message is received at least once, duplicate messages are acceptable.
• QoS 2: Ensure message is received exactly once, neither loss nor duplicate message is acceptable.

Underlying protocol for MQTT is usually TCP/IP which itself is meant to be reliable however a QoS 0 message loss might still occur due to excessive server load.

Irrespective of what your IoT application does and the selected platform, the life-cycle of all IoT devices are similar. A device connects to MQTT bridge (also called broker or server) on standard port 8883. It authenticates itself with MQTT CONNECT request's unique client id, username and password fields. Post successful connection it will subscribe to one or more topics at required QoS level. Beyond this point device will start parallel tasks: Waiting for published messages on its subscribed topics to act on them, publish its own state, events or reports, and periodically pinging bridge with keep-alive messages. Any time the device disconnects whole process is restarted.
Cloud platforms also support device connection over HTTPS protocol, but here we focus only on MQTT.

A Hypothetical IoT Application

To understand the cloud platform differences better lets define a hypothetical IoT application: where a IoT device is installed in a passenger bus cabin to monitor both the vehicle and its driver. There are six types of data the device reports or publishes.

1. Bus Identification: Registration number and route number.
2. Bus Location: Current GPS location.
3. Bus Speed: Current travel speed.
4. Bus Driver's Picture: Medium/Low resolution image taken at random intervals.
5. When Bus reaches its destination depot, it publishes this event to that depot so that it can get configuration for the next route and other maintenance activities. The depot also has a IoT device which can be a application running on a PC or local server there.
6. Associated time stamp for all the above events.

Now lets add some complexity to the above requirements: All the above data are reported only when certain conditions are met (Configurations) or in response to certain Commands:

1. Speed is reported only when certain threshold is crossed, along with Location.
2. Location is reported when it is queried.
3. Picture is reported when driver is found to be talking over phone, drinking, drowsy, etc.
4. Picture is reported when queried.

There are so many ways to map these requirements to topics and corresponding subscriptions. We begin with the knowledge that each published message costs money. If we create a common topic named /device-configurations to which all devices subscribe to get their configurations then this would be a inefficient and expensive choice, because a configuration message meant for a specific bus would be published to all the buses by the MQTT bridge. With this understanding lets begin the first iteration of design:

Application Topics:

• /bus/bus-id/configuration: bus-id can be any unique number or bus registration number. Each bus device will subscribe to its own configuration topic. A depot device and/or a cloud application will publish configuration messages to a specific bus on this topic.
• /bus/events: A common topic to which all bus devices will publish their events/reports (1-6) when corresponding conditions (A-D) are met. A cloud application will subscribe to this topic and process them further based on message contents. Messages can be binary data, however if they are in JSON format then built-in cloud features can be used to route data to other services based on message contents.
• /bus/bus-id/commands: Administrators can publish commands to specific bus to query specific data. Each bus device will subscribe to its own command topic and publish the result on events topic.
• /depot/depot-id/bus-arrived: depot-id can be any unique number or name of the depot. Each depot device will subscribe to its own topic. For a bus, before it begins its journey from a depot it will receive its configuration which will contain, among other details, destination depot's topic name. Bus devices will publish their arrival event on this topic. This is an interesting topic as it allows device to device communication with only using MQTT bridge that too without any pre-programming in the bus device about destination depot, there is no other cloud application involved.

Now we are ready to look into cloud platform differences and how they affect the design decisions made till now.

AWS IoT Core

AWS implementation of topics is very close to MQTT specification with some limits^[7]. Limits relevant here are: Topic names beginning with '$' character is reserved by AWS IoT services and also there is a restriction on how many forward slashes (sub-topics) a topic name can contain, which is set to 7. Devices can create any other topics (on-the-fly) and subscribe or publish to them as per the policies configured by application administrators.

The first iteration of the application can be migrated to AWS without any modification.

GCP IoT Core

GCP implementation takes a different approach to MQTT topics along with additional limits ^[8][9]. To use their IoT core services, Cloud Pub/Sub service is also required. Devices are added into registries and each registry has a default Cloud Pub/Sub topic. Devices can only publish or subscribe to predefined set of topics or their sub-topics:

• /devices/DEVICE_ID/config: Where DEVICE_ID is the ID or name you assign to a device while creating it. Each device can subscribe to its own config topic. Configuration messages are persistent, when a device connects and subscribe to its config topic, it will receive its latest configuration, and any further updates on it.
• /devices/DEVICE_ID/events: Each device can publish events to its own topic or its sub-topics like /devices/DEVICE_ID/events/critical. Published device events are forwarded to Cloud Pub/Sub topics added in the registry based on sub-topic name matching. If there is no match or no sub-topic then messages are forwarded to default topic. Subscribers of Pub/Sub topic will then process the device events.
• /devices/DEVICE_ID/state: Each device can publish its state to its own topic, no sub-topics are supported here. State topic is intended for reporting device's state in response to its configuration changes. State can be monitored from cloud platform.
• /devices/DEVICE_ID/commands/#: Wildcard '#' indicates that messages published to any sub-topics will also be received by this topic. Each device can subscribe to its own commands topic. Unlike configurations, commands are not preserved and are received only when device is connected to the MQTT bridge. Commands can be sent at much faster rate than Configurations.

The above list may seem restrictive but if you notice these predefined topics are logically very similar to topics of our application and there are built-in features associated with them. For example in AWS, /bus/bus-id/configuration is simply just another topic name for the bridge. For a bus to get its configuration it needs to somehow inform the cloud application, simply connecting to bridge and subscribing to configuration topic is not sufficient. Whereas in GCP, configurations can be pre-loaded by the cloud application and devices will automatically get it on next connection.

On the down side, device to device MQTT communication is not feasible in GCP so the idea of depot topic /depot/depot-id/bus-arrived needs redesign. Depot device now needs to be upgraded from MQTT application to Cloud application or it can subscribe to topics of Cloud Pub/Sub services instead and continue to run on local PC/server.

Second iteration of application topics:

• /bus/bus-id/configuration becomes /devices/DEVICE_ID/config
• /bus/events: Each device can publish events to its own /devices/DEVICE_ID/events topic. So /bus/events can be become default Pub/Sub registry topic where all device events will be forwarded to it.
• /bus/bus-id/commands becomes /devices/DEVICE_ID/commands/bus-commands
• /depot/depot-id/bus-arrived: this topic would be dropped and instead bus devices will publish their arrival events on /devices/DEVICE_ID/events/depot-arrived sub-topic. A new Cloud Pub/Sub topic /depot/bus-arrivals will be added to match depot-arrived sub-topic. A cloud application can further filter these messages and publish them to corresponding depot applications.

Now lets look at the security side of IoT cloud platforms.

IoT Device Security

AWS and GCP requires minimum TLS 1.2 for secure connection between devices and cloud services. Devices are expected to authenticate the server by verifying its X.509 public certificate with its root CA along with the expiry date. Devices should also be capable of updating the stored certificates and server endpoints in case these certificates are revoked.
How the cloud platform authenticates devices is different and that affects parts selection process.

Authentication with AWS IoT Core

AWS continues to follow TLS protocol and devices must support TLS client authentication. MQTT CONNECT request's username and password fields are not used. Cellular modules like SIM7600E supports SSL stacks with TLS 1.2 along with server and client authentication modes. So there is no need for any additional cryptographic libraries and MCU requirements becomes simple in terms of CPU Frequency/Power, Flash and SRAM memory, HW accelerators.

Authentication with GCP IoT Core

GCP again takes a different approach here, TLS client certificate verification is optional but JSON Web Token (JWT)^[10][11] based authentication is mandatory. JWT becomes the password field of the MQTT CONNECT request and username field remains ignored as in AWS. The idea is JWT generation needs less computation than TLS client verification however the trouble is this method is more common in web applications. None of the cellular modules with SSL stacks support JWT creation and we need cryptographic libraries like mbedTLS to just create tokens. This affects MCU selection, as now there should be enough memory and computation power to generate JWT in acceptable time.

JWTs have an expiry time of upto 24 hours, after this time device connection will be dropped by MQTT bridge and reconnection with a new token is required. Even if a device is disconnected due to any other reason token regeneration is still required.

Now lets take a deeper look into SIM7600E module, this is applicable to any cellular module with AT command interface.

How Secure Are Cellular Modules With AT Interface

There are two types of modules:

1. Modules which only supports TCP/IP stack and SSL needs to be implemented by MCU firmware.
2. Modules which also supports SSL protocols like in SIM7600E.

For the second types which may appear convenient, connection between module and remote server is secure but connection between MCU and module is not, it is just plain text AT commands over UART. MCU memory/debug access protection features won't help here.

All the attacker has to do is open your device and capture module's UART pins, which are well documented in datasheets. Captured data would include device private keys, application cloud requests and messages! Fortunately in SIM7600E SSL certificates can only be downloaded but cannot be read back. However during firmware OTA updates this feature won't help. So unless you are absolutely sure your device PCBs can't be reverse engineered in fields, cellular or even WiFi modules with SSL over AT interface is not much of use.

I wish manufacturer could add AT commands to enable/disable encryption over UART and the keys can be configured in production.

To make cellular IoT with external MCU completely secure, SSL stack should be implemented in MCU firmware. This would result in encrypted payloads of data packets over AT commands and along with memory/debug access protection reverse engineering firmware would be very difficult.

Conclusion

MQTT

AWS IoT Core is very flexible here, it allows device to device communication, arbitrary topics and subscriptions. Practical applications can be built with just the bridge and devices, for example suppose this Dash Camera is used as home surveillance cameras, one can simply interact with them with Smartphone over MQTT by just using the IoT core services.

GCP IoT Core with its pre-defined set of topics and Cloud Pub/Sub provides complete coverage of a typical IoT device requirements and with features like persistent configurations and state monitoring, many common implementations are simplified for us.

MCU Capabilities

If low-power/low-memory MCU is must for the application and some cellular module like SIM7600E is used then AWS is an obvious choice. However if your application is a mesh network of low powered devices and they connect to cloud services through a gateway which is just another device with high spec MCU then GCP is also a suitable choice. A gateway can be used to authenticate low spec devices to MQTT bridge.

For a large application device data collection is merely the first step in IoT, more cloud services are must for further processing of data and only after taking them into consideration along with cost we can conclude which cloud platform is more suitable for a given application.

Sample Project

The objective of this part's sample project is to explore IoT cloud connectivity. There is no softdevice or bootloader, these will be required in second part for OTA updates. MQTT client library chosen is AWS IoT embedded-c sdk instead of a neutral project like Eclipse Paho^[12], this is to keep at least one easy option. Another purpose of using AWS SDK is to show that there is nothing AWS specific inside the SDK, we can use it with any MQTT cloud platform.

There are multiple ways the project can be configured. Greyed blocks in project Stack are always required.

1. AWS IoT → GRPS SSL APIs.
   Easiest and the simplest one requiring least amount of memory.
2. AWS IoT → mbedTLS → GPRS TCP APIs.
   To solve UART snooping issue.
3. GCP IoT → mbedTLS + GRPS SSL APIs.
   mbedTLS is required to generate JWT.
4. GCP IoT → mbedTLS → GPRS TCP APIs.
   To solve UART snooping issue.

Project source code and more details are available on my GitHub public repository:
https://github.com/rkprojects/cloud-iot-with-nrf52↗