Publish Messages to AWS IoT core Rest API

Overview

This blog focuses on publishing data to AWS IoT Core from a device using HTTP protocol instead of widely used MQTT. Here, we will try sending to the broker first, and then later, we will securely send device data to the AWS services supported by AWS IoT rule actions known as basic ingest. Basic Ingest optimizes data flow by removing the publish/subscribe message broker from the ingestion path. To visualize the basic ingest, we will republish it to a separate topic using the AWS IoT Rule.

AWS IoT Core

Amazon Web Services (AWS) IoT Core is a robust platform that facilitates seamless communication between devices in the Internet of Things (IoT) ecosystem. Two primary communication protocols, HTTP and MQTT, are pivotal in enabling device interactions within the AWS IoT Core infrastructure.

MQTT, a lightweight and efficient publish/subscribe protocol, is well-suited for scenarios requiring real-time data exchange, low latency, and low bandwidth consumption. On the other hand, HTTP, a request-response protocol, is suitable for scenarios where devices need to communicate intermittently, leveraging standard web protocols.

The devices can Publish the data to AWS IoT Core by doing a post request to the endpoint with a topic-specific URL viz.

https://IoT_data_endpoint/topics/url_encoded_topic_name?qos=1

1	https://IoT_data_endpoint/topics/url_encoded_topic_name?qos=1

Where,

IoT_data_endpoint is an AWS IoT device data endpoint, an account and region-specific endpoint. Viz a2oxhbbblky26a-ats.iot.ap-south-1.amazonaws.com

url_encoded_topic_name is the full topic name of the message being sent.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Steps to Follow

Step 1 – Creating a Thing and Obtaining the AWS IoT device data endpoint

The first step is to create a thing in AWS IoT core and download the device certificate, root CA certificate, and public and private keys.

Here, one thing named “dynamodb2superset” is created, and the certificates and keys are saved in the local machine.

step1

Next, navigate to the bottom of the AWS IoT Menu, click Settings, and copy the AWS IoT device data endpoint.

step1b

Step 2 – Creating AWS IoT Rule for message routing

Here we are creating an AWS IoT rule named ‘testrule’ for basic ingest. The IoT rule action for this rule is set to republish in a separate topic named ‘republished’ for verification.

step2

While creating the rule action, a new AWS IAM role, ‘IoT_Core_republish’ is created.

Step 3 – Publishing to AWS IoT Core

We use the following Python script in a Jupyter Notebook to publish the data to AWS IoT Core here.

import random
import requests
from time import sleep
import json
from datetime import datetime
# Define the IoT Core HTTP endpoint URL
iotcore_endpoint = 'https://a2*********-ats.iot.ap-south-1.amazonaws.com:8443/topics/*******?qos=1'
# Define the paths to your client certificate and private key files
client_cert_file = r"C:\Users\RishiRaj\Downloads\dynamo2superset-certificate.pem"
private_key_file = r"C:\Users\RishiRaj\Downloads\dynamo2superset-private.pem"
current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
random_name = ['harrier','mig','sukhoi','northrop-gruman','lockheed martin', 'boeing']
message = {
    "date": current_time,
    "deviceid": random_name[int(random.uniform(0,6))],
    "temperature": random.uniform(10, 100),
    "humidity": random.uniform(50,90)
    }
senddata=str(message)
# Make the POST request to the IoT Core HTTP endpoint with client certificate and private key
response = requests.post(iotcore_endpoint,data=senddata, cert=(client_cert_file, private_key_file))
sleep(.1)
# Check the response status code to confirm if the request was successful
if response.status_code == 200:
    print("Data sent successfully.")
else:
    print(f"Failed to send data. Status code: {response.status_code}")
    print(response.text)

import random

import requests

from time import sleep

import json

from datetime import datetime

# Define the IoT Core HTTP endpoint URL

iotcore_endpoint = 'https://a2*********-ats.iot.ap-south-1.amazonaws.com:8443/topics/*******?qos=1'

# Define the paths to your client certificate and private key files

client_cert_file = r"C:\Users\RishiRaj\Downloads\dynamo2superset-certificate.pem"

private_key_file = r"C:\Users\RishiRaj\Downloads\dynamo2superset-private.pem"

current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")

random_name = ['harrier','mig','sukhoi','northrop-gruman','lockheed martin', 'boeing']

message = {

"date": current_time,

"deviceid": random_name[int(random.uniform(0,6))],

"temperature": random.uniform(10, 100),

"humidity": random.uniform(50,90)

}

senddata=str(message)

# Make the POST request to the IoT Core HTTP endpoint with client certificate and private key

response = requests.post(iotcore_endpoint,data=senddata, cert=(client_cert_file, private_key_file))

sleep(.1)

# Check the response status code to confirm if the request was successful

if response.status_code == 200:

print("Data sent successfully.")

else:

print(f"Failed to send data. Status code: {response.status_code}")

print(response.text)

Publishing data to AWS IoT core without Basic Ingest:

To Publish data to the IoT core broker, we need to modify the endpoint, as mentioned earlier. Here, we will publish a topic called ‘topic1’ with QoS 1. Hence, the new endpoint will be:

iotcore_endpoint = 'https://a2*********-ats.iot.ap-south-1.amazonaws.com:8443/topics/topic1?qos=1'

1	iotcore_endpoint = 'https://a2*********-ats.iot.ap-south-1.amazonaws.com:8443/topics/topic1?qos=1'

Once done, we can publish a sample payload using the code above. Following is a screenshot of the received payload.

step3

Publishing data to AWS IoT core with Basic Ingest

Similarly, we must change the endpoint to publish data to AWS IoT Core, bypassing the broker. The topic name will become $aws/rules/rule_name where rule_name is the AWS IoT Rule we publish. We publish the ‘testrule’ rule, created in the earlier step. Hence, the sample endpoint will be:

iotcore_endpoint = 'https://a2*********-ats.iot.ap-south-1.amazonaws.com:8443/topics/$aws/rules/testrule?qos=1'

1	iotcore_endpoint = 'https://a2*********-ats.iot.ap-south-1.amazonaws.com:8443/topics/$aws/rules/testrule?qos=1'

Following is a screenshot of the received payload, which was republished to another topic named ‘republished’ by ‘testrule’s IoT rule action.

step3b

Conclusion

The selection between utilizing MQTT or HTTP for publishing data to AWS IoT Core hinges upon the particular requirements and characteristics of the IoT application. MQTT, characterized by its lightweight publish/subscribe model, stands out in scenarios that demand real-time data exchange, low latency, and optimal bandwidth utilization. Its proficiency becomes evident when devices necessitate rapid and asynchronous communication, rendering it a fitting choice for applications with stringent real-time prerequisites.

Conversely, HTTP, functioning as a request-response protocol, proves advantageous when devices communicate intermittently or when simplicity, seamless integration, and adherence to standard web protocols are paramount. Its provision of a RESTful interface eases integration with existing web infrastructure and ensures compatibility across various devices.

Ultimately, the decision between MQTT and HTTP is contingent upon the specific requirements of the use case. MQTT is favored for applications prioritizing low latency, bidirectional communication, and efficient resource utilization. Meanwhile, HTTP is selected for scenarios emphasizing simplicity, ease of integration, and compatibility with standard web protocols. AWS IoT Core’s flexibility in supporting both MQTT and HTTP communication types empowers developers to customize their approach based on the distinctive demands of their individual IoT projects.

Drop a query if you have any questions regarding MQTT or HTTP and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, Microsoft Gold Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. What are the limitations of using HTTP in AWS IoT Core?

ANS: – HTTP, a request-response protocol, may not be as suitable for applications with rapid and asynchronous communication needs. HTTPS doesn’t support a clientId value like MQTT and iot:Connection.Thing.ThingName policy variable is not available, making it hard to manage IoT Policies. Moreover, this is a “Publish only” method, i.e., devices can send HTTPS messages to the IoT Core, not vice versa.

2. Can I switch between HTTP and MQTT for data ingestion in AWS IoT Core?

ANS: – Yes, AWS IoT Core provides the flexibility to use HTTP and MQTT for data ingestion. Developers can choose the protocol that aligns with the specific requirements of their IoT applications.

WRITTEN BY Rishi Raj Saikia

Rishi Raj Saikia is working as Sr. Research Associate - Data & AI IoT team at CloudThat. He is a seasoned Electronics & Instrumentation engineer with a history of working in Telecom and the petroleum industry. He also possesses a deep knowledge of electronics, control theory/controller designing, and embedded systems, with PCB designing skills for relevant domains. He is keen on learning new advancements in IoT devices, IIoT technologies, and cloud-based technologies.