How to Process and Present IoT Data on Microsoft Azure?

In my previous article on “ESP8266 Connectivity with Azure IoT Hub” we have seen, how to connect IoT device with Azure IoT Hub. We also discussed about how to configure and connect ESP8266 with Azure IoT Hub for hyper-scale messaging in a secured way.

We have data from different IoT devices on Cloud, but it is futile unless we can process that data and present it in some way. When we have tremendous number of IoT devices generating massive unstructured data every second, then you will eventually end up having pile of data with no meaning. Therefore, it is important to process that data and generate some meaningful information from it. As we are in Cloud Computing era, we have some superb services to help us process and present that data. There are many such services out there, but if we talk specifically about Microsoft Services, then we can use Azure Stream Analytics for processing and Microsoft Power BI for presenting data.

So, what are these services?

Azure Stream Analytics helps in streaming millions of events per second in real time and allow us to do processing over that data with great reliability and performance. To know more about Azure Stream Analytics, you can refer the article on ‘Azure Stream Analytics’ written by Arzan Amaria.

Microsoft Power BI which we are going to use for presenting the processed data is a SaaS Tool and is a collection of services and features to visualize, share and collaborate data.

Now that we know about Azure Stream Analytics and Microsoft Power BI, let’s see how to create connection between IoT Hub which will receive data from different IoT devices, then fetch and process that data using Stream Analytics and finally visualize & present data using Power BI.

Following are the steps to do the same:

Configuring Stream Analytics

1. Create Stream Analytics Job

   • Navigate to Microsoft Azure Portal http://portal.azure.com/ and login. Click on NEW icon at top right corner.
   • Select Data + Analytics > Stream Analytics Job
     
     
   • In the next blade, fill up the form with following details
     1. Job Name: You can use any name that you can identify easily later
     2. Resource Group: Select Create New and enter name for new Resource Group
     3. Location: You can leave as default
   • Click on Create. It may take some time for job to get provisioned and become available

2. Define the Input Data Stream
   

   Data will come as Data Stream from IoT Hub

   • Select Inputs from Overview in your Stream Analytics Job
     
   • Click on ADD in the new blade
   • In the next blade, fill up the form with following details
     1. Input Alias: Name for Input of Stream Analytics
     2. Source: IoT Hub
     3. IoT Hub: Choose the IoT which you created previously
     4. Leave other settings as default
   • Click on Create
   • On the Serialization settings form, leave things as is
   • Click on check mark at the right bottom corner (after some time a new input will be listed)

3. Define the Output Data Stream

   • Select Inputs from Overview in Stream Analytics Job
     
   • Click on Add in the new blade
   • In the next blade, fill up the form with following details
     1. Output Alias: DeviceBI
     2. Sink: Select Power BI
     3. Authorize Connection: Click on Authorize Button and enter credentials of Power BI
     4. Dataset Name: IoTDatasetSet
     5. Table Name: IoTDataTable
   • Click on Create to Proceed

4. Select Query from Input in Stream Analytics Job

   • Write the query according to your need, below is an example
     
     SELECT Id, Celsius, Humidity, hPa, System.Timestamp AS Timestamp INTO [DeviceBI] FROM [DeviceInputStream]

     1 2 3 4 5 6

     SELECT    Id, Celsius, Humidity, hPa, System.Timestamp AS Timestamp INTO      [DeviceBI] FROM      [DeviceInputStream]

   • Click on Save
   • After the query is saved, click on Start to start Stream Analytics job

Build Report in Power BI

1. Go to Datasets on left navigation panel, IoTDataSet should appear there within few minutes once Stream Analytics gets started
2. Click on IoTDataSet Dataset to open the Report Designer
3. Select Line Chart from Visualization Toolbox
4. From Field Toolbar on right navigation panel select the desired fields to be shown in graph (Celsius, hpa, humidity, etc.)
5. Drag and drop timestamp from fields to the Axis option in Visualization Toolbar
   

There are many other things which you can do using Stream Analytics and Power BI by processing the data in different ways and creating different type of graphs.

Interested to learn more about IoT? We conduct following BootCamps for IoT:

1. Fundamentals of IoT – Level 1
2. Working with Electronics in IoT – Level 2
3. Cloud Robotics and Advanced IoT Architecture – Level 3
4. Designing and Implementing IoT Solutions – Level 4

Feel free to ask questions or provide your valuable suggestions as comments below. Please share if you like the article.

WRITTEN BY CloudThat