Using AWS to ingest and analyze Data From An IoT Device: Example with Aurora And Atheana

Using AWS to ingest and analyze Data From An IoT Device: Example with Aurora And Atheana

Using AWS to ingest and analyze data from an IoT device: a simple example with Aurora and Atheana. If the image contains the face of a person, a record is written by the Lambda function in an Aurora MySQL serverless together with metadata output from the Rekognition ML algorithm: does the person have eyeglass?

With the Internet of Things quickly becoming a thing of the present (rather of the future…) the number of devices sending collected on the field is increasing exponentially and so does the amount of data, thus data ingestion and analysis has become of the hottest topics of the current IT landscape. AWS offers a wide range of services that allow us to ingest, collect, store, analyze, and visualize huge amounts of data quickly and efficiently.

In this brief article, we would like to present a very simple real-world application we developed as a proof of concept demonstration to show the data ingestion and analysis pipeline in AWS and IoT events and conferences.

We customized an existing Nespresso coffee machine to take photos of people making coffees using custom electronics, a Raspberry Pi Zero, and a micro camera. The image is immediately uploaded to S3 and an AWS Lambda triggered by the upload analyzes the image using Amazon Rekognition. After the analysis of the image is complete, if the image contains the face of a person, a record is written by the Lambda function in an Aurora MySQL serverless together with metadata output from the Rekognition ML algorithm: does the person have eyeglass? beard? mustaches? is she/he smiling? Finally, a very simple web application was developed and connected to the database to show statistics.

Furthermore, an AWS Athena query cleans the data and moves them to a new S3 bucket as parquet files.

Obviously, for our trivial application, many of these steps are redundant but they aim to demonstrate the power of AWS building blocks in creating very complex data pipelines.

A scheme of the proposed infrastructure is shown below.

Hereafter we describe all the steps of a common data ingestion and transformation and how we are doing them in our trivial application. Let’s dive deep!

serverless amazon-athena amazon-aurora iot

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Best Custom Web & Mobile App Development Company

Top Web & Mobile Application Development Company in India & USA. We specialize in Golang, Ruby on Rails, Symfony, Laravel PHP, Python, Angular, Mobile Apps, Blockchain, & Chatbots

The Best of IoT: Tutorials and Articles

See the top articles of the IoT Zone from the last 2 years, with topics like Raspberry Pi, MQTT, IoT security, IoT in agriculture and healthcare, Arduino, and more!

Top IoT App Developers India | Online IoT App Programmers

Hire Top IoT App developers in India on an hourly / full-time basis for your custom web development needs. Strict NDA, 16+ years exp & 2500+ clients|450+ Experts

2020/06/04時点 Aurora MySQL 5.6 vs 5.7 vs Serverless 機能比較一覧

Aurora MySQLの各エディション×各機能が使える/使えないの一覧表が欲しいなーと思ったので作ってみました。

Internet of Things (IoT): The Roadmap to a Smart Future - Mobinius

IoT Development Company. IoT is the roadmap to smart future. IoT is spreading its way in various sectors across the globe. Hire IoT Developers.