Using AWS to ingest and analyze data from an IoT device: a simple example with Aurora and Atheana. If the image contains the face of a person, a record is written by the Lambda function in an Aurora MySQL serverless together with metadata output from the Rekognition ML algorithm: does the person have eyeglass?
With the Internet of Things quickly becoming a thing of the present (rather of the future…) the number of devices sending collected on the field is increasing exponentially and so does the amount of data, thus data ingestion and analysis has become of the hottest topics of the current IT landscape. AWS offers a wide range of services that allow us to ingest, collect, store, analyze, and visualize huge amounts of data quickly and efficiently.
In this brief article, we would like to present a very simple real-world application we developed as a proof of concept demonstration to show the data ingestion and analysis pipeline in AWS and IoT events and conferences.
We customized an existing Nespresso coffee machine to take photos of people making coffees using custom electronics, a Raspberry Pi Zero, and a micro camera. The image is immediately uploaded to S3 and an AWS Lambda triggered by the upload analyzes the image using Amazon Rekognition. After the analysis of the image is complete, if the image contains the face of a person, a record is written by the Lambda function in an Aurora MySQL serverless together with metadata output from the Rekognition ML algorithm: does the person have eyeglass? beard? mustaches? is she/he smiling? Finally, a very simple web application was developed and connected to the database to show statistics.
Furthermore, an AWS Athena query cleans the data and moves them to a new S3 bucket as parquet files.
Obviously, for our trivial application, many of these steps are redundant but they aim to demonstrate the power of AWS building blocks in creating very complex data pipelines.
A scheme of the proposed infrastructure is shown below.
Hereafter we describe all the steps of a common data ingestion and transformation and how we are doing them in our trivial application. Let’s dive deep!
Top Web & Mobile Application Development Company in India & USA. We specialize in Golang, Ruby on Rails, Symfony, Laravel PHP, Python, Angular, Mobile Apps, Blockchain, & Chatbots
See the top articles of the IoT Zone from the last 2 years, with topics like Raspberry Pi, MQTT, IoT security, IoT in agriculture and healthcare, Arduino, and more!
Hire Top IoT App developers in India on an hourly / full-time basis for your custom web development needs. Strict NDA, 16+ years exp & 2500+ clients|450+ Experts
IoT Development Company. IoT is the roadmap to smart future. IoT is spreading its way in various sectors across the globe. Hire IoT Developers.