Amazon Timestream Is Finally Released — Is It Worth Your Time?

Amazon Timestream Is Finally Released — Is It Worth Your Time?

In this article, we will look at Amazon Timestream’s features, benefits, limitations, and pricing, followed by a short demo and conclusion.

Time series data is exponentially growing in volume and popularity. It’s essentially a collection of numerical values assigned to the specific timestamps and it’s used to describe how things change over time. Data from IoT devices, sensors, weather forecasts, click-stream, financial stock market data, or even your heart rate measurements — those are all examples of time-series.

The use cases that require to track changes over time are so pervasive that many products on the market have been designed specifically to store this type of data efficiently. During the re:Invent in 2018, AWS announced a new cloud service, Amazon Timestream, that should offer a 1000 times faster query performance and a reduction of costs by a factor of ten as compared to relational databases [1]. And the best of it is that it’s serverless, making it easy to scale as the amount of time series data stored over time keeps growing.

Even though the product has been announced already in 2018, it was not generally available until last week.

In this article, we will look at Amazon Timestream’s features, benefits, limitations, and pricing, followed by a short demo and conclusion.

Features

AWS has proven many times that they want to make it easy for their customers to use their services by abstracting away the IT Operations and things needed to start using their products. Timestream is no exception — there is no Ops, even if you wanted to do it. All that you can configure for your database is the database name, and for how long you would like to keep the data inside of this database. Technically speaking, you get to choose the retention period for the short-term in-memory storage tierand for the long-term magnetic store. This distinction is crucial, as it highlights one of the main advantages of using Timestream over typical databases. Let’s explain why.

Hot vs. cold data

Imagine that you are managing a fleet of servers, and you need to provide a real-time dashboard that displays all relevant metrics with respect to memory and CPU utilization, etc. For this purpose, you are continuously feeding new measurements over time to your time-series database. Since you want to see only the “fresh” recently updated data for metrics collected within the last couple of hours, you don’t need to keep the data from last month in memory — this would be a waste of expensive resources. At the same time, you likely don’t want to throw this last week’s data away because you may want to use it to analyze trends over time and detect anomalies that can only be detected when looking at a larger time window.

Image for post

Hot vs. cold data- image by the author

We often call those recently updated, frequently accessed records hot data. In contrast, everything that happened a long time ago and that you access only for a few specific analytics and data science use cases is referred to as *cold data *[2]. Amazon Timestream allows you to store both within a single database, which is really useful. Before Timestream, in order to provide low-latency dashboards, you would usually have to cache the hot data in some in-memory data stores such as Redis, and the cold data would have to be stored in some other database that is able to handle large amounts of data without breaking the bank.

data-engineering serverless software-engineering programming aws

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Software Developer vs Software Engineer — Differences: Bogus or Real?

In this article, see if there are any differences between software developers and software engineers. What you’re about to read mostly revolves around my personal thoughts, deductions, and offbeat imagination. If you have different sentiments, add them in the comment section, and let’s dispute! So, today’s topic…

Managing Data as a Data Engineer:  Understanding Data Changes

Understand how data changes in a fast growing company makes working with data challenging. In the last article, we looked at how users view data and the challenges they face while using data.

Data Observability: The Next Frontier of Data Engineering

Data Observability: The Next Frontier of Data Engineering. Introducing a better approach to building data pipelines

Managing Data as a Data Engineer — Understanding Users

Understanding how users view data and their pain points when using data. In this article, I would like to share some of the things that I have learnt while managing terabytes of data in a fintech company.

Do you have the Software Engineer and Data Scientist skills?

Becoming a reliable software engineer or data scientist developer, and prepare for production level coding requires a few techniques.