Let’s talk about this feature of Azure called stream analytics and how to detect an anomaly before it becomes a failure.

Data stream is a set of data that is coming through and is very transient, it’s not sitting in a traditional SQL database. If we had so , we can just run a batch job and run SQL query over that data and extract whatever insights we want under that data.

But what if we have data that is just passing through an event hub ?

How do we run queries, get reports, raise alerts if something becomes unusual ?

So we have this feature called Stream Analytics.

Streaming Analytics is a querying, alerting and monitoring tool that monitors streams instead of stationary databases. Now the use case that Microsoft presents is Internet of Things solution.
​
Say, if I have a watch or other devices and got millions of these devices out in the world and they are feeding back into an IOT Hub or a regular event hub. We may want to monitor those millions of events per second and detect something that is outside of a normal range and then to raise an alert or take a particular action based on detecting an above average quantity of those data.
​
Another example , say there are logs as people are visiting my website and writing log with 200 status codes and what resource they are viewing . It’s just a continuous stream of data of status code and resource path. So what if I can monitor log like that and be able to say if more than 20 of these 404/500 events happened within the last five minutes, I want to raise an alert by sending SMS message. It is real time analytics of a stream.

#anomaly-detection #machine-learning #streaming-analytics #azure