Akka-Streams: All About Graphs!

Graphs
Till now we know how to create a linear pipeline/linear graph. But in real life scenario we generally don’t have linear graphs to implement. The graphs can be complex. In Akka Streams computation graphs are written in a more graph-resembling DSL. It aims to make translating graph drawings (e.g. from notes taken from design discussions, or illustrations in protocol specifications) to and from code simpler.

Graphs are used to perform fan-in and fan-out operations. You can consider graph operations as junctions(multiple flows connected at a single point).

Fan-in : It takes multiple inputs and produces a single output.
Fan-out : It produces multiple outputs by taking a single input.

#akka #akka-streams #big data and fast data #scala #akka-streaming #graphs #graphs-in-akka #partialgraphs

What is GEEK

Buddha Community

Akka-Streams: All About Graphs!

Graphs
Till now we know how to create a linear pipeline/linear graph. But in real life scenario we generally don’t have linear graphs to implement. The graphs can be complex. In Akka Streams computation graphs are written in a more graph-resembling DSL. It aims to make translating graph drawings (e.g. from notes taken from design discussions, or illustrations in protocol specifications) to and from code simpler.

Graphs are used to perform fan-in and fan-out operations. You can consider graph operations as junctions(multiple flows connected at a single point).

Fan-in : It takes multiple inputs and produces a single output.
Fan-out : It produces multiple outputs by taking a single input.

#akka #akka-streams #big data and fast data #scala #akka-streaming #graphs #graphs-in-akka #partialgraphs

Getting Started with Akka-Streams

First we should know, what is a stream? So , the word stream means “steady flow of something” and here we have a flow of data. Stream is basically a sequence of data that can be infinite. Here, infinite means very huge size to fit in your system’s memory or just unknown to you.

Now, you understand streams, go through following to understand akka-streams:

Akka-Stream is a module built on top of Akka Actors. It means there are underlying actors that are actually working in transferring data.
It is a library to process and transfer a sequence of elements.
Akka Streams is nonblocking that means a certain operation does not hinder the progress of the calling thread.
It provides easy-to-use APIs to create streams that leverage the power of the Akka toolkit without explicitly defining actor behaviors and messages. That means, a relief from handling actors yourself.
We can focus on logic rather than focusing on how to manage actors and that can be much productive.

#akka #akka-streams #big data and fast data #scala #akka actor #scala #streams

Akka-Streams: All About Materialization!

In my previous blog, I discussed about the basics of akka-stream. Basically, it was an introduction to akka-stream to get you started. There I mentioned about materialization which is a really important concept in Akka-Streams. And in this blog I’m going to discuss about it and some related topics in detail. So, let’s begin
Akka-Streams Basics
Let’s revisit the basics first. So, the data processing in akka-streams is done as data flow from different stages of a graph. These stages have either one or more inputs and outputs.The basic building blocks are Sources (one output), Sinks (one input) and Flows (one input and one output). Using them, we can build linear pipelines/graphs.

#akka #akka-streams #big data and fast data #scala #akka streams #fusion #materialization

Wondering how to upgrade your skills in the pandemic? Here's a simple way you can do it.

Corona Virus Pandemic has brought the world to a standstill.

Countries are on a major lockdown. Schools, colleges, theatres, gym, clubs, and all other public places are shut down, the country’s economy is suffering, human health is on stake, people are losing their jobs and nobody knows how worse it can get.

Since most of the places are on lockdown, and you are working from home or have enough time to nourish your skills, then you should use this time wisely! We always complain that we want some ‘time’ to learn and upgrade our knowledge but don’t get it due to our ‘busy schedules’. So, now is the time to make a ‘list of skills’ and learn and upgrade your skills at home!

And for the technology-loving people like us, Knoldus Techhub has already helped us a lot in doing it in a short span of time!

If you are still not aware of it, don’t worry as Georgia Byng has well said,

“No time is better than the present”

– Georgia Byng, a British children’s writer, illustrator, actress and film producer.

No matter if you are a developer (be it front-end or back-end) or a data scientist, tester, or a DevOps person, or, a learner who has a keen interest in technology, Knoldus Techhub has brought it all for you under one common roof.

From technologies like Scala, spark, elastic-search to angular, go, machine learning, it has a total of 20 technologies with some recently added ones i.e. DAML, test automation, snowflake, and ionic.

How to upgrade your skills?

Every technology in Tech-hub has n number of templates. Once you click on any specific technology you’ll be able to see all the templates of that technology. Since these templates are downloadable, you need to provide your email to get the template downloadable link in your mail.

These templates helps you learn the practical implementation of a topic with so much of ease. Using these templates you can learn and kick-start your development in no time.

Apart from your learning, there are some out of the box templates, that can help provide the solution to your business problem that has all the basic dependencies/ implementations already plugged in. Tech hub names these templates as xlr8rs (pronounced as accelerators).

xlr8rs make your development real fast by just adding your core business logic to the template.

If you are looking for a template that’s not available, you can also request a template may be for learning or requesting for a solution to your business problem and tech-hub will connect with you to provide you the solution. Isn’t this helpful 🙂

Confused with which technology to start with?

To keep you updated, the Knoldus tech hub provides you with the information on the most trending technology and the most downloaded templates at present. This you’ll be informed and learn the one that’s most trending.

Since we believe:

“There’s always a scope of improvement“

If you still feel like it isn’t helping you in learning and development, you can provide your feedback in the feedback section in the bottom right corner of the website.

#ai #akka #akka-http #akka-streams #amazon ec2 #angular 6 #angular 9 #angular material #apache flink #apache kafka #apache spark #api testing #artificial intelligence #aws #aws services #big data and fast data #blockchain #css #daml #devops #elasticsearch #flink #functional programming #future #grpc #html #hybrid application development #ionic framework #java #java11 #kubernetes #lagom #microservices #ml # ai and data engineering #mlflow #mlops #mobile development #mongodb #non-blocking #nosql #play #play 2.4.x #play framework #python #react #reactive application #reactive architecture #reactive programming #rust #scala #scalatest #slick #software #spark #spring boot #sql #streaming #tech blogs #testing #user interface (ui) #web #web application #web designing #angular #coronavirus #daml #development #devops #elasticsearch #golang #ionic #java #kafka #knoldus #lagom #learn #machine learning #ml #pandemic #play framework #scala #skills #snowflake #spark streaming #techhub #technology #test automation #time management #upgrade

Video Streaming With Akka Streams

First of all, do not let the title deceive you — it will not be simple println(“Hello world”) but with Actor System. Today, we will implement your first (sorry if I assumed wrongly) video streaming service. Namely, I will use Akka HTTP and Streams to create a REST API capable of streaming a video file, in mp4 format, in a way that matches the expectations of HTML5 <video> tag. Besides, I will also add a few words about Akka and some components like Akka Streams to give you some theoretical background before you start coding. But first, one question.

Why Video Streaming?

There are three main reasons behind choosing video streaming as the topic for today’s article. The first one is that it is an absorbing and complex subject for me, especially on a large scale (like Netflix) and I have always wanted to learn more about it. The next reason is that it is a niche topic, so a project based around video streaming can be a great addition to anyone’s portfolio - this one is especially important for people who want to start their journey with Scala and Akka. Last but not least it is very interesting way to get familiar with Akka Streams, which in fact makes this whole operation much easier.

After this brief introduction, we can move to the first of the main topics for today.

What Is Akka And Is It Tasty?

In general, it is an open-source toolkit whose aim is to make the creation of multi-threaded and distributed applications easy and also provide runtime for such applications. Akka-based systems tend to scale very, very well. Akka is based around Actor Model and actor-based concurrency and draws lots of inspiration from Erlang. It is written in Scala but provides DSLs for both Scala and Java.

When it comes to the tastiness of Akka, if you prefer eating your source code then for sure it is very tasty. If you want to learn more about Akka I recommend starting from reading this.

Here most of the source code will be based around HTTP and Streams features and there will be almost no features from the standard Akka Actors package.

When we cover the absolute basics of Akka we can dive deeper into today’s text.

What Is Akka Streams?

Akka Streams are simply a package built on top of normal Akka Actors to help us with the processing of infinite (or finite but very, very large) sequences of data. The main motivation behind the creation of Akka Streams was problems with the correct configuration of actors in the actor system to achieve a stable flow of streamed data.

An important fact is that Akka Streams have a built-in back-pressure mechanism. Thanks to that one of the most complex problems in the streaming world, configuring the producer to react correctly when the consumer cannot keep up with load, is taken care of by a tool you use and you do not have to care about it too much.

Additionally, Akka Streams provides an API that is compliant with interfaces required by Reactive Streams SPI. As a side note, it is worth noting that Akka itself is one of the founding members of the Reactive Stream initiative.

Akka Streams checked so now we can jump to the next part of the theory.

What Is Akka HTTP?

Similarly to Akka Streams, it is a package provided by Akka creators. It is built on top of Akka Streams and Akka Actors and it helps you to interact with outside worlds via HTTP. It provides both sides of HTTP stack, so you can build a REST API and you can use it as HTTP client for sending requests to some external service.

So now, when you have some basic understating of tools which we will use to implement our backend, the last thing left to describe is the most important part of our application’s frontend.

Details Of HTML5

It is a new tag introduced in HTML 5 to replace adobe video player. As you may guess, its main responsibility is to embed media player, capable of playing videos, into an HTML document. The idea is very similar to plain old <img> tag.

Inside <video> tag you place <source> tag with two important attributes. The first one is **src **attribute used to point out video which we want to display. The second important field is type, responsible for keeping our video format

You can also write some text inside <video> </video> tags and it will be used as fallback in browsers that do not support the element. As for now, even Internet Explorer supports it so this scenario is almost impossible.

How Streaming With HTML

Although it sounds complex at the first glance, after some reading it becomes quite easy. Almost everything is ready and we only have to integrate different components.

On the backend side, most of the heavy lifting will be done by FileIO object, which will emit events chunked part of our file. Of course, the size of chunked elements is configurable. Moreover, the starting position is also configurable which will allow us to start playing our video from a certain point, not exactly from the beginning. All these, features are perfectly suited for <video> tag performing HTTP GET request with Range header to play video without downloading it first.

Examples of requests made by HTML video:

For anyone interested, request header – Range:bytes=x- is responsible for picking the starting position of our video. The first request will be send at the beginning of processing video while the second one can be sent when you decide to move to a certain point on the video timeline.

With this lengthy introduction, it is finally time for some coding. Are you happy?

Let’s Write Streaming Service

In the next few paragraphs, I will implement the backend of our streaming service and then I will create a simple web page in HTML to verify if it works correctly.

Because I like doing assumptions, I assumed that anyone interested knows some basics of Scala and SBT.

1. We will add all necessary dependencies to our **build.sbt **file, for this project we will need only 3 packages: akka-http, akka-actor-typed (in theory, package akka-actor is enough but remember to always type safe), akka-stream.

Scala

2. Now we can create the main class responsible for starting our application. I choose to extend App — for me, it is more convenient than creating main method. In the next step, we will put there Actor System and HTTP server starters.

Scala

3. After creating the main class we can add the code mentioned in previous step.

Scala

As for now, such configuration is enough. In the last step, we will add the call of bind method at the end to expose our REST API. The current configuration will run Actor System with name akka-video-stream and HTTP server on port 8090 of your local machine. Remember not to omit implicit keyword near Actor System definition, as such implicit parameter is required by signature of Http method.

4. Here we will finally implement REST API endpoint used to process requests from <video> tag.

Scala

As you can see, I have created an endpoint with URL “api/files/default”. It checks if **Range **header is present in request. If our server is able to find it, the client will get a response with code 200 (OK), otherwise, we return a response with code 416 (Range Not Satisfiable).

5. The fifth step is perfect for implementing the method which is the main course of the whole article.

Scala

Here I have done a few things:

• I have loaded the file I want to stream to the application, then based on the header from quest and file info I have calculated starting point of streaming and Content Range response header.
• With help of FileIO I have created a stream from the previously loaded file. Then I used this stream as data in HttpEntity.
• I have created HttpResposne with code 206 (Partial content), headers and responseEntity as body.

I also want to describe FileIO in more details as it is the most magical thing in this article. So, what exactly happened in this line: FileIO.fromPath(file.toPath, 1024, start) ?

It created Source (Akka Streams counterpart of Producer from Reactive Streams) from the content of file under the provided path. Each element emitted by the source will have exactly 1 MB of size. The first emitted element will be in the position marked by the start parameter, so if you set 0 as the start parameter value, the first emitted element will be the first MB of file under the provided path.

6. As we have implemented the main magic of today, we have to refactor our app to be able to use it.

We will start from changes in REST API definition:

complete(HttpResponse(StatusCodes.Ok)) => complete(stream(range))

So now instead, of simply returning OK, we call our stream method with range as a parameter and start streaming.

We have to remember that our API is still not exposed so we have to do the following modification to the fragment responsible for starting HTTP server.

Http().newServerAt(host, port) =>Http().newServerAt(host, port).bind(Streamer.route)

Et voila, now we have working backend and our REST API is accessible for anyone interested. Now we just have to test it.

#java #tutorial #scala #html #akka #streaming #streams