The current world is heavily dependent on data. Everyone is generating large amount. It is becoming challenge reading large amount of data and then process it and finally perform some action on that data.

In this post we will be creating a data pipeline where in we will be performing three responsibilities

Read data from Kafka topic
Process it using Logstash
Dump the data to elastic search and then visualize the data using Kibana

Links :

• Elasticsearch and Kibana Installation
  http://selftuts.com/elasticsearch-and-kibana-installation-using-docker-compose/
  https://www.youtube.com/watch?v=BYcXvhJTDpg

• Logstash Installation
  http://selftuts.com/install-logstash-on-ubuntu-18-04/
  https://www.youtube.com/watch?v=xE2IjWE6EQM

• Kafka Installation
  http://selftuts.com/install-kafka-and-kafka-manger-using-docker-compose/

• Kafka for beginners
  https://www.youtube.com/watch?v=Hl61x0s3yeQ&list=PLxoOrmZMsAWxXBF8h_TPqYJNsh3x4GyO4

• Data pipeline using Kafka and elasticsearch
  http://selftuts.com/create-data-pipeline-using-kafka-and-elasticsearch-logstash-kibana/

#kafka #elasticsearch #logstash #kibana