Practical Change Data Streaming Use Cases with Apache Kafka & Debezium

Practical Change Data Streaming Use Cases with Apache Kafka & Debezium

Best practices for running Debezium in production on and off Kubernetes, and the many use cases enabled by Kafka Connect's single message transformations. How to leverage CDC for reliable microservices integration, e.g. using the outbox pattern, as well as many other CDC applications (maintaining audit logs, driving streaming queries). What's the best framework we should use? How can we run this on Kubernetes?

Gunnar Morling discusses practical matters, best practices for running Debezium in production on and off Kubernetes, and the many use cases enabled by Kafka Connect's single message transformations. He talks about how to leverage CDC for reliable microservices integration, e.g. using the outbox pattern, as well as many other CDC applications (maintaining audit logs, driving streaming queries).

Angular 9 Tutorial: Learn to Build a CRUD Angular App Quickly

What's new in Bootstrap 5 and when Bootstrap 5 release date?

What’s new in HTML6

How to Build Progressive Web Apps (PWA) using Angular 9

What is new features in Javascript ES2020 ECMAScript 2020

Learn the basics of Microservices, Docker, and Kubernetes.

Learn the basics of Microservices, Docker, and Kubernetes.

Learn the basics of Microservices, Docker, and Kubernetes. Learn the basics of Microservices, Docker, and Kubernetes.

Introduction to Microservices, Docker, and Kubernetes

Learn the basics of Microservices, Docker, and Kubernetes. Code demo starts at 18:45. I mess up the terminal for the first few minutes, but I fix it by 21:50. Audio gets echoey a few times, but it goes away quickly. Sorry about that!

Deployment YAML: https://pastebin.com/rZa9Dm1w

Dockerfile: https://morioh.com/p/59a594cc28dc

How Kubernetes Helps to Enable DevOps

How Kubernetes Helps to Enable DevOps

The automation and infrastructural capabilities of Kubernetes makes it an ideal technological partner for DevOps. In this article, you'll see 10 ways Kubernetes enables DevOps

The automation and infrastructural capabilities of Kubernetes makes it an ideal technological partner for DevOps. In this article, you'll see 10 ways Kubernetes enables DevOps

DevOps was an idea before its time.

It actually took a while for technology to catch up and fully implement the principles and vision of DevOps, but that is how innovation works.

A new set of tools in our toolboxes (though it's now taking up more and more mindshare) is end-to-end automation, and Kubernetes helps you manage that. Kubernetes is an open source framework for "automating deployment, scaling, and management of containerized applications." Originally introduced by Google, Red Hat, and others pushed it forward the last 3-4 years.

At the 2017 All Day DevOps conference, Siamak Sadeghianfar laid out 10 ways Kubernetes enables DevOps. We thought it was worth revisiting what he had to say as **Kubernetes **only becomes more prominent.

**So, how exactly does Kubernetes enable DevOps? Let's dive into what Siamak had to say: **

1. Deployment automation.

You should automate every step of your delivery pipeline. Kubernetes automates the deployment of containers. Each component becomes a container image.

**2. Infrastructure as code. **

With **Kubernetes **your entire infrastructure is code. This means any part in your application (databases, ports, access controls, etc.) can be described in a way **Kubernetes **can use. For example, you store your infrastructure code in a version-control repository. **Kubernetes **takes the code, and based on instruction, deploys, and maintains your infrastructure. This happens automatically and consistently.

3. Configuration as code.

**Kubernetes **allows you to "configure as code." Traditionally, an admin had to run configuration scripts manually to make sure they get the right one. **Kubernetes **keeps the file in the source repo. This allows you to describe where the file needs to go in the container and how the application consumes it. Additionally, it can be version controlled.

4. Immutable infrastructure.

Since the inception of servers, there's been one problem: every time you put out a fire you change the state of the virtual machine. Eventually, you don't know exactly what the server looks like and you can't recreate it, even though you know it works. (These virtual machines are referred to as snowflakes.)

In Kubernetes, new containers are immutable; they are created when there is a problem with the original state. So, you can be confident that you know exactly how the new environment is configured.

5. On-Demand Infrastructure.

Developers can create hybrid services and infrastructure on-demand from the self-service catalog. This gives control to developers to get the resources they need, yet allows operations to control the configuration of the services. Kubernetes follows open service and API standards so you can expose cloud services.

6. Environment consistency.

Build once, and deploy into production-like environments everywhere. Kubernetes allows you to build a golden image and use the exact same image for every single place you deploy your container. This gives you a consistent, production-like environment so it is the exact same whether you are on your local Windows or Mac development machine, a test server, etc. This helps you "shift left" so that you can see issues when you are in development instead of in production.

7. Continuous Delivery pipeline.

This is a series of automated steps to test code before it goes in production. Jez Humble says the role of Continuous Delivery is, "an automated process to prove to you a change is a bad change and it should not go into production." Continuous Integration (CI) means every change is tested. Continuous Delivery (CD) automates all the way to production.

How can you tell if you have fully implemented Continuous Delivery? Siamak says that the test is if someone walks in and says, "Can you go to production right now?" can you do it without breaking a sweat?

8. Zero downtime deployments.

When you have several deployments a day, you can't pull down production to deploy. You must have safe, rolling updates without disrupting the production traffic. Kubernetes helps with blue/green deployments so that you can set up a new environment and switch to the new one without downtime.

9. A/B Testing.

How can you evaluate proposed changes, such as changing copy or the color of buttons? A/B testing delivers different versions to different customers so you can get real-world test results. Kubernetes manages routing traffic to different versions. The same version with slight modifications are possible.

10. Cross-functional collaboration.

This is shared access to environments with granular control. Kubernetes goes beyond the silos so everyone has the same set of environments, but you can grant access to different roles and allow different roles to do different things. For instance, developers may be able to push to production, while Infosec may have view access, and QA may have access to live containers, but not building or deploying. The entire team can access production, but operations is the only one who can make containers.

The growing popularity of Kubernetes is undeniable. Luckily, with these 10 capabilities, Siamak has laid out a great path for us all to follow as they continue to not only make their way into the DevOps pipeline, but in many ways enable it.

Why Kubernetes is Not Enough

Why Kubernetes is Not Enough

Farfetch’s mission is to be the global technology platform for luxury fashion. One of the transformations that we have been working on is migrating our 500 microservices to run on containers in Kubernetes. We built an abstraction layer called blueprints to capture all metadata of our services and its dependencies in a well-defined, source-controlled, always up-to-date framework. This has enabled us to be more agile and in control while we are adapting to the Kubernetes ecosystem.

Farfetch’s mission is to be the global technology platform for luxury fashion. We sell products from over 1000 boutiques and brands from around the world, ship to customers in 190 countries with the help of 3000 employees in 13 different offices. Our platform runs on an active-active geo-distributed infrastructure in multiple regions, on thousands of servers, petabyte scale storage and a global software defined network.

The nature of this global setup and scale brings unique challenges and opportunities. One of the transformations that we have been working on is migrating our 500 microservices to run on containers in Kubernetes. As we embarked on this journey we realized the need to improve upon not just our technology but also our processes. So we built an abstraction layer called blueprints to capture all metadata of our services and its dependencies in a well-defined, source-controlled, always up-to-date framework. This has enabled us to be more agile and in control while we are adapting to the Kubernetes ecosystem.