Enoch Barcenas

Enoch Barcenas

1600325754

Database Migrations on Kubernetes using Helm Hooks

TL;DR: you can leverage the pre-install and pre-upgrade Helm hooks to run database migrations before your application is installed or updated. Keep reading to understand why simpler solutions might not be good ideas and a couple of gotchas when using Helm hooks.

If your application uses a relational database, your schema will evolve over time. Before deploying a new version of you application, you will have to ensure that the database schema is up to date. This article is not about how to generate and manage schema migrations (there are multiple tools to handle that) but how to apply them as part of the application deployment process on Kubernetes.

Running migrations within the application

You could decide to run migrations automatically as part of your service startup. This might seem like a good idea: it guarantees that your service will not start before the migrations are applied and removes the risk of running an application on an outdated schema.

But when running on Kubernetes, this poses a couple of issues:

  • every time a new pod is created or restarted, it will try to run the migrations again. If your migration scripts are written correctly, this should not be an issue, but it’s not a clean design.
  • if the migration takes a while (think adding a column on a huge table), your pod might miss its readiness check and be killed before the migration is done. You could increase the initial delay for the pod’s readiness check, but then it makes it hard to understand if a pod is not starting because it’s applying database migrations or because of a different issue.
  • depending on the disruption budget of your deployment (how many pods can be added/removed at any point of time during a deployment), you may have 2 identical migrations running at the same time, resulting in conflicts and errors.

#kubernetes #helm #devops #database

What is GEEK

Buddha Community

Database Migrations on Kubernetes using Helm Hooks
Christa  Stehr

Christa Stehr

1602964260

50+ Useful Kubernetes Tools for 2020 - Part 2

Introduction

Last year, we provided a list of Kubernetes tools that proved so popular we have decided to curate another list of some useful additions for working with the platform—among which are many tools that we personally use here at Caylent. Check out the original tools list here in case you missed it.

According to a recent survey done by Stackrox, the dominance Kubernetes enjoys in the market continues to be reinforced, with 86% of respondents using it for container orchestration.

(State of Kubernetes and Container Security, 2020)

And as you can see below, more and more companies are jumping into containerization for their apps. If you’re among them, here are some tools to aid you going forward as Kubernetes continues its rapid growth.

(State of Kubernetes and Container Security, 2020)

#blog #tools #amazon elastic kubernetes service #application security #aws kms #botkube #caylent #cli #container monitoring #container orchestration tools #container security #containers #continuous delivery #continuous deployment #continuous integration #contour #developers #development #developments #draft #eksctl #firewall #gcp #github #harbor #helm #helm charts #helm-2to3 #helm-aws-secret-plugin #helm-docs #helm-operator-get-started #helm-secrets #iam #json #k-rail #k3s #k3sup #k8s #keel.sh #keycloak #kiali #kiam #klum #knative #krew #ksniff #kube #kube-prod-runtime #kube-ps1 #kube-scan #kube-state-metrics #kube2iam #kubeapps #kubebuilder #kubeconfig #kubectl #kubectl-aws-secrets #kubefwd #kubernetes #kubernetes command line tool #kubernetes configuration #kubernetes deployment #kubernetes in development #kubernetes in production #kubernetes ingress #kubernetes interfaces #kubernetes monitoring #kubernetes networking #kubernetes observability #kubernetes plugins #kubernetes secrets #kubernetes security #kubernetes security best practices #kubernetes security vendors #kubernetes service discovery #kubernetic #kubesec #kubeterminal #kubeval #kudo #kuma #microsoft azure key vault #mozilla sops #octant #octarine #open source #palo alto kubernetes security #permission-manager #pgp #rafay #rakess #rancher #rook #secrets operations #serverless function #service mesh #shell-operator #snyk #snyk container #sonobuoy #strongdm #tcpdump #tenkai #testing #tigera #tilt #vert.x #wireshark #yaml

Nels  Franecki

Nels Franecki

1620170460

How to migrate helm v2 to helm v3

This article will guide you about how to migrate helm v2 to helm v3 along with currently running resources in the environment.


In my previous article we have seen how to install helm version 2. Now in this article we are going to migrate currently running helm v2 configuration and its running resources to helm v3. To perform migration of helm v2 to helm v3 we are going to install plugin called 2to3 in the helm3.

#devops #helm #kubernetes #how to migrate helm v2 to helm v3 #migration helm v2 to helm v3

Enoch Barcenas

Enoch Barcenas

1600325754

Database Migrations on Kubernetes using Helm Hooks

TL;DR: you can leverage the pre-install and pre-upgrade Helm hooks to run database migrations before your application is installed or updated. Keep reading to understand why simpler solutions might not be good ideas and a couple of gotchas when using Helm hooks.

If your application uses a relational database, your schema will evolve over time. Before deploying a new version of you application, you will have to ensure that the database schema is up to date. This article is not about how to generate and manage schema migrations (there are multiple tools to handle that) but how to apply them as part of the application deployment process on Kubernetes.

Running migrations within the application

You could decide to run migrations automatically as part of your service startup. This might seem like a good idea: it guarantees that your service will not start before the migrations are applied and removes the risk of running an application on an outdated schema.

But when running on Kubernetes, this poses a couple of issues:

  • every time a new pod is created or restarted, it will try to run the migrations again. If your migration scripts are written correctly, this should not be an issue, but it’s not a clean design.
  • if the migration takes a while (think adding a column on a huge table), your pod might miss its readiness check and be killed before the migration is done. You could increase the initial delay for the pod’s readiness check, but then it makes it hard to understand if a pod is not starting because it’s applying database migrations or because of a different issue.
  • depending on the disruption budget of your deployment (how many pods can be added/removed at any point of time during a deployment), you may have 2 identical migrations running at the same time, resulting in conflicts and errors.

#kubernetes #helm #devops #database

Introduction to Helm - Package Manager for Kubernetes

**Introduction to Kubernetes and Helm **

Kubernetes is one of the best platforms to deploy and manage containerized applications. But deploying such containerized applications to Kubernetes can be challenging. You have to write a detailed YAML file to deploy resources like pods, deployments, and services on Kubernetes that’s where Helm comes into the play. Helm is a package manager for Kubernetes; it’s the yum and apt of Kubernetes. It allows us to deploy resources to Kubernetes quickly. It deploys charts which are the packages of application. Helm is also an official Kubernetes project in the Cloud Native Computing Foundation (CNCF) under the category of incubating projects.

Why we need Helm?

Helm makes deployments so easy in Kubernetes, all you need to add helm charts on your Kubernetes cluster. All the Helm charts are stored in Helm repository; you can search the required application chart on Helm registry and add them to your Kubernetes cluster with ease. For example, if you want to deploy a WordPress app on your Kubernetes, you have to create Yaml manifest files for deployment and service for both app and database, which can be quite complicated. Still, by using Helm you can deploy WordPress in a few minutes on your Kube cluster. Also if you want to deploy 50 microservices on Kubernetes using manifest files it consumes a whole lot of time and also there is an exorbitant probability of errors. Still, in case of Helm, you have to remember the name of required charts, and you can deploy these as quick as Flash.

How to install Helm?

The process of installing Helm is effortless, you can install it using the binaries, or you can use package managers.

From homebrew on macOS:

brew install Helm

From Chocolatey on windows:

choco install Kubernetes-helm

From installer script:

Helm now comes with an installer script, you can fetch this script to your system, and it automatically installs the latest version of Helm in your system.

$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3

$ chmod 700 get_helm.sh

$ ./get_helm.sh

#kubernetes #helm #helm charts #helm installation #helm overview #helm repository

Django-allauth: A simple Boilerplate to Setup Authentication

Django-Authentication 

A simple Boilerplate to Setup Authentication using Django-allauth, with a custom template for login and registration using django-crispy-forms.

Getting Started

Prerequisites

  • Python 3.8.6 or higher

Project setup

# clone the repo
$ git clone https://github.com/yezz123/Django-Authentication

# move to the project folder
$ cd Django-Authentication

Creating virtual environment

  • Create a virtual environment for this project:
# creating pipenv environment for python 3
$ virtualenv venv

# activating the pipenv environment
$ cd venv/bin #windows environment you activate from Scripts folder

# if you have multiple python 3 versions installed then
$ source ./activate

Configured Enviromment

Environment variables

SECRET_KEY = #random string
DEBUG = #True or False
ALLOWED_HOSTS = #localhost
DATABASE_NAME = #database name (You can just use the default if you want to use SQLite)
DATABASE_USER = #database user for postgres
DATABASE_PASSWORD = #database password for postgres
DATABASE_HOST = #database host for postgres
DATABASE_PORT = #database port for postgres
ACCOUNT_EMAIL_VERIFICATION = #mandatory or optional
EMAIL_BACKEND = #email backend
EMAIL_HOST = #email host
EMAIL_HOST_PASSWORD = #email host password
EMAIL_USE_TLS = # if your email use tls
EMAIL_PORT = #email port

change all the environment variables in the .env.sample and don't forget to rename it to .env.

Run the project

After Setup the environment, you can run the project using the Makefile provided in the project folder.

help:
 @echo "Targets:"
 @echo "    make install" #install requirements
 @echo "    make makemigrations" #prepare migrations
 @echo "    make migrations" #migrate database
 @echo "    make createsuperuser" #create superuser
 @echo "    make run_server" #run the server
 @echo "    make lint" #lint the code using black
 @echo "    make test" #run the tests using Pytest

Preconfigured Packages

Includes preconfigured packages to kick start Django-Authentication by just setting appropriate configuration.

PackageUsage
django-allauthIntegrated set of Django applications addressing authentication, registration, account management as well as 3rd party (social) account authentication.
django-crispy-formsdjango-crispy-forms provides you with a crispy filter and {% crispy %} tag that will let you control the rendering behavior of your Django forms in a very elegant and DRY way.

Contributing

  • Django-Authentication is a simple project, so you can contribute to it by just adding your code to the project to improve it.
  • If you have any questions, please feel free to open an issue or create a pull request.

Download Details:
Author: yezz123
Source Code: https://github.com/yezz123/Django-Authentication
License: MIT License

#django #python