1596588000
Multi-namespace Helm deploy in Kubernetes
Some background
My life in Site Reliability can be described as a perfect cocktail of building automation, reactive problem solving, and…posting “let me google that for you” links on slack channels. One of the major themes is figuring out how we scale based on customer demands. Currently, at LogicMonitor, we are running a hybrid environment where our core application as well as time series databases are running on servers in our physical data centers. These environments interact with microservices running in AWS. We called one of these hybrid production environments a “pod”(which in hindsight, is a poor choice of name once we introduced Kubernetes). We use the Atlassian product Bamboo for CI/CD.
In the last year and a half, the team went through the grueling but satisfying task of converting all our applications that were housed on ECS and EC2 instances into pods in Kubernetes, with each production “pod” being considered a namespace.
Deploying to k8s
Our deployment process was to use helm from within Bamboo. Each application built would produce a helm chart artifact and a docker image. Upon deployment, the image is pushed to a private docker repository and a helm install command is run with corresponding charts. However, with each production environment being considered a Kubernetes namespace, we needed to deploy to multiple namespaces per cluster, which was set by having an individual Bamboo deploy plan** per namespace, per application. **As of today we have 50 different prod environments and 8 microservices(for you math whizzes out there, that is 400 individual deploy plans). Sometimes, just for one application point release, it could take a developer well over an hour or two to deploy and verify all of production.
Building a new tool
So theres no way around this…if we want to effectively scale in infrastructure, we need to find a smarter way to deploy. Currently we use a variety of shell scripts that initiate the deployment process. In order to build a new tool, it needs to:
- Be able to query and list all the production namespaces
- Integrate helm/kubernetes libraries
- Deploy to multiple namespaces at once.
- Centralized logs for deployment progress
Introducing k8sdeploy
k8sdeploy is a go based tool, written with the goal of creating a cli that utilizes helm and kubernetes client libraries to deploy to multiple namespaces at once.
Initialization:
This creates the Helm Client and Client to Kubernetes**. **The current example below is for helmv2. Drastic changes with helm3 allow helm to directly communicate with k8s api server directly via kubeconfig.
// GetKubeClient generates a k8s client based on kubeconfig
func GetKubeClient(kubeconfig string) (*kubernetes.Clientset, error) {
config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
if err != nil {
panic(err.Error())
}
return kubernetes.NewForConfig(config)
}
//GetHelmClientv2 creates helm2 client based on kubeconfig
func GetHelmClientv2(kubeconfig string) *helm.Client {
config, _ := clientcmd.BuildConfigFromFlags(“”, kubeconfig)
client, _ := kubernetes.NewForConfig(config)
// port forward tiller (specific to helm2)
tillerTunnel, _ := portforwarder.New(“kube-system”, client, config)
// new helm client
host := fmt.Sprintf(“127.0.0.1:%d”, tillerTunnel.Local)
helmClient := helm.NewClient(helm.Host(host))
return helmClient
}
Shared Informer:
After the tool creates the client, it initializes a deployment watcher. This is a shared informer, which watches for changes in the current state of Kubernetes objects. In our case, upon deployment, we would create a channel to start and stop a shared informer for the ReplicaSet resource. The goal here is to not only log the deployment status (“1 of 2 updated replicas are available”), but also collate all the information in one stream which is crucial when deploying to multiple namespaces at once.
#kubernetes #programming #golang #automation #devops
What is GEEK
Buddha Community
1602964260
50+ Useful Kubernetes Tools for 2020 - Part 2
Introduction
Last year, we provided a list of Kubernetes tools that proved so popular we have decided to curate another list of some useful additions for working with the platform—among which are many tools that we personally use here at Caylent. Check out the original tools list here in case you missed it.
According to a recent survey done by Stackrox, the dominance Kubernetes enjoys in the market continues to be reinforced, with 86% of respondents using it for container orchestration.
And as you can see below, more and more companies are jumping into containerization for their apps. If you’re among them, here are some tools to aid you going forward as Kubernetes continues its rapid growth.
#blog #tools #amazon elastic kubernetes service #application security #aws kms #botkube #caylent #cli #container monitoring #container orchestration tools #container security #containers #continuous delivery #continuous deployment #continuous integration #contour #developers #development #developments #draft #eksctl #firewall #gcp #github #harbor #helm #helm charts #helm-2to3 #helm-aws-secret-plugin #helm-docs #helm-operator-get-started #helm-secrets #iam #json #k-rail #k3s #k3sup #k8s #keel.sh #keycloak #kiali #kiam #klum #knative #krew #ksniff #kube #kube-prod-runtime #kube-ps1 #kube-scan #kube-state-metrics #kube2iam #kubeapps #kubebuilder #kubeconfig #kubectl #kubectl-aws-secrets #kubefwd #kubernetes #kubernetes command line tool #kubernetes configuration #kubernetes deployment #kubernetes in development #kubernetes in production #kubernetes ingress #kubernetes interfaces #kubernetes monitoring #kubernetes networking #kubernetes observability #kubernetes plugins #kubernetes secrets #kubernetes security #kubernetes security best practices #kubernetes security vendors #kubernetes service discovery #kubernetic #kubesec #kubeterminal #kubeval #kudo #kuma #microsoft azure key vault #mozilla sops #octant #octarine #open source #palo alto kubernetes security #permission-manager #pgp #rafay #rakess #rancher #rook #secrets operations #serverless function #service mesh #shell-operator #snyk #snyk container #sonobuoy #strongdm #tcpdump #tenkai #testing #tigera #tilt #vert.x #wireshark #yaml
1595824440
Introduction to Helm - Package Manager for Kubernetes
**Introduction to Kubernetes and Helm **
Kubernetes is one of the best platforms to deploy and manage containerized applications. But deploying such containerized applications to Kubernetes can be challenging. You have to write a detailed YAML file to deploy resources like pods, deployments, and services on Kubernetes that’s where Helm comes into the play. Helm is a package manager for Kubernetes; it’s the yum and apt of Kubernetes. It allows us to deploy resources to Kubernetes quickly. It deploys charts which are the packages of application. Helm is also an official Kubernetes project in the Cloud Native Computing Foundation (CNCF) under the category of incubating projects.
Why we need Helm?
Helm makes deployments so easy in Kubernetes, all you need to add helm charts on your Kubernetes cluster. All the Helm charts are stored in Helm repository; you can search the required application chart on Helm registry and add them to your Kubernetes cluster with ease. For example, if you want to deploy a WordPress app on your Kubernetes, you have to create Yaml manifest files for deployment and service for both app and database, which can be quite complicated. Still, by using Helm you can deploy WordPress in a few minutes on your Kube cluster. Also if you want to deploy 50 microservices on Kubernetes using manifest files it consumes a whole lot of time and also there is an exorbitant probability of errors. Still, in case of Helm, you have to remember the name of required charts, and you can deploy these as quick as Flash.
How to install Helm?
The process of installing Helm is effortless, you can install it using the binaries, or you can use package managers.
From homebrew on macOS:
brew install Helm
From Chocolatey on windows:
choco install Kubernetes-helm
From installer script:
Helm now comes with an installer script, you can fetch this script to your system, and it automatically installs the latest version of Helm in your system.
$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh
#kubernetes #helm #helm charts #helm installation #helm overview #helm repository
1594162500
Multi-cloud Spending: 8 Tips To Lower Cost
A multi-cloud approach is nothing but leveraging two or more cloud platforms for meeting the various business requirements of an enterprise. The multi-cloud IT environment incorporates different clouds from multiple vendors and negates the dependence on a single public cloud service provider. Thus enterprises can choose specific services from multiple public clouds and reap the benefits of each.
Given its affordability and agility, most enterprises opt for a multi-cloud approach in cloud computing now. A 2018 survey on the public cloud services market points out that 81% of the respondents use services from two or more providers. Subsequently, the cloud computing services market has reported incredible growth in recent times. The worldwide public cloud services market is all set to reach $500 billion in the next four years, according to IDC.
By choosing multi-cloud solutions strategically, enterprises can optimize the benefits of cloud computing and aim for some key competitive advantages. They can avoid the lengthy and cumbersome processes involved in buying, installing and testing high-priced systems. The IaaS and PaaS solutions have become a windfall for the enterprise’s budget as it does not incur huge up-front capital expenditure.
However, cost optimization is still a challenge while facilitating a multi-cloud environment and a large number of enterprises end up overpaying with or without realizing it. The below-mentioned tips would help you ensure the money is spent wisely on cloud computing services.
- Deactivate underused or unattached resources
Most organizations tend to get wrong with simple things which turn out to be the root cause for needless spending and resource wastage. The first step to cost optimization in your cloud strategy is to identify underutilized resources that you have been paying for.
Enterprises often continue to pay for resources that have been purchased earlier but are no longer useful. Identifying such unused and unattached resources and deactivating it on a regular basis brings you one step closer to cost optimization. If needed, you can deploy automated cloud management tools that are largely helpful in providing the analytics needed to optimize the cloud spending and cut costs on an ongoing basis.
- Figure out idle instances
Another key cost optimization strategy is to identify the idle computing instances and consolidate them into fewer instances. An idle computing instance may require a CPU utilization level of 1-5%, but you may be billed by the service provider for 100% for the same instance.
Every enterprise will have such non-production instances that constitute unnecessary storage space and lead to overpaying. Re-evaluating your resource allocations regularly and removing unnecessary storage may help you save money significantly. Resource allocation is not only a matter of CPU and memory but also it is linked to the storage, network, and various other factors.
- Deploy monitoring mechanisms
The key to efficient cost reduction in cloud computing technology lies in proactive monitoring. A comprehensive view of the cloud usage helps enterprises to monitor and minimize unnecessary spending. You can make use of various mechanisms for monitoring computing demand.
For instance, you can use a heatmap to understand the highs and lows in computing visually. This heat map indicates the start and stop times which in turn lead to reduced costs. You can also deploy automated tools that help organizations to schedule instances to start and stop. By following a heatmap, you can understand whether it is safe to shut down servers on holidays or weekends.
#cloud computing services #all #hybrid cloud #cloud #multi-cloud strategy #cloud spend #multi-cloud spending #multi cloud adoption #why multi cloud #multi cloud trends #multi cloud companies #multi cloud research #multi cloud market
1601051854
Kubernetes in the Cloud: Strategies for Effective Multi Cloud Implementations
Kubernetes is a highly popular container orchestration platform. Multi cloud is a strategy that leverages cloud resources from multiple vendors. Multi cloud strategies have become popular because they help prevent vendor lock-in and enable you to leverage a wide variety of cloud resources. However, multi cloud ecosystems are notoriously difficult to configure and maintain.
This article explains how you can leverage Kubernetes to reduce multi cloud complexities and improve stability, scalability, and velocity.
Kubernetes: Your Multi Cloud Strategy
Maintaining standardized application deployments becomes more challenging as your number of applications and the technologies they are based on increase. As environments, operating systems, and dependencies differ, management and operations require more effort and extensive documentation.
In the past, teams tried to get around these difficulties by creating isolated projects in the data center. Each project, including its configurations and requirements were managed independently. This required accurately predicting performance and the number of users before deployment and taking down applications to update operating systems or applications. There were many chances for error.
Kubernetes can provide an alternative to the old method, enabling teams to deploy applications independent of the environment in containers. This eliminates the need to create resource partitions and enables teams to operate infrastructure as a unified whole.
In particular, Kubernetes makes it easier to deploy a multi cloud strategy since it enables you to abstract away service differences. With Kubernetes deployments you can work from a consistent platform and optimize services and applications according to your business needs.
The Compelling Attributes of Multi Cloud Kubernetes
Multi cloud Kubernetes can provide multiple benefits beyond a single cloud deployment. Below are some of the most notable advantages.
Stability
In addition to the built-in scalability, fault tolerance, and auto-healing features of Kubernetes, multi cloud deployments can provide service redundancy. For example, you can mirror applications or split microservices across vendors. This reduces the risk of a vendor-related outage and enables you to create failovers.
#kubernetes #multicloud-strategy #kubernetes-cluster #kubernetes-top-story #kubernetes-cluster-install #kubernetes-explained #kubernetes-infrastructure #cloud
1620039960
Deploying LOGIQ on K3s using a Helm Chart
We’re huge fans of Helm Charts and the simplicity they bring to complex application deployments on Kubernetes and MicroK8s. We showed you how you could use Helm Charts to deploy LOGIQ on MicroK8s in a previous post. As a follow-up to that article, we’d like to show you how Helm Charts are equally helpful in deploying complex applications on other certified Kubernetes distributions such as K3s. In this article, you’ll learn how we use Helm Charts to deploy the LOGIQ observability platform on K3s. If you’ve never used K3s or Helm Charts before, you can use this article to get acquainted with both.
What is K3s?
K3s is a highly available, lightweight, and fully compliant Kubernetes distribution. K3s is packaged as a single binary, thereby reducing dependencies and minimizing installation, run, and auto-update steps that you’d typically have to take while managing a production Kubernetes cluster. As K3s is lightweight, you can run a cluster on machines with as little as 512 MB RAM and upwards.
K3s comes bundled with a local storage provider, a service load balancer, a Helm controller, and the Traefik ingress controller. A single binary and process encapsulate the operation of all Kubernetes control plane components, allowing K3s to automate and manage complex cluster operations like distributing certificates.
#aiops #cncf #devops #kubernetes #monitoring #observability #helm #helm charts #k3s #kubernetes health monitoring #monitoring for kubernetes