A_mazon Elastic Kubernetes Service (EKS)_ is a service under the _AWS _umbrella that provides managed Kubernetes service. It significantly reduces the time to deploy, manage, & scale the infrastructure required to run containerized applications.

AWS has simplified EKS networking significantly with its container network interface (CNI) plugin. With no network overlays, Kubernetes pods (containers) & services (L4 load-balancers) get IPs from the same _VPC _allocated subnet as would an EC2 instance get.

Pod Networking in EKS

The AWS CNI plugin is responsible for allocating VPC IPs to Kubernetes nodes on which the pods run & configuring the necessary networking on each node. ​

The plugin consists of two primary components:

• The L-IPAM daemon is responsible for attaching elastic network interfaces (ENIs) to instances, assigning secondary IP addresses to these ENIs, etc
• The CNI plugin itself is responsible for wiring the host network & adding the correct interface to the pod namespace

AWS by default, reserves a large pool of IPs to an EKS (Kubernetes) node, that are always available to be used by the node. This pool of IPs, also known as **“Warm-Pool” **is determined by the instance type & cannot be shared with any other AWS service.

For example, an m4.4xlarge node can have up to 8 ENIs, & each ENI can have up to 30 IP addresses. When a worker node first joins the cluster, there is only 1 ENI along with all of its addresses in the ENI. Without any configuration, ipamD always try to keep one extra ENI.

So the total number of IPs reserved for an instance of type m4.4xlarge at any point of time will be 2*30=60 IPs.

This is particularly a large number of IPs for a single EKS node which might run a pod or two resource-intensive applications. Even with a buffer of 8 IPs that might be used by monitoring/logging/proxy DaemonSets, we are looking at approximately 10 IPs/node.

Hence with the default EKS setting, there are roughly about 50 IPs lying around & not being utilized!!! This results in inefficient utilization of your VPC subnet & in some cases if not properly planned, leads to subnet exhaustion.

NOTE: ENI & secondary IP address limitations by EC2 instance types are applicable. In general, larger instances can support more IP addresses. For more information, see IP Addresses Per Network Interface Per Instance Type document.

#ek #aws #devops #k8s #kubernetes