Azure Kubernetes Service (AKS) Terraform Module

terraform-azurerm-aks

Deploys a Kubernetes cluster on AKS with monitoring support through Azure Log Analytics

This Terraform module deploys a Kubernetes cluster on Azure using AKS (Azure Kubernetes Service) and adds support for monitoring with Log Analytics.

-> NOTE: If you have not assigned client_id or client_secret, A SystemAssigned identity will be created.

Notice on breaking changes

Please be aware that major version(e.g., from 6.8.0 to 7.0.0) update contains breaking changes that may impact your infrastructure. It is crucial to review these changes with caution before proceeding with the upgrade.

In most cases, you will need to adjust your Terraform code to accommodate the changes introduced in the new major version. We strongly recommend reviewing the changelog and migration guide to understand the modifications and ensure a smooth transition.

To help you in this process, we have provided detailed documentation on the breaking changes, new features, and any deprecated functionalities. Please take the time to read through these resources to avoid any potential issues or disruptions to your infrastructure.

  • Notice on Upgrade to v7.x
  • Notice on Upgrade to v6.x
  • Notice on Upgrade to v5.x

Remember, upgrading to a major version with breaking changes should be done carefully and thoroughly tested in your environment. If you have any questions or concerns, please don't hesitate to reach out to our support team for assistance.

Usage in Terraform 1.2.0

Please view folders in examples.

The module supports some outputs that may be used to configure a kubernetes provider after deploying an AKS cluster.

provider "kubernetes" {
  host                   = module.aks.host
  client_certificate     = base64decode(module.aks.client_certificate)
  client_key             = base64decode(module.aks.client_key)
  cluster_ca_certificate = base64decode(module.aks.cluster_ca_certificate)
}

There're some examples in the examples folder. You can execute terraform apply command in examples's sub folder to try the module. These examples are tested against every PR with the E2E Test.

Enable or disable tracing tags

We're using BridgeCrew Yor and yorbox to help manage tags consistently across infrastructure as code (IaC) frameworks. In this module you might see tags like:

resource "azurerm_resource_group" "rg" {
  location = "eastus"
  name     = random_pet.name
  tags = merge(var.tags, (/*<box>*/ (var.tracing_tags_enabled ? { for k, v in /*</box>*/ {
    avm_git_commit           = "3077cc6d0b70e29b6e106b3ab98cee6740c916f6"
    avm_git_file             = "main.tf"
    avm_git_last_modified_at = "2023-05-05 08:57:54"
    avm_git_org              = "lonegunmanb"
    avm_git_repo             = "terraform-yor-tag-test-module"
    avm_yor_trace            = "a0425718-c57d-401c-a7d5-f3d88b2551a4"
  } /*<box>*/ : replace(k, "avm_", var.tracing_tags_prefix) => v } : {}) /*</box>*/))
}

To enable tracing tags, set the variable to true:

module "example" {
  source               = "{module_source}"
  ...
  tracing_tags_enabled = true
}

The tracing_tags_enabled is default to false.

To customize the prefix for your tracing tags, set the tracing_tags_prefix variable value in your Terraform configuration:

module "example" {
  source              = "{module_source}"
  ...
  tracing_tags_prefix = "custom_prefix_"
}

The actual applied tags would be:

{
  custom_prefix_git_commit           = "3077cc6d0b70e29b6e106b3ab98cee6740c916f6"
  custom_prefix_git_file             = "main.tf"
  custom_prefix_git_last_modified_at = "2023-05-05 08:57:54"
  custom_prefix_git_org              = "lonegunmanb"
  custom_prefix_git_repo             = "terraform-yor-tag-test-module"
  custom_prefix_yor_trace            = "a0425718-c57d-401c-a7d5-f3d88b2551a4"
}

Pre-Commit & Pr-Check & Test

Configurations

We assumed that you have setup service principal's credentials in your environment variables like below:

export ARM_SUBSCRIPTION_ID="<azure_subscription_id>"
export ARM_TENANT_ID="<azure_subscription_tenant_id>"
export ARM_CLIENT_ID="<service_principal_appid>"
export ARM_CLIENT_SECRET="<service_principal_password>"

On Windows Powershell:

$env:ARM_SUBSCRIPTION_ID="<azure_subscription_id>"
$env:ARM_TENANT_ID="<azure_subscription_tenant_id>"
$env:ARM_CLIENT_ID="<service_principal_appid>"
$env:ARM_CLIENT_SECRET="<service_principal_password>"

We provide a docker image to run the pre-commit checks and tests for you: mcr.microsoft.com/azterraform:latest

To run the pre-commit task, we can run the following command:

$ docker run --rm -v $(pwd):/src -w /src mcr.microsoft.com/azterraform:latest make pre-commit

On Windows Powershell:

$ docker run --rm -v ${pwd}:/src -w /src mcr.microsoft.com/azterraform:latest make pre-commit

In pre-commit task, we will:

  1. Run terraform fmt -recursive command for your Terraform code.
  2. Run terrafmt fmt -f command for markdown files and go code files to ensure that the Terraform code embedded in these files are well formatted.
  3. Run go mod tidy and go mod vendor for test folder to ensure that all the dependencies have been synced.
  4. Run gofmt for all go code files.
  5. Run gofumpt for all go code files.
  6. Run terraform-docs on README.md file, then run markdown-table-formatter to format markdown tables in README.md.

Then we can run the pr-check task to check whether our code meets our pipeline's requirement(We strongly recommend you run the following command before you commit):

$ docker run --rm -v $(pwd):/src -w /src mcr.microsoft.com/azterraform:latest make pr-check

On Windows Powershell:

$ docker run --rm -v ${pwd}:/src -w /src mcr.microsoft.com/azterraform:latest make pr-check

To run the e2e-test, we can run the following command:

docker run --rm -v $(pwd):/src -w /src -e ARM_SUBSCRIPTION_ID -e ARM_TENANT_ID -e ARM_CLIENT_ID -e ARM_CLIENT_SECRET mcr.microsoft.com/azterraform:latest make e2e-test

On Windows Powershell:

docker run --rm -v ${pwd}:/src -w /src -e ARM_SUBSCRIPTION_ID -e ARM_TENANT_ID -e ARM_CLIENT_ID -e ARM_CLIENT_SECRET mcr.microsoft.com/azterraform:latest make e2e-test

To follow Ensure AKS uses disk encryption set policy we've used azurerm_key_vault in example codes, and to follow Key vault does not allow firewall rules settings we've limited the ip cidr on it's network_acls. On default we'll use the ip return by https://api.ipify.org?format=json api as your public ip, but in case you need use other cidr, you can assign on by passing an environment variable:

docker run --rm -v $(pwd):/src -w /src -e TF_VAR_key_vault_firewall_bypass_ip_cidr="<your_cidr>" -e ARM_SUBSCRIPTION_ID -e ARM_TENANT_ID -e ARM_CLIENT_ID -e ARM_CLIENT_SECRET mcr.microsoft.com/azterraform:latest make e2e-test

On Windows Powershell:

docker run --rm -v ${pwd}:/src -w /src -e TF_VAR_key_vault_firewall_bypass_ip_cidr="<your_cidr>" -e ARM_SUBSCRIPTION_ID -e ARM_TENANT_ID -e ARM_CLIENT_ID -e ARM_CLIENT_SECRET mcr.microsoft.com/azterraform:latest make e2e-test

Prerequisites

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Module Spec

The following sections are generated by terraform-docs and markdown-table-formatter, please DO NOT MODIFY THEM MANUALLY!

Requirements

NameVersion
terraform>= 1.3
azapi>= 1.4.0, < 2.0
azurerm>= 3.51.0, < 4.0
null>= 3.0
tls>= 3.1

Providers

NameVersion
azapi>= 1.4.0, < 2.0
azurerm>= 3.51.0, < 4.0
null>= 3.0
tls>= 3.1

Modules

No modules.

Resources

NameType
azapi_update_resource.aks_cluster_post_createresource
azurerm_kubernetes_cluster.mainresource
azurerm_kubernetes_cluster_node_pool.node_poolresource
azurerm_log_analytics_solution.mainresource
azurerm_log_analytics_workspace.mainresource
azurerm_role_assignment.acrresource
azurerm_role_assignment.network_contributorresource
azurerm_role_assignment.network_contributor_on_subnetresource
null_resource.kubernetes_version_keeperresource
null_resource.pool_name_keeperresource
tls_private_key.sshresource
azurerm_log_analytics_workspace.maindata source
azurerm_resource_group.maindata source
azurerm_user_assigned_identity.cluster_identitydata source

Outputs

NameDescription
aci_connector_linuxThe aci_connector_linux block of azurerm_kubernetes_cluster resource.
aci_connector_linux_enabledHas aci_connector_linux been enabled on the azurerm_kubernetes_cluster resource?
admin_client_certificateThe client_certificate in the azurerm_kubernetes_cluster's kube_admin_config block. Base64 encoded public certificate used by clients to authenticate to the Kubernetes cluster.
admin_client_keyThe client_key in the azurerm_kubernetes_cluster's kube_admin_config block. Base64 encoded private key used by clients to authenticate to the Kubernetes cluster.
admin_cluster_ca_certificateThe cluster_ca_certificate in the azurerm_kubernetes_cluster's kube_admin_config block. Base64 encoded public CA certificate used as the root of trust for the Kubernetes cluster.
admin_hostThe host in the azurerm_kubernetes_cluster's kube_admin_config block. The Kubernetes cluster server host.
admin_passwordThe password in the azurerm_kubernetes_cluster's kube_admin_config block. A password or token used to authenticate to the Kubernetes cluster.
admin_usernameThe username in the azurerm_kubernetes_cluster's kube_admin_config block. A username used to authenticate to the Kubernetes cluster.
aks_idThe azurerm_kubernetes_cluster's id.
aks_nameThe aurerm_kubernetes-cluster's name.
azure_policy_enabledThe azurerm_kubernetes_cluster's azure_policy_enabled argument. Should the Azure Policy Add-On be enabled? For more details please visit Understand Azure Policy for Azure Kubernetes Service
azurerm_log_analytics_workspace_idThe id of the created Log Analytics workspace
azurerm_log_analytics_workspace_nameThe name of the created Log Analytics workspace
azurerm_log_analytics_workspace_primary_shared_keySpecifies the workspace key of the log analytics workspace
client_certificateThe client_certificate in the azurerm_kubernetes_cluster's kube_config block. Base64 encoded public certificate used by clients to authenticate to the Kubernetes cluster.
client_keyThe client_key in the azurerm_kubernetes_cluster's kube_config block. Base64 encoded private key used by clients to authenticate to the Kubernetes cluster.
cluster_ca_certificateThe cluster_ca_certificate in the azurerm_kubernetes_cluster's kube_config block. Base64 encoded public CA certificate used as the root of trust for the Kubernetes cluster.
cluster_fqdnThe FQDN of the Azure Kubernetes Managed Cluster.
cluster_identityThe azurerm_kubernetes_cluster's identity block.
cluster_portal_fqdnThe FQDN for the Azure Portal resources when private link has been enabled, which is only resolvable inside the Virtual Network used by the Kubernetes Cluster.
cluster_private_fqdnThe FQDN for the Kubernetes Cluster when private link has been enabled, which is only resolvable inside the Virtual Network used by the Kubernetes Cluster.
generated_cluster_private_ssh_keyThe cluster will use this generated private key as ssh key when var.public_ssh_key is empty or null. Private key data in PEM (RFC 1421) format.
generated_cluster_public_ssh_keyThe cluster will use this generated public key as ssh key when var.public_ssh_key is empty or null. The fingerprint of the public key data in OpenSSH MD5 hash format, e.g. aa:bb:cc:.... Only available if the selected private key format is compatible, similarly to public_key_openssh and the ECDSA P224 limitations.
hostThe host in the azurerm_kubernetes_cluster's kube_config block. The Kubernetes cluster server host.
http_application_routing_enabledThe azurerm_kubernetes_cluster's http_application_routing_enabled argument. (Optional) Should HTTP Application Routing be enabled?
http_application_routing_zone_nameThe azurerm_kubernetes_cluster's http_application_routing_zone_name argument. The Zone Name of the HTTP Application Routing.
ingress_application_gatewayThe azurerm_kubernetes_cluster's ingress_application_gateway block.
ingress_application_gateway_enabledHas the azurerm_kubernetes_cluster turned on ingress_application_gateway block?
key_vault_secrets_providerThe azurerm_kubernetes_cluster's key_vault_secrets_provider block.
key_vault_secrets_provider_enabledHas the azurerm_kubernetes_cluster turned on key_vault_secrets_provider block?
kube_admin_config_rawThe azurerm_kubernetes_cluster's kube_admin_config_raw argument. Raw Kubernetes config for the admin account to be used by kubectl and other compatible tools. This is only available when Role Based Access Control with Azure Active Directory is enabled and local accounts enabled.
kube_config_rawThe azurerm_kubernetes_cluster's kube_config_raw argument. Raw Kubernetes config to be used by kubectl and other compatible tools.
kubelet_identityThe azurerm_kubernetes_cluster's kubelet_identity block.
locationThe azurerm_kubernetes_cluster's location argument. (Required) The location where the Managed Kubernetes Cluster should be created.
network_profileThe azurerm_kubernetes_cluster's network_profile block
node_resource_groupThe auto-generated Resource Group which contains the resources for this Managed Kubernetes Cluster.
oidc_issuer_urlThe OIDC issuer URL that is associated with the cluster.
oms_agentThe azurerm_kubernetes_cluster's oms_agent argument.
oms_agent_enabledHas the azurerm_kubernetes_cluster turned on oms_agent block?
open_service_mesh_enabled(Optional) Is Open Service Mesh enabled? For more details, please visit Open Service Mesh for AKS.
passwordThe password in the azurerm_kubernetes_cluster's kube_config block. A password or token used to authenticate to the Kubernetes cluster.
usernameThe username in the azurerm_kubernetes_cluster's kube_config block. A username used to authenticate to the Kubernetes cluster.

Download Details:

Author: Azure

Official Github: https://github.com/Azure/terraform-azurerm-aks 

License: MIT 
#azure #Microsoft #kubernetes 

Azure Kubernetes Service (AKS) Terraform Module
1.00 GEEK