When deploying an AKS cluster, even if you configure RBAC or AAD integration, local accounts will be enabled by default. This means that, given the right set of permitions, a user will be able to run the az get-credentials
command with the --admin
flag which will give you a non-audtibale access to the cluster.
But don’t worry! it’s possible to disable local account while deploying a new cluster or update an existing one, through the disable-local-accounts
flag when using Azure CLI.
If you disable local accounts on an existing AKS cluster, and want to revoke access to any user, be sure to rotate the cluster certificates
Once local accounts are disabled you’ll find another challenge: when using service principals or managed identities to connect to the cluster, you’ll need a non-interactive flow to get valid cluster credentials, otherwise when attempting to run commands against the cluster you’ll get prompted for authentication.
kubelogin is a tool that will help you with the non-interactive flow by providing the get-token
command, which you’ll learn to use together with the exec credential plugin in the following sample.
Using Terraform to create an AKS cluster with local accounts disabled.
Create variables.tf with the following contents:
1# Location of the services
2variable "location" {
3 default = "west europe"
4}
5
6# Resource Group Name
7variable "resource_group" {
8 default = "aks-no-local-accounts"
9}
10
11# Name of the AKS cluster
12variable "aks_name" {
13 default = "aksnolocalaccounts"
14}
15
16# Name of the Service Princiapl we'll use to connect to the cluster.
17variable "sp_name" {
18 default = "aksnolocalaccounts"
19}
Create providers.tf with the following contents:
1terraform {
2 required_version = ">= 0.13.5"
3 required_providers {
4 azurerm = {
5 version = "= 2.97.0"
6 }
7 azuread = {
8 version = "= 1.4.0"
9 }
10 kubernetes = {
11 version = "= 2.8.0"
12 }
13 }
14}
15
16provider "azurerm" {
17 features {}
18}
19
20# Configuring the kubernetes provider
21provider "kubernetes" {
22 host = azurerm_kubernetes_cluster.aks.kube_config.0.host
23 cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.cluster_ca_certificate)
24
25 # Using kubelogin to get an AAD token for the cluster.
26 exec {
27 api_version = "client.authentication.k8s.io/v1beta1"
28 command = "kubelogin"
29 args = [
30 "get-token",
31 "--environment",
32 "AzurePublicCloud",
33 "--server-id",
34 data.azuread_service_principal.aks_aad_server.application_id, # Application Id of the Azure Kubernetes Service AAD Server.
35 "--client-id",
36 azuread_application.sp.application_id, // Application Id of the Service Principal we'll create via terraform.
37 "--client-secret",
38 random_password.passwd.result, // The Service Principal's secret.
39 "-t",
40 data.azurerm_subscription.current.tenant_id, // The AAD Tenant Id.
41 "-l",
42 "spn" // Login using a Service Principal..
43 ]
44 }
45}
In the
kubernetes
provider definition theexec
plugin is used together with thekubelogin get-token
command.
Create main.tf with the following contents:
1# Create Resource Group
2resource "azurerm_resource_group" "rg" {
3 name = var.resource_group
4 location = var.location
5}
6
7# Get current Subscription
8data "azurerm_subscription" "current" {}
9
10# Get current Client
11data "azuread_client_config" "current" {}
12
13# We'll need the Application Id of the Azure Kubernetes Service AAD Server.
14data "azuread_service_principal" "aks_aad_server" {
15 display_name = "Azure Kubernetes Service AAD Server"
16}
17
18# Create VNET for AKS
19resource "azurerm_virtual_network" "vnet" {
20 name = "private-network"
21 address_space = ["10.0.0.0/8"]
22 location = azurerm_resource_group.rg.location
23 resource_group_name = azurerm_resource_group.rg.name
24}
25
26# Create the Subnet for AKS.
27resource "azurerm_subnet" "aks" {
28 name = "aks"
29 resource_group_name = azurerm_resource_group.rg.name
30 virtual_network_name = azurerm_virtual_network.vnet.name
31 address_prefixes = ["10.240.0.0/16"]
32}
33
34# Create the AKS cluster.
35# Cause this is a test node_count is set to 1
36resource "azurerm_kubernetes_cluster" "aks" {
37 name = var.aks_name
38 location = azurerm_resource_group.rg.location
39 resource_group_name = azurerm_resource_group.rg.name
40 dns_prefix = var.aks_name
41 local_account_disabled = true
42
43 default_node_pool {
44 name = "default"
45 node_count = 1
46 vm_size = "Standard_D2s_v3"
47 os_disk_size_gb = 30
48 os_disk_type = "Ephemeral"
49 vnet_subnet_id = azurerm_subnet.aks.id
50 }
51
52 # Using Managed Identity
53 identity {
54 type = "SystemAssigned"
55 }
56
57 network_profile {
58 network_plugin = "kubenet"
59 network_policy = "calico"
60 }
61
62 role_based_access_control {
63 enabled = true
64 azure_active_directory {
65 admin_group_object_ids = [
66 azuread_group.aks_admins.object_id,
67 ]
68 azure_rbac_enabled = false
69 managed = true
70 tenant_id = data.azurerm_subscription.current.tenant_id
71 }
72 }
73}
74
75resource "kubernetes_namespace" "ns" {
76 metadata {
77 name = "nolocalaccounts"
78 }
79}
Local accounts are disabled by using the
local_account_disabled
property. Also RBAC is enabled and an AAD group is configured as cluster administrator.
Create service_principal.tf with the following contents:
1resource "azuread_application" "sp" {
2 display_name = var.sp_name
3}
4
5resource "azuread_service_principal" "sp" {
6 application_id = azuread_application.sp.application_id
7}
8
9# Generate password for the Service Principal
10resource "random_password" "passwd" {
11 length = 32
12 min_upper = 4
13 min_lower = 2
14 min_numeric = 4
15 keepers = {
16 aks_app_id = azuread_application.sp.id
17 }
18}
19
20# Create kubecost's Service principal password
21resource "azuread_service_principal_password" "sp_password" {
22 service_principal_id = azuread_service_principal.sp.id
23 value = random_password.passwd.result
24 end_date = "2099-01-01T00:00:00Z"
25}
26
27resource "azuread_group" "aks_admins" {
28 display_name = "aks_admins"
29 owners = [data.azuread_client_config.current.object_id]
30
31 members = [
32 data.azuread_client_config.current.object_id,
33 azuread_service_principal.sp.object_id,
34 ]
35}
Once the Service Principal is created it is added to a AAD group (
aks_admins
) which is configured as cluster administrator in main.tf.
Deploy the cluster:
As prerequisite first download kubelogin:
1Invoke-WebRequest https://github.com/Azure/kubelogin/releases/download/v0.0.11/kubelogin-win-amd64.zip -OutFile kubelogin.zip
2Expand-Archive .\kubelogin.zip
3mv .\kubelogin\bin\windows_amd64\kubelogin.exe .\kubelogin.exe
4rm .\kubelogin\
5rm .\kubelogin.zip
Now run the following commands:
1terraform init
2terraform apply -auto-approve
If everything is ok, the deployment will finish without issues and you’ll be able to check for the existence of the nolocalaccounts
namespace in your cluster.
Please find the complete samples here
References:
Comments