AKS: Open Service Mesh & mTLS

Open Service Mesh (OSM) is a lightweight and extensible cloud native service mesh, easy to install and configure and with features as mTLS to secure your microservice environments.

Now that Open Service Mesh (OSM) integration with Azure Kubernetes Service (AKS) is GA (Check the announcement ) I’ll show you not only to deploy it but also how to add your microservices to the mesh so communication between them is encrypted.

Use terraform to Deploy an AKS cluster with OSM and Monitoring enabled

Create providers.tf with the following contents:

 1terraform {
 2  required_version = "> 0.14"
 3  required_providers {
 4    azurerm = {
 5      version = ">= 2.83.0"
 6    }
 7  }
 8}
 9
10provider "azurerm" {
11  features {}
12}

Note: be sure to use azurerm provide version 2.83.0 or higher.

Create variables.tf with the following contents:

 1# Location of the services
 2variable "location" {
 3  default = "west europe"
 4}
 5
 6# Resource Group Name
 7variable "resource_group" {
 8  default = "aks-osm"
 9}
10
11# Name of the AKS cluster
12variable "aks_name" {
13  default = "aks-osm"
14}

Create main.tf with the following contents:

 1# Create Resource Group
 2resource "azurerm_resource_group" "rg" {
 3  name     = var.resource_group
 4  location = var.location
 5}
 6
 7resource "azurerm_log_analytics_workspace" "workspace" {
 8  name                = var.aks_name
 9  location            = azurerm_resource_group.rg.location
10  resource_group_name = azurerm_resource_group.rg.name
11  sku                 = "PerGB2018"
12  retention_in_days   = 30
13}
14
15# Create the AKS cluster.
16resource "azurerm_kubernetes_cluster" "aks" {
17  name                = var.aks_name
18  location            = azurerm_resource_group.rg.location
19  resource_group_name = azurerm_resource_group.rg.name
20  dns_prefix          = var.aks_name
21
22  default_node_pool {
23    name            = "default"
24    node_count      = 3
25    vm_size         = "Standard_D2s_v3"
26    os_disk_size_gb = 30
27    os_disk_type    = "Ephemeral"
28  }
29
30  # Using Managed Identity
31  identity {
32    type = "SystemAssigned"
33  }
34
35  network_profile {
36    network_plugin = "azure"
37    network_policy = "calico"
38  }
39
40  role_based_access_control {
41    enabled = true
42  }
43
44  addon_profile {
45    kube_dashboard {
46      enabled = false
47    }
48    open_service_mesh {
49      enabled = true
50    }
51    oms_agent {
52      enabled                    = true
53      log_analytics_workspace_id = azurerm_log_analytics_workspace.workspace.id
54    }
55  }
56}

Note: open_service_mesh and oms_agent are enabled.

Create metrics_configmap.yaml with the following contents:

 1kind: ConfigMap
 2apiVersion: v1
 3data:
 4  schema-version: v1
 5  config-version: ver1
 6  osm-metric-collection-configuration: |-
 7    # OSM metric collection settings
 8    [osm_metric_collection_configuration]
 9      [osm_metric_collection_configuration.settings]
10          # Namespaces to monitor
11          monitor_namespaces = ["default"]    
12metadata:
13  name: container-azm-ms-osmconfig
14  namespace: kube-system

Note: monitor_namespaces is set to default, but you can add more namespaces to monitor.

Deploy the cluster

Runthe following commands:

1terraform init
2terraform plan -out tf.plan
3terraform apply ./tf.plan

If the deployment fails with the following message:

“OpenServiceMesh addon is not allowed since feature ‘Microsoft.ContainerService/AKS-OpenServiceMesh’ is not enabled. Please see https://aka.ms/aks/previews for how to enable features.”

Make sure you register the AKS-OpenServiceMesh feature for your subscription.

1az feature register --namespace "Microsoft.ContainerService" --name "AKS-OpenServiceMesh"

Check if the feature is registered:

1az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-OpenServiceMesh')].{Name:name,State:properties.state}"
2az provider register --namespace Microsoft.ContainerService

Once registered refresh the Microsoft.ContainerService registration:

1az provider register --namespace Microsoft.ContainerService

And once again check the status:

1az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService')].{Name:name,State:properties.state}"

Retry the deployment:

1terraform apply ./tf.plan

Check OSM status and version:

Get the cluster credentials:

1az aks get-credentials -g aks-osm -n aks-osm

Check the status of all OSM components:

1kubectl get deploy,po,svc -n kube-system --selector app=osm-controller

Check the OSM version:

1kubectl get deployment -n kube-system osm-controller -o yaml | grep -i image:

Check OSM configuration:

1kubectl get meshconfig osm-mesh-config -n kube-system -o yaml

Check that Permissive Traffic Policy Mode is enabled by default:

1kubectl get meshconfig osm-mesh-config -n kube-system -o yaml | grep -i enablePermissiveTrafficPolicyMode

In this mode, OSM automatically discovers services that are a part of the service mesh and programs traffic policy rules on each Envoy proxy sidecar to be able to communicate with these services.

Configure OSM to monitor a namespace

To tell OSM to monitor a namespace, a label must be added to the namespace. In the sample below, the namespace default is labled for monitoring.

1kubectl label ns default openservicemesh.io/monitored-by=osm

Note: The label openservicemesh.io/monitored-by does not enables sidecar injection.

If you also want to enable automatic side-car injection for the default namespace run:

1kubectl annotate namespace default openservicemesh.io/sidecar-injection=enabled

Configure OSM to enable metrics for a namespace

To tell OSM to enable metrics for a namespace, a label must be added to the namespace.

1kubectl annotate ns default "openservicemesh.io/metrics=enabled"

In order for Azure Monitor to read the metrics, deploy the metrics-configmap.yaml file to the cluster.

1kubectl apply -f ./metrics.configmap.yaml

Add Microservices to the mesh

Run an nginx server with the openservicemesh.io/sidecar-injection=enabled annotation, so OSM injects the envoy sidecar.

1k run nginx --image nginx --annotations="openservicemesh.io/sidecar-injection=enabled"
2k expose po nginx --port 80 --target-port 80

Now run a buybox pod with the openservicemesh.io/sidecar-injection=enabled annotation:

1kubectl run -it --rm busybox --image busybox --annotations="openservicemesh.io/sidecar-injection=enabled" -- sh

Form the shell prompt, run:

1wget -O- http://nginx

That’s it! That request was secured via mTLS

Open another terminal and run:

1kubectl get po

Because OSM injected the envoy sidecar into each pod, you’ll find that both the nginx and buybox pods have 2 containers.

Azure Monitor metrics

Run the following Kusto query to get the OSM metrics:

1InsightsMetrics
2| where Name contains "envoy"
3| extend t=parse_json(Tags)

Hope it helps!!!

Please find the complete sample here

References:


AKS: High Available Storage with Rook and Ceph
AKS: Open Service Mesh Traffic Access Control
comments powered by Disqus