The intent of this post is to help you understand how to connect an Azure Function to a Storage Account privately so all traffic flows through a VNet therefore enhancing the security of your solutions and blobs.
The Case:
Supose you have the following Azure Function written in C# which only copies a blob from one conatiner to another:
1using System.IO;
2using Microsoft.Azure.WebJobs;
3using Microsoft.Extensions.Logging;
4
5namespace Secured.Function
6{
7 public static class SecureCopy
8 {
9 [FunctionName("SecureCopy")]
10 public static void Run(
11 [BlobTrigger("input/{name}", Connection = "privatecfm_STORAGE")] Stream myBlob,
12 [Blob("output/{name}", FileAccess.Write, Connection = "privatecfm_STORAGE")] Stream copy,
13 ILogger log)
14 {
15 myBlob.CopyTo(copy);
16 }
17 }
18}
In this case the storage account used, for the blob trigger and the output binding, has a public endpoint exposed to the internet, which you can secure using features such as the Storage Account Firewall and the new private endpoints which will allow clients on a virtual network (VNet) to securely access data over a Private Link. The private endpoint uses an IP address from the VNet address space for your storage account service. [1]
With those features we can lockdown all inbound traffic to the Storage Account to only accept calls from inside a VNet, so the next step is to enable a feature for Azure Functions that will give your function app access to the resources in the VNet: Azure App Service VNet Integration feature [2].
The following sketch shows how this works:
- The Azure Function is integrated with a VNet using Regional VNet Integration (blue line).
- The Storage Account (shown on the right) has a Private Endpoint which assigns a private IP to the Storage Account.
- Traffic (red line) from the Azure Function flows through the VNet, the Private Endpoint and reaches the Storage Account.
- The Storage Account, shown on the left, is used for the core services of the Azure Function and,
at the time of writing, can’t be protected using private enpoints. Check Update 2020-08-25.
But wait there is one more thing, you will need to add an Azure Private DNS Zone to enable the Azure Function to resolve the name of the Storage Account so it uses the private ip for communication.)
Note: The solution will require use of the PremiumV2, or Elastic Premium pricing plan for the Azure Function.
Deploying the Infrastructure with Terraform
We’ll be using Terraform (version > 0.12) to deploy the solution. Start creating a providers.tf file with the following contents:
1terraform {
2 required_version = "> 0.12"
3}
4
5provider "azurerm" {
6 version = ">= 2.0"
7 features {}
8}
Define the following variables in a variables.tf file:
1# Azure Resource Location
2variable location {
3 default = "west europe"
4}
5
6# Azure Resource Group Name
7variable resource_group {
8 default = "private-endpoint"
9}
10
11# Name of the Storage Account you'll expose through the private endpoint
12variable sa_name {
13 default = "privatecfm"
14}
15
16# Name of the Storage Account backing the Azure Function
17variable function_required_sa {
18 default = "privatecfmfunc"
19}
Create a mainty.tf file with the following contents (Make sure you read the comments to understand the manifest):
1# Create Resource Group
2resource "azurerm_resource_group" "rg" {
3 name = var.resource_group
4 location = var.location
5}
6
7# Create VNet
8resource "azurerm_virtual_network" "vnet" {
9 name = "private-network"
10 address_space = ["10.0.0.0/16"]
11 location = azurerm_resource_group.rg.location
12 resource_group_name = azurerm_resource_group.rg.name
13 # Use Private DNS Zone. That's right we have to add this magical IP here.
14 # dns_servers = ["168.63.129.16"]
15}
16
17# Create the Subnet for the Azure Function. This is thge subnet where we'll enable Vnet Integration.
18resource "azurerm_subnet" "service" {
19 name = "service"
20 resource_group_name = azurerm_resource_group.rg.name
21 virtual_network_name = azurerm_virtual_network.vnet.name
22 address_prefixes = ["10.0.1.0/24"]
23
24 enforce_private_link_service_network_policies = true
25
26 # Delegate the subnet to "Microsoft.Web/serverFarms"
27 delegation {
28 name = "acctestdelegation"
29
30 service_delegation {
31 name = "Microsoft.Web/serverFarms"
32 actions = ["Microsoft.Network/virtualNetworks/subnets/action"]
33 }
34 }
35
36 service_endpoints = ["Microsoft.Storage"]
37}
38
39# Create the Subnet for the private endpoints. This is where the IP of the private enpoint will live.
40resource "azurerm_subnet" "endpoint" {
41 name = "endpoint"
42 resource_group_name = azurerm_resource_group.rg.name
43 virtual_network_name = azurerm_virtual_network.vnet.name
44 address_prefixes = ["10.0.2.0/24"]
45
46 enforce_private_link_endpoint_network_policies = true
47}
48
49# Get current public IP. We'll need this so we can access the Storage Account from our PC.
50data "http" "current_public_ip" {
51 url = "http://ipinfo.io/json"
52 request_headers = {
53 Accept = "application/json"
54 }
55}
56
57# Create the "private" Storage Account.
58resource "azurerm_storage_account" "sa" {
59 name = var.sa_name
60 resource_group_name = azurerm_resource_group.rg.name
61 location = azurerm_resource_group.rg.location
62 account_tier = "Standard"
63 account_replication_type = "GRS"
64 enable_https_traffic_only = true
65 # We are enabling the firewall only allowing traffic from our PC's public IP.
66 network_rules {
67 default_action = "Deny"
68 virtual_network_subnet_ids = []
69 ip_rules = [
70 jsondecode(data.http.current_public_ip.body).ip
71 ]
72 }
73}
74
75# Create input container
76resource "azurerm_storage_container" "input" {
77 name = "input"
78 container_access_type = "private"
79 storage_account_name = azurerm_storage_account.sa.name
80}
81
82# Create output container
83resource "azurerm_storage_container" "output" {
84 name = "output"
85 container_access_type = "private"
86 storage_account_name = azurerm_storage_account.sa.name
87}
88
89# Create the Private endpoint. This is where the Storage account gets a private IP inside the VNet.
90resource "azurerm_private_endpoint" "endpoint" {
91 name = "sa-endpoint"
92 location = azurerm_resource_group.rg.location
93 resource_group_name = azurerm_resource_group.rg.name
94 subnet_id = azurerm_subnet.endpoint.id
95
96 private_service_connection {
97 name = "sa-privateserviceconnection"
98 private_connection_resource_id = azurerm_storage_account.sa.id
99 is_manual_connection = false
100 subresource_names = ["blob"]
101 }
102}
103
104# Create the blob.core.windows.net Private DNS Zone
105resource "azurerm_private_dns_zone" "private" {
106 name = "privatelink.blob.core.windows.net"
107 resource_group_name = azurerm_resource_group.rg.name
108}
109
110resource "azurerm_private_dns_cname_record" "cname" {
111 name = "${var.sa_name}.blob.core.windows.net"
112 zone_name = azurerm_private_dns_zone.private.name
113 resource_group_name = azurerm_resource_group.rg.name
114 ttl = 300
115 record = "${var.sa_name}.privatelink.blob.core.windows.net"
116}
117
118# Create an A record pointing to the Storage Account private endpoint
119resource "azurerm_private_dns_a_record" "sa" {
120 name = var.sa_name
121 zone_name = azurerm_private_dns_zone.private.name
122 resource_group_name = azurerm_resource_group.rg.name
123 ttl = 3600
124 records = [azurerm_private_endpoint.endpoint.private_service_connection[0].private_ip_address]
125}
126
127# Link the Private Zone with the VNet
128resource "azurerm_private_dns_zone_virtual_network_link" "sa" {
129 name = "test"
130 resource_group_name = azurerm_resource_group.rg.name
131 private_dns_zone_name = azurerm_private_dns_zone.private.name
132 virtual_network_id = azurerm_virtual_network.vnet.id
133}
134
135# Create the Storage Account required by Azure Functions.
136resource "azurerm_storage_account" "function_required_sa" {
137 name = var.function_required_sa
138 resource_group_name = azurerm_resource_group.rg.name
139 location = azurerm_resource_group.rg.location
140 account_tier = "Standard"
141 account_replication_type = "GRS"
142 enable_https_traffic_only = true
143}
144
145# Create a container to hold the Azure Function Zip
146resource "azurerm_storage_container" "functions" {
147 name = "function-releases"
148 storage_account_name = azurerm_storage_account.function_required_sa.name
149 container_access_type = "private"
150}
151
152# Create a blob with the Azure Function zip
153resource "azurerm_storage_blob" "function" {
154 name = "securecopy.zip"
155 storage_account_name = azurerm_storage_account.function_required_sa.name
156 storage_container_name = azurerm_storage_container.functions.name
157 type = "Block"
158 source = "./securecopy.zip"
159}
160
161# Create a SAS token so the Function can access the blob and deploy the zip
162data "azurerm_storage_account_sas" "sas" {
163 connection_string = azurerm_storage_account.function_required_sa.primary_connection_string
164 https_only = false
165 resource_types {
166 service = false
167 container = false
168 object = true
169 }
170 services {
171 blob = true
172 queue = false
173 table = false
174 file = false
175 }
176 start = "2020-05-18"
177 expiry = "2025-05-18"
178 permissions {
179 read = true
180 write = false
181 delete = false
182 list = false
183 add = false
184 create = false
185 update = false
186 process = false
187 }
188}
189
190# Create the Azure Function plan (Elastic Premium)
191resource "azurerm_app_service_plan" "plan" {
192 name = "azure-functions-test-service-plan"
193 location = azurerm_resource_group.rg.location
194 resource_group_name = azurerm_resource_group.rg.name
195
196 kind = "elastic"
197 sku {
198 tier = "ElasticPremium"
199 size = "EP1"
200 capacity = 1
201 }
202}
203
204# Create Application Insights
205resource "azurerm_application_insights" "ai" {
206 name = "func-pe-test"
207 location = azurerm_resource_group.rg.location
208 resource_group_name = azurerm_resource_group.rg.name
209 application_type = "web"
210 retention_in_days = 90
211}
212
213# Create the Azure Function App
214resource "azurerm_function_app" "func_app" {
215 name = "func-pe-test"
216 location = azurerm_resource_group.rg.location
217 resource_group_name = azurerm_resource_group.rg.name
218 app_service_plan_id = azurerm_app_service_plan.plan.id
219 storage_account_name = azurerm_storage_account.function_required_sa.name
220 storage_account_access_key = azurerm_storage_account.function_required_sa.primary_access_key
221 version = "~3"
222
223 app_settings = {
224 https_only = true
225 APPINSIGHTS_INSTRUMENTATIONKEY = azurerm_application_insights.ai.instrumentation_key
226 privatecfm_STORAGE = azurerm_storage_account.sa.primary_connection_string
227 # With this setting we'll force all outbound traffic through the VNet
228 WEBSITE_VNET_ROUTE_ALL = "1"
229 WEBSITE_DNS_SERVER = "168.63.129.16"
230 # Properties used to deploy the zip
231 HASH = filesha256("./securecopy.zip")
232 WEBSITE_USE_ZIP = "https://${azurerm_storage_account.function_required_sa.name}.blob.core.windows.net/${azurerm_storage_container.functions.name}/${azurerm_storage_blob.function.name}${data.azurerm_storage_account_sas.sas.sas}"
233 }
234}
235
236# Enable Regional VNet integration. Function --> service Subnet
237resource "azurerm_app_service_virtual_network_swift_connection" "vnet_integration" {
238 app_service_id = azurerm_function_app.func_app.id
239 subnet_id = azurerm_subnet.service.id
240}
Now download, into your working folder, the securecopy.zip file containing the sample Azure Function or create a zip, with the same name, containing your own code.
Deploy the solution running the following commands:
1terraform init
2terraform apply
Test the solution
Use the Azure portal or Storage Explorer to upload a file to the input container, after a few seconds you should find a copy of the file in the output container.
VNET Integration Name Resolution Test
If the previous test didn’t work, please connect through KUDU or the Console to the Azure Function and run the following command:
1nameresolver <name of the storage account>.blob.core.windows.net
The output of the command should show 10.0.2.4 as the IP address. If that’s not the case you probably misconfigured something. [3]
Hope it helps and please find a copy of all the code here
Update 2020-08-25
I’ve added a new sample here using the same Storage Account for both: backing the Azure Function and the Blobs needed for the sample application.
With that script you must run Terraform apply twice: the first time with the Storage Account Firewall disabled so the Azure Function deployment runs without any issues and the second one enabling the Firewall so the Storage Account is protected.
This setup requires 4 Private Enpoints: one for each of the Storage Account Services required by the Azure Function runtime (blob. table, queue, files).
1terraform apply -var="sa_firewall_enabled=false"
2terraform apply -var="sa_firewall_enabled=true"
References
- [1] Use private endpoints for Azure Storage
- [2] Integrate your app with an Azure virtual network: Regional VNET Integration
- [3] Integrate your app with an Azure virtual network: Troubleshooting
Comments