HDInsight: Scale Horizontally

HDInsight: Scale Horizontally with Scale-HDInsightClusterNodes.ps1 a PowerShell workflow that will help you automate the process of scaling your cluster.

The script receives 4 parameters:

  • ResourceGroupName: The name of the resource group where the cluster resides
  • ClusterName: The name of your HDInsight cluster
  • Nodes: The number of nodes you want for the cluster
  • ConnectionName: The name of your automation connection account

and requires the following PowerShell modules: AzureRM.Profile, AzureRM.HDInsight

Here is the code:

 1<#
 2    .SYNOPSIS
 3        Scale-HDInsightClusterNodes is a simple PowerShell workflow runbook that will help you automate the process of scaling in or out your HDInsight clusters depending on your needs.
 4    
 5    .DESCRIPTION
 6        Scale-HDInsightClusterNodes is a simple PowerShell workflow runbook that will help you automate the process of scaling in or out your HDInsight clusters depending on your needs.
 7
 8    .PARAMETER ResourceGroupName
 9        The name of the resource group where the cluster resides
10    
11    .PARAMETER ClusterName
12        The name of your HDInsight cluster
13    
14    .PARAMETER Nodes
15        The number of nodes you want for the cluster
16
17    .PARAMETER ConnectionName
18        The name of your automation connection account
19   
20    .NOTES 
21        AUTHOR: Carlos Mendible 
22        LASTEDIT: June 13, 2017 
23#>
24Workflow Scale-HDInsightClusterNodes {
25    Param
26    (   
27        [Parameter(Mandatory = $true)]
28        [String]$ResourceGroupName,
29
30        [Parameter(Mandatory = $true)]
31        [String]$ClusterName,
32
33        [Parameter(Mandatory = $true)]
34        [Int]$Nodes,
35
36        [Parameter(Mandatory = $false)]
37        [String]$ConnectionName
38    )
39
40    # This Workflow requieres the following PowerShell modules: AzureRM.Profile, AzureRM.HDInsight
41
42    $automationConnectionName = $ConnectionName
43    if (!$ConnectionName) {
44        $automationConnectionName = "AzureRunAsConnection"
45    }
46	
47    # Get the connection by name (i.e. AzureRunAsConnection)
48    $servicePrincipalConnection = Get-AutomationConnection -Name $automationConnectionName         
49
50    Write-Output "Logging in to Azure..."
51    
52    Add-AzureRmAccount `
53        -ServicePrincipal `
54        -TenantId $servicePrincipalConnection.TenantId `
55        -ApplicationId $servicePrincipalConnection.ApplicationId `
56        -CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
57 
58    Write-Output "Scaling cluster $ClusterName to $Nodes nodes..."
59    Set-AzureRmHDInsightClusterSize `
60        -ResourceGroupName $ResourceGroupName `
61        -ClusterName $ClusterName `
62        -TargetInstanceCount $Nodes
63}

You can download the script from the Technet Script Gallery, import it as a runbook for your automation account in the Azure portal or simply copy the code from this post.

Hope it helps!


Deploy your ASP.NET Core Web API to AWS Lambda
dotnet new templates for AWS Lambda and Raspberry Pi
comments powered by Disqus