How to access Azure blob containers securely from hosted agents

My storage container needs to use a whitelist for access but the hosted agents IP is constantly changing!

Recently, I was on a call where security was a first class citizen! Woooo! The team wanted to use the latest and greatest Azure DevOps hosted agents but had secure files in storage containers they didn’t want publicly exposed.

Have no fear! It is pretty easy to get past this issue.

This team wanted to specifically use the whitelist feature of Azure storage blobs. One could also restrict access using keys. If you wanted to restrict access to those with keys only, you could just add the storage key as a variable in your build\release. Adding the IP is a bit more work, but not much work for the added security.

Here is a screen shot of what the security looks like for the storage account:

When I am running deployments, I really enjoy using the deploy.ps1, the source code for it can be found here.

Of course I am not going to be running this deploy.ps1 manually, we need to set it up to run in Azure DevOps releases.

A word of caution, to ensure the correct version of the AzureRM or Az cmdlets are installed. By default the hosted agents come with AzureRM 2.x so you will most likely need to update them to a more recent version. Here is an example of how I check and update the version:

If ($env:Agent_Name -eq 'Hosted Agent') {
  # verify AzureRM\Az modules are installed when using hosted agents

  Write-Output "Deploy is running on a hosted agent..."
  $azureRMInstalled = Get-InstalledModule -Name AzureRM -MinimumVersion '6.7.0' -ErrorAction SilentlyContinue
  $azInstalled = Get-Module Az.Compute -ErrorAction SilentlyContinue

  if ((-not $azureRMInstalled) -and (-not $azInstalled)) {
    Install-Module -Name AzureRM -RequiredVersion '6.7.0' -AllowClobber -Force
    Import-Module AzureRM -RequiredVersion '6.7.0' -Force
  } 
  elseif ($azInstalled) {
    Enable-AzureRmAlias
  }
}

The piece where we grab the public IP in the deploy.ps1 is pretty straight forward, we also want to force using TLS 1.2.

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$allowedIP = (Invoke-WebRequest -uri "http://ifconfig.me/ip").Content

I like to use the powershell task because this allows me to still run the deploy.ps1 locally vs some other azure specific task available in the release pipeline.

The artifact I use is not passed from the build, but the Azure Repo itself:

Here is what the stage in my release looks like using the PowerShell task. Ensure to scroll all the way over to the right

Notice everything is called with variables :).

When you are editing your release, just hit the variables tab next to tasks and set your own accordingly. I am using an azure service principal, so keep in mind that SPN must have the proper permissions on the subscription.

The last part of the deploy.ps1 runs an ARM template. We pass in the variables dynamically generated in the deploy.ps1 to the ARM template, this includes the agents IP address. In the ARM template, we just ensure that IP address is in the allow for the whitelist.

The full ARM template is here

Just add the task into your release pipeline and it will update each time with the hosted agents IP address. It will not save the previous IP address, each time the job runs, the agents IP that is allowed will be over written with the agent running the job. So there is no need to worry about old IP addresses sticking around.

The total time it takes to run is about 2.5 minutes. Of course, there is always room for improvement!