Deploy: Azure File Copy

Last Update: 4/28/2017

Team Services | TFS 2017 | TFS 2015 Update 3 | Previous versions: XAML Build, Release

icon Copy files to Microsoft Azure storage blobs or virtual machines (VMs).

The task is used to copy application files and other artifacts that are required in order to install the app; such as PowerShell scripts, PowerShell-DSC modules, and more.

When the target is Azure VMs, the files are first copied to an automatically generated Azure blob container and then downloaded into the VMs. The container is deleted after the files have been successfully copied to the VMs.

The task uses AzCopy, the command-line utility built for fast copying of data from and into Azure storage accounts.

To dynamically deploy Azure Resource Groups that contain virtual machines, use the Azure Resource Group Deployment task. This task has a sample template that can perform the required operations to set up the WinRM HTTPS protocol on the virtual machines, open the 5986 port in the firewall, and install the test certificate.

Demands

  • none

Arguments

Argument Description
Source Required. The source of the files to copy. Pre-defined system variables such as $(Build.Repository.LocalPath) can be used. Names containing wildcards such as *.zip are not supported.
Azure Connection Type Required. Select the type of service endpoint used to define the connection to Azure. Choose Azure Classic or Azure Resource Manager.
Azure Classic Subscription Required if you select Azure Classic for the Azure Connection Type parameter. The name of an Azure Classic service endpoint configured for the subscription where the target Azure service, virtual machine, or storage account is located.
Azure RM Subscription Required if you select Azure Resource Manager for the Azure Connection Type parameter. The name of an Azure Resource Manager service endpoint configured for the subscription where the target Azure service, virtual machine, or storage account is located. See Azure Resource Manager overview for more details.
Destination Type Required. The type of target destination for the files. Choose Azure Blob or Azure VMs.
Classic Storage Account Required if you select Azure Classic for the Azure Connection Type parameter. The name of an existing storage account within the Azure subscription.
RM Storage Account Required if you select Azure Resource Manager for the Azure Connection Type parameter. The name of an existing storage account within the Azure subscription.
Container Name Required if you select Azure Blob for the Destination Type parameter. The name of the container to which the files will be copied. If a container with this name does not exist, a new one will be created.
Blob Prefix Optional if you select Azure Blob for the Destination Type parameter. A prefix for the blob names, which can be used to filter the blobs. For example, using the build number enables easy filtering when downloading all blobs with the same build number.
Cloud Service Required if you select Azure Classic for the Azure Connection Type parameter and Azure VMs for the Destination Type parameter. The name of the Azure cloud service in which the virtual machines run.
Resource Group Required if you select Azure Resource Manager for the Azure Connection Type parameter and Azure VMs for the Destination Type parameter. The name of the Azure Resource Group in which the virtual machines run.
Select Machines By Depending on how you want to specify the machines in the group when using the Filter Criteria parameter, choose Machine Names or Tags.
Filter Criteria Optional. A list of machine names or tag names that identifies the machines that the task will target. The filter criteria can be:
- The name of an Azure Resource Group.
- An output variable from a previous task.
- A comma-delimited list of tag names or machine names.
Format when using machine names is a comma-separated list of the machine FDQNs or IP addresses.
Specify tag names for a filter as {TagName}:{Value} Example: Role:DB;OS:Win8.1
Admin Login Required if you select Azure VMs for the Destination Type parameter. The user name of an account that has administrative permissions for all the target VMs.
- Formats such as username, domain\username, machine-name\username, and .\username are supported.
- UPN formats such as username@domain.com and built-in system accounts such as NT Authority\System are not supported.
Password Required if you select Azure VMs for the Destination Type parameter. The password for the account specified as the Admin Login parameter. Use the padlock icon for a variable defined in the Variables tab to protect the value, and insert the variable name here.
Destination Folder Required if you select Azure VMs for the Destination Type parameter. The folder in the Azure VMs to which the files will be copied. Environment variables such as $env:windir and $env:systemroot are supported. Examples: $env:windir\FabrikamFiber\Web and c:\FabrikamFiber
Additional Arguments Optional. Any arguments you want to pass to the AzCopy.exe program for use when uploading to the blob and downloading to the VMs. See Transfer data with the AzCopy Command-Line Utility for more details. If you are using a Premium storage account, which supports only Azure page blobs, the pass /BlobType:Page as an additional argument.
Enable Copy Prerequisites Available if you select Azure Resource Manager for the Azure Connection Type parameter and Azure VMs for the Destination Type parameter. Setting this option configures the Windows Remote Management (WinRM) listener over HTTPS protocol on port 5986, using a self-signed certificate. This configuration is required for performing copy operation on Azure virtual machines.
- If the target virtual machines are accessed through a load balancer, ensure an inbound NAT rule is configured to allow access on port 5986.
- If the target virtual machines are associated with a Network Security Group (NSG), configure an inbound security rule to allow access on port 5986.
Copy in Parallel Available if you select Azure VMs for the Destination Type parameter. Setting this option causes the process to execute in parallel for the copied files. This can considerably reduce the overall time taken.
Clean Target Available if you select Azure VMs for the Destination Type parameter. Setting this option causes all of the files in the destination folder to be deleted before the copy process starts.
Test Certificate Available if you select Azure VMs for the Destination Type parameter. WinRM requires a certificate for the HTTPS transfer when copying files from the intermediate storage blob into the Azure VMs. If you set use a self-signed certificate, set this option to prevent the process from validating the certificate with a trusted certificate authority (CA).
Output - Storage Container URI Optional. The name of a variable that will be updated with the URI of the storage container into which the files were copied. Use this variable as an input to subsequent task steps.
Output - Storage Container SAS Token Optional. The name of a variable that will be updated with the Storage Access Security (SAS) token of the storage container into which the files were copied. Use this variable as an input to subsequent task steps. By default, the SAS token expires after 4 hours.
Control options See Control options

Q&A

What are the Azure PowerShell prerequisites for using this task?

The task requires Azure PowerShell to be installed on the machine running the automation agent. The recommended version is 1.0.2, but the task will work with version 0.9.8 and higher. You can use the Azure PowerShell Installer v1.0.2 to obtain this.

What are the WinRM prerequisites for this task?

The task uses Windows Remote Management (WinRM) HTTPS protocol to copy the files from the storage blob container to the Azure VMs. This requires the WinRM HTTPS service to be configured on the VMs, and a suitable certificate installed.

If the VMs have been created without opening the WinRM HTTPS ports, follow these steps:

  1. Configure an inbound access rule to allow HTTPS on port 5986 of each VM.
  2. Disable UAC remote restrictions.
  3. Specify the credentials for the task to access the VMs using an administrator-level login in the simple form username without any domain part.
  4. Install a certificate on the machine that runs the automation agent.
  5. Set the Test Certificate parameter of the task if you are using a self-signed certificate.

For more details, see this blog post.

What type of service endpoint should I choose?

The following table lists the storage accounts and the service endpoint connections that work with them. To identify whether a storage account is based on the classic APIs or the Resource Manager APIs, log into the Azure portal and browse for Storage accounts (Classic) or Storage accounts.

Storage account type Azure Service Connections in TFS/TS
Resource Manager Azure Resource Manager service endpoint
Classic Azure service endpoint with certificate-based or credentials-based authentication using a school or work account
  • For Azure classic resources, use an Azure service endpoint type with certificate or credentials-based authentication. If you are using credentials-based authentication, ensure that the credentials are for a school or work account. Microsoft accounts such as joe@live.com and joe@hotmail.com are not supported.

  • For Azure Resource Manager VMs, use an Azure Resource Manager service endpoint type. More details at Automating Azure Resource Group deployment using a Service Principal.

  • If you are using an Azure Resource Manager service endpoint type, or an Azure service endpoint type with certificate-based authentication, the task automatically filters appropriate classic storage accounts, the newer Azure Resource Manager storage accounts, and other fields. For example, the Resource Group or cloud service, and the virtual machines.

  • Note: Currently an Azure service endpoint type with credentials-based authentication does not filter the storage, Resource Group or cloud service, and virtual machine fields.

What happens if my Resource Group contains both Classic and Resource Manager VMs?

If the specified Resource Group contains both Azure Resource Manager and Classic VMs, the set of VMs that will be targeted depends on the connection type. For certificate-based connections and credentials-based connections, the copy operation will be performed only on Classic VMs. For Service Principal Name based connections, the copy operation will be performed on only Resource Manager VMs.

How do I create a school or work account for use with this task?

A suitable account can be easily created for use in a service endpoint:

  1. Use the Azure portal to create a new user account in Azure Active Directory.
  2. Add the Azure Active Directory user account to the co-administrators group in your Azure subscription.
  3. Sign into the Azure portal with this user account and change the password.
  4. Use the username and password of this account in the service endpoint connection. Deployments will be processed using this account.

Do I need an agent?

You need at least one agent to run your build or release. Get an agent.

I can't select a default agent queue and I can't queue my build or release. How do I fix this?

See queues.

I use Team Foundation Server on-premises and I don't see some of these features. Why not?

Some of these features are available only on Visual Studio Team Services and not yet available on-premises. Some features are available on-premises if you have upgraded to the latest version of TFS.

Help and support