Team Services | TFS 2017 | TFS 2015 | Previous versions (XAML builds)
Deploy and configure the test agent to run tests on a set of machines.
The test agent deployed by this task can collect data or run distributed tests using the Visual Studio Test task.
Demands and prerequisites
This task requires the target computer to have:
- Windows 7 Service Pack 1 or Windows 2008 R2 Service Pack 2 or higher
- .NET 4.5 or higher
- PSRemoting enabled by running the Enable-PSRemoting PowerShell script
Windows Remote Management (WinRM)
This task uses Windows Remote Management (WinRM) to access on-premises physical computers or virtual computers that are domain-joined or workgroup-joined.
To set up WinRM for on-premises physical computers or virtual machines
Follow the steps described in domain-joined
To set up WinRM for Microsoft Azure Virtual Machines
Azure Virtual Machines require WinRM to use the HTTPS protocol. You can use a self-signed Test Certificate. In this case, the automation agent will not validate the authenticity of the certificate as being issued by a trusted certification authority.
Azure Classic Virtual Machines. When you create a classic virtual machine from the Azure portal, the virtual machine is already set up for WinRM over HTTPS, with the default port 5986 already opened in the firewall and a self-signed certificate installed on the machine. These virtual machines can be accessed with no further configuration required. Existing Classic virtual machines can be also selected by using the Azure Resource Group Deployment task.
Azure Resource Group. If you have an Azure Resource Group already defined in the Azure portal, you must configure it to use the WinRM HTTPS protocol. You need to open port 5986 in the firewall, and install a self-signed certificate.
To dynamically deploy Azure Resource Groups that contain virtual machines, use the Azure Resource Group Deployment task. This task has a checkbox named Enable Deployment Pre-requisites. Select this to automatically set up the WinRM HTTPS protocol on the virtual machines, open port 5986 in the firewall, and install a test certificate. The virtual machines are then ready for use in the deployment task.
|Test Machine Group|
A comma-separated list of machine FQDNs or IP addresses, optionally including the port number. Can be:
The username of either a domain or a local administrative account on the target host(s).
The password for the administrative account specified above.
The protocol that will be used to connect to the target host, either HTTP or HTTPS.
Required. The username that the test agent will use.
Required. The password for the Username for the test agent. To protect the password, create a variable in the Variables tab and use the "padlock" icon.
|Run UI tests||
When set, the test agent will run as an interactive process. This is required when interacting with UI elements or starting applications during the tests. For example, Coded UI or Selenium tests that are running on full fidelity browsers will require this option to be set.
|Enable data collection only||
When set, the test agent will return previously collected data and not re-run the tests.
|Test agent version||
The version of the test agent to use.
|Test agent location||
Optional. The path to the test agent (vstf_testagent.exe) if different from the default path.
|Update test agent||
If set, and the test agent is already installed on the test machines, the task will check if a new version of the test agent is available.
Use this task for:
- Running automated tests against on-premises standard environments
- Running automated tests against existing Azure environments
- Running automated tests against newly provisioned Azure environments
The supported options for these scenarios are:
- On-premises and Visual Studio Team Services
- Build and release agents
- Hosted and on-premises agents are supported.
- The XPlat agent is not supported for any build/deploy/test tasks
- The agent must be able to communicate with all test machines. If the test machines are on-premises, the Hosted agent pool in Team Services cannot be used.
- The agent must have Internet access to download test agents.
If this is not the case, the test agent must be manually
downloaded, uploaded to a network location accessible to
the agent, and the Test Agent Location parameter used
to specify the location. The user must manually check for
new versions of the agent and update the test machines.
- Continuous integration/continuous deployment workflows
- Build/deploy/test tasks are supported in both Build and
Release Management workflows.
- Build/deploy/test tasks are supported in both Build and Release Management workflows.
- Machine group configuration
- Only Windows-based machines are supported inside a machine group for build/deploy/test tasks. Linux, iOS, or other platforms are not supported inside a machines group.
- Installing any version of Visual Studio on any of the test machines is not supported.
- Installing any older version of the test agent on any of the
test machines is not supported.
- Test machine topologies
- Azure-based test machines are fully supported, both existing test machines and newly provisioned test machines.
- Machines with the test agent installed must have network access to the TFS instance in use. Network-isolated test machines are not supported.
- Domain-joined test machines are supported.
- Workgroup-joined test machines must use HTTPS authentication configured during machine group creation.
- Usage Error Conditions
- Using the same test machines across different machine groups, and running builds (with any build/deploy/test tasks) in parallel against those machine groups is not supported.
- Cancelling an in-progress build or release that contains any build/deploy/test tasks is not supported. If you do cancel, behavior of subsequent builds may be unpredictable.
- Cancelling an ongoing test run queued through build/deploy/test tasks is not supported.
- Configuring the test agent and running tests as a non-administrator, or by using a service account, is not supported.
- Running tests for Universal Windows Platform apps is not supported. Use the Visual Studio Test task to run these tests.
How do I configure a Machine Group for testing?
Choose the Manage link next to the Machines
parameter to open the Machine Group page where you can create and manage your Machine Groups. See Testing in Continuous Integration and Continuous Deployment Workflows for more details.
How do I create an Azure Resource Group for testing?
When would I use the Enable Data Collection Only option?
An example would be in a client-server application model, where you deploy the test agent on the servers and use another task to deploy the test agent to test machines. This enables you to collect data from both server and client machines without triggering the execution of tests on the server machines.
Do I need a build agent?
You need at least one agent to run your build. Get an agent.
I can't select a default agent queue and I can't queue my build. How do I fix this?
I use Team Foundation Server on-premises and I don't see some of these features. Why not?
How can I submit feedback on this task?
Visit our UserVoice forum