Get started with Selenium testing in a continuous integration pipeline

Last Update: 4/7/2017

Visual Studio 2017 | Visual Studio 2015 | TFS 2017 | TFS 2015

Performing user interface testing as part of the build process is a great way of detecting unexpected changes and need not be difficult. This topic describes using Selenium to test your website in a continuous integration build.

In this topic:

For more information about Selenium browser automation, see:

Create the test project

As there is no template for Selenium testing, the easiest way to get started is to use the Unit Test template. This automatically adds the test framework references and enables you run and view the results from Visual Studio Test Explorer.

  1. In Visual Studio, open the File menu and choose New Project, then choose Test and select Unit Test Project. Alternatively, open the shortcut menu for the solution and choose Add then New Project and then Unit Test Project.

    For more details, see: Get started with developer testing tools.

  2. After the project is created, you must add the Selenium and browser driver references used by the browser to execute the tests. Open the shortcut menu for the Unit Test project and choose Manage NuGet Packages. Add the following packages to your project:

    • Selenium.WebDriver (for Firefox)
    • Selenium.WebDriver.IEDriver
    • Selenium.WebDriver.ChromeDriver
    • Selenium.WebDriver.PhantomJS.Xplatform

    Adding the browser driver packages to your solution

  3. The Unit Test project creates a default class named UnitTest1.cs. To author a Selenium Test, replace the contents of the file with the following code.

      namespace Partsunlimited.UITests
        using Microsoft.VisualStudio.TestTools.UnitTesting;
        using OpenQA.Selenium;
        using OpenQA.Selenium.Chrome;
        using OpenQA.Selenium.Firefox;
        using OpenQA.Selenium.IE;
        using OpenQA.Selenium.Remote;
        using OpenQA.Selenium.PhantomJS;
        using System;
        public class ChucksClass1
          private string baseURL = "";
          private RemoteWebDriver driver;
          private string browser;
          public TestContext TestContext { get; set; }
          public void TireSearch_Any()
            driver = new FirefoxDriver();
            driver.FindElementById("search - box").Clear();
            driver.FindElementById("search - box").SendKeys("tire");
            //do other Selenium things here!
          public void MyTestCleanup()
          public void MyTestInitialize()

    Replacing the code in UnitTest1.cs

  4. Run the Selenium test locally using Test Explorer.

    Running the tests in Visual Studio Test Explorer

Include the test in a continuous integration build

To include the Selenium test as part of a build, the source code must be in version control.

Checking the code into Visual Studio Team Services

Create a continuous integration build definition

  1. In your Visual Studio Team Services account where you checked in the test code, open the Build & Release hub and select the Builds tab.

  2. Create a new build definition using the Visual Studio build template. In the next page of the Create new build definition wizard:

    • Ensure that the repository and branch where you checked in your code is selected.

    • Select the Continuous integration... checkbox so that your solution builds after each check-in using continuous integration.

    • Select the Default agent queue in which you have installed an agent. If you have not installed an agent in the Default queue, choose the manage queues link and do that now. For information about installing a build agent, see Deploy a Windows build agent.

    Selenium tests will generally be run interactively, which would fail on the Hosted build controller.

  3. Choose Create to complete the Create new build definition wizard.

  4. Delete the Test Assemblies (Visual Studio Test) task step from the build definition, then add a Visual Studio Test Agent Deployment, Windows Machine File Copy, and Run Functional Tests task from the Test and Deploy tabs of the task catalog. Drag and drop then in that order immediately after the Publish symbols path task step.

    Tasks in the build definition

  5. Configure the tasks as shown in the following table.

    Nuget Installer
    Package: Nuget Installer

    Install and update NuGet package dependenciess.

    • Path to solution or packages.config: Select your app solution (.sln) file.
    • Installation type: Restore
    Visual Studio Build
    Build: Visual Studio Build

    Build with MSBuild and set the Visual Studio version property.

    • Solution: Select your app solution (.sln) file.
    • Platform: $(BuildPlatform)
    • Configuration: $(BuildConfiguration)
    • Visual Studio Version: Select the version used to create your app.
    Index Sources & Publish Symbols
    Test: Index Sources & Publish Symbols

    Index the source code and optionally publish symbols to a SymStore file share.

    • Search pattern: **\bin\**\*.pdb
    Visual Studio Test Agent Deployment
    Test: Visual Studio Test Agent Deployment

    Deploy and configure the test agent to run tests on a set of machines.

    • Machines: Comma-delimited list of machine names, or a variable containing the list.
    • Admin Login: Username for target server or a variable containing it.
    • Admin Password: Password for target server or a variable containing it.
    • Protocol: HTTP
    • Select Machines By: Machine Names
    • Agent Configuration | Username: Agent username or a variable containing it.
    • Agent Configuration | Password: Agent password or a variable containing it.
    • Agent Configuration | Interactive Process: Checked
    Windows Machine File Copy
    Deploy: Windows Machine File Copy

    Copy files to remote machines.

    • Source: $(Build.Repository.LocalPath)
    • Machines: Comma-delimited list of machine names, or a variable containing the list.
    • Admin Login: Username for target server or a variable containing it.
    • Password: Password for target server or a variable containing it.
    • Destination Folder: C:\Deploy or another folder on the target server.
    Run Functional Tests
    Test: Run Functional Tests

    Run Coded UI tests, Selenium tests, and functional tests on a set of machines using the test agent.

    • Machines: Comma-delimited list of machine names, or a variable containing the list.
    • Test Drop Location: C:\Deploy or the folder where you copied the files if different.
    • Execution Options | Test Selection: Test Assembly
    • Execution Options | Test Assembly: **\*Test*.dll
    Copy Files
    Test: Copy Files

    Copy files from a source folder to a target folder using match patterns.

    • Source Folder: $(build.sourcesdirectory)
    • Contents: **\bin\$(BuildConfiguration)\**
    • Target Folder: $(build.artifactstagingdirectory).
    Publish Build Artifacts
    Test: Publish Build Artifacts

    Publish Build artifacts to the server or a file share.

    • Path to Publish: Select your Azure subscription
    • Artifact Name: drop
    • Artifact Type: Server

    It's generally advisable to use custom variables for parameter values, especially where the same value is used in the parameters of more than one task. You can also secure and hide values by using custom variables. See Build Variables.

  6. Save the build definition and queue a new build.

  7. To validate the test results from a build, open the build summary from the Explorer tab.

    Selecting a build result

    The build summary includes a snapshot of the test results. There is also a Tests results page that highlights the build-on-build changes, including errors, stack traces, and the ability to easily create a bug that contains this information.

    The build summary and test results

Also see

For more information about Selenium browser automation, see:

For information about deploying PhantomJS.exe as part of your test, see this blog post.

Help and support

Submit bugs through Connect, make suggestions on Uservoice, and send quick thoughts using the Send-a-Smile icon link in the Visual Studio, Team Services, or TFS title bar. We look forward to your feedback.