Get started with continuous testing

Last Update: 5/23/2017

Team Services | TFS 2017 | TFS 2015 | Previous version

Make sure that your app still works after every check-in and build using Visual Studio Team Services. Find problems earlier by running tests automatically with each build. When your build is done, review your test results to start resolving the problems that you find.

Before you start

This overview shows how to get started by running unit tests with your build.

  1. Check in your solution to Visual Studio Team Services. Include your test projects.

  2. Create a build definition that builds your solution after each check-in, using continuous integration.

When you're ready to run other continuous tests, such as system tests, under more realistic conditions, create environments from physical or virtual machines that you've already set up. Then set up your build to run your app and tests in the environments that you created.

For information about setting up test machines in a test lab, see Use a lab environment for your devops

Run unit tests with your builds

Your build definition includes a test task that runs unit tests. For example, if you're building a Visual Studio solution in Team Services, your build definition includes a Visual Studio Test task. After your build starts, this task automatically runs all the unit tests in your solution - on the same build machine.

  1. If you want to customize your test run, go to your team project to find your build definition in the Explorer tab and edit it.

    Edit build definition

  2. Edit the Visual Studio Test task:

    Build def, edit VS Test task

  3. You can add filter criteria to run specific tests, enable code coverage, run tests from other unit test frameworks, and so on. For information about the option settings of the Visual Studio Test task, see Run Tests using Visual Studio task.

    Build definition: customize unit test run

  4. When you're done, save your build definition.

    Build definition: save

  5. Start the build.

    Build definition: queue build

  6. After the build finishes, you can review the test results to resolve any problems that happened. Go to the build to open the build summary.

    Go to Build hub, build definition, build summary

  7. Open the test run results summary.

    Go to test run results summary

  8. Compare your test results between this build and the last build.

    Here you'll find changes in new, failed, and passed tests, how long these tests took to run, how long these tests have been failing, and more. Organize your test results and open bugs directly for failed tests.

    Compare test result summaries between builds

  9. To start debugging a failed test, review the resulting error and stack trace.

    Error and stack trace for a failed test

Next: Run other continuous tests with your builds

When you're ready to test your app more extensively under more realistic conditions, on different platforms and configurations:

  1. Create environments from physical or virtual machines that you've already set up.

  2. Set up your build to run your app and tests in the environments that you created.

  3. After your build finishes, review your test results to start resolving problems that you found.

Q & A

Q: Where can I find details about configuring test agents?

A: See Install and configure test agents

Q: Where can I learn more about integrating tests with my build?

A: Try these blog posts and videos:

Q: How do I run tests from different unit test frameworks with my build?

A: First, set up your test frameworks in your development tool. For example, in Visual Studio:

  1. Set up the plug-in for your test framework, if you haven't already.

  2. Create a folder that has the custom binaries for your plug-in. (The plug-in package for your framework is a .vsix file. Change the extension to .zip so that you can unzip it.)

  3. Add these assemblies to version control and let the build controller know where to get them.

  4. In your build definition, provide the path to the test framework:

    Build def, VSTest task, custom test framework

Q: Can I find a specific test run?

A: Yes, by finding that test run here:

Find test run

Q: Can I find specific results from a test run?

A: Yes, after you find your test run, create a query to find the test results you want:

Find specific test results

See also

Help and support

Submit bugs through Connect, make suggestions on Uservoice, and send quick thoughts using the Send-a-Smile icon link in the Visual Studio, Team Services, or TFS title bar. We look forward to your feedback.