Repeat a test with different data

Last Update: 4/7/2017

Team Services | TFS 2017 | TFS 2015

When you write a manual test, you often want to specify that the test should be repeated several times with different test data. For example, if your users can add different quantities of a product to a shopping cart, then you want to check that a quantity of 200 works just as well as a quantity of 1.

To do this, you insert parameters in your test steps. Along with the test steps, you provide a table of parameter values. You can also share parameters and their data between test cases when you use the web portal with Team Foundation Server 2015 and higher or Team Services. That way you can run multiple test cases with the same data.

Add parameters to a test case

  1. Create a parameter by typing a name preceded by "@" in the actions and expected results of your test steps.

    Creating a parameter

  2. Underneath the list of steps, add combinations of parameter values. You might need to scroll down to see them.

Share parameters between test cases

  1. Convert existing parameters to shared parameters so that you can use them and the associated data in other test cases.

    Converting existing parameters to shared parameters

  2. After you've created a shared parameter set, open another test case and add the shared parameter set to that test case. You can search for the shared parameter set by name.

    Adding the shared parameter set to a test case

    The shared parameter set is displayed in the Parameter values section after you add it. You can now use these parameters in your test case steps.

  3. If the test case has different parameter names for these shared parameters, map the shared parameters to the local parameters to use the shared parameter data.

    Mapping a shared parameter to a local parameter

    When they are correctly mapped, the data associated with the shared parameter is displayed.

  4. Add, edit and rename your shared parameter sets in the Parameters tab. View the test cases that reference them in the Test cases pane.

    Adding, editing, viewing, and renaming a shared parameter

  5. Each shared parameter set is a work item. Open the Properties tab to view or make changes to this work item. For example, you can assign owners and track changes.

Run a test case with parameters

  1. Select a test case with parameters and start it running. The Test Runner shows the first row of parameter values.

    Test Runner showing the first row of parameter values

  2. When you've completed the steps, mark the test passed or failed. Then go on to the next iteration of the test, which uses the next row of parameter values.

    Going on to the next iteration of the test

  3. Use the drop down to navigate to other iterations.

    Navigating to other iterations

  4. If any of the parameter values are incorrect, fix them without canceling the test by choosing Edit from step's shortcut menu.

Review the test results

If you marked any test iteration as failed, the outcome of the entire test is shown as failed.

  1. Check the test result by opening the details pane.

    Checking the test result in the details pane

  2. Double-click a test result to view the test run details, and the test results for each iteration.

Speed up test iterations by using record and playback

It can be error-prone and tedious to work through a long table of parameter combinations. To speed things up, create an action recording when you run the test with the first set of parameter values, and then play it back for the other sets.

  1. Use Microsoft Test Manager to run the test.

  2. Select Create action recording before you choose Start.

  3. Complete the first test iteration and then move on to the next one.

  4. Mark each step as passed or failed as you work. Enter parameter values in the application exactly as they are displayed in the test script.

  5. Choose Play to run the test with the next set of parameter values. Your actions will be played back automatically, but you must still verify the results.

Record and playback doesn't work with all applications. For details, see Supported Configurations and Platforms for Coded UI Tests and Action Recordings.

Q&A

Q: Are parameters the best way to specify that the test should be run on different operating system platforms? And with different browsers, databases, and so on?

A: It's better to use test configurations for that. With test case parameters, you run the different parameter values one after another, which makes it difficult to switch from one platform to another.

Q: Can I use parameters in shared steps?

A: Yes. You set the parameter values in the test cases where you use the shared steps.

Q: Can I import parameter values from an Excel spreadsheet to my shared parameter sets?

A: Yes. Copy the data from your Excel spreadsheet and paste it into your shared parameters grid. You can also copy the data from your grid back into Excel if you need to.

See also

Help and support

Submit bugs through Connect, and make suggestions on Uservoice. We look forward to your feedback.