Collect more diagnostic data in manual tests

  • Article

TFS 2017 | TFS 2015
Visual Studio 2017 | Visual Studio 2015

Note

Microsoft Test Manager has been deprecated. It has been replaced with Azure Test Plans, a fully featured test management solution. Microsoft Test Manager 2017, which shipped with Microsoft Visual Studio 2017, is the last version available.

For Azure DevOps and TFS, see Collect diagnostic data while testing.

While you are testing your application, Microsoft Test Manager can collect data that will help diagnose any fault that you might find. If you create a bug report while you're testing, the data is automatically attached to the bug work item.

Test settings machine roles and adapters

You can decide what kinds of data you want to collect.

To use all the features described in this article, you must have one of the following subscriptions:

Or, configure Basic + Test Plans access level.

See Manual testing permissions and access.

What diagnostic data can I collect in a test?

The diagnostic data is collected in the test results. It will be added to a bug if you create one while performing the test.

Data you can collect How
- Link to the test case.
- The steps you marked as passed or failed.
- Any comments or attachments that you added.		 Run tests in the web portal.
+

- Operating system version and other system information.
- Your keystrokes and gestures.
- Screenshots, recorded automatically while you work.		 Run tests with Microsoft Test Manager.

(Use default test settings.)

Microsoft Test Manager has to be installed on the machine where you run the tests, or on a machine connected to a device where the test runs.
+

Data collected from your client or desktop application:

- Event logs
- IntelliTrace
- Video recording of the desktop
- Test impact analysis. This allows you to choose tests based on changes since a previous build.		 Use test settings when you perform the tests with Microsoft Test Manager.

Test settings files configure diagnostic data adapters. You can choose a test settings file when you run tests, or you can set a default test settings file for your test plan.
+

Data collected from your server software:

1. Event logs.
2. IntelliTrace
3. Test impact
4. Virtual machine snapshots of the servers, if you are using an SCVMM lab environment		 Use test settings to define the data you want to collect.

How do I create test settings?

You need test settings only if you want to collect more data than the default. The default setting collects basic system information from each lab machine and your keystrokes and gestures from the local machine.

  1. If the application you are testing is a website or has a server component, and you want to collect data from the servers:

    • Create a lab environment. It can be a standard environment or an SCVMM environment.

    • In the Properties of your test plan, set the test environment you want to use for manual tests.

    In test plan properties, choose a lab environment.

  2. Choose an existing test settings file, or create a new one.

    Select or create test settings.

    This sets the default selection for performing tests in this test plan. You can override the selection when you perform individual test runs.

  3. Give the test settings file a name.

    Set the name of the new test settings.

  4. Choose the lab environment you want to use for your tests. If you aren't using a lab environment, choose Local.

    On the Roles tab, choose the lab environment.

    Each test settings file matches one set of machine roles.

  5. For each machine role, choose the data you want to collect from that machine.

    Choose diagnostics for each machine role.

    The Local role is the client machine on which you'll perform the tests.

What are the diagnostic data options?

On the Data and Diagnostics page you can add and configure diagnostic adapters to collect data for each machine role in your lab environment. In most cases the diagnostic data is included with the test results.

Test setting Data and Diagnostic page

Diagnostic data adapter Configuration
Action Log: Allows you to record the actions you perform during your test, so that you can play them back rapidly on a subsequent occasion. The actions are also recorded as text descriptions in any bug report that you create. Record and play back manual tests.

Not all gestures and applications are recorded.
ASP.NET Client Proxy for IntelliTrace and Test Impact Select this adapter in a web client role. It is required if you are testing an ASP.NET application, and you want to collect Test Impact or Intellisense data on the web server role.
Event log

Collects events that your application wrote to the event logs.		 Choose Configure to select the types of events you want.

Your application has to write events using WriteEntry.
IntelliTrace: Generates an .itrace file that is linked to any bug that you create. From this IntelliTrace file, the local session can be simulated on another computer. Using IntelliTrace.
System information: Records information about the machine. No additional configuration.
Test impact: Enables the Recommended Tests feature in Testing Center, Track. This determines which tests are affected by the changes since a previous build, based on code coverage. If you are testing an ASP.NET application:

1. On the role where the IIS server will run, enable Test Impact and then choose Configure, Advanced, ASP.NET.
2. In the web client role, enable ASP.NET Client Proxy for IntelliTrace and Test Impact

Restart your server application after enabling this option.
Video Recorder records the desktop in real time while you work. To record audio, choose Configure.

Help and support

Report any problems on Developer Community, get advice on Stack Overflow, and get support via our Support page.