Load test in the cloud

Find performance problems in your app before customers do. Make sure your app can handle traffic for your next launch or marketing campaign. Start running cloud-based load tests in almost no time with Visual Studio Online.

Why load test in the cloud?

  • Quick: Just one click gets you a performance lab in the cloud, no setup needed. Scale up or down based on your needs.
  • Free load testing: Get 20,000 virtual user minutes every month to load test at no extra charge.
  • Real-time charts and graphs: Watch your app's performance while your tests run. Share the results when your tests are done.
  • Test where your users are: Test from different locations to reduce latency and simulate local conditions.
  • Get deeper insights about performance: Get full server and client diagnostics when you simulate real-life load patterns for your app.

To get more insight into how well your web app or site performs, start load testing in the cloud with Visual Studio Ultimate 2013, our sample load test project or your own load test project, and Visual Studio Online.

Or, if you have a Visual Studio Ultimate with MSDN subscription, you can run a basic load test directly in Visual Studio Online. No load test project necessary.

Load test with Visual Studio

Before you start:

  • Create your Visual Studio Online account, if you don't have one already. You can have any license assigned to you in Visual Studio Online when you use Visual Studio Ultimate 2013 to run load tests.

  • If you don't have a load test project, use our sample load test project with your web site or app. Just provide the address for the web site that you want to test. Or, if you have a load test project, jump ahead to set up your load test project for load testing with Visual Studio Online.

Get the sample load test project

  1. Download and install Visual Studio Ultimate 2013 or the latest update, if you haven't already.

  2. Download the sample load test project, and unzip the files.

  3. Open the GettingStartedWithLoadTesting.sln solution.

  4. Open the SampleWebTest.webtest file. Replace the URL with your app's web page.

    Double-click SampleWebTest.webtest. In Properties window, replace URL with your web page address

  5. Save the solution, then jump ahead to Connect to your Visual Studio Online account.

Set up your load test project to run tests in the cloud

Do you have a load test project from Visual Studio 2010 or Visual Studio 2012? No problem, just open your project in Visual Studio Ultimate 2013, then set up that project for load testing with Visual Studio Online.

  1. Download and install Visual Studio Ultimate 2013, if you haven't already.

  2. Open your load test project in Visual Studio Ultimate 2013.

  3. Edit the test settings file so that your project uses Visual Studio Online for load testing.

    Choose Visual Studio Online to run your load tests

    When you're ready to start load testing, connect Visual Studio Ultimate to Visual Studio Online.

Connect to your Visual Studio Online account

Before you can run load tests in the cloud, connect Visual Studio Ultimate to your Visual Studio Online account.

  1. If you haven't already, sign up for a Visual Studio Online account. Find out more about signing up for Visual Studio Online.

  2. In Team Explorer, connect to your Visual Studio Online account.

    Connect to your Visual Studio Online account

  3. View your available team projects.

    Click Select Team Projects

    a. If you haven't connected to your Visual Studio Online account before, add your account to the server list.

    Click Servers to add your Visual Studio Online account

    b. Enter your Visual Studio Online account name (youraccount.visualstudio.com).

    Enter your account name

    c. If you're prompted to sign in to Visual Studio Online, do that.

  4. Select your Visual Studio Online account from the list, then choose your team project. Now you can connect.

    Choose your team project

Run and analyze your load test

  1. In Solution Explorer, open the load test that you want to run.

    Solution Explorer: double-click your load test

  2. Now run your load test. This will run in the cloud using Visual Studio Online.

    On the load test toolbar, click Run

    Your test appears in the queue and waits for its turn to run. When Visual Studio Online is ready to run your test, the test status changes to "Acquiring resources".

    Your load test appears in the queue

    A large test run might take up to 10 minutes while Visual Studio Online sets up virtual machines and agents for you.

  3. You can watch your app's performance while the test runs. Look at the details to review errors, warnings, or other information about your test.

    Running your load test show the performance

  4. When the test is done, download the report to view the results.

    Click Download to save the report

    Click View Report to view results

    The results include performance counter data, threshold violations, and error information.

  5. Review your test's details. Find the number of users where your app's performance fails to meet your requirements by examining the step load pattern for virtual users.

    View load test run details

  6. Fix any performance issues that you find in your app's code, then rerun the test.

    To simulate real-world loads more closely, you can refine your test by specifying web performance test properties, load test scenario properties, and run settings properties.

View your load test runs

You can review past load test runs or current runs started by anyone on your team, at any time.

  1. Open your load test project, so you can open Load Test Manager.

    Open Load Test Manager

  2. Now you can view any load test runs started by anyone on your team.

    View all the past and current load test runs

  3. You can filter test runs by state, date, or user who created the test run.

    Filter load test runs by state, date, or user

Load test with Visual Studio Online

If you have Visual Studio Ultimate with MSDN subscription, you can run a basic load test on your web app or site directly from your Visual Studio Online account, no other tools necessary.

  1. Create your Visual Studio Online account, if you don't have one already.

  2. Sign in to your Visual Studio Online account (youraccount.visualstudio.com), if you're not signed in already.

  3. Go to the Load Test hub.

    From your account home page, go to Load Test

  4. Enter your load test details. To run your test closer to where your users are located, select a closer location for your load test. Then start your test when you're ready.

    Add load test information, then click Test now

  5. Watch your app's performance on the graph while the test runs, or come back later for the results.

    Real-time app performance chart during load test

  6. When your test is done, look at the results to see how well your app performed.

    For example, you can see your app's average response time, failed requests, and any errors that might have happened.

    Load test high-level results and summary

Try this next

Q&A

Q: How do I learn more about cloud-based load testing?

A: Watch this video, or check out the Cloud-based Load Testing blog here, or the Cloud-based Load Testing FAQ here.

Q: What are virtual user minutes?

A: Find out more about virtual user minutes (VUserMinutes).

Q: Are there any limits when running cloud-based load tests?

A: Yes, the free limit is 20,000 virtual user minutes per month. If you need more than this limit, ask the Visual Studio Online account owner to get more resources for cloud-based load testing.

Your account is charged when the number of used virtual user minutes goes above the free limit each month. Also, the account owner might set a limit on how many paid virtual user minutes that you can use each month.

If you're load testing directly from your Visual Studio Online account, your maximum load is 200 virtual users.

If you're using the Visual Studio Ultimate IDE during the trial period, your maximum load is 250 virtual users. This limit is removed after you get an active and valid license for Visual Studio Ultimate.

Q: Help, I'm having problems running load tests in the cloud!

A: Check out the latest troubleshooting advice.

Q: Where is my load test report stored after I download it?

A: Your downloaded reports are stored in a local SQL Server Express database. You can change the default location, if you want. You can also store all the reports together for everyone by changing the location for each user to the same database.

Q: My trial period for Visual Studio Ultimate ended, but I still want to run load tests?

A: To continue load testing after the trial, you'll need an active and valid Visual Studio Ultimate license. Find out more about licensing here.

Q: Can I load test an internal web site using Visual Studio Online?

A: No, you can't load test a web site that's only available to your company, like an internal staging or preproduction deployment. But, you can open up your web site's endpoint for limited IP addresses, or for users to run cloud-based load testing. Or, you can run a load test locally using Visual Studio.

Q: Can I run load tests locally and in the cloud from the same project?

A: Yes, your project can have multiple test settings files. Add another test settings file to your Solution Items folder.

Right-click test settings file. Click Active Load and Web Test Settings

Now you can use one settings file to run your tests locally and the other settings file to run your load tests in the cloud. To switch between them, open the file's shortcut menu, then select the test settings file that you want to use.

Q: Can I debug a load test while it's running in the cloud?

A: No, you have to run a load test locally to debug it.

Q: Can load tests use other test types, besides web performance tests, in their test mix?

A: Yes, you can include unit tests and coded web tests. But coded UI tests aren't supported by cloud-based load testing.

Q: What are test agents and why do I need them?

A: Test agents simulate the load on your web site by creating virtual users. You might need to use more agents, depending on your load test and the number of virtual users that you require. Your agent count (total cores) can't exceed 100, and your ratio of virtual users to agents must be 25 or more.

Update the number of agents (total cores) to use

For example, increase the number of agents that you're using, if you're getting status messages that an agent stopped working due to load, or if the downloaded report shows high CPU use for an agent.

If you run a test with 40 virtual users and 2 agents, the ratio is 20. This is too low, so increase the number of virtual users, or reduce the number of agents. If you use the default value of 0, this ratio does not apply.

Q: Can virtual users simulate pausing between test steps?

A: Yes, you can specify think times. Select a scenario in your load test and edit the think time in the Properties view.

Q: Where can I get more information about simulating real-world loads?

A: Find out more about how to specify web performance test properties, load test scenario properties and run settings properties.

Q: Are there load test features that aren't supported when you use run load tests in the cloud?

A: These features aren't currently supported:

  • Goal-based load patterns
  • Iteration-based runs
  • Network mix property
  • Agent to Use in test settings - use the core count property instead
  • SQL Trace properties in run settings
  • IP switching