Concurrent build and release pipelines in Visual Studio Team Services

Last Update: 3/17/2017

TFS 2017 | Team Services

A Team Services concurrent pipeline gives you the ability to run a single build or a single release at a time in your account.

We provide one free concurrent pipeline in each Team Services account. We also provide 240 minutes of total compute time per month from a hosted agent to run a build or a release. Each build or release job cannot run for more than 30 minutes.

For no additional charge you can keep hundreds or even thousands of build and release definitions in your account. You also can register any number of private agents with your account for no additional charge.

To run more than one build or release at a time, you need additional concurrent pipelines, which you can buy from the Visual Studio Marketplace.

How a concurrent pipeline is consumed

For example, a Team Services account has one concurrent pipeline. This allows users in that account to run only one build or release at a time. When additional builds and releases are triggered, they are queued and will wait for the previous one to complete.

A release requires a concurrent pipeline only when it is being actively deployed to an environment. Waiting for an approval or a manual intervention does not consume a concurrent pipeline.

Concurrent pipelines simple example

  1. FabrikamFiber CI Build 102 (master branch) is first to be started.
  2. Deployment of FabrikamFiber Release 11 is triggered by completion of FabrikamFiber CI Build 102.
  3. FabrikamFiber CI Build 101 (feature branch) is triggered. The build can't start yet because Release 11's deployment is active. So the build stays queued.
  4. Release 11 waits for approvals. Fabrikam CI Build 101 starts because a release waiting for approvals does not consume a concurrent pipeline.
  5. Release 11 is approved. It resumes only after Fabrikam CI Build 101 is completed.

Concurrent processing within a single build or release

Concurrent processing within a single build or release does not require additional concurrent pipelines. So long as you have enough agents, you can

  • In a build process, run multiple build configurations at the same time.

  • In a release process, deploy to multiple environments at the same time.

For example, suppose your Team Services account has three concurrent pipelines. You can have more than three agents running at the same time to perform parallel operations within builds and releases. For instance, notice below at 9 a.m. that five agents are actively running jobs from three concurrent pipelines.

Concurrent pipelines with additional agents example

Determine how many concurrent pipelines you need

You can begin by seeing if your teams can get by with the concurrent pipelines you've got by default. As the number of queued builds and releases exceeds the number of concurrent pipelines you have, your build and release queues will grow longer. When you find the queue delays are too long, you can purchase additional concurrent pipelines as needed.

Simple estimate

A simple rule of thumb: Estimate that you'll need one concurrent pipeline for every 10 users in your account.

Detailed estimate

In the following scenarios you might need multiple concurrent pipelines:

  • If you have multiple teams, and if each of them require a CI build, then you'll likely need a concurrent pipeline for each team.

  • If your CI build trigger applies to multiple branches, then you'll likely need a concurrent pipeline for each branch.

  • If you develop multiple applications using one account or server, then you'll likely need additional concurrent pipelines: one to deploy each application at the same time.

Purchase additional concurrent pipelines

If you need to more concurrent builds and releases, you can buy concurrent pipelines from the Visual Studio marketplace.

Private pipelines

If you want to run builds and releases on your own machines (private agents), then buy private pipelines for build and release. After you've done this, you can deploy your own private agents and use them with these concurrent pipelines.

Hosted pipelines

If want to run builds and releases on machines on a hosted agent, then buy hosted pipelines for build and release. With the first purchase of a hosted pipeline, the 240 minute limit on total build and release time as well as the 30 minute limit on a single job are waived. Each additional purchase of a hosted pipeline adds another hosted agent for running your builds and releases.

Sharing of concurrent pipelines among private and hosted agents

No matter which kind of concurrent pipelines you purchase, they are all shared and made available to both private and hosted agents. For example:

  1. You purchase a single private pipeline and a single hosted pipeline.

  2. You deploy two private agents in a pool.

  3. One build and one release are running in your private agent pool.

  4. A build is queued in the hosted pool. That build will not start until one of the builds in your private pool is completed.

View available pipelines

  1. Browse to Account settings, Build and Release, Resource limits.

    control-panel-account-build-and-release-resource-limits

    URL example: https://{your_account}/_admin/_buildQueue?_a=resourceLimits

  2. View the maximum number of concurrent pipelines that are available in your account.

  3. Select Pipelines queue... to display all the builds and releases that are actively consuming an available pipeline or that are queued waiting for a pipeline to be available.

Q&A

Who can use the Build and Release Management features?

Team Services users with basic access can author as many builds and releases as they want.

To approve releases, basic access is not necessary. Any user with stakeholder access can approve or reject releases.

I purchased agents. How will the change to concurrent pipelines affect me?

If you've purchased private agents, we'll convert them into private pipelines. If you've purchased hosted agents, we'll convert those into hosted pipelines. These conversions should not result in any changes to how much we charge you. In fact, now that we charge you for concurrent pipelines instead of agents, you might find that you don't need to buy as many private pipelines.

Can I control how concurrent pipelines are allocated to agent pools or team projects?

No. For instance, one team project in your account or collection may consume all the available pipelines at a given time. However, in this case, builds and releases from other projects are queued and allocated to pipelines in the order they are created. So, they will be served as other builds complete.

I use XAML build controllers with my account. How am I charged for those?

You can register one XAML build controller for each concurrent pipeline in your account. Your account gets one free concurrent pipeline by default, so you can register one XAML build controller for no additional charge. For each additional XAML build controller, you'll need an additional concurrent pipeline.

Why don't I see Visual Studio Enterprise benefits for concurrent pipelines in my Team Services account?

We do offer Visual Studio Enterprise customers some concurrent pipeline benefits in Team Foundation Server 2017. We don't yet offer this benefit in Team Services.

I'm using the Hosted VS2017 or Hosted Linux Preview queue and I'm getting only one agent at a time. Why?

We're offering the Hosted VS2017 and Hosted Linux Preview queues as a preview. So we provide only one of these agents at a time. Eventually we'll offer multiple hosted agents with these capabilities.