Last Update: 4/12/2017

The TFS Database Import Service for Visual Studio Team Services is currently in preview.

Troubleshooting

TfsMigrator could flag errors which need to be corrected prior to performing a migration. Below are the most common errors that are encountered when prepping for a migration. After correcting each error you will need to run TfsMigrator's validate command again to ensure the error(s) is/are actually gone.

Dealing with Size Warnings

In rare cases you might receive a warning like the one below.

The database metadata size is currently {databaseMetadata}MBs, which is above the recommended size of 204800MBs. Please contact vstsdataimport@microsoft.com to continue.

Metadata size refers to the size of your database without including files, code, and other binary data. The warning doesn’t imply that your collection is too big for import, rather its metadata size is larger than the vast majority of other databases. Please reach out to the vstsdataimport@microsoft.com and include the warning so that we can ensure your databases size is taken into special consideration during the migration.

Dealing with Collation Errors

Collation in this case is referring to the collection database’s collation. Collation controls the way string values are sorted. TFS supports a multitude of collations, but Team Services at this time only supports a single collation - SQL_Latin1_General_CP1_CI_AS. Receiving an error like the one below means that we don’t support the collation you’re using.

Collation is not supported, Database collation 'Finnish_CI_AS', Supported collations 'SQL_Latin1_General_CP1_CI_AS;Latin1_General_CI_AS'

We’re taking steps to expand the number of collations that we support. Currently, we support SQL_Latin1_General_CP1_CI_AS and Latin1_General_CI_AS. Customers using collations other than those can attempt an import, but may see unexpected results or the import could fail. For instance, customers will notice different ordering for strings containing non-English characters and non-English characters like 'é' will become equivalent to the English 'e' after the import has completed.

It's also possible that the import could fail due to something unique with the collation that your database is using. It's recommended that you complete a dry run first.

Note that importing from a SQL Azure VM increases the chance that the import might not work when using a non-supported collation. We're working on improving this moving forward, but we recommend that if you're using a non-supported collation from a SQL Azure VM that you perform a dry run or try to reduce your database size to fit within the limits to use the DACPAC method to import.

Dealing with Identity Errors

Identity errors aren't common when validating a collection, but when they do come up it's important to fix them prior to migration to avoid any undesired results. Generally, identity problems stem from valid operations on previous versions of TFS that are no longer valid on your current TFS version. For example, some users being members of a built-in valid users group was once allowed, but isn't in more recent versions. The most common identity errors and guidance on fixing them can be found below.

ISVError:100014

This error indicates that a permission is missing from a system group. System groups are well known groups in TFS and Team Services. For example, every collection that you create has “Project Collection Valid Users” and “Project Collection Administrators” groups. They’re created by default and it’s not possible to edit the permissions for these groups. What this error indicates is that somehow one or more of these groups is missing a permission that it's expected to have. In order to fix this, you will need to use TFSSecurity.exe to apply the expected permissions onto the flagged system groups. To get started you will need to identify which TFSSecurity command(s) will need to be run.

Project Collection Valid Users Error Message

Carefully examine the error message(s) TfsMigrator highlighted. If the group that was flagged ends with “0-0-0-0-3”, such as in the example below, then you will need to fix a missing permission for the “Project Collection Valid Users” group. Run the below command against TFSSecurity.exe after replacing the scope with the one from the error message and adding in your collection URL.

TFSSecurity.exe /a+ Identity "{scope}\\" Read sid:{Group SID} ALLOW /collection:{collectionUrl}*

In the below example you will need to take the scope and group SID from the error message, and add it the templated command above.

ISVError:100014 Missing permission for group:Microsoft.TeamFoundation.Identity;S-1-9-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-0-0-0-0-3 for scope:397c326b-b97c-4510-8271-75aac13de7a9. Expected:1 and Actual:0 

The final command will look like:

TFSSecurity.exe /a+ Identity "397c326b-b97c-4510-8271-75aac13de7a9\\" Read sid:S-1-9-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-0-0-0-0-3 ALLOW /collection:https://localhost:8080/tfs/defaultcollection

Project Collection Administrators Error Message

Carefully examine the error message(s) TfsMigrator highlighted. If the group that was flagged ends with “0-0-0-0-1”, such as in the example below, then you will need to fix a missing permission for the “Project Collection Administrators” group. Run the below commands against TFSSecurity.exe after replacing the scope with the one from the error message and adding in your collection.

TFSSecurity.exe /a+ Identity "{scope}\\" Read sid:{Group SID} ALLOW /collection:{collectionUrl}

TFSSecurity.exe /a+ Identity "{scope}\\" Write sid:{Group SID} ALLOW /collection:{collectionUrl}

TFSSecurity.exe /a+ Identity "{scope}\\" Delete sid:{Group SID} ALLOW /collection:{collectionUrl}

TFSSecurity.exe /a+ Identity "{scope}\\" ManageMembership sid:{Group SID} ALLOW /collection:{collectionUrl}

In the below example you will need to take the scope and group SID from the error message, and add it the templated command above.

ISVError:100014 Missing permission for group:Microsoft.TeamFoundation.Identity;S-1-9-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-0-0-0-0-1 for scope:0c7c2216-fa4b-4107-a203-82b324a147ef. Expected:15 and Actual:0 

The final command will look like:

TFSSecurity.exe /a+ Identity "0c7c2216-fa4b-4107-a203-82b324a147ef\\" Read sid:S-1-9-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-0-0-0-0-1 ALLOW /collectionhttps://localhost:8080/tfs/defaultcollection

TFSSecurity.exe /a+ Identity "0c7c2216-fa4b-4107-a203-82b324a147ef\\" Write sid:S-1-9-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-0-0-0-0-1 ALLOW /collectionhttps://localhost:8080/tfs/defaultcollection

TFSSecurity.exe /a+ Identity "0c7c2216-fa4b-4107-a203-82b324a147ef\\" Delete sid:S-1-9-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-0-0-0-0-1 ALLOW /collectionhttps://localhost:8080/tfs/defaultcollection

TFSSecurity.exe /a+ Identity "0c7c2216-fa4b-4107-a203-82b324a147ef\\" ManageMembership sid:S-1-9-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXXX-0-0-0-0-1 ALLOW /collectionhttps://localhost:8080/tfs/defaultcollection

If you have multiple errors that need to be corrected, it’s recommended that you put all of the commands into a batch file to execute them against TFSSecurity in an automated fashion. Once the commands have been executed you will need to run TfsMigrator validate again to ensure that the error(s) has\have been corrected. If the error(s) still persists, please contact vstsdataimport@microsoft.com.

ISVError:300005

ISVError:300005 indicates that a non-group identity is a member of an everyone group, more commonly known as the Valid Users groups. Valid Users groups are created by default for all projects and collections. They’re uneditable groups that only contain other TFS groups as members. In the case of ISVError:300005, a non TFS group identity, such as an AD group or user identity, has a direct membership in a Valid Users group.

Since Valid Users groups can’t be edited directly or through TFSSecurity.exe, correcting the invalid membership will need to be done by running a SQL statement against the configuration database to remove the offending identity. Carefully examine the error message(s) TfsMigrator highlighted. You will need copy down the GroupSid, MemberId, and ScopeId as these values will need to be placed into the templated command below.

DECLARE @p6 dbo.typ_GroupMembershipTable

INSERT into @p6 values('{GroupSid}','Microsoft.TeamFoundation.Identity','{MemberId}',0)

EXEC prc_UpdateGroupMembership @partitionId=1,@scopeId='{ScopeId}',@idempotent=1,@incremental=1,@insertInactiveUpdates=0,@updates=@p6,@eventAuthor='9EE20697-5343-43FC-8FC5-3D5D455D21C5',@updateGroupAudit=0

Below is an example ISVError:300005 message from TfsMigrator.

ISVError:300005 Unexpected non group identity was found to have direct membership to everyone group. GroupSid:S-1-9-1551374245-3746625149-2333054533-2458719197-2313548623-0-0-0-0-3, MemberId:76050ddf-4fd8-48c4-a1ff-859e44364519, ScopeId:7df650df-0f8b-4596-928d-13dd89e5f34f

Copy the GroupSid, MemberId, and ScopeId into the templated SQL command.

DECLARE @p6 dbo.typ_GroupMembershipTable

INSERT into @p6 values('S-1-9-1551374245-3746625149-2333054533-2458719197-2313548623-0-0-0-0-3','Microsoft.TeamFoundation.Identity','76050ddf-4fd8-48c4-a1ff-859e44364519',0)

EXEC prc_UpdateGroupMembership @partitionId=1,@scopeId='7df650df-0f8b-4596-928d-13dd89e5f34f',@idempotent=1,@incremental=1,@insertInactiveUpdates=0,@updates=@p6,@eventAuthor='9EE20697-5343-43FC-8FC5-3D5D455D21C5',@updateGroupAudit=0

Run the completed command against the TFS configuration database. This will need to be repeated for each ISVError:300005 instance that TfsMigrator found. Errors with the same scope ID can be batched into one command. Once the commands have been executed you will need to run TfsMigartor validate again to ensure that the errors have been corrected. If the errors still persist, please contact vstsdataimport@microsoft.com.

Process Validation Types

There are two types of process validation done by the tfsMigrator. First, it checks to see if your projects have been customized and can be imported using the xml process customization model. Second, it checks to see if any of your projects can be converted into the new inherited process customization model. Two log files are generated. TfsMigrator.log and TryMatchOobProcesses.log.

TfsMigrator.log TfsMigrator.log contains the logs that prevent your collection from being imported becuase of process errors. By default your projects will land under the xml process customization model. This is the most flexible as it allows for most customizations such as custom fields, work item types, and workflows. You must fix all the errors in this file in order to proceed with the data import.
TryMatchOobProcesses.log TryMatchOobProcesses.log file will be generated and contains the list of errors that the validator found for your projects land in the inherited model. TfsMigrator will look at your projects and determine if that project is using an OOB process such as Agile, Scrum, or CMMI. If it is, and does not contain and customizations, will we bring that project into the inherited model. Errors in this file will not prevent you from doing the data import.

Most customers have a mix of projects that have been customized (i.e. custom fields) and projects that are using an OOB process template. TfsMigrator checks each project and validates it accordingly. It is very possible you will have some projects that will be mapped to an OOB process and some projects will use the xml for process customization.

We recommend that for any project that has not been customized, that you review the TryMatchOobProcesses.log to determine if there are any errors. If so, make the adjustments accordingly so that the project can be mapped to an OOB process upon data import.

Dealing with Process Errors

Are your process templates customized? Are you using an older outdated process template? If so, you will most likely have process validation errors. TfsMigrator does an exhaustive check against your process templates. It checks to make sure that it is valid for Team Services. Odds are you will need to make some adjustments and apply them to your TFS collection.

If you are using an OOB Agile, Scrum, or CMMI process you probably won't see any errors in the tfsMigrator.log. Instead, check the TryMatchOobProcesses.log for errors. If you are error free then your project will map to an OOB process.

There are variety of customizations that will not work in Team Services. Make sure you review the list of customizations that are supported.

If you have projects that are using an older process template, the TfsMigrator will find several errors. This is because your process templates have not been updated to match the most recent process templates. To start, try running the Configure Features Wizard for each project. This will attempt to update your process templates with the most recent features. Doing so should drastically reduce the error count.

Finally, make sure you have witadmin on the machine that you intend to use to fix the process errors. This can be your local desktop. Witadmin is used in the automated scripts and is required whenever making changes to the process templates.

Step 1 – Review Errors

TfsMigrator.log file will be generated and contains the list of errors that the validation process found. To view the logs, open the TfsMigrator.log file. Search for the string "Validation - Starting validation of project 1". Each project is validated so you will need to scan through all the projects. Examine any lines that have a prefix of "[Error …".

Process logging file generated by TfsMigrator

We have documented the majority of the validation errors. For each validation error we have provided the error number, description, and the method to resolve.

Step 2 – Fix Errors

Now you know what projects have errors, the details of those errors, and how to fix them. Fixing the errors requires that you to change the xml and apply the changes back into the project.

We do not suggest using the TFS Power Tools. It is highly recommended that you modify the XML manually.

To get the process template from the project add the /SaveProcesses parameter when running the tfsMigrator command.

TfsMigrator validate /collection:{collection URL} /SaveProcesses

This command will extract the xml from the project and place it into the same folder as the logs. Extract the zip files to your local machine so that you can edit the files.

Now you need to fix the xml. Use the logs from the TfsMigrator.log file to determine the errors for each project.

Process logging file generated by TfsMigrator

Some errors will require you to do use a witamdin changefield command. Changing a field name is the most common example. To save yourself some time, we recommend you run the witadmin changfield ... command and then re-run the tfsMigrator tool. Doing this will re-export the xml with the corrected names. Otherwise you will need manually fix the fields in the xml as well.

Once you make a fix then you need to conform. Conform is defined as taking the XML you just changed and applying it back into TFS. To do this, depending on the changes you made, you will need to run one or more witadmin commands. To make this easier for you, we created a PowerShell script to automate the process. The script contains all of the witadmin commands needed to conform the entire process.

You can get the scripts at https://github.com/Microsoft/process-customization-scripts. Use the import/ConformProject.ps1 script.

.\conformproject.ps1 “<collection url>” “<project name>” “<process template folder>”

Conform project processes script running

When the script has completed you need to re-run the TfsMigrator to validate the collection. Follow steps 1 - 3 until the TfsMigrator generates no more validation errors.

If you are new to xml and witadmin, we suggest you make one fix at a time and then conform. Continue this loop until all errors are resolved.

Common Validation Errors

VS402841: Field X in work item type Bug has syncnamechanges=false but has rules making it an identity field. Identity fields must have syncnamechanges=true. Please update your process template to include this change.

In Team Services we added a rule so that every identity field must have the syncnamechanges=true attribute. In TFS that rule does not apply. Therefore, the TfsMigrator will identify this as an issue. Don't worry, making this change on TFS on-prem will not cause any harm.

To fix this you will need to run the witadmin changefield command. Syntax for the command will look something like this:

witadmin changefield /collection:http://AdventureWorksServer:8080/tfs/DefaultCollection /n:fieldname /syncnamechanges:true

For more information on the changfield command see https://msdn.microsoft.com/en-us/library/dd236909.aspx

TF402556: For field System.IterationId to be well defined, you must name it Iteration ID and set its type to Integer.

This error is typical for old process templates that have not been updated in some time. Try running the configure features wizard on each project. Alternatively you can run the follow witadmin command:

witadmin changefield /collection:http://AdventureWorksServer:8080/tfs/DefaultCollection /n:fieldname /name:newname

TF402571: Required element BugWorkItems is missing from Process Configuration.

This error is typically when a process has not been updated in a while. Try running the configure features wizard on each project to resolve.

TF402564: You've defined XX global lists. Only 64 are allowed.

By default, Team Services will support 64 global lists. You will typically run across this error if you have a large amount of build definitions. The global list named Builds – TeamProjectName gets created for each new build definition. You will need remove the outdated global lists.

Additional Resources

Dealing with Import Errors

Hit a failure when running your import? Failures in the import space fall into one of two categories. Verification failures happen when the import fails to start. The indication that this has occurred is when TfsMigrator attempts to queue an import, but returns an error instead. Import failures happen when the import was queued successfully in TfsMigrator, but failed after that point. The individual that queued the import will recieve a failure email if this happens.

Verification Failures

Verification failures happen when the import fails to start. Issues falling into this category mean that something with your import request isn't valid. Look up you error message below and follow the recommended guidance on how to resolve the error. After that your team can try to queue the import again.

VS403252: The specified import code {0} is not valid, expired, or is already in use.

This error means that something is wrong with your import code. Either it has already been successfully used for another import, it expired, or it isn't valid. Double check the code that you've placed in the import specification file against the codes that you were given as part of the preview. If you believe that there is a problem please reach out vstsdataimport@microsoft.com.

VS403253: Queuing an import requires an import code.

An import code was not provided in the import specification file. Open your import specification file and be sure that you've placed one of your team's import codes into the "ImportCode" parameter.

VS403254: Region {0} may not be used for the Import, it is not a supported region.

The region that you entered for your Team Services import isn't supported. Open your import specification file and update the region that you've provided with the correct short name for the region you want to import into. These could be, but aren't limited to: CUS, WEU, MA, EAU, SBR. These correspond to Central US, West Europe, India South, East Australia, and South Brazil respectively.

VS403249: The account {0} already exists. Please select a different name and try the import again.

All Team Services imports go into a new account that is created at import time. This error indicates that the account name your team has selected is already being used by an existing account. Select a different name and update the import specification file before retrying the import.

VS403250: The dacpac is not a detached TFS Collection database.
VS403286: The dacpac is from a TFS Configuration database. You must use a detached TFS Collection database.

The DACPAC is not built off a detached collection. The collection database will need to be detached and the DACPAC generated again.

VS403243: Unable to connect to the database using the provided SQL Connection String {0}.

Unable to make a connection to the database using the provided SQL Connection String. Review the parameters that were provided to ensure they’re correct and try again.

VS403260:  The database is not detached.

The database is not detached. It will need to be detached and the import queued again.

VS403261: The SQL connection string must use encryption.

The connection string must be excrypted otherwise the password will be sent in the clear. Please add "Encrypt=true" to your SQL connection string.

VS403262: The SQL connection string must use SQL Authentication, Integrated Authentication is not supported.

Please add "Integrated Security=False" to your SQL connection string.

VS403263: The User ID {0} must be member of the database role {1}.

This error means that your SQL login user does not have the required database role. Please make sure 'VSODBOROLE' is assigned to you login.

VS403264: The database is not a TFS Collection database, it cannot be used for import.

The connection string does not point to a TFS Collection database.

VS403255: The collection cannot be imported due to an ongoing post upgrade job. Please wait and try again later

The TFS Update has queued the file migration job. Imports cannot be performed until this job has completed. The completion time for this job is dependent on the size of the collection. Job progress can be tracked by running the below query on the collection database:

SELECT  COUNT (*) as remaining_files_to_migrate
FROM    tbl_FileReference
WHERE   PartitionId > 0
        AND MigrateFileId IS NOT NULL

Once the number of files remaining to migrate is zero you can run TfsMigrator.

VS403282: The source location parameter contains a new line character. Please ensure the SAS key is defined on a single line in the import specification file.

There is a new line character in the source location value, this could have been left over when copying the SAS key from your windows console, please remove the line break and try again.

VS403285: Invalid identity mapping file - {0}

Identity mapping file has unexpected (invalid) content. This error may be reported in different cases:

  • File may be completely empty
  • File may have single line with column names but no actual content (identity mapping entries)
  • File may not be parsed as CSV file (usually it happens when text editor used to edit generated mapping file didn't save it as proper CSV file)
  • File may have duplicated records for the same identity

    VS403271: It appears that your DACPAC was uploaded to East US. It’s required that customers targeting Central US for import put their DACPACs in Central US. Please move your DACPAC to Central US and requeue the import.

Your import files and DACPAC are not located in the required Azure region to complete the import to your target Team Services region. Please Create a new windows azure storage account in the required region and copy your files. Below is an example of how to copy your data using AzCopy.

AzCopy.exe /Source:https://accountSCUS.blob.core.windows.net/mycontainer /SourceKey:"primary access key" /Dest:https://accountCUS.blob.core.windows.net/mycontainer /DestKey:"primary access key" /S

Import Failures

When an imports fails the individual that queued the import will receive an email. In this case, your team will need to roll back by bringing your Team Foundation Server instance back online and attaching your collection. This will allow your team members to continue working. Once your team is back up and working again, follow the instructions in the failure email and file a support case to get assistance.

Note that during this public preview weekend support is NOT provided. You will need to bring your collection back online and file a support case.