The Clockwork SharePoint Cloud Migration Tool leverages the power of the Microsoft SharePoint Online Migration API to get documents, versions, and metadata into SharePoint Online. Using the API is Microsoft’s recommended approach to migration. It works as follows:
- build up a Migration Run
- validate the Migration Run data
- batch and upload the documents and metadata to Azure storage
- launch and monitor migration jobs
- validate migrated documents.
A migration run is a selection of data from the source DM system. It might contain just a few 100 documents or a few 100,000. The Clockwork Tool imports the metadata for the selected documents into a local SQLite database which is then used for monitoring the job progress.
- Check source files exist
- check if the file (or a file of the same name) is already in SharePoint
- check the users assigned to version created and modified-by metadata actually exist.
Batch and upload
The Microsoft Migration API migration jobs work most effectively with batches of around 250MB, but multiple jobs can be run simultaneously.
The Clockwork Tool groups the documents into batches and uploads the documents and associated metadata (as XML) to your Azure storage.
Launch and monitor
The Tool launches migration jobs and monitors them to maintain a consistent number of concurrently running jobs.
Once individual migration jobs are complete the Tool checks the files actually arrived in SharePoint and notes any discrepancies.
There are several factors that can affect the overall speed
- Customer internet speed can affect both the time it takes to validate documents, and also the time it takes to upload documents to Azure.
Documents can be uploaded in advance ready for launch of the migration jobs
- the Microsoft Migration API runs in your SharePoint Online tenancy as a background job and it backs off whenever the tenancy is under load. This means that during working hours (in the time zone of your SharePoint Online datacentre) performance is usually very slow, however after-hours Microsoft expect speeds of 250 GB/day are possible
- preparation of the migration run data can also take time as it depends on SQL Server speed, as well as the capabilities of the PC used to run the Migration Tool (where the SQLite database is run).
Files generally should not be migrated to the root of libraries. This is because the Migration API first uploads files to the root of the target library where it processes their metadata before moving them to the target destination. Files in the root of the library will prevent documents of the same name being uploaded.
Note: When the Migration API uploads a file to the root of the library it hides it so you don’t see it.
Similarly, because of the migration jobs running concurrently, there may be occasional errors if files of the same name (with different destination folders) happen to be uploaded at exactly the same time. However, the API error logs indicate if this happens so these occasional issues can be rectified.
For the traditional DM systems that use SQL Server the Tool uses stored procedures to retrieve the document data and insert it into the local SQLite database.
For iManage and OpenText DM systems the Tool has modules to connect immediately. For other systems we assist you with the connection.