Enterprise Wide PST Import - Using RoboCopy

This is Part 8 in a series of posts about my experience tackling the migration of PST files.
The first post in the series is here.
The next post in this series is here.

Our users are world-wide and even though we are concentrating on our Headquarters, many other users are seeing the benefit of importing their PST files. Especially those who travel a lot. Nearly everyone of these heavy travelers carry an external hard drive with their PST files. Now they can get those messages via OWA and they can discard the extra weight of the external hard drive.

We started getting many request for Archive mailbox from all over the world. The challenge is to copy the PST file to our staging area so it can be processed by the server.

Alas, Copy-Item seems to be really slow when working over a WAN line. And many of our pipes can get saturated. I started experimenting with Robocopy, and I am not entirely happy with the results, but it's better than copy-item.

I created a small stand alone script (RoboCopy-Item) that does the very simple  copy of a file. I am experimenting with the settings like /IPG:300 /Z, etc -- trying to find the best overall throughput for our environment. Fast, but not choking the WAN. I am still experimenting there.

In the Add-PSTImportQueue function we now look to see if a user is in HQ or outside somewhere. If they are outside HQ then we mark the job with a status of RoboCopy.

Then we use: Robocopy-PSTImportQueue which looks at the import queue and starts background jobs for each job with a RoboCopy status. I do this on a by user, by location selection. We don't want to have 50 jobs running for 30 users in 1 location. I keep it at 1 user per 1 location at any given time.

Then we use a Get-RoboCopyjob <jobnumber> that just gets the 1st 10 lines and that last 20 lines of a job, so you can quckly see the status of the job.

We toyed with the idea of incorporating this into the overall script so we could just run it, but too many things can go wrong and the powershell window with all the jobs running can get closed ot the server rebooted, etc.

We want to find a better way to do this.

Still bigger files -- over 1.5G -- are taking forever. We are using BITS to control the amount of traffic that can be allowed, so we have to go modify the Registry and restart BITS service. But copies from computers with that setting fills up the pipe to that location.

I'll update as we search for the better way.



Introduction: The Beginings
Part 1: Script Requirements
Part 2: Add-PSTImportQueue
Part 3: Process-PSTImportQueue
Part 4: Some Tools we added
Part 5: Set-PSTImportQueue
Part 6: About PST Capture
Part 7: More PST Import Tools
Part 8: Using RoboCopy
Part 9: Morning Status Report
Part 10: Using BITS Transfer
Part 11: Get the script / Set up
Part 12: The Functions